Drawing a sketch is a uniquely personal process that depends on previous knowledge, experiences, and current mood. In the sketching domain, deep generative models try to reconstruct, and interpolate sketches, however the success of these models also depends on users' needs. As a result, designing human-in-the-loop training gains importance from the data collection step to explaining the working mechanisms of models. In this repository, we explore possible approaches to build human-centered training and analysis processes.

1

What do we expect from interpolation?

One of the critical stages of interpretability is to explain the latent space behavior of generative models. However, recent papers do not put enough emphasis to improve the quality of latent space representation. We explored non-linear dimensionality reduction techniques (t-SNE and UMAP) on recent autoregressive and generative models (vanilla autoencoder, DCGAN, SketchRNN, Sketchformer) to explore the best practices and seek potential improvements to design more interactive training experiences for a human-in-the-loop approach. Our experiments and field studies revealed a list of considerations for visualizing the latent space in a more human-readable and developer-friendly way.

2

What do we expect from reconstruction?

3

What do we expect from latent space analysis?

You can explore the intra-class relations os Sketchformer: Open in Google CoLab