# Latent Representation & Generative Spaces

> One thread of 9 from the "embedding" map, covering the 17 papers in it. The other threads are not represented here.
>
> Every quotation was copied word for word from the paper's own text, and
> checked against that text. Quotes marked *inferred* failed that check and
> must be re-checked before use. Quotes marked *not re-checked* have not been
> matched against the paper's text as it now stands, so they carry no current
> verification either. Lines labelled *the tool's reading* are
> model judgment, not quotation, and carry no verification.
>
> **This is a scaffold, not prose.** The citations, quotes and structure are
> real; the argument is yours to write.

> Connects to: Self-Supervised & Contrastive Embeddings, Dimensionality Reduction & Visualization Embeddings.

Emerged from VAEs and GANs treating encoder outputs as learned latent embeddings (2013-2014), evolved to encompass disentangled representations, graphics codes, and style embeddings that enable controlled generation and manipulation (2015-2019), scaling to multimodal generative spaces (2019-2023).


**Auto-Encoding Variational Bayes** (2013) \cite{kingma2013auto}

What it did: Establish probabilistic encoder learning latent codes  *(the tool's reading)*

VAE introduces a recognition model that learns latent representations z through joint optimization with the generative model. These low-dimensional latent codes enable projection of high-dimensional data onto learned manifolds for visualization.  *(the tool's reading)*

> “From a coding theory perspective, the unobserved variables 𝐳 have an interpretation as a latent representation or code. In this paper we will therefore also refer to the recognition model q_ϕ(𝐳|𝐱) as a probabilistic encoder, since given a datapoint 𝐱 it produces a distribution (e.g. a Gaussian) over the possible values of the code 𝐳 from which the datapoint 𝐱 could have been generated.”
>
> ✓ verified: found word for word in the paper's own text


**Stochastic Backpropagation and Approximate Inference in Deep Generative Models** (2014) \cite{rezende2014stochastic}

What it did: Apply latent Gaussian variables to visualization embeddings  *(the tool's reading)*

Deep Latent Gaussian Models demonstrate that recognition models produce 2D latent embeddings for visualization where high-dimensional data projects into separable regions. This advances the VAE concept by showing practical utility of the learned latent space structure.  *(the tool's reading)*

> “We project the MNIST data set to a 2-dimensional latent space and use this 2D embedding as a visualisation of the data – an embedding for MNIST is shown in figure […]. The classes separate into different regions, suggesting that such embeddings can be useful in understanding the structure of high-dimensional data sets.”
>
> ✓ verified: found word for word in the paper's own text


**Semi-Supervised Learning with Deep Generative Models** (2014) \cite{kingma2014semi}

What it did: Use generative latent features for downstream classification  *(the tool's reading)*

Semi-supervised learning repurposes VAE embeddings as robust feature inputs to classifiers, showing that generative latent features are more separable than standard autoencoders. The embeddings now compute costs in low-dimensional spaces, improving efficiency.  *(the tool's reading)*

> “A commonly used approach is to construct a model that provides an embedding or feature representation of the data.”
>
> ✓ verified: found word for word in the paper's own text


**Unsupervised Representation Learning with Deep Convolutional Generative\n Adversarial Networks** (2015) \cite{radford2015unsupervised}

What it did: Discover semantic structure in generator latent space  *(the tool's reading)*

DCGANs reveal that generator latent noise vectors form semantically meaningful embedding spaces where face pose and other attributes vary linearly. This shifts focus from encoder embeddings to understanding how generator latent spaces inherently structure semantic concepts.  *(the tool's reading)*

> “In the context of evaluating learned representations of words […] demonstrated that simple arithmetic operations revealed rich linear structure in representation space. One canonical example demonstrated that the vector("King") - vector("Man") + vector("Woman") resulted in a vector whose nearest neighbor was the vector for Queen. We investigated whether similar structure emerges in the Z representation of our generators.”
>
> ✓ verified: found word for word in the paper's own text


**Deep Convolutional Inverse Graphics Network** (2015) \cite{kulkarni2015deep}

What it did: Learn disentangled interpretable latent representations  *(the tool's reading)*

Deep Convolutional Inverse Graphics Networks extend VAE encoders to learn disentangled graphics codes where specific neurons encode specific transformations. Training innovations force interpretability in embeddings at larger scales than prior VAE work.  *(the tool's reading)*

> “The decoder network is a function that transform a compact graphics code ( 200 dimensions) to a 150×150 image.”
>
> ✓ verified: found word for word in the paper's own text


**Context Encoders: Feature Learning by Inpainting** (2016) \cite{pathak2016context}

What it did: Leverage context prediction for semantic embeddings  *(the tool's reading)*

Context encoders learn semantically meaningful latent representations through inpainting rather than reconstruction, showing that context alone brings similar content close in embedding space. This demonstrates embeddings can capture semantic similarity via predictive objectives.  *(the tool's reading)*

> “This is similar in spirit to word2vec […] which learns word representation from natural language sentences by predicting a word given its context.”
>
> ✓ verified: found word for word in the paper's own text


**Conditional Image Generation with PixelCNN Decoders** (2016) \cite{vandenoord2016conditional}

What it did: Generate images from abstract latent embeddings  *(the tool's reading)*

Conditional PixelCNN decoders generate complex image distributions from latent embeddings, showing embeddings capture high-level semantic features. The approach enables smooth interpolation in embedding space and suggests embeddings from embeddings contain more abstract representations.  *(the tool's reading)*

> “The model can be conditioned on any vector, including descriptive labels or tags, or latent embeddings created by other networks.”
>
> ✓ verified: found word for word in the paper's own text


**WaveNet: A Generative Model for Raw Audio** (2016) \cite{vandenoord2016wavenet}

What it did: Apply embeddings to condition generative audio models  *(the tool's reading)*

WaveNet uses learned latent vector embeddings to globally condition generative models, extending embeddings beyond vision to discrete categorical inputs like speaker identity. This demonstrates embeddings generalize as universal conditioning mechanisms across modalities.  *(the tool's reading)*

> “Global conditioning is characterised by a single latent representation vech that influences the output distribution across all timesteps, e.g.a speaker embedding in a TTS model.”
>
> ✓ verified: found word for word in the paper's own text


**Quantum machine learning** (2016) \cite{biamonte2016quantum}

> “For quantum principal component analysis of classical data (qPCA […]), we choose a data vector v⃗_j at random, and use a quantum random access memory (qRAM […]) to map that vector into a quantum state: v⃗_j → |v_j⟩.”
>
> ✓ verified: found word for word in the paper's own text


**Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis** (2018) \cite{wang2018style}

> “GST embeddings can also be viewed as an external memory that stores style information extracted from training data. The reference signal guides memory writes at training time, and memory reads at inference time.”
>
> ✓ verified: found word for word in the paper's own text


**Representation Learning with Contrastive Predictive Coding** (2018) \cite{vandenoord2018representation}

What it did: Learn embeddings via contrastive predictive loss  *(the tool's reading)*

Contrastive Predictive Coding learns compact latent embeddings by predicting future observations rather than reconstructing inputs, using an InfoNCE loss that captures mutual information. A single framework produces embeddings across speech, images, text, and RL without task-specific design.  *(the tool's reading)*

> “In this paper we propose the following: first, we compress high-dimensional data into a much more compact latent embedding space in which conditional predictions are easier to model.”
>
> ✓ verified: found word for word in the paper's own text


**An Introduction to Variational Autoencoders** (2019) \cite{kingma2019introduction}

> “Its learning algorithm is a mix of classical (amortized, variational) expectation maximization but through the reparameterization trick ends up backpropagating through the many layers of the deep neural networks embedded inside of it.”
>
> ✓ verified: found word for word in the paper's own text


**Contrastive Multiview Coding** (2019) \cite{tian2019contrastive}

What it did: Extend contrastive learning to multiview embeddings  *(the tool's reading)*

Contrastive Multiview Coding shows that representation quality improves with multiple views, generalizing CPC beyond temporal prediction. This advances understanding that embedding quality depends on the diversity of contrastive signals across views.  *(the tool's reading)*

> “Given a set of sensory views, a deep representation is learnt by bringing views of the same scene together in embedding space, while pushing views of different scenes apart.”
>
> ✓ verified: found word for word in the paper's own text


**Data re-uploading for a universal quantum classifier** (2019) \cite{perezsalinas2019data}

> “However, if more layers are applied, each one will perform a different rotation, and many different rotations together have the capability of enabling a feature map. Data embedded in this feature space can be easily separated into classes employing the regions on the Bloch sphere.”
>
> ✓ verified: found word for word in the paper's own text


**VL-BERT: Pre-training of Generic Visual-Linguistic Representations** (2019) \cite{su2019bert}

> “For each input element, its embedding feature is the summation of four types of embedding, namely, token embedding, visual feature embedding, segment embedding, and sequence position embedding.”
>
> ✓ verified: found word for word in the paper's own text


**Bootstrap your own latent: A new approach to self-supervised Learning** (2020) \cite{grill2020bootstrap}

> “Generative approaches to representation learning build a distribution over data and latent embedding and use the learned embeddings as image representations.”
>
> ✓ verified: found word for word in the paper's own text


**Hierarchical Text-Conditional Image Generation with CLIP Latents** (2022) \cite{ramesh2022hierarchical}

What it did: Generate embeddings from text via diffusion priors  *(the tool's reading)*

DALL-E 2 explicitly generates CLIP image embeddings from text captions using diffusion priors rather than directly conditioning decoders on text embeddings. This introduces generative spaces over embeddings themselves, enabling diverse image variations from latent spaces.  *(the tool's reading)*

> “Given an image x, let z_i and z_t be its CLIP image and text embeddings, respectively.”
>
> ✓ verified: found word for word in the paper's own text


_[Your synthesis: what it enabled, what it left unsolved.]_


## Where this thread connects

Each crossing is where one line of work fed another. These are the tool's reading of
the corpus, not quotations.

- **Latent Representation & Generative Spaces** → **Self-Supervised & Contrastive Embeddings** (2018): Unsupervised latent learning
- **Dimensionality Reduction & Visualization Embeddings** → **Latent Representation & Generative Spaces** (2018): Manifold learning alignment