# Domain-Specific Metric Embeddings

> One thread of 9 from the "embedding" map, covering the 14 papers in it. The other threads are not represented here.
>
> Every quotation was copied word for word from the paper's own text, and
> checked against that text. Quotes marked *inferred* failed that check and
> must be re-checked before use. Quotes marked *not re-checked* have not been
> matched against the paper's text as it now stands, so they carry no current
> verification either. Lines labelled *the tool's reading* are
> model judgment, not quotation, and carry no verification.
>
> **This is a scaffold, not prose.** The citations, quotes and structure are
> real; the argument is yours to write.

> Connects to: Knowledge Graph & Relational Embeddings, Graph Neural Network Embeddings.

Developed for speaker identification and face recognition as fixed-dimensional acoustic/visual feature vectors (2015-2017), evolved to encompass person re-identification, metric learning objectives, and learned distance functions (2016-2018), increasingly integrated into recommendation and ranking pipelines.


**BPR: Bayesian Personalized Ranking from Implicit Feedback** (2012) \cite{rendle2012bpr}

> “Each row wu in W can be seen as a feature vector describing a user u and similarly each row hi of H describes an item i.”
>
> ✓ verified: found word for word in the paper's own text


**Bidirectional LSTM-CRF Models for Sequence Tagging** (2015) \cite{huang2015bidirectional}

> “It has been shown in […] that word embedding plays a vital role to improve sequence tagging performance. We downloaded[http://ronan.collobert.com/senna/] the embedding which has 130K vocabulary size and each word corresponds to a 50-dimensional embedding vector. To use this embedding, we simply replace the one hot encoding word representation with its corresponding 50-dimensional vector.”
>
> ✓ verified: found word for word in the paper's own text


**FaceNet: A unified embedding for face recognition and clustering** (2015) \cite{schroff2015facenet}

What it did: Introduced metric-optimized embeddings via triplet loss  *(the tool's reading)*

FaceNet pioneered learning embeddings by directly optimizing compact Euclidean feature spaces through triplet loss rather than intermediate classification layers. This established embeddings as explicit learned mappings where "squared L2 distances between embeddings directly correspond to face similarity."  *(the tool's reading)*

> “In this paper we present a system, called FaceNet, that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity.”
>
> ✓ verified: found word for word in the paper's own text


**A Discriminatively Learned CNN Embedding for Person Reidentification** (2016) \cite{zheng2016discriminatively}

What it did: Combined identification and verification losses for discriminative embeddings  *(the tool's reading)*

This work extended FaceNet's triplet-loss approach by proposing a siamese network that simultaneously combines identification loss and verification loss. The paper showed that "combining the two losses produces an embedding whose activation patterns are a union of the identification-only and verification-only embeddings, yielding more discriminative and robust features."  *(the tool's reading)*

> “Our network learns a discriminative embedding and a similarity measurement at the same time, thus making full usage of the re-ID annotations.”
>
> ✓ verified: found word for word in the paper's own text


**MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications** (2017) \cite{howard2017mobilenets}

> “It builds face embeddings based on the triplet loss.”
>
> ✓ verified: found word for word in the paper's own text


**Deep Speaker: an End-to-End Neural Speaker Embedding System** (2017) \cite{li2017deep}

What it did: Generalized metric embeddings to speaker domain with domain-specific architectures  *(the tool's reading)*

Deep Speaker transplanted the triplet-loss embedding methodology from faces into speaker recognition, introducing domain-specific improvements like ResCNN and GRU architectures for frame-level features. The system demonstrated "end-to-end training of frame-level feature extraction, pooling, and embedding generation using triplet loss based on cosine similarity."  *(the tool's reading)*

> “We present Deep Speaker, a neural speaker embedding system that maps utterances to a hypersphere where speaker similarity is measured by cosine similarity. The embeddings generated by Deep Speaker can be used for many tasks, including speaker identification, verification, and clustering.”
>
> ✓ verified: found word for word in the paper's own text


**Methods for interpreting and understanding deep neural networks** (2017) \cite{montavon2017methods}

> “Examples of domains that are not interpretable are abstract vector spaces (e.g. word embeddings […]), or domains composed of undocumented input features (e.g. sequences with unknown words or symbols).”
>
> ✓ verified: found word for word in the paper's own text


**Attentive Statistics Pooling for Deep Speaker Embedding** (2018) \cite{okabe2018attentive}

What it did: Refined speaker embeddings via domain-specific attention pooling  *(the tool's reading)*

Attentive Statistics Pooling advanced speaker embeddings by replacing conventional pooling with attention-weighted aggregation of frame-level features. This domain-specific innovation "proposes attentive statistics pooling for deep speaker embedding, which produces importance-weighted standard deviations as well as weighted means of frame-level features using an attention mechanism."  *(the tool's reading)*

> “an average pooling layer has been introduced to aggregate frame-level speaker feature vectors to obtain an utterance-level feature vector, i.e., speaker embedding, with a fixed number of dimensions.”
>
> ✓ verified: found word for word in the paper's own text


**Universal Sentence Encoder** (2018) \cite{cer2018universal}

> “The models take as input English strings and produce as output a fixed dimensional embedding representation of the string.”
>
> ✓ verified: found word for word in the paper's own text


**Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis** (2018) \cite{jia2018transfer}

> “a speaker encoder network, trained on a speaker verification task using an independent dataset of noisy speech without transcripts from thousands of speakers, to generate a fixed-dimensional embedding vector from only seconds of reference speech from a target speaker;”
>
> ✓ verified: found word for word in the paper's own text


**Collaborative Memory Network for Recommendation Systems** (2018) \cite{ebesu2018collaborative}

> “We use the terms user/item latent factors, memories and embeddings interchangeably.”
>
> ✓ verified: found word for word in the paper's own text


**Self-Supervised Learning of Pretext-Invariant Representations** (2019) \cite{misra2019self}

> “Specifically, we compute the representation of 𝐈, f(𝐯_𝐈), by extracting features, average pooling, and a linear projection to obtain a 128-dimensional representation.”
>
> ✓ verified: found word for word in the paper's own text


**Knowledge Distillation: A Survey** (2020) \cite{gou2020knowledge}

> “Based on idea of manifold learning, the student network is learned by feature embedding, which preserves the feature similarities of samples in the intermediate layers of the teacher networks […].”
>
> ✓ verified: found word for word in the paper's own text


**Ensemble deep learning: A review** (2021) \cite{ganaie2021ensemble}

> “The training set in the BIER is reweighed via the negative gradient of the loss function to project the input spaces (images) into a collection of independent output spaces. To make BIER more robust, Hierarchical Boosted deep metric learning […] incorporated the hierarchical label information into the embedding ensemble which improves the performance of the model on the large scale image retrieval application.”
>
> ✓ verified: found word for word in the paper's own text


_[Your synthesis: what it enabled, what it left unsolved.]_


## Where this thread connects

Each crossing is where one line of work fed another. These are the tool's reading of
the corpus, not quotations.

- **Knowledge Graph & Relational Embeddings** → **Domain-Specific Metric Embeddings** (2018): Knowledge-aware recommendations
- **Graph Neural Network Embeddings** → **Domain-Specific Metric Embeddings** (2018): Graph-based ranking