Graph Neural Network Embeddings

This paper extended convolutional neural networks to graph domains by proposing spectral multipliers based on the graph Laplacian's eigenvectors as a generalization of Fourier basis convolutions.

“Given a weighted graph, we can try to generalize a convolutional net by operating on the spectrum of the weights, given by the eigenvectors of its graph Laplacian.”◌ not checked against the paper’s text as it now stands

This paper introduced a method to learn spectral multipliers with O(1) parameters by using smooth interpolated kernels instead of one parameter per eigenvector.

“one can learn spectral multipliers which are smooth. Smoothness can be prescribed by learning only a subsampled set of frequency multipliers and using an interpolation kernel to obtain the rest, such as cubic splines.”◌ not checked against the paper’s text as it now stands

This paper demonstrated that the spectral construction recovers standard CNNs when applied to natural images by showing the Laplacian eigenvectors correspond to the Fourier basis for translation-invariant data.

“when applied to natural images, the construction in […] using the covariance as the similarity kernel recovers a standard convolutional network, without any prior knowledge.”◌ not checked against the paper’s text as it now stands

This paper established the connection between spectral filters and spatial localization through the duality that smooth spectral multipliers produce spatially localized filters.

“It results that a funtion x which is spatially localized has a smooth frequency response x̂ = V^T x.”◌ not checked against the paper’s text as it now stands

This paper proposed learning embeddings of spectral coordinates through a dual graph constructed from the similarity of signals in the spectral domain.

“A possible algorithmic stategy is to consider an input distribution X=(x_k)_k consisting on spatially localized signals and to construct a dual graph W by measuring the similarity of in the spectral domain: X= V^T X.”◌ not checked against the paper’s text as it now stands

The paper showed that the GCN model learns hidden layer representations that encode both local graph structure and node features.

“learns hidden layer representations that encode both local graph structure and features of nodes”◌ not checked against the paper’s text as it now stands

The paper demonstrated that node embeddings learned by the GCN capture semantic information useful for classification, as shown through t-SNE visualization of hidden layer activations.

“Right: t-SNE […] visualization of hidden layer activations of a two-layer GCN trained on the Cora dataset […] using 5% of labels. Colors denote document class.”◌ not checked against the paper’s text as it now stands

The paper uses graph convolutional networks to learn embeddings that capture latent representations of nodes in graph-structured data.

“We introduce the variational graph auto-encoder (VGAE), a framework for unsupervised learning on graph-structured data based on the variational auto-encoder (VAE). This model makes use of latent variables and is capable of learning interpretable latent representations for undirected graphs”◌ not checked against the paper’s text as it now stands

The paper demonstrates that embeddings learned by the VGAE model can be used effectively for link prediction by computing inner products between node embeddings.

The paper shows that incorporating node features into the embedding model significantly improves predictive performance on graph tasks.

“In contrast to most existing models for unsupervised learning on graph-structured data and link prediction, our model can naturally incorporate node features, which significantly improves predictive performance on a number of benchmark datasets.”◌ not checked against the paper’s text as it now stands

This paper used a graph embedding network called Structure2Vec to represent the evaluation function in a greedy algorithm for learning optimization policies.

“We will use a graph embedding network, called (S2V) […], to represent the policy in the greedy algorithm.”◌ not checked against the paper’s text as it now stands

The paper parameterized the Q-function using node embeddings computed through recursive feature aggregation across the graph topology.

“we will use the embedding μ_v^(T) for node v and the pooled embedding over the entire graph, ∑_u ∈Vμ_u^(T), as the surrogates for v and h(S), respectively”◌ not checked against the paper’s text as it now stands

This paper demonstrated that graph embeddings enable the learned policies to generalize to problem instances of different sizes than those used in training.

“This allows the policy to discriminate among nodes based on their usefulness, and generalizes to problem instances of different sizes.”◌ not checked against the paper’s text as it now stands

This paper proposed SNE, a framework that learns node representations by preserving both structural proximity and attribute proximity in social networks.

“We propose a generic Social Network Embedding framework (SNE), which learns representations for social actors (i.e., nodes) by preserving both the structural proximity and attribute proximity.”◌ not checked against the paper’s text as it now stands

This paper demonstrated that integrating network structure and attributes produces more informative node representations than structure-only methods.

“By jointly considering the attribute homophily and the network structure, we believe more informative node representations can be learned.”◌ not checked against the paper’s text as it now stands

This paper used deep neural networks with multiple layers to model both structural and attribute proximity, enabling better capture of complex interactions.

“To capture the complex interactions between features, we adopt a multi-layer neural network to take advantage of strong representation and generalization ability of deep learning.”◌ not checked against the paper’s text as it now stands

This paper introduced a framework for learning functions that generate node embeddings for previously unseen nodes by sampling and aggregating features from a node's local neighborhood.

“Instead of training individual embeddings for each node, we learn a function that generates embeddings by sampling and aggregating features from a node's local neighborhood.”◌ not checked against the paper’s text as it now stands

This paper extended graph convolutional networks to the inductive setting and proposed trainable aggregation functions beyond simple convolutions for generating node embeddings.

“In this work we both extend GCNs to the task of inductive unsupervised learning and propose a framework that generalizes the GCN approach to use trainable aggregation functions (beyond simple convolutions).”◌ not checked against the paper’s text as it now stands

This paper designed embedding generation through aggregating neighbor representations iteratively at different search depths, where nodes aggregate information from progressively further reaches of the graph.

“The intuition behind Algorithm […] is that at each iteration, or search depth, nodes aggregate information from their local neighbors, and as this process iterates, nodes incrementally gain more and more information from further reaches of the graph.”

The paper introduced user and item embeddings produced through graph convolutional message passing on a bipartite interaction graph.

“The auto-encoder produces latent features of user and item nodes through a form of message passing on the bipartite interaction graph.”◌ not checked against the paper’s text as it now stands

The paper reformulated the graph auto-encoder to produce user and item embedding matrices through the encoder function.

“For bipartite recommender graphs G = (𝒲, ℰ, ℛ), we can reformulate the encoder as [U, V] = f(X, M_1, …, M_R), where M_r ∈{0,1}^N_u × N_v is the adjacency matrix associated with rating type r∈ℛ, such that M_r contains 1's for those elements for which the original rating matrix M contains observed ratings with value r. U and V are now matrices of user and item embeddings with shape N_u× E and N_v× E, respectively.”◌ not checked against the paper’s text as it now stands

The paper described how the graph convolutional encoder transforms node features into embeddings through a specific two-step process combining message passing and dense transformations.

“To arrive at the final embedding of user node i, we transform the intermediate output h_i as follows: u_i = σ(W h_i). The item embedding v_i is calculated analogously with the same parameter matrix W.”◌ not checked against the paper’s text as it now stands

This paper unifies four popular network embedding methods (DeepWalk, LINE, PTE, and node2vec) by showing they all perform implicit matrix factorization with derivable closed-form matrices.

This paper establishes that LINE is a special case of DeepWalk when the context window size is set to one.

This paper reveals a theoretical connection between DeepWalk's implicit matrix and graph Laplacians, enabling a new NetMF algorithm for computing network embeddings.

This paper demonstrates that network embedding through skip-gram based methods implicitly factorizes specific matrices derived from random walks and network structure.

The paper develops a Graph Convolutional Network algorithm that generates embeddings of nodes incorporating both graph structure and node feature information.

“We develop a data-efficient Graph Convolutional Network (GCN) algorithm, which combines efficient random walks and graph convolutions to generate embeddings of nodes (i.e., items) that incorporate both graph structure as well as node feature information.”◌ not checked against the paper’s text as it now stands

The paper shows how embeddings can be used to make high-quality item-item recommendations by selecting nearest neighbors in the embedding space.

“We show how these embeddings can be used to make high-quality item-item recommendations by selecting nearest neighbors in the embedding space.”◌ not checked against the paper’s text as it now stands

The paper introduces importance pooling to weigh the importance of node features in aggregation based on random-walk similarity measures.

“We introduce a method to weigh the importance of node features in this aggregation based upon random-walk similarity measures, leading to a 46% performance gain in offline evaluation metrics.”◌ not checked against the paper’s text as it now stands

This paper proposes a unified framework for learning embeddings of attributed multiplex heterogeneous networks that supports both transductive and inductive learning.

“we formalize the problem of embedding learning for the Attributed Multiplex Heterogeneous Network and propose a unified framework to address this problem. The framework supports both transductive and inductive learning.”◌ not checked against the paper’s text as it now stands

The paper demonstrates that embeddings learned from the proposed framework achieve significant improvements in link prediction performance over state-of-the-art methods.

“with the learned embeddings from the proposed framework, we can achieve statistically significant improvements (e.g., 5.99-28.23% lift by F1 scores; p≪ 0.01, t-test) over previous state-of-the-art methods for link prediction.”◌ not checked against the paper’s text as it now stands

The paper provides theoretical analysis of the proposed embedding framework showing its connection to previous works and proving its better expressiveness.

“We also give the theoretical analysis of the proposed framework, showing its connection with previous works and proving its better expressiveness.”◌ not checked against the paper’s text as it now stands

This paper proposes to integrate user-item interactions into the embedding process by propagating embeddings on the user-item bipartite graph structure.

“In this work, we propose to integrate the user-item interactions — more specifically the bipartite graph structure — into the embedding process. We develop a new recommendation framework Neural Graph Collaborative Filtering (NGCF), which exploits the user-item graph structure by propagating embeddings on it.”◌ not checked against the paper’s text as it now stands

This paper develops embedding propagation layers that refine user and item embeddings by aggregating representations from neighboring users and items.

“Specifically, we devise an embedding propagation layer, which refines a user's (or an item's) embedding by aggregating the embeddings of the interacted items (or users).”◌ not checked against the paper’s text as it now stands

This paper stacks multiple embedding propagation layers to capture high-order connectivity information and encode collaborative signals into embeddings.

“By stacking multiple embedding propagation layers, we can enforce the embeddings to capture the collaborative signal in high-order connectivities.”◌ not checked against the paper’s text as it now stands

LightGCN learns user and item embeddings by linearly propagating them on the user-item interaction graph, combining embeddings from all layers through weighted sum.

“LightGCN learns user and item embeddings by linearly propagating them on the user-item interaction graph, and uses the weighted sum of the embeddings learned at all layers as the final embedding.”◌ not checked against the paper’s text as it now stands

The model uses only ID embeddings as initial representations which are refined through neighborhood aggregation without feature transformation or nonlinear activation.

“In LightGCN, the only trainable model parameters are the embeddings at the 0-th layer, i.e., e_u^(0) for all users and e_i^(0) for all items. When they are given, the embeddings at higher layers can be computed via LGC”◌ not checked against the paper’s text as it now stands

LightGCN combines embeddings at different layers to obtain final representations that capture different-order proximities in the user-item graph.

“The embeddings at different layers capture different semantics. E.g., the first layer enforces smoothness on users and items that have interactions, the second layer smooths users (items) that have overlap on interacted items (users), and higher-layers capture higher-order proximity”