Recurrent Glimpse-Based Visual Attention

This paper introduced a recurrent neural network model that selectively attends to different regions of an image sequentially rather than processing the entire image at once.

“The model is a recurrent neural network (RNN) which processes inputs sequentially, attending to different locations within the images (or video frames) one at a time, and incrementally combines information from these fixations to build up a dynamic internal representation of the scene or environment.”◌ not checked against the paper’s text as it now stands

This paper framed attention as a sequential decision problem where an agent learns task-specific policies for where to attend using reinforcement learning rather than fixed saliency rules.

“take inspiration from these results and develop a novel framework for attention-based task-driven visual processing with neural networks. Our model considers attention-based processing of a visual scene as a control problem”◌ not checked against the paper’s text as it now stands

This paper demonstrated that attention-based models can outperform standard convolutional neural networks on cluttered images by focusing computational resources only on relevant regions.

“Our results also suggest that an attention-based model may be better than a convolutional neural network at both dealing with clutter and scaling up to large input images.”◌ not checked against the paper’s text as it now stands

This paper showed that the attention model can learn to combine information from multiple sequential glimpses to make accurate classifications despite each individual glimpse being too small to see the entire object.

The paper introduced a deep recurrent neural network trained with reinforcement learning that learns to attend to relevant regions of input images for multiple object recognition.

“The proposed model is a deep recurrent neural network trained with reinforcement learning to attend to the most relevant regions of the input image.”◌ not checked against the paper’s text as it now stands

The paper's attention model directs its focus by using an emission network that acts as a controller to output the next glimpse location based on the recurrent network's internal state.

“The emission network takes the current state of recurrent network as input and makes a prediction on where to extract the next image patch for the glimpse network. It acts as a controller that directs attention based on the current internal states from the recurrent network.”◌ not checked against the paper’s text as it now stands

The paper demonstrated that the attention-based model learns to selectively process the image through sequential glimpses, with the model showing different learned attention policies for different tasks.

“It is surprising that the learned glimpses policy for predicting the next glimpse is very different in the addition task comparing to the predicting combination task. The model that learned to do addition toggles its glimpses between the two digits.”◌ not checked against the paper’s text as it now stands

This paper introduced a fully differentiable attention mechanism that allows the network to selectively focus on specific regions of an image during both reading and writing operations.

“The attention model in DRAW, however, is fully differentiable, making it possible to train with standard backpropagation.”◌ not checked against the paper’s text as it now stands

The paper demonstrated that attention enables the network to construct images sequentially by tracing lines, similar to human drawing, rather than progressively sharpening a blurred global image.

“Whereas without attention it progressively sharpens a blurred image in a global way, with attention it constructs the digit by tracing the lines—much like a person with a pen.”◌ not checked against the paper’s text as it now stands

The paper showed that the selective attention mechanism can be used to improve classification performance on cluttered images by allowing the network to iteratively focus on relevant regions and ignore clutter.

“As illustrated in Fig. […], having an iterative attention model allows the network to progressively zoom in on the relevant region of the image, and ignore the clutter outside it.”◌ not checked against the paper’s text as it now stands

The paper introduced an attention-based GRU modification that uses attention gates computed from question and memory information to update the hidden state, replacing the standard GRU update gate.

“By replacing the update gate u_i in the GRU (Equation […]) with the output of the attention gate g^t_i (Equation […]) in Equation […], the GRU can now use the attention gate for updating its internal state.”◌ not checked against the paper’s text as it now stands

The paper demonstrated that attention gates can be computed by interactions between input facts and both the question representation and episode memory state.

“We implement this attention by associating a single scalar value, the attention gate g^t_i, with each fact f_i during pass t. This is computed by allowing interactions between the fact and both the question representation and the episode memory state.”◌ not checked against the paper’s text as it now stands

The paper showed that the attention-based GRU is sensitive to position and ordering of input facts, which soft attention cannot preserve.

“For more complex queries, we would like for the attention mechanism to be sensitive to both the position and ordering of the input facts F. An RNN would be advantageous in this situation except they cannot make use of the attention gate from Equation […].”

The paper proposes an attention-based model that learns to selectively focus on discriminative parts of person image pairs through multiple glimpses and adaptive comparison.

“The end-to-end CAN learns to selectively focus on parts of pairs of person images after taking a few glimpses of them and adaptively comparing their appearance.”◌ not checked against the paper’s text as it now stands

The paper develops an attention mechanism that automatically localizes the most discriminative parts of persons in a recurrent manner of seeing and comparing images.

The paper generates attention maps directly from raw person images in an end-to-end trainable way rather than requiring pre-extracted features.

“our attention model can directly obtain the saliency-like attention maps from raw person image due to the end-to-end training pattern.”◌ not checked against the paper’s text as it now stands

The paper applies attention mechanisms with comparative components based on triplet comparisons rather than single image analysis.

The paper proposes a spatial attention module that automatically learns to focus on discriminative joints within each frame by assigning attention weights to different joints.

“We propose a spatial attention model to automatically explore and exploit the different degrees of importance of joints. With a soft attention mechanism, each joint within a frame is assigned a spatial attention weight based on the joint-selection gates.”◌ not checked against the paper’s text as it now stands

The paper introduces a temporal attention module that allocates different levels of importance to different frames in an action sequence rather than simply selecting key frames.

“For a sequence, the amount of valuable information provided by different frames is in general not equal. Only some of the frames (key frames) contain the most discriminative information while the other frames provide context information.”◌ not checked against the paper’s text as it now stands

The paper integrates spatial and temporal attention mechanisms into an end-to-end deep learning framework based on LSTM networks for skeleton-based action recognition.

“An end-to-end framework with two types of attention modules is designed based on the LSTM networks for skeleton based human action recognition.”

The paper proposes that the CNN learns an attention mechanism on the human body rather than background during identity discrimination, which the network leverages to adaptively locate and align pedestrians.

“Our key observation is that when the convolutional neural network (CNN) learns to discriminate between different identities, the learned feature maps usually exhibit strong activations on the human body rather than the background. The proposed network thus takes advantage of this attention mechanism to adaptively locate and align pedestrians within a bounding box.”◌ not checked against the paper’s text as it now stands

The paper visualizes that high responses in the Res4 Feature Maps are mostly concentrated on the pedestrian body, demonstrating the attention mechanism used for spatial localization.

“We visualize the Res4 Feature Maps in the base branch. We observe that high responses are mostly concentrated on the pedestrian body. So we use the Res4 Feature Maps to estimate the affine parameters.”◌ not checked against the paper’s text as it now stands

The paper uses high-level feature maps from the base branch that encode attention on the pedestrian to aid in estimating affine transformation parameters for alignment.

“On the other hand, since the Res2 Feature Maps are closer to the classification layer, it encodes the attention on the pedestrian and semantic cues for aiding identification.”

The paper proposes replacing standard MIL pooling operators with an attention mechanism implemented as a trainable weighted average of instances.

“we propose to replace widely-used permutation-invariant operators such as the maximum operator max and the mean operator mean by a trainable weighted average where weights are given by a two-layered neural network. The two-layered neural network corresponds to the attention mechanism”◌ not checked against the paper’s text as it now stands

The paper demonstrates that attention weights enable interpretability by identifying key instances that trigger the bag label.

“Notably, the attention weights allow us to find key instances, which could be further used to highlight possible ROIs.”◌ not checked against the paper’s text as it now stands

The paper shows that the attention-based MIL pooling provides insight into instance contributions while maintaining the requirement of permutation-invariance.

“an application of the proposed attention-based operator provides insight into the contribution of each instance to the bag label”◌ not checked against the paper’s text as it now stands