# Recurrent Glimpse-Based Visual Attention

> One thread of 5 from the "attention" map, covering the 8 papers in it. The other threads are not represented here.
>
> Every quotation was copied word for word from the paper's own text, and
> checked against that text. Quotes marked *inferred* failed that check and
> must be re-checked before use. Quotes marked *not re-checked* have not been
> matched against the paper's text as it now stands, so they carry no current
> verification either. Lines labelled *the tool's reading* are
> model judgment, not quotation, and carry no verification.
>
> **This is a scaffold, not prose.** The citations, quotes and structure are
> real; the argument is yours to write.

> Connects to: Self-Attention & Transformers.

Frames attention as a control problem where an RNN sequentially selects 'glimpses' or spatial regions to process, using reinforcement learning or differentiable soft-attention. Pioneered by models that treat visual attention as an iterative, foveation-like process. This thread bridges biological vision intuitions with learnable neural mechanisms but remains confined to recurrent, step-wise processing.


**Recurrent Models of Visual Attention** (2014) \cite{mnih2014recurrent}

What it did: Frames visual attention as sequential glimpse selection via RNN and reinforcement learning  *(the tool's reading)*

This paper introduces attention-based visual processing as a control problem where a recurrent agent sequentially selects limited image regions (glimpses) for processing at each time step. The approach uses an RNN combined with policy gradient learning for end-to-end training, moving beyond prior greedy or hardwired selection strategies.  *(the tool's reading)*

> “Instead humans focus attention selectively on parts of the visual space to acquire information when and where it is needed, and combine information from different fixations over time to build up an internal representation of the scene […], guiding future eye movements and decision making.”
>
> ✓ verified: found word for word in the paper's own text


**Multiple Object Recognition with Visual Attention** (2014) \cite{ba2014multiple}

What it did: Scales recurrent glimpse-based attention to real-world images and multiple objects  *(the tool's reading)*

This work extends the recurrent attention model (RAM) framework to handle real-world image complexity and multiple objects simultaneously, whereas prior RAM work only demonstrated performance on cluttered digit classification and toy control tasks.  *(the tool's reading)*

> “We present an attention-based model for recognizing multiple objects in images. The proposed model is a deep recurrent neural network trained with reinforcement learning to attend to the most relevant regions of the input image.”
>
> ✓ verified: found word for word in the paper's own text


**DRAW: A Recurrent Neural Network For Image Generation** (2015) \cite{gregor2015draw}

What it did: Replaces discrete REINFORCE-trained glimpses with fully differentiable Gaussian-filter attention  *(the tool's reading)*

DRAW introduces a fully differentiable two-dimensional spatial attention mechanism using 2D Gaussian filters that determine where the network reads from and writes to, explicitly mimicking human foveation. This replaces the discrete, REINFORCE-trained glimpse policy of prior RAM approaches while combining attention with sequential variational auto-encoding.  *(the tool's reading)*

> “Unlike the aforementioned works, we consider an explicitly two-dimensional form of attention, where an array of 2D Gaussian filters is applied to the image, yielding an image `patch' of smoothly varying location and zoom.”
>
> ✓ verified: found word for word in the paper's own text


**Dynamic Memory Networks for Visual and Textual Question Answering** (2016) \cite{xiong2016dynamic}

What it did: Adapts recurrent attention mechanism to gate facts in question-answering tasks  *(the tool's reading)*

This paper reinterprets attention as gating over input facts using global knowledge from questions and memory states to determine relevance, proposing an attention-based GRU variant. The mechanism shifts recurrent glimpse-based attention from visual navigation to multi-fact filtering in reasoning tasks.  *(the tool's reading)*

> “We implement this attention by associating a single scalar value, the attention gate g^t_i, with each fact f_i during pass t.”
>
> ✓ verified: found word for word in the paper's own text


**End-to-End Comparative Attention Networks for Person Re-Identification** (2016) \cite{liu2016end}

What it did: Applies recurrent LSTM-based attention to spatial masking for person re-identification  *(the tool's reading)*

This work implements recurrent attention via an LSTM that generates softmax-weighted spatial masks over CNN feature maps, enabling selective focus on discriminative local regions in person images. The approach maintains the recurrent glimpse paradigm while applying it to fine-grained feature selection rather than whole-image navigation.  *(the tool's reading)*

> “Briefly, similar to human visual processing, attention-based algorithms tend to selectively concentrate on a part of the information, and at the same time ignore other perceived information. Such a mechanism is usually called attention and can be employed to adaptively localize discriminative parts or regions of person images.”
>
> ✓ verified: found word for word in the paper's own text


**An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data** (2016) \cite{song2016end}

What it did: Extends recurrent attention to dual spatial-temporal gating in skeleton action recognition  *(the tool's reading)*

This paper frames attention as soft, content-dependent gates operating across both spatial dimensions (skeleton joints) and temporal dimensions (frames) within LSTM networks. The work generalizes recurrent attention beyond single-modality vision tasks to structured skeletal and temporal data for action understanding.  *(the tool's reading)*

> “With a soft attention mechanism, each joint within a frame is assigned a spatial attention weight based on the joint-selection gates. This enables our model to adaptively focus more on those discriminative joints.”
>
> ✓ verified: found word for word in the paper's own text


**Pedestrian Alignment Network for Large-scale Person Re-Identification** (2017) \cite{zheng2017pedestrian}

What it did: Replaces explicit recurrent LSTM attention with implicit CNN feature-based alignment  *(the tool's reading)*

This paper proposes that attention emerges implicitly from CNN training for identity classification without extra annotations or explicit LSTM/gating mechanisms. The work critiques the computational overhead of recurrent LSTM-based attention, shifting toward emergent alignment learned directly from the primary task.  *(the tool's reading)*

> “The proposed network thus takes advantage of this attention mechanism to adaptively locate and align pedestrians within a bounding box.”
>
> ✓ verified: found word for word in the paper's own text


**Attention-based Deep Multiple Instance Learning** (2018) \cite{ilse2018attention}

What it did: Reinterprets attention as permutation-invariant learned pooling for instance aggregation  *(the tool's reading)*

This paper abstracts attention away from sequential glimpse selection or spatial gating, reframing it as a trainable, permutation-invariant weighted-average pooling operator that aggregates instance embeddings. The mechanism replaces pre-defined pooling operators with a learnable two-layer neural network, moving attention from spatial/temporal selection to aggregate-level weighting.  *(the tool's reading)*

> “Interestingly, the proposed MIL pooling corresponds to a version of the attention mechanism […]. The main difference is that typically in the attention mechanism all instances are sequentially dependent while here we assume that all instances are independent.”
>
> ✓ verified: found word for word in the paper's own text


_[Your synthesis: what it enabled, what it left unsolved.]_


## Where this thread connects

Each crossing is where one line of work fed another. These are the tool's reading of
the corpus, not quotations.

- **Recurrent Glimpse-Based Visual Attention** → **Self-Attention & Transformers** (2017): Vision enters Transformer era