State Space Models & Mamba

Structured State Space (SSM) models and the recent family of Mamba variants have accelerated research into long-range sequence modeling, efficient visual and multimodal representation, and alternatives to transformers. Below is a curated list of notable publications you provided, organized by topic with direct links to each source.

Core SSM theory & foundations

HiPPO: Recurrent Memory with Optimal Polynomial Projections — arXiv
Introduces the HiPPO framework for principled recurrent memory: projection operators that preserve function approximation under online updates. HiPPO underpins many later SSM developments.
Efficiently Modeling Long Sequences with Structured State Spaces — arXiv
The S4 family: shows how structured state space layers can be implemented to model very long-range dependencies efficiently, with strong empirical results on sequence tasks.
Simplified state space layers for sequence modeling — Smith, Warrington & Linderman (2022) — arXiv
Proposes simplifications to SSM-layer implementations to reduce complexity while retaining modeling power — practical guidance for lighter-weight SSMs.

Mamba family — linear-time and selective state spaces

Mamba: Linear-Time Sequence Modeling with Selective State Spaces — arXiv
Presents Mamba, a selective-state-space design that attains linear-time complexity while preserving the SSM inductive biases; focuses on efficiency and scalability for long sequences.
Mamba-3: Improved Sequence Modeling using State Space Principles — arXiv
Hungry hungry hippos: Towards language modeling with state space models — arXiv
Explores applying SSMs to large-scale language modeling, detailing practical training recipes, scaling behavior, and challenges for language tasks.
The hidden attention of Mamba models — Ali, Zimerman & Wolf (2024) — arXiv
Analyzes how attention-like computations emerge in Mamba architectures and examines interpretability/behavioral parallels with attention-based models.

Vision & multimodal SSM adaptations

Vision mamba: Efficient visual representation learning with bidirectional state space model — arXiv
Adapts bidirectional SSM layers for visual representation learning, trading off compute for larger receptive fields in images and vision tokens.
Vmamba: Visual state space model — NeurIPS 2024
NeurIPS presentation of a visual SSM variant; emphasizes architectural choices that make SSMs effective on image and patch-based inputs.
Vl-mamba: Exploring state space models for multimodal learning — arXiv
Examines extensions of Mamba-style SSMs to multimodal inputs (vision + language), discussing fusion strategies and scaling considerations.
Localmamba: Visual state space model with windowed selective scan — Springer
Presents a windowed selective-scan variant tailored for visual data — balancing local processing with selective long-range aggregation.

Applications & domain-specific SSMs

Rsmamba: Remote sensing image classification with state space model — IEEE
Applies an SSM/Mamba-inspired architecture to remote sensing imagery classification, highlighting robustness to multi-scale patterns in aerial data.

Broader surveys & theoretical connections

State space model for new-generation network alternative to transformers: A survey — arXiv
Survey covering SSMs as a class of transformer alternatives — architectures, algorithms, empirical comparisons, and open problems.
Transformers are ssms: Generalized models and efficient algorithms through structured state space duality — arXiv
Explores a formal duality between transformers and SSMs, deriving generalized models and algorithmic implications for efficient implementations.

Quick takeaways