A Review of Hyperspectral Image Classification: From CNN, Transformer to Mamba

From an application perspective, the value of hyperspectral technology lies in "distinguishing differences." Many objects appear similar in color in ordinary images, but may show significant differences in their spectral curves. For example, different crops, different vegetation conditions, different material types, and even some anomalies that are difficult to detect with the naked eye can all be revealed through spectral features. However, to truly transform this complex spectral information into understandable categorical results, hyperspectral technology is indispensable.Hyperspectral image classificationtechnology.

The task of hyperspectral image classification can be simply understood as assigning a category label to each pixel in an image based on its spectral features and spatial context. In hyperspectral remote sensing, this task typically manifests as pixel-level land cover classification, that is, determining the land cover category of each pixel based on its neighborhood information. While it may seem similar to ordinary image classification, it is actually more complex. Ordinary images usually only have three channels: red, green, and blue, while hyperspectral images often contain dozens or even hundreds of continuous bands. Each pixel is not just a color point, but a complete spectral curve. To better understand the development of hyperspectral image classification, it can be roughly summarized into the following stages:

Early hyperspectral classification methods primarily relied on traditional machine learning. Researchers typically treated the spectral curve of each pixel as a high-dimensional vector and then used methods such as support vector machines, random forests, and K-nearest neighbors for classification. These methods are clear in structure, easy to implement, and can achieve certain results even with small sample conditions. However, they mostly focus on the spectral features of a single pixel, making insufficient use of surrounding spatial information. For ground features with similar spectra but different spatial distributions, relying solely on single-point spectra can easily lead to misclassification.

With the development of deep learning, convolutional neural networks (CNNs) have begun to be introduced into hyperspectral classification tasks. The advantage of CNNs lies in their ability to automatically extract local spatial features, allowing the model to consider not just a single pixel but also its surrounding neighborhood in a comprehensive judgment. Two-dimensional convolution primarily focuses on spatial texture and local structure.3D convolutionIt can handle both spatial and spectral dimensions simultaneously, and therefore has been widely used in hyperspectral classification. Compared with traditional methods, CNNs have driven hyperspectral classification from simple spectral recognition to joint spatial-spectral modeling.

In terms of experimental verification, CNN-based hyperspectral classification models are often used.Indian PinesPublic datasets such as Pavia University, Salinas, and University of Houston are used. Among them, Indian Pines and Salinas are more focused on agricultural landforms, while Pavia University and University of Houston are more focused on urban landforms and complex land cover scenarios.

(a) Spectral convolutional neural network

(b) Spatial Convolutional Neural Network

(c) Spatial-spectral convolutional neural network

Figure 2.Three basic modeling methods for hyperspectral CNN classification models:

Spectral, spatial, and spatial-spectral joint feature extraction

However, CNNs also have certain limitations. Convolutional operations are better at extracting local features, but their ability to model long-range dependencies and global relationships is relatively limited. In hyperspectral images, there are often complex relationships between different bands and different regions. To address this issue...TransformerIt is gradually entering the field of hyperspectral classification. Relying on the attention mechanism, Transformer can model the correlation between different locations and different bands, making it more suitable for capturing global spatial-spectral dependencies.

Figure 3.Hyperspectral Transformer classification model structure:

Spatial-spectral feature modeling based on self-attention mechanism

The introduction of the Transformer enables hyperspectral classification to move beyond local feature extraction and further towards global relationship modeling. It helps the model focus on a wider range of information, handling complex ground feature boundaries, distinguishing similar categories, and...Long-range spectral dependenceWhile it has certain advantages, the Transformer also introduces new problems. Hyperspectral data has many bands and high dimensionality, so directly using the attention mechanism would significantly increase computational cost and GPU memory consumption. Furthermore, hyperspectral labeled samples are usually few, making complex models more prone to overfitting under small sample conditions.

In recent years, Mamba and state-space models have provided new approaches for hyperspectral classification. Unlike Transformers, Mamba-type methods emphasize efficient sequence modeling, capable of handling long sequence information with lower computational complexity. For hyperspectral images, the spectral dimension itself is a continuous sequence, and spatial regions can also be expanded into sequences through scanning. Therefore, Mamba/SSM-type structures possess efficient long sequence modeling capabilities, thus providing new ideas for modeling spectral sequences and spatial-spectral sequences.

Figure 4.SS-Mamba hyperspectral classification model structure:

State-space modeling based on spatial and spectral dual branches

From a developmental perspective, hyperspectral classification has gone through several distinct stages: traditional machine learning mainly addresses the problem of "how to classify spectral vectors"; CNNs address the problem of "how to fuse local spatial information"; Transformers focus on the problem of "how to establish global relationships"; and Mamba/SSM-like methods further explore "how to efficiently model long-range spatial-spectral dependencies." These methods are not simply substitutions, but rather solutions to different core challenges at different stages. In recent years, Mamba/SSM-like methods, in addition to traditional benchmark datasets, have gradually begun to validate their generalization capabilities on larger-scale or more complex datasets such as WHU-Hi-HanChuan, WHU-Hi-HongHu, and WHU-Hi-LongKou.

Despite continuous model improvements, hyperspectral classification still faces numerous challenges. First, there is insufficient sample labeling. Hyperspectral data labeling typically requires specialized knowledge and is costly to acquire. Second, there is the problem of class imbalance; some classes have a large number of pixels, while others are very sparse, causing models to tend to favor the majority class. Third, data varies significantly across different sensors, regions, and imaging conditions; a model that performs well on one dataset may show a significant decline in performance in another scenario.

Therefore, the future development direction of hyperspectral classification is not just about pursuing high accuracy on a single dataset, but also about focusing more on the model's generalization ability, lightweight capability, and adaptability to real-world scenarios. On the one hand, it is necessary to design more effective spatial-spectral joint modeling structures; on the other hand, it is also necessary to combine ideas such as few-shot learning, transfer learning, self-supervised learning, and base models to improve the model's adaptability to different data sources and tasks.

Overall, the development of hyperspectral image classification is essentially a process in which models continuously learn "how to interpret spectra, how to understand space, and how to adapt to complex terrain features." From traditional machine learning to CNNs, from Transformers to Mamba, hyperspectral classification is moving from simple pixel-level recognition to more intelligent, efficient, and generalized spatial-spectral interpretation.

[1] Roy S K, Krishna G, Dubey S R, et al. HybridSN: Exploring 3-D–2-D CNN feature hierarchy for hyperspectral image classification[J]. IEEE geoscience and remote sensing letters, 2019, 17(2): 277-281.

[2]  Hong D, Han Z, Yao J, Gao L, Zhang B, Plaza A, Chanussot J. SpectralFormer: Rethinking Hyperspectral Image Classification With Transformers. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1–15.

[3]  Yao J, Hong D, Li C, Chanussot J. SpectralMamba: Efficient Mamba for Hyperspectral Image Classification. arXiv:2404.08489, 2024.

[4] Wang G, Zhang X, Peng Z, Zhang T, Jiao L. S²Mamba: A Spatial-Spectral State Space Model for Hyperspectral Image Classification. IEEE Transactions on Geoscience and Remote Sensing, 2025.

[5]  Ahmad M, Butt M H F, Khan A M, et al. Spatial–spectral morphological mamba for hyperspectral image classification[J]. Neurocomputing, 2025, 636: 129995.

[6] He Y, Tu B, Jiang P, et al. IGroupSS-mamba: Interval group spatial–spectral mamba for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 1-17.