Global Vision Media Focus
The company was featured on CCTV a total of 15 times, and its reach through Party media, central media, and local official media reached a total of 50 million people.
From MAE to Spectral Earth: A Fundamental Hyperspectral Model
Background Introduction
In recent years, with the breakthroughs in visual foundational models in the field of natural images, a question has gradually become clear: can hyperspectral remote sensing also build its own "foundational model"? Unlike RGB images, hyperspectral imagery typically contains hundreds of consecutive bands, possessing stronger physical meaning, higher dimensional redundancy, and scarcer labeled data. Traditional methods often rely on small-scale datasets for supervised training, resulting in limited model generalization ability and difficulty in cross-regional and cross-sensor transfer. Therefore, the hyperspectral field urgently needs a unified modeling paradigm capable of pre-training using massive amounts of unlabeled data.
Model framework
Against this backdrop, Masked Autoencoder (MAE) has become a crucial technical support for hyperspectral foundational models. First proposed by Meta, MAE's core idea is to randomly occlude a large proportion of the input image, encode only the visible portions, and then reconstruct the occluded content using a decoder. Unlike contrastive learning, MAE does not rely on similarity constraints between samples; instead, it forces the model to understand the internal structure of the data through a "reconstruction task." When more than 75% of the patch is occluded, the model must learn global spatial relationships and semantic organization rules, rather than simply memorizing local texture features. This mechanism gives MAE a powerful ability to learn structural representations.
Figure 1. MAE Structure Diagram
Spectral Earth, proposed within this framework, can be seen as a significant attempt to usher in a new era of fundamental models in the hyperspectral field. Based on the Vision Transformer architecture, this work learns a unified representation on large-scale hyperspectral data through self-supervised pre-training using MAE. Its core objective is not optimization for a single task, but rather the construction of transferable, generalized feature representations. After pre-training, the model was evaluated on multiple downstream remote sensing tasks, including land cover classification, crop type identification, and forest type mapping, validating the generalization ability of the pre-trained representations across different scenarios.
Figure 2. Spectral Earth pre-training/fine-tuning flowchart
Reproduction results
In downstream evaluation, Spectral Earth validated the model on multiple public datasets, including CORINE, CDL, NLCD, EuroCrops, TreeMap, BDForêt, and BNETD. Experimental results show that, under the Full Fine-Tuning strategy—that is, unfreezing all model parameters for overall fine-tuning—the ViT model pre-trained with MAE outperforms models trained from scratch on most tasks. This demonstrates that self-supervised pre-training does indeed provide hyperspectral models with more stable and transferable structural representation capabilities.
Table 1. Comparison of Reproduction Results
In the actual reproduction of MAE (ViT-S version) and its evaluation on downstream tasks, the benefits of pre-training are clearly evident. First, the model converges faster and performs more stably in small sample scenarios. Second, performance fluctuations are smaller on cross-regional datasets, indicating that it learns not specific data distribution features, but more general spectral structure information. Especially in crop identification and forest classification tasks, spectral modeling capability becomes a key factor in performance improvement.