Global Vision Media Focus
The company was featured on CCTV a total of 15 times, and its reach through Party media, central media, and local official media reached a total of 50 million people.
A Discussion on the Application of Clay Model in Remote Sensing Segmentation
Background Introduction
With the continuous improvement of remote sensing image acquisition capabilities, tasks such as land cover classification, water body identification, farmland extraction, wetland monitoring, and urban impervious surface segmentation place higher demands on the generalization ability of models. Traditional semantic segmentation methods typically rely on large-scale manually labeled data and are often trained on a single data source or a single region, easily leading to performance degradation when applied across regions, sensors, and time phases. In contrast, base models, pre-trained on large-scale remote sensing data, can learn more general surface representations before being transferred to specific downstream tasks, thus exhibiting a significant advantage in the field of remote sensing segmentation. Clay is an open-source Earth observation base model proposed against this backdrop. Officially positioned as an "Earth-oriented base model," it can receive remote sensing images along with location and time information, and output feature representations with spatiotemporal semantics for subsequent classification, regression, change detection, and segmentation tasks.
From an application perspective, Clay is particularly suitable for remote sensing segmentation scenarios with limited labeled samples but clear business requirements. For example, in land use/cover mapping, it can be used as an encoder backbone, leveraging pre-trained general features to improve segmentation performance under small sample conditions; in cross-regional migration tasks, it can also alleviate the problem of "training in one region and failing in another." The official documentation also points out that Clay supports multiple sensor inputs, including Sentinel-2, Landsat, Sentinel-1, NAIP, LINZ, and MODIS, giving it strong adaptability in multi-source remote sensing applications.
Model framework
The overall framework of Clay v1.5 can be summarized as "dynamic embedding + spatiotemporal location encoding + MAE backbone + teacher constraints". First, the model generates patch representations based on the number of bands and wavelength information of the input image through the dynamic embedding module, thus supporting inputs from different sensors and different combinations of bands. Second, the model introduces location encoding, encodes ground resolution (GSD), latitude and longitude, and temporal information into the feature representation, enabling the model to not only "see" the spectral content but also understand the spatial location and temporal attributes of the image.
In its core, Clay employs a Vision Transformer-based Masked Autoencoder structure to reconstruct occluded input blocks. According to the official documentation, reconstruction loss accounts for approximately 95% of the total loss, primarily responsible for learning the spectral-spatial structure of the remote sensing data; the remaining 5% comes from representational constraints provided by the teacher network. v1.5 uses DINOv2 as the teacher to enhance the semantic expressiveness of the embedding space. In terms of model size, Clay v1.5 has approximately 632 million parameters, including approximately 311 million parameters for the encoder, approximately 15 million parameters for the decoder, and approximately 304 million parameters for the teacher network. The official documentation also discloses that this version of the model was trained on approximately 70 million globally distributed remote sensing chips, indicating a solid pre-training foundation.
Figure 1. Clay Model
In segmentation tasks, Clay does not directly output semantic segmentation maps, but is more suitable as a feature extraction backbone. The common approach is to retain its encoder and attach a lightweight segmentation decoder, such as a SegFormer-style decoder, FPN, or U-Net decoder, to recover spatial details through multi-layer feature fusion, ultimately outputting pixel-level classification results. This "base model + task head" approach balances pre-trained representation capabilities with downstream task adaptability, and better meets the requirements of remote sensing segmentation for boundary and multi-scale details.
Reproduction results
This experiment appended a SegFormer-like decoder after the Clay encoder, utilizing intermediate layer feature fusion to predict segmentation results. The experiment used approximately 2000 random samples for training and validation. After approximately 30 epochs of training, the validation set achieved...weighted IoU 0.869The result.
Figure 2. Experimental data
Clay can be used as a unified feature backbone for tasks such as land cover mapping, water body segmentation, forest and low vegetation identification, and road and impervious surface extraction. However, it should be noted that Clay's pre-training objective is essentially still representation learning and reconstruction. Therefore, for fine boundary characterization, appropriate segmentation decoders, multi-scale feature fusion, and post-processing strategies are still needed to obtain higher quality results in engineering applications.