SEAR: Simple and Efficient Adaptation of Visual Geometric Transformers for RGB+Thermal 3D Reconstruction

ECCV 2026

Vsevolod Skorokhodov1, Chenghao Xu2, Shuo Sun3, Olga Fink2, Malcolm Mielle1,
1Schindler EPFL Lab, 2EPFL, 3Örebro University
Paper arXiv Code Data 🤗 Weights

SEAR estimates camera poses of RGB and Thermal images together.

Abstract

Foundational feed-forward visual geometry models enable accurate and efficient camera pose estimation and scene reconstruction by learning strong scene priors from massive RGB datasets. However, their effectiveness drops when applied to mixed sensing modalities, such as RGB-thermal (RGB-T) images. We observe that while a visual ge- ometry grounded transformer pretrained on RGB data generalizes well to thermal-only reconstruction, it struggles to align RGB and thermal modalities when processed jointly. To address this, we propose SEAR, a simple yet efficient fine-tuning strategy that adapts a pretrained geometry transformer to multimodal RGB-T inputs. Despite being trained on a relatively small RGB-T dataset, our approach significantly outperforms state-of-the-art methods for 3D reconstruction and camera pose esti- mation, achieving significant improvements over all metrics (e.g., over 29% in AUC@30) and delivering higher detail and consistency between modalities with negligible overhead in inference time compared to the original pretrained model. Notably, SEAR enables reliable multimodal pose estimation and reconstruction even under challenging conditions, such as low lighting and dense smoke. We validate our architecture through extensive ablation studies, demonstrating how the model aligns both modalities. Additionally, we introduce a new dataset featuring RGB and thermal sequences captured at different times, viewpoints, and il- lumination conditions, providing a robust benchmark for future work in multimodal 3D scene reconstruction.

Observation

RGB Thermal RGB+Thermal
First GIF Second GIF Third GIF
VGGT, pretrained on RGB data, generalizes well to thermal-only reconstruction, but struggles to align RGB and thermal modalities when they are processed jointly. This suggests that neither large-scale RGB-T datasets nor complex fine-tuning strategies are necessary for adapting VGGT to RGB-T inputs. Instead, we show that a small RGB-T dataset, combined with a simple fine-tuning strategy, is enough to effectively adapt the pretrained model to the new modality.

Comparison

Qualitative Results

COLMAP
MAELoFTR
MINIMAROMA
MP-SfM
DUSt3R
MASt3R
VGGT
MapAnything
SEAR
(Ours)
Ground Truth
Metallic Container
COLMAP Result
MA ELoFTR Result
MINIMA ROMA Result
MP-SfM
DUSt3R Result
MASt3R Result
VGGT Result
MapAnything
SEAR Result
Ground Truth
INR-Building
MA ELoFTR Result
MINIMA ROMA Result
DUSt3R Result
MP-SfM
MASt3R Result
VGGT Result
SEAR Result
MapAnything Result
Ground Truth
COLMAP Result
01_Annexet_No_
Radars_0
MA ELoFTR Result
MINIMA ROMA Result
DUSt3R Result
MP-SfM
MASt3R Result
VGGT Result
SEAR Result
MapAnything Result
Ground Truth
COLMAP Result
SmokeSeer
drone-bathroom
COLMAP
MA ELoFTR Result Result
MINIMA ROMA Result
MP-SfM Result
MASt3R Result
VGGT Result
SEAR Result
MapAnything Result
Ground Truth
COLMAP Result
Parterre
MA ELoFTR Result
MINIMA ROMA Result
DUSt3R Result
MP-SfM Result
MASt3R Result
VGGT Result
SEAR Result
MapAnything Result
Ground Truth
COLMAP Result
laptop
MA ELoFTR Result
MINIMA ROMA Result
DUSt3R Result
MP-SfM Result
MASt3R Result
VGGT Result
SEAR Result
MapAnything Result
Ground Truth
COLMAP Result
generator
MA ELoFTR Result
MINIMA ROMA Result
DUSt3R Result
MP-SfM Result
MASt3R Result
VGGT Result
SEAR Result
MapAnything Result
Ground Truth
COLMAP Result
Click on any image to view it in full size.

Interactive Results

Loading Point Clouds...

Left Click: Rotate | Right Click: Pan | Scroll: Zoom

Quantitative Results

Method Public Datasets SEAR Dataset
AUC ↑ RRA ↑ RTA ↑ PCA ↓ PCC ↓ Chamfer ↓ Reg (%) ↑ FPS ↑ AUC ↑ RRA ↑ RTA ↑ Reg (%) ↑ FPS ↑
COLMAP
+SPSG		 57.6 82.5 74.6 1.64 1.20 1.42 44.7 0.44 74.4 99.9 95.9 27.9 0.66
MAELoFTR 13.1 70.1 41.2 0.49 5.16 2.82 21.8 0.21 15.9 73.7 53.1 37.3 0.17
MINIMAROMA 41.0 68.3 63.0 0.97 1.09 1.03 98.7 0.05 48.2 65.9 68.9 100.0 0.04
MP-SfM 31.4 78.0 52.5 1.59 0.58 1.08 100.0 0.04 - - - - -
DUSt3R 18.9 45.9 42.1 0.72 4.72 2.72 100.0 0.66 18.7 51.0 39.2 100.0 0.67
MASt3R 30.8 69.7 55.8 0.66 0.27 0.46 100.0 0.24 39.1 54.9 56.3 100.0 0.25
VGGT 22.9 50.7 48.5 1.22 2.60 1.91 100.0 10.46 23.3 50.5 56.4 100.0 10.51
MapAnything 21.4 51.3 47.4 0.69 3.76 2.23 100.0 1.98 23.2 51.4 52.2 100.0 2.12
SEAR 70.0 90.6 87.6 0.47 0.06 0.27 100.0 9.94 62.8 83.7 84.2 100.0 10.22

Features Alignment

Features Alignment.

Features Alignment. PCA-based analysis of RGB-thermal feature alignment. The top row illustrates the feature evolution of our model on RGB-thermal inputs, while the bottom row shows the feature evolution of VGGT on the corresponding RGB-only inputs.

Our analysis suggests that the model does not learn a completely new representation for mixed-modality inputs, but instead operates largely within the original VGGT feature space. We run SEAR on RGB-thermal inputs and the original VGGT on the corresponding RGB-only images, and extract the intermediate outputs of the frame-attention layers in the AA modules. We then perform Principal Component Analysis (PCA) on features from the same layer and visualize the resulting embeddings in the figure above. It can be observed that, at each layer, the RGB features produced by SEAR and the original pre-trained VGGT model exhibit very similar structures. The layers 12-14 clearly show how our model aligns RGB and thermal features. After layer14, the combined RGB and thermal features closely resembles the RGB features produced by the pre-trained VGGT model for RGB-only images. Interestingly, VGGT reconstructs geometry starting from its middle layers, exactly where the RGB and thermal features are aligned in SEAR. For more details, we encourage readers to refer to the paper and the supplementary material.

Method

SEAR Method.

SEAR architecture. RGB and thermal images are first tokenized using DINOv2. For each modality, camera-specific tokens are concatenated with the corresponding DINO tokens. The combined tokens are then processed by an Alternating-Attention (AA) module with LoRA adapters. Finally, the refined tokens are passed to separate prediction heads for camera parameter estimation and depth estimation. Trainable parameters are highlighted with a flame symbol.

*Other Inputs

Other Inputs. The picture is a carousel. Tap to activate scrolling.

This result was not presented in the paper and in the supplementary material, but we found that VGGT can also handle many other inputs (RGB, Red-only, Inverse, Random 3x3 Convolution, Emboss, Gradient, magma, viridis, cool, hsv, prism, PiYG, Global Normals).

BibTeX


    @misc{skorokhodov2026searsimpleefficientadaptation,
      title={SEAR: Simple and Efficient Adaptation of Visual Geometric Transformers for RGB+Thermal 3D Reconstruction},
      author={Vsevolod Skorokhodov and Chenghao Xu and Shuo Sun and Olga Fink and Malcolm Mielle},
      year={2026},
      eprint={2603.18774},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2603.18774},
    }