Frozen DINOv2–CLIP Fusion and Binary Hashing for Remote Sensing Image Retrieval: A Controlled Evaluation
DOI:
https://doi.org/10.62411/jcta.16661Keywords:
Binary hashing, CLIP, Content-based image retrieval, DINOv2, Clinically Weighted Feature Fusion, Iterative quantization, Remote sensing, Vision foundation modelsAbstract
This study evaluates a controlled content-based remote sensing image retrieval pipeline that combines frozen vision foundation models with binary hashing. Across five stratified seeds on PatternNet and NWPU-RESISC45, calibrated DINOv2–CLIP score fusion achieved mAP values of 0.8143 ± 0.0007 and 0.5693 ± 0.0020, respectively, improving over the stronger single backbone by 0.0582 and 0.0514. The two similarity spaces were related but not redundant, with Spearman correlations of ρ = 0.510 and 0.515 across 200,000 sampled pairs. The contribution of texture features was dataset- and backbone-dependent: DINOv2–texture received a weight of 0.1 on PatternNet, whereas CLIP–texture and three-stream fusion assigned zero weight to texture; forcing texture into a 32-bit representation reduced mAP by 0.0147 and 0.0510. Among the unsupervised hashing methods, 64-bit iterative quantization (ITQ) exceeded the full-precision fused baseline by 0.0533 and 0.0445 mAP on PatternNet and NWPU-RESISC45, respectively. The corresponding paired parametric tests were significant, although the exact five-pair sign-flip test remained resolution-limited (p = 0.0625). A supervised CSQ-style head achieved mAP values of 0.9886 and 0.9087 at 64 bits; however, this improvement cannot be attributed to compression alone because gallery labels were used during training. In a controlled 200,000-vector benchmark constructed through deterministic repetition, 64-bit flat Hamming search required 1.6 MB and 0.674 ms/query, compared with 1.02 GB and 5.343 ms/query for 1,280-dimensional float vectors. Overall, the results support DINOv2–CLIP fusion and 64–128-bit ITQ as practical label-free choices, while restricting the scalability claim to the tested flat-search setting.References
B. Demir and L. Bruzzone, “Hashing-Based Scalable Remote Sensing Image Search and Retrieval in Large Archives,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 2, pp. 892–904, Feb. 2016, doi: 10.1109/TGRS.2015.2469138.
G. Cheng, X. Xie, J. Han, L. Guo, and G.-S. Xia, “Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 13, pp. 3735–3756, 2020, doi: 10.1109/JSTARS.2020.3005403.
W. Zhou, S. Newsam, C. Li, and Z. Shao, “PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval,” ISPRS J. Photogramm. Remote Sens., vol. 145, pp. 197–209, Nov. 2018, doi: 10.1016/j.isprsjprs.2018.01.004.
G. Cheng, J. Han, and X. Lu, “Remote Sensing Image Scene Classification: Benchmark and State of the Art,” Proc. IEEE, vol. 105, no. 10, pp. 1865–1883, Oct. 2017, doi: 10.1109/JPROC.2017.2675998.
A. Babenko, A. Slesarev, A. Chigorin, and V. Lempitsky, “Neural Codes for Image Retrieval,” in Lecture Notes in Computer Science, 2014, pp. 584–599. doi: 10.1007/978-3-319-10590-1_38.
A. Gordo, J. Almazán, J. Revaud, and D. Larlus, “Deep Image Retrieval: Learning Global Representations for Image Search,” in Lecture Notes in Computer Science, 2016, pp. 241–257. doi: 10.1007/978-3-319-46466-4_15.
F. Radenovic, G. Tolias, and O. Chum, “Fine-Tuning CNN Image Retrieval with No Human Annotation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 41, no. 7, pp. 1655–1668, Jul. 2019, doi: 10.1109/TPAMI.2018.2846566.
M. Oquab et al., “DINOv2: Learning Robust Visual Features without Supervision,” Trans. Mach. Learn. Res., Feb. 2024, doi: 10.48550/arxiv.2304.07193.
A. Radford et al., “Learning Transferable Visual Models from Natural Language Supervision,” in Proceedings of the International Conference on Machine Learning, 2021, vol. 139, pp. 8748–8763. [Online]. Available: https://proceedings.mlr.press/v139/radford21a.html
M. Caron et al., “Emerging Properties in Self-Supervised Vision Transformers,” in 2021 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2021, pp. 9630–9640. doi: 10.1109/ICCV48922.2021.00951.
M. Cherti et al., “Reproducible Scaling Laws for Contrastive Language-Image Learning,” in 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2023, pp. 2818–2829. doi: 10.1109/CVPR52729.2023.00276.
A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” in ICLR 2021, Jun. 2021. [Online]. Available: http://arxiv.org/abs/2010.11929
A. Vaswani et al., “Attention Is All You Need,” in 31st Conference on Neural Information Processing Systems (NIPS 2017), 2017. [Online]. Available: http://arxiv.org/abs/1706.03762
K. He, X. Chen, S. Xie, Y. Li, P. Dollar, and R. Girshick, “Masked Autoencoders Are Scalable Vision Learners,” in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2022, pp. 15979–15988. doi: 10.1109/CVPR52688.2022.01553.
X. Tan, Y. Zou, Z. Guo, K. Zhou, and Q. Yuan, “Deep Contrastive Self-Supervised Hashing for Remote Sensing Image Retrieval,” Remote Sens., vol. 14, no. 15, p. 3643, Jul. 2022, doi: 10.3390/rs14153643.
Y. Sun et al., “Unsupervised deep hashing through learning soft pseudo label for remote sensing image retrieval,” Knowledge-Based Syst., vol. 239, p. 107807, Mar. 2022, doi: 10.1016/j.knosys.2021.107807.
Y. Zhang, X. Zheng, and X. Lu, “Remote Sensing Image Retrieval by Deep Attention Hashing With Distance-Adaptive Ranking,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 16, pp. 4301–4311, 2023, doi: 10.1109/JSTARS.2023.3271303.
F. Liu et al., “RemoteCLIP: A Vision Language Foundation Model for Remote Sensing,” IEEE Trans. Geosci. Remote Sens., vol. 62, pp. 1–16, 2024, doi: 10.1109/TGRS.2024.3390838.
Y. Cong et al., “SatMAE: Pre-training Transformers for Temporal and Multi-spectral Satellite Imagery,” in Advances in Neural Information Processing Systems, 2022, vol. 35, pp. 197–211. [Online]. Available: https://doi.org/10.52202/068431-0015
M. Mendieta, B. Han, X. Shi, Y. Zhu, and C. Chen, “Towards Geospatial Foundation Models via Continual Pretraining,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2023, pp. 16760–16770. doi: 10.1109/ICCV51070.2023.01541.
J. Johnson, M. Douze, and H. Jegou, “Billion-Scale Similarity Search with GPUs,” IEEE Trans. Big Data, vol. 7, no. 3, pp. 535–547, Jul. 2021, doi: 10.1109/TBDATA.2019.2921572.
Y. A. Malkov and D. A. Yashunin, “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 4, pp. 824–836, Apr. 2020, doi: 10.1109/TPAMI.2018.2889473.
Y. Gong, S. Lazebnik, A. Gordo, and F. Perronnin, “Iterative Quantization: A Procrustean Approach to Learning Binary Codes for Large-Scale Image Retrieval,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 12, pp. 2916–2929, Dec. 2013, doi: 10.1109/TPAMI.2012.193.
J. Wang, T. Zhang, J. Song, N. Sebe, and H. T. Shen, “A Survey on Learning to Hash,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 4, pp. 769–790, Apr. 2018, doi: 10.1109/TPAMI.2017.2699960.
X. Luo et al., “A Survey on Deep Hashing Methods,” ACM Trans. Knowl. Discov. Data, vol. 17, no. 1, pp. 1–50, Feb. 2023, doi: 10.1145/3532624.
A. Gionis, P. Indyk, and R. Motwani, “Similarity Search in High Dimensions via Hashing,” in Proceedings of the International Conference on Very Large Data Bases, 1999, pp. 518–529. [Online]. Available: https://www.vldb.org/conf/1999/P49.pdf
M. S. Charikar, “Similarity estimation techniques from rounding algorithms,” in Proceedings of the thiry-fourth annual ACM symposium on Theory of computing, May 2002, pp. 380–388. doi: 10.1145/509907.509965.
H. Noh, A. Araujo, J. Sim, T. Weyand, and B. Han, “Large-Scale Image Retrieval with Attentive Deep Local Features,” in 2017 IEEE International Conference on Computer Vision (ICCV), Oct. 2017, pp. 3476–3485. doi: 10.1109/ICCV.2017.374.
Y. Li, Y. Zhang, X. Huang, H. Zhu, and J. Ma, “Large-Scale Remote Sensing Image Retrieval by Deep Hashing Neural Networks,” IEEE Trans. Geosci. Remote Sens., vol. 56, no. 2, pp. 950–965, Feb. 2018, doi: 10.1109/TGRS.2017.2756911.
S. Roy, E. Sangineto, B. Demir, and N. Sebe, “Metric-Learning-Based Deep Hashing Network for Content-Based Retrieval of Remote Sensing Images,” IEEE Geosci. Remote Sens. Lett., vol. 18, no. 2, pp. 226–230, Feb. 2021, doi: 10.1109/LGRS.2020.2974629.
L. Yuan et al., “Central Similarity Quantization for Efficient Image and Video Retrieval,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun. 2020, pp. 3080–3089. doi: 10.1109/CVPR42600.2020.00315.
C. D. Manning, P. Raghavan, and H. Schütze, Introduction to Information Retrieval. Cambridge University Press, 2008. doi: 10.1017/CBO9780511809071.
L. Fan, H. Zhao, and H. Zhao, “Distribution Consistency Loss for Large-Scale Remote Sensing Image Retrieval,” Remote Sens., vol. 12, no. 1, p. 175, Jan. 2020, doi: 10.3390/rs12010175.
P. Liu, X. Liu, Y. Wang, Z. Liu, Q. Zhou, and Q. Li, “An Intra-Class Ranking Metric for Remote Sensing Image Retrieval,” Remote Sens., vol. 15, no. 16, p. 3943, Aug. 2023, doi: 10.3390/rs15163943.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Jumi J, Tedjo Mulyono, Achmad Zaenuddin, Suwardi Suwardi
This work is licensed under a Creative Commons Attribution 4.0 International License.
