Description
Hyperspectral imaging provides dense spectral information for each pixel, making it a powerful tool for material identification and scene understanding. However, standalone hyperspectral data may fall short in representing spatial geometry, especially in complex environments such as geological structures. To address this limitation, we investigate the integration of multiple sensors capturing complementary information, specifically combining hyperspectral and 3D point cloud data, to improve segmentation performance in mineral mapping tasks. This work explores advanced sensor fusion techniques for deep learning-based segmentation of hyperspectral point clouds. Our approach focuses on training a Residual Multi-Layer Perceptron (Res-MLP) model using three key sensor fusion strategies: early fusion, intermediate fusion, and late fusion. Each strategy reflects a different stage of feature integration within the network architecture. Early fusion merges raw features from both modalities before any learning process, allowing the model to jointly learn low-level correlations. Intermediate fusion combines feature representations at deeper layers, aiming to capture higher-order dependencies. Late fusion integrates predictions or embeddings at the decision level, leveraging ensemble-like behavior to refine segmentation outcomes. By leveraging the complementary characteristics of hyperspectral imaging (rich spectral signatures) and 3D point clouds (detailed spatial geometry), the proposed fusion-based Res-MLP model achieves significant improvements in segmentation accuracy. Extensive experiments are conducted on the Tinto dataset, a well-established benchmark for geological mapping and mineral exploration. The dataset provides fully-labeled hyperspectral point cloud data collected over an open-pit mine. Our evaluation includes both qualitative visual analysis and quantitative metrics such as overall accuracy. Results consistently show that fusion-based models outperform single-modality approaches, confirming the hypothesis that sensor fusion significantly enhances deep learning performance in this domain. Additionally, the Res-MLP architecture proves to be lightweight and computationally efficient, making it suitable for large-scale or real-time applications in field geology and remote sensing. This study underscores the importance of integrating diverse sensing modalities in remote sensing workflows. As sensor technology continues to evolve and data availability grows, such fusion techniques will play a crucial role in advancing automated analysis, particularly in domains like geological mapping, mineral exploration, and earth observation. Our findings pave the way for future research into scalable and generalizable fusion architectures, and highlight the potential of combining spectral and spatial modalities for more intelligent environmental perception systems.
