Research on Open−pit Mine Road Obstacle Detection Method Based on Thermal Infrared and RGB Camera Fusion

Aiming at the drawbacks of single−modal object detection algorithms for road obstacle detection under harsh low−visibility operating conditions in open−pit mines—including dim nighttime illumination and high−concentration dust, which degrade image quality, reduce detection accuracy, and weaken model robustness—this paper proposes a visible−light and thermal infrared fused road obstacle detection algorithm for open−pit mines based on iterative GPT−Fusion. A deep network−driven local feature matching algorithm is adopted to realize feature registration between the two types of images. A cross−modal two−branch feature fusion network is constructed, where the GPT−Fusion module is embedded in the feature fusion stage. This module reorganizes two−dimensional visual features into one−dimensional token sequences to establish global contextual dependencies and conduct deep cross−modal feature interaction. Meanwhile, a small−object receptive field enhancement structure is embedded into the network, which significantly boosts the detection capability for distant tiny obstacles via a multi−branch dilated convolution mechanism. Finally, a spatial context pyramid module is integrated into the detection prediction head to adaptively aggregate spatial and channel−wise information, thereby effectively mitigating cross−modal discrepancies. Experimental results demonstrate that the improved model achieves superior detection performance on a self−built bimodal open−pit mine dataset, with a mean average precision mAP₅₀ of 89.4% and an average recall of 88.7%. The overall performance outperforms state−of−the−art single−modal and fusion networks, satisfying the safe obstacle detection requirements of unmanned mining trucks in complex and variable open−pit mine environments.