In this work, we demonstrate an innovative object detection framework based on depth and active infrared intensity images fusion with a time-of-flight (ToF) camera. A slide window weight fusion (SWWF) method provides fuse image with two modalities to localize targets. Then, the depth and intensity information is extracted to construct a joint feature space. Next, we utilize four machine learning methods to achieve object recognition. To verify this method, experiments are performed on an in-house dataset containing 1066 images, which are categorized into six different surface materials. Consequently, the approach performs well on localization with a 0.778 intersection over union (IoU). The best classification results are obtained with K-Nearest Neighbor (KNN) with a 98.01% total accuracy. Furthermore, our demonstrated method is less affected by various illumination conditions.