The role of 3D data in CNN-based object detection

Object detection is a challenging problem, typically addressed with artificial intelligence techniques which require extensive training and labelling of numerous images, especially in complex scenes. This paper investigates the use of three-dimensional (3D) information for object detection to enhance the performance of convolutional neural networks (CNN) conventionally trained with texture images. Our approach employed a fringe projection profilometry system as a machine vision system to acquire images of various objects, followed by training a CNN for single-shot object detection using both texture images and 3D images. Experimental results showed that training the CNN with 3D data markedly improved the average precision score from 76% with texture images to 91%. This illustrates that incorporating 3D data into CNN training significantly bolsters object detection precision, thereby underlining the critical importance and utility of 3D information in advancing machine vision systems.