Restriction lift date: 9999-01-01
Data processing for 3D time of flight depth imaging
Loading...
Files
Date
2021-06-26
Authors
Buckley, Steve
Journal Title
Journal ISSN
Volume Title
Publisher
University College Cork
Published Version
Abstract
In recent years, LiDAR has become used in an increasing number of ground based systems as a vital tool in the development of applications such as gaming, robotics and autonomous vehicles. As a result there is fast developing a requirement for large sensor arrays to provide high resolution 3D imaging. Due to the requirements of the next generation sensors, with pixel counts in excess of 106 (Mega-pixel Arrays), the data capture and processing techniques required to handle the resulting data is becoming more and more important. This work concentrates on this aspect of the sensor development by demonstrating solutions to read out both depth and intensity data for providing 3D plus greyscale (4Di) imaging. To investigate the various requirements of a Mega-pixel Array three LiDAR systems were developed. These were a single point ranging system, a long range 3D imaging system with scanning and a short range flash 3D and intensity imaging system (4Di).
The single point ranging module was designed as part of an EU funded project (INSPEX) to develop a smart cane for the visually impaired. This ranging device implemented histogram processing and a TDC (Time to Digital Converter) in an FPGA (Field Programmable Gate Array). As part of the histogram processing, a centre of gravity technique was implemented (Patent US 10,416,293,B2) to measure distances up to 10 m with the ability to detect variations down to ±1 mm.
The experience gained from this development was then taken forward into the development of a 3D imaging system capable of capturing images at a range in excess of 100 m with frame rates > 30 fps in high background light. This system utilised a bespoke 16x1 monolithic SiPM (Silicon Photomultiplier) sensor array and included the control of a mechanical scanner providing imaging with an angle of view of 80° in the X direction and 5° in the Y direction. Histogramming techniques were implemented to minimise the dead time between successive photon events to avoid pile-up due to high background noise. This design involved developing techniques for simultaneous histogram processing of pixels and data formatting for generating point cloud data.
From this, a third system was designed utilising a 400x100 SPAD (Single-Photon Avalanche Diode) array for short range flash imaging. This system was designed to image up to a distance of 10 m at frame rates > 100 fps. The design integrated 100 TDC and Histogramming channels plus configuration and readout data buses into a single FPGA. The combination of the SPAD array and FPGA provided a programmable autonomous imaging module that outputs both the 3D point cloud image plus intensity data (4Di).
The design of these three systems resulted in the development of histogramming and data processing techniques that could be taken forward when considering the design of next generation Mega-pixel Arrays. The development of the two 3D imaging systems also provided a good understanding of the different requirements and trade-offs for a sensor array based on the use case (short range flash and long range rolling shutter).
Finally, the issue of memory requirements for Mega-pixel arrays is addressed. For very large arrays, conventional histogramming techniques would result in excessive memory requirements that would not be practical to integrate into the sensor. Four techniques are demonstrated that compress these memory requirements. The four techniques, along with conventional histogramming, were implemented in a single FPGA with a common TDC. Each method was designed to work independently on the same common TDC data to provide a real-time true comparison. The performance of these different techniques are compared, and from the results, recommendations are made as to which technique should be adopted in the next generation Mega-pixel arrays based on three use cases of Short, Medium and Long range imaging.
Description
Keywords
LiDAR , SPAD , SiPM , Time of flight
Citation
Buckley, S. J. 2021. Data processing for 3D time of flight depth imaging. PhD Thesis, University College Cork.