To promote wider application of logistics robotics throughout the industry, two innovative 3D imaging systems are developed in this program to specifically deal with two pressing problems commonly faced in industrial scenarios. First, it's the Portable 3D Imaging System for moving objects with unconventional lightweight, lithography color filter. Its brand-new projection module enables image reconstruction up to 1 million pixels in only a single shot. Second, it's the 3D Image Reconstruction System for Transparent Objects equipped with the new laser coplanar algorithm based on the analysis of optical path of multiple reflections on transparent surfaces. These two systems open up a world of possibilities for a more extensive application of logistic robotics.

Sensing and Perception serve as the prerequisite requirements for any autonomous robotic applications in different logistics scenarios. In this project, item grasping and picking are the major focuses with a view to enhance the 3D localization accuracy of robotic arms and thereby improving logistic robotics’ overall performance. Through inputting auto-generated/captured data to learning-based framework, a software package for robust sensing and perception has been developed with high accuracy and efficiency. Not only does the software bring breakthroughs in two main areas, namely object segmentation/detection and object 6D pose estimation, it can also overcome the often-faced challenges, like segmenting closely-stacked items and localizing accurately even under severe occlusion.

Raw point-clouds produced from depth cameras and LiDAR sensors are often sparse, noisy, and non-uniform. Hence, the raw data is required to be amended, before it can be effectively used for rendering, analysis, or general processing. In this project, we developed deep learning based neural networks to enable automatic point-cloud upsampling. It allows for generating a point- cloud that is dense, complete, and uniform, as a faithful representation of the underlying surface. To enable an accurate 3D reconstruction, we also developed an edge-aware technique to facilitate the consolidation of point- clouds. Results demonstrate that our techniques can produce high-quality point-clouds with uniform point distribution and facilitate high-quality 3D mesh reconstruction.

3D point-cloud segmentation is an important task for 3D scene understanding. Given the set of points, the objective of segmentation is to cluster points with similar characteristics into homogeneous and meaningful regions. Semantic segmentation treats multiple objects of the same class as a single entity, while instance segmentation treats multiple objects of the same class as distinct individual objects. The segmentation results are helpful for analyzing the 3D scene in various aspects, such as locating objects, scene editing, etc. In this project, we aim to develop algorithms for both semantic and instance segmentation with high accuracy and real-time efficiency.