FOSS4G-Asia 2023 Seoul

The point cloud is becoming a valuable resource for building Digital Twins, the virtual representation of a real-world physical feature, with the LiDAR developments. The semantic information of each point (such as walls, floors, doors, windows, etc.) is essential to handle and analyzing point clouds. However, the raw point cloud captured from the LiDAR sensor doesn’t have semantics, and the annotation task for adding semantics is time-consuming and needs expert experience with commercial software. Therefore, we set up two goals for accelerating the 3D point clouds annotation task:
- Machine-based assistants, such as AI techniques, and
- Task-based collaboration.

Recently, deep learning has been used to derive semantic classes by automated classification and segmentation. Therefore, it can be used to address the manual task (bottleneck) in the traditional annotation process. Also, the data format for deploying is essential for collaboration. It should be easy to understand and efficient to share.

PCAS (Point Cloud Annotation System) is a 3D point cloud annotation system that enhances the quality and speed of annotation tasks with deep learning techniques. This system can be summarized as four key points:
- This system mainly develops based on Potree[1], an open-source library for point cloud visualization. It also utilizes various open sources, such as open3d[2] and torch-points3d[3], for handling point clouds with deep learning modules.
- This system supports three tools to accelerate annotation work: semi-automatic labeling with a deep learning module, 3D shape object-based labeling, and normal vector-based cluster labeling.
- The annotation task can be easily shared and tracked, similar to GitHub.
- The annotation task results are stored as an HDF5 file based on a predefined profile[4] for the labeled point clouds, easily sharing and managing semantic information.
This talk will introduce the four main points above, including the structure of PCAS with other open sources, the design purpose, and problems and solutions encountered during development.

References:
[1] Potree, WebGL point cloud viewer for large datasets, https://github.com/potree/potree
[2] torch-points3d, Pytorch framework for doing deep learning on point clouds, https://github.com/torch-points3d/torch-points3d
[3] Open3D, A Modern Library for 3D Data Processing, https://github.com/isl-org/Open3D
[4] The HDF5 profile for labeled point cloud data, https://docs.ogc.org/dp/21-077.html