Point Cloud Segmentation for Building Reuse: Construction of digital twins in early phase building reuse projects

Point cloud processing has come a long way in the past years. Advances in computer vision (CV) and machine learning (ML) have enabled its automated recognition and processing. However, few of those developments have made it through to the Architecture, Engineering and Construction (AEC) industry. Here, optimizing those workflows can reduce time spent on early-phase projects, which otherwise could be spent on developing innovative design solutions. Simplifying the processing of building point cloud scans makes it more accessible and therefore, usable for design, planning and decision-making. Furthermore, automated processing can also ensure that point clouds are processed consistently and accurately, reducing the potential for human error. This work is part of a larger effort to optimize early-phase design processes to promote the reuse of vacant buildings. It focuses on technical solutions to automate the reconstruction of point clouds into a digital twin as a simplified solid 3D element model. In this paper, various ML approaches, among others KPConv Thomas et al. (2019), ShapeConv Cao et al. (2021) and Mask-RCNN He et al. (2017), are compared in their ability to apply semantic as well as instance segmentation to point clouds. Further it relies on the S3DIS Armeni et al. (2017), NYU v2 Silberman et al. (2012) and Matterport Ramakrishnan et al. (2021) data sets for training. Here, the authors aim to establish a workflow that reduces the effort for users to process their point clouds and obtain object-based models. The findings of this research show that although pure point cloud-based ML models enable a greater degree of flexibility, they incur a high computational cost. We found, that using RGB-D images for classifications and segmentation simplifies the complexity of the ML model but leads to additional requirements for the data set. These can be mitigated in the initial process of capturing the building or by extracting the depth data from the point cloud.