FOSS4G-Asia 2023 Seoul

Automatic Image Registration of Aerial-Ground Cross-view Images based on Deep Learning Image Matching for Efficient Digital Twin Construction
11-30, 14:10–14:30 (Asia/Seoul), Seoul Archive

Driven by the interest in efficient monitoring and maintenance, digital twins are being established across various domains such as architecture, manufacturing, and urban planning. Constructing a digital twin requires diverse forms of data based on the target area and purpose. To visualize vast and complex regions, data acquisition is partitioned into aerial and ground levels. In the aerial level, aircraft and UAVs are utilized, while on the ground, methods like Mobile Mapping Systems (MMS) and Terrestrial Laser Scanners (TLS) are employed. However, the registration of cross-view images, such as aerial-ground images, is challenging due to differences in field of view, hindering smooth image matching. While manual registration through tie points established by humans is possible, it comes at a high cost and time, thereby impeding the efficiency of digital twin construction. Recent advancements in deep learning-based image matching are enhancing the potential for the registration of cross-view images. In this study, deep learning image matching is employed to automate the registration of cross-view images, contributing to efficient digital twin construction. Tie points are generated through deep learning image matching between nadir/oblique UAV images acquired via autonomous flights and ground images captured by terrestrial laser scanners. Utilizing these tie points, bundle adjustment is performed, and a subset of tie points is extracted based on reprojection error criteria. By incorporating these extracted tie points, image registration is performed again to produce the result. Experimental results demonstrate successful registration of previously unregistered ground images, and the addition of ground-exclusive areas enhanced the performance of the 3D model. This research is anticipated to contribute to the enhancement of efficiency and precision in digital twin construction across diverse domains, including spatial data analysis, 3D modeling, and national geospatial information systems.

Hwiyoung Kim is the leader of the Geospatial team at InnoPAM, a GeoAI company. He is involved in projects related to drone mapping, image georeferencing based on reference images, image local bundle adjustment, cross-view and multi-source image registration and 3D modeling. He is keen on the practical use and problem solving using GeoAI technology.

Junghoon Sung was born in Suwon, Korea, in 1994. He received a B.S.
degree in Urban Engineering from Hyupsung University, South Korea, in
2019. Currently, he is pursuing an M.S. degree in image processing at ChungAng University.