Elevating Autonomous Health Assessment to Combat Aging Infrastructures: Aerial Drone Photography Unveils Millimeter-Scale Bridge Deflections

What Significance of This Research?
Maintaining aging infrastructure is a global concern. In Japan, much of the infrastructure built during the economic boom of the '60s and '70s is now over 50 years old, including vast numbers of bridges, tunnels, and other vital components. In the last decade, Japan has witnessed several prominent infrastructure failures resulting in loss of life, such as the highway tunnel collapse in 2012 and the dam failure in 2018. With key infrastructure elements like bridges, tunnels, and sewage systems now over 50 years old, the risks of deterioration and accidents are pressing concerns. The deterioration of such infrastructure and the consequent accidents have emerged as critical issues, eliciting augmented government focus and heightened public vigilance. The prevailing shortfall of skilled labor in infrastructure maintenance exacerbates this challenge.
In response to this substantial challenge, the Ministry of Land, Infrastructure, Transport and Tourism in Japan developed a protocol in 2014. According to this protocol, bridges with a span exceeding 2 meters are subject to inspections at five-year intervals. The inspections consist of visual examinations conducted from vehicles during regular road patrols, designed to facilitate the early detection of structural damage.
Displacement Measurement Method using Digital Images
Concurrently, the National Institute of Advanced Industrial Science and Technology (AIST) in Japan has launched a research initiative aimed at the development of innovative sensing techniques for the monitoring of aging infrastructure. Supported by project grants (AIST strategic budget) and joint research (NEXCO East Japan, JR East Japan, Taiwan ITRI, etc.), our endeavor started to develop vision-based, high-precision displacement sensors for infrastructure assessment. The devised system encompasses a stationary camera mounted on a tripod, a set of test and reference markers affixed to the bridge girders, and displacement analysis framework employing the Sampling Moiré method 1,2. Over the course of the five-year project, a plethora of research outcomes were realized. Notably, we successfully gauged the deflection of an elevated bridge on the Shinkansen (Bullet train) in Japan 3 and a deteriorating concrete bridge in Taiwan 4, capturing images from a distance or beneath the bridge 5 to achieve sub-millimeter accuracy.
Despite achieving the initial goal, where the vision-based method rendered precision comparable to the conventional U-Doppler method, ongoing field experiments and comprehensive dialogues with our collaborators specializing in infrastructure maintenance have led to the recognition that reliance on stationary cameras is a considerable constraint for practical deployment, especially regarding critical structures like bridges spanning mountainous or aquatic environments.

Challenges and Expectations for Drone Camera Use
To overcome the drawbacks of using traditional stationary cameras, our team has turned to unmanned aerial vehicles (UAVs) as a potential solution. UAVs are expected to effectively resolve the difficulties of inconvenient camera mounting, particularly in challenging locations. Despite initial concerns regarding the practicality of using Unmanned Aerial Vehicles (UAVs) for sub-millimeter displacement measurements, the high precision of the sampling moiré method was reassuring. This method is distinguished by its ability to reliably detect displacements as small as 1/100th of a pixel in outdoor settings, a level of accuracy that is unrivaled by any other imaging technique we had at our disposal.
Inspiration from the Function of Human Ear
The crux of the issue lies in achieving highly accurate correction of drone image blur, reaching precision levels close to 1/100th of a pixel. Initially, we attempted to rectify image blurring using projective transformation, as suggested in prior studies, but this proved ineffective. Two months later, while reading a medical book at a café, I encountered a passage simplifying the workings of the human eye and ear. Humans can read distant billboard text even while in motion because the ear, equipped with three semicircular canals and two 6-axis sensors, continually and unconsciously balances and controls eye movements with high precision through muscles. Drawing an analogy, the gimbal function of a drone camera ensures smooth video capture with minimal motion blur. Inspired by this, we conceived the idea of equipping the drone with the equivalent of two "ears" by attaching reference markers on either side of the bridge near both girders, creating a stable ‘reference line’. By combining the center coordinates of these reference markers through image processing with pixel accuracy and the sampling moiré method with sub-pixel accuracy, we achieved accurate compensation of the reference line between the two reference points with an accuracy of 1/100th of a pixel.

Fig. 2 Deflection measurement of the bridge by using our proposed drone-based camera.
When we applied this method to inspect bridges in Kyoto, Japan, it yielded accurate deflection measurements for a 30 m truss bridge to a 110 m concrete bridge, aligning well with results from conventional sensors. To validate accuracy, we conducted verification at the Robot Test Field in Fukushima, Japan, using a real 35m scale bridge where the movement of a precise moving stage simulated deflection. The results confirmed that the average error over six cases was less than 0.2mm for movements ranging from 1mm to 5mm. Encouragingly, we are exploring the potential extension of this method to measure deflections in a 300m class suspension bridge.
Summary of This Research
Our ultimate visionary image is that the method developed in this research can enable drone-based infrastructure as a routine solution to combat the aging infrastructure faced by human society. The inspection drones perform the task autonomously, including precisely navigating to designated shooting locations over specific intervals and automatically measuring bridge deflections using internal GPS information. The proposed methodology could significantly shape the next generation of bridge inspection systems, allowing for autonomous processing, favorable flexibility, and preferred cost efficiency.
In the Future ...
The proposed approach would act as one integral technical component to new generation drone-based inspection techniques for infrastructure health assessment, allowing for autonomous processing, favorable flexibility, and preferred cost efficiency. The working image consists of inspection drones operating autonomously to execute their tasks, which include navigating precisely to predetermined locations at specific intervals and employing onboard GPS data to measure bridge deflections automatically. Our ultimate vision is that the approach developed through this research will establish drone-based infrastructure analysis as a routine solution to combat the issues presented by aging infrastructure and ensure societies' safety.
References
- Ri, S., Fujigaki, M. & Morimoto, Y. Sampling moiré method for accurate small deformation distribution measurement. Experimental Mechanics 50, 501-508 (2010).
- Ri, S. & Muramatsu, T. Theoretical error analysis of the sampling moiré method and phase compensation methodology for singe-shot phase analysis. Applied Optics 51, 3214-3223 (2012).
- Ri, S., Wang, Q., Tsuda, H., Shirasaki, H. & Kuribayashi, K. Displacement measurement of concrete bridges by the sampling Moiré method based on phase analysis of repeated pattern. Strain 56, e12351 (2020).
- Ri, S., Tsuda, H., Chang, K., Lo, F. & Lee, T. Dynamic deformation measurement by the sampling moiré method from video recording and its application to bridge engineering. Experimental Technique 44, 313-327 (2020).
- Ri, S., Wang, Q., Tsuda, H., Shirasaki, H. & Kuribayashi, K. Deflection measurement of bridge using images captured under the bridge by sampling moiré method. Experimental Technique 47, 1085-1095 (2023).
Thank you for reading!
Follow the Topic
-
Nature Communications
An open access, multidisciplinary journal dedicated to publishing high-quality research in all areas of the biological, health, physical, chemical and Earth sciences.
Related Collections
With collections, you can get published faster and increase your visibility.
Applications of Artificial Intelligence in Cancer
Publishing Model: Open Access
Deadline: Mar 31, 2025
Biology of rare genetic disorders
Publishing Model: Open Access
Deadline: Apr 30, 2025
Please sign in or register for FREE
If you are a registered user on Research Communities by Springer Nature, please sign in