Elevating Autonomous Health Assessment to Combat Aging Infrastructures: Aerial Drone Photography Unveils Millimeter-Scale Bridge Deflections

Elevating Autonomous Health Assessment to Combat Aging Infrastructures:  Aerial Drone Photography Unveils Millimeter-Scale Bridge Deflections
Like

Share this post

Choose a social network to share with, or copy the URL to share elsewhere

This is a representation of how your post may appear on social media. The actual post will vary between social networks

What Significance of This Research?

Maintaining aging infrastructure is a global concern. In Japan, much of the infrastructure built during the economic boom of the '60s and '70s is now over 50 years old, including vast numbers of bridges, tunnels, and other vital components. In the last decade, Japan has witnessed several prominent infrastructure failures resulting in loss of life, such as the highway tunnel collapse in 2012 and the dam failure in 2018. With key infrastructure elements like bridges, tunnels, and sewage systems now over 50 years old, the risks of deterioration and accidents are pressing concerns. The deterioration of such infrastructure and the consequent accidents have emerged as critical issues, eliciting augmented government focus and heightened public vigilance. The prevailing shortfall of skilled labor in infrastructure maintenance exacerbates this challenge.

In response to this substantial challenge, the Ministry of Land, Infrastructure, Transport and Tourism in Japan developed a protocol in 2014. According to this protocol, bridges with a span exceeding 2 meters are subject to inspections at five-year intervals. The inspections consist of visual examinations conducted from vehicles during regular road patrols, designed to facilitate the early detection of structural damage.

Displacement Measurement Method using Digital Images

Concurrently, the National Institute of Advanced Industrial Science and Technology (AIST) in Japan has launched a research initiative aimed at the development of innovative sensing techniques for the monitoring of aging infrastructure. Supported by project grants (AIST strategic budget) and joint research (NEXCO East Japan, JR East Japan, Taiwan ITRI, etc.), our endeavor started to develop vision-based, high-precision displacement sensors for infrastructure assessment. The devised system encompasses a stationary camera mounted on a tripod, a set of test and reference markers affixed to the bridge girders, and displacement analysis framework employing the Sampling Moiré method 1,2. Over the course of the five-year project, a plethora of research outcomes were realized. Notably, we successfully gauged the deflection of an elevated bridge on the Shinkansen (Bullet train) in Japan 3 and a deteriorating concrete bridge in Taiwan 4, capturing images from a distance or beneath the bridge 5 to achieve sub-millimeter accuracy.

Despite achieving the initial goal, where the vision-based method rendered precision comparable to the conventional U-Doppler method, ongoing field experiments and comprehensive dialogues with our collaborators specializing in infrastructure maintenance have led to the recognition that reliance on stationary cameras is a considerable constraint for practical deployment, especially regarding critical structures like bridges spanning mountainous or aquatic environments.

Conventional camera
Fig. 1  Deflection measurement of the bridge by using a camera fixed on a tripod on the ground.

Challenges and Expectations for Drone Camera Use

To overcome the drawbacks of using traditional stationary cameras, our team has turned to unmanned aerial vehicles (UAVs) as a potential solution. UAVs are expected to effectively resolve the difficulties of inconvenient camera mounting, particularly in challenging locations. Despite initial concerns regarding the practicality of using Unmanned Aerial Vehicles (UAVs) for sub-millimeter displacement measurements, the high precision of the sampling moiré method was reassuring. This method is distinguished by its ability to reliably detect displacements as small as 1/100th of a pixel in outdoor settings, a level of accuracy that is unrivaled by any other imaging technique we had at our disposal.

Inspiration from the Function of Human Ear

The crux of the issue lies in achieving highly accurate correction of drone image blur, reaching precision levels close to 1/100th of a pixel. Initially, we attempted to rectify image blurring using projective transformation, as suggested in prior studies, but this proved ineffective. Two months later, while reading a medical book at a café, I encountered a passage simplifying the workings of the human eye and ear. Humans can read distant billboard text even while in motion because the ear, equipped with three semicircular canals and two 6-axis sensors, continually and unconsciously balances and controls eye movements with high precision through muscles. Drawing an analogy, the gimbal function of a drone camera ensures smooth video capture with minimal motion blur. Inspired by this, we conceived the idea of equipping the drone with the equivalent of two "ears" by attaching reference markers on either side of the bridge near both girders, creating a stable ‘reference line’. By combining the center coordinates of these reference markers through image processing with pixel accuracy and the sampling moiré method with sub-pixel accuracy, we achieved accurate compensation of the reference line between the two reference points with an accuracy of 1/100th of a pixel.

Drone camera setup

Fig. 2 Deflection measurement of the bridge by using our proposed drone-based camera.

When we applied this method to inspect bridges in Kyoto, Japan, it yielded accurate deflection measurements for a 30 m truss bridge to a 110 m concrete bridge, aligning well with results from conventional sensors. To validate accuracy, we conducted verification at the Robot Test Field in Fukushima, Japan, using a real 35m scale bridge where the movement of a precise moving stage simulated deflection. The results confirmed that the average error over six cases was less than 0.2mm for movements ranging from 1mm to 5mm. Encouragingly, we are exploring the potential extension of this method to measure deflections in a 300m class suspension bridge.

Summary of This Research

Our ultimate visionary image is that the method developed in this research can enable drone-based infrastructure as a routine solution to combat the aging infrastructure faced by human society. The inspection drones perform the task autonomously, including precisely navigating to designated shooting locations over specific intervals and automatically measuring bridge deflections using internal GPS information. The proposed methodology could significantly shape the next generation of bridge inspection systems, allowing for autonomous processing, favorable flexibility, and preferred cost efficiency.

In the Future ...

The proposed approach would act as one integral technical component to new generation drone-based inspection techniques for infrastructure health assessment, allowing for autonomous processing, favorable flexibility, and preferred cost efficiency. The working image consists of inspection drones operating autonomously to execute their tasks, which include navigating precisely to predetermined locations at specific intervals and employing onboard GPS data to measure bridge deflections automatically. Our ultimate vision is that the approach developed through this research will establish drone-based infrastructure analysis as a routine solution to combat the issues presented by aging infrastructure and ensure societies' safety.

References

  1. Ri, S., Fujigaki, M. & Morimoto, Y. Sampling moiré method for accurate small deformation distribution measurement. Experimental Mechanics 50, 501-508 (2010).
  2. Ri, S. & Muramatsu, T. Theoretical error analysis of the sampling moiré method and phase compensation methodology for singe-shot phase analysis. Applied Optics 51, 3214-3223 (2012).
  3. Ri, S., Wang, Q., Tsuda, H., Shirasaki, H. & Kuribayashi, K. Displacement measurement of concrete bridges by the sampling Moiré method based on phase analysis of repeated pattern. Strain 56, e12351 (2020).
  4. Ri, S., Tsuda, H., Chang, K., Lo, F. & Lee, T. Dynamic deformation measurement by the sampling moiré method from video recording and its application to bridge engineering. Experimental Technique 44, 313-327 (2020).
  5. Ri, S., Wang, Q., Tsuda, H., Shirasaki, H. & Kuribayashi, K. Deflection measurement of bridge using images captured under the bridge by sampling moiré method. Experimental Technique 47, 1085-1095 (2023).

Thank you for reading!

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in

Follow the Topic

Engineering Mechanics
Technology and Engineering > Mathematical and Computational Engineering Applications > Engineering Mechanics
Environmental Civil Engineering
Technology and Engineering > Civil Engineering > Environmental Civil Engineering
Imaging Techniques
Physical Sciences > Materials Science > Materials Characterization Technique > Imaging Techniques

Related Collections

With collections, you can get published faster and increase your visibility.

Applications of Artificial Intelligence in Cancer

In this cross-journal collection between Nature Communications, npj Digital Medicine, npj Precision Oncology, Communications Medicine, Communications Biology, and Scientific Reports, we invite submissions with a focus on artificial intelligence in cancer.

Publishing Model: Open Access

Deadline: Mar 31, 2025

Biology of rare genetic disorders

This cross-journal Collection between Nature Communications, Communications Biology, npj Genomic Medicine and Scientific Reports brings together research articles that provide new insights into the biology of rare genetic disorders, also known as Mendelian or monogenic disorders.

Publishing Model: Open Access

Deadline: Apr 30, 2025