Towards high-resolution mmWave imaging with PSF optimization

We report a portable, affordable, and high-resolution 3D millimeter-wave (mmWave) imaging system, which overcomes the fundamental limitation of the mmWave device’s spatial resolution and augments its aperture size by over 50x on average.
Towards high-resolution mmWave imaging with PSF optimization
Like

Millimeter-wave (mmWave) imaging has enabled various applications because it is privacy-preserving, robust to illumination changes, and capable of penetrating various materials. For example, plenty of mmWave imaging systems have been deployed at major airports or train stations, which play a pivotal role in detecting concealed threats. Additionally, promising breakthroughs have also showcased the ability of mmWave imaging to provide non-invasive diagnostic information about skin cancer and breast cancer.

The fundamental capability that empowers mmWave imaging applications lies in its spatial resolution, which is intrinsically constrained by the aperture size. Existing high-resolution mmWave imaging systems are mainly built on the principles of either massive multiple-input multiple-output (MIMO) or synthetic
aperture radar (SAR). However, to produce high-resolution 3D mmWave images, hundreds of transceivers are still required with MIMO technology, which greatly increases the hardware complexity and system cost. On the other hand, SAR synthesizes a larger virtual aperture by maneuvering the radar to transmit signals from different locations. Nevertheless, bulky mechanical scanners or expensive tracking devices are indispensable to ensure that received signals from different locations can be combined coherently.

Recent advancements in mmWave radar technology have fostered the development of compact and affordable mmWave imaging systems by introducing SAR imaging to handheld settings. However, the major challenge in handheld SAR imaging is that the fluidity of hand movements introduces non-linearity and non-uniformity in the virtual apertures, resulting in phase errors in received signals and severe distortion in the resultant images. More importantly, since the phase of mmWave signal is extremely sensitive to mm-level distance variations, even sub-mm-level motion error can cause significant image distortion, as shown in Fig. 1b-e. One potential solution is to compensate for the motion error using the precise device trajectory. However, to achieve a 3D tracking accuracy of 0.1 mm, even the least expensive motion capture system made by OptiTrack costs more than $40,000, which is too expensive for consumer-level applications.

Fig. 1: The spatial asymmetry of motion errors.
Fig. 1: The spatial asymmetry of motion errors.  (a) An ideal planar aperture can be synthesized by mechanically moving a linear multi-input-multi-output (MIMO) array.  (b)-(e) The imaging result with different motion errors.  (f) The z-axis motion errors result in much larger phase errors than the x-axis and y-axis motion errors.

To tackle this challenge, we report a handheld mmWave imaging system that effectively combats motion errors, thus augmenting the aperture size of commercial-off-the-shelf mmWave devices by over 50x on average. Different from conventional approaches that focus on obtaining more precise device locations, our objective is to acquire the optimal point spread function (PSF) of the handheld synthetic array. This insight is established on two important phenomenons that we find during our investigation. First, we discover that motion errors along different directions exert different influences on image quality, as shown in Fig. 1f. SAR imaging is more susceptible to motion error along the range axis because the phase of mmWave signal is sensitive to range variations. This observation makes it possible to relieve the burden of motion compensation in 3D space and focus on the direction most susceptible to motion errors. Second, we find that the phase errors of different targets exhibit a local similarity, making it possible to approximate the phase error of the imaging target with another reference target. More specifically, directly estimating the phase error from image targets is challenging because the received signal phase is a combination of different parts of the target. In contrast, we analyze the relationship between phase errors of two different targets. It turns out that the phase errors of different targets can be very close if their locations are within a certain scope. Therefore, we transform the challenging problem of phase error estimation from the unknown imaging target into the tractable problem of phase error estimation from the known reference target. In this way, we can compensate for the phase error of the imaging target using the estimated phase error of the reference target.

Fig. 2: The optimization results of 1D and 2D point spread function (PSF). (a) represents the 2D PSF obtained from mechanical scanning, while (d) displays the corresponding 1D PSF. Notably, the PSFs of handheld scanning, depicted in (b) and (e) for 2D and 1D, respectively, suffer from distortion and blurring caused by motion errors. However, the proposed optimization technique successfully mitigates the impact of motion errors on the PSFs. (c) and (f) demonstrate the optimized 2D and 1D PSFs of handheld scanning, respectively. These optimized PSFs effectively combat the distortions induced by motion errors, leading to substantially improved image quality.

Now, our next problem is how to estimate phase errors from a reference target, which is also a non-trivial task. Our solution is based on the fact that phase errors will affect the quality of the point spread function (PSF), as shown in Fig. 2. Hence, we propose to estimate the phase error by optimizing the handheld PSF to be as close as the ideal one. This is achieved by optimizing the phase history of point scatters to be quadratic variational, which can be efficiently solved with the least square method. Based on these findings, we implement a 3D handheld mmWave imaging system with a MIMO radar and a low-cost tracking camera. The camera is used to get a coarse tracking result and resample the radar frames to uniform spacing. Then we estimate the phase error with PSF optimization and compensate for the phase errors of the imaging target.  Finally, we reconstruct the target with the frequency-domain imaging method. 

Fig. 3: Comparison with baseline. (a) Qualitative reconstruction results of different structural similarity index measure (SSIM) improvements. The empirical cumulative distribution functions of peak signal-to-noise ratio (b) and SSIM (c) are employed to quantitatively evaluate the resulting images, which demonstrate that our method brings a noteworthy average enhancement in both metrics.

Extensive experimental validations in Fig. 3 demonstrate the efficacy of our proposed imaging system in restoring targets from heavily distorted initial measurements, showcasing remarkable enhancements in both Peak Signal-to-Noise Ratio (PSNR, 4.54 dB) and Structure Similarity Index Measure (SSIM, 31.19%). Since our system does not rely on costly massive antennas or bulky motion controllers, it has the potential to become a standard component of subsequent handheld imaging systems, allowing diverse applications including security inspection, autonomous driving, and medical monitoring.

If you are interested in obtaining a comprehensive understanding of our research, you can read our complete paper, “A high-resolution handheld millimeter-wave imaging system with phase error estimation and compensation,” published in Communications Engineering: https://www.nature.com/articles/s44172-023-00156-2

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in

Subscribe to the Topic

Imaging Techniques
Physical Sciences > Materials Science > Materials Characterization Technique > Imaging Techniques
Internet of Things
Mathematics and Computing > Computer Science > Computer Engineering and Networks > Internet of Things
Mobile Computing
Mathematics and Computing > Computer Science > Software Engineering > Mobile Computing
Signal, Speech and Image Processing
Technology and Engineering > Electrical and Electronic Engineering > Signal, Speech and Image Processing

Related Collections

With collections, you can get published faster and increase your visibility.

Consumer waste valorization

This collection will focus on chemical, materials, mechanical and other engineering advances tackling post-consumer solid waste streams such as textiles and paper, food, electronics and plastics.

Publishing Model: Open Access

Deadline: May 31, 2024

Thermal Engineering for Sustainability

This collection will publish papers on the topic of thermal engineering with a particular focus on applications related to sustainability.

Publishing Model: Open Access

Deadline: Mar 30, 2024