Millimeter-wave (mmWave) imaging has enabled various applications because it is privacy-preserving, robust to illumination changes, and capable of penetrating various materials. For example, plenty of mmWave imaging systems have been deployed at major airports or train stations, which play a pivotal role in detecting concealed threats. Additionally, promising breakthroughs have also showcased the ability of mmWave imaging to provide non-invasive diagnostic information about skin cancer and breast cancer.

The fundamental capability that empowers mmWave imaging applications lies in its spatial resolution, which is intrinsically constrained by the aperture size. Existing high-resolution mmWave imaging systems are mainly built on the principles of either massive multiple-input multiple-output (MIMO) or synthetic
aperture radar (SAR). However, to produce high-resolution 3D mmWave images, hundreds of transceivers are still required with MIMO technology, which greatly increases the hardware complexity and system cost. On the other hand, SAR synthesizes a larger virtual aperture by maneuvering the radar to transmit signals from different locations. Nevertheless, bulky mechanical scanners or expensive tracking devices are indispensable to ensure that received signals from different locations can be combined coherently.

Recent advancements in mmWave radar technology have fostered the development of compact and affordable mmWave imaging systems by introducing SAR imaging to handheld settings. However, the major challenge in handheld SAR imaging is that the fluidity of hand movements introduces non-linearity and non-uniformity in the virtual apertures, resulting in phase errors in received signals and severe distortion in the resultant images. More importantly, since the phase of mmWave signal is extremely sensitive to mm-level distance variations, even sub-mm-level motion error can cause significant image distortion, as shown in Fig. 1b-e. One potential solution is to compensate for the motion error using the precise device trajectory. However, to achieve a 3D tracking accuracy of 0.1 mm, even the least expensive motion capture system made by OptiTrack costs more than $40,000, which is too expensive for consumer-level applications.

To tackle this challenge, we report a handheld mmWave imaging system that effectively combats motion errors, thus augmenting the aperture size of commercial-off-the-shelf mmWave devices by over 50x on average. Different from conventional approaches that focus on obtaining more precise device locations, our objective is to acquire the optimal point spread function (PSF) of the handheld synthetic array. This insight is established on two important phenomenons that we find during our investigation. First, we discover that motion errors along different directions exert different influences on image quality, as shown in Fig. 1f. SAR imaging is more susceptible to motion error along the range axis because the phase of mmWave signal is sensitive to range variations. This observation makes it possible to relieve the burden of motion compensation in 3D space and focus on the direction most susceptible to motion errors. Second, we find that the phase errors of different targets exhibit a local similarity, making it possible to approximate the phase error of the imaging target with another reference target. More specifically, directly estimating the phase error from image targets is challenging because the received signal phase is a combination of different parts of the target. In contrast, we analyze the relationship between phase errors of two different targets. It turns out that the phase errors of different targets can be very close if their locations are within a certain scope. Therefore, we transform the challenging problem of phase error estimation from the unknown imaging target into the tractable problem of phase error estimation from the known reference target. In this way, we can compensate for the phase error of the imaging target using the estimated phase error of the reference target.

Now, our next problem is how to estimate phase errors from a reference target, which is also a non-trivial task. Our solution is based on the fact that phase errors will affect the quality of the point spread function (PSF), as shown in Fig. 2. Hence, we propose to estimate the phase error by optimizing the handheld PSF to be as close as the ideal one. This is achieved by optimizing the phase history of point scatters to be quadratic variational, which can be efficiently solved with the least square method. Based on these findings, we implement a 3D handheld mmWave imaging system with a MIMO radar and a low-cost tracking camera. The camera is used to get a coarse tracking result and resample the radar frames to uniform spacing. Then we estimate the phase error with PSF optimization and compensate for the phase errors of the imaging target.  Finally, we reconstruct the target with the frequency-domain imaging method. 

Extensive experimental validations in Fig. 3 demonstrate the efficacy of our proposed imaging system in restoring targets from heavily distorted initial measurements, showcasing remarkable enhancements in both Peak Signal-to-Noise Ratio (PSNR, 4.54 dB) and Structure Similarity Index Measure (SSIM, 31.19%). Since our system does not rely on costly massive antennas or bulky motion controllers, it has the potential to become a standard component of subsequent handheld imaging systems, allowing diverse applications including security inspection, autonomous driving, and medical monitoring.

If you are interested in obtaining a comprehensive understanding of our research, you can read our complete paper, “A high-resolution handheld millimeter-wave imaging system with phase error estimation and compensation,” published in Communications Engineering: https://www.nature.com/articles/s44172-023-00156-2