Nanotechnology is rapidly changing many fields, including electronics, optics, materials science, medicine, and biology. These nanotechnologies depend heavily on fast, high-resolution imaging and characterization. Examples include fabricating complex three-dimensional (3D) nanostructures (e.g., metasurfaces, semiconductor devices), in-situ imaging of nanometer-scale reactions (e.g., catalysis, energy storage, quantum devices), and examining self-assembled 3D nanostructures in nature for inspiration- and curiosity-driven research (e.g., butterfly wing scales). Currently, most of such imaging is two-dimensional (2D) because rapid, easily accessible 3D imaging tools at the nanometer scale are absent and require specialized instrumentation or large facilities (e.g., synchrotrons). 

Transmission electron microscopes (TEM) are widely used nanoscale characterization tools that can reach unmatched 2D resolution. In our computational lens group, we develop algorithms to quantify and statistically infer hidden details in TEM micrographs. When we tried to develop such a scheme for analyzing in-situ TEM movies, we found that the beam’s coherence has primarily been treated as a nuisance many microscopists sought to “remove” or “ignore” in their images. But this coherence is extremely important for creating the phase contrast for single weakly scattering atoms. 

The characteristic of coherent imaging schemes is that they capture fixed-phase relationships between various parts of the specimen. Similarly, the high coherence of the electron beam enables the TEM imaging setup to capture an interference pattern (HRTEM micrograph) that encodes such phase relationships across various parts of the specimen. In other words, we figured that the fringes and speckles in an HRTEM micrograph are not “coherent nuisances” that limit the resolution; instead, they encode the local depth information of the specimen. Since we already know that the local thickness information can be measured from an energy-filtered HRTEM micrograph using Beer-Lambert’s law, we developed a 3D reconstruction method that pops out volume equivalent to the local thickness at the estimated local depth, which we call “pop-out 3D metrology” ¹. The method is now published in Communications Physics: https://www.nature.com/articles/s42005-023-01431-6 ².

 Our initial validation came from the simulated HRTEM micrographs (multi-slice) of specimens with parts at different depths that showed different local defocus values ³. We followed this with a set of proof-of-concept experiments. Apart from demonstrating feasibility, we also tried to fundamentally understand where the apparent plane of focus should be for a thick sample. Without knowing this, we cannot reliably “pop out” the thickness of the sample at the correct depth. The results we obtained from the simulations and experiments showed that the estimated depth points to the center of the mass of the specimen, and we have to pop out half of the thickness on either side of the estimated depth. 

Since we wanted to understand why local defocus values point to the center of mass of the scatterer and to show how depth information can be retrieved from a single 2D micrograph, we turned to a linearized kinematic scattering model (i.e., the first Born approximation) of the HRTEM image’s power spectra, using the multi-slice formalism. This model provided us with a few critical insights:

1. Whereas the HRTEM image contains scattering information from all the slices along the depth of the specimen, for amorphous samples with homogenous density, these slices together show an apparent defocus in the middle of the specimen.
2. Surprisingly, information about a sample’s local thickness not only shows up (expectedly) as an absorption contrast but also manifests in the image's transverse coherent point spread function.
3. Pop-out metrology can be done simultaneously on multiple specimen layers but with some additional priors.

 Extending the method to image multi-layered specimens opens the possibility of quick 3D metrology on a wide variety of amorphous specimens. This includes the self-assembled nanostructures in a butterfly wing scale (Junonia orythia), whose 3D densities we had previously reconstructed using ptychographic X-ray computed tomography (PXCT)⁴. Using this butterfly scale’s PXCT 3D reconstruction (Fig. 1b) as our ground truth, we tested our pop-out technique on this complex multi-layered sample and presented the results in the M&M 2023 meeting ⁵. Although the pop-out reconstruction shows artifacts, many 3D features were accurately reproduced from only a single 2D TEM image!

Here, we share a few notable advantages of pop-out 3D metrology:

• This type of metrology is fast since only a single 2D TEM image is needed to reconstruct a sample’s 3D volume without rotating the sample, as in tomography. This speed allows us to scale up 3D metrology to an extended specimen (i.e., a few µm² area imaged to ~10nm 3D resolution within a minute).
• The vast majority of TEMs with energy filters can already use the pop-out principle without modification.
• 3D dynamics: Our method can image the 3D structural dynamics with only a 2D TEM movie of a changing sample. Unlike tomography, whose temporal resolution is limited by the acquisition time of a full tilt series, our time resolution is limited only by the frame rate of the detector in movie-mode TEM (i.e., typically milliseconds) or pulse duration with the more exotic ultrafast TEM.

More broadly, our formulation of pop-out metrology is generalizable to any coherent imaging system through optically thick samples (i.e., X-rays, electrons, visible light photons, etc.). The manuscript picked electron microscopy only as a demonstrative modality rather than the only possible one. Given how accessible and fast pop-out 3D metrology is, from rapid fabrication feedback to study nanostructures and processes, this can be widely used as an accessible complementary tool to the existing slower form of metrology.

References:

1. Balakrishnan, D. et al. TEM-based single-shot coherent 3D imaging. in Imaging and Applied Optics Congress 2022 (3D, AOA, COSI, ISA, pcAOP) JW5A.3 (Optica Publishing Group, 2022).
2. Balakrishnan, D. et al. Single-shot, coherent, pop-out 3D metrology. Communications Physics 6, 1–15 (2023).
3. Kirkland, E. J. Advanced Computing in Electron Microscopy. (Springer Science & Business Media, 2013).
4. Prakash, A. et al. Nanoscale cuticle density variations correlate with pigmentation and color in butterfly wing scales. arXiv [physics.bio-ph] (2023).
5. Balakrishnan, D. et al. Potential Large-area Imaging of Butterfly Wing Scales with Transmission Electron Microscopy. Microsc. Microanal. 29, 1207–1208 (2023).