The story is mainly about a Eureka moment and a long road to realizing it.

An awkward problem

The time goes back to 2024. At that time, the concept of space-time metasurfaces was a popular topic among researchers (it is still popular now!). A simple analogy for it is like using a panel with time-varying responses to induce Doppler effect (controllable harmonics), but with much finer spatial resolution. This joint spatial-harmonic control of wavefronts have demonstrated some usage in wireless communications and in validation of novel physical phenomena. With some discussions with our colleagues, however, we found an awkward problem for this concept – there exists no algorithm to precisely generate the time-varying responses! Well, frankly, there are algorithms, but mostly very inefficient ones. Usually, the scale of metasurface is limited to a dozen units, otherwise the algorithm just took too long to produce a result or just simply failed to converge. In fact, small-scale metasurfaces could hardly accomplish anything practical, like focusing beams to monitoring heart beats etc. This motivated us to dive into this work in the beginning.

Eureka

In the spring of 2024, we had a Eureka moment. A question is first proposed. Why can’t the traditional phase-retrieval algorithms be applied to the harmonic dimension? Basically, these algorithms, which had matured for nearly half a century, are mostly used for spatial controls. So, the next key question is to find a bridge between the spatial and temporal dimensions and devise a way to combine them (remember, space-time holograms require both dimensions to be controlled.). To achieve it, a series of mathematical derivations was conducted and we found that the classical alternating-iteration framework was surprisingly workable once some minor compensation terms were introduced. After some numerical simulations, we validated that this framework indeed freed the algorithm from the scaling problem. Based on the initial success, we aspired for a more challenging task – harmonic-multiplexed computer-generated holography (CGH). In principle, CGH or holograms means generating complex light patterns, which also applies to the microwave frequencies. These technologies have been applied in virtual reality or similar display systems. Initially, we worried that the algorithm might still fail because of the complexity of the search space. The result, however, exceeded our expectation. Even when the pixel count approaches nearly a million, the algorithm still generates the holograms with acceptable quality and the computational cost is just a few seconds. Some revisions were made afterwards to introduce a tuning mechanism to balance the efficiency and hologram quality, which adds some utility for future applications. We were also pleased to see those old algorithms, as benchmarks, just failed to operate at such a scale, which demonstrated the advantage of our proposed work.

The Thorny Path

An idea, though precious, is easy to conceive but sometimes difficult to validate, especially by experiments. This is just the case of our work. The first challenge we faced was the scale problem. Numerical simulation demonstrated that to generate acceptable-quality holograms, the metasurface should contain at least thousands of programmable units. At that time, the largest programmable metasurface platform our group had ever fabricated had only 1024 units, which was also only equipped with an obsolete control system that was far from the requirement for space-time coding scheme (1kHz versus 1MHz). Therefore, we redesigned the system from scratch, including the excitation source, driver circuits, peripherals and even the scaffold that supports the system. Our second challenge happened in the measurement terminal. Unlike optics, we lacked a systematic field monitoring system. So, how to collect and demonstrate the hologram patterns? Based on some experiences from our previous works, we designed a customized scanning platform which include a set of instruments and mechanical submodules. With these prerequisites, we performed our first validation – a seven-letter hologram at seven harmonics. The results were encouraging. Then, we tried to push the boundary of the multiplexed number, from 7 to 26, then to 36, and finally to 62 in the demonstration reported in the manuscript. The bottleneck was reached only because of the bandwidth and memory limit of the collection instruments.

With these results, a good paper might be published. However, we thought the static hologram experiment still did not reflect the high efficiency of our algorithm and system. Hence we turned to some classical applications where high-dimensional multiplexing would help – like radar and wireless communications. In such systems, the capacity and delay could be assessed conveniently from quality-of-service evaluation. Based on this idea, the multi-object-tracking experiments were devised and conducted. Though some regrets still remain (like tracking does not form an actual closed-loop), we are still quite pleased to have demonstrated the potential practical applications of our ideas.

Future

The publication of the manuscript represents a temporary milestone for us. Currently, we are considering some truly practical applications of the space-time holography (STH). In our manuscript, we have demonstrated that thousands of beam-cloud channels could be generated independently with ease. This has inspired us to explore the possibility for positioning and imaging based on the idea of depth camera. In the meantime, the improvement of metasurface hardware is also urgently needed. To enter the true regime of time-varying material, the dynamic control speed of programmable system should be at least increased by several orders of magnitude, which calls for a refactoring at the architecture level. Though the algorithm itself has been validated in the fast-switching scenarios, we anticipate that new physics would emerge, and it deserves our finest attention and devotion.