Light spectrum encompasses rich information of the physical world. Two inks may look almost identical, yet the different transmission spectra can reveal their true identities. Spectral sensing relies on an encoding-decoding pipeline to extract application-relevant spectral information from light (Fig. 1). Conventionally, optical spectrometers employ optical components (prisms, gratings and interferometers) to spatially or temporally modulate incident light spectrum, encoding it into varying electrical signals; these raw sensory data are transferred to external computing units for a decoding process that resolves the spectrum. These systems can provide accurate spectral measurements, but they are often bulky and unsuitable for portable or resource-limited applications. In recent years, computational spectrometers have attracted increasing research focus because they can miniaturize the optical encoding process into compact sensor structures. However, the decoding is still performed outside; energy dissipation and latency caused by transferring large volumes of raw sensory data remains problematic. These challenges become especially important for portable instruments, edge devices, and Internet-of-Things components, where power and bandwidth are limited.

The problem leads to a question for our research team at the Hong Kong Polytechnic University: can we further bring decoding to the sensory terminal? The sensor could then output application-relevant spectral information only, while bypassing redundant raw measurement signals. In our work recently published in Nature Sensors, we present a spectral sensor that unifies both encoding and decoding (Fig. 2). A key insight of our approach comes from the computation capability of the capacitive transimpedance amplifier (CTIA) readout circuit. CTIAs are widely used in infrared (IR) image sensors, where they integrate overtime photocurrent (I_ph,m) from the upstream photodiodes. The readout voltage (V_out) is proportional to the photocurrent with the integration window (t_m) as a “linear coefficient”. When a sequence of distinct photocurrent values is fed into the CTIA as a continuous scanning waveform, the readout voltage essentially computes a vector product between the photocurrent vector (organized by the photocurrent sequence i_ph = [I_ph,m]) and the “time-weighting” vector (organized by the integration windows t = [t_m]).

We prepared a computational spectrometer device (non-uniform-composition antimony-sulfur-selenium heterojunctions) to serve as the photodiode for the complete spectral sensor architecture together with the CTIA readout and control circuits. The voltage-tunable spectral responsivity [R(λ,V)] of the photodiode enables in-sensor encoding: the incident spectrum (α_λ) is encoded into the photocurrent vector by scanning the photodiode across different voltages. Feeding the scanned photocurrent waveform into the CTIA allows the sensor to simultaneously execute the afore-mentioned vector product computation. It realizes designated in-sensor decoding computation, when the programmed time-weighting vectors map the decoding matrices for extracting application-specified spectral information. 

Different applications need different types of spectral information. Sometimes we want to reconstruct the complete spectrum. But in many practical cases, the key question is more direct: does this sample belong to class A or class B? Is this document authentic? etc. For these tasks, directly outputting application-relevant information (recognition results, etc.) can be more efficient than explicitly recording and transferring raw spectral data. As a proof-of-concept demonstration, we applied the spectral sensor to spectroscopic ink recognition. The programmed time-weighting vectors mapped the weight matrix of a neural network-based classifier for discriminating the 3 types of “inks” (1 Wine sample emulating stains on a document, and 2 Ink samples emulating genuine and falsified conetents). The 3 readout voltages derived from the sequential scans directly served as the classification indices; each achieved highest score on their corresponding classes. Furthermore, in a falsification detection task requiring binary classification between the 2 Ink samples, the spectral sensor maps the original and falsified inks to positive and negative voltages within a rapid, 0.12-s scan. For details, please refer to the article: “In-sensor spectral decoding for efficient spectroscopic recognition” at https://doi.org/10.1038/s44460-026-00085-5. 

In our experiments, the fully in-sensor approach reduces projected data amount by 92.5% and 97.5% for the ink recognition and falsification detection tasks, respectively. The data compression performance further accounts for a projected reduction of 91% in latency, and 89.4% in energy consumption. By reducing redundant data transfer and external computation, this approach may open new opportunities for portable spectral analysis, intelligent machine vision and energy-efficient edge sensing. Future works may focus on improving spectral resolution, increasing operation speed, scaling up the non-uniform-composition photodiode arrays, and integrating the system with mature imaging electronics.