The eyes have been a subject of great interest from ancient times to today’s modern research. Already thousands of years ago, ancient Greek, Egyptian and Chinese philosophers and physicians were interested in the physiology of the eye, and the curiosity only deepened as scientific knowledge grew. In the 19^th century, researchers like von Helmholtz developed what would become the basis for current day pupillometry, namely the measurement of pupil size and the response to light and other stimuli. Today, pupillometry is used daily for a variety of applications, from neuroscience to cognitive research, and assists physicians and clinical teams to assess medical conditions, behavior, and cognition.

Our lab focuses on the study of sleep and cognition, with a special interest in arousal and noradrenaline signaling in the brain. Arousal and the extent to which we are responsive to (or disconnected from) the environment change constantly during sleep, and we are keen to find new ways to monitor such processes. Although very useful, none of the currently available methods, such as EEG (electroencephalogram), or heart-rate monitoring, provide the combination of simple and complex information obtainable from pupillometry. We know that pupil size is regulated by the autonomous nervous system and that changes in pupil size are associated with the activity of certain neuromodulators, such as noradrenaline (which is involved in the fight-or-flight response). When these neuromodulation pathways are suppressed, we are less likely to respond to a stimulus and the pupil will be smaller^1,2. As a corollary, when we are alert – for example in response to a surprising or painful event - noradrenaline levels are high, and the pupil is larger. We hypothesized that we could use the connection between pupil size, arousal, and neuromodulatory signaling in the brain to improve our understanding of arousal changes during sleep and to monitor our responses to external and internal stimulation. However, we faced a major hurdle: since we close our eyes when we sleep, this rich source of information is not easily available to us.

This interest led us to explore possible ways of measuring pupillometry behind closed eyelids. We wanted to develop a non-invasive method that would require no contact with the sleeping person, in order to maintain natural sleeping patterns and simplify the measurement process.

In order to “see” through closed eyelids, we needed to work at an optical wavelength other than visible light, which is blocked by the eyelids. After several attempts with near-infrared wavelengths (used by commercial eye tracking systems) and mid-infrared imaging (also known as thermal imaging), we discovered that we could detect pupillary changes when using the short-wave infrared (SWIR) range. The ability of light to go through tissues depends on the composition and the optical absorption of the molecules within them. SWIR provides a good balance between the absorption of melanin and hemoglobin, which are responsible for most of the absorption in the lower wavelengths, to the absorption of water, which becomes more dominant at higher wavelengths. SWIR light sources and detectors are commercially available and easy to use, and when suitable illumination levels are used from a distance, they are completely safe to use for extended durations. Our experimental setup includes a SWIR light emitting diode (LED) and a camera with a sensor that can detect the required wavelengths.

Our objective was to study separately each of two main pillars of eye tracking, namely monitoring gaze direction (where the eyes are looking) and measuring pupil size changes (pupillometry). Gaze direction is important in several domains, such as identifying periods of rapid eye movement sleep. Measuring pupil size and the responses to light and other stimuli are important in many research and clinical contexts – ranging from monitoring arousal, stress, and pain, to assessing the neurological status in many brain-related disorders.

However, the experimental designed faced the challenge that we wanted to measure the changes occurring behind closed eyelids but, at the same time, we needed a comparable control, a gold-standard method to measure the actual pupil state in an open eye. We also wanted to minimize potential confounding variables in order to isolate the changes resulting from pupillary changes alone. Our solution was to conduct experiments on awake subjects, who could be instructed where to look, thereby directing eye movements and attention, while controlling surrounding conditions such as ambient light. We asked the subjects to hold one eye open (to be used as a control), while the other eye, imaged with our new SWIR-based method, was held closed with a finger. We then employed the known and robust pupillary light reflex (PLR), where both pupils constrict rapidly in response to a brief flash of light and then gradually dilate back to their original size. For the gaze direction experiments, we asked our subjects to look at different pre-determined targets across a screen.

Two separate approaches were employed to analyze the pupil size. The first was a relative method: when the pupil constricts, the dark pupil area becomes smaller. Therefore, there are fewer “black pixels” in the recorded images. Since, we assume that the pupil position is fixed when participants followed our instructions to gaze straight ahead, we term this method the “fixed circle” approach. The second analysis approach employs a deep learning-based approach, using a U-Net neural net analysis, in the hope that this can capture additional features that we would otherwise miss. The model was trained on images of closed and open eyes to predict how open eyes would look, based on images of the closed eye.

The results were very exciting. The fixed circle approach with closed eye data revealed a time-course of pupil response to light flashes that strongly resembles that found in the open eye (Figure 1).

Figure 1: Response to a 2 s white light stimulation. Comparison of pupil size changes in the open-eye (dark blue), closed-eye change in image brightness (orange), and a control region in the forehead change in brightness (purple). The results represent the grand average over 41 subjects, 9 events for each subject). We can see that the closed eye brightness change profile reveals similar dynamics to the typical pupil size PLR response, as seen from the open eye measurements.

It was important to ensure that the changes in brightness observed in the closed eye were not due to a reflection of the visible light flash from the eyelid. We suspected that this was not an issue because the temporal profile of the closed-eye brightness change was similar to that of the open-eye PLR response and lasted longer than the visible light stimulus. In addition, brightness changes in a control region on the forehead did not exhibit a similar brightness change. This option was addressed by additional experiments that separated the open eye and the screen from the closed eye with a barrier so that visible light did not reach the closed eye at all. Since this did not change the results, we became even more confident that we are truly observing changes in the eye. We also repeated the experiment with both eyes naturally closed in order to validate the technique under natural conditions, without a finger holding the eye shut. Similar results were obtained by the deep learning U-Net analysis. Both methods enabled us to obtain a statistically significant robust PLR response in nearly all participants. The gaze monitoring experiments confirmed that the dark circle we observed with our SWIR imaging-based method accurately reflects the target of the subjects’ gaze. Indeed, identification of the location with video tracking software (DeepLabCut^{TM 3,4}) could predict the gaze direction with an accuracy of 10-15 degrees.

We are very excited with the current results. They prove that closed eye pupillometry and gaze estimation is possible. We are now improving the methodology both in terms of imaging/optics and in terms of data analysis in order to detect PLR changes in naturally closed eyes and for any gaze direction. We would also like to track other pupil size changes, beyond responses to light flashes, for example in response to sudden stimuli or those occurring spontaneously. We can also predict additional new and exciting applications for this method. For example, in the field of sleep studies, pupil size shows correlation with sleep stages⁵, so that our method may enable future possibilities for touchless monitoring of sleep at home instead of at a sleep lab. In clinical medicine, a device based on our technology could assist in monitoring pain or arousal during anesthesia or unconsciousness. As with any new technique, our new ability to track gaze and pupil size in closed eyes can provide new information that was previously unavailable, and potentially open the door for future discoveries beyond those that we can imagine today.

1. N. Grujic, R. Polania, and D. Burdakov, “Neurobehavioral meaning of pupil size,” Neuron, Elsevier (2024) [doi:10.1016/j.neuron.2024.05.029].
2. H. Hayat et al., “Locus coeruleus norepinephrine activity mediates sensory-evoked awakenings from sleep,” Science Advances 6(15), eaaz4232, American Association for the Advancement of Science (2020) [doi:10.1126/sciadv.aaz4232].
3. A. Mathis et al., “DeepLabCut: markerless pose estimation of user-defined body parts with deep learning,” Nat Neurosci 21(9), 1281–1289 (2018) [doi:10.1038/s41593-018-0209-y].
4. T. Nath et al., “Using DeepLabCut for 3D markerless pose estimation across species and behaviors,” Nat Protoc 14(7), 2152–2176, Nature Publishing Group (2019) [doi:10.1038/s41596-019-0176-0].
5. M. Carro-Domínguez et al., “Pupil size reveals arousal level dynamics in human sleep,” bioRxiv, 2023–07, Cold Spring Harbor Laboratory (2023).