Toward vision beyond the human eye: A curvature-tunable neuromorphic bionic eye with ultradense waveguide-coupled pixels

A curvature-tunable neuromorphic bionic eye that features ultradense pixels, high detection efficiency, in-sensor computing and focus adaptation beyond that of human eye was created, which provides a pathway towards advanced artificial vision systems understanding dynamic 3D environments.

Published in Physics and Computational Sciences

Toward vision beyond the human eye: A curvature-tunable neuromorphic bionic eye with ultradense waveguide-coupled pixels
Like

Share this post

Choose a social network to share with, or copy the URL to share elsewhere

This is a representation of how your post may appear on social media. The actual post will vary between social networks

The human eye is an extraordinary imaging system, capable of seamlessly combining wide-field vision, high spatial resolution and efficient neural processing in a compact organ. For decades, scientists have sought to replicate these capabilities in artificial vision systems for intelligent robots and interactive artificial intelligence. Yet duplicating the human eye has proven remarkably difficult. Most artificial retinas cannot reproduce the slender, rod-like architecture of natural photoreceptors without sacrificing their ability to capture light, forcing engineers to trade-off between smaller pixels and higher sensitivity. However, even the human eye is not flawless. While the eye effortlessly refocuses on objects from far to near, its retina cannot change shape, meaning the image plane is never perfectly matched under every viewing condition. As a result, subtle optical aberrations that become increasingly noticeable over wide fields of view or long focusing ranges.

Our solution was inspired by the way the human retina is built. In contrast to fabricating conventional planar photodetecting pixels, we arranged light-sensitive materials around tiny vertical optical waveguides, creating slender pixels that resemble the rod and cone cells found in the eye (Fig.1). Light is guided along the entire length of the pixel, allowing it to harvest light much more efficiently without increasing its planar size. The design solves the dilemma of high photodetection efficiency and pixel downscaling, enabling an integration density of 7.84 × 106 pixels cm-2 (7000 PPI), exceeding human retinal density by more than one order of magnitude, together with an ultrahigh detectivity of 2.17 × 1014 Jones. Beyond sensing light, the pixels also exhibit an intrinsic visual memory, allowing them to emulate key synaptic functions and process visual information in sensor.

Fig. 1 | Design of neuromorphic bionic eye with ultradense waveguide-coupled pixels. a, Schematic illustration of the neuromorphic bionic eye (left panel), InNx/SU-8 vertical waveguide-coupled pixels array (middle panel), and individual pixel structure (right panel). b, Digital photograph and SEM images of InNx/SU-8 vertical pixels array. c, Schematic of the light-matter interaction within the InNx/SU-8 pixels. d, Electric field distribution within an individual InNx/SU-8 pixels. e, Comparison of integration density and specific detectivity.

Moreover, we integrated these ultradense pixels into a curvature-tunable artificial eye, enabling focus adaptation beyond the capabilities of the human eye (Fig.2). Compared with the fixed-shape retina in human eye, the sensor with ultradense waveguide-coupled pixels can dynamically change its shape to better match the focal surface, reducing optical distortions and maintaining sharp three-dimensional vision across a wide range of viewing conditions. The prototype achieved a 110° field of view while reducing field curvature by 45.7%. By combining in-sensor image denoising with dynamic optical aberration compensation, it improved the accuracy of artificial neural network recognition and enabled three-dimensional motion tracking with an accuracy of 96.1%, bringing artificial vision one step closer to the adaptability and intelligence of biological eyes.

Fig. 2 | System-level demonstration of neuromorphic bionic eye. a, Digital image of artificial vision system based on proposed neuromorphic bionic eye. b, Imaging results with and without curvature modulation. c, Spatial motion tracking in 3D space through readjustment and dynamic adaptation of focus. d, High-accuracy static image recognition by in-sensor denoising. e, Reconstructed spatial locations of motion object based on the curvature-tunable artificial eye.

Duplicating and mimicking the remarkable capabilities of human eye remains one of the grand challenges of artificial intelligence and robotics. Our work represents an important step towards artificial vision systems that go beyond simply capturing images to understanding the world around them, enabling intelligent machines to perceive their surroundings in ways that rival or even exceed human vision.

For more information, please refer to our recent publication in Nature Sensors, “A neuromorphic bionic eye with ultradense waveguide-coupled pixels for depth-tunable 3D vision” (https://www.nature.com/articles/s44460-026-00119-y).

Please sign in or register for FREE

If you are a registered user on Research Communities by Springer Nature, please sign in

Follow the Topic

Spotlight on Research from China
Research Publishing > Spotlight on Research from China
Optoelectronic Devices
Physical Sciences > Physics and Astronomy > Optics and Photonics > Optoelectronic Devices
Artificial Intelligence
Mathematics and Computing > Computer Science > Artificial Intelligence
Optical Sensor
Physical Sciences > Physics and Astronomy > Optics and Photonics > Optoelectronic Devices > Optical Sensor