With the advent of novel industrial applications of cameras, such as autonomous vehicles, drones, and virtual/augmented reality (VR/AR), the high-performance, miniaturized, and wide field-of-view (FoV) imaging device has become a key technological asset. In particular, an advanced camera with a compact form factor, a panoramic visual field (almost 360° FoV), and all-weather compatibility enables to envision a variety of next-generation imaging applications. However, the commercial cameras for 360° vision cannot suppress optical aberrations including image distortion due to the complicated configuration of multiple fisheye lenses. Moreover, the size of the entire imaging system is too bulky and heavy to apply it to various circumstances. Thus, there has been a need for the development of a novel imaging device as a replacement of the conventional 360° cameras.
Aiming for the development of a miniatured imaging system with panoramic visual field and amphibious imaging capability, we focused on the eyes of intertidal crabs (e.g., fiddler crab) (Fig. 1a). Unlike other types of crabs that live only on land or under water, the intertidal crabs have evolved ellipsoidal-shape eyes with flat corneal lenses, on which ommatidia are distributed to cover nearly 360°. Figure 1b shows photographs of the fiddler crab eye from various angles, and its magnified view (right inset: scanning electron microscope (SEM) image). The ommatidia are distributed on the almost entire surface of the ellipsoidal eye. Figure 1c shows flat cornea facet lenses (left frame: top view, SEM image) and a multi-layer structure of the corneal lens (right frame: cross-sectional view, transmission electron microscope (TEM) image). The flat lens with the multi-layer structure achieves a consistent focal length regardless of external refractive index change between air and water. A visual map that indicates the distribution range of the ommatidia confirms a panoramic vision of the fiddler crab (Figure 1d). From these structural and functional features, we could find out that the fiddler crab has an amphibious and panoramic vision, which inspired the design and fabrication of a novel amphibious and panoramic artificial vision.
In our recent publication, we have reported a novel panoramic and amphibious imaging device inspired by the fiddler crab vision. The artificial vision features panoramic FoV (horizontal FoV (FoVH): 300°, vertical FoV (FoVV): 160°), amphibious imaging capability, minimal optical aberrations, miniaturized form factor, and large depth-of-field (DoF) (1 cm ~ infinity). Specifically, we developed an artificial ommatidium by stacking polymeric layers with graded refractive indices (i.e., g-MLs) by mimicking the flat multi-layer corneal lens (Fig. 2a–b). This artificial corneal lens exhibits optical properties comparable to those of the natural fiddler crab eye lens. We also integrated a comb-shaped silicon photodetector array with the g-ML array (Fig. 2c), and mounted the integrated arrays on a spherical structure (Fig. 2d–e).
We demonstrated the imaging capability of the artificial vision by using a lab-made setup (Fig. 3). Imaging demonstrations were performed with five objects projected from all directions (Fig. 3a). We could successfully obtain the panoramic imaging result (Fig. 3b). Also, our artificial vision successfully achieved amphibious imaging, whose result was comparable to the optical simulation result (Fig. 3c–d). Even when a half of the device was immersed in water, the imaging performance was comparable to the case when the device was outside the water. According to these imaging demonstrations, we could verify that the artificial vision has a panoramic and amphibious imaging capability.
This work can bring a significant advance with regards to the development of a novel panoramic vision system with an amphibious imaging capability. Such an artificial vision can provide many new opportunities for the next-generation artificial vision applications including mobile electronics and robotics. For more detailed information, please see the latest article published in Nature Electronics: An amphibious artificial vision system with a panoramic visual field.