Exponential growth in robotics and driverless vehicles in the emerging era of Internet of Things (IoT) is imposing new requirements on the modern vision sensors. Some indispensable tasks include error-free sensing, scrupulous information processing, collision detection and timely avoidance. While state-of-the-art and general-purpose vision systems offer solutions, they can be bulky and power hungry.

Interestingly, energy and area efficient solutions for timely collision detection can be found in biological eyes, specifically in the insect vision systems, which have evolved in resource constrained environments allowing the animals to escape from predators or capture preys.

We found locust. While their recent devastation of agricultural fields has made them ill reputed, they provide a fascinating solution to task specific visual computation. Millions of locusts move inside dense swarms avoiding collisions. As we dug into the literature on the neural architecture in the visual pathway of locusts, we found that the collision avoidance warning is generated by a single neuron called the Lobula Giant Movement Detector (LGMD). This single neuronal cell performs multiplicative operation on two high-level features of the visual stimuli, i.e. angular velocity that generates a excitatory response, and angular projection that generates inhibitory response, to elicit a non-monotonic firing response that peaks before the impending collision allowing sufficient time for the insect to escape the collision.

We mimicked the functionality of LGMD neuron using a nanoscale collision detector. The detector is made from a monolayer molybdenum disulfide (MoS₂) photodetector that is stacked on top of a non-volatile and programmable floating gate memory architecture. Under no programming stimulus, the MoS₂ photodetector identifies a looming object through a monotonic increase in the device current (photoexcitation). On the contrary, under no visual stimulus, the device shows a monotonic decrease in the current subjected to programming voltage pulse trains (programming inhibition). When both stimuli are present simultaneously, the visual excitation and programming inhibition compete against each other and invoke a non-monotonic trend in the output current that mimics the LGMD escape response.

While VLSI implementations of insect inspired vision can be found in the literature, our approach is radically different. Instead of a layer by layer imitation of the entire neurobiological architecture inside the insect brain, we mimicked the task specific functionality of a single neuron using a single solid-state device that exploits in-memory computing and sensing at the same time. This minimizes hardware resources and energy overhead. Our biomimetic collision detector consumes little energy (in the range of nano-Joules) and occupies a small area (around 1 μm X 5 μm). We believe our demonstration has the capability to advance the development of task-specific, energy efficient and miniaturized collision avoidance systems.

If you are interested in our work, please refer to the paper published in Nature Electronics following the link: https://www.nature.com/articles/s41928-020-00466-9