In the push to develop new computing systems that mimic the brain, researchers in Singapore and China have devised an artificial retina device for the perception and recognition of objects emitting mid-infrared radiation (MIR). Inspired by how human eyesight works, the neuromorphic device is a step towards better MIR machine vision, which is an important technology for medical diagnosis, autonomous driving, intelligent night vision and military defence.
Current infrared machine vision has physically separated sensory and processing units, which creates large amounts of redundant data. This is not ideal because it results in computing and energy inefficiencies. In contrast, the human visual sensory system is very efficient, with a compact retina that perceives and processes visual data – more than 80% of our brain’s receive — which is then transmitted to the visual cortex of the brain for further processing. The retina’s photoreceptors receive continuous light stimuli ,which are converted into electrical potentials, and the latter are then encoded into trains of electrical pulses called spikes. A train of spikes containing the stimulus information then travels to the visual cortex.
Inspired by the biological retina, Fakun Wang and Fangchen Hu at Nanyang Technological University in Singapore, together with colleagues, have invented an optoelectronic retina based on a 2D van der Waals heterostructure. This heterostructure consists of a layer of black arsenic phosphorus (b-AsP) on top of a layer of molybdenum telluride (MoTe2). These materials were chosen for their fast response to light and their high absorption efficiency.
Optically driven
Previous studies focused on developing neuromorphic devices that are sensitive to light with visible and near-infrared (NIR) wavelengths. This study extends the range of wavelengths to the MIR. Another important novelty of this latest research is that the encoding function is driven optically, rather than electrically, which is promising for high-speed operation.
Programmable NIR laser pulses, applied simultaneously with MIR laser pulses, encode the information into spike trains. The stochastic NIR pulses change the MIR-excited current in the device, where a spike is generated when the current exceeds the threshold value. This emulates the encoding in the human retina. The device gives a stable response to light even for a NIR pulse frequency of 100 kHz, which guarantees high-precision MIR intensity coding.
Adaptive systems
Another important feature of intelligent systems is adaptation. To adapt to its visual environment the MIR vision system should have a wide dynamic working range of MIR intensities, and high encoding precision. The researchers tested their device with a metal mask with nine hollow figures of the number “3” illuminated by a MIR laser . This was used to imitate the real MIR targets such as a tissue sample. They found excellent encoding precision, with the encoded image matching the original image at a precision of over 97%. The team also showed that the NIR pulse parameters can be used to control the dynamic working range and precision.
Artificial eye has the potential to outperform human vision
Furthermore, they connected their device to what is considered one of the most efficient and brain-like artificial neural networks (ANNs) called a spiking neural network. In this ANN, neurons communicate by sending and receiving spikes as information carriers, much like in the brain. They used this system to classify MIR images of numerical figures in the MNIST data set, which is used to train image processing systems, and achieved an accuracy greater than 96%.
Wang, who led the research, says that their artificial retina is compatible with CMOS technology, and suggests two ways to further the research: “One is to improve device functions, such as integrating the memory function into this device, to realize the integration of perception, encoding, memory and processing. The other is to combine the device with guided-wave nanophotonics in order to achieve faster operating speeds and lower energy consumption.”
The research is described in Nature Communications.
