Source-linked AI summary
Gate-tunable van der Waals heterostructure for reconfigurable neural network vision sensor
Chen-Yu Wang, Shi-Jun Liang, Shuang Wang, Pengfei Wang, Zhuan Li, Zhongrui Wang, Anyuan Gao, Chen Pan, Chuan Liu, Jian Liu, Huafeng Yang, Xiaowei Liu, Wenhao Song, Cong Wang, Xiaomu Wang, Kunji Chen, Zhenlin Wang, Kenji Watanabe, Takashi Taniguchi, J. Joshua Yang, Feng Miao
TL;DR
Retina-inspired vision sensors face challenges in reproducing biological photoresponses while efficiently processing visual information. This paper develops a gate-tunable vdW heterostructure sensor whose pixels can sense, process, and classify images, reaching 100% accuracy in under 10 epochs.
Problem
Reproducing retinal photoresponses, particularly the OFF response, remains challenging for vision sensors designed to process visual information efficiently.
Method
The authors use gate-tunable positive and negative photoresponses in vdW vertical heterostructures to build reconfigurable pixels with individually adjustable classification weights.
Results
100% classification accuracy is reached with fewer than 10 training epochs.
Takeaways & Limitations
The retinomorphic vision sensor itself can function as a convolutional neural network for image recognition.
Takeaways & Limitations
The OFF photoresponse has a response time that is not comparable to the ON photoresponse.
Abstract
from arXiv · showhide
Early processing of visual information takes place in the human retina. Mimicking neurobiological structures and functionalities of the retina provide a promising pathway to achieving vision sensor with highly efficient image processing. Here, we demonstrate a prototype vision sensor that operates via the gate-tunable positive and negative photoresponses of the van der Waals (vdW) vertical heterostructures. The sensor emulates not only the neurobiological functionalities of bipolar cells and photoreceptors but also the unique synaptic connectivity between bipolar cells and photoreceptors. By tuning gate voltage for each pixel, we achieve reconfigurable vision sensor for simultaneously image sensing and processing. Furthermore, our prototype vision sensor itself can be trained to classify the input images, via updating the gate voltages applied individually to each pixel in the sensor. Our work indicates that vdW vertical heterostructures offer a promising platform for the development of neural network vision sensor.
Mimicking retinal cells with vertical heterostructure devices
The WSe2/h-BN/Al2O3 vertical heterostructure reproduces photoreceptor- and bipolar-cell-like positive and negative photoresponses through gate-voltage control. Arrays of these devices emulate retinal receptive fields, enabling reconfigurable simultaneous image sensing and processing, including edge enhancement.
- Device architecture: The compact vertical heterostructure integrates photoreceptor- and bipolar-cell-like functionality in a simple device architecture.The architecture vertically integrates the two cell types and is described as simpler and more compact than silicon retina structures.
- Gate-tunable photoresponses: Gate voltage switches the device between positive ON-photoresponse and negative OFF-photoresponse, mimicking bipolar-cell characteristics.Zero gate voltage produces increasing current under illumination, whereas negative gate voltage produces decreasing current.
- Device performance: <8 ms response time demonstrates fast OFF-photoresponse enabled by the sharp vdW interface and nanoscale Al2O3 layer.The response timescale is described as comparable to biological bipolar cells.
- Reconfigurable retinomorphic vision sensor: 13 vdW heterostructure devices assembled into a gate-controlled array emulate bipolar-cell receptive fields for edge detection.The array combines an OFF-photoresponse center with ON-photoresponse surrounding devices, and increasing activation changes the surrounding-side current variation.
Implementation of a convolutional neural network
The retinomorphic vision sensor implements a convolutional neural network by tuning per-pixel gate voltages as weights, enabling analog image classification. Training reaches 100% accuracy in fewer than 10 epochs, with target letters separated after 2 epochs.
- Implementation of a convolutional neural network: The sensor forms a convolutional neural network whose weights are updated by tuning the gate voltage applied to each pixel.The sensed image and per-pixel weights are combined through a dot product to calculate the total output current.
- Implementation of a convolutional neural network: Training uses nine binary 3 × 3 figures representing the letters ‘n’, ‘j’ and ‘u’, input into the sensor through laser.Backpropagation tunes each pixel’s back-gate voltage after every epoch.
- Implementation of a convolutional neural network: 100 % accuracy is reached with less than 10 epochs using the weighted average of three different convolution kernels.The accuracy over training epochs evaluates convergence of the neural-network outputs.
- Implementation of a convolutional neural network: After 2 epochs, the target letter can be well separated from the input images across the convolution kernels.The output evolution is evaluated for the three network outputs corresponding to the different letters.
- Implementation of a convolutional neural network: The prototype demonstrates a reconfigurable convolutional neural network for image recognition and points toward fully analog visual processing with vdW vertical heterostructures.The sensor itself performs image recognition beyond mimicking retinal-cell functionality.
Methods · Device fabrication
The device-fabrication process combined exfoliated WSe2 and h-BN with lithographically patterned electrodes, ALD-grown Al2O3, and PVA-based transfer. AFM characterized material thicknesses and device dimensions, followed by argon annealing before electrical measurements.
- Device fabrication: WSe2 and h-BN were obtained by mechanical exfoliation for device fabrication.
- Device fabrication: Bottom-gate Ti (5 nm)/Au (25 nm) and source/drain Pd (5 nm)/Au (75 nm) electrodes were patterned by electron-beam lithography and evaporation.
- Device fabrication: Al2O3 was grown on the bottom-gate electrodes using atomic layer deposition.
- Device fabrication: vdW vertical heterostructures were formed by transferring h-BN and WSe2 onto the substrate with the standard PVA method.
- Device fabrication: AFM confirmed thicknesses of WSe2 (2~20 nm), h-BN (10~40 nm), and Al2O3 (6~10 nm).Device robustness was positively correlated with material thickness.
- Device fabrication: Before electrical measurements, all devices were annealed at 573 K in argon for 2 hours to remove photoresist residue.
- Device fabrication: The Difference-of-Gaussians model describes biological receptive-field responses to light as the difference between center and surround Gaussian functions.The formulation uses spatial coordinates, central coordinates, and standard deviations for the spatially distributed photoresponse.
Electrical measurement
Electrical measurements connected the vdW heterostructure devices in parallel through a switch matrix and recorded their currents using dedicated instrumentation. Separate measurement channels were used for ON- and OFF-photoresponse devices, with gate voltage applied by a source measurement unit.
- Measurement setup: Devices were placed on a special PCB in a nitrogen atmosphere and connected in parallel through a switch matrix box.The setup used a controlled atmosphere and parallel device connection.
- Measurement instrumentation: Current measurements used a National Instruments PCIe-6351 data acquisition card and Stanford Research Systems SR570 current amplifier.These instruments provided data acquisition and current amplification.
- Measurement channels: Gate voltage was applied using a Keithley 2635A source measurement unit, with separate channels for ON-photoresponse and OFF-photoresponse devices.Supplementary Fig. 9a describes the two-channel measurement arrangement.
Signal processing
The signal-processing pipeline separately acquired ON and OFF channel currents, reduced surface-adsorbate noise, calibrated photocurrents, and reconstructed the measurements as images.
- ON and OFF channel currents were acquired separately before subsequent signal processing.
- Background-current fluctuations under background illumination were measured to reduce noise from H2O and O2 molecules absorbed on WSe2.
- ON- and OFF-photoresponse photocurrents were calibrated by multiplying them by weights obtained during pattern generation.
- Measured ON and OFF currents were added and reorganized into images, with all codes implemented in Igor software.