Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing

Guangdong Zhou; Jie Li; Qunliang Song; Lidan Wang; Zhijun Ren; Bai Sun; Xiaofang Hu; Wenhua Wang; Gaobo Xu; Xiaodie Chen; Lan Cheng; Feichi Zhou; Shukai Duan

doi:10.1038/s41467-023-43944-2

Full hardware implementation of neuromorphic visual system based on multimodal optoelectronic resistive memory arrays for versatile image processing

Nat Commun. 2023 Dec 20;14(1):8489. doi: 10.1038/s41467-023-43944-2.

Authors

Guangdong Zhou^#¹, Jie Li^#², Qunliang Song³, Lidan Wang¹, Zhijun Ren¹, Bai Sun⁴, Xiaofang Hu¹, Wenhua Wang³, Gaobo Xu³, Xiaodie Chen⁵, Lan Cheng⁶, Feichi Zhou⁷, Shukai Duan⁸

Affiliations

¹ College of Artificial Intelligence, Chongqing Key Laboratory of Brain-inspired Computing and Intelligent Chips, Key Laboratory of Luminescence Analysis and Molecular Sensors (Ministry of Education), Southwest University, Chongqing, 400715, China.
² School of Microelectronics, Southern University of Science and Technology, Shenzhen, 518055, China.
³ Faculty of Materials and Energy, Southwest University, Chongqing, 400715, China.
⁴ Frontier Institute of Science and Technology, Xi'an Jiaotong University, Shanxi, 710049, China.
⁵ Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, 999077, China.
⁶ State Key Laboratory of Silkworm Genome, College of Sericulture, Textile and Biomass Sciences, Southwest University, Chongqing, 400715, China.
⁷ School of Microelectronics, Southern University of Science and Technology, Shenzhen, 518055, China. zhoufc@sustech.edu.cn.
⁸ College of Artificial Intelligence, Chongqing Key Laboratory of Brain-inspired Computing and Intelligent Chips, Key Laboratory of Luminescence Analysis and Molecular Sensors (Ministry of Education), Southwest University, Chongqing, 400715, China. duansk@swu.edu.cn.

^# Contributed equally.

Abstract

In-sensor and near-sensor computing are becoming the next-generation computing paradigm for high-density and low-power sensory processing. To fulfil a high-density and efficient neuromorphic visual system with fully hierarchical emulation of the retina and visual cortex, emerging multimodal neuromorphic devices for multi-stage processing and a fully hardware-implemented system with versatile image processing functions are still lacking and highly desirable. Here we demonstrate an emerging multimodal-multifunctional resistive random-access memory (RRAM) device array based on modified silk fibroin protein (MSFP), exhibiting both optoelectronic RRAM (ORRAM) mode featured by unique negative and positive photoconductance memory and electrical RRAM (ERRAM) mode featured by analogue resistive switching. A full hardware implementation of the artificial visual system with versatile image processing functions is realised for the first time, including ORRAM mode array for the in-sensor image pre-processing (contrast enhancement, background denoising, feature extraction) and ERRAM mode array for near-sensor high-level image recognition, which hugely improves the integration density, and simply the circuit design and the fabrication and integration complexity.

Abstract

Grants and funding