Kev siv cov ntaub ntawv tiag tiag hauv ntiaj teb xav tau kev sib cog lus, tsis tshua muaj latency, lub tshuab hluav taws xob qis. Nrog rau qhov kev tshwm sim-tsav kev muaj peev xwm, kev sib txuas ntawm hlau-oxide-semiconductor hybrid memristive neuromorphic architectures muab lub hauv paus zoo tshaj plaws kho vajtse rau cov haujlwm zoo li no. Txhawm rau ua kom pom tag nrho cov peev txheej ntawm cov tshuab zoo li no, peb tawm tswv yim thiab sim ua kom pom cov txheej txheem sensor ua tiav rau cov khoom siv hauv lub ntiaj teb tiag tiag. Teeb duab kev tshoov siab los ntawm barn owl neuroanatomy, peb tau tsim lub bioinspired, tshwm sim-driven khoom localization system uas ua ke ib lub xeev-of-the-art piezoelectric micromechanical transducer transducer nrog xam daim duab-based neuromorphic resistive nco. Peb qhia cov kev ntsuas ntawm lub kaw lus fabricated uas suav nrog lub cim xeeb-raws li kev tiv thaiv coincidence detector, ncua kab circuitry, thiab tag nrho customizable ultrasonic transducer. Peb siv cov txiaj ntsig kev sim no los ntsuas qhov simulation ntawm qib system. Cov simulations no yog siv los ntsuas qhov kev daws teeb meem angular thiab lub zog ua haujlwm ntawm cov khoom siv hauv zos qauv. Cov txiaj ntsig tau pom tias peb txoj hauv kev tuaj yeem ua tau ntau qhov kev txiav txim ntawm qhov muaj zog ntau dua li microcontrollers ua haujlwm tib yam.
Peb tab tom nkag mus rau lub sijhawm ntawm kev suav txhua qhov chaw uas cov khoom siv thiab cov tshuab siv tau nce zuj zus los pab peb hauv peb lub neej txhua hnub. Cov tshuab no yuav tsum tau ua haujlwm tsis tu ncua, siv lub zog me me li sai tau thaum kawm los txhais cov ntaub ntawv uas lawv sau los ntawm ntau lub sensors hauv lub sijhawm tiag tiag thiab tsim tawm binary tawm los ntawm kev faib tawm lossis kev paub txog kev ua haujlwm. Ib qho ntawm cov kauj ruam tseem ceeb tshaj plaws uas yuav tsum tau ua kom ua tiav lub hom phiaj no yog rho tawm cov ntaub ntawv tseem ceeb thiab kev cog lus los ntawm cov ntaub ntawv tsis txaus ntseeg thiab feem ntau tsis tiav. Cov qauv kev tsim vaj tsev ua haujlwm feem ntau yog cov qauv ntsuas ntsuas ntawm tus nqi tas mus li thiab siab, tsim cov ntaub ntawv ntau txawm tias tsis muaj cov khoom siv tau zoo. Tsis tas li ntawd, cov txheej txheem no siv cov txheej txheem tsim cov teeb liab nyuaj ua ntej ua cov ntaub ntawv (feem ntau nrov) cov ntaub ntawv. Hloov chaw, biology muaj lwm txoj kev daws teeb meem rau kev ua cov ntaub ntawv tsis hnov tsw siv lub zog-npaum, asynchronous, kev tshwm sim-tsav mus kom ze (spikes) 2,3. Neuromorphic xam siv kev tshoov siab los ntawm cov kab ke lom neeg kom txo tau cov nqi ntawm kev siv hluav taws xob thiab kev nco zoo piv rau cov txheej txheem ua cov teeb liab ib txwm muaj 4,5,6. Tsis ntev los no, lub hom phiaj tseem ceeb ntawm lub hlwb raws li kev siv cov kev sib txuas ntawm lub hlwb (TrueNorth7, BrainScaleS8, DYNAP-SE9, Loihi10, Spinnaker11) tau tshwm sim. Cov txheej txheem no muab lub zog qis, qis latency daws teeb meem rau kev kawm tshuab thiab cortical circuit qauv. Txhawm rau kom siv lawv lub zog ua haujlwm tau zoo, cov txheej txheem neuromorphic yuav tsum tau txuas ncaj qha mus rau qhov tshwm sim-tsav sensors12,13. Txawm li cas los xij, niaj hnub no tsuas muaj ob peb lub cuab yeej kov uas ncaj qha muab cov ntaub ntawv tshwm sim-tsav. Cov piv txwv tseem ceeb yog dynamic pom qhov muag sensors (DVS) rau kev pom kev siv xws li kev taug qab thiab kev txav chaw 14,15,16,17 lub silicon cochlea18 thiab neuromorphic auditory sensors (NAS)19 rau kev hnov cov teeb liab ua, olfactory sensors20 thiab ntau yam piv txwv21,22 ntawm kov. . kev ntxhib los mos sensors.
Nyob rau hauv daim ntawv no, peb nthuav tawm ib qho kev tsim kho tshiab uas tau tsav los ntawm kev sib tham txog kev ua haujlwm siv rau cov khoom hauv zos. Ntawm no, thawj zaug, peb piav qhia txog qhov kawg-rau-kawg rau cov khoom hauv cheeb tsam tau txais los ntawm kev sib txuas lub xeev-of-the-art piezoelectric micromachined ultrasonic transducer (pMUT) nrog cov lej suav raws li neuromorphic resistive nco (RRAM). Nyob rau hauv-nco xam architectures siv RRAM yog ib qho kev cog lus rau kev txo hwj chim noj23,24,25,26,27,28,29. Lawv qhov kev tsis sib haum xeeb - tsis tas yuav tsum tau siv lub zog siv los khaws lossis hloov kho cov ntaub ntawv - yog ib qho zoo meej haum nrog asynchronous, tshwm sim-tsav qhov xwm txheej ntawm neuromorphic xam, ua rau ze-tsis muaj hwj chim noj thaum lub kaw lus tsis ua haujlwm. Piezoelectric micromachined ultrasonic transducers (pMUTs) yog pheej yig, miniaturized silicon-raws li ultrasonic transducers muaj peev xwm ua raws li transmitters thiab receivers30,31,32,33,34. Txhawm rau ua cov teeb liab tau txais los ntawm cov sensors built-in, peb kos kev tshoov siab los ntawm barn owl neuroanatomy35,36,37. Lub barn owl Tyto alba paub txog nws qhov zoo kawg li kev yos hav zoov hmo ntuj ua tsaug rau lub suab nrov hauv cheeb tsam zoo heev. Txhawm rau xam qhov chaw ntawm cov tsiaj nyeg, lub barn owl's localization system encodes lub sijhawm ya davhlau (ToF) thaum lub suab nthwv dej ntawm cov tsiaj nyeg ncav cuag txhua tus noog lub pob ntseg lossis lub suab receptors. Muab qhov kev ncua deb ntawm pob ntseg, qhov sib txawv ntawm ob qhov kev ntsuas ToF (Interaural Time Difference, ITD) ua rau nws muaj peev xwm ntsuas qhov azimuth txoj hauj lwm ntawm lub hom phiaj. Txawm hais tias cov kab ke lom neeg tsis zoo rau kev daws cov kab zauv sib npaug, lawv tuaj yeem daws cov teeb meem hauv cheeb tsam zoo heev. Lub barn owl lub paj hlwb siv cov txheej txheem ntawm coincidence detector (CD) 35 neurons (ie, neurons muaj peev xwm txheeb xyuas qhov kev sib raug zoo ntawm lub cev ntawm spikes uas propagate downward to convergent excitatory endings) 38,39 npaj rau hauv kev suav graphs los daws qhov teeb meem ntawm qhov chaw.
Cov kev tshawb fawb yav dhau los tau pom tias cov khoom siv hlau-oxide-semiconductor (CMOS) thiab RRAM-based neuromorphic hardware tshwm sim los ntawm cov kab mob tsis zoo ("auditory cortex") ntawm lub barn owl yog ib txoj hauv kev zoo rau kev suav txoj haujlwm siv ITD13, 40, 41, 42, 43 , 44, 45, 46. Txawm li cas los xij, lub peev xwm ntawm cov kab mob neuromorphic tiav uas txuas cov lus qhia rau cov paj hlwb neuromorphic tseem tsis tau pom dua. Qhov teeb meem tseem ceeb yog qhov sib txawv ntawm qhov sib txawv ntawm analog CMOS circuits, uas cuam tshuam rau qhov tseeb ntawm kev sib tw kom pom. Tsis ntev los no, lwm qhov kev siv tus lej ntawm ITD47 qhov kwv yees tau tshwm sim. Nyob rau hauv daim ntawv no, peb thov kom siv lub peev xwm ntawm RRAM los hloov tus nqi conductance nyob rau hauv ib tug uas tsis yog-volatile yam rau counteract variability nyob rau hauv analog circuits. Peb tau siv qhov kev sim uas muaj xws li ib qho pMUT xa cov membrane ua haujlwm ntawm qhov zaus ntawm 111.9 kHz, ob lub pMUT txais daim nyias nyias (sensors) simulating barn owl pob ntseg, thiab ib qho. Peb tau sim tus yam ntxwv ntawm pMUT kuaj pom thiab RRAM-raws li ITD suav cov duab los kuaj peb cov kab ke hauv cheeb tsam thiab ntsuas nws qhov kev daws teeb meem.
Peb sib piv peb cov qauv nrog kev siv digital ntawm microcontroller ua haujlwm tib lub zos siv cov qauv beamforming lossis neuromorphic txoj kev, nrog rau thaj chaw programmable rooj vag array (FPGA) rau kev kwv yees ITD tau npaj rau hauv kev siv. 47. Qhov kev sib piv no qhia txog kev sib tw fais fab kev ua tau zoo ntawm RRAM-based analog neuromorphic system.
Ib qho piv txwv zoo tshaj plaws ntawm qhov raug thiab ua tau zoo ntawm cov khoom hauv cheeb tsam tuaj yeem pom hauv barn owl35,37,48. Thaum tsaus ntuj thiab kaj ntug, lub barn owl (Tyto Alba) feem ntau tso siab rau kev mloog lus, nquag nrhiav cov tsiaj me xws li voles lossis nas. Cov kws paub txog kev hnov lus no tuaj yeem ua rau thaj chaw hnov lus los ntawm cov neeg raug tsim txom uas xav tsis thoob (kwv yees li 2 °) 35, raws li qhia hauv daim duab 1a. Barn owls infer qhov chaw ntawm lub suab hauv lub dav hlau azimuth (kab rov tav) los ntawm qhov sib txawv ntawm lub sij hawm tuaj ntawm lub davhlau (ITD) los ntawm lub suab mus rau ob lub pob ntseg. Lub tshuab xam zauv ITD tau thov los ntawm Jeffress49,50 uas tso siab rau neural geometry thiab yuav tsum muaj ob lub ntsiab lus tseem ceeb: ib qho axon, neuron lub paj hlwb ua haujlwm raws li txoj kab qeeb, thiab ib qho array ntawm coincidence detector neurons koom ua ke hauv kev suav. daim duab raws li qhia hauv daim duab 1b. Lub suab mus txog lub pob ntseg nrog lub sijhawm ncua sijhawm azimuth (ITD). Lub suab yog tom qab ntawd hloov mus rau hauv cov qauv dav dav hauv txhua pob ntseg. Cov axons ntawm sab laug thiab sab pob ntseg ua raws li cov kab qeeb thiab sib sau ua ke ntawm CD neurons. Raws li txoj cai, tsuas yog ib qho neuron nyob rau hauv ib qho array ntawm matched neurons yuav tau txais cov tswv yim nyob rau hauv ib lub sij hawm (qhov twg qhov ncua sij hawm tshem tawm raws nraim) thiab yuav tua ntau tshaj (cov neeg nyob sib ze yuav tua hluav taws, tab sis ntawm ib tug tsawg zaus). Ua kom muaj qee yam neurons encodes txoj haujlwm ntawm lub hom phiaj hauv qhov chaw yam tsis tas hloov ITD rau cov ces kaum. Lub tswv yim no tau sau tseg hauv daim duab 1c: piv txwv li, yog tias lub suab los ntawm sab xis thaum lub teeb liab tawm ntawm lub pob ntseg sab xis taug kev ntev dua li txoj kev los ntawm pob ntseg sab laug, them nyiaj rau cov naj npawb ntawm ITDs, piv txwv li, thaum neuron 2 sib tw. Hauv lwm lo lus, txhua CD teb rau qee yam ITD (tseem hu ua qhov kev ncua sijhawm zoo) vim qhov ncua sij hawm axonal. Yog li, lub hlwb hloov cov ntaub ntawv ntawm lub cev mus rau hauv cov ntaub ntawv spatial. Cov pov thawj Anatomical rau cov txheej txheem no tau pom 37,51. Phase-locked macronucleus neurons khaws cov ntaub ntawv ib ntus hais txog cov suab nrov: raws li lawv lub npe cuam tshuam, lawv tua hluav taws ntawm qee theem teeb liab. Kev sib tsoo detector neurons ntawm Jeffress qauv tuaj yeem pom hauv cov tub ntxhais laminar. Lawv tau txais cov ntaub ntawv los ntawm macronuclear neurons, uas nws axons ua raws li ncua kab. Tus nqi ncua sij hawm muab los ntawm txoj kab ncua sij hawm tuaj yeem piav qhia los ntawm qhov ntev ntawm axon, nrog rau lwm tus qauv myelination uas hloov pauv qhov nrawm nrawm. Kev tshoov siab los ntawm lub suab kaw lus ntawm lub barn owl, peb tau tsim lub tshuab biomimetic rau thaj chaw cov khoom. Ob lub pob ntseg yog sawv cev los ntawm ob lub pMUT txais. Lub suab qhov chaw yog pMUT transmitter nyob nruab nrab ntawm lawv (Daim duab 1a), thiab cov duab kos duab yog tsim los ntawm ib daim phiaj ntawm RRAM-raws li CD circuits (Fig. 1b, ntsuab), ua si lub luag hauj lwm ntawm CD neurons uas nws inputs yog ncua. los ntawm lub voj voog, cov kab qeeb (xiav) ua zoo li axons hauv cov khoom siv roj ntsha. Lub tswv yim kev hnov lus txawv ntawm kev khiav hauj lwm zaus ntawm tus plas, uas nws lub suab ua haujlwm nyob rau hauv 1-8 kHz ntau, tab sis pMUT sensors khiav hauj lwm ntawm txog 117 kHz yog siv nyob rau hauv txoj hauj lwm no. Kev xaiv ntawm ib qho ultrasonic transducer yog txiav txim siab raws li cov txheej txheem technical thiab optimization. Ua ntej, txwv qhov tau txais bandwidth rau ib zaus ib zaug ua kom zoo dua qhov ntsuas qhov tseeb thiab ua kom yooj yim rau cov kauj ruam tom qab ua tiav. Tsis tas li ntawd, kev ua haujlwm hauv ultrasound muaj qhov zoo dua uas cov pulses emitted tsis hnov, yog li tsis txhob cuam tshuam rau tib neeg, txij li lawv qhov kev hnov lus yog ~ 20-20 kHz.
lub barn owl tau txais suab nthwv dej los ntawm lub hom phiaj, nyob rau hauv cov ntaub ntawv no tsiv prey. Lub sijhawm ya davhlau (ToF) ntawm lub suab nthwv dej sib txawv rau txhua lub pob ntseg (tshwj tsis yog cov tsiaj nyeg ncaj qha rau pem hauv ntej ntawm tus noog). Cov kab dotted qhia txoj hauv kev uas lub suab nthwv dej coj mus txog lub pob ntseg pob ntseg. Prey tuaj yeem raug kho kom raug nyob rau hauv lub dav hlau kab rov tav raws li qhov sib txawv ntawm qhov ntev ntawm ob txoj hauv kev acoustic thiab qhov sib thooj ntawm lub sijhawm sib txawv (ITD) (sab laug duab tshwm sim los ntawm ref. 74, copyright 2002, Society for Neuroscience). Hauv peb lub cev, pMUT transmitter (tsaus xiav) tsim suab nthwv dej uas tawm ntawm lub hom phiaj. Reflected ultrasound tsis tau txais los ntawm ob lub pMUT receivers (lub teeb ntsuab) thiab ua tiav los ntawm neuromorphic processor (txoj cai). b Ib qho ITD (Jeffress) cov qauv kev suav piav qhia txog yuav ua li cas lub suab nkag mus rau hauv lub barn owl lub pob ntseg yog thawj zaug encoded raws li theem-locked spikes nyob rau hauv lub loj nucleus (NM) thiab ces siv ib tug geometrically teem kab sib chaws ntawm matched detector neurons nyob rau hauv lub lamellar nucleus. Kev Ua Haujlwm (Netherlands) (sab laug). Cov duab kos duab ntawm neuroITD suav nrog kev sib txuas cov kab ncua sij hawm thiab cov khoom sib koom ua ke ntawm cov neurons, owl biosensor system tuaj yeem ua qauv siv RRAM-based neuromorphic circuits (txoj cai). c Schematic ntawm lub ntsiab Jeffress mechanism, vim qhov sib txawv ntawm ToF, ob lub pob ntseg tau txais lub suab stimuli ntawm lub sijhawm sib txawv thiab xa axons los ntawm ob qho kawg mus rau lub ntes. Cov axons yog ib feem ntawm ib tug series ntawm coincidence detector (CD) neurons, txhua tus uas teb xaiv rau lub sij hawm sib raug zoo inputs. Raws li qhov tshwm sim, tsuas yog CDs uas nws cov khoom siv tuaj txog nrog lub sijhawm sib txawv tsawg tshaj plaws yog qhov txaus siab tshaj plaws (ITD tau them raws nraim). CD yuav ces encode lub hom phiaj angular txoj hauj lwm.
Piezoelectric micromechanical ultrasonic transducers yog scalable ultrasonic transducers uas tuaj yeem ua ke nrog advanced CMOS thev naus laus zis31,32,33,52 thiab muaj qis dua pib voltage thiab lub zog siv dua li cov tsoos volumetric transducers53. Hauv peb txoj haujlwm, lub membrane txoj kab uas hla yog 880 µm, thiab lub resonant zaus yog faib nyob rau hauv ntau ntawm 110–117 kHz (Fig. 2a, saib Methods kom paub meej). Nyob rau hauv ib pawg ntawm kaum xeem, qhov nruab nrab zoo tshaj yog hais txog 50 (ref. 31). Cov thev naus laus zis tau mus txog kev lag luam kom loj hlob thiab tsis yog bioinspired ib se. Kev sib xyaw cov ntaub ntawv los ntawm cov yeeb yaj kiab pMUT sib txawv yog cov txheej txheem paub zoo, thiab cov ntaub ntawv lub kaum sab xis tuaj yeem tau txais los ntawm pMUTs siv, piv txwv li, beamforming techniques31,54. Txawm li cas los xij, lub teeb liab ua tiav yuav tsum tau rho tawm cov ntaub ntawv lub kaum sab xis tsis haum rau kev ntsuas hluav taws xob tsawg. Cov txheej txheem npaj ua ke ua ke cov ntaub ntawv neuromorphic preprocessing circuit pMUT nrog RRAM-based neuromorphic xam daim duab tshwm sim los ntawm Jeffress qauv (Daim duab 2c), muab lwm txoj kev siv hluav taws xob zoo thiab muaj peev xwm-txhim kho kho vajtse. Peb tau ua ib qho kev sim nyob rau hauv uas ob lub pMUT sensors tau muab tso rau kwv yees li 10 cm sib nrug los siv qhov sib txawv ToF suab tau txais los ntawm ob qhov tau txais daim nyias nyias. Ib tug pMUT ua raws li lub transmitter zaum ntawm cov receivers. Lub hom phiaj yog PVC phaj 12 cm dav, nyob ntawm qhov deb D nyob rau hauv pem hauv ntej ntawm pMUT ntaus ntawv (Fig. 2b). Lub receiver sau cov suab uas tshwm sim los ntawm cov khoom thiab reacts ntau npaum li sai tau thaum lub suab nthwv dej. Rov ua qhov kev sim los ntawm kev hloov txoj haujlwm ntawm qhov khoom, txiav txim siab los ntawm qhov deb D thiab lub kaum sab xis θ. Kev tshoov siab los ntawm qhov txuas. 55, peb tawm tswv yim ua ntej neuromorphic ua ntej ntawm pMUT raw cov cim los hloov cov nthwv dej rau hauv cov peaks kom nkag mus rau cov duab neuromorphic suav. Lub ToF sib haum mus rau lub ncov amplitude yog muab rho tawm los ntawm txhua tus ntawm ob txoj kev thiab encoded raws li lub caij nyoog ntawm tus neeg ncov. Ntawm daim duab. 2c qhia tau hais tias lub circuitry yuav tsum tau sib cuam tshuam lub pMUT sensor nrog RRAM-raws li kev suav graph: rau txhua tus ntawm ob lub pMUT txais, cov teeb liab nyoos yog band-pass lim kom du, kho, thiab tom qab ntawd dhau mus rau qhov sib xyaw ua ke hauv kev kov yeej. lub dynamic threshold (Fig. 2d) tsim ib qho kev tshwm sim tso zis (spike) thiab firing (LIF) neuron: lub sij hawm tso zis spike encodes lub sij hawm ya davhlau. LIF qhov pib raug ntsuas tawm tsam pMUT cov lus teb, yog li txo qis pMUT kev hloov pauv ntawm cov cuab yeej mus rau lub cuab yeej. Nrog rau txoj hauv kev no, es tsis txhob khaws tag nrho lub suab nthwv dej hauv lub cim xeeb thiab ua tiav nws tom qab, peb tsuas yog tsim lub ncov sib xws rau ToF ntawm lub suab nthwv dej, uas tsim cov lus nkag mus rau lub cim xeeb kev ua haujlwm. Cov spikes raug xa ncaj qha mus rau cov kab ncua sij hawm thiab ua ke nrog kev sib tw nrhiav pom hauv cov duab neuromorphic suav. Vim hais tias lawv raug xa mus rau lub rooj vag ntawm cov transistors, tsis tas yuav tsum muaj qhov txuas ntxiv txuas ntxiv (saib Daim Duab Ntxiv 4 kom paub meej). Txhawm rau ntsuas qhov ntsuas qhov tseeb hauv cheeb tsam uas tau muab los ntawm pMUT thiab cov txheej txheem teeb tsa ua tiav, peb ntsuas ITD (uas yog, qhov sib txawv ntawm lub sijhawm ntawm cov xwm txheej ncov tsim los ntawm ob tus txais) raws li qhov kev ncua deb thiab lub kaum sab xis ntawm cov khoom sib txawv. Qhov kev tshuaj ntsuam ITD tau hloov mus rau cov ces kaum (saib Cov Txheej Txheem) thiab npaj tawm tsam qhov chaw ntawm cov khoom: qhov tsis paub tseeb hauv qhov ntsuas ITD tau nce nrog qhov deb thiab lub kaum sab xis rau cov khoom (Fig. 2e,f). Qhov teeb meem tseem ceeb yog qhov peak-rau-nruas piv (PNR) hauv pMUT teb. Qhov deb ntawm cov khoom, qhov qis dua lub teeb liab acoustic, yog li txo PNR (Daim duab 2f, kab ntsuab). Kev txo qis hauv PNR ua rau muaj qhov tsis paub tseeb hauv qhov kev kwv yees ntawm ITD, ua rau muaj qhov nce hauv qhov tseeb hauv cheeb tsam (Daim duab 2f, kab xiav). Rau ib qho khoom ntawm qhov deb ntawm 50 cm ntawm lub transmitter, qhov tseeb ntawm lub kaw lus yog kwv yees li 10 °. Qhov kev txwv no los ntawm cov yam ntxwv ntawm lub sensor tuaj yeem txhim kho. Piv txwv li, lub siab xa los ntawm lub emitter tuaj yeem nce ntxiv, yog li ua kom lub zog tsav lub pMUT daim nyias nyias. Lwm qhov kev daws teeb meem los nthuav tawm cov teeb liab kis tau tus mob yog txuas ntau lub transmitters 56. Cov kev daws teeb meem no yuav ua rau kom muaj kev tshawb pom ntau ntxiv ntawm cov nqi ntawm cov nqi hluav taws xob ntau ntxiv. Kev txhim kho ntxiv tuaj yeem ua rau sab txais. Lub pMUT's receiver suab nrov pem teb tuaj yeem txo qis los ntawm kev txhim kho kev sib txuas ntawm pMUT thiab thawj theem amplifier, uas tam sim no ua tiav nrog cov xov hlau sib txuas thiab RJ45 cables.
Daim duab ntawm pMUT siv lead ua nrog rau 880 µm daim nyias nyias ua ke ntawm 1.5 hli suab. b Daim duab ntawm kev ntsuas ntsuas. Lub hom phiaj yog nyob ntawm azimuth txoj hauj lwm θ thiab nyob deb D. Lub pMUT transmitter generates ib 117.6 kHz teeb liab uas bounces tawm lub hom phiaj thiab mus txog ob pMUT receivers nrog txawv sij hawm-ntawm-flight (ToF). Qhov sib txawv no, txhais tau tias yog qhov sib txawv ntawm lub sijhawm sib txawv (ITD), encodes txoj haujlwm ntawm ib qho khoom thiab tuaj yeem kwv yees los ntawm kev kwv yees cov lus teb ntawm ob lub receiver sensors. c Schematic ntawm pre-processing cov kauj ruam rau converting lub raw pMUT teeb liab mus rau spike sequences (piv txwv li cov tswv yim rau lub neuromorphic xam graph). Cov pMUT sensors thiab neuromorphic xam cov duab tau raug tsim thiab sim, thiab cov neuromorphic ua ntej ua raws li software simulation. d Cov lus teb ntawm pMUT daim nyias nyias thaum tau txais ib lub teeb liab thiab nws qhov kev hloov pauv mus rau qhov sib npaug. e Kev sim hauv cheeb tsam qhov tseeb qhov tseeb raws li kev ua haujlwm ntawm lub kaum sab xis (Θ) thiab qhov nrug (D) rau lub hom phiaj khoom. Txoj kev rho tawm ITD yuav tsum muaj qhov kev daws teeb meem tsawg kawg nkaus ntawm kwv yees li 4 ° C. f angular raug (xiav kab) thiab sib thooj peak-rau-suab nrov piv (cov kab ntsuab) piv rau cov khoom nyob deb rau Θ = 0.
Resistive nco khaws cov ntaub ntawv nyob rau hauv lub xeev uas tsis yog-volatile conductive. Lub hauv paus ntsiab lus ntawm txoj kev yog tias kev hloov pauv ntawm cov khoom ntawm qib atomic ua rau muaj kev hloov pauv hauv nws cov hluav taws xob conductivity57. Ntawm no peb siv lub cim xeeb oxide-raws li resistive uas muaj 5nm txheej ntawm hafnium dioxide sandwiched ntawm sab saum toj thiab hauv qab titanium thiab titanium nitride electrodes. Cov khoom siv hluav taws xob ntawm RRAM tuaj yeem hloov pauv los ntawm kev siv lub tshuab hluav taws xob tam sim no / qhov hluav taws xob uas tsim los yog rhuav tshem cov hluav taws xob ntawm cov pa oxygen ntawm cov electrodes. Peb sib koom ua ke xws li cov cuab yeej58 rau hauv tus txheej txheem 130 nm CMOS txheej txheem los tsim cov khoom siv rov ua dua tshiab ntawm cov neuromorphic circuit siv lub ntsuas ntsuas qhov sib txawv thiab cov kab ncua sij hawm (Fig. 3a). Qhov tsis-volatile thiab analog xwm ntawm cov cuab yeej, ua ke nrog cov xwm txheej-tsav ntawm lub neuromorphic Circuit Court, txo hwj chim noj. Lub Circuit Court muaj ib qho instant on/off muaj nuj nqi: nws ua hauj lwm tam sim ntawd tom qab qhib, cia lub hwj chim yuav tsum tau muab tua tag nrho thaum lub Circuit Court tsis ua hauj lwm. Lub hauv paus tsev blocks ntawm lub tswv yim tswv yim yog qhia nyob rau hauv daim duab. 3b ib. Nws muaj N parallel ib leeg-resistor ib leeg-transistor (1T1R) cov qauv uas encode synaptic hnyav los ntawm qhov hnyav tam sim no, txhaj rau hauv cov synapse ntawm ib khub sib txawv (DPI) 59, thiab thaum kawg txhaj rau hauv synapse nrog kev koom ua ke thiab leak. activated (LIF) neuron 60 (saib Methods kom paub meej). Cov input surges yog siv rau lub rooj vag ntawm 1T1R qauv nyob rau hauv daim ntawv ntawm ib theem ntawm voltage pulses nrog ib tug ntev ntawm qhov kev txiav txim ntawm pua pua ntawm nanoseconds. Resistive nco tau muab tso rau hauv lub siab conductive xeev (HCS) los ntawm kev siv ib tug zoo sab nraud siv rau Vtop thaum Vbottom yog grounded, thiab rov qab mus rau ib tug tsawg conductive xeev (LCS) los ntawm kev siv ib tug zoo voltage rau Vbottom thaum Vtop yog grounded. Qhov nruab nrab tus nqi ntawm HCS tuaj yeem tswj tau los ntawm kev txwv qhov programming tam sim no (ua raws) ntawm SET (ICC) los ntawm lub rooj vag-qhov voltage ntawm series transistor (Fig. 3c). Cov haujlwm ntawm RRAM hauv Circuit Court yog ob npaug: lawv ncaj qha thiab hnyav cov khoom siv pulses.
Scanning electron microscope (SEM) duab ntawm xiav HfO2 1T1R RRAM ntaus ntawv kev koom ua ke nyob rau hauv 130 nm CMOS technology nrog selector transistors (650 nm dav) nyob rau hauv ntsuab. b Basic tsev blocks ntawm lub tswv yim neuromorphic schema. Lub input voltage pulses (peaks) Vin0 thiab Vin1 haus tam sim no Iweight, uas yog proportional rau lub conduction xeev G0 thiab G1 ntawm 1T1R qauv. Qhov tam sim no tau txhaj rau hauv DPI synapses thiab ua rau LIF neurons zoo siab. RRAM G0 thiab G1 tau teeb tsa hauv HCS thiab LCS feem. c Kev ua haujlwm ntawm kev sib txuas ua ke kom nruj rau ib pawg ntawm 16K RRAM cov khoom siv raws li kev ua haujlwm ntawm ICC tam sim no sib txuam, uas tswj tau qib kev coj ua zoo. d Circuit Court ntsuas nyob rau hauv (a) qhia tias G1 (hauv LCS) zoo thaiv cov tswv yim los ntawm Vin1 (ntsuab), thiab qhov tseeb cov zis neuron lub membrane voltage teb tsuas yog xiav input los ntawm Vin0. RRAM zoo txiav txim siab cov kev sib txuas hauv Circuit Court. e Kev ntsuas ntawm Circuit Court nyob rau hauv (b) qhia cov nyhuv ntawm tus nqi conductance G0 ntawm daim nyias nyias voltage Vmem tom qab siv lub voltage mem tes Vin0. Qhov kev coj ua ntau dua, cov lus teb muaj zog dua: yog li, RRAM ntaus ntawv siv I / O kev sib txuas hnyav. Kev ntsuas tau ua nyob rau hauv Circuit Court thiab ua kom pom qhov muaj nuj nqi ntawm RRAM, routing thiab hnyav ntawm cov khoom siv pulses.
Ua ntej, vim muaj ob lub xeev kev coj ua yooj yim (HCS thiab LCS), RRAMs tuaj yeem thaiv lossis nco cov tswv yim tawm tswv yim thaum lawv nyob hauv LCS lossis HCS xeev, feem. Yog li ntawd, RRAM zoo txiav txim siab cov kev sib txuas hauv Circuit Court. Qhov no yog lub hauv paus rau muaj peev xwm reconfigure lub architecture. Txhawm rau ua kom pom qhov no, peb yuav piav qhia txog kev siv cov khoom siv hluav taws xob hauv Circuit Court hauv daim duab 3b. Lub RRAM sib raug rau G0 yog programmed rau hauv HCS, thiab qhov thib ob RRAM G1 yog programmed rau hauv LCS. Input pulses yog siv rau ob qho tib si Vin0 thiab Vin1. Cov teebmeem ntawm ob ntu ntawm cov khoom siv pulses tau txheeb xyuas hauv cov zis neurons los ntawm kev sau cov neuron membrane voltage thiab cov teeb liab tso zis siv oscilloscope. Qhov kev sim tau ua tiav thaum tsuas yog HCS cov cuab yeej (G0) txuas nrog lub paj hlwb ntawm lub paj hlwb kom txhawb nqa cov membrane nro. Qhov no yog pom nyob rau hauv daim duab 3d, qhov twg lub xiav mem tes tsheb ciav hlau ua rau lub membrane voltage tsim nyob rau hauv lub membrane capacitor, thaum lub ntsuab pulse tsheb ciav hlau ua rau lub membrane voltage tas li.
Qhov thib ob tseem ceeb ua haujlwm ntawm RRAM yog kev siv cov khoom sib txuas. Siv RRAM qhov kev hloov pauv kev coj ua piv txwv, I / O kev sib txuas tuaj yeem hnyav raws li. Hauv qhov kev sim thib ob, G0 ntaus ntawv tau programmed rau ntau theem ntawm HCS, thiab cov tswv yim pulse tau siv rau VIn0 cov tswv yim. Lub tswv yim mem tes rub tawm tam sim no (Iweight) los ntawm lub cuab yeej, uas yog proportional to conductance and thecorresponding potential drop Vtop − Vbot. Qhov hnyav tam sim no yog tom qab txhaj rau hauv DPI synapses thiab LIF tso tawm cov neurons. Lub membrane voltage ntawm cov zis neurons raug kaw siv lub oscilloscope thiab tso tawm nyob rau hauv daim duab 3d. Lub ncov voltage ntawm lub neuron membrane nyob rau hauv cov lus teb rau ib tug tib input mem tes yog proportional rau lub conductance ntawm resistive nco, qhia tau hais tias RRAM yuav siv tau raws li ib tug programmable caij ntawm synaptic hnyav. Ob qhov kev sim ua ntej no qhia tau hais tias qhov kev thov RRAM-raws li neuromorphic platform muaj peev xwm siv tau cov ntsiab lus ntawm cov txheej txheem Jeffress yooj yim, uas yog cov kab ncua sij hawm thiab qhov sib txuam ntsuas ntsuas Circuit Court. Lub Circuit Court platform yog tsim los ntawm stacking successive blocks ntawm ib sab, xws li cov blocks nyob rau hauv daim duab 3b, thiab txuas lawv lub rooj vag mus rau ib tug ntau input kab. Peb tsim, tsim, thiab sim ib lub platform neuromorphic uas muaj ob lub paj hlwb uas tau txais ob lub tswv yim (Fig. 4a). Daim duab kos duab qhia hauv daim duab 4b. Lub sab sauv 2 × 2 RRAM matrix tso cai rau cov tswv yim pulses coj mus rau ob qhov tso zis neurons, thaum qis dua 2 × 2 matrix tso cai rov qab sib txuas ntawm ob lub neurons (N0, N1). Peb pom tau hais tias lub platform no tuaj yeem siv nrog kev ncua kab teeb tsa thiab ob qhov sib txawv ntawm qhov sib txawv ntawm qhov ntsuas ntsuas, raws li tau pom los ntawm kev sim ntsuas hauv daim duab 4c-e.
Daim duab hluav taws xob tsim los ntawm ob qhov tso zis neurons N0 thiab N1 tau txais ob lub tswv yim 0 thiab 1. Cov plaub sab saum toj ntawm cov array txhais cov kev sib txuas synaptic los ntawm cov tswv yim rau cov zis, thiab plaub lub hlwb hauv qab txhais cov kev sib txuas ntawm cov neurons. Cov xim RRAMs sawv cev rau cov cuab yeej teeb tsa hauv HCS ntawm sab xis: cov khoom siv hauv HCS tso cai rau kev sib txuas thiab sawv cev rau qhov hnyav, thaum cov khoom siv hauv LCS thaiv cov tswv yim pulses thiab lov tes taw kev sib txuas rau cov zis. b Daim duab ntawm Circuit Court (a) nrog yim RRAM modules tseem ceeb hauv xiav. c ncua kab yog tsim los ntawm tsuas yog siv cov dynamics ntawm DPI synapses thiab LIF neurons. RRAM ntsuab yog teem rau conductance siab txaus kom muaj peev xwm induce ib tug glitch ntawm cov zis tom qab lub input ncua sij hawm Δt. d Schematic illustration of direction-insensitive CD nrhiav kom tau lub sij hawm nyob ntawm cov teeb liab. Output neuron 1, N1, hluav taws kub ntawm inputs 0 thiab 1 nrog ncua luv luv. e Direction rhiab CD Circuit Court, ib lub Circuit Court uas kuaj pom thaum input 1 mus txog input 0 thiab tuaj txog tom qab input 0. Cov zis ntawm Circuit Court yog sawv cev los ntawm neuron 1 (N1).
Txoj kab qeeb (Daim duab 4c) tsuas yog siv tus cwj pwm tsis zoo ntawm DPI synapses thiab LIF neurons los tsim cov tswv yim tawm ntawm Vin1 mus rau Vout1 los ntawm kev ncua Tdel. Tsuas yog G3 RRAM txuas nrog Vin1 thiab Vout1 yog programmed hauv HCS, tus so ntawm RRAMs yog programmed hauv LCS. G3 ntaus ntawv tau programmed rau 92.6 µs los xyuas kom meej tias txhua qhov input mem tes nce lub membrane voltage ntawm cov zis neuron txaus kom ncav cuag qhov pib thiab tsim kom muaj cov zis tawm qeeb. Qhov ncua sij hawm Tdel yog txiav txim los ntawm cov synaptic thiab neural lub sij hawm tas mus li. Coincidence detectors ntes tau qhov tshwm sim ntawm ib ntus correlated tab sis spatially faib input signals. Kev taw qhia-insensitive CD tso siab rau ib tus neeg cov khoom siv sib txuas mus rau ib qho kev tso tawm neuron (Daim duab 4d). Ob lub RRAMs txuas Vin0 thiab Vin1 rau Vout1, G2 thiab G4 feem yog programmed rau kev ua haujlwm siab. Tib lub sijhawm tuaj txog ntawm cov spikes ntawm Vin0 thiab Vin1 nce qhov hluav taws xob ntawm N1 neuron daim nyias nyias saum toj ntawm qhov yuav tsum tau ua kom muaj qhov tso zis tawm. Yog hais tias ob lub inputs sib nrug heev nyob rau hauv lub sij hawm, tus nqi ntawm lub membrane voltage sau los ntawm thawj cov tswv yim yuav muaj sij hawm mus rau lwj, tiv thaiv lub membrane muaj peev xwm N1 mus txog qhov pib nqi. G1 thiab G2 yog programmed rau kwv yees li 65 µs, uas ua kom ntseeg tau tias ib qho input surge tsis ua kom cov membrane voltage txaus kom ua rau cov zis tawm. Kev tshawb nrhiav qhov sib txawv ntawm cov xwm txheej uas tau muab faib rau hauv qhov chaw thiab lub sijhawm yog qhov kev ua haujlwm tseem ceeb uas siv rau hauv ntau yam kev paub txog kev paub xws li kev kho qhov muag raws li kev tiv thaiv kev tiv thaiv thiab lub suab hauv zos. Yog li, suav cov kev taw qhia-sensitive thiab tsis hnov tsw CDs yog ib qho tseem ceeb hauv tsev thaiv rau kev tsim kho qhov muag pom thiab audio localization systems. Raws li qhia los ntawm cov yam ntxwv ntawm lub sij hawm tsis tu ncua (saib Daim duab ntxiv 2), lub tswv yim Circuit Court siv ib qho kev tsim nyog ntawm plaub qhov kev txiav txim ntawm qhov loj ntawm lub sij hawm teev. Yog li, nws tuaj yeem ua tau raws li qhov yuav tsum tau ua ntawm kev pom thiab lub suab. Directional-sensitive CD yog ib lub voj voog uas nkag siab rau qhov kev txiav txim siab ntawm kev tuaj txog ntawm pulses: los ntawm sab xis mus rau sab laug thiab rov ua dua. Nws yog ib qho tseem ceeb ntawm lub tsev thaiv nyob rau hauv lub hauv paus ntsiab lus ntawm kev tshawb nrhiav network ntawm Drosophila qhov muag system, siv los xam cov lus taw qhia thiab ntes kev sib tsoo 62. Yuav kom ua tiav cov kev taw qhia-rhiab CD, ob lub tswv yim yuav tsum raug coj mus rau ob lub paj hlwb sib txawv (N0, N1) thiab kev sib txuas lus yuav tsum tau tsim los ntawm lawv (Fig. 4e). Thaum thawj cov tswv yim tau txais, TSIS muaj qhov cuam tshuam los ntawm kev nce qhov hluav taws xob hla nws daim nyias nyias saum tus nqi pib thiab xa tawm qhov nce siab. Qhov kev tshwm sim tso tawm no, nyob rau hauv lem, hluav taws N1 ua tsaug rau cov kev taw qhia kev twb kev txuas tseem ceeb nyob rau hauv ntsuab. Yog hais tias qhov kev tshwm sim tawm tswv yim Vin1 tuaj txog thiab muaj zog N1 thaum nws cov membrane voltage tseem siab, N1 tsim ib qho kev tshwm sim uas qhia tau hais tias qhov sib tw tau pom ntawm ob lub tswv yim. Kev taw qhia kev sib txuas tso cai rau N1 kom tso tawm cov zis tsuas yog tias cov tswv yim 1 tuaj tom qab cov tswv yim 0. G0, G3, thiab G7 yog programmed rau 73.5 µS, 67.3 µS, thiab 40.2 µS, ntsig txog, kom ntseeg tau tias ib qho kev sib tw ntawm cov tswv yim Vin0 ua rau qeeb. tso zis spike, thaum N1's membrane muaj peev xwm tsuas yog ncav cuag qhov pib thaum ob qho kev tawm tswv yim tawg tuaj txog hauv sync. .
Variability yog ib qho ntawm qhov tsis zoo hauv cov qauv neuromorphic systems63,64,65. Qhov no ua rau muaj kev coj cwj pwm ntawm cov neurons thiab synapses. Piv txwv ntawm cov kev tsis zoo no suav nrog 30% (txhais tau tias tus qauv sib txawv) qhov sib txawv ntawm cov tswv yim nce, lub sij hawm tas li, thiab lub sij hawm refractory, rau npe tab sis ob peb (saib Methods). Qhov teeb meem no tseem ceeb tshaj thaum muaj ntau yam neural circuits txuas ua ke, xws li kev taw qhia-sensitive CD uas muaj ob lub neurons. Txhawm rau ua haujlwm kom raug, lub sijhawm nce thiab lwj ntawm ob lub paj hlwb yuav tsum zoo sib xws li qhov ua tau. Piv txwv li, qhov sib txawv loj hauv cov tswv yim nce tuaj yeem ua rau ib qho neuron ua rau muaj kev cuam tshuam rau lub tswv yim pulse thaum lwm cov neuron tsis tshua teb. Ntawm daim duab. Daim duab 5a qhia tau hais tias randomly xaiv neurons teb txawv rau tib lub tswv yim mem tes. Qhov kev hloov pauv neural no muaj feem cuam tshuam, piv txwv li, rau kev ua haujlwm ntawm cov kev taw qhia-sensitive CDs. Nyob rau hauv lub tswvyim qhia nyob rau hauv daim duab. 5b, c, qhov kev nkag siab ntawm neuron 1 yog ntau dua li ntawm neuron 0. Yog li, neuron 0 yuav tsum muaj peb lub tswv yim pulses (es tsis yog 1) kom ncav cuag qhov pib, thiab neuron 1, raws li xav tau, xav tau ob lub ntsiab lus nkag. Kev ua raws li lub sij hawm-dependent biomimetic plasticity (STDP) yog ib txoj hauv kev los txo cov kev cuam tshuam ntawm qhov tsis zoo thiab tsis muaj zog ntawm cov hlab ntsha thiab synaptic circuits ntawm kev ua haujlwm43. Ntawm no peb xav kom siv cov yas coj cwj pwm ntawm kev nco nco ua ib qho kev cuam tshuam rau kev txhim kho ntawm neural input thiab txo cov teebmeem ntawm kev hloov pauv hauv cov kab mob neuromorphic. Raws li qhia hauv daim duab. 4e, theem kev coj ua cuam tshuam nrog RRAM synaptic huab hwm coj tau zoo hloov kho cov lus teb neural membrane voltage. Peb siv qhov rov ua haujlwm RRAM programming zoo. Rau ib qho kev tawm tswv yim, qhov kev coj ua qhov tseem ceeb ntawm qhov hnyav synaptic yog reprogrammed kom txog thaum lub hom phiaj tus cwj pwm tau txais (saib Cov Txheej Txheem).
Ib qho kev sim ntsuas ntawm cov lus teb ntawm cuaj qhov kev xaiv xaiv ib tus neeg neurons rau tib lub tswv yim mem tes. Cov lus teb sib txawv ntawm cov pej xeem, cuam tshuam cov tswv yim nce thiab lub sij hawm tas li. b Kev sim ntsuas qhov cuam tshuam ntawm cov neurons ntawm qhov hloov pauv ntawm cov neurons cuam tshuam rau cov kev taw qhia-sensitive CD. Ob qhov kev taw qhia-rhiab CD tso tawm cov neurons teb txawv rau cov tswv yim stimuli vim neuron-rau-neuron variability. Neuron 0 muaj qhov kev tawm tswv yim qis dua li neuron 1, yog li nws yuav siv peb lub tswv yim pulses (es tsis yog 1) los tsim kom muaj qhov tso zis tawm. Raws li qhov xav tau, neuron 1 nce mus txog qhov pib nrog ob qhov xwm txheej nkag. Yog tias cov tswv yim 1 tuaj txog Δt = 50 µs tom qab neuron 0 hluav taws kub, CD nyob twj ywm vim Δt ntau dua lub sij hawm tas li ntawm neuron 1 (kwv yees 22 µs). c yog txo los ntawm Δt = 20 µs, yog li ntawd input 1 peaks thaum neuron 1′s firing tseem siab, uas ua rau ib txhij nrhiav tau ntawm ob lub input txheej xwm.
Ob lub ntsiab lus siv hauv ITD suav kem yog cov kab qeeb thiab cov kev taw qhia tsis zoo CD. Ob lub circuits yuav tsum tau calibration meej los xyuas kom meej cov khoom zoo positioning kev ua tau zoo. Txoj kab ncua yuav tsum xa ib qho kev ncua ntev ntawm cov tswv yim ncov (Fig. 6a), thiab CD yuav tsum tau qhib tsuas yog thaum cov tswv yim poob rau hauv lub hom phiaj nrhiav kom paub ntau yam. Rau txoj kab ncua sij hawm, qhov hnyav synaptic ntawm cov khoom sib txuas (G3 hauv daim duab 4a) tau reprogrammed kom txog thaum lub hom phiaj ncua tau txais. Teem lub siab ntev nyob ib ncig ntawm lub hom phiaj ncua kom nres qhov kev pab cuam: qhov me dua qhov kam rau ua, qhov nyuaj dua nws yog kom ua tiav txoj kab ncua. Ntawm daim duab. Daim duab 6b qhia txog cov txiaj ntsig ntawm cov txheej txheem ncua sij hawm calibration: nws tuaj yeem pom tau tias cov tswv yim npaj tuaj yeem muab tag nrho cov kev ncua sij hawm xav tau hauv cov qauv tsim (los ntawm 10 mus rau 300 μs). Qhov siab tshaj plaws ntawm calibration iterations cuam tshuam qhov zoo ntawm cov txheej txheem calibration: 200 iterations tuaj yeem txo qhov yuam kev kom tsawg dua 5%. Ib qho calibration iteration sib raug rau qhov teeb tsa / rov pib ua haujlwm ntawm RRAM cell. Cov txheej txheem tuning kuj tseem ceeb heev rau kev txhim kho qhov tseeb ntawm CD module tam sim ze qhov kev tshwm sim. Nws siv kaum calibration iterations kom ua tiav tus nqi zoo (piv txwv li, tus nqi ntawm cov xwm txheej raug txheeb xyuas raws li qhov cuam tshuam) saum toj 95% (xiav kab hauv daim duab 6c). Txawm li cas los xij, txoj kev tu ncua tsis cuam tshuam cov xwm txheej tsis zoo (uas yog, qhov zaus ntawm cov xwm txheej uas raug txheeb xyuas tsis raug raws li qhov cuam tshuam). Lwm txoj kev pom nyob rau hauv cov kab mob lom rau kev kov yeej lub sij hawm txwv ntawm txoj kev ua kom sai sai yog redundancy (uas yog, ntau cov ntawv luam ntawm tib yam khoom siv los ua ib qho kev ua haujlwm). Kev tshoov siab los ntawm biology66, peb muab tso rau ntau lub CD circuits hauv txhua CD module ntawm ob txoj kab qeeb kom txo tau qhov cuam tshuam ntawm qhov tsis zoo. Raws li qhia hauv daim duab. 6c (kab ntsuab), tso peb lub ntsiab lus CD hauv txhua qhov CD module tuaj yeem txo qis lub tswb tsis tseeb kom tsawg dua 10–2.
Qhov cuam tshuam ntawm neuronal variability ntawm ncua kab circuits. b Kev ncua kab hluav taws xob tuaj yeem raug ntsuas kom ncua sij hawm loj los ntawm kev teeb tsa lub sij hawm tsis tu ncua ntawm LIF neurons thiab DPI synapses rau cov txiaj ntsig loj. Kev nce tus naj npawb ntawm iterations ntawm RRAM calibration txheej txheem ua kom nws muaj peev xwm txhim kho qhov tseeb ntawm lub hom phiaj ncua sij hawm: 200 iterations txo qhov yuam kev tsawg dua 5%. Ib qho iteration sib raug rau SET / RESET ua haujlwm ntawm RRAM cell. Txhua CD module hauv c Jeffress qauv tuaj yeem siv tau siv N parallel CD cov ntsiab lus kom yooj yim dua nrog rau kev ua tsis tiav. d Ntau RRAM calibration iterations nce qhov tseeb qhov zoo (xiav kab), thaum tus nqi tsis zoo yog ywj siab ntawm tus naj npawb ntawm iterations (kab ntsuab). Muab ntau CD cov ntsiab lus nyob rau hauv parallel kom tsis txhob muaj qhov yuam kev ntawm CD module qhov sib tw.
Tam sim no peb ntsuas qhov kev ua tau zoo thiab kev siv hluav taws xob ntawm qhov kawg-rau-kawg kev sib koom ua ke cov khoom hauv cheeb tsam uas pom hauv daim duab 2 siv kev ntsuas ntawm cov khoom siv suab nrov ntawm pMUT sensor, CD, thiab ncua kab hluav taws xob uas ua rau cov duab neuromorphic xam. Jeffress qauv (Fig. 1a). Raws li rau neuromorphic xam daim duab, ntau dua cov CD modules, qhov zoo dua qhov kev daws teeb meem, tab sis kuj yog qhov siab dua lub zog ntawm lub cev (Fig. 7a). Kev sib haum xeeb tuaj yeem ncav cuag los ntawm kev sib piv qhov tseeb ntawm tus kheej cov khoom (pMUT sensors, neurons, thiab synaptic circuits) nrog rau qhov tseeb ntawm tag nrho cov kab ke. Qhov kev daws teeb meem ntawm txoj kab ncua yog txwv los ntawm lub sij hawm tas mus li ntawm cov simulated synapses thiab neurons, uas nyob rau hauv peb lub tswv yim tshaj 10 µs, uas sib haum mus rau ib tug angular daws teeb meem ntawm 4 ° (saib Methods). Cov nodes siab dua nrog CMOS thev naus laus zis yuav tso cai rau kev tsim cov neural thiab synaptic circuits nrog cov sij hawm qis dua, ua rau muaj qhov tseeb dua ntawm cov kab lus qeeb. Txawm li cas los xij, hauv peb lub cev, qhov tseeb yog txwv los ntawm qhov yuam kev pMUT hauv kev kwv yees txoj haujlwm angular, piv txwv li 10 ° (xiav kab rov tav kab hauv daim duab 7a). Peb kho tus naj npawb ntawm CD modules ntawm 40, uas sib haum rau ib qho kev daws teeb meem ntawm 4 °, piv txwv li, qhov tseeb angular ntawm daim duab suav (lub teeb xiav kab rov tav kab hauv daim duab 7a). Nyob rau theem system, qhov no muab qhov kev daws teeb meem ntawm 4 ° thiab qhov tseeb ntawm 10 ° rau cov khoom nyob 50 cm nyob rau hauv pem hauv ntej ntawm lub sensor system. Tus nqi no yog piv rau cov neuromorphic suab localization tshuab qhia hauv ref. 67. Ib qho kev sib piv ntawm cov txheej txheem npaj nrog lub xeev ntawm cov kos duab tuaj yeem pom nyob rau hauv Cov Lus Ntxiv 1. Ntxiv pMUTs ntxiv, ua kom cov suab nrov nce qib, thiab txo cov suab nrov hauv hluav taws xob yog ua tau txoj hauv kev los txhim kho qhov tseeb hauv cheeb tsam. ) yog kwv yees li ntawm 9.7. nz. 55. Muab 40 CD chav nyob rau ntawm daim duab suav, SPICE simulation kwv yees lub zog ntawm ib qho kev ua haujlwm (piv txwv li, lub zog ntawm qhov khoom) yog 21.6 nJ. Cov kab mob neuromorphic tsuas yog qhib thaum muaj qhov kev tshwm sim tuaj txog, piv txwv li thaum lub suab nrov nthwv dej mus txog ib qho pMUT receiver thiab tshaj qhov pib tshawb pom, txwv tsis pub nws tseem tsis ua haujlwm. Qhov no zam kev siv hluav taws xob tsis tsim nyog thaum tsis muaj cov teeb liab tawm tswv yim. Xav txog qhov zaus ntawm kev ua haujlwm hauv cheeb tsam ntawm 100 Hz thiab lub sijhawm ua haujlwm ntawm 300 µs ib qho kev ua haujlwm (qhov siab tshaj plaws ua tau ITD), lub zog noj ntawm cov paj hlwb neuromorphic suav yog 61.7 nW. Nrog neuromorphic ua ntej siv rau txhua tus txais pMUT, lub zog noj ntawm tag nrho cov kab ke mus txog 81.6 nW. Txhawm rau kom nkag siab txog lub zog ua haujlwm ntawm cov lus pom zoo neuromorphic txoj hauv kev piv rau cov khoom siv ib txwm siv, peb muab cov lej no rau lub zog uas yuav tsum tau ua tib yam haujlwm ntawm lub tshuab hluav taws xob tsis tshua muaj hluav taws xob niaj hnub siv cov neuromorphic lossis cov pa beamforming68 Skill. Txoj hauv kev neuromorphic txiav txim siab txog theem analog-rau-digital converter (ADC), tom qab los ntawm kev lim band-pass thiab lub hnab ntawv rho tawm theem (Teeger-Kaiser method). Thaum kawg, qhov pib ua haujlwm yog ua kom rho tawm ToF. Peb tau tshem tawm qhov kev suav ntawm ITD raws li ToF thiab kev hloov pauv mus rau kwv yees txoj haujlwm angular txij li qhov no tshwm sim ib zaug rau txhua qhov kev ntsuas (saib Cov Txheej Txheem). Piv txwv li tus nqi piv txwv ntawm 250 kHz ntawm ob txoj kev (pMUT receivers), 18 band pass lim kev ua haujlwm, 3 lub hnab ntawv rho tawm haujlwm, thiab 1 qhov pib ua haujlwm rau ib tus qauv, tag nrho cov kev siv hluav taws xob yog kwv yees li ntawm 245 microwatts. Qhov no siv lub microcontroller's low-power mode69, uas tig rau thaum lub algorithms tsis ua, uas txo hwj chim noj mus rau 10.8 µW. Kev siv hluav taws xob ntawm lub teeb liab beamforming kev daws teeb meem tau thov hauv qhov siv. 31, nrog 5 pMUT receivers thiab 11 beams uniformly faib nyob rau hauv lub azimuth dav hlau [-50°, +50°], yog 11.71 mW (saib cov txheej txheem rau cov ntsiab lus). Tsis tas li ntawd, peb tshaj tawm cov kev siv hluav taws xob ntawm FPGA47-based Time Difference Encoder (TDE) kwv yees ntawm 1.5 mW raws li kev hloov pauv rau Jeffress qauv rau cov khoom hauv zos. Raws li cov kev kwv yees no, txoj kev xav neuromorphic txo qis kev siv hluav taws xob los ntawm tsib qhov kev txiav txim siab piv rau microcontroller siv cov txheej txheem beamforming classical rau cov khoom siv hauv zos. Kev siv txoj hauv kev neuromorphic rau kev teeb tsa ua haujlwm ntawm lub microcontroller classic txo qis kev siv hluav taws xob los ntawm kwv yees li ob qhov kev txiav txim siab. Qhov kev ua tau zoo ntawm qhov kev thov tuaj yeem piav qhia los ntawm kev sib xyaw ua ke ntawm asynchronous resistive-nco analog circuit muaj peev xwm ua haujlwm hauv-nco suav thiab tsis muaj kev hloov pauv analog-rau-digital xav tau kom pom cov cim.
ib qho kev daws teeb meem Angular (xiav) thiab kev siv hluav taws xob (ntsuab) ntawm kev ua haujlwm hauv cheeb tsam nyob ntawm seb muaj pes tsawg tus CD modules. Qhov tsaus xiav kab rov tav bar sawv cev rau qhov tseeb qhov tseeb ntawm PMUT thiab lub teeb xiav kab rov tav bar sawv cev rau qhov tseeb ntawm qhov tseeb ntawm cov duab neuromorphic. b Kev siv hluav taws xob ntawm cov txheej txheem tau hais tseg thiab sib piv nrog ob qhov kev sib tham txog kev siv microcontroller thiab kev siv digital ntawm Lub Sijhawm Difference Encoder (TDE)47 FPGA.
Txhawm rau txo qis kev siv hluav taws xob ntawm lub hom phiaj hauv cheeb tsam, peb tau tsim, tsim thiab siv qhov ua tau zoo, kev tshwm sim-tsav RRAM-raws li neuromorphic circuit uas ua cov ntaub ntawv teeb liab tsim los ntawm cov sensors built-in los xam qhov chaw ntawm lub hom phiaj ntawm qhov khoom tiag. sij hawm. . Thaum cov txheej txheem ua haujlwm ib txwm ua tsis tu ncua cov qauv kuaj pom cov teeb liab thiab ua cov kev suav kom rho tawm cov ntaub ntawv muaj txiaj ntsig, cov lus pom zoo neuromorphic ua kev suav asynchronously raws li cov ntaub ntawv muaj txiaj ntsig tuaj txog, ua kom lub zog ua haujlwm tau zoo los ntawm tsib qhov kev txiav txim ntawm qhov loj. Tsis tas li ntawd, peb qhia txog qhov hloov pauv ntawm RRAM-based neuromorphic circuits. Lub peev xwm ntawm RRAM hloov pauv kev coj ua yam tsis hloov pauv (plasticity) them rau qhov sib txawv ntawm qhov sib txawv ntawm ultra-low power analog DPI's synaptic thiab neural circuits. Qhov no ua rau RRAM-raws li Circuit Court muaj ntau yam thiab muaj zog. Peb lub hom phiaj tsis yog kom rho tawm cov haujlwm nyuaj lossis cov qauv ntawm cov teeb liab, tab sis txhawm rau txheeb xyuas cov khoom hauv lub sijhawm. Peb lub kaw lus tseem tuaj yeem ua kom zoo dua cov teeb liab thiab thaum kawg xa nws mus rau cov txheej txheem ua haujlwm ntxiv kom txiav txim siab nyuaj dua thaum xav tau. Hauv cov ntsiab lus ntawm cov ntawv thov hauv zos, peb cov txheej txheem neuromorphic preprocessing tuaj yeem muab cov ntaub ntawv hais txog qhov chaw ntawm cov khoom. Cov ntaub ntawv no tuaj yeem siv tau, piv txwv li, rau kev tshawb nrhiav kev txav los yog kev paub txog kev piav tes piav taw. Peb hais txog qhov tseem ceeb ntawm kev sib txuas cov khoom siv hluav taws xob tsis tshua muaj zog xws li pMUTs nrog cov khoom siv hluav taws xob qis qis. Rau qhov no, cov txheej txheem neuromorphic yog qhov tseem ceeb raws li lawv tau coj peb los tsim cov kev siv hluav taws xob tshiab ntawm cov txheej txheem kev siv tshuab lom neeg xws li Jeffress qauv. Nyob rau hauv cov ntsiab lus ntawm kev siv sensor fusion, peb lub kaw lus tuaj yeem ua ke nrog ntau qhov sib txawv ntawm cov xwm txheej raws li cov khoom siv kom tau txais cov ntaub ntawv raug ntau dua. Txawm hais tias owls yog qhov zoo tshaj plaws ntawm kev nrhiav tsiaj nyob hauv qhov tsaus ntuj, lawv muaj qhov muag pom tau zoo thiab ua kom pom kev sib xyaw ua ke thiab pom kev tshawb nrhiav ua ntej ntes prey70. Thaum ib qho kev hnov mob neuron hluav taws kub hnyiab, tus plas tau txais cov ntaub ntawv nws xav tau los txiav txim siab nyob rau hauv qhov kev taw qhia kom pib nws qhov kev tshawb nrhiav pom, yog li tsom nws cov xim ntawm ib feem me me ntawm qhov pom kev. Kev sib xyaw ua ke ntawm qhov muag pom (DVS lub koob yees duab) thiab cov lus pom zoo mloog (raws li pMUT) yuav tsum tau tshawb nrhiav rau kev txhim kho ntawm cov neeg ua haujlwm ywj pheej yav tom ntej.
Lub pMUT sensor nyob rau ntawm PCB nrog ob lub receivers kwv yees li 10 cm sib nrug, thiab lub transmitter nyob nruab nrab ntawm cov receivers. Hauv kev ua haujlwm no, txhua daim nyias nyias yog ib qho kev ncua bimorph qauv uas muaj ob txheej ntawm piezoelectric aluminium nitride (AlN) 800 nm tuab sandwiched ntawm peb txheej molybdenum (Mo) 200 nm tuab thiab coated nrog ib txheej 200 nm tuab. sab saum toj passivating SiN txheej raws li tau piav nyob rau hauv kev siv. 71. Cov electrodes sab hauv thiab sab nrauv yog siv rau hauv qab thiab sab saum toj txheej ntawm molybdenum, thaum nruab nrab molybdenum electrode yog unpatterned thiab siv los ua hauv av, uas ua rau ib daim nyias nyias nrog plaub khub ntawm electrodes.
Qhov kev tsim qauv no tso cai rau kev siv cov membrane deformation, uas ua rau kev sib kis tau zoo dua thiab tau txais kev nkag siab zoo. Xws li pMUT feem ntau nthuav tawm qhov kev xav tau ntawm 700 nm / V raws li cov emitter, muab qhov siab ntawm 270 Pa / V. Raws li ib tug receiver, ib tug pMUT zaj duab xis qhia ib tug luv luv Circuit Court rhiab heev ntawm 15 nA / Pa, uas yog ncaj qha mus rau lub piezoelectric coefficient ntawm AlN. Qhov kev hloov pauv ntawm qhov hluav taws xob hauv AlN txheej ua rau muaj kev hloov pauv ntawm qhov zaus resonant, uas tuaj yeem raug them los ntawm kev siv DC kev tsis ncaj ncees rau pMUT. DC rhiab heev yog ntsuas ntawm 0.5 kHz / V. Rau kev ua tus cwj pwm acoustic, lub microphone yog siv rau pem hauv ntej ntawm pMUT.
Txhawm rau ntsuas lub ncha mem tes, peb muab tso rau ib lub phaj nrog thaj tsam li ntawm 50 cm2 nyob rau hauv pem hauv ntej ntawm pMUT kom muaj kev cuam tshuam cov suab nrov nthwv dej. Ob qhov kev ncua deb ntawm daim hlau thiab lub kaum sab xis txheeb ze rau lub dav hlau pMUT yog tswj los ntawm cov tuav tshwj xeeb. Lub Tectronix CPX400DP voltage qhov kev tsis txaus ntseeg peb pMUT daim nyias nyias, kho qhov zaus resonant mus rau 111.9 kHz31, thaum lub transmitters yog tsav los ntawm Tectronix AFG 3102 mem tes generator tuned rau lub resonant zaus (111.9 kHz. Cov tam sim no nyeem los ntawm plaub qhov chaw nres nkoj ntawm txhua tus pMUT txais tau hloov mus rau qhov hluav taws xob siv qhov tshwj xeeb sib txawv tam sim no thiab qhov hluav taws xob hluav taws xob, thiab cov txiaj ntsig tau tshwm sim yog digitized los ntawm Spektrum cov ntaub ntawv nrhiav tau. Qhov kev txwv ntawm kev kuaj pom tau tshwm sim los ntawm pMUT teeb liab tau txais nyob rau hauv cov xwm txheej sib txawv: peb tau txav lub reflector mus rau qhov sib txawv [30, 40, 50, 60, 80, 100] cm thiab hloov lub pMUT txhawb lub kaum sab xis ([0, 20, 40] o ) Daim duab 2b qhia txog qhov kev daws teeb meem ntawm lub cev ITD nyob ntawm seb qhov sib thooj angular nyob rau hauv degrees.
Kab lus no siv ob qhov sib txawv tawm ntawm lub txee RRAM circuits. Thawj yog ib qho array ntawm 16,384 (16,000) li (128 × 128 pab kiag li lawm) nyob rau hauv ib tug 1T1R configuration nrog ib tug transistor thiab ib tug resistor. Qhov thib ob nti yog cov neuromorphic platform qhia hauv daim duab 4a. RRAM cell muaj 5 nm tuab HfO2 zaj duab xis kos rau hauv TiN / HfO2 / Ti / TiN pawg. Cov pawg RRAM tau koom ua ke rau hauv cov kab rov qab (BEOL) ntawm tus txheej txheem 130nm CMOS. RRAM-raws li neuromorphic circuits nthuav tawm qhov kev sib tw tsim qauv rau txhua lub tshuab hluav taws xob analog uas RRAM cov cuab yeej koom ua ke nrog ib txwm siv CMOS thev naus laus zis. Hauv particular, lub xeev conduction ntawm RRAM ntaus ntawv yuav tsum tau nyeem thiab siv raws li ib tug muaj nuj nqi variable rau lub system. Txog rau qhov kawg no, ib lub voj voog tsim, tsim thiab sim uas nyeem cov tam sim no los ntawm lub cuab yeej thaum lub tswv yim mem tes tau txais thiab siv qhov tam sim no los luj cov lus teb ntawm ib tug sib txawv khub integrator (DPI) synapse. Qhov no Circuit Court yog qhia nyob rau hauv daim duab 3a, uas sawv cev rau lub hauv paus tsev blocks ntawm lub neuromorphic platform nyob rau hauv daim duab 4a. Lub tswv yim mem tes qhib lub rooj vag ntawm 1T1R ntaus ntawv, inducing ib tug tam sim no los ntawm RRAM proportional rau lub ntaus ntawv tus conductance G (Iweight = G (Vtop – Vx)). Lub inverting input ntawm lub lag luam amplifier (op-amp) Circuit Court muaj qhov tsis tu ncua DC bias voltage Vtop. Lub op-amp cov lus tawm tswv yim tsis zoo yuav muab Vx = Vtop los ntawm kev muab qhov sib npaug tam sim no los ntawm M1. Tam sim no Iweight tau txais los ntawm lub cuab yeej yog txhaj rau hauv DPI synapse. Lub zog tam sim no yuav ua rau muaj kev cuam tshuam ntau dua, yog li RRAM kev coj ua tau zoo siv qhov hnyav synaptic. Qhov exponential synaptic tam sim no yog txhaj los ntawm cov membrane capacitor ntawm Leaky Integration thiab Excitation (LIF) neurons, qhov twg nws yog kev sib xyaw ua ke. Yog hais tias qhov pib voltage ntawm daim nyias nyias (qhov hloov hluav taws xob ntawm lub inverter) tau kov yeej, qhov tso zis ntawm cov neuron yog qhib, ua rau cov zis tawm. Qhov no mem tes rov qab thiab shunts lub neuron lub membrane capacitor rau hauv av, ua rau nws tawm. Qhov no Circuit Court yog ntxiv nrog ib tug mem tes expander (tsis muaj nyob rau hauv daim duab. 3a), uas shapes cov zis mem tes ntawm LIF neuron mus rau lub hom phiaj pulse dav. Multiplexers kuj tau tsim rau hauv txhua kab, tso cai rau qhov hluav taws xob siv rau sab saum toj thiab hauv qab electrodes ntawm RRAM ntaus ntawv.
Kev ntsuas hluav taws xob suav nrog kev tshuaj xyuas thiab sau cov cwj pwm tsis zoo ntawm analog circuits, nrog rau kev ua haujlwm thiab nyeem RRAM cov khoom siv. Ob leeg cov kauj ruam yuav tsum muaj cov cuab yeej tshwj xeeb, tag nrho cov uas txuas nrog lub rooj tsavxwm sensor tib lub sijhawm. Kev nkag mus rau RRAM li hauv cov kab hluav taws xob neuromorphic yog nqa los ntawm cov cuab yeej sab nraud los ntawm multiplexer (MUX). Lub MUX cais 1T1R ntawm tes los ntawm qhov seem ntawm lub voj voog uas nws nyob, tso cai rau cov cuab yeej nyeem thiab / lossis programmed. Txhawm rau ua haujlwm thiab nyeem cov khoom siv RRAM, Keithley 4200 SCS tshuab yog siv ua ke nrog Arduino microcontroller: thawj zaug rau kev tsim cov mem tes thiab nyeem tam sim no, thiab qhov thib ob rau kev nkag mus ceev rau ib tus neeg 1T1R cov ntsiab lus hauv lub cim xeeb array. Thawj qhov kev ua haujlwm yog tsim cov khoom siv RRAM. Cov hlwb raug xaiv ib tug los ntawm ib tug thiab ib tug zoo voltage yog siv ntawm sab saum toj thiab hauv qab electrodes. Nyob rau hauv cov ntaub ntawv no, tam sim no yog txwv rau qhov kev txiav txim ntawm kaum ntawm microamperes vim hais tias ntawm cov khoom ntawm lub rooj vag voltage coj mus rau lub selector transistor. Lub RRAM cell tuaj yeem mus ncig ntawm lub xeev qis (LCS) thiab lub xeev siab (HCS) siv RESET thiab SET haujlwm, raws li. Kev ua haujlwm SET yog ua los ntawm kev siv lub voj voos hluav taws xob nrog lub sijhawm ntawm 1 μs thiab qhov siab tshaj plaws ntawm 2.0-2.5 V rau lub tshuab hluav taws xob sab saud, thiab lub tshuab hluav taws xob sib txuas ntawm cov duab zoo sib xws nrog lub ncov voltage ntawm 0.9-1.3 V rau lub rooj vag ntawm lub selector transistor. Cov txiaj ntsig no tso cai rau kev hloov pauv RRAM kev coj ua ntawm 20-150 µs ntu. Rau RESET, 1 µs dav, 3 V ncov mem tes yog siv rau hauv qab electrode (ntsis kab) ntawm lub xovtooj thaum lub rooj vag voltage nyob rau hauv thaj tsam ntawm 2.5-3.0 V. Cov inputs thiab outputs ntawm cov analog circuits yog dynamic signals. . Rau cov tswv yim, peb interleaved ob HP 8110 mem tes generators nrog Tektronix AFG3011 teeb liab generators. Lub input mem tes muaj qhov dav ntawm 1 µs thiab nce / poob ntug ntawm 50 ns. Hom mem tes no yog suav tias yog ib qho glitch nyob rau hauv analog glitch raws circuits. Raws li rau lub teeb liab tso zis, lub teeb liab tso zis tau kaw siv Teledyne LeCroy 1 GHz oscilloscope. Qhov kev tau txais ceev ntawm oscilloscope tau raug pov thawj tsis yog qhov txwv tsis pub muaj nyob rau hauv kev tsom xam thiab tau txais cov ntaub ntawv hauv Circuit Court.
Siv cov dynamics ntawm analog electronics los simulate tus cwj pwm ntawm neurons thiab synapses yog ib qho zoo nkauj thiab muaj txiaj ntsig kev daws teeb meem los txhim kho kev ua tau zoo. Qhov tsis zoo ntawm qhov kev suav hauv qab no yog tias nws yuav sib txawv ntawm lub tswv yim mus rau lub tswv yim. Peb ntsuas qhov sib txawv ntawm cov neurons thiab synaptic circuits (Ntxiv daim duab 2a,b). Ntawm tag nrho cov kev tshwm sim ntawm kev sib txawv, cov uas cuam tshuam nrog lub sij hawm tas mus li thiab cov tswv yim tau txais muaj kev cuam tshuam loj tshaj plaws ntawm qib system. Lub sij hawm tas li ntawm LIF neuron thiab DPI synapse yog txiav txim los ntawm RC Circuit Court, qhov twg tus nqi ntawm R yog tswj los ntawm kev tsis ncaj ncees voltage siv rau lub rooj vag ntawm lub transistor (Vlk rau neuron thiab Vtau rau lub synapse), txiav txim siab qhov tus nqi xau. Input nce yog txhais raws li lub ncov voltage mus txog los ntawm lub synaptic thiab neuronal membrane capacitors stimulated los ntawm ib tug input mem tes. Lub input nce yog tswj los ntawm lwm yam bias transistor uas modulates cov input tam sim no. Lub Monte Carlo simulation calibrated ntawm ST Microelectronics '130nm txheej txheem tau ua los sau qee cov tswv yim nce thiab sijhawm tas li cov txheeb cais. Cov txiaj ntsig tau nthuav tawm hauv Daim Duab Ntxiv 2, qhov twg cov tswv yim nce thiab lub sij hawm tas li raug suav tias yog qhov ua haujlwm ntawm qhov tsis sib xws ntawm qhov ntsuas qhov ntsuas qhov to. Cov cim ntsuab qhia qhov txawv ntawm tus qauv ntawm lub sij hawm tas li los ntawm qhov nruab nrab. Ob leeg neurons thiab synaptic circuits muaj peev xwm qhia tau ntau yam ntawm lub sij hawm tas mus li nyob rau hauv qhov ntau ntawm 10-5-10-2 s, raws li qhia nyob rau hauv Supplementary Fig. Input amplification (Ntxiv daim duab 2e,d) ntawm neuronal thiab synapse variability yog kwv yees li 8% thiab 3%, feem. Xws li qhov tsis txaus yog sau tseg zoo nyob rau hauv cov ntaub ntawv: ntau yam kev ntsuas tau ua nyob rau hauv array ntawm DYNAP chips los ntsuas qhov tsis sib xws ntawm cov pejxeem ntawm LIF63 neurons. Cov synapses hauv BrainScale sib xyaw cov teeb liab nti tau ntsuas thiab lawv qhov tsis sib xws tau txheeb xyuas, thiab cov txheej txheem ntsuas tau raug npaj kom txo qis cov txiaj ntsig ntawm qhov system-theem variability64.
Kev ua haujlwm ntawm RRAM hauv cov kab hluav taws xob neuromorphic yog ob npaug: lub ntsiab lus architecture (routing inputs to outputs) thiab kev siv ntawm synaptic hnyav. Cov cuab yeej tom kawg tuaj yeem siv los daws qhov teeb meem ntawm qhov hloov pauv ntawm cov qauv neuromorphic circuits. Peb tau tsim cov txheej txheem calibration yooj yim uas suav nrog reprogramming RRAM ntaus ntawv kom txog rau thaum lub Circuit Court raug soj ntsuam raws li qee qhov yuav tsum tau ua. Rau ib qho kev tawm tswv yim, cov zis tau saib xyuas thiab RRAM yog reprogrammed kom txog thaum lub hom phiaj tus cwj pwm ua tiav. Lub sijhawm tos ntawm 5 s tau qhia txog kev ua haujlwm ntawm kev ua haujlwm los daws qhov teeb meem ntawm RRAM so uas ua rau muaj kev hloov pauv hloov pauv hloov pauv (Cov Ntaub Ntawv Ntxiv). Synaptic luj yog kho los yog calibrated raws li qhov yuav tsum tau ntawm lub neuromorphic Circuit Court ua qauv. Cov txheej txheem calibration tau sau tseg hauv cov algorithms ntxiv [1, 2] uas tsom mus rau ob qhov tseem ceeb ntawm cov neuromorphic platforms, ncua kab thiab cov kev taw qhia tsis zoo CD. Rau ib lub voj voog nrog txoj kab qeeb, lub hom phiaj tus cwj pwm yog muab cov khoom siv hluav taws xob nrog kev ncua Δt. Yog hais tias qhov tseeb Circuit Court ncua tsawg dua li lub hom phiaj tus nqi, qhov hnyav synaptic ntawm G3 yuav tsum raug txo qis (G3 yuav tsum tau rov pib dua thiab tom qab ntawd muab tso rau qis dua tam sim no Icc). Hloov pauv, yog tias qhov kev ncua sij hawm ntau dua li lub hom phiaj tus nqi, qhov kev coj ua ntawm G3 yuav tsum tau nce ntxiv (G3 yuav tsum tau rov pib dua ua ntej thiab tom qab ntawd teem rau tus nqi siab dua Icc). Cov txheej txheem no rov ua dua kom txog rau thaum qhov ncua sij hawm tsim los ntawm lub voj voog sib tw rau lub hom phiaj tus nqi thiab kev kam rau siab tau teem tseg kom nres cov txheej txheem calibration. Rau kev taw qhia-insensitive CDs, ob lub RRAM li, G1 thiab G3, tau koom nrog hauv cov txheej txheem calibration. Qhov Circuit Court no muaj ob lub tswv yim, Vin0 thiab Vin1, ncua los ntawm dt. Lub Circuit Court yuav tsum tsuas yog teb rau qeeb qeeb hauv qab qhov sib piv [0,dtCD]. Yog tias tsis muaj qhov tso tawm qhov siab tshaj plaws, tab sis qhov kev nkag siab siab yog nyob ze, ob qho tib si RRAM yuav tsum tau txhawb nqa los pab cov neuron mus txog qhov pib. Piv txwv li, yog hais tias lub Circuit Court teb rau ib tug ncua sij hawm uas tshaj lub hom phiaj ntawm dtCD, cov conductance yuav tsum tau txo. Rov ua cov txheej txheem kom txog thaum tau txais tus cwj pwm raug. Kev ua raws li tam sim no tuaj yeem hloov kho los ntawm kev tsim-hauv analog circuit hauv ref. 72.73 ib. Nrog rau qhov no built-in Circuit Court, xws li cov txheej txheem yuav ua tau ib ntus kom calibrate lub system los yog rov siv nws rau lwm daim ntawv thov.
Peb ntsuas lub hwj chim noj ntawm peb cov teeb liab neuromorphic ua mus kom ze ntawm tus qauv 32-ntsis microcontroller68. Hauv qhov kev ntsuam xyuas no, peb xav tias kev ua haujlwm nrog kev teeb tsa tib yam li hauv daim ntawv no, nrog ib lub pMUT transmitter thiab ob lub pMUT txais. Txoj kev no siv lub lim bandpass, ua raws li lub hnab ntawv rho tawm cov kauj ruam (Teeger-Kaiser), thiab thaum kawg ib qho kev pib ua haujlwm yog siv rau lub teeb liab kom rho tawm lub sijhawm ntawm lub davhlau. Kev suav ntawm ITD thiab nws qhov kev hloov pauv mus rau kev tshawb pom cov ces kaum raug tshem tawm hauv kev ntsuam xyuas. Peb xav txog kev siv band pass lim siv qhov kev txiav txim thib 4 infinite impulse teb lim xav tau 18 ntab taw tes ua haujlwm. Lub hnab ntawv rho tawm siv peb qhov kev ua haujlwm ntab ntxiv, thiab kev ua haujlwm kawg yog siv los teeb tsa qhov pib. Tag nrho ntawm 22 ntab taw tes ua haujlwm yuav tsum tau ua ua ntej lub teeb liab. Cov teeb liab kis tau yog luv luv ntawm 111.9 kHz sine waveform tsim txhua 10 ms uas ua rau qhov chaw ua haujlwm zaus ntawm 100 Hz. Peb siv tus nqi piv txwv ntawm 250 kHz kom ua raws li Nyquist thiab 6 ms qhov rais rau txhua qhov kev ntsuas kom ntes tau ntau yam ntawm 1 meter. Nco ntsoov tias 6 milliseconds yog lub sij hawm ya ntawm ib yam khoom uas yog 1 meter deb. Qhov no muab lub zog noj ntawm 180 µW rau A / D hloov pauv ntawm 0.5 MSPS. Lub teeb liab ua ntej yog 6.60 MIPS (cov lus qhia ib ob), tsim 0.75 mW. Txawm li cas los xij, lub microcontroller tuaj yeem hloov mus rau hom qis zog 69 thaum lub algorithm tsis ua haujlwm. Hom no muab kev siv hluav taws xob zoo li qub ntawm 10.8 μW thiab lub sijhawm sawv ntawm 113 μs. Muab lub moos zaus ntawm 84 MHz, lub microcontroller ua tiav tag nrho cov haujlwm ntawm cov neuromorphic algorithm li ntawm 10 ms, thiab cov algorithm xam lub luag haujlwm voj voog ntawm 6.3%, yog li siv lub zog qis. Lub zog dissipation yog 244.7 μW. Nco ntsoov tias peb tshem tawm ITD cov zis los ntawm ToF thiab hloov dua siab tshiab rau lub kaum sab xis kom pom, yog li kev kwv yees lub zog siv ntawm microcontroller. Qhov no muab tus nqi ntxiv rau lub zog ua haujlwm ntawm qhov kev thov. Raws li ib qho kev sib piv ntxiv, peb ntsuas qhov kev siv hluav taws xob ntawm cov txheej txheem beamforming classical tau npaj tseg hauv qhov kev siv. 31.54 thaum embedded nyob rau hauv tib lub microcontroller68 ntawm 1.8V mov voltage. Tsib qhov sib npaug sib npaug pMUT daim nyias nyias yog siv los txais cov ntaub ntawv rau beamforming. Raws li rau kev ua nws tus kheej, txoj kev siv beamforming yog ncua sijhawm. Nws tsuas yog siv qhov ncua sij hawm rau txoj kab uas sib haum mus rau qhov xav tau qhov sib txawv ntawm lub sij hawm tuaj txog ntawm ib txoj kab thiab txoj kab siv. Yog tias cov cim qhia nyob rau theem, cov lej ntawm cov cim no yuav muaj lub zog siab tom qab lub sijhawm hloov pauv. Yog tias lawv tawm ntawm theem, kev cuam tshuam kev puas tsuaj yuav txwv lub zog ntawm lawv cov nyiaj. hauv kev sib raug zoo. Ntawm daim duab. 31, ib qho piv txwv ntawm 2 MHz raug xaiv rau lub sijhawm hloov cov ntaub ntawv los ntawm tus lej ntawm cov qauv. Ib txoj hauv kev yooj yim dua yog kom tswj tau tus qauv coarser tus nqi ntawm 250 kHz thiab siv Finite Impulse Response (FIR) lim los ua kom muaj feem cuam tshuam qeeb. Peb yuav xav tias qhov nyuaj ntawm lub beamforming algorithm yog txiav txim siab los ntawm lub sij hawm hloov, txij li txhua channel yog convolved nrog FIR lim nrog 16 kais dej nyob rau hauv txhua txoj kev. Txhawm rau xam tus naj npawb ntawm MIPS xav tau rau kev ua haujlwm no, peb xav txog lub qhov rais ntawm 6ms ib qhov kev ntsuas kom ntes tau ntau yam ntawm 1 meter, 5 channels, 11 beamforming cov lus qhia (ntau +/- 50 ° hauv 10 ° cov kauj ruam). 75 kev ntsuas ib thib ob thawb lub microcontroller mus rau nws qhov siab tshaj plaws ntawm 100 MIPS. Txuas. 68, ua rau lub zog dissipation ntawm 11.26 mW rau tag nrho cov hluav taws xob dissipation ntawm 11.71 mW tom qab ntxiv rau onboard ADC pab.
Cov ntaub ntawv txhawb nqa cov txiaj ntsig ntawm txoj kev tshawb fawb no muaj los ntawm tus kws sau ntawv, FM, raws li kev thov tsim nyog.
Indiveri, G. & Sandamirskaya, Y. Qhov tseem ceeb ntawm qhov chaw thiab lub sijhawm rau kev ua cov teeb liab hauv cov neeg ua haujlwm neuromorphic: Kev sib tw ntawm kev tsim cov hluav taws xob qis, cov neeg ua haujlwm ywj pheej uas cuam tshuam nrog ib puag ncig. Indiveri, G. & Sandamirskaya, Y. Qhov tseem ceeb ntawm qhov chaw thiab lub sijhawm rau kev ua cov teeb liab hauv cov neeg ua haujlwm neuromorphic: Kev sib tw ntawm kev tsim cov hluav taws xob qis, cov neeg ua haujlwm ywj pheej uas cuam tshuam nrog ib puag ncig.Indiveri G. thiab Sandamirskaya Y. Qhov tseem ceeb ntawm qhov chaw thiab lub sijhawm rau kev ua cov teeb liab hauv cov neeg ua haujlwm neuromorphic: qhov kev sib tw ntawm kev tsim cov neeg ua haujlwm tsis muaj zog uas tsis muaj zog cuam tshuam nrog ib puag ncig. Indiveri, G. & Sandamirskaya, Y.的挑战. Indiveri, G. & Sandamirskaya, Y.Indiveri G. thiab Sandamirskaya Y. Qhov tseem ceeb ntawm qhov chaw thiab lub sijhawm rau kev ua cov teeb liab hauv cov neeg ua haujlwm neuromorphic: qhov kev sib tw ntawm kev tsim cov neeg ua haujlwm tsis muaj zog uas tsis muaj zog cuam tshuam nrog ib puag ncig.IEEE Kev Lag Luam Kev Lag Luam. Phau ntawv Journal 36, 16–28 (2019).
Thorpe, SJ Peak Lub Sijhawm Tuaj Txog: Ib Txoj Haujlwm Zoo Neural Network Coding Scheme. hauv Eckmiller, R., Hartmann, G. & Hauske, G. (eds). hauv Eckmiller, R., Hartmann, G. & Hauske, G. (eds).hauv Eckmiller, R., Hartmann, G. thiab Hauske, G. (eds.).Hauv Eckmiller, R., Hartmann, G., thiab Hauske, G. (eds.). Parallel processing in neural systems and computers 91–94 (North-Holland Elsevier, 1990).
Levy, WB & Calvert, VG Kev Sib Txuas Lus siv 35 npaug ntawm lub zog ntau dua li kev suav hauv tib neeg cortex, tab sis ob qho tib si tus nqi yuav tsum tau kwv yees tus lej synapse. Levy, WB & Calvert, VG Kev Sib Txuas Lus siv 35 npaug ntawm lub zog ntau dua li kev suav hauv tib neeg cortex, tab sis ob qho tib si tus nqi yuav tsum tau kwv yees tus lej synapse.Levy, WB thiab Calvert, WG Kev Sib Txuas Lus siv 35 npaug ntawm lub zog ntau dua li kev suav hauv tib neeg cortex, tab sis ob qho tib si tus nqi yuav tsum tau kwv yees tus lej ntawm synapses. Levy, WB & Calvert, VG Communication 消耗的能量是人类皮层计算的35倍，但这两种成本都需要预测突触数量。 Levy, WB & Calvert, VG CommunicationLevy, WB thiab Calvert, WG Kev Sib Txuas Lus siv 35 npaug ntawm lub zog ntau dua li kev suav hauv tib neeg cortex, tab sis ob qho tib si tus nqi yuav tsum kwv yees tus naj npawb ntawm cov synapses.txheej txheem. National Academy ntawm Science. kev kawm. US 118, https://doi.org/10.1073/pnas.2008173118 (2021).
Dalgaty, T., Vianello, E., De Salvo, B. & Casas, J. Insect-inspired neuromorphic xam. Dalgaty, T., Vianello, E., De Salvo, B. & Casas, J. Insect-inspired neuromorphic xam.Dalgati, T., Vianello, E., DeSalvo, B. and Casas, J. Kab-inspired neuromorphic xam.Dalgati T., Vianello E., DeSalvo B. and Casas J. Insect-inspired neuromorphic xam. Tam sim no. Kev xav. Kab science. 30, 59–66 (2018).
Roy, K., Jaiswal, A. & Panda, P. Towards spike-based machine intelligence with neuromorphic xam. Roy, K., Jaiswal, A. & Panda, P. Towards spike-based machine intelligence with neuromorphic xam. Roy, K., Jaiswal, A. & Panda, P. Towards Spike-Based Machine Intelligence with Neuromorphic Computing.Roy K, Jaiswal A, thiab Panda P. Pulse-raws li kev txawj ntse siv cov paj hlwb neuromorphic. Xwm txheej 575, 607–617 (2019).
Indiveri, G. & Liu, S.-C. Indiveri, G. & Liu, S.-C.Indiveri, G. and Liu, S.-K. Indiveri, G. & Liu, S.-C. Indiveri, G. & Liu, S.-C.Indiveri, G. and Liu, S.-K.Kev nco thiab cov ntaub ntawv ua haujlwm hauv cov tshuab neuromorphic. txheej txheem. IEEE 103, 1379–1397 (2015).
Akopyan F. et al. Truenorth: Tsim thiab cuab yeej rau 65 mW 1 lab neuron programmable synaptic nti. IEEE kev lag luam. Computer tsim ntawm integrated circuit systems. 34, 1537–1557 (2015).
Schemmel, J. et al. Nyob demo: scaled down version ntawm BrainScaleS neuromorphic system ntawm phaj nplai. 2012 IEEE International Symposium ntawm Circuits thiab Systems (ISCAS), (IEEE ed.) 702–702 (2012).
Moradi, S., Qiao, N., Stefanini, F. & Indiveri, G. Ib tug scalable multicore architecture nrog heterogeneous nco qauv rau dynamic neuromorphic asynchronous processors (DYNAPs). Moradi, S., Qiao, N., Stefanini, F. & Indiveri, G. Ib tug scalable multicore architecture nrog heterogeneous nco qauv rau dynamic neuromorphic asynchronous processors (DYNAPs).Moradi S., Qiao N., Stefanini F. thiab Indiviri G. A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (DYNAP). Moradi, S., Qiao, N., Stefanini, F. & Indiveri, G. Moradi, S., Qiao, N., Stefanini, F. & Indiveri, G. Ib hom kev nthuav dav multi-core architecture, nrog cov qauv cim xeeb rau kev ua haujlwm tsis zoo (DYNAP).Moradi S., Qiao N., Stefanini F. thiab Indiviri G. A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (DYNAP).IEEE Kev Lag Luam ntawm Biomedical Science. hluav taws xob system. 12, 106–122 (2018).
Davis, M. et al. Loihi: Ib tug neuromorphic multi-core processor nrog embedded kawm. IEEE Micro 38, 82–99 (2018).
Furber, SB, Galluppi, F., Tuam Tsev, S. & Plana, LA The SpiNNaker project. Furber, SB, Galluppi, F., Tuam Tsev, S. & Plana, LA The SpiNNaker project.Ferber SB, Galluppi F., Tuam Tsev S. thiab Plana LA SpinNaker project.Ferber SB, Galluppi F., Tuam Tsev S. thiab Plana LA SpinNaker project. txheej txheem. IEEE 102, 652–665 (2014).
Liu, S.-K. & Delbruck, T. Neuromorphic sensory systems. & Delbruck, T. Neuromorphic sensory systems.thiab Delbrück T. Neuromorphic sensory systems. & Delbruck, T. 神经形态感觉系统. & Delbruck, T.thiab Delbrück T. Neuromorphic sensory system.Tam sim no. Kev xav. Neurobiology. 20, 288–295 (2010).
Chop, T. et al. Neuromorphic sensory kev koom ua ke rau kev sib koom ua ke lub suab hauv zos thiab kev sib tsoo kom tsis txhob sib tsoo. Hauv 2019 ntawm IEEE Conference on Biomedical Circuits thiab Systems (BioCAS), (IEEE Ed.) 1–4 (2019).
Risi, N., Aimar, A., Donati, E., Solinas, S. & Indiveri, G. A spike-based neuromorphic architecture of stereo vision. Risi, N., Aimar, A., Donati, E., Solinas, S. & Indiveri, G. A spike-based neuromorphic architecture of stereo vision.Risi N, Aymar A, Donati E, Solinas S, thiab Indiveri G. A spike-based neuromorphic stereovision architecture. Risi, N., Aimar, A., Donati, E., Solinas, S. & Indiveri, G. 一种基于脉冲的立体视觉神经形态结构. Risi, N., Aimar, A., Donati, E., Solinas, S. & Indiveri, G.Risi N, Aimar A, Donati E, Solinas S, thiab Indiveri G. Spike-based neuromorphic architecture rau lub suab tsis pom kev.pem hauv ntej. Neurobotics 14, 93 (2020).
Osswald, M., Ieng, S.-H., Benosman, R. & Indiveri, G. A spiking neural network model of 3Dperception for event-based neuromorphic stereo vision systems. Osswald, M., Ieng, S.-H., Benosman, R. & Indiveri, G. A spiking neural network model of 3Dperception for event-based neuromorphic stereo vision systems.Oswald, M., Ieng, S.-H., Benosman, R., and Indiveri, G. A 3D Pulsed Neural Network Perception Model for Event-Based Neuromorphic Stereo Vision Systems. Osswald, M., Ieng, S.-H., Benosman, R. & Indiveri, G. 基于事件的神经形态立体视觉系统的3Dperception 脉冲神经网 Osswald, M., Ieng, S.-H., Benosman, R. & Indiveri, G. 3Dperception 脉冲神经网络模型.Oswald, M., Ieng, S.-H., Benosman, R., and Indiveri, G. Spiked 3Dperception Neural Network Model for an Event-Based Neuromorphic Stereo Vision System.kev kawm. Qhia 7, 1–11 (2017).
Dalgaty, T. et al. Kab-inspired yooj yim nrhiav kom pom muaj xws li tiv thaiv nco thiab bursty neural networks. Bionic biohybrid system. 10928, 115–128 (2018).
D'Angelo, G. et al. Kev tshwm sim-raws li eccentric motion detection siv lub sij hawm sib txawv coding. pem hauv ntej. Neurology. 14, 451 (2020).
Lub sij hawm xa tuaj: Nov-17-2022