PaperlayerView exact revision on arXiv

Source-linked AI summary

The Reciprocal tau-Cell: A Log-Domain Circuit for On-Chip Learning

Craig Jin, Thomas Chaffey

arXiv:2610.20018v1eess.SP

TL;DR

Hardware gradient descent often requires a separate backward pass, limiting its implementation in analog circuits. This paper uses a reciprocal τ-cell and hardware linearisation so the same circuit computes gradients in a second run. In simulations, in-situ learning raises test accuracy on the four worst mismatched circuits from 53–63% to 88–96%.

  • Problem

    Hardware gradient descent requires computing gradients, which existing approaches implement with a separate digital backward pass, digital periphery, or dedicated update circuits.

    Source: §I · p. 1

  • Method

    The reciprocal τ-cell uses symmetric connections and latches the active-cell pattern so the same circuit can compute the gradient in a linearized second run.

    Source: §I · p. 1, B. Gradient computation · p. 2

  • Results

    In the four worst mismatch draws, in-situ learning raised test accuracy from 53–63% to 88–96%.

    Source: G. Why learn on chip? · p. 4

  • Takeaways & Limitations

    The reciprocal τ-cell allows a monDEQ to be implemented and trained in a CMOS circuit using the circuit itself to compute the gradient.

    Source: §V · p. 4

    The study learns static mappings between vectors; learning dynamic sequence mappings in situ remains an open problem.

    Source: §V · p. 4

Abstract

from arXiv · show

Training hardware by gradient descent requires a gradient to be computed using a separate backwards pass. If a circuit is reciprocal, meaning every connection is symmetric, the backward pass is the same network run a second time with the error as its input. The reciprocal tau-cell, a weak-inversion log-domain circuit, makes this practical in standard CMOS. Each cell is a current-mode leaky integrator with a built-in ReLU; each connection is two transistors whose signed strength is a ratio of bias currents. The equilibrium of the network is the solution to a convex program, and latching the cells that are off will hardware linearise the circuit, making its second run exactly linear. Transistor-level SKY130 simulations show that the second run gives the gradient within 1.9%; that hardware linearisation permits large, noise-robust error signals; that a 13-cell network learns the Iris data set; and that learning in situ restores mismatched circuits on which software-trained weights fail, from 53-63% to 88-96% test accuracy.

I. INTRODUCTION

Hardware gradient descent commonly requires a separate backward pass, while existing analog approaches use digital periphery or dedicated update circuits. The paper presents a reciprocal τ-cell with hardware linearisation, enabling the same circuit to compute gradients in a second run.

I. INTRODUCTION

  • Gradient computation in hardware is difficult: prior approaches use a separate digital backward pass [1], digital periphery,, or dedicated log-domain update circuits [4].Source: §I · p. 1
  • With symmetric connections, a network’s backward pass is a second run of the same circuit with the error as input.This reciprocity underlies several existing learning approaches –[18].Source: §I · p. 1
  • Prior passive resistor or memristor networks lack programmable dynamics, duplicate layers for negative weights, and require small injected currents that can be buried in noise, [17],,.Source: §I · p. 1
  • The reciprocal τ-cell implements hardware linearisation in standard CMOS, with a two-transistor connection whose signed strength is a bias-current ratio shared in both directions.The paper also tests gradient accuracy, noise robustness, mismatch, benchmark learning, and in-situ learning in transistor-level simulations.Source: §I · p. 1

A. Equilibrium behavior

Each cell rectifies its summed input and relaxes toward it, producing an equilibrium. With symmetric connections and sufficiently limited feedback, that equilibrium is unique and solves a convex optimisation problem.

A. Equilibrium behavior

  • Each cell rectifies its input plus neighboring currents and relaxes toward that sum with time constant τ.The reciprocal τ-cell uses a translinear loop to implement this leaky-integrator behavior with a built-in rectifier.Source: A. Equilibrium behavior · p. 1, Figure 1 · p. 2
  • When W = W⊤ and I − W ≻ 0, the equilibrium is unique, reached from any starting point, and solves a convex optimisation problem.Source: A. Equilibrium behavior · p. 1
  • Without learning, the circuit solves non-negative quadratic programs; with learning, it realizes a monDEQ that behaves like an infinitely deep network with tied weights, evaluated in one settling time [18],.Source: A. Equilibrium behavior · pp. 1–2

B. Gradient computation

For symmetric connections, the error response of the same network run with error input supplies the backward quantities needed for weight gradients. Keeping the active and inactive cells fixed makes this second run the circuit’s linear response.

B. Gradient computation

  • In a two-cell example, injecting the output error into the same reciprocal circuit yields d1 and d2 such that d1x2 + d2x1 is exactly the weight gradient.Source: B. Gradient computation · p. 2
  • For many cells, fixing the on/off matrix D makes the second run’s error response d = (I − DW)−1De, while each weight gradient uses currents at its two cells.Removing off cells leaves a symmetric active-set system, so the response corresponds to the same circuit with the same cells on and off.Source: B. Gradient computation · p. 2

III. THE CIRCUIT

The circuit uses weak-inversion transistor behavior and a translinear loop to implement a current-mode cell. Its loop makes the cell’s dynamics linear in current while it is on.

III. THE CIRCUIT

  • In weak inversion, transistor gate–source voltages encode logarithms of currents, and products of currents around a loop balance by the translinear principle.Source: A. Operation of a cell · p. 2
  • At the cell’s summing node, input and neighbor currents feed a diode-connected transistor that rectifies the total and takes its logarithm.Source: A. Operation of a cell · p. 2
  • The M1–M4 loop drives capacitor current so that the cell is exactly linear in current while on, with a time constant set by a bias current.A cascode suppresses the Early effect of M2.Source: A. Operation of a cell · p. 2

B. A weight is a voltage shift

Each connection uses two transistors to encode a signed weight as a bias-current ratio, with the same weight code enforcing reciprocity in both directions.

B. A weight is a voltage shift

  • A two-transistor connection uses the sender’s output voltage shift to multiply current and encode the weight.The diode is biased by a static current on the sender’s capacitor node.Source: B. A weight is a voltage shift · p. 2
  • The signed weight strength is set by a ratio of bias currents.A current DAC or bias generator sets the ratio, and an equal sink returns the bias current.Source: B. A weight is a voltage shift · p. 2
  • One code drives both directions, making the connection weights symmetric by construction.Negative weights draw current out of the cell, while positive weights are pushed in by a PMOS mirror before the ReLU.Source: B. A weight is a voltage shift · p. 2

C. Hardware linearisation

Hardware linearisation preserves the free-run active set for a second, linear response to injected error. Two measurements recover that response exactly when the active cells remain on.

C. Hardware linearisation

  • After the free run, comparators latch each cell’s active state, and switches silence cells that were off.The procedure retains the free-run pattern of on and off cells for the backward run.Source: C. Hardware linearisation · p. 2
  • The learning step holds the network state, latches the active cells, then takes two backward measurements.Figure 3(b) depicts the sequence after the free run.Source: C. Hardware linearisation · p. 2, Figure 3 · p. 3
  • A pedestal replaces the inputs on active cells, and outputs are measured with and without added error; their difference is exact while those cells stay on.Each weight is updated by −η(d_i x_j + d_j x_i).Source: C. Hardware linearisation · p. 3

IV. SIMULATION RESULTS

The evaluation uses transistor-level SKY130 simulations and compares the circuit’s gradient against a finite-difference reference for the simulated circuit.

IV. SIMULATION RESULTS

  • The study uses transistor-level ngspice simulations with SKY130 models at the typical corner and 27 °C.The setup specifies W/L = 4 µm/8 µm, C = 1 pF, I0 = 5 nA, and a 1.8 V supply; mismatch uses random transistor threshold offsets, with SKY130 predicting about 1.3 mV.Source: A. Simulation setup · p. 3
  • Every black transistor is simulated, while orange blocks, current sources, and update arithmetic are idealized.Each phase of the learning procedure is solved as a settled DC operating point.Source: A. Simulation setup · p. 3
  • The reference gradient is obtained by perturbing each shared weight and re-solving the circuit, and error is measured after best uniform rescaling.A uniform scale is treated as harmless because the learning rate absorbs it.Source: A. Simulation setup · p. 3

B. Do the cell and the connection work as designed?

The simulations validate the cell and connection, show that the reciprocal second run measures the gradient, and find that latching supports accurate gradients at larger error signals despite readout noise.

B. Do the cell and the connection work as designed?

  • The cell copies its input within ±2% from 0.5 to 60 nA, and programmed weights are reproduced within 0.015 for negative and 0.03 for positive values.The cell time constants are 51, 11, and 2.9 µs at I0 = 1, 5, and 20 nA; six mutually inhibiting cells settle within 1% in 0.12 ms and draw 0.26 µW.Source: B. Do the cell and the connection work… · p. 3

C. Does the second run give the gradient?

D. Why hardware linearise?

  • At 14 nA, latched gradient error falls to 1.3%, while an unlatched nudge reaches 36% at 10 nA and 52% at 14 nA.The test adds 20 pA rms readout noise; the unlatched error switches the off cell on at larger amplitudes.Source: D. Why hardware linearise? · p. 3, Figure 4 · p. 4

E. How much does mismatch matter?

Mismatch increases gradient error, though the median remains 3.1% at 1 mV; on Iris, the circuit tracks exact-gradient software and reaches 90.0% test accuracy.

E. How much does mismatch matter?

  • At 1 mV, median gradient error rises from 1.3% to 3.1%; at 3 mV, cell, connection, and combined offsets yield 7.9%, 6.7%, and 16%.Cell offsets produce a diagonally symmetrisable GW whose per-cell factor on d could be calibrated out; connection offsets break reciprocity, which rescaling cannot restore in loops of three or more cells.Source: E. How much does mismatch matter? · p. 4

F. Does the circuit learn a real task?

  • The Iris experiment uses 120 training and 30 test flowers, with four input, six hidden, and three output cells.The 13-cell network classifies each flower by its largest output.Source: F. Does the circuit learn a real task? · p. 4
  • 90.0% test accuracy after 30 epochs, matching the software network’s 90.0% test accuracy and three test errors, despite 96.7% versus 98.3% training accuracy.The circuit follows the software training curve epoch by epoch using the exact gradient.Source: F. Does the circuit learn a real task? · p. 4

G. Why learn on chip?

In situ learning raises test accuracy in the worst mismatched circuits; the reciprocal tau-cell also supports CMOS implementation and training of a monDEQ, while area and dynamic-sequence learning remain open constraints.

G. Why learn on chip?

  • 88–96% test accuracy after in-situ retraining, up from 53–63% in the four worst mismatch draws.The reported accuracy is the mean of the last three epochs; copied software weights fall to a median 86.7% at 3 mV mismatch and 76.7% at 5 mV.Source: G. Why learn on chip? · p. 4

V. CONCLUSION

  • The reciprocal tau-cell implements and trains a monDEQ in CMOS using hardware linearisation, targeting low-power edge applications such as learning at the sensor.A current DAC per pair dominates area, and dynamic sequence mappings learned in situ remain an open problem.Source: §V · p. 4
Loading 2610.20018v1…