Performance Analysis of Edge and In-Sensor AI Processors: A Comparative Review

I. INTRODUCTION

• Edge AI workloads span wearables, IoT nodes, sensor networks, micro-robots, and AR/VR systems, all operating under stringent energy and thermal constraints.Sensing, signal conditioning, memory access, and wireless transmission share the constrained power budget, making system-level efficiency important alongside algorithmic and architectural optimization.Source: §I · p. 1
• Specialized edge processors trade clock speed, local memory, and dedicated MAC arrays against power consumption and programmability.Architectures range from neuromorphic and compute-in-memory approaches to programmable ARM and RISC-V platforms, in-sensor processors, and higher-power edge NPUs.Source: §I · pp. 1–2
• The IMX500 achieves the highest compute density and lowest EDP, GAP9 delivers competitive MAC/J, and STM32N6 provides the shortest latency at substantially higher power.The comparison runs the same 336 million MAC PicoSAM2 segmentation model on all three platforms.Source: §I · p. 2
• The review combines a survey of sub-200-mW architectures with a model-matched benchmark to clarify the edge-AI landscape and its design trade-offs.Figure 1 maps peak performance against power consumption for current accelerators and highlights the operating region of ultra-low-power devices.Source: §I · p. 2, Figure 1 · p. 2

A. Edge Computing and Edge AI Tasks

• In-sensor computing embeds intelligence in the sensing pipeline to reduce data movement at its source.Edge systems acquire, digitize, and buffer sensor data through mixed-signal front ends; traditional architectures rely on high-fidelity ADCs.Source: A. Edge Computing and Edge AI Tasks · p. 2
• In-sensor scenarios may operate below 10 mW, making hardware-aware optimization necessary because full-precision inference is typically prohibitive.Quantization, pruning, and knowledge distillation are cited as common optimization methods.Source: A. Edge Computing and Edge AI Tasks · p. 2

B. Energy Consumption Components

• Sensing-chain energy scales with sampling rate and bit depth for vision or high-frequency signals, so low-power trigger paths can keep high-fidelity ADCs dormant until relevant events.The sensing chain is negligible for low-bandwidth modalities.Source: B. Energy Consumption Components · p. 2
• Wireless communication typically consumes nanojoules to microjoules per bit, versus picojoules for a modern MAC, so local inference can reduce overall energy by minimizing radio activity.Computational energy also depends on model arithmetic intensity and CPU, DSP, or NPU frequency and voltage settings.Source: B. Energy Consumption Components · p. 2

C. Metrics to Evaluate Low-Power AI Platforms

• Latency per inference measures the end-to-end time for one model forward pass, reflecting clock speed, memory hierarchy, and accelerator utilization.Source: C. Metrics to Evaluate Low-Power AI… · p. 3
• Inference efficiency (MAC/cycle) measures sustained throughput utilization, including dataflow efficiency and memory stalls.Higher values indicate a better match between the accelerator or DSP and the model structure.Source: C. Metrics to Evaluate Low-Power AI… · p. 3
• Energy efficiency (MAC/J) quantifies useful operations per joule, consolidating dynamic power, leakage, and memory movement.Optimized datapaths, short memory routes, or reduced-precision execution can yield higher MAC/J.Source: C. Metrics to Evaluate Low-Power AI… · p. 3
• Energy–Delay Product combines energy per inference with latency, penalizing platforms that are fast but power-hungry or energy-efficient but slow.A lower EDP indicates a more balanced platform across latency and energy cost.Source: C. Metrics to Evaluate Low-Power AI… · p. 3
• Energy per inference measures the total energy to process one input sample, a priority in ultra-low-power applications because it affects how many events a battery can sustain.Source: C. Metrics to Evaluate Low-Power AI… · p. 3
• Peak power is the maximum instantaneous draw, while average power covers the duty cycle and informs battery sizing for a target lifetime.Peak power matters for sources with high internal impedance; average power includes active and sleep periods.Source: C. Metrics to Evaluate Low-Power AI… · p. 3
• Memory footprint distinguishes Flash for model weights from SRAM for activation maps and input buffers during execution.Source: C. Metrics to Evaluate Low-Power AI… · p. 3

A. MCU Processors with ML Extensions

• Modern Edge AI MCUs combine sub-threshold voltage operation, clock gating, and DSP or low-power ML extensions to balance throughput with tight energy budgets.Limited on-chip SRAM and Flash resources drive efficient memory hierarchies and quantization.Source: A. MCU Processors with ML Extensions · p. 3
• Apollo510 uses ARM Cortex-M55 and SPOT technology for extremely low active power while maintaining real-time inference.Source: A. MCU Processors with ML Extensions · p. 3
• STM32U5/H5 use Cortex-M33 cores with advanced power modes or higher-frequency computing, while Nordic’s nRF54 separates radio and application domains across multicore Cortex-M33.Renesas RA and Infineon DA series extend low-power MCU designs with secure features, expanded memory, and peripheral integration.Source: A. MCU Processors with ML Extensions · p. 3

B. MCU with Dedicated ML Accelerators

• GAP9 provides a scalable RISC-V cluster, configurable precision, and 1.6 MB of L2 SRAM for software-defined ML acceleration.The passage also describes MAX78000, Sony CXD5602, NXP i.MX RT1170, Kendryte K210/K230, and STM32N6/STM32V8 accelerator designs.Source: B. MCU with Dedicated ML Accelerators · p. 3

C. Reconfigurable Logic and FPGA-based Processors

• Ultra-low-power FPGAs offer custom datapaths and fine-grained precision control, trading flexibility for modestly higher static power.Examples include iCE40 UltraPlus for submW BNN inference, EOS S3 for dynamic workload partitioning, and PolarFire for burst-mode inference.Source: C. Reconfigurable Logic and FPGA-based… · p. 3

D. Neuromorphic Processors

• Neuromorphic processors use event-driven SNNs with co-located memory and compute for always-on sensing.Examples range from Loihi 2 and TrueNorth to sub-10 mW spiking vision systems from SynSense.Source: D. Neuromorphic Processors · pp. 3–4

E. Processing-In-Memory (PIM) and Compute-In-Memory (CIM)

• PIM/CIM performs MAC operations inside memory arrays, enabling parallel, energy-efficient matrix-vector multiplication for edge inference.Analog CIM uses RRAM/SRAM current-summation MACs; Mythic M1076 performs flash-based analog MVM without DRAM accesses.Source: E. Processing-In-Memory (PIM) and… · p. 4

F. In-Sensor Computing

• In-sensor inference runs directly in the readout pipeline, avoiding high-bandwidth video transmission and reducing system power.Sony IMX500/501 embeds DSP and NPU logic to output metadata, while OmniVision sensors perform presence and face detection below 5 mW.Source: F. In-Sensor Computing · p. 4

IV. PERFORMANCE EVALUATION

• A representative always-on segmentation workload was benchmarked across three heterogeneous processors—GAP9, STM32N6, and Sony IMX500—with evaluation focused on latency, energy per inference, energy efficiency, and EDP.Source: §IV · pp. 4–5
• Segmentation preserves large activation tensors and adds skip connections, multiscale aggregation, and upsampling, increasing memory and bandwidth pressure.These characteristics expose compute-memory bottlenecks and dataflow inefficiencies in ultra-low-power systems.Source: §IV · p. 5
• PicoSAM2 is a compact 336 million MAC U-Net-derived model with a 3×96×96 RGB input and a lightweight encoder-decoder pipeline.It is designed for low-latency inference on constrained hardware.Source: §IV · p. 5
• The four hardware metrics are latency, inference efficiency (MAC/cycle), energy efficiency (MAC/J), and EDP.EDP integrates energy cost and responsiveness into one figure of merit.Source: §IV · p. 5
• 13.7 ms latency on STM32N6 is lowest, compared with 14.3 ms on IMX500 and 42.1 ms on GAP9.The passage attributes STM32N6's lead to its higher clock frequency and partly links GAP9's highest latency to its lower cluster clock.Source: §IV · p. 5
• 86.2 MAC/cycle on IMX500 is about 3 times STM32N6's 29.5 and 4 times GAP9's 20.8.The evaluation attributes IMX500's advantage to spatial data reuse and reduced memory traffic.Source: §IV · p. 5
• 1359.6 MMAC/J on IMX500 exceeds GAP9's 182.15 MMAC/J and STM32N6's 21.5 MAC/J.GAP9 benefits from low dynamic power; the passage attributes IMX500's efficiency to its CMOS-stacked design.Source: §IV · p. 5
• 3.4 mJ·s EDP for IMX500 is lowest; GAP9 records 74.88 mJ·s and STM32N6 206.8 mJ·s.The review identifies IMX500 as strongest in utilization, energy efficiency, and energy-delay balance; GAP9 remains competitive for low-power MCU-class deployments, while STM32N6 favors raw speed at higher power cost.Source: §IV · p. 5, Figure 2 · p. 5

• There is no one-size-fits-all Edge AI solution, as shown by benchmarking the PicoSAM2 segmentation model.Source: §V · p. 5
• The review focuses on ultra-low-power Edge AI processors operating within the sub-200 mW envelope for battery-powered and energy-harvesting IoT devices.Source: §V · p. 5
• System designers should consider responsiveness, energy efficiency measured in MAC/J, and the combined balance captured by EDP when deploying edge devices.Source: §V · p. 5