Multi-Objective Reinforced Evolution in Mobile Neural Architecture Search

1. Introduction

• The paper identifies a prior RL–EA integration, RENAS, as failing to converge.It also notes that NAS evaluation commonly uses incompletely trained models, with less training data introducing biases.Source: §1 · pp. 1–2
• MoreMNAS combines NSGA-II and reinforcement learning for multi-objective neural architecture search.The authors describe this as overcoming drawbacks of each method.Source: §1 · p. 2
• Its cell-based search space supports genetic crossover and mutation, while hierarchical reinforced mutation balances exploration and exploitation.The method is also described as using minimal human expertise as an early guide and imposing practical constraints.Source: §1 · p. 2
• MoreMNAS models dominate SRCNN, FSRCNN, and VDSR in the reported experiments.The paper also reports multi-objective super-resolution models near the Pareto front in a single run.Source: §1 · p. 2

2.1. Single-Objective Oriented NAS

• Most early NAS methods in the single-objective category use validation accuracy as their sole objective.This defines the scope of the single-objective-oriented subsection.Source: §2.1 · p. 2

2.1.1. REINFORCEMENT LEARNING-BASED APPROACHES

• RL-based NAS includes Q-learning agents that store architecture and validation results in replay memory and use ϵ-greedy exploration.Policy-gradient methods use recurrent controllers and validation accuracy rewards, but are described as prone to convergence problems and mainly single-objective.Source: §2.1.1 · p. 2

2.1.2. EVOLUTION-BASED APPROACHES

• Evolutionary NAS evolves populations through crossover, mutation, or network morphism, eliminating less competitive models based on fitness.The passage places this work within the broader neuroevolution literature.Source: §2.1.2 · p. 2

2.1.3. REINFORCED EVOLUTION-BASED APPROACHES

• RENAS replaces random mutation with reinforced mutation, but its genetic algorithm does not preserve advantages across generations, so convergence is not assured.Previously generated better models may be removed during evolution.Source: §2.1.3 · p. 2

2.2. Multi-Objective Oriented NAS

• Multi-objective NAS seeks models along a Pareto front; linear combinations of objectives are described as suboptimal.Related approaches include NSGA-II-based NEMO and NSGA-Net, while PPP-Net selects Pareto-optimal models each generation and uses an RNN regressor to speed evaluation.Source: §2.2 · p. 3

3.1. Building Single Image Super-Resolution as a CMOP

• Mobile SISR must account for limited device resources and inference responsiveness, alongside a minimum acceptable PSNR.The paper says weighted summation is impractical in this setting.Source: §3.1 · p. 3
• The formulation maximizes benchmark PSNR while minimizing FLOPS and parameter count, subject to lower and upper bounds.The model architecture is the decision variable; lower bounds for FLOPS and parameters may be set to zero, and the PSNR upper bound to infinity.Source: §3.1 · p. 3
• Because the objectives compete or conflict, no model can achieve an optimum across all objectives.Source: §3.1 · p. 3

3.2. Architecture

• The architecture pipeline comprises a cell-based search space, an NSGA-II-variant model-generation controller, and a multi-reward evaluator.The controller samples models and receives performance feedback after evaluation.Source: §3.2 · p. 3
• The search space contains cells with repeated basic operator blocks, and cells within one model may differ in structure.For simplicity, the cells share the same search space of repeated blocks.Source: §3.2 · p. 3
• Unlike NSGA-Net, MoreMNAS performs crossover on cell-based architectures rather than binary encodings and uses two-level hierarchical mutation.The authors describe the mutation process as balancing exploration and exploitation.Source: §3.2 · p. 3

3.3. Search Space

• The search varies a central backbone between fixed feature-extraction and reconstruction stages, following super-resolution’s three-part processing pipeline.The design allows connection topologies such as skip and dense connections.Source: §3.3 · p. 4
• Each model comprises cells that may use different structures, with each cell built from repeated basic operators.Operators include convolution variants, kernel sizes, filter counts, skip connections, and repetition counts.Source: §3.3 · p. 4
• Using the same operator set across seven cells in their experiments, each with 192 choices, the search domain has complexity 192^7, and the approach maps cell codes one-to-one to neural architectures.Source: §3.3 · p. 4
• Unlike classification-oriented search spaces, this super-resolution design excludes pooling, batch normalization, and down-sampling.The paper reports that pooling and batch normalization can deteriorate super-resolution results, while down-sampling irreversibly blurs features.Source: §3.3 · p. 4

3.4.1. POPULATION INITIALIZATION

• Population initialization randomly selects an operator for each cell, repeats the cell to form a model, and generates 2N individuals for the first generation.The paper calls this procedure uniform initialization.Source: §3.4.1 · pp. 4–5

3.4.2. NON-DOMINATED SORTING

• The method uses NSGA-II’s non-dominated sorting with an improved crowding-distance definition that distinguishes individuals within the same cuboid.The improved distance is defined for an individual by objective and sorting order.Source: §3.4.2 · p. 5

3.4.3. CELL-BASED CROSSOVER

• Cell-based crossover exchanges a cell between two parent models to form a child, with single-point crossover given as an example.Two-point and k-point crossover are also noted as possible strategies.Source: §3.4.3 · p. 5

3.4.4. HIERARCHICAL MUTATION

• Crossover mainly supports exploitation, while mutation introduces exploration; hierarchical mutation addresses the sensitivity of reinforced-only approaches to reward design and hyperparameter tuning.The authors also identify limited use of training knowledge in pure evolution-based NAS as a motivation.Source: §3.4.4 · p. 5
• Hierarchical mutation combines reinforced-dominant, natural, and prior-regularized mutation, with reinforced control used for objectives that are difficult to predict.In super-resolution, the paper identifies −PSNR as difficult to predict, while multi-adds and parameter count are treated as easier to predict.Source: §3.4.4 · p. 5
• The reinforced controller uses an LSTM to sample a sequence of cell actions from the search space, building a complete model and using its objective scores as learned guidance.The reward is delayed until all cells have been generated; model reward is based on the mean squared error between generated and ground-truth high-resolution images.Source: §3.4.4 · pp. 5–6
• Natural mutation continues throughout evolution, while elitist preservation is described as preventing model degradation during model generation.The method also includes a small-probability prior-regularized mutation that repeats one selected operator across the model’s positions.Source: §3.4.4 · p. 6

3.5.1. SCALABILITY

• Total cost grows in direct proportion to generated models, so additional computing resources can boost the pipeline linearly.The method can evaluate FLOPS and parameter constraints before training; models that violate either can skip training.Source: §3.5.1 · p. 6, §3.5.2 · p. 6

3.5.2. HANDLING CONSTRAINTS

• FLOPS and parameter counts are available immediately after model-meta is given, but PSNR requires full training unless regression is used.Training can be skipped when FLOPS or parameter constraints are violated.Source: §3.5.2 · p. 6
• The PSNR constraint keeps feasible models and excludes low-parameter models that could otherwise occupy valuable Pareto-front positions.Source: §3.5.2 · p. 6

4.1. Setup and Implementation Details

• The pipeline generated 11,200 models across 200 generations and 56 individuals per generation; a complete run took about seven days on one Tesla V100 machine with eight GPUs.Source: §4.1 · p. 6

4.1.1. BASE MODEL

• The base model's feature-extraction and restoration stages each use 32 3 × 3 convolution filters with unit stride.Source: §4.1.1 · p. 7

4.1.2. EVALUATOR

• Evaluators train on DIV2K for 200 epochs using 800 high-resolution images randomly cropped to 80 × 80 and bicubic-downsampled to low resolution.Training uses random flips and rotations, Adam, and L1 loss.Source: §4.1.2 · p. 7

4.1.3. FULL TRAINING

• Selected Pareto-front models receive full training for 4,800 epochs with 128 × 128 crops.The full-training setup also changes the initial learning rate by model size and decays it every 80,000 back-propagations.Source: §4.1.3 · p. 7

4.2. Comparison with Human Designed State of the Art Models

• C matches FSRCNN with fewer multi-adds and dominates SRCNN on score, parameters, and multi-adds; B/D also compete with DRRN on Urban100 with fewer multi-adds.MoreMNAS-A uses inverted bottleneck blocks interleaved with grouped convolutions, while B uses them separately.Source: §4.2 · p. 7
• At ×2 scale, MOREMNAS-A and B achieve higher PSNR and SSIM than VDSR across four evaluation sets with fewer multi-adds; MOREMNAS-D dominates VDSR on three objectives using one-quarter its multi-adds.Source: §4.2 · p. 7
• Latest models push the Pareto front further during the evolutionary process.Source: Figure 8 · p. 8, §4.2 · p. 8

• MoreMNAS combines NSGA-II and reinforcement learning to seek better trade-offs among competing objectives.The authors describe benefits including no model recession during evolution, reinforced-mutation exploitation, and objective balancing through NSGA-II and roulette-wheel selection.Source: §5 · p. 8
• In super-resolution, lightweight models sometimes dominate human-designed models such as SRCNN and VDSR across PSNR, multi-adds, and parameter count.The authors state that the algorithm may also apply beyond mobile settings.Source: §5 · p. 8
• Future work targets regression-based pipeline acceleration, hyperparameter tuning, and replacing reinforcement with policy-gradient methods such as PPO and POP3D.Source: §5 · p. 8