Source-linked AI summary
SmartPhotoCrafter: Unified Reasoning, Generation and Optimization for Automatic Photographic Image Editing
Ying Zeng, Miaosen Luo, Guangyuan Li, Yang Yang, Ruiyang Fan, Linxiao Shi, Qirui Yang, Jian Zhang, Chengcheng Liu, Siming Zheng, Jinwei Chen, Bo Li, Peng-Tao Jiang
TL;DR
Automatic photographic editing is challenging because users may not know what to edit, while existing systems lack sufficient image-quality perception and subtle photometric sensitivity. SmartPhotoCrafter couples an Image Critic’s reasoning with a Photographic Artist through multi-stage training and coordinated reinforcement learning. It achieves strong performance across diverse enhancement scenarios while maintaining naturalness and visual coherence.
Problem
Existing instruction-conditioned editors depend on users’ editing knowledge, while current IQA and agent methods have limited sensitivity or integration for unified photographic restoration and retouching.
Method
SmartPhotoCrafter uses an Image Critic to infer aesthetic and photometric edits, conditions a Photographic Artist on its reasoning, and trains both through staged adaptation and coordinated reinforcement learning.
Results
SmartPhotoCrafter achieves strong performance across diverse enhancement scenarios while maintaining naturalness and visual coherence.
Takeaways & Limitations
Coupling semantic understanding with photorealistic generation supports automatic photographic enhancement with fidelity, edit consistency, and perceptual quality improvement.
Takeaways & Limitations
Current IQA methods remain limited in detecting subtle photometric deviations and are not typically integrated as photographic-aware editing objectives.
Abstract
from arXiv · showhide
Traditional photographic image editing typically requires users to possess sufficient aesthetic understanding to provide appropriate instructions for adjusting image quality and camera parameters. However, this paradigm relies on explicit human instruction of aesthetic intent, which is often ambiguous, incomplete, or inaccessible to non-expert users. In this work, we propose SmartPhotoCrafter, an automatic photographic image editing method which formulates image editing as a tightly coupled reasoning-to-generation process. The proposed model first performs image quality comprehension and identifies deficiencies by the Image Critic module, and then the Photographic Artist module realizes targeted edits to enhance image appeal, eliminating the need for explicit human instructions. A multi-stage training pipeline is adopted: (i) Foundation pretraining to establish basic aesthetic understanding and editing capabilities, (ii) Adaptation with reasoning-guided multi-edit supervision to incorporate rich semantic guidance, and (iii) Coordinated reasoning-to generation reinforcement learning to jointly optimize reasoning and generation. During training, SmartPhotoCrafter emphasizes photo-realistic image generation, while supporting both image restoration and retouching tasks with consistent adherence to color- and tone-related semantics. We also construct a stage-specific dataset, which progressively builds reasoning and controllable generation, effective cross-module collaboration, and ultimately high-quality photographic enhancement. Experiments demonstrate that SmartPhotoCrafter outperforms existing generative models on the task of automatic photographic enhancement, achieving photo-realistic results while exhibiting higher tonal sensitivity to retouching instructions. Project page: https://github.com/vivoCameraResearch/SmartPhotoCrafter.
1 Introduction
SmartPhotoCrafter addresses the difficulty of automatic photographic enhancement by coupling image-quality reasoning with targeted generation. Its multi-stage training and stage-specific data develop reasoning, controllable editing, cross-module collaboration, and photorealistic enhancement.
- Automatic photographic editing must improve aesthetic quality while preserving the original semantic content.
- Existing instruction-conditioned editors require users to know what and how to edit, while lacking image-quality perception for fully automatic enhancement.
- SmartPhotoCrafter couples an Image Critic’s aesthetic and photometric reasoning with a Photographic Artist that generates enhanced outputs.
- Its three-stage pipeline comprises foundation pretraining, reasoning-conditioned adaptation, and coordinated reinforcement learning for joint reasoning and generation optimization.
- The coordinated reinforcement-learning framework optimizes both modules and improves sensitivity to fine-grained tonal adjustments and high-quality color distributions.
- Stage-specific datasets provide supervision for IQA reasoning, edit suggestions, and high-quality photographic targets, progressively supporting controllable generation and collaboration.
2 Related Works
Related work spans automatic editing, multimodal reasoning, image-quality assessment, and retouching, but existing methods leave a gap between semantic editing and fine-grained photographic control. SmartPhotoCrafter targets this gap with unified reasoning-to-generation and retouching-aware optimization.
- Automatic editing seeks semantic alignment with desired transformations while minimizing manual intervention, with diffusion and vision-language models improving edit realism and diversity.
- Unified generative editors support multimodal reasoning but remain limited in the subtle photometric adjustments required for high-quality photographic enhancement.
- Current IQA methods often miss subtle photometric deviations and are not typically integrated as photographic-aware objectives in editing pipelines.
- Existing agent frameworks often struggle to jointly handle photographic restoration and retouching, leaving structural recovery and photometric refinement insufficiently unified.
- Prior methods typically trade off semantic transformation with precise photometric control or fine-grained adjustment with global aesthetic enhancement and restoration.
3.1 Problem Formalization
SmartPhotoCrafter formulates enhancement as reasoning-guided generation: an Image Critic analyzes quality and proposes edits, while a Photographic Artist produces a semantically consistent, photometrically precise result.
- The task maps a distorted or aesthetically suboptimal input X to an enhanced output Xe that preserves content and structure relative to reference Xgt.
- The Image Critic produces aesthetic and photometric reasoning R, actionable edit suggestions E, and a predicted quality score S.
- The Photographic Artist generates Xe from input image X and the Image Critic’s reasoning latent Hc.
- Together, the modules unify aesthetic understanding with high-fidelity editing, combining interpretable guidance with fine-grained, controllable image transformation.
3.2 Multi-Stage Training Pipeline
The training pipeline progressively establishes basic capabilities, aligns generation with critic reasoning, and jointly optimizes discrete reasoning with continuous photographic generation.
- The pipeline is designed to learn interpretable reasoning, fine-grained photometric sensitivity, and high-quality enhancement under photographic-aware supervision.
- Foundation Pretraining: Foundation pretraining uses IQA and editing data to establish basic reasoning, restoration, and retouching capabilities for the Image Critic and Photographic Artist.
- Reasoning-Conditioned Adaptation: Reasoning-conditioned adaptation conditions the Photographic Artist on the Image Critic’s latent reasoning representation to produce semantically grounded edits.
- Reasoning-Conditioned Adaptation: The reasoning-aware latent Hc concatenates context and reasoning hidden states and serves as the conditioning signal for generating enhanced image Xe.
- Coordinated Reasoning-to-Generation Reinforcement Learning: Coordinated reinforcement learning uses GRPO for the Image Critic’s discrete reasoning and DiffusionNFT for the Photographic Artist’s continuous generation.
- Coordinated Reasoning-to-Generation Reinforcement Learning: DiffusionNFT aligns the Photographic Artist with aesthetic preferences by reformulating policy alignment as contrastive interpolation in the velocity field.
3.3 Data Construction
The stage-specific data construction progressively supports image-quality reasoning, controllable photographic generation, and collaboration between the Image Critic and Photographic Artist. It covers restoration, retouching, depth-of-field control, and unified reasoning-to-generation supervision.
- Data Construction: Stage-specific datasets progressively shape reasoning capability, controllable generation, and cross-module collaboration.The data-generation pipeline annotates reasoning and suggestions, creates enhancement pairs, and builds unified understanding-generation data.
- Image Critic: Image Critic data combines IQA and editing datasets with chain-of-thought reasoning, quality scores, and structured edit suggestions.The construction covers multiple distortion types and uses retained MOS scores for IQA supervision.
- Photographic Artist: Photographic Artist training covers distortion removal and controllable retouching for exposure, contrast, saturation, color temperature, and bokeh.Restoration uses degradation-specific input–ground-truth pairs, while retouching uses parameterized color and tonal adjustments at multiple intensities.
- Cross-module Collaboration: Coordinated reinforcement learning jointly enhances the Image Critic and Photographic Artist for photographic-aware reasoning and image enhancement.The unified optimization paradigm is summarized in Figure 3.
- Photographic Artist: Multi-edit samples combine restoration with randomly stacked retouching adjustments to support compositional editing.The construction also includes multi-level blur data for depth-of-field control.
- Cross-module Adaptation: Reasoning-conditioned adaptation aligns the Image Critic’s latent representations with the Photographic Artist’s conditional generation process.An online strategy applies the critic’s structured editing instructions to synthesize dynamic reference targets.
3.4 Reward Design for Photographic-aware Editing
The editing reward combines semantic compliance, photometric control, and perceptual consistency so generated images follow the critic’s instructions while preserving scene structure. Its attribute-level supervision targets subtle tonal changes without requiring pixel-wise reconstruction.
- Overall Reward: The multi-level reward simultaneously evaluates semantic compliance, photometric fidelity, and perceptual similarity.The design addresses the limited sensitivity of holistic perceptual rewards to subtle exposure and contrast variations.
- Reward Coupling: The semantic-compliance reward gates the combined reward, while photometric and perceptual terms regulate adjustment magnitude and structural fidelity.The formulation uses λ1=1.0 and λ2=0.5 to balance the latter components.
- Semantic Compliance: Semantic Compliance Reward verifies whether color- and tone-related suggestions from the Image Critic are reflected in the edited image.It isolates photometric attributes and aggregates binary compliance indicators into a targeted supervision signal.
- Photometric Control: Photometric Control Reward measures attribute-wise global photometric discrepancies rather than enforcing rigid pixel-level reconstruction.It encourages relative improvement toward the ground truth across attributes such as exposure, contrast, and color temperature.
- Perceptual Consistency: Perceptual Consistency Reward uses LPIPS to preserve structural layouts, object identities, and fine-grained texture patterns.A smaller perceptual distance yields a higher reward and complements photometric supervision.
3.5 Reward Design for Photographic-aware Understanding
The understanding reward design trains the Image Critic to produce structured, improvement-sensitive aesthetic reasoning and actionable edit suggestions. It combines format compliance, score ranking, and complementary supervision from restoration and retouching data.
- Reward Design: The Image Critic’s structured reward captures global aesthetic judgment and fine-grained improvement guidance.Its responsibilities include image-quality reasoning, scoring, and edit-suggestion generation.
- Reasoning Format: Reasoning Format Reward penalizes deviations from the canonical reasoning → suggestion → score template.This enforces structured aesthetic analysis in the chain-of-thought process.
- Score Ranking: Score Ranking Reward compares critic scores for an input and its edited result to encourage higher scores after effective enhancement.The resulting ordering places quality-enhanced versions above their originals when restoration or retouching is effective.
- Edit Suggestions: Edit suggestions use explicit corrective constraints for restoration and exploration-driven supervision for retouching.The data sources include restoration pairs, FiveK color grading data, and AVA-based high-aesthetic images with synthetic degradations.
- Edit Suggestions: Jointly combining restoration supervision with retouching exploration helps the Image Critic produce interpretable and actionable edit suggestions.This design bridges visual understanding and controllable image enhancement.
4 Experiments
SmartPhotoCrafter is evaluated across automatic enhancement, multi-edit instruction adherence, and image restoration, using perceptual, semantic, structural, and distributional criteria. It achieves strong results while balancing aesthetic improvement, photorealism, and content preservation.
- Experimental Setup: SmartPhotoCrafter is evaluated against open-source generative baselines under automatic enhancement settings that require models to identify deficiencies without user instructions.Metrics include MUSIQ and NIMA for perceptual quality, plus DINO, CLIP, FID, and LPIPS for semantic, structural, and distributional consistency.
- Experimental Setup: The experiments use datasets spanning image quality assessment, restoration, aesthetic retouching, and unified photographic enhancement.The evaluation includes FiveK, degraded AVA images, and image restoration testsets, while training uses complementary IQA, distortion, restoration, and retouching data.
- Automatic Photographic Enhancement: SmartPhotoCrafter achieves the best DINO and CLIP scores and the lowest FID and LPIPS in automatic photographic enhancement, while ranking second on MUSIQ.The results indicate a balance between perceptual quality, semantic alignment, distribution fidelity, and photorealistic editing.
- Multi-Edit Instruction Adherence: SmartPhotoCrafter outperforms compared methods across all multi-edit instruction-adherence metrics, including LPIPS 0.09, DINO 0.97, and CLIP 0.96.It also achieves the highest PSNR and SSIM and the lowest FID, supporting fidelity, structural consistency, and alignment with high-quality reference-image distributions.
- Qualitative Results: SmartPhotoCrafter produces natural enhancement by improving color while preserving scene structure and textural details without artifacts or content distortion.Qualitative comparisons contrast this balance with under-enhanced, over-enhanced, or AI-stylized outputs from other methods.
- Image Restoration: On deblurring and dehazing, SmartPhotoCrafter achieves the best or second-best performance across all reported metrics, with strong perceptual closeness and reconstruction fidelity.It obtains the lowest LPIPS, DISTS, and FID while maintaining competitive PSNR and SSIM, and shows a favorable balance between enhancement strength and content preservation.
4.4 Ablation Study
The ablation study examines reinforcement learning and the photometric reward rphoto. Reinforcement learning alone improves perceptual and aesthetic scores but harms distributional fidelity, whereas adding rphoto yields more balanced improvements.
- Study Design: The ablation evaluates reinforcement learning and the photometric reward rphoto on FiveK and degraded AVA-GT paired data.Input images contain suboptimal styles, while ground-truth images serve as expert-level references.
- Reinforcement Learning: RL without rphoto raises MUSIQ from 67.82 to 68.25 and NIMA from 5.57 to 5.58, but worsens FID from 30.61 to 38.51.DINO and CLIP remain unchanged, suggesting that unconstrained RL improves perceptual or aesthetic scores at the expense of distributional fidelity.
- Photometric Reward: Adding rphoto reduces FID from 38.51 to 27.96 and increases NIMA from 5.58 to 5.66 in the full reinforcement-learning model.DINO and CLIP reach 0.98 and 0.96, respectively, indicating improved semantic and structural alignment.
- Photometric Reward: The photometric reward provides fine-grained supervision for realistic tonal adjustments while preventing over-optimization.The full reward combination supports both high perceptual quality and faithful distribution alignment.
5 Conclusion
SmartPhotoCrafter couples multimodal aesthetic reasoning with photorealistic image enhancement through a unified reasoning-to-generation framework. It performs strongly across enhancement scenarios while preserving naturalness and visual coherence, but does not yet address higher-level composition.
- Conclusion: SmartPhotoCrafter integrates multimodal understanding with photorealistic enhancement by using visual-language reasoning to condition image synthesis.Its rewards jointly encourage image fidelity, consistency with inferred edits, and perceptual quality improvement.
- Conclusion: The framework achieves strong performance across diverse enhancement scenarios while maintaining naturalness and visual coherence.The conclusion characterizes coupling semantic understanding with generative modeling as promising for intelligent photographic enhancement systems.
- Limitations and Future Work: The current framework focuses on restoration and photometric adjustments while preserving original content and structural layout.Higher-level compositional factors remain unexplored and are identified as future work.
A More Qualitative Results
Qualitative case studies show that SmartPhotoCrafter analyzes image deficiencies and produces targeted, photorealistic enhancements that improve aesthetics while preserving scene content and structure.
- SmartPhotoCrafter produces visually pleasing results with improved aesthetics, tonal balance, and details across diverse shooting scenarios.
- The enhanced images improve color balance and detail while faithfully preserving the original content and structural composition.
- The method performs realistic enhancement rather than re-synthesis, maintaining photorealistic outputs without artifacts or unnatural appearances.
- IQA case studies combine Image Critic analyses, input images, and Photographic Artist results across images with varying quality levels.
- The Image Critic analyzes diverse image attributes and summarizes actionable editing suggestions, enabling the Photographic Artist to generate improved enhancement results.