What Will This Copper Look Like Later? Forecasting Surface Appearance and Rendering It as a PBR Material

Teejuta Sriwaranon; Borworntat Dendumrongkul; Tanapat Chamted; Pizzanu Kanongchaiyos

Source-linked AI summary

What Will This Copper Look Like Later? Forecasting Surface Appearance and Rendering It as a PBR Material

Teejuta Sriwaranon, Borworntat Dendumrongkul, Tanapat Chamted, Pizzanu Kanongchaiyos

arXiv:2608.28102v1Aug 28, 2026cs.GRcs.CV

TL;DR

The paper asks whether a particular copper specimen’s later oxidized appearance can be forecast from a fixed-camera observation and rendered as standard PBR maps. It compares learned and closed-form forecasters under whole-recording holdout, finding that only the parameter-free global color extrapolation transfers reliably to unseen specimens. The resulting recommendation is to use that forecaster for new specimens and learned models only for continuing already observed footage.

Problem
The paper addresses how to predict the later appearance of a particular photographed copper surface and place that future state in a rendered scene.
Method
The system forecasts future copper frames from fixed-camera observations, converts them into albedo, normal, roughness and metallic PBR maps, and evaluates transfer with entire recordings held out.
Results
Only the closed-form global color extrapolation transfers to unseen specimens; trained forecasters are less accurate than copying the last frame under this evaluation.
Takeaways & Limitations
Use the parameter-free forecaster for new specimens and the learned model only to continue a specimen with existing footage.
Takeaways & Limitations
The evidence demonstrates transfer within copper but does not measure how copper specimens vary generally, and curved geometry may violate the assumption of uniform aging across the surface.

Abstract

from arXiv · show

Digital design requires predicting how a metal surface will look later in its oxidation; this paper presents such a pipeline for copper. Given a fixed-camera observation, the system forecasts appearance 10 accelerated units ahead and converts it into the albedo, normal, roughness and metallic maps a renderer consumes. Forecasting is evaluated as an authoring tool would use it, on a copper specimen the system has not observed: an entire recording is held out, so training and checkpoint selection use one specimen and the test set is the whole of a second, recorded on a different day and condition. Under this protocol a learned spatio-temporal model with a monotone oxidation state, the most accurate forecaster within a single recording, is less accurate than copying the last observed frame on an unseen specimen, in both directions, as are three further trained architectures. The only forecaster that transfers is a closed-form global color extrapolation with no trained parameters, improving on copy-last-frame by 13.4% and 50.6%, with a margin that increases with horizon to +16.7% and +55.5% at t+10. Two controls qualify this: correcting every frame for the photometric drift measured on a non-oxidizing reference region leaves both margins intact, ruling out uncontrolled exposure as their source, and a moving-block bootstrap over the 6 independent windows each recording contains separates the larger margin from zero but leaves the smaller one not individually significant. The mechanism is measured: a learned susceptibility map encodes where corrosion begins on the training specimen and misleads on a new one, whereas the global color trajectory is what specimens share. The pipeline therefore deploys the closed-form forecaster for unseen specimens and the learned model only for continuing one already observed. Code, splits, protocol and leakage audit are released.

1 Introduction

The paper forecasts the future appearance of a particular copper specimen from observation and converts that forecast into renderable PBR material maps. It emphasizes transfer to unseen specimens and finds that parameter-free global color extrapolation transfers better than trained forecasters under that protocol.

The pipeline predicts how a photographed copper specimen will look later in oxidation and converts the result into albedo, normal, roughness and metallic maps.
The evaluation holds out an entire recording so the test specimen differs in specimen, day and condition from training.
The study is limited to copper, with stronger-than-usual within-metal transfer evaluation but no claim of cross-metal transfer.
Two candidate forecasters span a learned monotone-oxidation model and a parameter-free closed-form global color extrapolation.
Every trained model is worse than copying the last frame on an unseen specimen, while only the parameter-free extrapolation transfers.
The work distinguishes future-state prediction from procedural weathering, measured time-varying appearance capture, spatio-temporal forecasting, and heritage restoration.

3 The Pipeline

The pipeline observes a fixed-camera copper recording, forecasts future frames, and materializes them as ordinary PBR textures. Patina coverage controls physically motivated roughness and metallic responses, while calibrated thresholds prevent noise from collapsing the material to bare metal.

The system has three stages: observe the specimen, forecast the next K frames, and materialize a predicted frame as a metallic-roughness PBR material.
One accelerated unit is a fixed fraction of the recording’s aging trajectory, not calendar time.
Patina coverage measures how far the predicted frame has moved from the clean-state canvas formed by the observed context.
The noise-calibrated dead-zone τ and specimen-specific normalizer s determine coverage; an unsuitable τ makes coverage identically zero and collapses the material to bare metal.
Oxide coverage increases roughness and decreases metallic response, while a recovered height field supplies normals and the predicted frame supplies albedo.
The renderer consumes ordinary metallic-roughness textures directly rather than a bespoke shading model.

4 Data and Protocol

The study uses two fixed-camera copper recordings under different conditions and evaluates forecasting with whole-recording holdout. Horizons are normalized aging units, and the metric measures error only where the specimen changes.

The dataset contains 2 copper recordings: one chamber-aged specimen and one open-air specimen recorded on a different day.
Both recordings use stationary cameras, but differ in resolution, frame rate, illumination, duration, specimen and noise.
One accelerated unit represents the same fraction of an aging trajectory across recordings, while calendar-time conversion requires matched natural-exposure data.
Leave-one-recording-out training uses one recording and tests on the complete second recording, preventing memorization of a specimen’s corrosion pattern.
Given 5 observed frames at 128×128, the task predicts the next 10 using foreground-masked MSE over pixels that actually change.
Trained models use AdamW and window-consistent spatial and photometric augmentation that preserves each window’s temporal signal.

5 Two Candidate Forecasters

The candidate forecasters represent two transfer strategies: a learned spatial model with monotone oxidation dynamics and a parameter-free global color model. Their key difference is whether they learn where corrosion occurs on the specimen.

The comparison includes patinanet, a learned model, and affinergb, a parameter-free closed-form global color extrapolation.
patinanet predicts non-negative oxidation increments, diffuses those increments, and keeps the per-pixel oxidation state non-decreasing.
The learned model separates a global pace of oxidation from a static spatial susceptibility map that encodes where corrosion begins.
Its monotonicity applies to oxidation state rather than appearance, because thin-film interference can produce non-monotone color changes.
affinergb fits one global 3×3 color matrix and bias, extrapolates the transforms linearly, and applies the result to the last observed frame.
The learned model uses a specimen-tied spatial prior, whereas affinergb learns no spatial information.
Three additional trained architectures broaden the comparison beyond one learned design.

6 Generalization to an Unseen Specimen

Under leave-one-recording-out evaluation, trained forecasters fail to generalize across copper specimens, while a parameter-free global color extrapolation transfers in both directions. Photometric correction leaves its advantage intact, and the measured mechanism implicates specimen-specific spatial priors rather than shared global color change.

The test holds out an entire recording from a different copper specimen, day, and condition, with both transfer directions reported.Training and checkpoint selection use one recording; the complete second recording is the test set.
+13.4% and +50.6%: affinergb is the only one of five entries beating copy-last-frame on the unseen specimen in the two directions.Every trained model is worse than persistence, including patinanet at −43.2% and −72.3%.
+16.7% and +55.5% at t+10: affinergb’s advantage over persistence increases with horizon in the two directions.Its gains are +5.8%, +9.6%, and +16.7% at t+1, t+5, and t+10 in one direction; the reverse gains are +19.8%, +47.4%, and +55.5%.
Photometric correction does not remove affinergb’s advantage, indicating that the improvement is not explained by uncontrolled exposure or white-balance drift.The chamber reference changes by 0.001 while the specimen changes by 0.128, and the advantage survives gain, offset, and combined corrections.
A learned susceptibility map correlates with corrosion on the training specimen at r = 0.68 but with the held-out specimen at r = 0.00.The spatial prior becomes misleading on a new specimen, whereas affinergb extrapolates the global color trajectory shared across specimens.
The monotonicity guarantee holds, with 0 negative steps in either direction, but it does not provide transfer.Visual changes over t+10 are small—1.2% and 0.6% of full scale—so evaluation requires a noise-calibrated metric rather than visual inspection.

7 Characterizing the Shipped Forecaster

The shipped forecaster is characterized on a held-out specimen by varying context length, color-transform capacity, and extrapolation order. Ten context frames, a full 3×3 color transform, and first-order extrapolation provide the practical configuration.

Context length: Ten context frames improve performance in both directions, making context length the most consequential tested choice.With n = 3, the forecaster is about −7% versus persistence in the harder direction; with n = 10, it is 34% and 58% better in the two directions.
Evaluation protocol: The sweep evaluates the untrained shipped forecaster on a held-out specimen using MSEfg in both transfer directions.Each configuration varies one axis, samples 60 windows, and uses a shifting usable window set as context length changes.
Context length: The deployed pipeline should observe ten frames before forecasting, although trained-model comparisons elsewhere use five.The usable window set changes with context length, so references in this sweep differ slightly from the full-window results at n=5.
Color transform: A full 3×3 color matrix outperforms per-channel and single-brightness transforms in both transfer directions.Reducing the transform from 3×3 to per-channel increases error from 1.72e−04 to 1.93e−04 and from 5.43e−05 to 8.86e−05; single brightness gains are worse again.
Extrapolation order: First-order extrapolation is effective, whereas freezing the transform is worse and quadratic extrapolation is roughly 11× worse in both directions.The reported quadratic errors are 1.83e−03 and 5.80e−04, while linear drift degrades gently beyond its fitted range.

8 Rendering the Forecast

The forecast is converted into PBR textures and evaluated by rendering the held-out specimen under fixed geometry and lighting. Both forecasters outperform a static material across horizons, while the closed-form method leads at ordinary horizons before long-range gamut drift causes a crossover.

The measured mechanism is that a learned susceptibility map tracks corrosion on the training specimen but is uncorrelated on the held-out specimen.The correlations are r = 0.68 for the training specimen and r = 0.00 for the held-out specimen.
Both forecasters beat the static control at every tested horizon on both rendering metrics.LIP and LPIPS compare renders driven by predicted textures with renders driven by true future frames under identical geometry and lighting.
At +10, LIP is 0.230 versus 0.453 for the static control, while LPIPS is 0.163 versus 0.388.
The closed-form forecaster leads at +5, +10 and +20, but the learned model overtakes it at +50.
At t+85, extrapolation pushes 8% of predicted pixels outside the representable range in the chamber-to-air direction, producing the crossover.In reverse, 0% leave the range and the closed-form forecaster remains more accurate even at t+85.

9 The Authoring Loop

The authoring loop captures a fixed-camera aging sequence, fits a closed-form color transform, and converts the forecast into ordinary PBR textures. It avoids model loading and training, allowing horizon changes through a fast refit-and-render workflow.

Capture: The loop captures ten frames under fixed framing and stable illumination, with no material identification, registration, or calibration target.
Forecast: The forecast is a least-squares color-transform extrapolation taking 0.24 s per specimen on one CPU core at 128×128.No model, checkpoint, or GPU is required.
Materialize and render: The predicted frame becomes albedo, normal, roughness, and metallic maps that existing engines can consume as ordinary metallic-roughness textures.
Materialize and render: Changing the aging horizon reruns only the forecast step, allowing interactive horizon scrubbing without re-authoring each material.
The absence of training, model maintenance, and per-metal checkpoints makes the pipeline practical to deploy.This follows from the finding that learned models fail to transfer while the parameter-free forecaster does not require a model to ship.

10 Applications

Because the output is a standard texture set tied to an observed specimen’s trajectory, the pipeline supports artifact visualization, design preview, and accelerated-aging installations. Its parameter-free fit keeps the workflow short for new samples.

The texture-set output integrates into existing authoring workflows without requiring new ones.
Heritage visualization: Conservators can show how a specific bronze or copper artifact is becoming, while combining future aging with generative restoration of its past.
Design and pre-visualization: Architects can preview copper-cladding development on a real material sample through a photograph, forecast, and render loop.The forecaster needs no training on the new sample.
Media art and projection mapping: The time-indexed material sequence can drive projection onto physical or architectural surfaces at any playback speed.Accelerated units describe progression rather than calendar time.
The parameter-free fit removes model shipping, checkpoint selection, and training between photographing a new specimen and forecasting it.

11 Limitations

The evidence establishes transfer for copper but leaves specimen variability, condition effects, spatially varying aging, and calendar-time mapping unresolved. Long horizons and limited statistical replication further constrain deployment claims.

The study uses two copper recordings, leaving exactly one held-out specimen per transfer direction.This demonstrates non-memorization but does not measure how copper specimens vary in general.
Specimen shift is confounded with condition shift because the recordings differ in specimen, environment, and noise level.The data cannot separate these effects.
The six independent windows and single seed leave wide intervals and provide no variance estimate for individual trained architectures.The supported claim is the joint pattern, not a precise effect size for any single row.
Mapping accelerated units to calendar years requires matched outdoor exposure data and analytical confirmation that are unavailable here.
Below roughly five context frames, forecasting is worse than doing nothing, while extrapolation degrades beyond the supported horizon.The proposed deployment rule is to refuse forecasting when context is insufficient.
A context window straddling a color reversal is a predicted failure mode because the non-global-linear trajectory would be fitted through the turn.Neither recording measures this case; broader gold–brown–blue–green coverage is needed.
A global transform cannot represent aging that varies across a surface, such as at drip edges, sheltered regions, or bimetallic junctions.A transferable learned spatial model remains an open problem.

Code and Data Availability

The paper releases the scripts, split builder, audit, forecasters, renderer, and analysis needed to reproduce its results end to end. The corpus contains two fixed-camera recordings of copper specimens.

The released materials include extraction scripts, a leave-one-recording-out split builder, leakage audit, forecasters, renderer, and analysis tools.
The corpus consists of 2 fixed-camera recordings of copper specimens.
A single command generates every table and figure from the run artifacts, enabling end-to-end reproduction.

12 Conclusion

The paper builds an end-to-end pipeline from fixed-camera copper observations to PBR maps and finds that transfer performance favors a parameter-free global extrapolation over trained models. It recommends different forecasters for new specimens and for continuing specimens already observed, while showing that whole-specimen holdout is necessary to expose specimen-specific learning.

The pipeline maps a fixed-camera copper observation to PBR maps consumed by a standard rendering engine.
On unseen specimens, every trained architecture performed worse than doing nothing, while a parameter-free global color extrapolation beat copy-last-frame in both transfer directions with a horizon-growing margin.The passage attributes the result to a learned spatial prior about where corrosion begins.
For new specimens, practitioners should use the parameter-free forecaster; the learned model is reserved for continuing a specimen with existing footage.
Holding out an entire specimen reveals that the learned model captured specimen-specific information rather than the oxidation process.Splitting each recording into time ranges would instead have made the learned model appear to be the clear winner.