PRAISE: A Reproducible Workflow for Non-Programmer Scientists to Implement Published AI Models

• Implementation can be difficult because scientists may lack data-science expertise, suitable computing infrastructure, or familiarity with installation, training, and validation.Source: §1 · p. 1
• PRAISE is a hands-on workflow that helps non-programmer scientists reproducibly implement a published AI model in their own practical application.Source: §1 · p. 1
• Existing workflows and resources address parts of AI implementation, but leave the route from a published model to working deployment undescribed.Source: §1 · p. 1
• Implementation means the full route of selecting, installing, running, and validating a model; reproduction, adaptation, and deployment name distinct stages or aims.Source: §1 · p. 1
• Readers can follow the paper sequentially or begin with the workflow in Section 4 and consult Sections 2 and 3 as needed.Source: §1 · p. 2

2.1.1. Dataset size

• Dataset size needs to represent the underlying population; small datasets may overfit, and the sufficient size depends on data complexity, architecture, and task.Performance generally improves with increasing dataset size until saturation; suggested samples-per-feature rules range from 3 to 300.Source: §2.1.1 · p. 2

2.1.2. Non-ideal dataset properties

• Gaps, biased representation, skewed classes, and randomly sampled negatives can constrain model applicability or introduce false negatives.A toxicity dataset missing compound categories may perform poorly on them, while a dataset dominated by similar molecules may specialize the model.Source: §2.1.2 · p. 2

2.1.3. Dataset splitting

• Training, validation, and test data are separated to evaluate on unseen data; the leakage protocol also checks for improper access to information during training.Splits can be randomized, so reporting the seed supports reproducibility; leakage can inflate apparent performance, and the proposed checklist examines pipeline stages.Source: §2.1.3 · p. 2, §2.3 · p. 4

2.2.1. Structure of neural networks

• Neural networks transform inputs through layers of nodes, with weights and biases optimized during training to model relationships and produce predictions.Nodes combine weighted inputs with a bias and apply a nonlinear activation function, allowing complex relationships to be captured.Source: §2.2.1 · pp. 2–3, §2.2.2 · p. 3

2.2.2. Loss function minimization

• Training minimizes prediction loss by propagating errors backward, updating weights and biases, and repeating the process across batches and epochs.The loss measures discrepancy between predictions and target values; it should generally decrease over successive epochs, though it can fluctuate early.Source: §2.2.2 · p. 3

2.2.3. Loss function monitoring and early stopping

• Monitoring training and validation loss can reveal overfitting; minimum validation loss can identify a stopping point whose weights are used for new predictions.Source: §2.2.3 · p. 3

2.2.4. Common types of machine learning

• Supervised learning uses known target values, unsupervised learning finds patterns in inputs, and reinforcement learning learns through actions and their consequences.Source: §2.2.4 · p. 3

2.3. Verifying model training and predictive performance

• Training diagnostics inspect completion, loss, and learned parameters, while validation metrics assess predictions; ROC-AUC can miss minority-class precision, and MCC incorporates all confusion-matrix outcomes.Weight histograms can be compared across training stages or runs; ROC-AUC values near 1 indicate strong performance, 0.5 random classification, and below 0.5 worse-than-random ranking.Source: §2.3 · pp. 3–4

3. Computational stack and development tools for AI models

• The stack links hardware and software layers, with information flowing through them during model execution; users configure it through development tools and environment and package managers.All layers and dependencies must be installed in compatible versions before the model can run.Source: §3 · p. 4, Figure 1 · p. 5

3.1. Computing resources

• A capable GPU is recommended for AI workloads; local PCs, cloud resources, and hybrid setups offer different ways to meet computing needs.A hybrid approach can start installation, troubleshooting, and exploratory runs locally before moving larger workloads to the cloud, reducing startup cloud-GPU use and costs.Source: §3.1 · p. 4, §3.1.2 · p. 4, §3.1.3 · p. 4

3.1.2. Cloud computing resources

• Cloud computing provides scalable off-site resources, but sensitive data may face cross-border disclosure rules or provider use, so anonymization or synthetic data can reduce privacy risks.The paper advises against uploading privacy-sensitive information to off-site infrastructure.Source: §3.1.2 · p. 4

3.2. Operating system

• Operating-system compatibility is not guaranteed, and even minor version differences can cause instability or errors; reproducing the original environment with a container is advised.For GPU workloads, runtimes such as CUDA provide an interface between software and NVIDIA GPUs, including memory allocation and parallel task execution.Source: §3.2 · p. 4, §3.3 · p. 5

3.4. Libraries and packages

• Libraries and packages provide reusable functions for data handling and visualization, while neural-network frameworks manage model operations and gradients during training.Each dependency must be present in a compatible version; frameworks such as TensorFlow and PyTorch track forward-pass operations and compute backward-pass gradients.Source: §3.4 · p. 5, §3.4.1 · p. 5

3.5. User code: application scripts and notebooks

• User scripts or notebooks coordinate data loading and preparation, model training and validation, and presentation of predictions as figures or tables.The code calls packages from lower stack layers and can be executed in a development tool.Source: §3.5 · p. 5

3.6. Environment and package managers

• Environment and package managers isolate dependencies, helping address version incompatibilities; development tools provide interfaces for editing and executing code, with debugging and file-management features.Model developers’ environment specifications, such as .yml files, should be followed where possible.Source: §3.6 · pp. 5–6, §3.7 · p. 6

3.8. Generative AI as a coding aid

• Generative AI can lower coding barriers through guidance, feedback, code generation, explanations, and suggestions, while domain expertise remains the primary requirement for implementing models from literature.The paper reports that AI coding support can improve coding speed and quality without impairing users’ ability to understand or modify the resulting code.Source: §3.8 · pp. 6–7

4. PRAISE: the workflow for the implementation of published AI models

• PRAISE sets expected outputs and checks for each workflow step, adding fallback routes when checks fail.The workflow covers implementation from use case formulation to deployment; Section 5 addresses potential issues.Source: §4 · p. 7

4.1. Use case formulation

• A use case should define a problem before choosing AI, since conventional IT or data-management approaches may be more suitable.The defined problem also helps determine what kind of AI or ML is required.Source: §4.1 · p. 7

4.1.1. Simultaneous problem and solution discovery

• Problem and solution discovery proceed iteratively until they align, with candidate solutions helping refine an initially vague problem.The guide brings together problem and solution discovery, then performs regulatory compliance afterwards.Source: §4.1.1 · p. 7, Figure 3 · p. 7

4.1.2. Risk assessment and compliance

• Risk assessment and compliance checks address data security and regulatory constraints from the start, including privacy, intellectual property, and non-disclosure requirements.The paper also advises particular caution for personal data and sensitive domains such as eligibility decisions, law, migration, or elections.Source: §4.1.2 · p. 7

4.1.3. Implementation and documentation

• The use case document records the problem, intended use, candidate model, risk level, and performance acceptance criteria set before training.Complete documentation throughout the workflow supports troubleshooting and reproduction; a private Git repository provides a chronological change log.Source: §4 · p. 7, §4.1.3 · p. 8

4.2. Dataset preparation

• Dataset validity should be assessed against the intended use case, including size, gaps, biases, and missing negative examples.Assessment should consider how well the data represent expected situations and how imperfections may affect training performance.Source: §4.2 · p. 8
• When coverage is inadequate, data can be supplemented from public repositories, responsible scraping, or newly generated experimental or synthetic data.The resulting dataset is split into training, validation, and test subsets while avoiding leakage as much as possible.Source: §4.2 · p. 8
• The leakage protocol checks split order, overlap, subset use, and reproducibility, including a fixed-seed split script.Preprocessing follows the split; related records stay together, validation guides training choices, and the test set is reserved for final metrics.Source: §4.2 · p. 8
• A data manifest records data sources, record counts per subset, and the splitting script and seed.Source: §4.2 · p. 8

4.3. Computing infrastructure selection

• Infrastructure must meet model requirements without being too limited to run the model or so large that management and costs become excessive.The selected infrastructure is recorded as the output of this step.Source: §4.3 · p. 8

4.3.1. Constraints on computational infrastructure

• Memory needs depend on model and input details, and training may require more resources than inference with a provided trained model.If retraining is unnecessary and pretrained weights are available, users need not match the developer’s computing setup.Source: §4.3.1 · p. 8

4.3.2. Local hardware computing

• Local hardware is ideal when it is close to the model authors’ specifications, but purchasing dedicated equipment may require substantial investment.Cloud computing can support models that local hardware cannot, though prolonged GPU use costs more; hybrid setup can use local resources before heavier cloud runs.Source: §4.3.2 · p. 9, §4.3.3 · p. 9

4.3.4. OS selection

• Using the operating system employed by the model authors is the rule of thumb; containers or virtualization are alternatives when it is unavailable, often with reduced performance.Source: §4.3.4 · p. 9

4.4. Running an AI model: from installation to computing

• Model code, datasets, and documentation are typically obtained from the authors’ repository, while the required environment, files, and paths must be configured locally.Development tools vary in usability and complexity, and open-source availability does not guarantee security, so code and dependencies should be reviewed.Source: §4.4.1 · p. 9, §4.4.2 · p. 9, §4.4.3 · p. 9

4.4.5. Basic after-training checks

• After training or inference, expected runtime, output files, and their contents provide basic checks that execution completed as intended.Unexpectedly short runtime, missing files, or identical predictions can indicate that training did not complete or the model did not learn.Source: §4.4.4 · p. 9, §4.4.5 · p. 9

4.5. Model validation

• Validation asks whether model results warrant deployment, with weight-distribution analysis offering a way to compare training runs and identify undertrained or overtrained layers.WeightWatcher automates analysis of layer weight matrices and reports a quality value per layer.Source: §4.5 · p. 9
• Predictive metrics are calculated on validation data during training and on test data after training, with repeated seeds or splits indicating stability.Performance acceptability depends on the use case, and test subsets furthest from training data can assess coverage of dataset gaps.Source: §4.5 · p. 10
• Performance indicators need not exactly match the original report; fixed-seed runs across three datasets and six networks still showed accuracy variations of up to 2.9%.The use case document records whether validation met its requirements and guides investigation when it did not.Source: §4.5 · p. 10

4.6. Deployment

• After acceptance criteria are met, deployment tests representative new inputs and checks that users can supply inputs, interpret outputs, and access the model on available hardware.Source: §4.6 · p. 10
• The reproducibility package includes the use case document, data manifest and split script, infrastructure and environment records, run logs, model files, and metrics.The paper identifies this as meeting the bronze standard and recommends public release of code, data, and documentation.Source: §4.6 · p. 10

5. Fallback troubleshooting by workflow iteration

• When a checkpoint fails, troubleshooting routes send users to a decision point and back one or more steps; such iterations are common.The fallback workflow extends the ideal sequence to account for difficulties encountered in practice.Source: §5 · p. 10, Figure 4 · p. 11

5.1. Analyzing dataset mismatch

• When available data do not cover the use case, reformulate the use case to fit the data or proceed with minor gaps documented as validity limitations.Validation can assess performance separately in poorly represented regions of the applicability domain.Source: §5.1 · p. 10

5.2. Assessing computational infrastructure

• Diagnose computational bottlenecks first; if code changes are insufficient, reassess infrastructure, which spans libraries, runtime, operating system, and hardware.If infrastructure changes are impractical or unaffordable, reduce dataset size or reformulate the use case; these remedies may be combined.Source: §5.2 · p. 10, §5.2.1 · p. 10

5.2.2. Adjust the dataset or reformulate the use case

• If infrastructure remains fixed, reduce dataset size without losing needed characteristics or modify the use-case questions to fit available computing capacity.The project’s earlier implementation work can clarify which datasets and questions the infrastructure can handle.Source: §5.2.1 · p. 10, §5.2.2 · p. 11

5.3. Revising user code

• Revise code to address crashes, memory limits, long runtimes, or unexpected outputs; downsizing options trade resource use against training behavior.Smaller batches produce noisier gradients; gradient checkpointing trades computation for memory, precision scaling lowers memory requirements, and gradient accumulation supports larger models on limited hardware. Validate performance against the published results; consider pretrained models if downsizing is insufficient.Source: §5.3 · p. 11, §5.3.1 · p. 11, §5.3.2 · p. 11

5.3.5. Alternative types of machine learning

• If code changes do not resolve computational bottlenecks, consider modifying the training procedure or switching to another machine-learning type, then reassess infrastructure or the use case.Different training implementations may change computational and memory requirements.Source: §5.3.5 · p. 12

5.4. Diagnosing validation issues

• When validation is unacceptable, diagnose the data, scripts, and notebooks; changes in data splits or added records can shift metrics and model coverage.An unmodified reproduction is expected to reach metrics comparable to the published results, though validation outcomes may vary.Source: §5.4 · p. 12, §5.4.1 · p. 12

5.4.2. Permutation Testing for Data Leakage

• If performance is unexpectedly high, shuffled-label permutation tests should approach ROC-AUC ≈0.5 and MCC ≈0; higher scores may indicate leakage or a training or evaluation error.Compare training and evaluation scripts with the published implementation, checking data handling, preprocessing, configuration, optimization, and evaluation; reformulate the use case if diagnostics do not resolve the issue.Source: §5.4.2 · p. 12, §5.4.3 · p. 12

6.1. First pass: reproducing ProSmith

• The project aimed to reproduce ProSmith’s held-out test performance and deploy predictions for new enzyme-substrate pairs to prioritize candidates and mutants.The case addressed matching enzymes to substrates for a chemical recycling project.Source: §6.1 · p. 12

6.2. Iteration 1: unstable retraining on Windows

• After repeated retraining errors on Windows, moving to cloud-based Linux enabled baseline training to run without those errors and cut all training steps from about four days to under two.The operating system was identified as the likely cause, though the errors had not been traced to a specific cause.Source: §6.2 · p. 13

6.3. Iteration 2: training data did not cover the use case

• Validation found the enzymes and substrates of interest absent from training data, prompting literature-based dataset supplementation and targeted preprocessing and gradient-boosting changes.Added enzyme-substrate pairs were traced to original publications to verify their labels.Source: §6.3 · p. 13

6.4. Iteration 3: simulated negatives in the training data

• Because the training data used simulated negatives, ProSmith was retrained on VIPER, which contains experimentally determined negative outcomes.The VIPER data were converted to ProSmith format, shuffled, and split using Kroll et al. (2024)’s ratio with a fixed random seed.Source: §6.4 · p. 13

6.5. Iteration 4: suspected data leakage

• A suspected leakage issue arose because validation data were used during gradient boosting; the students restructured training to use training data only and save a single model and its ensemble weights.That choice kept the same models for validation and testing but meant training on fewer records; the table caption reports that the suspected leak fix was found not to be data leakage.Source: §6.5 · pp. 13–14, Table 1 · p. 14

6.6. Comparison of performance metrics across training sessions

• Positive MCC scores showed that ProSmith remained predictive on supplemented datasets, and the restructured pipeline met the use-case criteria for deployment.Each training was run once, so values were indicative; the lower MCC after iteration 4 was deemed consistent with the smaller training set and reported run-to-run variation.Source: §6.6 · p. 14

6.7. Deployment

• The deployed SURF Research Cloud script predicts new enzyme-substrate interactions and maps predictions to input records, but experimental validation is still needed before they guide enzyme selection.The model could also predict how point mutations affect enzyme-substrate interaction.Source: §6.7 · p. 14

6.8. Case study outcome

• Three BSc chemistry students and their tutor, all without programming experience, reproduced and extended ProSmith within four months using PRAISE and four issue-driven iterations.The students independently handled installation, data management, training, and validation, adapting the pipeline for additional datasets.Source: §6.8 · p. 14

7. Discussion

• PRAISE divides literature-model implementation into explicit, verifiable steps and includes fallback loops for common issues.Source: §7 · p. 14
• Dataset and model validation support reproducibility, but prediction accuracy is separate from scientific value, which users must assess themselves.Feature analysis is excluded because learned vector inputs in complex models such as ProSmith lack direct chemical interpretation.Source: §7 · p. 14
• Basic computer and data skills remain necessary, while model or dataset availability and documentation quality determine whether implementation can begin.Users may also need expertise beyond the workflow and generative-AI coding helpers, or computing resources their institution cannot provide.Source: §7 · pp. 14–15
• Absent data covering the research question, predictions are unreliable regardless of model quality.Source: §7 · p. 15
• Implementing a literature-based AI model is presented as a learnable skill that can make non-programmer scientists evaluators and improvers of AI models.Source: §7 · p. 15

Data availability statement

• The use-case results do not constitute publishable data for this paper, and its files and validation results were stored in a private GitHub repository for internal use.Results will be made available in a future publication if deployment leads to novel scientific insights.Source: Data availability statement · p. 15

Declaration of generative AI and AI-assisted technologies in the manuscript preparation process

• The authors used Claude for literature search, code interpretation and writing, grammar checks, and shortening text, then reviewed and edited the content.The authors take full responsibility for the published article.Source: Declaration of generative AI and… · p. 15

Funding

• The research was carried out within the C-Recycle consortium and co-funded by the European Union via the Just Transition Fund, project JTFZ-00101.The funder had no role in study design, data collection, analysis, or the decision to publish.Source: Funding · p. 15