A Fortran-Keras Deep Learning Bridge for Scientific Computing

1 Introduction

• Fortran remains in use across scientific fields, and rewriting large, complex legacy codebases in mainstream languages is difficult.Its backward compatibility lets modern compilers build code written in the 1960s and 1970s.Source: §1 · p. 1
• Recent growth in training-set size and computing power has contributed to increased deep-learning use across scientific and other domains.Scientific applications include physically informed networks and neural methods for learning partial differential equations.Source: §1 · pp. 1–2
• Fortran programmers have few options for implementing deep neural networks because modern deep-learning libraries are not available in Fortran.Source: §1 · p. 2
• FKB offers a two-way bridge for using Keras models in Fortran, avoiding the arduous, time-consuming alternative of rewriting deep-learning libraries.The bridge is intended to connect environments where deep-learning resources are plentiful with those where they are scarce.Source: §1 · p. 2

2 Fortran Projects

• FKB is intended to make neural-network integration possible in existing large-scale Fortran projects.The paper identifies climate and ocean simulators, computational-fluid-dynamics packages, and molecular-physics tools as potential applications.Source: §2 · pp. 2–3
• Fortran is used in climate and ocean models including CESM, NEMO, HYCOM, and WRF.The passage describes applications ranging from global and regional ocean circulation to atmospheric simulations.Source: §2 · p. 2
• Fortran-based projects also support finite-element mechanics, earthquake simulation, computational fluid dynamics, and molecular modeling.Examples include ANSYS, ABAQUS, SPECFEM3D, Code Saturne, NEK5000, CHARMM, and NWChem.Source: §2 · pp. 2–3
• Figure 1 reports programming-language usage from a 2018 Kaggle survey and deep-learning-framework usage from a 2019 Kaggle report.Source: Figure 1 · p. 3

3 The Python Anchor (Deep Learning)

• Keras is selected as the Python anchor because of its reported practitioner popularity, convenience, and combination of customizability and ease of use.It is an API built on TensorFlow that supports implementing, training, and testing networks.Source: §3 · p. 3
• Figure 2 depicts FKB's position within the Fortran and Python ecosystems.Source: §3 · p. 3, Figure 2 · p. 4
• FKB/P sits on top of Keras to access models produced there and transmit them to the Fortran anchor, FKB/F.This structure supports integrating deep neural-network architectures into Fortran applications.Source: §3 · p. 4

4 The Fortran Anchor (Scientific Computing)

• Running a Python instance from Fortran is described as too slow and resource-intensive for large-scale computation, while hand-coded or single-use implementations are also noted.Source: §4 · p. 4
• Neural Fortran provides a native Fortran foundation that FKB extends to convert Keras models for use in existing Fortran projects.The passage reports that Neural Fortran supports arbitrary-size networks with data-based parallelism and has benchmark compute performance comparable to Keras with lower memory use.Source: §4 · p. 4
• FKB supports training networks in Python and transferring them to Fortran simulations, and sending Fortran models back to Python for analysis, expansion, and optimization.The Python-side transfer can support hyperparameter searches using available tools.Source: §4 · p. 4

5 Features of FKB

• FKB addresses the burden of testing many climate-simulation networks by bridging Keras and Fortran, enabling iterative tuning in Python and testing in Fortran.The prior hard-coded approach required rewriting and recompiling Fortran for each model with different hyperparameters.Source: §5 · p. 5

5.1 FKB/P

• FKB/P translates saved Keras architectures, weights, and biases into equivalent Fortran networks, and can translate Fortran modifications back to HDF5.Fortran-built networks can also be moved to Keras for further evaluation or hyperparameter tuning.Source: §5.1 · p. 5

5.2 FKB/F

• FKB/F extends Neural Fortran with features that increase flexibility and support communication across the bridge.The paper develops this side of FKB through customizable layers, training, losses, and ensembles.Source: §5.2 · p. 5

5.2.1 Custom Layers

• FKB encapsulates forward and backward operations in extendable layer types, allowing arbitrary layers and user-defined operations.It supports dense layers, dropout, and batch normalization.Source: §5.2.1 · p. 6

5.2.2 Training in Fortran

• FKB supports online backpropagation in Fortran, enabling models to be trained there or updated with corrections during simulation.Online training can also compensate for deficiencies in a primary model or address differences between offline and online data distributions.Source: §5.2.2 · p. 7

5.2.3 Custom Loss Functions

• Users can define application-specific losses, including penalties for physical-constraint violations, but must also provide derivatives because FKB lacks automatic differentiation.The cross-entropy example supplies separate scalar-loss and derivative functions.Source: §5.2.3 · pp. 7–8

5.2.4 Ensembles

• FKB ensembles average predictions across models, which can smooth member errors and reduce instability from physical-constraint violations.The implementation can add Gaussian noise to each member’s input and parallelize predictions with OpenMP.Source: §5.2.4 · p. 8

6 Case Study

• The training data quadrupled CRM extent from 32 to 128 km; earlier fits to the enriched data had failed and crashed within weeks online.Source: §6 · p. 9
• SHERPA random search trained 108 Keras candidates, which FKB transferred into Fortran for online SPCAM3 tests without rewriting Fortran code.The candidates were each trained for 25 epochs with early stopping monitoring validation loss.Source: §6 · pp. 9–10
• Although most stable runs had major climate errors, the best-offline-error network combined stability with temperature biases below 2 K, competitive with.Temperature RMSEs for many other runs frequently exceeded 10 K; stability alone was not sufficient for prognostic success.Source: §6 · p. 10, Figure 3 · p. 10
• Offline validation MSE correlated negatively with online stability (Spearman correlation −0.73; p = 4.961e−19).Online performance was measured by time-to-failure in the coupled climate model.Source: §6 · p. 10, Figure 4 · p. 11
• SGD, Adam, and RMSProp correlated with online performance at −0.6670, 0.5936, and 0.0586, respectively; SGD models never exceeded 1,000 steps.Optimizer choice had the largest absolute correlation among the examined hyperparameters; network depth and parameter count were less predictive of coupled skill.Source: §6 · p. 11, Table 2 · p. 12
• 87,840 steps: the five best models completed five-year simulations, versus 128 steps for Rasp et al.’s model.Two of the five also had simulated tropospheric temperature RMSE below 2 °C.Source: §6 · pp. 11–12

• FKB provides a two-way Fortran–Python bridge that lets researchers transfer neural-network models to and from Keras.Source: §7 · p. 12
• Researchers can access many Keras API features directly in Fortran, including custom layers and loss functions.Source: §7 · p. 12
• Precompilation and linking let FKB be leveraged in existing large-scale simulators, demonstrated with SPCAM3 atmospheric simulations.Source: §7 · p. 12