freeCAM

Manual

This manual describes how to use freeCAM. Specifically, it covers ➊ reading the live state of a running model from a notebook, ➋ editing and watching the model in the browser with the Workflow Builder, ➌ the layers of one MPI rank and the classes in each, and ➍ the Python classes of the physics processes and the kernels each one can replace.

Reading the live state

Rank 0's mean profiles of temperature, both winds, water vapour, vertical velocity and pressure, read live from the model
Figure 1. Mean profiles of temperature, zonal and meridional wind, water vapor, vertical pressure velocity, and pressure on rank 0, read from the running model.

Every CAM field is exposed as a distributed NumPy array. Specifically, a notebook can request ➊ statistics over all ranks, ➋ the columns owned by one rank, or ➌ a plot, all without stopping the model. Figure 1 shows rank 0's mean profiles of six fields, read from the running model.

driver.cam.state.T.stats(rank="global")
driver.cam.state.plot(rank=0,
    variables=('T', 'u', 'v', 'q', 'omega', 'pmid'))

Editing and watching the model in the browser

The Workflow Builder is a browser page attached to the same model. In it, a user can edit the step, generate the corresponding freeCAM code, run it, and watch the state on a globe. The page is launched from Python, and the ranks keep each step's state in memory so that the globe can display it.

import freecam as fc

driver = fc.Driver(case="PI-atm", nsteps=1488,          # one model month
                   record_state={"store": "memory", "keep_steps": 1500})
ui = driver.ui(port=8793)
print(ui.url)   # open it; Run starts the model, the Globe tab follows it

From a shell, uv run python -i -c '...' keeps the page up. Note that without record_state, ui() still keeps the state of the last 1000 steps in memory.

Reordering the step and generating its code

Every process in the step can be moved, switched off, removed, or replaced. A change that departs from the validated default is marked experimental, since only the default has been checked bit-for-bit against the original. Generate then writes the step as freeCAM code in one of three forms: ➊ a complete program, ➋ a notebook, or ➌ the setup for a running driver.

The Workflow Builder: shallow convection is moved ahead of deep convection, the check asks for Experimental, and Generate writes the code, in which shallow_convection now comes before deep_convection
1 / 9
Figure 2. Reordering the step in the Workflow Builder. Shallow convection is moved ahead of deep convection, the check marks the change as experimental, and Generate writes the code in the new order.

Watching the state on the globe

Run starts the model, and the Globe tab follows it. Specifically, the Globe tab shows the state in two ways. ➊ Inside one step, it shows what each process alone did to the state. Figure 3 shows the seven processes that change the temperature at step 24. ➋ Step by step, it shows how the state evolves. Figure 4 shows one model day of the same run, in which the air over land warms in daylight and cools at night, by 5 to 7 K over the Sahara and the Kalahari, while over the ocean it changes by less than 1 K. Further, a click on the globe picks a column and shows its profile and the latitude-pressure section along its meridian, as shown in Figure 5. Note that a run recorded to disk with record_state=True can be opened later with freecam globe <run>/state.

Temperature at 848 hPa at the start of a step, then the change made by each process in turn: vertical diffusion, gravity-wave drag, dynamics, deep convection, shallow convection, cloud macro/microphysics and radiation
1 / 8
Figure 3. Temperature at 848 hPa at the start of step 24, followed by the change made by each process in turn: vertical diffusion, gravity-wave drag, dynamics, deep convection, shallow convection, cloud macro/microphysics, and radiation.
Temperature at the lowest model level through 50 steps, one model day: the land warms by day and cools by night, the ocean barely changes
1 / 50
Figure 4. Temperature at the lowest model level over 50 steps, i.e., one model day. The land warms by day and cools by night, while the ocean barely changes.
The globe viewer: temperature at 848 hPa over the globe, the column picked at 23.5N 12.8E, and the latitude-pressure section along its meridian
Figure 5. The globe viewer, showing temperature at 848 hPa, the column picked at 23.5°N 12.8°E, and the latitude-pressure section along its meridian.

The layers

One rank of the model, top to bottom. Control flows down, and only down: the Fortran never calls back into Python. A pause hands control back to Python and waits for a resume.

Usernotebook, script or command line
Python APIfc.Driver / fc.FreeCAM: prepares an isolated run directory; the first live operation launches one persistent MPI session.src/freecam/pi_cam/facade.py
Command line--python-stages, --segmented-original-kernels, --disable-actions, --stage-execution.src/freecam/pi_cam/cli.py
Workflow BuilderEdits, checks and generates a workflow in the browser; reads the same runner manifest to say which kernels are bindable and validated.web/, src/freecam/pi_cam/workflow_builder/
cloudpickled commands over multiprocessing.connection, one worker per MPI rank
Python control layerruns on every one of the 512 MPI ranks
Workflow: one ordered list of actions per step boundary import → physics before the coupler (cam_run1) → dynamics → physics after the coupler (cam_run2) → history → clock → boundary export. Actions can be enabled, disabled, moved and run; a notebook-defined fc.Physics can be inserted without rebuilding CAM. Installing a stage class disables the original action and puts the class's tend() in its slot. src/freecam/pi_cam/plan.py, runtime_processes.py, model/python_processes.py
9 pausable classesDryAdjustment, ShallowConvection, DeepConvection, ConvectiveTracerTransport, VerticalDiffusion, GravityWaveDrag, AerosolWetDeposition, AerosolDryDeposition, ChemistryTendencies. Each is a declaration: its action, its runner prefix, its swappable kernels.src/freecam/physics/pausable.py
RadiationA split stage: the plan stops before radiation_tend and resumes after it; the class runs between the halves over the radt runner.src/freecam/physics/radiation.py
CloudMacroMicrophysicsStage 7, the first class built, over the stage7 runner with mmacro_pcond and micro_mg_tend.src/freecam/physics/cloud_macro_microphysics.py
EnergyFixer and 11 InertStagesOwn their actions whole and expose no kernel: the energy fixer because its core allocates its tendency inside the call; the inert eleven because their bodies do nothing in this configuration, proven by one gate with all of them disabled.src/freecam/physics/pausable.py
NativeStage.tend(): two paths, chosen by the kernel slots
native-wholeNo kernel replaced. The class calls its original Fortran action once through run_action(): no runner, no views, no copies. This is the default path and costs about one percent.
or
segmentedSome kernel replaced. SegmentedStage.run() drives the image's runner:
  1. create once, then start with the replacement mask
  2. on NEEDS_KERNEL: frame gives the paused call's arguments where they live
  3. the model answers the batch, or original runs the very call for a gate
  4. write-back into the live lanes of every output, then resume
  5. repeat until DONE, once per chunk this rank owns
src/freecam/physics/stage.py, segments.py; src/freecam/pi_cam/segment_runner.py
StatePoolRank-local NumPy arrays that are zero-copy views of live Fortran state; distributed get, stats and mutation through the facade.src/freecam/pi_cam/state.py
Kernel coverage ledgerEvery action's class, kernels, gates and performance evidence, rebuilt from the records and checked for staleness.validation/physics_kernel_decoupling.json, src/freecam/pi_cam/kernel_coverage.py
ctypes over a bind(C) ABI: pycam_<prefix>_{create, start, frame, resume, original, error, reset, destroy}_v1
Generated adaptersconvert pointers, shapes, scalars and handles; copy no numerics
Pausable runnersOne state machine per process. The original driver is hoisted verbatim into a module (its locals become module state), cut into pieces at the kernel call sites; the runner re-expresses the if / do / select skeleton around the pauses and carries out a piece's return, cycle or exit.native/pi_cam/support/pycam_<prefix>_{glue,driver,runner}.F90 (31 modules)
Specs and generatorTen YAML specs name the pinned source ranges, the pauses, the dummies, carries and options; the generator writes the modules and the frame table, and hashes every range so a moved source fails --check.native/pi_cam/pausable/*.yaml, tools/pi_cam_pausable.py
FramesThe paused call's arguments in the callee's order: address, rank, shape, dtype, intent. Derived types are served component by component; an optional a site omits is an empty slot.native/pi_cam/segment_frames.yaml
ManifestWhich stage has a runner, what its entries are called, which kernels it pauses at, and the gate records that validated each pause.native/pi_cam/segment_runners.yaml
Hosts and carriesViews of phys_state, phys_tend, pbuf, cam_in and cam_out for any stage; views of the locals tphysbc and tphysac carry between actions.pycam_stage_hosts.F90; control patches 0039, 0043
Module-state patchesOne public statement each, naming the private variables a hoisted driver reads; the patched module is compiled for its interface only and the oracle's object is linked.control patches 0044 to 0048
one fixed-address image, linked from the oracle build's own objects
Original iCESM1.3.1 Fortranthe numerical source of truth, byte for byte
physpkg with control patchesAction dispatch, stage boundaries and carry accessors added around the original statements; no numerical statement changes.
The 17 kernels and everything around themConvection, turbulence, radiation, microphysics, deposition, chemistry, the SE dynamical core: all compiled once in the oracle build and reused unchanged.
libfreecam_pi_cam.soThe image freeCAM loads; its manifest records every ABI symbol and the SHA-256 of the numerical library.build/pi_cam_promoted/
MCT x2a / a2x arrays as zero-copy views
CESM coupler and the other componentsonline boundary provider: live CLM, CICE, DOCN, RTM
The default boundary is the real coupled system, run in the same job. An offline provider replays captured boundary datasets for the replay cases.src/freecam/pi_cam/boundary.py
Figure 6. One rank of freeCAM, top to bottom: the user's entry points, the Python control layer, the generated adapters, the original iCESM Fortran, and the CESM coupler with the other components.

The classes and their kernels

Every pause has been answered by the original kernel through Python in a 512-rank, 50-step run compared with the pinned iCESM reference with no numerical tolerance. The pause count is per 50 steps; radiation runs every other step, wet deposition pauses for every mode, phase and species.

Table 1. The Python classes, the runners behind them, and the kernels each one pauses at, with the 50-step gate that validated every pause.

ClassActionRunnerKernels (frame slots)Pauses / 50 stepsGate
DryAdjustmentcam_run1.dry_adjustmentdadadjdadadj (7)100bit-for-bit
ShallowConvectioncam_run1.shallow_convectionshcucompute_uwshcu_inv (54)100bit-for-bit
DeepConvectioncam_run1.deep_convectionzmdeepzm_convr (70), zm_conv_evap (22), momtran (25)100 eachbit-for-bit
ConvectiveTracerTransportcam_run1.convective_tracer_transport_leafzmtranconvtran (21), transporting with the mass fluxes DeepConvection left in the module's own arrays100bit-for-bit
CloudMacroMicrophysicscam_run1.cloud_macro_microphysicsstage7mmacro_pcond, micro_mg_tend (from their reviewed contracts)100 eachbit-for-bit
AerosolWetDepositioncam_run1.aerosol_wet_deposition_leafawetwetdepa_v2 (32), two call sites3000bit-for-bit
Radiationcam_run1.radiation (split)radtrad_rrtmg_sw (58), rad_rrtmg_lw (35)50 eachbit-for-bit
ChemistryTendenciescam_run2.chemistry_tendencies_leafchemgas_phase_chemdr (34), the whole gas-phase driver as one kernel100bit-for-bit
VerticalDiffusioncam_run2.vertical_diffusionvdiffcompute_tms (14), compute_eddy_diff (55), compute_vdiff (42) at two call sites100, 100, 200bit-for-bit
AerosolDryDepositioncam_run2.aerosol_dry_deposition_leafadrymodal_aero_depvel_part (13), four call sites800bit-for-bit
GravityWaveDragcam_run2.gravity_wave_draggwdgw_drag_prof (33), the band and the pressure coordinates served by component100bit-for-bit
EnergyFixercam_run1.energy_fixernoneno kernel: check_energy_fix allocates its tendency inside the call and reads private module state0owned whole
11 InertStagesrayleigh_friction, charge_neutrality, qbo_relaxation, ion_drag, sea_salt_rebin, modal_aerosol_preparation, carma (3), tracer and age-of-air tendenciesnoneno kernel: the bodies do nothing in this configuration0disabled at once, bit-for-bit

One further run installed all eleven classes and paused every one of the 17 kernels in the same 50 steps (5300 pauses): bit-for-bit. The nine numerical actions that are disabled by default are alternate forms of the same work (a parent stage whose leaves run, or a leaf whose stage runs whole) and are owned through them.