feat: osiris wrapper - #279
Conversation
functions (should probably be moved to another branch). Added density and temperature profiles. Fixed LaTeX issues
regenerates from the saved NetCDF artifacts alone — no rerun, no raw MS/ tree. Made list_diagnostics/load_series/load_hist_energy dispatch between the MS/ HDF5 tree and a binary/ NetCDF dir, made field-energy source-agnostic, and now persist HIST/energy.nc in save_run_datasets. Added 3 tests
- regen harness: rebuild the full canned plot set from saved NetCDFs (no rerun) - f(p) and delta-f lineouts; temperature profile from phase-space Maxwellian fits - number-density profiles (initial/final/late-mean); 2-panel equal-aspect omega-k - phase-space & spacetime: space on x-axis, cropped to box, log-contrast floor - proper-LaTeX titles (prose vs math) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ion+reflection plot (should probably be moved to osiris-lpi repo)
sync-up.sh script doesn't delete ongoing osiris runs
|
is this ready to go? |
|
Gave it a read through; should be good to go. Idk if the tests are actually all that useful but I left them in ¯_(ツ)_/¯ |
BaseOsiris.write_units() previously returned {} so OSIRIS runs logged an empty units.yaml. Derive the physical reference scales (wp0, tp0, n0, v0, x0, c_light, beta, box_length, sim_duration) from the deck's simulation.n0 (density) or simulation.omega_p0 (frequency); when both are present, n0 wins, as in OSIRIS. This mirrors the canonical key set the other adept solvers emit so OSIRIS runs are comparable in MLflow.
Adds skin_depth_normalization and skin_depth_normalization_from_frequency to normalization.py. OSIRIS has no single global reference temperature (species carry per-species thermal momenta), so the temperature-dependent keys (T0/nuee/logLambda_ee) are omitted.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
|
Read through this — looks good to merge from my side. The wrapper architecture is clean, the security surface on the subprocess runner is fine (list-form Hardcoded local paths in the example configs. Otherwise LGTM — nice work on the docstrings and the NetCDF regen path. (Separately: the |
|
Fixed the hardcoded paths. Also updated the |
write_units now derives w_laser (rad/s), laser_wavelength (nm), laser_a0, and laser_intensity — the peak intensity of a linearly polarized drive in W/cm^2 (eps0 c E0^2/2 with E0 = a0 m_e c w_laser/e, i.e. the ICF convention I * lam_um^2 = 1.37e18 * a0^2) — from the deck's antenna / zpulse_speckle / zpulse section and the reference wp0. Laser-less decks are unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The engine convention for _vlasov1d (and _vlasov2d, _pic1d) is the RMS/standard-deviation thermal speed: Maxwellian exp(-v^2/2) at T=1, L0 = lambda_De, code wavenumber k*lambda_De. normalization.py:91 was the sole outlier with v0 = sqrt(2 T0/m_e), which made every logged physical unit (v0, x0, c_light, box_length) wrong by sqrt(2), every dimensional string input (um box sizes, gradient scale lengths, laser k0) off by sqrt(2), and a 'normalizing_temperature: 2000eV' run physically a 4000 eV plasma. Dimensionless numeric-input dynamics were unaffected. Also fixes a sign error in the NRL Coulomb logarithm (log(n^0.5 / T^-1.25) == log(n^0.5 * T^+1.25)), which produced logLambda_ee = -11.8 and a negative logged nuee at 2000 eV / 1.5e21cc; the correct values are +7.22 and a positive rate. vth_norm() is deliberately untouched: its only callers are in vfp1d, which is self-consistently built on the sqrt(2T/m) convention. Adds tests/test_vlasov1d/test_units_boundary.py: dimensional input in, dimensional quantity out, checked against CODATA constants and the NRL formulary - the units-boundary test class whose absence let this bug survive every existing suite. See VLASOV1D_CONVENTIONS_AUDIT.md (F1, F2) and ADEPT_CONVENTIONS_AUDIT.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…in write_units _tf1d re-implements the debye normalization inline; it carried the same v0 = sqrt(2T/m) outlier (its engine, like _vlasov1d, runs in sqrt(T/m) units) and the same log(n^0.5 / T^-1.25) sign error. Diagnostics-only: the logged units were wrong; dynamics are unaffected (grid beta is written but never consumed). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…T0/mass - Dougherty.compute_vbar returned the raw first moment n*u while its callers (find_self_consistent_beta, the drag term) treat it as a mean velocity. The drag therefore centered on n*u and momentum was not conserved wherever n(x) != 1. Now returns sum(v f)/sum(f). - The Krook target Maxwellian was hard-coded to exp(-v^2/2) (T=1, m=1): any species with T0 != 1 was dragged toward T=1. It is now built with variance T0/mass from the species' bulk parameters, which are threaded through cfg.grid.species_params (new T0 entry). - New test drives the collision operator in isolation with a 50% density modulation and a finite drift: per-cell density, momentum, and energy conservation, and the relaxed mean velocity must equal u0 (the old operator drifts it toward n*u0). Audit findings F4, F5. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rgy monitor - Field moments p and q were centered on the raw first moment n*u; they are now centered on the true mean velocity. The first moment is saved under the honest name 'j' and 'v' is now j/n (previously 'v' held the flux while the same integrand was called 'mean_j' in the scalars). - The field-moment '-flogf' computed +int f log|f| dv, the exact negative of the scalar of the same name; both now agree. - Adds mean_kinetic_energy / mean_field_energy / mean_total_energy scalars (electrostatic energy monitor, with the 1/2 factors). A conservation monitor of this kind would have caught the sqrt(2) convention bug far earlier. Audit findings F6, F7, F12.2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
dx was xmax/nx and the interpolation period was xmax; both are now the box length (xmax - xmin). Latent for all shipped configs (xmin: 0.0) but wrong spacing, kx grid, and advection wrap for any xmin != 0. Audit finding F11. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GridConfig.c_light (set only by wavepacket.yaml) was never read - beta/c_light are always derived from the normalization. The AmpereSolver class docstring claimed j = sum_s (q_s/m_s) int v f dv; the code correctly has no 1/m_s. Audit findings F12.1, F12.4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The ey driver pair was a backscatter-matched triad under the old c_hat = 11.30 (w_pump - w_seed = 1.16 = old EPW frequency) with placeholder k0 = 1.0 on both drivers. Retuned to a matched triad under the corrected c_hat = 15.984: pump (k0 = +0.162949, w0 = 2.79), seed (k0 = -0.086080, w0 = 1.700942), EPW at k*lambda_De = 0.249. Matching condition and conventions documented in the config. Audit finding F3. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- config.md gains a 'Normalization convention' section: v0 = sqrt(T0/m_e) (sigma convention), L0 = lambda_De, k in 1/lambda_De, Maxwellian exp(-v^2/2) at T=1, Bohm-Gross w^2 = 1 + 3k^2. - Species v0/T0 documented as code-units-only (they bypass normalize()); drift and thermal width now share the same unit. - FP/Krook 'baseline' documented as a rate in units of wp0 (no 2pi), with the newly logged nuee_norm as the reference scale, and the O(1) caveat between the Dougherty nu and the NRL nu_ee. - Super-Gaussian alpha documented (code comment + config.md): it fixes <v^4>/<v^2> = 3 T0/mass for all m; the variance equals T0/mass only at m=2 (x1.24 at m=3, x1.37 at m=4). Documentation only, per review. Audit findings F8, F9, F10 (nuee_norm logging itself landed with the species_params change in an earlier commit). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The fixtures locked in the sqrt(2)-wrong units (c_light 11.302, v0 2.652e7 m/s, x0 12.14 nm, negative nuee). Regenerated under the corrected normalization and the new config surface: c_light 15.984, v0 1.876e7 m/s, x0 8.584 nm, logLambda_ee +7.22, positive nuee, new nuee_norm, species_params T0, c_light knob removed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Post-mortem of job 56005549 (srs-Ln100-Te4-5x-ions): at the every-3-step field cadence the per-run drainer thread sustained only ~2/3 of the ~16 MB/s dump rate, so the /dev/shm staging backlog grew ~2.2 GiB/min, filled the node's 256 GiB of RAM in ~1.8 h, and every later OSIRIS HDF5 write failed on ENOSPC (truncated 2 KB dumps, 2 OOM kills, 26 h TIMEOUT at 16% of tmax). Details in osiris-lpi NOTES.md + dev_docs/ stream-drainer-recommendations.md. Throughput (the drainer needed ~1.5x; this buys ~20x): - StreamWriter batches appends and lands them chunk-aligned: one resize_dimension + one slice write per batch instead of a gzip read-modify-write of the whole ~1 MiB chunk per row (~40x write amplification). Measured 23x on the production field shape (1765 vs 76 appends/s; the job needed ~55/s per sim). - Default compression gzip 4 -> 1 (the drainer is compression-bound; offline postproc can recompress). - Discovery is cached: the full MS/ rglob walk, whose cost grows with the backlog, runs every rediscover_every polls instead of every poll. - No thread pool on purpose: h5py serializes all HDF5 calls behind a process-global lock, so batching is where the throughput is. Safety: - Backlog spill valve (staging mode): past spill_backlog_files/_bytes, or under floor_free_bytes on the staging fs, a diagnostic flips to mirror-only draining (plain copy to persist MS/, ~10x cheaper), so the ramdisk keeps draining no matter what; the NetCDF is caught up from the mirror at finalize (mirror consumed + pruned in discard_grid_h5 mode). OSIRIS must never see ENOSPC on a dump. - Corrupt dumps are quarantined (*.h5.bad) and the stream continues; previously one bad dump dropped the writer and the watcher re-hit the same file every poll, forever. Bookkeeping is now iteration-based (recovered from the iter coordinate on resume), not positional, so skipped dumps cannot shift slots. Observability: - One stats line per stats_every_s (streamed/spilled/backlog/ quarantined + staging free space) so a smoke can assert the steady-state backlog is flat while OSIRIS runs; checking the ramdisk after the job proves nothing. - Repeated identical errors log once per error_log_every occurrences. API and on-disk schema unchanged (runner.py untouched); all 17 existing stream tests pass unmodified, plus new coverage for quarantine and the spill valve in both staging modes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nches The dispersion overlays on the (k, omega) plots assumed the waves live at the simulation reference density (omega_p = 1). SRS decks are normalized to critical density with the plasma near quarter-critical, so the Bohm-Gross and EM branches were drawn a factor of 2 too high in omega. - plot_omega_k / plot_omega_k_figure / save_canned_plots take omega_p (= sqrt(n/n_ref)); the Langmuir label shows the value when != 1. - New BAM overlay (show_bam): shaded beam-acoustic band omega = k*v_phi, v_phi in bam_vph (default 2.8-5 v_th, the BAM root phase-velocity range of Yin et al., PRE 73, 025401 (2006)). Driven SRS moves the ES daughter wave below the Langmuir branch into this band. - Manifest wiring via canned_plot_kwargs: output.overlay_density (0.25 -> omega_p = 0.5) and output.bam (bool). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
output.ion_acoustic: true draws omega = +/- k c_s for every ion species
in the run's rendered deck (os-stdin in the run dir or its parent, so the
regen/binary layout works too). Per-species c_s in code units comes from
c_s^2 = (Z T_e + 3 T_i)/m_i c^2 = v_th_e^2 / rqm + 3 u_th_i^2
(Z cancels through rqm = m_i/Z), walking the deck's ordered species/udist
pairs; the electron temperature is the first rqm<0 species' uth. Ion-free
or deck-less runs draw nothing, so the option is safe to leave on.
Deck keys keep their slice suffixes (uth(1:3)), hence the base-name
lookup helper.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
# Conflicts: # adept/_base_.py # adept/_vlasov1d/modules.py # adept/_vlasov1d/solvers/pushers/fokker_planck.py # docs/source/solvers/vlasov1d/config.md # tests/test_vlasov1d/test_units_boundary.py
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Should be good to go. Recent changes are mostly plotting / quality of life items -- core wrapper remains largely unchanged |
joglekara
left a comment
joglekara
left a comment
There was a problem hiding this comment.
Went through this — the code itself looks careful (the crash-salvage logic and the reap/mirror discipline in stream.py in particular). The gaps I found are all wiring rather than logic.
The one thing I'd like before merge: wire the tests into CI
tests/test_osiris/ is 115 tests across 9 files and none of them have ever run on CI — .github/workflows/cpu-tests.yaml has no osiris paths-filter and no test-osiris job.
The nice thing is that they don't need OSIRIS. I ran the suite at 5bb8dd8 with OSIRIS_BIN/OSIRIS_BIN_1D explicitly unset:
107 passed, 8 skipped in 32.95s
The 8 skips are the only ones that genuinely can't run headless — 7 in test_io_and_plots.py want a pre-existing run dir (OSIRIS_TWOSTREAM_RUN) and exactly 1 wants a built osiris-1D.e. Everything else is pure Python: fake bash binaries for the subprocess driver, synthetic HDF5 for the converter/post/plots, and the four checked-in decks for the parser.
| file | tests | covers |
|---|---|---|
test_stream.py |
20 | concurrent H5→NetCDF converter, reap/mirror/quarantine |
test_deck_roundtrip.py |
18 | parser vs. the 4 real decks |
test_plots_new_views.py |
18 | plotting |
test_post_netcdf.py |
17 | post-processing / consolidation |
test_adaptive_box.py |
13 | density-gradient box scaling |
test_runner.py |
7/8 | subprocess driver, error classification, binary discovery |
test_units.py |
7 | skin-depth normalization |
test_diagnostics_plots.py |
7 | diagnostics |
33 s for that is a good trade. The deck parser and the stream converter are the two I most want guarded — a silent regression in either corrupts a real run's data rather than failing loudly.
Second reason: adding the filter folds osiris into the &shared anchor, so edits to adept/_base_.py and adept/normalization.py start triggering it. This PR touches both, which is exactly the coupling worth catching.
Should be ~15 lines copying the pic1d block.
Two papercuts (happy for these to be follow-ups)
.gitignore ignores directories nothing writes to. It adds osiris_runs/ and scratch/, but the default run_root is ./checkpoints everywhere — runner.py:120, base.py:175, both configs, and the docs. checkpoints/ isn't ignored, so a local run leaves untracked GB-scale h5/nc sitting in the repo.
docs/osiris-adept-usage.md is outside the docs build. It's 348 good lines but it sits outside docs/source/, so Sphinx never picks it up, and there's no docs/source/solvers/osiris/config.md. Every other solver has one and CLAUDE.md asks for it on new config options.
Notes, not asks
h5pyis imported at module scope inadept/osiris/io.pybut isn't inpyproject.toml— it arrives transitively viah5netcdf. Works today, silent break if that backend ever changes.stage_discard_h5(default off) unlinks the scratch HDF5 afterw.append()but before the NetCDF batch flushes, so a crash inside that window loses those slices from both copies. It's documented as lossy-by-design — just flagging that it's not one to enable for a run that's expensive to repeat.write_units()returns{}for the shipped two-stream deck, since that deck sets neithern0noromega_p0, sounits.yamlcomes out empty for the example config. Populates fine on the LPI decks (13 keys). Working as documented, just mildly surprising.
Things I checked that are clean
- Deck parser round-trips all four real decks exactly at dict level, and no unquoted token gets silently promoted to a quoted string (that was my main worry — it would change deck semantics while still passing the round-trip assertion).
subprocess.Popentakes a list, noshell=True, env merge is explicit.- Every deletion path in
stream.pyearly-returns whenpersist_dir is None, so the default non-staged path deletes nothing; the mirror doescopyfile→os.replace. - The
_base_.pychange is a genuine no-op for the other solvers —ADEPTModule.__init__only doesself.cfg = cfg, sooriginal_cfgequals what was being dumped before.
Add an osiris paths filter (adept/osiris/** + tests/test_osiris/**) and a test-osiris job. The suite needs no OSIRIS binary: the runner smoke test and real-data IO tests skip unless OSIRIS_BIN_1D / a run dir are provided. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rence Move docs/osiris-adept-usage.md to docs/source/solvers/osiris/overview.md inline with the other solvers, and pull the manifest schema, override semantics, binary resolution, and adaptive box sizing into a new config.md reference. Register both pages in the index.rst toctrees, solvers.md, and the top-level doc link lists. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
OSIRIS tests wired in, docs updated/rearranged, and the |
2 participants
Todo: