Quit omc on every exit path and check for leftovers - #358
Merged
Conversation
Since f0c8470 testmodel.py has a daemon thread sampling memory, and on Python 3.14 a live daemon thread means no __del__ runs at sys.exit. The main omc session was never quit explicitly and relied on OMPython's __del__, so from 2026-09-08 every test left its omc alive: ~19200 OOM kills per run and ~6000 models killed while loading, unchanged by c7a7b4d, which fixed the timeout path only. quit_omc now waits for the session's processes and kills what does not exit; writeResultAndExit always quits both sessions and whatever is left below the script; an excepthook writes the result first. OMPython receives with no timeout, so an omc killed mid-command left the main thread in a C-level recv where no signal handler runs: 194 models were charged the full outer timeout with no phase data. guardSession replaces sendExpression with one that polls and notices omc dying within a second. test.py marks each test with OMLT_JOB=<name> in the environment and scans /proc/*/environ afterwards (3 ms), killing and logging anything that survived. The Jenkins exit trap prints memory.stat and the largest processes. Assisted-by: Claude Fable 5.1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since f0c8470 testmodel.py has a daemon thread sampling memory, and on Python 3.14 a live daemon thread means no del runs at sys.exit. The main omc session was never quit explicitly and relied on OMPython's del, so from 2026-09-08 every test left its omc alive: ~19200 OOM kills per run and ~6000 models killed while loading, unchanged by c7a7b4d, which fixed the timeout path only.
quit_omc now waits for the session's processes and kills what does not exit; writeResultAndExit always quits both sessions and whatever is left below the script; an excepthook writes the result first.
OMPython receives with no timeout, so an omc killed mid-command left the main thread in a C-level recv where no signal handler runs: 194 models were charged the full outer timeout with no phase data. guardSession replaces sendExpression with one that polls and notices omc dying within a second.
test.py marks each test with OMLT_JOB= in the environment and scans /proc/*/environ afterwards (3 ms), killing and logging anything that survived. The Jenkins exit trap prints memory.stat and the largest processes.
Assisted-by: Claude Fable 5.1