Skip to content

Fix prod_set div-by-zero on fast CPUs and nice -n -20 EPERM in unprivileged guests#755

Closed
charles-typ wants to merge 1 commit into
facebookresearch:v2-betafrom
charles-typ:export-D111817632-to-v2-beta
Closed

Fix prod_set div-by-zero on fast CPUs and nice -n -20 EPERM in unprivileged guests#755
charles-typ wants to merge 1 commit into
facebookresearch:v2-betafrom
charles-typ:export-D111817632-to-v2-beta

Conversation

@charles-typ

Copy link
Copy Markdown
Contributor

Summary:
Two robustness fixes to the WDL prod_set runner, both hit while running the
suite on a fast Arm (Olympus) host and inside an unprivileged VM guest.

Problem 1 - scoring crash on fast cores. concurrency_concurrent_hash_map_bench
and synchronization_lifo_sem_bench fail with "score: error (float division by
zero)". On fast CPUs some sub-metrics (e.g. single_thread_lifo_trypost/trywait,
CHM empty()/size() ops) complete in 0 ns, below the timer resolution.
compute_score_from_time then does sum_baseline[key] / sum_c[key] and raises
ZeroDivisionError, losing the whole workload's score. A 0 ns sample is not a
real data point, so we skip those keys (the geomean of the remaining
sub-metrics is still valid). If every sub-metric is 0, geomean([]) raises and
the existing main() handler already reports a per-benchmark error without
aborting the set.

Problem 2 - nice -n -20 EPERM. run_prod.sh hard-codes nice -n -20, which needs
root/CAP_SYS_NICE. In an unprivileged environment (e.g. a VM guest) every
workload invocation fails with EPERM. We now probe nice -n -20 once and fall
back to default priority when it is not permitted, so the run proceeds instead
of failing.

Differential Revision: D111817632

…ileged guests

Summary:
Two robustness fixes to the WDL prod_set runner, both hit while running the
suite on a fast Arm (Olympus) host and inside an unprivileged VM guest.

Problem 1 - scoring crash on fast cores. concurrency_concurrent_hash_map_bench
and synchronization_lifo_sem_bench fail with "score: error (float division by
zero)". On fast CPUs some sub-metrics (e.g. single_thread_lifo_trypost/trywait,
CHM empty()/size() ops) complete in 0 ns, below the timer resolution.
compute_score_from_time then does sum_baseline[key] / sum_c[key] and raises
ZeroDivisionError, losing the whole workload's score. A 0 ns sample is not a
real data point, so we skip those keys (the geomean of the remaining
sub-metrics is still valid). If every sub-metric is 0, geomean([]) raises and
the existing main() handler already reports a per-benchmark error without
aborting the set.

Problem 2 - nice -n -20 EPERM. run_prod.sh hard-codes nice -n -20, which needs
root/CAP_SYS_NICE. In an unprivileged environment (e.g. a VM guest) every
workload invocation fails with EPERM. We now probe nice -n -20 once and fall
back to default priority when it is not permitted, so the run proceeds instead
of failing.

Differential Revision: D111817632
@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 13, 2026
@meta-codesync

meta-codesync Bot commented Jul 13, 2026

Copy link
Copy Markdown

@charles-typ has exported this pull request. If you are a Meta employee, you can view the originating Diff in D111817632.

meta-codesync Bot pushed a commit that referenced this pull request Jul 14, 2026
…ileged guests (#755)

Summary:
Pull Request resolved: #755

Two robustness fixes to the WDL prod_set runner, both hit while running the
suite on a fast Arm (Olympus) host and inside an unprivileged VM guest.

Problem 1 - scoring crash on fast cores. concurrency_concurrent_hash_map_bench
and synchronization_lifo_sem_bench fail with "score: error (float division by
zero)". On fast CPUs some sub-metrics (e.g. single_thread_lifo_trypost/trywait,
CHM empty()/size() ops) complete in 0 ns, below the timer resolution.
compute_score_from_time then does sum_baseline[key] / sum_c[key] and raises
ZeroDivisionError, losing the whole workload's score. A 0 ns sample is not a
real data point, so we skip those keys (the geomean of the remaining
sub-metrics is still valid). If every sub-metric is 0, geomean([]) raises and
the existing main() handler already reports a per-benchmark error without
aborting the set.

Problem 2 - nice -n -20 EPERM. run_prod.sh hard-codes nice -n -20, which needs
root/CAP_SYS_NICE. In an unprivileged environment (e.g. a VM guest) every
workload invocation fails with EPERM. We now probe nice -n -20 once and fall
back to default priority when it is not permitted, so the run proceeds instead
of failing.

Reviewed By: YifanYuan3, ahmadelyoussef

Differential Revision: D111817632

fbshipit-source-id: 6e132cf9f2e2a598e4a801e07711881718785bde
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant