Skip to content

Add relative tolerance to the rtp limit comparison - #114

Open
soulrap wants to merge 1 commit into
engineio:mainfrom
soulrap:rtp-limit-tolerance
Open

Add relative tolerance to the rtp limit comparison#114
soulrap wants to merge 1 commit into
engineio:mainfrom
soulrap:rtp-limit-tolerance

Conversation

@soulrap

@soulrap soulrap commented Aug 27, 2026

Copy link
Copy Markdown

Problem

verify_mode_volatility compares each statistic against its limit with a strict
val > limit. The optimizer targets the rtp limit exactly, so its result can
land a few ulps on either side of it. When it lands above, the run is reported as
failing a limit it actually hit.

Two of the sample games reproduce this on a clean full run
(run_sims + run_optimization + run_analysis + run_format_checks, 1e4 sims per mode):

VIOLATED: rtp  VALUE:0.967 --- LIMIT: 0.967

The printed value is round(val, 4), which hides the cause. The exact values,
recomputed from the published lookup tables:

game mode rtp result
0_0_expwilds superspin 0.96700000000123722188 flagged (+1.2e-12)
0_0_scatter bonus 0.96700000000008785861 flagged (+8.8e-14)
0_0_expwilds base 0.96699999947532203403 passed
0_0_scatter base 0.96699999991445092640 passed

Which side of the limit a mode lands on is incidental to the optimizer's weight
search, so the failure is not reproducible in any meaningful sense.

Change

Compare against the limit with a relative tolerance, applied to rtp only:

limit_rel_tol = {"rtp": 1e-9}

for key, limit in mode_limits.items():
    val = getattr(MathStats, key, None)
    if val is None or val <= limit:
        continue
    if isclose(val, limit, rel_tol=limit_rel_tol.get(key, 0.0)):
        continue
    violated_attributes[key] = val

Three decisions behind it:

  1. rtp only. Limits absent from limit_rel_tol get rel_tol=0.0, and since
    val > limit already excludes equality, that is exactly the previous strict
    behaviour. Only rtp is a value the optimizer deliberately drives onto the
    boundary, so only rtp has a reason to be loosened.
  2. Relative, not absolute. 1e-9 relative is ~9.7e-10 absolute at 0.967 and
    rescales on its own if the limit is ever retargeted (0.97, 0.95, ...).
  3. 1e-9 in size. The largest residue observed was 1.2e-12, leaving ~800x
    headroom, while the smallest meaningful rtp regression (0.9671) sits five
    orders of magnitude away. The tolerance cannot mask a real deviation.

Verification

Six cases through verify_mode_volatility:

scenario rtp expected result
0_0_expwilds/superspin actual 0.96700000000123720 pass pass
0_0_scatter/bonus actual 0.96700000000008790 pass pass
exactly at limit 0.967 pass pass
below limit 0.96699999947532200 pass pass
real violation 0.9671 VIOLATED VIOLATED
unoptimized run 17.7082 VIOLATED VIOLATED

Other metrics keep the strict comparison: cvar = 800.0000000001 against a limit
of 800 is still reported as violated.

execute_all_tests was then re-run against the existing publish files for both
affected games. All five modes report SHA-256 OK, payout hash OK with no
VIOLATED line.

All six sample games were run end to end with optimization enabled (13 modes
total, every one landing on rtp 0.967) to confirm nothing else regressed.

The optimizer targets the rtp limit exactly, so its result can land a few
ulps above it - 0_0_scatter/bonus and 0_0_expwilds/superspin both reported
VIOLATED at 0.967 + 1e-12. Compare with a relative tolerance for rtp only;
the other limits keep the strict comparison.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant