Skip to content

Restore build-cache reuse for plugins with precompiled GSPs - #16132

Draft
maczikasz wants to merge 1 commit into
apache:7.0.xfrom
maczikasz:fix/gsp-precompiled-page-reproducibility
Draft

Restore build-cache reuse for plugins with precompiled GSPs#16132
maczikasz wants to merge 1 commit into
apache:7.0.xfrom
maczikasz:fix/gsp-precompiled-page-reproducibility

Conversation

@maczikasz

Copy link
Copy Markdown

Description

Fixes #16131.

Grails plugins that ship precompiled GSPs currently produce a different jar on every checkout,
even when the source is byte-for-byte identical
. Nothing in the page changed — only the file's
timestamp on disk, which every fresh git clone or CI checkout resets.

Gradle's build cache decides whether it can reuse earlier output by hashing a task's inputs. A jar
whose bytes change on every checkout invalidates that hash for every task downstream of it, so the
cache can never serve those tasks and they re-run from scratch. Contributors rebuild downstream
modules on every fresh clone; CI never benefits from an earlier run. It is invisible in the way
that matters — the build succeeds and the output is correct, it is only slower.

Over 2026-07-29 → 2026-08-05 on develocity.apache.org, roughly 616 hours of avoidable CI task
re-execution
in this project trace to jars carrying precompiled GSPs, the largest single
contributor being :grails-test-examples-spring-security-ui-simple:assetCompile. Those are summed
task re-executions rather than wall-clock, since much of the work runs in parallel.

Root cause

GroovyPageCompiler bakes the .gsp source file's modification time into every generated page
class:

gpp.lastModified = gspfile.lastModified()   // GroovyPageCompiler.groovy

GroovyPageParser emits that value as a static final long LAST_MODIFIED constant, making it part
of the compiled class's ABI. Being ABI-level is what makes this expensive rather than cosmetic:
Gradle's COMPILE_CLASSPATH normalization hashes only the public API and ignores everything else,
but a changed constant survives even that, because constants are inlined into callers.

Archive reproducibility is not the gap — the jars already use normalized entry timestamps
(1980-02-01). The divergence is inside the class bytes.

The reload path, and why this is two changes rather than one

LAST_MODIFIED is not unused at runtime. GroovyPageMetaInfo.checkIfReloadableResourceHasChanged
reads the field directly and compares it against the live source timestamp to decide whether a
precompiled page is stale. Fixing the emitted value alone would make that comparison always report
a change, so this PR also guards it:

if (currentLastmodified > 0 && lastModified > 0 &&
        Math.abs(currentLastmodified - lastModified) > LASTMODIFIED_CHECK_GRANULARITY) {

0 now means "no source timestamp recorded", so staleness detection is skipped rather than firing
on every check. Behaviour for pages carrying a real timestamp is unchanged.

Blast radius, for reviewers:

  • For GSPs in binary plugin jars — the case this PR is about — the reload path was already
    unreachable: DefaultGroovyPageLocator.resolveViewInBinaryPlugin nulls the resource callable, and
    the plugin jars ship no .gsp sources to resolve.
  • The guard covers the remaining case: an application's own precompiled pages with reloading
    enabled.
  • getLastModified() is public API and will now return 0 for reproducibly compiled pages. Nothing
    in this repository calls it. Please tell me if you would like an Upgrade Note for that.

The LAST_MODIFIED field is deliberately retained rather than removed, because GroovyPageMetaInfo
resolves it reflectively via findField.

Evidence

Two CI builds of this project at the same commit (01037bdf), compared on the ASF Develocity
instance — publicly readable, no login required:

Task-input comparison: kat5373gl55fw vs lenhcqeavcokm

For :grails-fields:jar and :grails-spring-security:jar, compileGroovy, compileJava and
processResources are identical. The only diverging input is build/gsp-classes/main.

Reproduced locally by changing only the source mtimes, with file content unchanged:

mtimes resulting class bytes
Before 3 distinct 3 distinct sets
After 3 distinct byte-identical

A sibling closure class emitted by the same task in the same run
(gsp_..._table_gsp$_run_closure1.class), which carries no LAST_MODIFIED constant, was
byte-identical in every run both before and after. That rules out general Groovy compiler
nondeterminism and isolates the timestamp as the sole cause.

Contributor Checklist

Issue and Scope

Code Quality

  • Added tests: GroovyPageMetaInfoReloadSpec covers the reload semantics in both directions.
    Reverting the guard makes exactly the "no recorded timestamp" case fail and leaves the others
    passing, so it is a genuine regression guard.
  • ./gradlew build --rerun-tasksnot run in full. I ran
    ./gradlew clean aggregateViolations :grails-test-report:check --continue, which completed
    with 9 failures, all integrationTest tasks failing on
    ContainerLaunchException: Container startup failed for image selenium/standalone-chrome.
    That image publishes no linux/arm64 manifest and cannot start on Apple Silicon; the failures
    are environmental and unrelated to this change. All unit tests pass — see below.
  • Code style: checkstyleMain, checkstyleTest, codenarcMain, codenarcTest clean for
    grails-gsp-core, and the aggregate CHECKSTYLE, CODENARC, PMD and SPOTBUGS violation
    reports all report "No violations found".
  • No mass reformatting, style-only changes, or large-scale refactoring.
  • Generative AI tooling was used; see Attribution below.

Licensing and Attribution

  • Contributed under the Apache License 2.0; the new source file carries the Apache license
    header.
  • I have the necessary rights to submit this contribution.
  • Generative AI tooling was used in preparing this contribution, following the
    ASF policy on generative tooling.
    The cache miss and the diverging input were located by Gradle's
    Build Caching Optimizer, an agent
    that analyses Develocity build data; it is still in development and is being trialled against
    open-source builds. The root cause, the fix and the verification above were established
    against the sources and the build scans linked here.

Documentation

  • No user-facing documentation change included — this is an internal compilation and caching
    fix with no API addition. Tell me if you would prefer one.
  • Not a new feature, so no What's New entry.
  • No Upgrade Note included. The one candidate is getLastModified() returning 0 for
    precompiled pages; I did not judge that user-visible enough to warrant one, but I will add it
    if you disagree.
  • The description explains what was changed and why.

GroovyPageCompiler baked the .gsp source file's modification time into every
generated page class. GroovyPageParser emits that value as a
`static final long LAST_MODIFIED` constant, so it forms part of the compiled
class's ABI. A fresh checkout gives every .gsp a new modification time, so
identical sources compiled on two machines produce different bytes.

Because the divergence is ABI-level it is not hidden by classpath
normalization: consumers re-key even under COMPILE_CLASSPATH, which otherwise
ignores everything but the ABI. Any jar bundling precompiled GSPs therefore
invalidates the build cache for every downstream task on every fresh checkout.
Archive reproducibility was not the gap -- the jars already use normalized
entry timestamps.

Emit a fixed LAST_MODIFIED so precompilation is reproducible. Verified by
compiling the same sources with only the mtime changed:

  before: 3 distinct mtimes -> 3 distinct sets of class bytes
  after:  3 distinct mtimes -> byte-identical output

A sibling closure class from the same task, carrying no LAST_MODIFIED
constant, was byte-identical in every run both before and after, confirming
the timestamp is the sole source of divergence.

The value is read at runtime, so fixing it is not sufficient on its own.
GroovyPageMetaInfo.checkIfReloadableResourceHasChanged compares the field
against the live source timestamp to decide whether a precompiled page is
stale; a fixed value would make that comparison always report a change. Guard
it so that a lastModified of 0 means "no source timestamp recorded" and
staleness detection is skipped rather than firing on every check. Behaviour
for pages carrying a real timestamp is unchanged.

For GSPs in binary plugin jars the reload path was already unreachable --
DefaultGroovyPageLocator.resolveViewInBinaryPlugin nulls the resource
callable, and those jars ship no .gsp sources. The guard covers an
application's own precompiled pages with reloading enabled.

The LAST_MODIFIED field is retained rather than removed because
GroovyPageMetaInfo resolves it reflectively via findField.

Fixes apache#16131
@testlens-app

testlens-app Bot commented Aug 11, 2026

Copy link
Copy Markdown

✅ All tests passed ✅

🏷️ Commit: 5079430
▶️ Tests: 48887 executed
⚪️ Checks: 51/51 completed


Learn more about TestLens at testlens.app.

@codeconsole

Copy link
Copy Markdown
Contributor

Thanks @maczikasz for looking into this! I expanded on your solution a bit here #16142

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants