Make precompiled GSPs reproducible without disabling runtime reloading

grails-doc/src/en/guide/theWebLayer/gsp/makingChangesToADeployedApplication.adoc

@@ -46,8 +46,10 @@ There are also some system properties to control GSP reloading:
 |===
 |Name|Description|Default
 |grails.gsp.enable.reload|system property for enabling the GSP reload mode (alternative to adding it in the file-based application configuration|
-|grails.gsp.reload.interval|interval between checking the lastmodified time of the gsp source file, unit is milliseconds|5000
-|grails.gsp.reload.granularity|the number of milliseconds leeway to give before deciding a file is out of date. this is needed because different roundings usually cause a 1000ms difference in lastmodified times|1000
+|grails.gsp.reload.interval|interval between checks of the gsp source file, unit is milliseconds|5000
+|grails.gsp.reload.granularity|the number of milliseconds leeway to give before deciding a file is out of date. this is needed because different roundings usually cause a 1000ms difference in lastmodified times. Applies only to pages compared by modification time — see below|2000
 |===
 
 GSP reloading is supported for precompiled GSPs since Grails 1.3.5.
+
+A precompiled GSP records a checksum of the page it was compiled from, and is reloaded when the source no longer matches that checksum. Comparing content rather than modification times means a page that was copied or checked out afresh — and so carries a new modification time but the same content — is not needlessly recompiled, and an edit is detected however close together two writes fall. Pages compiled at runtime are compared the same way. A page precompiled by a version of Grails earlier than 8.0.0 carries no checksum, and is still compared by modification time.

grails-doc/src/en/guide/upgrading/upgrading80x.adoc

@@ -2354,3 +2354,41 @@ resolved unambiguously before can become ambiguous.
 
 Removing the packages from `grails.spring.bean.packages`, or the stray classes from those packages, restores
 the previous set of beans.
+
+
+==== 45. Precompiled GSPs Record a Checksum Instead of a Modification Time
+
+A precompiled GSP used to bake the modification time of its `.gsp` source into the generated class, as a
+`LAST_MODIFIED` constant. Git stores no modification times, so every fresh clone or CI checkout gave each
+source a new one, and the generated classes differed on every checkout even when the sources were
+byte-for-byte identical. Because `LAST_MODIFIED` was a compile-time constant it was part of the class's ABI,
+so the difference survived even Gradle's compile-classpath normalization and every task downstream of a jar
+containing precompiled GSPs missed the build cache.
+
+Pages compiled by Grails 8.0.0 and later record a `SOURCE_CHECKSUM` of the page source instead, and
+`LAST_MODIFIED` is emitted as `0`. Identical sources now compile to identical bytes on every machine.
+
+Runtime reloading, described in
+<<makingChangesToADeployedApplication,Making Changes to a Deployed Application>>, is unaffected: a
+precompiled page is reloaded when its source no longer matches the recorded checksum. Detection is now more
+accurate than it was, because a page that was merely copied or checked out afresh is no longer treated as
+changed, and an edit is caught however close together two writes fall. Pages precompiled by an earlier
+version carry no checksum and continue to be compared by modification time.
+
+Pages compiled at runtime record a checksum as well, so development-mode reloading changes in two ways.
+An edit is now picked up whenever the source's modification time or length moves, which any ordinary save
+does; previously a save landing within `grails.gsp.reload.granularity` milliseconds of the recorded time went
+unnoticed until a later edit moved it clear of that window. In the other direction, a page whose modification
+time moves without its content changing is no longer recompiled, so touching a file — or a build step copying
+views into place — no longer reloads pages that did not actually change. If you were using `touch` to force a
+page to recompile, edit the page instead.
+
+One public API changes behaviour: `GroovyPageMetaInfo.getLastModified()` returns `0` for pages precompiled
+by Grails 8.0.0 or later, since no modification time is recorded for them. Use `getSourceChecksum()` to
+identify the source a page was compiled from.
+
+One cross-version note: `LAST_MODIFIED = 0` only means "nothing recorded" to a runtime that understands the
+checksum. An application still running an earlier Grails that consumes a plugin precompiled by 8.0.0 or later
+evaluates the old condition instead, so with GSP reloading enabled every such page is declared stale on its
+first check and recompiled from source at runtime. The behaviour is reload-only and self-correcting, but it
+is worth knowing before mixing versions.

grails-gsp/core/src/main/groovy/org/grails/gsp/GroovyPageMetaInfo.java

@@ -76,6 +76,8 @@ public class GroovyPageMetaInfo implements GrailsApplicationAware {
     private Class<?> pageClass;
     private Constructor<?> pageClassConstructor;
     private long lastModified;
+    private String sourceChecksum;
+    private volatile SourceStamp checksumStamp;
     private InputStream groovySource;
     private String contentType;
     private int[] lineNumbers;
@@ -128,7 +130,15 @@ public GroovyPageMetaInfo(Class<?> pageClass) {
         }
         contentType = (String) ReflectionUtils.getField(ReflectionUtils.findField(pageClass, GroovyPageParser.CONSTANT_NAME_CONTENT_TYPE), null);
         jspTags = (Map) ReflectionUtils.getField(ReflectionUtils.findField(pageClass, GroovyPageParser.CONSTANT_NAME_JSP_TAGS), null);
+        // Read unguarded, unlike SOURCE_CHECKSUM below: LAST_MODIFIED has been emitted by every version of the
+        // compiler, so a page class without it cannot exist. That makes the constant permanent ABI which must
+        // keep being emitted even though GroovyPageCompiler now always writes 0 -- pages precompiled by earlier
+        // versions still carry a real timestamp and are still compared by it.
         lastModified = (Long) ReflectionUtils.getField(ReflectionUtils.findField(pageClass, GroovyPageParser.CONSTANT_NAME_LAST_MODIFIED), null);
+        Field sourceChecksumField = ReflectionUtils.findField(pageClass, GroovyPageParser.CONSTANT_NAME_SOURCE_CHECKSUM);
+        if (sourceChecksumField != null) {
+            sourceChecksum = (String) ReflectionUtils.getField(sourceChecksumField, null);
+        }
         expressionCodecName = (String) ReflectionUtils.getField(ReflectionUtils.findField(pageClass, GroovyPageParser.CONSTANT_NAME_EXPRESSION_CODEC), null);
         staticCodecName = (String) ReflectionUtils.getField(ReflectionUtils.findField(pageClass, GroovyPageParser.CONSTANT_NAME_STATIC_CODEC), null);
         outCodecName = (String) ReflectionUtils.getField(ReflectionUtils.findField(pageClass, GroovyPageParser.CONSTANT_NAME_OUT_CODEC), null);
@@ -315,6 +325,12 @@ public void setPageClass(Class<?> pageClass) {
         initializePluginPath();
     }
 
+    /**
+     * @return the modification time of the source this page was compiled from, or {@code 0} if none was
+     * recorded. Pages precompiled by Grails 8.0.0 or later always report {@code 0}: the compiler records a
+     * checksum of the source instead, so that identical sources compile to identical bytes. Use
+     * {@link #getSourceChecksum()} to identify the source such a page was compiled from.
+     */
     public long getLastModified() {
         return lastModified;
     }
@@ -323,6 +339,22 @@ public void setLastModified(long lastModified) {
         this.lastModified = lastModified;
     }
 
+    /**
+     * @return the checksum of the GSP source this page was compiled from, or {@code null} if none was recorded
+     * @since 8.0.0
+     */
+    public String getSourceChecksum() {
+        return this.sourceChecksum;
+    }
+
+    /**
+     * @param sourceChecksum the checksum of the GSP source this page was compiled from
+     * @since 8.0.0
+     */
+    public void setSourceChecksum(String sourceChecksum) {
+        this.sourceChecksum = sourceChecksum;
+    }
+
     public InputStream getGroovySource() {
         return groovySource;
     }
@@ -404,6 +436,87 @@ public void applyLastModifiedFromResource(Resource resource) {
         this.lastModified = establishLastModified(resource);
     }
 
+    /**
+     * The modification time and length a page's source had when it last matched the recorded checksum.
+     */
+    private record SourceStamp(long lastModified, long contentLength) {
+    }
+
+    /**
+     * Decides whether the given source still hashes to the checksum this page recorded.
+     * <p>
+     * Hashing means reading the page in full, so the modification time and length observed the last time the
+     * two matched are kept, and the read is skipped while neither has moved. That returns the steady-state
+     * cost of a reload-enabled application to one stat per page per check interval, which is what the
+     * timestamp comparison used to cost.
+     * <p>
+     * Note what role the timestamp plays here: it is a fast path for skipping work, never the thing that
+     * decides staleness. Anything that moves it without changing the page -- a fresh checkout, a copy, a
+     * touch -- costs one hash and then correctly reports no change, where the old comparison reported the
+     * page stale. The one edit this misses is an edit preserving both the modification time and the exact
+     * length, still a narrower gap than the {@code grails.gsp.reload.granularity} window it replaces.
+     *
+     * @param resource the source to compare against the recorded checksum
+     * @return true if the source no longer matches
+     */
+    private boolean hasSourceChecksumChanged(Resource resource) {
+        SourceStamp stamp = readSourceStamp(resource);
+        if (stamp != null && stamp.equals(this.checksumStamp)) {
+            return false;
+        }
+        String currentChecksum = establishChecksum(resource);
+        if (currentChecksum == null) {
+            return false;
+        }
+        if (this.sourceChecksum.equals(currentChecksum)) {
+            this.checksumStamp = stamp;
+            return false;
+        }
+        return true;
+    }
+
+    /**
+     * @param resource the Resource to stamp
+     * @return its modification time and length, or null if either could not be read -- in which case the
+     * caller must hash rather than assume the source is unchanged
+     */
+    private SourceStamp readSourceStamp(Resource resource) {
+        long modified = establishLastModified(resource);
+        if (modified <= 0) {
+            return null;
+        }
+        try {
+            long length = resource.contentLength();
+            return length >= 0 ? new SourceStamp(modified, length) : null;
+        }
+        catch (IOException e) {
+            return null;
+        }
+    }
+
+    /**
+     * Attempts to checksum the given resource. If it cannot be read, {@code null} is returned, which is
+     * treated the same way an unobtainable modification time is: the page is left alone rather than
+     * reloaded on the strength of a failed read.
+     *
+     * @param resource the Resource to digest
+     * @return the checksum, or null if it could not be established
+     */
+    private String establishChecksum(Resource resource) {
+        if (resource == null) {
+            return null;
+        }
+        try {
+            return GroovyPageParser.checksumOf(resource.getContentAsByteArray());
+        }
+        catch (IOException e) {
+            if (LOG.isDebugEnabled()) {
+                LOG.debug("Unable to checksum GSP source [" + resource + "], leaving the compiled page in place", e);
+            }
+            return null;
+        }
+    }
+
     /**
      * Attempts to establish what the last modified date of the given resource is. If the last modified date cannot
      * be etablished -1 is returned
@@ -474,10 +587,27 @@ public Resource checkIfReloadableResourceHasChanged(final PrivilegedAction<Resou
             public Resource call() {
                 Resource resource = resourceCallable.run();
                 if (resource != null && resource.exists()) {
+                    // A page compiled by GroovyPageCompiler records a checksum of its source rather than the
+                    // source's modification time, which git does not preserve across a checkout. Comparing
+                    // content answers the question directly: a page that was merely touched is not stale, and
+                    // an edit is caught however close together the writes fall.
+                    if (sourceChecksum != null) {
+                        return hasSourceChecksumChanged(resource) ? resource : null;
+                    }
+                    // Pages precompiled before SOURCE_CHECKSUM existed carry a real timestamp and no checksum,
+                    // so they still land here. Only 0 means "nothing recorded": that is the value GroovyPageCompiler
+                    // deliberately emits, so comparing against it would report every page stale forever.
+                    //
+                    // -1 is a different thing and must keep its old meaning. establishLastModified returns it
+                    // when a resource's timestamp could not be read at all, and applyLastModifiedFromResource
+                    // stores it as-is. Such a page self-heals: the first check able to read a real mtime sees a
+                    // difference, reloads once, and records a valid timestamp. Excluding -1 here would strand it
+                    // until restart, silently.
                     long currentLastmodified = establishLastModified(resource);
                     // granularity is required since lastmodified information is rounded somewhere in copying & war (zip) file information
                     // usually the lastmodified time is 1000L apart in files and in files extracted from the zip (war) file
-                    if (currentLastmodified > 0 && Math.abs(currentLastmodified - lastModified) > LASTMODIFIED_CHECK_GRANULARITY) {
+                    if (currentLastmodified > 0 && lastModified != 0 &&
+                            Math.abs(currentLastmodified - lastModified) > LASTMODIFIED_CHECK_GRANULARITY) {
                         return resource;
                     }
                 }

grails-gsp/core/src/main/groovy/org/grails/gsp/GroovyPagesTemplateEngine.java

@@ -64,7 +64,6 @@
 import grails.config.Settings;
 import grails.core.GrailsApplication;
 import grails.core.GrailsClass;
-import grails.io.IOUtils;
 import grails.util.CacheEntry;
 import grails.util.Environment;
 import grails.util.GrailsUtil;
@@ -583,8 +582,13 @@ protected GroovyPageMetaInfo buildPageMetaInfo(InputStream inputStream, Resource
 
         GroovyPageParser parser;
         String path = getPathForResource(res);
+        // Buffer the raw bytes rather than decoding straight off the stream, so the page can be checksummed
+        // exactly as it is stored. The checksum has to be taken over the stored bytes, not over gspSource or
+        // the decorated source below, because the runtime re-reads the resource raw when checking staleness.
+        byte[] gspBytes;
         try {
-            String gspSource = IOUtils.toString(inputStream, getGspEncoding());
+            gspBytes = inputStream.readAllBytes();
+            String gspSource = new String(gspBytes, getGspEncoding());
             parser = new GroovyPageParser(name, path, path, decorateGroovyPageSource(new StringBuilder(gspSource)).toString(),
                     grailsApplication != null ? grailsApplication.getConfig() : null);
         }
@@ -597,6 +601,11 @@ protected GroovyPageMetaInfo buildPageMetaInfo(InputStream inputStream, Resource
         // Make a new metaInfo
         GroovyPageMetaInfo metaInfo = createPageMetaInfo(parser, in);
         metaInfo.applyLastModifiedFromResource(res);
+        // Record a checksum here as well as a timestamp, so a page compiled at runtime is checked for staleness
+        // the same way a precompiled one is. It costs nothing extra -- the bytes are already in hand -- and it
+        // catches the edits the timestamp comparison misses, which in development is the case that matters:
+        // two saves inside the grails.gsp.reload.granularity window.
+        metaInfo.setSourceChecksum(GroovyPageParser.checksumOf(gspBytes));
         try {
             metaInfo.setPageClass(compileGroovyPage(in, name, path, metaInfo));
             metaInfo.setHtmlParts(parser.getHtmlPartsArray());