Cache anonymous public traffic safely at CloudFront

[alexeygrigorev]

Outcome

Enable positive-TTL CloudFront caching only for explicitly classified anonymous public GET/HEAD responses, while keeping every authenticated, credential-bearing, personalized, mutation, preview, management, learner, registration, health, and operational request fail-closed and uncacheable.

The same edge boundary forwards a trustworthy optional country suggestion to #108, blocks broad abuse before Django, and gives the owner a predictable cost choice backed by current traffic and exact feature eligibility.

Product, architecture, and deployed-source authority

• 01 — Platform request/content activation flows
• 02 — URL, query, canonical, and preview behavior
• 03 — Atomic content activation
• 06 — Private Studio/admin API responses
• 07 — Web security, rate limits, privacy, observability, and targets
• 08 — CloudFront, protected ALB origin, noindex, Terraform portability, and cost
• 10 — Infrastructure/deployment/security verification
• Current accepted aws-infra module/root from Create the portable sandbox network, DNS, edge, and TLS foundation #67–Historical: Bootstrap web.dtcdev.click (superseded by automatic development delivery) #70.

Current trusted source facts:

• the distribution has one default behavior with min/default/max TTL 0;
• it forwards all viewer headers, cookies, and query strings to Django;
• it has an optional web_acl_id input but the website module does not create the required ACL;
• CloudFront reaches the ALB over HTTPS/TLS 1.2;
• ALB ingress is limited to the inventoried AWS CloudFront origin-facing IPv4/IPv6 prefix lists;
• the ALB forwards only when both the public Host and generated X-Origin-Verify value match;
• development X-Robots-Tag noindex, nofollow is forced at the edge and remains required;
• application private/no-store responses and the Enforce non-indexable development and preview behavior #36 zero-TTL source contract are already tested.

This issue replaces the temporary all-dynamic cache posture; it must update #36 tests rather than weakening noindex or private-response guarantees.

Route classification is a code-owned contract

Create one versioned route-cache registry consumed by Django tests, cache-header middleware, deployed smoke, and Terraform/policy assertions. Every route is private/disabled unless explicitly classified. Adding a route without a classification fails CI.

Route class	Initial examples	Methods	Edge TTL / stale	Browser policy	Cache key
Fingerprinted static	/static/ versioned filenames	GET, HEAD	365 days; no stale error object	public, max-age=31536000, immutable	normalized path + gzip/brotli only; no cookies/query/viewer headers
Stable release asset	code-owned active-release asset paths that are not fingerprinted	GET, HEAD	24 hours; invalidated on activation	public, max-age=3600	path + encoding; no cookies/query/viewer headers
Editorial detail	approved article, podcast, person, book, docs, FAQ, wiki detail routes	GET, HEAD	600 seconds; stale-if-error up to 24 hours	max-age=0, must-revalidate; ETag/Last-Modified	canonical path + encoding; no query
Public hub/feed/sitemap	approved hubs, feeds, sitemap and explicit public JSON feeds	GET, HEAD	300 seconds; stale-if-error up to 1 hour	max-age=0, must-revalidate	canonical path; only exact page when registry allows pagination
Public course/event catalog/detail	anonymous-stable catalog/detail pages only	GET, HEAD	60 seconds; stale-if-error at most 5 minutes	max-age=0, must-revalidate	canonical path; exact allowlisted pagination only
Code-owned permanent redirect	explicit public alias/redirect manifest only	GET, HEAD	24 hours	public, max-age=300	normalized source path; query handled by redirect contract
Public 404	clean, credential-free, query-free unknown GET/HEAD	GET, HEAD	30 seconds; no stale-if-error	max-age=0	normalized path
Search/arbitrary query	search, filters not listed above	GET, HEAD	disabled in MVP	private/no-store	none
Private/dynamic	/accounts/, /studio/, /admin/, /cadmin/, learner/dashboard/enrollment/submission routes, onboarding/Slack, course/event registration forms or management links, preview, exports, authenticated/private API	all	disabled, including error caching	private, no-store	none
Operational	/health/, readiness, metrics, webhook/callback, job/provider endpoints	all	disabled	private, no-store or explicit operational equivalent	none
Unsafe/error	POST/PUT/PATCH/DELETE, 400/401/403/405/409/429/5xx, any response with Set-Cookie, private, no-store, Vary: *, CSRF, identity, PII, or capability state	applicable	disabled	no-store	none

Exact route names and path patterns are generated/reviewed from the actual URL registry; the examples above do not authorize an unlisted route. Public compatibility APIs remain disabled unless their owning contract explicitly proves anonymous, stable, bounded JSON.

All cache policies use min_ttl = 0 so an origin/edge no-store decision wins. Error-cache TTL is zero except the exact public 404 class. CloudFront never caches an origin error merely because an older successful representation exists. Stale-if-error is limited to already cached, anonymous public content and never applies to time-sensitive registration state or any private route.

Anonymous cache gate and fail-closed segmentation

A response is eligible for shared caching only when every condition is true:

• request method is GET or HEAD;
• the route registry class is public-cacheable;
• the viewer classifier proves the request has no Authorization, session/auth/preview/management credential, signed URL/cookie, CSRF-bearing cookie, or unknown/malformed credential-like state;
• Django renders an anonymous-stable representation and explicitly returns public plus the class s-maxage;
• response has no Set-Cookie, private, no-store, Vary: *, CSRF token, account-sensitive navigation/data, PII, or staff/learner state;
• status is allowed by the matrix.

Use a deterministic, versioned viewer-request edge function without a key-value store to remove any viewer-supplied internal classification header and set an internal anonymous-v1 or private marker. Any malformed cookie/header/query encoding, unknown credential-shaped cookie, Authorization syntax, preview/management token, or classifier failure becomes private.

The marker participates in the public-HTML cache key and is forwarded to Django. Explicit private path behaviors have the zero-TTL policy regardless of origin headers. On a mixed public path, credential-bearing requests go to origin and an edge origin-response guard forces private, no-store before cache storage, even if application code regresses. If this cannot be proved with the selected CloudFront function/Lambda@Edge and policy ordering, that route remains zero-TTL; security is not weakened to create a HIT.

Django remains authoritative too: every authenticated request and explicit private route returns private, no-store. Public templates must not place account-sensitive navigation/data in the anonymous cacheable representation. If a route cannot render one anonymous-stable representation without setting a cookie, it stays disabled in MVP.

Query, header, cookie, and poisoning policy

• Fingerprinted/static/detail/feed/sitemap keys contain no query parameter.
• A registry entry may allow one canonical positive-integer page parameter for a named hub. Duplicate, empty, overlong, out-of-range, invalidly encoded, or unexpected parameters make the response no-store or a safe 400; they never create cache variants.
• Known tracking keys may be stripped by a single safe canonical GET/HEAD redirect. They are never reflected into HTML or forwarded while omitted from the key.
• Search text and arbitrary filters are not cached in MVP.
• Host, User-Agent, Referer, Accept-Language, CloudFront country, X-Forwarded-*, viewer-supplied internal headers, and arbitrary cookies are not cache-key inputs.
• Accept-Encoding uses CloudFront gzip/brotli normalization, not raw attacker-controlled variants.
• Origin requests receive only the per-behavior header/cookie/query allowlist. The all-viewer policy is removed from cacheable behaviors and retained only where a private route truly needs that request context.
• Duplicate headers/query keys, alternate Host, path normalization, encoded separators, and poisoning canaries have negative tests.

Trusted country suggestion

Only onboarding/profile routes that need it receive CloudFront-Viewer-Country; those routes remain zero-TTL.

• CloudFront removes a viewer-supplied lookalike and adds its own geolocation header through the origin request policy.
• The existing CloudFront-prefix-list plus host plus generated origin-verification boundary remains intact. No implementation or evidence reads, prints, reconstructs, or returns the generated origin secret.
• Django enables trust only in the configured CloudFront deployment, accepts exactly a known uppercase ISO alpha-2 code, treats CloudFront special/unknown codes as missing, and exposes a request-scoped suggestion rather than a confirmed value.
• Country is not in any public cache key. Raw IP/header values are not retained for Unify member profile onboarding, Slack access, and course registration #108.
• Local/direct-origin tests default to no suggestion. A trusted test seam uses a boolean fixture, never a copied origin secret.

Invalidation and freshness

Network side effects occur after commit through a durable, idempotent invalidation job.

Content activation:

1. Candidate validation computes the changed public paths plus dependent hubs, redirects, search pages, sitemap, feeds, and stable asset paths before activation.
2. The activation transaction stores that route manifest and a unique invalidation intent keyed by distribution plus content release.
3. After commit, a worker submits/coalesces invalidations, records the provider ID/state, retries bounded transient failures, and alerts on terminal failure.
4. The first release may use exactly one coalesced /* invalidation for each activated release. Route/tag optimization is follow-up; it cannot reduce correctness.
5. A failed invalidation cannot roll back the already atomic content pointer. Bounded TTL keeps the old safe representation for no longer than its class TTL; failure is visible and never leaves indefinite mixed state.

Application deployment:

• fingerprinted assets require no invalidation;
• after the new web revision is ready and before release finalization, submit at most one idempotent /* invalidation keyed by exact application SHA and wait for Completed within the documented bound;
• failure fails finalization/alerts and keeps or restores a known-good application revision; rerun reuses the same logical intent;
• rollback invalidates again under the rollback release identity so templates/routes cannot remain mixed.

Invalidation paths contain public paths only, never query secrets, email, profile data, management links, or preview tokens. Operations expose safe counts/state, not raw sensitive URLs.

WAF and application abuse controls

Attach one Terraform-managed CloudFront-scope web ACL. No console-only rule, emergency edit, pricing subscription, or advanced paid product is allowed.

Initial rules:

• AWS managed common web-threat, known-bad-input, and IP-reputation protections selected within the chosen plan/rule quota;
• malformed/oversize request and disallowed-method/path protections;
• rate-based rules with these count-mode starting thresholds per source IP over five minutes:
  • ordinary cacheable public GET/HEAD: 2,000;
  • search, unknown-query, or other origin-bound anonymous reads: 300;
  • /api/ reads: 300;
  • signup/login/profile/Slack/course/event registration paths: 60;
• a separately bounded emergency block/rate rule controlled through reviewed Terraform input.

Run managed and rate rules in count mode for at least seven representative days in development, record aggregate matches/false positives, then enable blocking only for reviewed rules. Known exploit/IP-reputation matches may move sooner only with deterministic fixtures and no legitimate-user regression. Blocked/rate-limited requests never reach ALB/Django/ECS/database/email and return a safe non-cacheable response.

Application services retain stricter business limits: login/verification/resend/profile/registration by normalized identity plus safe IP class, and admin API by principal/capability. Edge IP limits are not account authorization and do not solve distributed botnets alone.

Do not trust User-Agent as crawler identity and do not create an unconditional verified-bot bypass. Production SEO crawlers may operate under ordinary public limits. robots.txt expresses crawl preference but is not enforcement. Use plan-provided common/self-identifying bot analytics or controls only when the selected tier supports them; targeted/advanced bot, fraud, account-takeover, CAPTCHA, or challenge products are not enabled in MVP. High-rate/cache-busting/known-reputation traffic is blocked through the baseline rules; distributed anomalies trigger alarms and the reviewed emergency rule.

Cost-plan decision and alarms

Use the cheapest option that satisfies the exact accepted distribution, cache, WAF, logging, Terraform, and bot-control contract. Do not reduce security, observability, or cache correctness to fit a pricing tier.

Before any plan subscription or apply, record a redacted read-only comparison using current AWS documentation and the latest 30 days of workload-only metrics (or the available shorter window plus explicit projection):

• request count, transfer, cache hit/miss, WAF evaluated/blocked, log ingestion, invalidation, edge-function/Lambda@Edge, ALB origin transfer/request, ECS, RDS, and residual service cost;
• normal, 10x viral, cache-busting agent, and distributed-bot scenarios;
• exact distribution/account eligibility, number of behaviors/WAF rules, function associations, standard versus real-time logging, common versus targeted bot features, and any unsupported association;
• pay-as-you-go total and the Free, Pro, Business, and Premium flat-rate features/allowances then current.

Safe selection rule:

1. Free is eligible only if all required logging, WAF rules, cache behaviors/policies, and usage fit; no required evidence may be dropped.
2. Prefer Pro when it is the cheapest flat tier whose real subscription eligibility check accepts the exact candidate and whose allowances cover the forecast with headroom.
3. If Pro rejects a required configuration, compare pay-as-you-go with Business using measured/projection totals; choose the cheaper sufficient option. Business is not selected merely to obtain advanced bots that are out of MVP.
4. Premium/advanced products require a new owner-approved cost issue.
5. If flat-plan lifecycle cannot be managed reproducibly through accepted infrastructure automation, retain pay-as-you-go until it can; do not make a console-only subscription.

At grooming time, current AWS documentation lists Free at $0/month, Pro at $15/month with 10M requests/50TB and 25 WAF rules, Business at $200/month, and no overage charges for flat plans; standard logging begins at Pro and targeted bot controls begin at Business. Recheck at implementation because pricing/features can change. AWS says requests blocked by WAF/DDoS do not count against flat-plan allowances; residual ALB/ECS/RDS, non-included edge compute, and unrelated services still require alarms.

Set workload budget/anomaly and traffic alarms with named owner/runbook for:

• 50%, 80%, and 100% of the selected allowance or forecast;
• cache hit ratio below 70% for cacheable route classes after warm-up;
• origin-request rate above twice the reviewed normal peak for 15 minutes;
• WAF block/rate anomaly, 4xx/5xx, invalidation failure/age, and edge-function error;
• ALB/ECS/RDS cost or load rising despite WAF/cache controls.

The emergency action is a reviewed Terraform rate/block toggle or cache disable/TTL-zero rollback. No alarm action broadens caching or exposes the origin.

Current official evidence to recheck:

• CloudFront flat-rate plan features, allowances, covered costs, and unsupported features
• CloudFront pricing comparison

Observability and privacy

Expose aggregate route class, viewer class, cache status, age bucket, invalidation state/latency, WAF rule label/action, origin-request rate, bytes, status, and function errors. Use bounded labels.

Do not log raw Cookie, Authorization, session/CSRF value, complete query, IP, country suggestion, origin-verification value, preview/management token, Slack link, profile field, or response body. Standard CloudFront/WAF logs use encrypted storage, bounded retention, least privilege, and redaction/field omission. Real-time logs are not required.

Development remains noindex, nofollow on HIT, MISS, redirect, error, asset, and WAF denial. Positive caching never changes production canonicals or makes a preview indexable.

Non-goals

• No caching of authenticated/private/personalized/registration/management/search responses, unsafe methods, or arbitrary query variants.
• No public origin, removal/readback of the generated origin guard, VPC/topology redesign, Route 53 zone/delegation change, production account mutation, or unrelated infrastructure import.
• No advanced/targeted Bot Control, fraud/account-takeover product, CAPTCHA, challenge, real-time logs, Origin Shield, multi-origin failover, or guaranteed botnet elimination in MVP.
• No SEO/indexing change, analytics/tracking addition, public-member country personalization, application microservice, or Redis cache.
• No content activation feature beyond the invalidation contract owned with its source issue.

Acceptance criteria

• One generated route-cache registry classifies every current route and CI fails on an unclassified route or a mismatch between Django, Terraform behavior, and deployed smoke expectations.
• The exact matrix above is implemented with min_ttl 0; representative anonymous public GET/HEAD reaches MISS then HIT/Age, while search/private/unsafe/error classes never store.
• Viewer classification removes spoofed internal headers, treats malformed/unknown credential state as private, and prevents authenticated/private responses from entering or being served from shared cache even after a warmed anonymous response.
• Application and edge both enforce private/no-store for all explicit private routes and credentialed mixed-route requests; Set-Cookie/CSRF/PII/identity/Vary:* canaries are never cached.
• Cache/origin request keys contain only the exact allowlists; Host/header/cookie/query duplication, encoding, normalization, tracking, unknown parameter, and cache-poisoning variants fail safely without unbounded misses.
• CloudFront country reaches only the zero-TTL consumer routes, cannot be viewer-forged or direct-origin supplied, validates to an optional suggestion, and stores/logs no raw IP/header.
• Content and deployment invalidation intents are durable, idempotent, coalesced, observable, secret-free, bounded, and follow the exact activation/deploy/rollback failure semantics.
• Terraform creates/attaches the reviewed WAF ACL and rules; count-mode evidence precedes block mode; blocked traffic is proven not to reach ALB/application/database/email work.
• WAF plus application limits cover ordinary content, search/query abuse, login/signup/Slack/registration, APIs, cache-busting agents, known-bad/IP reputation, and an explicit distributed-bot emergency path without User-Agent-only trust.
• A current, redacted cost/eligibility report applies the cheapest-sufficient selection rule to pay-as-you-go and every eligible flat tier; selected plan/configuration is reproducible, budgeted, and contains no silent advanced product.
• Cache/WAF/invalidation/allowance/origin alarms and runbooks use bounded non-PII metrics and a tested TTL-zero/emergency-rule rollback.
• Development noindex/no-follow, production canonical behavior, TLS, exact hosted-zone ownership, CloudFront-only origin lock, and infrastructure portability remain unchanged.
• Website and aws-infra Terraform/policy/unit/integration/security/migration/OpenAPI/full-Django/core-Playwright gates pass through repository-standard commands; no state/plan/credential/secret is committed.
• [HUMAN] After authorized deployment, redacted live evidence proves the full anonymous MISS->HIT/private bypass matrix, country suggestion, invalidation, WAF block-before-origin, selected plan/allowance alarms, noindex, and TTL-zero rollback without obtaining or exposing protected values.

Automated and fault scenarios

1. For every route class: GET/HEAD/unsafe method, empty/allowed/unknown/duplicate/malformed query, gzip/brotli, 200/redirect/404/4xx/429/5xx, Set-Cookie, ETag/304, expiry, stale origin, and invalid origin Cache-Control.
2. Warm anonymous cache, then request as two sessions, Authorization principals, CSRF-only cookie, preview/management token, malformed cookie/header, and anonymous again. Assert body/header/cache key/age never crosses a private boundary.
3. Poisoning matrix for Host, forwarded/internal/country headers, case/duplicate headers, encoded path separators, cache-buster queries, tracking redirect, Vary, conflicting cache directives, and compression variants.
4. Content activation changes detail/hub/feed/sitemap/redirect/stable asset; duplicate and concurrent activation; invalidation throttle/timeout/terminal failure; deploy, finalization failure, rollback, and job replay.
5. Country present/missing/unknown/lowercase/forged, direct origin, local/test mode, and Unify member profile onboarding, Slack access, and course registration #108 visible confirmation.
6. WAF fixtures for common exploit, bad input, known reputation, ordinary burst, cache-buster, dynamic-route rate, distributed sources, legitimate NAT/shared IP, accessibility client, and emergency toggle. Prove count/block results and no origin work.
7. Cost model for actual baseline, normal, 10x, cache-busting, WAF-blocked attack, and distributed bot; exercise flat eligibility failure and deterministic fallback without changing configuration/security.
8. Terraform negative fixtures reject zero/positive TTL drift, private behavior using public policy, all-viewer forwarding on cacheable routes, absent WAF/origin lock, excessive rule/behavior quota, unreviewed paid feature, console-only state, missing alarms, and production/sandbox coupling.
9. Kill/timeout edge function, invalidation worker, Django, and origin; prove classifier fails private, stale is bounded by class, alarms fire, and TTL-zero rollback restores safe dynamic service.

Browser and deployed acceptance

At desktop about 1440x900 and mobile about 390x844 using only synthetic accounts:

1. Load approved public editorial and catalog pages twice; inspect X-Cache/Age, body/canonical/noindex, assets, responsive layout, keyboard behavior, and non-error screenshots.
2. Sign in after warming those URLs; revisit as two accounts and via back/refresh/logout. No member state, navigation, CSRF, profile, enrollment, or Studio capability crosses cache.
3. Visit login, profile/Slack, registration form, learner dashboard, Studio, Django admin, admin API denial, health, search/query, preview fixture, deliberate 404, and 5xx fixture; each matches the matrix.
4. Exercise an editable country suggestion without placing it or account data in a cache key, URL, screenshot, or another account response.
5. Activate synthetic content and deploy/roll back a synthetic revision in the authorized environment; verify invalidation and bounded freshness.
6. Trigger only safe synthetic WAF count/block fixtures; verify denial is readable/non-cacheable and origin metrics show no corresponding work.
7. Capture no auth/token/secret/PII values. Store screenshots only under the project-local .tmp/screenshots/issue-109/ and inspect every image.

Required specification amendments

Implementation must update these exact authority files:

• _docs/specs/01-platform-architecture.md — anonymous cache gate and durable invalidation flow;
• _docs/specs/02-url-link-seo-compatibility.md — route/query/redirect/error cache and canonical interaction;
• _docs/specs/03-github-content-and-people.md — activation route manifest, invalidation, and bounded stale behavior;
• _docs/specs/04-courses-and-cohorts.md — public catalog/detail versus learner/registration cache classes;
• _docs/specs/05-events-registration-email.md — public event versus registration/management/private cache and rate limits;
• _docs/specs/06-studio-and-admin-api.md — explicit edge zero-TTL/private behavior for Studio/admin API and credentialed mixed routes;
• _docs/specs/07-security-privacy-operations.md — classifier trust, poisoning, WAF/bot/cost/observability/rollback rules;
• _docs/specs/08-aws-sandbox-terraform.md — cache/origin-request policies, edge function, country forwarding, WAF, plan selection, alarms, and portability;
• _docs/specs/09-migration-rollout-roadmap.md — deployment/content invalidation ordering and rollback;
• _docs/specs/10-verification-strategy.md — local/Terraform/deployed cache, WAF, cost, and failure matrix;
• _docs/specs/open-decisions.md — record the cheapest-sufficient pricing rule and deferred advanced bot products.

Update #36 source/deployed cache assertions from blanket zero TTL to the generated public/private matrix while preserving every noindex/private/canonical/analytics guard.

Dependencies and delivery

• Accepted Create the portable sandbox network, DNS, edge, and TLS foundation #67–Historical: Bootstrap web.dtcdev.click (superseded by automatic development delivery) #70 infrastructure and Enforce non-indexable development and preview behavior #36 application/edge safety are the baseline.
• Add least-privilege OIDC delivery for website infrastructure #78 is a hard prerequisite for any repeated sandbox Terraform plan/apply; Migrate legacy development Terraform, state, and OIDC identifiers safely #94 state/OIDC migration must be coordinated so Cache anonymous public traffic safely at CloudFront #109 targets the accepted current root/state identity. Source work and local policy tests may be prepared without AWS mutation.
• Import and render the main site collections with exact legacy routes #39/content activation consumes the invalidation service. The initial bounded wildcard remains correct if the optimized route manifest lands later.
• Add observability, service targets, backup/restore, and failure runbooks #66 owns the broader observability platform; this issue owns the minimum cache/WAF/invalidation/cost alarms required to operate its controls.
• Unify member profile onboarding, Slack access, and course registration #108 consumes the country suggestion but neither issue blocks the others functional path.
• Production pricing/account/domain remain separate; the development selection is not silently promoted.

This is a cross-repository website plus aws-infra change. Each repository follows the independent engineer/tester/PM lifecycle. No pull request. Live cache/WAF/pricing verification is HUMAN, so commits use Refs #109, retain/add human, and the issue remains open until the deployed criterion passes.

[alexeygrigorev]

PM grooming — engineer-ready, with cheapest-sufficient cost evidence required

I replaced the intake checklist with the complete cache/security/operations contract.

The issue now fixes:

• one generated route-class registry shared by Django, Terraform policy, and smoke tests;
• exact TTL/key/stale/error behavior for fingerprinted assets, stable assets, editorial detail, hubs/feeds, public course/event pages, redirects, 404s, search, private routes, and unsafe/error responses;
• an anonymous-only cache gate with a non-KVS viewer classifier, spoofed-header removal, private fallback for malformed/unknown credential state, and an origin-response no-store guard;
• allowlisted query/header/cookie forwarding and poisoning defenses;
• CloudFront-only, zero-TTL forwarding of a validated country suggestion to Unify member profile onboarding, Slack access, and course registration #108;
• durable content/deployment/rollback invalidation with a correct first-release wildcard fallback;
• staged managed WAF/rate rules, app-level business throttles, no User-Agent-only crawler trust, and no advanced/targeted bot or fraud product in MVP;
• aggregate cache/WAF/invalidation/cost alarms and a tested TTL-zero rollback.

Cost decision

The issue does not force pay-as-you-go or Business. It requires a current eligibility check against the exact candidate and selects the cheapest sufficient option without weakening security or observability:

1. Free only if every required behavior, WAF rule, logging control, and allowance fits.
2. Prefer Pro if the real subscription eligibility check accepts the exact candidate and forecast.
3. If Pro rejects a required feature, compare measured pay-as-you-go with Business and select the cheaper sufficient option.
4. Premium and advanced products require a separate owner-approved cost issue.

Current AWS documentation says flat plans include CloudFront/WAF/DDoS and have no overage charges; WAF/DDoS-blocked requests do not consume plan allowance. The current plan table lists Free $0, Pro $15 with 10M requests/50TB and 25 WAF rules, and Business $200; it also shows standard logging beginning at Pro and targeted bot controls at Business. The public feature comparison currently places custom caching rules at Business+, so implementation must substantiate actual Pro eligibility for our precise cache/origin policies rather than assuming either acceptance or rejection. Recheck all features/prices at implementation time:

• https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/flat-rate-pricing-plan.html
• https://aws.amazon.com/cloudfront/pricing/

The body names exact amendments to specs 01–10 where relevant plus open-decisions and #36 cache assertions. #78 remains the hard gate for repeated Terraform apply, and #94 must be coordinated for the accepted state/root identity. #39, #66, and #108 are integration consumers/partners as described, not excuses to weaken the first safe wildcard/metrics/no-country path.

Live cache/WAF/plan verification remains a redacted HUMAN gate. Cross-repository commits therefore use Refs #109 and the issue stays open with human. No repository edit, Terraform plan/apply, AWS mutation, pricing subscription, or deployment was performed during grooming.

[alexeygrigorev]

Software Engineer Report — documentation-only first slice

Prepared the uncommitted normative documentation slice shared by #108 and #109 from exact origin/main commit 114be271a36966eb5dbeabd1e811ea910632fa57 on branch issues-108-109-specs in .tmp/website-108-109-specs.

#109 contract recorded

• Replaced the blanket zero-TTL plan with one generated, deny-by-default route registry and the exact public/private TTL, stale, browser-policy, query, redirect, 404, unsafe/error, and cache-key matrix.
• Recorded the fail-closed anonymous viewer classifier, spoofed-header removal, credential/private bypass, origin-response no-store guard, per-behavior forwarding allowlists, and poisoning defenses.
• Preserved existing editorial content and URLs, robots.txt, sitemap, canonicals, structured data, production indexing behavior, and development noindex/nofollow on HIT, MISS, redirect, error, asset, and WAF denial.
• Recorded zero-TTL trusted CloudFront country forwarding to Unify member profile onboarding, Slack access, and course registration #108, durable content/deploy/rollback invalidation ordering, first-release bounded wildcard, provider failure behavior, and TTL-zero rollback.
• Recorded staged managed/rate WAF controls, application business limits, no User-Agent crawler trust, deferred advanced bot/fraud/challenge products, redacted observability, cheapest-sufficient plan eligibility/cost rule, allowance/origin alarms, portability, Add least-privilege OIDC delivery for website infrastructure #78/Migrate legacy development Terraform, state, and OIDC identifiers safely #94 gates, and complete automated/browser/HUMAN failure matrices.

Exact files changed

• _docs/architecture/app-boundaries.md
• _docs/specs/README.md
• _docs/specs/01-platform-architecture.md
• _docs/specs/02-url-link-seo-compatibility.md
• _docs/specs/03-github-content-and-people.md
• _docs/specs/04-courses-and-cohorts.md
• _docs/specs/05-events-registration-email.md
• _docs/specs/06-studio-and-admin-api.md
• _docs/specs/07-security-privacy-operations.md
• _docs/specs/08-aws-development-terraform.md
• _docs/specs/09-migration-rollout-roadmap.md
• _docs/specs/10-verification-strategy.md
• _docs/specs/open-decisions.md

_docs/specs/08-aws-sandbox-terraform.md remains the intentional legacy-link file; its canonical target, 08-aws-development-terraform.md, contains the amendments. The adopted course-platform provenance README was read as baseline evidence and not rewritten.

Checks

• git diff --check — passed.
• make terminology-check — passed: development terminology inventory classified.
• make lint — passed.
• make format-check — passed: 333 files already formatted.
• make check-links — passed: 7 tests.
• make check-seo — passed: 76 tests.
• make compatibility-artifacts-check — passed: 2,965-row manifest valid; generated comparison exactly matches the checked artifact.
• Read-only uv run python local Markdown-reference check — passed: 17 local targets across all 13 changed docs.

Scope and handoff

Documentation only. No Django/app/migration/OpenAPI/Terraform or aws-infra code, AWS plan/apply, WAF/CloudFront/pricing subscription, secret, deployment, production data, screenshot, commit, or push was created. Changes remain uncommitted for independent tester and PM review. Implementation and HUMAN acceptance criteria remain unchecked.

[alexeygrigorev]

QA Review — documentation-only first slice

Verdict: FAIL

I independently reviewed the uncommitted 13-document slice on branch issues-108-109-specs in .tmp/website-108-109-specs, based exactly on 114be271a36966eb5dbeabd1e811ea910632fa57. I read the full groomed #108/#109 issue bodies and comments, both engineer reports, repository process/tester guidance, the complete diff, and the existing URL/SEO/noindex/content/adoption authorities.

The #109-specific normative material is broadly complete: it records the exact route/TTL/stale/browser/key matrix, fail-closed viewer classification and response guard, query/poisoning rules, zero-TTL trusted country forwarding, durable content/deploy/rollback invalidation, staged WAF/application limits, cheapest-sufficient plan rule, alarms, redacted observability, TTL-zero rollback, #78/#94 gates, production SEO preservation, and development noindex/nofollow on HIT/MISS/redirect/error/asset/WAF denial.

Blocking findings

1. The shared authority set is internally contradictory about course registration.
   _docs/specs/open-decisions.md:33-35 still defines event and course registration as accountless and says not to create a learner account until ownership verification. The new Unify member profile onboarding, Slack access, and course registration #108 authority at _docs/specs/04-courses-and-cohorts.md:111-114 requires a durable account first and no anonymous CourseRegistration. Since this Cache anonymous public traffic safely at CloudFront #109 slice jointly amends open-decisions.md and is meant to be coherent with Unify member profile onboarding, Slack access, and course registration #108, decision 6 must be narrowed to event registration and the account-owned course flow stated explicitly. The stale course-registration-verification wording at _docs/specs/05-events-registration-email.md:71 and open-decisions.md:63 should be aligned as well.

2. The shared-profile snapshot boundary is not exact.
   _docs/specs/04-courses-and-cohorts.md:125-132 places normalized email, campaign/cohort, course comment, privacy, and marketing evidence inside the item it calls the “shared-profile snapshot.” The groomed contract says those are registration-owned alongside, not fields of, the deliberately minimized shared-profile snapshot. This ambiguity can drive the wrong persistence/cache/privacy design and should be corrected before implementation.

3. The checked-source compatibility gate fails after preparing the exact pinned inputs.
   I built all five pinned source revisions under project-local .tmp/legacy-compatibility-sources/ without source edits, then ran make compatibility-source-artifacts-check. It failed with:
   checked source artifacts are stale: generated-path-baseline.jsonl, source-build-provenance.json.
   This is not proven to originate in the 13-file diff, but it prevents a fully green independent compatibility gate and needs reconciliation or a documented accepted baseline.

Passing evidence

• git diff --check 114be271a36966eb5dbeabd1e811ea910632fa57 — passed.
• make terminology-check — passed.
• make lint — passed.
• make format-check — passed; 333 files formatted.
• make check-links — passed; 7 tests.
• make check-seo — passed; 76 tests.
• make compatibility-artifacts-check — passed; 2,965-row manifest valid and generated comparison exact.
• Independent local Markdown reference validation — passed; 17 local targets across all 13 changed docs.
• Focused stale-contract searches found no remaining blanket positive-cache prohibition: zero TTL is retained only for private/unclassified/unsafe/operational routes, country consumers, rollout safety, and rollback.
• The existing /blog/guide-to-free-online-courses-at-datatalks-club.html content contract, production robots.txt/sitemap/canonicals/structured data/indexing, and development non-indexability are explicitly preserved.

No application or Terraform implementation changed in this documentation-only slice, so browser screenshots, deployed cache/WAF checks, AWS actions, and HUMAN criteria are not applicable yet. No acceptance checkbox was changed.

[alexeygrigorev]

Software Engineer Rework Report — documentation-only slice

Corrected the independent-QA blockers in the shared #108/#109 specification candidate at exact base 114be271a36966eb5dbeabd1e811ea910632fa57. The work remains uncommitted for the original independent tester to re-review.

Authority corrections

• Narrowed open decision 6 to accountless event registration verification. It now states that event registration creates no learner account, while course registration requires a durable account with verified email ownership and a completed MemberProfile before a confirmed registration can be created. It explicitly forbids an anonymous CourseRegistration.
• Replaced the stale course-registration-verification purpose in specification 05 and open decision 11 with verified-account course/cohort registration confirmation. Neither authority can now be read as authorizing the retired anonymous pending-course-registration flow.
• Split the course-registration persistence boundary exactly as groomed:
  • the deliberately minimized immutable shared-profile snapshot contains only profile UUID, completion schema version, profile revision, snapshot timestamp, optional certificate/display name, confirmed country/derived region, organization, work status, professional role, and seniority;
  • normalized verified email, target campaign/cohort snapshot, course-specific comment, versioned privacy-notice acknowledgement evidence, and separate optional marketing-consent evidence are registration-owned alongside—and never inside—the shared-profile snapshot.
• Applied that exact separation consistently in app boundaries, specifications 01/04/07/09/10, verification/browser/reporting language, and open decision 19. Profile edits and campaign repointing alter neither historical set outside the approved legal deletion/anonymization workflow.
• Clarified the specification index's definition-of-ready wording and the migration roadmap's event-only accountless verification wording.

Correction edits are limited to:

• _docs/architecture/app-boundaries.md
• _docs/specs/README.md
• _docs/specs/01-platform-architecture.md
• _docs/specs/04-courses-and-cohorts.md
• _docs/specs/05-events-registration-email.md
• _docs/specs/07-security-privacy-operations.md
• _docs/specs/09-migration-rollout-roadmap.md
• _docs/specs/10-verification-strategy.md
• _docs/specs/open-decisions.md
• _docs/compatibility/generated-path-baseline.jsonl
• _docs/compatibility/source-build-provenance.json

The previously amended specifications 02, 03, 06, and 08 were read and left unchanged by this correction.

Pinned-source artifact reconciliation

Reproduced the reported failure before changing artifacts:

make compatibility-source-artifacts-check → checked source artifacts are stale: generated-path-baseline.jsonl, source-build-provenance.json.

The canonical builder verified all five project-local checkouts as clean, detached, exact-origin/exact-revision inputs, then regenerated artifacts through uv run python scripts/build_pinned_legacy_sources.py --artifacts-only:

• main site ee43d3fa0929faf691178d79f19528e6f15a83e5: 2,301 files, tree 4ac3b1088f4d24fd33077a4bcb519e88136af1a233fd9d0fa4a21d9e508550b0;
• docs 3f23e006ffdaa498bbc69697408853b6f5eb37dc: 174 files, tree b952b74d200da24eeb0c409f464f6ce0f256df8348626d6229d261a1dcfbf669;
• FAQ c8da1deea9e24945922702994de101dd90a5380a: 152 files, unchanged tree a54dbc8e1b5f47167a667ea571930665b11af17ce15dc672e77354d9ccf60671;
• Podwiki 988b79d0d655bf4755945c3118544cb9e0dbead6: 310 files, tree 4e5bbedca55e577d187800f8952f4e1cfc8acb5dd46cf42e529d606e8f72f791;
• course platform route source 98a235283904b4ef9ad29e196298540756cf1bcc: no generated-site tree, as designed.

Only the two artifacts named by the failed check differ in Git. The generated baseline remains exactly 2,937 rows with zero added/removed paths. Exactly 209 rows changed only content_sha256: 205 main-site, 2 docs, and 2 Podwiki; by kind, 203 HTML, 3 JSON, and 3 XML. No path, classification, expected status, source identity/path/revision, machine-contract marker, or schema field changed. The other four regenerated source artifacts remained byte-identical. Provenance now binds the exact prepared output trees above.

Checked artifact digests are:

• generated-path-baseline.jsonl: d078787a465b916e8005b11405b960bdf55906a5fe586f5f4c1f1507d01c719b;
• source-build-provenance.json: 67d56790277e8e0af06c97f657de14db39121eeea1b3f480b7a273f36ae5d847.

No compatibility builder/source code or pinned checkout was edited.

Verification

• git diff --check 114be271a36966eb5dbeabd1e811ea910632fa57 — pass.
• make terminology-check — pass.
• make lint — pass.
• make format-check — pass: 333 files already formatted.
• make check-links — pass: 7 tests.
• make check-seo — pass: 76 tests.
• make compatibility-artifacts-check — pass: 2,965-row manifest valid; 4,918-difference generated comparison remains byte-identical to the checked ledger.
• make compatibility-source-artifacts-check — pass: all 6 source artifacts checked against the exact prepared pinned trees.
• make test-compatibility — pass: 289 tests.
• Read-only local Markdown-reference audit across all 13 specification documents — pass: 17 local references, zero missing targets.
• Focused stale-contract searches — no combined event/course accountless-verification wording and no course-registration-verification purpose remain. The only anonymous-course phrase is the explicit prohibition against creating one.

Scope and handoff

This remains a documentation and checked-compatibility-artifact slice only. No Django/app/model/migration/OpenAPI/Terraform/aws-infra implementation, browser page, AWS action, WAF/CloudFront/pricing subscription, secret, production data, screenshot, commit, push, or deployment was created. Implementation and deployed/HUMAN acceptance criteria remain pending and unchecked.

Worktree: .tmp/website-108-109-specs
Branch: issues-108-109-specs

[alexeygrigorev]

QA Review — specification-amendment retest

PASS for the corrected, uncommitted specification slice on issues-108-109-specs at base 114be271a36966eb5dbeabd1e811ea910632fa57.

Contract review

• The generated route-classification contract fails closed: only explicitly classified anonymous public GET/HEAD responses may receive bounded positive edge caching; authenticated, credential-shaped, personalized, learner, registration, management, search, operational, unsafe, and error responses remain zero-TTL/private/no-store.
• Cache HIT/MISS does not alter production canonical, robots, sitemap, structured-data, meaningful-content, links, or asset contracts.
• Development remains non-indexable on all response classes: X-Robots-Tag: noindex, nofollow, disallowing development robots.txt, no production sitemap, and production canonicals only for equivalent public pages.
• Existing SEO-bearing editorial pages—including /blog/guide-to-free-online-courses-at-datatalks-club.html—retain path, meaningful content, canonical, metadata, links, and sitemap behavior. Lack of observed indexing is explicitly not permission to rewrite or retire them.
• Trusted country suggestion remains confined to zero-TTL onboarding/profile routes and is never a confirmed profile value, cache-key input, raw logged header, or stored IP.
• Invalidation, WAF, poisoning defenses, cost selection, alarms, TTL-zero rollback, privacy/redaction, and cross-repository responsibility are specified consistently.
• The Unify member profile onboarding, Slack access, and course registration #108 course/member corrections are also consistent here: accountless registration means events; courses use verified durable accounts, and shared-profile snapshots are distinct from registration-owned values/evidence.

Diff and generated-artifact review

• Scope is 15 files: 13 Markdown authority documents and exactly 2 generated compatibility artifacts. No application code, dependency, migration, or runtime configuration changed.
• generated-path-baseline.jsonl: 2,937 unique rows before and after; 0 paths added, 0 removed; exactly 209 rows changed, and content_sha256 is the only changed field (205 main-site, 2 docs, 2 Podwiki; 203 HTML, 3 JSON, 3 XML).
• source-build-provenance.json: only output_tree_sha256 changed for the main site, docs, and Podwiki. Source identities, repositories, pinned revisions, path/public settings, schema, row counts, file counts, and deterministic overrides are unchanged.
• Rebuilt output trees are exact: main site 2,301 files/4ac3b1088f4d24fd33077a4bcb519e88136af1a233fd9d0fa4a21d9e508550b0; docs 174/b952b74d200da24eeb0c409f464f6ce0f256df8348626d6229d261a1dcfbf669; FAQ 152/a54dbc8e1b5f47167a667ea571930665b11af17ce15dc672e77354d9ccf60671; Podwiki 310/4e5bbedca55e577d187800f8952f4e1cfc8acb5dd46cf42e529d606e8f72f791; course platform has no generated tree.
• Added-line sensitive-data scan found 0 email addresses, AWS access keys, private keys, bearer credentials, Slack join URLs, or assigned secrets. No PII/secret evidence was captured.

Commands and results

• git diff --check 114be271a36966eb5dbeabd1e811ea910632fa57 — pass
• make terminology-check — pass
• make lint — pass
• make format-check — 333 files already formatted
• make typecheck — success, 258 source files
• make migrations-check — no changes detected
• make check-links — 7 passed
• make check-seo — 76 passed
• make compatibility-artifacts-check — 2,965 manifest rows valid; the 4,918-row differences artifact reproduced exactly
• make compatibility-source-artifacts-check — checked 6 source artifacts
• make test-compatibility — 289 passed
• make test-playwright-core — 44 passed, 3 deployed-only tests skipped
• Independent modified-Markdown reference check — 17 local references checked, 0 missing
• Independent baseline/provenance structural comparisons — pass

Screenshots are not applicable to this documentation/generated-artifact-only slice; no rendered page or template changed. I did not edit files, check implementation acceptance boxes, commit, push, deploy, mutate AWS, or expose protected values.

The broader website/aws-infra implementation and [HUMAN] deployed verification criteria remain pending; this verdict covers the required specification amendments only.