Harden scoped credentials and versioned transactional template rendering

[alexeygrigorev]

External parent epic: DataTalksClub/website#21

Audited baseline: c0fdfac632f057f1e6a467111613928c4899b13b

Outcome

Provide the tenant-scoped credential and immutable template/rendering contract required before the DataTalks.Club website can manage or send Relay-owned transactional templates.

Relay remains the canonical template store and renderer. It supports one editable draft per client/key, immutable monotonically numbered published versions, safe typed rendering, and separate least-privilege runtime/template-management credentials. Preview and dry-run exercise the same validator/renderer as real delivery while producing zero persistent or outbound side effects.

Baseline gaps

At the audited commit:

• ClientApiKey is hashed/revocable but has no endpoint scopes or expiry/overlap lifecycle;
• EmailTemplate is one mutable row and PUT /api/transactional/templates/{key} uses update_or_create;
• no draft revision or immutable published version exists;
• render_template_string renders with autoescape=False;
• required context checks only presence/non-empty values and do not reject extra/wrong-type data;
• dry-run rendering calls upsert_contact, so the documented zero-write behavior is false; and
• sends identify only template_key, so later mutation is not contractually isolated from queued work.

Scope

Scoped and expiring client credentials

• Extend named client API keys with an explicit non-empty scope set, expires_at, revocation, safe public identifier, creator/audit metadata, and last-used approximation.
• Enforce scope at every client API endpoint before object lookup so wrong-scope and cross-tenant requests fail without existence leakage.
• Support at least:
  • website runtime: transactional:send, transactional:read, templates:read;
  • template management: templates:read, templates:write, transactional:dry_run.
• Preserve hash-only key storage and one-time display. Rotation creates a separately identifiable key and permits an explicit bounded overlap; revocation is immediate.
• Reject expired/revoked/unscoped credentials consistently and redact every credential from logs, errors, audits, docs examples, and test output.
• Add expiry visibility/metrics suitable for an external alert; provisioning real keys and alert destinations remain out of scope.

Immutable template lifecycle

Implement client-scoped APIs with a versioned OpenAPI contract:

• GET /api/transactional/templates/{key} returns redacted catalog metadata, editable draft revision, and current published-version reference.
• PUT /api/transactional/templates/{key}/draft requires expected_revision; a stale revision returns 409 without mutation.
• POST /api/transactional/templates/{key}/publish requires the current draft revision plus an idempotency key and creates one immutable monotonically numbered published version.
• GET /api/transactional/templates/{key}/versions/{version} returns the named immutable version.
• Republishing a historical version for rollback creates a new version; no published row is updated or deleted.
• Retire or deny the old mutable published-content PUT /api/transactional/templates/{key} for newly scoped website credentials. Do not maintain two writable canonical paths.

Each published version owns subject, plain text, HTML/authored rich source, typed required/optional context schema, example context, sender ID, validation policy, timestamps, and immutable content checksum.

Safe zero-write rendering

• POST /api/transactional/templates/{key}/render renders an exact draft revision or published version without creating/updating a contact, message, event, callback, job, queue payload, or provider request.
• Transactional dry-run uses the same validator and renderer and is also zero-write.
• Preview, dry-run, test-send preparation, and real-send preparation share the same rendering implementation.
• Escape variables by default.
• Allow only an explicit small template expression/filter set; reject arbitrary template tags, filters, includes, attribute traversal, or callable access.
• Sanitize explicitly authored rich content and reject unsafe protocols, script/event-handler content, header/address injection, and unsupported constructs.
• Validate missing, extra, and wrong-type context fields with stable safe errors.
• Require meaningful plain/HTML content and link parity without tracking pixels.
• Rendering must be deterministic for the same version/context/options.

Queued snapshot contract

Update the transactional request/model boundary so every future real or test send must name both template_key and immutable template_version. Before provider queueing, Relay snapshots the exact version, resolved sender, recipient, subject, plain text, HTML, and delivery-affecting options. Later draft/publish/republish actions cannot change an already prepared message.

Request-hash idempotency, lease/state semantics, and provider ambiguity are owned by the follow-up delivery-lifecycle issue, not this issue.

Expected files

Implementation is expected to remain within Relay-owned surfaces such as:

• mailing/models.py and new forward-only mailing/migrations/ files;
• mailing/services/auth.py;
• mailing/services/api.py and mailing/services/api_errors.py;
• mailing/services/transactional_catalog.py;
• mailing/services/transactional_rendering.py;
• mailing/services/transactional.py;
• mailing/views.py, mailing/urls.py, and mailing/services/api_docs.py;
• docs/api.md and generated/OpenAPI contract documentation;
• focused tests under mailing/tests/.

Do not rewrite unrelated campaign, Mailchimp, inbound-email, Taskdeck, or deployment behavior.

Acceptance criteria

• Separate runtime and template-management keys can be created with exact scopes, expiry, safe identity, one-time secret display, overlap rotation, and immediate revocation while only hashes persist.
• Missing/wrong scope, expired/revoked key, and cross-client key/template/version access fail closed before existence disclosure.
• Draft edits require expected_revision and concurrent/stale edits cannot lose work.
• Publish is idempotent and produces one immutable numbered version; historical republish creates a new version and preserves all history.
• No API or model operation can mutate a published version.
• A prepared/queued message remains byte-for-byte tied to its template version and sender/content snapshot after later edit/publish/rollback.
• Preview and dry-run create zero contact/message/event/callback/job/queue/provider writes or calls.
• Missing/extra/wrong-type context, malicious source/context/URL, unsupported tags/filters, and header injection fail with safe stable errors.
• Variables escape by default; allowed rich content is sanitized; plain and HTML outputs have meaning/link parity and no tracking pixel.
• Relay OpenAPI identifies the supported versioned contract and tests fail on incompatible schema drift.
• Logs, audit metadata, API responses, and test output contain no raw credentials, protected context, recipient body, or rendered body beyond the explicitly authorized render response.
• Focused unit, database concurrency, API, migration, and contract tests pass without AWS credentials or external network access.

Mandatory tests

1. Create/read draft; correct/stale concurrent revision; publish replay with the same key; changed publish under one key; new version; historical republish.
2. Queue/prepare version N, publish N+1, and prove the N snapshot is unchanged.
3. Exact client/version access plus wrong-scope, expired, revoked, overlap-rotation, and cross-tenant denial.
4. Missing/extra/wrong-type/Unicode/long context and malicious HTML/script/URL/header/template constructs.
5. Preview/dry-run database before/after counts for every contact/message/event/callback/job table plus queue/provider spies proving zero side effects.
6. Schema/migration tests proving published immutability and uniqueness under concurrent publish.

Validation

Use uv and deterministic local fakes only:

git diff --check
uv run python manage.py makemigrations --check --dry-run
uv run pytest mailing/tests/test_transactional_api.py mailing/tests/test_transactional_dry_run.py mailing/tests/test_client_api_key_migration.py
make lint
make test

No real AWS credentials, SES simulator request, Relay sandbox call, key provisioning, callback delivery, or email send is permitted in this issue.

Browser and screenshots

This is a backend/API/OpenAPI contract change with no required visible operator-page change. Browser testing and screenshots are not applicable unless implementation deliberately changes a visible Relay page; such UI expansion requires PM confirmation before entering scope.

Dependencies

• The external ownership decision is website #21.
• No website implementation, owner decision, live credential, or deployment is required for deterministic implementation and tests.
• The delivery idempotency/state issue depends on this issue's immutable template_key + template_version and snapshot contract.
• The website must not enable real sends until this contract is deployed to the sandbox at an exact commit/OpenAPI version and passes pinned consumer tests.

Non-goals

• No website repository change.
• No request-hash idempotency, concurrent first-send recovery, provider lease/state/ambiguity, or claim-time suppression; those belong to the delivery-lifecycle issue.
• No generic transport callback or unmatched provider-event reconciliation; those belong to the callback/reconciliation issue.
• No campaign/newsletter redesign, tracking pixel, engagement analytics, or arbitrary template code.
• No production deployment, production sender/domain, broad recipient access, live credential provisioning/rotation, infrastructure mutation, or external email.

Process handoff

Follow docs/PROCESS.md in an isolated Relay issue worktree. The engineer implements and tests without committing; a separate tester verifies exact criteria and commands; a product manager accepts; only then is the issue committed with Closes #N and merged/pushed through the repository lifecycle without a pull request.

[alexeygrigorev]

Groomed on creation

Owned Relay slice for the scoped/expiring credential, immutable draft/publish/version, safe rendering, and zero-write preview/dry-run contract required by website #21.

Pinned to audited baseline c0fdfac632f057f1e6a467111613928c4899b13b with explicit files, API behavior, concurrency/security tests, uv-backed validation, non-goals, and process handoff. No live credential, deployment, provider call, or email send is authorized.