auth: import FileTokenCache into CLI and wire DualWrite#5056
Merged
simonfaltum merged 5 commits intomainfrom Apr 22, 2026
Merged
auth: import FileTokenCache into CLI and wire DualWrite#5056simonfaltum merged 5 commits intomainfrom
simonfaltum merged 5 commits intomainfrom
Conversation
First of a three-PR sequence that moves file-based OAuth token cache management from the SDK to the CLI. This PR adds a CLI-local copy of FileTokenCache under libs/auth/storage, a DualWrite helper that mirrors the SDK's historical dualWrite + hostCacheKey convention, wires u2m.WithTokenCache at every NewPersistentAuth call site (including CLICredentials, which is used by every non-auth command), and switches auth logout to the CLI FileTokenCache for token removal. The SDK is unchanged so behavior is byte-for-byte identical: two redundant writes to the same file with the same keys and tokens. See documents/fy2027-q2/cli-ga/2026-04-21-move-token-cache-to-cli-plan.md.
CLI's forbidigo rules forbid os.UserHomeDir (use env.UserHomeDir) and os.IsNotExist (use errors.Is(err, fs.ErrNotExist)). Thread ctx through NewFileTokenCache so the env-based home directory lookup works. Co-authored-by: Isaac
simonfaltum
added a commit
that referenced
this pull request
Apr 22, 2026
The file factory was calling the SDK's cache.NewFileTokenCache(), which is being phased out in favor of the CLI's own storage.NewFileTokenCache(ctx) imported in #5056. Route ResolveCache through it so that legacy and plaintext modes share a single file cache implementation owned by the CLI. Co-authored-by: Isaac
This was referenced Apr 22, 2026
Replaces the caller-side storage.DualWrite helper with a DualWritingTokenCache wrapper. Every write through the wrapper under the primary key is also mirrored under the host key, so refresh paths (Token, ForceRefreshToken) preserve cross-SDK compatibility after the SDK stops dual-writing internally, not just Challenge paths. Co-authored-by: Isaac
mihaimitrea-db
approved these changes
Apr 22, 2026
hectorcast-db
approved these changes
Apr 22, 2026
Contributor
hectorcast-db
left a comment
hectorcast-db
left a comment
There was a problem hiding this comment.
LGTM minus the test issue above
Callers (cmd.RunE closures) now construct the FileTokenCache and pass it to discoveryLogin, runInlineLogin, and loadToken. Previously each of those helpers built the file cache internally, which meant unit tests hitting discoveryLogin or loadToken would create/touch ~/.databricks/token-cache.json on the developer's machine. Tests now pass the in-memory cache helper, so the real file is no longer a side effect of running the suite. Addresses review feedback from Mihai on login.go and token.go. Co-authored-by: Isaac
simonfaltum
added a commit
that referenced
this pull request
Apr 22, 2026
The file factory was calling the SDK's cache.NewFileTokenCache(), which is being phased out in favor of the CLI's own storage.NewFileTokenCache(ctx) imported in #5056. Route ResolveCache through it so that legacy and plaintext modes share a single file cache implementation owned by the CLI. Co-authored-by: Isaac
simonfaltum
added a commit
that referenced
this pull request
Apr 22, 2026
The file factory was calling the SDK's cache.NewFileTokenCache(), which is being phased out in favor of the CLI's own storage.NewFileTokenCache(ctx) imported in #5056. Route ResolveCache through it so that legacy and plaintext modes share a single file cache implementation owned by the CLI. Co-authored-by: Isaac
renaudhartert-db
approved these changes
Apr 22, 2026
simonfaltum
added a commit
that referenced
this pull request
Apr 22, 2026
The file factory was calling the SDK's cache.NewFileTokenCache(), which is being phased out in favor of the CLI's own storage.NewFileTokenCache(ctx) imported in #5056. Route ResolveCache through it so that legacy and plaintext modes share a single file cache implementation owned by the CLI. Co-authored-by: Isaac
simonfaltum
added a commit
that referenced
this pull request
Apr 22, 2026
The file factory was calling the SDK's cache.NewFileTokenCache(), which is being phased out in favor of the CLI's own storage.NewFileTokenCache(ctx) imported in #5056. Route ResolveCache through it so that legacy and plaintext modes share a single file cache implementation owned by the CLI. Co-authored-by: Isaac
deco-sdk-tagging Bot
added a commit
to databricks/databricks-sdk-go
that referenced
this pull request
Apr 23, 2026
## Release v0.130.0 ### Breaking Changes * Remove the `Experimental_IsUnifiedHost` field (and the `DATABRICKS_EXPERIMENTAL_IS_UNIFIED_HOST` environment variable) from `Config`. Unified host detection is now automatic via the `/.well-known/databricks-config` endpoint. * Remove the unused `ErrWorkspaceIDInAccountClient` exported variable. It was never returned from any production path, and its message contradicted the unified host workflow where a single profile with both `AccountID` and `WorkspaceID` produces both clients. * Remove the file-based OAuth token cache from `credentials/u2m/cache`. The removed symbols are `cache.NewFileTokenCache`, `cache.FileTokenCacheOption`, `cache.WithFileLocation`, and the private `tokenCacheFile` struct. The `TokenCache` interface, `ErrNotFound` sentinel, `HostCacheKeyProvider`, and `DiscoveryOAuthArgument` remain exported. `NewPersistentAuth` now defaults to a new in-memory cache (`cache.NewInMemoryTokenCache`) when no `WithTokenCache` option is passed; consumers that relied on the previous file-backed default must supply their own persistent cache. See databricks/cli#5056 for the companion CLI change that moves the file cache into the CLI. ### New Features and Improvements * Add `u2m.WithDiscoveryHost` option to override the default `https://login.databricks.com` host used by the discovery login flow. Intended for testing and development against non-production environments. * Add support for unified hosts. A single configuration profile can now be used for both account-level and workspace-level operations when the host supports it and both `AccountID` and `WorkspaceID` are available. ### Bug Fixes * Fix CLI token source `--profile` fallback: `--profile` is a global Cobra flag that old CLIs accept silently instead of reporting "unknown flag", making the previous error-based detection dead code. Now uses `databricks version` to detect CLI capabilities at init time ([#1605](#1605)). ### Internal Changes * Pass `--force-refresh` to Databricks CLI `auth token` command to bypass the CLI's internal token cache ([#1628](#1628)). ### API Changes * Add [w.TemporaryVolumeCredentials](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/catalog#TemporaryVolumeCredentialsAPI) workspace-level service. * Add `GetPermissionLevels`, `GetPermissions`, `SetPermissions` and `UpdatePermissions` methods for [w.KnowledgeAssistants](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/knowledgeassistants#KnowledgeAssistantsAPI) workspace-level service. * Add `ThumbnailUrl` field for [apps.App](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/apps#App). * Add `JiraOptions`, `OutlookOptions` and `SmartsheetOptions` fields for [pipelines.ConnectorOptions](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#ConnectorOptions). * Add `GoogleAdsConfig` field for [pipelines.SourceConfig](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/pipelines#SourceConfig). * Add `ReplaceExisting` field for [postgres.CreateBranchRequest](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/postgres#CreateBranchRequest). * Add `ReplaceExisting` field for [postgres.CreateEndpointRequest](https://pkg.go.dev/github.com/databricks/databricks-sdk-go/service/postgres#CreateEndpointRequest).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack
Entry point for the opt-in secure token storage work. Review and merge top-to-bottom:
This PR is also the first of a separate 3-PR sequence that moves file token cache ownership from the SDK to the CLI. That sequence (SDK PR removing the internal dual-write, then SDK bump in the CLI) can proceed in parallel with #5008 and #5013.
Why
First of 3 PRs moving file-based OAuth token cache ownership from the Go SDK to the CLI. Today the SDK owns both the cache interface and the file-backed implementation, including the dual-write-under-host-key convention. Long-term we want the SDK to stop owning persistence: the OAuth flow and cache interface stay, but file format and host-key conventions move to the CLI. This unblocks secure storage backends and Renaud's Session model on a cleaner foundation.
This PR imports the cache into the CLI and wires it everywhere. Nothing is deleted from the SDK yet. PRs 2 and 3 (SDK PR, then SDK bump) finish the move.
Changes
Before: CLI relied on the SDK's default
FileTokenCache. Dual-write to the legacy host-based key happened insidePersistentAuth.Challenge()andrefresh()via the SDK's internaldualWrite.Now: CLI owns its own
FileTokenCacheatlibs/auth/storage/file_cache.go, a near-verbatim copy from the SDK (same JSON schema, same path~/.databricks/token-cache.json, same permissions). A newstorage.DualWritingTokenCachewraps the file cache so that every write through it under the primary key is also mirrored under the legacy host key. Everyu2m.NewPersistentAuthcall site in the CLI now passesu2m.WithTokenCache(storage.NewDualWritingTokenCache(fileCache, arg)).Because mirroring happens inside the cache's
Storemethod, every SDK-internal write (Challenge, refresh, discovery) dual-writes automatically. No call site needs to remember to invoke a helper, so refresh paths (Token(),ForceRefreshToken()) preserve cross-SDK compatibility just like login paths do.The SDK is unchanged. It still dual-writes internally, so the two writes hit the same file with the same keys and bytes, i.e. idempotent. Zero user-visible behavior change.
Files touched:
libs/auth/storage/file_cache.go+ test (new)libs/auth/storage/dual_writing_cache.go+ test (new)cmd/auth/login.go,cmd/auth/token.go,cmd/auth/logout.golibs/auth/credentials.goNEXT_CHANGELOG.mdLint-driven deltas from SDK:
os.UserHomeDir()is forbidden in the CLI, usesenv.UserHomeDir(ctx)instead. Required threadingctxthroughNewFileTokenCache.os.IsNotExist(err)is forbidden, useserrors.Is(err, fs.ErrNotExist).Known edge case: Tokens that exist only under the legacy host key (users who logged in before profile-keyed writes existed and never re-ran
auth login --profile) keep working for now because the SDK's internal dualWrite still runs. After PR 2 (SDK stops dual-writing), re-login will be required for those users. Minimal impact.Test plan
file_cache_test.go(port of SDK tests)dual_writing_cache_test.gocovering primary-key mirroring, non-primary passthrough, no host-key, host-key-equals-primary, discovery with populated/emptyGetDiscoveredHost, and Lookup delegationmake checksandmake testpassdatabricks auth login,auth token,auth logouton a live profile before merging