Skip to content

secrets-guard.md

Latest commit

 

History

History
400 lines (277 loc) · 9.03 KB

secrets-guard.md

File metadata and controls

400 lines (277 loc) · 9.03 KB
Raw

Secrets Guard - Automatic Secret Detection and Redaction

Secrets Guard automatically scans your code for sensitive credentials and API keys before sharing with AI assistants or documentation tools. It prevents accidental leakage of secrets by using industry-standard detection tools.

Quick Start

Secrets Guard is enabled by default. Just use CopyTree normally:

copytree

Output:

🔒 Secrets Guard: 2 files excluded, 3 secrets redacted
📎 144 files [980 KB] copied to clipboard

How It Works

  1. Excludes high-risk files entirely (.env, *.pem, credentials.json, etc.)
  2. Scans file content using Gitleaks for 200+ secret patterns
  3. Redacts inline secrets with typed markers like ***REDACTED:AWS-ACCESS-KEY***
  4. Reports findings without exposing the actual secret values

Installation

Install Gitleaks

macOS:

brew install gitleaks

Linux:

# Download latest release
curl -sSL https://github.com/gitleaks/gitleaks/releases/download/v8.19.0/gitleaks_8.19.0_linux_x64.tar.gz | tar -xz
sudo mv gitleaks /usr/local/bin/

Windows:

# Using Chocolatey
choco install gitleaks

# Or Scoop
scoop install gitleaks

Verify Installation

gitleaks version

Note: If Gitleaks is not installed, Secrets Guard will disable itself automatically and show a warning with installation instructions.

CLI Options

Enable/Disable

# Explicitly enable (default)
copytree --secrets redact

# Disable for trusted repos
copytree --secrets off

Redaction Modes

Choose how secrets are marked:

# Typed mode (default) - shows secret type
copytree --redaction typed
# Output: ***REDACTED:AWS-ACCESS-KEY***

# Generic mode - simple marker
copytree --redaction generic
# Output: ***REDACTED***

# Hash mode - includes hash for debugging
copytree --redaction hash
# Output: ***REDACTED:AWS-ACCESS-KEY:a3f5d9ab***

CI Mode

Fail the build if secrets are found:

copytree --secrets fail

Exit code will be non-zero if any secrets are detected, making it perfect for CI/CD pipelines.

Configuration

Global config

Add to your config.yaml — ~/Library/Application Support/CopyTree/config.yaml on macOS, ~/.config/copytree/config.yaml on Linux, %APPDATA%\CopyTree\config.yaml on Windows. Run copytree config show --sources to see the path in use.

secretsGuard:
  enabled: true
  redactionMode: typed
  failOnSecrets: false
  maxFileBytes: 1000000
  # exclude (default) | scan | fail
  oversizePolicy: exclude
  exclude:
    # Additional patterns beyond the defaults
    - internal-secrets.json
    - '**/test/fixtures/**'
    - '**/examples/**'
  gitleaks:
    # Only needed when the binary is not on PATH, or you have your own rules.
    binaryPath: gitleaks
    configPath: /path/to/.gitleaks.toml

Every key here is validated. The schema is closed, so a typo is reported rather than silently ignored:

copytree config validate

Excluded File Patterns

These files are always excluded (never scanned or included):

Environment files:

  • .env, .env.*, .env.local, .env.production, etc.

Private keys:

  • *.pem, *.key, *.p12, *.pfx, *.p8
  • id_rsa, id_dsa, id_ecdsa, id_ed25519

Credentials:

  • credentials.json, secrets.json, auth.json
  • *-credentials.json, *-secrets.json

Service accounts:

  • service-account-*.json
  • firebase-adminsdk-*.json
  • google-credentials.json

Keystores:

  • *.jks, *.keystore, gradle.properties

Config files:

  • .npmrc, .pypirc, .aws/credentials, .docker/config.json

Terraform:

  • *.tfstate, *.tfstate.backup

See full list in src/pipeline/stages/SecretsGuardStage.js

Detected Secret Types

When Gitleaks is installed it is used, and detects 200+ patterns including:

  • AWS: Access keys (AKIA*, ASIA*), Secret keys
  • Google: API keys (AIza*), Service account JSON
  • GitHub: Tokens (ghp__, github_pat__)
  • Slack: Tokens (xoxb-, xoxp-)
  • Private Keys: RSA, DSA, EC, SSH keys
  • Database URLs: With embedded credentials
  • JWT Tokens: Bearer tokens
  • Generic Secrets: High-entropy strings

Without it, a built-in scanner runs instead (src/utils/secretPatterns.js), covering AWS keys, private key headers, the published token prefixes above, and credential-shaped assignments.

What redaction replaces

Only the credential itself. A detection reports the span of the secret, not the statement around it, so redacted code still parses:

// before
const apiKey = 'sk_live_EXAMPLE_NOT_A_REAL_KEY';
// after
const apiKey = '***REDACTED:PROVIDER_TOKEN***';

This is what makes the built-in scanner safe to leave on by default. Its output is source handed to a model, and a rule that swallows the surrounding syntax produces a context that is wrong in a way nothing downstream can detect.

The generic rules are correspondingly conservative. A value has to be a quoted literal or an environment-style assignment, clear an entropy floor, and not be a recognisable placeholder. So these are left alone:

const token = payload.token.trim(); // a property read
const bearer = extractBearerToken(header); // a function call
apiKey: '<YOUR_API_KEY_HERE>'; // a placeholder
password: 'process.env.DB_PASS'; // a reference

Findings never carry the matched bytes. They travel into stats, into events, and onto thrown errors, all of which an embedder is liable to log, so each one carries a rule id, a position, a length, and a truncated SHA-256 fingerprint.

Handling False Positives

Inline Suppression

Add gitleaks:allow comment on the same line:

const testKey = 'AKIAIOSFODNN7EXAMPLE'; // gitleaks:allow

Allowlist Patterns

In config:

allowlist: [
  '**/test/**', // All test files
  '**/fixtures/**', // Test fixtures
  '**/examples/**', // Example code
  'docs/api-examples.md', // Specific files
];

Custom Gitleaks Config

Create .gitleaks.toml to customize rules:

# Disable specific rules
[[rules]]
id = "generic-api-key"
enabled = false

# Adjust entropy threshold
[[rules]]
id = "high-entropy-string"
entropy = 5.0  # Increase to reduce false positives

Then configure CopyTree to use it:

gitleaks: {
  configPath: './.gitleaks.toml';
}

Examples

Basic Usage

# Default: enabled, redacts inline with typed markers
copytree

# Preview the selection without reading a byte of content
copytree plan .

Development Workflow

# Working with trusted internal repo
copytree --secrets off

# Sharing code with AI (paranoid mode)
copytree --redaction generic --secrets fail

CI/CD Pipeline

# GitHub Actions
- name: Check for secrets
  run: copytree . --secrets fail --no-content

# GitLab CI
secrets_check:
  script:
    - copytree --secrets fail --format json -o report.json
  artifacts:
    paths:
      - report.json

Troubleshooting

Gitleaks Not Found

Problem: Secrets Guard disabled with warning

Solution:

# macOS
brew install gitleaks

# Verify
gitleaks version

Too Many False Positives

Problem: Legitimate code being redacted

Solutions:

  1. Use inline gitleaks:allow comments
  2. Add paths to allowlist
  3. Create custom .gitleaks.toml
  4. Disable specific rules

Performance Issues

Problem: Scanning is slow

Solutions:

// Reduce file size limit
maxFileBytes: 500000, // 500KB

// Increase parallelism
parallelism: 8,

// Skip large files entirely
exclude: ['*.bundle.js', '*.min.js']

Binary Files

Problem: Binary files causing errors

Solution: Binary files are automatically skipped. No action needed.

Security Considerations

What Secrets Guard Does

  • ✅ Detects 200+ known secret patterns
  • ✅ Excludes obviously sensitive files
  • ✅ Redacts secrets inline while preserving context
  • ✅ Never logs or stores raw secret values
  • ✅ Works in-memory (no temp files)

What It Doesn't Do

  • ❌ Not a substitute for .gitignore
  • ❌ Not perfect - false negatives possible
  • ❌ Doesn't scan git history
  • ❌ Doesn't rotate leaked credentials
  • ❌ Doesn't guarantee 100% detection

Best Practices

  1. Defense in depth: Use Secrets Guard + .gitignore + .copytreeignore
  2. Review output: Check redactions make sense
  3. Allowlist sparingly: Only for known-safe test data
  4. Rotate if leaked: If a secret escapes, rotate it immediately
  5. Use secret managers: Avoid hardcoded secrets entirely

Reference

Related Documentation

External Resources

Need help? Open an issue on GitHub