Skip to content

migration.md

Latest commit

 

History

History
487 lines (353 loc) · 13.5 KB

migration.md

File metadata and controls

487 lines (353 loc) · 13.5 KB
Raw

Migration Guide: stackctl.sh to stackctl

DRAFT -- Pending CLI Stabilization

This document tracks the migration from stackctl.sh to the standalone stackctl binary. The CLI contracts, release workflow, and GitHub Actions integration are still evolving and have not yet reached a stable 1.0. Sections marked with ⚠️ may change before the first stable release.

This guide documents the migration from the repository-local ./stackctl.sh script to the standalone stackctl binary. It covers configuration migration, command mapping, behavior differences, and rollback instructions.

Overview

AniTrend/local-stack historically shipped a tools/stackctl.sh script plus Python-based generation and rendering tools (generate_stacks.py, render_compose.py). The stackctl binary replaces this entire toolchain with a single Deno-compiled binary, eliminating the Python and script dependencies.

Before After
./stackctl.sh up stackctl up
Python 3 + dependencies Single binary, no runtime
Per-repo local script System-wide install (Homebrew)
Shell-based config via env vars ~/.stackctl YAML config
Manual profile switching Built-in profile overlays

Prerequisites

  • Docker with Swarm mode enabled (same as before)
  • stackctl binary — installed via one of:
    • Homebrew: brew install AniTrend/tap/stackctl
    • GitHub Releases: download the appropriate tarball (stackctl-v<version>-<target-triple>.tar.gz) from the latest release. Supported triples: x86_64-unknown-linux-gnu, aarch64-unknown-linux-gnu, x86_64-apple-darwin, aarch64-apple-darwin.
    • Manual: deno install -n stackctl --allow-read --allow-write --allow-env --allow-run --allow-sys jsr:@anitrend/stackctl
  • SOPS + age (optional) — only needed for stackctl secrets commands

Quick Start

# Verify installation
stackctl --version

# Initialize config in your project
stackctl init

# Deploy all stacks
stackctl sync

# Check environment
stackctl doctor

Configuration Migration

Before: Environment Variables

The old stackctl.sh used shell environment variables and .env files:

export COMPOSE_DIR="./docker-compose"
export RENDER_DIR="./.rendered"
export STACKS_DIR="./stacks"
export STACK_PREFIX="mystack"
export STACKCTL_PROFILE="dev"

After: YAML Config File

Create a .stackctl file (generated via stackctl init):

project: myproject

stack:
  # Root directory for service compose files (each must declare x-stack metadata)
  directory: ./stack
  # Stack names to manage (empty = all discovered)
  names: []
  # Default Docker network
  network: myproject_default
  # Override files (profile or explicit)
  overrides: []

render:
  # Output directory for rendered YAML
  outputDirectory: ./.rendered
  # Fail on unresolved variables
  strict: false

Converting Environment Variables

Old Environment Variable New Config Field Example
COMPOSE_DIR stack.directory ./docker-compose
RENDER_DIR render.outputDirectory ./.rendered
STACKS_DIR No equivalent (generated to stacks/) —
STACK_PREFIX project mystack
STACKCTL_PROFILE --profile flag or STACKCTL_PROFILE env dev

Command Parity

These commands have reached functional parity with the old stackctl.sh script:

Old (./stackctl.sh) New (stackctl) Notes
./stackctl.sh up stackctl up Replaces shell-based deploy
./stackctl.sh down stackctl down —
./stackctl.sh status stackctl status Now with --json output
./stackctl.sh logs stackctl logs Improved streaming
./stackctl.sh reload stackctl reload Full config-aware pipeline
./stackctl.sh doctor stackctl doctor More comprehensive checks

New Capabilities (No stackctl.sh Equivalent)

The standalone binary adds capabilities that were previously handled by separate Python scripts or not available at all:

Command Purpose
stackctl generate Explicit stack regeneration
stackctl render Explicit environment interpolation
stackctl secrets SOPS/age integration
stackctl env .env scaffolding
stackctl plan Inspect operations without executing
stackctl init Config file generation
stackctl sync Full pipeline (generate → render → deploy)

Step-by-Step Migration

Step 1: Export Current Configuration

Record your current stackctl.sh environment:

echo "COMPOSE_DIR=${COMPOSE_DIR:-./docker-compose}"
echo "RENDER_DIR=${RENDER_DIR:-./.rendered}"
echo "STACK_PREFIX=${STACK_PREFIX}"
echo "STACKCTL_PROFILE=${STACKCTL_PROFILE:-dev}"

Step 2: Run stackctl init

# Interactive detection (scans for docker-compose files)
stackctl init

# Or with explicit values
stackctl init --project myproject --preset standard

This creates .stackctl in your project root. Edit it to match your recorded configuration from Step 1.

Step 3: Verify Configuration

stackctl doctor

Fixes any issues reported:

  • Missing Docker or Swarm mode
  • Invalid or missing .stackctl config
  • Missing override files
  • Missing stack directories

Step 4: Dry-Run a Deployment

# See what would happen without making changes
stackctl sync --dry-run
stackctl up --dry-run

Review the output carefully. The pipeline is:

Config → Discover → Generate → Override → Render → Deploy

Step 5: Deploy

# Deploy all stacks
stackctl sync

# Or deploy incrementally
stackctl up my-stack-name

Step 6: Verify

stackctl status
stackctl logs my-service

Profile Handling

Before

STACKCTL_PROFILE=prod ./stackctl.sh up

After

Profiles use separate config overlays:

# Using flag
stackctl up --profile prod

# Using environment variable
STACKCTL_PROFILE=prod stackctl up

Profile overlays are loaded in this order (later wins):

  1. Built-in defaults
  2. .stackctl (base)
  3. .stackctl.<profile> (e.g., .stackctl.prod)
  4. .stackctl.local (local overrides, gitignored)
  5. .stackctl.local.<profile> (local profile overrides)

Override File Support

stackctl supports explicit override files in addition to profile overlays. Override files use Docker Compose override semantics:

  • Scalars: replaced
  • Maps: deep-merged
  • Sequences: appended
stackctl up --override ./overrides/production.yml --override ./overrides/region-eu.yml

Override files are applied after profile merging but before render.

Compose Metadata (x-stack)

stackctl uses the x-stack compose metadata key to group docker-compose files into named stacks during discovery and generation. Every compose file must declare which stack it belongs to.

Supported Forms

Two forms are supported:

Scalar (legacy) -- a plain stack name:

services:
  api:
    image: myapp/api
x-stack: api

Object (v1) -- a map with a name field:

services:
  api:
    image: myapp/api
x-stack:
  name: api

Both forms are equivalent and normalize to the same stack name. Scalar and object forms can coexist within the same stack group; they are merged by normalized name.

Object Form Constraints

The object form currently accepts the name field only. Adding unknown fields will cause an error:

# Invalid -- "labels" is not a recognized field
x-stack:
  name: api
  labels: [production]

During discovery, files with invalid x-stack metadata are skipped for grouping and reported as errors. During explicit loading (e.g. stackctl generate), invalid metadata causes the command to fail with a descriptive message.

The x-stack key is source-only metadata. It is stripped from generated and rendered stack output and never appears in deployed compose files.

Rollback

Rollback a Deployment

# Remove a specific stack
stackctl down my-stack-name

# Re-deploy previous version
docker stack deploy --compose-file .rendered/my-stack-name.rendered.yml my-stack-name

Rollback stackctl Binary

# Homebrew
brew switch stackctl <previous-version>

# Manual
cp /usr/local/bin/stackctl /usr/local/bin/stackctl.new
# ... download previous version
mv stackctl.previous /usr/local/bin/stackctl

Revert to stackctl.sh

The old stackctl.sh remains in your repository and is unaffected by stackctl installation. To revert:

  1. Uninstall stackctl: brew uninstall stackctl
  2. Delete .stackctl config: rm .stackctl
  3. Continue using ./stackctl.sh as before

Generated files (stacks/*.yml, .rendered/*.yml) are compatible between both tools for the same configuration.

Troubleshooting

Docker Not Running

✗ Docker is not running or not accessible

Ensure Docker is running and your user has access:

docker info

Swarm Mode Not Active

✗ Docker Swarm mode is not active

Initialize Swarm mode:

docker swarm init

Stack Not Found

✗ Stack "myapp" not found in /path/to/project

Check that your compose files declare x-stack metadata and are located within the configured stack.directory. See Compose Metadata (x-stack) for supported forms.

# docker-compose.yml -- either form works:
services:
  api:
    image: myapp/api
x-stack: myapp              # scalar form
# or:
# x-stack:
#   name: myapp             # object form

Config Validation Errors

stackctl validates configuration at startup. Run stackctl doctor for a complete diagnostic. Common issues:

  • Missing project: Set the project name in .stackctl
  • Missing stack.network: Set the Docker network name
  • Empty stack.names: Leave as [] to discover all stacks, or list specific stack names
  • Invalid render.outputDirectory: Must be a valid path

Unresolved Environment Variables

In strict mode (render.strict: true), unused variables cause failure. Switch to non-strict mode or provide the variables:

# Non-strict mode
echo 'render:\n  strict: false' >> .stackctl

# Provide variable
export MY_VAR=value
stackctl up

Permission Issues

stackctl requires these permissions:

  • --allow-read — read compose files, config, env files
  • --allow-write — write generated/rendered stacks
  • --allow-env — read environment variables
  • --allow-run — execute Docker, sops, age
  • --allow-sys — system info for doctor

When installed via Homebrew, permissions are pre-configured.

Behavior Differences

Generated Stack Paths

  • Old: Relative paths in generated stacks reference the repo root
  • New: Paths are absolutized to the project root during rendering

This means .rendered/*.yml files are self-contained and can be used independently of the working directory.

⚠️ Generated files are not safe to deploy raw. Stack files in stacks/ (generated) and .rendered/ (rendered) contain ${VAR} placeholders that must be resolved through the render pipeline before deployment. Deploying a generated stack file directly without running stackctl render or stackctl sync will result in unresolved environment variables in your running services.

Deterministic Output

stackctl produces deterministic YAML output:

  • Keys are sorted alphabetically
  • Stack files are ordered by stack name
  • Runs produce identical output for identical input

This enables drift detection in CI.

Error Reporting

  • Old: First error stops the pipeline
  • New: All errors are collected and reported at once
  • Exit codes: 0=success, 1=validation/drift failure, 2=config error, 3=missing dependency, 4=unexpected error

Signal Handling

  • Old: Ctrl-C may leave processes running
  • New: SIGINT is forwarded to child processes; secrets deploy runs cleanup on interruption

Using stackctl in GitHub Actions

⚠️ The GitHub Actions integration is under active development and its location may change before the first stable release.

Add the setup-stackctl composite action to your workflow to install the stackctl binary on any GitHub Actions runner (Linux x64/arm64, macOS x64/arm64):

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Setup stackctl
        uses: AniTrend/stackctl/.github/actions/setup-stackctl@v1
        with:
          version: latest # or a specific version like "0.1.0"

      - name: Verify installation
        run: stackctl --version

      - name: Run stackctl sync
        run: stackctl sync

The action selects the correct tarball (stackctl-v<version>-<target-triple>.tar.gz) for the runner's OS and architecture, verifies the SHA256 checksum, and adds the binary to PATH for all subsequent steps.