Add supervisor pattern: /add-supervisor skill (custom LangGraph + Supervisor API) - #33
Open
alexbaur wants to merge 3 commits into
Open
Add supervisor pattern: /add-supervisor skill (custom LangGraph + Supervisor API)#33alexbaur wants to merge 3 commits into
alexbaur wants to merge 3 commits into
Conversation
Introduces a supervisor agent as a post-scaffold pattern (like eval gates,
governance, monitoring) applied once a project has >1 agent. A supervisor
routes user queries across the project's agents and other managed sub-agents.
The core contribution is a Selection Matrix that picks the best-fit supervisor
pattern from three options based on the user's needs, architecture, and the
complexity required — because the three are NOT interchangeable; they differ in
who owns the routing loop and what artifact lands in the bundle:
custom Hand-written LangGraph supervisor. GA, fully DAB-declarable
(an agent App under src/agents/), gated by the same CI eval
loop as any agent. The default.
supervisor_api Databricks-managed loop (Responses API) wrapped in a
declarable App. Beta (AI Gateway + UC OTel-traces preview).
agent_bricks_mas Agent Bricks Supervisor tile. NOT a DAB resource — scaffolds
a bundle-declared bootstrap job that provisions it imperatively
and a consumed-endpoint reference. UI GA / SDK Beta.
Integration is deliberately lightweight and maps to what already exists:
- Models the supervisor as "just another agent" (custom/api) so CI's
detect_patterns -> eval_gate picks it up with zero workflow changes.
- Reuses the add_agent.py wiring approach; renders graph/tools/deps variants.
- Records the choice in .agentops-stacks/manifest.yml under `supervisor:` — the
same contract CI/tooling already read.
- No new bundle-init input (choice made post-scaffold when it's meaningful).
- Honors the standing "declarative over notebook-created; document why" rule:
the MAS tile can't be a DAB resource, so it's a documented bootstrap job.
Files:
- plugin/skills/add-supervisor/SKILL.md — skill + Selection Matrix + security posture
- plugin/commands/add-supervisor.md — /add-supervisor command
- plugin/skills/agentops-stacks/scripts/add_supervisor.py — engine (3 patterns)
- .../scripts/supervisor_templates/ — graph (custom, api), tools, MAS job+notebook
- template/.../docs/supervisor-patterns.md.tmpl — pattern deep-dive in rendered project
- template/.../.agentops-stacks/manifest.yml.tmpl — supervisor contract placeholder
- workflows/single-account-single-agent.json — machine-readable decision matrix
- plugin/skills/agentops-lifecycle/SKILL.md — Step 3.5 routing note
- plugin/README.md, install_skills.sh — register the new skill/command
- tests/test_add_supervisor.py — 13 unit tests (no CLI needed), all passing
Co-authored-by: Isaac
Review pass found two defects that would break the generated supervisor App: 1. supervisor_api graph: LangChain message .type "ai" was passed to the Responses API as role "ai" (invalid — expects "assistant"). Added an explicit _ROLE_MAP so human/ai/system/tool map to valid roles. 2. agent.py inheritance: the script copied the source agent's agent.py, which — when the base agent had Lakebase memory — imports get_async_checkpointer from graph. The supervisor graph.py doesn't define it, so the App crashed on import. Now overwrite agent.py with a dedicated stateless supervisor handler (agent_supervisor.py.tmpl). Adds tests covering the agent.py overwrite, including the Lakebase-base case. Co-authored-by: Isaac
Agent Bricks Supervisor (MAS) is being deprecated and will not be available going forward, so the /add-supervisor feature is reduced to the two patterns that are first-class DAB citizens and remain supported: custom Hand-written LangGraph supervisor (GA, default). supervisor_api Databricks-managed loop wrapped in a declarable App (Beta). Both scaffold the supervisor as an agent App, so it deploys and is eval-gated like any other agent — unchanged from before. Removed: - agent_bricks_mas from VALID_TYPES, the scaffold_mas_bootstrap() path, the manifest endpoint branch, and the MAS next-steps in add_supervisor.py - supervisor_templates/mas_bootstrap_notebook.py.tmpl and mas_bootstrap_job.yml.tmpl - the D3a (no-code MAS) gate and the agent_bricks_mas pattern entry from the workflow decision matrix (D3 is now a single supervisor_api gate) - MAS rows/prose from SKILL.md, docs/supervisor-patterns.md, the lifecycle Step 3.5 table, the manifest template comment, the command file, README, and the installer help text - the two MAS-specific unit tests Verified: 12 unit tests pass; workflow JSON valid; end-to-end scaffold of both patterns renders databricks.yml that parses (yaml.safe_load) and graph/agent.py that compile, with no leftover placeholders; an invalid --type agent_bricks_mas is now rejected by argparse. No residual MAS references remain in feature files. Co-authored-by: Isaac
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Introduces a supervisor agent to AgentOps Stacks as a post-scaffold pattern (applied once a project has >1 agent, alongside eval gates / governance / monitoring). A supervisor routes user queries across the project's agents and other managed sub-agents.
The core contribution is a Selection Matrix that picks the best-fit supervisor pattern based on the user's needs, architecture, and required complexity. This PR ships two patterns — both first-class DAB citizens scaffolded as agent Apps:
src/agents/src/agents/Decision matrix (first gate that fires wins)
databricks.yml) → customWhy this fits AgentOps Stacks
detect_patterns → eval_gatepicks it up with zero workflow changes.add_agent.pywiring approach; renders graph/agent/tools/deps variants per pattern..agentops-stacks/manifest.ymlundersupervisor:— the same contract CI/tooling already read.bundle initinput — the choice is made post-scaffold, when it's meaningful.Files
plugin/skills/add-supervisor/SKILL.md— skill + Selection Matrix + per-pattern security postureplugin/commands/add-supervisor.md—/add-supervisorcommandplugin/skills/agentops-stacks/scripts/add_supervisor.py— engine (2 patterns).../scripts/supervisor_templates/— graph (custom, api), agent handler, toolstemplate/.../docs/supervisor-patterns.md.tmpl— pattern deep-dive in the rendered projecttemplate/.../.agentops-stacks/manifest.yml.tmpl— supervisor contract placeholderworkflows/single-account-single-agent.json— machine-readable decision matrix inrouting_notesplugin/skills/agentops-lifecycle/SKILL.md— Step 3.5 routing noteplugin/README.md,install_skills.sh— register the new skill/commandtests/test_add_supervisor.py— 12 unit tests (no CLI needed), all passingTesting
add_supervisor.pypass — build a fake project tree, assert both patterns wiredatabricks.yml/ manifest / layout correctly; no Databricks CLI needed.databricks.ymlparses (yaml.safe_load), renderedgraph.py/agent.pycompile, no leftover placeholders; an invalid--type agent_bricks_masis now rejected by argparse.test_create_project.pytests require Databricks CLI v1.1.0 (unrelated to this change; unaffected).This pull request and its description were written by Isaac.