Skip to content
jiva-studioPublic

About

Deterministic Multi-Agent TDD Harness & Verification FSM for Autonomous Software Engineering

Resources

Stars

1 star

Watchers

0 watching

Forks

Latest commit

Β 

History

38 Commits

Folders and files

NameName
Last commit message
Last commit date
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

πŸ₯ Band

Declarative, Claims-Driven Agent Harness & Verification FSM for Autonomous Software Engineering.

Band is a lightweight, zero-dependency execution harness and external verification engine. It drives AI agents through arbitrary, declarative multi-stage pipelines (TDD, mutation analysis, adversarial audits, multi-tier quality gates) with deterministic state machines and external Stop-hook enforcement.

flowchart LR
    A["πŸ’‘ <b>Feature Request</b>"] --> B["🎯 <b>1. Intent Discovery</b><br/><code>/intent</code>"] --> C["πŸ“ <b>2. Spec & Contract</b><br/><code>/spec</code>"] --> D["πŸ₯ <b>3. Autonomous FSM</b><br/><code>/band</code>"] --> E["πŸŽ‰ <b>Verified Code</b>"]
Loading

πŸ”„ The 3-Step Autonomous Engineering Lifecycle

Band structures autonomous software engineering into three strictly gated phases:

🎯 1. Intent Discovery (/intent)

  • Problem Solved: Prevents context pollution and goal drift (Anthropic Research, 2024) where AI models jump into premature code assumptions before understanding the business domain.
  • 5-Lens Interview: Clarifies JTBD, Adversarial edge cases, Non-Goals, Pre-Mortem failure modes, and Business Invariants.
  • Anti-Pollution Rule: Strictly forbids technical design, file paths, or code snippets in the intent document.
  • Artifact & Gate: .agents/tasks/<slug>/intent.md verified via sh .agents/bin/band --validate-intent

πŸ“ 2. Technical Specification (/spec)

  • Problem Solved: Prevents hallucinatory API design and blast-radius regressions (SWE-bench / Jimenez et al., 2024) by requiring codebase reconnaissance before coding.
  • Blast Radius Matrix: Explicitly maps affected packages, modified files, and downstream consumers.
  • Contract Generation: Creates .agents/tasks/<slug>/done.yaml with declarative verification claims.
  • Artifact & Gate: .agents/tasks/<slug>/done.yaml verified via sh .agents/bin/band --validate

πŸ₯ 3. Autonomous Orchestration (/band)

  • Problem Solved: Eliminates premature task completion and self-correction failure (Huang et al., 2023) by intercepting agent exits with a deterministic FSM and external Stop-hook.
  • Stage Boundaries: Enforces allow (whitelist) and deny (blacklist) file boundaries per stage at the Git level.
  • Stop-Hook Interception: Intercepts agent exit attempts, verifies claims with real compilers and test runners, and feeds back failure traces.
  • Artifact & Gate: Automated Multi-Agent Pipeline driven by .agents/settings.json (Claude Code, via .claude -> .agents) or .agents/hooks.json

βš™οΈ How the FSM Engine Works

flowchart LR
    Dispatch["1. Stage Dispatch<br/>(Role & Directive)"] --> Guard["2. Git Boundary<br/>(allow / deny)"] --> Agent["3. Agent Execution"] --> Hook{"4. Stop-Hook<br/>Claims Passed?"}
    Hook -- "No" --> Retry["Circuit Breaker & Feedback"] --> Agent
    Hook -- "Yes" --> Next["Advance to Stage N+1"]
Loading
  1. Declarative Stages (pipelines/*.yaml): Ordered steps (e.g. Red βž” Green βž” Mutation βž” Review βž” Gate).
  2. Subagent Scoping: Subagents are given minimal directives with strict file boundaries (allow / deny).
  3. External Stop-Hook: When an agent attempts to stop or complete, the hook intercepts and evaluates stage claims.
  4. Circuit Breaker: Trips if identical failures repeat across consecutive attempts, preventing infinite hallucination loops.
  5. Zero External Dependencies: Built on the Python 3 standard library (subprocess, json, hashlib, unittest).

🧠 Scientific Grounding & Theoretical Foundation

Autonomous LLM agents deployed on real-world codebases exhibit predictable failure modes when left unconstrained. Band's architecture directly solves these empirical problems:

1. Separation of Intent from Specification (Anti-Pollution & Goal Drift)

  • The Problem: When an AI agent is asked to simultaneously understand business requirements and write technical code, it suffers from severe context pollution and goal drift (Anthropic Research, 2024): the model jumps to premature technical assumptions (e.g. ad-hoc database schemas), skips critical business edge cases, and loses sight of non-goals without structured decomposition (Wei et al., 2022).
  • Band Solution: Two-phase decoupling via /intent and /spec. Phase 1 captures human intent, user journeys, invariants, and strict non-goals in pure natural language (zero code). Phase 2 takes the validated intent, conducts codebase reconnaissance, and outputs an exact architectural blueprint and verification contract (done.yaml).

2. Inability of LLMs to Self-Correct Without Deterministic Oracles

3. Phase Contamination & Test Modification Gaming

4. Role Specialization & Standard Operating Procedures (SOP)

5. Mutation Testing as an Empirical Adequacy Criterion

πŸ“¦ Pipelines & Task Contracts

Built-in Pipeline Catalog

Pipeline Target Use Case Stages Executed
hardened Mission-critical core domain logic & security features red-phase βž” green-phase βž” mutation-gate βž” adversarial-review βž” gatekeeper
standard Regular product features and component development red-phase βž” green-phase βž” gatekeeper
fast Quick hotfixes, CSS styling, and minor UI tweaks implementation βž” gatekeeper
docs Technical documentation, architecture guides, schemas authoring βž” critic βž” gatekeeper

Task Specification (.agents/tasks/<slug>/done.yaml)

A Task Contract (done.yaml) is the single source of truth for task completion:

slug: feat-user-auth
target: "services/auth"
pipeline: "hardened"

claims:
  # L1: Build & Unit Tests
  - id: l1-build
    kind: make
    target: check-package
    params:
      PKG: "@services/auth"

  # L2: Mutation Testing (Stryker diff against modified files)
  - id: l2-mutation
    kind: mutation
    target: "@services/auth"
    mode: diff

  # L3: AI Adversarial Critic Review
  - id: l3-critic
    kind: critic
    checks:
      - "No unhandled nulls or security bypasses"
      - "Follows intent.md non-goals"

  # L4: Clean Diff Hygiene
  - id: l4-hygiene
    kind: hygiene
    no_stubs: true
    no_skipped_tests: true

πŸš€ Quickstart

Install Band into the root of any repository:

curl -fsSL https://raw.githubusercontent.com/jiva-studio/band/main/install.sh | bash
sh .agents/bin/band --init      # wire hooks, link .claude -> .agents
sh .agents/bin/band --doctor    # verify interpreter, hooks, Makefile targets

This installs the clean .agents/ structure:

.agents/
β”œβ”€β”€ bin/band               # Launcher: sh .agents/bin/band <args> (works from any directory)
β”œβ”€β”€ settings.json          # Claude Code settings incl. Band hooks (seen as .claude/settings.json)
β”œβ”€β”€ .gitignore             # settings.local.json, worktrees/ (harness-local state)
β”œβ”€β”€ hooks.json             # Stop-hook and PreToolUse guard configuration (non-Claude harnesses)
β”œβ”€β”€ pipelines/             # Declarative pipeline definitions (standard, hardened, fast, docs)
β”œβ”€β”€ band/                  # Python FSM verification engine & zero-dependency YAML loader
β”œβ”€β”€ skills/                # Agent skills (/band-install, /intent, /spec, /band, /coder)
β”œβ”€β”€ agents/                # Subagent definitions (optional)
└── tasks/                 # Task folders with intent.md, spec.md, done.yaml

Running Band

Always invoke Band through the launcher, never with a bare python3 -m band (the package lives in .agents/, so it is not importable from the repository root):

sh .agents/bin/band --status

The launcher adds .agents/ to PYTHONPATH and picks the interpreter in this order: $BAND_PYTHON (may contain arguments, e.g. BAND_PYTHON="uv run --no-project python"), python3, python (>= 3.8), then uv run --no-project python. This makes it work on hosts such as NixOS where python3 is not on PATH but uv is.

Claude Code integration

.agents/ is the only canonical agent directory. band --init makes .claude a symlink to it (.claude -> .agents), so Claude Code reads .agents/settings.json, .agents/skills/, .agents/agents/ and .agents/commands/ directly:

  • If .claude does not exist, the symlink is created.
  • If a real .claude/ directory exists, its contents are migrated into .agents/ first: settings.json is deep-merged (lists unioned), skills/, agents/ and commands/ entries and other files are moved. Existing .agents entries are never overwritten. Migration is all-or-nothing: on any conflict (same entry with different content, a differing scalar setting, invalid JSON) nothing is changed and --init reports SKIPPED with the reasons; band --doctor keeps warning until it is resolved.
  • .agents/.gitignore ignores harness-local state arriving through the link (settings.local.json, worktrees/).

Band's hooks are merged into .agents/settings.json (existing settings and hooks are preserved; re-running is idempotent). The guard also protects this file (and .claude/settings.json, the same file) from agent edits; ~/.claude/settings.json and settings.local.json are not affected:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Edit|Write|MultiEdit|NotebookEdit|Bash",
        "hooks": [{ "type": "command", "command": "sh \"${CLAUDE_PROJECT_DIR}/.agents/bin/band\" --guard --harness claude-code", "timeout": 30 }]
      }
    ],
    "Stop": [
      {
        "hooks": [{ "type": "command", "command": "sh \"${CLAUDE_PROJECT_DIR}/.agents/bin/band\" --hook --harness claude-code", "timeout": 600 }]
      }
    ]
  }
}
  • PreToolUse guard reads Claude Code's stdin payload (tool_name, tool_input.file_path / notebook_path / command). A protected write is denied with exit code 2 and hookSpecificOutput.permissionDecision: "deny"; allowed calls exit 0 silently, so Claude Code's own permission prompts still apply.
  • Stop hook blocks with exit code 2 (reason on stderr, plus hookSpecificOutput {"hookEventName": "Stop", "decision": "continue", "reason": ...} on stdout) while stage claims fail, and exits 0 when the pipeline passes, is paused, or no task is active. An internal Band error never traps the agent: the stop is allowed and a systemMessage is shown.
  • .agents/hooks.json keeps the original dialect ({"decision": "allow" | "deny" | "continue"}) for other harnesses; --harness auto (the default) detects Claude Code payloads by hook_event_name.

Critic claim limits

The critic claim reviews the complete task diff β€” committed, uncommitted and untracked changes against the merge-base of the task's base branch (claim base > done.yaml base_branch > $BAND_BASE_BRANCH > origin/HEAD > main > master). It never reviews truncated input: if the diff exceeds max_diff_chars (default 200000) or intent.md exceeds max_intent_chars (default 50000), the claim fails with an explicit message. The CLI timeout defaults to 300 s (timeout on the claim or BAND_CRITIC_TIMEOUT).

πŸ’¬ Developer Workflow

Developers trigger the workflow directly through chat slash commands:

> /band-install                           # 0. Auto-configures Makefile, mutation testing & doctor
> /intent Add course cover image picker   # 1. Runs 5-lens interview & locks intent.md
> /spec                                   # 2. Explores codebase, spins worktree & generates done.yaml
> /band                                   # 3. Runs autonomous multi-agent FSM pipeline
  • /band-install β€” Inspects repository stack, generates tailored Makefile verification targets and mutation test configurations, and runs band --doctor.
  • /intent <feature> β€” Conducts the 5-lens interview and produces .agents/tasks/<slug>/intent.md.
  • /spec β€” Creates an isolated Git Worktree (.agents/worktrees/<slug>), conducts codebase reconnaissance, maps blast radius, and generates a validated done.yaml.
  • /band β€” Spawns specialized subagents through the FSM pipeline under external Stop-hook supervision until 100% verified.
  • /coder β€” Implementation agent for single-agent tasks and subagent coding phases.

πŸ“„ License

MIT Β© Jiva Studio

About

Deterministic Multi-Agent TDD Harness & Verification FSM for Autonomous Software Engineering

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages