Architecture

The board & objective state machine

How OperationKit turns a card on a board into a verified, auditable piece of work — and why the state machine is the key to making that reliable.

The objective as atomic unit

Every piece of work in OperationKit is an objective: a single SQLite row, a card on the board, and — while it runs — a tmux session with a live agent terminal. These three representations are the same object. The card moves because the row changes state; the terminal exists because the row is in working; the row gains evidence because the terminal produced output.

This three-way correspondence is intentional. It gives you one place to look for the status of any piece of work (the board), one place for the agent to write evidence (the filesystem), and one place for the orchestrator to make routing decisions (the database). Splitting these into separate systems — a project tracker here, a job queue there, a log aggregator somewhere else — creates a synchronization problem that becomes a trust problem: you can never be sure the card reflects what the agent actually did.

Three objective types

Not all work needs the same review overhead. OperationKit defines three types, each with a different path through the state machine:

The type is set when the objective is created and doesn't change. This means the review overhead is a deliberate decision made at intake, not something that slips through accidentally. A bug that accidentally ships as a project gets more review than necessary; a project that ships as a task bypasses a safety gate. Getting the type right matters.

Why a state machine

The core alternative to a state machine is a list of steps a human or agent follows — a checklist, a runbook, a prompt. Checklists are useful but not enforced: an agent can skip a step, a human can approve without reading, and nothing in the system prevents the next step from running before the prior one is verified.

A state machine makes entry and exit conditions explicit and enforced. An objective in ai_review cannot skip to done without a reviewer session producing a pass verdict and the state-poller recording the transition. The transition is auditable: the row's state history, the reviewer's ARTIFACT.md, and the timestamp are all in the database. If something goes wrong, there is a complete record of what state the objective was in at each point in time.

Yao et al.'s Tree of Thoughts (2023) showed that decomposing a problem into discrete, evaluable intermediate states dramatically improves reliability on complex tasks. The insight generalizes beyond single-model reasoning: at the system level, decomposing work into states with explicit evaluation criteria is what separates a reproducible engineering process from one that works sometimes. The state machine is OperationKit's implementation of that insight at the board level.

State machine

An objective transitions through six states. The transitions are driven by the state-poller, a server-side process that watches for terminal-session exit events and fires the appropriate state change. No transition requires human intervention except the human_review → done gate on project-type objectives.

The fail path

When the autonomous reviewer returns a fail verdict, the state machine reverts the objective to working rather than closing it. This is not an error state — it's a designed iteration loop. The reviewer's findings are prepended to the next working session's context so the agent enters its retry with a concrete list of what failed, rather than repeating the original approach.

This loop is bounded: the objective tracks a retry count, and after a configurable maximum the objective surfaces for human triage rather than spinning indefinitely. The loop implements the Reflexion pattern (Shinn et al., 2023): verbal feedback stored as episodic context drives improvement without gradient updates. Madaan et al.'s Self-Refine (2023) extends this result to a broader range of generation tasks, showing that structured critique followed by targeted revision consistently outperforms single-shot generation — even when the same model produces both the output and the critique.

The autonomous reviewer

The reviewer is not a self-evaluation. It is a separate Claude session spawned by the state-poller after the working session exits. It has no memory of the working session's reasoning, no access to the working session's scratch notes, and no prior relationship with the task. It arrives cold and reads only what the working session chose to expose as its deliverable.

The masking property

Every objective has two files in its memory directory:

• NOTES.md — the working agent's private scratch. Hypotheses, dead ends, self-narration, intermediate state. The agent writes freely here because nothing in this file is graded.
• ARTIFACT.md — the deliverable record. What was produced, where it lives (branch, file paths, PR link), and evidence for each acceptance criterion. This is the only file the reviewer reads.

The reviewer is masked off NOTES.md by the harness. It sees ARTIFACT.md and the branch, nothing else from the working session. This separation is the masking property, and it exists for a specific reason: a model evaluating its own reasoning is susceptible to the same blind spots that produced any errors in that reasoning. The working agent's scratch contains its rationalization of why its approach was correct — showing that to the reviewer poisons the independence of the grade.

The masking property means the reviewer grades the deliverable, not the agent's confidence in the deliverable. These are different things. A working agent that is certain it succeeded but left placeholder text in ARTIFACT.md will fail review — because the evidence wasn't written, not because the underlying work wasn't done. This forces the working agent to produce legible output, not just internally coherent reasoning.

Why independence matters

Hong et al.'s MetaGPT (2023) showed that assigning distinct roles with structured handoffs across agent boundaries — rather than having a single agent perform all roles — substantially reduces the error rate on complex software engineering tasks. The key mechanism is that each role has a different information set and a different success criterion: the coder is not also the tester; the tester is not also the reviewer. OperationKit applies this principle at the review boundary: the working agent's success criterion is producing an output; the reviewer's success criterion is verifying that output against the acceptance criteria. The same agent cannot honestly do both at once.

WIP limiting

OperationKit caps concurrent working sessions at 25. Objectives beyond that cap queue at the queue state and wait for a slot to open. This is not a resource constraint — it is a design choice, and it follows directly from Lean manufacturing and Kanban theory.

Benson & DeMaria Anderson's Personal Kanban (2011) articulates the core finding: constraining WIP is the primary lever for improving throughput and reducing lead time. The intuition is that work in progress that is not being actively progressed is not free — it holds context, creates coordination overhead, blocks downstream dependencies, and degrades the quality of attention given to each active item. A system with 100 open items and 5 workers does not deliver 20 items per worker; it delivers confused, partially-done work across all 100.

For an agent system, the cost of unbounded concurrency is more concrete:

• Context-switch cost. Each working session holds a context window. A session that is interrupted, resumes from a stale context, or conflicts with a sibling session on shared files incurs a real error rate penalty.
• Coordination complexity. Two sessions editing the same file, the same branch, or the same database table produce race conditions. At low concurrency these are manageable; at high concurrency they become the dominant failure mode.
• Review queue depth. If working sessions outpace the reviewer's capacity to evaluate them, review becomes a bottleneck that creates false confidence — objectives accumulate in ai_review, pressure builds to approve them faster, and quality gates erode.

The 25-session cap is not a universal optimum. It reflects the resource profile of a single-server deployment with a standard Claude subscription. You can tune it via MAX_CONCURRENT_SESSIONS in .env, but the right direction is to lower it until throughput improves — not raise it until the server struggles.

The delegator / strategy pattern

Some objectives are too large for a single working session to complete from scratch. OperationKit handles this through the delegator pattern: a persistent top-level objective that does not produce a deliverable itself, but instead spawns child objectives and routes work based on what each child produces.

How it works

A delegator objective is an ordinary project-type objective whose working session implements an orchestration loop rather than a direct implementation task:

1. The delegator session wakes, reads its own NOTES.md for state from the prior iteration, and reads the ARTIFACT.md of any child objectives that completed since its last wake.
2. It decides what to do next — spawn a new child, retry a failed child with new instructions, aggregate results, or declare the parent objective complete.
3. It writes its decision to NOTES.md and exits. The session slot is freed.
4. When a child objective transitions to done, the harness fires a [child-complete] event on the parent. The state-poller wakes the delegator's next session with that event in context.

The delegator re-enters working on each [child-complete] event, reads the child's output, and decides the next step. This is the strategy pattern from software architecture applied to agent orchestration: the delegator holds the strategic plan; each child worker holds a specific tactic. Neither needs to know how the other works internally.

Why this beats a monolithic prompt

The alternative is a single long-running agent session that plans, decomposes, executes, and verifies all in one context window. This fails at scale for three reasons:

• Context length. A context window that holds the plan, all intermediate outputs, and all error history quickly reaches capacity. Work done early in the session is compressed or lost by the time the agent needs to reference it.
• Error propagation. A mistake in step 3 of 20 may not surface until step 18. In a monolithic session, the agent must trace back through its own reasoning to identify the root cause — often with degraded context. In a delegator model, child 3 fails review, the delegator reads the finding, and spawns child 3-retry with corrected instructions.
• Parallelism. A delegator can spawn multiple children simultaneously, each working on an independent subtask. A monolithic session is sequential by definition.

Hong et al.'s MetaGPT framework (2023) formalizes this intuition: structured role decomposition with explicit handoffs between agents outperforms a single capable agent on multi-step engineering tasks because it breaks the correlation between the errors — each role's failure mode is different, so they don't compound.

References

The design decisions above are grounded in published research. The citations below point to the specific papers and books that informed each choice.

• Tree of Thoughts: Deliberate Problem Solving with Large Language Models

  Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023). Advances in Neural Information Processing Systems, 36.

  Why it matters here: Demonstrates that decomposing a problem into discrete, evaluable intermediate states dramatically improves reliability on complex tasks. Motivates OperationKit's state machine design: each state is an evaluable intermediate with explicit entry and exit conditions, not just a label on a kanban card.

  arxiv.org/abs/2305.10601 ↗

• Reflexion: Language Agents with Verbal Reinforcement Learning

  Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023). Advances in Neural Information Processing Systems, 36.

  Why it matters here: Shows that storing a natural-language critique of a prior attempt in episodic memory and prepending it to the next attempt outperforms naive retry without any weight updates. Directly informs the fail path: reviewer findings are prepended to the working session's next context, not just logged.

  arxiv.org/abs/2303.11366 ↗

• Self-Refine: Iterative Refinement with Self-Feedback

  Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., … & Clark, P. (2023). Advances in Neural Information Processing Systems, 36.

  Why it matters here: Establishes that structured critique followed by targeted revision consistently outperforms single-shot generation across a wide range of tasks. Validates the ai_review → working → ai_review loop as a design pattern with measurable quality improvement, not just organizational discipline.

  arxiv.org/abs/2303.17651 ↗

• MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework

  Hong, S., Zheng, X., Chen, J., Cheng, Y., Wang, J., Zhang, C., … & Wu, C. (2023). arXiv preprint.

  Why it matters here: Shows that assigning distinct roles with structured handoffs — coder, tester, reviewer — reduces error rates on software engineering tasks by breaking the correlation between errors. Directly motivates the working/reviewer role split, the masking property, and the delegator/worker decomposition.

  arxiv.org/abs/2308.00352 ↗

• Personal Kanban: Mapping Work, Navigating Life

  Benson, J., & DeMaria Anderson, T. (2011). Modus Cooperandi Press.

  Why it matters here: Articulates the foundational Lean/Kanban finding that constraining WIP is the primary lever for improving throughput and reducing lead time. OperationKit's 25-session cap applies this principle directly: the right direction when throughput is low is to reduce concurrency, not increase it. Limiting WIP is not a resource constraint — it is a quality decision.