CraftResearch Tech Logo
CraftResearch.tech
LAB NOTE 01Published May 15, 2024 · 8 min readAuthor: Sakshyam Pokhrel

Deterministic Multi-Agent State Verification in Desktop Environments

Why probabilistic chat assistants inevitably hallucinate during multi-file refactors, and how orchestrating specialized agents inside sandboxed terminal environments with deterministic test verification creates reliable autonomous workflows.

1. The Single-Turn Assistant Bottleneck

Traditional AI coding assistants treat software modification as a text completion task. While effective for single-function generation or snippet explanations, this paradigm fundamentally breaks when tasked with multi-file architectural refactors.

When an agent cannot execute shell commands, compile source code, or inspect compiler telemetry, it operates blind to runtime side effects. A single incorrect import or mismatched type definition cascades across dependent modules without the agent ever receiving negative feedback.

“Software reliability cannot rely on statistical probability alone. It requires deterministic verification loops where compiler exit codes and test suites act as the authoritative ground truth.”

— Sakshyam Pokhrel, CraftResearch Tech

2. The Multi-Agent Supervisor Architecture

Inside CraftCode, we separate agent roles into distinct lifecycle stages:

  • 01. Research & AST Analysis: Scans dependency trees, reads types, and establishes existing invariants before modifying code.
  • 02. Planning & Atomic Changes: Deconstructs the user goal into isolated, single-responsibility file edits.
  • 03. Sandboxed Execution: Applies edits and triggers real build/test commands via Model Context Protocol (MCP) tool bindings.
  • 04. Verification & Self-Correction: Intercepts non-zero exit codes, parses compiler stack traces, and automatically repairs the failure before presenting changes.

3. Empirical Takeaways for AI Engineering

Through hundreds of autonomous refactoring runs, we have verified three primary architectural rules:

  1. Always prefer exact diff replacements over full-file rewrites: Full file regenerations introduce token truncation errors and unintended deletions.
  2. Maintain persistent structured project context: Storing architectural decision records in standardized schemas prevents agents from losing context across long sessions.
  3. Strict timeout bounds on background daemons: Long-running processes must report status asynchronously without blocking the primary agent loop.
ABOUT THE AUTHOR

Sakshyam Pokhrel

Developer, AI product builder, and founder of CraftResearch Tech. Creator of CraftCode.

Visit founder portfolio (pokhrel.tech) ↗
Collaborate on AI research