Scientific Debugging Loop

Structure automated program repair into explicit stages of hypothesis generation, targeted probe validation, conclusion making, and patching.

When autonomous coding agents attempt program repair directly from bug descriptions, they frequently hallucinate root causes, modify unrelated code, and introduce regressions. The scientific debugging pattern (formalized in systems like AutoSD) structures the repair agent's control flow into four explicit stages:

  1. Hypothesis generation: The agent analyzes issue descriptions, stack traces, and failure symptoms to propose specific causal hypotheses explaining why the failure occurs.
  2. Execution-based validation: Before touching production code, the agent constructs targeted minimal reproducing tests or diagnostic probes to empirically test each hypothesis against the execution environment.
  3. Conclusion making: The agent evaluates the probe outputs, eliminates falsified hypotheses, and isolates the confirmed defect mechanism.
  4. Automated patching and verification: The agent crafts a targeted fix addressing only the verified defect mechanism and confirms that both the reproduction probe and the existing test suite pass.

By decoupling defect diagnosis from code modification, this pattern prevents compound mistake amplification and grounds test-time code refinement in empirical execution feedback rather than speculative edits. It integrates naturally with plan-validate-execute and iterative refinement feedback channels.

Sources

  • Tang & Runkler (2024), sec. 3.5