Skip to content

Check the work before finishing

When the model answers, your code checks the work. If the check fails, the model keeps working.

import subprocess
from pathlib import Path

from alpineagents import Agent, Message, State, loop, tool, waiting_for_user


@tool
def write_file(path: str, content: str) -> None:
    """Create a file, or replace its content"""
    Path(path).write_text(content)


def failing_tests() -> str | None:
    """The pytest output if any test fails, otherwise None."""
    result = subprocess.run(["pytest", "-q"], capture_output=True, text=True)
    return None if result.returncode == 0 else result.stdout[-3000:]


@loop(until=waiting_for_user, limit=40)
def fix_until_green(agent: Agent, state: State):
    agent.think(state)
    if state.pending_calls:
        agent.use_tools(state)
        return
    failures = failing_tests()
    if failures:
        state.add_message(Message.notice(f"The tests still fail. Fix them, then answer.\n{failures}"))


agent = Agent(model="claude-sonnet-5", tools=[write_file], loop=fix_until_green)
print(agent.run("Fix the failing tests"))
  1. When the reply asks for tools, the turn runs them and ends with return.
  2. When the reply is an answer, failing_tests() runs the tests.
  3. If they fail, state.add_message(Message.notice(...)) adds the output to the context.
  4. The notice comes after the answer, so waiting_for_user is false and the loop continues.
  5. If the tests pass, nothing is added. waiting_for_user is true and the loop stops before the next turn.

limit=40 still caps the run if the model cannot fix the tests. waiting_for_user is explained in Write your own loop.

Why a notice

Message.notice(text) makes a message from your code, marked with [notice] so the model can tell it apart from the person. Use Message.user(text) for messages from the person. Both go in with state.add_message(...) (Keep the conversation).