Check the work before finishing

When the model answers, your code checks the work. If the check fails, the model keeps working.

import subprocess

from alpineagents import Agent, State, loop


def failing_tests() -> str | None:
    """The pytest output if any test fails, otherwise None."""
    result = subprocess.run(["pytest", "-q"], capture_output=True, text=True)
    return None if result.returncode == 0 else result.stdout[-3000:]


@loop(until=State.is_answered, limit=40)
def fix_until_green(agent: Agent, state: State):
    agent.think(state)
    if state.wants_tools():
        agent.use_tools(state)
        return
    failures = failing_tests()
    if failures:
        state.add_notice(f"The tests still fail. Fix them, then answer.\n{failures}")

1. When the reply asks for tools, the turn runs them and ends with return.
2. When the reply is an answer, failing_tests() runs the tests.
3. If they fail, state.add_notice(...) adds the output to the context.
4. The notice comes after the answer, so State.is_answered is false and the loop continues.
5. If the tests pass, nothing is added. State.is_answered is true and the loop stops before the next turn.

limit=40 still caps the run if the model cannot fix the tests.

Why a notice

add_notice adds a message from your code, marked with [notice] so the model can tell it apart from the person. Use add_user_message for messages from the person.

Related

• State: add a message
• Stop conditions