Use other models¶

Model strings¶

Agent(model=...) accepts a string:

String	Model	API key
`"claude-sonnet-5"`, any name starting with `claude-`	`Anthropic`	`ANTHROPIC_API_KEY`
`"anthropic/<name>"`	`Anthropic`	`ANTHROPIC_API_KEY`
`"gpt-5"`, any name starting with `gpt-`	`OpenAICompatible` at api.openai.com	`OPENAI_API_KEY`
`"openai/<name>"`	`OpenAICompatible` at api.openai.com	`OPENAI_API_KEY`
`"ollama/<name>"`, for example `"ollama/llama3:8b"`	`OpenAICompatible` at `http://localhost:11434/v1`	Not needed

Any other string raises ValueError with the strings you can use instead.

Options with a model string¶

resolve_model turns a model string into a Model, the same way Agent(model=...) does. Keyword options go to the Model it picks:

from alpineagents import Agent
from alpineagents.models import resolve_model

model = resolve_model("anthropic/claude-sonnet-5", max_tokens=16_000, thinking=True)
# the same as Anthropic("claude-sonnet-5", max_tokens=16_000, thinking=True)

agent = Agent(model=model)
print(agent.run("Say hello"))

"ollama/<name>" uses http://localhost:11434/v1 unless you pass base_url=, for an Ollama server on another machine.
An option the picked Model does not take raises TypeError that lists the options it takes.

With base_url=, the string is not split into a provider and a name. The model is always OpenAICompatible, and the whole string is the model name the server receives. Routers such as OpenRouter name their models this way:

import os

from alpineagents import Agent
from alpineagents.models import resolve_model

model = resolve_model(
    "anthropic/claude-sonnet-5",
    base_url="https://openrouter.ai/api/v1",
    api_key=os.environ["OPENROUTER_API_KEY"],
)
# the same as OpenAICompatible("anthropic/claude-sonnet-5", base_url=..., api_key=...)

agent = Agent(model=model)
print(agent.run("Say hello"))

A proxy that speaks the Anthropic API rather than the OpenAI one needs Anthropic itself: Anthropic("claude-sonnet-5", base_url="https://my-proxy/anthropic").

Model settings¶

Create the Model yourself to change its settings:

import os

from alpineagents import Agent, Anthropic, OpenAICompatible

claude = Anthropic("claude-sonnet-5", thinking=True, max_tokens=16_000)

ollama = OpenAICompatible(
    "qwen3:8b",
    base_url="http://localhost:11434/v1",
    context_window=32_000,
)

openrouter = OpenAICompatible(
    "moonshotai/kimi-k2",
    base_url="https://openrouter.ai/api/v1",
    api_key=os.environ["OPENROUTER_API_KEY"],
)

agent = Agent(model=claude)

OpenAICompatible works with any server that speaks the OpenAI Chat Completions API: OpenAI, Ollama, vLLM, OpenRouter and others.
Set context_window to the model's real window. compact_if_full and agent.context_used(state) use it.
Anthropic sends max_tokens=64000 unless you give one, or the model's own output limit if it is lower. It looks the limit up once per model name with the Anthropic Models API; if that fails (a proxy without it), it sends 64000 and, if the API says that is too many, sends the request again with the model's limit. OpenAICompatible sends no max_tokens unless you give one.
Creating a Model uses no network and needs no API key. The key is read at the first request.
Retries are left to the provider SDK: retries=2 by default.

See the Models API for every setting.

Replies cut off by the output limit¶

A reply that hit the output limit has reply.cut_off true, and so does its message in the State: state.messages[-1].cut_off. stop_reason keeps what the provider said ("max_tokens", or "length" from an OpenAI-compatible server). The text of such a reply may end mid-sentence; it is still the answer. A tool call cut in the middle does not run, and the model gets an input error telling it to split the work.

When the limit cut the reply off before any text or tool call, which happens when a model thinks for the whole max_tokens, think raises CutOffError instead of taking the empty reply for an answer. The reply and its tokens stay in the State. Raise max_tokens, or lower the thinking, and go on with a new message:

from alpineagents import Agent, Anthropic, CutOffError, Message, State

agent = Agent(model=Anthropic("claude-haiku-5-5"))
state = State(messages=[Message.user("Map the product")])
try:
    agent.run(state)
except CutOffError as e:
    print(f"cut off after {e.reply.usage.output_tokens} tokens of thinking")
    bigger = agent.copy(model=Anthropic("claude-haiku-5-5", max_tokens=128_000))
    state.add_message(Message.user("Go on, and write the files one at a time."))
    bigger.run(state)

A Model of your own reports a cut-off reply with one of the stop_reason values in CUT_OFF_STOP_REASONS ("max_tokens", "model_context_window_exceeded", "length" or "max_output_tokens").

Cost¶

state.usage.cost is None unless the Model has prices. Prices are dollars per million tokens:

from alpineagents import Agent, Anthropic, Message, Price, State

model = Anthropic("claude-sonnet-5", price=Price(input=3, output=15, cache_read=0.3))
agent = Agent(model=model)
state = State(messages=[Message.user("Say hello")])
agent.run(state)
print(state.usage.cost)

Each Reply records the model that answered in reply.model, as the provider reported it. It can differ from the requested name: an alias resolved to a dated id, or a router that picked another model. The cost still uses the requested Model's price.

Models with no price¶

usage.cost is None when the Model has no price, and a None anywhere makes a total None. That includes the total of a run with subagents, which adds up every request of the tree. A model you pay for with a subscription, or run on your own machine, costs nothing extra, which is not the same as an unknown price. Say so with a price of zero:

from alpineagents import OpenAICompatible, Price

free = Price(input=0, output=0)
local = OpenAICompatible("llama3:8b", base_url="http://localhost:11434/v1", price=free)

Its requests add 0 to the cost, so the total stays a number. Each request is priced with the price of the Model that made it, so the cost of a lead and its subagents is right even when they use different models. Token counts add up across models too, so read them as a sum over all the models.

Switch models in one conversation¶

A State does not belong to the Agent that ran it. Any Agent may run, think, use tools and compact the same State, so a conversation can start on a fast model and go on with a strong one. agent.copy(model=...) makes the second Agent with every other setting the same:

from alpineagents import Agent, Message, State

fast = Agent(model="claude-haiku-5", system="You are a helpful assistant.")
strong = fast.copy(model="claude-opus-5")

state = State(messages=[Message.user("Outline a plan to move the users table to a new schema")])
print(fast.run(state))

state.add_message(Message.user("Now write the data backfill. It must not lock the table"))
print(strong.run(state))

Both runs add to state. The second model sees the whole conversation, including the replies of the first.
state.history records the model of every think request in a model_request entry, so you can see which model wrote each reply. Tokens of ask and compact requests count in state.usage, but they write no model_request entry. usage.cost is None when any model used has no price.
Thinking blocks and other provider-specific blocks are sent back only to the provider that wrote them.
Providers cache a long context per model, so the first request after a switch pays for the whole context again.
The windows differ. Check strong.context_used(state) before you switch to a model with a smaller window, and compact first if needed: strong.compact(state).
Switching never warns. ResumeWarning is only for the first run after store.load (State and history).

Write a Model¶

Subclass Model to add a provider. Implement respond and context_window:

from alpineagents import Agent, Message, Model, Reply, Usage
from alpineagents.types import TextBlock


class Echo(Model):
    """Replies with the last user message. Uses no network."""

    name = "echo"
    provider = "echo"

    @property
    def context_window(self) -> int:
        return 100_000

    def respond(self, request, on_text=None, on_event=None):
        text = request.messages[-1].text
        if on_text:
            on_text(text)
        return Reply(
            message=Message("assistant", (TextBlock(text),)),
            usage=Usage(requests=1),
            context_tokens=self.count_tokens(request),
            model=self.name,
        )


agent = Agent(model=Echo())
print(agent.run("hello"))

Rules for a Model:

Return a Reply. Do not change the State. The Agent records the reply.
User messages can hold Image blocks. Convert them to the provider's image format, or raise a clear error if the provider cannot read images.
Put the model that actually answered in Reply.model: the name the provider reported, or the requested name if it did not say.
Use no network and no credentials in __init__. Create the SDK client at the first request.
Call on_text with each piece of text as it streams in, if on_text is not None.
Wrap provider exceptions that finally fail as RateLimitError, AuthError, ContextTooLongError or ProviderError, with raise ... from e.
Print nothing. Report events such as a fallback to another model with on_event(ModelEvent(...)).

Model has working defaults for everything else, including arespond for async code.

Test your agent: FakeModel replaces the model in tests
Handle unreliable models: wrap a Model to retry