LLM providers¶

Active Graph ships two concrete LLMProvider implementations. Both expose identical Protocol surface — complete(), estimate_cost(), count_tokens() — so a runtime swapping one for the other doesn't reshape any call site. Choose by the model family you want; everything else is the same.

from activegraph import Graph, Runtime
from activegraph.llm import AnthropicProvider, OpenAIProvider

rt = Runtime(Graph(), llm_provider=AnthropicProvider())  # or:
rt = Runtime(Graph(), llm_provider=OpenAIProvider())

Installing¶

Pick one of three extras. They install cleanly and don't conflict.

pip install "activegraph[anthropic]"   # AnthropicProvider only
pip install "activegraph[openai]"      # OpenAIProvider only
pip install "activegraph[llm]"         # both providers

The [openai] extra also pulls in tiktoken so client-side token counting is accurate; see the count_tokens row below for what happens when tiktoken is missing.

API keys¶

Both providers read their API key from the environment, never from code or a checked-in config:

export ANTHROPIC_API_KEY='...'
export OPENAI_API_KEY='...'

Override the env-var name via the api_key_env= constructor kwarg if you need a different one (per-environment key rotation, for example).

Default model resolution¶

Each provider declares a default_model — the model name the runtime uses when an @llm_behavior doesn't pin one explicitly:

@llm_behavior(name="extractor", output_schema=Claim)
def extractor(event, graph, ctx, llm_output):
    ...

With AnthropicProvider() this resolves to "claude-sonnet-4-5"; with OpenAIProvider() it resolves to "gpt-4o-mini". The runtime stamps the resolved name onto the behavior at registration time (inside Runtime(...)'s first registry materialization), so swapping providers is a one-line change:

rt = Runtime(Graph(), llm_provider=OpenAIProvider())  # gpt-4o-mini
rt = Runtime(Graph(), llm_provider=AnthropicProvider())  # claude-sonnet-4-5

Pass model="..." on the decorator to override:

@llm_behavior(name="extractor", output_schema=Claim, model="gpt-4o")
def extractor(event, graph, ctx, llm_output):
    ...

Cross-provider model-name validation¶

When a behavior pins model="..." explicitly, the runtime checks the name against each shipped provider's recognizes_model():

Provider	Recognized prefixes
`AnthropicProvider`	`claude-`
`OpenAIProvider`	`gpt-`, `o1-`, `o3-`, `o4-`

If the configured provider doesn't recognize the name but a different shipped provider does, the runtime raises InvalidRuntimeConfiguration at registration time with a structured error naming both providers. This catches the most common shape of provider-swap misconfiguration — an @llm_behavior copied from an Anthropic example into an OpenAI-configured runtime — before the first network call, instead of letting the provider 404 silently.

Names no shipped provider recognizes (custom deployments, OpenAI fine-tunes like ft:gpt-4o-mini:org::id, internal naming conventions) pass through silently. The validation is permissive by design: only recognized cross-provider mismatches fire.

Side-by-side¶

Aspect	`AnthropicProvider`	`OpenAIProvider`
`default_model` (used when `@llm_behavior` omits `model=`)	`"claude-sonnet-4-5"`	`"gpt-4o-mini"`
Recognized model families (per `recognizes_model()`)	`claude-*`	`gpt-`, `o1-`, `o3-`, `o4-`
API key env	`ANTHROPIC_API_KEY`	`OPENAI_API_KEY`
SDK	`anthropic>=0.40`	`openai>=1.0`
Structured output	Instruction-based by default: schema + example instance embedded in the system prompt by `build_system_prompt`; provider parses JSON via the shared `parse_structured_response` helper. Opt into native constrained decoding with `Runtime(native_structured_output=True)` — sends Messages API `output_config` on supported `claude-*` families	Same default path. Native mode sends Chat Completions `response_format={"type": "json_schema", ..., "strict": true}` on supported families (`gpt-4o`, `gpt-4.1`, `gpt-5`, `o3`, `o4`)
`count_tokens()`	Server-side via `messages.count_tokens` (1 roundtrip per call when `budget.max_cost_usd` is set and no cache hit)	Client-side via `tiktoken` when available; char/4 heuristic fallback with a one-time debug log if tiktoken is missing
Tool use	Supported (`Tool.to_definition()` emits Anthropic shape)	Supported. The provider translates framework/Anthropic-shaped tool definitions into OpenAI Chat Completions `function` tools and extracts returned `tool_calls` into the shared `ToolCall` shape
Tool-name wire rewriting (v1.3)	Pack-scoped canonical names (`pack.tool`) are outside the API's `[a-zA-Z0-9_-]` alphabet; the provider rewrites `.` → `__` on the wire and maps returned calls back, so the runtime and the event log only ever see canonical names	Same rewriting
Exception mapping (v1.3)	`llm.rate_limited` on 429-shaped errors; `llm.auth_error` on 401/403-shaped errors (terminal, never retried); `llm.request_error` on other 4xx (terminal); `llm.network_error` for the rest (timeouts, connection errors, 5xx — retried)	Same mapping
Reasoning-model parameters	n/a (`max_tokens` is universal)	`o1`/`o3`/`o4`/`gpt-5` families get `max_completion_tokens` and no `temperature`/`top_p` (the API rejects the GPT-4-era parameters). Override the family table with the `reasoning_model_prefixes=` kwarg
Pricing	Family-prefix lookup; override with `pricing=` kwarg	Family-prefix lookup; override with `pricing=` kwarg

Native structured output (opt-in)¶

CONTRACT v1.3 #1. With Runtime(native_structured_output=True), the runtime resolves a structured-output mode per behavior at registration time: native constrained decoding when the provider supports it for the resolved model and the behavior's output_schema fits the native subset (every field required, no numeric/string constraint keywords, no recursion); the prompt-embedded path otherwise. Nothing changes at the @llm_behavior surface — the schema stays output_schema=.

Three things to know before flipping the flag:

Prompt hashes change. Native mode drops the schema block from the system prompt and adds a mode field to the hash input, so LLM caches recorded in prompt mode won't be hit, and replay_strict=True will (correctly) raise ReplayDivergenceError replaying a log recorded under the other mode. Match the flag to how the log was recorded.
Fallback is silent but audited. A behavior that can't go native (provider capability, model family, or schema subset) uses the prompt path; the resolved mode is recorded on every llm.requested event's structured_output_mode payload field.
Validation doesn't move. Responses still flow through parse_structured_response, so llm.parse_error / llm.schema_violation semantics are identical in both modes.

RecordedLLMProvider replays whatever mode the fixture was recorded in — construct it with structured_output_mode="native" to serve native-mode fixtures (the default "prompt" keeps every pre-v1.3 fixture reachable unchanged).

Embedding providers (v1.3; recorded runtime path in v1.8)¶

EmbeddingProvider is the runtime's second provider seam, next to LLMProvider. Memory and retrieval capabilities still live in packs, but calls now go through Runtime.embed or the packs-facing ctx.embed so external I/O is recorded and replayable:

from activegraph import Runtime
from activegraph.llm import EmbeddingProvider

class MyEmbedder:                      # any object with the Protocol shape
    default_model = "text-embedding-3-small"
    def embed(self, *, texts, model):
        ...                            # call your embedding API
        return vectors                 # one list[float] per input text

rt = Runtime(graph, embedding_provider=MyEmbedder())
vectors = rt.embed(["first document", "second document"])

# Inside a behavior or pack:
# vectors = ctx.embed(texts, model="text-embedding-3-small")

Forks inherit the parent's embedding provider (override per-fork with fork(embedding_provider=...)); Runtime.load(..., embedding_provider=...) wires one at load time. Set replay_embedding_cache=True on load/fork to hydrate EmbeddingCache from recorded embedding.responded events. Strict replay enables the cache automatically, verifies the request-hash sequence, and never calls the provider.

embedding.requested records the model and a content hash, not the source text. embedding.responded records validated ordered vectors. Calling rt.embedding_provider.embed(...) directly remains possible for compatibility but bypasses both events and the cache, so it forfeits ActiveGraph replay guarantees.

The runtime ships exactly one implementation: HashEmbeddingProvider, a deterministic, dependency-free test double (token-hash buckets, L2-normalized). Its vectors reflect token overlap, not semantics — use it to test embedding plumbing offline; wire a real provider (OpenAI embeddings, Voyage, a local model) for retrieval quality. Real embedding implementations are deliberately not shipped in the runtime: no network dependencies, no API keys.

Mixing with `RecordedLLMProvider`¶

The fixture-backed provider is provider-agnostic: fixtures are keyed by prompt-content hash, and the model name (claude-… or gpt-…) is part of the hash input. Fixtures recorded against one provider replay against RecordedLLMProvider regardless of which live provider you switch to next.

from activegraph.llm import RecordingLLMProvider, OpenAIProvider

inner = OpenAIProvider()
provider = RecordingLLMProvider(inner, fixtures_dir="tests/fixtures/llm")

RecordingLLMProvider wraps either concrete provider the same way. Record once against a live key, commit the fixtures, run tests against RecordedLLMProvider thereafter.

Writing a custom provider¶

LLMProvider is a runtime-checkable Protocol. Any class with the three methods is a provider — no inheritance required, no registration step:

from decimal import Decimal
from activegraph.llm import LLMMessage, LLMResponse, LLMProvider

class MyProvider:
    default_model = "my-model-name"   # v1.0.2 #1 — used when @llm_behavior omits model=

    def complete(self, *, system, messages, model, max_tokens,
                 temperature, top_p, output_schema, timeout_seconds,
                 tools=None, structured_output_mode="prompt") -> LLMResponse:
        ...

    def estimate_cost(self, *, input_tokens, output_tokens, model) -> Decimal:
        ...

    def count_tokens(self, *, system, messages, model) -> int:
        ...

    def recognizes_model(self, name: str) -> bool:  # v1.0.2 #1
        return name.startswith("my-")

    def supports_native_structured_output(self, model: str) -> bool:  # v1.3 #1
        return False

assert isinstance(MyProvider(), LLMProvider)

default_model, recognizes_model, and supports_native_structured_output are additive (v1.0.2 #1 / v1.3 #1). Custom providers that omit them keep working at every call site — the runtime guards each lookup with getattr(...) — they just require an explicit model= on every @llm_behavior, don't participate in cross-provider validation, and resolve to the prompt-embedded structured-output path. (The isinstance check above requires the full current method set; the runtime itself never isinstance-checks providers.)

If your provider exposes the framework's instruction-based structured-output path (most do), reuse parse_structured_response(text, schema) from activegraph.llm.parsing for byte-identical error semantics with the shipped providers — same llm.parse_error and llm.schema_violation reason codes for the same response shapes.

See CONTRACT v1.0.1 #5 for the provider-commitment surface: which methods are stable and which behaviors are provider-dependent (count_tokens). Native structured-output mode is specified in CONTRACT v1.3 #1.