Files
jobboard/docs/llm-backends.md
T
WolfandClaude Opus 5.5 9d666d0937 Bring in pipeline, auth, deploy and docs from the boilerplate
roledesk is built on the boilerplate from here on: pipeline 2.0 (queue and
scheduler), auth (one password, sessions, passkeys), the deploy kit and its
docs, plus the coupling and packaging tests. Downstream copies, unedited.

Refreshed from boilerplate/d56949f7d0f6c0bfe5745478e09007bfad67ae87

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 12:29:16 +02:00

242 lines
12 KiB
Markdown

# Asking Claude, from a deployed app
One question — *ask Claude this, get JSON back* — and two ways to answer it, chosen by which credential is present in the environment. The caller asks without knowing which one ran.
api One call to the Claude API. Wants ANTHROPIC_API_KEY. No local
install, no subprocess, per-token billing. The unattended path.
claude_code The Claude Agent SDK drives the `claude` CLI, on the
subscription it is signed in with, authenticated by
CLAUDE_CODE_OAUTH_TOKEN. Nothing billed per token.
This is the reusable part of the deployment: the compose file passes both variables through, the app prefers the key when there is one, and switching from the subscription to the API is a line in `.env` and `docker compose up -d` — no rebuild, no code change.
## The two credentials
**`CLAUDE_CODE_OAUTH_TOKEN`** comes from `claude setup-token`, run interactively on a machine you are already signed in on. It lasts a year. Generating it needs browser authorization, so no deployment script can automate it — you produce it on your laptop and paste it into the server's `.env` with an editor. Then `docker compose up -d app`, because a `restart` will not pick it up.
**`ANTHROPIC_API_KEY`** is per-token billing from the Console. Nothing to renew, nothing to sign in to, and it is what production should run on once the thing is real.
Set one. If both are set, `auto` takes the key. Neither, and every call fails with a notice naming the reason, which is the correct outcome — better a batch that fails saying "no credentials" than one that waits for a human who is not there.
The API key and the OAuth token are not interchangeable and do not substitute for each other. `ANTHROPIC_API_KEY` in the environment does not make the CLI work, and an OAuth token does not authenticate an API call.
**Neither belongs in the image or in the compose file.** They live in the `.env` beside it, `chmod 600`, injected as environment variables at container creation. A compose file gets pasted into chat windows; a file you had to `cat` deliberately does not.
## Why there is a `backend` setting and not just detection
Detection answers "what can run here". The setting answers "what should". They differ in two cases worth naming: forcing `api` on a machine that also has the CLI, so a benchmark measures the thing you meant; and `off`, which makes everything needing a model fail on purpose, with a notice that says which switch did it. An app that silently degrades to no-model is worse than one that stops.
## What the `claude_code` path deliberately does not get
No tools (`tools=[]`, `allowed_tools=[]`), no filesystem settings (`setting_sources=None`, so no CLAUDE.md, no skills, no MCP servers leak into a classification call), no session to resume. One text-in, JSON-out turn — the same shape as the API call it stands in for. The CLI is built for agent work, and left to its defaults it will bring all of that with it.
## Thinking has to be stated, not defaulted
The CLI thinks unprompted; the Messages API thinks only when asked. So "say nothing about thinking" means opposite things on the two sides, and the two backends stop being interchangeable. State it explicitly to both. It is not a small difference: on a batch of twenty-five short classifications, the CLI's default cost 128 s against 17 s for the same answers.
Some models support manual extended thinking only and reject `adaptive`. Check before spending a call, not after.
## Both take the same JSON schema
The Agent SDK accepts the Messages API's `json_schema` output format and hands the parsed object back on `ResultMessage.structured_output`. That is what makes this two backends rather than two implementations: neither path is the degraded one, and moving between them is a config line.
## Reference implementation
Self-contained — no imports from your project, both SDKs optional. It is the shape, not the whole thing: the real one in newsdesk also streams partial answers out for a progress bar, retries, and classifies API errors by status code so a 401 and a 429 tell you different things.
```python
"""Ask Claude a question, constrained to a JSON schema, get a dict back.
Two backends, one seam. `ANTHROPIC_API_KEY` selects the API; otherwise the
Claude Agent SDK drives the `claude` CLI on a subscription, authenticated by
`CLAUDE_CODE_OAUTH_TOKEN`. Both are optional dependencies:
uv pip install anthropic # the api backend
uv pip install claude-agent-sdk # the claude_code backend
"""
from __future__ import annotations
import asyncio
import importlib.util
import json
import os
from dataclasses import dataclass
from typing import Any
class AgentError(Exception):
"""The backend was there and the call still did not produce an answer."""
@dataclass(slots=True)
class AgentConfig:
#: "auto", "api", "claude_code", or "off" to make every call fail.
backend: str = "auto"
model: str = "claude-haiku-4-5-20251001"
timeout: float = 300.0
max_tokens: int = 8192
#: "off" or "adaptive". Stated to both backends, never defaulted.
thinking: str = "off"
@dataclass(slots=True)
class Reply:
data: dict[str, Any]
backend: str
model: str
usage: dict[str, Any]
def backend(settings: AgentConfig) -> str:
"""Which backend will answer, or "" for nobody."""
if settings.backend == "off":
return ""
if settings.backend in ("api", "claude_code"):
return settings.backend
if any(os.environ.get(name) for name in
("ANTHROPIC_API_KEY", "ANTHROPIC_AUTH_TOKEN", "ANTHROPIC_PROFILE")):
return "api"
# The SDK bundles its own `claude`, so having the package is enough. Not
# proof the call will work — the CLI may be signed out — but that surfaces
# as an AgentError on the call, which the caller handles anyway.
if importlib.util.find_spec("claude_agent_sdk") is not None:
return "claude_code"
return ""
def _thinking(setting: str) -> dict[str, str]:
return {"type": "adaptive"} if setting == "adaptive" else {"type": "disabled"}
def ask_json(settings: AgentConfig, *, system: str, question: dict[str, Any],
schema: dict[str, Any]) -> Reply:
"""Ask `question`, constrained to `schema`, and return the parsed answer.
`system` should be byte-identical across the calls in a run — it is what
the API path caches. `question` is the part that changes.
"""
name = backend(settings)
if not name:
raise AgentError("no model backend is available: set ANTHROPIC_API_KEY "
"or install claude-agent-sdk with CLAUDE_CODE_OAUTH_TOKEN")
if name == "api":
return _ask_api(settings, system, question, schema)
return _ask_claude_code(settings, system, question, schema)
def _ask_api(settings: AgentConfig, system: str, question: dict[str, Any],
schema: dict[str, Any]) -> Reply:
try:
import anthropic
except ImportError as exc:
raise AgentError("the anthropic SDK isn't installed") from exc
try:
client = anthropic.Anthropic(timeout=settings.timeout)
response = client.messages.create(
model=settings.model,
max_tokens=settings.max_tokens,
# The instructions are byte-identical across every call; only the
# payload below changes. Caching them is most of the saving.
system=[{"type": "text", "text": system,
"cache_control": {"type": "ephemeral"}}],
messages=[{"role": "user", "content": json.dumps(question)}],
thinking=_thinking(settings.thinking),
output_config={"format": {"type": "json_schema", "schema": schema}},
)
except anthropic.APIError as exc:
# Carry the status through in the text: 401 ("your key is wrong") and
# 429 ("slow down") want different reactions, and the message body
# alone does not always separate them.
status = getattr(exc, "status_code", "")
raise AgentError(f"the API call failed: {status} {exc}".strip()) from exc
text = next((b.text for b in response.content if b.type == "text"), "")
return Reply(data=json.loads(text), backend="api",
model=response.model or settings.model,
usage=response.usage.model_dump() if response.usage else {})
def _ask_claude_code(settings: AgentConfig, system: str, question: dict[str, Any],
schema: dict[str, Any]) -> Reply:
try:
from claude_agent_sdk import (
ClaudeAgentOptions, ClaudeSDKError, ResultMessage, query,
)
except ImportError as exc:
raise AgentError("the Claude Agent SDK isn't installed") from exc
options = ClaudeAgentOptions(
system_prompt=system,
model=settings.model or None,
# A classification call, not an agent: nothing to run, nothing to read,
# and no project settings — so no CLAUDE.md, no skills and no MCP
# servers get in front of the model on the way to an answer.
tools=[],
allowed_tools=[],
permission_mode="dontAsk",
setting_sources=None,
thinking=_thinking(settings.thinking),
output_format={"type": "json_schema", "schema": schema},
# The API takes max_tokens as a parameter; the CLI takes it only as an
# environment variable. Without this the setting is quietly API-only.
# The SDK merges this into the inherited environment rather than
# replacing it, so naming one variable does not cost the CLI its PATH.
env={"CLAUDE_CODE_MAX_OUTPUT_TOKENS": str(settings.max_tokens)},
)
async def ask() -> Reply:
answer: Reply | None = None
async for message in query(prompt=json.dumps(question), options=options):
if not isinstance(message, ResultMessage):
continue
if message.is_error:
raise AgentError("; ".join(message.errors or []) or message.result
or f"the agent ended as {message.subtype}")
data = message.structured_output
if not isinstance(data, dict):
# Ended without structured output — a limit, an interruption.
# `result` is the raw text if there is any, worth one attempt.
data = json.loads(message.result or "")
answer = Reply(data=data, backend="claude_code",
model=settings.model, usage=message.usage or {})
if answer is None:
raise AgentError("the agent returned no result")
return answer
try:
# asyncio.run, so this is a synchronous call like the other backend.
# From inside a running loop, await `ask()` directly instead.
return asyncio.run(asyncio.wait_for(ask(), settings.timeout))
except TimeoutError as exc:
raise AgentError(f"no answer within {settings.timeout:.0f}s") from exc
except ClaudeSDKError as exc:
# The SDK has its own exception hierarchy — a CLI that exited, a line
# it could not parse, an answer that overran max_tokens — and none of
# it is an AgentError. Left alone, those escape a caller that catches
# AgentError per item and take down a whole run. Every failure of this
# backend is an AgentError or none of them are.
raise AgentError(str(exc)) from exc
```
Using it:
```python
settings = AgentConfig()
schema = {
"type": "object",
"properties": {"tags": {"type": "array", "items": {"type": "string"}}},
"required": ["tags"],
}
reply = ask_json(settings, system="Tag this article. Answer with the schema.",
question={"title": title, "body": body}, schema=schema)
print(reply.data["tags"], reply.backend)
```
## The two failures that will actually happen
**A changed token in `.env` appears to do nothing.** `docker compose restart` does not update environment variables. `docker compose up -d app` recreates the container, which does.
**The token expired and nothing said so.** The setup token lasts a year, and the failure looks like a model call failing rather than a credential problem. Read the error text — this is why the SDK's own message is passed through intact rather than summarised into "model call failed".