Bring in desk from news

The plugin contract, the model task and the tools roledesk imports, copied
rather than reached through an editable install of news, which also exposes
a package named pipeline. news stays the source of truth for desk.

Refreshed from news/desk at a275c38a1aa0fe80a7ec7dd5781ecccf683f71c6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Opus 5.5 committed 2026-09-23 12:29:17 +02:00
1 parent 9d666d0937
commit 248ea1636d
36 files changed
+8393

No files matched your search

+161
View File
@@ -0,0 +1,161 @@
"""The settings sections an app composes into its own config.
`Section` is the base: a block of YAML validated into a model, with an
unrecognised key being an error rather than a silent no-op. `AgentConfig`
is how a model is reached and `ModelTaskConfig` is how a task for one is
paced — both read by the pipeline, so both live here rather than in any one
app's config. An app declares its own `Config(Section)` with these as fields
and whatever else it needs next to them.
"""
from __future__ import annotations
from typing import Any, Literal, Protocol, get_args, get_origin
from pydantic import BaseModel, ConfigDict, Field, model_validator
#: The bare words YAML 1.1 resolves to each boolean, in the order a field gets
#: asked about them. See `Section._spell_yaml_booleans_back`.
_YAML_WORDS = {False: ("off", "no", "false"), True: ("on", "yes", "true")}
def _literal_strings(annotation: Any) -> set[str]:
"""Every string a `Literal` annotation accepts, through any wrapping."""
if get_origin(annotation) is Literal:
return {a for a in get_args(annotation) if isinstance(a, str)}
return {s for arg in get_args(annotation) for s in _literal_strings(arg)}
class Section(BaseModel):
"""A block of settings.
`extra="forbid"` is the whole point: an unrecognised key is a mistake — a
typo, or a setting that was renamed and never updated — and saying so is
more useful than ignoring it. `protected_namespaces=()` because pydantic
reserves the `model_` prefix for its own methods and two settings here are
honestly named `model` and `model_weight`.
"""
model_config = ConfigDict(extra="forbid", protected_namespaces=())
@model_validator(mode="before")
@classmethod
def _spell_yaml_booleans_back(cls, data: Any) -> Any:
"""Let a setting whose value is spelled `off` be written as `off`.
YAML 1.1 reads the bare words `off no false on yes true` as booleans,
so `backend: off` arrives here as `False` and fails against a `str`
field — with an error that helpfully lists `off` as one of the values
it will accept. The documented spelling being the one spelling that
doesn't work is not a thing to answer with a note in the example file;
`agent.backend: off` is what the example and the module that reads it
both tell you to write.
So a boolean landing on a field whose `Literal` lists a word for that
boolean is read as the word. Nothing else moves: `off` on a field that
has no `off` to offer is still an error, and a real `bool` field is
untouched, because `True` and `False` are what it asked for.
"""
if not isinstance(data, dict):
return data
spelled = data
for name, value in data.items():
if not isinstance(value, bool):
continue
field = cls.model_fields.get(name)
if field is None:
continue
choices = _literal_strings(field.annotation)
word = next((w for w in _YAML_WORDS[value] if w in choices), None)
if word is not None:
# A copy, because this dict is the caller's — `load` hands the
# parsed YAML straight in, and reading a file shouldn't edit it.
spelled = dict(spelled) if spelled is data else spelled
spelled[name] = word
return spelled
class AgentConfig(Section):
"""Who answers when a job needs a model. See desk/tools/agent.py."""
#: `auto` prefers the API when there are credentials and otherwise drives
#: the `claude` CLI on your subscription through the Claude Agent SDK.
#: Naming one skips the detection; `off` means no model is reachable, so
#: every batch that needs one fails with a notice saying which switch did
#: it. A stage whose batches all fail still reports and the cascade
#: carries on — turning the model off stops the model, not the pull.
backend: Literal["auto", "api", "claude_code", "off"] = "auto"
#: Overrides the per-task model everywhere. Empty means each job picks its
#: own — tagging asks for Haiku, because it is Haiku work.
model: str = ""
#: Seconds for one call, either backend. The Claude Code path spawns a
#: process and is the slower of the two by a few seconds.
timeout: float = Field(300.0, gt=0)
max_tokens: int = Field(8192, ge=1)
#: Where the `claude` binary is, if it isn't on PATH.
cli_path: str = ""
credentials_database: str = Field("", exclude=True, repr=False)
class ModelTaskConfig(Section):
"""What every task for the model is governed by. See desk/modeltask.py."""
#: Off is off everywhere: the pull, the button and `run("tag")` alike all
#: find nothing to do and say so. Not a refusal — a task that is off has
#: an empty backlog, so a cascade carries straight on past it.
enabled: bool = True
#: Whether a *pull* runs it, and nothing else — the button never reads
#: this. `auto` runs it during a pull when `agent.backend` resolves to
#: something and skips it quietly when it does not, so a machine with no
#: model pulls in silence rather than filing a failure per batch about
#: being set up the way it was meant to be. `always` runs it either way,
#: and a missing backend fails each batch saying so. `never` keeps it out
#: of the pull altogether, leaving the button.
on_ingest: Literal["auto", "always", "never"] = "auto"
model: str = ""
#: Whether the model reasons before it answers. Per *task* rather than per
#: backend on purpose: both backends take the same shapes, so how hard to
#: think is a property of the work, not of how the model is reached.
#:
#: `off` by default because neither task here is thinking work and the cost
#: is not marginal — tagging twenty-five headlines measured 128s with
#: thinking and 17s without, for tags that came back the same. Turn it on
#: for a task whose answer is a judgement rather than a lookup.
#:
#: `effort` is deliberately not offered. The CLI has it and the Messages
#: API does not, and a setting that silently means nothing on one backend
#: is exactly the asymmetry this seam exists to avoid.
thinking: Literal["off", "adaptive"] = "off"
#: Rows per model call — how the task slices its own backlog, and the one
#: number here that is genuinely the task's. Twenty-five is what one call
#: and one schema hold; it is prompt economics, and no generic queue could
#: know it.
#:
#: How many run at once is not here and should not be: that is how many
#: workers there are, and two tasks that both want a model share them
#: rather than each helping itself. See `workers` on the app's config.
batch_size: int = Field(25, ge=1)
class WorkersConfig(Section):
"""How many workers of each kind the queue has.
A property of the machine and the subscription, not of any one task, which
is why it is here and not next to `batch_size`. `llm` is what one Claude
subscription answers without queueing behind itself; `http` is wider
because a fetch is mostly waiting; `cpu` is one because the ranker and the
embedder want the machine. A resource nobody names gets one worker.
"""
llm: int = Field(4, ge=1)
http: int = Field(8, ge=1)
cpu: int = Field(1, ge=1)
class HasAgent(Protocol):
"""What the pipeline asks of an app's config: that it has an `agent`
section. `modeltask` reads nothing else from it — the rest is handed to
the task's own `settings` callable, which knows the app."""
@property
def agent(self) -> AgentConfig: ...