"""A task for the model: rows that are waiting, one call per chunk, answers written back. The machine tagging and summarising both run in. `agent.py` answers *how do I reach a model* — the API, or the local `claude`. This module answers the layer above: *what do I send it, and what do I do with what comes back*. Everything between those two lives here, once. ModelTask the parts: waiting, build_batch, system, schema, apply, missed one one batch: one model call, one apply. What a job is. of that as a `Task`: the batches, and how to find them run every batch here and now, for a caller with no queue Tagging and summarising used to be near-twins — the same loop, the same failure handling, the same job body and the same resume, spelled out twice and drifting by a line each merge. What actually differs between them is small and is all *content*: which rows are waiting, what goes in the batch, what the model is told, what shape it must answer in, what to do with the answer. That is a `ModelTask`; this module is everything that is not one. The third one — clustering, or whatever comes next — is a prompt, a schema and two functions over the store, and no loop. """ from __future__ import annotations import logging from collections.abc import Callable, Sequence from dataclasses import dataclass from typing import Any from desk.config import HasAgent, ModelTaskConfig from pipeline.context import NULL, RunContext from pipeline.task import Batch, Tally, Task, resolve from desk.models import ModelTaskResult, Notice from desk.notices import classify, stops_the_run from desk.tools import agent log = logging.getLogger(__name__) #: `notices.py`'s code for "the model's safeguards would not take this text". #: Named here because `one` has to recognise it by more than its wording: it #: is the only failure that says something about a particular row, and so the #: only one worth asking about again with fewer rows. REFUSED = "agent.refused" @dataclass(frozen=True, slots=True) class ModelTask[R: ModelTaskResult]: """What one task for the model consists of. See tagging.TASK for one.""" #: The job kind. kind: str label: str #: For progress lines and the idle result: "tagging" / "tags" / #: "nothing untagged". verb: str noun: str idle: str #: The key under which `build_batch` lists its rows. Both of newsdesk's #: say "items", and they used to say "articles" and "segments": a batch is #: a mixed archive and naming it after one shape was a claim about the #: rows in it that stopped being true. items: str #: Which config section governs it. settings: Callable[[Any], ModelTaskConfig] #: How many rows are waiting: `(config, store)`. pending: Callable[[Any, Any], int] #: Which rows are waiting, by uid, most recent first: `(config, store, limit)`. #: What the task splits over — one batch per model call — so it asks for #: identifiers rather than the payload, which each batch builds for itself #: when it runs. waiting: Callable[[Any, Any, int], list[str]] #: The payload the model works from, its prompt included: #: `(config, store, limit, uids)`. Given uids it builds that batch, and #: asks the store again rather than trusting the list — a row may have been #: done in the seconds since the task shared the work out. build_batch: Callable[..., dict[str, Any]] # All three take the config because *which rows are waiting* is no longer a # question the store can answer on its own: summarising is governed by what # each installed plugin declares about its own items, and the plugins are # found through the config. A task that does not care ignores the argument. #: The system prompt, given the store rather than fixed, because tagging's #: carries the vocabulary and that grows between one run and the next. system: Callable[[Any], str] schema: dict[str, Any] #: Write an answer back: `(store, answer, batch, config)`. Tolerant of a #: sloppy response, and the only place the answer's shape is trusted. apply: Callable[[Any, dict[str, Any], dict[str, Any] | None, Any], R] #: `(store, uids, why="")`: the model was handed these rows and said #: nothing usable about them. Count that against each one still waiting, #: and answer how many of them that has now given up on. `why` is #: `refused` when the safeguards turned the row down, so the Given up list #: can say so. #: #: Every task needs one and there is no default, because the default is the #: bug. A row the model cannot satisfy — a segment whose only honest #: abstract is shorter than the guard, a uid it drops from every reply — is #: otherwise found by the next `waiting`, sent again, and paid for again, #: on every pull, for as long as the row exists. Where the count is kept is #: the store's business; that there is a bound at all is this module's. missed: Callable[..., int] #: The result type, so totals start as the right kind of zero. result: type[R] #: `module:function` run once when every batch of the run is over, given #: `(config, store, tally)` and the run context. "" for a task with #: nothing to settle, which summarising is: an abstract is written for one #: segment by the batch that read it, and no other batch has an opinion. #: #: For the decision a batch is in no position to make. Tagging's new tags #: are the case: the cap is *per run*, and a batch that can see neither #: the other thirty-nine nor what they proposed can only enforce it #: forty times over. So a batch reports what it wants, and this is where #: the run's own answer is worked out — once, over the whole tally. #: #: It returns the fields of `result` that settling added, and both paths #: take it from there: queued, the final job reports it; direct, `_fold` #: absorbs it into the total. Same function either way. settle: str = "" #: The field every answer in the reply names its row by. Both tasks say #: `uid`, which is not a coincidence — an answer that cannot say which row #: it is about is not applicable — and counting it in the streamed reply is #: what moves the progress bar per item rather than per batch. A task whose #: answers are not one-per-row sets it to "" and gets the old behaviour. answer_key: str = "uid" class _Answered: """Counts finished answers in a reply that is still arriving. The model writes its answers one after another, so the `"uid"` that opens each one is a reliable "another row is done" marker tens of seconds before the request that carries it completes. Counting a substring is a blunt instrument and deliberately so: the alternative is incremental JSON parsing, which is a great deal of machinery to be *approximately as* right about a number that only drives a bar. It cannot overcount — the key appears once per answer, and the payload it is counted in is the model's own output, not the batch we sent — and if it undercounts, the bar catches up when the chunk lands and the real total is known. `tail` is why this is a class rather than a closure: a key can be split across two deltas, so the last few characters of each are carried into the next. Carrying `len(needle) - 1` is exactly enough to catch one that spans the join, and too few for anything already counted to be seen twice. """ def __init__(self, key: str, report: Callable[[int], None]) -> None: self._needle = f'"{key}"' self._report = report self._tail = "" self.count = 0 def __call__(self, piece: str) -> None: window = self._tail + piece found = window.count(self._needle) self._tail = window[1 - len(self._needle):] if len(self._needle) > 1 else "" if found: self.count += found self._report(self.count) def one[R: ModelTaskResult](task: ModelTask[R], config: HasAgent, store: Any, uids: Sequence[str], *, context: RunContext = NULL) -> R: """One batch: one model call, one apply. What a job is. The unit the queue admits. It used to be a whole backlog inside a single job, which opened four threads of its own — so the queue counted one thing running where four requests were in the air, and four model workers was a number nobody was keeping. Now the job and the request are the same thing and the count is true. The prompt is built here rather than when the batch was handed out: for tagging that carries the vocabulary as it stands when this batch starts, which is what somebody adding a tag by hand mid-run should reach. What it does *not* carry is what a sibling batch proposed a second ago — nothing of this run's proposals exists yet, because they are settled once, when every batch is over. See `settle`. Raises `agent.AgentError` when no model can be reached, or when the one that could gave nothing. The job fails, saying which — and the notice says what to do about it, because both of those are things you fix rather than things to work around. A row the model answers is done with. One it leaves out, or answers in a way `apply` will not take, is charged a turn against `task.missed` — so it comes round again a bounded number of times rather than on every pull for as long as it exists. A batch the model's safeguards refuse outright is split and asked again in halves rather than failed, because a refusal is about the text of one row and the other rows in it are owed their answers. Down to a single row it is charged like any other unanswered one and the run carries on. See `_narrow`. It counts its own rows and nothing else. As a job that is all there is; run through `Task.run` instead, the caller shifts the progress into the whole backlog, which is the only place that knows how big it is. """ context.check() cfg = task.settings(config) batch = task.build_batch(config, store, len(uids) or cfg.batch_size, list(uids)) rows = batch[task.items] if not rows: # Done by someone else between the task sharing the work out and # now, which is ordinary: # every task rediscovers its backlog and the batches overlap at the # edges. Nothing to ask, nothing to write. return task.result(reason=task.idle) name = agent.backend(config.agent) if not name: # The notice goes out from here rather than being left to the failure, # because nothing was asked and so the `except` below never runs. Left # to itself this reached the screen only as "Tagging a batch failed", # filed as a bug, about a machine that is working correctly and is one # install or one config line from working here too. The configured # backend is what is passed, not the one that ran: there wasn't one, # and `off` is the difference between a missing install and a switch. unreachable = agent.AgentError("no model is reachable from here") _report(context, classify(unreachable, backend=config.agent.backend)) raise unreachable total = len(rows) def report(done: int) -> None: done = min(done, total) context.progress(done, total, f"{task.verb} {task.items}, {done} of {total}") context.progress(0, total, f"asking {name} about {total} {task.items}") counted = (_Answered(task.answer_key, report) if task.answer_key else None) try: context.check() reply = agent.ask_json( config.agent, system=task.system(store), question={task.items: rows}, schema=task.schema, model=config.agent.model or cfg.model, thinking=cfg.thinking, using=name, on_partial=counted, ) except agent.AgentError as exc: notice = classify(exc, backend=name) # A refusal is the one failure that is about the rows rather than # about the call, and so the one that answers "which of them?" by # being asked again with fewer. Everything else falls through to the # raise below unchanged. if notice.code == REFUSED and len(rows) > 1: return _narrow(task, config, store, _uids(task, rows), context=context) log.exception("%s batch of %d failed on %s", task.verb, len(rows), name) # The notice carries the sentence and the fix — "Claude Code is signed # out" and the command that mends it, rather than the token that # expired — to where you will see it. Then the batch fails: its rows # keep their watermark and come round on the next run — uncharged, # because nothing was asked of a model. # # Its siblings are unaffected only while the failure was this batch's. # When it is the account's, `_report` stops the ones still queued: see # `notices.stops_the_run`. if notice.code == REFUSED: # Down to one row, so the notice can name it — and has to. "The # safeguards wouldn't take this text" is not a sentence anyone can # act on without knowing which of the day's articles it is about, # and the row is about to leave the backlog quietly. `key` is the # title too, so two refusals in one run are two lines rather than # one line with a count of two. about = rows[0].get("title") or _uids(task, rows)[0] notice = notice.model_copy(update={ "text": f"{notice.text} — {about}", "key": str(about)}) _report(context, notice) if notice.code == REFUSED: # Down to one row, and that row is the one. Charged and not # raised: the rule about never charging a failure is a rule about # failures that prove nothing, and this one has just proved # exactly which row it is about. Left uncharged it is found by # the next `waiting`, sent again, refused again, and poisons # whichever batch it lands in for as long as the row exists — # which is how two articles held eighteen others' abstracts. totals = task.result(reason=notice.text) totals.failed = 1 totals.given_up = task.missed(store, _uids(task, rows), why="refused") totals.pending = task.pending(config, store) return totals raise # Apply even if killed during the call: the spend has happened. The worker # records cancellation at return while preserving the paid-for answer. totals = task.apply(store, reply.data, batch, config) # Whatever is still waiting now is what the model was asked about and did # not settle, so it is charged one turn — before `pending` is read, so a # row this call gave up on has already left the number the run reports. # Charged on a reply and never on a failure: a batch that never reached a # model cost nothing and proved nothing about its rows. totals.given_up = task.missed(store, uids) totals.batches = 1 totals.backend = reply.backend totals.pending = task.pending(config, store) report(total) return totals def _uids(task: ModelTask[Any], rows: Sequence[dict[str, Any]]) -> list[str]: """Which rows a built batch is about, from the batch itself. `one` is given uids, but what it sent is `rows` — `build_batch` asks the store again and drops whatever was finished in between. On the path this serves, that difference is the whole point: a row nobody sent is a row nobody may charge. """ return [str(row[task.answer_key]) for row in rows if row.get(task.answer_key)] def _narrow[R: ModelTaskResult](task: ModelTask[R], config: HasAgent, store: Any, uids: Sequence[str], *, context: RunContext) -> R: """Ask about these rows again in two halves, and add up what comes back. A refusal names no row. It is one answer about the whole request, so the only way to learn which row it is about is to send fewer — and the reason to bother is that the rest of the batch is fine and is owed its abstracts. The split costs a handful of extra calls, once, against one call that summarises nobody and is made again on every pull. Halves rather than one row at a time, because a clean half lands whole: the cost is set by how many rows are flagged rather than by how many are in the batch, and two flagged rows in one batch is ordinary — they arrive from the same day's news. Each half is a whole `one`, so a half that meets an unrelated failure raises it exactly as it would have unsplit, and the batch fails the way it always did. The bad case, if the classifier ever took against the *prompt* rather than any row: every single-row call is refused, every row is charged, and three pulls empty the backlog into given-up. Bounded and visible — the status line counts them and `reopen_given_up` asks again — but it is why the notice names the text rather than the row. """ middle = len(uids) // 2 totals = task.result() for half in (uids[:middle], uids[middle:]): if not half: continue landed = one(task, config, store, half, context=context) totals.absorb(landed) # `absorb` leaves the fields that are about the run rather than the # chunk, and which backend answered is one of them — the same reading # `_fold` takes of the same shape. totals.backend = landed.backend or totals.backend totals.pending = task.pending(config, store) return totals def _report(context: RunContext, notice: Notice) -> None: """Put one failure on screen, and stop the run if it will meet the same one. The two halves belong together because they are one judgement made once. A batch that cannot reach a model, or is told the spend limit is reached, has learned something about the *account* and not about its own rows — and the twenty-two batches queued behind it would each pay a call to be told it again, and each put the same line on screen. One batch fails, one notice goes out, and the rest are cancelled rather than spent. Which failures those are is the catalogue's to say and not this module's: a timeout or a reply that was not JSON is this batch's bad luck, and the next batch deserves its turn. See `notices.stops_the_run`. """ context.notice(notice) if stops_the_run(notice): context.stop(notice.text) # ----------------------------------------------------------------- the task def batches(task: ModelTask[Any], limit: int | None = None) -> Any: """The finder: what is waiting, cut into one payload per model call. How many to a call is the task's own business — twenty-five articles is what one call and one schema hold, and no generic queue could know it. How many run at once is the queue's, because that is how many workers it has. It asks for identifiers rather than the rows, and each batch asks the store again when it runs: a row may have been done in the seconds since the work was shared out. `enabled: false` is answered here, and answered as nothing to do rather than as a refusal. Here, because this is the one place every route to the work passes through: the stage of a cascade, the button, and `run("tag")` are three callers of this same finder, and a switch only one of them read would be a switch that turned the task off in one place. As nothing to do, because a task that finds nothing still queues its final job — so a cascade whose tagging is off carries on to whatever follows rather than ending on it. """ def find(config: HasAgent, store: Any) -> list[Batch]: cfg = task.settings(config) if not cfg.enabled: return [] uids = task.waiting(config, store, limit if limit is not None else 100_000) size = max(cfg.batch_size, 1) cut = [uids[at:at + size] for at in range(0, len(uids), size)] return [Batch({"uids": chunk}, label=_label(task, len(chunk), n, len(cut))) for n, chunk in enumerate(cut, start=1)] return find def _label(task: ModelTask[Any], rows: int, n: int, of: int) -> str: """What one batch is called in the queue: "Tagging 25 items, 2 of 9". The kind's own label is "Tagging a batch" on all nine of them, which is true and useless — the queue lists nine identical rows and nothing says which is which or how much is left. The slice is the only thing that tells them apart, and this is the one place it is known: the finder cut them, and the batch that runs half an hour later knows only its own uids. Its shape is the progress line's, capitalised because it is a name rather than a sentence about what is happening. The tail is dropped for a task of one batch, where "1 of 1" says nothing. """ said = f"{task.verb} {rows} {task.items}".capitalize() return said if of < 2 else f"{said}, {n} of {of}" def _fold(task: ModelTask[Any]) -> Any: """Add the batches up, for the direct path only. Under the queue each batch is its own job with its own row, and there is nothing to add. The settle runs here too, off the same tally shape the final job builds, because "what may this run add" is a question about the run and not about the queue. Otherwise `run("tag")` in a notebook would be the one caller with no cap at all. """ def fold(config: HasAgent, store: Any, results: list[Any]) -> dict[str, Any]: if not results: return task.result(reason=task.idle).model_dump() totals = task.result() for result in results: if isinstance(result, Exception): # One batch that could not be answered is not a failed run: as # eight jobs it would be one failing and seven landing. Its rows # keep their watermark and come round next time. totals.failed += 1 totals.reason = str(result) continue landed = task.result.model_validate(result) totals.absorb(landed) # `absorb` adds up counts and leaves the fields that are about the # run rather than the batch. Which backend answered is one of # those, and the run wants the last one to have said anything. totals.backend = landed.backend or totals.backend if task.settle: answered = tuple(r for r in results if isinstance(r, dict)) settled = resolve(task.settle)( config, store, Tally(done=len(answered), results=answered), context=NULL) or {} totals.absorb(task.result.model_validate(settled)) # After the settle, so a tag it added — and the rows it went back and # tagged — are counted in what the run reports it did. totals.pending = task.pending(config, store) return totals.model_dump() return fold def of(task: ModelTask[Any], *, limit: int | None = None, name: str = "", callback: str = "", carries: dict[str, Any] | None = None) -> Task: """The `Task` for one model task: its batches, and how to find them. A `ModelTask` is the *content* — which rows are waiting, what the model is told, what to do with the answer. This is that content as a step of the pipeline, which is the same shape as every other step and is queued and killed the same way. """ return Task( kind=f"{task.kind}_batch", name=name or task.kind, finder=batches(task, limit), callback=callback, settle=task.settle, carries=dict(carries or {}), size=lambda payload: len(payload.get("uids") or ()), fold=_fold(task), ) def run[R: ModelTaskResult](task: ModelTask[R], config: HasAgent, store: Any, limit: int | None = None, *, context: RunContext = NULL) -> dict[str, Any]: """Every batch, here and now, with no queue. The direct path and nothing else. What a Claude Code session, a notebook and a test get: one call that blocks and hands back the total. Under the queue this same task is `of(...)` started against the queue, its batches are independent jobs, and there is no total until the final job reads how they went — which is the whole point of splitting it. Same `one` either way, and the same batches from the same finder, so there is no second implementation to drift. """ return of(task, limit=limit).run(config, store, context=context)