roledesk is built on the boilerplate from here on: pipeline 2.0 (queue and scheduler), auth (one password, sessions, passkeys), the deploy kit and its docs, plus the coupling and packaging tests. Downstream copies, unedited. Refreshed from boilerplate/d56949f7d0f6c0bfe5745478e09007bfad67ae87 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
137 lines
6.5 KiB
Python
137 lines
6.5 KiB
Python
"""Turning a failure into one line worth putting on screen.
|
|
|
|
The whole of this module's job is a handoff. Something broke; the log has the
|
|
traceback and always will; what the log cannot do is *tell you it happened*
|
|
while you are looking at the app wondering why nothing appeared. So a notice
|
|
is pushed to whoever is watching — and, since nobody usually is, written down.
|
|
|
|
**It is stored, and it was not always.** The first design kept nothing: a
|
|
notices table would be a second record of what the log already records, with
|
|
rows that go stale and a dismiss button to maintain. That argument assumed
|
|
somebody was watching when the notice was raised, and somebody who could read
|
|
a server's log when they were not. Neither holds for a pull that runs on a
|
|
schedule and a reader holding a phone. What it cost was three things, each
|
|
reported as a defect of its own:
|
|
|
|
a line that faded was gone twelve seconds, then nowhere to find it
|
|
nothing open, nothing heard the usual case, and the notice was lost
|
|
every run said it again "already said" was known only to a tab
|
|
|
|
So `Jobs.record_notice` keeps one row per `(code, key)`, and the three are one
|
|
fix. The same failure said again is that row with a higher count, which is how
|
|
a client tells news from a repeat: it interrupts for the first and only counts
|
|
the rest. A line somebody dismissed is still in the log, so dismissing means
|
|
later rather than never. A row goes when somebody clears it, or a fortnight
|
|
after it was last said — by which time it has stopped being true, which is the
|
|
only staleness the first design was right to worry about.
|
|
|
|
The table is not the log and does not try to be. No traceback, no history of
|
|
each occurrence: a sentence, what to do, the raw error, how often, and when.
|
|
|
|
This module is only the part the queue itself needs: the notice every failed
|
|
job gets, the check that stops it being said twice, and the rendering a
|
|
terminal wants. **Classifying a failure is the app's job**, because a useful
|
|
sentence names things only the app knows — its model backend, its feeds, its
|
|
config keys. An app writes its own vocabulary next to its handlers and passes
|
|
notices in through `RunContext.notice`; newsdesk's is `desk/notices.py`.
|
|
|
|
What a notice has to carry is not the full story but enough to orient you:
|
|
|
|
text what happened, in a sentence, in terms of what you lost by it.
|
|
kind whether this is the world misbehaving — an expired login, a feed
|
|
gone 403 — or a bug of ours. That is the fork in what you do next,
|
|
and guessing it from an error string is exactly the work you should
|
|
not have to do at a glance.
|
|
fix the command, when there is one.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import logging
|
|
from collections.abc import Iterable
|
|
|
|
from pipeline.models import Notice, NoticeKind
|
|
|
|
log = logging.getLogger(__name__)
|
|
|
|
#: Words that mean the failure happened at the other end of a wire. An app's
|
|
#: own classifier will want more than this; `whose_fault` is the cheap default
|
|
#: for the failures the queue meets on its own.
|
|
THEIR_FAULT = (
|
|
"http", "url", "timeout", "connection", "ssl", "certificate", "socket",
|
|
"dns", "resolve", "403", "404", "429", "500", "502", "503", "refused",
|
|
"unreachable", "forbidden", "not found", "unavailable", "moved",
|
|
)
|
|
|
|
|
|
def job_failed(kind: str, label: str, error: BaseException | str) -> Notice:
|
|
"""A job that ended in `failed`.
|
|
|
|
A handler raising something it did not expect is the definition of ours to
|
|
fix, so this is the one place that says `bug` outright rather than working
|
|
it out.
|
|
"""
|
|
return Notice(
|
|
code="job.failed", level="error", kind="bug", key=kind,
|
|
text=f"{label or kind} failed.",
|
|
detail=str(error).strip(),
|
|
where=f"the `job … ({kind}) failed` traceback in the log",
|
|
)
|
|
|
|
|
|
def already_said(said: Iterable[Notice], error: str) -> bool:
|
|
"""Whether one of these notices is about this failure.
|
|
|
|
The caller is the worker, which announces `job_failed` for every job that
|
|
ends in `failed` — and must not, when the handler already announced a
|
|
classified notice about the very exception that ended it. "agent.backend
|
|
is `off`, so nothing that needs a model runs" followed by "Tagging a batch
|
|
failed — looks like a bug" is two contradictory lines about one failure,
|
|
and the wrong one is the louder: it sends you to a traceback about a
|
|
machine that is doing exactly what it was configured to do.
|
|
|
|
Matched on `detail` rather than on identity, because what the handler
|
|
classified is an exception and what reaches the worker is that same
|
|
exception rendered `Type: message` — the same string with a prefix. Which
|
|
is also the line in the right place: a notice about something else the job
|
|
met on the way, a source that 403'd during a pull that then crashed over
|
|
something unrelated, does not match, and that crash still gets its own
|
|
line as the unexplained failure it is.
|
|
"""
|
|
return any(notice.detail.strip() and notice.detail.strip() in error
|
|
for notice in said)
|
|
|
|
|
|
def render(notice: Notice) -> str:
|
|
"""The same notice, for a terminal.
|
|
|
|
The browser is the first-class destination, but it is not the only one: a
|
|
handler run from a terminal hits every one of these failures and deserves
|
|
the same sentence, the same command to run, and the same "is this mine to
|
|
fix". Without this the CLI got the code and the text and silently lost the
|
|
two fields that make a notice worth having.
|
|
|
|
Rendered rather than logged in pieces so that the wording lives in one
|
|
place. An app changes a rule and both destinations change together.
|
|
"""
|
|
lines = [notice.text]
|
|
if notice.fix:
|
|
lines.append(f" -> {notice.fix}")
|
|
if notice.kind == "bug":
|
|
lines.append(f" this looks like a bug rather than a bad day; see {notice.where}"
|
|
if notice.where else " this looks like a bug rather than a bad day")
|
|
return "\n".join(lines)
|
|
|
|
|
|
def whose_fault(error: BaseException | str) -> NoticeKind:
|
|
"""Whether a failure is the other end's or ours.
|
|
|
|
A 403 is the outlet's business and there is nothing here to fix. A
|
|
`KeyError: 'entries'` is us assuming a shape the other end didn't have, and
|
|
it will keep happening until someone changes the code — so it belongs in
|
|
front of you as a defect, not as weather.
|
|
"""
|
|
text = f"{type(error).__name__} {error}".lower() if isinstance(
|
|
error, BaseException) else str(error).lower()
|
|
return "expected" if any(hint in text for hint in THEIR_FAULT) else "bug"
|