Extract is bounded to the ranking: pull → embed → score → extract → score → summarise
The free stages run first and their score bounds the two model stages. Extract reads only the best extract.per_agent roles of each agent at extract.min_score or more, then score runs again so the gates read what it filled; summarise gets a min_score of its own. The cascade steps by position, since score is in it twice. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
c4f2bcc0c9
commit
7db77e83a3
15 files changed
+215
-89
No files matched your search
@@ -6,7 +6,7 @@ task in `pipeline.TASKS` and the default schedule in `pipeline.SCHEDULES`.
|
||||
embed.py one vector per open role, for fit
|
||||
score.py every open role scored for every enabled search agent
|
||||
summarise.py about 60 words on each of the best roles, on Haiku
|
||||
cascade.py pull → extract → embed → score → summarise, what `TASKS`
|
||||
cascade.py pull → embed → score → extract → score → summarise, what `TASKS`
|
||||
offers, the `pull` job a schedule fires, and that schedule
|
||||
sources.py what a pull reads, changed by hand: boards and searches
|
||||
added, switched on or off, pruned by region
|
||||
|
||||
+22
-20
@@ -1,18 +1,21 @@
|
||||
"""The tasks roledesk offers, and the order a pull runs them in.
|
||||
|
||||
pull → extract → embed → score → summarise
|
||||
pull → embed → score → extract → score → summarise
|
||||
|
||||
newsdesk's cascade convention: a cascade is a tuple of task names, and the
|
||||
final job of one task starts the next (`step` is the callback it carries).
|
||||
Nothing is alive between two tasks.
|
||||
Nothing is alive between two tasks. A stage is addressed by its position in
|
||||
the tuple, not its name, since `score` runs twice.
|
||||
|
||||
Every stage is always in the chain, and a stage that is switched off finds
|
||||
nothing: `extract.enabled: false`, or an `on_ingest` this pull does not
|
||||
satisfy, is an extract stage with no batches whose final job moves straight
|
||||
on. Extract runs before score so the gates read what it filled; before embed
|
||||
too, since embedding does not read its columns and the order costs nothing.
|
||||
Summarise runs last because it reads the scores: it takes only the best
|
||||
`summarise.per_agent` roles of each enabled agent.
|
||||
on. The free stages go first: embed and score cost no tokens, and the score
|
||||
they produce is what bounds the two that do. Extract reads only the best
|
||||
`extract.per_agent` roles of each enabled agent, not below
|
||||
`extract.min_score`; the second score then applies the gates to the columns
|
||||
it filled. Summarise runs last, on the best `summarise.per_agent` roles not
|
||||
below `summarise.min_score`.
|
||||
|
||||
A batch that failed, or a stage somebody killed, does not stop the chain: the
|
||||
work comes round on the next pull, and a rescore of what did arrive is worth
|
||||
@@ -37,7 +40,7 @@ from pipeline import NULL, TASKS, RunContext, Tally, Task, declare, job
|
||||
STEP = "app.jobs.cascade:step"
|
||||
|
||||
CASCADES: dict[str, tuple[str, ...]] = {
|
||||
"pull": ("pull", "extract", "embed", "score", "summarise"),
|
||||
"pull": ("pull", "embed", "score", "extract", "score", "summarise"),
|
||||
}
|
||||
|
||||
|
||||
@@ -92,21 +95,19 @@ STAGES: dict[str, Callable[[str], Task]] = {
|
||||
}
|
||||
|
||||
|
||||
def task(name: str, *, cascade: str = "") -> Task:
|
||||
def task(name: str, *, cascade: str = "", at: int = 0) -> Task:
|
||||
"""One stage as a `Task`. With `cascade` its final job carries `step`,
|
||||
and the next stage follows; without, it stops when its batches do."""
|
||||
and the stage after position `at` follows; without, it stops when its
|
||||
batches do."""
|
||||
return replace(STAGES[name](cascade), name=name,
|
||||
callback=STEP if cascade else "",
|
||||
carries={"cascade": cascade, "task": name})
|
||||
carries={"cascade": cascade, "task": name, "at": at})
|
||||
|
||||
|
||||
def follows(cascade: str, name: str) -> str:
|
||||
"""What runs after `name`, or "" at the end or for an unknown name."""
|
||||
def follows(cascade: str, at: int) -> str:
|
||||
"""The stage after position `at`, or "" at the end or off the cascade."""
|
||||
tasks = CASCADES.get(cascade, ())
|
||||
if name not in tasks:
|
||||
return ""
|
||||
at = tasks.index(name)
|
||||
return tasks[at + 1] if at + 1 < len(tasks) else ""
|
||||
return tasks[at + 1] if 0 <= at < len(tasks) - 1 else ""
|
||||
|
||||
|
||||
def step(config: Any, store: Any, tally: Tally, payload: dict[str, Any], *,
|
||||
@@ -114,9 +115,10 @@ def step(config: Any, store: Any, tally: Tally, payload: dict[str, Any], *,
|
||||
"""The callback: make the next task. A killed stage is skipped and the
|
||||
chain goes on, as in newsdesk's cascade."""
|
||||
name, done = payload.get("cascade", ""), payload.get("task", "")
|
||||
following = follows(name, done)
|
||||
at = int(payload.get("at", -1))
|
||||
following = follows(name, at)
|
||||
if following:
|
||||
context.task(task(following, cascade=name))
|
||||
context.task(task(following, cascade=name, at=at + 1))
|
||||
out: dict[str, Any] = {"task": done, "next": following}
|
||||
if tally.killed:
|
||||
out["skipped"] = "cancelled"
|
||||
@@ -125,7 +127,7 @@ def step(config: Any, store: Any, tally: Tally, payload: dict[str, Any], *,
|
||||
|
||||
def chain(name: str) -> Task:
|
||||
"""The first stage of cascade `name`, carrying `step`, so the rest follow."""
|
||||
return task(CASCADES[name][0], cascade=name)
|
||||
return task(CASCADES[name][0], cascade=name, at=0)
|
||||
|
||||
|
||||
@job("pull", "Pulling every board", resource="http")
|
||||
@@ -154,4 +156,4 @@ for _name in ("extract", "embed", "score", "summarise"):
|
||||
# setting: a cadence edited or switched off on the Pulls page survives
|
||||
# every restart (see pipeline/schedule.py).
|
||||
declare("pull", "pull", "every 6h",
|
||||
label="Pull every board, then extract, embed, score and summarise")
|
||||
label="Pull every board, then embed, score, extract, rescore and summarise")
|
||||
+15
-7
@@ -8,12 +8,17 @@ refused batch, and failing with a notice when no model is reachable are
|
||||
`desk.modeltask`'s.
|
||||
|
||||
A role is waiting while it is open, has never been answered (`extracted_at IS
|
||||
NULL`) and has not been given up on. The answer only fills columns that are
|
||||
still empty: a board's structured field wins over what a model read out of the
|
||||
prose. See `Store.set_extracted`.
|
||||
NULL`), has not been given up on, and is in the best `extract.per_agent` of an
|
||||
enabled agent's ungated scores at `extract.min_score` or more. That bound is
|
||||
what keeps the cost proportional to the rankings rather than to the boards:
|
||||
the pull scores every role for free first, then reads the few hundred that
|
||||
rank, and scores again so the gates see what was filled. A batch built for
|
||||
named uids finishes what it was dealt whatever their rank. The answer only
|
||||
fills columns that are still empty: a board's structured field wins over what
|
||||
a model read out of the prose. See `Store.set_extracted`.
|
||||
|
||||
Off by default (`extract.enabled: false`), because on the API backend every
|
||||
batch is a paid call and ~30k open roles is ~1,500 of them.
|
||||
batch is a paid call.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -128,7 +133,8 @@ def _item(role: Role) -> dict[str, Any]:
|
||||
def build_batch(config: Config, store: Store, limit: int = 20,
|
||||
uids: Sequence[str] | None = None) -> dict[str, Any]:
|
||||
return {"instructions": SYSTEM,
|
||||
"roles": [_item(r) for r in store.extract_waiting(limit, uids)]}
|
||||
"roles": [_item(r) for r in store.extract_waiting(
|
||||
limit, config.extract.per_agent, config.extract.min_score, uids)]}
|
||||
|
||||
|
||||
# -------------------------------------------------------------------- answer
|
||||
@@ -199,9 +205,11 @@ TASK: ModelTask[ExtractResult] = ModelTask(
|
||||
kind="extract", label="Reading postings for gate fields",
|
||||
verb="extracting", noun="fields", idle="nothing to extract", items="roles",
|
||||
settings=lambda config: config.extract,
|
||||
pending=lambda config, store: store.extract_pending(),
|
||||
pending=lambda config, store: store.extract_pending(
|
||||
config.extract.per_agent, config.extract.min_score),
|
||||
waiting=lambda config, store, limit: [
|
||||
r.uid for r in store.extract_waiting(limit)],
|
||||
r.uid for r in store.extract_waiting(
|
||||
limit, config.extract.per_agent, config.extract.min_score)],
|
||||
build_batch=build_batch,
|
||||
system=lambda store: SYSTEM,
|
||||
schema=SCHEMA,
|
||||
|
||||
@@ -6,9 +6,9 @@ what the model is told, the schema it answers in, what to do with the
|
||||
answer), and running it is `desk.modeltask`'s.
|
||||
|
||||
A role is waiting while it has no summary, has some text, and is in the top
|
||||
`summarise.per_agent` of an enabled agent's ungated scores. That bound is what
|
||||
keeps the cost flat: a pull summarises the roles somebody will look at, not the
|
||||
30k that are open. A batch built for named uids (the read page's Summarise
|
||||
`summarise.per_agent` of an enabled agent's ungated scores at
|
||||
`summarise.min_score` or more. That bound is what keeps the cost flat: a pull
|
||||
summarises the roles somebody will look at, not the 30k that are open. A batch built for named uids (the read page's Summarise
|
||||
button) takes those roles whatever their rank. See `Store.summarise_waiting`.
|
||||
|
||||
The text is `article_text` when the posting page was fetched, else
|
||||
@@ -121,7 +121,8 @@ def _item(role: Role) -> dict[str, Any]:
|
||||
|
||||
def _waiting(config: Config, store: Store, limit: int,
|
||||
uids: Sequence[str] | None = None) -> list[Role]:
|
||||
return store.summarise_waiting(limit, config.summarise.per_agent, uids)
|
||||
return store.summarise_waiting(limit, config.summarise.per_agent, uids,
|
||||
min_score=config.summarise.min_score)
|
||||
|
||||
|
||||
def build_batch(config: Config, store: Store, limit: int = 10,
|
||||
@@ -165,7 +166,8 @@ TASK: ModelTask[SummariseResult] = ModelTask(
|
||||
verb="summarising", noun="summaries", idle="nothing to summarise",
|
||||
items="summaries",
|
||||
settings=lambda config: config.summarise,
|
||||
pending=lambda config, store: store.summarise_pending(config.summarise.per_agent),
|
||||
pending=lambda config, store: store.summarise_pending(
|
||||
config.summarise.per_agent, config.summarise.min_score),
|
||||
waiting=lambda config, store, limit: [r.uid for r in _waiting(config, store, limit)],
|
||||
build_batch=build_batch,
|
||||
system=lambda store: SYSTEM,
|
||||
|
||||
Reference in new issue
Block a user