Extract and summarise bounds are each agent's own limits, set on its form

agents.limits holds extract_top, extract_min, summarise_top and
summarise_min; the scope query reads them per ranked row, and a row saved
before this behaves like a fresh agent. The four global config keys go.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Fable 5.1 committed 2026-09-28 16:02:06 +02:00
1 parent e602eb3388
commit cebe62ddc9
20 files changed
+229 -147

No files matched your search

+1 -1
View File
@@ -49,7 +49,7 @@ The cog opens `/settings`: a rail of pages on the left, the page beside it, stic
Likes, hidden companies and votes belong to a search agent, not to the account: two agents rank the same roles differently, and one may hide a company the other favours. An agent also learns from votes: its `taste` weight scores a role by how close it is to the roles the agent liked minus the ones it passed.
The reader, `/roles/{uid}`, shows the full posting: the same vote and Save controls, the company's own affinity, the tags and the summary when there is one. Opening a role marks it read, and so does a vote. A search role's posting is a teaser; the reader offers a Fetch button that scrapes the original listing once and stores the text. A Summarise button asks for a short summary of that one role; `config.yaml`'s `summarise:` block turns summarising on for the pull itself, bounded to the top `per_agent` roles of each agent so the cost stays flat.
The reader, `/roles/{uid}`, shows the full posting: the same vote and Save controls, the company's own affinity, the tags and the summary when there is one. Opening a role marks it read, and so does a vote. A search role's posting is a teaser; the reader offers a Fetch button that scrapes the original listing once and stores the text. A Summarise button asks for a short summary of that one role; `config.yaml`'s `summarise:` block turns summarising on for the pull itself, bounded to the top roles of each agent by that agent's own `limits` so the cost stays flat.
## Docs
+8 -17
View File
@@ -22,7 +22,7 @@ from typing import Any, Literal
import yaml
from pydantic import Field, ValidationError, model_validator
from app.models import Gate, Weights
from app.models import Gate, Limits, Weights
from auth import AuthConfig
from desk.config import AgentConfig, ModelTaskConfig, Section, WorkersConfig
@@ -43,37 +43,27 @@ class ExtractConfig(ModelTaskConfig):
"""Turning a posting's title and text into the gate columns. See
app/jobs/extract.py. Off by default, so nobody is billed by surprise:
on the API backend every batch is a paid call. On, it reads only the
best `per_agent` roles of each enabled agent that score at least
`min_score`, so the cost follows the ranking rather than the boards."""
best roles of each enabled agent, as deep and as low as that agent's
own `limits` say, so the cost follows the ranking rather than the
boards."""
enabled: bool = False
model: str = "claude-haiku-4-5-20251001"
batch_size: int = Field(20, ge=1)
#: How deep into each enabled agent's ranking extract reads: the backstop.
per_agent: int = Field(500, ge=0)
#: Below this score a role is not read, however few are above it. Loose,
#: since a fresh role with the worst fit scores about 0.27 on the default
#: weights and a good one 0.6 or more.
min_score: float = Field(0.3, ge=0, le=1)
class SummariseConfig(ModelTaskConfig):
"""About 60 words on a role, for the list and the read page. See
app/jobs/summarise.py. Off by default like extract; on, a pull summarises
only the best `per_agent` of each enabled agent, so the cost is bounded
whatever the backlog. The read page's Summarise button asks for one role
whether this is on or not."""
only the best of each enabled agent, as deep and as low as that agent's
own `limits` say, so the cost is bounded whatever the backlog. The read
page's Summarise button asks for one role whether this is on or not."""
enabled: bool = False
model: str = "claude-haiku-4-5-20251001"
#: A description is capped at 8000 characters per role, so ten is about
#: 20k tokens of input per call.
batch_size: int = Field(10, ge=1)
#: How deep into each enabled agent's ranking a pull summarises.
per_agent: int = Field(30, ge=0)
#: Below this score a role is not summarised. Stricter than extract's,
#: since a summary is the expensive answer.
min_score: float = Field(0.5, ge=0, le=1)
class EmbedConfig(Section):
@@ -96,6 +86,7 @@ class AgentSeed(Section):
intent: str = ""
gates: list[Gate] = Field(default_factory=list)
weights: Weights = Field(default_factory=Weights)
limits: Limits = Field(default_factory=Limits)
enabled: bool = True
+3 -4
View File
@@ -12,10 +12,9 @@ nothing: `extract.enabled: false`, or an `on_ingest` this pull does not
satisfy, is an extract stage with no batches whose final job moves straight
on. The free stages go first: embed and score cost no tokens, and the score
they produce is what bounds the two that do. Extract reads only the best
`extract.per_agent` roles of each enabled agent, not below
`extract.min_score`; the second score then applies the gates to the columns
it filled. Summarise runs last, on the best `summarise.per_agent` roles not
below `summarise.min_score`.
roles of each enabled agent, as deep and as low as that agent's own `limits`
say; the second score then applies the gates to the columns it filled.
Summarise runs last, on the best roles by the same agent's summarise limits.
A batch that failed, or a stage somebody killed, does not stop the chain: the
work comes round on the next pull, and a rescore of what did arrive is worth
+7 -10
View File
@@ -8,9 +8,10 @@ refused batch, and failing with a notice when no model is reachable are
`desk.modeltask`'s.
A role is waiting while it is open, has never been answered (`extracted_at IS
NULL`), has not been given up on, and is in the best `extract.per_agent` of an
enabled agent's ungated scores at `extract.min_score` or more. That bound is
what keeps the cost proportional to the rankings rather than to the boards:
NULL`), has not been given up on, and is in the best of an enabled agent's
ungated scores as deep and as low as that agent's own `limits` say
(`extract_top`, `extract_min`). That bound is what keeps the cost
proportional to the rankings rather than to the boards:
the pull scores every role for free first, then reads the few hundred that
rank, and scores again so the gates see what was filled. A batch built for
named uids finishes what it was dealt whatever their rank. The answer only
@@ -133,8 +134,7 @@ def _item(role: Role) -> dict[str, Any]:
def build_batch(config: Config, store: Store, limit: int = 20,
uids: Sequence[str] | None = None) -> dict[str, Any]:
return {"instructions": SYSTEM,
"roles": [_item(r) for r in store.extract_waiting(
limit, config.extract.per_agent, config.extract.min_score, uids)]}
"roles": [_item(r) for r in store.extract_waiting(limit, uids)]}
# -------------------------------------------------------------------- answer
@@ -205,11 +205,8 @@ TASK: ModelTask[ExtractResult] = ModelTask(
kind="extract", label="Reading postings for gate fields",
verb="extracting", noun="fields", idle="nothing to extract", items="roles",
settings=lambda config: config.extract,
pending=lambda config, store: store.extract_pending(
config.extract.per_agent, config.extract.min_score),
waiting=lambda config, store, limit: [
r.uid for r in store.extract_waiting(
limit, config.extract.per_agent, config.extract.min_score)],
pending=lambda config, store: store.extract_pending(),
waiting=lambda config, store, limit: [r.uid for r in store.extract_waiting(limit)],
build_batch=build_batch,
system=lambda store: SYSTEM,
schema=SCHEMA,
+1 -1
View File
@@ -276,7 +276,7 @@ def seed_agents(config: Config, store: Store) -> list[str]:
for seed in config.agents:
try:
store.save_agent(seed.name, seed.intent, seed.gates, seed.weights,
seed.enabled)
seed.enabled, limits=seed.limits)
except ValueError as exc:
log.warning("agents: %s", exc)
continue
+6 -7
View File
@@ -6,9 +6,10 @@ what the model is told, the schema it answers in, what to do with the
answer), and running it is `desk.modeltask`'s.
A role is waiting while it has no summary, has some text, and is in the top
`summarise.per_agent` of an enabled agent's ungated scores at
`summarise.min_score` or more. That bound is what keeps the cost flat: a pull
summarises the roles somebody will look at, not the 30k that are open. A batch built for named uids (the read page's Summarise
of an enabled agent's ungated scores as deep and as low as that agent's own
`limits` say (`summarise_top`, `summarise_min`). That bound is what keeps the
cost flat: a pull summarises the roles somebody will look at, not the 30k
that are open. A batch built for named uids (the read page's Summarise
button) takes those roles whatever their rank. See `Store.summarise_waiting`.
The text is `article_text` when the posting page was fetched, else
@@ -121,8 +122,7 @@ def _item(role: Role) -> dict[str, Any]:
def _waiting(config: Config, store: Store, limit: int,
uids: Sequence[str] | None = None) -> list[Role]:
return store.summarise_waiting(limit, config.summarise.per_agent, uids,
min_score=config.summarise.min_score)
return store.summarise_waiting(limit, uids)
def build_batch(config: Config, store: Store, limit: int = 10,
@@ -166,8 +166,7 @@ TASK: ModelTask[SummariseResult] = ModelTask(
verb="summarising", noun="summaries", idle="nothing to summarise",
items="summaries",
settings=lambda config: config.summarise,
pending=lambda config, store: store.summarise_pending(
config.summarise.per_agent, config.summarise.min_score),
pending=lambda config, store: store.summarise_pending(),
waiting=lambda config, store, limit: [r.uid for r in _waiting(config, store, limit)],
build_batch=build_batch,
system=lambda store: SYSTEM,
+18 -2
View File
@@ -388,15 +388,31 @@ class Weights(Model):
taste: float = Field(0.2, ge=0)
class Limits(Model):
"""How far down an agent's ranking the two model stages read. Each pair
is a top N, the backstop, and a minimum score, the real stop: a role
is read when it is among the agent's best `extract_top` ungated scores
and not below `extract_min`, and summarised likewise. A fresh role with
the worst fit scores about 0.27 on the default weights, a good one 0.6
or more, so extract's floor is loose and summarise's, the long answer,
stricter."""
extract_top: int = Field(500, ge=0)
extract_min: float = Field(0.3, ge=0, le=1)
summarise_top: int = Field(30, ge=0)
summarise_min: float = Field(0.5, ge=0, le=1)
class Agent(Model):
"""A row of `agents`: a named search with its own intent, gates and
weights. The cached intent vector is a column this does not declare."""
"""A row of `agents`: a named search with its own intent, gates, weights
and limits. The cached intent vector is a column this does not declare."""
id: int
name: str
intent: str = ""
gates: Annotated[list[Gate], BeforeValidator(lambda v: _json(v, "[]"))] = []
weights: Annotated[Weights, BeforeValidator(lambda v: _json(v, "{}"))] = Weights()
limits: Annotated[Limits, BeforeValidator(lambda v: _json(v, "{}"))] = Limits()
enabled: bool = True
created_at: str
+21
View File
@@ -20,6 +20,10 @@
const $ = (sel, root = document) => root.querySelector(sel);
const DEFAULT_WEIGHTS = { fit: 0.6, recency: 0.2, affinity: 0.2, taste: 0.2 };
// What the model reads after a pull, per agent: the best `extract_top`
// roles scoring at least `extract_min` get their gate columns filled, the
// best `summarise_top` at `summarise_min` or more a summary.
const DEFAULT_LIMITS = { extract_top: 500, extract_min: 0.3, summarise_top: 30, summarise_min: 0.5 };
const NEW = "new";
/** @type {object[]} every agent, as the server lists them */
@@ -108,6 +112,11 @@
return `fit ${w.fit} · newness ${w.recency} · companies ${w.affinity} · liked ${w.taste}${gates}`;
}
function limitsLine(row) {
const l = Object.assign({}, DEFAULT_LIMITS, row.limits);
return `reads ${l.extract_top} at ${l.extract_min} · summarises ${l.summarise_top} at ${l.summarise_min}`;
}
function card(row, fallback) {
const div = el("article", "agent-card");
if (!row.enabled) div.dataset.off = "true";
@@ -127,6 +136,7 @@
? el("p", "agent-intent", intent)
: el("p", "agent-intent empty", "Nothing described yet, so fit has nothing to measure against. Edit says what this agent looks for."));
div.appendChild(el("p", "agent-weights", weightsLine(row)));
div.appendChild(el("p", "agent-limits", limitsLine(row)));
const actions = el("div", "agent-card-actions");
const edit = el("button", "secondary", "Edit");
@@ -151,6 +161,7 @@
const div = el("article", "agent-card editing");
const f = $("#agent-form-template").content.querySelector("form").cloneNode(true);
const weights = Object.assign({}, DEFAULT_WEIGHTS, row ? row.weights : {});
const limits = Object.assign({}, DEFAULT_LIMITS, row ? row.limits : {});
$("#agent-form-title", f).textContent = row ? `Edit ${row.name}` : "New agent";
$("#a-name", f).value = row ? row.name : "";
$("#a-intent", f).value = row ? row.intent : "";
@@ -159,6 +170,10 @@
$("#a-recency", f).value = weights.recency;
$("#a-affinity", f).value = weights.affinity;
$("#a-taste", f).value = weights.taste;
$("#a-extract-top", f).value = limits.extract_top;
$("#a-extract-min", f).value = limits.extract_min;
$("#a-summarise-top", f).value = limits.summarise_top;
$("#a-summarise-min", f).value = limits.summarise_min;
$("#a-delete", f).hidden = !row;
f.addEventListener("input", () => { f.dataset.dirty = "true"; });
f.addEventListener("submit", (event) => save(event, row));
@@ -227,6 +242,12 @@
affinity: Number($("#a-affinity", f).value),
taste: Number($("#a-taste", f).value),
},
limits: {
extract_top: Number($("#a-extract-top", f).value),
extract_min: Number($("#a-extract-min", f).value),
summarise_top: Number($("#a-summarise-top", f).value),
summarise_min: Number($("#a-summarise-min", f).value),
},
// Saving an agent means it is wanted, so a new one starts on; the
// card's switch is where an existing one is switched.
enabled: row ? row.enabled : true,
+4 -1
View File
@@ -1083,7 +1083,7 @@ table.applications td[data-label="Saved"] { white-space: nowrap; color: var(--di
white-space: pre-line; overflow-wrap: anywhere;
}
.agent-intent.empty { font-family: inherit; font-size: .9rem; color: var(--dim); font-style: italic; }
.agent-weights { margin: 0; font-size: .8rem; color: var(--faint); font-variant-numeric: tabular-nums; }
.agent-weights, .agent-limits { margin: 0; font-size: .8rem; color: var(--faint); font-variant-numeric: tabular-nums; }
.agent-card-actions { display: flex; align-items: center; flex-wrap: wrap; gap: .5rem 1rem; margin-top: .8rem; }
.agent-card-actions a { font-size: .9rem; font-weight: 600; text-decoration: none; }
@media (hover: hover) {
@@ -1147,6 +1147,9 @@ table.applications td[data-label="Saved"] { white-space: nowrap; color: var(--di
.weights { display: flex; gap: .75rem; flex-wrap: wrap; border: 1px solid var(--line); border-radius: .3rem; }
/* A label that wraps to two lines pushes its box down; the boxes stay in line. */
.weights label { flex: 1 1 6rem; justify-content: space-between; }
/* The limits are a pair per stage, count then score, so two to a row. */
.weights.limits label { flex: 1 1 40%; }
.agent-form fieldset + .hint { margin-top: -.3rem; }
.agent-actions { display: flex; gap: .5rem; flex-wrap: wrap; }
.agent-form details summary { cursor: pointer; color: var(--dim); font-size: .9rem; }
.agent-form details[open] summary { margin-bottom: .5rem; }
+8 -2
View File
@@ -167,7 +167,9 @@ END;
-- A search agent: a named saved search with its own intent text, gates and
-- weights. gates is a list of {field, values, mode: require|prefer, weight}.
-- intent_vec caches the intent's vector; intent_key is sha1 of the embedding
-- model and the intent text, so editing either makes it stale.
-- model and the intent text, so editing either makes it stale. limits is how
-- deep into this agent's ranking extract and summarise read, as JSON: a key
-- it lacks means the `Limits` default.
CREATE TABLE IF NOT EXISTS agents (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
@@ -177,7 +179,8 @@ CREATE TABLE IF NOT EXISTS agents (
enabled INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL,
intent_key TEXT NOT NULL DEFAULT '',
intent_vec BLOB
intent_vec BLOB,
limits TEXT NOT NULL DEFAULT '{}'
);
-- One agent's stance on a company: -1 hidden, 0 neutral, 1 liked, 2
@@ -331,6 +334,9 @@ ALTER TABLE companies DROP COLUMN affinity;
ALTER TABLE companies ADD COLUMN ats TEXT NOT NULL DEFAULT '';
ALTER TABLE companies ADD COLUMN country TEXT NOT NULL DEFAULT '';
ALTER TABLE companies ADD COLUMN city TEXT NOT NULL DEFAULT '';
"""),
(8, "agents.limits: how deep into its ranking the model stages read is each agent's own", """
ALTER TABLE agents ADD COLUMN limits TEXT NOT NULL DEFAULT '{}';
"""),
)
+46 -43
View File
@@ -14,7 +14,7 @@ from collections.abc import Callable, Iterator, MutableMapping, Sequence
from contextlib import closing, contextmanager
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any
from typing import Any, Literal
from uuid import uuid4
import numpy as np
@@ -22,8 +22,8 @@ import numpy as np
from desk.models import Item
from app.models import (
AFFINITY, REMOTE, STATUSES, Agent, Application, ApplicationEvent, Candidate,
Company, Gate, Role, ScoredRole, Subscription, Weights, company_key, country_of,
place_key, title_key,
Company, Gate, Limits, Role, ScoredRole, Subscription, Weights, company_key,
country_of, place_key, title_key,
)
from app.storage import schema
@@ -619,24 +619,29 @@ class Store:
# -------------------------------------------------------------- extract
def _top_scope(self, per_agent: int, min_score: float) -> tuple[str, list[Any]]:
"""The `WHERE` fragment for a role among the `per_agent` best ungated
scores of an enabled agent, not below `min_score`: what the two model
stages, extract and summarise, are bounded to. The role is `r`. A
company an agent hid needs no clause: `score` writes no rows for it
under that agent."""
return ("""r.uid IN (SELECT uid FROM (
SELECT s.uid, s.score, ROW_NUMBER() OVER (
def _top_scope(self, stage: Literal["extract", "summarise"]) -> tuple[str, list[Any]]:
"""The `WHERE` fragment for a role among the best ungated scores of
an enabled agent, as deep and as low as that agent's own `limits`
say for `stage`: what the two model stages, extract and summarise,
are bounded to. The rank is numbered per agent and the agent's
limits ride out of the window subquery beside it, so each row is
held to the limits of the agent that ranked it; a key the JSON lacks
is the `Limits` default. The role is `r`. A company an agent hid
needs no clause: `score` writes no rows for it under that agent."""
defaults = Limits()
return (f"""r.uid IN (SELECT uid FROM (
SELECT s.uid, s.score, a.limits, ROW_NUMBER() OVER (
PARTITION BY s.agent_id
ORDER BY s.score DESC, top.first_seen DESC, s.uid) AS n
FROM scores s
JOIN agents a ON a.id = s.agent_id AND a.enabled = 1
JOIN roles top ON top.uid = s.uid AND top.closed_at IS NULL
WHERE s.gated_reason = '')
WHERE n <= ? AND score >= ?)""", [per_agent, min_score])
WHERE n <= COALESCE(json_extract(limits, '$.{stage}_top'), ?)
AND score >= COALESCE(json_extract(limits, '$.{stage}_min'), ?))""",
[getattr(defaults, f"{stage}_top"), getattr(defaults, f"{stage}_min")])
def _extract_scope(self, per_agent: int, min_score: float,
uids: Sequence[str] | None) -> tuple[str, list[Any]]:
def _extract_scope(self, uids: Sequence[str] | None) -> tuple[str, list[Any]]:
"""The `WHERE` for a role `extract` may read: open, never answered,
not given up. Given `uids`, those of them and nothing else, since a
batch asks again when it runs and must finish what it was dealt;
@@ -646,21 +651,21 @@ class Store:
if uids is not None:
where.append("r.uid IN (SELECT value FROM json_each(?))")
return " AND ".join(where), [json.dumps(list(uids))]
top, params = self._top_scope(per_agent, min_score)
top, params = self._top_scope("extract")
return " AND ".join([*where, top]), params
def extract_pending(self, per_agent: int, min_score: float = 0.0) -> int:
where, params = self._extract_scope(per_agent, min_score, None)
def extract_pending(self) -> int:
where, params = self._extract_scope(None)
row = self._row(f"SELECT COUNT(*) AS n FROM roles r WHERE {where}", params)
return int(row["n"]) if row else 0
def extract_waiting(self, limit: int, per_agent: int, min_score: float = 0.0,
uids: Sequence[str] | None = None) -> list[Role]:
"""Roles `extract` has yet to read, newest first: the best
`per_agent` of every enabled agent's scores not below `min_score`, so
the backlog follows the rankings rather than the boards. Given
`uids`, only those of them still waiting, whatever their rank."""
where, params = self._extract_scope(per_agent, min_score, uids)
def extract_waiting(self, limit: int, uids: Sequence[str] | None = None) -> list[Role]:
"""Roles `extract` has yet to read, newest first: the best of every
enabled agent's scores, as deep and as low as that agent's own
`limits` say, so the backlog follows the rankings rather than the
boards. Given `uids`, only those of them still waiting, whatever
their rank."""
where, params = self._extract_scope(uids)
return [Role.from_row(r) for r in self._rows(
f"SELECT r.* FROM roles r WHERE {where} "
"ORDER BY r.first_seen DESC, r.uid LIMIT ?", [*params, limit])]
@@ -722,8 +727,7 @@ class Store:
#: roles in some agent's top list.
_SUMMARY_MISSED = "summarise.missed:"
def _summary_scope(self, per_agent: int, min_score: float,
uids: Sequence[str] | None) -> tuple[str, list[Any]]:
def _summary_scope(self, uids: Sequence[str] | None) -> tuple[str, list[Any]]:
"""The `WHERE` for a role `summarise` may be sent: no summary yet and
some text to summarise. Given `uids`, those of them and nothing else,
open or not and given up or not, because somebody asked for them.
@@ -733,7 +737,7 @@ class Store:
if uids is not None:
where.append("r.uid IN (SELECT value FROM json_each(?))")
return " AND ".join(where), [json.dumps(list(uids))]
top, params = self._top_scope(per_agent, min_score)
top, params = self._top_scope("summarise")
where += [
"r.closed_at IS NULL",
f"""NOT EXISTS (SELECT 1 FROM state
@@ -743,20 +747,18 @@ class Store:
]
return " AND ".join(where), params
def summarise_pending(self, per_agent: int, min_score: float = 0.0) -> int:
where, params = self._summary_scope(per_agent, min_score, None)
def summarise_pending(self) -> int:
where, params = self._summary_scope(None)
row = self._row(f"SELECT COUNT(*) AS n FROM roles r WHERE {where}", params)
return int(row["n"]) if row else 0
def summarise_waiting(self, limit: int, per_agent: int,
uids: Sequence[str] | None = None,
min_score: float = 0.0) -> list[Role]:
"""Roles `summarise` has yet to write, newest first. The top
`per_agent` of every enabled agent's scores not below `min_score`, so
the backlog is bounded by the number of agents rather than by the
number of open roles; given `uids`, those still without a summary,
whatever their rank. A batch asks again when it runs."""
where, params = self._summary_scope(per_agent, min_score, uids)
def summarise_waiting(self, limit: int, uids: Sequence[str] | None = None) -> list[Role]:
"""Roles `summarise` has yet to write, newest first: the top of every
enabled agent's scores, as deep and as low as that agent's own
`limits` say, so the backlog is bounded by the agents rather than by
the number of open roles; given `uids`, those still without a
summary, whatever their rank. A batch asks again when it runs."""
where, params = self._summary_scope(uids)
return [Role.from_row(r) for r in self._rows(
f"SELECT r.* FROM roles r WHERE {where} "
"ORDER BY r.first_seen DESC, r.uid LIMIT ?", [*params, limit])]
@@ -866,24 +868,25 @@ class Store:
def save_agent(self, name: str, intent: str = "", gates: Sequence[Gate] = (),
weights: Weights | None = None, enabled: bool = True,
agent_id: int | None = None) -> Agent:
agent_id: int | None = None, limits: Limits | None = None) -> Agent:
"""Create an agent, or with `agent_id` replace that one's fields.
Raises ValueError on an empty or taken name, KeyError on an unknown id."""
name = name.strip()
if not name:
raise ValueError("an agent needs a name")
fields = (name, intent, json.dumps([g.model_dump() for g in gates]),
(weights or Weights()).model_dump_json(), int(enabled))
(weights or Weights()).model_dump_json(),
(limits or Limits()).model_dump_json(), int(enabled))
try:
with self.transaction() as conn:
if agent_id is None:
agent_id = conn.execute(
"""INSERT INTO agents (name, intent, gates, weights, enabled,
created_at) VALUES (?,?,?,?,?,?)""",
"""INSERT INTO agents (name, intent, gates, weights, limits,
enabled, created_at) VALUES (?,?,?,?,?,?,?)""",
(*fields, self.now())).lastrowid
elif not conn.execute(
"""UPDATE agents SET name=?, intent=?, gates=?, weights=?,
enabled=? WHERE id=?""",
limits=?, enabled=? WHERE id=?""",
(*fields, agent_id)).rowcount:
raise KeyError(agent_id)
except sqlite3.IntegrityError as exc:
+8
View File
@@ -30,6 +30,14 @@
<label>Companies you like <input type="number" id="a-affinity" min="0" step="0.05" value="0.2"></label>
<label>Roles you liked <input type="number" id="a-taste" min="0" step="0.05" value="0.2"></label>
</fieldset>
<fieldset class="weights limits">
<legend>What the model reads</legend>
<label>Extract the best <input type="number" id="a-extract-top" min="0" step="10" value="500"></label>
<label>down to a score of <input type="number" id="a-extract-min" min="0" max="1" step="0.05" value="0.3"></label>
<label>Summarise the best <input type="number" id="a-summarise-top" min="0" step="5" value="30"></label>
<label>down to a score of <input type="number" id="a-summarise-min" min="0" max="1" step="0.05" value="0.5"></label>
</fieldset>
<span class="hint">Every pull scores every open role for free; these bound what the model then reads, per agent. The count is the backstop, the score the real stop. A new role with the worst fit scores about 0.27, a good one 0.6 or more.</span>
<details class="advanced">
<summary>Advanced</summary>
<label>Gates
+4 -3
View File
@@ -16,8 +16,8 @@ from pydantic import BaseModel, Field
from app import jobs
from app.models import (
AFFINITY, Agent, Application, Company, Gate, Role, ScoredRole, Status, Subscription,
Weights, company_key,
AFFINITY, Agent, Application, Company, Gate, Limits, Role, ScoredRole, Status,
Subscription, Weights, company_key,
)
from app.storage import Store
from app.web.deps import ConfigDep, StoreDep
@@ -538,6 +538,7 @@ class AgentBody(BaseModel):
intent: str = ""
gates: list[Gate] = []
weights: Weights = Weights()
limits: Limits = Limits()
enabled: bool = True
@@ -547,7 +548,7 @@ def save_agent(body: AgentBody, store: StoreDep) -> Agent:
page starts after a save."""
try:
return store.save_agent(body.name, body.intent, body.gates, body.weights,
body.enabled, agent_id=body.id)
body.enabled, agent_id=body.id, limits=body.limits)
except ValueError as exc:
raise HTTPException(409, str(exc)) from exc
except KeyError:
+15 -13
View File
@@ -49,28 +49,21 @@ agent:
# Title and description in, gate columns out (app/jobs/extract.py). Off by
# default: on the API backend every batch of 20 roles is a paid call. On, a
# pull reads only the best per_agent roles of every enabled agent (the
# backstop) that score at least min_score (the real stop), after a first
# score; a second score then applies the gates to what was filled. A fresh
# role with the worst fit scores about 0.27 on the default weights, a good
# one 0.6 or more, so 0.3 is loose.
# pull reads only the best roles of every enabled agent, as deep and as low
# as that agent's own limits say (see agents below), after a first score; a
# second score then applies the gates to what was filled.
extract:
enabled: false
model: claude-haiku-4-5-20251001
per_agent: 500
min_score: 0.3
# About 60 words on a role: what it is, the stack, where, the pay if stated
# (app/jobs/summarise.py). Off by default. On, each pull summarises the best
# per_agent roles of every enabled agent that have none yet and score at least
# min_score, so the cost stays bounded however many roles are open; stricter
# than extract's floor, since a summary is the long answer. The Summarise
# button on a role's page asks for that one role either way.
# roles of every enabled agent that have none yet, by that agent's limits, so
# the cost stays bounded however many roles are open. The Summarise button on
# a role's page asks for that one role either way.
summarise:
enabled: false
model: claude-haiku-4-5-20251001
per_agent: 30
min_score: 0.5
# A role and each agent's intent as vectors, for the fit term of a score
# (app/jobs/embed.py). Needs the `embeddings` extra; without it `embed` posts
@@ -94,6 +87,14 @@ sources: {}
# 0.2}; taste is how close a role is to the ones the agent liked minus the
# ones it passed. A country is an ISO code, filled from the board's location
# line; EU is a role open anywhere in Europe.
# limits is how deep into this agent's ranking the two model stages read,
# each a top N (the backstop) and a minimum score (the real stop): extract
# reads the best extract_top roles at extract_min or more, summarise the best
# summarise_top at summarise_min or more. A fresh role with the worst fit
# scores about 0.27 on the default weights and a good one 0.6 or more. The
# defaults are {extract_top: 500, extract_min: 0.3, summarise_top: 30,
# summarise_min: 0.5}; a wide monitoring search raises them, a narrow hunt
# lowers them.
# The Search agents sub-tab lists and edits them; the order menu on Roles
# picks one by name. Two more, beside the starter in config.yaml:
agents: []
@@ -107,6 +108,7 @@ agents: []
# - {field: country, values: [AT, BE, BG, HR, CY, CZ, DK, EE, FI, FR, DE, GR, HU, IE, IT,
# LV, LT, LU, MT, NL, PL, PT, RO, SK, SI, ES, SE, IS, LI, NO,
# GB, CH, EU], mode: require}
# limits: {extract_top: 1000, extract_min: 0.3, summarise_top: 50, summarise_min: 0.5}
# - name: Europe hybrid
# intent: >-
# Senior backend or platform engineer in a European tech hub, hybrid or
+3 -6
View File
@@ -17,20 +17,17 @@ workers:
extract:
enabled: false
model: claude-haiku-4-5-20251001
per_agent: 500
min_score: 0.3
summarise:
enabled: false
model: claude-haiku-4-5-20251001
per_agent: 30
min_score: 0.5
embed:
model: sentence-transformers/all-MiniLM-L6-v2
sources: {}
# The starter ranking, imported once into an empty table. Edit it under
# Settings, Ranking afterwards; this list is not read again. EU in a country
# gate is a role open anywhere in Europe ("Remote - Europe", "EMEA"). Two more
# are in config.example.yaml.
# gate is a role open anywhere in Europe ("Remote - Europe", "EMEA"). How deep
# into the ranking extract and summarise read is each agent's `limits`, see
# config.example.yaml, which has two more agents.
agents:
- name: Copenhagen engineering
intent: >-
+2 -2
View File
@@ -90,7 +90,7 @@ What was decided for roledesk and the one reason for each. The longer reasoning
**The pull cascade is `pull → embed → score → extract → score → summarise`.** Every stage is always in the chain and a stage that is off finds nothing, as in newsdesk. The free stages run first: embed and score cost no tokens, and their score is what bounds the two model stages. Extract reads only what ranks, then score runs again so the gates read what it filled. A stage is addressed by its position in the tuple, since `score` is in it twice. Extract reads `on_ingest` only as a stage of a pull, so with `auto` and no model reachable a pull passes it quietly.
**Extract is bounded like summarise: the top `per_agent` roles of each agent at `min_score` or more.** Extract has short answers but would otherwise touch every open role, and the seed boards alone produced 29,000; at Haiku's rates that is tens of dollars for a first pass over roles nobody will look at. Bounded to the ranking, a pull reads a few hundred. `per_agent` is the backstop (default 500) and `min_score` the real stop (default 0.3, loose): a fixed count stays flat whatever the boards produce, where a percentage would not. A batch dealt named uids finishes them whatever their rank, so a running batch is not cut short by a rescore.
**Extract is bounded like summarise: the top roles of each agent, by that agent's own `limits`.** Extract has short answers but would otherwise touch every open role, and the seed boards alone produced 29,000; at Haiku's rates that is tens of dollars for a first pass over roles nobody will look at. Bounded to the ranking, a pull reads a few hundred. `extract_top` is the backstop (default 500) and `extract_min` the real stop (default 0.3, loose): a fixed count stays flat whatever the boards produce, where a percentage would not. The bounds are per agent rather than config, because one agent may be a wide monitoring search that should read a thousand and another a narrow hunt that wants fifty; one global number fits neither. A batch dealt named uids finishes them whatever their rank, so a running batch is not cut short by a rescore.
## The pages
@@ -134,7 +134,7 @@ What was decided for roledesk and the one reason for each. The longer reasoning
**A search role's posting text is fetched on demand, not in the pull.** `POST /api/roles/{uid}/text` scrapes `original_url` only when the read page asks for it, because following Jobindex's redirect for every role during a pull would count as a click in Jobindex's own statistics; one click for a posting somebody actually opened is fair.
**Summarising is bounded to the top `per_agent` roles of each agent at `min_score` or more, and off by default.** The pull's `summarise` stage only considers roles with no summary yet that are in the top `summarise.per_agent` (default 30) of an enabled agent's scores and at `summarise.min_score` (default 0.5, stricter than extract's since a summary is the long answer), so the cost stays flat whatever the backlog of open roles is. `summarise.enabled: false` matches `extract`'s own default; the read page's Summarise button asks for one role's summary either way.
**Summarising is bounded to the top roles of each agent, by that agent's own `limits`, and off by default.** The pull's `summarise` stage only considers roles with no summary yet that are in the top `summarise_top` (default 30) of an enabled agent's scores and at `summarise_min` (default 0.5, stricter than extract's since a summary is the long answer), so the cost stays flat whatever the backlog of open roles is. The bounds live on the agent, beside its gates and weights, for the same reason as extract's: a wide monitoring search and a narrow hunt want different depths, and the agent is where the difference is already expressed. `summarise.enabled: false` matches `extract`'s own default; the read page's Summarise button asks for one role's summary either way.
**The schedules element's narrow layout is fixed upstream, in `../boilerplate`.** `<desk-schedules>` is a shadow root the app cannot lay out from outside, and `pipeline/` is a downstream copy refreshed with `git archive`, so a fix landing only in roledesk would be reverted by the next refresh.
+16 -12
View File
@@ -14,6 +14,7 @@ from desk.models import Item
from desk.tools import agent
from app.config import Config
from app.jobs import extract
from app.models import Limits
from app.storage import Store
@@ -155,38 +156,41 @@ def test_an_unknown_uid_is_ignored_and_the_role_left_out_is_charged(
role = store.role(posting("1").uid)
assert role is not None and role.country == "" and role.extracted_at is None
assert result["rejected"] == ["not-in-the-batch"]
assert store.extract_pending(500) == 1
assert store.extract_pending() == 1
modeltask.run(extract.TASK, config, store)
modeltask.run(extract.TASK, config, store)
assert store.extract_pending(500) == 0, "given up after three unanswered replies"
assert store.extract_pending() == 0, "given up after three unanswered replies"
def test_only_the_best_of_each_agent_above_the_floor_is_read(
store: Store, tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
"""The bound that keeps extract cheap: the top `per_agent` of an enabled
agent's scores at `min_score` or more, whatever the boards produced. A
role nobody ranks is never sent; a batch dealt a uid finishes it."""
"""The bound that keeps extract cheap: the top of an enabled agent's
scores, as deep and as low as that agent's own `limits` say, whatever
the boards produced. A role nobody ranks is never sent; a batch dealt a
uid finishes it."""
for sid in ("best", "second", "low", "unscored", "gated"):
store.upsert_role(posting(sid))
one = store.save_agent("One")
one = store.save_agent("One", limits=Limits(extract_top=1, extract_min=0.5))
two = store.save_agent("Two", enabled=False)
store.replace_scores(one.id, [(posting("best").uid, 0.9, ""),
(posting("second").uid, 0.8, ""),
(posting("low").uid, 0.2, ""),
(posting("gated").uid, 0.95, "country unknown")])
store.replace_scores(two.id, [(posting("unscored").uid, 0.99, "")])
config = Config(database=str(tmp_path / "x.db"),
extract={"enabled": True, "per_agent": 1, "min_score": 0.5})
config = Config(database=str(tmp_path / "x.db"), extract={"enabled": True})
model = stub(monkeypatch, lambda roles: [answer(r["uid"]) for r in roles])
assert store.extract_pending(1, 0.5) == 1
assert store.extract_pending(5, 0.5) == 2, "the floor holds the low one back"
assert store.extract_pending(5, 0.0) == 3, "the disabled agent's pick never counts"
assert store.extract_pending() == 1
store.save_agent("One", agent_id=one.id, limits=Limits(extract_top=5, extract_min=0.5))
assert store.extract_pending() == 2, "the floor holds the low one back"
store.save_agent("One", agent_id=one.id, limits=Limits(extract_top=5, extract_min=0))
assert store.extract_pending() == 3, "the disabled agent's pick never counts"
store.save_agent("One", agent_id=one.id, limits=Limits(extract_top=1, extract_min=0.5))
modeltask.run(extract.TASK, config, store)
assert [[r["uid"] for r in batch] for batch in model.asked] == [[posting("best").uid]]
assert [r.uid for r in store.extract_waiting(10, 1, 0.5, uids=[posting("low").uid])] == \
assert [r.uid for r in store.extract_waiting(10, uids=[posting("low").uid])] == \
[posting("low").uid], "a dealt uid is read whatever its rank"
+29 -16
View File
@@ -11,7 +11,7 @@ import pytest
from pydantic import ValidationError
from desk.models import Item
from app.models import Application, company_key, place_key, title_key
from app.models import Application, Limits, company_key, place_key, title_key
from app.storage import LATEST, Store
@@ -62,6 +62,7 @@ def test_schema_creates_on_an_empty_file(tmp_path: Path) -> None:
"application_events", "agents", "scores", "state"} <= tables
assert "idx_roles_cluster" in indexes
assert store.migrations_applied == []
assert LATEST == 8
# A second open finds it current and applies nothing.
assert Store(path).migrations_applied == []
@@ -88,7 +89,7 @@ def test_a_v1_file_migrates_to_the_shape_a_new_file_has(tmp_path: Path) -> None:
for column in ("permit", "languages", "extract_tries", "summary", "summarised_at",
"article_text", "article_at", "read_at"):
conn.execute(f"ALTER TABLE roles DROP COLUMN {column}")
for column in ("intent_key", "intent_vec"):
for column in ("intent_key", "intent_vec", "limits"): # v3, v8
conn.execute(f"ALTER TABLE agents DROP COLUMN {column}")
# v1 had affinity on the company; the vote columns v4 added and v6
# drops again are not in a fresh file, so there is nothing to put back.
@@ -117,6 +118,7 @@ def _downgrade_to_v4(path: Path) -> None:
conn.execute("ALTER TABLE companies ADD COLUMN affinity INTEGER NOT NULL DEFAULT 0")
for column in ("ats", "country", "city"): # v7
conn.execute(f"ALTER TABLE companies DROP COLUMN {column}")
conn.execute("ALTER TABLE agents DROP COLUMN limits") # v8
conn.execute("DROP TABLE agent_companies")
conn.execute("DROP TABLE role_votes")
conn.execute("PRAGMA user_version = 4")
@@ -734,26 +736,37 @@ def test_a_summarys_tags_land_with_it_and_not_on_a_rejected_late_answer(
assert role is not None and role.tags == ["backend", "python"]
def test_summarise_waits_on_the_top_of_each_enabled_agent(store: Store) -> None:
"""Two agents each contribute their own top one; a closed role and a
gated score are not in anybody's top. A hidden company needs no case:
`score` writes no rows for it under the agent that hid it."""
def test_summarise_waits_on_the_top_of_each_enabled_agent_by_its_own_limits(
store: Store) -> None:
"""Each agent contributes its own top, as deep and as low as its own
`limits` say; a closed role and a gated score are in nobody's top. A
hidden company needs no case: `score` writes no rows for it under the
agent that hid it. An agent saved before v8 has `limits` of '{}', which
reads as the defaults."""
for sid in "abcd":
store.upsert_role(posting(sid))
a, b, c, d = (posting(s).uid for s in "abcd")
with store.transaction() as conn:
conn.execute("UPDATE roles SET closed_at = ? WHERE uid = ?", (store.now(), d))
one = store.save_agent("One")
two = store.save_agent("Two")
one = store.save_agent("One", limits=Limits(summarise_top=1, summarise_min=0))
two = store.save_agent("Two", limits=Limits(summarise_top=1, summarise_min=0))
store.replace_scores(one.id, [(d, 0.99, ""), (a, 0.9, ""), (b, 0.5, "")])
store.replace_scores(two.id, [(a, 0.95, "remote"), (c, 0.7, ""), (b, 0.8, "")])
waiting = {r.uid for r in store.summarise_waiting(10, per_agent=1)}
assert {r.uid for r in store.summarise_waiting(10)} == {a, b}
assert waiting == {a, b}
assert {r.uid for r in store.summarise_waiting(10, per_agent=2)} == {a, b, c}
assert store.summarise_pending(per_agent=2) == 3
assert [r.uid for r in store.summarise_waiting(10, 1, uids=[d])] == [d]
# The floor: b's best score is 0.8 under Two, c's 0.7, a's 0.9.
assert {r.uid for r in store.summarise_waiting(10, 2, min_score=0.75)} == {a, b}
assert store.summarise_pending(2, min_score=0.85) == 1
# Two goes deeper: its second, c, joins; One's second, b, was in already.
store.save_agent("Two", agent_id=two.id, limits=Limits(summarise_top=2, summarise_min=0))
assert {r.uid for r in store.summarise_waiting(10)} == {a, b, c}
assert store.summarise_pending() == 3
assert [r.uid for r in store.summarise_waiting(10, uids=[d])] == [d]
# The floor is Two's own: c at 0.7 drops, One's b at 0.5 stays.
store.save_agent("Two", agent_id=two.id,
limits=Limits(summarise_top=2, summarise_min=0.75))
store.save_agent("One", agent_id=one.id, limits=Limits(summarise_top=2, summarise_min=0))
assert {r.uid for r in store.summarise_waiting(10)} == {a, b}
# '{}' is the defaults, 30 at 0.5: One's b at 0.5 stays, Two's c at 0.7 is back.
with store.transaction() as conn:
conn.execute("UPDATE agents SET limits = '{}'")
assert {r.uid for r in store.summarise_waiting(10)} == {a, b, c}
assert store.agents()[0].limits == Limits()
+9 -7
View File
@@ -12,6 +12,7 @@ import pytest
from app.config import Config
from app.jobs import cascade, summarise
from app.models import Limits
from app.storage import Store
from desk import modeltask
from desk.models import Item
@@ -63,18 +64,19 @@ def store(tmp_path: Path) -> Store:
@pytest.fixture
def config(tmp_path: Path) -> Config:
return Config(database=str(tmp_path / "roledesk.db"),
summarise={"enabled": True, "per_agent": 2})
return Config(database=str(tmp_path / "roledesk.db"), summarise={"enabled": True})
def ranked(store: Store, *ids: str, agent_name: str = "Backend",
enabled: bool = True) -> list[str]:
"""Store a role per id and score them for one agent, best first."""
"""Store a role per id and score them for one agent, best first. The
agent summarises its top two: the bound is the agent's own."""
uids = []
for sid in ids:
store.upsert_role(posting(sid))
uids.append(posting(sid).uid)
one = store.save_agent(agent_name, "python backend", enabled=enabled)
one = store.save_agent(agent_name, "python backend", enabled=enabled,
limits=Limits(summarise_top=2))
store.replace_scores(one.id, [(uid, 1.0 - n / 10, "") for n, uid in enumerate(uids)])
return uids
@@ -154,7 +156,7 @@ def test_a_role_with_no_text_is_skipped_not_sent(
summarise.run_batch(config, store, uids=[empty])
assert model.asked == []
assert store.summarise_pending(config.summarise.per_agent) == 0
assert store.summarise_pending() == 0
def test_a_named_uid_is_summarised_whatever_its_rank_or_the_switch(
@@ -182,10 +184,10 @@ def test_an_unknown_or_empty_answer_is_rejected_and_the_role_given_up_after_thre
assert result["rejected"] == ["not-in-the-batch", uid]
assert store.role(uid).summary == "" # type: ignore[union-attr]
assert store.summarise_pending(2) == 1
assert store.summarise_pending() == 1
modeltask.run(summarise.TASK, config, store)
modeltask.run(summarise.TASK, config, store)
assert store.summarise_pending(2) == 0, "given up after three unanswered replies"
assert store.summarise_pending() == 0, "given up after three unanswered replies"
def test_disabled_finds_nothing_and_asks_nobody(
+20
View File
@@ -621,6 +621,12 @@ def test_the_agents_page_lists_and_edits_search_agents(client: TestClient) -> No
for weight in ("a-fit", "a-recency", "a-affinity", "a-taste"):
assert f'id="{weight}"' in page
assert 'id="a-taste" min="0" step="0.05" value="0.2"' in page
# What the model reads after a pull: a count and a score floor per stage.
assert "<legend>What the model reads</legend>" in template
for limit in ("a-extract-top", "a-extract-min", "a-summarise-top", "a-summarise-min"):
assert f'id="{limit}"' in template
assert 'id="a-extract-top" min="0" step="10" value="500"' in page
assert 'id="a-summarise-min" min="0" max="1" step="0.05" value="0.5"' in page
assert 'id="a-new"' in page and 'id="a-delete"' in page
assert "Save and score" in page
@@ -756,6 +762,20 @@ def test_a_gate_on_an_unknown_field_is_refused(client: TestClient) -> None:
assert client.post("/api/agents", json=body).status_code == 422
def test_an_agents_limits_are_saved_listed_and_checked(client: TestClient) -> None:
"""How deep the model stages read is the agent's own; a score outside
0..1 is refused."""
default = client.post("/api/agents", json={"name": "Wide"}).json()
assert default["limits"] == {"extract_top": 500, "extract_min": 0.3,
"summarise_top": 30, "summarise_min": 0.5}
narrow = {"extract_top": 50, "extract_min": 0.6, "summarise_top": 5, "summarise_min": 0.7}
saved = client.post("/api/agents", json={"name": "Narrow", "limits": narrow})
assert saved.status_code == 200 and saved.json()["limits"] == narrow
assert [a["limits"]["extract_top"] for a in client.get("/api/agents").json()] == [50, 500]
bad = client.post("/api/agents", json={"name": "Bad", "limits": {"extract_min": 2}})
assert bad.status_code == 422
def test_roles_by_agent_are_ordered_by_score_and_hide_the_gated(
client: TestClient, store: Store) -> None:
for sid in ("low", "high", "gated"):