Rank by the cut: fetch gate overrides in scoring, the cut per agent, stages read the cut, fetch text renamed from enrich, the feed kept to the fetch gate, and migration 32

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Opus 5.5 committed 2026-10-04 15:11:43 +02:00
1 parent 060ccf66e6
commit 617356e60c
35 files changed
+1054 -751

No files matched your search

+132 -67
View File
@@ -1,21 +1,29 @@
"""`score`: every open role, scored for every enabled search agent.
"""`score`: every open role, scored for every enabled search agent, and each
agent's cut.
A role is scored per agent, `(uid, agent) -> score`, because a role is only
good for a given search. For each agent:
good for a given search. Each agent first applies its fetch gate:
require gates a role that fails one keeps its row, with score 0 and the
reason ("remote not remote")
any gates one group: a role that matches none of them, and has the
column filled for at least one, is gated ("none of
country DK / remote remote")
prefer gates `weight` when the role matches, 0 when it does not
reason ("remote not remote"); the region, which nothing
lifts
any gates one group, the place part: a role that matches none of
them, and has the column filled for at least one, is
gated ("none of country DK / remote remote"); skipped for
a company set to All countries
exclude gates a role that matches one of the values keeps its row, with
score 0 and the value ("title account executive")
score 0 and the value ("title account executive"); those
on `title` are skipped for a company set to All roles
A gate rejects on evidence, never on absence: a role whose column is empty
is not gated by it. `extract` is what fills an empty column, and it reads
only the roles that rank, so gating on an empty column would keep those
roles out of extract's reach for good.
only the roles in the cut, so gating on an empty column would keep those
roles out of extract's reach for good. A hidden company (`affinity = -1`)
is not scored at all; a favourite passes the gate like any other company.
The final score, the feed's order, is the weighted sum of four terms over
the sum of the weights, so it stays 0..1:
fit cos(role vector, prompt vector) as a level: the embedding
model's calibrated `lo` is 0 and `hi` 1, clipped outside
(see `scale`), so a role's fit does not depend on what
@@ -29,25 +37,29 @@ roles out of extract's reach for good.
over this agent's votes; with only likes it is the liked mean,
with only passes the negated passed mean; no votes, no term
Beside the score, for an agent with a CV, `cv_fit`: how close the role is to
the CV, the best cosine between the role and the CV's chunks
(`embed.cv_chunks`, embedded as the prompts are), scaled like fit. It is no
term of the score; the judge reads the closest roles the score leaves out
(`Store.judge_rows`) and CV match orders the roles not judged yet by it.
A role with no vector is scored without `fit` and `taste`, unless `fit` is
all there is, when it is gated as "no vector". An agent with no votes scores
every role without `taste`; the direction is newsdesk's
`centroid(liked) - centroid(disliked)`. An agent with no prompt that has
text has no `fit` either, so nothing of it enters the cut by fit and its
scores do not decide what summarise reads (`Store._summarise_top`); a run
names such agents in one notice.
Every term is 0..1. The score is the weighted sum over the sum of the weights
(the agent's four and its prefer gates'), so it stays 0..1. A favourite
company (`affinity = 2`) skips the `any` group but not the `require` or
`exclude` gates, and
is scored like any other company with its affinity term at 1: nothing puts
its roles above a better fit. A hidden one (`-1`) is not scored at all. A
role with no vector is scored without `fit` and `taste`, unless `fit` is all
there is, when it is gated as "no vector".
An agent with no votes scores every role without `taste`; the direction is
newsdesk's `centroid(liked) - centroid(disliked)`. An agent with no prompt
that has text has no `fit` either: its scores are not a match, so it is left
out of what extract and summarise read (`Store._top_scope`), and a run names
such agents in one notice.
Beside the score, `fit` is stored as it is, and for an agent with a CV,
`cv_fit`: how close the role is to the CV, the best cosine between the role
and the CV's chunks (`embed.cv_chunks`, embedded as the prompts are), scaled
like fit. Neither carries the affinity term.
The cut is the roles that get fetch text, extract and judge (`cut_of`). It
is computed from two levels, each with the company's affinity as a bonus of
`w.affinity * affinity / 3` over the sum of the weights, the same size as in
the final score (a liked company about 0.05 above neutral, a favourite about
0.1): the fit level `fit + bonus` and the CV level `cv_fit + bonus`. Over
the roles that pass the fetch gate, the best `cut_fit_top` by fit level at
or above `cut_fit_min` are `fit`; the best `cut_cv_top` by CV level at or
above `cut_cv_min` that are not in yet are `cv`; every role this agent
liked or anybody saved that is not in yet is `kept`. Newness and taste are
no part of it, and the two lists are not merged into one number.
Recomputed in full on every run, and each agent's rows are replaced in one
transaction: nothing is incremental, so nothing can drift.
@@ -67,7 +79,7 @@ import numpy as np
from pipeline import NULL, Notice, RunContext, job
from app.config import Calibration, Config
from app.jobs import embed
from app.models import Agent, Candidate, Gate, Weights
from app.models import Agent, Candidate, Gate, Limits, Weights
from app.storage import Store
log = logging.getLogger(__name__)
@@ -130,10 +142,10 @@ def scale(cosines: dict[str, float],
embedding model's calibrated `band` (`app.config.CALIBRATION`).
Raw cosines on multilingual-e5-small sit between about 0.75 and 0.87 for
real postings against a real intent, so unscaled a prefer gate weighted
0.2 moved a score more than the whole spread of fit. The band stretches
that spread to 0..1 with fixed constants, so a role's level is the same
in every run and a `limits` bar is an absolute threshold.
real postings against a real intent, so unscaled the whole spread of fit
was smaller than a recency step. The band stretches that spread to 0..1
with fixed constants, so a role's level is the same in every run and a
minimum of the cut is an absolute threshold.
Without a band, for a model nobody calibrated, it falls back to min-max
across the run: the closest role is 1 and the farthest 0, which ranks
@@ -165,19 +177,20 @@ def _reason(gate: Gate) -> str:
def failed_gates(candidate: Candidate, gates: Sequence[Gate]) -> list[str]:
"""Why this role is gated, as reasons: every require gate it fails, every
exclude gate it matches, and the `any` group when it matches none of them.
A gate with an empty column (state -1) neither passes nor fails, so the
group fails only when some member had evidence. A favourite company's
roles skip the `any` group but not the require and exclude gates, so a
star widens a search to a company's roles within e.g. Europe, not to the
whole world, and not to the titles the agent leaves out."""
"""Why the fetch gate rejects this role, as reasons: every require gate
it fails, every exclude gate it matches, and the `any` group when it
matches none of them. A gate with an empty column (state -1) neither
passes nor fails, so the group fails only when some member had evidence.
A company set to All countries skips the `any` group, the place part,
and one set to All roles the exclude gates on `title`, the titles to
leave out; the require gates, the region, apply to every company."""
states = list(zip(gates, candidate.gates, strict=True))
reasons = [_reason(g) for g, state in states if g.mode == "require" and state == 0]
reasons += [f"{g.field} {g.values[state - 1]}" for g, state in states
if g.mode == "exclude" and state > 0]
if g.mode == "exclude" and state > 0
and not (candidate.all_roles and g.field == "title")]
group = [(g, state) for g, state in states if g.mode == "any"]
if candidate.affinity != 2 and group and not any(state == 1 for _, state in group) \
if not candidate.all_countries and group and not any(state == 1 for _, state in group) \
and any(state == 0 for _, state in group):
reasons.append("none of " + ", ".join(_wanted(g) for g, _ in group))
return reasons
@@ -186,17 +199,15 @@ def failed_gates(candidate: Candidate, gates: Sequence[Gate]) -> list[str]:
def score_one(candidate: Candidate, gates: Sequence[Gate], weights: Weights,
fit: float | None, now: datetime, *, taste: float | None = None,
missing: str = "no vector") -> tuple[float, str]:
"""One role for one agent: `(score, gated_reason)`. `fit` and `taste` are
already scaled to 0..1 (see `scale`), or None when there is nothing to
measure: no role vector, no prompt vector, no votes. A None term leaves
the denominator; `missing` says why fit is out."""
"""One role for one agent: `(final score, gated_reason)`. `fit` and
`taste` are already scaled to 0..1 (see `scale`), or None when there is
nothing to measure: no role vector, no prompt vector, no votes. A None
term leaves the denominator; `missing` says why fit is out."""
if failed := failed_gates(candidate, gates):
return 0.0, ", ".join(failed)
terms = [(weights.recency, recency(candidate.posted_at, now)),
(weights.affinity, affinity_term(candidate.affinity))]
terms += [(g.weight, 1.0 if state == 1 else 0.0)
for g, state in zip(gates, candidate.gates, strict=True) if g.mode == "prefer"]
if fit is not None:
terms.append((weights.fit, fit))
if taste is not None:
@@ -208,6 +219,44 @@ def score_one(candidate: Candidate, gates: Sequence[Gate], weights: Weights,
return sum(w * t for w, t in terms) / total, ""
def bonus(affinity: int, weights: Weights) -> float:
"""What a company's affinity adds to both levels of the cut: the share
the affinity term gives it in the final score above a company at -1, so
a liked company sits about 0.05 above a neutral one and a favourite
about 0.1 on the default weights. 0 when every weight is 0."""
total = weights.fit + weights.recency + weights.affinity + weights.taste
return weights.affinity * affinity / 3 / total if total else 0.0
def cut_of(candidates: Sequence[Candidate], passed: set[str], fits: dict[str, float],
near: dict[str, float], kept: set[str], weights: Weights,
limits: Limits) -> dict[str, str]:
"""`uid -> why` for the roles in the cut, among `candidates` whose uid
is in `passed` (the fetch gate let them through): `fit` for the best
`limits.cut_fit_top` by fit level at or above `cut_fit_min`, `cv` for
the best `cut_cv_top` by CV level at or above `cut_cv_min` not in yet,
and `kept` for those in `kept` (liked or saved) not in yet. A level is
`fits` or `near` plus the company's `bonus`; a role without one is in
no list by it. Ties go to the newer role, then the lower uid."""
by_uid = {c.uid: c for c in candidates if c.uid in passed}
# Newest first, then uid, as the sort below keeps equal levels in order.
ordered = sorted(sorted(by_uid.values(), key=lambda c: c.uid),
key=lambda c: c.posted_at, reverse=True)
cut: dict[str, str] = {}
for why, levels, top, floor in (("fit", fits, limits.cut_fit_top, limits.cut_fit_min),
("cv", near, limits.cut_cv_top, limits.cut_cv_min)):
scored = [(levels[c.uid] + bonus(c.affinity, weights), c.uid) for c in ordered
if c.uid in levels and c.uid not in cut]
scored = [(level, uid) for level, uid in scored if level >= floor]
scored.sort(key=lambda pair: pair[0], reverse=True)
for _, uid in scored[:top]:
cut[uid] = why
for c in ordered:
if c.uid in kept and c.uid not in cut:
cut[c.uid] = "kept"
return cut
# ------------------------------------------------------------------ vectors
def intent_key(model: str, intent: str) -> str:
@@ -301,12 +350,13 @@ def _cv(store: Store, agent: Agent, model: str, encoder: _Encoder) -> np.ndarray
def no_intent(names: Sequence[str]) -> Notice:
"""The notice for enabled agents without a prompt that has text: their
scores have no fit, so `Store._top_scope` keeps them from feeding extract
and summarise. The code keeps its name from when an agent had one intent."""
scores have no fit, so nothing of theirs enters the cut by fit and
their final scores do not choose what summarise reads. The code keeps
its name from when an agent had one intent."""
return Notice(
code="score.no_intent", level="warning", kind="expected", key="score",
text=f"No prompt with text, so ranked without fit and not read by extract "
f"or summarise: {', '.join(names)}.",
text=f"No prompt with text, so ranked without fit and nothing in the cut by fit: "
f"{', '.join(names)}.",
fix="Give the agent a prompt under Settings, Search agents.",
where="app.jobs.score")
@@ -314,7 +364,8 @@ def no_intent(names: Sequence[str]) -> Notice:
def score_agent(store: Store, agent: Agent, vectors: dict[str, np.ndarray],
prompts: Sequence[tuple[str, np.ndarray]], now: datetime,
calibration: Calibration | None = None,
cv: np.ndarray | None = None) -> dict[str, int]:
cv: np.ndarray | None = None,
saved: set[str] | None = None) -> dict[str, int]:
"""Two passes: the cosines of the roles this agent scores, to each of its
`prompts` and to its taste direction, then every score with both scaled
by the model's `calibration`, or across the run without one. A role's fit
@@ -327,9 +378,12 @@ def score_agent(store: Store, agent: Agent, vectors: dict[str, np.ndarray],
share. The voted roles are scored like any other; a liked one sits near
the direction it helped make, which is what a like should do. With `cv`
(`_cv`), each of those roles also gets its `cv_fit`, its `closeness` to
the CV scaled like fit, beside the score and not in it."""
the CV scaled like fit. Then the cut (`cut_of`), with this agent's likes
and the `saved` roles (`Store.saved_uids`, read here when not given)
kept."""
candidates = store.score_candidates(agent.id, agent.gates)
direction = taste_direction(vectors, store.votes(agent.id))
votes = store.votes(agent.id)
direction = taste_direction(vectors, votes)
cosines: dict[str, dict[str, float]] = {name: {} for name, _ in prompts}
tastes: dict[str, float] = {}
near: dict[str, float] = {}
@@ -351,22 +405,29 @@ def score_agent(store: Store, agent: Agent, vectors: dict[str, np.ndarray],
fits[uid], best[uid] = value, name
tastes = scale(tastes, calibration.taste if calibration else None)
near = scale(near, calibration.cv_band if calibration else None)
rows: list[tuple[str, float, str, str, float | None]] = []
gated = 0
scored: list[tuple[str, float, str, float | None]] = []
passed: set[str] = set()
for candidate in candidates:
fit = fits.get(candidate.uid)
value, reason = score_one(
candidate, agent.gates, agent.weights, fit, now,
taste=tastes.get(candidate.uid),
missing="no vector" if prompts else "no prompt vector")
gated += bool(reason)
if not reason:
passed.add(candidate.uid)
scored.append((candidate.uid, value, reason, fit))
kept = {uid for uid, vote in votes.items() if vote == 1}
kept |= store.saved_uids() if saved is None else saved
cut = cut_of(candidates, passed, fits, near, kept, agent.weights, agent.limits)
rows = []
for uid, value, reason, fit in scored:
counted = fit is not None and agent.weights.fit and not reason
cv_fit = near.get(candidate.uid)
rows.append((candidate.uid, round(value, 6), reason,
best[candidate.uid] if counted else "",
None if cv_fit is None else round(cv_fit, 6)))
cv_fit = near.get(uid)
rows.append((uid, round(value, 6), reason, best[uid] if counted else "",
None if cv_fit is None else round(cv_fit, 6),
None if fit is None else round(fit, 6), cut.get(uid, "")))
store.replace_scores(agent.id, rows)
return {"scored": len(rows), "gated": gated}
return {"scored": len(rows), "gated": len(rows) - len(passed), "cut": len(cut)}
@job("score", "Scoring roles for every agent", solo=True, resource="cpu")
@@ -386,17 +447,21 @@ def run_score(config: Config, store: Store, *,
if blind := [agent.name for agent in agents
if not any(p.text.strip() for p in agent.prompts)]:
context.notice(no_intent(blind))
scored = gated = 0
scored = gated = cut = 0
saved = store.saved_uids()
for n, agent in enumerate(agents):
context.check()
context.progress(n, len(agents), f"scoring for {agent.name}, {n + 1} of {len(agents)}")
counts = score_agent(store, agent, vectors, _prompts(store, agent, model, encoder),
now, calibration, cv=_cv(store, agent, model, encoder))
now, calibration, cv=_cv(store, agent, model, encoder),
saved=saved)
scored += counts["scored"]
gated += counts["gated"]
log.info("%-20s %6d scored, %6d gated", agent.name, counts["scored"], counts["gated"])
cut += counts["cut"]
log.info("%-20s %6d scored, %6d gated, %5d in the cut", agent.name, counts["scored"],
counts["gated"], counts["cut"])
context.progress(len(agents), len(agents), "")
return {"agents": len(agents), "scored": scored, "gated": gated,
return {"agents": len(agents), "scored": scored, "gated": gated, "cut": cut,
"vectors": len(vectors)}