Score roles per search agent, and run pull into extract, embed and score

embed stores one vector per open role; score rates every open role for
every enabled agent from require and prefer gates, fit against the
agent's intent, recency and company affinity, keeping gated rows with a
reason. Schema v3 adds vectors and the cached intent vector. The page
gets an agent order, the score on each row, and an Agents tab. Two
starter agents in roles.yaml are imported once.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Opus 5.5 committed 2026-09-23 12:14:21 +02:00
1 parent 4dfb8831aa
commit bc60324fab
22 files changed
+1457 -44

No files matched your search

+18
View File
@@ -26,5 +26,23 @@ extract:
enabled: false
model: claude-haiku-4-5-20251001
# A role and each agent's intent as vectors, for the fit term of a score
# (roledesk/jobs/embed.py). Needs sentence-transformers; without it `embed`
# posts a notice and scores leave fit out. Changing the model re-embeds.
embed:
model: sentence-transformers/all-MiniLM-L6-v2
per_run: 50000
# Starter search agents, imported once into an empty `agents` table. Each gate
# is {field, values, mode: require|prefer, weight}; fields are place, country,
# remote, seniority, function, salary_min, currency, permit, languages and
# company. weights default to {fit: 0.6, recency: 0.2, affinity: 0.2}.
agents:
- name: Berlin backend
intent: Senior backend engineer, Python services and PostgreSQL.
gates:
- {field: place, values: [Berlin], mode: require}
- {field: remote, values: [hybrid, remote], mode: prefer, weight: 0.2}
# One block per plugin. Empty until the ats plugin lands.
sources: {}
+14
View File
@@ -63,6 +63,20 @@ at least the listing.
**Summary and tagging are off; `extract` replaces them.** Every profile declares `SummaryPolicy(enabled=False)` and `TaggingPolicy(enabled=False)`. `extract` is a roledesk `ModelTask` on Haiku that turns title and description into the gate columns, and fills only the columns a plugin left empty.
**A score is the weighted sum over the sum of the weights.** The agent's `fit`, `recency` and `affinity` weights and every prefer gate's `weight` form one denominator, so a score stays in 0..1 and the 0.9 floor for a favourite means the same thing whatever the weights are. A prefer gate normalised by the prefer weights alone, then weighted as one term, comes to the same number.
**Gates match loosely where the column is free text.** `place` is a case-insensitive substring, like the place filter, so `Berlin` matches "Berlin, Germany". `company` compares company keys. `languages` passes when every language the role requires is one of the values. `salary_min` is at least the smallest value. The rest are case-insensitive equality. The gates are evaluated in SQL as one `CASE` column each, so a `require` gate is the `WHERE` it reads as while the gated row and its reason survive. The reason lists every failed require gate, not just the first.
**A role with no vector is scored without `fit`.** The fit weight leaves the denominator, so a role `embed` has not reached yet ranks on recency, affinity and prefer gates instead of sinking to the bottom. It is gated as "no vector" only when fit is the only weighted term. An agent whose intent could not be embedded (empty, or no sentence-transformers) says "no intent vector" instead.
**Each run replaces an agent's scores.** `replace_scores` deletes and inserts in one transaction, so a role closed or muted since the last run drops out rather than keeping a stale score, and a reader sees one list or the other.
**Starter agents are imported once, by a state marker.** newsdesk seeds whenever its table is empty. Here `agents:seeded` in `state` is set after the import, so deleting every agent survives a restart.
**`embed` and `score` run in the server process.** newsdesk isolates `embed` in a child process with a timeout. roledesk's run is one bounded pass over open roles, and in-process keeps the embedder replaceable in tests. The cost is torch's memory staying in the server after the first run; isolate it if that becomes the problem.
**The pull cascade is `pull → extract → embed → score`.** Every stage is always in the chain and a stage that is off finds nothing, as in newsdesk. Extract precedes score so the gates read what it filled, and reads `on_ingest` only as a stage of a pull, so with `auto` and no model reachable a pull passes it quietly.
## The alpha page
**The template renders an empty shell; `static/app.js` fills it and drives every action.** `index.html` carries only the filter bar's `<select>` options (from `roledesk.models` and `plugins.ats.FUNCTIONS`, so they can't drift from what the store accepts) and the section containers. Every row, count and button state is a fetch from the JS, including the initial load — one code path for "show this" rather than a server-rendered first page plus a client re-render for every filter change after.
+10 -8
View File
@@ -1,4 +1,4 @@
"""`roledesk serve | job pull|extract | companies seed|add|check | db init`."""
"""`roledesk serve | job pull|extract|embed|score | companies seed|add|check | db init`."""
from __future__ import annotations
@@ -37,7 +37,7 @@ def build_parser() -> argparse.ArgumentParser:
serve.add_argument("--port", type=int, default=8790)
job = sub.add_parser("job", help="queue a job or task and drain the queue once")
job.add_argument("kind", choices=["pull", "extract"])
job.add_argument("kind", choices=["pull", "extract", "embed", "score"])
companies = sub.add_parser("companies", help="the target company list")
companies_sub = companies.add_subparsers(dest="action", required=True)
@@ -76,8 +76,8 @@ def main(argv: list[str] | None = None) -> int:
def run_job(args: argparse.Namespace) -> int:
"""Queue `kind` — a job, or a task from `pipeline.TASKS` such as `pull` —
then run everything queued until the queue is idle, and print how it
ended. A task's own result is its final job's, which is the one whose
`task_id` equals the run this started."""
ended: one line per task, from each task's final job. `pull` is a
cascade, so it prints a line for every stage it went through."""
from pipeline import FINAL, JobQueue, Jobs, TASKS
from roledesk import jobs
from roledesk.storage import Store
@@ -90,12 +90,14 @@ def run_job(args: argparse.Namespace) -> int:
if args.kind in TASKS:
task_id = queue.start_task(TASKS[args.kind]())
queue.drain()
final = next((j for j in queue.jobs.of_task(task_id) if j.kind == FINAL), None)
if final is not None:
started = queue.jobs.of_task(task_id)
if not started:
return 0
finals = [j for j in queue.jobs.of_cascade(started[0].cascade_id) if j.kind == FINAL]
for final in finals:
print(json.dumps({"id": final.id, "state": final.state,
"result": final.result, "error": final.error}))
return 0 if final.state == "done" else 1
return 0
return 0 if all(f.state == "done" for f in finals) else 1
queued = queue.enqueue(args.kind)
queue.drain()
if queued is not None:
+27
View File
@@ -14,6 +14,7 @@ import yaml
from pydantic import Field, ValidationError
from desk.config import AgentConfig, ModelTaskConfig, Section, WorkersConfig
from roledesk.models import Gate, Weights
DEFAULT_PATH = Path(os.environ.get("ROLEDESK_CONFIG", "roles.yaml"))
@@ -28,6 +29,29 @@ class ExtractConfig(ModelTaskConfig):
batch_size: int = Field(20, ge=1)
class EmbedConfig(Section):
"""Turning a role into a vector for `fit`. Needs sentence-transformers;
without it `embed` says so and does nothing, and `score` leaves fit out.
See roledesk/jobs/embed.py."""
#: Recorded on every vector. Changing it drops the old vectors, which are
#: a different geometry and never comparable.
model: str = "sentence-transformers/all-MiniLM-L6-v2"
#: Roles embedded per run at most, newest first.
per_run: int = Field(50_000, ge=1)
class AgentSeed(Section):
"""A search agent as `roles.yaml` lists it. Imported into an empty
`agents` table once; after that the table is the answer."""
name: str
intent: str = ""
gates: list[Gate] = Field(default_factory=list)
weights: Weights = Field(default_factory=Weights)
enabled: bool = True
class Config(Section):
"""The whole file."""
@@ -36,6 +60,9 @@ class Config(Section):
workers: WorkersConfig = Field(default_factory=WorkersConfig)
agent: AgentConfig = Field(default_factory=AgentConfig)
extract: ExtractConfig = Field(default_factory=ExtractConfig)
embed: EmbedConfig = Field(default_factory=EmbedConfig)
#: Starter agents, imported once into an empty table.
agents: list[AgentSeed] = Field(default_factory=list)
#: One block per plugin: `enabled`, and whatever that plugin reads.
sources: dict[str, Any] = Field(default_factory=dict)
#: Where this was loaded from. Never written back into the file.
+9 -3
View File
@@ -1,12 +1,17 @@
"""The work roledesk queues. Importing this package registers every job.
"""The work roledesk queues. Importing this package registers every job and
every task in `pipeline.TASKS`.
pull.py read every configured plugin and store the roles it yields
extract.py title and description into the gate columns, on Haiku
embed.py one vector per open role, for fit
score.py every open role scored for every enabled search agent
cascade.py pull → extract → embed → score, and what `TASKS` offers
"""
from __future__ import annotations
from roledesk.jobs import extract, pull
from roledesk.jobs import cascade, embed, extract, pull, score
from roledesk.jobs.score import seed_agents
#: What `JobQueue(bootstrap=...)` and `pipeline.worker --bootstrap` are given.
BOOTSTRAP = "roledesk.jobs:bootstrap"
@@ -22,4 +27,5 @@ def bootstrap(config_path: str = "") -> tuple[object, object]:
return config, Store(config.database)
__all__ = ["BOOTSTRAP", "bootstrap", "extract", "pull"]
__all__ = ["BOOTSTRAP", "bootstrap", "cascade", "embed", "extract", "pull", "score",
"seed_agents"]
+105
View File
@@ -0,0 +1,105 @@
"""The tasks roledesk offers, and the order a pull runs them in.
pull → extract → embed → score
newsdesk's cascade convention: a cascade is a tuple of task names, and the
final job of one task starts the next (`step` is the callback it carries).
Nothing is alive between two tasks.
Every stage is always in the chain, and a stage that is switched off finds
nothing: `extract.enabled: false`, or an `on_ingest` this pull does not
satisfy, is an extract stage with no batches whose final job moves straight
on. Extract runs before score so the gates read what it filled; before embed
too, since embedding does not read its columns and the order costs nothing.
A batch that failed, or a stage somebody killed, does not stop the chain: the
work comes round on the next pull, and a rescore of what did arrive is worth
having either way.
"""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import replace
from functools import partial
from typing import Any
from desk.tools import agent
from pipeline import TASKS, Tally, Task
from roledesk.jobs import embed, extract, pull, score # noqa: F401 -- registers the jobs
STEP = "roledesk.jobs.cascade:step"
CASCADES: dict[str, tuple[str, ...]] = {
"pull": ("pull", "extract", "embed", "score"),
}
def _once(config: Any, store: Any) -> list[dict[str, Any]]:
"""One batch with nothing in its payload: a task that is a single job."""
return [{}]
def _extract(cascade: str) -> Task:
"""`extract` as a stage, reading `on_ingest` when it is one of a pull.
`never`, or `auto` with no model reachable, is a stage that finds
nothing. Standing on its own, as the button, it reads neither."""
stage = extract.extract_task()
if not cascade:
return stage
find = stage.finder
def during_a_pull(config: Any, store: Any) -> list[Any]:
when = config.extract.on_ingest
if when == "never" or (when == "auto" and not agent.backend(config.agent)):
return []
return find(config, store)
return replace(stage, finder=during_a_pull)
STAGES: dict[str, Callable[[str], Task]] = {
"pull": lambda cascade: pull.pull_task(),
"extract": _extract,
"embed": lambda cascade: Task(kind="embed", finder=_once, label="Embedding roles"),
"score": lambda cascade: Task(kind="score", finder=_once,
label="Scoring roles for every agent"),
}
def task(name: str, *, cascade: str = "") -> Task:
"""One stage as a `Task`. With `cascade` its final job carries `step`,
and the next stage follows; without, it stops when its batches do."""
return replace(STAGES[name](cascade), name=name,
callback=STEP if cascade else "",
carries={"cascade": cascade, "task": name})
def follows(cascade: str, name: str) -> str:
"""What runs after `name`, or "" at the end or for an unknown name."""
tasks = CASCADES.get(cascade, ())
if name not in tasks:
return ""
at = tasks.index(name)
return tasks[at + 1] if at + 1 < len(tasks) else ""
def step(config: Any, store: Any, tally: Tally, payload: dict[str, Any], *,
context: Any) -> dict[str, Any]:
"""The callback: make the next task. A killed stage is skipped and the
chain goes on, as in newsdesk's cascade."""
name, done = payload.get("cascade", ""), payload.get("task", "")
following = follows(name, done)
if following:
context.task(task(following, cascade=name))
out: dict[str, Any] = {"task": done, "next": following}
if tally.killed:
out["skipped"] = "cancelled"
return out
#: What `roledesk job <name>` and the Jobs view offer. `pull` is the whole
#: chain; the others are one stage each, on their own.
TASKS["pull"] = partial(task, "pull", cascade="pull")
for _name in ("extract", "embed", "score"):
TASKS[_name] = partial(task, _name)
+79
View File
@@ -0,0 +1,79 @@
"""`embed`: one vector per open role, for the `fit` term of a score.
The encoder is `desk.tools.embedding`; the table is roledesk's own `vectors`
(see `Store.put_vectors`). What gets encoded is the title and the opening of
the description, which is what says what a role is.
sentence-transformers is optional, as in newsdesk. Without it this task puts
one notice on screen and does nothing; `score` then leaves `fit` out and ranks
on what it has.
"""
from __future__ import annotations
import logging
from typing import Any
from desk.tools import embedding
from pipeline import NULL, Notice, RunContext, job
from roledesk.config import Config
from roledesk.models import Role
from roledesk.storage import Store
log = logging.getLogger(__name__)
#: How much of the description the encoder sees. MiniLM truncates at 256
#: tokens anyway; 2000 characters is past that for any language.
BODY_CHARS = 2000
#: Roles encoded per call, and per commit: a kill loses at most one chunk.
CHUNK = 64
def document(role: Role) -> str:
return f"{role.title}\n{role.description[:BODY_CHARS]}"
def load_embedder(model: str) -> embedding.Embedder:
"""The encoder. A seam of its own so a test hands in a fake and nothing
downloads a model; `score` loads through it too."""
return embedding.load(model)
def unavailable(exc: BaseException) -> Notice:
"""The notice for a machine without sentence-transformers. One code and
key for `embed` and `score`, so the two say it as one line."""
return Notice(
code="embed.unavailable", level="warning", kind="expected", key="embed",
text="sentence-transformers is not installed, so roles are not embedded "
"and scores leave fit out.",
fix="Install it into roledesk's environment: "
"`uv pip install sentence-transformers`.",
detail=str(exc), where="roledesk.jobs.embed")
@job("embed", "Embedding roles", solo=True, resource="cpu")
def run_embed(config: Config, store: Store, *,
context: RunContext = NULL) -> dict[str, Any]:
"""Embed every open role that has no vector from the configured model."""
model = config.embed.model
try:
embedder = load_embedder(model)
except ImportError as exc:
log.warning("%s", exc)
context.notice(unavailable(exc))
return {"embedded": 0, "skipped": True, "reason": "sentence-transformers missing"}
dropped = store.drop_vectors(keep=model)
if dropped:
log.info("dropped %d vectors from a previous embedding model", dropped)
roles = store.roles_to_embed(model, config.embed.per_run)
for at in range(0, len(roles), CHUNK):
context.check()
context.progress(at, len(roles), f"embedding roles, {at} of {len(roles)}")
chunk = roles[at:at + CHUNK]
vectors = embedder.encode([document(r) for r in chunk])
store.put_vectors(model, [(r.uid, v) for r, v in zip(chunk, vectors, strict=True)])
context.progress(len(roles), len(roles),
f"embedding roles, {len(roles)} of {len(roles)}")
return {"embedded": len(roles), "dropped": dropped}
+3 -6
View File
@@ -27,7 +27,7 @@ from typing import Any
from desk import modeltask
from desk.modeltask import ModelTask
from pipeline import NULL, RunContext, TASKS, Task, job
from pipeline import NULL, RunContext, Task, job
from roledesk.config import Config
from roledesk.models import PERMITS, REMOTE, SENIORITIES, ExtractResult, Role
from roledesk.storage import Store
@@ -219,9 +219,6 @@ def run_batch(config: Config, store: Store, *, context: RunContext = NULL,
def extract_task() -> Task:
"""Finds nothing while `extract.enabled` is false, so the task, the
button and the cascade stage are all no-ops with the switch off."""
return replace(modeltask.of(TASK), label=TASK.label)
#: `roledesk job extract` and the Jobs view's Extract. Finds nothing while
#: `extract.enabled` is false, so pressing it with the switch off is a no-op.
TASKS["extract"] = extract_task
+4 -7
View File
@@ -20,7 +20,7 @@ import logging
from datetime import UTC, datetime
from typing import Any
from pipeline import NULL, RunContext, TASKS, Tally, Task, job
from pipeline import NULL, RunContext, Tally, Task, job
from roledesk.config import Config
from roledesk.storage import Store
@@ -103,11 +103,8 @@ def close_boards(config: Config, store: Store, tally: Tally, *,
return {"boards": answered, "failed": failed, "roles": roles, "closed": closed}
def _pull_task() -> Task:
def pull_task() -> Task:
"""The pull on its own. `roledesk.jobs.cascade` makes it the first stage
of the chain that `TASKS["pull"]` starts."""
return Task(kind="pull_batch", finder=_boards, name="pull",
label="Reading every board", settle=SETTLE)
#: Offered as `roledesk job pull` and to `<desk-jobs>`'s Pull button, the same
#: way newsdesk's cascade stages register themselves in `TASKS`.
TASKS["pull"] = _pull_task
+217
View File
@@ -0,0 +1,217 @@
"""`score`: every open role, scored for every enabled search agent.
A role is scored per agent, `(uid, agent) -> score`, because a role is only
good for a given search. For each agent:
require gates a role that fails one keeps its row, with score 0 and the
reason ("place unknown", "remote not remote")
prefer gates `weight` when the role matches, 0 when it does not
fit cos(role vector, intent vector), mapped to 0..1
recency 0.5 ** (days since first_seen / 14)
affinity (companies.affinity + 1) / 3
Every term is 0..1. The score is the weighted sum over the sum of the weights
(the agent's three and its prefer gates'), so it stays 0..1. A favourite
company (`affinity = 2`) skips the gates and is floored at 0.9; a muted one
(`-1`) is not scored at all. A role with no vector is scored without `fit`,
unless `fit` is all there is, when it is gated as "no vector".
Recomputed in full on every run, and each agent's rows are replaced in one
transaction: nothing is incremental, so nothing can drift.
"""
from __future__ import annotations
import hashlib
import logging
import math
from collections.abc import Sequence
from datetime import datetime
from typing import Any
import numpy as np
from pipeline import NULL, RunContext, job
from roledesk.config import Config
from roledesk.jobs import embed
from roledesk.models import Agent, Candidate, Gate, Weights
from roledesk.storage import Store
log = logging.getLogger(__name__)
HALF_LIFE_DAYS = 14.0
#: Where a favourite company's roles start, whatever else they score.
FAVOURITE_FLOOR = 0.9
#: The `state` key that records the starter agents were imported.
SEEDED = "agents:seeded"
# ------------------------------------------------------------------- terms
def recency(first_seen: str, now: datetime) -> float:
"""1.0 for a role first seen now, halving every `HALF_LIFE_DAYS`."""
try:
seen = datetime.fromisoformat(first_seen)
except ValueError:
return 0.0
days = max((now - seen).total_seconds(), 0.0) / 86400
return math.pow(0.5, days / HALF_LIFE_DAYS)
def affinity_term(affinity: int) -> float:
"""-1..2 to 0..1. -1 never gets here; neutral is 1/3."""
return (affinity + 1) / 3
def fit_term(role: np.ndarray, intent: np.ndarray) -> float:
"""Cosine to 0..1. Rescaled rather than clipped, as newsdesk does, so a
role pointing away from the intent sits below an unrelated one."""
norms = float(np.linalg.norm(role) * np.linalg.norm(intent))
if norms == 0:
return 0.5
return (float(np.dot(role, intent)) / norms + 1.0) / 2.0
def _reason(gate: Gate, state: int) -> str:
if state < 0:
return f"{gate.field} unknown"
if gate.field == "salary_min":
return f"salary_min under {min(gate.values, key=lambda v: float(v))}"
return f"{gate.field} not {' / '.join(str(v) for v in gate.values)}"
def score_one(candidate: Candidate, gates: Sequence[Gate], weights: Weights,
fit: float | None, now: datetime, *,
missing: str = "no vector") -> tuple[float, str]:
"""One role for one agent: `(score, gated_reason)`. `fit` is None when
the role or the intent has no vector; `missing` says which."""
favourite = candidate.affinity == 2
if not favourite:
failed = [_reason(g, state) for g, state in zip(gates, candidate.gates, strict=True)
if g.mode == "require" and state != 1]
if failed:
return 0.0, ", ".join(failed)
terms = [(weights.recency, recency(candidate.first_seen, now)),
(weights.affinity, affinity_term(candidate.affinity))]
terms += [(g.weight, 1.0 if state == 1 else 0.0)
for g, state in zip(gates, candidate.gates, strict=True) if g.mode == "prefer"]
if fit is not None:
terms.append((weights.fit, fit))
total = sum(w for w, _ in terms)
if not total:
if favourite:
return FAVOURITE_FLOOR, ""
# Fit was the only term with any weight, and there is no fit.
return 0.0, missing if fit is None and weights.fit else "no weights"
score = sum(w * t for w, t in terms) / total
return (max(score, FAVOURITE_FLOOR) if favourite else score), ""
# ------------------------------------------------------------------ vectors
def intent_key(model: str, intent: str) -> str:
return hashlib.sha1(f"{model}\n{intent}".encode()).hexdigest()
class _Encoder:
"""The embedder, loaded on first use only: an agent whose intent vector
is cached costs no model load at all."""
def __init__(self, model: str, context: RunContext) -> None:
self.model = model
self.context = context
self._embedder: Any = None
self.failed = False
def encode(self, text: str) -> np.ndarray | None:
if self.failed:
return None
if self._embedder is None:
try:
self._embedder = embed.load_embedder(self.model)
except ImportError as exc:
self.failed = True
self.context.notice(embed.unavailable(exc))
return None
return np.asarray(self._embedder.encode([text])[0])
def _intent(store: Store, agent: Agent, model: str,
encoder: _Encoder) -> np.ndarray | None:
if not agent.intent.strip():
return None
key = intent_key(model, agent.intent)
cached = store.intent_vector(agent.id, key)
if cached is not None:
return cached
vec = encoder.encode(agent.intent)
if vec is not None:
store.set_intent_vector(agent.id, key, vec)
return vec
# ------------------------------------------------------------------- the job
def score_agent(store: Store, agent: Agent, vectors: dict[str, np.ndarray],
intent: np.ndarray | None, now: datetime) -> dict[str, int]:
rows: list[tuple[str, float, str]] = []
gated = 0
for candidate in store.score_candidates(agent.gates):
role_vec = vectors.get(candidate.uid)
fit = fit_term(role_vec, intent) if role_vec is not None and intent is not None \
else None
value, reason = score_one(
candidate, agent.gates, agent.weights, fit, now,
missing="no vector" if intent is not None else "no intent vector")
gated += bool(reason)
rows.append((candidate.uid, round(value, 6), reason))
store.replace_scores(agent.id, rows)
return {"scored": len(rows), "gated": gated}
@job("score", "Scoring roles for every agent", solo=True, resource="cpu")
def run_score(config: Config, store: Store, *,
context: RunContext = NULL) -> dict[str, Any]:
seed_agents(config, store)
agents = store.agents(enabled_only=True)
model = config.embed.model
vectors = store.vectors(model) if agents else {}
encoder = _Encoder(model, context)
now = store.clock()
scored = gated = 0
for n, agent in enumerate(agents):
context.check()
context.progress(n, len(agents), f"scoring for {agent.name}, {n + 1} of {len(agents)}")
counts = score_agent(store, agent, vectors, _intent(store, agent, model, encoder), now)
scored += counts["scored"]
gated += counts["gated"]
log.info("%-20s %6d scored, %6d gated", agent.name, counts["scored"], counts["gated"])
context.progress(len(agents), len(agents), "")
return {"agents": len(agents), "scored": scored, "gated": gated,
"vectors": len(vectors)}
# ------------------------------------------------------------------ seeding
def seed_agents(config: Config, store: Store) -> list[str]:
"""Import `roles.yaml`'s `agents:` into an empty table, once. The state
marker is what makes it once: deleting every agent afterwards is a
choice, and a restart must not undo it."""
if store.state.get(SEEDED) or store.agents():
return []
added = []
for seed in config.agents:
try:
store.save_agent(seed.name, seed.intent, seed.gates, seed.weights,
seed.enabled)
except ValueError as exc:
log.warning("agents: %s", exc)
continue
added.append(seed.name)
if added:
store.state[SEEDED] = store.now()
log.info("imported %d agents from %s", len(added), config.path or "roles.yaml")
return added
+66 -1
View File
@@ -15,7 +15,7 @@ import re
import unicodedata
from typing import Annotated, Any, Literal, get_args
from pydantic import BeforeValidator
from pydantic import BeforeValidator, Field
from desk.models import JsonDict, Model, ModelTaskResult
@@ -152,6 +152,14 @@ class ExtractResult(ModelTaskResult):
return self.extracted
class ScoredRole(Role):
"""A role as `/api/roles` serves it: with one agent's score and gate
reason when an agent was asked for, and without otherwise."""
score: float | None = None
gated_reason: str = ""
class Company(Model):
key: str
name: str
@@ -202,6 +210,63 @@ class Application(Model):
updated_at: str = ""
# ------------------------------------------------------------------ agents
#: The role columns a gate can name. `company` matches on the company key;
#: `place` is a case-insensitive substring, like the place filter; the rest are
#: case-insensitive equality, except `salary_min` (at least the value) and
#: `languages` (every language the role requires is one of the values).
GateField = Literal["place", "country", "remote", "seniority", "function", "salary_min",
"currency", "permit", "languages", "company"]
GATE_FIELDS: tuple[str, ...] = get_args(GateField)
GateMode = Literal["require", "prefer"]
class Gate(Model):
"""One condition of an agent. `require` gates a role that fails it (the
row is kept with the reason); `prefer` adds `weight` to its score when it
matches."""
field: GateField
values: list[str | int | float] = Field(min_length=1)
mode: GateMode = "require"
weight: float = Field(1.0, ge=0)
class Weights(Model):
"""How much each score term counts. Summed with the prefer gates' weights
into the denominator, so a score stays in 0..1 whatever the numbers."""
fit: float = Field(0.6, ge=0)
recency: float = Field(0.2, ge=0)
affinity: float = Field(0.2, ge=0)
class Agent(Model):
"""A row of `agents`: a named search with its own intent, gates and
weights. The cached intent vector is a column this does not declare."""
id: int
name: str
intent: str = ""
gates: Annotated[list[Gate], BeforeValidator(lambda v: _json(v, "[]"))] = []
weights: Annotated[Weights, BeforeValidator(lambda v: _json(v, "{}"))] = Weights()
enabled: bool = True
created_at: str
class Candidate(Model):
"""What scoring reads about one open role for one agent: when it was
first seen, its company's affinity, and per gate `1` match, `0` no
match, `-1` the column is empty."""
uid: str
first_seen: str
affinity: int = 0
gates: list[int] = []
class ApplicationEvent(Model):
id: int
application_id: str
+27 -1
View File
@@ -142,6 +142,8 @@ END;
-- A search agent: a named saved search with its own intent text, gates and
-- weights. gates is a list of {field, values, mode: require|prefer, weight}.
-- intent_vec caches the intent's vector; intent_key is sha1 of the embedding
-- model and the intent text, so editing either makes it stale.
CREATE TABLE IF NOT EXISTS agents (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
@@ -149,7 +151,9 @@ CREATE TABLE IF NOT EXISTS agents (
gates TEXT NOT NULL DEFAULT '[]',
weights TEXT NOT NULL DEFAULT '{}',
enabled INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL
created_at TEXT NOT NULL,
intent_key TEXT NOT NULL DEFAULT '',
intent_vec BLOB
);
-- A role is scored per agent. A gated role keeps its row, with the reason.
@@ -162,6 +166,16 @@ CREATE TABLE IF NOT EXISTS scores (
PRIMARY KEY (uid, agent_id)
);
-- One vector per role, little-endian float32. Only comparable to another from
-- the same model, so every read is scoped by it.
CREATE TABLE IF NOT EXISTS vectors (
uid TEXT PRIMARY KEY REFERENCES roles(uid) ON DELETE CASCADE,
model TEXT NOT NULL,
dim INTEGER NOT NULL,
vec BLOB NOT NULL,
embedded_at TEXT NOT NULL
);
-- The string map a desk plugin is handed as self.state. One namespace shared
-- by every plugin, so a plugin prefixes its keys.
CREATE TABLE IF NOT EXISTS state (
@@ -182,6 +196,7 @@ CREATE INDEX IF NOT EXISTS idx_roles_board ON roles(source, subscription_id
CREATE INDEX IF NOT EXISTS idx_application_events
ON application_events(application_id, at);
CREATE INDEX IF NOT EXISTS idx_scores_agent ON scores(agent_id, score DESC);
CREATE INDEX IF NOT EXISTS idx_vectors_model ON vectors(model);
"""
#: The table whose absence marks a file that has never been a roledesk
@@ -199,6 +214,17 @@ ALTER TABLE roles ADD COLUMN permit TEXT NOT NULL DEFAULT ''
CHECK (permit IN ('', 'sponsors', 'no_sponsorship'));
ALTER TABLE roles ADD COLUMN languages TEXT NOT NULL DEFAULT '[]';
ALTER TABLE roles ADD COLUMN extract_tries INTEGER NOT NULL DEFAULT 0;
"""),
(3, "vectors, and the agent's cached intent vector, for score", """
CREATE TABLE vectors (
uid TEXT PRIMARY KEY REFERENCES roles(uid) ON DELETE CASCADE,
model TEXT NOT NULL,
dim INTEGER NOT NULL,
vec BLOB NOT NULL,
embedded_at TEXT NOT NULL
);
ALTER TABLE agents ADD COLUMN intent_key TEXT NOT NULL DEFAULT '';
ALTER TABLE agents ADD COLUMN intent_vec BLOB;
"""),
)
+207 -7
View File
@@ -17,10 +17,13 @@ from pathlib import Path
from typing import Any
from uuid import uuid4
import numpy as np
from desk.models import Item
from roledesk.models import (
AFFINITY, REMOTE, STATUSES, Application, ApplicationEvent, Company, Role,
Subscription, company_key, place_key, title_key,
AFFINITY, REMOTE, STATUSES, Agent, Application, ApplicationEvent, Candidate,
Company, Gate, Role, ScoredRole, Subscription, Weights, company_key, place_key,
title_key,
)
from roledesk.storage import schema
@@ -73,6 +76,43 @@ def _like(text: str) -> str:
return f"%{escaped}%"
#: Vectors are stored little-endian float32, fixed here so a file written on
#: one machine reads on another. newsdesk's encoding, written again.
_DTYPE = np.dtype("<f4")
def pack(vec: np.ndarray) -> bytes:
return np.asarray(vec, dtype=_DTYPE).tobytes()
def unpack(blob: bytes) -> np.ndarray:
return np.frombuffer(blob, dtype=_DTYPE)
def _gate_sql(gate: Gate) -> tuple[str, str, list[Any]]:
"""One gate as `(empty, match, params)`: SQL that is true when the role's
column is empty, and SQL that is true when it matches, over `roles r`.
`params` belong to `match`; `empty` takes none."""
column = f"r.{gate.field}"
values = [str(v).strip().lower() for v in gate.values if str(v).strip()]
marks = ", ".join("?" for _ in values) or "NULL"
if gate.field == "salary_min":
floor = min(_int_or_none(v) or 0 for v in gate.values)
return f"{column} IS NULL", f"{column} >= ?", [floor]
if gate.field == "place":
likes = " OR ".join("LOWER(r.place) LIKE ? ESCAPE '\\'" for _ in values) or "0"
return "r.place = ''", f"({likes})", [_like(v) for v in values]
if gate.field == "company":
keys = [company_key(v) for v in values]
return ("r.company_key = ''",
f"r.company_key IN ({', '.join('?' for _ in keys) or 'NULL'})", keys)
if gate.field == "languages":
return ("r.languages IN ('', '[]')",
f"NOT EXISTS (SELECT 1 FROM json_each(r.languages) "
f"WHERE LOWER(value) NOT IN ({marks}))", values)
return f"{column} = ''", f"LOWER({column}) IN ({marks})", values
def _int_or_none(value: Any) -> int | None:
if value is None or value == "" or isinstance(value, bool):
return None
@@ -229,7 +269,8 @@ class Store:
place: str | None = None, open_only: bool = True,
q: str | None = None, function: str | None = None,
remote: str | None = None, days: int | None = None,
hide_muted: bool = False) -> list[Role]:
hide_muted: bool = False, agent: int | None = None,
gated: bool = False) -> list[ScoredRole]:
"""Newest first by `first_seen`. `company` and `place` are
case-insensitive substrings of the columns a board wrote — not the
key, so a search does not have to know the exact slug a company folds
@@ -243,7 +284,11 @@ class Store:
finally reached it — can otherwise fill the whole page with roles
that then all turn out to be from the one company someone muted,
leaving nothing to show for a default query that would otherwise
have plenty."""
have plenty.
`agent` orders by that agent's score instead, highest first, and
carries `score` and `gated_reason` on each row. Only roles it has
scored are listed, and a gated one only when `gated` is set."""
where, params = [], []
if company:
where.append("company LIKE ? ESCAPE '\\'")
@@ -268,11 +313,22 @@ class Store:
if hide_muted:
where.append("company_key NOT IN "
"(SELECT key FROM companies WHERE affinity = -1)")
sql = "SELECT * FROM roles"
joins: list[Any] = []
if agent is None:
sql = "SELECT roles.* FROM roles"
order = "roles.first_seen DESC, roles.uid"
else:
sql = ("SELECT roles.*, s.score, s.gated_reason FROM roles "
"JOIN scores s ON s.uid = roles.uid AND s.agent_id = ?")
joins.append(agent)
if not gated:
where.append("s.gated_reason = ''")
order = "s.score DESC, roles.first_seen DESC, roles.uid"
if where:
sql += " WHERE " + " AND ".join(where)
sql += " ORDER BY first_seen DESC, uid LIMIT ?"
return [Role.from_row(r) for r in self._rows(sql, [*params, limit])]
sql += f" ORDER BY {order} LIMIT ?"
return [ScoredRole.from_row(r)
for r in self._rows(sql, [*joins, *params, limit])]
def mark_closed(self, source: str, seen_ids: Sequence[str],
subscription_id: int) -> int:
@@ -353,6 +409,150 @@ class Store:
f"WHERE {where}", (wanted,))
return int(crossed)
# -------------------------------------------------------------- vectors
def roles_to_embed(self, model: str, limit: int) -> list[Role]:
"""Open roles with no vector from `model`, newest first, so a backlog
embedded from the top improves what is on screen first."""
return [Role.from_row(r) for r in self._rows(
"""SELECT r.* FROM roles r
LEFT JOIN vectors v ON v.uid = r.uid AND v.model = ?
WHERE v.uid IS NULL AND r.closed_at IS NULL
ORDER BY r.first_seen DESC, r.uid LIMIT ?""", (model, limit))]
def put_vectors(self, model: str, pairs: Sequence[tuple[str, np.ndarray]]) -> int:
stamp = self.now()
with self.transaction() as conn:
return conn.executemany(
"""INSERT INTO vectors (uid, model, dim, vec, embedded_at)
VALUES (?,?,?,?,?)
ON CONFLICT(uid) DO UPDATE SET model=excluded.model,
dim=excluded.dim, vec=excluded.vec,
embedded_at=excluded.embedded_at""",
[(uid, model, int(vec.shape[-1]), pack(vec), stamp)
for uid, vec in pairs]).rowcount
def vectors(self, model: str) -> dict[str, np.ndarray]:
"""Every vector from `model`, by uid. The whole table: at ~30k roles of
384 floats that is ~45 MB, and one read beats one per agent."""
return {r["uid"]: unpack(r["vec"]) for r in self._rows(
"SELECT uid, vec FROM vectors WHERE model = ?", (model,))}
def vector_counts(self) -> dict[str, int]:
return {r["model"]: int(r["n"]) for r in self._rows(
"SELECT model, COUNT(*) AS n FROM vectors GROUP BY model")}
def drop_vectors(self, keep: str) -> int:
"""Delete vectors from every model but `keep`: another model's are a
different geometry, never compared, and only make counts lie."""
return self._write("DELETE FROM vectors WHERE model <> ?", (keep,))
# --------------------------------------------------------------- agents
def agents(self, enabled_only: bool = False) -> list[Agent]:
sql = "SELECT * FROM agents"
if enabled_only:
sql += " WHERE enabled = 1"
return [Agent.from_row(r)
for r in self._rows(sql + " ORDER BY name COLLATE NOCASE")]
def agent(self, agent_id: int) -> Agent | None:
row = self._row("SELECT * FROM agents WHERE id=?", (agent_id,))
return Agent.from_row(row) if row else None
def save_agent(self, name: str, intent: str = "", gates: Sequence[Gate] = (),
weights: Weights | None = None, enabled: bool = True,
agent_id: int | None = None) -> Agent:
"""Create an agent, or with `agent_id` replace that one's fields.
Raises ValueError on an empty or taken name, KeyError on an unknown id."""
name = name.strip()
if not name:
raise ValueError("an agent needs a name")
fields = (name, intent, json.dumps([g.model_dump() for g in gates]),
(weights or Weights()).model_dump_json(), int(enabled))
try:
with self.transaction() as conn:
if agent_id is None:
agent_id = conn.execute(
"""INSERT INTO agents (name, intent, gates, weights, enabled,
created_at) VALUES (?,?,?,?,?,?)""",
(*fields, self.now())).lastrowid
elif not conn.execute(
"""UPDATE agents SET name=?, intent=?, gates=?, weights=?,
enabled=? WHERE id=?""",
(*fields, agent_id)).rowcount:
raise KeyError(agent_id)
except sqlite3.IntegrityError as exc:
raise ValueError(f"an agent named {name!r} already exists") from exc
assert agent_id is not None
found = self.agent(agent_id)
assert found is not None
return found
def delete_agent(self, agent_id: int) -> bool:
"""Its scores go with it, by the foreign key."""
return self._write("DELETE FROM agents WHERE id=?", (agent_id,)) > 0
def intent_vector(self, agent_id: int, key: str) -> np.ndarray | None:
"""The agent's cached intent vector, if it was made under `key`."""
row = self._row("SELECT intent_key, intent_vec FROM agents WHERE id=?",
(agent_id,))
if row is None or row["intent_key"] != key or row["intent_vec"] is None:
return None
return unpack(row["intent_vec"])
def set_intent_vector(self, agent_id: int, key: str, vec: np.ndarray) -> None:
self._write("UPDATE agents SET intent_key=?, intent_vec=? WHERE id=?",
(key, pack(vec), agent_id))
# --------------------------------------------------------------- scores
def score_candidates(self, gates: Sequence[Gate]) -> list[Candidate]:
"""Every open role of a company that is not muted, with how it meets
each gate: 1 match, 0 no match, -1 the column is empty. The gates are
evaluated here, in SQL, so a `require` gate is the `WHERE` it reads
as, written as a column so a gated role keeps its row and reason."""
columns: list[str] = []
params: list[Any] = []
for n, gate in enumerate(gates):
empty, match, values = _gate_sql(gate)
columns.append(
f"CASE WHEN {empty} THEN -1 WHEN {match} THEN 1 ELSE 0 END AS g{n}")
params += values
select = ", ".join(["r.uid", "r.first_seen",
"COALESCE(c.affinity, 0) AS affinity", *columns])
rows = self._rows(
f"""SELECT {select} FROM roles r
LEFT JOIN companies c ON c.key = r.company_key
WHERE r.closed_at IS NULL AND COALESCE(c.affinity, 0) <> -1""",
params)
return [Candidate(uid=r["uid"], first_seen=r["first_seen"],
affinity=int(r["affinity"]),
gates=[int(r[f"g{n}"]) for n in range(len(gates))])
for r in rows]
def replace_scores(self, agent_id: int,
rows: Sequence[tuple[str, float, str]]) -> int:
"""`(uid, score, gated_reason)` for every role this agent scored,
replacing whatever it had: a role closed or muted since the last run
drops out rather than keeping a stale score. One transaction, so a
reader sees the old list or the new one."""
stamp = self.now()
with self.transaction() as conn:
conn.execute("DELETE FROM scores WHERE agent_id=?", (agent_id,))
conn.executemany(
"""INSERT INTO scores (uid, agent_id, score, gated_reason, scored_at)
VALUES (?,?,?,?,?)""",
[(uid, agent_id, score, reason, stamp) for uid, score, reason in rows])
return len(rows)
def scores(self, agent_id: int) -> dict[str, tuple[float, str]]:
"""`uid -> (score, gated_reason)` for one agent."""
return {r["uid"]: (float(r["score"]), str(r["gated_reason"]))
for r in self._rows(
"SELECT uid, score, gated_reason FROM scores WHERE agent_id=?",
(agent_id,))}
# ------------------------------------------------------------ companies
def add_company(self, name: str, *, key: str = "", domain: str = "",
+54 -7
View File
@@ -1,4 +1,4 @@
"""The JSON routes the page uses: roles, companies, applications.
"""The JSON routes the page uses: roles, companies, applications, agents.
`/api/jobs*`, `/_jobs/jobs.js` and `/ws/jobs` are `pipeline.web`'s, mounted by
`connect` in `app.py`. This is roledesk's own API, over the same store.
@@ -9,10 +9,12 @@ from __future__ import annotations
from datetime import timedelta
from typing import Annotated
from fastapi import APIRouter, HTTPException, Query
from pydantic import BaseModel
from fastapi import APIRouter, HTTPException, Query, Response
from pydantic import BaseModel, Field
from roledesk.models import AFFINITY, Application, Company, Role, Status
from roledesk.models import (
AFFINITY, Agent, Application, Company, Gate, ScoredRole, Status, Weights,
)
from roledesk.web.deps import StoreDep
router = APIRouter(prefix="/api", tags=["roledesk"])
@@ -30,14 +32,21 @@ def list_roles(
remote: str = "",
days: int | None = None,
show_muted: bool = False,
agent: int | None = None,
gated: bool = False,
limit: Annotated[int, Query(ge=1, le=1000)] = 200,
) -> list[Role]:
) -> list[ScoredRole]:
"""Open roles, newest first. See `Store.roles` for what each filter does.
A muted company's roles are left out unless `show_muted` is set."""
A muted company's roles are left out unless `show_muted` is set. With
`agent`, ordered by that agent's score and carrying `score` and
`gated_reason`; a gated role only with `gated=1`."""
if agent is not None and store.agent(agent) is None:
raise HTTPException(404, f"no such agent: {agent}")
return store.roles(limit=limit, q=q or None, company=company or None,
place=place or None, function=function or None,
remote=remote or None, days=days, hide_muted=not show_muted)
remote=remote or None, days=days, hide_muted=not show_muted,
agent=agent, gated=gated)
# --------------------------------------------------------------- companies
@@ -122,3 +131,41 @@ def set_status(app_id: str, body: StatusBody, store: StoreDep) -> Application:
raise HTTPException(422, str(exc)) from exc
except KeyError:
raise HTTPException(404, f"no such application: {app_id}") from None
# ------------------------------------------------------------------ agents
@router.get("/agents")
def list_agents(store: StoreDep) -> list[Agent]:
return store.agents()
class AgentBody(BaseModel):
"""Create when `id` is missing, replace that agent's fields when given."""
id: int | None = None
name: str = Field(min_length=1)
intent: str = ""
gates: list[Gate] = []
weights: Weights = Weights()
enabled: bool = True
@router.post("/agents")
def save_agent(body: AgentBody, store: StoreDep) -> Agent:
"""Scores are not recomputed here: that is the `score` task, which the
page starts after a save."""
try:
return store.save_agent(body.name, body.intent, body.gates, body.weights,
body.enabled, agent_id=body.id)
except ValueError as exc:
raise HTTPException(409, str(exc)) from exc
except KeyError:
raise HTTPException(404, f"no such agent: {body.id}") from None
@router.delete("/agents/{agent_id}", status_code=204)
def delete_agent(agent_id: int, store: StoreDep) -> Response:
if not store.delete_agent(agent_id):
raise HTTPException(404, f"no such agent: {agent_id}")
return Response(status_code=204)
+1
View File
@@ -24,6 +24,7 @@ def create_app(config_path: str | None = None) -> FastAPI:
# `Store` first, so the file is at the current schema before the queue
# adds its own table to it.
store = Store(config.database)
jobs.seed_agents(config, store)
app = FastAPI(title="roledesk", docs_url="/api/docs")
connect(app, Jobs(config.database), config, store,
workers=config.workers.model_dump(),
+34
View File
@@ -159,6 +159,40 @@ button:focus-visible, a:focus-visible, input:focus-visible, select:focus-visible
}
.role.muted { opacity: .55; }
.role.gated { opacity: .7; }
.show-muted[hidden] { display: none; }
.role-score.badge:empty { display: none; }
.role-score.badge { color: var(--signal); border-color: var(--signal-weak); font-variant-numeric: tabular-nums; }
.role-gated { color: var(--mute); font-size: .85rem; }
/* ---------------------------------------------------------------- agents */
.agents { list-style: none; margin: 0 0 1rem; padding: 0; display: flex; flex-direction: column; gap: .4rem; }
.agent {
display: flex; align-items: center; gap: .75rem; flex-wrap: wrap;
background: var(--panel); border: 1px solid var(--rule); border-radius: 6px; padding: .5rem .75rem;
}
.agent-meta { color: var(--faint); font-size: .85rem; flex: 1 1 12rem; }
.agent-form {
display: flex; flex-direction: column; gap: .6rem; max-width: 44rem;
background: var(--panel); border: 1px solid var(--rule); border-radius: 6px; padding: .75rem 1rem;
}
.agent-form h2 { font-size: 1rem; margin: 0; }
.agent-form label { display: flex; flex-direction: column; gap: .2rem; font-size: .9rem; }
.agent-form label.inline { flex-direction: row; align-items: center; gap: .4rem; }
.agent-form input[type="text"], .agent-form textarea, .agent-form input[type="number"] {
font: inherit; color: inherit; background: var(--bg); border: 1px solid var(--rule);
border-radius: 4px; padding: .35rem .5rem; width: 100%;
}
.agent-form textarea { font-family: ui-monospace, "Cascadia Mono", Consolas, monospace; font-size: .85rem; }
.agent-form .hint { color: var(--faint); font-size: .8rem; overflow-wrap: anywhere; }
.weights { display: flex; gap: .75rem; flex-wrap: wrap; border: 1px solid var(--rule); border-radius: 4px; }
.weights label { flex: 1 1 6rem; }
.agent-actions { display: flex; gap: .5rem; flex-wrap: wrap; }
.agent-actions button, .agent button {
background: var(--bg); border: 1px solid var(--rule); border-radius: 4px; padding: .3rem .8rem; cursor: pointer;
}
.agent-actions button.primary { background: var(--signal); border-color: var(--signal); color: #fff; }
/* ------------------------------------------------------------ applications */
+161 -3
View File
@@ -1,5 +1,5 @@
// roledesk's page: the filter bar, the role list, the tracker, and the tab
// switch between them. The template renders an empty shell (see index.html);
// roledesk's page: the filter bar, the role list, the tracker, the search
// agents, and the tab switch between them. The template renders an empty shell (see index.html);
// everything on screen comes from the API, and every action — Save, an
// affinity click, a status change — is a fetch from here, not a form post.
"use strict";
@@ -12,6 +12,8 @@ let companies = new Map();
let trackedByRole = new Map();
/** @type {object[]} the last /api/roles response, already muted-filtered by the server */
let roles = [];
/** @type {object[]} every search agent, from /api/agents */
let agents = [];
const $ = (sel, root = document) => root.querySelector(sel);
const $$ = (sel, root = document) => Array.from(root.querySelectorAll(sel));
@@ -42,7 +44,9 @@ function initTabs() {
const name = tab.dataset.tab;
$("#roles-view").hidden = name !== "roles";
$("#applications-view").hidden = name !== "applications";
$("#agents-view").hidden = name !== "agents";
if (name === "applications") loadApplications();
if (name === "agents") loadAgents();
});
});
}
@@ -64,6 +68,11 @@ function filterParams() {
if (remote) params.set("remote", remote);
if (days) params.set("days", days);
if ($("#f-show-muted").checked) params.set("show_muted", "true");
const agent = $("#f-agent").value;
if (agent) {
params.set("agent", agent);
if ($("#f-gated").checked) params.set("gated", "1");
}
return params;
}
@@ -104,6 +113,13 @@ function renderRoles() {
node.querySelector(".role-remote").textContent = role.remote || "";
node.querySelector(".role-function").textContent = role.function || "";
node.querySelector(".role-age").textContent = relativeAge(role.first_seen);
if (role.score !== null && role.score !== undefined) {
const badge = node.querySelector(".role-score");
badge.textContent = role.gated_reason ? "gated" : role.score.toFixed(2);
badge.title = "Score for the selected agent";
node.querySelector(".role-gated").textContent = role.gated_reason || "";
if (role.gated_reason) li.classList.add("gated");
}
node.querySelector(".affinity").replaceWith(
affinityControl(role.company_key, company ? company.affinity : 0));
@@ -260,6 +276,140 @@ async function setStatus(id, status) {
}
}
// ---------------------------------------------------------------- agents
async function loadAgents() {
agents = await api("/api/agents");
renderAgentSelect();
renderAgents();
}
// The filter bar's order: "Newest", or one agent's score. Keeps the current
// choice when the list is refreshed, unless that agent is gone.
function renderAgentSelect() {
const select = $("#f-agent");
const current = select.value;
select.replaceChildren(new Option("Newest", ""));
for (const agent of agents) {
const label = agent.enabled ? agent.name : `${agent.name} (off)`;
select.appendChild(new Option(label, String(agent.id)));
}
select.value = agents.some((a) => String(a.id) === current) ? current : "";
$("#show-gated").hidden = !select.value;
}
function renderAgents() {
const list = $("#agents-list");
list.replaceChildren();
for (const agent of agents) {
const li = document.createElement("li");
li.className = "agent";
const name = document.createElement("strong");
name.textContent = agent.name;
const meta = document.createElement("span");
meta.className = "agent-meta";
const gates = agent.gates.map((g) => `${g.mode} ${g.field}`).join(", ") || "no gates";
meta.textContent = `${agent.enabled ? "" : "off · "}${gates}`;
const edit = document.createElement("button");
edit.type = "button";
edit.textContent = "Edit";
edit.addEventListener("click", () => fillAgentForm(agent));
li.append(name, meta, edit);
list.appendChild(li);
}
$("#agents-count").textContent = `${agents.length} agent${agents.length === 1 ? "" : "s"}`;
}
function fillAgentForm(agent) {
$("#agent-form-title").textContent = agent ? `Edit ${agent.name}` : "New agent";
$("#a-id").value = agent ? String(agent.id) : "";
$("#a-name").value = agent ? agent.name : "";
$("#a-intent").value = agent ? agent.intent : "";
$("#a-gates").value = JSON.stringify(agent ? agent.gates : [], null, 2);
const weights = agent ? agent.weights : { fit: 0.6, recency: 0.2, affinity: 0.2 };
$("#a-fit").value = weights.fit;
$("#a-recency").value = weights.recency;
$("#a-affinity").value = weights.affinity;
$("#a-enabled").checked = agent ? agent.enabled : true;
$("#a-delete").hidden = !agent;
$("#agent-status").textContent = "";
}
async function saveAgent(event) {
event.preventDefault();
const status = $("#agent-status");
let gates;
try {
gates = JSON.parse($("#a-gates").value || "[]");
} catch (err) {
status.textContent = `Gates are not valid JSON: ${err.message}`;
return;
}
const id = $("#a-id").value;
const body = {
name: $("#a-name").value.trim(),
intent: $("#a-intent").value,
gates,
weights: {
fit: Number($("#a-fit").value),
recency: Number($("#a-recency").value),
affinity: Number($("#a-affinity").value),
},
enabled: $("#a-enabled").checked,
};
if (id) body.id = Number(id);
try {
const saved = await api("/api/agents", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body),
});
await loadAgents();
fillAgentForm(saved);
await rescore(status);
} catch (err) {
status.textContent = err.message;
}
}
async function deleteAgent() {
const id = $("#a-id").value;
if (!id || !confirm("Delete this agent and its scores?")) return;
try {
await api(`/api/agents/${encodeURIComponent(id)}`, { method: "DELETE" });
fillAgentForm(null);
await loadAgents();
await loadRoles();
} catch (err) {
$("#agent-status").textContent = err.message;
}
}
// Start the `score` job and refetch the roles once it has finished, so the
// order on the Roles tab is the saved agent's.
async function rescore(status) {
status.textContent = "Saved. Scoring…";
const job = await api("/api/jobs", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ kind: "score", payload: {} }),
});
for (let i = 0; i < 300; i += 1) {
await new Promise((resolve) => setTimeout(resolve, 1000));
const snapshot = await api("/api/jobs");
const busy = [...snapshot.running, ...snapshot.queued].some((j) => j.id === job.id);
if (!busy) {
const done = snapshot.recent.find((j) => j.id === job.id);
status.textContent = done && done.state !== "done"
? `Scoring ${done.state}: ${done.error || ""}`
: "Saved and scored.";
await loadRoles();
return;
}
}
status.textContent = "Saved. Scoring is still running; see Jobs below.";
}
// -------------------------------------------------------------------- init
async function loadCompanies() {
@@ -269,7 +419,7 @@ async function loadCompanies() {
async function init() {
initTabs();
await Promise.all([loadCompanies(), loadApplications()]);
await Promise.all([loadCompanies(), loadApplications(), loadAgents()]);
await loadRoles();
const refetch = debounce(loadRoles, 300);
@@ -280,6 +430,14 @@ async function init() {
$("#f-remote").addEventListener("change", loadRoles);
$("#f-days").addEventListener("change", loadRoles);
$("#f-show-muted").addEventListener("change", loadRoles);
$("#f-agent").addEventListener("change", () => {
$("#show-gated").hidden = !$("#f-agent").value;
loadRoles();
});
$("#f-gated").addEventListener("change", loadRoles);
$("#agent-form").addEventListener("submit", saveAgent);
$("#a-new").addEventListener("click", () => fillAgentForm(null));
$("#a-delete").addEventListener("click", deleteAgent);
$("#filters").addEventListener("submit", (e) => e.preventDefault());
}
+36
View File
@@ -13,11 +13,15 @@
<nav class="tabs" id="tabs">
<button type="button" class="tab" data-tab="roles" aria-pressed="true">Roles</button>
<button type="button" class="tab" data-tab="applications" aria-pressed="false">Applications</button>
<button type="button" class="tab" data-tab="agents" aria-pressed="false">Agents</button>
</nav>
</header>
<section id="roles-view" class="view">
<form id="filters" class="filters" autocomplete="off">
<select id="f-agent" aria-label="Order">
<option value="">Newest</option>
</select>
<input type="search" id="f-q" placeholder="Search titles" aria-label="Search titles">
<input type="text" id="f-company" placeholder="Company" aria-label="Company">
<input type="text" id="f-place" placeholder="Place" aria-label="Place">
@@ -43,6 +47,9 @@
<label class="show-muted">
<input type="checkbox" id="f-show-muted"> Show muted
</label>
<label class="show-muted" id="show-gated" hidden>
<input type="checkbox" id="f-gated"> Show gated
</label>
</form>
<p id="roles-count" class="count" role="status"></p>
@@ -59,6 +66,33 @@
</table>
</section>
<section id="agents-view" class="view" hidden>
<p id="agents-count" class="count" role="status"></p>
<ul id="agents-list" class="agents"></ul>
<form id="agent-form" class="agent-form" autocomplete="off">
<h2 id="agent-form-title">New agent</h2>
<input type="hidden" id="a-id">
<label>Name <input type="text" id="a-name" required></label>
<label>Intent <span class="hint">a CV paragraph, or what this search is for</span>
<textarea id="a-intent" rows="5"></textarea></label>
<label>Gates <span class="hint">JSON: [{"field": "place", "values": ["Berlin"], "mode": "require"}, {"field": "remote", "values": ["remote"], "mode": "prefer", "weight": 0.2}]</span>
<textarea id="a-gates" rows="5" spellcheck="false">[]</textarea></label>
<fieldset class="weights">
<legend>Weights</legend>
<label>Fit <input type="number" id="a-fit" min="0" step="0.05" value="0.6"></label>
<label>Recency <input type="number" id="a-recency" min="0" step="0.05" value="0.2"></label>
<label>Affinity <input type="number" id="a-affinity" min="0" step="0.05" value="0.2"></label>
</fieldset>
<label class="inline"><input type="checkbox" id="a-enabled" checked> Enabled</label>
<div class="agent-actions">
<button type="submit" class="primary">Save and score</button>
<button type="button" id="a-new">New</button>
<button type="button" id="a-delete" hidden>Delete</button>
</div>
<p id="agent-status" class="count" role="status"></p>
</form>
</section>
<section id="jobs">
<h2>Jobs</h2>
<script type="module" src="/_jobs/jobs.js"></script>
@@ -76,6 +110,8 @@
<span class="role-place"></span>
<span class="role-remote badge"></span>
<span class="role-function badge"></span>
<span class="role-score badge"></span>
<span class="role-gated"></span>
<span class="role-age"></span>
</div>
<button type="button" class="save"></button>
+21
View File
@@ -10,3 +10,24 @@ extract:
enabled: false
model: claude-haiku-4-5-20251001
sources: {}
embed:
model: sentence-transformers/all-MiniLM-L6-v2
# Starter search agents, imported once into an empty table. Edit them on the
# page afterwards; this list is not read again.
agents:
- name: Berlin backend
intent: >-
Senior backend engineer. Python and Go services, PostgreSQL, event-driven
systems, APIs and data pipelines on Kubernetes in the cloud. Wants
ownership of production systems in a product company.
gates:
- {field: place, values: [Berlin], mode: require}
- {field: remote, values: [hybrid, remote], mode: prefer, weight: 0.2}
- name: Remote EU stretch
intent: >-
Staff or lead engineer role, fully remote from Europe. Platform and
infrastructure work, distributed systems, developer tooling, technical
leadership across teams.
gates:
- {field: remote, values: [remote], mode: require}
- {field: country, values: [DE, NL, AT, CH], mode: prefer, weight: 0.2}
+313
View File
@@ -0,0 +1,313 @@
"""`embed`, `score` and the pull cascade, with the embedding backend replaced
by a deterministic fake. Nothing here downloads a model or reaches a board."""
from __future__ import annotations
import sys
from collections.abc import Iterator, Sequence
from datetime import UTC, datetime, timedelta
from pathlib import Path
from typing import Any
import numpy as np
import pytest
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
from desk.models import Item # noqa: E402
from pipeline import FINAL, TASKS, JobQueue, Jobs, RunContext # noqa: E402
from plugins.ats import ATSSource # noqa: E402
from roledesk.config import AgentSeed, Config # noqa: E402
from roledesk.jobs import cascade, embed, score # noqa: E402
from roledesk.models import Candidate, Gate, Weights # noqa: E402
from roledesk.storage import Store # noqa: E402
NOW = datetime(2026, 9, 23, 9, 0, tzinfo=UTC)
#: The fake's whole vocabulary: one dimension per word, plus a constant one so
#: no text is the zero vector.
WORDS = ("backend", "python", "frontend", "react", "data")
class FakeEmbedder:
model_id = "fake"
def __init__(self) -> None:
self.calls: list[list[str]] = []
def encode(self, texts: Sequence[str]) -> np.ndarray:
self.calls.append(list(texts))
rows = []
for text in texts:
lower = text.lower()
vec = np.asarray([lower.count(w) for w in WORDS] + [0.1], dtype=np.float32)
rows.append(vec / np.linalg.norm(vec))
return np.vstack(rows)
class Clock:
def __init__(self) -> None:
self.at = NOW
def __call__(self) -> datetime:
return self.at
@pytest.fixture
def fake(monkeypatch: pytest.MonkeyPatch) -> FakeEmbedder:
embedder = FakeEmbedder()
monkeypatch.setattr(embed, "load_embedder", lambda model: embedder)
return embedder
@pytest.fixture
def clock() -> Clock:
return Clock()
@pytest.fixture
def store(tmp_path: Path, clock: Clock) -> Iterator[Store]:
s = Store(tmp_path / "roledesk.db")
s.clock = clock
yield s
@pytest.fixture
def config(tmp_path: Path) -> Config:
return Config(database=str(tmp_path / "roledesk.db"), embed={"model": "fake"})
def role(store: Store, source_id: str, title: str = "Backend Engineer", *,
company: str = "Acme", body: str = "", **raw: Any) -> str:
item = Item(source_id=source_id, location=f"https://boards.example/{source_id}",
title=title, body=body, published_at=datetime(2026, 9, 20, tzinfo=UTC),
raw={"company": company, **raw})
item.source = "ats"
store.upsert_role(item)
return item.uid
def run(config: Config, store: Store, job: Any) -> dict[str, Any]:
notices: list[Any] = []
result = dict(job(config, store, context=RunContext(on_notice=notices.append)))
result["notices"] = [n.code for n in notices]
return result
# ------------------------------------------------------------------ embed
def test_embed_writes_a_vector_per_open_role_once(
store: Store, config: Config, fake: FakeEmbedder) -> None:
role(store, "1")
closed = role(store, "2")
with store.transaction() as conn:
conn.execute("UPDATE roles SET closed_at = ? WHERE uid = ?", (NOW.isoformat(), closed))
assert run(config, store, embed.run_embed)["embedded"] == 1
assert run(config, store, embed.run_embed)["embedded"] == 0
assert store.vector_counts() == {"fake": 1}
def test_embed_without_sentence_transformers_says_so_and_does_nothing(
store: Store, config: Config, monkeypatch: pytest.MonkeyPatch) -> None:
def missing(model: str) -> Any:
raise ImportError("no sentence_transformers")
monkeypatch.setattr(embed, "load_embedder", missing)
role(store, "1")
result = run(config, store, embed.run_embed)
assert result["embedded"] == 0 and result["notices"] == ["embed.unavailable"]
assert store.vector_counts() == {}
# ------------------------------------------------------------------ terms
def test_recency_halves_every_fourteen_days() -> None:
assert score.recency(NOW.isoformat(), NOW) == 1.0
assert score.recency((NOW - timedelta(days=14)).isoformat(), NOW) == pytest.approx(0.5)
assert score.recency((NOW - timedelta(days=28)).isoformat(), NOW) == pytest.approx(0.25)
def test_a_role_without_a_vector_is_scored_without_fit_unless_fit_is_all() -> None:
candidate = Candidate(uid="u", first_seen=NOW.isoformat(), affinity=0)
value, reason = score.score_one(candidate, [], Weights(), None, NOW)
assert reason == "" and value == pytest.approx((0.2 * 1.0 + 0.2 / 3) / 0.4)
only_fit = Weights(fit=1, recency=0, affinity=0)
assert score.score_one(candidate, [], only_fit, None, NOW) == (0.0, "no vector")
# ------------------------------------------------------------------ score
def test_a_require_gate_keeps_the_row_with_a_reason(
store: Store, config: Config, fake: FakeEmbedder) -> None:
berlin = role(store, "1", place="Berlin, Germany")
munich = role(store, "2", place="Munich")
nowhere = role(store, "3")
agent = store.save_agent("Berlin", "python backend",
[Gate(field="place", values=["Berlin"])])
run(config, store, score.run_score)
scores = store.scores(agent.id)
assert scores[berlin][1] == "" and scores[berlin][0] > 0
assert scores[munich] == (0.0, "place not Berlin")
assert scores[nowhere] == (0.0, "place unknown")
shown = store.roles(agent=agent.id)
assert [r.uid for r in shown] == [berlin]
assert {r.uid for r in store.roles(agent=agent.id, gated=True)} == {berlin, munich, nowhere}
def test_a_prefer_gate_adds_its_weight_to_the_score(
store: Store, config: Config, fake: FakeEmbedder) -> None:
remote = role(store, "1", remote="remote")
office = role(store, "2", remote="onsite")
agent = store.save_agent("Remote", "", [Gate(field="remote", values=["remote"],
mode="prefer", weight=0.5)])
run(config, store, score.run_score)
scores = store.scores(agent.id)
# No intent, so fit is out: recency 0.2 + affinity 0.2 + the gate's 0.5.
assert scores[remote][0] - scores[office][0] == pytest.approx(0.5 / 0.9, abs=1e-5)
assert scores[office][1] == ""
def test_a_favourite_skips_the_gates_and_is_floored(
store: Store, config: Config, fake: FakeEmbedder) -> None:
store.add_company("Fave", affinity=2)
fave = role(store, "1", company="Fave", place="Munich")
agent = store.save_agent("Berlin", "", [Gate(field="place", values=["Berlin"])])
run(config, store, score.run_score)
value, reason = store.scores(agent.id)[fave]
assert reason == "" and value >= 0.9
def test_a_muted_company_is_never_scored(
store: Store, config: Config, fake: FakeEmbedder) -> None:
store.add_company("Muted", affinity=-1)
muted = role(store, "1", company="Muted")
kept = role(store, "2")
agent = store.save_agent("Any")
run(config, store, score.run_score)
assert set(store.scores(agent.id)) == {kept}
assert muted not in {r.uid for r in store.roles(agent=agent.id, gated=True)}
def test_an_older_role_scores_lower(
store: Store, config: Config, clock: Clock, fake: FakeEmbedder) -> None:
clock.at = NOW - timedelta(days=14)
old = role(store, "1")
clock.at = NOW
new = role(store, "2")
agent = store.save_agent("Any")
run(config, store, score.run_score)
scores = store.scores(agent.id)
assert scores[new][0] > scores[old][0]
def test_two_agents_score_the_same_role_differently(
store: Store, config: Config, fake: FakeEmbedder) -> None:
uid = role(store, "1", "Backend Engineer", body="Python services")
backend = store.save_agent("Backend", "python backend developer",
weights=Weights(fit=1, recency=0, affinity=0))
frontend = store.save_agent("Frontend", "react frontend developer",
weights=Weights(fit=1, recency=0, affinity=0))
run(config, store, embed.run_embed)
result = run(config, store, score.run_score)
assert result["agents"] == 2
assert store.scores(backend.id)[uid][0] > store.scores(frontend.id)[uid][0]
def test_the_intent_vector_is_cached_until_the_intent_changes(
store: Store, config: Config, fake: FakeEmbedder) -> None:
role(store, "1")
agent = store.save_agent("Backend", "python backend")
run(config, store, score.run_score)
run(config, store, score.run_score)
assert fake.calls == [["python backend"]]
store.save_agent("Backend", "react frontend", agent_id=agent.id)
run(config, store, score.run_score)
assert fake.calls[-1] == ["react frontend"]
def test_a_disabled_agent_is_not_scored(
store: Store, config: Config, fake: FakeEmbedder) -> None:
role(store, "1")
agent = store.save_agent("Off", enabled=False)
assert run(config, store, score.run_score)["agents"] == 0
assert store.scores(agent.id) == {}
def test_starter_agents_are_imported_once(store: Store, tmp_path: Path) -> None:
config = Config(database=str(tmp_path / "roledesk.db"),
agents=[AgentSeed(name="A"), AgentSeed(name="B")])
assert score.seed_agents(config, store) == ["A", "B"]
for agent in store.agents():
store.delete_agent(agent.id)
assert score.seed_agents(config, store) == []
assert store.agents() == []
def test_the_shipped_starter_agents_validate() -> None:
from roledesk import config as config_mod
names = [a.name for a in config_mod.load(ROOT / "roles.yaml").agents]
assert names == ["Berlin backend", "Remote EU stretch"]
# ---------------------------------------------------------------- cascade
def test_the_pull_cascade_runs_extract_embed_then_score(
store: Store, config: Config, fake: FakeEmbedder, tmp_path: Path,
monkeypatch: pytest.MonkeyPatch) -> None:
store.add_subscription("ats", "greenhouse:acme", label="Acme", company="Acme",
board="greenhouse")
agent = store.save_agent("Backend", "python backend")
def fake_fetch(self: ATSSource) -> Iterator[Item]:
item = Item(source_id="greenhouse:acme:1", location="https://x.example/1",
title="Backend Engineer", body="Python",
published_at=datetime(2026, 9, 20, tzinfo=UTC),
raw={"company": "Acme", "board": "greenhouse", "slug": "acme"})
yield item
self.state["ats:ok:greenhouse:acme"] = self.state.get("ats:run", "")
self.state["ats:ids:greenhouse:acme"] = '["greenhouse:acme:1"]'
monkeypatch.setattr(ATSSource, "fetch", fake_fetch)
queue = JobQueue(Jobs(tmp_path / "roledesk.db"), config, store,
workers={"cpu": 1, "http": 1, "llm": 1})
task_id = queue.start_task(TASKS["pull"]())
queue.drain(timeout=60)
cascade_id = queue.jobs.of_task(task_id)[0].cascade_id
jobs = sorted(queue.jobs.of_cascade(cascade_id), key=lambda j: j.id)
stages = [j.result.get("task") for j in jobs if j.kind == FINAL]
assert stages == list(cascade.CASCADES["pull"])
work = [j.kind for j in jobs if j.kind != FINAL]
assert work == ["pull_batch", "embed", "score"], "extract is off, so it finds nothing"
assert all(j.state == "done" for j in jobs)
assert len(store.scores(agent.id)) == 1
def test_follows_walks_the_cascade_and_stops_at_its_end() -> None:
assert cascade.follows("pull", "pull") == "extract"
assert cascade.follows("pull", "extract") == "embed"
assert cascade.follows("pull", "embed") == "score"
assert cascade.follows("pull", "score") == ""
assert cascade.follows("pull", "nope") == ""
+5 -1
View File
@@ -87,11 +87,15 @@ def test_a_v1_file_migrates_to_the_shape_a_new_file_has(tmp_path: Path) -> None:
with sqlite3.connect(old) as conn:
for column in ("permit", "languages", "extract_tries"):
conn.execute(f"ALTER TABLE roles DROP COLUMN {column}")
for column in ("intent_key", "intent_vec"):
conn.execute(f"ALTER TABLE agents DROP COLUMN {column}")
conn.execute("DROP TABLE vectors")
conn.execute("PRAGMA user_version = 1")
assert Store(old).migrations_applied == list(range(2, LATEST + 1))
# ALTER appends, so the order differs; every read is by column name.
assert sorted(_columns(old, "roles")) == sorted(_columns(fresh, "roles"))
for table in ("roles", "agents", "vectors"):
assert sorted(_columns(old, table)) == sorted(_columns(fresh, table))
def test_state_is_a_mapping(store: Store) -> None:
+46
View File
@@ -203,3 +203,49 @@ def test_index_renders_the_shell(client: TestClient) -> None:
assert "roledesk" in res.text
assert 'id="roles-list"' in res.text
assert "<desk-jobs>" in res.text
# ------------------------------------------------------------------ agents
def test_agents_are_created_updated_listed_and_deleted(client: TestClient) -> None:
body = {"name": "Berlin", "intent": "python backend",
"gates": [{"field": "place", "values": ["Berlin"], "mode": "require"}]}
created = client.post("/api/agents", json=body)
assert created.status_code == 200
agent = created.json()
assert agent["weights"] == {"fit": 0.6, "recency": 0.2, "affinity": 0.2}
assert agent["gates"][0]["weight"] == 1.0
updated = client.post("/api/agents", json={**body, "id": agent["id"], "enabled": False,
"weights": {"fit": 1, "recency": 0,
"affinity": 0}})
assert updated.json()["enabled"] is False and updated.json()["weights"]["fit"] == 1
assert [a["name"] for a in client.get("/api/agents").json()] == ["Berlin"]
assert client.post("/api/agents", json=body).status_code == 409
assert client.delete(f"/api/agents/{agent['id']}").status_code == 204
assert client.get("/api/agents").json() == []
assert client.delete(f"/api/agents/{agent['id']}").status_code == 404
def test_a_gate_on_an_unknown_field_is_refused(client: TestClient) -> None:
body = {"name": "Bad", "gates": [{"field": "salary", "values": [1]}]}
assert client.post("/api/agents", json=body).status_code == 422
def test_roles_by_agent_are_ordered_by_score_and_hide_the_gated(
client: TestClient, store: Store) -> None:
for sid in ("low", "high", "gated"):
store.upsert_role(posting(sid))
uids = {r.source_id: r.uid for r in store.roles()}
agent = store.save_agent("A")
store.replace_scores(agent.id, [(uids["low"], 0.2, ""), (uids["high"], 0.8, ""),
(uids["gated"], 0.0, "place unknown")])
shown = client.get(f"/api/roles?agent={agent.id}").json()
assert [r["source_id"] for r in shown] == ["high", "low"]
assert shown[0]["score"] == 0.8
everything = client.get(f"/api/roles?agent={agent.id}&gated=1").json()
assert everything[-1]["gated_reason"] == "place unknown"
assert client.get("/api/roles").json()[0]["score"] is None
assert client.get("/api/roles?agent=999").status_code == 404