Score roles per search agent, and run pull into extract, embed and score
embed stores one vector per open role; score rates every open role for every enabled agent from require and prefer gates, fit against the agent's intent, recency and company affinity, keeping gated rows with a reason. Schema v3 adds vectors and the cached intent vector. The page gets an agent order, the score on each row, and an Agents tab. Two starter agents in roles.yaml are imported once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
4dfb8831aa
commit
bc60324fab
22 files changed
+1457
-44
No files matched your search
@@ -26,5 +26,23 @@ extract:
|
||||
enabled: false
|
||||
model: claude-haiku-4-5-20251001
|
||||
|
||||
# A role and each agent's intent as vectors, for the fit term of a score
|
||||
# (roledesk/jobs/embed.py). Needs sentence-transformers; without it `embed`
|
||||
# posts a notice and scores leave fit out. Changing the model re-embeds.
|
||||
embed:
|
||||
model: sentence-transformers/all-MiniLM-L6-v2
|
||||
per_run: 50000
|
||||
|
||||
# Starter search agents, imported once into an empty `agents` table. Each gate
|
||||
# is {field, values, mode: require|prefer, weight}; fields are place, country,
|
||||
# remote, seniority, function, salary_min, currency, permit, languages and
|
||||
# company. weights default to {fit: 0.6, recency: 0.2, affinity: 0.2}.
|
||||
agents:
|
||||
- name: Berlin backend
|
||||
intent: Senior backend engineer, Python services and PostgreSQL.
|
||||
gates:
|
||||
- {field: place, values: [Berlin], mode: require}
|
||||
- {field: remote, values: [hybrid, remote], mode: prefer, weight: 0.2}
|
||||
|
||||
# One block per plugin. Empty until the ats plugin lands.
|
||||
sources: {}
|
||||
@@ -63,6 +63,20 @@ at least the listing.
|
||||
|
||||
**Summary and tagging are off; `extract` replaces them.** Every profile declares `SummaryPolicy(enabled=False)` and `TaggingPolicy(enabled=False)`. `extract` is a roledesk `ModelTask` on Haiku that turns title and description into the gate columns, and fills only the columns a plugin left empty.
|
||||
|
||||
**A score is the weighted sum over the sum of the weights.** The agent's `fit`, `recency` and `affinity` weights and every prefer gate's `weight` form one denominator, so a score stays in 0..1 and the 0.9 floor for a favourite means the same thing whatever the weights are. A prefer gate normalised by the prefer weights alone, then weighted as one term, comes to the same number.
|
||||
|
||||
**Gates match loosely where the column is free text.** `place` is a case-insensitive substring, like the place filter, so `Berlin` matches "Berlin, Germany". `company` compares company keys. `languages` passes when every language the role requires is one of the values. `salary_min` is at least the smallest value. The rest are case-insensitive equality. The gates are evaluated in SQL as one `CASE` column each, so a `require` gate is the `WHERE` it reads as while the gated row and its reason survive. The reason lists every failed require gate, not just the first.
|
||||
|
||||
**A role with no vector is scored without `fit`.** The fit weight leaves the denominator, so a role `embed` has not reached yet ranks on recency, affinity and prefer gates instead of sinking to the bottom. It is gated as "no vector" only when fit is the only weighted term. An agent whose intent could not be embedded (empty, or no sentence-transformers) says "no intent vector" instead.
|
||||
|
||||
**Each run replaces an agent's scores.** `replace_scores` deletes and inserts in one transaction, so a role closed or muted since the last run drops out rather than keeping a stale score, and a reader sees one list or the other.
|
||||
|
||||
**Starter agents are imported once, by a state marker.** newsdesk seeds whenever its table is empty. Here `agents:seeded` in `state` is set after the import, so deleting every agent survives a restart.
|
||||
|
||||
**`embed` and `score` run in the server process.** newsdesk isolates `embed` in a child process with a timeout. roledesk's run is one bounded pass over open roles, and in-process keeps the embedder replaceable in tests. The cost is torch's memory staying in the server after the first run; isolate it if that becomes the problem.
|
||||
|
||||
**The pull cascade is `pull → extract → embed → score`.** Every stage is always in the chain and a stage that is off finds nothing, as in newsdesk. Extract precedes score so the gates read what it filled, and reads `on_ingest` only as a stage of a pull, so with `auto` and no model reachable a pull passes it quietly.
|
||||
|
||||
## The alpha page
|
||||
|
||||
**The template renders an empty shell; `static/app.js` fills it and drives every action.** `index.html` carries only the filter bar's `<select>` options (from `roledesk.models` and `plugins.ats.FUNCTIONS`, so they can't drift from what the store accepts) and the section containers. Every row, count and button state is a fetch from the JS, including the initial load — one code path for "show this" rather than a server-rendered first page plus a client re-render for every filter change after.
|
||||
|
||||
+10
-8
@@ -1,4 +1,4 @@
|
||||
"""`roledesk serve | job pull|extract | companies seed|add|check | db init`."""
|
||||
"""`roledesk serve | job pull|extract|embed|score | companies seed|add|check | db init`."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -37,7 +37,7 @@ def build_parser() -> argparse.ArgumentParser:
|
||||
serve.add_argument("--port", type=int, default=8790)
|
||||
|
||||
job = sub.add_parser("job", help="queue a job or task and drain the queue once")
|
||||
job.add_argument("kind", choices=["pull", "extract"])
|
||||
job.add_argument("kind", choices=["pull", "extract", "embed", "score"])
|
||||
|
||||
companies = sub.add_parser("companies", help="the target company list")
|
||||
companies_sub = companies.add_subparsers(dest="action", required=True)
|
||||
@@ -76,8 +76,8 @@ def main(argv: list[str] | None = None) -> int:
|
||||
def run_job(args: argparse.Namespace) -> int:
|
||||
"""Queue `kind` — a job, or a task from `pipeline.TASKS` such as `pull` —
|
||||
then run everything queued until the queue is idle, and print how it
|
||||
ended. A task's own result is its final job's, which is the one whose
|
||||
`task_id` equals the run this started."""
|
||||
ended: one line per task, from each task's final job. `pull` is a
|
||||
cascade, so it prints a line for every stage it went through."""
|
||||
from pipeline import FINAL, JobQueue, Jobs, TASKS
|
||||
from roledesk import jobs
|
||||
from roledesk.storage import Store
|
||||
@@ -90,12 +90,14 @@ def run_job(args: argparse.Namespace) -> int:
|
||||
if args.kind in TASKS:
|
||||
task_id = queue.start_task(TASKS[args.kind]())
|
||||
queue.drain()
|
||||
final = next((j for j in queue.jobs.of_task(task_id) if j.kind == FINAL), None)
|
||||
if final is not None:
|
||||
started = queue.jobs.of_task(task_id)
|
||||
if not started:
|
||||
return 0
|
||||
finals = [j for j in queue.jobs.of_cascade(started[0].cascade_id) if j.kind == FINAL]
|
||||
for final in finals:
|
||||
print(json.dumps({"id": final.id, "state": final.state,
|
||||
"result": final.result, "error": final.error}))
|
||||
return 0 if final.state == "done" else 1
|
||||
return 0
|
||||
return 0 if all(f.state == "done" for f in finals) else 1
|
||||
queued = queue.enqueue(args.kind)
|
||||
queue.drain()
|
||||
if queued is not None:
|
||||
|
||||
@@ -14,6 +14,7 @@ import yaml
|
||||
from pydantic import Field, ValidationError
|
||||
|
||||
from desk.config import AgentConfig, ModelTaskConfig, Section, WorkersConfig
|
||||
from roledesk.models import Gate, Weights
|
||||
|
||||
DEFAULT_PATH = Path(os.environ.get("ROLEDESK_CONFIG", "roles.yaml"))
|
||||
|
||||
@@ -28,6 +29,29 @@ class ExtractConfig(ModelTaskConfig):
|
||||
batch_size: int = Field(20, ge=1)
|
||||
|
||||
|
||||
class EmbedConfig(Section):
|
||||
"""Turning a role into a vector for `fit`. Needs sentence-transformers;
|
||||
without it `embed` says so and does nothing, and `score` leaves fit out.
|
||||
See roledesk/jobs/embed.py."""
|
||||
|
||||
#: Recorded on every vector. Changing it drops the old vectors, which are
|
||||
#: a different geometry and never comparable.
|
||||
model: str = "sentence-transformers/all-MiniLM-L6-v2"
|
||||
#: Roles embedded per run at most, newest first.
|
||||
per_run: int = Field(50_000, ge=1)
|
||||
|
||||
|
||||
class AgentSeed(Section):
|
||||
"""A search agent as `roles.yaml` lists it. Imported into an empty
|
||||
`agents` table once; after that the table is the answer."""
|
||||
|
||||
name: str
|
||||
intent: str = ""
|
||||
gates: list[Gate] = Field(default_factory=list)
|
||||
weights: Weights = Field(default_factory=Weights)
|
||||
enabled: bool = True
|
||||
|
||||
|
||||
class Config(Section):
|
||||
"""The whole file."""
|
||||
|
||||
@@ -36,6 +60,9 @@ class Config(Section):
|
||||
workers: WorkersConfig = Field(default_factory=WorkersConfig)
|
||||
agent: AgentConfig = Field(default_factory=AgentConfig)
|
||||
extract: ExtractConfig = Field(default_factory=ExtractConfig)
|
||||
embed: EmbedConfig = Field(default_factory=EmbedConfig)
|
||||
#: Starter agents, imported once into an empty table.
|
||||
agents: list[AgentSeed] = Field(default_factory=list)
|
||||
#: One block per plugin: `enabled`, and whatever that plugin reads.
|
||||
sources: dict[str, Any] = Field(default_factory=dict)
|
||||
#: Where this was loaded from. Never written back into the file.
|
||||
|
||||
@@ -1,12 +1,17 @@
|
||||
"""The work roledesk queues. Importing this package registers every job.
|
||||
"""The work roledesk queues. Importing this package registers every job and
|
||||
every task in `pipeline.TASKS`.
|
||||
|
||||
pull.py read every configured plugin and store the roles it yields
|
||||
extract.py title and description into the gate columns, on Haiku
|
||||
embed.py one vector per open role, for fit
|
||||
score.py every open role scored for every enabled search agent
|
||||
cascade.py pull → extract → embed → score, and what `TASKS` offers
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from roledesk.jobs import extract, pull
|
||||
from roledesk.jobs import cascade, embed, extract, pull, score
|
||||
from roledesk.jobs.score import seed_agents
|
||||
|
||||
#: What `JobQueue(bootstrap=...)` and `pipeline.worker --bootstrap` are given.
|
||||
BOOTSTRAP = "roledesk.jobs:bootstrap"
|
||||
@@ -22,4 +27,5 @@ def bootstrap(config_path: str = "") -> tuple[object, object]:
|
||||
return config, Store(config.database)
|
||||
|
||||
|
||||
__all__ = ["BOOTSTRAP", "bootstrap", "extract", "pull"]
|
||||
__all__ = ["BOOTSTRAP", "bootstrap", "cascade", "embed", "extract", "pull", "score",
|
||||
"seed_agents"]
|
||||
@@ -0,0 +1,105 @@
|
||||
"""The tasks roledesk offers, and the order a pull runs them in.
|
||||
|
||||
pull → extract → embed → score
|
||||
|
||||
newsdesk's cascade convention: a cascade is a tuple of task names, and the
|
||||
final job of one task starts the next (`step` is the callback it carries).
|
||||
Nothing is alive between two tasks.
|
||||
|
||||
Every stage is always in the chain, and a stage that is switched off finds
|
||||
nothing: `extract.enabled: false`, or an `on_ingest` this pull does not
|
||||
satisfy, is an extract stage with no batches whose final job moves straight
|
||||
on. Extract runs before score so the gates read what it filled; before embed
|
||||
too, since embedding does not read its columns and the order costs nothing.
|
||||
|
||||
A batch that failed, or a stage somebody killed, does not stop the chain: the
|
||||
work comes round on the next pull, and a rescore of what did arrive is worth
|
||||
having either way.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from dataclasses import replace
|
||||
from functools import partial
|
||||
from typing import Any
|
||||
|
||||
from desk.tools import agent
|
||||
from pipeline import TASKS, Tally, Task
|
||||
from roledesk.jobs import embed, extract, pull, score # noqa: F401 -- registers the jobs
|
||||
|
||||
STEP = "roledesk.jobs.cascade:step"
|
||||
|
||||
CASCADES: dict[str, tuple[str, ...]] = {
|
||||
"pull": ("pull", "extract", "embed", "score"),
|
||||
}
|
||||
|
||||
|
||||
def _once(config: Any, store: Any) -> list[dict[str, Any]]:
|
||||
"""One batch with nothing in its payload: a task that is a single job."""
|
||||
return [{}]
|
||||
|
||||
|
||||
def _extract(cascade: str) -> Task:
|
||||
"""`extract` as a stage, reading `on_ingest` when it is one of a pull.
|
||||
`never`, or `auto` with no model reachable, is a stage that finds
|
||||
nothing. Standing on its own, as the button, it reads neither."""
|
||||
stage = extract.extract_task()
|
||||
if not cascade:
|
||||
return stage
|
||||
find = stage.finder
|
||||
|
||||
def during_a_pull(config: Any, store: Any) -> list[Any]:
|
||||
when = config.extract.on_ingest
|
||||
if when == "never" or (when == "auto" and not agent.backend(config.agent)):
|
||||
return []
|
||||
return find(config, store)
|
||||
|
||||
return replace(stage, finder=during_a_pull)
|
||||
|
||||
|
||||
STAGES: dict[str, Callable[[str], Task]] = {
|
||||
"pull": lambda cascade: pull.pull_task(),
|
||||
"extract": _extract,
|
||||
"embed": lambda cascade: Task(kind="embed", finder=_once, label="Embedding roles"),
|
||||
"score": lambda cascade: Task(kind="score", finder=_once,
|
||||
label="Scoring roles for every agent"),
|
||||
}
|
||||
|
||||
|
||||
def task(name: str, *, cascade: str = "") -> Task:
|
||||
"""One stage as a `Task`. With `cascade` its final job carries `step`,
|
||||
and the next stage follows; without, it stops when its batches do."""
|
||||
return replace(STAGES[name](cascade), name=name,
|
||||
callback=STEP if cascade else "",
|
||||
carries={"cascade": cascade, "task": name})
|
||||
|
||||
|
||||
def follows(cascade: str, name: str) -> str:
|
||||
"""What runs after `name`, or "" at the end or for an unknown name."""
|
||||
tasks = CASCADES.get(cascade, ())
|
||||
if name not in tasks:
|
||||
return ""
|
||||
at = tasks.index(name)
|
||||
return tasks[at + 1] if at + 1 < len(tasks) else ""
|
||||
|
||||
|
||||
def step(config: Any, store: Any, tally: Tally, payload: dict[str, Any], *,
|
||||
context: Any) -> dict[str, Any]:
|
||||
"""The callback: make the next task. A killed stage is skipped and the
|
||||
chain goes on, as in newsdesk's cascade."""
|
||||
name, done = payload.get("cascade", ""), payload.get("task", "")
|
||||
following = follows(name, done)
|
||||
if following:
|
||||
context.task(task(following, cascade=name))
|
||||
out: dict[str, Any] = {"task": done, "next": following}
|
||||
if tally.killed:
|
||||
out["skipped"] = "cancelled"
|
||||
return out
|
||||
|
||||
|
||||
#: What `roledesk job <name>` and the Jobs view offer. `pull` is the whole
|
||||
#: chain; the others are one stage each, on their own.
|
||||
TASKS["pull"] = partial(task, "pull", cascade="pull")
|
||||
for _name in ("extract", "embed", "score"):
|
||||
TASKS[_name] = partial(task, _name)
|
||||
@@ -0,0 +1,79 @@
|
||||
"""`embed`: one vector per open role, for the `fit` term of a score.
|
||||
|
||||
The encoder is `desk.tools.embedding`; the table is roledesk's own `vectors`
|
||||
(see `Store.put_vectors`). What gets encoded is the title and the opening of
|
||||
the description, which is what says what a role is.
|
||||
|
||||
sentence-transformers is optional, as in newsdesk. Without it this task puts
|
||||
one notice on screen and does nothing; `score` then leaves `fit` out and ranks
|
||||
on what it has.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any
|
||||
|
||||
from desk.tools import embedding
|
||||
from pipeline import NULL, Notice, RunContext, job
|
||||
from roledesk.config import Config
|
||||
from roledesk.models import Role
|
||||
from roledesk.storage import Store
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
#: How much of the description the encoder sees. MiniLM truncates at 256
|
||||
#: tokens anyway; 2000 characters is past that for any language.
|
||||
BODY_CHARS = 2000
|
||||
|
||||
#: Roles encoded per call, and per commit: a kill loses at most one chunk.
|
||||
CHUNK = 64
|
||||
|
||||
|
||||
def document(role: Role) -> str:
|
||||
return f"{role.title}\n{role.description[:BODY_CHARS]}"
|
||||
|
||||
|
||||
def load_embedder(model: str) -> embedding.Embedder:
|
||||
"""The encoder. A seam of its own so a test hands in a fake and nothing
|
||||
downloads a model; `score` loads through it too."""
|
||||
return embedding.load(model)
|
||||
|
||||
|
||||
def unavailable(exc: BaseException) -> Notice:
|
||||
"""The notice for a machine without sentence-transformers. One code and
|
||||
key for `embed` and `score`, so the two say it as one line."""
|
||||
return Notice(
|
||||
code="embed.unavailable", level="warning", kind="expected", key="embed",
|
||||
text="sentence-transformers is not installed, so roles are not embedded "
|
||||
"and scores leave fit out.",
|
||||
fix="Install it into roledesk's environment: "
|
||||
"`uv pip install sentence-transformers`.",
|
||||
detail=str(exc), where="roledesk.jobs.embed")
|
||||
|
||||
|
||||
@job("embed", "Embedding roles", solo=True, resource="cpu")
|
||||
def run_embed(config: Config, store: Store, *,
|
||||
context: RunContext = NULL) -> dict[str, Any]:
|
||||
"""Embed every open role that has no vector from the configured model."""
|
||||
model = config.embed.model
|
||||
try:
|
||||
embedder = load_embedder(model)
|
||||
except ImportError as exc:
|
||||
log.warning("%s", exc)
|
||||
context.notice(unavailable(exc))
|
||||
return {"embedded": 0, "skipped": True, "reason": "sentence-transformers missing"}
|
||||
|
||||
dropped = store.drop_vectors(keep=model)
|
||||
if dropped:
|
||||
log.info("dropped %d vectors from a previous embedding model", dropped)
|
||||
roles = store.roles_to_embed(model, config.embed.per_run)
|
||||
for at in range(0, len(roles), CHUNK):
|
||||
context.check()
|
||||
context.progress(at, len(roles), f"embedding roles, {at} of {len(roles)}")
|
||||
chunk = roles[at:at + CHUNK]
|
||||
vectors = embedder.encode([document(r) for r in chunk])
|
||||
store.put_vectors(model, [(r.uid, v) for r, v in zip(chunk, vectors, strict=True)])
|
||||
context.progress(len(roles), len(roles),
|
||||
f"embedding roles, {len(roles)} of {len(roles)}")
|
||||
return {"embedded": len(roles), "dropped": dropped}
|
||||
@@ -27,7 +27,7 @@ from typing import Any
|
||||
|
||||
from desk import modeltask
|
||||
from desk.modeltask import ModelTask
|
||||
from pipeline import NULL, RunContext, TASKS, Task, job
|
||||
from pipeline import NULL, RunContext, Task, job
|
||||
from roledesk.config import Config
|
||||
from roledesk.models import PERMITS, REMOTE, SENIORITIES, ExtractResult, Role
|
||||
from roledesk.storage import Store
|
||||
@@ -219,9 +219,6 @@ def run_batch(config: Config, store: Store, *, context: RunContext = NULL,
|
||||
|
||||
|
||||
def extract_task() -> Task:
|
||||
"""Finds nothing while `extract.enabled` is false, so the task, the
|
||||
button and the cascade stage are all no-ops with the switch off."""
|
||||
return replace(modeltask.of(TASK), label=TASK.label)
|
||||
|
||||
|
||||
#: `roledesk job extract` and the Jobs view's Extract. Finds nothing while
|
||||
#: `extract.enabled` is false, so pressing it with the switch off is a no-op.
|
||||
TASKS["extract"] = extract_task
|
||||
@@ -20,7 +20,7 @@ import logging
|
||||
from datetime import UTC, datetime
|
||||
from typing import Any
|
||||
|
||||
from pipeline import NULL, RunContext, TASKS, Tally, Task, job
|
||||
from pipeline import NULL, RunContext, Tally, Task, job
|
||||
|
||||
from roledesk.config import Config
|
||||
from roledesk.storage import Store
|
||||
@@ -103,11 +103,8 @@ def close_boards(config: Config, store: Store, tally: Tally, *,
|
||||
return {"boards": answered, "failed": failed, "roles": roles, "closed": closed}
|
||||
|
||||
|
||||
def _pull_task() -> Task:
|
||||
def pull_task() -> Task:
|
||||
"""The pull on its own. `roledesk.jobs.cascade` makes it the first stage
|
||||
of the chain that `TASKS["pull"]` starts."""
|
||||
return Task(kind="pull_batch", finder=_boards, name="pull",
|
||||
label="Reading every board", settle=SETTLE)
|
||||
|
||||
|
||||
#: Offered as `roledesk job pull` and to `<desk-jobs>`'s Pull button, the same
|
||||
#: way newsdesk's cascade stages register themselves in `TASKS`.
|
||||
TASKS["pull"] = _pull_task
|
||||
@@ -0,0 +1,217 @@
|
||||
"""`score`: every open role, scored for every enabled search agent.
|
||||
|
||||
A role is scored per agent, `(uid, agent) -> score`, because a role is only
|
||||
good for a given search. For each agent:
|
||||
|
||||
require gates a role that fails one keeps its row, with score 0 and the
|
||||
reason ("place unknown", "remote not remote")
|
||||
prefer gates `weight` when the role matches, 0 when it does not
|
||||
fit cos(role vector, intent vector), mapped to 0..1
|
||||
recency 0.5 ** (days since first_seen / 14)
|
||||
affinity (companies.affinity + 1) / 3
|
||||
|
||||
Every term is 0..1. The score is the weighted sum over the sum of the weights
|
||||
(the agent's three and its prefer gates'), so it stays 0..1. A favourite
|
||||
company (`affinity = 2`) skips the gates and is floored at 0.9; a muted one
|
||||
(`-1`) is not scored at all. A role with no vector is scored without `fit`,
|
||||
unless `fit` is all there is, when it is gated as "no vector".
|
||||
|
||||
Recomputed in full on every run, and each agent's rows are replaced in one
|
||||
transaction: nothing is incremental, so nothing can drift.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import logging
|
||||
import math
|
||||
from collections.abc import Sequence
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
|
||||
from pipeline import NULL, RunContext, job
|
||||
from roledesk.config import Config
|
||||
from roledesk.jobs import embed
|
||||
from roledesk.models import Agent, Candidate, Gate, Weights
|
||||
from roledesk.storage import Store
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
HALF_LIFE_DAYS = 14.0
|
||||
|
||||
#: Where a favourite company's roles start, whatever else they score.
|
||||
FAVOURITE_FLOOR = 0.9
|
||||
|
||||
#: The `state` key that records the starter agents were imported.
|
||||
SEEDED = "agents:seeded"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- terms
|
||||
|
||||
def recency(first_seen: str, now: datetime) -> float:
|
||||
"""1.0 for a role first seen now, halving every `HALF_LIFE_DAYS`."""
|
||||
try:
|
||||
seen = datetime.fromisoformat(first_seen)
|
||||
except ValueError:
|
||||
return 0.0
|
||||
days = max((now - seen).total_seconds(), 0.0) / 86400
|
||||
return math.pow(0.5, days / HALF_LIFE_DAYS)
|
||||
|
||||
|
||||
def affinity_term(affinity: int) -> float:
|
||||
"""-1..2 to 0..1. -1 never gets here; neutral is 1/3."""
|
||||
return (affinity + 1) / 3
|
||||
|
||||
|
||||
def fit_term(role: np.ndarray, intent: np.ndarray) -> float:
|
||||
"""Cosine to 0..1. Rescaled rather than clipped, as newsdesk does, so a
|
||||
role pointing away from the intent sits below an unrelated one."""
|
||||
norms = float(np.linalg.norm(role) * np.linalg.norm(intent))
|
||||
if norms == 0:
|
||||
return 0.5
|
||||
return (float(np.dot(role, intent)) / norms + 1.0) / 2.0
|
||||
|
||||
|
||||
def _reason(gate: Gate, state: int) -> str:
|
||||
if state < 0:
|
||||
return f"{gate.field} unknown"
|
||||
if gate.field == "salary_min":
|
||||
return f"salary_min under {min(gate.values, key=lambda v: float(v))}"
|
||||
return f"{gate.field} not {' / '.join(str(v) for v in gate.values)}"
|
||||
|
||||
|
||||
def score_one(candidate: Candidate, gates: Sequence[Gate], weights: Weights,
|
||||
fit: float | None, now: datetime, *,
|
||||
missing: str = "no vector") -> tuple[float, str]:
|
||||
"""One role for one agent: `(score, gated_reason)`. `fit` is None when
|
||||
the role or the intent has no vector; `missing` says which."""
|
||||
favourite = candidate.affinity == 2
|
||||
if not favourite:
|
||||
failed = [_reason(g, state) for g, state in zip(gates, candidate.gates, strict=True)
|
||||
if g.mode == "require" and state != 1]
|
||||
if failed:
|
||||
return 0.0, ", ".join(failed)
|
||||
|
||||
terms = [(weights.recency, recency(candidate.first_seen, now)),
|
||||
(weights.affinity, affinity_term(candidate.affinity))]
|
||||
terms += [(g.weight, 1.0 if state == 1 else 0.0)
|
||||
for g, state in zip(gates, candidate.gates, strict=True) if g.mode == "prefer"]
|
||||
if fit is not None:
|
||||
terms.append((weights.fit, fit))
|
||||
total = sum(w for w, _ in terms)
|
||||
if not total:
|
||||
if favourite:
|
||||
return FAVOURITE_FLOOR, ""
|
||||
# Fit was the only term with any weight, and there is no fit.
|
||||
return 0.0, missing if fit is None and weights.fit else "no weights"
|
||||
score = sum(w * t for w, t in terms) / total
|
||||
return (max(score, FAVOURITE_FLOOR) if favourite else score), ""
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ vectors
|
||||
|
||||
def intent_key(model: str, intent: str) -> str:
|
||||
return hashlib.sha1(f"{model}\n{intent}".encode()).hexdigest()
|
||||
|
||||
|
||||
class _Encoder:
|
||||
"""The embedder, loaded on first use only: an agent whose intent vector
|
||||
is cached costs no model load at all."""
|
||||
|
||||
def __init__(self, model: str, context: RunContext) -> None:
|
||||
self.model = model
|
||||
self.context = context
|
||||
self._embedder: Any = None
|
||||
self.failed = False
|
||||
|
||||
def encode(self, text: str) -> np.ndarray | None:
|
||||
if self.failed:
|
||||
return None
|
||||
if self._embedder is None:
|
||||
try:
|
||||
self._embedder = embed.load_embedder(self.model)
|
||||
except ImportError as exc:
|
||||
self.failed = True
|
||||
self.context.notice(embed.unavailable(exc))
|
||||
return None
|
||||
return np.asarray(self._embedder.encode([text])[0])
|
||||
|
||||
|
||||
def _intent(store: Store, agent: Agent, model: str,
|
||||
encoder: _Encoder) -> np.ndarray | None:
|
||||
if not agent.intent.strip():
|
||||
return None
|
||||
key = intent_key(model, agent.intent)
|
||||
cached = store.intent_vector(agent.id, key)
|
||||
if cached is not None:
|
||||
return cached
|
||||
vec = encoder.encode(agent.intent)
|
||||
if vec is not None:
|
||||
store.set_intent_vector(agent.id, key, vec)
|
||||
return vec
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- the job
|
||||
|
||||
def score_agent(store: Store, agent: Agent, vectors: dict[str, np.ndarray],
|
||||
intent: np.ndarray | None, now: datetime) -> dict[str, int]:
|
||||
rows: list[tuple[str, float, str]] = []
|
||||
gated = 0
|
||||
for candidate in store.score_candidates(agent.gates):
|
||||
role_vec = vectors.get(candidate.uid)
|
||||
fit = fit_term(role_vec, intent) if role_vec is not None and intent is not None \
|
||||
else None
|
||||
value, reason = score_one(
|
||||
candidate, agent.gates, agent.weights, fit, now,
|
||||
missing="no vector" if intent is not None else "no intent vector")
|
||||
gated += bool(reason)
|
||||
rows.append((candidate.uid, round(value, 6), reason))
|
||||
store.replace_scores(agent.id, rows)
|
||||
return {"scored": len(rows), "gated": gated}
|
||||
|
||||
|
||||
@job("score", "Scoring roles for every agent", solo=True, resource="cpu")
|
||||
def run_score(config: Config, store: Store, *,
|
||||
context: RunContext = NULL) -> dict[str, Any]:
|
||||
seed_agents(config, store)
|
||||
agents = store.agents(enabled_only=True)
|
||||
model = config.embed.model
|
||||
vectors = store.vectors(model) if agents else {}
|
||||
encoder = _Encoder(model, context)
|
||||
now = store.clock()
|
||||
scored = gated = 0
|
||||
for n, agent in enumerate(agents):
|
||||
context.check()
|
||||
context.progress(n, len(agents), f"scoring for {agent.name}, {n + 1} of {len(agents)}")
|
||||
counts = score_agent(store, agent, vectors, _intent(store, agent, model, encoder), now)
|
||||
scored += counts["scored"]
|
||||
gated += counts["gated"]
|
||||
log.info("%-20s %6d scored, %6d gated", agent.name, counts["scored"], counts["gated"])
|
||||
context.progress(len(agents), len(agents), "")
|
||||
return {"agents": len(agents), "scored": scored, "gated": gated,
|
||||
"vectors": len(vectors)}
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ seeding
|
||||
|
||||
def seed_agents(config: Config, store: Store) -> list[str]:
|
||||
"""Import `roles.yaml`'s `agents:` into an empty table, once. The state
|
||||
marker is what makes it once: deleting every agent afterwards is a
|
||||
choice, and a restart must not undo it."""
|
||||
if store.state.get(SEEDED) or store.agents():
|
||||
return []
|
||||
added = []
|
||||
for seed in config.agents:
|
||||
try:
|
||||
store.save_agent(seed.name, seed.intent, seed.gates, seed.weights,
|
||||
seed.enabled)
|
||||
except ValueError as exc:
|
||||
log.warning("agents: %s", exc)
|
||||
continue
|
||||
added.append(seed.name)
|
||||
if added:
|
||||
store.state[SEEDED] = store.now()
|
||||
log.info("imported %d agents from %s", len(added), config.path or "roles.yaml")
|
||||
return added
|
||||
+66
-1
@@ -15,7 +15,7 @@ import re
|
||||
import unicodedata
|
||||
from typing import Annotated, Any, Literal, get_args
|
||||
|
||||
from pydantic import BeforeValidator
|
||||
from pydantic import BeforeValidator, Field
|
||||
|
||||
from desk.models import JsonDict, Model, ModelTaskResult
|
||||
|
||||
@@ -152,6 +152,14 @@ class ExtractResult(ModelTaskResult):
|
||||
return self.extracted
|
||||
|
||||
|
||||
class ScoredRole(Role):
|
||||
"""A role as `/api/roles` serves it: with one agent's score and gate
|
||||
reason when an agent was asked for, and without otherwise."""
|
||||
|
||||
score: float | None = None
|
||||
gated_reason: str = ""
|
||||
|
||||
|
||||
class Company(Model):
|
||||
key: str
|
||||
name: str
|
||||
@@ -202,6 +210,63 @@ class Application(Model):
|
||||
updated_at: str = ""
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ agents
|
||||
|
||||
#: The role columns a gate can name. `company` matches on the company key;
|
||||
#: `place` is a case-insensitive substring, like the place filter; the rest are
|
||||
#: case-insensitive equality, except `salary_min` (at least the value) and
|
||||
#: `languages` (every language the role requires is one of the values).
|
||||
GateField = Literal["place", "country", "remote", "seniority", "function", "salary_min",
|
||||
"currency", "permit", "languages", "company"]
|
||||
GATE_FIELDS: tuple[str, ...] = get_args(GateField)
|
||||
|
||||
GateMode = Literal["require", "prefer"]
|
||||
|
||||
|
||||
class Gate(Model):
|
||||
"""One condition of an agent. `require` gates a role that fails it (the
|
||||
row is kept with the reason); `prefer` adds `weight` to its score when it
|
||||
matches."""
|
||||
|
||||
field: GateField
|
||||
values: list[str | int | float] = Field(min_length=1)
|
||||
mode: GateMode = "require"
|
||||
weight: float = Field(1.0, ge=0)
|
||||
|
||||
|
||||
class Weights(Model):
|
||||
"""How much each score term counts. Summed with the prefer gates' weights
|
||||
into the denominator, so a score stays in 0..1 whatever the numbers."""
|
||||
|
||||
fit: float = Field(0.6, ge=0)
|
||||
recency: float = Field(0.2, ge=0)
|
||||
affinity: float = Field(0.2, ge=0)
|
||||
|
||||
|
||||
class Agent(Model):
|
||||
"""A row of `agents`: a named search with its own intent, gates and
|
||||
weights. The cached intent vector is a column this does not declare."""
|
||||
|
||||
id: int
|
||||
name: str
|
||||
intent: str = ""
|
||||
gates: Annotated[list[Gate], BeforeValidator(lambda v: _json(v, "[]"))] = []
|
||||
weights: Annotated[Weights, BeforeValidator(lambda v: _json(v, "{}"))] = Weights()
|
||||
enabled: bool = True
|
||||
created_at: str
|
||||
|
||||
|
||||
class Candidate(Model):
|
||||
"""What scoring reads about one open role for one agent: when it was
|
||||
first seen, its company's affinity, and per gate `1` match, `0` no
|
||||
match, `-1` the column is empty."""
|
||||
|
||||
uid: str
|
||||
first_seen: str
|
||||
affinity: int = 0
|
||||
gates: list[int] = []
|
||||
|
||||
|
||||
class ApplicationEvent(Model):
|
||||
id: int
|
||||
application_id: str
|
||||
|
||||
@@ -142,6 +142,8 @@ END;
|
||||
|
||||
-- A search agent: a named saved search with its own intent text, gates and
|
||||
-- weights. gates is a list of {field, values, mode: require|prefer, weight}.
|
||||
-- intent_vec caches the intent's vector; intent_key is sha1 of the embedding
|
||||
-- model and the intent text, so editing either makes it stale.
|
||||
CREATE TABLE IF NOT EXISTS agents (
|
||||
id INTEGER PRIMARY KEY,
|
||||
name TEXT NOT NULL UNIQUE,
|
||||
@@ -149,7 +151,9 @@ CREATE TABLE IF NOT EXISTS agents (
|
||||
gates TEXT NOT NULL DEFAULT '[]',
|
||||
weights TEXT NOT NULL DEFAULT '{}',
|
||||
enabled INTEGER NOT NULL DEFAULT 1,
|
||||
created_at TEXT NOT NULL
|
||||
created_at TEXT NOT NULL,
|
||||
intent_key TEXT NOT NULL DEFAULT '',
|
||||
intent_vec BLOB
|
||||
);
|
||||
|
||||
-- A role is scored per agent. A gated role keeps its row, with the reason.
|
||||
@@ -162,6 +166,16 @@ CREATE TABLE IF NOT EXISTS scores (
|
||||
PRIMARY KEY (uid, agent_id)
|
||||
);
|
||||
|
||||
-- One vector per role, little-endian float32. Only comparable to another from
|
||||
-- the same model, so every read is scoped by it.
|
||||
CREATE TABLE IF NOT EXISTS vectors (
|
||||
uid TEXT PRIMARY KEY REFERENCES roles(uid) ON DELETE CASCADE,
|
||||
model TEXT NOT NULL,
|
||||
dim INTEGER NOT NULL,
|
||||
vec BLOB NOT NULL,
|
||||
embedded_at TEXT NOT NULL
|
||||
);
|
||||
|
||||
-- The string map a desk plugin is handed as self.state. One namespace shared
|
||||
-- by every plugin, so a plugin prefixes its keys.
|
||||
CREATE TABLE IF NOT EXISTS state (
|
||||
@@ -182,6 +196,7 @@ CREATE INDEX IF NOT EXISTS idx_roles_board ON roles(source, subscription_id
|
||||
CREATE INDEX IF NOT EXISTS idx_application_events
|
||||
ON application_events(application_id, at);
|
||||
CREATE INDEX IF NOT EXISTS idx_scores_agent ON scores(agent_id, score DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_vectors_model ON vectors(model);
|
||||
"""
|
||||
|
||||
#: The table whose absence marks a file that has never been a roledesk
|
||||
@@ -199,6 +214,17 @@ ALTER TABLE roles ADD COLUMN permit TEXT NOT NULL DEFAULT ''
|
||||
CHECK (permit IN ('', 'sponsors', 'no_sponsorship'));
|
||||
ALTER TABLE roles ADD COLUMN languages TEXT NOT NULL DEFAULT '[]';
|
||||
ALTER TABLE roles ADD COLUMN extract_tries INTEGER NOT NULL DEFAULT 0;
|
||||
"""),
|
||||
(3, "vectors, and the agent's cached intent vector, for score", """
|
||||
CREATE TABLE vectors (
|
||||
uid TEXT PRIMARY KEY REFERENCES roles(uid) ON DELETE CASCADE,
|
||||
model TEXT NOT NULL,
|
||||
dim INTEGER NOT NULL,
|
||||
vec BLOB NOT NULL,
|
||||
embedded_at TEXT NOT NULL
|
||||
);
|
||||
ALTER TABLE agents ADD COLUMN intent_key TEXT NOT NULL DEFAULT '';
|
||||
ALTER TABLE agents ADD COLUMN intent_vec BLOB;
|
||||
"""),
|
||||
)
|
||||
|
||||
|
||||
+207
-7
@@ -17,10 +17,13 @@ from pathlib import Path
|
||||
from typing import Any
|
||||
from uuid import uuid4
|
||||
|
||||
import numpy as np
|
||||
|
||||
from desk.models import Item
|
||||
from roledesk.models import (
|
||||
AFFINITY, REMOTE, STATUSES, Application, ApplicationEvent, Company, Role,
|
||||
Subscription, company_key, place_key, title_key,
|
||||
AFFINITY, REMOTE, STATUSES, Agent, Application, ApplicationEvent, Candidate,
|
||||
Company, Gate, Role, ScoredRole, Subscription, Weights, company_key, place_key,
|
||||
title_key,
|
||||
)
|
||||
from roledesk.storage import schema
|
||||
|
||||
@@ -73,6 +76,43 @@ def _like(text: str) -> str:
|
||||
return f"%{escaped}%"
|
||||
|
||||
|
||||
#: Vectors are stored little-endian float32, fixed here so a file written on
|
||||
#: one machine reads on another. newsdesk's encoding, written again.
|
||||
_DTYPE = np.dtype("<f4")
|
||||
|
||||
|
||||
def pack(vec: np.ndarray) -> bytes:
|
||||
return np.asarray(vec, dtype=_DTYPE).tobytes()
|
||||
|
||||
|
||||
def unpack(blob: bytes) -> np.ndarray:
|
||||
return np.frombuffer(blob, dtype=_DTYPE)
|
||||
|
||||
|
||||
def _gate_sql(gate: Gate) -> tuple[str, str, list[Any]]:
|
||||
"""One gate as `(empty, match, params)`: SQL that is true when the role's
|
||||
column is empty, and SQL that is true when it matches, over `roles r`.
|
||||
`params` belong to `match`; `empty` takes none."""
|
||||
column = f"r.{gate.field}"
|
||||
values = [str(v).strip().lower() for v in gate.values if str(v).strip()]
|
||||
marks = ", ".join("?" for _ in values) or "NULL"
|
||||
if gate.field == "salary_min":
|
||||
floor = min(_int_or_none(v) or 0 for v in gate.values)
|
||||
return f"{column} IS NULL", f"{column} >= ?", [floor]
|
||||
if gate.field == "place":
|
||||
likes = " OR ".join("LOWER(r.place) LIKE ? ESCAPE '\\'" for _ in values) or "0"
|
||||
return "r.place = ''", f"({likes})", [_like(v) for v in values]
|
||||
if gate.field == "company":
|
||||
keys = [company_key(v) for v in values]
|
||||
return ("r.company_key = ''",
|
||||
f"r.company_key IN ({', '.join('?' for _ in keys) or 'NULL'})", keys)
|
||||
if gate.field == "languages":
|
||||
return ("r.languages IN ('', '[]')",
|
||||
f"NOT EXISTS (SELECT 1 FROM json_each(r.languages) "
|
||||
f"WHERE LOWER(value) NOT IN ({marks}))", values)
|
||||
return f"{column} = ''", f"LOWER({column}) IN ({marks})", values
|
||||
|
||||
|
||||
def _int_or_none(value: Any) -> int | None:
|
||||
if value is None or value == "" or isinstance(value, bool):
|
||||
return None
|
||||
@@ -229,7 +269,8 @@ class Store:
|
||||
place: str | None = None, open_only: bool = True,
|
||||
q: str | None = None, function: str | None = None,
|
||||
remote: str | None = None, days: int | None = None,
|
||||
hide_muted: bool = False) -> list[Role]:
|
||||
hide_muted: bool = False, agent: int | None = None,
|
||||
gated: bool = False) -> list[ScoredRole]:
|
||||
"""Newest first by `first_seen`. `company` and `place` are
|
||||
case-insensitive substrings of the columns a board wrote — not the
|
||||
key, so a search does not have to know the exact slug a company folds
|
||||
@@ -243,7 +284,11 @@ class Store:
|
||||
finally reached it — can otherwise fill the whole page with roles
|
||||
that then all turn out to be from the one company someone muted,
|
||||
leaving nothing to show for a default query that would otherwise
|
||||
have plenty."""
|
||||
have plenty.
|
||||
|
||||
`agent` orders by that agent's score instead, highest first, and
|
||||
carries `score` and `gated_reason` on each row. Only roles it has
|
||||
scored are listed, and a gated one only when `gated` is set."""
|
||||
where, params = [], []
|
||||
if company:
|
||||
where.append("company LIKE ? ESCAPE '\\'")
|
||||
@@ -268,11 +313,22 @@ class Store:
|
||||
if hide_muted:
|
||||
where.append("company_key NOT IN "
|
||||
"(SELECT key FROM companies WHERE affinity = -1)")
|
||||
sql = "SELECT * FROM roles"
|
||||
joins: list[Any] = []
|
||||
if agent is None:
|
||||
sql = "SELECT roles.* FROM roles"
|
||||
order = "roles.first_seen DESC, roles.uid"
|
||||
else:
|
||||
sql = ("SELECT roles.*, s.score, s.gated_reason FROM roles "
|
||||
"JOIN scores s ON s.uid = roles.uid AND s.agent_id = ?")
|
||||
joins.append(agent)
|
||||
if not gated:
|
||||
where.append("s.gated_reason = ''")
|
||||
order = "s.score DESC, roles.first_seen DESC, roles.uid"
|
||||
if where:
|
||||
sql += " WHERE " + " AND ".join(where)
|
||||
sql += " ORDER BY first_seen DESC, uid LIMIT ?"
|
||||
return [Role.from_row(r) for r in self._rows(sql, [*params, limit])]
|
||||
sql += f" ORDER BY {order} LIMIT ?"
|
||||
return [ScoredRole.from_row(r)
|
||||
for r in self._rows(sql, [*joins, *params, limit])]
|
||||
|
||||
def mark_closed(self, source: str, seen_ids: Sequence[str],
|
||||
subscription_id: int) -> int:
|
||||
@@ -353,6 +409,150 @@ class Store:
|
||||
f"WHERE {where}", (wanted,))
|
||||
return int(crossed)
|
||||
|
||||
# -------------------------------------------------------------- vectors
|
||||
|
||||
def roles_to_embed(self, model: str, limit: int) -> list[Role]:
|
||||
"""Open roles with no vector from `model`, newest first, so a backlog
|
||||
embedded from the top improves what is on screen first."""
|
||||
return [Role.from_row(r) for r in self._rows(
|
||||
"""SELECT r.* FROM roles r
|
||||
LEFT JOIN vectors v ON v.uid = r.uid AND v.model = ?
|
||||
WHERE v.uid IS NULL AND r.closed_at IS NULL
|
||||
ORDER BY r.first_seen DESC, r.uid LIMIT ?""", (model, limit))]
|
||||
|
||||
def put_vectors(self, model: str, pairs: Sequence[tuple[str, np.ndarray]]) -> int:
|
||||
stamp = self.now()
|
||||
with self.transaction() as conn:
|
||||
return conn.executemany(
|
||||
"""INSERT INTO vectors (uid, model, dim, vec, embedded_at)
|
||||
VALUES (?,?,?,?,?)
|
||||
ON CONFLICT(uid) DO UPDATE SET model=excluded.model,
|
||||
dim=excluded.dim, vec=excluded.vec,
|
||||
embedded_at=excluded.embedded_at""",
|
||||
[(uid, model, int(vec.shape[-1]), pack(vec), stamp)
|
||||
for uid, vec in pairs]).rowcount
|
||||
|
||||
def vectors(self, model: str) -> dict[str, np.ndarray]:
|
||||
"""Every vector from `model`, by uid. The whole table: at ~30k roles of
|
||||
384 floats that is ~45 MB, and one read beats one per agent."""
|
||||
return {r["uid"]: unpack(r["vec"]) for r in self._rows(
|
||||
"SELECT uid, vec FROM vectors WHERE model = ?", (model,))}
|
||||
|
||||
def vector_counts(self) -> dict[str, int]:
|
||||
return {r["model"]: int(r["n"]) for r in self._rows(
|
||||
"SELECT model, COUNT(*) AS n FROM vectors GROUP BY model")}
|
||||
|
||||
def drop_vectors(self, keep: str) -> int:
|
||||
"""Delete vectors from every model but `keep`: another model's are a
|
||||
different geometry, never compared, and only make counts lie."""
|
||||
return self._write("DELETE FROM vectors WHERE model <> ?", (keep,))
|
||||
|
||||
# --------------------------------------------------------------- agents
|
||||
|
||||
def agents(self, enabled_only: bool = False) -> list[Agent]:
|
||||
sql = "SELECT * FROM agents"
|
||||
if enabled_only:
|
||||
sql += " WHERE enabled = 1"
|
||||
return [Agent.from_row(r)
|
||||
for r in self._rows(sql + " ORDER BY name COLLATE NOCASE")]
|
||||
|
||||
def agent(self, agent_id: int) -> Agent | None:
|
||||
row = self._row("SELECT * FROM agents WHERE id=?", (agent_id,))
|
||||
return Agent.from_row(row) if row else None
|
||||
|
||||
def save_agent(self, name: str, intent: str = "", gates: Sequence[Gate] = (),
|
||||
weights: Weights | None = None, enabled: bool = True,
|
||||
agent_id: int | None = None) -> Agent:
|
||||
"""Create an agent, or with `agent_id` replace that one's fields.
|
||||
Raises ValueError on an empty or taken name, KeyError on an unknown id."""
|
||||
name = name.strip()
|
||||
if not name:
|
||||
raise ValueError("an agent needs a name")
|
||||
fields = (name, intent, json.dumps([g.model_dump() for g in gates]),
|
||||
(weights or Weights()).model_dump_json(), int(enabled))
|
||||
try:
|
||||
with self.transaction() as conn:
|
||||
if agent_id is None:
|
||||
agent_id = conn.execute(
|
||||
"""INSERT INTO agents (name, intent, gates, weights, enabled,
|
||||
created_at) VALUES (?,?,?,?,?,?)""",
|
||||
(*fields, self.now())).lastrowid
|
||||
elif not conn.execute(
|
||||
"""UPDATE agents SET name=?, intent=?, gates=?, weights=?,
|
||||
enabled=? WHERE id=?""",
|
||||
(*fields, agent_id)).rowcount:
|
||||
raise KeyError(agent_id)
|
||||
except sqlite3.IntegrityError as exc:
|
||||
raise ValueError(f"an agent named {name!r} already exists") from exc
|
||||
assert agent_id is not None
|
||||
found = self.agent(agent_id)
|
||||
assert found is not None
|
||||
return found
|
||||
|
||||
def delete_agent(self, agent_id: int) -> bool:
|
||||
"""Its scores go with it, by the foreign key."""
|
||||
return self._write("DELETE FROM agents WHERE id=?", (agent_id,)) > 0
|
||||
|
||||
def intent_vector(self, agent_id: int, key: str) -> np.ndarray | None:
|
||||
"""The agent's cached intent vector, if it was made under `key`."""
|
||||
row = self._row("SELECT intent_key, intent_vec FROM agents WHERE id=?",
|
||||
(agent_id,))
|
||||
if row is None or row["intent_key"] != key or row["intent_vec"] is None:
|
||||
return None
|
||||
return unpack(row["intent_vec"])
|
||||
|
||||
def set_intent_vector(self, agent_id: int, key: str, vec: np.ndarray) -> None:
|
||||
self._write("UPDATE agents SET intent_key=?, intent_vec=? WHERE id=?",
|
||||
(key, pack(vec), agent_id))
|
||||
|
||||
# --------------------------------------------------------------- scores
|
||||
|
||||
def score_candidates(self, gates: Sequence[Gate]) -> list[Candidate]:
|
||||
"""Every open role of a company that is not muted, with how it meets
|
||||
each gate: 1 match, 0 no match, -1 the column is empty. The gates are
|
||||
evaluated here, in SQL, so a `require` gate is the `WHERE` it reads
|
||||
as, written as a column so a gated role keeps its row and reason."""
|
||||
columns: list[str] = []
|
||||
params: list[Any] = []
|
||||
for n, gate in enumerate(gates):
|
||||
empty, match, values = _gate_sql(gate)
|
||||
columns.append(
|
||||
f"CASE WHEN {empty} THEN -1 WHEN {match} THEN 1 ELSE 0 END AS g{n}")
|
||||
params += values
|
||||
select = ", ".join(["r.uid", "r.first_seen",
|
||||
"COALESCE(c.affinity, 0) AS affinity", *columns])
|
||||
rows = self._rows(
|
||||
f"""SELECT {select} FROM roles r
|
||||
LEFT JOIN companies c ON c.key = r.company_key
|
||||
WHERE r.closed_at IS NULL AND COALESCE(c.affinity, 0) <> -1""",
|
||||
params)
|
||||
return [Candidate(uid=r["uid"], first_seen=r["first_seen"],
|
||||
affinity=int(r["affinity"]),
|
||||
gates=[int(r[f"g{n}"]) for n in range(len(gates))])
|
||||
for r in rows]
|
||||
|
||||
def replace_scores(self, agent_id: int,
|
||||
rows: Sequence[tuple[str, float, str]]) -> int:
|
||||
"""`(uid, score, gated_reason)` for every role this agent scored,
|
||||
replacing whatever it had: a role closed or muted since the last run
|
||||
drops out rather than keeping a stale score. One transaction, so a
|
||||
reader sees the old list or the new one."""
|
||||
stamp = self.now()
|
||||
with self.transaction() as conn:
|
||||
conn.execute("DELETE FROM scores WHERE agent_id=?", (agent_id,))
|
||||
conn.executemany(
|
||||
"""INSERT INTO scores (uid, agent_id, score, gated_reason, scored_at)
|
||||
VALUES (?,?,?,?,?)""",
|
||||
[(uid, agent_id, score, reason, stamp) for uid, score, reason in rows])
|
||||
return len(rows)
|
||||
|
||||
def scores(self, agent_id: int) -> dict[str, tuple[float, str]]:
|
||||
"""`uid -> (score, gated_reason)` for one agent."""
|
||||
return {r["uid"]: (float(r["score"]), str(r["gated_reason"]))
|
||||
for r in self._rows(
|
||||
"SELECT uid, score, gated_reason FROM scores WHERE agent_id=?",
|
||||
(agent_id,))}
|
||||
|
||||
# ------------------------------------------------------------ companies
|
||||
|
||||
def add_company(self, name: str, *, key: str = "", domain: str = "",
|
||||
|
||||
+54
-7
@@ -1,4 +1,4 @@
|
||||
"""The JSON routes the page uses: roles, companies, applications.
|
||||
"""The JSON routes the page uses: roles, companies, applications, agents.
|
||||
|
||||
`/api/jobs*`, `/_jobs/jobs.js` and `/ws/jobs` are `pipeline.web`'s, mounted by
|
||||
`connect` in `app.py`. This is roledesk's own API, over the same store.
|
||||
@@ -9,10 +9,12 @@ from __future__ import annotations
|
||||
from datetime import timedelta
|
||||
from typing import Annotated
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Query
|
||||
from pydantic import BaseModel
|
||||
from fastapi import APIRouter, HTTPException, Query, Response
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from roledesk.models import AFFINITY, Application, Company, Role, Status
|
||||
from roledesk.models import (
|
||||
AFFINITY, Agent, Application, Company, Gate, ScoredRole, Status, Weights,
|
||||
)
|
||||
from roledesk.web.deps import StoreDep
|
||||
|
||||
router = APIRouter(prefix="/api", tags=["roledesk"])
|
||||
@@ -30,14 +32,21 @@ def list_roles(
|
||||
remote: str = "",
|
||||
days: int | None = None,
|
||||
show_muted: bool = False,
|
||||
agent: int | None = None,
|
||||
gated: bool = False,
|
||||
limit: Annotated[int, Query(ge=1, le=1000)] = 200,
|
||||
) -> list[Role]:
|
||||
) -> list[ScoredRole]:
|
||||
"""Open roles, newest first. See `Store.roles` for what each filter does.
|
||||
|
||||
A muted company's roles are left out unless `show_muted` is set."""
|
||||
A muted company's roles are left out unless `show_muted` is set. With
|
||||
`agent`, ordered by that agent's score and carrying `score` and
|
||||
`gated_reason`; a gated role only with `gated=1`."""
|
||||
if agent is not None and store.agent(agent) is None:
|
||||
raise HTTPException(404, f"no such agent: {agent}")
|
||||
return store.roles(limit=limit, q=q or None, company=company or None,
|
||||
place=place or None, function=function or None,
|
||||
remote=remote or None, days=days, hide_muted=not show_muted)
|
||||
remote=remote or None, days=days, hide_muted=not show_muted,
|
||||
agent=agent, gated=gated)
|
||||
|
||||
|
||||
# --------------------------------------------------------------- companies
|
||||
@@ -122,3 +131,41 @@ def set_status(app_id: str, body: StatusBody, store: StoreDep) -> Application:
|
||||
raise HTTPException(422, str(exc)) from exc
|
||||
except KeyError:
|
||||
raise HTTPException(404, f"no such application: {app_id}") from None
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ agents
|
||||
|
||||
@router.get("/agents")
|
||||
def list_agents(store: StoreDep) -> list[Agent]:
|
||||
return store.agents()
|
||||
|
||||
|
||||
class AgentBody(BaseModel):
|
||||
"""Create when `id` is missing, replace that agent's fields when given."""
|
||||
|
||||
id: int | None = None
|
||||
name: str = Field(min_length=1)
|
||||
intent: str = ""
|
||||
gates: list[Gate] = []
|
||||
weights: Weights = Weights()
|
||||
enabled: bool = True
|
||||
|
||||
|
||||
@router.post("/agents")
|
||||
def save_agent(body: AgentBody, store: StoreDep) -> Agent:
|
||||
"""Scores are not recomputed here: that is the `score` task, which the
|
||||
page starts after a save."""
|
||||
try:
|
||||
return store.save_agent(body.name, body.intent, body.gates, body.weights,
|
||||
body.enabled, agent_id=body.id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(409, str(exc)) from exc
|
||||
except KeyError:
|
||||
raise HTTPException(404, f"no such agent: {body.id}") from None
|
||||
|
||||
|
||||
@router.delete("/agents/{agent_id}", status_code=204)
|
||||
def delete_agent(agent_id: int, store: StoreDep) -> Response:
|
||||
if not store.delete_agent(agent_id):
|
||||
raise HTTPException(404, f"no such agent: {agent_id}")
|
||||
return Response(status_code=204)
|
||||
@@ -24,6 +24,7 @@ def create_app(config_path: str | None = None) -> FastAPI:
|
||||
# `Store` first, so the file is at the current schema before the queue
|
||||
# adds its own table to it.
|
||||
store = Store(config.database)
|
||||
jobs.seed_agents(config, store)
|
||||
app = FastAPI(title="roledesk", docs_url="/api/docs")
|
||||
connect(app, Jobs(config.database), config, store,
|
||||
workers=config.workers.model_dump(),
|
||||
|
||||
@@ -159,6 +159,40 @@ button:focus-visible, a:focus-visible, input:focus-visible, select:focus-visible
|
||||
}
|
||||
|
||||
.role.muted { opacity: .55; }
|
||||
.role.gated { opacity: .7; }
|
||||
.show-muted[hidden] { display: none; }
|
||||
.role-score.badge:empty { display: none; }
|
||||
.role-score.badge { color: var(--signal); border-color: var(--signal-weak); font-variant-numeric: tabular-nums; }
|
||||
.role-gated { color: var(--mute); font-size: .85rem; }
|
||||
|
||||
/* ---------------------------------------------------------------- agents */
|
||||
|
||||
.agents { list-style: none; margin: 0 0 1rem; padding: 0; display: flex; flex-direction: column; gap: .4rem; }
|
||||
.agent {
|
||||
display: flex; align-items: center; gap: .75rem; flex-wrap: wrap;
|
||||
background: var(--panel); border: 1px solid var(--rule); border-radius: 6px; padding: .5rem .75rem;
|
||||
}
|
||||
.agent-meta { color: var(--faint); font-size: .85rem; flex: 1 1 12rem; }
|
||||
.agent-form {
|
||||
display: flex; flex-direction: column; gap: .6rem; max-width: 44rem;
|
||||
background: var(--panel); border: 1px solid var(--rule); border-radius: 6px; padding: .75rem 1rem;
|
||||
}
|
||||
.agent-form h2 { font-size: 1rem; margin: 0; }
|
||||
.agent-form label { display: flex; flex-direction: column; gap: .2rem; font-size: .9rem; }
|
||||
.agent-form label.inline { flex-direction: row; align-items: center; gap: .4rem; }
|
||||
.agent-form input[type="text"], .agent-form textarea, .agent-form input[type="number"] {
|
||||
font: inherit; color: inherit; background: var(--bg); border: 1px solid var(--rule);
|
||||
border-radius: 4px; padding: .35rem .5rem; width: 100%;
|
||||
}
|
||||
.agent-form textarea { font-family: ui-monospace, "Cascadia Mono", Consolas, monospace; font-size: .85rem; }
|
||||
.agent-form .hint { color: var(--faint); font-size: .8rem; overflow-wrap: anywhere; }
|
||||
.weights { display: flex; gap: .75rem; flex-wrap: wrap; border: 1px solid var(--rule); border-radius: 4px; }
|
||||
.weights label { flex: 1 1 6rem; }
|
||||
.agent-actions { display: flex; gap: .5rem; flex-wrap: wrap; }
|
||||
.agent-actions button, .agent button {
|
||||
background: var(--bg); border: 1px solid var(--rule); border-radius: 4px; padding: .3rem .8rem; cursor: pointer;
|
||||
}
|
||||
.agent-actions button.primary { background: var(--signal); border-color: var(--signal); color: #fff; }
|
||||
|
||||
/* ------------------------------------------------------------ applications */
|
||||
|
||||
|
||||
+161
-3
@@ -1,5 +1,5 @@
|
||||
// roledesk's page: the filter bar, the role list, the tracker, and the tab
|
||||
// switch between them. The template renders an empty shell (see index.html);
|
||||
// roledesk's page: the filter bar, the role list, the tracker, the search
|
||||
// agents, and the tab switch between them. The template renders an empty shell (see index.html);
|
||||
// everything on screen comes from the API, and every action — Save, an
|
||||
// affinity click, a status change — is a fetch from here, not a form post.
|
||||
"use strict";
|
||||
@@ -12,6 +12,8 @@ let companies = new Map();
|
||||
let trackedByRole = new Map();
|
||||
/** @type {object[]} the last /api/roles response, already muted-filtered by the server */
|
||||
let roles = [];
|
||||
/** @type {object[]} every search agent, from /api/agents */
|
||||
let agents = [];
|
||||
|
||||
const $ = (sel, root = document) => root.querySelector(sel);
|
||||
const $$ = (sel, root = document) => Array.from(root.querySelectorAll(sel));
|
||||
@@ -42,7 +44,9 @@ function initTabs() {
|
||||
const name = tab.dataset.tab;
|
||||
$("#roles-view").hidden = name !== "roles";
|
||||
$("#applications-view").hidden = name !== "applications";
|
||||
$("#agents-view").hidden = name !== "agents";
|
||||
if (name === "applications") loadApplications();
|
||||
if (name === "agents") loadAgents();
|
||||
});
|
||||
});
|
||||
}
|
||||
@@ -64,6 +68,11 @@ function filterParams() {
|
||||
if (remote) params.set("remote", remote);
|
||||
if (days) params.set("days", days);
|
||||
if ($("#f-show-muted").checked) params.set("show_muted", "true");
|
||||
const agent = $("#f-agent").value;
|
||||
if (agent) {
|
||||
params.set("agent", agent);
|
||||
if ($("#f-gated").checked) params.set("gated", "1");
|
||||
}
|
||||
return params;
|
||||
}
|
||||
|
||||
@@ -104,6 +113,13 @@ function renderRoles() {
|
||||
node.querySelector(".role-remote").textContent = role.remote || "";
|
||||
node.querySelector(".role-function").textContent = role.function || "";
|
||||
node.querySelector(".role-age").textContent = relativeAge(role.first_seen);
|
||||
if (role.score !== null && role.score !== undefined) {
|
||||
const badge = node.querySelector(".role-score");
|
||||
badge.textContent = role.gated_reason ? "gated" : role.score.toFixed(2);
|
||||
badge.title = "Score for the selected agent";
|
||||
node.querySelector(".role-gated").textContent = role.gated_reason || "";
|
||||
if (role.gated_reason) li.classList.add("gated");
|
||||
}
|
||||
|
||||
node.querySelector(".affinity").replaceWith(
|
||||
affinityControl(role.company_key, company ? company.affinity : 0));
|
||||
@@ -260,6 +276,140 @@ async function setStatus(id, status) {
|
||||
}
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------- agents
|
||||
|
||||
async function loadAgents() {
|
||||
agents = await api("/api/agents");
|
||||
renderAgentSelect();
|
||||
renderAgents();
|
||||
}
|
||||
|
||||
// The filter bar's order: "Newest", or one agent's score. Keeps the current
|
||||
// choice when the list is refreshed, unless that agent is gone.
|
||||
function renderAgentSelect() {
|
||||
const select = $("#f-agent");
|
||||
const current = select.value;
|
||||
select.replaceChildren(new Option("Newest", ""));
|
||||
for (const agent of agents) {
|
||||
const label = agent.enabled ? agent.name : `${agent.name} (off)`;
|
||||
select.appendChild(new Option(label, String(agent.id)));
|
||||
}
|
||||
select.value = agents.some((a) => String(a.id) === current) ? current : "";
|
||||
$("#show-gated").hidden = !select.value;
|
||||
}
|
||||
|
||||
function renderAgents() {
|
||||
const list = $("#agents-list");
|
||||
list.replaceChildren();
|
||||
for (const agent of agents) {
|
||||
const li = document.createElement("li");
|
||||
li.className = "agent";
|
||||
const name = document.createElement("strong");
|
||||
name.textContent = agent.name;
|
||||
const meta = document.createElement("span");
|
||||
meta.className = "agent-meta";
|
||||
const gates = agent.gates.map((g) => `${g.mode} ${g.field}`).join(", ") || "no gates";
|
||||
meta.textContent = `${agent.enabled ? "" : "off · "}${gates}`;
|
||||
const edit = document.createElement("button");
|
||||
edit.type = "button";
|
||||
edit.textContent = "Edit";
|
||||
edit.addEventListener("click", () => fillAgentForm(agent));
|
||||
li.append(name, meta, edit);
|
||||
list.appendChild(li);
|
||||
}
|
||||
$("#agents-count").textContent = `${agents.length} agent${agents.length === 1 ? "" : "s"}`;
|
||||
}
|
||||
|
||||
function fillAgentForm(agent) {
|
||||
$("#agent-form-title").textContent = agent ? `Edit ${agent.name}` : "New agent";
|
||||
$("#a-id").value = agent ? String(agent.id) : "";
|
||||
$("#a-name").value = agent ? agent.name : "";
|
||||
$("#a-intent").value = agent ? agent.intent : "";
|
||||
$("#a-gates").value = JSON.stringify(agent ? agent.gates : [], null, 2);
|
||||
const weights = agent ? agent.weights : { fit: 0.6, recency: 0.2, affinity: 0.2 };
|
||||
$("#a-fit").value = weights.fit;
|
||||
$("#a-recency").value = weights.recency;
|
||||
$("#a-affinity").value = weights.affinity;
|
||||
$("#a-enabled").checked = agent ? agent.enabled : true;
|
||||
$("#a-delete").hidden = !agent;
|
||||
$("#agent-status").textContent = "";
|
||||
}
|
||||
|
||||
async function saveAgent(event) {
|
||||
event.preventDefault();
|
||||
const status = $("#agent-status");
|
||||
let gates;
|
||||
try {
|
||||
gates = JSON.parse($("#a-gates").value || "[]");
|
||||
} catch (err) {
|
||||
status.textContent = `Gates are not valid JSON: ${err.message}`;
|
||||
return;
|
||||
}
|
||||
const id = $("#a-id").value;
|
||||
const body = {
|
||||
name: $("#a-name").value.trim(),
|
||||
intent: $("#a-intent").value,
|
||||
gates,
|
||||
weights: {
|
||||
fit: Number($("#a-fit").value),
|
||||
recency: Number($("#a-recency").value),
|
||||
affinity: Number($("#a-affinity").value),
|
||||
},
|
||||
enabled: $("#a-enabled").checked,
|
||||
};
|
||||
if (id) body.id = Number(id);
|
||||
try {
|
||||
const saved = await api("/api/agents", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
await loadAgents();
|
||||
fillAgentForm(saved);
|
||||
await rescore(status);
|
||||
} catch (err) {
|
||||
status.textContent = err.message;
|
||||
}
|
||||
}
|
||||
|
||||
async function deleteAgent() {
|
||||
const id = $("#a-id").value;
|
||||
if (!id || !confirm("Delete this agent and its scores?")) return;
|
||||
try {
|
||||
await api(`/api/agents/${encodeURIComponent(id)}`, { method: "DELETE" });
|
||||
fillAgentForm(null);
|
||||
await loadAgents();
|
||||
await loadRoles();
|
||||
} catch (err) {
|
||||
$("#agent-status").textContent = err.message;
|
||||
}
|
||||
}
|
||||
|
||||
// Start the `score` job and refetch the roles once it has finished, so the
|
||||
// order on the Roles tab is the saved agent's.
|
||||
async function rescore(status) {
|
||||
status.textContent = "Saved. Scoring…";
|
||||
const job = await api("/api/jobs", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ kind: "score", payload: {} }),
|
||||
});
|
||||
for (let i = 0; i < 300; i += 1) {
|
||||
await new Promise((resolve) => setTimeout(resolve, 1000));
|
||||
const snapshot = await api("/api/jobs");
|
||||
const busy = [...snapshot.running, ...snapshot.queued].some((j) => j.id === job.id);
|
||||
if (!busy) {
|
||||
const done = snapshot.recent.find((j) => j.id === job.id);
|
||||
status.textContent = done && done.state !== "done"
|
||||
? `Scoring ${done.state}: ${done.error || ""}`
|
||||
: "Saved and scored.";
|
||||
await loadRoles();
|
||||
return;
|
||||
}
|
||||
}
|
||||
status.textContent = "Saved. Scoring is still running; see Jobs below.";
|
||||
}
|
||||
|
||||
// -------------------------------------------------------------------- init
|
||||
|
||||
async function loadCompanies() {
|
||||
@@ -269,7 +419,7 @@ async function loadCompanies() {
|
||||
|
||||
async function init() {
|
||||
initTabs();
|
||||
await Promise.all([loadCompanies(), loadApplications()]);
|
||||
await Promise.all([loadCompanies(), loadApplications(), loadAgents()]);
|
||||
await loadRoles();
|
||||
|
||||
const refetch = debounce(loadRoles, 300);
|
||||
@@ -280,6 +430,14 @@ async function init() {
|
||||
$("#f-remote").addEventListener("change", loadRoles);
|
||||
$("#f-days").addEventListener("change", loadRoles);
|
||||
$("#f-show-muted").addEventListener("change", loadRoles);
|
||||
$("#f-agent").addEventListener("change", () => {
|
||||
$("#show-gated").hidden = !$("#f-agent").value;
|
||||
loadRoles();
|
||||
});
|
||||
$("#f-gated").addEventListener("change", loadRoles);
|
||||
$("#agent-form").addEventListener("submit", saveAgent);
|
||||
$("#a-new").addEventListener("click", () => fillAgentForm(null));
|
||||
$("#a-delete").addEventListener("click", deleteAgent);
|
||||
$("#filters").addEventListener("submit", (e) => e.preventDefault());
|
||||
}
|
||||
|
||||
|
||||
@@ -13,11 +13,15 @@
|
||||
<nav class="tabs" id="tabs">
|
||||
<button type="button" class="tab" data-tab="roles" aria-pressed="true">Roles</button>
|
||||
<button type="button" class="tab" data-tab="applications" aria-pressed="false">Applications</button>
|
||||
<button type="button" class="tab" data-tab="agents" aria-pressed="false">Agents</button>
|
||||
</nav>
|
||||
</header>
|
||||
|
||||
<section id="roles-view" class="view">
|
||||
<form id="filters" class="filters" autocomplete="off">
|
||||
<select id="f-agent" aria-label="Order">
|
||||
<option value="">Newest</option>
|
||||
</select>
|
||||
<input type="search" id="f-q" placeholder="Search titles" aria-label="Search titles">
|
||||
<input type="text" id="f-company" placeholder="Company" aria-label="Company">
|
||||
<input type="text" id="f-place" placeholder="Place" aria-label="Place">
|
||||
@@ -43,6 +47,9 @@
|
||||
<label class="show-muted">
|
||||
<input type="checkbox" id="f-show-muted"> Show muted
|
||||
</label>
|
||||
<label class="show-muted" id="show-gated" hidden>
|
||||
<input type="checkbox" id="f-gated"> Show gated
|
||||
</label>
|
||||
</form>
|
||||
|
||||
<p id="roles-count" class="count" role="status"></p>
|
||||
@@ -59,6 +66,33 @@
|
||||
</table>
|
||||
</section>
|
||||
|
||||
<section id="agents-view" class="view" hidden>
|
||||
<p id="agents-count" class="count" role="status"></p>
|
||||
<ul id="agents-list" class="agents"></ul>
|
||||
<form id="agent-form" class="agent-form" autocomplete="off">
|
||||
<h2 id="agent-form-title">New agent</h2>
|
||||
<input type="hidden" id="a-id">
|
||||
<label>Name <input type="text" id="a-name" required></label>
|
||||
<label>Intent <span class="hint">a CV paragraph, or what this search is for</span>
|
||||
<textarea id="a-intent" rows="5"></textarea></label>
|
||||
<label>Gates <span class="hint">JSON: [{"field": "place", "values": ["Berlin"], "mode": "require"}, {"field": "remote", "values": ["remote"], "mode": "prefer", "weight": 0.2}]</span>
|
||||
<textarea id="a-gates" rows="5" spellcheck="false">[]</textarea></label>
|
||||
<fieldset class="weights">
|
||||
<legend>Weights</legend>
|
||||
<label>Fit <input type="number" id="a-fit" min="0" step="0.05" value="0.6"></label>
|
||||
<label>Recency <input type="number" id="a-recency" min="0" step="0.05" value="0.2"></label>
|
||||
<label>Affinity <input type="number" id="a-affinity" min="0" step="0.05" value="0.2"></label>
|
||||
</fieldset>
|
||||
<label class="inline"><input type="checkbox" id="a-enabled" checked> Enabled</label>
|
||||
<div class="agent-actions">
|
||||
<button type="submit" class="primary">Save and score</button>
|
||||
<button type="button" id="a-new">New</button>
|
||||
<button type="button" id="a-delete" hidden>Delete</button>
|
||||
</div>
|
||||
<p id="agent-status" class="count" role="status"></p>
|
||||
</form>
|
||||
</section>
|
||||
|
||||
<section id="jobs">
|
||||
<h2>Jobs</h2>
|
||||
<script type="module" src="/_jobs/jobs.js"></script>
|
||||
@@ -76,6 +110,8 @@
|
||||
<span class="role-place"></span>
|
||||
<span class="role-remote badge"></span>
|
||||
<span class="role-function badge"></span>
|
||||
<span class="role-score badge"></span>
|
||||
<span class="role-gated"></span>
|
||||
<span class="role-age"></span>
|
||||
</div>
|
||||
<button type="button" class="save"></button>
|
||||
|
||||
+21
@@ -10,3 +10,24 @@ extract:
|
||||
enabled: false
|
||||
model: claude-haiku-4-5-20251001
|
||||
sources: {}
|
||||
embed:
|
||||
model: sentence-transformers/all-MiniLM-L6-v2
|
||||
# Starter search agents, imported once into an empty table. Edit them on the
|
||||
# page afterwards; this list is not read again.
|
||||
agents:
|
||||
- name: Berlin backend
|
||||
intent: >-
|
||||
Senior backend engineer. Python and Go services, PostgreSQL, event-driven
|
||||
systems, APIs and data pipelines on Kubernetes in the cloud. Wants
|
||||
ownership of production systems in a product company.
|
||||
gates:
|
||||
- {field: place, values: [Berlin], mode: require}
|
||||
- {field: remote, values: [hybrid, remote], mode: prefer, weight: 0.2}
|
||||
- name: Remote EU stretch
|
||||
intent: >-
|
||||
Staff or lead engineer role, fully remote from Europe. Platform and
|
||||
infrastructure work, distributed systems, developer tooling, technical
|
||||
leadership across teams.
|
||||
gates:
|
||||
- {field: remote, values: [remote], mode: require}
|
||||
- {field: country, values: [DE, NL, AT, CH], mode: prefer, weight: 0.2}
|
||||
@@ -0,0 +1,313 @@
|
||||
"""`embed`, `score` and the pull cascade, with the embedding backend replaced
|
||||
by a deterministic fake. Nothing here downloads a model or reaches a board."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from collections.abc import Iterator, Sequence
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import numpy as np
|
||||
import pytest
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
if str(ROOT) not in sys.path:
|
||||
sys.path.insert(0, str(ROOT))
|
||||
|
||||
from desk.models import Item # noqa: E402
|
||||
from pipeline import FINAL, TASKS, JobQueue, Jobs, RunContext # noqa: E402
|
||||
from plugins.ats import ATSSource # noqa: E402
|
||||
from roledesk.config import AgentSeed, Config # noqa: E402
|
||||
from roledesk.jobs import cascade, embed, score # noqa: E402
|
||||
from roledesk.models import Candidate, Gate, Weights # noqa: E402
|
||||
from roledesk.storage import Store # noqa: E402
|
||||
|
||||
NOW = datetime(2026, 9, 23, 9, 0, tzinfo=UTC)
|
||||
|
||||
#: The fake's whole vocabulary: one dimension per word, plus a constant one so
|
||||
#: no text is the zero vector.
|
||||
WORDS = ("backend", "python", "frontend", "react", "data")
|
||||
|
||||
|
||||
class FakeEmbedder:
|
||||
model_id = "fake"
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.calls: list[list[str]] = []
|
||||
|
||||
def encode(self, texts: Sequence[str]) -> np.ndarray:
|
||||
self.calls.append(list(texts))
|
||||
rows = []
|
||||
for text in texts:
|
||||
lower = text.lower()
|
||||
vec = np.asarray([lower.count(w) for w in WORDS] + [0.1], dtype=np.float32)
|
||||
rows.append(vec / np.linalg.norm(vec))
|
||||
return np.vstack(rows)
|
||||
|
||||
|
||||
class Clock:
|
||||
def __init__(self) -> None:
|
||||
self.at = NOW
|
||||
|
||||
def __call__(self) -> datetime:
|
||||
return self.at
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake(monkeypatch: pytest.MonkeyPatch) -> FakeEmbedder:
|
||||
embedder = FakeEmbedder()
|
||||
monkeypatch.setattr(embed, "load_embedder", lambda model: embedder)
|
||||
return embedder
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def clock() -> Clock:
|
||||
return Clock()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def store(tmp_path: Path, clock: Clock) -> Iterator[Store]:
|
||||
s = Store(tmp_path / "roledesk.db")
|
||||
s.clock = clock
|
||||
yield s
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def config(tmp_path: Path) -> Config:
|
||||
return Config(database=str(tmp_path / "roledesk.db"), embed={"model": "fake"})
|
||||
|
||||
|
||||
def role(store: Store, source_id: str, title: str = "Backend Engineer", *,
|
||||
company: str = "Acme", body: str = "", **raw: Any) -> str:
|
||||
item = Item(source_id=source_id, location=f"https://boards.example/{source_id}",
|
||||
title=title, body=body, published_at=datetime(2026, 9, 20, tzinfo=UTC),
|
||||
raw={"company": company, **raw})
|
||||
item.source = "ats"
|
||||
store.upsert_role(item)
|
||||
return item.uid
|
||||
|
||||
|
||||
def run(config: Config, store: Store, job: Any) -> dict[str, Any]:
|
||||
notices: list[Any] = []
|
||||
result = dict(job(config, store, context=RunContext(on_notice=notices.append)))
|
||||
result["notices"] = [n.code for n in notices]
|
||||
return result
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ embed
|
||||
|
||||
def test_embed_writes_a_vector_per_open_role_once(
|
||||
store: Store, config: Config, fake: FakeEmbedder) -> None:
|
||||
role(store, "1")
|
||||
closed = role(store, "2")
|
||||
with store.transaction() as conn:
|
||||
conn.execute("UPDATE roles SET closed_at = ? WHERE uid = ?", (NOW.isoformat(), closed))
|
||||
|
||||
assert run(config, store, embed.run_embed)["embedded"] == 1
|
||||
assert run(config, store, embed.run_embed)["embedded"] == 0
|
||||
assert store.vector_counts() == {"fake": 1}
|
||||
|
||||
|
||||
def test_embed_without_sentence_transformers_says_so_and_does_nothing(
|
||||
store: Store, config: Config, monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
def missing(model: str) -> Any:
|
||||
raise ImportError("no sentence_transformers")
|
||||
|
||||
monkeypatch.setattr(embed, "load_embedder", missing)
|
||||
role(store, "1")
|
||||
|
||||
result = run(config, store, embed.run_embed)
|
||||
|
||||
assert result["embedded"] == 0 and result["notices"] == ["embed.unavailable"]
|
||||
assert store.vector_counts() == {}
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ terms
|
||||
|
||||
def test_recency_halves_every_fourteen_days() -> None:
|
||||
assert score.recency(NOW.isoformat(), NOW) == 1.0
|
||||
assert score.recency((NOW - timedelta(days=14)).isoformat(), NOW) == pytest.approx(0.5)
|
||||
assert score.recency((NOW - timedelta(days=28)).isoformat(), NOW) == pytest.approx(0.25)
|
||||
|
||||
|
||||
def test_a_role_without_a_vector_is_scored_without_fit_unless_fit_is_all() -> None:
|
||||
candidate = Candidate(uid="u", first_seen=NOW.isoformat(), affinity=0)
|
||||
value, reason = score.score_one(candidate, [], Weights(), None, NOW)
|
||||
assert reason == "" and value == pytest.approx((0.2 * 1.0 + 0.2 / 3) / 0.4)
|
||||
|
||||
only_fit = Weights(fit=1, recency=0, affinity=0)
|
||||
assert score.score_one(candidate, [], only_fit, None, NOW) == (0.0, "no vector")
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ score
|
||||
|
||||
def test_a_require_gate_keeps_the_row_with_a_reason(
|
||||
store: Store, config: Config, fake: FakeEmbedder) -> None:
|
||||
berlin = role(store, "1", place="Berlin, Germany")
|
||||
munich = role(store, "2", place="Munich")
|
||||
nowhere = role(store, "3")
|
||||
agent = store.save_agent("Berlin", "python backend",
|
||||
[Gate(field="place", values=["Berlin"])])
|
||||
|
||||
run(config, store, score.run_score)
|
||||
|
||||
scores = store.scores(agent.id)
|
||||
assert scores[berlin][1] == "" and scores[berlin][0] > 0
|
||||
assert scores[munich] == (0.0, "place not Berlin")
|
||||
assert scores[nowhere] == (0.0, "place unknown")
|
||||
shown = store.roles(agent=agent.id)
|
||||
assert [r.uid for r in shown] == [berlin]
|
||||
assert {r.uid for r in store.roles(agent=agent.id, gated=True)} == {berlin, munich, nowhere}
|
||||
|
||||
|
||||
def test_a_prefer_gate_adds_its_weight_to_the_score(
|
||||
store: Store, config: Config, fake: FakeEmbedder) -> None:
|
||||
remote = role(store, "1", remote="remote")
|
||||
office = role(store, "2", remote="onsite")
|
||||
agent = store.save_agent("Remote", "", [Gate(field="remote", values=["remote"],
|
||||
mode="prefer", weight=0.5)])
|
||||
|
||||
run(config, store, score.run_score)
|
||||
|
||||
scores = store.scores(agent.id)
|
||||
# No intent, so fit is out: recency 0.2 + affinity 0.2 + the gate's 0.5.
|
||||
assert scores[remote][0] - scores[office][0] == pytest.approx(0.5 / 0.9, abs=1e-5)
|
||||
assert scores[office][1] == ""
|
||||
|
||||
|
||||
def test_a_favourite_skips_the_gates_and_is_floored(
|
||||
store: Store, config: Config, fake: FakeEmbedder) -> None:
|
||||
store.add_company("Fave", affinity=2)
|
||||
fave = role(store, "1", company="Fave", place="Munich")
|
||||
agent = store.save_agent("Berlin", "", [Gate(field="place", values=["Berlin"])])
|
||||
|
||||
run(config, store, score.run_score)
|
||||
|
||||
value, reason = store.scores(agent.id)[fave]
|
||||
assert reason == "" and value >= 0.9
|
||||
|
||||
|
||||
def test_a_muted_company_is_never_scored(
|
||||
store: Store, config: Config, fake: FakeEmbedder) -> None:
|
||||
store.add_company("Muted", affinity=-1)
|
||||
muted = role(store, "1", company="Muted")
|
||||
kept = role(store, "2")
|
||||
agent = store.save_agent("Any")
|
||||
|
||||
run(config, store, score.run_score)
|
||||
|
||||
assert set(store.scores(agent.id)) == {kept}
|
||||
assert muted not in {r.uid for r in store.roles(agent=agent.id, gated=True)}
|
||||
|
||||
|
||||
def test_an_older_role_scores_lower(
|
||||
store: Store, config: Config, clock: Clock, fake: FakeEmbedder) -> None:
|
||||
clock.at = NOW - timedelta(days=14)
|
||||
old = role(store, "1")
|
||||
clock.at = NOW
|
||||
new = role(store, "2")
|
||||
agent = store.save_agent("Any")
|
||||
|
||||
run(config, store, score.run_score)
|
||||
|
||||
scores = store.scores(agent.id)
|
||||
assert scores[new][0] > scores[old][0]
|
||||
|
||||
|
||||
def test_two_agents_score_the_same_role_differently(
|
||||
store: Store, config: Config, fake: FakeEmbedder) -> None:
|
||||
uid = role(store, "1", "Backend Engineer", body="Python services")
|
||||
backend = store.save_agent("Backend", "python backend developer",
|
||||
weights=Weights(fit=1, recency=0, affinity=0))
|
||||
frontend = store.save_agent("Frontend", "react frontend developer",
|
||||
weights=Weights(fit=1, recency=0, affinity=0))
|
||||
|
||||
run(config, store, embed.run_embed)
|
||||
result = run(config, store, score.run_score)
|
||||
|
||||
assert result["agents"] == 2
|
||||
assert store.scores(backend.id)[uid][0] > store.scores(frontend.id)[uid][0]
|
||||
|
||||
|
||||
def test_the_intent_vector_is_cached_until_the_intent_changes(
|
||||
store: Store, config: Config, fake: FakeEmbedder) -> None:
|
||||
role(store, "1")
|
||||
agent = store.save_agent("Backend", "python backend")
|
||||
|
||||
run(config, store, score.run_score)
|
||||
run(config, store, score.run_score)
|
||||
assert fake.calls == [["python backend"]]
|
||||
|
||||
store.save_agent("Backend", "react frontend", agent_id=agent.id)
|
||||
run(config, store, score.run_score)
|
||||
assert fake.calls[-1] == ["react frontend"]
|
||||
|
||||
|
||||
def test_a_disabled_agent_is_not_scored(
|
||||
store: Store, config: Config, fake: FakeEmbedder) -> None:
|
||||
role(store, "1")
|
||||
agent = store.save_agent("Off", enabled=False)
|
||||
assert run(config, store, score.run_score)["agents"] == 0
|
||||
assert store.scores(agent.id) == {}
|
||||
|
||||
|
||||
def test_starter_agents_are_imported_once(store: Store, tmp_path: Path) -> None:
|
||||
config = Config(database=str(tmp_path / "roledesk.db"),
|
||||
agents=[AgentSeed(name="A"), AgentSeed(name="B")])
|
||||
assert score.seed_agents(config, store) == ["A", "B"]
|
||||
for agent in store.agents():
|
||||
store.delete_agent(agent.id)
|
||||
assert score.seed_agents(config, store) == []
|
||||
assert store.agents() == []
|
||||
|
||||
|
||||
def test_the_shipped_starter_agents_validate() -> None:
|
||||
from roledesk import config as config_mod
|
||||
|
||||
names = [a.name for a in config_mod.load(ROOT / "roles.yaml").agents]
|
||||
assert names == ["Berlin backend", "Remote EU stretch"]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- cascade
|
||||
|
||||
def test_the_pull_cascade_runs_extract_embed_then_score(
|
||||
store: Store, config: Config, fake: FakeEmbedder, tmp_path: Path,
|
||||
monkeypatch: pytest.MonkeyPatch) -> None:
|
||||
store.add_subscription("ats", "greenhouse:acme", label="Acme", company="Acme",
|
||||
board="greenhouse")
|
||||
agent = store.save_agent("Backend", "python backend")
|
||||
|
||||
def fake_fetch(self: ATSSource) -> Iterator[Item]:
|
||||
item = Item(source_id="greenhouse:acme:1", location="https://x.example/1",
|
||||
title="Backend Engineer", body="Python",
|
||||
published_at=datetime(2026, 9, 20, tzinfo=UTC),
|
||||
raw={"company": "Acme", "board": "greenhouse", "slug": "acme"})
|
||||
yield item
|
||||
self.state["ats:ok:greenhouse:acme"] = self.state.get("ats:run", "")
|
||||
self.state["ats:ids:greenhouse:acme"] = '["greenhouse:acme:1"]'
|
||||
|
||||
monkeypatch.setattr(ATSSource, "fetch", fake_fetch)
|
||||
queue = JobQueue(Jobs(tmp_path / "roledesk.db"), config, store,
|
||||
workers={"cpu": 1, "http": 1, "llm": 1})
|
||||
|
||||
task_id = queue.start_task(TASKS["pull"]())
|
||||
queue.drain(timeout=60)
|
||||
|
||||
cascade_id = queue.jobs.of_task(task_id)[0].cascade_id
|
||||
jobs = sorted(queue.jobs.of_cascade(cascade_id), key=lambda j: j.id)
|
||||
stages = [j.result.get("task") for j in jobs if j.kind == FINAL]
|
||||
assert stages == list(cascade.CASCADES["pull"])
|
||||
work = [j.kind for j in jobs if j.kind != FINAL]
|
||||
assert work == ["pull_batch", "embed", "score"], "extract is off, so it finds nothing"
|
||||
assert all(j.state == "done" for j in jobs)
|
||||
assert len(store.scores(agent.id)) == 1
|
||||
|
||||
|
||||
def test_follows_walks_the_cascade_and_stops_at_its_end() -> None:
|
||||
assert cascade.follows("pull", "pull") == "extract"
|
||||
assert cascade.follows("pull", "extract") == "embed"
|
||||
assert cascade.follows("pull", "embed") == "score"
|
||||
assert cascade.follows("pull", "score") == ""
|
||||
assert cascade.follows("pull", "nope") == ""
|
||||
@@ -87,11 +87,15 @@ def test_a_v1_file_migrates_to_the_shape_a_new_file_has(tmp_path: Path) -> None:
|
||||
with sqlite3.connect(old) as conn:
|
||||
for column in ("permit", "languages", "extract_tries"):
|
||||
conn.execute(f"ALTER TABLE roles DROP COLUMN {column}")
|
||||
for column in ("intent_key", "intent_vec"):
|
||||
conn.execute(f"ALTER TABLE agents DROP COLUMN {column}")
|
||||
conn.execute("DROP TABLE vectors")
|
||||
conn.execute("PRAGMA user_version = 1")
|
||||
|
||||
assert Store(old).migrations_applied == list(range(2, LATEST + 1))
|
||||
# ALTER appends, so the order differs; every read is by column name.
|
||||
assert sorted(_columns(old, "roles")) == sorted(_columns(fresh, "roles"))
|
||||
for table in ("roles", "agents", "vectors"):
|
||||
assert sorted(_columns(old, table)) == sorted(_columns(fresh, table))
|
||||
|
||||
|
||||
def test_state_is_a_mapping(store: Store) -> None:
|
||||
|
||||
@@ -203,3 +203,49 @@ def test_index_renders_the_shell(client: TestClient) -> None:
|
||||
assert "roledesk" in res.text
|
||||
assert 'id="roles-list"' in res.text
|
||||
assert "<desk-jobs>" in res.text
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ agents
|
||||
|
||||
def test_agents_are_created_updated_listed_and_deleted(client: TestClient) -> None:
|
||||
body = {"name": "Berlin", "intent": "python backend",
|
||||
"gates": [{"field": "place", "values": ["Berlin"], "mode": "require"}]}
|
||||
created = client.post("/api/agents", json=body)
|
||||
assert created.status_code == 200
|
||||
agent = created.json()
|
||||
assert agent["weights"] == {"fit": 0.6, "recency": 0.2, "affinity": 0.2}
|
||||
assert agent["gates"][0]["weight"] == 1.0
|
||||
|
||||
updated = client.post("/api/agents", json={**body, "id": agent["id"], "enabled": False,
|
||||
"weights": {"fit": 1, "recency": 0,
|
||||
"affinity": 0}})
|
||||
assert updated.json()["enabled"] is False and updated.json()["weights"]["fit"] == 1
|
||||
|
||||
assert [a["name"] for a in client.get("/api/agents").json()] == ["Berlin"]
|
||||
assert client.post("/api/agents", json=body).status_code == 409
|
||||
assert client.delete(f"/api/agents/{agent['id']}").status_code == 204
|
||||
assert client.get("/api/agents").json() == []
|
||||
assert client.delete(f"/api/agents/{agent['id']}").status_code == 404
|
||||
|
||||
|
||||
def test_a_gate_on_an_unknown_field_is_refused(client: TestClient) -> None:
|
||||
body = {"name": "Bad", "gates": [{"field": "salary", "values": [1]}]}
|
||||
assert client.post("/api/agents", json=body).status_code == 422
|
||||
|
||||
|
||||
def test_roles_by_agent_are_ordered_by_score_and_hide_the_gated(
|
||||
client: TestClient, store: Store) -> None:
|
||||
for sid in ("low", "high", "gated"):
|
||||
store.upsert_role(posting(sid))
|
||||
uids = {r.source_id: r.uid for r in store.roles()}
|
||||
agent = store.save_agent("A")
|
||||
store.replace_scores(agent.id, [(uids["low"], 0.2, ""), (uids["high"], 0.8, ""),
|
||||
(uids["gated"], 0.0, "place unknown")])
|
||||
|
||||
shown = client.get(f"/api/roles?agent={agent.id}").json()
|
||||
assert [r["source_id"] for r in shown] == ["high", "low"]
|
||||
assert shown[0]["score"] == 0.8
|
||||
everything = client.get(f"/api/roles?agent={agent.id}&gated=1").json()
|
||||
assert everything[-1]["gated_reason"] == "place unknown"
|
||||
assert client.get("/api/roles").json()[0]["score"] is None
|
||||
assert client.get("/api/roles?agent=999").status_code == 404
|
||||
Reference in new issue
Block a user