score is solo, so a request during a run was deduped into the running job,
which had already read the old state. _score_after_change returns a queued
score when one has not started, else enqueues a plain score and, if that
dedupes to a running job, one follow-up with depends_on it, after="done" and
dedup_key score:after:<id>. POST /api/score answers with that job and
agents.js waits on it; the affinity route answers with score_job.
The flaky affinity test was pipeline's race, not the app's: Jobs.enqueue
re-raises the UNIQUE IntegrityError when the live job it collided with
finishes before the SELECT that should hand it back. The app retries.
Co-Authored-By: Claude Opus <noreply@anthropic.com>
_get streams the body with a running byte count, so an oversized board is
refused at MAX_BYTES instead of after it is held whole, and bounds the whole
request, redirects included, by DEADLINE, checked between chunks. The
per-phase timeouts are httpx.Timeout(10, read=30), set from 84 live requests
against mongodb, feedzai and the five Greenhouse boards with most open roles,
with and without pay_transparency=true: the body arrives in under 1 s, the
first byte waited up to 16.6 s, and the flag made no measurable difference.
_paged works unchanged through it (tested over a mock transport, 429 included).
Correct pull.py's BATCH_SIZE comment: a batch reads its boards in sequence,
and the parallelism is the http pool's.
Co-Authored-By: Claude Opus <noreply@anthropic.com>
Every ats parser now writes one human-readable pay line whether or not it
converts to yearly salary_min/salary_max/currency: hourly and daily rates,
Greenhouse's several pay zones, and a range with no currency all survive as
text instead of being dropped outright. Stored in roles.salary_text
(migration 12), replaced whole on every pull like title, so a board that
stops stating pay clears it rather than leaving a stale line. salary_state
is derived, not stored ('normalised', 'stated', or ''), and is what the
feed's new Pay filter and badge read; gates and scoring are unchanged, still
reading only the normalised columns.
Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
country_of folds with _place_fold (ł, đ, ø, æ map to ASCII; _fold keeps building
company_key unchanged), knows every ISO country by English name and European
ones by their own, and the cities among the 150 most frequent unresolved
places on the live data. Within one location a country beats a US state or
Canadian province, which beats a city. A state code that is also an ISO code
counts as the state only when the line names a US place. Several countries,
all European, are EU; otherwise ''. A code in parentheses is read when no
name matched.
Migration 12 recomputes country from place for roles extract has not read
and whose board sent no country in raw. On the live data: open roles with no
country 2,827 -> 1,183 of 16,841; 1,705 open roles change, 48 from a code.
Co-Authored-By: Claude Opus <noreply@anthropic.com>
scale() maps a cosine to clip((cos - lo) / (hi - lo), 0, 1) with constants
from CALIBRATION in app/config.py, keyed by model and replaceable under
embed.calibration. multilingual-e5-small: fit 0.75..0.88, taste -0.14..0.19
(taste from synthetic votes). An uncalibrated model falls back to min-max
and logs a warning once per run.
The extract and summarise bars default to 0.55 and 0.65, about 300 and 35
roles for the starter agent on the measured data; stored limits are kept.
An agent without an intent is left out of the model-stage scope, and each
score run names such agents in one score.no_intent notice.
Co-Authored-By: Claude Opus <noreply@anthropic.com>
MOVED and _moved answered 301 for twelve paths the settings cog moved
off of a few days ago. A browser caches a 301 indefinitely, so a path
reused later would be unreachable through no fault of that page, and
a one-person app has no outstanding bookmark to protect. Drop the
table, the routes, its export from app.web, and the tests that pinned
it; the old paths now answer plain 404. Keep the test that every
section of the Sources page exists, rewritten without MOVED.
Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
Part A: Ashby (?includeCompensation=true), Greenhouse (pay_transparency=true,
already free on the list request), Lever (salaryRange), Recruitee (salary)
and Personio (salaryInformation) all give structured pay; Workable and
SmartRecruiters do not (checked live). Every host converts to a yearly
salary_min/salary_max/currency, multiplying a monthly figure by 12 and
skipping an hourly or daily one rather than annualising a number nobody
stated. A range with no currency is ignored, and Greenhouse's own multi-zone
ranges are skipped outright since nothing says which zone the role is at.
Part B: classify() matched substrings, so " it" matched inside "Italy",
filing European sales roles as engineering; it now matches on word
boundaries (full boundary for the handful of short tokens that are
themselves the start of an unrelated word, prefix matching kept for the
stems). Title is checked before department (a hand-checked sample found
title-first right about twice as often), nothing matching returns '' rather
than "other", and a modest set of German/Dutch/Danish/Norwegian/Swedish/
French keywords covers the seed's other languages.
function is not a board field, so it gets the opposite rule from every
other raw column: extract's answer overwrites the classifier's guess
outright, and once extracted_at is set no later pull may put the guess
back. Migration 9 recomputes function for every not-yet-extracted role from
its stored title and department.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
recency and the newest feed both counted from first_seen, when the app
itself first stored a role, so a newly added board's whole backlog scored
as freshly posted for two weeks: 10,223 of 15,415 board roles were posted
more than 30 days before they were first seen (average gap in days: Ashby
78, Greenhouse 143, Lever 226, Personio 333).
Add roles.posted_at (migration 9): the earlier of first_seen and a trusted
host's published_at, where TRUSTED_POSTED_HOSTS (app/models.py) records
which of the ats and search hosts report a creation date rather than one
that moves on an edit, checked against live boards and the live database.
A future or pre-2015 date is not trusted either. upsert_role sets it on
insert and keeps the earliest on update, so an edit on the board can only
move it earlier, never later. recency, the feed's newest order, the "new
since" days filter and _top_scope's tie-break now read posted_at;
first_seen is unchanged and still what the UI shows as when the app found
a role. The row's age in roles.js now shows the posted date instead.
A partial index (posted_at DESC, uid) WHERE closed_at IS NULL serves the
newest feed; sqlite3's bundled build has no STAT4, so its planner
otherwise sorts every open role instead of using it, measured 30x slower
on the live database, so Store.roles() forces it for that query shape.
Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
A second pull (schedule, Run button, `app job pull` in another process) could
start while one was fetching and overwrite the stored run marker, so the first
run's boards failed their stamp check or a settle closed on the other run's id
lists. `claim_run` sets `pull:run` in one conditional statement, only while it
equals `pull:settled` or is older than an hour; a refused `pull` job queues no
stage and says so in its result and a notice, and the CLI exits 1. Batches
carry the marker in their payload, the plugins stamp with it, and the settle
reads it off its own batches' payloads before releasing the run.
Co-Authored-By: Claude Opus <noreply@anthropic.com>
`app.jobs.sources.set_board` attaches a board to an existing company,
replaces the one it has (closing the old board's open roles) or creates
the company, with row and subscription changed in one transaction by
`Store.set_company_board`. A board that lists zero roles is kept. `add`,
`discover --add`, the seed importer (moved out of app/cli.py) and the new
`POST|DELETE /api/companies/{key}/board` all go through it; a seed re-run
attaches a board to a row that never had one. A company's subscription is
rewritten in place, and detaching hides it.
`companies.ats_slug` (migration 9) keeps the slug of a host the app
cannot read yet, and discovery returns such a host before probing slugs.
The Companies rows get Set board / Change, the host select is rendered
from `plugins.ats.HOSTS`, and a row with nothing found reads "board not
found" and is in the "Not readable yet" filter.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Workable and SmartRecruiters page their lists, and both parsers read page one
only, so the settle closed every role past it (Delivery Hero: 980 stated, 100
open). `_paged` follows nextPage and offset, and the plugin stamps
`ats:complete` only when the distinct ids read equal the stated total; a 429
after page one keeps what was read and closes nothing. Adzuna completes on its
`count`, Jobindex counts raw items and raises on an answer with no channel.
A board whose fetch raises is recorded under `ats:failing:<target>` with the
first failure's time and the latest reason; after STALE_DAYS the settle closes
its roles, and its next answer reopens them. `/api/companies`, the Companies
list and `app companies check` show it.
Co-Authored-By: Claude Opus <noreply@anthropic.com>
Measured on the live roles against the "Copenhagen engineering" agent's
intent: the English-only all-MiniLM-L6-v2 separates engineering from
non-engineering roles with an AUC of 0.68 overall but only 0.74 restricted
to Danish-country roles (a Danish engineering role beats the rest's
non-engineering median only 68% of the time). multilingual-e5-small reaches
0.70 overall and 1.00 on Danish-country roles, for about 3x the CPU time.
paraphrase-multilingual-MiniLM-L12-v2 measured worse than the English-only
baseline (AUC 0.60) and was dropped.
e5 needs "query: "/"passage: " prefixes; app/jobs/embed.py adds them by
matching the model name so a model change can't leave a stale prefix
setting behind. Added tests for the prefixing and for the two invariants
the model change relies on: embed drops the old model's vectors before
re-embedding, and an agent's cached intent vector is keyed by model so a
model change is a cache miss, never a stale-dimension read.
Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
A require gate on an empty column gated the role ("country unknown"), and extract, the stage that fills the column, is bounded to the roles that rank, so 2,281 of 15,556 open roles could never come back: every board that writes "Remote", "Hybrid" or "Distributed" as its location. An empty column now neither passes nor fails a gate; the role ranks, extract reads it, the rescore decides.
`any` is a third gate mode: the any gates are one group a role must match one of, so a ranking can say "in Denmark, or remote", which require alone could not. The group is gated only when some member had evidence against it.
`remote_of(place)` fills `remote` from the location line the way `country_of` fills `country`, and the startup backfill does both, since Greenhouse writes "Hybrid" as the location and gives no flag.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Run one stage goes: the cascade is the unit of work and app job keeps the
per-stage entry points. The pull button is disabled while a pull runs and
the lede no longer hardcodes the cadence. Messages folds into Runs as
Problems. pull_batch checks for a cancel before every board, one plugin per
target, and both plugins now close their HTTP client.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
agents.limits holds extract_top, extract_min, summarise_top and
summarise_min; the scope query reads them per ranked row, and a row saved
before this behaves like a fresh agent. The four global config keys go.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`app companies prune`, `/api/prune` and the Sources disclosure switched off every board with no open role in a region. It existed because the first seed was 186 US boards; the European seed makes it moot, and it was wrong on its own terms: a board with no European role today (a company between openings, or a fetch that timed out) was switched off for good. A company that hires nowhere near should not be in the list, and a ranking's country gate handles the rest. `EUROPE` in models had no other reader and goes with it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The seed was jobhunt-hq's 186 US boards, chosen because they could be read. It is now a hand-picked list of European tech employers, with `board` filled by the new `app.jobs.discover` from each company's own careers site and every board fetched once before it landed: 175 have one, and the rest keep the ATS host their site runs on (Teamtailor 32, Workday 12, SuccessFactors 9, ...) and a careers URL, as rows the app cannot read yet. Companies shows them as "not readable yet" with a link, behind a filter, so the next parser to write is the host that filter shows most.
`app companies discover NAME DOMAIN [--add]` reads the home page, its careers links and the usual paths for a link into a known host, one level down on a careers page, and probes Greenhouse, Lever, Ashby, SmartRecruiters, Recruitee and Personio by slug when the pages link nothing. It is what the LinkedIn flow will call.
Migration 7 adds companies.ats, country and city. The Workable parser now reads the v3 shape its list endpoint actually sends (location object, department list, shortcode and no URL or text); the first populated board showed it.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The free stages run first and their score bounds the two model stages.
Extract reads only the best extract.per_agent roles of each agent at
extract.min_score or more, then score runs again so the gates read what it
filled; summarise gets a min_score of its own. The cascade steps by
position, since score is in it twice.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A search agent is a persona, and every like or pass belongs to the one
being viewed. The rail now opens on Viewing as: a full-width chooser,
the name alone with one agent, a note with none, remembered in the URL
and localStorage. The vote buttons, the reader's affinity control, the
feed, the tag counts and the reader fetch all carry that agent; before,
votes sent none and every one landed on the default agent. Best first
and Newest first are back as the segmented control under Order,
independent of the agent. The rail selects no longer clip descenders:
Chrome ignores line-height on a select, so the fixed box is gone in
favour of padding and a min-height.
Settings render inside a sticky rail (Search agents; Sources with Job
boards, Company boards, Runs, Messages; Account), a strip under the
header on a phone, so Runs is one click away however long the company
list gets. Sources shows one section at a time, chosen by the fragment,
so the old redirects still land. Search agents are cards, newest first:
name, an On/Off switch with its word, the intent in full, the weights,
Edit in place, New agent on top, and a link to Roles as that agent.
Switching one on rescores it.
replace_scores writes nothing for an agent deleted while score was
running, instead of failing the run on the foreign key.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The per-agent migration becomes v6 behind main's v5. A vote still stamps
the role's read_at, tag counts hide the agent's hidden companies, and
/api/filters takes the agent like /api/roles.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Votes on roles and company affinity move to role_votes and agent_companies,
one row per search agent, so two agents rank the same roles differently. The
score gains taste, the cosine to mean(liked) - mean(passed) of the agent's
votes, as newsdesk ranks articles. The API takes agent on roles, votes and
companies and falls back to the default agent, the enabled one with the
lowest id.
Settings become Search agents (a list again, with New and Delete), Sources
(job boards, company boards and runs with the message log, one section per
plugin) and Account. Old paths answer 301 to their section.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The Roles tab takes newsdesk's shell, ported from its index.html and
app.css rather than rewritten: a filter rail (Show read, Best first or
Newest first, first seen, tag chips with counts, the old filters behind a
disclosure), the feed in newsdesk's row layout (score gutter, title,
company, place, badges, age, tags, the summary as excerpt, judge buttons)
and the reader as a third column that replaces the list on a phone. The
router pushes /roles/{uid}, which the server also renders with the reader
open, so the separate read page and role.js go. j/k, u/n/d, S, Enter,
Escape, o, /, f and c work as in newsdesk. The phone rules come with their
comments.
Two things the row lacked: roles.read_at, stamped when the reader opens a
role or a vote is cast, behind include_read on /api/roles; and role_tags,
written by summarise alongside the summary and served with counts by
/api/filters, filtered in SQL before LIMIT. Migration 5.
PyCharm run configurations in .run/ for serve, pull, seed and pytest.
core.autocrlf=input is set locally so a Windows-side checkout commits LF.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
README's tabs table now matches two tabs and the settings cog, and
mentions the read page, votes, and the Fetch/Summarise buttons.
docs/decisions.md gets a new "The phone" section recording each
decision the mobile plan made, confirmed against the merged code.
docs/mobile-plan.md is deleted per its own package C.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two tabs and a cog, settings sub-tabs, news's feed row and reader, votes,
paging, on-demand posting text and a bounded summarise stage, Companies and
boards merged, forms on top, plain ledes. Work packages for a fan-out.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
What a pull reads can now be changed on a page instead of only the command
line: try a search and see its first ten results before saving it, edit or
remove one, follow a company's ATS board, switch any board or search on or
off, and preview and apply a prune by region. Each row shows its open roles
and the countries they are in.
The page and `app companies`/`app searches` call the same functions in
app/jobs/sources.py. Switching a target off closes its open roles, and the
pull's settle closes them again if a batch in flight reopened them. Roles
pulled before `country` existed are backfilled at startup.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ats boards only read companies someone listed, and the seed list is
US-centred: 31 of 29,000 open roles were in Denmark. A `search` plugin runs
saved queries on boards that span companies, pulled in the same task as the
boards: Jobindex (Denmark, RSS, no key) and Adzuna (UK and most of the EU,
free key, built to its documented API). Jobnet and Jooble were checked and
left out.
`country` is now filled from the board's location line by a lookup, with EU
for Europe-wide roles, so country gates work without extract. `app companies
prune --in` unsubscribes boards with no open role in a region. The starter
agents and searches are Copenhagen and Europe.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One entry replaces "own repo, depending on news" and "desk and pipeline
are never edited here": built on the boilerplate with desk copied from
news, all four downstream copies, fixes upstream first, package app. The
superseded 8 MB cap entry is gone and the 96 MB one is unwrapped. Long
entries are cut to the decision and one reason, and the stale names
(roles.yaml, the one alpha page, app.js) follow the restructure. New: pull
as a job on a 6h schedule, and the coupling rule's place in the layers.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With fit as (cos + 1) / 2 the real run spanned about 0.55 to 0.79, so a
prefer gate at weight 0.2 moved a score more than the whole spread of fit.
fit is now the cosine min-max scaled across the roles an agent scores in
that run: the closest is 1, the farthest 0, a single role 1, and a role a
require gate rules out stays out of the range. The starter agents' prefer
weights drop to 0.05, enough to break a tie. A regression test pins the
case: two close cosines that the old mapping let a 0.05 gate reorder.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
roledesk is built on the boilerplate from here on: pipeline 2.0 (queue and
scheduler), auth (one password, sessions, passkeys), the deploy kit and its
docs, plus the coupling and packaging tests. Downstream copies, unedited.
Refreshed from boilerplate/d56949f7d0f6c0bfe5745478e09007bfad67ae87
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
embed stores one vector per open role; score rates every open role for
every enabled agent from require and prefer gates, fit against the
agent's intent, recency and company affinity, keeping gated rows with a
reason. Schema v3 adds vectors and the cached intent vector. The page
gets an agent order, the score on each row, and an Agents tab. Two
starter agents in roles.yaml are imported once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
roledesk job extract reads open, unanswered roles 20 to a call and fills
place, country, remote, seniority, salary, currency, function, permit and
languages, only into columns the board left empty. Schema v2 adds
roles.permit, roles.languages and roles.extract_tries. Off by default.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/, GET /api/roles|companies|applications, POST /api/companies/{key}/affinity
and /api/applications[/{id}/status]. The Jinja2 template renders an empty
shell; static/app.js fills it and drives Save, affinity and status changes.
<desk-jobs> from pipeline.web mounts at the bottom so Pull can be started
from the page.
Store.roles gains q, function, remote, days and hide_muted, and company/
place move from a key match to a substring on the text a board wrote.
hide_muted filters in SQL before LIMIT: filtering after it shipped first and
broke immediately on real data, since Databricks' freshly pulled 884 roles
were the newest 200-plus rows in the table and muting it left nothing to
show.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
openings.go's 8 MB cap was written against arbitrary URLs; every URL here is
on a fixed allow-list of seven ATS hosts, and the real pull of 2026-09-23
refused Anduril, Anthropic, OpenAI, Databricks, Carvana, Axon and Airwallex
for it. A board that still exceeds 96 MB now retries once without
content=true (Greenhouse only), trading the description for the listing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The real run against the seeded 186 companies refused seven of the
largest boards outright, Anduril worst at 41 MB, because the cap and
requesting full descriptions pull against each other on exactly the
employers that post the most roles. Left as is for now; the fix is a
retry without content=true, tracked as an open decision.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
seed/companies.json: 179 companies from jobhunt-hq's seed.json (MIT),
reduced to key/name/board/careers_url, plus a hand-added DACH set.
`roledesk companies seed|add|check` import it, add one company from its
board with a live check first, and report dead boards.
roledesk/jobs/pull.py is now a pipeline task: ten boards per pull_batch job
on the http resource, upserting every role the ats plugin yields. Its final
job's settle closes every board this run answered in full and no longer
lists a role from, reading the plugin's own self.state stamps rather than
each batch job's return value, so one bad board in a batch does not cost the
other nine their closure. `roledesk job pull` runs it, and it registers
itself in pipeline.TASKS so <desk-jobs> can start it too.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
roledesk/storage/schema.py creates the tables at v1 with WAL and the
versioned-migration machinery from newsdesk; store.py adds upsert_role
(unpacks raw into gate columns, first_seen on insert, last_seen every
time), roles(), mark_closed, companies and subscriptions CRUD, and
save_application writing an append-only application_events row on every
status entered. Key helpers company_key, title_key, place_key live in
models.py. docs/decisions.md records the key decisions.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>