Files
jobboard/docs/decisions.md
T
WolfandClaude Opus b021838959 Calibrate fit and taste per embedding model instead of min-max per run
scale() maps a cosine to clip((cos - lo) / (hi - lo), 0, 1) with constants
from CALIBRATION in app/config.py, keyed by model and replaceable under
embed.calibration. multilingual-e5-small: fit 0.75..0.88, taste -0.14..0.19
(taste from synthetic votes). An uncalibrated model falls back to min-max
and logs a warning once per run.

The extract and summarise bars default to 0.55 and 0.65, about 300 and 35
roles for the starter agent on the measured data; stored limits are kept.
An agent without an intent is left out of the model-stage scope, and each
score run names such agents in one score.no_intent notice.

Co-Authored-By: Claude Opus <noreply@anthropic.com>
2026-09-29 14:59:34 +02:00

40 KiB

Key decisions

What was decided for roledesk and the one reason for each. The longer reasoning is in ../news/docs/JobBoardPlan/roles-build-plan.md and jobhunt-hq-assessment.md.

Shape

Built on the boilerplate, with desk copied from news. pipeline/ (queue and scheduler, pipeline 2.0), auth/ and deploy/ come from ../boilerplate and desk/ from ../news, all four as downstream copies refreshed with git archive. A fix lands upstream first, or the next refresh reverts it. The package is app, per the boilerplate rule, since the deploy kit and the APP_* variables assume it.

The domain noun is role. jobs/ keeps meaning the work the app queues, so "job" never has two senses in one sentence.

Own database, queue and port. roledesk.db, config.yaml, port 8790. A shared file with newsdesk would buy "articles about this company" by cosine, which is worth less than two apps that can migrate independently.

Storage is written, not imported. desk has no storage package on purpose, so ~120 lines of connection plumbing are copied from newsdesk. If the same edit lands in both a second time, a shared Connection moves into desk below pipeline.

Layers are models, config, storage, jobs, web, main, cli. tests/test_architecture.py holds the direction and the doors within app; tests/test_coupling.py holds that pipeline, auth and desk import nothing from app or each other, except desk from pipeline, and that a plugin imports desk only.

Data

Clustering key is (company_key, title_key, place_key). newsdesk's location_key OR title_key collapses two outlets running one story, which is right for wire copy and wrong for ads, where the same title at two companies is two jobs.

place, country and remote are separate columns. location already means where an Item's original lives. remote alone over-matches, because boards write "UK | Remote", so a gate on remote also reads country.

raw is transport, columns are storage. No Item.facets. A plugin puts structured fields in raw under the column's name and upsert_role unpacks them into indexed columns, which is where gates query them. An empty value never overwrites a filled column, so a terse board does not erase what extract found.

extract fills, the board wins. set_extracted writes a model's value only into an empty column. A board's structured field is first-party, and upsert_role would put it back on the next pull anyway, so letting the model replace it would make a column flip between two answers. place brings place_key with it, which only ever moves a role from the empty place key to a real one.

function is the one column where the model wins, because the board never filled it. classify(department, title) in plugins/ats.py is ats's own keyword guess, not a board field, so "the board wins" does not apply to it: set_extracted writes a non-empty function over the guess outright, rather than only into an empty column, and upsert_role then leaves function alone for any role with extracted_at set, whatever the classifier says on the next pull. A role not yet extracted still gets a fresh guess every pull, the same as any other raw field. One rule, so function cannot flip between the classifier's answer and the model's on alternating pulls, which is exactly what "the board wins" above was written to prevent for a real board field.

classify matches whole words, not substrings, and checks the title before the department. The ported version matched substrings, so " it" (meant to catch an "IT" department) matched inside "Italy" and "Italia", filing European sales roles ("Commercial Italy") as engineering; it now requires a word boundary, doubled on both sides for the handful of short tokens ("it", "seo", "qa", "ux", ...) that are themselves the start of an unrelated word ("Seoul", "Qatar"), while the rest keep a leading boundary only, since most are deliberate stems ("manag", "engineer", "recruit") meant to also match "management", "engineering", "recruiting". A department is often a generic umbrella ("Operations", "Engineering & Product") that says less than the title does, so the title is checked first and the department is the fallback; a hand-checked sample of 60 roles where the two orders disagreed found title-first right about twice as often. Nothing matching returns '', not "other", which claimed a knowledge the classifier does not have. A modest set of German, Dutch, Danish, Norwegian, Swedish and French keywords (developer/engineer, sales) covers the seed's other languages; it is deliberately not complete, since extract is the rest.

Structured pay is unpacked the same as any other board field, except that Greenhouse and Ashby need an extra query parameter to get it. Ashby sends it with ?includeCompensation=true, Lever's salaryRange and Recruitee's salary are already on the plain list response, and Greenhouse needs pay_transparency=true on the same request it already makes (no per-role fetch); Personio's XML rarely carries a salaryInformation block, but reads the same way when it does. Workable and SmartRecruiters give neither, checked live against their real boards (2026-09-29). Every host converts to yearly salary_min/salary_max/currency: a monthly figure is multiplied by 12, an hourly or a daily one is left out rather than annualised into a number nobody stated, and a range with no currency is ignored. Greenhouse can give several ranges for one role (one per office or pay zone); with more than one there is no way to tell which office the role in hand is at, so all of them are skipped rather than one being picked at random.

extract reads the board's location too. The model gets title, the board's place text, and the first 3000 characters of the description. Greenhouse's location line ("Berlin, Germany") is often the only place a country is stated, and it is part of the posting, so the "only what the posting states" contract still holds.

country is filled from place by a lookup, before extract. country_of maps the folded location line to an ISO code from a table of European countries and cities, plus the non-European places that fill the seed boards, and returns nothing when a line names two countries. It resolves about 90% of the seed boards' roles for free, where extract would be a paid call per role for the same answer. EU, ISO's reserved code, stands for "Remote - Europe" and "EMEA", so a Europe gate is a list of codes that includes it. A US state after a comma counts only when no name matched, because DE and IN are countries too.

permit and languages are columns, and so is extract_tries. permit answers the plan's open item 1 (visa sponsorship as a gate field) while nothing depends on it yet. newsdesk keeps give-up counts in raw; here upsert_role rewrites raw on every pull, which would reset the count, so the counter is its own column.

Postings are persisted. Storing every posting is what gives first_seen, when the app itself first stored the role, and closed_at, set by mark_closed when a complete fetch of a board no longer lists a role. A role listed again is open again.

posted_at (migration 10) is the earlier of first_seen and a trusted host's published_at; first_seen is left alone, as when the app found the role. first_seen being the only honest posted date was true of Greenhouse's updated_at, which resets on every edit; plugins.ats already reads first_published instead, and that plus the other hosts' dates — Lever's createdAt, Ashby's publishedAt, SmartRecruiters' releasedDate, Recruitee's published_at/created_at, Personio's createdAt, Workable's published, and the two search providers' (Jobindex's RSS pubDate, Adzuna's created) — are creation dates, checked against live boards and the live database (app.models.TRUSTED_POSTED_HOSTS, with the evidence in its comment). A date in the future or before 2015 is not trusted either, host or no host. Without this, a newly added board's whole backlog scored as freshly posted for two weeks: on the live database, 10,223 of 15,415 board roles were posted more than 30 days before they were first seen (average gap in days: Ashby 78, Greenhouse 143, Lever 226, Personio 333), and open roles with recency (app.jobs.score.recency) above 0.5 fell from all 15,558 of them to 3,089 (19.9%) once it read posted_at instead. upsert_role keeps the earliest posted_at a role has ever had: an edit on the board can only move its own date later, never earlier, so it can only move posted_at earlier too, when the board finally reports the true one.

The tracker schema is taken from jobhunt-hq (MIT). applications with status in saved, applied, screen, technical, onsite, offer, rejected, withdrawn, next_action, next_date, and an append-only application_events written by the store on every status entered. The events table is what "days to first screen" and conversion by source are computed from.

subscriptions names its labels company and board. newsdesk's outlet and section are news words; roledesk has no rows to migrate, so it uses its own names from the first migration.

Ingestion

Closure is decided from self.state, not from what a pull_batch job returns. The ats plugin stamps each board it parses without raising with the run's marker, the ids it saw and whether they are the whole board, and the pull's settle closes a board only when its stamp matches the marker its own batches carried and the read was complete. So a batch that raises part way still closes the boards it finished, and a board no batch reached stays open instead of being closed on an empty id list.

One pull runs at a time, claimed in the database. The schedule, the Run button and app job pull, which drains a queue of its own in another process, could each start a pull while one was fetching; the second overwrote the stored run marker, so the first run's boards failed their stamp check and stayed open, or a settle closed on the other run's id lists, and with extract on two cascades could send the same roles to the model twice. claim_run sets pull:run to a new marker in one conditional statement, only while it equals pull:settled or is older than an hour, so of two starters in two processes one wins; the settle writes pull:settled when it ends. A refused pull job queues no stage and says "a pull is already running" in its result and a notice, and the CLI prints the same and exits 1. The marker travels in every batch's payload, the plugins stamp with the marker they were handed, and the settle reads its marker off its own batches' payloads, which a batch that raised still has. The hour is the escape for a run that died: a killed process or a killed cascade never settles, and blocking every pull until someone clears a key would be worse than one duplicated fetch after an hour. The guard lives in app because pipeline's queue dedups a job against a live job of its kind, not a cascade against a live cascade, and solo=True on the pull job would not help, since that job ends in milliseconds once it has asked for the chain.

Boards follow companies; searches find them. ats only reads companies someone listed, so it cannot surface a role at a company nobody knew of, and the seed list came from a US-centred project: of 29,000 open roles, 31 were in Denmark. The search plugin runs saved queries on boards that span companies, as search subscriptions whose params are the query, and pulls them in the same task as the boards.

The seed is a list of companies, not of boards. The first seed was jobhunt-hq's 186 US boards, chosen because they could be read, and 31 of their 29,000 roles were in Denmark. The second is 362 European employers picked by hand, with board filled by app.jobs.discover from each company's own careers site and checked once by fetching it; 175 have one. The rest stay in the list with the ats host their site runs on, so that "cannot read yet" is a filter on the Companies section rather than a company that was never written down, and the next parser to write is whichever host that filter shows most. Teamtailor is first, on this list. A re-run attaches the board an entry names to a row that never had one, so a seed update that gains boards reaches an existing database; it never replaces a board, including a detached one, and never touches affinity, notes or status. Its boards go through set_board without the fetch, since they were read when the seed was written and app companies check finds the ones that died since.

discover reads the site before it guesses. A careers page that links its board is the authority, so the home page, its own careers links, then the usual paths are read for a link into a known host, and a page's own "open positions" link one level down. Only when no page names a host are Greenhouse, Lever, Ashby, SmartRecruiters, Recruitee and Personio probed with the company's slug, because a board drawn by a script leaves no link. A probe hit on a common name can be another tenant (recruitee:google exists, and is not Google), which is why the seed's boards were fetched and read before they landed, and why discover --add reports the roles it found. For the same reason a page that names only an unreadable host ends discovery with that host: probing anyway, as the first version did, could replace a known Workday site with a stranger's Greenhouse board.

A company row carries ats, ats_slug, country and city. ats is what discovery found whether or not the app reads it; a company with an ats and no board subscription is one it cannot read yet. ats_slug is the name that host knows the company by (the acme of acme.teamtailor.com), so the parser written later has its targets without a second discovery run; with a board it is the board's slug. country and city come from the seed for now and are the first of the company facts the feed will filter by; funding, size and industry come from a source that has them, not from a column filled by hand.

Jobindex and Adzuna, not Jobnet or Jooble. Jobindex, Denmark's largest board, has an RSS twin for every search page, with phrase and required-word syntax and a region id; no key. Adzuna has a documented API with a free key for the UK and most of the EU, but no Denmark. Jobnet's search endpoint answers 401 outside its own site's session, and Jooble gives a key only on a manual application and answered a Cloudflare challenge. LinkedIn and Indeed have no public search API.

A search closes a role only when it listed everything. A search result is not a complete list, so a role missing from one run has not closed. An Adzuna search listed everything when it read as many distinct results as the count it states; a Jobindex search, whose feed states no count, when a page within its page limit came back with fewer than 20 items, counted before the parser drops any, so one malformed item does not pass for the end. Such a search closes like a board; the rest close once no search has listed them for 21 days. A Jobindex answer with no RSS channel raises instead of reading as an empty, complete search. A role two searches found belongs to the last one that listed it, so one search never closes a role the other still lists.

Jobindex ads are read from the feed, not followed to the employer. Each item links to Jobindex's own page, and its /c?t= redirect leads to the employer's site or ATS. Following it per role would count as a click in Jobindex's statistics and costs a request per role, so discovering a company's ATS board from a search result is left for later.

Switching a board or search off closes its open roles. Only a pull that reads a target closes its roles, so a target no longer read would keep them open for good. Switched back on, the next pull reopens what it still lists. A batch already in flight when the switch happens reopens them again, so the pull's settle closes every switched-off target once more.

The Sources tab and the CLI share app.jobs.sources. Adding a board or a search and switching one are one function each, called by app companies, app searches and /api/sources*, so the two cannot disagree about what "off" does. A search or board is run once before it is saved, and refused if it does not answer. A search that finds nothing is refused too, since that is usually a typo in the query; a board that lists nothing is kept, since a company between openings is still one to follow.

Every way a company gets a board is app.jobs.sources.set_board. app companies add|set-board|discover --add|seed, /api/boards and /api/companies/{key}/board all call it, so attaching a board to an existing row, replacing one and creating a company are one code path; the seed importer moved out of app/cli.py for this. The company row and its subscription change in one transaction (Store.set_company_board), so a board another company already has leaves no company row behind. Its host and slug become the row's ats and ats_slug; other facts only fill empty columns, so a person's or the seed's values survive.

A company has at most one ats subscription, rewritten in place. Replacing a board changes the row's target and closes the old board's open roles, as switching a board off does; detaching switches it off and hides it (show 0), which is how a removed search is kept. Deleting the row instead would leave a batch already in flight writing roles against a subscription id that no longer exists, and adding a second row would clash on the label, which is the company's name. A detached board is left out of the Companies and Sources lists, and the next set_board reuses the row.

Discovery that finds nothing readable does not detach a board. discover --add on a company that is read from a board and whose site now names only, say, Workday changes nothing and says so. A board that answers is stronger evidence than a page that names no board; set-board or Detach is the explicit step.

SmartRecruiters' board list has no description. Its /postings endpoint gives title, location and department; the job text is only on the per-posting endpoint, and one extra request per role would multiply a board's request count by its role count. description stays empty for this host.

The body cap is 96 MB, with one retry for Greenhouse. openings.go's 8 MB cap was written against arbitrary URLs, while ats only fetches a fixed allow-list of seven ATS hosts, and the real run found ?content=true boards between 9 and 41 MB. A board that still exceeds 96 MB retries once without ?content=true, trading the description for the listing.

Workable's list endpoint has no URL and no text. The parser was first written to Workable's published job object, since every slug tried in the build answered an empty list; the first populated board (Hugging Face, 2026-09-28) showed the v3 shape instead: location an object, department a list, a shortcode and no URL, and no description. The job page is apply.workable.com/{slug}/j/{shortcode}/, and description stays empty as for SmartRecruiters rather than costing one request per role.

Workable and SmartRecruiters are read to the end, and a board closes roles only when it read the total it states. Workable answers 10 roles a page with a nextPage token and SmartRecruiters at most 100 a page by offset, and both parsers read page one only, so the settle took page one for the whole board and closed what fell off it: Delivery Hero stated 980 roles and 100 were open. _paged follows the pages 0.5 s apart, up to 100 pages, and the plugin stamps ats:complete only when the distinct ids read equal total or totalFound. The count is taken on the raw results, so a row the parser drops does not block closure, and on distinct ids, so a role repeated by a shifted offset does not fake completeness. A 429 after page one ends the read: the roles read are stored and nothing closes, because a partial list is no evidence that the rest closed. A 429 on page one is a failure like any other. The other five hosts answer with the whole board and are complete whenever they parse.

A board that fails on every fetch for 21 days closes its roles. A board whose fetch raised was only logged, so a company that moved ATS kept its roles open for good behind a 404. The ats plugin now records ats:failing:<target> in state with the time of the first failure in a row and the latest reason, and a fetch that answers removes it. Once the first failure is STALE_DAYS old, the same window a search role gets, the settle closes the board's roles; the next answer reopens what it lists through upsert_role, as for a board switched back on. The record is plugin state rather than a column because the plugin only sees self.state. /api/companies returns it as failing_since and failing_reason, the Companies list shows "not answering since", and app companies check prints it beside its live result.

pull is a job that starts the cascade, on a default schedule of every 6h. A schedule enqueues a job kind and a cascade is started by one, so the pull job only asks for the chain; declared beside it in app/jobs/cascade.py, the schedule is a starting point that an edit under Runs overrides for good. pull is not also a task, which would draw two buttons under Runs.

Ranking

Scoring is per search agent. A role is scored as (uid, agent) -> score, because a role is only good for a given search. Each gate has a mode: require becomes a SQL WHERE, prefer becomes a weighted term in the score. Gated roles keep their row with a reason. companies.affinity = 2 skips the gates and floors the score, -1 mutes.

fit is the cosine between a role and the agent's intent, mapped to a level by constants calibrated per embedding model: clip((cos - lo) / (hi - lo), 0, 1). (cos + 1) / 2 spanned only about 0.87 to 0.95 on multilingual-e5-small, so a prefer gate at 0.2 outweighed all of fit. Min-max across the roles an agent scored in the run fixed the spread but made fit a rank: the best role of a poor set got 1, a role's fit and the 0.9 favourite floor's meaning drifted from run to run, and the limits bars that bound the paid stages compared an absolute number with a relative one. The constants are CALIBRATION in app/config.py, keyed by model name, and an entry under embed.calibration replaces a model's built-in one; keyed by model, a change of embed.model cannot carry another geometry's constants along. A model with none falls back to min-max per run, with gated roles left out of the range, and the log warns once per run. taste is mapped the same way with its own pair. The starter agents' prefer weights are 0.05.

multilingual-e5-small's constants are fit 0.75 to 0.88 and taste -0.14 to 0.19, measured on a copy of the live database. Over 16,841 open roles and five different intents the raw fit cosine pooled to p1 0.7528, p50 0.7989, p99 0.8528, p99.5 0.8595, p99.9 0.8711 and max 0.9031; per intent p1 ranged 0.7456 to 0.7700 and p99.9 0.8413 to 0.8773. hi is above every intent's p99.9, so about 5 of the "Copenhagen engineering" agent's 4,156 ungated roles clip to 1; at 0.86 (p99.5) 75 would, and lose their order where it matters most. The span 0.13 is close to the 0.145 min-max span it replaces, so a prefer weight keeps about its old size against fit. The copy had no votes, so taste comes from synthetic ones, 30 liked and 30 passed roles for each of three intents (pooled p1 -0.1404, p50 0.0429, p99.9 0.1924), and should be re-measured once real votes exist.

The default bars are 0.55 for extract and 0.65 for summarise. On the same data, the Copenhagen agent's ungated roles clearing each bar, min-max then calibrated, were: 0.3: 3,467 and 3,520; 0.4: 1,837 and 2,090; 0.5: 519 and 728; 0.55: 200 and 307; 0.6: 56 and 111; 0.65: 15 and 35. So 0.55 sends about 300 roles to extract and 0.65 about 35 to summarise, which its top 30 trims; the top-N backstops (500 and 30) stay for an agent whose bar admits more. The old defaults, 0.3 and 0.5, admit 3,520 and 728 on the calibrated scale, where only the top N would stop them. An agent keeps the limits stored with it, so one saved under the old defaults keeps 0.3 and 0.5 until it is edited: a migration cannot tell a stored default from a choice.

An agent without an intent feeds neither model stage. Its scores have no fit term, so they order roles by recency, affinity and prefer gates, and a fresh role of any kind clears a bar. Store._top_scope leaves such an agent out, and each score run raises one score.no_intent notice naming the enabled agents without an intent. The agent still ranks its own list.

The embedding backend is effectively required. The intent is a CV or a paragraph per agent, and TF-IDF cannot relate "Kubernetes" to "container orchestration". The model is intfloat/multilingual-e5-small, not the English-only all-MiniLM-L6-v2 roledesk started with: the corpus turned European, and on a sample of the live roles (all open Danish-country roles plus a random sample of the rest, language labelled by a stop-word count) against the "Copenhagen engineering" agent's intent, MiniLM separates engineering from non-engineering roles with an AUC of 0.68 overall but only 0.74 restricted to Danish-country roles, where a Danish-country engineering role beats the rest's non-engineering median only 68% of the time; multilingual-e5-small reaches an AUC of 0.70 overall and 1.00 on Danish-country roles, for about 3x the CPU time. A second multilingual candidate, paraphrase-multilingual-MiniLM-L12-v2, measured worse than plain MiniLM (AUC 0.60) and was dropped. e5 needs its own "query: " / "passage: " prefixes, which app/jobs/embed.py adds by matching the model name, not a separate setting.

Summary and tagging are off; extract replaces them. Every profile declares SummaryPolicy(enabled=False) and TaggingPolicy(enabled=False). extract is a roledesk ModelTask on Haiku that turns title and description into the gate columns, and fills only the columns a plugin left empty.

A score is the weighted sum over the sum of the weights. The agent's fit, recency and affinity weights and every prefer gate's weight form one denominator, so a score stays in 0..1 and the 0.9 floor for a favourite means the same thing whatever the weights are.

A gate rejects on evidence, not on absence, and any gates are one group. A require gate used to gate a role whose column was empty ("country unknown"), and extract, which fills that column, is bounded to the roles that rank, so 2,281 of 15,556 open roles were out of reach for good: every board that writes "Remote", "Hybrid" or "Distributed" as its location. Now an empty column neither passes nor fails; the role ranks, extract reads it, and the rescore gates it on what it found. any is the third mode: the any gates form one group a role must match one of, so "in Denmark, or remote" is two any gates, where require alone could only say "and". remote is also read off the location line, as country is, since Greenhouse puts "Hybrid" there and nowhere else.

Gates match loosely where the column is free text. place is a case-insensitive substring, so Berlin matches "Berlin, Germany"; company compares company keys, languages passes when every required language is a value, salary_min is at least the smallest value, and the rest are case-insensitive equality. Each gate is one SQL CASE column, so a require gate filters while the gated row keeps every failed gate as its reason.

A role with no vector is scored without fit. The fit weight leaves the denominator, so a role embed has not reached yet ranks on recency, affinity and prefer gates instead of sinking to the bottom. It is gated as "no vector" only when fit is the only weighted term, and as "no intent vector" when the intent could not be embedded.

Each run replaces an agent's scores. replace_scores deletes and inserts in one transaction, so a role closed or muted since the last run drops out rather than keeping a stale score, and a reader sees one list or the other.

Starter agents are imported once, by a state marker. newsdesk seeds whenever its table is empty. Here agents:seeded in state is set after the import, so deleting every agent survives a restart.

embed and score run in the server process. newsdesk isolates embed in a child process with a timeout. roledesk's run is one bounded pass over open roles, and in-process keeps the embedder replaceable in tests. The cost is torch's memory staying in the server after the first run; isolate it if that becomes the problem.

The pull cascade is pull → embed → score → extract → score → summarise. Every stage is always in the chain and a stage that is off finds nothing, as in newsdesk. The free stages run first: embed and score cost no tokens, and their score is what bounds the two model stages. Extract reads only what ranks, then score runs again so the gates read what it filled. A stage is addressed by its position in the tuple, since score is in it twice. Extract reads on_ingest only as a stage of a pull, so with auto and no model reachable a pull passes it quietly.

Extract is bounded like summarise: the top roles of each agent, by that agent's own limits. Extract has short answers but would otherwise touch every open role, and the seed boards alone produced 29,000; at Haiku's rates that is tens of dollars for a first pass over roles nobody will look at. Bounded to the ranking, a pull reads a few hundred. extract_top is the backstop (default 500) and extract_min the real stop (default 0.55, about 300 roles for the starter agent): a fixed count stays flat whatever the boards produce, where a percentage would not. The bounds are per agent rather than config, because one agent may be a wide monitoring search that should read a thousand and another a narrow hunt that wants fifty; one global number fits neither. A batch dealt named uids finishes them whatever their rank, so a running batch is not cut short by a rescore.

extract_batch and summarise_batch are isolate=True, with a timeout. context.check() only runs between the calls modeltask.one makes, not inside one, so a call that hangs on either backend was otherwise unkillable short of restarting the server; a process boundary is what lets a Kill, or the timeout itself, actually end it. One call is already bounded at agent.timeout, so the deadline is a few calls' worth (30 minutes for extract, 15 for summarise): a batch still running after that is stuck, not slow. The cost: an isolated job's context has no working stop, task or fan_out (see pipeline/child.py's context_for), so a refusal or an account-level failure inside a batch can no longer stop the rest of that task's queued siblings — they run on to the same failure instead of being spared it.

The pages

Each roledesk tab renders an empty shell; static/roles.js fills it and drives every action. Roles, Applications, Agents and Companies extend the boilerplate's base.html; the server renders only select options (from app.models, plugins.ats.FUNCTIONS and plugins.ats.HOSTS, so they cannot drift from what the store and the plugin accept). One code path for "show this" beats a server-rendered first page plus a client re-render after every filter change.

roles(company=, place=) match a case-insensitive substring of what the board wrote. The API's company and place fields are free text a person types, not a slug, so "Trade Republic" finds trade-republic-bank without knowing the key.

A muted company is excluded in SQL, before LIMIT. Filtering after LIMIT shipped first and showed nothing once Databricks' 884 freshly pulled roles filled the newest 200 rows and Databricks was muted.

The phone

Two tabs and a cog. TABS in app/web/pages.py becomes Roles and Applications; the other six pages (Agents, Companies, Sources, Jobs, Schedules, Messages, Account) move behind a settings cog at /settings/<name>, because nine tabs wrapped to three rows on a phone and six of them were setup or system pages. The old paths answer 301 to their new place, query string kept, so a bookmark or a ?agent= link still lands.

Settings are three sub-tabs: Search agents, Sources, Account. Six sub-tabs were three setup pages and three system pages for two things, what the app ranks by and where roles come from, so each is one page now and Account stays its own. The old paths answer 301 to the section they became.

Sources is one page, one section per plugin. Job boards are the search plugin's queries, company boards are ats's, and the runs that read both end the page with the message log. A source's section says which plugin it is, and following a company, hiding it and switching its board off are one list. The UI says "job board" for what the API and CLI call a search, since "search agent" already means something else.

Search agents are back in the UI as a list. One person may run several: their own hunt, and one that monitors roles for someone else, each with its own intent and gates. The list has New and Delete; a delete takes the agent's votes, liked companies and scores with it.

Settings pages sit beside a rail, and Sources shows one section at a time, chosen by the URL fragment. The company boards list runs to 50 or 60 companies, so Runs at the end of a stacked page was out of reach and jump links still meant scrolling back up. The rail (settings/_rail.html, from SETTINGS and SECTIONS in app/web/pages.py) is sticky beside the content, and a strip under the header below 60rem. The fragment rather than a path per section, because the old paths already 301 to /settings/sources#runs and the like, and a hidden section stays in the DOM, so each section's script runs unchanged.

Runs is the queue's own elements plus a Problems log. <desk-jobs> and <desk-schedules> draw what is running and when the pull runs next, the app adds only "Run the pull now", disabled while a pull cascade is live, and the message log is the section's last block rather than a section of its own; #messages still lands on it. "Run one stage" went because the cascade is the unit of work: a stage run alone is a partial pull whose result the next stage never sees. app job <stage> and POST /api/jobs keep the per-stage entry points, which is what "Save and score" uses.

Search agents are cards, edited in place. An agent is defined by what it looks for, so the intent is the card's body and the weights a small line under it; a list of names and weights hid the part that matters. Edit turns that card into the form and New puts a blank one on top, because the new agent sorts first by id and a form at the bottom of the page was out of sight of the card it changed.

Affinity and votes are per agent. A company hidden for one hunt may be a favourite for another, and a vote says what one agent should learn from, so both carry the agent id, and the Roles tab and the company list ask for one agent's. Without an agent id the API uses the default agent, the enabled one with the lowest id, so the CLI and older callers keep working.

taste is the liked-minus-passed centroid, as in newsdesk. An agent's votes become one vector, the mean of the liked roles' embeddings minus the mean of the passed ones', and a role's taste term is its cosine to that vector, calibrated like fit, weighted beside fit, recency and affinity. It is the one term an agent learns rather than is told, and it costs no model call.

Roles are a feed with a reader, not links out. GET /roles/{uid} renders the feed with its reader open on that role instead of sending the reader to the board, because 29,194 of 29,196 open board roles already have a description averaging 6,300 characters, and a search role's short teaser is worth reading in place before deciding whether to fetch the rest.

A vote moves the rank at read time and never hides. Store.roles multiplies the served score by the vote (0.5 passed, 1.2 liked, capped at 1) instead of triggering a rescore, so a vote's effect on Best match shows at once and needs no job. A vote never removes a role from the list either, because a stray tap must stay visible and a single further tap must be able to undo it.

Read state is a column on roles, and a vote stamps it too. read_at sits beside vote, the same shape as newsdesk's items.read_at, so the feed filters and counts by it directly rather than joining a side table on every page load. set_vote stamps it only when it is still NULL, since voting is reading in newsdesk as well; clearing a vote leaves it alone, because taking back an opinion is not un-reading the posting.

Tags are a table, written only by summarise. role_tags(uid, tag) follows newsdesk's tags table rather than a JSON array on roles, so tag_counts and the any/all filter are ordinary SQL (GROUP BY, EXISTS) instead of scanning every row's array. summarise already reads the whole posting once per role, so writing 3 to 8 tags alongside the summary costs no extra model call.

Tag filtering runs in SQL before LIMIT, the same as hide_muted. An EXISTS or COUNT(DISTINCT tag) clause in _roles_from keeps the guarantee the muted-company exclusion already established: a filter narrows the rows a page is built from, not the page after it is already built, so a tag matching only older roles is not silently empty on page one.

"Hidden" is the word the UI uses for affinity -1; the code keeps mute and hide_muted. Renaming the columns would touch every reader of agents and companies for no behavioural gain, so only the toggle's title, the filter label and the dimmed row say "hidden" in the UI.

A company affinity change enqueues score. POST /api/companies/{key}/affinity queues the score job right after the update, deduped to a live run of the same kind so ten clicks are one pass. Otherwise an affinity change is invisible in Best match until the next 6-hourly pull, and an unmuted company's roles have no score rows at all until then, so they are missing from that order entirely.

A search role's posting text is fetched on demand, not in the pull. POST /api/roles/{uid}/text scrapes original_url only when the read page asks for it, because following Jobindex's redirect for every role during a pull would count as a click in Jobindex's own statistics; one click for a posting somebody actually opened is fair.

Summarising is bounded to the top roles of each agent, by that agent's own limits, and off by default. The pull's summarise stage only considers roles with no summary yet that are in the top summarise_top (default 30) of an enabled agent's scores and at summarise_min (default 0.65, about 35 roles for the starter agent, stricter than extract's since a summary is the long answer), so the cost stays flat whatever the backlog of open roles is. The bounds live on the agent, beside its gates and weights, for the same reason as extract's: a wide monitoring search and a narrow hunt want different depths, and the agent is where the difference is already expressed. summarise.enabled: false matches extract's own default; the read page's Summarise button asks for one role's summary either way.

The schedules element's narrow layout is fixed upstream, in ../boilerplate. <desk-schedules> is a shadow root the app cannot lay out from outside, and pipeline/ is a downstream copy refreshed with git archive, so a fix landing only in roledesk would be reverted by the next refresh.

The phone CSS is copied from health and news, not written from scratch. Both apps have already been through several rounds of phone fixes, and the comments in their app.css say what each rule avoids, so adapting their fragments keeps those fixes rather than rediscovering the same edge cases, such as iOS zooming on a 16px input or a card losing width to a side gutter.

Outside the app

Unposted roles and the network stay outside. Their state is inside: an application row with no role_uid, a contact, next_action and next_date, and a company status such as researching or contacted.

LinkedIn enters only as alert mail over IMAP. Scraping it is banned by contract and enforced. The alert mails also carry personalised recommendations no public API exposes.