54 KiB
Key decisions
What was decided for roledesk and the reason for each. The target design is the two ADRs in documentation/; where a paragraph names an ADR section, the rule is there and this file adds only the reasoning. Host details are in plugins/README.md.
Shape
Own database, queue and port. roledesk.db, config.yaml, port 8790. A shared file with newsdesk would buy "articles about this company" by cosine, which is worth less than two apps that can migrate independently.
Storage is written, not imported. desk has no storage package on purpose, so the connection plumbing is copied from newsdesk. If the same edit lands in both a second time, a shared Connection moves into desk below pipeline.
Layers are models, places, dupes, seniority, config, storage, prep, jobs, web, main, cli. places, dupes and seniority read what a board wrote and know no database, so they sit under config and storage may use them. tests/test_architecture.py holds the direction and the doors within app; tests/test_coupling.py holds that pipeline, auth and desk import nothing from app or each other, except desk from pipeline, and that a plugin imports desk only.
Data
Copies are matched on towns, not on the stored place key (app/dupes.py; the rule is ADR Fetch prep and score 1.4). Host does not matter, since a search provider and the company's board list the same role, and one board may list one role under several ids. The stored (company_key, title_key, place_key) finds none of the cross-host pairs, since Ballerup and Ballerup, Denmark differ, and newsdesk's location_key OR title_key would merge the same title at two companies. The same title in Berlin and Munich stays two roles, so a place filter finds each.
A group is rows that all match each other, never a chain. A row with no place matches Belgium and the Netherlands, and joining on one match merged both on the live data. Rows are taken sharpest first (a town, then only a country, then neither) and join a group only when they match every member.
One posting per place is folded too, on every host. A host that lists one job once per office gives rows of the same company and title whose towns never meet; dupes.fold merges those of one host when their texts share at least SAME_TEXT (30%) of their words, and the kept role keeps each copy's place and link. It applies to every host, not only to readers that declare places="copies", because most such rows came from hosts that do not declare it. A remote listing and an office listing stay two roles: the kept role takes a copy's places and not its remote, and the fetch gate reads a remote role by its area, so a remote listing folded behind an office elsewhere was rejected with it, and an office listing behind a remote role elsewhere likewise.
A copy is stored as roles.dup_of and rebuilt from scratch after every pull. Every reader of open roles adds dup_of IS NULL, which keeps idx_roles_posted usable where a window function over the filtered rows would scan them all. Where a group changed, the kept role takes the earliest read_at and each agent's latest vote from its copies; an unchanged group is left alone, so marking a role unread again sticks. The per-board counts on Sources still count copies, since they report what a board holds.
place, country and remote are separate columns. desk's Item.location already means where the original lives.
raw is transport, columns are storage. A plugin puts structured fields in raw under the column's name and upsert_role unpacks them into indexed columns, which is where gates query them. An empty value never overwrites a filled column, so a terse board does not erase what extract found.
A board's field wins over extract's, so a column cannot flip (ADR Fetch prep and score 4.1). set_extracted writes a model's value only into an empty column. A board's field is first-party and upsert_role would put it back on the next pull, so letting the model replace it would make the column flip between two answers. Seniority and remote are the exceptions, below.
function is the one column where the model wins outright, because no board fills it. classify(department, title) in plugins/ats.py is a keyword guess, not a board field: set_extracted writes a non-empty function over it, and upsert_role leaves function alone once extracted_at is set, so it cannot flip between the classifier and the model on alternating pulls.
classify matches whole words and checks the title before the department. Substring matching filed "Commercial Italy" as engineering through " it". Short tokens that start unrelated words ("it", "seo", "qa") need a boundary on both sides, while the rest keep a leading boundary only, since they are stems ("manag", "engineer"). A department is often an umbrella ("Operations") that says less than the title; on a hand-checked sample of 60 roles where the two orders disagreed, title first was right about twice as often. No match returns '', not "other", which would claim a knowledge the classifier does not have.
Several pay ranges for one role give no figures. Greenhouse can state one range per office or pay zone, and with more than one there is no way to tell which office the role in hand is at, so none is picked.
Pay that does not normalise is kept as a line, out of the gate columns. roles.salary_text is one readable line written whether or not the pay conformed (an hourly rate, several pay zones, no currency), so a person can do their own sums. It is replaced whole on every pull, '' included, since the usual keep-filled rule would leave a stale line when a board stops stating pay. salary_state is derived, not stored, and is what the Pay filter and badge read; gates and scoring read only salary_min, salary_max and currency.
extract reads the board's location line beside the text. It gets the title, the board's place and the opening 2,500 and closing 1,500 characters of the description (HEAD and TAIL in app/jobs/extract.py). Greenhouse's location line ("Berlin, Germany") is often the only place a country is stated, and it is part of the posting, so the "only what the posting states" contract holds. Salary, place and remote sit in the opening lines, languages and the deadline in the closing ones.
country comes from the places by lookup, not from extract. A lookup gives the answer a paid call per role would. prep takes the one country the resolved places lie in, EU when they lie in several European countries, and the host's stated country only as a fallback (app/prep/country.py).
extract_tries is a column. newsdesk keeps give-up counts in raw; here upsert_role rewrites raw on every pull, which would reset the count.
Postings are persisted. Storing every posting gives first_seen, when the app first stored the role, and closed_at; a role listed again is open again.
posted_at is the earlier of first_seen and a trusted host's date, and only moves earlier (ADR Fetch prep and score 5.1.1). Without it a newly added board's backlog counted as freshly posted for two weeks: on a copy of the live data two thirds of board roles were posted more than 30 days before the app first saw them. Greenhouse's updated_at resets on every edit, so its reader reads first_published. A date in the future or before 2015 is trusted from no host. upsert_role keeps the earliest posted_at a role has had, since an edit on the board can only move its own date later. first_seen stays when the app found the role.
The tracker schema is taken from jobhunt-hq (MIT). applications with its statuses, next_action and next_date, and an append-only application_events written on every status entered, which is what days to first screen and conversion by source are computed from.
Ingestion
Closure is decided from self.state, not from what a pull_batch job returns. Each plugin stamps a board it parsed with the run's marker, the ids it saw and whether the read was complete, and the settle closes a board only when its stamp carries the marker of its own batches and the read was complete. A batch that raises part way still closes the boards it finished, and a board no batch reached stays open instead of being closed on an empty id list. A failing board's record is plugin state for the same reason: the plugin sees only self.state.
One pull runs at a time, claimed in the database. The schedule, the Run button and app job pull in another process could each start a pull while one was fetching; the second overwrote the run marker, so the first run's boards failed their stamp check or a settle closed on another run's id lists, and with extract on two cascades could send the same roles to the model twice. claim_run sets pull:run in one conditional statement, only while it equals pull:settled or is older than an hour. The hour is the escape for a run that died: blocking every pull until someone clears a key would be worse than one duplicated fetch. The guard lives in app because pipeline's queue dedups a job against a live job of its kind, not a cascade against a live cascade, and solo=True on pull would not help, since that job ends in milliseconds once it has asked for the chain.
Boards follow companies; searches find them. ats reads only companies someone listed, so it cannot surface a role at a company nobody knew of, and the first seed came from a US-centred project with almost no Danish roles. The search plugin runs saved queries on sites that span companies, as search subscriptions whose params are the query, in the same pull as the boards.
The seed is a list of companies, not of boards. A company whose careers site the app cannot read yet stays in the list with the host its site runs on, so "cannot read yet" is a filter rather than a company never written down, and the next reader to write is whichever host that filter shows most. A re-run attaches the board an entry names to a row that never had one, so a seed update reaches an existing database; it never replaces a board, a detached one included, and never touches affinity, notes or status. Seed boards are attached without the check request (set_board(check=False)), since they were read when the seed was written.
discover reads the site before it guesses. A careers page that links its board is the authority, so the home page, its careers links and the usual paths are read for a link into a known host. Only when no page names a host are the hosts with a cheap per-slug endpoint probed with the company's slug, because a board drawn by a script leaves no link. A probe hit on a common name can be another tenant (recruitee:google is not Google), so a page that names only an unreadable host ends discovery with that host: probing anyway could replace a known Workday site with a stranger's Greenhouse board.
Discovery that finds nothing readable does not detach a board. A board that answers is stronger evidence than a page that names no board; set-board or Detach is the explicit step.
A company row carries ats and ats_slug. ats is what discovery found whether or not the app reads it, so a company with an ats and no board is one the app cannot read yet. ats_slug is the name that host knows the company by, so a reader written later has its targets without a second discovery run.
An employer known only from a search gets a company row when it is first judged, and that it came from a search is derived. Search roles carry a company_key with no row, so they could not be hidden or liked; Store.set_affinity creates a row for such a key with only its name. A company with no ats subscription (not even a detached one) and no ats, careers_url, domain or hand_reason is from job boards (from_search). Derived rather than a column, because every way of following a company writes one of those, so a search-only company that later gets a domain or a board is followed from then on without a flag to flip.
Jobindex ads are read from the feed, and a posting is fetched only on demand. Following each item's link would count as a click in Jobindex's statistics and cost a request per role, so the pull never follows it, and discovering a company's board from a search result is left for later. POST /api/roles/{uid}/text (app/jobs/posting.py) follows the "Se jobbet" link for a role somebody opened, since one click for a posting somebody opened is fair, and keeps it as roles.posting_url, which the reader's link-out then prefers.
A search's read never closes by its list (ADR Fetch prep and score 1.3). A search that read as many results as its provider stated used to close every role its query no longer returned, liked and saved ones included, although a result list is never a company's whole list. Search roles close by age alone.
A read that stalls is incomplete, and a sweep's cursor may be a pair. A reader that sees the host send a page again hands it over empty with a cursor, and the runner ends the read incomplete; a cursor of None there read as the last page, and on a host that states no total as a complete read that closed what the pages had not listed. With a stated total a page of ids already read is not a stall, since a resumed sweep starts on one when the list grew at the top. A cursor is kept across pulls when it survives JSON apart from a tuple coming back as a list, so Workday's and SuccessFactors' sweeps resume.
Switching a board or search off closes its open roles. Only a pull that reads a target closes its roles, so a target no longer read would keep them open for good. Switched back on, the next pull reopens what it still lists. A batch in flight when the switch happens reopens them, so the settle closes every switched-off target once more.
A destructive switch asks first; a reopening one queues a pull. Switching a board or search off, replacing a board and detaching one each close roles in a tap, so the page asks first when there are roles to lose, naming the count. Switching a target on, following or setting a board, setting All locations or All roles on and saving a search each queue pull at once instead of waiting up to six hours for the schedule. The pull is queued in the API route, not in the page's script, so a script calling the endpoint gets what a click gets; claim_run's refusal is not an error, since the run in flight or the next one reads the change.
The Sources tab and the CLI share app.jobs.sources. Adding and switching a board or a search are one function each, called by the CLI and /api/sources*, so the two cannot disagree about what "off" does. A search or board is run once before it is saved and refused if it does not answer. A search that finds nothing is refused too, since that is usually a typo; a board that lists nothing is kept, since a company between openings is still one to follow.
Every way a company gets a board is app.jobs.sources.set_board. The CLI, the seed, /api/boards and /api/companies/{key}/board all call it, so attaching, replacing and creating a company are one code path. Row and subscription change in one transaction (Store.set_company_board), so a board another company already has leaves no company row behind. Other facts only fill empty columns, so a person's or the seed's values survive.
A company has at most one ats subscription, rewritten in place. Replacing a board changes the row's target and closes the old board's roles; detaching switches it off and hides it (show 0). Deleting the row instead would leave a batch in flight writing roles against a subscription id that no longer exists, and a second row would clash on the label, which is the company's name. The next set_board reuses the row.
A company listed inside its owner's board is read by hand. No field separates its roles reliably from the owner's, so it is on the Watchlist with reason group.
No reader is built on a key lifted from a site's script. Such a key can change without notice and marks an endpoint that is not open; those companies are on the Watchlist with reason key.
The body cap is 96 MiB (ADR Plugins 3.4). openings.go's 8 MB cap was written against arbitrary URLs, while the readers fetch only their hosts' allow-lists, and Greenhouse boards with ?content=true measured between 9 and 41 MB. Greenhouse retries once without the text, trading the description for the listing.
A request is streamed, with a deadline over the whole request (ADR Plugins 3.4). The body is counted as it arrives and refused at MAX_BYTES, where reading it first held the whole body in memory before the check. httpx's timeouts are per phase, so a host that keeps sending a byte now and then never timed out; the deadline covers redirects and body. On Greenhouse's largest boards a body of up to 10 MB arrived in under a second once it started, while the first byte waited up to 16.6 s, so a read may wait 30 s, about twice the worst seen, and the deadline is 60 s. openings.go's 10 s read timeout had failed two of those boards.
A paged read counts distinct ids of the raw results (ADR Plugins 2.1). Counting raw results means a row the reader drops does not block closing, and counting distinct ids means a role repeated by a shifted offset does not fake completeness.
Teamtailor is read from the career site's feed, and a slug with a dot is the site's own host. The official API needs a key per company; the feed needs none. Most career sites run on the company's domain, so teamtailor:careers.acme.com reads that host. A target may name any host that resolves, the same trust as any board the owner adds, as long as it is a DNS name whose last label is not a number.
A host is a module in plugins/readers/, beside ats.py. plugins/ats.py held eight hosts in six parallel tables and discovery three more, so two people adding a host edited the same lines. The package sits beside ats.py and is not plugins/ats/, since desk.sources.registry globs plugins/*.py and the tests patch names on the ats module.
A host is handed the fetch. A reader receives get, which ats.py binds to its _get at call time, so the allow-list, the redirect rule, the byte cap and the deadline stay in one place and a test that patches ats._get reaches every host. The fetch refuses an IP address, a local host name and any port but 443 before it asks, because a Teamtailor or SuccessFactors slug is a host name somebody typed.
One runner calls every reader, and acts on what each declares. Capabilities were implied by which functions a module had, and the search providers sat outside the contract with their own getter. plugins/readers/_runner.py owns the guarded fetch, paging, series, completeness, the fallback feed, 429s and failures; plugins/ats.py and plugins/search.py are thin Source classes over it and keep their state prefixes, so no stored key moved. A paged read cut by MAX_PAGES continues from its cursor on the next pull without reading page 1 again, so a new role on page 1 waits one sweep; the owner accepted that over reading the head every pull.
Fetch text reads only the roles in the cut, after scoring (ADR Fetch prep and score 3). The pull used to read role pages per board in the order roles were found, so a board of 800 roles took four pulls to reach the roles that mattered. The stage was called enrich until 2026-10-04 and was renamed because it reads data and calls no model.
robots.txt is decided when a reader is built (ADR Plugins 3.7). Adzuna's API host disallows / for every agent; the owner reads its documented API with the account's key anyway, since the rule is aimed at crawlers.
The company switches are global, not per agent, and separate from affinity (ADR Fetch prep and score 1.1, overrides). They decide what is fetched, and one fetch serves every agent. All locations exists because, for a company the owner cares about, a role in Freiburg may be open to remote or worth asking about for Denmark, while Microsoft Germany is not wanted. The wide reach of an All locations company includes the agents' own reach, so the switch never reads less than the board would without it. A board's own params override both the agents' reach and All locations (app/jobs/reach.py). A board under 300 roles is read whole, since filtering it saves little and risks closing roles.
A board's size is set when it is added (ADR Fetch prep and score 1.2). A new board without a size used to be read whole on its first pull, worldwide, until the page cap; so set_board sizes a new board, and one without a size is sized in its batch before its reach is chosen.
A role has a list of places, resolved against a gazetteer. Readers kept one place of several, joined them, or the host posted a copy per place. A place text resolves by its own words (Gazetteer.resolve), never by the role's stated country, which names one country of a multi-place line: San Francisco; London, UK with US is in Europe through London. The gates and the Place filter read every place of a kept role, its folded copies' included, so OpenAI's San Francisco role with a Dublin copy meets a Europe gate.
A role in several cities leads with the one nearest the viewing agent (places.Nearness). Mistral's Forward Deployed role, kept as its Montréal posting with a Paris copy, read "Montréal +3" and opened the Montréal page, while for the owner it is a Paris role. The gazetteer holds no coordinates, so within a tier the agent's home (the first value of the first any gate on country or place) stands in for distance. Only the row and the reader follow the order; the gates and the copies rule do not read it.
A remote role has a remote area: the places a remote hire may live in (role_remote, app/prep/area.py). The Place field listed "USA (Remote)", "Remote US" and "United States (Remote)" as three towns, and no filter said which remote roles someone in Denmark could take. "Remote" alone names no place, so such a role is remote with no stated area. Anywhere is a standalone place with nothing under it, so the Place filter never reaches every role through it.
A remote chip matches by overlap, and Anywhere is in every one. remote:country:DK ("Remote · Denmark") finds a remote role whose area is Denmark or in it, a place Denmark lies in (Europe, Øresund), or Anywhere: the roles a person living in Denmark can take. Danish employers barely post remote roles, so the remote roles open to Denmark are mostly those scoped to Europe or to anywhere, and the owner wanted Remote · Denmark to hold them rather than have a separate "Remote · anywhere" chip; remote:anywhere is a valid token that is never suggested. A remote role with no stated area matches no remote:<place> chip.
Plain Remote is every remote role within the world gate. The token remote:any ("Remote") and the plain remote line of the location gate both mean every remote role inside the agent's require gates on country. Such a gate reads a remote role that has an area through the area alone, so "Home Based - Worldwide", "Germany (Remote)" and "Remote - Europe" meet Europe and "Remote US" does not. The form's Remote line offers no, anywhere within the world gate, or only when open to someone in a list of places (remote_from, matched as the chips are). Remote suggestions are kept to the asking agent's world gate, so an agent kept to Europe is not offered Remote · United States.
For remote, the model wins, as it does for seniority (ADR Fetch prep and score 4.1). Black Forest Labs' Freiburg roles are OnSite on Ashby while the text offers remote work with an office week a month, and the board's word kept them onsite. The prompt says remote when the hire works from home most of the time, even with a regular office week, and hybrid when there are set office days every week. "Remote sensing", "remote control" and "remote monitoring" in a title are not remote (app/prep/words.py).
A company has aliases. company_aliases maps a folded name to a company, so "Orsted" and "Ørsted" are one company without changing company_key, the primary key of companies. The enrichment columns (sector, stage, size band, founded, last funding) stay empty until a reader states them rather than being filled by hand.
A description keeps its shape as marked lines. _shared.html_text turns HTML into a line per paragraph, • before a list item and ## before a heading. Plain text with marks, not sanitised HTML, is stored, because summarise, extract and embed read the same column, and the reader escapes each line before drawing it. Titles and teasers stay one line.
Readers scrape; prep decides (ADR Plugins 1 and 2.5). Mapping lived in 25 readers, each spelling its tables its own way, so changing what "Experienced" means touched a plugin. upsert_role stores the host's words in *_host columns, and when a host table changes, its fingerprint makes the next prep map every open role again from the stored words, with no pull. A host's remote: False stays unknown rather than onsite, because SmartRecruiters, Ashby, Recruitee and Workable send it for hybrid roles too.
Unmapped places and words are fixed in code, from a list on Sources. Settings › Sources › Unmapped lists the place texts no place resolves and the host words no table maps, with no editor: the owner tells Claude what a text is, and the unmapped skill says which dictionary or host table takes it. Code rather than add_place_alias rows, because a fix then reaches a new database and is reviewed like any other change. The prep fingerprint covers places.seed(), so the next prep adds the new spellings and maps every open role again.
Seniority has three sources, and the model wins (ADR Fetch prep and score 4.1). seniority_stated is the host's word through its table, seniority_title is seniority.from_title and seniority_model is extract's; roles.seniority is the first of model, stated, title. The model gets the other two as hints it may overrule, since host vocabularies differ and most roles state no level. app seniority shows per host how often the stated level agrees with the model.
pull is a job that starts the cascade, on a default schedule of every 6 h. A schedule enqueues a job kind and a cascade is started by one, so the pull job only asks for the chain; declared beside it in app/jobs/cascade.py, the schedule is a starting point that an edit under Runs overrides for good. pull is not also a task, which would draw two buttons under Runs.
Every whole read is guarded, not only filtered ones (ADR Plugins 3.3). A Greenhouse or Lever body whose shape changed is HTTP 200 and zero rows, which the settle took for a complete read and closed every role on. _runner.read raises Suspect on any read that is not a sweep and lists under SUSPECT_SHARE of the same read's last count, once that count is at least SUSPECT_MIN (10) for a plain read and 1 for a filtered, narrowed or empty="suspect" read. A board that really shrank costs a Recheck, which resets the count (keep_size), or 21 days of the same answer.
A read that could not ask for remote keeps the remote roles it has. On a host with no remote filter a read for "DK, SE, remote" goes out as "DK, SE" and counts as filtered, so its settle closed every remote role outside those countries. The listing now records remote_unasked, those remote roles close by age only, and the Companies line says "remote not asked". New remote roles on such a board are still not read; reading it whole instead is with the owner (docs/open.md).
A read that asks nothing is not a read. When an ids dimension maps to no host id, the read asked nothing, and counting it as complete and empty would close the board; the listing is incomplete and the Companies line ends "not closing".
Every target is guarded in its batch. A row that failed to store used to fail the whole pull_batch: that board had neither a stamp nor a failing record, and the boards after it in the batch, the same ones every pull since batches are ordered, were never read, closed or aged. run_batch wraps each target, records the failure under the source's failing key and moves on; the failing key is removed only after the stamp.
The settle says what needs a look. Its counts went into the job result, which nothing renders, so a settle could close twelve thousand roles with the Problems log empty. close_boards raises one pull.settled notice naming every target unanswered or read incomplete (not one mid-sweep, or it would fire every pull) and every board that closed more than CLOSE_NOTICE (50) roles or more than half of at least CLOSE_SHARE_MIN (10) open ones.
Ranking
Scoring is per search agent. A role is scored as (uid, agent), because a role is only good for a given search. A gated role keeps its row with the reason, which Show gated on Roles lists.
Three decisions per agent: the fetch gate, the cut and the feed scores (ADR Fetch prep and score). One blended score used to pick what the models read and order the feed, and the owner could not tell which rule did what. Newest first used to show roles the agent rejected, so a change of order changed which roles were shown; every order now hides them. Wishes such as hybrid are feed filters, not score terms (migration 32 dropped the prefer gates), so a role is never judged twice, by the score and by a filter.
Saved filters are per agent and stored on the server (saved_filters). They are per agent because an agent's filters change with what it hunts, and on the server so they follow the owner across devices.
An agent is the group: it holds several prompts, and prompt closeness is the best prompt's. A hunt spans several kinds of role (AI and data leadership, customer-facing technical leadership, AI research), and one blended paragraph ranks a role by its distance to the blend. Each prompt in agent_prompts is embedded on its own and cached under the sha1 of model and text, so an edit re-embeds one prompt; scores.prompt names the winner. Votes, liked companies, gates, weights and limits stay per agent, so a vote counts for every prompt; a market watch is a second agent with its own. The embedder separates kinds of role but not seniority: an individual contributor's posting on the same subject ranks beside a head's, so seniority comes from extract, the Seniority filter or votes.
Closeness is a cosine mapped to a level by constants calibrated per embedding model: clip((cos - lo) / (hi - lo), 0, 1). (cos + 1) / 2 spanned only about 0.87 to 0.95 on multilingual-e5-small, so a term weighted 0.2 outweighed all of prompt closeness. Min-max across a run's roles fixed the spread but made it a rank: the best role of a poor set got 1, a level's meaning drifted from run to run, and the cut's minimum compared an absolute number with a relative one. The constants are CALIBRATION in app/config.py, keyed by model name so a change of embed.model cannot carry another geometry's constants; embed.calibration in the config replaces a model's entry. A model with none falls back to min-max per run. Job like (taste) is mapped the same way with its own pair.
multilingual-e5-small's bands are fit 0.75 to 0.88, taste -0.14 to 0.19 and cv 0.77 to 0.90, measured on a copy of the live database. Over 16,841 open roles and five intents the raw prompt cosine pooled to p1 0.753, p99.5 0.860 and p99.9 0.871; hi sits above every intent's p99.9, so only a handful of an agent's roles clip to 1, where 0.86 would clip about 75 and lose their order where it matters most. Taste came from synthetic votes (30 liked and 30 passed for each of three intents) and should be re-measured once real votes exist. A CV chunk is longer than a prompt and sits closer to every posting: the prompt band clipped 88 of one agent's 7,270 roles to 1, so the 50 closest tied on posting date, while (0.77, 0.90) clips one.
The embedding backend is effectively required, and it is intfloat/multilingual-e5-small. Prompts and a CV are paragraphs, and TF-IDF cannot relate "Kubernetes" to "container orchestration". roledesk started with the English-only all-MiniLM-L6-v2, but the corpus turned European: on a sample of the live roles against an engineering prompt, MiniLM separated engineering from other roles with an AUC of 0.68 overall and 0.74 on Danish-country roles, multilingual-e5-small with 0.70 overall and 1.00 on Danish-country roles, for about 3x the CPU time. paraphrase-multilingual-MiniLM-L12-v2 measured worse than MiniLM (AUC 0.60). e5 needs its "query: " and "passage: " prefixes, which app/jobs/embed.py adds by matching the model name rather than by a setting.
desk's own summary and tagging are off. Every profile declares SummaryPolicy(enabled=False) and TaggingPolicy(enabled=False); roledesk's extract fills the gate columns and its summarise writes the summary and tags, as ModelTasks of its own.
A gate rejects on evidence, not on absence, and the location lines are one group (ADR Fetch prep and score 1.1). A gate used to reject a role whose column was empty, and extract, which fills that column, reads only the cut, so every role from a board that writes "Remote" or "Hybrid" as its location was out of reach for good. The location lines are any gates that form one group a role must match one of, so "in Denmark, or remote" is two any gates where require alone could only say "and". Three cases count as an unknown location rather than as evidence. A place text the gazetteer cannot resolve is one: "Storkøbenhavn" and "Headquarters" were read as "not Malmö" and rejected about 650 open roles, most of them in Greater Copenhagen. A role whose country and place are both unknown passes whatever its remote says, since a board that writes only "Hybrid" may sit in one of the agent's towns. A place that holds the agent's places, Øresund or Europe for Denmark, is no evidence against either, but only when the role names no place elsewhere: London; Europe is in London. Such a role enters the cut on its level, extract reads it and the gate is applied again on what it found.
A favourite only raises the level; the company switches do the widening (ADR Fetch prep and score 2.1). Affinity used to skip gates, which made it both a widening and a ranking; widening now belongs to All locations and All roles, which are the same for every agent because fetching is shared.
Gates match loosely where the column is free text, and each gate is one SQL column. country and place match over the gazetteer as the Place filter does, company compares company keys, title matches whole words of title_key, languages passes when every language the role requires is one of the agent's, salary_min is at least the smallest value, and the rest are case-insensitive equality (_gate_sql in app/storage/store.py). Each gate is one SQL CASE column, so the gated row keeps every failed gate as its reason.
Role exclusion matches title phrases, not function. Sales titles (Account Executive, Client Partner) and individual data engineering ranked high, since their postings read like the sales engineering and data leadership the prompts describe. function = sales could not remove them: the keyword classifier files "Solutions Engineer" and "pre-sales" under sales, so it would also have gated Field CTO, Staff Solutions Engineer and every AI Deployment Strategist near the agent's top. A value matches as whole words of title_key, so data engineer gates "Senior MLOps/Data Engineer" and not "Data Engineering Manager", and the reason names the value that matched ("title account executive"). Bare sales and vice president stay off the owner's list because they would take Sales Engineer and VP roles, and account manager because it would take Technical Account Manager.
A role that gains text is extracted again. Extract reads a role once, and a role read from its title alone (fetch text capped or failing, or the role entering the cut late) kept title-only place, remote and level for good. The pull that brings a better text now resets extracted_at beside dropping the vector, so both stages read the text.
A role with no vector is scored without prompt closeness and job like. Their weights leave the denominator, so a role embed has not reached ranks on newness and affinity instead of sinking to the bottom. It is gated as "no vector" only when fit is the only weighted term, and as "no prompt vector" when no prompt could be embedded.
Each run replaces an agent's scores. replace_scores deletes and inserts in one transaction, so a role closed or hidden since the last run drops out rather than keeping a stale score, and a reader sees one list or the other.
Starter agents are imported once, by a state marker. newsdesk seeds whenever its table is empty; here agents:seeded in state is set after the import, so deleting every agent survives a restart.
embed and score run in the server process. newsdesk isolates embed in a child process with a timeout. roledesk's run is one bounded pass over open roles, and in-process keeps the embedder replaceable in tests. The cost is torch's memory staying in the server after the first run; isolate it if that becomes the problem.
Every stage is always in the pull cascade (CASCADES in app/jobs/cascade.py). A stage that is off finds nothing, as in newsdesk. The free stages run first, since embed and score cost no tokens and score writes the cut that bounds everything after it. A stage is addressed by its position in the tuple, since score is in it three times.
The cut is one selection with one depth (ADR Fetch prep and score 2.2). Fetch text, extract and judge read the same cut, and a second bar inside it would defeat its purpose. The cut keeps the fetch gate because without it only 118 of one agent's 500 best by prompt closeness were roles it could take.
extract_batch, judge_batch and summarise_batch run isolated, with a timeout. context.check() runs only between the calls modeltask.one makes, not inside one, so a call that hangs was unkillable short of restarting the server; a process boundary lets a Kill, or the timeout, end it. One call is bounded at agent.timeout, so the deadline is a few calls' worth (30 minutes for extract, 15 for judge and summarise): a batch still running after that is stuck, not slow. The cost is that an isolated job's context has no working stop, task or fan_out (context_for in pipeline/child.py), so a refusal or an account-level failure inside a batch cannot spare the task's queued siblings.
Judge reads a role against a written profile, after the /rank of github.com/MadsLorentzen/ai-job-search. Prompt closeness compares topics; it cannot tell a step down, years of experience or a deal-breaker, which a model reading both texts can. The profile is the CV, its languages with a level, and two soft texts, "What I like" (like_text) and "What I don't like" (avoid_text), each covering the work, the workplace and situations with no split into tasks versus environment, since the prompts already say what the owner wants. A deal-breaker is a hard rule and belongs in the fetch gate, so the form has no must-haves box; must_text is set only through the API. Culture is left out because it is guessed from ad text, and location because the fetch gate does it.
The model reports facts and code applies the rules. For each must-have line the model answers "met", "broken" only when the posting clearly contradicts it, or "unclear" when the text does not say, never inferred from silence; a must-have it leaves out counts as unclear. The cap is code's: a broken must-have makes career count as at most BROKEN_CAP (4) when a role is served (app/models.py), so the stored career stays the model's and the reader shows both. The CV match weights are applied when a role is served too, so changing them needs no model call. The verdicts store each line's text, so the reader still shows what a judgement was made against after the text is edited. The input_key carries judge.FORMAT (4) and the texts, so a change to the prompt's contract makes every judgement again.
CV closeness is a free, local signal beside the judge, never in its place. The score run cuts the CV into chunks (embed.cv_chunks, up to CV_CHUNK_CHARS = 1,200 characters, inside e5's 512 tokens), embeds each as a prompt is, and stores each role's best cosine over the chunks, as a level, in scores.cv_fit. It is a list of the cut of its own because on one agent 43 of the 50 roles closest to the CV were outside its 150 best by score.
A posting's languages and deadline give a flag, not a score. Extract reads languages_required (language, level, required or plus) and a deadline from the closing lines, where both usually sit; Jobnet and Jobbank state a deadline, kept as deadline_host, which prep takes first. The language flag is computed when a role is served: ⚠ when a required language is missing or above the agent's level, ★ when a plus one is the agent's. Closing soon is a deadline from today to four days on, in UTC.
The pages
Each page renders a shell that its own script fills. The server renders only select options, from app.models, plugins.ats.FUNCTIONS and plugins.ats.HOSTS, so they cannot drift from what the store and the plugin accept. One code path for "show this" beats a server-rendered first page plus a client re-render after every filter change.
The feed's first request names the agent, and waits only when it cannot. The server gates nothing for a request without an agent, so a first request sent before the agents were known listed every open role until they arrived, and for good when /api/agents failed. The request carries the agent of the address, else the one this browser viewed last; with neither it waits for the agents. Companies, tag counts and saved filters still fill in when they arrive and the feed does not wait on them, since awaiting them once left the feed blank behind a silent catch. When the agents cannot be had, the feed shows its error state with a retry (state.feedError).
A page starts a row earlier for each vote or read since the last one. The server sorts afresh on every request, and a pass, or a read with Show read off, takes a role out of the part already loaded, so the next page asked from the count of rows on screen skipped one role and showed another twice. The offset the server has read is kept apart from the rows on screen, the next page steps back one row per change and drops the rows already shown, and the end of the list is the server's to say. A rescore during scrolling can still shift the list; only a snapshot of the order on the server would rule that out.
The rail's counts cover what the feed shows. Tag, company and place counts ignored the fetch gate, so a chip promised more roles than the list held (Novo Nordisk 134 against 102). With an agent named, each count keeps to the roles that agent's gate passes.
company= and place= on /api/roles match a case-insensitive substring of what the board wrote. They are free text a person types, not a slug, so "Trade Republic" finds trade-republic-bank without knowing the key, and other pages link with them.
Every filter runs in SQL before LIMIT. Filtering after LIMIT showed nothing once a hidden company's freshly pulled roles filled the newest 200 rows. Hidden companies, tags (an EXISTS or COUNT(DISTINCT tag) clause) and every other filter narrow the rows a page is built from, so a filter matching only older roles is not silently empty on page one.
GET /api/roles serves a row, not a role. The whole Role, description, raw and article text included, went to a feed that shows a title, a few badges and an excerpt. Store.roles_page selects only what a row draws (RoleRow), with the excerpt cut in SQL, which made a page about a tenth of the size; the reader fetches the full role by uid when it opens one.
Seniority is a ladder of rungs with two tracks. One line from intern to exec put a Principal below a Team Lead and a Staff engineer below any manager, while the owner counts a Principal Solutions Engineer as very senior. From staff up, each rung pairs an individual contributor's level with a people lead's, as big tech firms grade them (LADDER in app/models.py), and roles.seniority stays one column of words, so gates, extract and the host tables work on words. "X Manager" is a people level only where it clearly names someone who runs a team, since a product or account manager is an individual contributor. Principal sits one rung above Staff, although at some firms it is the top individual grade and at others a senior manager's peer.
A filter that most roles cannot answer keeps the ones that do not say. The Seniority filter keeps roles with no level unless its box is unticked, since most roles state none. The Pay filter leaves out roles that state no pay unless its box is ticked, since most state none and the filter would otherwise do nothing. GET /api/roles/facets counts each level and pay state under the other filters, so the rail shows what a click would add or lose.
The Seniority filter is a range of rungs and a track, not a button per level. "At least Staff" is a range, and "manager roles only" is the Staff rung on the people track, so From, To and Track cover what the buttons did; only a scattered pick such as junior and director is lost. The page carries RUNGS and LADDER as JSON, so the words live in models.py only.
Company and Place are chips picked from suggestions. Only a suggestion becomes a chip, so a filter names something the database holds: a company key or a place token. app/static/chips.js has no dependency; its tests run with node --test tests/js/*.test.mjs.
A filter is a history entry, text is not. Each filter set pushes an entry and back takes it off. The search box and typed text replace the entry, or back would step through a search a letter at a time, and so do the agent and the order. Every pushed entry carries the address it was pushed over, so closing the reader or taking a company off is history.back() when the entry under it is that list.
A board's limits are words on its row, and a filter. The readers declare what they cannot deliver and the runner records how a board was read, and none of it reached the Companies list. app/jobs/limits.py turns both into phrases per board (WORDS), shown after the size and offered under the Companies filter's Limits; board_open says how many open roles the board itself has, since the company's count includes search roles.
The phone
Three tabs and a cog. TABS in app/web/pages.py is Roles, Applications and Watchlist, and the settings pages sit behind a cog at /settings/<name>, because nine tabs wrapped to three rows on a phone and most were setup or system pages. Settings are three pages, Search agents, Sources and Account, since six were three setup and three system pages for two things, what the app ranks by and where roles come from.
Watchlist is a tab: the companies the app cannot read, rated yes or no. The owner goes through their careers pages by hand and wanted that list one click from Roles. A company is on it when it has no board and is not known from a search only, or when its board is failing. The rating is companies.watch, global and apart from affinity, which is per agent and ranks roles the app does read. careers_opened_at is stamped when the careers link is followed, and On the list sorts by it, never opened first, so a round starts with what is overdue. On a phone the third tab leaves the Roles search field a few letters wide, so while it has the focus the tabs and the queue chip step aside.
A Watchlist company says why it is there (companies.hand_reason, HAND_REASONS). Attaching a board clears it. A company with a reason is never "known from a search only", so one added with a name alone is listed. No second table and no checked_at: a company has one careers link, and careers_opened_at already says when somebody last looked. A careers link somebody typed is kept, since discovery and the seed fill it only when empty. A name changes only while the company has no board, since the subscription is found by the key the name folds to.
The old paths' 301s are gone. A browser caches a 301 indefinitely, so a path a later page reused would be unreachable; a one-person app with no outstanding bookmarks does not need to carry that risk.
Sources is one page in sections: Job boards, Company boards, Runs and Unmapped. Job boards are the search plugin's queries and company boards are ats's. The UI says "job board" for what the API and CLI call a search, since "search agent" already means something else.
Settings pages sit beside a rail, and Sources shows one section at a time, chosen by the URL fragment. The company boards list runs long, so Runs at the end of a stacked page was out of reach and jump links still meant scrolling back up. The rail (settings/_rail.html, from SETTINGS and SECTIONS) is sticky beside the content, and a strip under the header on a narrow screen. A fragment rather than a path per section, because a hidden section stays in the DOM, so each section's script runs unchanged.
Runs is the queue's own elements plus a Problems log. <desk-jobs> and <desk-schedules> draw what is running and when the pull runs next; the app adds "Run the pull now", disabled while a pull cascade is live, and the stage switches. "Run one stage" went because the cascade is the unit of work: a stage run alone is a partial pull whose result the next stage never sees. app job <stage> and /api/jobs keep the per-stage entry points.
Search agents are cards, edited in place. One person may run several agents: their own hunt and one that watches roles for someone else. An agent is defined by what it looks for, so its prompts are the card's body and the weights a small line under it. Edit turns that card into the form and New puts a blank one on top, because a form at the bottom of the page was out of sight of the card it changed. Job boards and Watchlist companies are edited the same way, one open at a time.
A gate the form cannot say lands in a row under Advanced, so a load and a save lose nothing. The form asks the fetch gate in its four parts by name rather than as the stored require and any sets with ISO codes, which the owner could not read; any other gate is a row under Other rules. Spellings of a town that differ by accents only share one chip, and both are stored.
Viewing as on Roles is a select, also with one agent. The owner looked for the select to switch agent and found it gone after the second agent was switched off. It lists the enabled agents, since an agent that is off has no fresh scores.
Affinity and votes are per agent. A company hidden for one hunt may be a favourite for another, and a vote says what one agent should learn from, so both carry the agent id. Without an agent id the API uses the default agent, the enabled one with the lowest id, so the CLI and older callers keep working.
Roles are a feed with a reader, not links out. GET /roles/{uid} renders the feed with its reader open on that role instead of sending the reader to the board, because nearly every board role carries its full text, and a search role's teaser is worth reading in place before fetching the rest.
Every link built from someone else's data goes through safeUrl (app.js). A board, a search result, a careers page and an application's url are free text from outside the app; an http(s) check before it lands in an href keeps a javascript: link or a bare string from being clickable.
A vote moves the rank when the feed is served and never hides (ADR Fetch prep and score 5.1.3). Applied at read time rather than by a rescore, a vote shows at once and needs no job; never hiding means a stray tap stays visible and one more tap undoes it.
Read state is a column on roles, and a vote stamps it too. read_at is on roles, as newsdesk's items.read_at, so the feed filters and counts by it without joining a side table. A vote stamps it only when it is NULL, since voting is reading; clearing a vote leaves it, because taking back an opinion is not un-reading the posting.
A role's buttons run ▼ ● ▲ ★, and saving a role likes it. The order is low to high, so ▲ sits beside ★. A role worth saving is a liked one, so judge.js casts ▲ for the viewing agent after a save, unless ▲ is cast already; unsaving leaves the vote alone. This is done in the page, not in POST /api/applications, because an application belongs to no agent and a vote does.
Tags are a table, written only by summarise. role_tags(uid, tag) follows newsdesk's tags table rather than a JSON array on roles, so tag counts and the any/all filter are ordinary SQL. summarise already reads the whole posting once per role, so writing 3 to 8 tags beside the summary costs no extra call.
A company affinity change or an agent save queues a score that will see it. score is solo, so the queue dedups a request into the live job; a queued one sees the change, but a running one read the old state, and deduping into it reported "Saved and scored" for a score that did not include the save. So a request during a run queues one follow-up with depends_on the running job, keyed score:after:<id>, which makes ten clicks during one run one extra pass. The routes answer with that job's id and the page waits on it.
Summarise is off unless switched on, and saving or liking a role summarises it at once (ADR Fetch prep and score 6). Saving or liking queues a summarise_batch for that one uid, under the key the Summarise button uses so a double tap is one job, when the stage is on, the role has text and no summary and a model is reachable, rather than waiting for the next pull.
The model stages and fetch text are switched on the page, and a stored setting beats config.yaml. extract.enabled and summarise.enabled were read at startup and both false, and nothing on the page said so. The state keys stage:extract, stage:summarise, stage:judge and stage:fetch_text override the file, and stages.stage_on in app/jobs/stages.py is the one function that decides, so a stage in a pull and app job <stage> agree; with no key the file applies, so an existing deployment behaves as before until someone uses the switch. A stored setting rather than a rewrite of config.yaml, because rewriting it from the app would lose its comments and a restart is what the switch avoids. Turning a stage on starts nothing; the next pull runs it.
Outside the app
Unposted roles and the network stay outside. Their state is inside: an application row with no role_uid, a contact, next_action and next_date, and a company status such as researching or contacted.
LinkedIn enters only as alert mail over IMAP (not built yet). Scraping it is banned by contract and enforced. The alert mails also carry personalised recommendations no public API exposes.