From 0c4f3c9bbe1cd333d447d0e799eca8cf633682ef Mon Sep 17 00:00:00 2001 From: Wolf Date: Tue, 6 Oct 2026 13:49:46 +0200 Subject: [PATCH] Add to the flow ADR what the review found missing: the gate's unknown location, prep, the remote read on hosts without a remote filter, the per-office fold within one country, the embedding and its calibration, the models and their inputs, the feed's read state and Show gated, and must-haves taken out Co-Authored-By: Claude Fable 5.1 --- documentation/ADR Fetch prep and score.md | 67 ++++++++++++++--------- 1 file changed, 40 insertions(+), 27 deletions(-) diff --git a/documentation/ADR Fetch prep and score.md b/documentation/ADR Fetch prep and score.md index 8a1578d..6628f22 100644 --- a/documentation/ADR Fetch prep and score.md +++ b/documentation/ADR Fetch prep and score.md @@ -1,10 +1,10 @@ -# Architecture +# ADR Fetch prep and score The target design of roledesk's ranking: how a role gets from a board to the Roles feed for one search agent, and the key decisions behind it. What gets built is tested against this document. - **Fetch gate**: the agent's hard rules, a yes or no per role, in four parts: world, location, role exclusion and role fetch. -- **Input analytics score** and **input analytics filter**: what decides the **cut**, the roles that pass the analytics filter and get fetch text and the `model analytics score`. -- **Feed scores**: the two orders of the feed, the want feed score and the match feed score. +- **Input analytics score** and **input analytics filter**: what decides the **cut**, the roles that pass the analytics filter and get fetch text and the `model analytics score`, which is the `CV match` (4.2.1). +- **Feed scores**: the two scores that order the feed, the want feed score and the match feed score; Newest is the third order. - **Fetch text**: the second HTTP read of a role's own page, without a model. - **Extract**, **judge**, **summarise**: the model calls (Haiku). - **Affinity**: an agent's setting for a company: hidden, neutral, liked or favourite. @@ -17,6 +17,7 @@ The target design of roledesk's ranking: how a role gets from a board to the Rol world · location · role exclusion · role fetch, with All locations and All roles as overrides company boards: whole under 300 roles, else filtered at the host where it can search providers: their own saved query + then prep, no tokens: host words, places, remote, country; the copies folded │ ▼ 2 Input analytics score, no tokens ─ prompt closeness, CV closeness, affinity → the cut @@ -28,7 +29,7 @@ The target design of roledesk's ranking: how a role gets from a board to the Rol 4 Model analytics score ─ extract the cut → fetch gate and cut again → judge the cut │ ▼ - 5 Feed ─ every role that passes the fetch gate, in two orders; + 5 Feed ─ every role that passes the fetch gate, in three orders; filters and saved filters narrow it │ ▼ @@ -51,7 +52,7 @@ The stricter location filter. A role passes the `location fetch gate` when it ma #### 1.1.3 Role exclusion gate -A list of titles to leave out: phrases that reject a role by its title. No host can filter by exclusion, so this gate is applied to every role after fetch. +A list of titles to leave out: phrases that reject a role by its title, matched on whole words. No host can filter by exclusion, so this gate is applied to every role after fetch. #### 1.1.4 Role fetch gate @@ -64,26 +65,34 @@ The fetch terms: words a big board's host must match before it sends a role. The - A search provider is read by its own saved query, which replaces the `role fetch gate` (1.1.4). - A hidden company never passes. -The gate rejects on evidence only. A role with an unknown location passes until extract has read it. +The gate rejects on evidence only. A role with an unknown location passes until extract has read it. A location is unknown when no place of the role resolved and no country is stated, an unresolved place text included. A place that holds the agent's places (Øresund or Europe for Denmark) is no evidence against the role, unless the role also names a place elsewhere. A role in several places passes when one is inside. A hybrid or onsite role at an unknown location passes. A role whose text says remote while its fields do not (1.5) passes the location gate until extract has read it. The world gate reads a remote role by its remote area. ### 1.2 Reading a board - A company board with fewer than 300 roles is read whole. - A company board with 300 or more roles is asked for the fetch gate at the host: the world and location gates, and the fetch terms, where the host can filter on them. A host that cannot filter on a part sends the board whole for that part. - A board's size is measured once, when it is added or rechecked, not on every read. -- One fetch serves all enabled agents. The host is asked for the union of their world and location gates and the union of their fetch terms; one enabled agent without fetch terms means no terms. Each agent then applies its own gate to what arrived. +- One fetch serves all enabled agents. The host is asked for, per agent, its location gate, or its world gate when it sets no location, united over the agents, and for the union of their fetch terms; one enabled agent without fetch terms means no terms. Each agent then applies its own gate to what arrived. +- A host that filters on countries but not on remote is asked for the world gate's countries when an agent wants remote roles, so a remote role listed under any country in the world gate still arrives; the read says that remote was not asked. +- A board's own saved params win over the agents' gates on the dimensions they set. With no enabled agent every board is read whole. ### 1.3 Closing - A role closes when a complete read of its board no longer lists it. -- A read that is not complete closes nothing: one narrowed by fetch terms, a search provider's, a partial or failed read. Roles on such boards close by age. This is accepted because ranking relies on recency, not on closing. +- A read that is not complete closes nothing: one narrowed by fetch terms, a search provider's, a partial or failed read. Roles on such boards close by age. This is accepted because ranking relies on recency, not on closing. A read filtered by location is complete for what it asked: it closes the roles inside its filter that it no longer lists. +- Switching a board or search off closes its roles. A closed role that a read lists again reopens. A search is read 5 pages deep, 10 at most. - A role that no read has listed for 21 days closes. A board that has failed on every read for 21 days closes its roles. - A liked or saved role is never closed by a filtered read. ### 1.4 Copies - One posting listed several times (by a search provider and by the company's board, or by one board under several ids) is one role. Two listings are one posting when they share company, title and country, and their towns overlap; a listing without a town matches any town. -- The kept copy is the company board's over a search provider's, then the one with text, then the earliest posted. Only the kept copy is scored, read by a model and shown; it lists the others. +- The kept copy is the company board's over a search provider's, then the one with text, then the earliest posted. Only the kept copy is scored, read by a model and shown; it lists the others. Votes and read state move to the kept copy. +- A host that posts one job once per office gives listings of one title whose towns never meet. Those of one host are one role when their texts share 30% of their words, within one country, remote and office kept apart; the kept copy carries every office's place. + +### 1.5 Prep + +After fetch, without a model: a host's words for seniority, employment and remote map through a table per host; each place text resolves against a gazetteer of towns, regions, countries and continents, where a region may lie in two countries; remote and the remote area are read from the place words; the country comes from the resolved places; a role whose title or text says remote ("fully remote", "work from home", not "no remote") is marked as possibly remote for the gate (1.1). A text nothing maps is listed under Unmapped for the owner to add, and until then the role's location counts as unknown. ## 2 Input analytics score and filter @@ -94,17 +103,18 @@ Each role is embedded and scored semantically: - **Prompt closeness** is its level against the best of the agent's prompts. - **CV closeness** is its level against the closest part of the CV. -Both are calibrated levels from 0 to 1. A company's affinity adds to both levels the share it has in the `want feed score`: the company like weight times the company like's rise above neutral (5.1.1), over the sum of the want feed score weights. On the start weights a liked company adds about 0.056, a favourite about 0.111. The bonus counts before the count and the minimum apply. +Both are calibrated levels from 0 to 1. A company's affinity adds to both levels the share it has in the `want feed score`: the company like weight times the company like's rise above neutral (5.1.1), over the sum of all four want feed score weights, also when one of them has left the sum. On the start weights a liked company adds about 0.056, a favourite about 0.111. The bonus counts before the count and the minimum apply. + +The embedding is `intfloat/multilingual-e5-small`, with its query and passage prefixes, over the title and the first 2,000 characters of the text, else the teaser. The CV is cut into chunks of at most 1,200 characters and the closest chunk counts. A level is a linear clip of the cosine between two bounds measured per model: prompt closeness 0.75 to 0.88, CV closeness 0.77 to 0.90, job like -0.14 to 0.19. With several prompts each is scaled on its own and the best counts. #### 2.2 Input analytics filter The `closeness`es are the main input for the `input analytics filter`, which determines which roles get `fetch text` and the `model analytics score`. A role passes the `input analytics filter` when it: -- passes the `fetch gate` (1.1.1 to 1.1.3) - +- passes the `fetch gate` (1.1.1 to 1.1.3), and - is among the best N by `prompt closeness`, each at least X (500 and 0.6 to start), or - is among the best M by `CV closeness` that the first list leaves out, each at least Y (150 and 0.6 to start), or -- is liked or saved not matter the `closeness`. +- is liked by this agent or saved under any agent, whatever its `closeness`. The two lists are kept apart because they answer different questions (what I want, what I am), and a merged number lets the stronger side crowd the other out. For a broad agent the count stops each list; for a narrow agent the minimum stops it, so roles that do not fit are not read just to fill the count. @@ -114,43 +124,42 @@ An agent without a CV has no CV list and no judge. An agent without prompts has ## 3 Fetch text -Some job boards have one API for the listings and another for the job specs. `fetch text` reads the role's own page for every role in the cut that came without text. Once the full text is fetched, the role is embedded again and its levels recomputed. A role that enters the cut after this recompute gets its text on the next pull. +Some job boards have one API for the listings and another for the job specs. `fetch text` reads the role's own page for every role in the cut that came without text, on hosts that have a role page, at most 200 a board and 1,000 a run, a failed read retried after 7 days; Jobindex is never read in a pull, since each read counts as a click there. Once the full text is fetched, the role is embedded again, its levels recomputed, and extract reads it again if it had read the title alone. A role that enters the cut after this recompute gets its text on the next pull. ## 4 Model analytics -All roles in the cut go through the detailed analysis and metadata extraction. Extract and judge are two calls: `extract` needs no CV, it is faster, and it is the final step of the ingestion, so it decides what finally passes the `analytics filter`. +All roles in the cut go through the detailed analysis and metadata extraction. Extract and judge are two calls: `extract` needs no CV, it is faster, and it is the final step of the ingestion, so it decides what finally passes the `analytics filter`. All three calls go to Haiku (`claude-haiku-4-5`), extract 20 roles a call, judge 5, summarise 10. A stage that is switched off, or has no model to ask, is skipped, and the Runs page says so. A cut role without text is neither judged nor summarised. ### 4.1 Extract Often information is only in the free text, e.g. that a role in a specific location is actually remote. So every role has raw and processed fields for e.g. salary, location, remote and seniority. Extract reads what is hidden in the free text. A field the board states wins over what extract reads, except seniority and remote, where extract wins. -**Extract** reads the cut first. It fills location, remote area, seniority, languages and deadline. +**Extract** reads the cut first, the first 2,500 and last 1,500 characters of each role. It fills location, remote and remote area, seniority, languages, deadline, pay and currency, function and work permit. A role the model leaves unanswered three times is given up. -The fetch gate and the cut are then applied again with what extract filled. A role can drop out, such as one that turns out to be outside the agent's locations, or enter, such as one that turns out to be remote. A role that enters is extracted on the next pull. +The fetch gate and the cut are then applied again with what extract filled. A role can drop out, such as one that turns out to be outside the agent's locations, or enter, such as one that turns out to be remote. A role that enters is judged in this pull and extracted on the next. ### 4.2 Judge -Once extract has completed the ingestion, `judge` reads each role in the cut against the agent's CV, languages, likes and dislikes. It outputs: +Once extract has completed the ingestion, `judge` reads each role in the cut, up to 8,000 characters of it, against the agent's CV, languages, prompts, likes and dislikes. It outputs: - the `CV match` - one to three strengths and gaps -- a verdict on each must-have -Each role is extracted once. A judgement is redone only when its input changes: a change to the CV, a prompt, or the like or dislike text re-judges the whole cut (about an hour for 650 roles); a changed posting re-judges that role only. +Each role is judged once. A judgement is redone only when its input changes: a change to the CV, the languages, a prompt, or the like or dislike text re-judges the whole cut (about an hour for 650 roles); a changed posting re-judges that role only. #### 4.2.1 CV match Judge rates how closely the job spec matches the experience in the CV on three dimensions: -- skills, -- experience, -- career, +- skills, where a language the posting requires above the level spoken counts as a missing skill, +- experience, +- career, rated against the prompts, likes and dislikes rather than the CV, -each from 0 to 10. A model's rating is an estimate, not a measurement, so a coarse scale serves better than 0 to 100. The **CV match** is the three numbers weighted as given in the agent settings (30/25/30 to start). A must-have the role breaks caps career at 4. Roles with the same CV match are ordered by CV closeness. +each from 0 to 10. A model's rating is an estimate, not a measurement, so a coarse scale serves better than 0 to 100. The **CV match** is the three numbers weighted as given in the agent settings (30/25/30 to start), times 10, so 0 to 100. Roles with the same CV match are ordered by CV closeness. ## 5 Feed -The feed shows every role that passes the `world fetch gate`, the `location fetch gate` and the `role exclusion gate` (1.1.1 to 1.1.3), loaded as you scroll, in every order, newest first included. No order shows a role the fetch gate rejects. A hidden company's roles are never shown. +The feed shows every open role, kept copies only, that passes the `world fetch gate`, the `location fetch gate` and the `role exclusion gate` (1.1.1 to 1.1.3), loaded as you scroll, 60 at a time, in every order, newest first included. No order shows a role the fetch gate rejects. A hidden company's roles are never shown. Opening a role or voting on it marks it read; Show read, on by default, hides the read ones when switched off. Saving a role (★) also likes it. ### 5.1 Feed scores @@ -161,6 +170,7 @@ The `want feed score` consists of `prompt closeness`, newness, company likes and - `newness` counts from the posting date, the earlier of when the app first saw the role and the date the board states (only for boards whose dates are reliable), and halves every 14 days. - `company like` is 1/3 for a neutral company, 2/3 for a liked one and 1 for a favourite. - `job like` is described in 5.1.3. +- A role without an embedding yet is scored without `prompt closeness` and `job like`; their weights leave the sum. #### 5.1.2 Match feed score, "What fits me" @@ -185,16 +195,17 @@ The user likes or passes on roles in the feed, per agent. Each role has an embed - The Roles page says that the feed is already filtered by the agent's fetch gate, location included. - The company view says what is filtered: which parts of the fetch gate apply to this company, for example "every role in Europe, not filtered by country" for an All locations company. Looking up a company never silently hides roles elsewhere. +- Show gated lists the roles the fetch gate rejects, each with the reason, so what is kept out can be checked. It is the one way a rejected role is shown. ## 6 After the ranking -Summarise is not part of any score. It runs after the feed scores, on the top N of the `want feed score` and the top N of the `match feed score` (N set per agent, 30 to start, so at most 60 roles), and on every role liked or saved. +Summarise is not part of any score. It runs after the feed scores, on the top N of the `want feed score` and the top N of the `match feed score` (N set per agent, 30 to start, so at most 60 roles), and on every role liked or saved. It writes about 60 words and 3 to 8 tags, once per role, and needs the role's text. ## Settings Per agent, on the form: -- name and prompts +- name, an On/Off switch, and prompts - the fetch gate: world (the outer countries), location (countries, towns, remote), role exclusion (titles to leave out), role fetch (fetch terms) - CV, languages, likes and dislikes - the analytics filter: count and minimum by prompt closeness, count and minimum by CV closeness @@ -202,3 +213,5 @@ Per agent, on the form: - summarise size Per agent, per company: affinity (hidden, neutral, liked, favourite). Per company, for every agent: All locations and All roles. + +The pull runs every 6 hours.