From 3012afdd57782d72998f4976925ca2d006b8e6da Mon Sep 17 00:00:00 2001 From: Wolf Date: Sun, 4 Oct 2026 08:23:38 +0200 Subject: [PATCH] Write the ranking flow as one shared picture: fetch, cheap pass, second fetch, model pass, feed, with what is decided, proposed and open Co-Authored-By: Claude Opus 5.5 --- docs/ranking-flow.md | 90 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 docs/ranking-flow.md diff --git a/docs/ranking-flow.md b/docs/ranking-flow.md new file mode 100644 index 0000000..04d0126 --- /dev/null +++ b/docs/ranking-flow.md @@ -0,0 +1,90 @@ +# Ranking flow + +This is how a role gets from a board to the Roles feed, for one search agent. It is the shared picture to change the flow against. Each stage says what the code does **today**, what was **decided** on 2026-10-04 (not built yet), what is **proposed** and what is still **open**. + +``` + company boards (ATS) search providers + reach: Where lines + fetch terms, their own saved query + only boards ≥300 roles whose host filters + │ │ + ▼ ▼ + 1 Fetch ─ list pages: title, place, teaser or text + │ + ▼ + 2 Cheap pass, no tokens ─ prep, embed, fit to the prompts, closeness to the CV, gates + │ + ▼ analysis score → the cut (at most N, each at least X) + 3 Second fetch ─ enrich reads the role's own page for cut roles without text → embed, rescore + │ + ▼ + 4 Model pass (Haiku) ─ extract, summarise, judge, on the cut only + │ + ▼ + 5 Feed ─ two orders: "what I'm looking for" (final score) and "what fits me" (CV match) + front-end filters narrow either order +``` + +## 1 Fetch + +**Today.** The Where lines become the reach: a country and remote filter the host applies. This only happens on boards with 300 or more roles whose host can filter by place. Smaller boards, and hosts that cannot filter, are read whole. Fetch terms only apply to companies set to All countries (Mistral AI and OpenAI today), and only on hosts that take a query (Workday, SuccessFactors, Oracle, Phenom, Eightfold, SmartRecruiters). Search providers send their own saved query and take neither filter. + +**Decided.** Where lines and fetch terms become one block on the agent form and follow one rule: each applies to every board with 300 or more roles whose host can filter on it. Fetch terms no longer depend on All countries. Agent 2 has no fetch terms today, so nothing changes on the live server until some are set. + +**Consequence.** The pull never closes a role against a read narrowed by terms, because that read is not the whole board. With terms on every big board, roles there close only by age. Wolf ranks on recency, not on `closed_at`, so this is acceptable unless he says otherwise. + +## 2 Cheap pass + +**Today.** Every open role gets one score per agent: `(0.6 fit + 0.2 newness + 0.2 company affinity + 0.2 taste + prefer gates) / sum of the weights`. Fit is the calibrated cosine to the best of the agent's prompts. The gates (Where lines, titles to leave out) set a failing role's score to 0 with a reason. A gate rejects on evidence only, so a role with no known location still ranks until extract fills it in (323 of agent 2's 7,263 ungated roles). Closeness to the CV (`cv_fit`) is stored beside the score but is not part of it. That one score drives enrich, extract, summarise, judge and the feed order. + +**Decided.** The score is split in two: + +- The **analysis score** decides which roles get a second fetch and a model read. It uses fit to the prompts, closeness to the CV and company affinity, so a liked company gets more of its roles read. It leaves out newness (a pull runs every 6 hours and a role is read once), taste (votes come after the read) and prefer gates. +- The **final score** orders the feed (section 5). + +Prefer gates leave the agent. Hybrid becomes a feed filter. + +**Proposed: location guards the analysis cut.** For agent 2 on 2026-10-04 (24,711 open roles): + +| Gates on the cut | Roles passing | Fit ≥ 0.6 | Inside today's Where, of the top 500 by fit | Inside today's Where, of the top 150 | +|---|---|---|---|---| +| Today: Europe, Where lines, titles | 7,263 | 2,086 | 500 | 150 | +| Europe and titles, no Where lines | 14,704 | 4,855 | 118 | 28 | +| Titles only | 22,297 | 9,161 | 66 | 18 | + +Without the Where lines, about four in five model reads go to roles outside DK, SE, the Skåne towns and remote. Keeping today's coverage would then take a budget about four times larger. Companies on the follow-everywhere list (section 5) skip the Where lines, as favourites do today. + +**Open.** + +- How fit and CV closeness combine into one cut. Today the judge reads the best 150 by score plus the 50 closest to the CV. Options are the higher of the two levels, or the union of two top lists. +- How affinity enters: as a weighted term, or by widening the cut for liked companies. +- Whether a role liked or saved outside the cut is pushed into the model pass. Today saved and liked roles are summarised whatever their score. + +## 3 Second fetch + +**Today.** Enrich reads the role's own page for open roles without text, on hosts that list without it. It uses its own bar from the config (`enrich.top` 500, `enrich.min_score` 0.5, at most 200 per board and 1,000 per run). Embed and score run again for the roles it filled. + +**Decided.** Enrich reads the analysis cut, so every role the model will read has its text first. + +## 4 Model pass + +**Today.** These three stages run on Haiku. Each limit is a count *and* a minimum score: at most N roles, each scoring at least X. The count is a ceiling and the score is the real stop. + +- **Extract** reads at most 500 roles scoring at least 0.55. It fills location, remote area, seniority, languages and deadline. A rescore follows so the gates see these columns. +- **Summarise** reads at most 30 roles scoring at least 0.65, plus the roles you saved or liked. +- **Judge** reads at most 150 roles scoring at least 0.55, plus the 50 closest to the CV that those leave out. It scores skills, experience and career 0 to 100, plus a verdict on each must-have. These are weighted 30/25/30 into `cv_score`. + +**Decided.** The form wording becomes "at most N, each scoring at least X", in place of "the best N down to a score of X". + +**Open.** Whether extract and judge become one call. Arguments for keeping two: extract is per role and needs no CV, while judge is per agent and role with the CV in the system prompt; and extract's location and remote answers feed the gates before judge reads anything. The argument for one call is that the posting is read once instead of twice. + +## 5 Feed + +**Today.** There are two orders. **Score** uses the same score as section 2. **CV match** shows the judged roles first by `cv_score`, then the unjudged ones by CV closeness. Roles gated by the agent are hidden unless Show gated is ticked, so the front-end place filter can only narrow inside the Where lines. A favourite company (affinity 2) skips the Where lines but also gains in the score; that is why OpenAI's London roles rank. All countries and All roles are separate company flags that only change what is fetched. + +**Decided.** There are two orders: "what I'm looking for" (final score) and "what fits me" (CV match). The final score includes newness and likes. It is not one blended number. + +**Open.** + +- What the final score is made of: whether fit stays at 0.6, and whether it reads extracted facts such as seniority. +- Location in the feed: does the agent's Where keep hiding roles, or is location only a front-end filter on top of the analysis cut? +- A follow-everywhere company group. It is shown as a feed filter, skips the Where lines in the cut, and is fetched across all countries without fetch terms. It would replace three things: favourite as a workaround, All countries and All roles.