Delete the ranking flow plan: its four packages are merged and its reasoning is in decisions.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Opus 5.5 committed 2026-10-04 15:37:18 +02:00
1 parent 553042f1fc
commit f7917a6fad
1 file changed
-198
-198
View File
@@ -1,198 +0,0 @@
# Plan: ranking flow
This plan takes the code from what it does **today** to the target in `architecture.md`. Each stage gives today's behaviour, what was **decided** on 2026-10-04 and the measurements behind it. Work packages follow once it is cut into them. When the work lands, the reasoning is folded into `decisions.md` and this file is deleted.
## Three scores
Each agent decides about a role three times: a gate on what comes in, a gate on what the models read, and an order. Everything after that is the owner's filters, saved per agent.
- **Fetch gate**, a yes or no: the agent's hard rules. These are the required countries (Europe), the Where lines (countries, towns, remote) and the titles to leave out. A company set to **All roles** passes everything but the required countries: its roles skip the Where lines and the titles to leave out. The gate applies at every stage: at the host when the host can filter, in the cheap pass, again after extract (now with the real location and remote area), and in the feed. A role that fails it is never read by a model and never shown.
- **Analysis score**: decides which roles that pass the fetch gate make the **cut**, the roles that get fetch text and the model pass.
- **Final score**: orders the feed.
Names: **fetch text** is the second HTTP read of a role's own page, with no model (the code calls it `enrich`). **Extract**, **judge** and **summarise** are the model calls. **Fetch terms** are words a big board must match to send a role; they narrow the read at the host and are not part of the fetch gate.
```
company boards (ATS) search providers
fetch gate at the host + fetch terms, their own saved query
only boards ≥300 roles whose host filters
│ │
▼ ▼
1 Fetch ─ list pages: title, place, teaser or text
│
▼
2 Cheap pass, no tokens ─ prep, embed, fit to the prompts, closeness to the CV, fetch gate
│
▼ analysis score → the cut
3 Fetch text ─ the role's own page, for cut roles without text → embed, rescore
│
▼
4 Model pass (Haiku) ─ extract the cut → fetch gate and cut again → judge the cut
│
▼
5 Feed ─ every role that passes the fetch gate, in two orders:
"what I'm looking for" (final score) and "what fits me" (CV match);
front-end filters and saved filters narrow it
│
▼
6 After the ranking ─ summarise the top of the feed
```
## 1 Fetch
**Today.** The Where lines become the reach: a country and remote filter the host applies. This only happens on boards with 300 or more roles whose host can filter by place. Smaller boards, and hosts that cannot filter, are read whole. Fetch terms only apply to companies set to All countries (Mistral AI and OpenAI today), and only on hosts that take a query (Workday, SuccessFactors, Oracle, Phenom, Eightfold, SmartRecruiters). Search providers send their own saved query and take neither filter.
**Decided.** The fetch gate's location part and the fetch terms become one block on the agent form and follow one rule: each applies to every board with 300 or more roles whose host can filter on it. Fetch terms no longer depend on All countries. Agent 2 has no fetch terms today, so nothing changes on the live server until some are set. The titles to leave out cannot be asked of a host, so that part of the fetch gate starts at the cheap pass.
**Consequence.** The pull never closes a role against a read narrowed by terms, because that read is not the whole board. With terms on every big board, roles there close only by age. Accepted, since Wolf ranks on recency, not on `closed_at`.
## 2 Cheap pass
**Today.** Every open role gets one score per agent: `(0.6 fit + 0.2 newness + 0.2 company affinity + 0.2 taste + prefer gates) / sum of the weights`. Fit is the calibrated cosine to the best of the agent's prompts. The gates (Where lines, titles to leave out) set a failing role's score to 0 with a reason. A gate rejects on evidence only, so a role with no known location still ranks until extract fills it in (323 of agent 2's 7,263 roles that pass). Closeness to the CV (`cv_fit`) is stored beside the score but is not part of it. That one score drives enrich, extract, summarise, judge and the feed order.
**Decided.** The one score is split into the analysis score (this section) and the final score (section 5). The analysis score uses fit to the prompts, closeness to the CV and company affinity. It leaves out newness (a pull runs every 6 hours and a role is read once), taste (votes come after the read) and prefer gates. Prefer gates leave the agent; hybrid becomes a feed filter.
**Decided: the cut is two lists, joined.** One list is by prompt fit. The other is by CV closeness: the roles closest to the CV that the first list leaves out. The two are not merged into one number, because they answer different questions (what I want, what I am), and a merged number would let the stronger side crowd the other out. Each list has a count and a minimum level on the form. For a broad agent the count stops the list; for a narrow agent the minimum does, so roles that do not fit are not read just to fill the count. Starting values: 500 by prompt fit and 150 by CV closeness, each at least 0.6. The minimums are levels of plain fit and plain CV closeness, not of today's blended score. Among agent 2's roles that pass the fetch gate, 2,086 reach fit 0.6, 1,580 reach CV closeness 0.6, and 2,437 reach either, so for agent 2 the counts decide.
**Decided: affinity is a bonus on the level, as today in spirit.** A role's level on both lists rises by the company's affinity. Today the affinity term is weighted 0.2 out of 1.25, which puts a liked company about 0.05 above a neutral one and a favourite about 0.1 above. The cut keeps those sizes. The four company settings are hidden (-1), neutral (0), liked (1) and favourite (2). A hidden company is not scored at all, so it never reaches the cut or the feed. There is no level that pushes a company down without hiding it, and none is planned. Affinity remains a nudge, not a pass: a favourite's role that matches nothing stays out of the cut. Favourite only raises the level. Today it also skips the Where lines; that moves to All roles (section 5).
**Decided: a role you like or save always makes the cut.** This is part of the analysis score, not a separate manual path: a like or a save passes the cut whatever the role's levels, so the role gets fetch text, extract and judge. It goes through on the next pull or agent save, not the moment you click.
**Decided: the cut only takes roles that pass the fetch gate, location included.** For agent 2 on 2026-10-04 (24,711 open roles):
| Rules applied before the cut | Roles passing | Fit ≥ 0.6 | Inside today's Where, of the top 500 by fit | Inside today's Where, of the top 150 |
|---|---|---|---|---|
| Today: Europe, Where lines, titles | 7,263 | 2,086 | 500 | 150 |
| Europe and titles, no Where lines | 14,704 | 4,855 | 118 | 28 |
| Titles only | 22,297 | 9,161 | 66 | 18 |
Without the Where lines, about four in five model reads go to roles outside DK, SE, the Skåne towns and remote, and keeping today's coverage would take a budget about four times larger.
## 3 Fetch text
**Today.** The `enrich` stage reads the role's own page for open roles without text, on hosts that list without it. It uses its own bar from the config (`enrich.top` 500, `enrich.min_score` 0.5, at most 200 per board and 1,000 per run). Embed and score run again for the roles it filled.
**Decided.** Fetch text reads the whole analysis cut, so every role the model will read has its text first. The stage is renamed from `enrich` to fetch text so it is not mistaken for a model call.
## 4 Model pass
**Today.** Extract and judge run on Haiku (summarise is in section 6). Each has a count and a minimum score on the blended score; for agent 2 the count decides every stage.
- **Extract** reads at most 500 roles scoring at least 0.55. It fills location, remote area, seniority, languages and deadline. A rescore follows so the gates see these columns.
- **Judge** reads at most 150 roles scoring at least 0.55, plus the 50 closest to the CV that those leave out. It scores skills, experience and career 0 to 100, plus a verdict on each must-have. These are weighted 30/25/30 into `cv_score`.
**Decided.** One depth: the cut is the only selection, and both scoring model stages read all of it. A second score inside the cut would defeat its purpose. The order is:
1. **Extract** reads the whole cut. It is the small, fast call (about 1.5 s a role on the live backend).
2. The fetch gate and the cut are applied again with what extract filled, now knowing which roles are really remote and where the rest are. Some roles drop out; some enter, such as roles that turn out to be remote. A role that enters is extracted on the next pull.
3. **Judge** reads the cut as recomputed (about 6 s a role).
**Decided: extract and judge stay two calls.** Extract needs no CV and its location and remote answers feed the fetch gate before judge reads anything. Judge works per agent and role, with the CV in the system prompt.
**Cost.** Every model read happens once per role (judge once per role and CV), so a larger cut costs mostly a one-off catch-up. With a cut of about 650 roles, that catch-up is about 16 minutes of extract and about an hour of judge on the live backend (`claude_code`, the subscription, so no per-token bill). After it, each pull reads only the roles new to the cut. The recurring cost is a re-judge: changing the CV, a prompt, or the like or dislike text changes every judgement's input, so the whole cut is judged again, about an hour for 650 roles. A changed posting text re-judges that role only. Accepted: the re-judge takes a while but is correct, since the old judgements were made against the old text.
## 5 Feed
**Today.** There are two orders. **Score** uses the same score as section 2. **CV match** shows the judged roles first by `cv_score`, then the unjudged ones by CV closeness. Roles that fail the gates are hidden unless Show gated is ticked, so the front-end place filter can only narrow inside the Where lines. A favourite company (affinity 2) skips the Where lines but also gains in the score; that is why OpenAI's London roles rank. All countries and All roles are separate company flags that only change what is fetched.
**Decided.**
- The feed shows every role that passes the fetch gate, loaded as you scroll, not only the cut. Newest first stays inside the fetch gate as well, so no order shows roles outside the agent's places.
- There are two orders: "what I'm looking for" (final score) and "what fits me" (CV match). The final score includes newness and likes. It is not one blended number.
- Front-end filters only narrow further. The Roles page says that the feed is already filtered by the agent's fetch gate, location included.
- **All countries** and **All roles** stay two company switches, now defined as overrides of the fetch gate: All countries bypasses the place part (Where lines) and All roles the role part (fetch terms and titles to leave out); the region (Europe) always applies. Neither changes any score: favourite is what pushes a company's roles up. Both apply to every agent, as today. Use case: small companies worth a speculative approach ("have you thought about hiring in Denmark?").
- The company view says plainly what is filtered: for an All roles company, "every role in Europe, not filtered by country or title"; for any other company, that it shows the roles the agent's fetch gate passes. Looking up a company must not silently hide roles in other countries.
- **Saved filters**, per agent. The current filter settings can be saved under a name, loaded with one click and deleted. There is no editing: to change one, save a new one and delete the old.
**Decided: the final score** is today's formula without the prefer gates: 0.6 fit to the prompts, 0.2 newness, 0.2 company affinity, 0.2 roles you liked, over the sum of the weights. Everything else, such as seniority, location and pay, is a filter, not a score term, so a role is never judged twice by the score and by a filter.
## 6 After the ranking
**Today.** Summarise reads at most 30 roles scoring at least 0.65, plus the roles you saved or liked. It runs inside the model pass, on the same score.
**Decided.** Summarise is not part of the preparation or the scoring. It runs once the final scores are set, on the top of the feed by the final score, with its own size on the form (50 to start, as a slider). Roles you saved or liked are summarised too, as today.
## Planner choices
Two points the architecture leaves open, settled by the planner so the build can start:
- On a narrow read (the agents' places), when fetch terms times the place series exceed `MAX_SERIES`, the terms are dropped and the places kept, because the places are what the agent's fetch gate needs. A wide read keeps today's rule: the place part is dropped and the terms kept.
- Stored agent limits are reset to the new defaults by the migration. The old bars were measured on the blended score and mean nothing on plain fit.
## Contract between the packages
1. **`Limits`** keys: `cut_fit_top` 500, `cut_fit_min` 0.6, `cut_cv_top` 150, `cut_cv_min` 0.6, `summarise_top` 50. `extract_*`, `judge_*` and `summarise_min` are removed.
2. **`Gate`**: modes `require`, `any`, `exclude`. `prefer` and the `weight` field are removed. `Weights` is unchanged.
3. **`scores`** gets `fit REAL` (the prompt fit level, before the bonus) and `cut TEXT NOT NULL DEFAULT ''`, one of `fit`, `cv`, `kept` or empty. `score` is the final score; `cv_fit` stays.
4. **Analysis levels**: `bonus = w.affinity * affinity / 3 / (w.fit + w.recency + w.affinity + w.taste)`; the fit level is `fit + bonus` and the CV level `cv_fit + bonus`. Per agent, over roles that pass the fetch gate: the best `cut_fit_top` by fit level at or above `cut_fit_min` are `fit`. Then the best `cut_cv_top` by CV level at or above `cut_cv_min`, among the roles not already in, are `cv`. Roles this agent liked, and saved roles (`applications.role_uid`), not already in, are `kept`. Ties break on `posted_at` DESC, then `uid`.
5. **Fetch gate in scoring**: `require` gates always apply. The `any` group is skipped for a company with `companies.all_countries = 1` and no longer for a favourite. `exclude` gates on `title` are skipped for a company with `companies.all_roles = 1`. A hidden company is not scored.
6. **Final score**: today's `score_one` without prefer terms.
7. **Stage selection**:
- fetch text: open roles without text, in the cut of any enabled agent, within the existing per-board, per-run and retry budgets.
- extract: waiting roles in the cut of any enabled agent.
- judge: `(agent, role)` for roles in that agent's cut, for an enabled agent with a CV and a prompt.
- summarise: roles without a summary that have text, and are either in the best `summarise_top` by final score of an enabled agent with a prompt, or saved, or liked.
8. **Cascades**:
- `pull`: pull, prep, embed, score, fetch_text, prep, embed, score, extract, score, judge, summarise.
- `fetch_text` (after an agent save or an affinity change): fetch_text, prep, embed, score, extract, score, judge, summarise.
- The skip past prep, embed and score when fetch text filled nothing stays.
9. **Rename `enrich` to fetch text**:
- Python: module `app/jobs/fetch_text.py`; job kinds `fetch_text` and `fetch_text_batch`; `enrich_after` becomes `fetch_text_after`.
- Stage switch: name `fetch_text`, state key `stage:fetch_text`, and the `/api/stages` names extract, summarise, judge, fetch_text.
- Config: section `fetch_text:` with `enabled`, `per_board`, `per_run` and `retry_days`; `top` and `min_score` are removed.
- Job label: "Fetching the text of the roles in the cut".
10. **Feed**: when the request names an agent, every order, newest included, joins `scores` and hides roles that fail the fetch gate unless `gated=1`.
11. **Saved filters**:
- Table `saved_filters(id INTEGER PRIMARY KEY, agent_id INTEGER NOT NULL REFERENCES agents(id) ON DELETE CASCADE, name TEXT NOT NULL, query TEXT NOT NULL, created_at TEXT NOT NULL)`, with names unique per agent regardless of case.
- `GET /api/agents/{id}/filters` returns `[{id, name, query, created_at}]` ordered by name.
- `POST /api/agents/{id}/filters {name, query}` returns 201 with the row; 409 for a duplicate name, 404 for an unknown agent, 422 for an empty name.
- `DELETE /api/filters/{id}` returns 204, or 404.
- `query` is the feed's query string as `roles.js` `addressFor` writes it, without the agent.
12. **Migration 32**, in one step:
- rewrite every `agents.limits` to the new defaults;
- drop prefer gates and every gate's `weight` from `agents.gates`;
- add the two `scores` columns and the `saved_filters` table;
- rename the state key `stage:enrich` to `stage:fetch_text`;
- delete queued `enrich` and `enrich_batch` jobs.
## Work packages
Wave 1 runs in parallel. Wave 2 starts once wave 1 is merged into main. Each package works in its own worktree, made from main by the planner, and does not merge or rebase.
### WP1: ranking core, in Python (Opus), wave 1
Owns `app/models.py`, `app/storage/schema.py`, `app/storage/store.py`, `app/config.py`, `app/cli.py`, `app/web/api.py`, `app/jobs/{score,extract,judge,summarise,enrich,cascade,stages,posting,prep}.py`, `config.example.yaml`, `plugins/ats.py`, `plugins/readers/__init__.py` and `plugins/README.md` (the enrich wording only), `README.md`, and every Python test except those WP2 owns. It builds contract points 1 to 12 with tests, including a test that loading an agent stored before migration 32 works.
### WP2: fetch gate at the host (Sonnet), wave 1
Owns `app/jobs/reach.py`, `plugins/readers/_runner.py`, `app/jobs/limits.py`, the Limits options in `app/templates/settings/sources.html`, and `tests/test_{fetch_terms,overrides,reach,runner,web_reach,limits,readers_filters}.py`.
- Fetch terms go to every board with 300 or more roles (`why` is `agents` or `countries`) whose host takes `query="one"`, unless the company is set to All roles. `ask_for` no longer requires `wide`.
- Apply the planner's choice on series.
- Update the reach line, the docstrings and the limit words: "text per role" now says fetch text reads each role's page for the roles in the cut.
### WP3: settings pages (Sonnet), wave 2
Owns `app/templates/settings/agents.html`, `app/static/agents.js`, `app/static/remote.js`, `tests/js/remote.test.mjs`, the Runs and Agent match wording in `app/templates/settings/sources.html`, `app/static/jobs.js`, `app/static/companies.js`, the overrides text in `app/static/app.js`, and the form assertions in `tests/test_web.py`.
- The agent form follows Settings in `architecture.md`:
- a fetch gate block in three parts: Region (scope), Place (countries, towns, remote), Role (fetch terms, titles to leave out);
- the feed order weights and the CV match weights;
- the cut: the best N by fit, each at least X; the best M by CV closeness not already in, each at least Y; liked and saved roles always in;
- summarise the top N of the feed.
- Hybrid and prefer leave the form. The card lines follow the new limits.
- The tooltips say what All countries (place part) and All roles (role part) bypass.
- The Runs page names fetch text.
### WP4: Roles feed (Opus), wave 2
Owns `app/templates/roles.html`, `app/static/roles.js`, `app/static/app.css`, `app/static/chips.js`.
- A rail note under Viewing as says the feed is already filtered by the agent's fetch gate, places included.
- Show gated is offered in every order when an agent is chosen.
- Saved filters in the rail, per agent: save the current filters under a name, apply with one click, delete. Patterns come from `../news` (the `#active` pills).
- The company strip says which parts of the fetch gate apply to the company, and no longer says that a favourite skips the places.
## Done when
`uv run pytest` and `node --test tests/js/*.test.mjs` pass on main, every statement in `architecture.md` holds, the reasoning is folded into `decisions.md`, and this plan is deleted. The live server picks the change up on Wolf's restart. Migration 32 runs then, and the first pull judges the whole cut (about an hour).