Fold the readers plan into the decisions and delete it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Opus 5.5 committed 2026-09-30 10:06:50 +02:00
1 parent ca80219bbb
commit b567144612
3 files changed
+18 -183

No files matched your search

+17 -3
View File
@@ -68,7 +68,7 @@ What was decided for roledesk and the one reason for each. The longer reasoning
**Jobindex and Adzuna, not Jobnet or Jooble.** Jobindex, Denmark's largest board, has an RSS twin for every search page, with phrase and required-word syntax and a region id; no key. Adzuna has a documented API with a free key for the UK and most of the EU, but no Denmark. Jobnet's search endpoint answers 401 outside its own site's session, and Jooble gives a key only on a manual application and answered a Cloudflare challenge. LinkedIn and Indeed have no public search API.
**Platsbanken is the third search provider, for Sweden.** The JobTech API (`jobsearch.api.jobtechdev.se/search`) needs no key, states a total, so completeness follows Adzuna's rule, and filters by the taxonomy id of a län (`PLATSBANKEN_REGIONS`; Skåne is `CaRE_1nn_cSU`). It refuses an offset above 2000, which the page limit never reaches. The ads carry no usable remote field, so `remote` is read off the headline. Free text ranks by all words rather than requiring them; a quoted phrase narrows it.
**Platsbanken is the third search provider, for Sweden.** The JobTech API (`jobsearch.api.jobtechdev.se/search`) needs no key, states a total, so completeness follows Adzuna's rule, and filters by the taxonomy id of a län (`platsbanken.REGIONS`; Skåne is `CaRE_1nn_cSU`). It refuses an offset above 2000, which the page limit never reaches. The ads carry no usable remote field, so `remote` is read off the headline. Free text ranks by all words rather than requiring them; a quoted phrase narrows it.
**A search closes a role only when it listed everything.** A search result is not a complete list, so a role missing from one run has not closed. An Adzuna search listed everything when it read as many distinct results as the `count` it states; a Jobindex search, whose feed states no count, when a page within its page limit came back with fewer than 20 items, counted before the parser drops any, so one malformed item does not pass for the end. Such a search closes like a board; the rest close once no search has listed them for 21 days. A Jobindex answer with no RSS channel raises instead of reading as an empty, complete search. A role two searches found belongs to the last one that listed it, so one search never closes a role the other still lists.
@@ -104,9 +104,23 @@ What was decided for roledesk and the one reason for each. The longer reasoning
**A host is handed the fetch.** A module imports `desk` and `._shared` only and receives `get`, which `ats.py` binds to its own `_get` at call time. The allow-list, the redirect rule, the byte cap and the deadline stay in one place, and a test that patches `ats._get` reaches every host. `_get` refuses an IP address, a local or numeric host name and any port but 443 before it asks, because a Teamtailor or SuccessFactors slug is a host name somebody typed.
**A description stated only on the role's own page is read once, 200 roles per board per pull.** Workday, SuccessFactors, Oracle, Phenom, BambooHR, Jobylon, Rippling and Gem list a role without its text. A module's `detail` is called for roles not yet read, those never tried before those that failed, and the ids read are kept in `ats:detailed:<target>`; a known role is sent without a description on later pulls, which `upsert_role` ignores. `detail` may fill `DETAIL_FIELDS` only (description, location, remote, currency, salary_min, salary_max, posted), the columns a later pull with an empty listing value leaves alone; title, url, department and `salary_text` come from the listing, since the listing rewrites them every pull. Completeness is decided by the listing alone, so a board of 800 roles closes its roles from the first pull and has all its text after four.
**Per-role text is read by the enrich run, ranked roles first.** Workday, SuccessFactors, Oracle, Phenom, BambooHR, Jobylon, Rippling and Gem list a role without its text. The pull reads listings only. After scoring, the `enrich` stage reads the role pages of roles without text that an enabled agent ranks in its top `enrich.top` (500) at `enrich.min_score` (0.5) or more, best score first, 200 per board and 1,000 per run, and only the roles it filled are embedded and scored again. It follows every pull, a saved agent and a star change, not a vote. A read that brought no text is tried again after `retry_days` (7). `detail` may fill `DETAIL_FIELDS` only, the columns a later pull with an empty listing value leaves alone; title, url, department and `salary_text` come from the listing. Completeness is decided by the listing alone. The pull used to read 200 pages per board in the order roles were found, so a board of 800 roles took four pulls to reach the roles that mattered.
**robots.txt is honoured, by RFC 9309.** A module names the page a person opens, and a board whose site disallows that page for `*` or for the app's own token fails with "robots.txt disallows it" and closes its roles after 21 days like any failing board. The matcher takes the longest rule, an Allow on a tie, `*` and `$`, and case-sensitive paths, or paths without case for a host that serves any case, as Workday does. A 4xx means no rules; a 5xx, a 429 or a timeout fails the board for that pull. `urllib.robotparser` was replaced because it takes the first rule and knows no wildcard, which refused Infineon and let `/*company` through. This keeps Novonesis, Darktrace, Criteo, Snyk, Bang & Olufsen, Pandora, Terma, Zenegy and Umbraco off the automatic route. The SuccessFactors reader uses `/search/` and `/job/` only, never `/services/` or a `career*.successfactors.eu` back end.
**robots.txt is decided when a reader is built, not checked at run time.** The builder reads the host's robots.txt, the owner decides per reader whether to read it anyway, and `READS.robots` records the decision and the date. The runtime matcher went with the readers rework. Jobindex is read in full although its robots.txt disallows `page=` and `geoareaid=`; Adzuna's API host disallows `/` for every agent, and whether a keyed API is read regardless is open with the owner; Adzuna has no key configured and has never run. HR Manager's portal disallows everything but the application form; no attached company uses it. The companies once kept off the automatic route by robots.txt (Novonesis, Darktrace, Criteo, Snyk, Bang & Olufsen, Pandora, Terma, Zenegy, Umbraco) stay on the Watchlist with reason `robots` until each is decided. The SuccessFactors reader uses `/search/` and `/job/` only.
**Every source is a reader module that declares what it can do.** The 22 hosts and the 3 search providers are modules in `plugins/readers/`, each with `READS = Reads(...)`: board or search, whole or paged listing, where the text is (listing, excerpt, detail, none), how completeness is known (stated, end, whole, never), page size and the host's ceiling, the filters it takes per dimension (none, one value a request, many, host ids through `facets`), whether it states one place, several, or one posting per place, which fields it states and whether its date is trusted, a key, a provisional title, whether an unknown slug answers an empty board, and the robots decision. Capabilities were implied by which functions a module had, and the search providers sat outside the contract with their own getter. `problems()` checks the declaration against the functions at import, and `tests/test_readers_contract.py` runs every module against fixtures, so a new reader that drops a place or a teaser fails there. The contract is the docstring of `plugins/readers/__init__.py`; the types are `_contract.py`.
**One runner calls every reader.** `plugins/readers/_runner.py` owns the guarded fetch, paging, one request series per filter value unioned by id, completeness, the fallback feed, 429s and failures. `plugins/ats.py` and `plugins/search.py` are thin `Source` classes over it and keep their state prefixes, so no stored key moved. A paged read cut by `MAX_PAGES` saves its cursor and continues on the next pull; the sweep that reaches the end closes what it did not list. A host ceiling (Workday 2,000) is not swept; a filter has to get under it.
**The agents set the reach of a pull.** `app/jobs/reach.py` computes it once per pull: the union of the enabled agents' `any` gates on country, town (as its country) and remote, or everything when one agent has none. A favourite company is read with its agent's `require` reach, Europe today, so a company worth asking is read beyond Denmark and Sweden. A board's `params` override both. Boards under 300 roles, and hosts with a one-country filter over more than 8 countries, are read whole. SmartRecruiters, Eightfold, SuccessFactors, Phenom, Workday and Oracle filter at the source; countries and remote go as separate series, because the hosts intersect two facets. A complete filtered read closes roles outside the filter, except roles with a vote or an application; a filtered read, or one from a host whose unknown slug answers empty, that lists under 10% of the last read with the same filter counts as failed.
**A role has a list of places, resolved against a gazetteer.** Readers kept one place of several, joined them, or the host posted a copy per place. A reader now returns every place and never multiplies a row. `places` (city, region, country, continent, with `place_includes` for Øresund across two countries) and `place_aliases` are seeded from the dictionaries that were in code; `role_places` links each place of a role, and an unknown text is listed by `unresolved_places` until an alias links it. The Place filter and the gates join over the tree, which halved their time and dropped the `EU` fallback. Copies of one posting per place, same host, company and title and texts at least 30% alike, fold into one role that keeps each copy's place and link; the rule applies to every host, since the 5,024 such roles came mostly from hosts that do not declare copies.
**A company has aliases.** `company_aliases` maps a folded name (marks stripped, ø to o, legal form dropped) to a company, so "Orsted" and "Ørsted" are one company without changing `company_key`. `merge_companies` moves roles, subscriptions and ratings. Empty enrichment columns (sector, stage, size band, founded, last funding, source and date) wait for the startupmap.one and The Hub readers.
**A list teaser is kept.** A short description in a listing (Phenom, Oracle, Jobylon, Jobindex, Adzuna) is `roles.teaser`, shown and embedded until the full text arrives, when the vector is dropped and the text replaces it.
**Seniority has three sources.** `seniority_stated` is the host's word mapped by a table in its reader, `seniority_title` is `seniority.from_title`, and `seniority_model` is `extract`'s verdict; `roles.seniority` is the first of model, stated, title. `extract` reads roles without a model value, every agent's first pick before anyone's second, with the other two as hints it may overrule, since host vocabularies differ and 57% of roles state nothing. `app seniority` shows per host how often the stated level agrees with the model.
**No reader is built on a key lifted from a site's script.** Maersk's API, New Work's and Welcome to the Jungle's search index answer only with a key the page's JavaScript carries, which can change without notice and marks an endpoint that is not open. They are on the Watchlist with reason `key`.
+1 -1
View File
@@ -2,7 +2,7 @@
Planned 2026-09-29, after wave 0 and wave 1 of the hosts plan landed. What landed is in `decisions.md` under Ingestion and The phone; what is open without a plan is in `open.md`. This file and `docs/plan-hosts/` are deleted when the packages below land or are dropped.
WP9 (the contract part) and WP11 are superseded by `docs/plan-readers.md`; WP10 and WP12 to WP15 build against its contract once its WP A has landed.
WP9 (the contract part) and WP11 were superseded by the readers rework (see `decisions.md`, Ingestion); WP10 and WP12 to WP15 build against the reader contract in `plugins/readers/__init__.py`.
## Where it stands
-179
View File
@@ -1,179 +0,0 @@
# Plan: readers as plugins with a declared contract
Planned 2026-09-30. Turns every source of roles (the 22 ATS hosts, the 3 search providers, the company-specific readers of `plan-hosts-wave-2.md`) into a reader module with one documented API, and one runner that reads what each reader declares and handles paging, filters, per-role text, limits and failures for all of them. Pattern taken from `../news` (`desk/sources/spec.py`, `docs/plugin-rework.md`): the plugin declares, the application does. Folded into `decisions.md` and deleted when the packages land.
This plan supersedes WP9 (provisional title, as far as the contract goes) and WP11 (board filters) of `plan-hosts-wave-2.md`, and covers items 2, 7, 17 and the "Plugin contract" and "Filtering at the source" sections of `review-ingest.md`.
## How far we are
About two thirds of the way for boards, none of the way for searches.
| Part | Today | Gap |
|---|---|---|
| One module per host | yes, 22 in `plugins/atshosts/`, checked at import by `problems()` | none |
| Documented contract | docstring of `plugins/atshosts/__init__.py` | covers hosts only |
| Capabilities | inferred from which functions exist (`page` means paged, `detail` means per-role text) | not declared; nothing about filters, caps, fields, date trust, places |
| Runner | `plugins/ats.py`: guarded `get`, robots.txt, paging, per-role reads, failure state | fused with the `ats` Source; search does not use it |
| Search providers | three functions in `plugins/search.py` with their own getter | no robots.txt, no size cap, no host allow-list, not modules |
| Row | a bare dict with fixed keys; `ats.py` builds `raw` from a fixed list | a host cannot hand over country, seniority, employment type, several places |
| Date trust | `TRUSTED_POSTED_HOSTS` in `app/models.py` | a host fact kept outside the host |
| Filters at the source | none for boards; searches have per-provider params built in `app/jobs/sources.py:search_params` | no way for a reader to say what it filters, nor for the runner to pass it |
| Oversized boards | read incomplete on every pull (`MAX_PAGES` 100) | no reading across pulls |
| Per-role text | 200 per board per pull, never-tried first, inside the pull | not ordered by score |
## What a reader must announce
The list below is what the 25 readers do today, plus what the wave-2 readers need. The user's list (whole board, paged, list then text, location filter) is complete for the listing. It misses the traits under "also".
| Trait | Values | Who, today |
|---|---|---|
| Scope | `board` (one company, its whole list) or `search` (many companies, never a whole list) | 22 hosts, 3 providers |
| Listing | `whole` (one request) or `paged` | whole: Greenhouse, Lever, Ashby, Recruitee, Personio, Deel, Pinpoint, BambooHR, Jobylon, Rippling. Paged: the other 12 and the 3 providers |
| Text | `listing` (full text in the list), `excerpt` (a teaser in the list), `detail` (one request per role), `none` | detail: Workday, SuccessFactors, Oracle, Phenom, Eightfold, Gem, Jobylon, BambooHR, Rippling. Excerpt: Jobindex, Adzuna. None: SmartRecruiters, Workable |
| Completeness | `stated` (the host gives a total), `end` (an empty page ends it), `whole` (one answer is all), `never` (a search result is never a whole list) | |
| Page size and host ceiling | integers; the ceiling is where the host stops answering whatever we ask | Workday 2,000; Platsbanken offset 2,000; Jobindex 200; Eightfold 10 a page |
| Filters | per dimension (country, place, remote, query): `none`, `one` (one value per request), `many` (a list in one request), `ids` (values must be looked up on the host first) | country one: SmartRecruiters, Eightfold. Many: Phenom. Ids: Workday, Oracle, SuccessFactors. Query: the 3 providers. Greenhouse, Ashby, Recruitee: none |
| Places | `one`, `many`, `copies` (see below) | |
| **Also:** fields stated | pay (`figures`, `line`, `none`), posted (`trusted`, `listed`, `none`), remote, country, seniority, employment | seniority: Personio, SmartRecruiters, Recruitee, Jobylon, none mapped |
| Access | key name (Adzuna), a smaller fallback feed (Greenhouse), the robots.txt decision taken when the reader was built | |
| Title | `final` or `provisional` (a sitemap reader knows a URL before a title) | none yet; WP10 of wave 2 needs it |
| Empty answer | `trusted` or `suspect`: whether an unknown slug answers an empty board | suspect: Ashby, SmartRecruiters, Gem |
Rate limits (429) and the per-request limits stay runner rules, the same for every reader. No reader has a need that differs.
## Several places for one role
A role at "Berlin, Munich, Remote (DE)" arrives in three shapes today:
- **One place of several kept.** Ashby, Lever, Personio, Greenhouse, Workable, Recruitee, Platsbanken read the first and drop the rest. SuccessFactors keeps the last.
- **All places joined in one line.** Teamtailor, Deel, Workday, Phenom, Oracle, Eightfold, Jibe join them with `"; "`. `country` becomes `EU` over several countries, which the country gate does not expand.
- **One posting per place.** The host itself lists copies with their own ids (1,741 groups, 5,021 roles). `dupes.py` leaves them apart.
The rule (owner, 2026-09-30): **a role has a list of places.** A reader never multiplies a row per place and never drops a place. `Row.places` is a list of `Place(text, remote)`. Copies the host publishes per place are a host fact that a reader cannot undo. The reader declares `places="copies"`, and `dupes.py` folds them by company and title into one role, keeping each copy's place with its own outbound link.
The contract test (WP A) feeds every reader that declares `places="many"` a fixture with at least two places and fails the reader if fewer come out.
### Places and companies as symbols
Places and companies become rows that other rows link to, not text matched at query time. Today the place knowledge sits in code (`_CITIES`, `_REGIONS`, `_COUNTRY_NAMES` in `app/models.py`, regions and aliases in `app/places.py`) and is matched word by word against `roles.place_fold` for every query.
- `places(id, name, kind, parent_id)` with `kind` one of `city`, `region`, `country`, `continent`: Copenhagen belongs to Hovedstaden, which belongs to DK, which belongs to Europe. Seeded from the dictionaries in code.
- `place_aliases(fold, place_id)`: `kobenhavn`, `kopenhamn`, `copenhagen` point to Copenhagen, and `skane` and `skåne` to Skåne. Typing either spelling finds the place.
- `role_places(uid, place_id, text, url)`: one row per place of a role. `place_id` is empty for a text the gazetteer does not know. That text is added to an `unresolved` list, which a person or a later job resolves, and it is linked once resolved. `url` is the copy's own link when copies were folded.
- A country filter or gate is a join over the tree: DK expands to every place under DK, and Europe to every country under Europe. The `EU` fallback and the per-request word matching go. This also enables counts per place.
- `roles.place` stays the displayed line: the first place, then "+2" when there are more.
- `company_aliases(fold, company_key)`: "Orsted" and "Ørsted" become one company without changing the primary key. Companies gain empty columns for enrichment (sector, stage, size band, founded, last funding, source and date of each fact), which the startupmap.one and The Hub readers fill later.
## The API
The package `plugins/atshosts/` becomes `plugins/readers/`, and the search providers move into it. The contract stays the package docstring, extended, with `plugins/readers/_contract.py` holding the types and `problems()`. `plugins/README.md` stays the list of sources and their measured limits.
```python
# plugins/readers/workday.py
NAME = "workday"
LABEL = "Workday"
PATTERNS = (r"...",)
READS = Reads(
scope="board",
listing="paged", page_size=20, ceiling=2000,
text="detail",
complete="stated",
filters=Filters(country="ids", remote="ids"),
places="many",
fields=Fields(posted="listed", pay="none", remote="stated"),
robots="read: robots.txt allows /wday/cxs/ (checked 2026-09-29)",
empty="trusted",
)
def slug_ok(slug) -> bool
def allowed(slug) -> Callable[[str], bool]
def careers_url(slug) -> str
# listing="whole"
def feed_url(slug, ask: Ask) -> str
def parse(body, slug) -> list[Row]
# listing="paged"
def page(get, slug, cursor, ask: Ask) -> Page
def rows(decoded, slug) -> list[Row]
# text="detail"
def detail(get, slug, row) -> dict
# filters with "ids"
def facets(get, slug) -> dict[str, dict[str, str]] # {"country": {"DE": "<host id>"}}
# optional, as today
def fallback_url, robots_page, probe_url, probe_ok, probe_slugs
```
`Ask` is what the runner wants from one series of requests: `countries`, `places`, `remote`, `query`, `params` (a search's saved params). A reader reads only the dimensions it declared, and the runner never hands it a value it did not declare. With `one`, the runner makes one series per value and unions them by id. With `ids`, it calls `facets` once a day per board, keeps the result in state and puts host ids into `Ask`. `Row` becomes a `TypedDict` with `places`, and optional `country`, `seniority`, `employment`. The runner puts every stated key into `raw`, so a new field needs no runner change.
`problems()` checks the declaration against the functions: `paged` needs `page` and `rows`, `text="detail"` needs `detail`, an `ids` filter needs `facets`, `scope="search"` needs `complete="never"` or `stated`, `key` needs a config name. A module that fails refuses to import, as today.
`TRUSTED_POSTED_HOSTS` goes: `fields.posted="trusted"` says the same thing in the module.
**robots.txt is decided when a reader is built, not checked at run time** (owner, 2026-09-30). Whoever builds a reader reads the host's robots.txt, reports what it disallows, and the owner decides per reader whether to read the host anyway. `robots` is a required string that records the decision and the date. The runtime check, its state keys and its tests go. Jobindex is read with paging and area. HR Manager, left out because its robots.txt disallows reading, goes back to the owner for a decision.
## The runner
`plugins/readers/_runner.py`, taken out of `plugins/ats.py`. `plugins/ats.py` and `plugins/search.py` stay as thin `Source` classes, so subscriptions and state keys (`ats:*`, `search:*`) keep their names and no migration is needed.
For one target, the runner:
1. Loads the reader and its subscription.
2. Resolves `Ask`. The reach is set in the front end by the agents that are switched on:
- Base reach: the union of their `any` gates on country, town and remote, or everything when one agent has none.
- Favourite companies (favourite = 2, which already skips the `any` gates): only the `require` gates, which today means Europe. Mistral or Databricks are read across Europe, so a role outside Denmark still shows up and can prompt a question to them.
- Board override: `params` on the board's subscription.
The reach is then narrowed to the filters the reader declares. A search's own params go in `Ask.params`. Boards under 300 roles get no filter.
3. Applies the key.
4. Reads the listing: whole, paged, or one series per filter value, unioned by id. Through one guarded `get` for every reader: host allow-list, https only, size cap, deadline, fallback feed. A teaser in the list is kept (`Row.teaser`).
5. **Sweeps across pulls.** A paged read that hits `MAX_PAGES` saves its cursor and the ids seen, and the next pull continues from there. A sweep that reaches the end is complete and closes what it did not list. A host ceiling (Workday 2,000) is not swept: the reader must filter below it or stays incomplete.
6. Decides completeness from the declaration, and applies the guards on closing: a filtered read closes roles outside the filter, except roles with a vote, an application or a note. A filtered or `empty="suspect"` read that returns under 10% of the last count counts as failed.
7. Records failures (`failing` since and reason, 21 days to close), 429 handling and waits, as today.
The runner makes two runs (owner, 2026-09-30):
1. **Listing run** (the pull). The agents hand the reach to the runner. The runner calls each reader with its filters and handles paging and sweeps. Readers return standard rows, with every place of a role. Then `dupes.py` folds copies, and `embed` and `score` run.
2. **Enrich run** (new stage `enrich`). After scoring, the runner is handed the ranked roles. For each role that still needs a call (text through `detail`, and later other per-role facts), it fetches the highest-scored first, within a budget per board and per run. It drops the vector of each role whose text improved, and `embed` and `score` run again for those roles only.
The enrich run follows every rescore that changes the ranking materially: new roles from a pull, an agent created or edited, a company starred or unstarred. A vote on a role does not trigger it. A role never scored above the agents' thresholds gets no extra call. That is the budget which keeps the enrich run from reading every board in full.
### Seniority: stated, then the model
43% of open roles have a level, all taken from title words. A sample of 59 without a level was 85% individual contributors. Personio, SmartRecruiters, Recruitee and Jobylon state a level that nothing reads. `extract` has never run.
A role keeps three sources of a level, and the displayed `roles.seniority` is the first that is filled, in this order:
1. `seniority_model`: the model's verdict, from `extract` run as an afterburner. It reads roles whose model value is empty. The prompt includes the stated and title values as hints, and the model may overrule them.
2. `seniority_stated`: the host's own word, mapped to the app's vocabulary (`app.seniority.SENIORITIES`) by a table per reader in `_shared.py`. The raw word stays in `raw`. A word the table does not know maps to empty and is logged, so the table can grow.
3. `seniority_title`: `seniority.from_title`, as today.
The filters read `roles.seniority`. A later comparison of stated and model values measures how far each host can be trusted.
## Decisions taken by the owner, 2026-09-30
1. **Filters** come from the agents that are switched on. Favourite companies are read across the `require` reach (Europe), and a board can override its filter.
2. **robots.txt** is decided per reader at build time. Nothing checks it at run time. Jobindex is read in full.
3. **Places** are a list per role. Copies per place are folded, each keeping its own link. Places and companies become symbols with aliases.
4. **A teaser is kept.** A short description in the list (Phenom and Oracle: 1,066 roles, plus Jobindex and Adzuna excerpts) goes into `Row.teaser` and `roles.teaser`. Until the full text arrives, it is shown and embedded, and a reader that sees one must hand it over. When the full text arrives, it replaces the teaser for embedding and the vector is dropped, as today.
5. **Seniority** has a stated value from the plugin and a model value from an afterburner that fills what is empty and may overrule it.
## Work packages
Each package owns the files listed and their tests. The planner makes each worktree from local `main` (see memory: isolation worktrees branch from stale origin), merges, and removes the `xfail` marks a package fixed.
| WP | Model | Builds | Owns | Needs |
|---|---|---|---|---|
| A Contract | Opus | rename `atshosts` to `readers`; `_contract.py` with `Reads`, `Filters`, `Fields`, `Ask`, `Place`, `Row` (with `places`, `teaser`), `problems()`; the extended docstring with the robots.txt rule; a conformance test over every module (declaration against functions, `places="many"` and teaser fixtures) marking today's gaps `xfail` per module | `plugins/readers/__init__.py`, `_contract.py`, `_shared.py`, every `import` of `atshosts`, `tests/test_readers_contract.py` | |
| B Runner | Opus | `_runner.py` out of `ats.py`; runtime robots.txt check removed; `Ask` from params; series union; sweep; closing guards; `ats.py` and `search.py` thin; `raw` built from every stated `Row` key | `plugins/ats.py`, `plugins/search.py`, `plugins/readers/_runner.py`, `app/jobs/pull.py`, `tests/test_ats.py`, `tests/test_search.py`, `tests/test_pull.py` | A |
| C Providers | Sonnet | `jobindex.py`, `adzuna.py`, `platsbanken.py` as reader modules with teaser and places; `search_params` built from the declarations | the three modules, `app/jobs/sources.py`, their tests | A |
| D1 Hosts a to l | Sonnet | `READS`, all places, teaser, host-stated seniority mapped to `seniority_stated`, date trust in `fields`, robots decision string for Ashby to Lever | those modules and tests | A |
| D2 Hosts o to w | Sonnet | the same for Oracle to Workday | those modules and tests | A |
| E1 Places and teaser storage | Opus | `places`, `place_aliases`, `role_places`, `roles.teaser`, the three seniority columns, migration seeded from the dictionaries in code; `upsert_role` writes them; gates and the Place filter as joins over the tree; `TRUSTED_POSTED_HOSTS` removed; embed falls back to the teaser | `app/storage/*`, `app/models.py`, `app/places.py`, `app/jobs/embed.py`, their tests | A |
| F Reach and filters | Opus | reach from the agents and favourites into `Ask`; `facets` and `Ask` for SmartRecruiters, Eightfold, Workday, SuccessFactors, Phenom, Oracle; board override on the Companies page; each reader's declaration shown on the Sources page | `app/jobs/reach.py`, those six modules, `app/web/api.py`, `app/static/companies.js`, `app/static/sources.js` | B, D1, D2, E1 |
| G Enrich run | Opus | `enrich` as its own stage over the ranked roles, highest score first; triggered after a pull's scoring, after an agent is saved and after a star changes, not after a vote; re-embed and rescore only the roles it filled | `app/jobs/detail.py`, `app/jobs/cascade.py`, `app/jobs/stages.py`, their tests | B |
| E2 Copies and place list | Sonnet | `dupes.py` folds copies per place into one role with per-place links; the reader shows the place list with links; the teaser is shown when there is no text | `app/dupes.py`, `app/static/roles.js`, `app/templates/*`, their tests | E1 |
| I Seniority | Sonnet | `seniority_stated`, `seniority_model`, `seniority_title` and the precedence; `extract` as an afterburner over roles without a model value, with the hints | `app/seniority.py`, `app/jobs/extract.py`, their tests (the columns come from E1) | E1 |
| H Companies as symbols | Sonnet | `company_aliases`, alias folding on insert and lookup, empty enrichment columns | `app/storage/*` after E1, `app/jobs/sources.py` after C, their tests | E1, C |
Order: A first. Then B, C, D1, D2 and E1 in parallel. Then F, G, E2, H and I in parallel; H adds a migration after E1's. After the merges, the planner attaches Infineon and Microsoft on Eightfold with a country filter, updates `plugins/README.md`, and folds this plan into `decisions.md`.