Fold the prep plan into the decisions, keep the filter survey as reference
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
1538c776d6
commit
795ca3628c
5 files changed
+9
-80
No files matched your search
+5
-1
@@ -120,7 +120,11 @@ What was decided for roledesk and the one reason for each. The longer reasoning
|
||||
|
||||
**A list teaser is kept.** A short description in a listing (Phenom, Oracle, Jobylon, Jobindex, Adzuna) is `roles.teaser`, shown and embedded until the full text arrives, when the vector is dropped and the text replaces it.
|
||||
|
||||
**Seniority has three sources.** `seniority_stated` is the host's word mapped by a table in its reader, `seniority_title` is `seniority.from_title`, and `seniority_model` is `extract`'s verdict; `roles.seniority` is the first of model, stated, title. `extract` reads roles without a model value, every agent's first pick before anyone's second, with the other two as hints it may overrule, since host vocabularies differ and 57% of roles state nothing. `app seniority` shows per host how often the stated level agrees with the model.
|
||||
**Readers scrape; `prep` decides.** A reader hands on the host's own words (`seniority`, `employment`, `remote`, and `country` only when the host states one per role) and does light cleanup only: text from HTML, dates, the places of a role as a list, pay figures. Mapping lived in 25 readers, each spelling its tables its own way, so changing what "Experienced" means touched a plugin. The `prep` stage (`app/prep/`) maps centrally after every pull and every enrich run. Seniority, employment and remote go through the host's table in `app/prep/hosts/<reader>.py` (whole words, or `MATCH` fragments by substring or token), then a shared word map. Country comes from the resolved places (several European countries give `EU`), and the host's country is only a fallback. Places and company aliases are resolved, and copies and duplicates are folded over the whole database. `upsert_role` stores the host words (`*_host` columns, migration 22) and marks the role `prep_due`. When a table changes, its fingerprint makes the next prep map every open role again from the stored words, with no pull. A word no table knows is listed in `unmapped` (`app prep unmapped`, the Sources page). A host's `remote: False` stays unknown rather than onsite, because SmartRecruiters, Ashby, Recruitee and Workable send it for hybrid roles too.
|
||||
|
||||
**Fetch terms narrow what big boards send across Europe.** An agent's `fetch_terms` (migration 23) are words a big board must match when it is read with the wide reach: a company tagged All roles, or a one-country host whose reach was dropped for needing over 8 requests. The terms of a pull are the union over the enabled agents; one enabled agent without terms switches them off. The narrow reach takes no terms, because it already leaves at most about 150 roles on any big board. The survey of 2026-09-30 found that every host's keyword matches the whole text, so a term cuts less than a title filter would ("engineer" left 1,713 of Bosch's 4,830). SmartRecruiters, SuccessFactors, Workday, Phenom, Eightfold and Oracle take one term a request, with one series per term unioned by id; over 8 series the place filter goes first, then the terms. A term-filtered read is never the whole board: it writes no size, prunes no per-role state and closes nothing. Category and level filters wait until something maps agent terms to host ids. A town in a gate is still fetched as its country.
|
||||
|
||||
**Seniority has three sources.** `seniority_stated` is the host's word mapped by that host's table in `app/prep/hosts/`, `seniority_title` is `seniority.from_title`, and `seniority_model` is `extract`'s verdict; `roles.seniority` is the first of model, stated, title. `extract` reads roles without a model value, every agent's first pick before anyone's second, with the other two as hints it may overrule, since host vocabularies differ and 57% of roles state nothing. `app seniority` shows per host how often the stated level agrees with the model.
|
||||
|
||||
**No reader is built on a key lifted from a site's script.** Maersk's API, New Work's and Welcome to the Jungle's search index answer only with a key the page's JavaScript carries, which can change without notice and marks an endpoint that is not open. They are on the Watchlist with reason `key`.
|
||||
|
||||
|
||||
Reference in new issue
Block a user