Split the All tag into two company overrides: All countries and All roles (WP L)
Fetching has two filters, and a company now overrides each on its own.
All countries (companies.all_countries) is the old All roles tag: its big
boards are read with the agents' wide reach. All roles (companies.all_roles,
new meaning) drops the agents' fetch terms and any later role filter.
- Migration 24 copies the old tag to all_countries and keeps all_roles for
those companies, and moves a board's {"terms": false} to its company.
- The runner no longer reads {"terms": false}; the plan leaves the terms out
for a company set to All roles.
- POST /api/companies/{key}/all-countries and /all-roles, each queueing a
pull when turned on; CompanyOut carries both.
- Two toggles replace All on the Companies row, the reader and the Roles
strip; the reach line states countries and role types in plain words;
the filter select gets both options.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
795ca3628c
commit
e52857dea5
23 files changed
+602
-215
No files matched your search
+2
-2
@@ -112,7 +112,7 @@ What was decided for roledesk and the one reason for each. The longer reasoning
|
||||
|
||||
**One runner calls every reader.** `plugins/readers/_runner.py` owns the guarded fetch, paging, one request series per filter value unioned by id, completeness, the fallback feed, 429s and failures. `plugins/ats.py` and `plugins/search.py` are thin `Source` classes over it and keep their state prefixes, so no stored key moved. A paged read cut by `MAX_PAGES` saves its cursor and continues on the next pull; the sweep that reaches the end closes what it did not list. A host ceiling (Workday 2,000) is not swept; a filter has to get under it.
|
||||
|
||||
**The agents set the reach of a pull.** `app/jobs/reach.py` computes it once per pull: the union of the enabled agents' `any` gates on country, town (as its country) and remote, or everything when one agent has none. A company tagged **All roles** (`companies.all_roles`, migration 21) is read with the wide reach: the union of the enabled agents' `require` reach, Europe today, and the union of the `any` gates, so a tag never reads less than the board would untagged. An agent with no `require` gate on place makes the wide reach everywhere. For a company the owner cares about, a role in Freiburg may be open to remote, or worth asking about for Denmark, while Microsoft Germany is not wanted. The tag is separate from favourite and liked, and global rather than per agent, because it decides what is fetched and fetching is shared. Favourite (affinity 2) affects scoring only, where it skips the `any` gates; it used to widen the fetch to its agent's `require` reach, and migration 21 tagged every company that was a favourite of any agent so nothing read before is lost. The tag also replaced a board's `{"reach": "all"}` param, which read everywhere: migration 21 tagged those companies and logged each such board, since no everywhere level is left. A board's `params` override the agents' reach and the tag. Boards under 300 roles, and hosts with a one-country filter over more than 8 countries, are read whole. SmartRecruiters, Eightfold, SuccessFactors, Phenom, Workday and Oracle filter at the source; countries and remote go as separate series, because the hosts intersect two facets. A complete filtered read closes roles outside the filter, except roles with a vote or an application; a filtered read, or one from a host whose unknown slug answers empty, that lists under 10% of the last read with the same filter counts as failed.
|
||||
**The agents set the reach of a pull.** `app/jobs/reach.py` computes it once per pull: the union of the enabled agents' `any` gates on country, town (as its country) and remote, or everything when one agent has none. Fetching has two filters, countries and role types, and a company has an override for each. A company set to **All countries** (`companies.all_countries`, migration 24) is read with the wide reach: the union of the enabled agents' `require` reach, Europe today, and the union of the `any` gates, so the override never reads less than the board would without it. An agent with no `require` gate on place makes the wide reach everywhere. For a company the owner cares about, a role in Freiburg may be open to remote, or worth asking about for Denmark, while Microsoft Germany is not wanted. The overrides are separate from favourite and liked, and global rather than per agent, because they decide what is fetched and fetching is shared. They matter only for a board of 300 roles or more on a host that filters, so a small company is affected once it grows past that. The Companies list states both halves per board, e.g. "follows agents: DK, SE, remote · all roles" or "all countries (Europe) · follows agents' terms: engineer, product, data", or why the board is read whole. Favourite (affinity 2) affects scoring only, where it skips the `any` gates; it used to widen the fetch to its agent's `require` reach, and migration 21 tagged every company that was a favourite of any agent so nothing read before is lost. That tag, first named All roles in `companies.all_roles`, also replaced a board's `{"reach": "all"}` param, which read everywhere: migration 21 tagged those companies and logged each such board, since no everywhere level is left. Migration 24 renamed the tag to All countries and gave its companies (Mistral AI and OpenAI on live) All roles as well, since the owner wanted their every role. A board's `params` override the agents' reach and All countries. Boards under 300 roles, and hosts with a one-country filter over more than 8 countries, are read whole. SmartRecruiters, Eightfold, SuccessFactors, Phenom, Workday and Oracle filter at the source; countries and remote go as separate series, because the hosts intersect two facets. A complete filtered read closes roles outside the filter, except roles with a vote or an application; a filtered read, or one from a host whose unknown slug answers empty, that lists under 10% of the last read with the same filter counts as failed.
|
||||
|
||||
**A role has a list of places, resolved against a gazetteer.** Readers kept one place of several, joined them, or the host posted a copy per place. A reader now returns every place and never multiplies a row. `places` (city, region, country, continent, with `place_includes` for Øresund across two countries) and `place_aliases` are seeded from the dictionaries that were in code; `role_places` links each place of a role, and an unknown text is listed by `unresolved_places` until an alias links it. The Place filter and the gates join over the tree, which halved their time and dropped the `EU` fallback. Copies of one posting per place, same host, company and title and texts at least 30% alike, fold into one role that keeps each copy's place and link; the rule applies to every host, since the 5,024 such roles came mostly from hosts that do not declare copies.
|
||||
|
||||
@@ -122,7 +122,7 @@ What was decided for roledesk and the one reason for each. The longer reasoning
|
||||
|
||||
**Readers scrape; `prep` decides.** A reader hands on the host's own words (`seniority`, `employment`, `remote`, and `country` only when the host states one per role) and does light cleanup only: text from HTML, dates, the places of a role as a list, pay figures. Mapping lived in 25 readers, each spelling its tables its own way, so changing what "Experienced" means touched a plugin. The `prep` stage (`app/prep/`) maps centrally after every pull and every enrich run. Seniority, employment and remote go through the host's table in `app/prep/hosts/<reader>.py` (whole words, or `MATCH` fragments by substring or token), then a shared word map. Country comes from the resolved places (several European countries give `EU`), and the host's country is only a fallback. Places and company aliases are resolved, and copies and duplicates are folded over the whole database. `upsert_role` stores the host words (`*_host` columns, migration 22) and marks the role `prep_due`. When a table changes, its fingerprint makes the next prep map every open role again from the stored words, with no pull. A word no table knows is listed in `unmapped` (`app prep unmapped`, the Sources page). A host's `remote: False` stays unknown rather than onsite, because SmartRecruiters, Ashby, Recruitee and Workable send it for hybrid roles too.
|
||||
|
||||
**Fetch terms narrow what big boards send across Europe.** An agent's `fetch_terms` (migration 23) are words a big board must match when it is read with the wide reach: a company tagged All roles, or a one-country host whose reach was dropped for needing over 8 requests. The terms of a pull are the union over the enabled agents; one enabled agent without terms switches them off. The narrow reach takes no terms, because it already leaves at most about 150 roles on any big board. The survey of 2026-09-30 found that every host's keyword matches the whole text, so a term cuts less than a title filter would ("engineer" left 1,713 of Bosch's 4,830). SmartRecruiters, SuccessFactors, Workday, Phenom, Eightfold and Oracle take one term a request, with one series per term unioned by id; over 8 series the place filter goes first, then the terms. A term-filtered read is never the whole board: it writes no size, prunes no per-role state and closes nothing. Category and level filters wait until something maps agent terms to host ids. A town in a gate is still fetched as its country.
|
||||
**Fetch terms narrow what big boards send across Europe.** An agent's `fetch_terms` (migration 23) are words a big board must match when it is read with the wide reach: a company set to All countries, or a one-country host whose reach was dropped for needing over 8 requests. A company set to **All roles** (`companies.all_roles`, migration 24) takes no terms on any of its boards, and will take no category or level filter either. It replaced a board's `{"terms": false}` param, which migration 24 moved to the board's company. On its own, All roles changes nothing today, since the narrow reach takes no terms; it is kept for the role filters to come. The terms of a pull are the union over the enabled agents; one enabled agent without terms switches them off. The narrow reach takes no terms, because it already leaves at most about 150 roles on any big board. The survey of 2026-09-30 found that every host's keyword matches the whole text, so a term cuts less than a title filter would ("engineer" left 1,713 of Bosch's 4,830). SmartRecruiters, SuccessFactors, Workday, Phenom, Eightfold and Oracle take one term a request, with one series per term unioned by id; over 8 series the place filter goes first, then the terms. A term-filtered read is never the whole board: it writes no size, prunes no per-role state and closes nothing. Category and level filters wait until something maps agent terms to host ids. A town in a gate is still fetched as its country.
|
||||
|
||||
**Seniority has three sources.** `seniority_stated` is the host's word mapped by that host's table in `app/prep/hosts/`, `seniority_title` is `seniority.from_title`, and `seniority_model` is `extract`'s verdict; `roles.seniority` is the first of model, stated, title. `extract` reads roles without a model value, every agent's first pick before anyone's second, with the other two as hints it may overrule, since host vocabularies differ and 57% of roles state nothing. `app seniority` shows per host how often the stated level agrees with the model.
|
||||
|
||||
|
||||
Reference in new issue
Block a user