Fetch terms narrow what big boards read with the wide reach send (WP F2)
An agent has fetch_terms (migration 23, API, a chip field in the agent
editor). The terms of a pull are the union over the enabled agents, none
when one of them has none. They go only to wide reads: a company tagged All
roles, or a reach ask_for drops as too many countries for a one host; a
board's {"terms": false} switches them off. SmartRecruiters q,
SuccessFactors q, Workday searchText, Phenom keywords, Eightfold query and
Oracle keyword declare query="one"; the runner makes one series per term
times the country series, drops the places first over MAX_SERIES, then the
terms. A term read is narrowed: it writes no ats:size, closes nothing and
prunes no per-role state, and the 10% guard holds it to its own
fingerprint. The Companies reach line counts the terms.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
3917b31209
commit
0fd45b333a
28 files changed
+839
-113
No files matched your search
+10
-7
@@ -69,11 +69,14 @@ Numbers are from the live database after the pull of 2026-09-29. For the "list,
|
||||
| robots.txt | read before the list for the 14 hosts that name a page: Workday, SuccessFactors, Oracle, Phenom, Eightfold, Jibe, Gem, BambooHR, Pinpoint, Deel, Jobylon, Rippling, HiBob, HR Manager; kept for one day per board | A disallow fails the board with "robots.txt disallows it" and counts toward the 21 days. An unreachable file (5xx, 429, timeout) fails the board for this pull and is asked again next pull. A 4xx means no rules, which allows. `Crawl-delay` is not honoured. The eight API hosts (Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee, Personio, Teamtailor) and the search providers read no robots.txt. |
|
||||
| 429 | first page: the board fails; a later page: the read stops, keeps what it read and is incomplete; a per-role read: the per-role reads of that board stop until the next pull | Nothing is closed on a list cut short by a 429. A search that fails is skipped for the pull. |
|
||||
|
||||
How far a board is read: a pull asks each board over 300 roles for the reach of the search agents that are switched on (`app/jobs/reach.py`): the union of their `any` gates on country, town (as its country) and remote, or every role when one agent can take a role from anywhere. A company starred as favourite is asked for its agent's `require` reach, Europe today; a host that asks one country a request reads a favourite whole instead of 40 times. A board's own reach, set on the Companies list, wins. A complete filtered read closes the roles outside the reach, except those with a vote or an application.
|
||||
How far a board is read: a pull asks each board over 300 roles for the reach of the search agents that are switched on (`app/jobs/reach.py`): the union of their `any` gates on country, town (as its country) and remote, or every role when one agent can take a role from anywhere. A company tagged All roles is asked for the agents' wide reach, the union of their `require` reach, Europe today; a host that asks one country a request reads such a board whole instead of 30 times, unless fetch terms narrow it. A board's own reach, set on the Companies list, wins. A complete filtered read closes the roles outside the reach, except those with a vote or an application.
|
||||
|
||||
Fetch terms narrow a wide read by words. Each agent has a list of them (`fetch_terms`, set in the agent editor); the terms of a pull are the union over the agents that are on, and none when one of those agents has none. They go only to a board read with the wide reach: a company tagged All roles, or a board whose reach is more countries than a one-country host is asked for (`MAX_SERIES`, 8). The narrow reach (Denmark, Sweden) leaves at most about 150 roles on any big board, so it takes no terms, and nor does a board under 300 roles, one never read whole, or one with its own reach. `{"terms": false}` in a board's params switches them off. A host that takes a keyword gets one series per term, unioned by id, each combined with the country series (an AND on every host the survey tested). When terms times the country series exceed 8, the place part of the reach goes first, whole, and the terms stay; more than 8 terms read the board unfiltered. Every host matches the whole text, not only the title, so a term like `engineer` cuts less than it looks. A read with terms is never the whole board: it records no size, closes no role it did not list, and the 10% guard holds it only to a read with the same terms and places. The search providers take no terms: each saved search has its own query.
|
||||
|
||||
What no source does:
|
||||
|
||||
- Filter on the host's side for a host other than SmartRecruiters, SuccessFactors, Workday, Oracle, Phenom and Eightfold. Greenhouse, Ashby, Recruitee and the rest are read whole.
|
||||
- Filter by category or level at the source. Workday, Oracle, Jibe and Eightfold state category ids, but nothing maps an agent's terms to them yet (`docs/plan-prep-survey.md`).
|
||||
- Read more than one board per company. DFDS is read from its Danish Teamtailor board only.
|
||||
- Use a key taken from a site's script. Maersk, New Work and Welcome to the Jungle are on the Watchlist for that reason.
|
||||
- Read a page that needs a browser to run its script.
|
||||
@@ -225,7 +228,7 @@ What no source does:
|
||||
| Remote | `location.remote`, true or false (122 remote) |
|
||||
| Pay | none: the list states no pay |
|
||||
| Found by discovery | yes, from a `jobs.`, `careers.` or `api.smartrecruiters.com` link; a guessed name is probed |
|
||||
| Filter at the source | `?country=dk`, one country a request; remote is ignored by the API, so a remote role outside the reach is not read. Bosch DK: 2 of 4,830 (2026-09-30). Which countries is the agents' reach, see below |
|
||||
| Filter at the source | `?country=dk`, one country a request; remote is ignored by the API, so a remote role outside the reach is not read. Bosch DK: 2 of 4,830 (2026-09-30). Which countries is the agents' reach, see below. Fetch terms as `?q=engineer`, one a request, AND with the country; it matches title and text (Bosch: 1,713 of 4,830) |
|
||||
| Limitations | 10,000 roles per board (100 pages). The identifier is read without case, so a lowercased slug works. An unknown identifier answers an empty list, not an error: a mistyped slug reads as a complete board with no roles and would close a company's roles. |
|
||||
|
||||
The reader's "Fetch the posting" button shows for these roles, since their description is empty; it reads one posting page on demand and does not feed `embed` or `extract`. Whether it reads a SmartRecruiters page cleanly was not checked.
|
||||
@@ -310,7 +313,7 @@ The reader's "Fetch the posting" button shows for these roles, since their descr
|
||||
| Remote | none |
|
||||
| Pay | none |
|
||||
| Found by discovery | yes, from a link to a role, `/go/` page or search on the site's own host; a link to the SAP back end names the host only; not probed. A site on the company's own domain whose pages link only the back end is set by hand |
|
||||
| Filter at the source | `optionsFacetsDD_country=DK`, one country a request; no remote facet. Vestas DK: 64 of 668 (2026-09-30). Which countries is the agents' reach, see below |
|
||||
| Filter at the source | `optionsFacetsDD_country=DK`, one country a request; no remote facet. Vestas DK: 64 of 668 (2026-09-30). Which countries is the agents' reach, see below. Fetch terms as the form's `q=engineer`, one a request, AND with the country (Vestas DK: 40 of 64) |
|
||||
| Limitations | Only the default locale's roles are listed: a role that exists only in another locale (Carlsberg's French brand, 37 of 105) is not read. Only `/search/` and `/job/` are read, never `/services/` or the `successfactors.eu` back end, so a company with only the back end (Bang & Olufsen) cannot be read. 100 pages per board, so 1,000 to 10,000 roles depending on the page size. A third layout or a bot wall that answers 200 fails the board with "not a SuccessFactors search page". Read from this machine only; not yet from the deployed server. |
|
||||
|
||||
### Workday
|
||||
@@ -327,7 +330,7 @@ The reader's "Fetch the posting" button shows for these roles, since their descr
|
||||
| Remote | `remoteType`, where the tenant sends it: remote, hybrid, onsite (585 with a value) |
|
||||
| Pay | none: no pay field; pay appears only inside the text |
|
||||
| Found by discovery | yes, from a `myworkdayjobs.com` link with a site; not probed |
|
||||
| Filter at the source | the tenant's own facets in `appliedFacets`, ids read by `facets` once a day: the facet whose values are countries, and any value that reads as remote. Two facets intersect, so remote is a series of its own. LEGO DK: 97 of 455 (2026-09-30). A tenant with no country facet is read whole. Which countries is the agents' reach, see below |
|
||||
| Filter at the source | the tenant's own facets in `appliedFacets`, ids read by `facets` once a day: the facet whose values are countries, and any value that reads as remote. Two facets intersect, so remote is a series of its own. LEGO DK: 97 of 455 (2026-09-30). A tenant with no country facet is read whole. Which countries is the agents' reach, see below. Fetch terms as `searchText` in the body, one a request, AND with the facets (LEGO: 106 for `engineer`) |
|
||||
| Limitations | 2,000 roles per board (100 pages of 20). A tenant has several sites and each is its own board. robots.txt paths are matched without case, as Workday serves any case. Every tenant is behind Cloudflare and was read from this machine only. |
|
||||
|
||||
### Oracle Recruiting Cloud
|
||||
@@ -344,7 +347,7 @@ The reader's "Fetch the posting" button shows for these roles, since their descr
|
||||
| Remote | `WorkplaceTypeCode`: remote, hybrid, onsite (741 with a value) |
|
||||
| Pay | none: the resource states no pay |
|
||||
| Found by discovery | yes, from a candidate site or role link on the pod, or a REST call naming `siteNumber`; not probed |
|
||||
| Filter at the source | `selectedLocationsFacet` with the pod's geography ids, and `selectedWorkplaceTypesFacet=ORA_REMOTE` as a series of its own. `locationsFacet` lists the ten biggest countries only, so a country missing from a full list is read unfiltered: fa-evmr (Nokia) has neither DK nor SE listed and is read whole. Which countries is the agents' reach, see below |
|
||||
| Filter at the source | `selectedLocationsFacet` with the pod's geography ids, and `selectedWorkplaceTypesFacet=ORA_REMOTE` as a series of its own. `locationsFacet` lists the ten biggest countries only, so a country missing from a full list is read unfiltered: fa-evmr (Nokia) has neither DK nor SE listed and is read whole. Which countries is the agents' reach, see below. Fetch terms as `keyword=` in the finder, one a request; it matches the text (Nokia: 468 of 554 for `engineer`) |
|
||||
| Limitations | The host is the tenant's Fusion pod, not the company's domain (`jobs.nokia.com` redirects to it). Links are built for locale `en`. The department fields were empty on every role of the four boards. |
|
||||
|
||||
### Phenom
|
||||
@@ -361,7 +364,7 @@ The reader's "Fetch the posting" button shows for these roles, since their descr
|
||||
| Remote | per role only, from the page's `checkRemote` (120 with a value) |
|
||||
| Pay | none |
|
||||
| Found by discovery | yes, from a `/<country>/<lang>/search-results` link; a link to Phenom itself names the host only; not probed |
|
||||
| Filter at the source | `selected_fields.country`, a list of English names in one request; no remote. Philips NL and DE: 111 of 873 (2026-09-30). Which countries is the agents' reach, see below |
|
||||
| Filter at the source | `selected_fields.country`, a list of English names in one request; no remote. Philips NL and DE: 111 of 873 (2026-09-30). Which countries is the agents' reach, see below. Fetch terms as `keywords` in the body, one a request, AND with the countries; it matches the text, not the title |
|
||||
| Limitations | The locale is built as `<lang>_<country>`, the spelling of the three sites read; another site may spell it differently. 10,000 roles per board. A role page without a description fails the read and is retried on a later pull. |
|
||||
|
||||
### Eightfold
|
||||
@@ -378,7 +381,7 @@ The reader's "Fetch the posting" button shows for these roles, since their descr
|
||||
| Remote | `workLocationOption`: onsite, hybrid, remote (628 with a value) |
|
||||
| Pay | none |
|
||||
| Found by discovery | yes, from an `eightfold.ai` link that carries `domain=`; a bare `eightfold.ai` link names the host only; not probed |
|
||||
| Filter at the source | `location=<country name>`, one country a request; no remote. Ericsson SE: 154 of 618 (2026-09-30). Which countries is the agents' reach, see below |
|
||||
| Filter at the source | `location=<country name>`, one country a request; no remote. Ericsson SE: 154 of 618 (2026-09-30). Which countries is the agents' reach, see below. Fetch terms as `query=`, one a request, AND with the location; it matches title, text and skills (Ericsson SE: 102 of 154 for `software`) |
|
||||
| Limitations | With 10 per request and 100 pages, a board over 1,000 roles is never read to the end; unless the country filter brings it under the cap: Infineon (1,289, 163 in DK and SE) and Microsoft (about 2,300, 60) become readable that way. Verified on three tenants only. |
|
||||
|
||||
### Gem
|
||||
|
||||
Reference in new issue
Block a user