Commit Graph
79 Commits
Author SHA1 Message Date
WolfandClaude Opus 5.5 a0882e9659 Size a board once with board_size, before a pull reads it with no size, and name what a site's ids leave out
set_board sizes a board from page 1's total, or reads to the end on a host that states none, instead of reading it whole up to page 100. A pull no longer rewrites ats:size; a board with none is handed the agents' reach marked unsized and is sized in its batch first, or by its whole read on a host that cannot narrow the reach. reach.sizes drops the open-role fallback. The Companies line names countries a Workday or Oracle site does not list and a missing remote option.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:43:52 +02:00
WolfandClaude Opus 5.5 e52857dea5 Split the All tag into two company overrides: All countries and All roles (WP L)
Fetching has two filters, and a company now overrides each on its own.
All countries (companies.all_countries) is the old All roles tag: its big
boards are read with the agents' wide reach. All roles (companies.all_roles,
new meaning) drops the agents' fetch terms and any later role filter.

- Migration 24 copies the old tag to all_countries and keeps all_roles for
  those companies, and moves a board's {"terms": false} to its company.
- The runner no longer reads {"terms": false}; the plan leaves the terms out
  for a company set to All roles.
- POST /api/companies/{key}/all-countries and /all-roles, each queueing a
  pull when turned on; CompanyOut carries both.
- Two toggles replace All on the Companies row, the reader and the Roles
  strip; the reach line states countries and role types in plain words;
  the filter select gets both options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:50:45 +02:00
WolfandClaude Opus 5.5 795ca3628c Fold the prep plan into the decisions, keep the filter survey as reference
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:01:35 +02:00
WolfandClaude Opus 5.5 0fd45b333a Fetch terms narrow what big boards read with the wide reach send (WP F2)
An agent has fetch_terms (migration 23, API, a chip field in the agent
editor). The terms of a pull are the union over the enabled agents, none
when one of them has none. They go only to wide reads: a company tagged All
roles, or a reach ask_for drops as too many countries for a one host; a
board's {"terms": false} switches them off. SmartRecruiters q,
SuccessFactors q, Workday searchText, Phenom keywords, Eightfold query and
Oracle keyword declare query="one"; the runner makes one series per term
times the country series, drops the places first over MAX_SERIES, then the
terms. A term read is narrowed: it writes no ats:size, closes nothing and
prunes no per-role state, and the 10% guard holds it to its own
fingerprint. The Companies reach line counts the terms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:59:27 +02:00
WolfandClaude Opus 5.5 55757c41e5 Integration check: wp-p-prep with wp-r1-raw and wp-r2-raw
Resolves tests/test_readers_shared.py, adds MATCH_MODE to the pinpoint,
recruitee and rippling tables, and removes table_word, country_if_single and
remote_word from plugins/readers/_shared.py with their tests and the Helpers
paragraph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:31:31 +02:00
Wolf 45604e9316 Merge branch 'wp-r1-raw' into scratch-p-integration 2026-09-30 11:28:38 +02:00
WolfandClaude Opus 5.5 d04ceae7e0 Prep stage: host words stored raw, mapped centrally in app/prep after every pull
upsert_role stores the host's own words (roles.*_host), the place texts
unresolved when they change, and the company key unresolved on insert, and
marks the role prep_due. app.prep maps seniority, employment, remote and
country through the host tables in app/prep/hosts, resolves places and
company aliases, lists unknown words in `unmapped`, remaps every open role
when the tables' fingerprint changes, and runs rebuild_dupes, which moves
out of the pull's settle. Migration 22 adds the columns and the table and
fills the host words from raw. Cascade: pull, prep, embed, score, enrich,
prep, embed, score, extract, score, summarise. `app prep [unmapped]
[--remap]`, /api/prep/unmapped and a fold on the Sources page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:19:22 +02:00
WolfandClaude Opus 5.5 06eb929eaf Readers oracle to workday and the providers hand on the host's words; their word tables move to app/prep/hosts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:07:07 +02:00
WolfandClaude Opus 5.5 6c7ce1090d Readers ashby to lever hand on the host's words; their word tables move to app/prep/hosts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:54:19 +02:00
WolfandClaude Opus 5.5 5d5b7e7d3f Tag a company All roles to read its big boards with the agents' wide reach; favourite scores only
Migration 21 adds companies.all_roles, tags every favourite of any agent and each company whose board had params {"reach": "all"} (logged per board, the param removed). reach.plan reads a tagged company's boards with the union of the enabled agents' require reach and the base reach; favourite no longer widens the fetch. POST /api/companies/{key}/all-roles sets it and queues a pull when on. The Companies list, the reader and the company strip draw the toggle beside the affinity buttons, the list filters on it, and the board's reach editor drops its All roles option.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:44:41 +02:00
WolfandClaude Opus 5.5 8d9a059e24 Read Adzuna's keyed API although its robots.txt disallows /: the owner's decision
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:14:42 +02:00
WolfandClaude Opus 5.5 903b69b6d3 Share the readers' place, table-lookup and country helpers, unify the employment vocabulary, and list the enrich switch with the stages
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:13:24 +02:00
WolfandClaude Opus 5.5 ca80219bbb Merge branch 'wp-f-reach'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:04:26 +02:00
WolfandClaude Opus 5.5 dafca5435e Show and set each board's reach on the Companies list, list the readers' declarations on Sources, and document filtering at the source
A company row says what a pull asks its board for; its editor sets the board's own reach (the agents', all roles, or countries and remote) through /api/sources/{id}/reach. /api/readers lists each reader's listing, text, filters, places and robots decision, folded under the Follow form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:03:41 +02:00
WolfandClaude Opus 5.5 3462a4abe7 Read big boards within the agents' reach: countries and remote from the enabled agents, Europe for favourites, filtered at the source by six hosts
app/jobs/reach.py derives each board's reach once per pull and the pull hands it to the runner through the batch payload. SmartRecruiters, Eightfold and SuccessFactors filter one country a request, Phenom a list, Workday and Oracle by host ids from facets. Remote beside countries is a series of its own, since two facets intersect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:55:18 +02:00
WolfandClaude Opus 5.5 77b45a4b0f Read role pages in an enrich stage after scoring, best roles first
The pull reads listings only. `enrich` (app/jobs/enrich.py) takes the open
roles without text that an enabled agent ranks ungated in its best
`enrich.top` at `enrich.min_score` or above, best score first, on boards
whose reader has `detail`, within `enrich.per_board` and `enrich.per_run`,
one batch per board. `plugins.ats.read_board_details` reads them and keeps
the per-board state; a read that gave no text is read again after
`enrich.retry_days` (finding 18). Filled roles are upserted, which drops
their vectors, and embed and score follow; `cascade.step` passes over both
when nothing was filled.

The pull cascade is pull, embed, score, enrich, embed, score, extract, score,
summarise. Saving an agent and changing a company's affinity queue the
`enrich` job (enrich, embed, score) behind their rescore; a vote does not.
The stage has a switch, `stage:enrich`, over `enrich.enabled`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:53:29 +02:00
WolfandClaude Opus 5.5 1d7b286374 Merge branch 'wp-d1-hosts'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:32:46 +02:00
WolfandClaude Opus 5.5 902089a5e2 Runner: readers read from their declaration, ats and search thin over it
plugins/readers/_runner.py takes the guarded fetch, the listing (whole,
paged, one series per filter value unioned by id), completeness from READS,
the sweep of a board over the page cap across pulls, the 10% guard on a
filtered or empty="suspect" read, facets kept a day, the detail reads, the
raw an Item carries and the failure bookkeeping out of plugins/ats.py.
Ask comes from a subscription's params (ask_for, open for the reach), with
a declared key resolved into Ask.params["key"] and refused when missing.

The runtime robots.txt check goes (decided at build time, READS.robots).
plugins/search.py reads the providers of readers.PROVIDERS through the
runner and applies the saved pages param. A filtered board's close keeps
roles with a vote or an application. DETAIL_FIELDS gain places, teaser,
country and employment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:28:33 +02:00
WolfandClaude Opus 5.5 dec5fd5009 Merge branch 'wp-d2-hosts'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:14:55 +02:00
WolfandClaude Opus 5.5 4316624769 Declare READS for Oracle, Personio, Phenom, Pinpoint, Recruitee, Rippling, SmartRecruiters, SuccessFactors, Teamtailor, Workable and Workday
Every row carries places (Personio additionalOffices, Workable and Recruitee
locations[], every SuccessFactors address, Workday's own read), the list
teaser of Phenom and Oracle is kept in teaser, and the level, employment and
country a host states are mapped. Workday "N Locations" and "Country - City",
and Recruitee "Remote job", are no longer places. Phenom takes dateCreated
as posted. robots_page and ROBOTS_ANY_CASE go; READS.robots records the
decision. Signatures are feed_url(slug, ask) and page(get, slug, cursor, ask),
with an empty Ask as the default so the runner of today still calls them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:14:27 +02:00
WolfandClaude Opus 5.5 f2e652a8da Hosts a to l declare READS: places, teaser, stated seniority and employment
Ashby, BambooHR, Deel, Eightfold, Gem, Greenhouse, HiBob, HR Manager, Jibe,
Jobylon and Lever move to feed_url(slug, ask) and page(get, slug, cursor, ask),
declare READS with the robots decision, and drop robots_page. Rows carry
places (Ashby secondaryLocations, Lever allLocations, Greenhouse offices in
place of a Hybrid placeholder), Jobylon keeps its summary as teaser and maps
experience and employment, Eightfold reads creationTs as the posting date and
Ashby reads workplaceType. Fixtures <name>-places.* and jobylon-teaser.html
drive the conformance checks; the eleven names leave UNDECLARED.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:11:53 +02:00
WolfandClaude Opus 5.5 b78c9dfc67 Search providers as reader modules: jobindex, adzuna, platsbanken
Each declares READS with scope=search, keeps every place and hands the excerpt over as teaser. search_params and PROVIDERS follow the declarations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:08:25 +02:00
WolfandClaude Opus 5.5 9f526356c0 Reader contract: READS declaration, Ask, Row with places and teaser, conformance test
plugins/readers/_contract.py holds Reads, Filters, Fields, Key, Ask, Place and
Row, and problems(), which checks a module's READS against its functions and
signatures. The package loader runs it for modules that declare READS and
lists the others with undeclared(); READERS holds every module, HOSTS the
boards and the undeclared, PROVIDERS the searches. The package docstring
documents the traits, Ask, facets, places, teaser and the robots.txt rule.

tests/test_readers_contract.py runs over every reader: READS declared (xfail
per module in UNDECLARED), declaration against functions, places="many"
and teaser fixtures, plus unit tests of the check with fake modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:55:47 +02:00
WolfandClaude Opus 5.5 73dec9028e Rename plugins/atshosts to plugins/readers
Every import, test and live doc follows the new path; behaviour is unchanged.
tests/test_atshosts.py becomes tests/test_readers.py. plan-readers.md and
review-ingest.md keep the old name, since they describe the state before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:48:30 +02:00
WolfandClaude Fable 5.1 30b36df2fb Describe every source in plugins/README.md: how it is read, what it delivers and its limits
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 22:23:43 +02:00
WolfandClaude Fable 5.1 31abad4c8e Embed a role again when its text arrives, and keep the place its own page gave
A role listed before its page was read kept a title-only vector, and the listing's place replaced the page's on every later pull.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 22:23:43 +02:00
Wolf c7c5aebdb6 Merge branch 'wp7-small-hosts' 2026-09-29 20:15:47 +02:00
Wolf f38da929c9 Merge branch 'wp4-successfactors' 2026-09-29 20:15:47 +02:00
WolfandClaude Sonnet 5.5 d80754e985 Read HR Manager's public position feed by skip and take
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:13:23 +02:00
Wolf c18c93a55e Merge branch 'wp5-oracle-phenom' 2026-09-29 20:13:16 +02:00
Wolf 0eff7c9d13 Merge branch 'wp6-json-hosts' 2026-09-29 20:12:28 +02:00
WolfandClaude Sonnet 5.5 679d9ec938 Let discovery find a SuccessFactors board from a role, landing or search link
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:12:13 +02:00
WolfandClaude Sonnet 5.5 85bfe62fec Read Eightfold career sites through the PCSX search
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:11:18 +02:00
WolfandClaude Sonnet 5.5 cf182ad921 Read a Phenom role's description from the page state when the page has no JobPosting, and fail a page with neither
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:10:52 +02:00
WolfandClaude Sonnet 5.5 27a7723a1d Read HiBob's careers boards through the company header the site sends
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:10:37 +02:00
WolfandClaude Sonnet 5.5 254645c297 Read SuccessFactors career sites by their search pages and role microdata
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:10:11 +02:00
WolfandClaude Sonnet 5.5 8ee6f9b546 Read Phenom career sites through their search widget, with the description from each role page's JSON-LD
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:09:08 +02:00
WolfandClaude Sonnet 5.5 eac483ff13 Read Rippling's boards, merging a role's places, and its pay and description from the role's record
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:08:37 +02:00
WolfandClaude Sonnet 5.5 bd4121be91 Read Jibe career sites from their jobs API
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:08:35 +02:00
WolfandClaude Sonnet 5.5 34d1f74e3a Read Oracle Recruiting Cloud boards from the REST listing, with the description from the role's own resource
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:07:56 +02:00
WolfandClaude Fable 5.1 bb8a9a3b8f Read robots.txt without case for a host that serves a path in any case, as Workday does
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 20:07:14 +02:00
WolfandClaude Sonnet 5.5 a41995da8e Read Jobylon's boards from the embed page and each role's page
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:06:16 +02:00
WolfandClaude Sonnet 5.5 1fc9668110 Read Gem boards through the job board's GraphQL API
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:05:04 +02:00
WolfandClaude Sonnet 5.5 9efc6d386c Read Workday career sites through the tenant's own list and role endpoints
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:04:23 +02:00
WolfandClaude Sonnet 5.5 68a3858355 Read Deel's job boards from the page's server payload
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:02:49 +02:00
WolfandClaude Sonnet 5.5 76187d5862 Read BambooHR boards from their careers list and role pages
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:02:45 +02:00
WolfandClaude Sonnet 5.5 0b76b16daa Read Pinpoint boards from their postings.json
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-09-29 20:00:52 +02:00
WolfandClaude Opus 5.5 2ff5046993 State the final host contract in the package docstring and the plan, with ranks, case, detail fields and robots.txt
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:47:09 +02:00
WolfandClaude Opus 5.5 4fb06191cf Load the host modules when the registry reads ats.py by path without the repo root on sys.path
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:44:03 +02:00
WolfandClaude Opus 5.5 60136765b2 Refuse a fetch to another port than 443, an IP address or a local host name, at every hop
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 19:42:58 +02:00