Commit Graph
283 Commits
Author SHA1 Message Date
WolfandClaude Opus 5.5 a0882e9659 Size a board once with board_size, before a pull reads it with no size, and name what a site's ids leave out
set_board sizes a board from page 1's total, or reads to the end on a host that states none, instead of reading it whole up to page 100. A pull no longer rewrites ats:size; a board with none is handed the agents' reach marked unsized and is sized in its batch first, or by its whole read on a host that cannot narrow the reach. reach.sizes drops the open-role fallback. The Companies line names countries a Workday or Oracle site does not list and a missing remote option.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:43:52 +02:00
WolfandClaude Opus 5.5 a19c528e3f Plan board size set once, two strikes for swept boards, SmartRecruiters and Workable text, Jobindex link-out
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:19:07 +02:00
WolfandClaude Opus 5.5 5ed36b7301 Fold the seniority plan into the decisions and delete it
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:16:36 +02:00
WolfandClaude Opus 5.5 d529f6c0ce Merge the seniority range filter: From, To and Track in the rail (wp-sen-b-range)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:14:45 +02:00
Wolf dd9e359609 Merge the seniority ladder's levels: title rules, host tables, extract, migration 25 (wp-sen-a-levels)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

# Conflicts:
#	app/prep/hosts/jobylon.py
#	app/prep/hosts/recruitee.py
2026-09-30 13:14:45 +02:00
WolfandClaude Opus 5.5 b2c9a9ec29 Link the German towns, Villach, Americas and Asia Pacific; map the unmapped host words
Fifteen towns join Germany and Villach joins Austria. Americas is a spelling of
the United States, and Asia Pacific (APAC) is a continent over no country.
The Lever, HiBob, Jobylon, Personio, SmartRecruiters and Recruitee tables
learn the 40 words the Unmapped list showed, most of Lever's commitments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:14:21 +02:00
WolfandClaude Opus 5.5 322db49773 Title levels on the ladder: principal, senior manager, distinguished, lead split (WP A)
from_title matches a title twice, for the highest people level and the
highest individual grade. A people level wins on its own rung; a grade
above that rung lifts it one rung (Principal Engineering Manager is a
senior manager). Senior Staff is principal, Senior Principal, Distinguished
and Technical Fellow are distinguished; Associate Director, AVP and Senior
Manager are senior managers. lead is now a team lead only (Team Lead, Shift
Lead, Supervisor, Teamleiter); Tech Lead, Lead X and X Lead(er) are staff.
A slash names two levels, so Senior/Staff is staff.

- Jobylon "lead" maps to staff; Recruitee maps student_college, manager,
  senior_manager and executive.
- extract's prompt describes the ladder and its two tracks; enum from LEVELS.
- prep VERSION 2, so the next prep maps every open role again.
- Migration 25 clears seniority_model where it is staff, lead, manager or
  director, so extract reads those roles again, and sets roles.seniority to
  the first filled of model, stated and title.
- The coverage test of tests/test_seniority.py passes; its xfail is gone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:11:26 +02:00
WolfandClaude Opus 5.5 e03305ff87 Say how fetch terms are separated and that they match the whole ad
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:07:25 +02:00
WolfandClaude Opus 5.5 06ff9f549b List unlinked places and unmapped words under Sources, fix them in code with the unmapped skill
Settings > Sources > Unmapped shows the place texts no place resolves, with
their roles' countries and companies, and the host words no table maps, and
copies both as text. Prep fingerprints places.seed(), so a town or spelling
added to the code is seeded and every open role is mapped again at the next
prep. app prep unlinked lists the places on the command line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:06:22 +02:00
WolfandClaude Opus 5.5 4e390d97ca Filter seniority by a range of rungs and a track: level_from, level_to, track; From, To and Track in the rail (WP B)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:03:31 +02:00
WolfandClaude Opus 5.5 ae978594b4 Plan seniority as a ladder with two tracks and a range filter; the ladder in models
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:58:12 +02:00
WolfandClaude Opus 5.5 d858020068 Merge branch 'wp-l-two-overrides'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:51:13 +02:00
WolfandClaude Opus 5.5 e52857dea5 Split the All tag into two company overrides: All countries and All roles (WP L)
Fetching has two filters, and a company now overrides each on its own.
All countries (companies.all_countries) is the old All roles tag: its big
boards are read with the agents' wide reach. All roles (companies.all_roles,
new meaning) drops the agents' fetch terms and any later role filter.

- Migration 24 copies the old tag to all_countries and keeps all_roles for
  those companies, and moves a board's {"terms": false} to its company.
- The runner no longer reads {"terms": false}; the plan leaves the terms out
  for a company set to All roles.
- POST /api/companies/{key}/all-countries and /all-roles, each queueing a
  pull when turned on; CompanyOut carries both.
- Two toggles replace All on the Companies row, the reader and the Roles
  strip; the reach line states countries and role types in plain words;
  the filter select gets both options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:50:45 +02:00
WolfandClaude Opus 5.5 795ca3628c Fold the prep plan into the decisions, keep the filter survey as reference
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:01:35 +02:00
WolfandClaude Opus 5.5 1538c776d6 Merge branch 'wp-f2-terms'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:00:00 +02:00
WolfandClaude Opus 5.5 0fd45b333a Fetch terms narrow what big boards read with the wide reach send (WP F2)
An agent has fetch_terms (migration 23, API, a chip field in the agent
editor). The terms of a pull are the union over the enabled agents, none
when one of them has none. They go only to wide reads: a company tagged All
roles, or a reach ask_for drops as too many countries for a one host; a
board's {"terms": false} switches them off. SmartRecruiters q,
SuccessFactors q, Workday searchText, Phenom keywords, Eightfold query and
Oracle keyword declare query="one"; the runner makes one series per term
times the country series, drops the places first over MAX_SERIES, then the
terms. A term read is narrowed: it writes no ats:size, closes nothing and
prunes no per-role state, and the 10% guard holds it to its own
fingerprint. The Companies reach line counts the terms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:59:27 +02:00
WolfandClaude Opus 5.5 3917b31209 Merge the prep stage with the readers that hand on host words (wp-p-prep, wp-r1-raw, wp-r2-raw)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:32:15 +02:00
WolfandClaude Opus 5.5 55757c41e5 Integration check: wp-p-prep with wp-r1-raw and wp-r2-raw
Resolves tests/test_readers_shared.py, adds MATCH_MODE to the pinpoint,
recruitee and rippling tables, and removes table_word, country_if_single and
remote_word from plugins/readers/_shared.py with their tests and the Helpers
paragraph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:31:31 +02:00
Wolf 45604e9316 Merge branch 'wp-r1-raw' into scratch-p-integration 2026-09-30 11:28:38 +02:00
WolfandClaude Opus 5.5 56fad284a6 Prep reads remote from a Danish or Swedish place line, and from a search role's title
The search providers stop mapping remote_word themselves (R2), so prep reads
the same words: in any role's place line after remote_of, and in the title of
a role from a search.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:23:34 +02:00
WolfandClaude Opus 5.5 d04ceae7e0 Prep stage: host words stored raw, mapped centrally in app/prep after every pull
upsert_role stores the host's own words (roles.*_host), the place texts
unresolved when they change, and the company key unresolved on insert, and
marks the role prep_due. app.prep maps seniority, employment, remote and
country through the host tables in app/prep/hosts, resolves places and
company aliases, lists unknown words in `unmapped`, remaps every open role
when the tables' fingerprint changes, and runs rebuild_dupes, which moves
out of the pull's settle. Migration 22 adds the columns and the table and
fills the host words from raw. Cascade: pull, prep, embed, score, enrich,
prep, embed, score, extract, score, summarise. `app prep [unmapped]
[--remap]`, /api/prep/unmapped and a fold on the Sources page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:19:22 +02:00
WolfandClaude Opus 5.5 06eb929eaf Readers oracle to workday and the providers hand on the host's words; their word tables move to app/prep/hosts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:07:07 +02:00
WolfandClaude Opus 5.5 6c7ce1090d Readers ashby to lever hand on the host's words; their word tables move to app/prep/hosts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:54:19 +02:00
WolfandClaude Opus 5.5 a66aac95cf Merge branch 'wp-k-all-roles'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:45:03 +02:00
WolfandClaude Opus 5.5 5d5b7e7d3f Tag a company All roles to read its big boards with the agents' wide reach; favourite scores only
Migration 21 adds companies.all_roles, tags every favourite of any agent and each company whose board had params {"reach": "all"} (logged per board, the param removed). reach.plan reads a tagged company's boards with the union of the enabled agents' require reach and the base reach; favourite no longer widens the fetch. POST /api/companies/{key}/all-roles sets it and queues a pull when on. The Companies list, the reader and the company strip draw the toggle beside the affinity buttons, the list filters on it, and the board's reach editor drops its All roles option.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:44:41 +02:00
WolfandClaude Opus 5.5 5fa05d5f73 Plan prep: narrow the terms package to what the survey found
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:40:40 +02:00
WolfandClaude Opus 5.5 ce3857c14d Merge branch 'wp-s-survey'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:40:26 +02:00
WolfandClaude Opus 5.5 535eb86f3e Survey which source-side filters the hosts take, and how towns reach the fetch
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:40:06 +02:00
WolfandClaude Opus 5.5 11ba94cfd7 Plan: readers only scrape, a prep stage maps centrally, agents' terms narrow big boards
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:32:01 +02:00
WolfandClaude Opus 5.5 8d9a059e24 Read Adzuna's keyed API although its robots.txt disallows /: the owner's decision
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:14:42 +02:00
WolfandClaude Opus 5.5 87202c163d Merge branch 'wp-j-cleanup'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:13:40 +02:00
WolfandClaude Opus 5.5 903b69b6d3 Share the readers' place, table-lookup and country helpers, unify the employment vocabulary, and list the enrich switch with the stages
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:13:24 +02:00
WolfandClaude Opus 5.5 b567144612 Fold the readers plan into the decisions and delete it
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:06:50 +02:00
WolfandClaude Opus 5.5 ca80219bbb Merge branch 'wp-f-reach'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:04:26 +02:00
WolfandClaude Opus 5.5 dafca5435e Show and set each board's reach on the Companies list, list the readers' declarations on Sources, and document filtering at the source
A company row says what a pull asks its board for; its editor sets the board's own reach (the agents', all roles, or countries and remote) through /api/sources/{id}/reach. /api/readers lists each reader's listing, text, filters, places and robots decision, folded under the Follow form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:03:41 +02:00
WolfandClaude Opus 5.5 3462a4abe7 Read big boards within the agents' reach: countries and remote from the enabled agents, Europe for favourites, filtered at the source by six hosts
app/jobs/reach.py derives each board's reach once per pull and the pull hands it to the runner through the batch payload. SmartRecruiters, Eightfold and SuccessFactors filter one country a request, Phenom a list, Workday and Oracle by host ids from facets. Remote beside countries is a series of its own, since two facets intersect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:55:18 +02:00
WolfandClaude Opus 5.5 38a80d5b55 Merge branch 'wp-g-enrich'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:53:57 +02:00
WolfandClaude Opus 5.5 77b45a4b0f Read role pages in an enrich stage after scoring, best roles first
The pull reads listings only. `enrich` (app/jobs/enrich.py) takes the open
roles without text that an enabled agent ranks ungated in its best
`enrich.top` at `enrich.min_score` or above, best score first, on boards
whose reader has `detail`, within `enrich.per_board` and `enrich.per_run`,
one batch per board. `plugins.ats.read_board_details` reads them and keeps
the per-board state; a read that gave no text is read again after
`enrich.retry_days` (finding 18). Filled roles are upserted, which drops
their vectors, and embed and score follow; `cascade.step` passes over both
when nothing was filled.

The pull cascade is pull, embed, score, enrich, embed, score, extract, score,
summarise. Saving an agent and changing a company's affinity queue the
`enrich` job (enrich, embed, score) behind their rescore; a vote does not.
The stage has a switch, `stage:enrich`, over `enrich.enabled`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:53:29 +02:00
WolfandClaude Opus 5.5 695d38c339 Merge branch 'wp-e2-copies'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:49:10 +02:00
WolfandClaude Opus 5.5 3d1fed7c0e Fold per-place copies of one host into one role that lists every place with its own link
The reader shows every place, each linked to its copy, and the teaser marked as a short description when there is no text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:48:49 +02:00
WolfandClaude Opus 5.5 06b464790a Merge branch 'wp-h-companies'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:45:59 +02:00
WolfandClaude Opus 5.5 9172b2da72 Companies as symbols: company_aliases, merge, duplicates, empty enrichment columns (migration 20)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:45:43 +02:00
WolfandClaude Opus 5.5 7896adad27 Merge branch 'wp-i-seniority'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:44:07 +02:00
WolfandClaude Opus 5.5 7ca1f02e26 Extract as the afterburner for the level: model verdict with stated and title hints, best rank first, agreement per host
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:43:52 +02:00
WolfandClaude Opus 5.5 3adafd8950 Read date trust only from the readers' declarations, now that every reader declares
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:39:07 +02:00
WolfandClaude Opus 5.5 5f0f4ad5b7 Merge branch 'wp-e1-places'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:33:07 +02:00
WolfandClaude Opus 5.5 7f5530f35e Import Ask after the path set-up in the reader tests, so the file runs on its own
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:33:06 +02:00
WolfandClaude Opus 5.5 1d7b286374 Merge branch 'wp-d1-hosts'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:32:46 +02:00
WolfandClaude Opus 5.5 bbd7522163 Places as a gazetteer, role_places, teaser and three seniority sources
Migration 19 adds places, place_aliases, place_includes (seeded from the
dictionaries in app/models.py and app/places.py, Europe as a continent over
the European countries) and role_places, one row per place of a role,
resolved by app.places.Gazetteer, NULL when unknown. upsert_role writes them
from raw["places"], else the place line cut at ";"; the migration builds them
from each role's place column.

The Place filter and the country and place gates are joins over the tree:
country:DK is any place under Denmark, region:skane any town in Skane, a
country gate is met by any place in its countries, and the EU fallback of
the word matcher is no longer used. A place: token the gazetteer does not
know still matches whole words. Store gains gazetteer, role_places,
unresolved_places, add_place_alias, add_place and relink_places.

roles.teaser is kept-filled and embedded while there is no description; a
vector is dropped when a better text arrives. seniority_model, _stated and
_title sit behind roles.seniority, which stays the first filled of them.

TRUSTED_POSTED_HOSTS gives way to trusted_posted(host), which reads
READS.fields.posted and falls back to the old set for undeclared readers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:31:45 +02:00
WolfandClaude Opus 5.5 650ae3f13a Merge branch 'wp-b-runner'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:29:15 +02:00