Commit Graph
119 Commits
Author SHA1 Message Date
WolfandClaude Fable 5.1 62cf1c738c Merge sweeps that resume on a tuple cursor, a stalled read counted incomplete, searches closing by age only, and Lever and Oracle failing on a wrong-shaped answer (wp-rev-runner)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-05 09:56:07 +02:00
WolfandClaude Fable 5.1 c4c17fb019 Count a page of ids already read as a stall only where the host states no total or the reader hands the page over empty, so a resumed sweep reads on when the list grew at the top
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-05 09:56:06 +02:00
WolfandClaude Fable 5.1 7f37270eb3 Keep the sweep of a reader whose cursor is a tuple, so Workday and SuccessFactors boards over 100 pages finish across pulls, and end a read incomplete on a page with nothing new while a cursor follows, whatever the total, which SuccessFactors now hands back on a repeated page
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-05 09:51:16 +02:00
WolfandClaude Fable 5.1 85aed0807b Fail the read when Lever answers something other than a list or Oracle an answer without its search wrapper, instead of reading an empty board that is complete
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-05 09:32:06 +02:00
WolfandClaude Fable 5.1 caed82d416 Cut the plugins README to the per-reader sections, which the ADRs and the docstrings now cover for the flow and the rules
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-10-05 09:29:38 +02:00
WolfandClaude Opus 5.5 d80f26a930 Bring the plugins README and a style comment to All locations and fetch terms on every big board
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 21:43:01 +02:00
WolfandClaude Opus 5.5 6d82287402 Bring the Python side to the ADR: the judge scores 0 to 10 with the cap at 4 and migration 33 clears every judgement, summarise reads the top 30 of the want and the match order, a hidden company's roles are never in the feed, and the company switch is called All locations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 21:40:32 +02:00
WolfandClaude Opus 5.5 558189d747 Merge the ranking core: fetch gate overrides in scoring, the cut per agent, stages read the cut, fetch text, the feed kept to the fetch gate, saved filters backend, migration 32 (wp-core)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:18:35 +02:00
WolfandClaude Opus 5.5 8b6c6d5ac3 Test migration 32, the cut, saved filters and the override rescore, and bring the READMEs and config comments to fetch text and the cut
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:16:38 +02:00
WolfandClaude Opus 5.5 617356e60c Rank by the cut: fetch gate overrides in scoring, the cut per agent, stages read the cut, fetch text renamed from enrich, the feed kept to the fetch gate, and migration 32
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 15:11:43 +02:00
WolfandClaude Sonnet 5.5 5fe5a31747 Send fetch terms to every big board whose host takes one query, and on a series overflow let a narrow read keep its places and drop the terms
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-04 14:50:33 +02:00
WolfandClaude Opus 5.5 dd733c9fb5 Say that a Jobnet query requires every word, as a live count shows, not that it matches loosely
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 21:17:13 +02:00
WolfandClaude Opus 5.5 04f4c09a21 Merge text for search readers with detail (wp-src-searchdetail)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 12:29:34 +02:00
WolfandClaude Sonnet 5.5 b148647ac3 Read a search role's own page for its text in enrich and on the Fetch button when its reader has detail.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 12:29:19 +02:00
WolfandClaude Opus 5.5 10dec33dce Decode HTML entities in freehire titles and companies, and separate the region ids in the searches help
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 12:28:19 +02:00
WolfandClaude Opus 5.5 a9c387c130 Merge the wiring of the new search providers (wp-src-wiring)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 12:24:56 +02:00
WolfandClaude Sonnet 5.5 6e6e655203 Wire freehire, Jobnet and Akademikernes Jobbank into the CLI, the Sources form, dupes and the docs
Provider names, the CLI's choices and the dupes rule's search hosts now come from readers.PROVIDERS, and the regions, work modes and remote words from the modules' REGIONS, MODES and REMOTE, read with getattr so main works before and after the readers merge.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 12:24:34 +02:00
WolfandClaude Opus 5.5 07e3bb4756 Merge the jobbank search reader (wp-src-jobbank)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 12:21:18 +02:00
WolfandClaude Opus 5.5 ee17fe3e30 Merge the jobnet search reader (wp-src-jobnet)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 12:21:18 +02:00
WolfandClaude Sonnet 5.5 0ed0360b09 Add the Jobnet search reader, which reads jobnet.dk's backend with the x-csrf header and 100 ads a page
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 12:21:00 +02:00
WolfandClaude Sonnet 5.5 59e43f18b7 Add the freehire search reader: tech roles of many countries with full text, yearly pay and work mode
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 12:19:15 +02:00
WolfandClaude Sonnet 5.5 43c0bc7523 Read Akademikernes Jobbank's search feed, with each ad's text from its own page
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-10-03 12:18:56 +02:00
WolfandClaude Opus 5.5 c497fe965d Keep a description's headings, bullets and paragraphs, and draw them in the reader
The readers flattened a posting's HTML into one line with strip_html, so
sections and bullets ran together. _shared.html_text gives a line per
paragraph, '• ' before a list item and '## ' before a heading; every
reader's description goes through it, sections read per role get their
own headings, and the reader pane draws the lines as h3, ul and p.
Excerpts leave the marks out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 21:40:12 +02:00
WolfandClaude Fable 5.1 a211cf7414 Fold the review of 2026-09-30 into the decisions and open items, and delete the plan
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 16:11:27 +02:00
WolfandClaude Fable 5.1 a63eed2e56 Say remote not asked on the Companies line, show a failing search on its row, forget a removed search's record, and mark a read that asked nothing as not closing
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 16:11:27 +02:00
WolfandClaude Fable 5.1 fe2a1b0347 Merge main into wp-review-contract
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 15:57:46 +02:00
WolfandClaude Fable 5.1 fac3487925 Merge main into wp-review-losses
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 15:54:33 +02:00
WolfandClaude Fable 5.1 2cc83b1d84 Correct the contract docstring: detail is read by enrich, the example is Workday's, the touch points of a new reader
Adds the search providers' PARAMS, clean_params and company, and a test
that the example READS is workday.py's.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 15:54:13 +02:00
WolfandClaude Fable 5.1 24a165818c Read searches through the runner, so a search has the 10% guard, the shared stamp and a search:failing record when it does not answer
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 15:53:47 +02:00
WolfandClaude Fable 5.1 36bc639384 Guard every read that is not a sweep, treat a read that asks nothing as incomplete, and mark a filtered read that could not ask remote, so no board loses its roles to a changed answer or a filter it could not send
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 15:53:47 +02:00
WolfandClaude Fable 5.1 2349fc424b Require READS, search params and a dated robots record; record robots.txt per requested path
Every reader declares, so undeclared(), the scope() fallback, robots_page and
ROBOTS_ANY_CASE go. A search provider needs PARAMS and clean_params, a row may
carry company, places gains none. Each robots record names the paths the
reader requests, from robots.txt read again on 2026-09-30. Platsbanken states
remote, HR Manager states no place.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 15:51:13 +02:00
WolfandClaude Fable 5.1 74f7c8ad2e Bring plugins/README.md in line with the code of 2026-09-30
Add a Flow section with the state keys, describe per-role text as read by enrich after scoring, move SmartRecruiters and Workable to list-then-each-role, refresh the Overview counts, fix the shared rules, the per-host rows, the search providers and the Watchlist table, and point at _runner.py and READS for what they own instead of restating them.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-30 15:33:21 +02:00
WolfandClaude Opus 5.5 4cf81437a2 Fold the board size, two strikes, per-role text and Jobindex link-out plan into the decisions
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:49:35 +02:00
WolfandClaude Opus 5.5 8c66aeb445 Merge board size set once, Recheck, and the Companies line naming what a site does not know (wp-a-size)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:45:27 +02:00
WolfandClaude Opus 5.5 27e24d0342 Merge the Jobindex link-out: the Se jobbet link kept as posting_url (wp-d-jobindex)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:45:27 +02:00
WolfandClaude Opus 5.5 a0882e9659 Size a board once with board_size, before a pull reads it with no size, and name what a site's ids leave out
set_board sizes a board from page 1's total, or reads to the end on a host that states none, instead of reading it whole up to page 100. A pull no longer rewrites ats:size; a board with none is handed the agents' reach marked unsized and is sized in its batch first, or by its whole read on a host that cannot narrow the reach. reach.sizes drops the open-role fallback. The Companies line names countries a Workday or Oracle site does not list and a missing remote option.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:43:52 +02:00
WolfandClaude Opus 5.5 802656aca4 Prune done work from the docs: fold the ingest review's open findings into open.md and delete it, cut the hosts wave-2 plan to what is unbuilt, drop the smaller-hosts survey
The plugins README no longer describes a run-time robots.txt check or boards that never close past the page cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:40:43 +02:00
WolfandClaude Opus 5.5 54b56ac370 Read the Se jobbet link off a Jobindex vis-job page
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:26:20 +02:00
WolfandClaude Opus 5.5 19401cfc7a Merge two strikes for boards swept across pulls (wp-b-strikes)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:25:33 +02:00
WolfandClaude Opus 5.5 70aa4b3117 Read the text of SmartRecruiters and Workable roles through detail, keeping Workable's shortcode in the row
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:23:58 +02:00
WolfandClaude Opus 5.5 a91032b60f Close a role of a swept board only when two finished sweeps in a row missed it
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 13:23:57 +02:00
WolfandClaude Opus 5.5 e52857dea5 Split the All tag into two company overrides: All countries and All roles (WP L)
Fetching has two filters, and a company now overrides each on its own.
All countries (companies.all_countries) is the old All roles tag: its big
boards are read with the agents' wide reach. All roles (companies.all_roles,
new meaning) drops the agents' fetch terms and any later role filter.

- Migration 24 copies the old tag to all_countries and keeps all_roles for
  those companies, and moves a board's {"terms": false} to its company.
- The runner no longer reads {"terms": false}; the plan leaves the terms out
  for a company set to All roles.
- POST /api/companies/{key}/all-countries and /all-roles, each queueing a
  pull when turned on; CompanyOut carries both.
- Two toggles replace All on the Companies row, the reader and the Roles
  strip; the reach line states countries and role types in plain words;
  the filter select gets both options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:50:45 +02:00
WolfandClaude Opus 5.5 795ca3628c Fold the prep plan into the decisions, keep the filter survey as reference
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 12:01:35 +02:00
WolfandClaude Opus 5.5 0fd45b333a Fetch terms narrow what big boards read with the wide reach send (WP F2)
An agent has fetch_terms (migration 23, API, a chip field in the agent
editor). The terms of a pull are the union over the enabled agents, none
when one of them has none. They go only to wide reads: a company tagged All
roles, or a reach ask_for drops as too many countries for a one host; a
board's {"terms": false} switches them off. SmartRecruiters q,
SuccessFactors q, Workday searchText, Phenom keywords, Eightfold query and
Oracle keyword declare query="one"; the runner makes one series per term
times the country series, drops the places first over MAX_SERIES, then the
terms. A term read is narrowed: it writes no ats:size, closes nothing and
prunes no per-role state, and the 10% guard holds it to its own
fingerprint. The Companies reach line counts the terms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:59:27 +02:00
WolfandClaude Opus 5.5 55757c41e5 Integration check: wp-p-prep with wp-r1-raw and wp-r2-raw
Resolves tests/test_readers_shared.py, adds MATCH_MODE to the pinpoint,
recruitee and rippling tables, and removes table_word, country_if_single and
remote_word from plugins/readers/_shared.py with their tests and the Helpers
paragraph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:31:31 +02:00
Wolf 45604e9316 Merge branch 'wp-r1-raw' into scratch-p-integration 2026-09-30 11:28:38 +02:00
WolfandClaude Opus 5.5 d04ceae7e0 Prep stage: host words stored raw, mapped centrally in app/prep after every pull
upsert_role stores the host's own words (roles.*_host), the place texts
unresolved when they change, and the company key unresolved on insert, and
marks the role prep_due. app.prep maps seniority, employment, remote and
country through the host tables in app/prep/hosts, resolves places and
company aliases, lists unknown words in `unmapped`, remaps every open role
when the tables' fingerprint changes, and runs rebuild_dupes, which moves
out of the pull's settle. Migration 22 adds the columns and the table and
fills the host words from raw. Cascade: pull, prep, embed, score, enrich,
prep, embed, score, extract, score, summarise. `app prep [unmapped]
[--remap]`, /api/prep/unmapped and a fold on the Sources page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:19:22 +02:00
WolfandClaude Opus 5.5 06eb929eaf Readers oracle to workday and the providers hand on the host's words; their word tables move to app/prep/hosts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 11:07:07 +02:00
WolfandClaude Opus 5.5 6c7ce1090d Readers ashby to lever hand on the host's words; their word tables move to app/prep/hosts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:54:19 +02:00
WolfandClaude Opus 5.5 5d5b7e7d3f Tag a company All roles to read its big boards with the agents' wide reach; favourite scores only
Migration 21 adds companies.all_roles, tags every favourite of any agent and each company whose board had params {"reach": "all"} (logged per board, the param removed). reach.plan reads a tagged company's boards with the union of the enabled agents' require reach and the base reach; favourite no longer widens the fetch. POST /api/companies/{key}/all-roles sets it and queues a pull when on. The Companies list, the reader and the company strip draw the toggle beside the affinity buttons, the list filters on it, and the board's reach editor drops its All roles option.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:44:41 +02:00