Commit Graph
256 Commits
Author SHA1 Message Date
WolfandClaude Opus 5.5 535eb86f3e Survey which source-side filters the hosts take, and how towns reach the fetch
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:40:06 +02:00
WolfandClaude Opus 5.5 11ba94cfd7 Plan: readers only scrape, a prep stage maps centrally, agents' terms narrow big boards
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:32:01 +02:00
WolfandClaude Opus 5.5 8d9a059e24 Read Adzuna's keyed API although its robots.txt disallows /: the owner's decision
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:14:42 +02:00
WolfandClaude Opus 5.5 87202c163d Merge branch 'wp-j-cleanup'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:13:40 +02:00
WolfandClaude Opus 5.5 903b69b6d3 Share the readers' place, table-lookup and country helpers, unify the employment vocabulary, and list the enrich switch with the stages
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:13:24 +02:00
WolfandClaude Opus 5.5 b567144612 Fold the readers plan into the decisions and delete it
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:06:50 +02:00
WolfandClaude Opus 5.5 ca80219bbb Merge branch 'wp-f-reach'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:04:26 +02:00
WolfandClaude Opus 5.5 dafca5435e Show and set each board's reach on the Companies list, list the readers' declarations on Sources, and document filtering at the source
A company row says what a pull asks its board for; its editor sets the board's own reach (the agents', all roles, or countries and remote) through /api/sources/{id}/reach. /api/readers lists each reader's listing, text, filters, places and robots decision, folded under the Follow form.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 10:03:41 +02:00
WolfandClaude Opus 5.5 3462a4abe7 Read big boards within the agents' reach: countries and remote from the enabled agents, Europe for favourites, filtered at the source by six hosts
app/jobs/reach.py derives each board's reach once per pull and the pull hands it to the runner through the batch payload. SmartRecruiters, Eightfold and SuccessFactors filter one country a request, Phenom a list, Workday and Oracle by host ids from facets. Remote beside countries is a series of its own, since two facets intersect.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:55:18 +02:00
WolfandClaude Opus 5.5 38a80d5b55 Merge branch 'wp-g-enrich'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:53:57 +02:00
WolfandClaude Opus 5.5 77b45a4b0f Read role pages in an enrich stage after scoring, best roles first
The pull reads listings only. `enrich` (app/jobs/enrich.py) takes the open
roles without text that an enabled agent ranks ungated in its best
`enrich.top` at `enrich.min_score` or above, best score first, on boards
whose reader has `detail`, within `enrich.per_board` and `enrich.per_run`,
one batch per board. `plugins.ats.read_board_details` reads them and keeps
the per-board state; a read that gave no text is read again after
`enrich.retry_days` (finding 18). Filled roles are upserted, which drops
their vectors, and embed and score follow; `cascade.step` passes over both
when nothing was filled.

The pull cascade is pull, embed, score, enrich, embed, score, extract, score,
summarise. Saving an agent and changing a company's affinity queue the
`enrich` job (enrich, embed, score) behind their rescore; a vote does not.
The stage has a switch, `stage:enrich`, over `enrich.enabled`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:53:29 +02:00
WolfandClaude Opus 5.5 695d38c339 Merge branch 'wp-e2-copies'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:49:10 +02:00
WolfandClaude Opus 5.5 3d1fed7c0e Fold per-place copies of one host into one role that lists every place with its own link
The reader shows every place, each linked to its copy, and the teaser marked as a short description when there is no text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:48:49 +02:00
WolfandClaude Opus 5.5 06b464790a Merge branch 'wp-h-companies'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:45:59 +02:00
WolfandClaude Opus 5.5 9172b2da72 Companies as symbols: company_aliases, merge, duplicates, empty enrichment columns (migration 20)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:45:43 +02:00
WolfandClaude Opus 5.5 7896adad27 Merge branch 'wp-i-seniority'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:44:07 +02:00
WolfandClaude Opus 5.5 7ca1f02e26 Extract as the afterburner for the level: model verdict with stated and title hints, best rank first, agreement per host
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:43:52 +02:00
WolfandClaude Opus 5.5 3adafd8950 Read date trust only from the readers' declarations, now that every reader declares
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:39:07 +02:00
WolfandClaude Opus 5.5 5f0f4ad5b7 Merge branch 'wp-e1-places'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:33:07 +02:00
WolfandClaude Opus 5.5 7f5530f35e Import Ask after the path set-up in the reader tests, so the file runs on its own
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:33:06 +02:00
WolfandClaude Opus 5.5 1d7b286374 Merge branch 'wp-d1-hosts'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:32:46 +02:00
WolfandClaude Opus 5.5 bbd7522163 Places as a gazetteer, role_places, teaser and three seniority sources
Migration 19 adds places, place_aliases, place_includes (seeded from the
dictionaries in app/models.py and app/places.py, Europe as a continent over
the European countries) and role_places, one row per place of a role,
resolved by app.places.Gazetteer, NULL when unknown. upsert_role writes them
from raw["places"], else the place line cut at ";"; the migration builds them
from each role's place column.

The Place filter and the country and place gates are joins over the tree:
country:DK is any place under Denmark, region:skane any town in Skane, a
country gate is met by any place in its countries, and the EU fallback of
the word matcher is no longer used. A place: token the gazetteer does not
know still matches whole words. Store gains gazetteer, role_places,
unresolved_places, add_place_alias, add_place and relink_places.

roles.teaser is kept-filled and embedded while there is no description; a
vector is dropped when a better text arrives. seniority_model, _stated and
_title sit behind roles.seniority, which stays the first filled of them.

TRUSTED_POSTED_HOSTS gives way to trusted_posted(host), which reads
READS.fields.posted and falls back to the old set for undeclared readers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:31:45 +02:00
WolfandClaude Opus 5.5 650ae3f13a Merge branch 'wp-b-runner'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:29:15 +02:00
WolfandClaude Opus 5.5 902089a5e2 Runner: readers read from their declaration, ats and search thin over it
plugins/readers/_runner.py takes the guarded fetch, the listing (whole,
paged, one series per filter value unioned by id), completeness from READS,
the sweep of a board over the page cap across pulls, the 10% guard on a
filtered or empty="suspect" read, facets kept a day, the detail reads, the
raw an Item carries and the failure bookkeeping out of plugins/ats.py.
Ask comes from a subscription's params (ask_for, open for the reach), with
a declared key resolved into Ask.params["key"] and refused when missing.

The runtime robots.txt check goes (decided at build time, READS.robots).
plugins/search.py reads the providers of readers.PROVIDERS through the
runner and applies the saved pages param. A filtered board's close keeps
roles with a vote or an application. DETAIL_FIELDS gain places, teaser,
country and employment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:28:33 +02:00
WolfandClaude Opus 5.5 779b136757 Plan readers: listing run and enrich run, and what triggers the enrich run
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:16:29 +02:00
WolfandClaude Opus 5.5 dec5fd5009 Merge branch 'wp-d2-hosts'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:14:55 +02:00
WolfandClaude Opus 5.5 4316624769 Declare READS for Oracle, Personio, Phenom, Pinpoint, Recruitee, Rippling, SmartRecruiters, SuccessFactors, Teamtailor, Workable and Workday
Every row carries places (Personio additionalOffices, Workable and Recruitee
locations[], every SuccessFactors address, Workday's own read), the list
teaser of Phenom and Oracle is kept in teaser, and the level, employment and
country a host states are mapped. Workday "N Locations" and "Country - City",
and Recruitee "Remote job", are no longer places. Phenom takes dateCreated
as posted. robots_page and ROBOTS_ANY_CASE go; READS.robots records the
decision. Signatures are feed_url(slug, ask) and page(get, slug, cursor, ask),
with an empty Ask as the default so the runner of today still calls them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:14:27 +02:00
WolfandClaude Opus 5.5 f2e652a8da Hosts a to l declare READS: places, teaser, stated seniority and employment
Ashby, BambooHR, Deel, Eightfold, Gem, Greenhouse, HiBob, HR Manager, Jibe,
Jobylon and Lever move to feed_url(slug, ask) and page(get, slug, cursor, ask),
declare READS with the robots decision, and drop robots_page. Rows carry
places (Ashby secondaryLocations, Lever allLocations, Greenhouse offices in
place of a Hybrid placeholder), Jobylon keeps its summary as teaser and maps
experience and employment, Eightfold reads creationTs as the posting date and
Ashby reads workplaceType. Fixtures <name>-places.* and jobylon-teaser.html
drive the conformance checks; the eleven names leave UNDECLARED.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:11:53 +02:00
WolfandClaude Opus 5.5 e90fc81c14 Merge branch 'wp-c-providers'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:08:57 +02:00
WolfandClaude Opus 5.5 b78c9dfc67 Search providers as reader modules: jobindex, adzuna, platsbanken
Each declares READS with scope=search, keeps every place and hands the excerpt over as teaser. search_params and PROVIDERS follow the declarations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 09:08:25 +02:00
WolfandClaude Opus 5.5 4c0da58da7 Plan readers: seniority as stated, title and model values
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:57:30 +02:00
WolfandClaude Opus 5.5 f4b0ffc28f Merge branch 'wp-a-contract'
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:56:58 +02:00
WolfandClaude Opus 5.5 9f526356c0 Reader contract: READS declaration, Ask, Row with places and teaser, conformance test
plugins/readers/_contract.py holds Reads, Filters, Fields, Key, Ask, Place and
Row, and problems(), which checks a module's READS against its functions and
signatures. The package loader runs it for modules that declare READS and
lists the others with undeclared(); READERS holds every module, HOSTS the
boards and the undeclared, PROVIDERS the searches. The package docstring
documents the traits, Ask, facets, places, teaser and the robots.txt rule.

tests/test_readers_contract.py runs over every reader: READS declared (xfail
per module in UNDECLARED), declaration against functions, places="many"
and teaser fixtures, plus unit tests of the check with fake modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:55:47 +02:00
WolfandClaude Opus 5.5 73dec9028e Rename plugins/atshosts to plugins/readers
Every import, test and live doc follows the new path; behaviour is unchanged.
tests/test_atshosts.py becomes tests/test_readers.py. plan-readers.md and
review-ingest.md keep the old name, since they describe the state before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:48:30 +02:00
WolfandClaude Opus 5.5 af337ef558 Plan readers: owner decisions on reach, robots.txt at build time, places and companies as symbols, teaser kept
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:46:29 +02:00
WolfandClaude Opus 5.5 10fdd60988 Plan readers as plugins: declared traits, one API, one runner, one row per posting with all its places
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-30 08:34:49 +02:00
WolfandClaude Opus 5.5 667e1d35f4 Review the ingest path: text, places, dates, coverage and filtering at the source
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:43:19 +02:00
WolfandClaude Opus 5.5 82b7902eff Regroup copies when a board or search is switched off, and fold the duplicates plan into the decisions
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:11:33 +02:00
Wolf e1c197b488 Merge branch 'wp-j-dupes-ui' 2026-09-29 23:09:52 +02:00
Wolf fafd3c7636 Merge branch 'wp-i-dupes-storage' 2026-09-29 23:09:52 +02:00
WolfandClaude Opus 5.5 545c932f7b Store copies of one posting as roles.dup_of and leave them out of the feed, the counts and the model stages
Migration 18 adds roles.dup_of with a partial index and runs the first rebuild. Store.rebuild_dupes feeds app.dupes.assign the open roles of every (company_key, title_key) with more than one, rewrites dup_of where it changed, clears it on closed roles, and hands read_at and votes to a changed group's kept role. The pull's settle and `app dupes` run it. RoleRow.copies counts a row's open copies; Role.copies lists them and Role.dup_of names a copy's kept role.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:09:20 +02:00
WolfandClaude Opus 5.5 e28a5cd68a Name what a copy's note links to
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:05:33 +02:00
WolfandClaude Opus 5.5 374f74b742 Show a role's copies: ×N beside the company in the feed, Also posted and A copy of in the reader
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 23:05:07 +02:00
WolfandClaude Fable 5.1 55e353e764 Drop the open items that were done and say what seniority still lacks
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 22:54:31 +02:00
WolfandClaude Fable 5.1 a9ecd1b595 Fold the feed filters plan into the decisions and name what it leaves open
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 22:33:24 +02:00
Wolf db3b1a2933 Merge branch 'wp-g-rail' 2026-09-29 22:30:10 +02:00
WolfandClaude Opus 5.5 ae7c0eeb0e Make Company and Place chip inputs, add Seniority levels and Pay counts to the rail, and keep every filter on a company click
Company and Place are chipInputs on the suggest routes, their chips sent as company_keys and places and read back from the address, place tokens named through /api/places/labels. The old ?company= and ?place= show as a plain chip to take off. Seniority is a row of level chips with "and roles with no level", Pay keeps its select with "and roles that state no pay", each with its count from /api/roles/facets, asked after every reload. Feed rows show the seniority badge. More filters opens by default beside the feed and stays shut on a phone.

A company's name now links to the list on screen with that company as its one chip. The strip's sentence says what ★ does, or did. setAffinity passes the route's answer to onChange, so the strip waits on its score_job. The rail's tag chips get their pill border back, which the quiet rail button rule had taken.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 22:29:23 +02:00
WolfandClaude Fable 5.1 30b36df2fb Describe every source in plugins/README.md: how it is read, what it delivers and its limits
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 22:23:43 +02:00
WolfandClaude Fable 5.1 31abad4c8e Embed a role again when its text arrives, and keep the place its own page gave
A role listed before its page was read kept a title-only vector, and the listing's place replaced the page's on every later pull.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-29 22:23:43 +02:00
Wolf fd89381caa Merge branch 'wp-e-api' 2026-09-29 22:17:52 +02:00