Ashby, BambooHR, Deel, Eightfold, Gem, Greenhouse, HiBob, HR Manager, Jibe,
Jobylon and Lever move to feed_url(slug, ask) and page(get, slug, cursor, ask),
declare READS with the robots decision, and drop robots_page. Rows carry
places (Ashby secondaryLocations, Lever allLocations, Greenhouse offices in
place of a Hybrid placeholder), Jobylon keeps its summary as teaser and maps
experience and employment, Eightfold reads creationTs as the posting date and
Ashby reads workplaceType. Fixtures <name>-places.* and jobylon-teaser.html
drive the conformance checks; the eleven names leave UNDECLARED.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each declares READS with scope=search, keeps every place and hands the excerpt over as teaser. search_params and PROVIDERS follow the declarations.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
plugins/readers/_contract.py holds Reads, Filters, Fields, Key, Ask, Place and
Row, and problems(), which checks a module's READS against its functions and
signatures. The package loader runs it for modules that declare READS and
lists the others with undeclared(); READERS holds every module, HOSTS the
boards and the undeclared, PROVIDERS the searches. The package docstring
documents the traits, Ask, facets, places, teaser and the robots.txt rule.
tests/test_readers_contract.py runs over every reader: READS declared (xfail
per module in UNDECLARED), declaration against functions, places="many"
and teaser fixtures, plus unit tests of the check with fake modules.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every import, test and live doc follows the new path; behaviour is unchanged.
tests/test_atshosts.py becomes tests/test_readers.py. plan-readers.md and
review-ingest.md keep the old name, since they describe the state before.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A role listed before its page was read kept a title-only vector, and the listing's place replaced the page's on every later pull.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
_get streams the body with a running byte count, so an oversized board is
refused at MAX_BYTES instead of after it is held whole, and bounds the whole
request, redirects included, by DEADLINE, checked between chunks. The
per-phase timeouts are httpx.Timeout(10, read=30), set from 84 live requests
against mongodb, feedzai and the five Greenhouse boards with most open roles,
with and without pay_transparency=true: the body arrives in under 1 s, the
first byte waited up to 16.6 s, and the flag made no measurable difference.
_paged works unchanged through it (tested over a mock transport, 429 included).
Correct pull.py's BATCH_SIZE comment: a batch reads its boards in sequence,
and the parallelism is the http pool's.
Co-Authored-By: Claude Opus <noreply@anthropic.com>
Every ats parser now writes one human-readable pay line whether or not it
converts to yearly salary_min/salary_max/currency: hourly and daily rates,
Greenhouse's several pay zones, and a range with no currency all survive as
text instead of being dropped outright. Stored in roles.salary_text
(migration 12), replaced whole on every pull like title, so a board that
stops stating pay clears it rather than leaving a stale line. salary_state
is derived, not stored ('normalised', 'stated', or ''), and is what the
feed's new Pay filter and badge read; gates and scoring are unchanged, still
reading only the normalised columns.
Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
Part A: Ashby (?includeCompensation=true), Greenhouse (pay_transparency=true,
already free on the list request), Lever (salaryRange), Recruitee (salary)
and Personio (salaryInformation) all give structured pay; Workable and
SmartRecruiters do not (checked live). Every host converts to a yearly
salary_min/salary_max/currency, multiplying a monthly figure by 12 and
skipping an hourly or daily one rather than annualising a number nobody
stated. A range with no currency is ignored, and Greenhouse's own multi-zone
ranges are skipped outright since nothing says which zone the role is at.
Part B: classify() matched substrings, so " it" matched inside "Italy",
filing European sales roles as engineering; it now matches on word
boundaries (full boundary for the handful of short tokens that are
themselves the start of an unrelated word, prefix matching kept for the
stems). Title is checked before department (a hand-checked sample found
title-first right about twice as often), nothing matching returns '' rather
than "other", and a modest set of German/Dutch/Danish/Norwegian/Swedish/
French keywords covers the seed's other languages.
function is not a board field, so it gets the opposite rule from every
other raw column: extract's answer overwrites the classifier's guess
outright, and once extracted_at is set no later pull may put the guess
back. Migration 9 recomputes function for every not-yet-extracted role from
its stored title and department.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
A second pull (schedule, Run button, `app job pull` in another process) could
start while one was fetching and overwrite the stored run marker, so the first
run's boards failed their stamp check or a settle closed on the other run's id
lists. `claim_run` sets `pull:run` in one conditional statement, only while it
equals `pull:settled` or is older than an hour; a refused `pull` job queues no
stage and says so in its result and a notice, and the CLI exits 1. Batches
carry the marker in their payload, the plugins stamp with it, and the settle
reads it off its own batches' payloads before releasing the run.
Co-Authored-By: Claude Opus <noreply@anthropic.com>
`app.jobs.sources.set_board` attaches a board to an existing company,
replaces the one it has (closing the old board's open roles) or creates
the company, with row and subscription changed in one transaction by
`Store.set_company_board`. A board that lists zero roles is kept. `add`,
`discover --add`, the seed importer (moved out of app/cli.py) and the new
`POST|DELETE /api/companies/{key}/board` all go through it; a seed re-run
attaches a board to a row that never had one. A company's subscription is
rewritten in place, and detaching hides it.
`companies.ats_slug` (migration 9) keeps the slug of a host the app
cannot read yet, and discovery returns such a host before probing slugs.
The Companies rows get Set board / Change, the host select is rendered
from `plugins.ats.HOSTS`, and a row with nothing found reads "board not
found" and is in the "Not readable yet" filter.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Workable and SmartRecruiters page their lists, and both parsers read page one
only, so the settle closed every role past it (Delivery Hero: 980 stated, 100
open). `_paged` follows nextPage and offset, and the plugin stamps
`ats:complete` only when the distinct ids read equal the stated total; a 429
after page one keeps what was read and closes nothing. Adzuna completes on its
`count`, Jobindex counts raw items and raises on an answer with no channel.
A board whose fetch raises is recorded under `ats:failing:<target>` with the
first failure's time and the latest reason; after STALE_DAYS the settle closes
its roles, and its next answer reopens them. `/api/companies`, the Companies
list and `app companies check` show it.
Co-Authored-By: Claude Opus <noreply@anthropic.com>