Commit Graph
64 Commits
Author SHA1 Message Date
WolfandClaude Sonnet 5 44ff7c0131 Unpack structured pay from five ATS hosts; fix the function classifier
Part A: Ashby (?includeCompensation=true), Greenhouse (pay_transparency=true,
already free on the list request), Lever (salaryRange), Recruitee (salary)
and Personio (salaryInformation) all give structured pay; Workable and
SmartRecruiters do not (checked live). Every host converts to a yearly
salary_min/salary_max/currency, multiplying a monthly figure by 12 and
skipping an hourly or daily one rather than annualising a number nobody
stated. A range with no currency is ignored, and Greenhouse's own multi-zone
ranges are skipped outright since nothing says which zone the role is at.

Part B: classify() matched substrings, so " it" matched inside "Italy",
filing European sales roles as engineering; it now matches on word
boundaries (full boundary for the handful of short tokens that are
themselves the start of an unrelated word, prefix matching kept for the
stems). Title is checked before department (a hand-checked sample found
title-first right about twice as often), nothing matching returns '' rather
than "other", and a modest set of German/Dutch/Danish/Norwegian/Swedish/
French keywords covers the seed's other languages.

function is not a board field, so it gets the opposite rule from every
other raw column: extract's answer overwrites the classifier's guess
outright, and once extracted_at is set no later pull may put the guess
back. Migration 9 recomputes function for every not-yet-extracted role from
its stored title and department.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-29 14:29:12 +02:00
Wolf c6246af994 Merge branch 'worktree-agent-ac3c9345edd29a1c6'
# Conflicts:
#	app/cli.py
#	app/jobs/sources.py
#	app/static/app.css
#	app/static/companies.js
#	app/web/api.py
#	tests/test_web.py
2026-09-29 14:15:10 +02:00
WolfandClaude Opus b43fc54565 One pull at a time: the run is claimed in the database and its marker rides in each batch
A second pull (schedule, Run button, `app job pull` in another process) could
start while one was fetching and overwrite the stored run marker, so the first
run's boards failed their stamp check or a settle closed on the other run's id
lists. `claim_run` sets `pull:run` in one conditional statement, only while it
equals `pull:settled` or is older than an hour; a refused `pull` job queues no
stage and says so in its result and a notice, and the CLI exits 1. Batches
carry the marker in their payload, the plugins stamp with it, and the settle
reads it off its own batches' payloads before releasing the run.

Co-Authored-By: Claude Opus <noreply@anthropic.com>
2026-09-29 14:12:17 +02:00
WolfandClaude Opus 5.5 3b209d628a Every way a company gets a board is set_board, and an unreadable host keeps its slug
`app.jobs.sources.set_board` attaches a board to an existing company,
replaces the one it has (closing the old board's open roles) or creates
the company, with row and subscription changed in one transaction by
`Store.set_company_board`. A board that lists zero roles is kept. `add`,
`discover --add`, the seed importer (moved out of app/cli.py) and the new
`POST|DELETE /api/companies/{key}/board` all go through it; a seed re-run
attaches a board to a row that never had one. A company's subscription is
rewritten in place, and detaching hides it.

`companies.ats_slug` (migration 9) keeps the slug of a host the app
cannot read yet, and discovery returns such a host before probing slugs.
The Companies rows get Set board / Change, the host select is rendered
from `plugins.ats.HOSTS`, and a row with nothing found reads "board not
found" and is in the "Not readable yet" filter.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-29 14:04:52 +02:00
WolfandClaude Opus c7da12b93d Paged boards are read to their stated total, and a board failing for 21 days closes
Workable and SmartRecruiters page their lists, and both parsers read page one
only, so the settle closed every role past it (Delivery Hero: 980 stated, 100
open). `_paged` follows nextPage and offset, and the plugin stamps
`ats:complete` only when the distinct ids read equal the stated total; a 429
after page one keeps what was read and closes nothing. Adzuna completes on its
`count`, Jobindex counts raw items and raises on an answer with no channel.

A board whose fetch raises is recorded under `ats:failing:<target>` with the
first failure's time and the latest reason; after STALE_DAYS the settle closes
its roles, and its next answer reopens them. `/api/companies`, the Companies
list and `app companies check` show it.

Co-Authored-By: Claude Opus <noreply@anthropic.com>
2026-09-29 14:02:49 +02:00
WolfandClaude Fable 5.1 8532e4e739 Runs is the queue's elements and a Problems log; a Kill lands between boards
Run one stage goes: the cascade is the unit of work and app job keeps the
per-stage entry points. The pull button is disabled while a pull runs and
the lede no longer hardcodes the cadence. Messages folds into Runs as
Problems. pull_batch checks for a cancel before every board, one plugin per
target, and both plugins now close their HTTP client.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-28 16:10:38 +02:00
WolfandClaude Fable 5.1 51d1e67ff9 A European seed of 362 employers, readable or not; discover finds a company's board from its site
The seed was jobhunt-hq's 186 US boards, chosen because they could be read. It is now a hand-picked list of European tech employers, with `board` filled by the new `app.jobs.discover` from each company's own careers site and every board fetched once before it landed: 175 have one, and the rest keep the ATS host their site runs on (Teamtailor 32, Workday 12, SuccessFactors 9, ...) and a careers URL, as rows the app cannot read yet. Companies shows them as "not readable yet" with a link, behind a filter, so the next parser to write is the host that filter shows most.

`app companies discover NAME DOMAIN [--add]` reads the home page, its careers links and the usual paths for a link into a known host, one level down on a careers page, and probes Greenhouse, Lever, Ashby, SmartRecruiters, Recruitee and Personio by slug when the pages link nothing. It is what the LinkedIn flow will call.

Migration 7 adds companies.ats, country and city. The Workable parser now reads the v3 shape its list endpoint actually sends (location object, department list, shortcode and no URL or text); the first populated board showed it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-28 15:27:14 +02:00
WolfandClaude Opus 5.5 6b40702a9a Sources tab: searches, company boards and the prune by region, in the app
What a pull reads can now be changed on a page instead of only the command
line: try a search and see its first ten results before saving it, edit or
remove one, follow a company's ATS board, switch any board or search on or
off, and preview and apply a prune by region. Each row shows its open roles
and the countries they are in.

The page and `app companies`/`app searches` call the same functions in
app/jobs/sources.py. Switching a target off closes its open roles, and the
pull's settle closes them again if a batch in flight reopened them. Roles
pulled before `country` existed are backfilled at startup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-24 15:15:04 +02:00
WolfandClaude Opus 5.5 047f7cd8ad Find roles by search, not only by company: Jobindex and Adzuna searches
The ats boards only read companies someone listed, and the seed list is
US-centred: 31 of 29,000 open roles were in Denmark. A `search` plugin runs
saved queries on boards that span companies, pulled in the same task as the
boards: Jobindex (Denmark, RSS, no key) and Adzuna (UK and most of the EU,
free key, built to its documented API). Jobnet and Jooble were checked and
left out.

`country` is now filled from the board's location line by a lookup, with EU
for Europe-wide roles, so country gates work without extract. `app companies
prune --in` unsubscribes boards with no open role in a region. The starter
agents and searches are Copenhagen and Europe.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 19:55:50 +02:00
WolfandClaude Opus 5.5 59a301c351 Wire the app like the boilerplate: door, queue, schedules, eight tabs
app/main.py assembles it in the boilerplate's order, auth.connect first,
then the queue with the scheduler and the pages. TABS is Roles,
Applications, Agents, Companies, Jobs, Schedules, Messages and Account:
the alpha page's three views become templates on the boilerplate's
base.html, driven by static/roles.js (the old app.js) through api() and
toasts.js, over the unchanged JSON routes. Companies lists every board with
its open roles and the affinity control. A 422 is one sentence, so a toast
never reads [object Object].

Config takes the boilerplate's shape: APP_CONFIG or config.yaml, a missing
file is the defaults, an unknown key is an error. roles.yaml is gone;
config.yaml is the committed default and carries the starter agents.

pull is now a job that asks for the pull cascade, because a schedule
enqueues a job kind; its default schedule, every 6h, is declared beside it
in app/jobs/cascade.py. pull_batch's targets are keyword-only so the Jobs
tab offers no batch button. `app worker` is the boilerplate's; `app job`
still drains a queue of its own and fires no schedules.

test_coupling holds pipeline, auth and desk apart (desk may import
pipeline) and plugins to desk; test_architecture keeps app's layers, now
with main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 12:48:12 +02:00
WolfandClaude Opus 5.5 7a63311714 Rename the package roledesk to app
The boilerplate's rule: the deploy kit's Dockerfile, the APP_CONFIG,
APP_PASSWORD and APP_DATA variables and its commands all assume a package
called app, so a project renames its display title and nothing else. The
console script is `app`, and the launch entry runs `uv run app serve --port
8790`. The domain noun stays `role`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 12:30:10 +02:00
WolfandClaude Sonnet 5 867ebe6405 Raise the ats body cap to 96 MB, retry Greenhouse without content=true
openings.go's 8 MB cap was written against arbitrary URLs; every URL here is
on a fixed allow-list of seven ATS hosts, and the real pull of 2026-09-23
refused Anduril, Anthropic, OpenAI, Databricks, Carvana, Axon and Airwallex
for it. A board that still exceeds 96 MB now retries once without
content=true (Greenhouse only), trading the description for the listing.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 11:39:39 +02:00
WolfandClaude Sonnet 5 21ed472a87 Read Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee and
Personio boards

The `ats` plugin: one Source per host's public JSON (Personio: XML), each
parsed to a common row and yielded as a `role` item. The field map, the
redirect allow-list, the 8 MB cap and the 10 s timeout are ported from
jobhunt-hq's openings.go (MIT); the nine-function classifier is ported from
its seedgen. Workable, SmartRecruiters, Recruitee and Personio are new,
written from one real board of each fetched while this was built. Fixtures
under tests/fixtures/ats/ keep every test off the network.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-09-23 11:27:14 +02:00
WolfandClaude Opus 5.5 36dbcbb29e Skeleton: roledesk on desk and pipeline from ../news
Package layers models, config, storage, jobs, web, cli, held by
tests/test_architecture.py with desk and pipeline as external trees and
plugins/ limited to desk. roles.yaml is the default config, roledesk.db
its own file, port 8790. create_app mounts pipeline.web.connect over
Jobs(config.database); `roledesk job pull` drains a no-op pull and
`roledesk db init` creates the file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 10:59:17 +02:00