Seed 186 companies, add roledesk companies, and pull and close every board
seed/companies.json: 179 companies from jobhunt-hq's seed.json (MIT), reduced to key/name/board/careers_url, plus a hand-added DACH set. `roledesk companies seed|add|check` import it, add one company from its board with a live check first, and report dead boards. roledesk/jobs/pull.py is now a pipeline task: ten boards per pull_batch job on the http resource, upserting every role the ats plugin yields. Its final job's settle closes every board this run answered in full and no longer lists a role from, reading the plugin's own self.state stamps rather than each batch job's return value, so one bad board in a batch does not cost the other nine their closure. `roledesk job pull` runs it, and it registers itself in pipeline.TASKS so <desk-jobs> can start it too. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
21ed472a87
commit
0f59452b2a
5 files changed
+1684
-10
No files matched your search
@@ -30,6 +30,14 @@ What was decided for roledesk and the one reason for each. The longer reasoning
|
||||
|
||||
**`subscriptions` names its labels `company` and `board`.** newsdesk's `outlet` and `section` are news words; roledesk has no rows to migrate, so it uses its own names from the first migration.
|
||||
|
||||
## Ingestion
|
||||
|
||||
**Closure is decided from `self.state`, not from what a `pull_batch` job returns.** The `ats` plugin stamps `ats:ok:<target>` with the run's own marker and `ats:ids:<target>` with the ids it saw on every board it parses without raising. The `pull` task's settle then closes a board only when its stamp matches the marker the finder wrote before any batch ran. A batch job that raises part way through a ten-board chunk still leaves the boards it had already finished stamped, so one bad board does not cost its batch-mates their closure — and a board a batch never got to (the whole job crashed, or was never claimed) stays open rather than being closed on an empty id list.
|
||||
|
||||
**SmartRecruiters' board list has no description.** Its `/postings` endpoint gives title, location and department but not the job text; that is only on the per-posting detail endpoint, and fetching one extra request per role would multiply a board's request count by however many roles it lists. `description` stays empty for this host; title, location, department and the derived `function` are still filled.
|
||||
|
||||
**Workable's shape is confirmed, not a populated example.** Every slug tried against `apply.workable.com/api/v3/accounts/{slug}/jobs` during this build returned the documented `{"total": 0, "results": []}` before Cloudflare rate-limited further requests from this session, so the parser is written from Workable's long-published job object rather than from a fetched row. Worth a fixture-level recheck against a live board with open roles the next time this plugin is touched.
|
||||
|
||||
## Ranking
|
||||
|
||||
**Scoring is per search agent.** A role is scored as `(uid, agent) -> score`, because a role is only good for a given search. Each gate has a mode: `require` becomes a SQL `WHERE`, `prefer` becomes a weighted term in the score. Gated roles keep their row with a reason. `companies.affinity = 2` skips the gates and floors the score, `-1` mutes.
|
||||
|
||||
Reference in new issue
Block a user