Rank by the cut: fetch gate overrides in scoring, the cut per agent, stages read the cut, fetch text renamed from enrich, the feed kept to the fetch gate, and migration 32

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Opus 5.5 committed 2026-10-04 15:11:43 +02:00
1 parent 060ccf66e6
commit 617356e60c
35 files changed
+1054 -751

No files matched your search

+3 -3
View File
@@ -37,8 +37,8 @@ whose reader cannot narrow that reach is read whole anyway, and sized by
that read once it is complete, rather than by a second request.
A pull reads listings only. A host that states a role's text on the role's
own page (`text="detail"`) is read there by `app/jobs/enrich.py` after
scoring, best roles first, through `read_board_details`. A search provider
own page (`text="detail"`) is read there by `app/jobs/fetch_text.py` after
scoring, for the roles in the agents' cut, through `read_board_details`. A search provider
whose reader has `detail` (Jobbank) is read the same way, under its search
target `provider:name`; `read_detail` is the one read the Fetch button makes.
@@ -435,7 +435,7 @@ class ATSSource(Source):
return f"{host}:{slug}:{source_id}"
# A host with `detail` states a role's text on the role's own page,
# which the `enrich` stage reads after scoring (`read_board_details`),
# which the `fetch_text` stage reads after scoring (`read_board_details`),
# never the pull. Its rows go out without a description, so the text
# a role's page gave stays, and a role read on its own page goes out
# without the listing's place and remote as well, so what its page
+6 -6
View File
@@ -215,11 +215,11 @@ is shown and embedded until `description` arrives. A reader that sees one
keeps it.
**Detail.** `detail(get, slug, row)` reads one role's own page and returns a
dict of the fields it found. The pull reads listings only; the `enrich`
stage (`app/jobs/enrich.py`) calls `detail` after scoring, so the requests
go where the ranking is: the open roles without text that an agent ranks
among its best `enrich.top` at `enrich.min_score` or above, best first, at
most `enrich.per_board` from one board and `enrich.per_run` in all, one
dict of the fields it found. The pull reads listings only; the `fetch_text`
stage (`app/jobs/fetch_text.py`) calls `detail` after scoring, so the
requests go where the model pass will read: the open roles without text in
the cut of an enabled agent, best first, at most `fetch_text.per_board`
from one board and `fetch_text.per_run` in all, one
batch per board through `ats.read_board_details` and
`_runner.read_details`, `_runner.PAGE_DELAY` apart, the roles never tried
before those whose read failed. Only `_runner.DETAIL_FIELDS` are taken:
@@ -230,7 +230,7 @@ any other key is left out with a debug log line, `seniority` among them.
`title`, `url`, `department` and `salary_text` cannot be filled from
`detail`: the listing rewrites them every pull. A 404 or a 410 counts as read; any
other failure is logged and retried on a later run; a read that gave no text
is retried after `enrich.retry_days`; a 429 ends the reads of that board for
is retried after `fetch_text.retry_days`; a 429 ends the reads of that board for
the run. A role read in detail is sent by later pulls without the listing's
place and remote, so what its own page said about them stays. Completeness
is decided by the listing alone.