Rank by the cut: fetch gate overrides in scoring, the cut per agent, stages read the cut, fetch text renamed from enrich, the feed kept to the fetch gate, and migration 32
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
060ccf66e6
commit
617356e60c
35 files changed
+1054
-751
No files matched your search
+3
-3
@@ -37,8 +37,8 @@ whose reader cannot narrow that reach is read whole anyway, and sized by
|
||||
that read once it is complete, rather than by a second request.
|
||||
|
||||
A pull reads listings only. A host that states a role's text on the role's
|
||||
own page (`text="detail"`) is read there by `app/jobs/enrich.py` after
|
||||
scoring, best roles first, through `read_board_details`. A search provider
|
||||
own page (`text="detail"`) is read there by `app/jobs/fetch_text.py` after
|
||||
scoring, for the roles in the agents' cut, through `read_board_details`. A search provider
|
||||
whose reader has `detail` (Jobbank) is read the same way, under its search
|
||||
target `provider:name`; `read_detail` is the one read the Fetch button makes.
|
||||
|
||||
@@ -435,7 +435,7 @@ class ATSSource(Source):
|
||||
return f"{host}:{slug}:{source_id}"
|
||||
|
||||
# A host with `detail` states a role's text on the role's own page,
|
||||
# which the `enrich` stage reads after scoring (`read_board_details`),
|
||||
# which the `fetch_text` stage reads after scoring (`read_board_details`),
|
||||
# never the pull. Its rows go out without a description, so the text
|
||||
# a role's page gave stays, and a role read on its own page goes out
|
||||
# without the listing's place and remote as well, so what its page
|
||||
|
||||
@@ -215,11 +215,11 @@ is shown and embedded until `description` arrives. A reader that sees one
|
||||
keeps it.
|
||||
|
||||
**Detail.** `detail(get, slug, row)` reads one role's own page and returns a
|
||||
dict of the fields it found. The pull reads listings only; the `enrich`
|
||||
stage (`app/jobs/enrich.py`) calls `detail` after scoring, so the requests
|
||||
go where the ranking is: the open roles without text that an agent ranks
|
||||
among its best `enrich.top` at `enrich.min_score` or above, best first, at
|
||||
most `enrich.per_board` from one board and `enrich.per_run` in all, one
|
||||
dict of the fields it found. The pull reads listings only; the `fetch_text`
|
||||
stage (`app/jobs/fetch_text.py`) calls `detail` after scoring, so the
|
||||
requests go where the model pass will read: the open roles without text in
|
||||
the cut of an enabled agent, best first, at most `fetch_text.per_board`
|
||||
from one board and `fetch_text.per_run` in all, one
|
||||
batch per board through `ats.read_board_details` and
|
||||
`_runner.read_details`, `_runner.PAGE_DELAY` apart, the roles never tried
|
||||
before those whose read failed. Only `_runner.DETAIL_FIELDS` are taken:
|
||||
@@ -230,7 +230,7 @@ any other key is left out with a debug log line, `seniority` among them.
|
||||
`title`, `url`, `department` and `salary_text` cannot be filled from
|
||||
`detail`: the listing rewrites them every pull. A 404 or a 410 counts as read; any
|
||||
other failure is logged and retried on a later run; a read that gave no text
|
||||
is retried after `enrich.retry_days`; a 429 ends the reads of that board for
|
||||
is retried after `fetch_text.retry_days`; a 429 ends the reads of that board for
|
||||
the run. A role read in detail is sent by later pulls without the listing's
|
||||
place and remote, so what its own page said about them stays. Completeness
|
||||
is decided by the listing alone.
|
||||
|
||||
Reference in new issue
Block a user