Fold the review of 2026-09-30 into the decisions and open items, and delete the plan

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Fable 5.1 committed 2026-09-30 16:11:27 +02:00
1 parent a63eed2e56
commit a211cf7414
4 files changed
+29 -86

No files matched your search

+18
View File
@@ -138,6 +138,22 @@ What was decided for roledesk and the one reason for each. The longer reasoning
**`pull` is a job that starts the cascade, on a default schedule of every 6h.** A schedule enqueues a job kind and a cascade is started by one, so the `pull` job only asks for the chain; declared beside it in `app/jobs/cascade.py`, the schedule is a starting point that an edit under Runs overrides for good. `pull` is not also a task, which would draw two buttons under Runs.
**Every whole read is guarded.** The 10% guard held only filtered, narrowed and `empty="suspect"` reads, so 206 of 309 boards, holding two thirds of the open board roles, could close every role on one answer that parsed to nothing: a Greenhouse or Lever body whose shape changed is HTTP 200 and zero rows, and the settle took it for a complete read. Now `_runner.read` raises `Suspect` on any read that is not a sweep and lists under `SUSPECT_SHARE` of what the same read listed last time, once that count is at least `SUSPECT_MIN` (10) for a plain read; filtered, narrowed and suspect reads keep their floor of one. The board is then failing with that reason, its roles stay open, and the Companies row says so. A board that really shrank costs a Recheck, which resets the count through `keep_size`, or `STALE_DAYS` of the same answer, after which the read is accepted and the count reset, since by then its roles have closed anyway.
**A read that could not ask remote keeps the remote roles it has.** `ask_for` keeps only the dimensions a reader declares, so on SmartRecruiters, Phenom, SuccessFactors and Eightfold the reach "DK, SE, remote" went out as "DK, SE", the read counted as filtered, and the settle closed every remote role outside those countries: 72 on 2026-09-29 and 30 (Philips, Bosch, Coolblue, Delivery Hero), with the line saying "follows agents: DK, SE". Now the listing records `remote_unasked` (`ats:remote_unasked:<target>`), `_close_unlisted` and `_close_swept` spare roles whose `remote` is `remote`, those close when the board has not listed them for `STALE_DAYS`, and the line says "remote not asked on SmartRecruiters". New remote roles on such a board are still not read; reading it whole instead is the owner's call (`open.md`).
**A read that asks nothing is not a read.** When an `ids` dimension maps to no host id, `series` used to return nothing, the listing was complete and empty, and the settle closed the board while the line read "site lists none of DK, SE: nothing asked", a fact about the site. The listing is incomplete now, logged at warning, and the line ends "not closing".
**Every target is guarded in its batch, and the failing record goes after the stamp.** `ats._board` dropped `ats:failing` before the rows were stored, and a row that failed in `item_of` or `upsert_role` failed the whole `pull_batch` job: that board had neither a stamp nor a record, and the boards after it in the batch, the same ones every pull since batches are ordered, were not read, never closed and never aged. `run_batch` now wraps each target, records the failure under the source's failing key and moves on; a Kill still propagates. The failing key is removed after `stamp`.
**Searches share the runner's bookkeeping.** `search.py` logged a failure and moved on, wrote its own `search:ok/complete/ids`, and read through `read_listing` rather than `read`, so it had no guard: a Platsbanken answer with `hits` renamed read as complete and empty and closed the search at once. It now reads through `_runner.read` (no sweep), records `search:failing:<target>` through `record_failure`, stamps through `stamp`, and the search row shows "not answering since" as a board row does; `failing_boards` reads both prefixes and a search failing for `STALE_DAYS` closes its roles. A removed search forgets its record.
**The settle speaks.** Its counts went into the job result, which nothing renders: the settle of 2026-09-30 10:06 closed 12,588 roles with one target unanswered and one incomplete, and the Problems log stayed empty. `close_boards` now raises one `pull.settled` notice when something needs a look: every target unanswered (with its failing reason) or read incomplete (boards only, and not one mid-sweep, or the notice would fire every pull), every board that closed more than `CLOSE_NOTICE` (50) roles or more than half of at least `CLOSE_SHARE_MIN` (10) open ones, and the totals. Its result key `failed` is `unanswered`. `run_pull` and `_boards` release the run claim when building the chain raises, so a finder that fails no longer refuses every pull for an hour with "a pull is already running".
**Narrowed boards close by age.** A read by fetch terms closed nothing, so a role that left such a board stayed open for good. A complete narrowed read now closes the board's roles it did not list and that were last seen more than `STALE_DAYS` ago, the rule searches have; no agent has terms yet, so this is latent.
**Declarations are checked against fixtures.** Of the `READS` traits only `posted`, `filters`, `complete`, `ceiling`, `empty`, `key` and `scope` were read at run time; `pay`, `remote`, `country`, `seniority`, `employment`, `teaser`, `places` and `text` were checked by nothing, so Platsbanken declared `remote="none"` while its rows set it and HR Manager declared `places="one"` and returned none. `tests/test_readers_contract.py` now parses each reader's listing and detail fixtures and asserts every declared field, a missing fixture fails instead of skipping, `places` has `none`, a search module needs `PARAMS` and `clean_params` and its rows `company`, every prep table names a reader and every reader stating words has a table, and `robots` must carry a date and name the paths the reader requests: seven records named a page the reader never asked for. Every host's robots.txt was read again on 2026-09-30; `api.smartrecruiters.com` disallows `/` for every agent but LinkedInBot, which is with the owner. The paths for modules without `READS`, `robots_page` and `ROBOTS_ANY_CASE` are gone, and `discover._UNREADABLE` lost the ten hosts that have modules.
## Ranking
**Scoring is per search agent.** A role is scored as `(uid, agent) -> score`, because a role is only good for a given search. Each gate has a mode: `require` becomes a SQL `WHERE`, `prefer` becomes a weighted term in the score. Gated roles keep their row with a reason. `companies.affinity = 2` skips the `any` group, `-1` mutes.
@@ -210,6 +226,8 @@ What was decided for roledesk and the one reason for each. The longer reasoning
**`GET /api/roles` serves a row, not a role.** It used to answer `roles.*`, the whole `Role` (`description`, `feed_text`, `raw` and `article_text` included, about 5.5 KB a role on average) for a feed that only ever shows a title, a few badges and an excerpt; a 1000-row restore was several megabytes of text a phone never renders. `Store.roles_page` selects only the fields a row draws (`RoleRow`), with `excerpt` built by `app.models.excerpt` off a `SUBSTR` of `summary` or `description`, never the column whole. Measured on a copy of the live database: a 60-row page went from 392 KiB to 39 KiB and a 1000-row restore from 6.6 MB to 631 KiB, both about a tenth. The reader is unaffected: it already fetched the whole role by uid when it opened one (`GET /api/roles/{uid}`, still the full `Role`); what changed is that the list row it painted from first, before that fetch lands, no longer carries a body to paint with, so the reader now waits on the fetch for the posting text and the original link the same way it always waited for the summary.
**A board's limits are words on its row, and a filter.** The readers declare what they cannot deliver and the runner records how a board was read, and none of it reached the Companies list beyond the reach line and "not answering since". `app/jobs/limits.py` turns both into words per board, in a fixed order: `failing`, `read incomplete`, `swept`, `filtered by reach`, `remote not asked`, `narrowed by terms`, `nothing asked`, `size 0`, `over the ceiling`, `text per role` or `no text` or `excerpt only`, `no pay`, `date not trusted`, `without text: N`, `text failed: M`, `read whole`. The row shows them after the size, the Companies filter offers each as an option under Limits, and `board_open` beside the company's count says how many open roles the board itself has, since the company's count includes search roles. The readers list on Sources shows every trait a reader declares.
## The phone
**Two tabs and a cog.** `TABS` in `app/web/pages.py` becomes Roles and Applications; the other seven pages (Agents, Companies, Sources, Jobs, Schedules, Messages, Account) move behind a settings cog at `/settings/<name>`, because nine tabs wrapped to three rows on a phone and six of them were setup or system pages. The Watchlist became a third tab and the settings pages became three sub-tabs, both below.