Prune done work from the docs: fold the ingest review's open findings into open.md and delete it, cut the hosts wave-2 plan to what is unbuilt, drop the smaller-hosts survey

The plugins README no longer describes a run-time robots.txt check or boards that never close past the page cap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Opus 5.5 committed 2026-09-30 13:40:43 +02:00
1 parent a44bf5dacb
commit 802656aca4
8 files changed
+70 -472

No files matched your search

+28 -19
View File
@@ -1,26 +1,35 @@
# Open
What is left after the phone rework of 2026-09-25, for whoever picks the repo up next, in the order it bites.
What is known and not yet fixed, checked against the code on 2026-09-30. What landed is in `decisions.md`; the unbuilt readers are planned in `plan-hosts-wave-2.md`.
- **The checkout moved.** `CLAUDE.md`, `README.md`, `desk/README.md` and the refresh commands assume `../boilerplate` and `../news` beside the checkout. Keep them beside it in the sandbox or change the paths. `.venv` holds absolute paths, so `uv sync --all-extras` again after the move.
- **The boilerplate moved on.** Since roledesk's last refresh (`77052a8`, the schedules fix) it has `4573f6d` (the kit: the layout patterns every app built for itself, as classes) and `1679e88` (deploy from a WSL checkout, the sandbox setup). Refresh `deploy/` and `pipeline/` with `git archive`, and see whether `app/static/app.css` should use the kit's classes instead of its own copies of the same rules.
- **Summarising is off** until the switch under Runs (default `summarise.enabled: false`) is used, and needs a model backend, as extract does. The Summarise button on a read page works either way once a backend is configured.
- **The owner's agent has no rule on seniority.** Fit compares subject, not level, so an individual contributor's posting ranks beside a head's. Since 2026-09-30 the level is a rung on a ladder with two tracks (`LADDER` in `app/models.py`), filled for half of open roles; a prefer gate still names words (e.g. `principal`, `senior_manager`, `distinguished`, `director`, `exec`). A gate on "at least a rung" would read as the rail's filter does. Extract fills the rest.
- **Votes do not train fit.** A vote only moves the rank at read time. Feeding votes into the prompt vectors is worth doing once there are a few hundred.
- **76 companies are read by hand**, on the Watchlist with a careers link opened on 2026-09-29: 35 `unread`, 10 `robots`, 9 Teamtailor ones without a career host, 8 `group`, 7 `blocked`, 4 `gone`, 3 `key`. The remaining readers are planned in `docs/plan-hosts-wave-2.md`.
- **The hosts added on 2026-09-29 were read from this machine only.** Every Workday tenant is behind Cloudflare; read one Workday and one SuccessFactors board from the deployed server before relying on them.
- **An unknown SmartRecruiters or Gem id answers an empty listing, not 404**, so a mistyped slug reads as a complete board with no roles and would close a company's roles. `set_board` reads the board first, which catches a typo only when the person notices "0 roles".
- **`Crawl-delay` is not honoured.** Booking.com's robots.txt asks for 5 seconds; the shared reader waits `PAGE_DELAY`, 0.5. A read of its board is two requests.
- **Eightfold returns 10 roles a page and `MAX_PAGES` is 100**, so Infineon (1,289 roles) and Microsoft (about 2,300) would never be read complete and are not attached. The endpoint takes a location filter; a target has nowhere to state one.
- **A SuccessFactors site on a company's own domain is not discovered from its own pages**, which link only the back end and the image CDN. It needs the rule Teamtailor has in `discover.py`: a page with the markers means the page's host is the board, checked by reading `/search/`. Until then such a board is set by hand.
- **HR Manager's portal disallows reading in robots.txt, its feed host has no robots.txt.** The module checks the feed's host, so a board would be read. Seaborg, the one company on it, is not attached until the owner decides whether the feed counts as allowed.
- **Six new hosts' dates are not trusted** (Eightfold, Jibe, Gem, Deel, HiBob, HR Manager; Pinpoint states none), so their backlog counts as posted when first seen. Check each date on a live board and add the host to `TRUSTED_POSTED_HOSTS`.
- **SmartRecruiters and Workable roles have no text.** Their lists carry none and their modules have no `detail`, so 7,391 open roles (Bosch, Wise, Ubisoft, Delivery Hero) are ranked on the title alone. Both hosts have a per-posting endpoint; a `detail` in each module fills the text 200 roles per board per pull.
- **A posting fetched by hand feeds the reader only.** "Fetch the posting" stores `article_text`, which `summarise` reads and `embed` and `extract` do not.
- **A list's teaser is not stored.** Phenom's and Oracle's lists carry a short description, which is dropped so that it cannot replace the full text; until the role's page is read the role has no text at all.
## Text
- **A hand-fetched posting is not embedded or extracted.** "Fetch the posting" stores `article_text`, which only `summarise` reads. `embed` and `extract` read `description or teaser`, so a Jobindex role keeps being ranked on its teaser. The fix is one text rule, `article_text or description or teaser`, in embed and extract, and queuing embed and score after a fetch.
- **An edited posting is never embedded again.** The vector is dropped only when a teaser is replaced by a description. A hash of the embedded text on `vectors` would re-embed on any change.
- **Extract marks a title-only role done for good.** `upsert_role` does not reset `extracted_at` when text arrives. Extract is on, so this bites. `_EXTRACT_WAITING` re-reads roles whose `seniority_model` is empty, which covers level and not pay.
- **Lever keeps only the intro paragraph.** `lever.py` reads `description` and drops `lists[]` and `additional`, about 80% of the text on Spotify's board. Recruitee drops `requirements` the same way.
- **Pay in the text is not in the fields.** About 1,470 roles state a currency range in the text only, mostly Greenhouse and US. Pay figures are filled for 16.5% of open roles. Extract can return `salary_min` and `salary_max` but has read about 500 roles so far.
- **A title-only role has no mark in the feed.** 1,304 open roles have no text, 869 of them Eightfold. A mark would show which fits are guesses.
## Ranking
- **The owner's agent has no rule on seniority.** Fit compares subject, not level. The ladder and the rail's From/To/Track filter exist (`LADDER` in `app/models.py`), and the level is filled for about half of open roles. A gate still names words (`principal`, `senior_manager`, `director`, `exec`); a gate on "at least a rung" would read as the rail's filter does.
- **Votes train `taste` only.** The liked-minus-passed centroid weighs 0.2 on both agents. Feeding votes into the prompt vectors is worth doing once there are a few hundred.
- **A starred company's roles crowd the top of Best first.** With the companies weight at 0.2 a favourite's roles gain about 0.13; after starring Databricks it held 33 of the top 60 rows. Lower the weight on the agent's card, or star fewer companies.
- **The owner's agent passes any role in Sweden and Germany.** Its `any` group holds `country` DK, SE, DE, which is why SAP in Garching ranks. For Denmark, remote and Skåne only, the group is `country` DK, `remote` remote and `place` `region:skane`.
## Sources
- **The boilerplate moved on.** `deploy/` is at boilerplate `1679e88` and lacks `wsl-setup.sh` and `wsl-setup.ps1`; `pipeline/` and `auth/` differ from boilerplate HEAD (three languages, installable phone). Refresh them with `git archive`, and decide whether `app/static/app.css` should use the kit's classes (`4573f6d`) instead of its own copies.
- **The hosts added on 2026-09-29 were read from this machine only.** Every Workday tenant is behind Cloudflare; read one Workday and one SuccessFactors board from the deployed server before relying on them.
- **Six hosts' dates are not trusted.** Eightfold, Jibe, Gem, Deel, HiBob and HR Manager declare `posted="listed"`, so their backlog counts as posted when first seen. Eightfold now maps `creationTs`, Jibe's `posted_date` equals the page's `datePosted`, and HR Manager's `pubDate` rises with the id; each needs one check on a live board before it declares `posted="trusted"`. Platsbanken's local time is read as UTC.
- **A mistyped SmartRecruiters or Gem slug attaches as an empty board.** Both answer an empty listing for an unknown id. `empty="suspect"` and the 10% guard keep a known board from closing its roles, but `set_board` keeps a new board that lists nothing, which is caught only when the person notices "0 roles".
- **`Crawl-delay` is not honoured.** Booking.com's robots.txt asks for 5 seconds; the runner waits `PAGE_DELAY`, 0.5. A read of its board is two requests.
- **A SuccessFactors site on a company's own domain is not discovered.** Its pages link only the back end and the image CDN. It needs the rule Teamtailor has in `discover.py`: a page with the markers means the page's host is the board, checked by reading `/search/`. Until then such a board is set by hand.
- **HR Manager: decisions and data disagree.** `decisions.md` and `plugins/README.md` say no attached company uses it, but the live database has `hrmanager:teknologisk` attached with 0 open roles. Seaborg stays unattached until the owner decides whether the feed counts as allowed.
- **Stripe's places carry a "Locations" suffix.** 282 open roles have place text like "Singapore Locations" or "Dublin; Ireland Locations".
## The pages
- **★ means two things on the Roles tab.** On a row it saves the role, in the company strip it marks the company a favourite.
- **Place suggestions take about 110 ms**, most of it splitting 5,600 place lines into towns on every request. Caching the towns per distinct line would cut it.
- **`company_key` keeps `ø`**, so "Orsted" and "Ørsted" are two companies. Folding it changes the primary key of `companies`.
- **`company_key` keeps `ø`.** Aliases make "Orsted" and "Ørsted" one company; folding the key itself would change the primary key of `companies`.