Files
jobboard/docs/open.md
T

56 lines
9.4 KiB
Markdown

# Open
What is known and not yet fixed, checked against the code on 2026-09-30. What landed is in `decisions.md`; the unbuilt readers are planned in `plan-hosts-wave-2.md`.
## Text
- **A hand-fetched posting is not embedded or extracted.** "Fetch the posting" stores `article_text`, which only `summarise` reads. `embed` and `extract` read `description or teaser`, so a Jobindex role keeps being ranked on its teaser. The fix is one text rule, `article_text or description or teaser`, in embed and extract, and queuing embed and score after a fetch.
- **An edited posting is never embedded again.** The vector is dropped only when the text ranks better than before (a title-only role gaining a teaser, or a teaser replaced by a description). A hash of the embedded text on `vectors` would re-embed on any change.
- **Extract marks a title-only role done for good.** `upsert_role` does not reset `extracted_at` when text arrives. Extract is on, so this bites. `_EXTRACT_WAITING` re-reads roles whose `seniority_model` is empty, which covers level and not pay.
- **Lever keeps only the intro paragraph.** `lever.py` reads `description` and drops `lists[]` and `additional`, about 80% of the text on Spotify's board. Recruitee drops `requirements` the same way.
- **Pay in the text is not in the fields.** About 1,470 roles state a currency range in the text only, mostly Greenhouse and US. Pay figures are filled for 17.9% of open roles. Extract can return `salary_min` and `salary_max` but has read 554 roles so far.
- **A title-only role has no mark in the feed.** 1,180 open roles have no description, teaser or fetched posting: 677 of them Workday, 256 SmartRecruiters, 150 Workable, 57 SuccessFactors and 26 Eightfold. A mark would show which fits are guesses.
## Ranking
- **The owner's agent has no rule on seniority.** Fit compares subject, not level. The ladder and the rail's From/To/Track filter exist (`LADDER` in `app/models.py`), and the level is filled for about half of open roles. A gate still names words (`principal`, `senior_manager`, `director`, `exec`); a gate on "at least a rung" would read as the rail's filter does.
- **Votes train `taste` only.** The liked-minus-passed centroid weighs 0.2 on both agents. Feeding votes into the prompt vectors is worth doing once there are a few hundred.
- **A starred company's roles crowd the top of Agent match.** With the companies weight at 0.2 a favourite's roles gain about 0.13; after starring Databricks it held 33 of the top 60 rows. Lower the weight on the agent's card, or star fewer companies.
## Sources
- **The boilerplate moved on.** `pipeline/` equals boilerplate `a1c6f0d` once line endings are ignored: 87 files of the working tree have CRLF although the index has LF, so a plain `diff -r` shows every file as different (use `--strip-trailing-cr`). `auth/` lacks `c6a90ff`, the phone commit (`auth/README.md`, `auth/static/account.js`, `auth/templates/login.html`). `deploy/` is at `1679e88` and lacks the WSL scripts of `e1b3746` (`wsl-setup.sh`, `wsl-setup.ps1`), and `docs/release.md` lacks its `wsl.md` sentence. `desk/` is news `a275c38` less the prune and lags news by four commits (`b0e1f6f`, `f01a6bd`, `3bb6775`, `7b9c8a2`). The three languages of boilerplate `7bb7da2` are starter code in `app/`, not part of the copies. Refresh with `git archive`, and decide whether `app/static/app.css` should use the kit's classes (`4573f6d`) instead of its own copies.
- **The hosts added on 2026-09-29 were read from this machine only.** Every Workday tenant is behind Cloudflare; read one Workday and one SuccessFactors board from the deployed server before relying on them.
- **Six hosts' dates are not trusted.** Eightfold, Jibe, Gem, Deel, HiBob and HR Manager declare `posted="listed"`, so their backlog counts as posted when first seen. Eightfold now maps `creationTs`, Jibe's `posted_date` equals the page's `datePosted`, and HR Manager's `pubDate` rises with the id; each needs one check on a live board before it declares `posted="trusted"`. Platsbanken's local time is read as UTC.
- **A mistyped SmartRecruiters, Gem or Ashby slug attaches as an empty board.** All three answer an empty listing for an unknown id and declare `empty="suspect"`. The 10% guard keeps a known board from closing its roles, but `set_board` keeps a new board that lists nothing, which is caught only when the person notices "0 roles".
- **`Crawl-delay` is not honoured.** Booking.com's robots.txt asks for 5 seconds; the runner waits `PAGE_DELAY`, 0.5. A read of its board is two requests.
- **A SuccessFactors site on a company's own domain is not discovered.** Its pages link only the back end and the image CDN. It needs the rule Teamtailor has in `discover.py`: a page with the markers means the page's host is the board, checked by reading `/search/`. Until then such a board is set by hand.
- **HR Manager is attached although its robots.txt disallows reading.** `hrmanager:teknologisk` (the Danish Technological Institute) has 7 open roles, while `READS.robots` says "disallowed by robots.txt; owner to decide". Seaborg stays unattached until the owner decides whether the feed counts as allowed.
- **Whether SmartRecruiters' identifier is case-sensitive is unsettled.** The module docstring says it is (`BoschGroup`), `decisions.md` says it is not, checked live; the live boards hold the identifiers with case. A live request with a lowercased identifier settles it.
- **The seed lags the live database.** The seed has 286 boards and the database 310: it lacks Microsoft, Infineon, Novonesis, Darktrace, Criteo, NVIDIA, HR Manager and seventeen more, and names five of them as `unread` or `robots`. `app companies seed` never replaces a board, so the seed is brought in line by hand.
- **Stripe's places carry a "Locations" suffix.** 316 open roles (285 without copies) have place text like "Singapore Locations" or "Dublin; Ireland Locations".
## The pages
- **★ means two things on the Roles tab.** On a row it saves the role, in the company strip it marks the company a favourite.
- **Place suggestions take about 110 ms**, most of it splitting 5,600 place lines into towns on every request. Caching the towns per distinct line would cut it.
- **`company_key` keeps `ø`.** Aliases make "Orsted" and "Ørsted" one company; folding the key itself would change the primary key of `companies`.
## From the review of 2026-09-30
- **Remote roles are not read from hosts that cannot filter on remote.** On SmartRecruiters, Phenom, SuccessFactors and Eightfold the reach "DK, SE, remote" is asked as "DK, SE"; since 2026-09-30 the filtered read keeps the remote roles it has and the line says "remote not asked", but new remote roles on those 21 boards are never fetched, and the 72 closed on 2026-09-29 and 30 (Philips 34, Bosch 25, Coolblue 10, Delivery Hero 3) reopen only if a read lists them. Reading such boards whole is the owner's call; Bosch is about 70 requests a pull.
- **SmartRecruiters' API host disallows `/` for every agent but LinkedInBot** (read 2026-09-30), which covers the listing and the detail requests; `READS.robots` says "owner to decide" and the eleven boards are read meanwhile. Jobylon allows `*` but disallows named job crawlers (JobrapidoBot, JoobleBot, AdzunaBot); ours is not one of them. HR Manager's record says the owner decided on 2026-09-29 to read the API host; confirm it.
- **`has_key` re-implements `resolve_key`**, and the form hard-codes the labels of Jobindex, Adzuna and Platsbanken; since 2026-10-03 the CLI, dupes, `search_params` and the other providers in the form come from `readers.PROVIDERS`.
- **A search role's detail places are not kept across pulls.** `plugins/search.py` has no `_KEPT_FROM_DETAIL`, so the next pull may overwrite the town Jobbank's `detail` added; the text is kept. Per-role detail state of a search is never pruned, since a search never lists everything.
- **Jobbank's JSON-LD `addressLocality` is sometimes not a town** ("Ørsted" on an Ørsted ad), and the feed's regions (`Øresundsregionen`, `Udlandet (øvrige)`) will show under Unmapped.
- **freehire states seniority and employment type, but the reader declares neither**, since a stated word needs a table in `app/prep/hosts/`. Jobnet keeps `utm_*` on employer links; dupes match on company, title and town, so it does not matter there.
- **`ats.py` keeps compatibility names that only tests reach for:** `_PARSERS`, `_parse_dt`, `_greenhouse_salary`, `_parse_teamtailor`, `_Refused` and `DETAIL_FIELDS`.
- **`title="provisional"` cannot work while `DETAIL_FIELDS` has no `title`.** Wave 2's WP9 must own `_runner.py` (`plan-hosts-wave-2.md`).
- **Some `posted="trusted"` declarations rest on the detail read** (Workday, BambooHR, Rippling) **or on a fallback the host moves** (Greenhouse `updated_at`, Phenom `postedDate`).
- **Dupes fold same-host roles with the same title and town without comparing text.** 99 open copies share under 30% of their words with the kept role.
- **Fetch text never reads a role outside the cut.** A role without text is ranked on its title until it enters the cut.
- **A pull never re-measures a board's size.** Recheck is the only way, so a board that grows past 300 roles stays read whole until someone rechecks it.
- **Fetch terms are configured on no agent**, so no board is read with terms today.
- **`SMALL` (300), `STALE_DAYS` (21), the guard's 10% and `DETAIL_CAP` (200) are code constants**, and the pages hard-code 300 in tooltips.
- **`The todo list.md` holds items the code has done** (Jobindex robots, Eightfold cap, places, aliases, teaser, ladder, hybrid on Ashby); the owner prunes it.