Storage: roles, companies, subscriptions, applications, agents

roledesk/storage/schema.py creates the tables at v1 with WAL and the
versioned-migration machinery from newsdesk; store.py adds upsert_role
(unpacks raw into gate columns, first_seen on insert, last_seen every
time), roles(), mark_closed, companies and subscriptions CRUD, and
save_application writing an append-only application_events row on every
status entered. Key helpers company_key, title_key, place_key live in
models.py. docs/decisions.md records the key decisions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Opus 5.5 committed 2026-09-23 11:03:51 +02:00
1 parent 36dbcbb29e
commit ac5eca874a
5 files changed
+949 -7

No files matched your search

+45
View File
@@ -0,0 +1,45 @@
# Key decisions
What was decided for roledesk and the one reason for each. The longer reasoning is in `../news/docs/JobBoardPlan/roles-build-plan.md` and `jobhunt-hq-assessment.md`.
## Shape
**Own repo, depending on news for `desk` and `pipeline`.** `pyproject.toml` takes `newsdesk` as an editable path dependency on `../news`. `desk` imports no app and news's architecture test enforces it, so it works as a library. Copying it would fork the plugin contract; `pipeline` alone may be copied later if depending on a repo named news becomes the problem.
**`desk` and `pipeline` are never edited here.** A change roles needs lands in news, and roledesk picks it up through the editable install.
**The domain noun is `role`.** `jobs/` keeps meaning the work the app queues, so "job" never has two senses in one sentence.
**Own database, queue and port.** `roledesk.db`, `roles.yaml`, port 8790. A shared file with newsdesk would buy "articles about this company" by cosine, which is worth less than two apps that can migrate independently now that `Jobs` takes a path.
**Storage is written, not imported.** `desk` has no storage package on purpose, so ~120 lines of connection plumbing are copied from newsdesk. If the same edit lands in both a second time, a shared `Connection` moves into `desk` below `pipeline`.
**Layers are `models, config, storage, jobs, web, cli`.** `tests/test_architecture.py` holds the direction, the doors, and that a plugin imports `desk` only, since a plugin is app-agnostic.
## Data
**Clustering key is `(company_key, title_key, place_key)`.** newsdesk's `location_key OR title_key` collapses two outlets running one story, which is right for wire copy and wrong for ads, where the same title at two companies is two jobs.
**`place`, `country` and `remote` are separate columns.** `location` already means where an `Item`'s original lives. `remote` alone over-matches, because boards write "UK | Remote", so a gate on remote also reads `country`.
**`raw` is transport, columns are storage.** No `Item.facets`. A plugin puts structured fields in `raw` under the column's name and `upsert_role` unpacks them into indexed columns, which is where gates query them. An empty value never overwrites a filled column, so a terse board does not erase what `extract` found.
**Postings are persisted.** Storing every posting is what gives `first_seen`, which is the only honest posted date since Greenhouse resets `updated_at` on every edit, and `closed_at`, set by `mark_closed` when a complete fetch of a board no longer lists a role. A role listed again is open again.
**The tracker schema is taken from jobhunt-hq (MIT).** `applications` with status in `saved, applied, screen, technical, onsite, offer, rejected, withdrawn`, `next_action`, `next_date`, and an append-only `application_events` written by the store on every status entered. The events table is what "days to first screen" and conversion by source are computed from.
**`subscriptions` names its labels `company` and `board`.** newsdesk's `outlet` and `section` are news words; roledesk has no rows to migrate, so it uses its own names from the first migration.
## Ranking
**Scoring is per search agent.** A role is scored as `(uid, agent) -> score`, because a role is only good for a given search. Each gate has a mode: `require` becomes a SQL `WHERE`, `prefer` becomes a weighted term in the score. Gated roles keep their row with a reason. `companies.affinity = 2` skips the gates and floors the score, `-1` mutes.
**`fit` is cosine between the role and the agent's intent text.** The intent is a CV or a paragraph per agent. This is why the embedding backend is effectively required: TF-IDF cannot relate "Kubernetes" to "container orchestration".
**Summary and tagging are off; `extract` replaces them.** Every profile declares `SummaryPolicy(enabled=False)` and `TaggingPolicy(enabled=False)`. `extract` is a roledesk `ModelTask` on Haiku that turns title and description into the gate columns, and skips rows a plugin already filled from structured fields.
## Outside the app
**Unposted roles and the network stay outside.** Their state is inside: an application row with no `role_uid`, a `contact`, `next_action` and `next_date`, and a company `status` such as researching or contacted.
**LinkedIn enters only as alert mail over IMAP.** Scraping it is banned by contract and enforced. The alert mails also carry personalised recommendations no public API exposes.