Files
jobboard/docs/architecture.md
T

194 lines
12 KiB
Markdown

# Architecture
The target design of roledesk's ranking: how a role gets from a board to the Roles feed for one search agent, and the key decisions behind it. What gets built is tested against this document.
- **Fetch gate**: the agent's hard rules, a yes or no per role, in four parts: world, location, role exclusion and role fetch.
- **Analytics score** and **analytics filter**: what decides the **cut**, the roles that pass the analytics filter and get fetch text and the model pass.
- **Feed scores**: the two orders of the feed, the want feed score and the match feed score.
- **Fetch text**: the second HTTP read of a role's own page, without a model.
- **Extract**, **judge**, **summarise**: the model calls (Haiku).
- **Affinity**: an agent's setting for a company: hidden, neutral, liked or favourite.
- **All locations**, **All roles**: company switches, the same for every agent, that bypass the location and the role parts of the fetch gate.
## Flow
```
1 Fetch ─ all relevant roles, limited by the fetch gate:
world · location · role exclusion · role fetch, with All locations and All roles as overrides
company boards: whole under 300 roles, else filtered at the host where it can
search providers: their own saved query
│
▼
2 Analytics score, no tokens ─ prompt closeness, CV closeness, affinity → the cut
│
▼
3 Fetch text ─ the role's own page, for cut roles without text
│
▼
4 Model pass ─ extract the cut → fetch gate and cut again → judge the cut
│
▼
5 Feed ─ every role that passes the fetch gate, in two orders;
filters and saved filters narrow it
│
▼
6 After the ranking ─ summarise the top of both orders
```
## 1 Fetch
The aim is to fetch every relevant role while keeping out the thousands that are not, before anything is known about them in detail. The fetch gate does that.
### 1.1 Fetch gate
#### 1.1.1 World fetch gate
The outer boundary of the agent, e.g. the countries of a region. Every role must be in it. `All locations` falls back to this boundary.
#### 1.1.2 Location fetch gate
The stricter location filter. A role passes the `location fetch gate` when it matches at least one of the set locations: a country, a town, or remote. If none is set, a role can be anywhere within the `world fetch gate`.
#### 1.1.3 Role exclusion gate
A list of titles to leave out: phrases that reject a role by its title. No host can filter by exclusion, so this gate is applied to every role after fetch.
#### 1.1.4 Role fetch gate
The fetch terms: words a big board's host must match before it sends a role. The `role fetch gate` is only applied at the host, as a blunt tool for avoiding thousands of roles from a big board. It is ignored afterwards, since `prompt closeness` is the more nuanced downstream filter.
#### Overrides
- A company set to **All locations** bypasses the `location fetch gate` (1.1.2). Its roles are fetched and kept across the entire `world fetch gate`.
- A company set to **All roles** bypasses the `role exclusion gate` and the `role fetch gate` (1.1.3 and 1.1.4): no fetch terms and no titles to leave out. It is for small companies worth a speculative approach, whatever the role.
- A search provider is read by its own saved query, which replaces the `role fetch gate` (1.1.4).
- A hidden company never passes.
The gate rejects on evidence only. A role with an unknown location passes until extract has read it.
### 1.2 Reading a board
- A company board with fewer than 300 roles is read whole.
- A company board with 300 or more roles is asked for the fetch gate at the host: the world and location gates, and the fetch terms, where the host can filter on them. A host that cannot filter on a part sends the board whole for that part.
- A board's size is measured once, when it is added or rechecked, not on every read.
- One fetch serves all enabled agents. The host is asked for the union of their world and location gates and the union of their fetch terms; one enabled agent without fetch terms means no terms. Each agent then applies its own gate to what arrived.
### 1.3 Closing
- A role closes when a complete read of its board no longer lists it.
- A read that is not complete closes nothing: one narrowed by fetch terms, a search provider's, a partial or failed read. Roles on such boards close by age. This is accepted because ranking relies on recency, not on closing.
- A role that no read has listed for 21 days closes. A board that has failed on every read for 21 days closes its roles.
- A liked or saved role is never closed by a filtered read.
### 1.4 Copies
- One posting listed several times (by a search provider and by the company's board, or by one board under several ids) is one role. Two listings are one posting when they share company, title and country, and their towns overlap; a listing without a town matches any town.
- The kept copy is the company board's over a search provider's, then the one with text, then the earliest posted. Only the kept copy is scored, read by a model and shown; it lists the others.
## 2 Analytics score and filter
Each role is embedded and scored semantically:
- **Prompt closeness** is its level against the best of the agent's prompts.
- **CV closeness** is its level against the closest part of the CV.
Both are calibrated levels from 0 to 1, the same in every run. A liked company adds 0.05 to both levels, a favourite 0.1. The bonus counts before the count and the minimum apply.
The levels drive the `analytics filter`. A role passes it when it passes the fetch gate (1.1.1 to 1.1.3) and:
- is among the best N by prompt closeness, each at least X (500 and 0.6 to start), or
- is among the best M by CV closeness that the first list leaves out, each at least Y (150 and 0.6 to start), or
- is liked or saved, whatever its levels.
The two lists are kept apart because they answer different questions (what I want, what I am), and a merged number lets the stronger side crowd the other out. For a broad agent the count stops each list; for a narrow agent the minimum stops it, so roles that do not fit are not read just to fill the count.
Newness and job likes are not part of the `analytics score`. A role is read once, pulls run every few hours, and votes come after the read.
An agent without a CV has no CV list and no judge. An agent without prompts has no prompt list and no prompt closeness in the want feed score.
The `analytics filter` determines which roles get `fetch text` and the model pass.
## 3 Fetch text
Some job boards have one API for the listings and another for the job specs. `fetch text` reads the role's own page for every role in the cut that came without text. Once the full text is fetched, the role is embedded again and its levels recomputed. A role that enters the cut after this recompute gets its text on the next pull.
## 4 Model pass
All roles in the cut go through the detailed analysis and metadata extraction. Extract and judge are two calls: `extract` needs no CV, it is faster, and it is the final step of the ingestion, so it decides what finally passes the `analytics filter`.
### 4.1 Extract
Often information is only in the free text, e.g. that a role in a specific location is actually remote. So every role has raw and processed fields for e.g. salary, location, remote and seniority. Extract reads what is hidden in the free text. A field the board states wins over what extract reads, except seniority and remote, where extract wins.
**Extract** reads the cut first. It fills location, remote area, seniority, languages and deadline.
The fetch gate and the cut are then applied again with what extract filled. A role can drop out, such as one that turns out to be outside the agent's locations, or enter, such as one that turns out to be remote. A role that enters is extracted on the next pull.
### 4.2 Judge
Once extract has completed the ingestion, `judge` reads each role in the cut against the agent's CV, languages, likes and dislikes. It outputs:
- the `CV match`
- one to three strengths and gaps
- a verdict on each must-have
Each role is extracted once. A judgement is redone only when its input changes: a change to the CV, a prompt, or the like or dislike text re-judges the whole cut (about an hour for 650 roles); a changed posting re-judges that role only.
#### 4.2.1 CV match
Judge rates how closely the job spec matches the experience in the CV on three dimensions, skills, experience and career, each from 0 to 5. The **CV match** is the three numbers weighted as given in the agent settings (30/25/30 to start). A must-have the role breaks caps career at 2. Roles with the same CV match are ordered by CV closeness.
## 5 Feed
The feed shows every role that passes the `world fetch gate`, the `location fetch gate` and the `role exclusion gate` (1.1.1 to 1.1.3), loaded as you scroll, in every order, newest first included. No order shows a role the fetch gate rejects. A hidden company's roles are never shown.
### 5.1 Feed scores
#### 5.1.1 Want feed score, "What I'm looking for"
The `want feed score` consists of `prompt closeness`, newness, company likes and job likes, with weights set per agent, to start `(0.6 prompt closeness + 0.2 newness + 0.2 company like + 0.2 job like) / sum of the weights`.
- `newness` counts from the posting date, the earlier of when the app first saw the role and the date the board states (only for boards whose dates are reliable), and halves every 14 days.
- `company like` is 1/3 for a neutral company, 2/3 for a liked one and 1 for a favourite.
- `job like` is described in 5.1.3.
#### 5.1.2 Match feed score, "What fits me"
The `match feed score` orders the judged roles by `CV match`, then the rest by `CV closeness` (embedding only, as they had no model pass).
#### 5.1.3 Job likes
The user likes or passes on roles in the feed, per agent. Each role has an embedding, so the votes mark areas of the embedding space. The liked direction is the mean of the liked roles' embeddings minus the mean of the passed roles' embeddings. A role's `job like` is its cosine to that direction, calibrated to a level from 0 to 1 like the closeness levels. Roles near the ones liked rise and roles near the ones passed on sink, whether or not they were voted on themselves.
- With only likes, the direction is the mean of the liked roles; with only passes, away from the mean of the passed roles.
- With no votes, `job like` drops out and its weight leaves the sum.
### 5.2 UI
#### Filters and saved filters (per agent)
- Everything else (seniority, location, remote or hybrid, pay) is a filter, not a score term. Filters only narrow what the fetch gate passed.
- For quick browsing, the current filter settings can be saved under a name, loaded with one click, and deleted. There is no editing; a change is a new saved filter and the old one deleted.
#### No roles silently missing
- The Roles page says that the feed is already filtered by the agent's fetch gate, location included.
- The company view says what is filtered: which parts of the fetch gate apply to this company, for example "every role in Europe, not filtered by country" for an All locations company. Looking up a company never silently hides roles elsewhere.
## 6 After the ranking
Summarise is not part of any score. It runs after the feed scores, on the top N of the `want feed score` and the top N of the `match feed score` (N set per agent, 30 to start, so at most 60 roles), and on every role liked or saved.
## Settings
Per agent, on the form:
- name and prompts
- the fetch gate: world (the outer countries), location (countries, towns, remote), role exclusion (titles to leave out), role fetch (fetch terms)
- CV, languages, likes and dislikes
- the analytics filter: count and minimum by prompt closeness, count and minimum by CV closeness
- the want feed score weights, and the CV match weights
- summarise size
Per agent, per company: affinity (hidden, neutral, liked, favourite). Per company, for every agent: All locations and All roles.