architecture doc

This commit is contained in:
Wolf committed 2026-10-04 20:36:26 +02:00
1 parent f7917a6fad
commit c98f8dcf28
1 file changed
+104 -44
+104 -44
View File
@@ -1,27 +1,25 @@
# Architecture
The target design of roledesk's ranking: how a role gets from a board to the Roles feed for one search agent, and the key decisions behind it. What gets built is tested against this document. The way from today's code to here is in `plan-ranking-flow.md` while that plan is open; the detailed reasoning is in `decisions.md`.
## Terms
The target design of roledesk's ranking: how a role gets from a board to the Roles feed for one search agent, and the key decisions behind it. What gets built is tested against this document.
- **Fetch gate**: the agent's hard rules, a yes or no per role, in three parts: region, place and role.
- **Analysis score**: what decides the **cut**, the roles that get fetch text and the model pass.
- **Final score**: the order of the feed.
- **Analytics score** and **analytics filter**: what decides the **cut**, the roles that get fetch text and the model pass.
- **Feed score**: the order of the feed.
- **Fetch text**: the second HTTP read of a role's own page, without a model.
- **Extract**, **judge**, **summarise**: the model calls (Haiku).
- **Affinity**: an agent's setting for a company: hidden, neutral, liked or favourite.
- **All countries**, **All roles**: company switches, the same for every agent, that bypass the place and the role part of the fetch gate.
- **All locations**, **All roles**: company switches, the same for every agent, that bypass the place and the role part of the fetch gate.
## Flow
```
1 Fetch ─ all relevant roles, limited by the fetch gate:
region · place · role, with All countries and All roles as overrides
world · location · role exclusion · role fetch, with All locations and All roles as overrides
company boards: whole under 300 roles, else filtered at the host where it can
search providers: their own saved query
│
▼
2 Analysis score, no tokens ─ fit to the prompts, closeness to the CV, affinity → the cut
2 Analytics score, no tokens ─ fit to the prompts, closeness to the CV, affinity → the cut
│
▼
3 Fetch text ─ the role's own page, for cut roles without text
@@ -43,15 +41,27 @@ The aim is to fetch every relevant role while keeping out the thousands that are
### 1.1 Fetch gate
1. **Region**: the required countries, Europe. Every role must be in them, and no override lifts this.
2. **Place**: the Where lines. A role passes when it matches at least one line: a country, a town or remote. With every line empty, a role can be anywhere in the region.
3. **Role**: the fetch terms, words a big board's host must match before it sends a role, and the titles to leave out, phrases that reject a role by its title.
#### 1.1.1. World fetch gate
Overrides:
The outer boundary of the agent,. e.g. the countries, region, etc. Every role must be in them, `all locations` will default to this boundary.
- A company set to **All countries** bypasses the place part. Its roles are fetched and kept from the whole region.
- A company set to **All roles** bypasses the role part: no fetch terms and no titles to leave out. It is for small companies worth a speculative approach, whatever the role.
- A search provider is read by its own saved query, which replaces the fetch terms.
#### 1.1.2 Location fetch gate
The stricter location filter. A role passes the `location fetch gate` when it matches at least one of the set locations: a country, a town, or remote. If none is set, a role can be anywhere within the `world filter`.
#### 1.1.3 Role exclusion gate
A list of titles the titles to leave out, phrases that reject a role by its title. Note, that no host is able to filter by exclusion. This gate is mentioned in the fetch group but really only applied downstream.
#### 1.1.4 Role fetch gate
The fetch terms, words a big board's host must match before it sends a role. Note that the `role fetch filter` is only applied at fetch time as a blunt tool for avoiding thousands of roles from a big board. It is ignored otherwise as the `Prompt closeness` is the more nuanced downstream filter of this.
#### Overrides:
- A company set to **All locations** bypasses the `location fetch gate` 1.1.2. Its roles are fetched and kept across the entire `world fetch gate`.
- A company set to **All roles** bypasses the `role exclusion gate` and the `role fetch gate` (1.1.3 and 1.1.4): no fetch terms and no titles to leave out. It is for small companies worth a speculative approach, whatever the role.
- A search provider is read by its own saved query, which replaces `role fetch gate` 1.1.4.
- A hidden company never passes.
The gate rejects on evidence only. A role with an unknown location passes until extract has read it.
@@ -59,49 +69,99 @@ The gate rejects on evidence only. A role with an unknown location passes until
### 1.2 Reading a board
- A company board with fewer than 300 roles is read whole.
- A company board with 300 or more roles is asked for the fetch gate at the host: the region and place, and the fetch terms, each where the host can filter on it. A host that cannot filter on a part sends the board whole for that part.
- Whatever the host did, the gate is applied to every role that arrives.
- A company board with 300 or more roles is asked for the fetch gate at the host: the region and place, and the fetch terms, if the host can filter on it. A host that cannot filter on a part sends the board whole for that part.
- A read narrowed by fetch terms is not the whole board, so it closes no role. Roles on such boards close by age. This is accepted because ranking relies on recency, not on closing.
## 2 Analysis score and the cut
## 2 Analytics score and filter
Each role is embedded and scored semantically:
- **Prompt closeness** is its level against the best of the agent's prompts.
- **CV closeness** is its level against the closest part of the CV.
Both are calibrated levels from 0 to 1, the same in every run. Additionally a liked company gets a bonus of 0.05 above neutral, a favourite about 0.1.
The scores are driving the `analytics filter`.
- Filters set a maximum number of jobs and minimum score (both need to apply) for each of `prompt` and `cv` closeness separately. They answer different questions (what I want, what I am), and a merged number lets the stronger side crowd the other out.
- Every liked or saved job passes the analytics filter`, whatever its levels.
- Newness and taste are not part of the `analytics score`. A role is read once and pulls run every few hours, and votes come after the read.
- Only jobs which pass the `Fetch gate`s (1.1.1 to 1.1.3).
The `analytics filter`determines which jobs go into further analysis and full `text fetch`.
- Free and local: each role is embedded once. **Fit** is its level against the best of the agent's prompts. **CV closeness** is its level against the closest part of the CV. Both are calibrated levels from 0 to 1, the same in every run.
- Affinity is a bonus on both levels, in the same proportion as in the final score: a liked company about 0.05 above neutral, a favourite about 0.1. It is a nudge, not a pass.
- The cut holds the roles that pass the fetch gate and are:
- among the best N by fit, each at least X (500 and 0.6 to start), or
- among the best M by CV closeness that the first list leaves out, each at least Y (150 and 0.6 to start), or
- liked or saved, whatever their levels.
- Newness and taste are not part of the analysis score. A role is read once and pulls run every few hours, and votes come after the read.
- The two lists are not merged into one number. They answer different questions (what I want, what I am), and a merged number lets the stronger side crowd the other out.
- For a broad agent the count stops each list; for a narrow agent the minimum stops it, so roles that do not fit are not read just to fill the count.
- The cut takes only roles that pass the fetch gate. Measured on 2026-10-04 for agent 2, without the Where lines only 118 of the 500 best by fit were roles the agent could take.
## 3 Fetch text
- Fetch text reads the role's own page for every role in the cut that came without text. The role is embedded again and its levels recomputed.
Some job boards have a separate API for the job listings and the actual job specs. `fetch text` reads the role's own page for every role in the cut that came without text. and passed the `analytics filter`. One the full text is fetched, the roles are embedded again and its levels recomputed.
## 4 Model pass
- One depth: the cut is the only selection, and extract and judge both read all of it.
- **Extract** reads the cut first. It fills location, remote area, seniority, languages and deadline.
- The fetch gate and the cut are then applied again with what extract filled. A role can drop out, such as one that turns out to be outside the agent's places, or enter, such as one that turns out to be remote. A role that enters is extracted on the next pull.
- **Judge** reads the cut as recomputed. It reads each role against the agent's CV, languages, likes and dislikes, and gives skills, experience and career from 0 to 100, one to three strengths and gaps, and a verdict on each must-have. The **CV match** is the three numbers under the agent's weights (30/25/30 to start). Changing the weights asks the model for nothing.
- Extract and judge are two calls. Extract needs no CV, and its location and remote answers must reach the fetch gate before judge reads anything.
- Each role is extracted once. A judgement is redone only when its input changes. A change to the CV, a prompt, or the like or dislike text re-judges the whole cut (about an hour for 650 roles); a changed posting re-judges that role only.
All jobs passing the `analytics filter` are running though the more detailed analytics and meta data extraction. Extract and judge are two calls because `extract` needs no CV, it is faster and it is the final step of the ingestion as as such deciding what eventually passes the `analytics filter`.
#### 4.1 Extract
Often information is only in the free text, e.g. if a role in a specific location is actually remote. So every role has raw and processed fields for e.g., salary, location, remote, seniority. Extract is extracting any information hidden in the free text and as such correcting the official fields in some cases.
**Extract** reads the cut first. It fills location, remote area, seniority, languages and deadline.
The fetch gate and the cut are then applied again with what extract filled. A role can drop out, such as one that turns out to be outside the agent's places, or enter, such as one that turns out to be remote. A role that enters is extracted on the next pull.
#### 4.2 Judge
Once extract has completely the ingestion, `judge` reads each role against the agent's CV, languages, likes and dislikes. It outputs:
- the `CV match`
- one to three strengths and gaps,
- a verdict on each must-have.
Each role is extracted once. A judgement is redone only when its input changes. i.e., a change to the CV, a prompt, or the like or dislike text re-judges the whole cut (about an hour for 650 roles); a changed posting re-judges that role only.
##### 4.2.1 CV match
It provides scores how closely the jobspec matches the experience as given by the CV for the 3 dimensions skills, experience and career in a rating from 0 to 5. The **CV match** is the three numbers weighted as given in the agent settings (30/25/30 to start)
## 5 Feed
- The feed shows every role that passes the agent's fetch gate, loaded as you scroll, in every order, newest first included. No order shows a role the fetch gate rejects.
- **What I'm looking for** orders by the final score: `(0.6 fit + 0.2 newness + 0.2 affinity + 0.2 taste) / sum of the weights`. Newness halves every 14 days. Taste is the role's closeness to the roles the agent liked, away from those it passed. The weights are set per agent.
- **What fits me** orders the judged roles by CV match, then the rest by CV closeness.
- Everything else (seniority, location, remote or hybrid, pay) is a filter, not a score term. Filters only narrow what the fetch gate passed.
The feed shows every role that passes the `world fetch gate`, `location fetch gate` and the `role exclusion gate` (1.1.1 - 1.1.3), lazy loaded as you scroll, in every order, newest first included. No order shows a role the fetch gate rejects.
#### 5.1 Feed Scores
##### 5.1.1 Want feed score or "What I'm looking"
The `want feed score` consists of `prompt closeness`, newness, company likes and job likes with weights set in the setting, currently`(0.6 fit + 0.2 newness + 0.2 company like + 0.2 job likes) / sum of the weights`.
- `newness` halves every 14 days.
- A `company like` and a `job like` is set by the user in the app
##### 5.1.2 Match feed score or "What fits me"
The `match feed score` orders the judged roles by `CV match`, then the rest by `CV closeness` (embedding only as no model pass)
##### 5.1.3 Job likes
#### 5.2 UI
##### Saved filters (per agent):
- Everything else (seniority, location, remote or hybrid, pay) is a filter, not a score term. Filters only narrow what the fetch gate passed.
- To allow for quick browsing to see the correct roles, the current filter settings can we saved saved under a name, loaded with one click, and deleted. There is no editing; a change is a new saved filter and the old one deleted.
**To make sure no roles are silently missing:**
- The Roles page says that the feed is already filtered by the agent's fetch gate, location included.
- **Saved filters**, per agent: the current filter settings are saved under a name, loaded with one click, and deleted. There is no editing; a change is a new saved filter.
- The company view says what is filtered: which parts of the fetch gate apply to this company, for example "every role in Europe, not filtered by country" for an All countries company. Looking up a company never silently hides roles elsewhere.
- The company view says what is filtered: which parts of the fetch gate apply to this company, for example "every role in Europe, not filtered by country" for an All locations company. Looking up a company never silently hides roles elsewhere.
## 6 After the ranking
- Summarise is not part of any score. It runs after the final score, on the top of the feed by the final score (50 to start, its own setting), and on every role liked or saved.
Summarise is not part of any score. It runs after the `analytics score`, on the top scored jobs (number set set in settings, 50 to start), and on every role liked or saved.
## Settings
@@ -111,7 +171,7 @@ Per agent, on the form:
- the fetch gate: region (required countries), place (Where lines), role (fetch terms, titles to leave out)
- CV, languages, likes and dislikes
- the cut: count and minimum by fit, count and minimum by CV closeness
- the final score weights, and the CV match weights
- the `feed score` weights, and the CV match weights
- summarise size
Per agent, per company: affinity (hidden, neutral, liked, favourite). Per company, for every agent: All countries and All roles.
Per agent, per company: affinity (hidden, neutral, liked, favourite). Per company, for every agent: All locations and All roles.