Split the All tag into two company overrides: All countries and All roles (WP L)

Fetching has two filters, and a company now overrides each on its own.
All countries (companies.all_countries) is the old All roles tag: its big
boards are read with the agents' wide reach. All roles (companies.all_roles,
new meaning) drops the agents' fetch terms and any later role filter.

- Migration 24 copies the old tag to all_countries and keeps all_roles for
  those companies, and moves a board's {"terms": false} to its company.
- The runner no longer reads {"terms": false}; the plan leaves the terms out
  for a company set to All roles.
- POST /api/companies/{key}/all-countries and /all-roles, each queueing a
  pull when turned on; CompanyOut carries both.
- Two toggles replace All on the Companies row, the reader and the Roles
  strip; the reach line states countries and role types in plain words;
  the filter select gets both options.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Opus 5.5 committed 2026-09-30 12:50:45 +02:00
1 parent 795ca3628c
commit e52857dea5
23 files changed
+599 -212

No files matched your search

+131 -56
View File
@@ -10,15 +10,15 @@ on. Computed once per pull (`plan`) and handed to each board's batch as the
function, a remote value other than `remote`, a place the gazetteer does function, a remote value other than `remote`, a place the gazetteer does
not know), can take a role from anywhere, so the base reach is then not know), can take a role from anywhere, so the base reach is then
everywhere. everywhere.
- **Wide reach, for the companies tagged All roles** (`companies.all_roles`, - **Wide reach, for the companies set to All countries**
global): the union over the enabled agents of their `require` reach, the (`companies.all_countries`, global): the union over the enabled agents of
intersection of each one's `require` gates on country and place, Europe their `require` reach, the intersection of each one's `require` gates on
today, expanded to its countries through the gazetteer, and the base country and place, Europe today, expanded to its countries through the
reach, so a tag never reads less than the board would untagged. An agent gazetteer, and the base reach, so the override never reads less than the
with no `require` gate on place makes it everywhere. A role in Freiburg board would without it. An agent with no `require` gate on place makes it
may be open to remote, or worth asking the company about for Denmark. everywhere. A role in Freiburg may be open to remote, or worth asking the
Favourite (affinity 2) is scoring only, where it skips the `any` gates; it company about for Denmark. Favourite (affinity 2) is scoring only, where
does not widen what is fetched. it skips the `any` gates; it does not widen what is fetched.
- **Board override**: a board whose subscription `params` name a country, a - **Board override**: a board whose subscription `params` name a country, a
place or remote gets no reach here; `ask_for` reads the params instead. place or remote gets no reach here; `ask_for` reads the params instead.
- **Small boards**: a board that listed under `SMALL` roles when last read - **Small boards**: a board that listed under `SMALL` roles when last read
@@ -33,12 +33,14 @@ board sends the roles that match one of them rather than every role:
- The terms of a pull are the union over the enabled agents. An enabled - The terms of a pull are the union over the enabled agents. An enabled
agent with none wants every role, so then there are none. agent with none wants every role, so then there are none.
- They go only with the wide reach: a company tagged All roles (`wide` in - They go only with the wide reach: a company set to All countries (`wide`
the reach), and a board whose reach `ask_for` drops as too many countries in the reach), and a board whose reach `ask_for` drops as too many
for a `one` host. The narrow reach (DK, SE) already leaves at most about countries for a `one` host. The narrow reach (DK, SE) already leaves at
150 roles on any big board, and small, unknown and overridden boards get most about 150 roles on any big board, and small, unknown and overridden
no reach at all, so none of those take terms. boards get no reach at all, so none of those take terms.
- A board's params switch them off with `{"terms": false}`. - A company set to All roles (`companies.all_roles`, global) takes none on
any of its boards, and no later role filter (category, level) either. It
changes nothing on the narrow reach today, which takes no terms.
- Only a reader that declares `query="one"` takes them, one series a term; - Only a reader that declares `query="one"` takes them, one series a term;
`ask_for` has the rule for terms times the reach's series. `ask_for` has the rule for terms times the reach's series.
- A read with terms is never the whole board: it sets no `ats:size`, and - A read with terms is never the whole board: it sets no `ats:size`, and
@@ -196,22 +198,61 @@ def fetch_terms(agents: Iterable[Agent]) -> tuple[str, ...]:
return tuple(found[k] for k in sorted(found)) if listed else () return tuple(found[k] for k in sorted(found)) if listed else ()
def wide_names(agents: Iterable[Agent], gaz: places.Gazetteer) -> str:
"""The wide reach as the agents' `require` gates name it, for the
Companies list: "Europe" for a gate on that place, and for a gate that
lists countries one by one, the continent they all lie in; up to three
names as written; "everywhere" when an agent has no such gate; "" when
none of that fits, which `line` says as a count."""
names: dict[str, str] = {}
for agent in agents:
gates = [g for g in agent.gates if g.mode == "require"
and g.field in ("country", "place") and not gate_reach(g, gaz).everywhere]
if not gates:
return "everywhere"
for gate in gates:
values = [str(v).strip() for v in gate.values if str(v).strip()]
if len(values) > 3:
values = [_continent(gate_reach(gate, gaz).countries, gaz)]
for text in values:
names.setdefault(text.casefold(), text)
return ", ".join(names.values()) if len(names) <= 3 and "" not in names else ""
def _continent(codes: Iterable[str], gaz: places.Gazetteer) -> str:
"""The continent every one of `codes` lies in, "" for none. A code the
gazetteer has no country for (EU) is skipped."""
known = {p.key for p in gaz.places.values() if p.kind == "country"}
wanted = set(codes) & known
for place in gaz.places.values():
if place.kind != "continent" or not wanted:
continue
under = {gaz.places[p].key for p in gaz.under((place.id,))
if gaz.places[p].kind == "country"}
if wanted <= under:
return place.name
return ""
@dataclass @dataclass
class Plan: class Plan:
"""The reach of one pull: `base` for every board, `wide` for the boards """The reach of one pull: `base` for every board, `wide` for the boards
of the companies in `tagged` (tagged All roles), each board's size by of the companies in `all_countries`, each board's size by target, the
target, and the fetch `terms` of the wide reads.""" fetch `terms` of the wide reads, which skip the companies in
`all_roles`, and `wide_names`, what the agents call the wide reach."""
base: Reach base: Reach
wide: Reach = EVERYWHERE wide: Reach = EVERYWHERE
sizes: dict[str, int] = field(default_factory=dict) sizes: dict[str, int] = field(default_factory=dict)
tagged: frozenset[str] = frozenset() all_countries: frozenset[str] = frozenset()
terms: tuple[str, ...] = () terms: tuple[str, ...] = ()
all_roles: frozenset[str] = frozenset()
wide_names: str = ""
def why(self, sub: Subscription) -> tuple[Reach, str]: def why(self, sub: Subscription) -> tuple[Reach, str]:
"""The reach `sub` is read with, and why: `override`, `small`, """The reach `sub` is read with, and why: `override`, `small`,
`unknown` (a board never read whole), `tag` (its company is tagged `unknown` (a board never read whole), `countries` (its company is
All roles) or `agents`.""" set to All countries) or `agents`."""
if OVERRIDE_KEYS & set(sub.params or {}): if OVERRIDE_KEYS & set(sub.params or {}):
return EVERYWHERE, "override" return EVERYWHERE, "override"
size = self.sizes.get(sub.target) size = self.sizes.get(sub.target)
@@ -219,20 +260,26 @@ class Plan:
return EVERYWHERE, "unknown" return EVERYWHERE, "unknown"
if size < SMALL: if size < SMALL:
return EVERYWHERE, "small" return EVERYWHERE, "small"
if company_key(sub.company or sub.label) in self.tagged: if company_key(sub.company or sub.label) in self.all_countries:
return self.wide, "tag" return self.wide, "countries"
return self.base, "agents" return self.base, "agents"
def narrows_roles(self, sub: Subscription) -> bool:
"""Whether `sub`'s board may be narrowed by role type: its company is
not set to All roles."""
return company_key(sub.company or sub.label) not in self.all_roles
def for_subscription(self, sub: Subscription) -> dict[str, Any] | None: def for_subscription(self, sub: Subscription) -> dict[str, Any] | None:
"""The `reach` a board's batch hands `ask_for`; None for none. With """The `reach` a board's batch hands `ask_for`; None for none. With
fetch terms, a tagged board's carries them and `wide`, and a board on fetch terms, and unless its company is set to All roles, a board on
the agents' reach carries them for `ask_for` to use if it drops the the wide reach carries them and `wide`, and a board on the agents'
countries as too many (see the module docstring).""" reach carries them for `ask_for` to use if it drops the countries as
too many (see the module docstring)."""
found, why = self.why(sub) found, why = self.why(sub)
out = found.params() out = found.params()
if self.terms and why in ("tag", "agents") and (sub.params or {}).get("terms") is not False: if self.terms and why in ("countries", "agents") and self.narrows_roles(sub):
out = (out or {}) | {"terms": list(self.terms)} out = (out or {}) | {"terms": list(self.terms)}
if why == "tag": if why == "countries":
out["wide"] = True out["wide"] = True
return out return out
@@ -255,8 +302,8 @@ def sizes(store: Store, subs: Iterable[Subscription]) -> dict[str, int]:
def plan(store: Store, subs: Iterable[Subscription] | None = None) -> Plan: def plan(store: Store, subs: Iterable[Subscription] | None = None) -> Plan:
"""The reach of a pull, from the enabled agents and the companies tagged """The reach of a pull, from the enabled agents and the companies' two
All roles, and the sizes of `subs` (every `ats` subscription when overrides, and the sizes of `subs` (every `ats` subscription when
None).""" None)."""
gaz = store.gazetteer() gaz = store.gazetteer()
agents = store.agents(enabled_only=True) agents = store.agents(enabled_only=True)
@@ -267,10 +314,14 @@ def plan(store: Store, subs: Iterable[Subscription] | None = None) -> Plan:
if not agents: if not agents:
base = wide = EVERYWHERE base = wide = EVERYWHERE
subs = list(store.subscriptions(source="ats") if subs is None else subs) subs = list(store.subscriptions(source="ats") if subs is None else subs)
tagged = frozenset(c.key for c in store.companies() if c.all_roles) rows = store.companies()
# A tag never reads less than the board would untagged: the base's # All countries never reads less than the board would without it: the
# remote, which a `require` of Europe does not name, is kept. # base's remote, which a `require` of Europe does not name, is kept.
return Plan(base, wide | base, sizes(store, subs), tagged, fetch_terms(agents)) return Plan(base, wide | base, sizes(store, subs),
all_countries=frozenset(c.key for c in rows if c.all_countries),
terms=fetch_terms(agents),
all_roles=frozenset(c.key for c in rows if c.all_roles),
wide_names=wide_names(agents, gaz) if agents else "everywhere")
def reaches(store: Store, subs: Iterable[Subscription]) -> Mapping[str, dict[str, Any]]: def reaches(store: Store, subs: Iterable[Subscription]) -> Mapping[str, dict[str, Any]]:
@@ -282,38 +333,62 @@ def reaches(store: Store, subs: Iterable[Subscription]) -> Mapping[str, dict[str
#: Why a board is read whole, as the Companies list says it. #: Why a board is read whole, as the Companies list says it.
_WHOLE = {"small": f"under {SMALL} roles", "unknown": "not read whole yet", _WHOLE = {"small": f"under {SMALL} roles", "unknown": "size not known yet",
"override": "set here", "tag": "company tag, too many countries to ask", "override": "set here", "countries": "all countries (everywhere)",
"agents": "an agent takes any place"} "agents": "an agent takes any place"}
#: Why a board is read filtered, as the Companies list says it.
_SAID = {"override": "set here", "tag": "company tag"} def _places(ask: readers.Ask) -> str:
words = sorted(ask.countries) if len(ask.countries) <= 4 \
else [f"{len(ask.countries)} countries"]
if ask.remote:
words.append("remote")
return ", ".join(words)
def _roles(ask: readers.Ask) -> str:
"""The role-type half of the line: the agents' terms the board is
asked for, or all roles."""
if not ask.terms:
return "all roles"
terms = ", ".join(ask.terms) if len(ask.terms) <= 5 else f"{len(ask.terms)} terms"
return f"follows agents' terms: {terms}"
def line(found: Plan, sub: Subscription) -> str: def line(found: Plan, sub: Subscription) -> str:
"""What a pull asks `sub`'s board for, once its reader has narrowed the """What a pull asks `sub`'s board for, once its reader has narrowed the
reach, in a few words: "DK, SE, remote · agents", "33 countries, remote · reach, in plain words, countries first and role types second:
company tag", "57 countries · company tag · 3 terms", "all places · "follows agents: DK, SE, remote · all roles", "all countries (Europe) ·
company tag · 3 terms", "all roles · under 300 roles", "all roles · follows agents' terms: engineer, product, data", "all countries (Europe)
Greenhouse cannot filter".""" · all roles", "set here: DE · all roles", "all places, too many countries
to ask · follows agents' terms: engineer", or, for a board read whole,
why: "under 300 roles: read whole", "Greenhouse cannot filter: read
whole"."""
module = readers.HOSTS.get(sub.target.partition(":")[0]) module = readers.HOSTS.get(sub.target.partition(":")[0])
if module is None: if module is None:
return "" return ""
given, why = found.why(sub) given, why = found.why(sub)
ask = _runner.ask_for(module, sub.params, found.for_subscription(sub)) ask = _runner.ask_for(module, sub.params, found.for_subscription(sub))
terms = f" · {len(ask.terms)} term{'s' if len(ask.terms) != 1 else ''}" if ask.terms else "" roles = _roles(ask)
if _runner.filtered(ask): if _runner.filtered(ask):
words = sorted(ask.countries) if len(ask.countries) <= 4 \ if why == "countries":
else [f"{len(ask.countries)} countries"] named = found.wide_names or f"{len(found.wide.countries)} countries"
if ask.remote: return f"all countries ({named}) · {roles}"
words.append("remote") if why == "override":
return f"{', '.join(words)} · {_SAID.get(why, why)}{terms}" return f"set here: {_places(ask)} · {roles}"
if terms: return f"follows agents: {_places(ask)} · {roles}"
return f"all places · {_SAID.get(why, why)}{terms}" if ask.terms:
if why in ("agents", "tag") and given.everywhere: if given.everywhere:
return f"all roles · {'company tag' if why == 'tag' else _WHOLE[why]}" return f"all countries (everywhere) · {roles}"
if why in ("agents", "tag") and not module.READS.filters.declared(): return f"all places, too many countries to ask · {roles}"
return f"all roles · {module.LABEL} cannot filter" if why in ("agents", "countries") and given.everywhere:
if why == "agents": return f"{_WHOLE[why]}: read whole"
return f"all roles · {module.LABEL} cannot ask for these" if why in ("agents", "countries", "override") and not module.READS.filters.declared():
return f"all roles · {_WHOLE[why]}" return f"{module.LABEL} cannot filter: read whole"
reads = module.READS.filters
if why in ("agents", "countries") and reads.country == "one" \
and len(given.countries) > _runner.MAX_SERIES:
return "too many countries to ask: read whole"
if why in ("agents", "countries"):
return f"{module.LABEL} cannot ask for these: read whole"
return f"{_WHOLE[why]}: read whole"
+6 -3
View File
@@ -1003,9 +1003,12 @@ class Company(Model):
#: Why the company is read by hand, one of `HAND_REASONS`; "" when it is #: Why the company is read by hand, one of `HAND_REASONS`; "" when it is
#: not on that list. #: not on that list.
hand_reason: str = "" hand_reason: str = ""
#: 1 when every board of the company is read with the agents' wide reach, #: The two fetch overrides (`app/jobs/reach.py`), global rather than an
#: Europe today, rather than their places (`app/jobs/reach.py`). Global, #: agent's, since fetching is shared. `all_countries` 1: every big board
#: not an agent's: it is about what is fetched, and fetching is shared. #: of the company is read with the agents' wide reach, Europe today,
#: rather than their places. `all_roles` 1: no role-type narrowing, no
#: fetch terms today, on any of its boards.
all_countries: int = 0
all_roles: int = 0 all_roles: int = 0
created_at: str created_at: str
updated_at: str updated_at: str
+15 -4
View File
@@ -1113,9 +1113,10 @@ table.applications td[data-label="Saved"] { white-space: nowrap; color: var(--di
.affinity button[data-value="-1"][aria-pressed="true"] { .affinity button[data-value="-1"][aria-pressed="true"] {
color: var(--accent-fg); background: var(--error); border-color: var(--error); color: var(--accent-fg); background: var(--error); border-color: var(--error);
} }
/* The All roles tag, global and apart from the stances: green, the colour /* The two fetch overrides, All countries and All roles, global and apart
the app gives what widens (`--good`). */ from the stances: green, the colour the app gives what widens (`--good`). */
.affinity button[data-value="all"][aria-pressed="true"] { .affinity button[data-value="countries"][aria-pressed="true"],
.affinity button[data-value="roles"][aria-pressed="true"] {
color: var(--accent-fg); background: var(--good); border-color: var(--good); color: var(--accent-fg); background: var(--good); border-color: var(--good);
} }
@@ -1308,8 +1309,18 @@ table.applications td[data-label="Saved"] { white-space: nowrap; color: var(--di
font-size: .96rem; text-decoration: none; color: inherit; font-size: .96rem; text-decoration: none; color: inherit;
} }
.crow .name { font-weight: 600; } .crow .name { font-weight: 600; }
/* A company row has one control more: the All roles tag after the stances. */ /* A company row has one control more: the two fetch overrides after the stances. */
#companies-list .crow { grid-template-columns: 1fr auto auto auto; } #companies-list .crow { grid-template-columns: 1fr auto auto auto; }
.affinity button[data-value="countries"], .affinity button[data-value="roles"] {
white-space: nowrap;
}
/* At 390px their words do not fit beside the name and the stances, so they
take a line of their own above the board's reach, which they set. */
@media (max-width: 640px) {
#companies-list .crow > .affinity[aria-label="Fetch overrides"] {
grid-column: 1 / -1; justify-self: start;
}
}
/* Only a name that is a link: the Watchlist draws one without a page to /* Only a name that is a link: the Watchlist draws one without a page to
open as text, which must not look as if it led somewhere. */ open as text, which must not look as if it led somewhere. */
@media (hover: hover) { @media (hover: hover) {
+58 -27
View File
@@ -16,7 +16,8 @@
* Beside `api` on `window`, the few helpers more than one page script uses: * Beside `api` on `window`, the few helpers more than one page script uses:
* `count`, `cell`, `debounce`, `safeUrl`, and the company list with its * `count`, `cell`, `debounce`, `safeUrl`, and the company list with its
* affinity control (`loadCompanies`, `companies`, `affinityControl`, * affinity control (`loadCompanies`, `companies`, `affinityControl`,
* `setAffinity`) and the All roles tag beside it (`allRolesControl`). One * `setAffinity`) and the two fetch overrides beside it, All countries and
* All roles (`overridesControl`). One
* copy each, since two copies of a helper is one of them * copy each, since two copies of a helper is one of them
* being fixed. * being fixed.
* *
@@ -184,55 +185,85 @@
if (onChange) await onChange(updated); if (onChange) await onChange(updated);
} }
/* The All roles tag, one more toggle in the affinity buttons' fragment /* The two fetch overrides, one more pair of toggles in the affinity
beside them, in a group of its own since it is global rather than an buttons' fragment beside them, in a group of their own since they are
agent's stance: every board of the company is read with the agents' global rather than an agent's stance (app/jobs/reach.py). Fetching has
wide reach, Europe today, rather than their places (app/jobs/reach.py). On queues a pull on the server, as the other two filters, and by default a company follows the agents on both:
reopening switches do; off asks nothing, since no role closes until the - All countries reads its big boards (300 roles or more) with the
next pull reads the board with its filter again. `onChange` runs after agents' wide reach, Europe today, rather than their places.
the server accepted it, with the company it answered. */ - All roles reads them with no role-type narrowing: none of the agents'
function allRolesControl(companyKey, current, onChange) { fetch terms, which only apply with the wide reach, and none of the
role filters that may come later.
On queues a pull on the server, as the other reopening switches do; off
asks nothing, since no role closes until the next pull reads the board
with its filters again. `company` is the shared list's row;
`onChange` runs after the server accepted a switch, with the company it
answered. */
const OVERRIDES = [
{ value: "countries", field: "all_countries", path: "all-countries", text: "All countries",
on: "All countries is on: this company's big boards are read across the agents' wide "
+ "reach (Europe), not only the agents' places. The default is the agents' places. "
+ "Boards under 300 roles are read whole either way. Turning it off asks nothing "
+ "first: no role closes until the next pull reads the board again.",
off: "All countries: read this company's big boards across the agents' wide reach "
+ "(Europe) instead of the agents' places, the default. Boards under 300 roles are "
+ "read whole already. Turning it on queues a pull.",
said: (on) => (on ? "all countries in the agents' wide reach" : "the agents' places") },
{ value: "roles", field: "all_roles", path: "all-roles", text: "All roles",
on: "All roles is on: this company's boards are read with no role-type narrowing, "
+ "none of the agents' fetch terms. The default follows the terms, which only apply "
+ "with All countries or on a host that cannot ask for each country. Turning it off "
+ "asks nothing first.",
off: "All roles: no role-type narrowing for this company, none of the agents' fetch "
+ "terms, the default, nor later role filters. The terms only apply with All "
+ "countries or on a host that cannot ask for each country, so on its own this "
+ "changes nothing today. Turning it on queues a pull.",
said: (on) => (on ? "every role type" : "the agents' terms") },
];
function overridesControl(companyKey, company, onChange) {
const span = document.createElement("span"); const span = document.createElement("span");
span.className = "affinity"; span.className = "affinity";
span.setAttribute("role", "group"); span.setAttribute("role", "group");
span.setAttribute("aria-label", "Company tag"); span.setAttribute("aria-label", "Fetch overrides");
for (const o of OVERRIDES) {
const current = Boolean(company && company[o.field]);
const button = document.createElement("button"); const button = document.createElement("button");
button.type = "button"; button.type = "button";
button.dataset.value = "all"; button.dataset.value = o.value;
button.textContent = "All"; button.textContent = o.text;
button.title = current button.title = current ? o.on : o.off;
? "All roles within the agents' wide reach (Europe), not only their places. Turning it " button.setAttribute("aria-label", o.text);
+ "off asks nothing first: no role closes until the next pull reads the board again." button.setAttribute("aria-pressed", String(current));
: "All roles: read this company's boards for all roles within the agents' wide reach " button.addEventListener("click", () => setOverride(companyKey, o, !current, onChange));
+ "(Europe), not only their places. Turning it on queues a pull.";
button.setAttribute("aria-label", "All roles");
button.setAttribute("aria-pressed", String(Boolean(current)));
button.addEventListener("click", () => setAllRoles(companyKey, !current, onChange));
span.appendChild(button); span.appendChild(button);
}
return span; return span;
} }
async function setAllRoles(companyKey, on, onChange) { async function setOverride(companyKey, o, on, onChange) {
if (!companyKey) return; if (!companyKey) return;
let updated; let updated;
try { try {
updated = await api(`/api/companies/${encodeURIComponent(companyKey)}/all-roles`, { updated = await api(`/api/companies/${encodeURIComponent(companyKey)}/${o.path}`, {
method: "POST", body: { on }, method: "POST", body: { on },
}); });
} catch (err) { } catch (err) {
return; return;
} }
const company = companies.get(companyKey); const company = companies.get(companyKey);
if (company) company.all_roles = Boolean(updated.all_roles); if (company) {
company.all_countries = Boolean(updated.all_countries);
company.all_roles = Boolean(updated.all_roles);
}
window.toast.info(on window.toast.info(on
? `${updated.name}: all roles within the agents' wide reach. ` ? `${updated.name}: ${o.said(true)}. Reading now, or with the pull right after.`
+ "Reading now, or with the pull right after." : `${updated.name}: back to ${o.said(false)} from the next pull.`);
: `${updated.name}: back to its reach from the next pull.`);
if (onChange) await onChange(updated); if (onChange) await onChange(updated);
} }
Object.assign(window, { Object.assign(window, {
api, count, cell, debounce, safeUrl, companies, loadCompanies, affinityControl, setAffinity, api, count, cell, debounce, safeUrl, companies, loadCompanies, affinityControl, setAffinity,
allRolesControl, overridesControl,
}); });
})(); })();
+23 -15
View File
@@ -5,8 +5,8 @@
* joined with each company's board subscription — its id and whether it is * joined with each company's board subscription — its id and whether it is
* on — so nothing here fetches `/api/sources` as well. The affinity buttons * on — so nothing here fetches `/api/sources` as well. The affinity buttons
* are `affinityControl`, the same ones the Roles tab and the read page draw, * are `affinityControl`, the same ones the Roles tab and the read page draw,
* and beside them `allRolesControl`, the All roles tag, which is global and * and beside them `overridesControl`, the two fetch overrides All countries
* drawn with or without an agent. * and All roles, which are global and drawn with or without an agent.
* *
* Liked and hidden are per search agent, so the `#c-agent` select above the * Liked and hidden are per search agent, so the `#c-agent` select above the
* list says whose the list shows and whose the buttons set. It opens on the * list says whose the list shows and whose the buttons set. It opens on the
@@ -20,18 +20,20 @@
* `/api/companies/{key}/board`, which reads the board once before keeping * `/api/companies/{key}/board`, which reads the board once before keeping
* it; Detach stops reading it and keeps the company. * it; Detach stops reading it and keeps the company.
* *
* A row with a board says what a pull asks it for ("reads DK, SE · agents", * A row with a board says what a pull asks it for, countries first and role
* `reach` from app/jobs/reach.py), and its editor has a second line that * types second ("reads follows agents: DK, SE, remote · all roles", `reach`
* sets the board's own reach: the agents', or its own countries and remote, * from app/jobs/reach.py), so the row shows whether the company follows the
* posted to `/api/sources/{id}/reach`. All roles within the agents' wide * agents' filters or overrides them. Its editor has a second line that sets
* reach (Europe) is the company's All roles tag instead, which the board's * the board's own reach: the agents', or its own countries and remote,
* own countries win over. * posted to `/api/sources/{id}/reach`. The wide reach (Europe) and no role
* narrowing are the company's two overrides instead, and the board's own
* countries win over All countries.
*/ */
(function () { (function () {
"use strict"; "use strict";
const { const {
api, count, debounce, safeUrl, companies, loadCompanies, affinityControl, allRolesControl, api, count, debounce, safeUrl, companies, loadCompanies, affinityControl, overridesControl,
} = window; } = window;
const $ = (sel, root = document) => root.querySelector(sel); const $ = (sel, root = document) => root.querySelector(sel);
@@ -212,8 +214,8 @@
} }
/** The open row's reach editor, for a row with a board: the agents' reach /** The open row's reach editor, for a row with a board: the agents' reach
or the board's own countries and remote. The wide reach is the or the board's own countries and remote. The editor points to the
company's All roles tag, which the editor points to. */ company's two overrides, which are set beside the stars. */
function reachEditor(company, rerender) { function reachEditor(company, rerender) {
if (company.subscription_id == null) return null; if (company.subscription_id == null) return null;
const own = company.reach_params || {}; const own = company.reach_params || {};
@@ -244,9 +246,14 @@
shown(); shown();
const note = document.createElement("span"); const note = document.createElement("span");
note.className = "muted small"; note.className = "muted small";
note.textContent = company.all_roles note.textContent = [
? "The All roles tag is on; a reach set here wins over it." company.all_countries
: "All roles within the agents' wide reach (Europe): the All tag beside the stars."; ? "All countries is on; countries set here win over it."
: "All countries, beside the stars, reads the agents' wide reach (Europe).",
company.all_roles
? "All roles is on: no fetch terms."
: "All roles drops the agents' fetch terms.",
].join(" ");
form.append(mode, codes, remote, save, note); form.append(mode, codes, remote, save, note);
form.addEventListener("submit", async (event) => { form.addEventListener("submit", async (event) => {
event.preventDefault(); event.preventDefault();
@@ -309,6 +316,7 @@
case "favourite": return company.affinity === 2; case "favourite": return company.affinity === 2;
case "liked": return company.affinity === 1; case "liked": return company.affinity === 1;
case "hidden": return company.affinity === -1; case "hidden": return company.affinity === -1;
case "all_countries": return company.all_countries === true;
case "all_roles": return company.all_roles === true; case "all_roles": return company.all_roles === true;
case "on": return company.on === true; case "on": return company.on === true;
case "off": return company.on === false; case "off": return company.on === false;
@@ -343,7 +351,7 @@
if (agent != null) { if (agent != null) {
row.appendChild(affinityControl(company.key, company.affinity, async () => render(), agent)); row.appendChild(affinityControl(company.key, company.affinity, async () => render(), agent));
} }
row.appendChild(allRolesControl(company.key, company.all_roles, async () => { row.appendChild(overridesControl(company.key, company, async () => {
await loadCompanies(agentId()); await loadCompanies(agentId());
render(); render();
})); }));
+10 -10
View File
@@ -81,7 +81,7 @@
(function () { (function () {
"use strict"; "use strict";
const { api, esc, debounce, safeUrl, companies, loadCompanies, affinityControl, allRolesControl, const { api, esc, debounce, safeUrl, companies, loadCompanies, affinityControl, overridesControl,
judgeButtons, loadTracked, chipInput } = window; judgeButtons, loadTracked, chipInput } = window;
const $ = (sel, root = document) => root.querySelector(sel); const $ = (sel, root = document) => root.querySelector(sel);
@@ -1071,19 +1071,19 @@
el.querySelector(".strip-drop").onclick = dropCompany; el.querySelector(".strip-drop").onclick = dropCompany;
} }
const slot = el.querySelector(".affinity-slot"); const slot = el.querySelector(".affinity-slot");
const drawn = `${affinity}:${company ? company.all_roles : ""}`; const drawn = `${affinity}:${company ? `${company.all_countries}:${company.all_roles}` : ""}`;
if (slot.dataset.affinity !== drawn) { if (slot.dataset.affinity !== drawn) {
slot.dataset.affinity = drawn; slot.dataset.affinity = drawn;
slot.replaceChildren(affinityControl(key, affinity, slot.replaceChildren(affinityControl(key, affinity,
(updated) => affinityChanged(key, company, updated), state.filter.agent)); (updated) => affinityChanged(key, company, updated), state.filter.agent));
// The All roles tag needs a company row, which a first stance makes. // The two overrides need a company row, which a first stance makes.
if (company) slot.append(" ", allRolesControl(key, company.all_roles, renderStrip)); if (company) slot.append(" ", overridesControl(key, company, renderStrip));
// What ★ does, before; what it did, after. // What ★ does, before; what it did, after.
el.querySelector(".strip-note").textContent = affinity === 2 el.querySelector(".strip-note").textContent = affinity === 2
? "★ Favourite: roles from every place the agent's required gates allow are listed. " ? "★ Favourite: roles from every place the agent's required gates allow are listed. "
+ "All fetches them from a big board." + "All countries fetches them from a big board."
: "★ Favourite also lists this company's roles outside your places, as far as the " : "★ Favourite also lists this company's roles outside your places, as far as the "
+ "agent's required gates allow. All fetches them from a big board."; + "agent's required gates allow. All countries fetches them from a big board.";
} }
const n = state.total; const n = state.total;
el.querySelector(".strip-count").textContent = state.loaded el.querySelector(".strip-count").textContent = state.loaded
@@ -1362,9 +1362,9 @@
} }
/* The company's ★ ▲ ⊘, the one place a company is judged outside the /* The company's ★ ▲ ⊘, the one place a company is judged outside the
Companies page, and its All roles tag beside them. A company seen only Companies page, and its two fetch overrides beside them. A company seen
through a search has no row until its first change creates one, so the only through a search has no row until its first change creates one, so
list is reloaded then, and the tag, which needs a row, waits for it. A the list is reloaded then, and the overrides, which need a row, wait for it. A
change dims or undims its rows in the feed. */ change dims or undims its rows in the feed. */
function renderAffinity(slot, r) { function renderAffinity(slot, r) {
const company = companies.get(r.company_key); const company = companies.get(r.company_key);
@@ -1375,7 +1375,7 @@
renderFeed(); renderFeed();
}, state.filter.agent)); }, state.filter.agent));
if (company) { if (company) {
slot.append(" ", allRolesControl(r.company_key, company.all_roles, slot.append(" ", overridesControl(r.company_key, company,
async () => renderAffinity(slot, r))); async () => renderAffinity(slot, r)));
} }
} }
+50 -4
View File
@@ -195,9 +195,11 @@ CREATE TABLE IF NOT EXISTS role_tags (
-- worth checking by hand, 0 not; careers_opened_at when its careers page was -- worth checking by hand, 0 not; careers_opened_at when its careers page was
-- last opened from there, which is also when a person last looked at it. -- last opened from there, which is also when a person last looked at it.
-- hand_reason is why a person reads the careers page (models.HAND_REASONS), -- hand_reason is why a person reads the careers page (models.HAND_REASONS),
-- '' when nobody has to. all_roles 1 reads every board of the company with -- '' when nobody has to. The two fetch overrides, global since fetching is
-- the agents' wide reach, Europe today (app/jobs/reach.py); global, since -- shared (app/jobs/reach.py): all_countries 1 reads every big board of the
-- fetching is shared. -- company with the agents' wide reach, Europe today, rather than their
-- places; all_roles 1 reads them with no role-type narrowing, no fetch terms
-- today. Until migration 24, all_roles meant what all_countries means now.
CREATE TABLE IF NOT EXISTS companies ( CREATE TABLE IF NOT EXISTS companies (
key TEXT PRIMARY KEY, key TEXT PRIMARY KEY,
name TEXT NOT NULL, name TEXT NOT NULL,
@@ -222,7 +224,8 @@ CREATE TABLE IF NOT EXISTS companies (
last_funding_currency TEXT NOT NULL DEFAULT '', last_funding_currency TEXT NOT NULL DEFAULT '',
facts_source TEXT NOT NULL DEFAULT '', facts_source TEXT NOT NULL DEFAULT '',
facts_at TEXT NOT NULL DEFAULT '', facts_at TEXT NOT NULL DEFAULT '',
all_roles INTEGER NOT NULL DEFAULT 0 all_roles INTEGER NOT NULL DEFAULT 0,
all_countries INTEGER NOT NULL DEFAULT 0
); );
-- Every folded spelling of a company (models.company_fold): orsted is Ørsted -- Every folded spelling of a company (models.company_fold): orsted is Ørsted
@@ -604,6 +607,10 @@ ALTER TABLE companies ADD COLUMN hand_reason TEXT NOT NULL DEFAULT '';
# A Python step: it adds the column only where it is missing. # A Python step: it adds the column only where it is missing.
(23, "agents.fetch_terms: words that narrow what big boards read with the wide reach " (23, "agents.fetch_terms: words that narrow what big boards read with the wide reach "
"send", lambda conn: _add_fetch_terms(conn)), "send", lambda conn: _add_fetch_terms(conn)),
# A Python step: params is JSON and the company is found by company_key.
(24, "companies.all_countries takes over the All roles tag, and companies.all_roles "
"means no role narrowing; a board's {\"terms\": false} becomes its company's "
"all_roles", lambda conn: _split_all_roles(conn)),
) )
#: What a file created from `SCHEMA` today is already at. #: What a file created from `SCHEMA` today is already at.
@@ -879,6 +886,45 @@ def _add_all_roles(conn: sqlite3.Connection) -> None:
"to tag; it reads the agents' reach now", target, key) "to tag; it reads the agents' reach now", target, key)
def _split_all_roles(conn: sqlite3.Connection) -> None:
"""Migration 24's step. Fetching has two filters, countries and role
types, and each gets its own company override. `all_countries` is the
old All roles tag, the wide reach, copied from `all_roles` when the
column is added. `all_roles` keeps its column and now means no role-type
narrowing: a company that had the tag keeps it too, since the tag was
given to the favourites whose every role the owner wanted. A board whose
`params` hold `{"terms": false}` gives its company `all_roles`, and the
param goes. Runs twice without harm: the copy happens only with the new
column."""
columns = {r[1] for r in conn.execute("PRAGMA table_info(companies)")}
if "all_countries" not in columns:
conn.execute("ALTER TABLE companies ADD COLUMN all_countries INTEGER NOT NULL DEFAULT 0")
copied = conn.execute("UPDATE companies SET all_countries=1 WHERE all_roles=1").rowcount
log.info("migration 24: %d companies tagged All roles are All countries and All roles",
copied)
moved = 0
for sub_id, target, company, label, raw in conn.execute(
"SELECT id, target, company, label, params FROM subscriptions "
"WHERE source = 'ats'").fetchall():
try:
params = json.loads(raw or "{}")
except ValueError:
continue
if not isinstance(params, dict) or "terms" not in params:
continue
off = params.pop("terms") is False
conn.execute("UPDATE subscriptions SET params=? WHERE id=?", (json.dumps(params), sub_id))
key = company_key(company or label)
if not off:
continue
if conn.execute("UPDATE companies SET all_roles=1 WHERE key=?", (key,)).rowcount:
moved += 1
else:
log.warning("migration 24: %s read without terms, but %r has no company row "
"to mark All roles; it takes the agents' terms now", target, key)
log.info("migration 24: %d boards' {\"terms\": false} became All roles", moved)
_V22_COLUMNS = ("seniority_host", "employment_host", "remote_host", "country_host", "employment") _V22_COLUMNS = ("seniority_host", "employment_host", "remote_host", "country_host", "employment")
_V22 = """ _V22 = """
+20 -10
View File
@@ -125,7 +125,7 @@ _EXCERPT_RAW = 240
_COMPANY_FIELDS = ("name", "domain", "status", "careers_url", "notes", "ats", "country", "city", _COMPANY_FIELDS = ("name", "domain", "status", "careers_url", "notes", "ats", "country", "city",
"ats_slug", "watch", "careers_opened_at", "hand_reason", "sector", "stage", "ats_slug", "watch", "careers_opened_at", "hand_reason", "sector", "stage",
"size_band", "founded", "last_funding", "last_funding_currency", "size_band", "founded", "last_funding", "last_funding_currency",
"facts_source", "facts_at", "all_roles") "facts_source", "facts_at", "all_roles", "all_countries")
#: A company as the store serves it: the row and the asked-for agent's affinity. #: A company as the store serves it: the row and the asked-for agent's affinity.
_COMPANY_SELECT = """SELECT c.*, COALESCE(ac.affinity, 0) AS affinity FROM companies c _COMPANY_SELECT = """SELECT c.*, COALESCE(ac.affinity, 0) AS affinity FROM companies c
@@ -1938,9 +1938,16 @@ class Store:
self._alias(conn, key, fields["name"]) self._alias(conn, key, fields["name"])
return self._company(key) return self._company(key)
def set_all_countries(self, key: str, on: bool) -> Company:
"""Switch a company's All countries override: its big boards are read
with the agents' wide reach, Europe today, rather than their places,
from the next pull (`app/jobs/reach.py`). Global, not per agent.
Raises KeyError for an unknown company."""
return self.update_company(key, all_countries=int(bool(on)))
def set_all_roles(self, key: str, on: bool) -> Company: def set_all_roles(self, key: str, on: bool) -> Company:
"""Tag a company All roles, or untag it: its boards are read with the """Switch a company's All roles override: its boards are read with no
agents' wide reach, Europe today, from the next pull role-type narrowing, no fetch terms today, from the next pull
(`app/jobs/reach.py`). Global, not per agent. Raises KeyError for an (`app/jobs/reach.py`). Global, not per agent. Raises KeyError for an
unknown company.""" unknown company."""
return self.update_company(key, all_roles=int(bool(on))) return self.update_company(key, all_roles=int(bool(on)))
@@ -2001,10 +2008,11 @@ class Store:
def merge_companies(self, keep: str, other: str) -> Company: def merge_companies(self, keep: str, other: str) -> Company:
"""Fold company `other` into `keep`, in one transaction: its roles and """Fold company `other` into `keep`, in one transaction: its roles and
subscriptions move to `keep`, each agent's stance moves unless the subscriptions move to `keep`, each agent's stance moves unless the
agent has one on `keep` already, an All roles tag carries over, its agent has one on `keep` already, the All countries and All roles
aliases and its name become aliases of `keep`, and its row goes. Copies are not recomputed overrides carry over, its aliases and its name become aliases of
(`rebuild_dupes`). Raises KeyError for an unknown company and `keep`, and its row goes. Copies are not recomputed (`rebuild_dupes`).
ValueError for merging one into itself.""" Raises KeyError for an unknown company and ValueError for merging one
into itself."""
if keep == other: if keep == other:
raise ValueError("a company cannot be merged into itself") raise ValueError("a company cannot be merged into itself")
with self.transaction() as conn: with self.transaction() as conn:
@@ -2025,9 +2033,11 @@ class Store:
(keep, other)) (keep, other))
conn.execute("DELETE FROM company_aliases WHERE company_key=?", (other,)) conn.execute("DELETE FROM company_aliases WHERE company_key=?", (other,))
self._alias(conn, keep, rows[other]) self._alias(conn, keep, rows[other])
conn.execute("UPDATE companies SET all_roles = MAX(all_roles, " conn.execute("UPDATE companies SET "
"(SELECT all_roles FROM companies WHERE key=?)) WHERE key=?", "all_roles = MAX(all_roles, (SELECT all_roles FROM companies WHERE key=?)), "
(other, keep)) "all_countries = MAX(all_countries, "
"(SELECT all_countries FROM companies WHERE key=?)) WHERE key=?",
(other, other, keep))
conn.execute("DELETE FROM companies WHERE key=?", (other,)) conn.execute("DELETE FROM companies WHERE key=?", (other,))
return self._company(keep) return self._company(keep)
+1 -1
View File
@@ -55,7 +55,7 @@
<span>Only roles matching</span> <span>Only roles matching</span>
<div id="a-terms"></div> <div id="a-terms"></div>
</div> </div>
<span class="hint">Fetch terms narrow what a big board sends when it is read across Europe: a company tagged All roles, or a host that cannot ask for each country. The board then lists only roles whose text has one of these words, one request per word. Empty means no narrowing, and while any agent that is on has none, no board is narrowed. Boards read for Denmark or Sweden, and boards under 300 roles, are never narrowed. The words decide what is fetched, not how it ranks.</span> <span class="hint">Fetch terms narrow what a big board sends when it is read across Europe: a company set to All countries, or a host that cannot ask for each country. A company set to All roles is never narrowed. The board then lists only roles whose text has one of these words, one request per word. Empty means no narrowing, and while any agent that is on has none, no board is narrowed. Boards read for Denmark or Sweden, and boards under 300 roles, are never narrowed. The words decide what is fetched, not how it ranks.</span>
</fieldset> </fieldset>
<fieldset class="weights"> <fieldset class="weights">
<legend>How much each counts</legend> <legend>How much each counts</legend>
+1
View File
@@ -106,6 +106,7 @@
<option value="favourite">Favourite</option> <option value="favourite">Favourite</option>
<option value="liked">Liked</option> <option value="liked">Liked</option>
<option value="hidden">Hidden</option> <option value="hidden">Hidden</option>
<option value="all_countries">All countries</option>
<option value="all_roles">All roles</option> <option value="all_roles">All roles</option>
<option value="on">On</option> <option value="on">On</option>
<option value="off">Off</option> <option value="off">Off</option>
+35 -17
View File
@@ -450,8 +450,9 @@ class CompanyOut(BaseModel):
`careers_opened_at` are the Watchlist's (`Company`'s), the latter also `careers_opened_at` are the Watchlist's (`Company`'s), the latter also
when a person last looked at the careers page. `by_hand` marks a company when a person last looked at the careers page. `by_hand` marks a company
with no board shown that is not `from_search`, and `hand_reason` says why with no board shown that is not `from_search`, and `hand_reason` says why
it is read by hand (`app.models.HAND_REASONS`). `all_roles` is the it is read by hand (`app.models.HAND_REASONS`). `all_countries` and
company tag that reads its boards with the agents' wide reach.""" `all_roles` are the company's two fetch overrides: its big boards read
with the agents' wide reach, and with no role-type narrowing."""
key: str key: str
name: str name: str
@@ -474,6 +475,7 @@ class CompanyOut(BaseModel):
from_search: bool = False from_search: bool = False
hand_reason: str = "" hand_reason: str = ""
by_hand: bool = False by_hand: bool = False
all_countries: bool = False
all_roles: bool = False all_roles: bool = False
#: The board's own reach in its params (`jobs.reach.OVERRIDE_KEYS`), and #: The board's own reach in its params (`jobs.reach.OVERRIDE_KEYS`), and
#: what a pull asks its board for (`jobs.reach.line`). #: what a pull asks its board for (`jobs.reach.line`).
@@ -518,6 +520,7 @@ def _company_outs(store: Store, agent: int | None) -> list[CompanyOut]:
failing_since=since, failing_reason=reason, failing_since=since, failing_reason=reason,
from_search=from_search, hand_reason=c.hand_reason, from_search=from_search, hand_reason=c.hand_reason,
by_hand=c.key not in boards and not from_search, by_hand=c.key not in boards and not from_search,
all_countries=bool(c.all_countries),
all_roles=bool(c.all_roles), all_roles=bool(c.all_roles),
reach_params={k: v for k, v in (sub.params if sub else {}).items() reach_params={k: v for k, v in (sub.params if sub else {}).items()
if k in jobs.reach.OVERRIDE_KEYS}, if k in jobs.reach.OVERRIDE_KEYS},
@@ -715,27 +718,41 @@ def set_watch(key: str, body: WatchBody, store: StoreDep) -> Company:
raise HTTPException(404, f"no such company: {key}") from None raise HTTPException(404, f"no such company: {key}") from None
class AllRolesBody(BaseModel): class OverrideBody(BaseModel):
on: bool on: bool
@router.post("/companies/{key}/all-roles") def _override(store: Store, queue: Any, key: str, on: bool, setter: str) -> Company:
def set_all_roles(key: str, body: AllRolesBody, store: StoreDep, queue: QueueDep) -> Company: """Switch one of a company's two fetch overrides. On queues a pull right
"""Tag a company All roles, or untag it: global, apart from affinity. after, as the other reopening switches do; off queues nothing and asks
Its big boards are then read with the agents' wide reach, Europe today, nothing first, since no role closes until the next pull reads the board
rather than their places (`app/jobs/reach.py`). with its filters again. 404 for an unknown key."""
On queues a pull right after, as the other reopening switches do; off
queues nothing and asks nothing first, since no role closes until the
next pull reads the board with its filter again. 404 for an unknown key."""
try: try:
company = store.set_all_roles(key, body.on) company = getattr(store, setter)(key, on)
except KeyError: except KeyError:
raise HTTPException(404, f"no such company: {key}") from None raise HTTPException(404, f"no such company: {key}") from None
if body.on: if on:
queue.enqueue("pull") queue.enqueue("pull")
return company return company
@router.post("/companies/{key}/all-countries")
def set_all_countries(key: str, body: OverrideBody, store: StoreDep,
queue: QueueDep) -> Company:
"""All countries on or off: global, apart from affinity. On, the
company's big boards are read with the agents' wide reach, Europe today,
rather than their places (`app/jobs/reach.py`). See `_override`."""
return _override(store, queue, key, body.on, "set_all_countries")
@router.post("/companies/{key}/all-roles")
def set_all_roles(key: str, body: OverrideBody, store: StoreDep, queue: QueueDep) -> Company:
"""All roles on or off: global, apart from affinity. On, the company's
boards are read with no role-type narrowing, no fetch terms today
(`app/jobs/reach.py`). See `_override`."""
return _override(store, queue, key, body.on, "set_all_roles")
class CareersBody(BaseModel): class CareersBody(BaseModel):
careers_url: str careers_url: str
@@ -839,8 +856,9 @@ def switch_source(sub_id: int, body: SwitchBody, store: StoreDep,
class ReachBody(BaseModel): class ReachBody(BaseModel):
#: `agents`: the enabled agents' reach; `own`: the `countries` (ISO #: `agents`: the enabled agents' reach; `own`: the `countries` (ISO
#: codes) and remote given here. Every role is the company's All roles #: codes) and remote given here. The wide reach and no role narrowing
#: tag (`/api/companies/{key}/all-roles`), not a board's reach. #: are the company's overrides (`/api/companies/{key}/all-countries`,
#: `/all-roles`), not a board's reach.
mode: Literal["agents", "own"] mode: Literal["agents", "own"]
countries: list[str] = [] countries: list[str] = []
remote: bool = False remote: bool = False
@@ -851,8 +869,8 @@ def set_reach(sub_id: int, body: ReachBody, store: StoreDep) -> dict[str, Any]:
"""A board's own reach, which wins over the agents' (see """A board's own reach, which wins over the agents' (see
`app/jobs/reach.py`), or none. The next pull reads the board so; a `app/jobs/reach.py`), or none. The next pull reads the board so; a
narrower reach closes the roles outside it then, except those with a narrower reach closes the roles outside it then, except those with a
vote or an application. A company tagged All roles reads every role vote or an application. A reach set here wins over the company's All
whatever its board's reach says.""" countries override."""
sub = store.subscription(sub_id) sub = store.subscription(sub_id)
if sub is None or sub.source != "ats": if sub is None or sub.source != "ats":
raise HTTPException(404, f"no such board: {sub_id}") raise HTTPException(404, f"no such board: {sub_id}")
+2 -2
View File
@@ -112,7 +112,7 @@ What was decided for roledesk and the one reason for each. The longer reasoning
**One runner calls every reader.** `plugins/readers/_runner.py` owns the guarded fetch, paging, one request series per filter value unioned by id, completeness, the fallback feed, 429s and failures. `plugins/ats.py` and `plugins/search.py` are thin `Source` classes over it and keep their state prefixes, so no stored key moved. A paged read cut by `MAX_PAGES` saves its cursor and continues on the next pull; the sweep that reaches the end closes what it did not list. A host ceiling (Workday 2,000) is not swept; a filter has to get under it. **One runner calls every reader.** `plugins/readers/_runner.py` owns the guarded fetch, paging, one request series per filter value unioned by id, completeness, the fallback feed, 429s and failures. `plugins/ats.py` and `plugins/search.py` are thin `Source` classes over it and keep their state prefixes, so no stored key moved. A paged read cut by `MAX_PAGES` saves its cursor and continues on the next pull; the sweep that reaches the end closes what it did not list. A host ceiling (Workday 2,000) is not swept; a filter has to get under it.
**The agents set the reach of a pull.** `app/jobs/reach.py` computes it once per pull: the union of the enabled agents' `any` gates on country, town (as its country) and remote, or everything when one agent has none. A company tagged **All roles** (`companies.all_roles`, migration 21) is read with the wide reach: the union of the enabled agents' `require` reach, Europe today, and the union of the `any` gates, so a tag never reads less than the board would untagged. An agent with no `require` gate on place makes the wide reach everywhere. For a company the owner cares about, a role in Freiburg may be open to remote, or worth asking about for Denmark, while Microsoft Germany is not wanted. The tag is separate from favourite and liked, and global rather than per agent, because it decides what is fetched and fetching is shared. Favourite (affinity 2) affects scoring only, where it skips the `any` gates; it used to widen the fetch to its agent's `require` reach, and migration 21 tagged every company that was a favourite of any agent so nothing read before is lost. The tag also replaced a board's `{"reach": "all"}` param, which read everywhere: migration 21 tagged those companies and logged each such board, since no everywhere level is left. A board's `params` override the agents' reach and the tag. Boards under 300 roles, and hosts with a one-country filter over more than 8 countries, are read whole. SmartRecruiters, Eightfold, SuccessFactors, Phenom, Workday and Oracle filter at the source; countries and remote go as separate series, because the hosts intersect two facets. A complete filtered read closes roles outside the filter, except roles with a vote or an application; a filtered read, or one from a host whose unknown slug answers empty, that lists under 10% of the last read with the same filter counts as failed. **The agents set the reach of a pull.** `app/jobs/reach.py` computes it once per pull: the union of the enabled agents' `any` gates on country, town (as its country) and remote, or everything when one agent has none. Fetching has two filters, countries and role types, and a company has an override for each. A company set to **All countries** (`companies.all_countries`, migration 24) is read with the wide reach: the union of the enabled agents' `require` reach, Europe today, and the union of the `any` gates, so the override never reads less than the board would without it. An agent with no `require` gate on place makes the wide reach everywhere. For a company the owner cares about, a role in Freiburg may be open to remote, or worth asking about for Denmark, while Microsoft Germany is not wanted. The overrides are separate from favourite and liked, and global rather than per agent, because they decide what is fetched and fetching is shared. They matter only for a board of 300 roles or more on a host that filters, so a small company is affected once it grows past that. The Companies list states both halves per board, e.g. "follows agents: DK, SE, remote · all roles" or "all countries (Europe) · follows agents' terms: engineer, product, data", or why the board is read whole. Favourite (affinity 2) affects scoring only, where it skips the `any` gates; it used to widen the fetch to its agent's `require` reach, and migration 21 tagged every company that was a favourite of any agent so nothing read before is lost. That tag, first named All roles in `companies.all_roles`, also replaced a board's `{"reach": "all"}` param, which read everywhere: migration 21 tagged those companies and logged each such board, since no everywhere level is left. Migration 24 renamed the tag to All countries and gave its companies (Mistral AI and OpenAI on live) All roles as well, since the owner wanted their every role. A board's `params` override the agents' reach and All countries. Boards under 300 roles, and hosts with a one-country filter over more than 8 countries, are read whole. SmartRecruiters, Eightfold, SuccessFactors, Phenom, Workday and Oracle filter at the source; countries and remote go as separate series, because the hosts intersect two facets. A complete filtered read closes roles outside the filter, except roles with a vote or an application; a filtered read, or one from a host whose unknown slug answers empty, that lists under 10% of the last read with the same filter counts as failed.
**A role has a list of places, resolved against a gazetteer.** Readers kept one place of several, joined them, or the host posted a copy per place. A reader now returns every place and never multiplies a row. `places` (city, region, country, continent, with `place_includes` for Øresund across two countries) and `place_aliases` are seeded from the dictionaries that were in code; `role_places` links each place of a role, and an unknown text is listed by `unresolved_places` until an alias links it. The Place filter and the gates join over the tree, which halved their time and dropped the `EU` fallback. Copies of one posting per place, same host, company and title and texts at least 30% alike, fold into one role that keeps each copy's place and link; the rule applies to every host, since the 5,024 such roles came mostly from hosts that do not declare copies. **A role has a list of places, resolved against a gazetteer.** Readers kept one place of several, joined them, or the host posted a copy per place. A reader now returns every place and never multiplies a row. `places` (city, region, country, continent, with `place_includes` for Øresund across two countries) and `place_aliases` are seeded from the dictionaries that were in code; `role_places` links each place of a role, and an unknown text is listed by `unresolved_places` until an alias links it. The Place filter and the gates join over the tree, which halved their time and dropped the `EU` fallback. Copies of one posting per place, same host, company and title and texts at least 30% alike, fold into one role that keeps each copy's place and link; the rule applies to every host, since the 5,024 such roles came mostly from hosts that do not declare copies.
@@ -122,7 +122,7 @@ What was decided for roledesk and the one reason for each. The longer reasoning
**Readers scrape; `prep` decides.** A reader hands on the host's own words (`seniority`, `employment`, `remote`, and `country` only when the host states one per role) and does light cleanup only: text from HTML, dates, the places of a role as a list, pay figures. Mapping lived in 25 readers, each spelling its tables its own way, so changing what "Experienced" means touched a plugin. The `prep` stage (`app/prep/`) maps centrally after every pull and every enrich run. Seniority, employment and remote go through the host's table in `app/prep/hosts/<reader>.py` (whole words, or `MATCH` fragments by substring or token), then a shared word map. Country comes from the resolved places (several European countries give `EU`), and the host's country is only a fallback. Places and company aliases are resolved, and copies and duplicates are folded over the whole database. `upsert_role` stores the host words (`*_host` columns, migration 22) and marks the role `prep_due`. When a table changes, its fingerprint makes the next prep map every open role again from the stored words, with no pull. A word no table knows is listed in `unmapped` (`app prep unmapped`, the Sources page). A host's `remote: False` stays unknown rather than onsite, because SmartRecruiters, Ashby, Recruitee and Workable send it for hybrid roles too. **Readers scrape; `prep` decides.** A reader hands on the host's own words (`seniority`, `employment`, `remote`, and `country` only when the host states one per role) and does light cleanup only: text from HTML, dates, the places of a role as a list, pay figures. Mapping lived in 25 readers, each spelling its tables its own way, so changing what "Experienced" means touched a plugin. The `prep` stage (`app/prep/`) maps centrally after every pull and every enrich run. Seniority, employment and remote go through the host's table in `app/prep/hosts/<reader>.py` (whole words, or `MATCH` fragments by substring or token), then a shared word map. Country comes from the resolved places (several European countries give `EU`), and the host's country is only a fallback. Places and company aliases are resolved, and copies and duplicates are folded over the whole database. `upsert_role` stores the host words (`*_host` columns, migration 22) and marks the role `prep_due`. When a table changes, its fingerprint makes the next prep map every open role again from the stored words, with no pull. A word no table knows is listed in `unmapped` (`app prep unmapped`, the Sources page). A host's `remote: False` stays unknown rather than onsite, because SmartRecruiters, Ashby, Recruitee and Workable send it for hybrid roles too.
**Fetch terms narrow what big boards send across Europe.** An agent's `fetch_terms` (migration 23) are words a big board must match when it is read with the wide reach: a company tagged All roles, or a one-country host whose reach was dropped for needing over 8 requests. The terms of a pull are the union over the enabled agents; one enabled agent without terms switches them off. The narrow reach takes no terms, because it already leaves at most about 150 roles on any big board. The survey of 2026-09-30 found that every host's keyword matches the whole text, so a term cuts less than a title filter would ("engineer" left 1,713 of Bosch's 4,830). SmartRecruiters, SuccessFactors, Workday, Phenom, Eightfold and Oracle take one term a request, with one series per term unioned by id; over 8 series the place filter goes first, then the terms. A term-filtered read is never the whole board: it writes no size, prunes no per-role state and closes nothing. Category and level filters wait until something maps agent terms to host ids. A town in a gate is still fetched as its country. **Fetch terms narrow what big boards send across Europe.** An agent's `fetch_terms` (migration 23) are words a big board must match when it is read with the wide reach: a company set to All countries, or a one-country host whose reach was dropped for needing over 8 requests. A company set to **All roles** (`companies.all_roles`, migration 24) takes no terms on any of its boards, and will take no category or level filter either. It replaced a board's `{"terms": false}` param, which migration 24 moved to the board's company. On its own, All roles changes nothing today, since the narrow reach takes no terms; it is kept for the role filters to come. The terms of a pull are the union over the enabled agents; one enabled agent without terms switches them off. The narrow reach takes no terms, because it already leaves at most about 150 roles on any big board. The survey of 2026-09-30 found that every host's keyword matches the whole text, so a term cuts less than a title filter would ("engineer" left 1,713 of Bosch's 4,830). SmartRecruiters, SuccessFactors, Workday, Phenom, Eightfold and Oracle take one term a request, with one series per term unioned by id; over 8 series the place filter goes first, then the terms. A term-filtered read is never the whole board: it writes no size, prunes no per-role state and closes nothing. Category and level filters wait until something maps agent terms to host ids. A town in a gate is still fetched as its country.
**Seniority has three sources.** `seniority_stated` is the host's word mapped by that host's table in `app/prep/hosts/`, `seniority_title` is `seniority.from_title`, and `seniority_model` is `extract`'s verdict; `roles.seniority` is the first of model, stated, title. `extract` reads roles without a model value, every agent's first pick before anyone's second, with the other two as hints it may overrule, since host vocabularies differ and 57% of roles state nothing. `app seniority` shows per host how often the stated level agrees with the model. **Seniority has three sources.** `seniority_stated` is the host's word mapped by that host's table in `app/prep/hosts/`, `seniority_title` is `seniority.from_title`, and `seniority_model` is `extract`'s verdict; `roles.seniority` is the first of model, stated, title. `extract` reads roles without a model value, every agent's first pick before anyone's second, with the other two as hints it may overrule, since host vocabularies differ and 57% of roles state nothing. `app seniority` shows per host how often the stated level agrees with the model.
+2 -2
View File
@@ -69,9 +69,9 @@ Numbers are from the live database after the pull of 2026-09-29. For the "list,
| robots.txt | read before the list for the 14 hosts that name a page: Workday, SuccessFactors, Oracle, Phenom, Eightfold, Jibe, Gem, BambooHR, Pinpoint, Deel, Jobylon, Rippling, HiBob, HR Manager; kept for one day per board | A disallow fails the board with "robots.txt disallows it" and counts toward the 21 days. An unreachable file (5xx, 429, timeout) fails the board for this pull and is asked again next pull. A 4xx means no rules, which allows. `Crawl-delay` is not honoured. The eight API hosts (Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee, Personio, Teamtailor) and the search providers read no robots.txt. | | robots.txt | read before the list for the 14 hosts that name a page: Workday, SuccessFactors, Oracle, Phenom, Eightfold, Jibe, Gem, BambooHR, Pinpoint, Deel, Jobylon, Rippling, HiBob, HR Manager; kept for one day per board | A disallow fails the board with "robots.txt disallows it" and counts toward the 21 days. An unreachable file (5xx, 429, timeout) fails the board for this pull and is asked again next pull. A 4xx means no rules, which allows. `Crawl-delay` is not honoured. The eight API hosts (Greenhouse, Lever, Ashby, Workable, SmartRecruiters, Recruitee, Personio, Teamtailor) and the search providers read no robots.txt. |
| 429 | first page: the board fails; a later page: the read stops, keeps what it read and is incomplete; a per-role read: the per-role reads of that board stop until the next pull | Nothing is closed on a list cut short by a 429. A search that fails is skipped for the pull. | | 429 | first page: the board fails; a later page: the read stops, keeps what it read and is incomplete; a per-role read: the per-role reads of that board stop until the next pull | Nothing is closed on a list cut short by a 429. A search that fails is skipped for the pull. |
How far a board is read: a pull asks each board over 300 roles for the reach of the search agents that are switched on (`app/jobs/reach.py`): the union of their `any` gates on country, town (as its country) and remote, or every role when one agent can take a role from anywhere. A company tagged All roles is asked for the agents' wide reach, the union of their `require` reach, Europe today; a host that asks one country a request reads such a board whole instead of 30 times, unless fetch terms narrow it. A board's own reach, set on the Companies list, wins. A complete filtered read closes the roles outside the reach, except those with a vote or an application. How far a board is read: a pull asks each board over 300 roles for the reach of the search agents that are switched on (`app/jobs/reach.py`): the union of their `any` gates on country, town (as its country) and remote, or every role when one agent can take a role from anywhere. A company set to All countries is asked for the agents' wide reach, the union of their `require` reach, Europe today; a host that asks one country a request reads such a board whole instead of 30 times, unless fetch terms narrow it. A board's own reach, set on the Companies list, wins. A complete filtered read closes the roles outside the reach, except those with a vote or an application.
Fetch terms narrow a wide read by words. Each agent has a list of them (`fetch_terms`, set in the agent editor); the terms of a pull are the union over the agents that are on, and none when one of those agents has none. They go only to a board read with the wide reach: a company tagged All roles, or a board whose reach is more countries than a one-country host is asked for (`MAX_SERIES`, 8). The narrow reach (Denmark, Sweden) leaves at most about 150 roles on any big board, so it takes no terms, and nor does a board under 300 roles, one never read whole, or one with its own reach. `{"terms": false}` in a board's params switches them off. A host that takes a keyword gets one series per term, unioned by id, each combined with the country series (an AND on every host the survey tested). When terms times the country series exceed 8, the place part of the reach goes first, whole, and the terms stay; more than 8 terms read the board unfiltered. Every host matches the whole text, not only the title, so a term like `engineer` cuts less than it looks. A read with terms is never the whole board: it records no size, closes no role it did not list, and the 10% guard holds it only to a read with the same terms and places. The search providers take no terms: each saved search has its own query. Fetch terms narrow a wide read by words. Each agent has a list of them (`fetch_terms`, set in the agent editor); the terms of a pull are the union over the agents that are on, and none when one of those agents has none. They go only to a board read with the wide reach: a company set to All countries, or a board whose reach is more countries than a one-country host is asked for (`MAX_SERIES`, 8). The narrow reach (Denmark, Sweden) leaves at most about 150 roles on any big board, so it takes no terms, and nor does a board under 300 roles, one never read whole, or one with its own reach. A company set to All roles takes none on any of its boards. A host that takes a keyword gets one series per term, unioned by id, each combined with the country series (an AND on every host the survey tested). When terms times the country series exceed 8, the place part of the reach goes first, whole, and the terms stay; more than 8 terms read the board unfiltered. Every host matches the whole text, not only the title, so a term like `engineer` cuts less than it looks. A read with terms is never the whole board: it records no size, closes no role it did not list, and the 10% guard holds it only to a read with the same terms and places. The search providers take no terms: each saved search has its own query.
What no source does: What no source does:
+6 -5
View File
@@ -340,10 +340,11 @@ def ask_for(module: ModuleType, subscription_params: Mapping[str, Any] | None =
A module without `READS` gets `Ask()`. A module without `READS` gets `Ask()`.
Fetch terms (`reach["terms"]`) go only to a board reader that declares Fetch terms (`reach["terms"]`) go only to a board reader that declares
`query="one"`, and only on a wide read: `reach["wide"]` (a company `query="one"`, and only on a wide read: `reach["wide"]` (a company set
tagged All roles), or a reach this rule just dropped as too many to All countries), or a reach this rule just dropped as too many
countries. A board whose params say `"terms": false` or name a query of countries. A company set to All roles gets no terms in its reach (see
their own takes none. Terms times the reach's series (`_cost`) may cost `app/jobs/reach.py`), and a board whose params name a query of their
own takes none. Terms times the reach's series (`_cost`) may cost
at most `MAX_SERIES`: over it, the place part of the reach goes first, at most `MAX_SERIES`: over it, the place part of the reach goes first,
whole, since remote without its countries would read only the remote whole, since remote without its countries would read only the remote
roles; over it with terms alone, the terms go too and the board is read roles; over it with terms alone, the terms go too and the board is read
@@ -365,7 +366,7 @@ def ask_for(module: ModuleType, subscription_params: Mapping[str, Any] | None =
wanted = {name: value for name, value in wanted.items() if name in declared_now} wanted = {name: value for name, value in wanted.items() if name in declared_now}
terms = _terms(given.get("terms")) terms = _terms(given.get("terms"))
if (not wide or reads.scope != "board" or reads.filters.query != "one" if (not wide or reads.scope != "board" or reads.filters.query != "one"
or params.get("terms") is False or "query" in wanted): or "query" in wanted):
terms = () terms = ()
if terms and len(terms) * _cost(reads, wanted) > MAX_SERIES: if terms and len(terms) * _cost(reads, wanted) > MAX_SERIES:
wanted = {name: value for name, value in wanted.items() wanted = {name: value for name, value in wanted.items()
+7 -4
View File
@@ -148,7 +148,7 @@ def test_migration_20_seeds_an_old_file_and_leaves_collisions(tmp_path: Path) ->
"VALUES (?, ?, '', '')", (key, name)) "VALUES (?, ?, '', '')", (key, name))
conn.execute("PRAGMA user_version = 19") conn.execute("PRAGMA user_version = 19")
reopened = Store(path) reopened = Store(path)
assert reopened.migrations_applied == [20, 21, 22, 23] assert reopened.migrations_applied == [20, 21, 22, 23, 24]
assert reopened.duplicate_companies() == [("orsted", ["orsted", "ørsted"])] assert reopened.duplicate_companies() == [("orsted", ["orsted", "ørsted"])]
assert len(reopened.companies()) == 3 assert len(reopened.companies()) == 3
@@ -183,11 +183,14 @@ def test_migration_21_tags_the_favourites_and_the_boards_that_read_all_roles(tmp
store.set_affinity(off.id, "liked", 1) # type: ignore[arg-type] store.set_affinity(off.id, "liked", 1) # type: ignore[arg-type]
with sqlite3.connect(path) as conn: with sqlite3.connect(path) as conn:
conn.execute("ALTER TABLE companies DROP COLUMN all_roles") conn.execute("ALTER TABLE companies DROP COLUMN all_roles")
conn.execute("ALTER TABLE companies DROP COLUMN all_countries")
conn.execute("PRAGMA user_version = 20") conn.execute("PRAGMA user_version = 20")
reopened = Store(path) reopened = Store(path)
assert reopened.migrations_applied == [21, 22, 23] assert reopened.migrations_applied == [21, 22, 23, 24]
assert {c.key: c.all_roles for c in reopened.companies()} == { # The tag of 21 is All countries and All roles since 24.
"mistral-ai": 1, "acme": 0, "black-forest-labs": 1, "liked": 0} tagged = {"mistral-ai": 1, "acme": 0, "black-forest-labs": 1, "liked": 0}
assert {c.key: c.all_roles for c in reopened.companies()} == tagged
assert {c.key: c.all_countries for c in reopened.companies()} == tagged
params = {s.target: s.params for s in reopened.subscriptions(source="ats")} params = {s.target: s.params for s in reopened.subscriptions(source="ats")}
assert params == {"smartrecruiters:mistral": {"note": "kept"}, assert params == {"smartrecruiters:mistral": {"note": "kept"},
"workday:acme": {"countries": ["DK"]}, "greenhouse:ghost": {}} "workday:acme": {"countries": ["DK"]}, "greenhouse:ghost": {}}
+1 -1
View File
@@ -246,7 +246,7 @@ def test_a_v17_file_gains_dup_of_with_its_first_rebuild(tmp_path: Path) -> None:
migrated = Store(path) migrated = Store(path)
assert migrated.migrations_applied == [18, 19, 20, 21, 22, 23] assert migrated.migrations_applied == [18, 19, 20, 21, 22, 23, 24]
assert dup_of(migrated) == {first.uid: None, second.uid: first.uid} assert dup_of(migrated) == {first.uid: None, second.uid: first.uid}
with sqlite3.connect(path) as conn: with sqlite3.connect(path) as conn:
assert conn.execute("SELECT 1 FROM sqlite_master WHERE name = 'idx_roles_dup_of'" assert conn.execute("SELECT 1 FROM sqlite_master WHERE name = 'idx_roles_dup_of'"
+17 -10
View File
@@ -72,7 +72,7 @@ def _board(store: Store, target: str, company: str, size: int | None = 1000,
store.state[f"ats:size:{target}"] = str(size) store.state[f"ats:size:{target}"] = str(size)
if tagged: if tagged:
from app.models import company_key from app.models import company_key
store.set_all_roles(company_key(company), True) store.set_all_countries(company_key(company), True)
def _sub(store: Store, target: str) -> Any: def _sub(store: Store, target: str) -> Any:
@@ -123,12 +123,12 @@ def test_the_narrow_reach_takes_no_terms(store: Store) -> None:
sub = _sub(store, "smartrecruiters:BoschGroup") sub = _sub(store, "smartrecruiters:BoschGroup")
ask = _runner.ask_for(smartrecruiters, sub.params, plan.for_subscription(sub)) ask = _runner.ask_for(smartrecruiters, sub.params, plan.for_subscription(sub))
assert ask == Ask(countries=("DK", "SE")) assert ask == Ask(countries=("DK", "SE"))
assert reach.line(plan, sub) == "DK, SE · agents" assert reach.line(plan, sub) == "follows agents: DK, SE · all roles"
@pytest.mark.parametrize("size, params", [ @pytest.mark.parametrize("size, params", [
(299, {}), (None, {}), (5000, {"countries": ["DE"]}), (5000, {"terms": False}), (299, {}), (None, {}), (5000, {"countries": ["DE"]}),
], ids=["small", "never read whole", "own reach", "terms off"]) ], ids=["small", "never read whole", "own reach"])
def test_small_unknown_overridden_and_switched_off_boards_take_no_terms( def test_small_unknown_overridden_and_switched_off_boards_take_no_terms(
store: Store, size: int | None, params: dict[str, object]) -> None: store: Store, size: int | None, params: dict[str, object]) -> None:
_wolf(store, TERMS) _wolf(store, TERMS)
@@ -148,7 +148,13 @@ def test_the_agents_reach_takes_terms_once_it_is_too_many_countries_to_ask(
sub = _sub(store, "smartrecruiters:BoschGroup") sub = _sub(store, "smartrecruiters:BoschGroup")
ask = _runner.ask_for(smartrecruiters, sub.params, plan.for_subscription(sub)) ask = _runner.ask_for(smartrecruiters, sub.params, plan.for_subscription(sub))
assert ask == Ask(terms=("data", "engineer", "product")) assert ask == Ask(terms=("data", "engineer", "product"))
assert reach.line(plan, sub) == "all places · agents · 3 terms" assert reach.line(plan, sub) == \
"all places, too many countries to ask · follows agents' terms: data, engineer, product"
# All roles drops them there too; the countries stay dropped.
store.set_all_roles("bosch", True)
plan = reach.plan(store)
assert _runner.ask_for(smartrecruiters, sub.params, plan.for_subscription(sub)) == Ask()
assert reach.line(plan, sub) == "too many countries to ask: read whole"
# ------------------------------------------------------ terms times the reach # ------------------------------------------------------ terms times the reach
@@ -234,16 +240,16 @@ def test_the_reach_line_counts_the_terms(store: Store) -> None:
plan = reach.plan(store) plan = reach.plan(store)
philips = _sub(store, "phenom:www.careers.philips.com/global/en") philips = _sub(store, "phenom:www.careers.philips.com/global/en")
assert reach.line(plan, philips) == \ assert reach.line(plan, philips) == \
f"{len(plan.wide.countries)} countries · company tag · 3 terms" "all countries (Europe) · follows agents' terms: data, engineer, head of"
assert reach.line(plan, _sub(store, "smartrecruiters:BoschGroup")) == \ assert reach.line(plan, _sub(store, "smartrecruiters:BoschGroup")) == \
"all places · company tag · 3 terms" "all places, too many countries to ask · follows agents' terms: data, engineer, head of"
def test_one_term_is_one_term(store: Store) -> None: def test_one_term_is_one_term(store: Store) -> None:
_wolf(store, ["engineer"]) _wolf(store, ["engineer"])
_board(store, "smartrecruiters:BoschGroup", "Bosch") _board(store, "smartrecruiters:BoschGroup", "Bosch")
assert reach.line(reach.plan(store), _sub(store, "smartrecruiters:BoschGroup")) == \ assert reach.line(reach.plan(store), _sub(store, "smartrecruiters:BoschGroup")) == \
"all places · company tag · 1 term" "all places, too many countries to ask · follows agents' terms: engineer"
# ------------------------------------------------------ each host's request # ------------------------------------------------------ each host's request
@@ -431,7 +437,7 @@ def test_migration_23_gives_every_agent_no_terms(tmp_path: Path) -> None:
conn.execute("ALTER TABLE agents DROP COLUMN fetch_terms") conn.execute("ALTER TABLE agents DROP COLUMN fetch_terms")
conn.execute("PRAGMA user_version = 22") conn.execute("PRAGMA user_version = 22")
migrated = Store(path) migrated = Store(path)
assert migrated.migrations_applied == [23] assert migrated.migrations_applied == [23, 24]
assert migrated.agents()[0].fetch_terms == [] assert migrated.agents()[0].fetch_terms == []
@@ -469,4 +475,5 @@ def test_the_companies_list_says_the_terms(client: TestClient, app: FastAPI) ->
_wolf(store, ["engineer", "data", "head of"]) _wolf(store, ["engineer", "data", "head of"])
_board(store, "smartrecruiters:BoschGroup", "Bosch") _board(store, "smartrecruiters:BoschGroup", "Bosch")
row = next(c for c in client.get("/api/companies").json() if c["key"] == "bosch") row = next(c for c in client.get("/api/companies").json() if c["key"] == "bosch")
assert row["reach"] == "all places · company tag · 3 terms" assert row["reach"] == \
"all places, too many countries to ask · follows agents' terms: data, engineer, head of"
+155
View File
@@ -0,0 +1,155 @@
"""A company's two fetch overrides (see "The agents set the reach of a pull"
and "Fetch terms" in `docs/decisions.md`): All countries reads its big boards
with the agents' wide reach, All roles with no role-type narrowing. The four
combinations, the reach line, migration 24 and a merge. No network."""
from __future__ import annotations
import json
import sqlite3
import sys
from pathlib import Path
import pytest
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
from app.jobs import reach # noqa: E402
from app.models import Gate # noqa: E402
from app.storage import Store # noqa: E402
from plugins.readers import Ask, _runner, phenom # noqa: E402
TERMS = ["engineer", "product", "data"]
PHILIPS = "phenom:www.careers.philips.com/global/en"
@pytest.fixture
def store(tmp_path: Path) -> Store:
found = Store(tmp_path / "roledesk.db")
# Agent 2 of 2026-09-30: Europe required; Denmark, Sweden or remote.
found.save_agent("Wolf", fetch_terms=TERMS, gates=[
Gate(field="place", values=["Europe"], mode="require"),
Gate(field="country", values=["DK", "SE"], mode="any"),
Gate(field="remote", values=["remote"], mode="any")])
return found
def _board(store: Store, target: str, company: str, size: int = 5000) -> None:
store.add_subscription("ats", target, label=company, company=company,
board=target.partition(":")[0])
store.add_company(company)
store.state[f"ats:size:{target}"] = str(size)
def _read(store: Store, target: str) -> tuple[Ask, str]:
"""What the next pull asks the board for, and the Companies list's line."""
plan = reach.plan(store)
sub = next(s for s in store.subscriptions(source="ats") if s.target == target)
return _runner.ask_for(phenom, sub.params, plan.for_subscription(sub)), reach.line(plan, sub)
@pytest.mark.parametrize("countries, roles, line", [
(False, False, "follows agents: DK, SE · all roles"),
(False, True, "follows agents: DK, SE · all roles"),
(True, False, "all countries (Europe) · follows agents' terms: data, engineer, product"),
(True, True, "all countries (Europe) · all roles"),
], ids=["neither", "all roles alone", "all countries alone", "both"])
def test_the_four_combinations(store: Store, countries: bool, roles: bool, line: str) -> None:
_board(store, PHILIPS, "Philips")
store.set_all_countries("philips", countries)
store.set_all_roles("philips", roles)
ask, said = _read(store, PHILIPS)
assert said == line
# Phenom asks for countries, many in one request, and not for remote.
if countries:
assert {"DE", "FR", "DK", "SE"} <= set(ask.countries) and len(ask.countries) > 30
else:
assert ask.countries == ("DK", "SE")
# Terms go only with the wide reach, and never to a company set to All roles.
assert ask.terms == (("data", "engineer", "product") if countries and not roles else ())
def test_all_roles_alone_changes_nothing_today(store: Store) -> None:
_board(store, PHILIPS, "Philips")
before = _read(store, PHILIPS)
store.set_all_roles("philips", True)
assert _read(store, PHILIPS) == before
@pytest.mark.parametrize("target, size, line", [
(PHILIPS, 120, "under 300 roles: read whole"),
("greenhouse:acme", 5000, "Greenhouse cannot filter: read whole"),
], ids=["small", "greenhouse"])
def test_a_board_read_whole_says_why_whatever_the_overrides(
store: Store, target: str, size: int, line: str) -> None:
_board(store, target, "Acme", size)
for countries, roles in ((False, False), (True, False), (True, True)):
store.set_all_countries("acme", countries)
store.set_all_roles("acme", roles)
assert _read(store, target)[1] == line
def test_a_merge_keeps_both_overrides(store: Store) -> None:
store.add_company("Mistral")
store.add_company("Mistral AI")
store.set_all_countries("mistral", True)
store.set_all_roles("mistral-ai", True)
kept = store.merge_companies("mistral-ai", "mistral")
assert (kept.all_countries, kept.all_roles) == (1, 1)
# ------------------------------------------------------------ migration 24
def test_migration_24_splits_the_tag_and_moves_terms_off(tmp_path: Path) -> None:
path = tmp_path / "old.db"
store = Store(path)
for name in ("Mistral AI", "Acme", "Plain"):
store.add_company(name)
store.add_subscription("ats", "workday:acme", label="Acme", company="Acme",
board="workday", params={"terms": False, "note": "kept"})
store.add_subscription("ats", "workday:plain", label="Plain", company="Plain",
board="workday", params={"countries": ["DK"]})
# A board whose company has no row: its params are cleaned, no flag lands.
store.add_subscription("ats", "workday:ghost", label="Ghost", company="Ghost",
board="workday", params={"terms": False})
with sqlite3.connect(path) as conn:
conn.execute("ALTER TABLE companies DROP COLUMN all_countries")
# Before 24, all_roles was the All roles tag, the wide reach.
conn.execute("UPDATE companies SET all_roles=1 WHERE key='mistral-ai'")
conn.execute("PRAGMA user_version = 23")
migrated = Store(path)
assert migrated.migrations_applied == [24]
flags = {c.key: (c.all_countries, c.all_roles) for c in migrated.companies()}
assert flags == {"mistral-ai": (1, 1), "acme": (0, 1), "plain": (0, 0)}
params = {s.target: s.params for s in migrated.subscriptions(source="ats")}
assert params == {"workday:acme": {"note": "kept"}, "workday:plain": {"countries": ["DK"]},
"workday:ghost": {}}
def test_migration_24_runs_twice_without_harm(tmp_path: Path) -> None:
from app.storage.schema import _split_all_roles
path = tmp_path / "db.db"
store = Store(path)
store.add_company("Acme")
store.set_all_roles("acme", True)
with sqlite3.connect(path) as conn:
_split_all_roles(conn)
row = conn.execute("SELECT all_countries, all_roles FROM companies").fetchone()
assert json.dumps(list(row)) == "[0, 1]"
def test_the_wide_reach_is_named_as_the_gates_name_it(store: Store) -> None:
gaz = store.gazetteer()
europe = Gate(field="country", values=["AT", "DK", "SE", "FR", "EU"], mode="require")
nordics = Gate(field="country", values=["DK", "SE"], mode="require")
anywhere = Gate(field="country", values=["DK"], mode="any")
agents = [a.model_copy(update={"gates": [g]}) for a, g in
((store.agents()[0], europe), (store.agents()[0], nordics))]
# Countries one by one are named by their continent; a short list as written.
assert reach.wide_names(agents[:1], gaz) == "Europe"
assert reach.wide_names(agents[1:], gaz) == "DK, SE"
assert reach.wide_names([agents[0].model_copy(update={"gates": [anywhere]})],
gaz) == "everywhere"
+1 -1
View File
@@ -299,7 +299,7 @@ def test_migration_22_fills_the_host_words_from_raw(tmp_path: Path) -> None:
migrated = Store(path) migrated = Store(path)
assert migrated.migrations_applied == [22, 23] assert migrated.migrations_applied == [22, 23, 24]
one = row(migrated, "1") one = row(migrated, "1")
assert (one["seniority_host"], one["employment_host"], one["remote_host"], assert (one["seniority_host"], one["employment_host"], one["remote_host"],
one["country_host"]) == ("senior", "full_time", "true", "DK") one["country_host"]) == ("senior", "full_time", "true", "DK")
+15 -16
View File
@@ -111,14 +111,14 @@ def test_a_company_tagged_all_roles_is_read_across_the_require_reach(store: Stor
_wolf(store) _wolf(store)
_board(store, "workday:mistral.wd3/Jobs", "Mistral AI") _board(store, "workday:mistral.wd3/Jobs", "Mistral AI")
_board(store, "workday:acme.wd3/Jobs", "Acme") _board(store, "workday:acme.wd3/Jobs", "Acme")
store.set_all_roles("mistral-ai", True) store.set_all_countries("mistral-ai", True)
plan = reach.plan(store) plan = reach.plan(store)
subs = {s.target.split(".")[0]: s for s in store.subscriptions(source="ats")} subs = {s.target.split(".")[0]: s for s in store.subscriptions(source="ats")}
wide, why = plan.why(subs["workday:mistral"]) wide, why = plan.why(subs["workday:mistral"])
# Europe, and the base's remote, which a tag never reads less than. # Europe, and the base's remote, which a tag never reads less than.
assert why == "tag" and {"DE", "FR", "DK"} <= wide.countries and wide.remote assert why == "countries" and {"DE", "FR", "DK"} <= wide.countries and wide.remote
assert reach.line(plan, subs["workday:mistral"]) == \ # The gate lists Europe's countries one by one; the line names the continent.
f"{len(wide.countries)} countries, remote · company tag" assert reach.line(plan, subs["workday:mistral"]) == "all countries (Europe) · all roles"
assert plan.why(subs["workday:acme"]) == (plan.base, "agents") assert plan.why(subs["workday:acme"]) == (plan.base, "agents")
@@ -135,39 +135,38 @@ def test_a_favourite_does_not_widen_the_fetch(store: Store) -> None:
def test_a_tag_too_wide_for_a_one_host_reads_the_board_whole(store: Store, host: str) -> None: def test_a_tag_too_wide_for_a_one_host_reads_the_board_whole(store: Store, host: str) -> None:
_wolf(store) _wolf(store)
_board(store, f"{host}:mistral", "Mistral AI") _board(store, f"{host}:mistral", "Mistral AI")
store.set_all_roles("mistral-ai", True) store.set_all_countries("mistral-ai", True)
[sub] = store.subscriptions(source="ats") [sub] = store.subscriptions(source="ats")
assert reach.line(reach.plan(store), sub) == \ assert reach.line(reach.plan(store), sub) == "too many countries to ask: read whole"
"all roles · company tag, too many countries to ask"
def test_a_tag_asks_phenom_for_europe_in_one_request(store: Store) -> None: def test_a_tag_asks_phenom_for_europe_in_one_request(store: Store) -> None:
_wolf(store) _wolf(store)
_board(store, "phenom:mistral", "Mistral AI") _board(store, "phenom:mistral", "Mistral AI")
store.set_all_roles("mistral-ai", True) store.set_all_countries("mistral-ai", True)
[sub] = store.subscriptions(source="ats") [sub] = store.subscriptions(source="ats")
found = reach.plan(store) found = reach.plan(store)
assert reach.line(found, sub) == f"{len(found.wide.countries)} countries · company tag" assert reach.line(found, sub) == "all countries (Europe) · all roles"
def test_an_agent_without_a_required_place_makes_the_tag_everywhere(store: Store) -> None: def test_an_agent_without_a_required_place_makes_the_tag_everywhere(store: Store) -> None:
_wolf(store) _wolf(store)
store.save_agent("Anywhere", gates=[Gate(field="country", values=["US"], mode="any")]) store.save_agent("Anywhere", gates=[Gate(field="country", values=["US"], mode="any")])
_board(store, "workday:mistral.wd3/Jobs", "Mistral AI") _board(store, "workday:mistral.wd3/Jobs", "Mistral AI")
store.set_all_roles("mistral-ai", True) store.set_all_countries("mistral-ai", True)
plan = reach.plan(store) plan = reach.plan(store)
[sub] = store.subscriptions(source="ats") [sub] = store.subscriptions(source="ats")
assert plan.why(sub) == (reach.EVERYWHERE, "tag") assert plan.why(sub) == (reach.EVERYWHERE, "countries")
assert plan.for_subscription(sub) is None assert plan.for_subscription(sub) is None
assert reach.line(plan, sub) == "all roles · company tag" assert reach.line(plan, sub) == "all countries (everywhere): read whole"
def test_a_boards_own_reach_wins_over_the_tag(store: Store) -> None: def test_a_boards_own_reach_wins_over_the_tag(store: Store) -> None:
_wolf(store) _wolf(store)
_board(store, "workday:mistral.wd3/Jobs", "Mistral AI", params={"countries": ["FR"]}) _board(store, "workday:mistral.wd3/Jobs", "Mistral AI", params={"countries": ["FR"]})
store.set_all_roles("mistral-ai", True) store.set_all_countries("mistral-ai", True)
[sub] = store.subscriptions(source="ats") [sub] = store.subscriptions(source="ats")
assert reach.line(reach.plan(store), sub) == "FR · set here" assert reach.line(reach.plan(store), sub) == "set here: FR · all roles"
@pytest.mark.parametrize("size, params, why", [ @pytest.mark.parametrize("size, params, why", [
@@ -248,8 +247,8 @@ def test_the_pull_reads_a_tagged_company_with_the_wide_reach(
_wolf(store) _wolf(store)
_board(store, "phenom:jobs.mistral.ai/global/en", "Mistral AI") _board(store, "phenom:jobs.mistral.ai/global/en", "Mistral AI")
_board(store, "smartrecruiters:bfl", "Black Forest Labs") _board(store, "smartrecruiters:bfl", "Black Forest Labs")
store.set_all_roles("mistral-ai", True) store.set_all_countries("mistral-ai", True)
store.set_all_roles("black-forest-labs", True) store.set_all_countries("black-forest-labs", True)
asked: dict[str, Ask] = {} asked: dict[str, Ask] = {}
def fake(module: object, slug: str, get: object, ask: Ask, **kwargs: object def fake(module: object, slug: str, get: object, ask: Ask, **kwargs: object
+6 -6
View File
@@ -70,7 +70,7 @@ def test_schema_creates_on_an_empty_file(tmp_path: Path) -> None:
"application_events", "agents", "scores", "state"} <= tables "application_events", "agents", "scores", "state"} <= tables
assert "idx_roles_cluster" in indexes assert "idx_roles_cluster" in indexes
assert store.migrations_applied == [] assert store.migrations_applied == []
assert LATEST == 23 assert LATEST == 24
# A second open finds it current and applies nothing. # A second open finds it current and applies nothing.
assert Store(path).migrations_applied == [] assert Store(path).migrations_applied == []
@@ -187,7 +187,7 @@ def test_a_v14_file_gains_the_watchlist_columns_and_leaves_companies_unrated(
migrated = Store(path) migrated = Store(path)
assert migrated.migrations_applied == [15, 16, 17, 18, 19, 20, 21, 22, 23] assert migrated.migrations_applied == [15, 16, 17, 18, 19, 20, 21, 22, 23, 24]
acme = migrated.company("acme") acme = migrated.company("acme")
assert acme is not None assert acme is not None
assert (acme.ats, acme.careers_url) == ("teamtailor", "https://a.example") assert (acme.ats, acme.careers_url) == ("teamtailor", "https://a.example")
@@ -392,7 +392,7 @@ def test_v14_makes_each_intent_the_agents_one_prompt_with_its_vector(tmp_path: P
migrated = Store(path) migrated = Store(path)
assert migrated.migrations_applied == [14, 15, 16, 17, 18, 19, 20, 21, 22, 23] assert migrated.migrations_applied == [14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24]
assert migrated.agent(backend.id).prompts == [ # type: ignore[union-attr] assert migrated.agent(backend.id).prompts == [ # type: ignore[union-attr]
Prompt(name="Backend", text="python backend")] Prompt(name="Backend", text="python backend")]
assert migrated.agent(blank.id).prompts == [] # type: ignore[union-attr] assert migrated.agent(blank.id).prompts == [] # type: ignore[union-attr]
@@ -416,7 +416,7 @@ def test_v16_gives_a_company_no_reason_and_keeps_its_row(tmp_path: Path) -> None
migrated = Store(path) migrated = Store(path)
assert migrated.migrations_applied == [16, 17, 18, 19, 20, 21, 22, 23] assert migrated.migrations_applied == [16, 17, 18, 19, 20, 21, 22, 23, 24]
acme = migrated.company("acme") acme = migrated.company("acme")
assert acme is not None assert acme is not None
assert (acme.careers_url, acme.notes, acme.hand_reason) == ( assert (acme.careers_url, acme.notes, acme.hand_reason) == (
@@ -445,7 +445,7 @@ def test_v17_folds_every_place_and_levels_the_titles_extract_has_not_read(
migrated = Store(path) migrated = Store(path)
assert migrated.migrations_applied == [17, 18, 19, 20, 21, 22, 23] assert migrated.migrations_applied == [17, 18, 19, 20, 21, 22, 23, 24]
with sqlite3.connect(path) as conn: with sqlite3.connect(path) as conn:
rows = {uid: (fold, level) for uid, fold, level in conn.execute( rows = {uid: (fold, level) for uid, fold, level in conn.execute(
"SELECT uid, place_fold, seniority FROM roles")} "SELECT uid, place_fold, seniority FROM roles")}
@@ -687,7 +687,7 @@ def test_v19_gives_every_role_its_places_and_splits_its_level(tmp_path: Path) ->
migrated = Store(path) migrated = Store(path)
assert migrated.migrations_applied == [19, 20, 21, 22, 23] assert migrated.migrations_applied == [19, 20, 21, 22, 23, 24]
assert [p.name for p in migrated.role_places(joined.uid)] == ["Copenhagen", "Malmö"] assert [p.name for p in migrated.role_places(joined.uid)] == ["Copenhagen", "Malmö"]
assert [p.name for p in migrated.role_places(detailed.uid)] == ["Aarhus"] assert [p.name for p in migrated.role_places(detailed.uid)] == ["Aarhus"]
assert migrated.role_places(stated.uid) == [] assert migrated.role_places(stated.uid) == []
+5 -3
View File
@@ -538,8 +538,9 @@ def test_companies_lists_board_and_open_role_count(
"ats": "greenhouse", "ats_slug": "", "careers_url": "", "ats": "greenhouse", "ats_slug": "", "careers_url": "",
"country": "DK", "city": "", "domain": "", "watch": None, "country": "DK", "city": "", "domain": "", "watch": None,
"careers_opened_at": "", "failing_since": "", "failing_reason": "", "careers_opened_at": "", "failing_since": "", "failing_reason": "",
"from_search": False, "hand_reason": "", "by_hand": False, "all_roles": False, "from_search": False, "hand_reason": "", "by_hand": False,
"reach_params": {}, "reach": "all roles · not read whole yet"} "all_countries": False, "all_roles": False,
"reach_params": {}, "reach": "size not known yet: read whole"}
[other] = [c for c in res.json() if c["key"] == "other"] [other] = [c for c in res.json() if c["key"] == "other"]
assert (other["board"], other["subscription_id"], other["on"], other["ats"]) == ( assert (other["board"], other["subscription_id"], other["on"], other["ats"]) == (
"", None, None, "") "", None, None, "")
@@ -1388,7 +1389,8 @@ def test_a_company_is_added_by_hand_with_a_reason_and_a_link(
"ats": "", "ats_slug": "", "careers_url": "https://revolut.example/careers", "ats": "", "ats_slug": "", "careers_url": "https://revolut.example/careers",
"country": "", "city": "", "watch": None, "careers_opened_at": "", "country": "", "city": "", "watch": None, "careers_opened_at": "",
"failing_since": "", "failing_reason": "", "from_search": False, "failing_since": "", "failing_reason": "", "from_search": False,
"hand_reason": "added", "by_hand": True, "all_roles": False, "reach_params": {}, "reach": ""} "hand_reason": "added", "by_hand": True, "all_countries": False, "all_roles": False,
"reach_params": {}, "reach": ""}
assert store.company("revolut").hand_reason == "added" # type: ignore[union-attr] assert store.company("revolut").hand_reason == "added" # type: ignore[union-attr]
other = client.post("/api/companies", json={"name": "Meta", "hand_reason": "blocked"}) other = client.post("/api/companies", json={"name": "Meta", "hand_reason": "blocked"})
+32 -15
View File
@@ -53,7 +53,7 @@ def _row(client: TestClient) -> dict[str, object]:
def test_a_company_row_says_what_its_board_is_asked_for(client: TestClient, bosch: int) -> None: def test_a_company_row_says_what_its_board_is_asked_for(client: TestClient, bosch: int) -> None:
row = _row(client) row = _row(client)
# SmartRecruiters cannot ask for remote, so only the country is asked. # SmartRecruiters cannot ask for remote, so only the country is asked.
assert (row["reach"], row["reach_params"]) == ("DK · agents", {}) assert (row["reach"], row["reach_params"]) == ("follows agents: DK · all roles", {})
def test_a_boards_own_reach_is_kept_in_its_params_and_wins( def test_a_boards_own_reach_is_kept_in_its_params_and_wins(
@@ -61,14 +61,14 @@ def test_a_boards_own_reach_is_kept_in_its_params_and_wins(
res = client.post(f"/api/sources/{bosch}/reach", res = client.post(f"/api/sources/{bosch}/reach",
json={"mode": "own", "countries": ["de", "se"], "remote": True}) json={"mode": "own", "countries": ["de", "se"], "remote": True})
assert res.status_code == 200, res.text assert res.status_code == 200, res.text
assert res.json()["reach"] == "DE, SE · set here" assert res.json()["reach"] == "set here: DE, SE · all roles"
sub = store.subscription(bosch) sub = store.subscription(bosch)
assert sub is not None assert sub is not None
assert sub.params == {"note": "kept", "countries": ["DE", "SE"], "remote": True} assert sub.params == {"note": "kept", "countries": ["DE", "SE"], "remote": True}
assert _row(client)["reach_params"] == {"countries": ["DE", "SE"], "remote": True} assert _row(client)["reach_params"] == {"countries": ["DE", "SE"], "remote": True}
res = client.post(f"/api/sources/{bosch}/reach", json={"mode": "agents"}) res = client.post(f"/api/sources/{bosch}/reach", json={"mode": "agents"})
assert res.json()["reach"] == "DK · agents" assert res.json()["reach"] == "follows agents: DK · all roles"
assert store.subscription(bosch).params == {"note": "kept"} # type: ignore[union-attr] assert store.subscription(bosch).params == {"note": "kept"} # type: ignore[union-attr]
@@ -97,30 +97,47 @@ def test_a_search_or_an_unknown_board_has_no_reach(client: TestClient, store: St
_ANY_STATE = ("queued", "running", "done", "failed", "cancelled") _ANY_STATE = ("queued", "running", "done", "failed", "cancelled")
def test_the_all_roles_tag_reads_the_wide_reach_and_on_queues_a_pull( def test_all_countries_reads_the_wide_reach_and_on_queues_a_pull(
client: TestClient, app: FastAPI, store: Store, bosch: int) -> None: client: TestClient, app: FastAPI, store: Store, bosch: int) -> None:
queue = app.state.queue queue = app.state.queue
res = client.post("/api/companies/bosch/all-roles", json={"on": True}) res = client.post("/api/companies/bosch/all-countries", json={"on": True})
assert res.status_code == 200, res.text assert res.status_code == 200, res.text
assert res.json()["all_roles"] == 1 assert (res.json()["all_countries"], res.json()["all_roles"]) == (1, 0)
# On reopens roles, so it queues a pull as switching a board on does. # On reopens roles, so it queues a pull as switching a board on does.
assert queue.jobs.last("pull", states=_ANY_STATE) is not None assert queue.jobs.last("pull", states=_ANY_STATE) is not None
row = _row(client) row = _row(client)
# Wolf requires no place, so the wide reach is everywhere. # Wolf requires no place, so the wide reach is everywhere.
assert (row["all_roles"], row["reach"]) == (True, "all roles · company tag") assert (row["all_countries"], row["all_roles"], row["reach"]) == (
# A board's own countries win over the tag. True, False, "all countries (everywhere): read whole")
# A board's own countries win over it.
client.post(f"/api/sources/{bosch}/reach", json={"mode": "own", "countries": ["DE"]}) client.post(f"/api/sources/{bosch}/reach", json={"mode": "own", "countries": ["DE"]})
assert _row(client)["reach"] == "DE · set here" assert _row(client)["reach"] == "set here: DE · all roles"
def test_turning_the_tag_off_queues_nothing( def test_all_roles_is_its_own_switch_and_on_queues_a_pull(
client: TestClient, app: FastAPI, store: Store, bosch: int) -> None: client: TestClient, app: FastAPI, store: Store, bosch: int) -> None:
store.set_all_roles("bosch", True) res = client.post("/api/companies/bosch/all-roles", json={"on": True})
res = client.post("/api/companies/bosch/all-roles", json={"on": False}) assert res.status_code == 200, res.text
assert res.status_code == 200 and res.json()["all_roles"] == 0 assert (res.json()["all_countries"], res.json()["all_roles"]) == (0, 1)
assert app.state.queue.jobs.last("pull", states=_ANY_STATE) is not None
row = _row(client)
# The narrow reach takes no terms, so on its own it changes nothing today.
assert (row["all_countries"], row["all_roles"], row["reach"]) == (
False, True, "follows agents: DK · all roles")
@pytest.mark.parametrize("path, setter", [("all-countries", "set_all_countries"),
("all-roles", "set_all_roles")])
def test_turning_an_override_off_queues_nothing(
client: TestClient, app: FastAPI, store: Store, bosch: int, path: str,
setter: str) -> None:
getattr(store, setter)("bosch", True)
res = client.post(f"/api/companies/bosch/{path}", json={"on": False})
assert res.status_code == 200
assert (res.json()["all_countries"], res.json()["all_roles"]) == (0, 0)
assert app.state.queue.jobs.last("pull", states=_ANY_STATE) is None assert app.state.queue.jobs.last("pull", states=_ANY_STATE) is None
assert _row(client)["reach"] == "DK · agents" assert _row(client)["reach"] == "follows agents: DK · all roles"
assert client.post("/api/companies/nobody/all-roles", json={"on": True}).status_code == 404 assert client.post(f"/api/companies/nobody/{path}", json={"on": True}).status_code == 404
def test_the_readers_are_listed_with_what_they_declare(client: TestClient) -> None: def test_the_readers_are_listed_with_what_they_declare(client: TestClient) -> None: