Drop pruning boards by region: a fix for the US seed, and a wrong one
`app companies prune`, `/api/prune` and the Sources disclosure switched off every board with no open role in a region. It existed because the first seed was 186 US boards; the European seed makes it moot, and it was wrong on its own terms: a board with no European role today (a company between openings, or a fetch that timed out) was switched off for good. A company that hires nowhere near should not be in the list, and a ranking's country gate handles the rest. `EUROPE` in models had no other reader and goes with it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
3aee48997c
commit
e602eb3388
12 files changed
+11
-233
No files matched your search
@@ -9,7 +9,7 @@ task in `pipeline.TASKS` and the default schedule in `pipeline.SCHEDULES`.
|
||||
cascade.py pull → embed → score → extract → score → summarise, what `TASKS`
|
||||
offers, the `pull` job a schedule fires, and that schedule
|
||||
sources.py what a pull reads, changed by hand: boards and searches
|
||||
added, switched on or off, pruned by region
|
||||
added, switched on or off
|
||||
discover.py which ATS a company's careers site runs on, from its pages
|
||||
"""
|
||||
|
||||
|
||||
+3
-38
@@ -1,7 +1,6 @@
|
||||
"""What a pull reads, changed by hand: add a board or a search, switch one on
|
||||
or off, prune boards by region. `app companies`, `app searches` and the
|
||||
settings pages all call these, so the command line and the pages cannot disagree
|
||||
about what "off" means.
|
||||
or off. `app companies`, `app searches` and the settings pages all call these,
|
||||
so the command line and the pages cannot disagree about what "off" means.
|
||||
|
||||
A board or search switched off has its open roles closed at once. Nothing
|
||||
else would ever close them, since only a pull that reads a target closes its
|
||||
@@ -14,7 +13,7 @@ import os
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from app.config import Config
|
||||
from app.models import EUROPE, Subscription, company_key
|
||||
from app.models import Subscription, company_key
|
||||
from app.storage import Store
|
||||
from app.jobs.pull import search_target
|
||||
|
||||
@@ -25,40 +24,6 @@ if TYPE_CHECKING:
|
||||
PROVIDERS = ("jobindex", "adzuna")
|
||||
|
||||
|
||||
def region_codes(region: str) -> set[str]:
|
||||
"""`europe`, or comma-separated ISO codes such as `DK,SE,EU`, as a set of
|
||||
codes. `europe` may be one entry among codes: `europe,US`."""
|
||||
codes = {c.strip().upper() for c in (region or "").split(",") if c.strip()}
|
||||
if "EUROPE" in codes:
|
||||
codes = (codes - {"EUROPE"}) | set(EUROPE)
|
||||
if not codes:
|
||||
raise ValueError("name a region: europe, or country codes such as DK,SE,EU")
|
||||
return codes
|
||||
|
||||
|
||||
def prune_plan(store: Store, region: str) -> tuple[int, list[tuple[Subscription, int]]]:
|
||||
"""Every fetched board none of whose open roles is in the region, with its
|
||||
open role count, largest first; and how many roles had `country` filled
|
||||
from `place` on the way. A board whose roles all name no country is kept:
|
||||
it says nothing about where that company hires."""
|
||||
wanted = region_codes(region)
|
||||
filled = store.backfill_country()
|
||||
countries = store.open_countries_by_subscription()
|
||||
plan = [(sub, sum(countries[sub.id].values()))
|
||||
for sub in store.subscriptions(source="ats", fetchable=True)
|
||||
if set(countries.get(sub.id, {})) - {""}
|
||||
and not wanted.intersection(countries[sub.id])]
|
||||
return filled, sorted(plan, key=lambda p: -p[1])
|
||||
|
||||
|
||||
def prune(store: Store, region: str) -> dict[str, Any]:
|
||||
"""Switch off every board `prune_plan` names."""
|
||||
filled, plan = prune_plan(store, region)
|
||||
closed = sum(set_fetch(store, sub.id, False)[1] for sub, _ in plan)
|
||||
return {"countries_filled": filled, "pruned": len(plan), "roles_closed": closed,
|
||||
"boards_left": len(store.subscriptions(source="ats", fetchable=True))}
|
||||
|
||||
|
||||
def set_fetch(store: Store, sub_id: int, fetch: bool) -> tuple[Subscription, int]:
|
||||
"""Switch one board or search on or off. Off closes its open roles and
|
||||
returns how many; see the module docstring."""
|
||||
|
||||
Reference in new issue
Block a user