Every way a company gets a board is set_board, and an unreadable host keeps its slug

`app.jobs.sources.set_board` attaches a board to an existing company,
replaces the one it has (closing the old board's open roles) or creates
the company, with row and subscription changed in one transaction by
`Store.set_company_board`. A board that lists zero roles is kept. `add`,
`discover --add`, the seed importer (moved out of app/cli.py) and the new
`POST|DELETE /api/companies/{key}/board` all go through it; a seed re-run
attaches a board to a row that never had one. A company's subscription is
rewritten in place, and detaching hides it.

`companies.ats_slug` (migration 9) keeps the slug of a host the app
cannot read yet, and discovery returns such a host before probing slugs.
The Companies rows get Set board / Change, the host select is rendered
from `plugins.ats.HOSTS`, and a row with nothing found reads "board not
found" and is in the "Not readable yet" filter.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Opus 5.5 committed 2026-09-29 14:04:52 +02:00
1 parent 0558ef19ed
commit 3b209d628a
18 files changed
+1038 -178

No files matched your search

+5 -3
View File
@@ -8,7 +8,7 @@ Built on the [boilerplate](../boilerplate): its job queue and scheduler (`pipeli
```bash
uv sync --all-extras
uv run app companies seed # 362 European employers from seed/companies.json, 175 with a readable board
uv run app companies seed # 362 European employers from seed/companies.json, 175 with a readable board; a re-run adds new rows and boards
uv run app job pull # read every board and search, embed and score them, extract the best, rescore, summarise
uv run app serve # http://127.0.0.1:8790
```
@@ -24,9 +24,11 @@ uv run app searches add jobindex "Copenhagen golang" --q "golang go-udvikler" --
uv run app searches add adzuna "Berlin backend" --q backend --country de --where Berlin
uv run app searches list
uv run app companies discover "Pleo" pleo.io --add # find its board on its own site, then follow it
uv run app companies set-board lunar greenhouse:lunar # attach a board to a company, or replace the one it has
uv run app companies detach lunar # stop reading its board; the company stays
```
The seed is a hand-picked list of European tech employers, not a list of boards: a company whose careers site runs on something the app cannot read yet (Workday, SuccessFactors, Teamtailor, its own pages) is still a row, shown under Companies as "not readable yet" with a link to its careers page, so the gap is visible and one parser at a time closes it. `companies discover` reads a company's site for a link into one of the boards it knows, probes the hosts with a cheap per-slug endpoint when the pages link nothing, and with `--add` follows what it found the same way the seed does. A probe hit is a guess by name: check the roles it lists before trusting it for a common name.
The seed is a hand-picked list of European tech employers, not a list of boards: a company whose careers site runs on something the app cannot read yet (Workday, SuccessFactors, Teamtailor, its own pages) is still a row, with that host and its slug there where one was found, shown under Companies as "not readable yet" with a link to its careers page, so the gap is visible and one parser at a time closes it. A row where nothing was found reads "board not found"; the "Not readable yet" filter shows both. Each row has a Set board control (Change once it has one) to attach, replace or detach its board by hand. Replacing or detaching closes the old board's open roles; a board is read once before it is kept, and one that lists no roles yet is kept too. `companies discover` reads a company's site for a link into one of the boards it knows, probes the hosts with a cheap per-slug endpoint only when no page names any host, and with `--add` sets the board of a new or existing company, or records the host it found when nothing readable was. A probe hit is a guess by name: check the roles it lists before trusting it for a common name.
A Jobindex query is its own syntax: words are OR, `"a b"` is a phrase, `+word` is required; `--area 1` is Region Hovedstaden. An Adzuna search needs a free key from developer.adzuna.com in `ADZUNA_APP_ID` and `ADZUNA_APP_KEY`. `searches add` runs the search once and saves it only if it finds something. Switching a board or search off closes its open roles; the next pull after switching it back on reopens the ones it still lists. The `embeddings` extra is what `fit` needs, `agent` is for `extract` on a model, `passkeys` for the Account tab's passkeys.
@@ -44,7 +46,7 @@ The cog opens `/settings`: a rail of pages on the left, the page beside it, stic
| | |
|---|---|
| **Search agents** | a card per agent, newest first: name, an On/Off switch, what it looks for, and its weights. Edit turns the card into the form in place (name, intent, weights, require/any/prefer gates behind an Advanced disclosure); New agent puts a blank card on top. Saving or switching on rescores; deleting one takes its votes, liked companies and scores with it |
| **Sources** | one page showing one section at a time, picked in the rail or by the URL fragment: job boards (queries on Jobindex or Adzuna: try, save, edit, remove), company boards (every company whose board is read, its open roles and its affinity for the picked agent, following a company) and runs (Run the pull now, what is running, the schedule, and the Problems log) |
| **Sources** | one page showing one section at a time, picked in the rail or by the URL fragment: job boards (queries on Jobindex or Adzuna: try, save, edit, remove), company boards (every company, its board or the host it cannot be read from, its open roles and its affinity for the picked agent, following a company, setting or changing a company's board) and runs (Run the pull now, what is running, the schedule, and the Problems log) |
| **Account** | the password, sessions and passkeys. Open on loopback, locked on anything else |
Likes, hidden companies and votes belong to a search agent, not to the account: two agents rank the same roles differently, and one may hide a company the other favours. An agent also learns from votes: its `taste` weight scores a role by how close it is to the roles the agent liked minus the ones it passed.
+65 -54
View File
@@ -3,7 +3,7 @@
app serve [--host H] [--port P] [--reload]
app job pull|extract|embed|score [--payload JSON]
app worker [--workers http=8,cpu=1] [--until-idle]
app companies seed|add|discover|check
app companies seed|add|set-board|detach|discover|check
app searches add|list|remove
app db init
@@ -67,17 +67,26 @@ def build_parser() -> argparse.ArgumentParser:
companies = sub.add_parser("companies", help="the target company list")
companies_sub = companies.add_subparsers(dest="action", required=True)
seed = companies_sub.add_parser(
"seed", help="import seed/companies.json, subscribing to each readable board")
"seed", help="import seed/companies.json, and the boards it names for rows without one")
seed.add_argument("--path", default="seed/companies.json")
add = companies_sub.add_parser("add", help="add one company from its board")
add = companies_sub.add_parser(
"add", help="follow a company by its board, attaching or replacing an existing one's")
add.add_argument("board", help="host:slug, e.g. greenhouse:acme")
add.add_argument("name")
set_board = companies_sub.add_parser(
"set-board", help="attach a board to a company, or replace the one it has")
set_board.add_argument("key", help="the company key, e.g. trade-republic")
set_board.add_argument("board", help="host:slug, e.g. greenhouse:acme")
detach = companies_sub.add_parser(
"detach", help="stop reading a company's board and close its roles; the row stays")
detach.add_argument("key")
discover = companies_sub.add_parser(
"discover", help="find a company's board from its website; --add follows it")
discover.add_argument("name")
discover.add_argument("domain", help="its website, e.g. pleo.io")
discover.add_argument("--add", action="store_true",
help="add the company, with its board when one is readable")
help="add or update the company: its board when one is readable, "
"else the host it runs on")
companies_sub.add_parser("check", help="report boards that do not answer")
searches = sub.add_parser("searches", help="saved searches on cross-company job boards")
@@ -230,69 +239,41 @@ def run_companies(args: argparse.Namespace) -> int:
config = config_mod.load(args.config)
store = Store(config.database)
if args.action == "seed":
print(json.dumps(seed_companies(store, args.path)))
return 0
return seed_companies(store, args.path)
if args.action == "add":
return add_company(store, args.board, args.name)
if args.action == "set-board":
return set_board(store, args.key, args.board)
if args.action == "detach":
return detach_board(store, args.key)
if args.action == "discover":
return discover_company(store, args.name, args.domain, add=args.add)
return check_companies(store)
def seed_companies(store: Store, path: str) -> dict[str, int]:
"""Import `seed/companies.json`: one `companies` row per entry, and one
`ats` subscription for each entry with a readable `board`. An entry
without one is the point of the list as much as one with: a company worth
following whose careers site the app cannot read yet keeps its `ats`
host and `careers_url`, and the Companies section says so. A key that
already exists is skipped, so re-running only ever adds what is missing;
it never edits a row."""
from app.models import company_key
from plugins.ats import careers_url, is_known_target
def seed_companies(store: Store, path: str) -> int:
"""`app companies seed`: `app.jobs.sources.seed_companies`, its counts as
JSON and each entry that did not land whole on stderr."""
from app import jobs
data = json.loads(Path(path).read_text(encoding="utf-8"))
added = readable = skipped = 0
for row in data.get("companies", []):
name = row["name"]
key = row.get("key") or company_key(name)
if store.company(key) is not None:
skipped += 1
continue
board = row.get("board") or ""
if board and not is_known_target(board):
print(f"companies seed: {key}: {board!r} is not a known board; kept without it",
file=sys.stderr)
board = ""
host = board.partition(":")[0]
store.add_company(
name, key=key, domain=row.get("domain", ""),
careers_url=row.get("careers_url") or (careers_url(board) if board else ""),
ats=row.get("ats") or host, country=row.get("country", ""),
city=row.get("city", ""))
added += 1
if not board:
continue
try:
store.add_subscription("ats", board, label=name, company=name, board=host)
readable += 1
except ValueError as exc:
# The company row landed; the subscription is the only thing
# missing, and worth saying rather than losing silently.
print(f"companies seed: {key}: {exc}", file=sys.stderr)
return {"added": added, "readable": readable, "skipped": skipped,
"total": len(data.get("companies", []))}
result = jobs.sources.seed_companies(store, path)
for problem in result.pop("problems"):
print(f"companies seed: {problem}", file=sys.stderr)
print(json.dumps(result))
return 0
def discover_company(store: Store, name: str, domain: str, *, add: bool) -> int:
"""`app companies discover "Pleo" pleo.io [--add]`: what the company's
site says its board is, as JSON. With `--add`, the company is added the
way the seed adds it: with its board when one is readable, as a row the
app cannot read yet otherwise."""
site says its board is, as JSON. With `--add`, the company is added or
updated: its board attached or replaced when one is readable, else the
host and slug it runs on recorded; see `add_discovered`."""
from app import jobs
found = jobs.discover.discover(name, domain)
out: dict[str, Any] = {"name": name, "domain": domain, "board": found.board,
"ats": found.ats, "careers_url": found.careers_url,
"ats": found.ats, "ats_slug": found.ats_slug,
"careers_url": found.careers_url,
"evidence": found.evidence}
if add:
try:
@@ -306,8 +287,8 @@ def discover_company(store: Store, name: str, domain: str, *, add: bool) -> int:
def add_company(store: Store, board: str, name: str) -> int:
"""`app companies add greenhouse:acme "Acme"`: fetch the board first
and refuse if it does not answer, is not a known host, or the company
already exists. Port of seedgen's `-add`."""
and refuse if it does not answer or is not a known host. A company that
exists gets the board attached or replaced. Port of seedgen's `-add`."""
from app import jobs
from app.models import company_key
@@ -320,12 +301,42 @@ def add_company(store: Store, board: str, name: str) -> int:
return 0
def set_board(store: Store, key: str, board: str) -> int:
"""`app companies set-board trade-republic greenhouse:traderepublic`:
attach the board to an existing company or replace the one it has."""
from app import jobs
try:
sub, found, closed = jobs.sources.set_board(store, key, board)
except ValueError as exc:
print(f"companies set-board: {exc}", file=sys.stderr)
return 1
print(json.dumps({"key": key, "board": sub.target, "roles": found, "closed": closed}))
return 0
def detach_board(store: Store, key: str) -> int:
"""`app companies detach acme`: stop reading its board, close its roles."""
from app import jobs
try:
closed = jobs.sources.detach_board(store, key)
except KeyError:
print(f"companies detach: no company {key!r}", file=sys.stderr)
return 1
except ValueError as exc:
print(f"companies detach: {exc}", file=sys.stderr)
return 1
print(json.dumps({"key": key, "closed": closed}))
return 0
def check_companies(store: Store) -> int:
"""`app companies check`: every `ats` board, fetched once. Prints a
line per board that did not answer and exits 1 if any did not."""
from plugins.ats import fetch_board
subs = store.subscriptions(source="ats")
subs = [s for s in store.subscriptions(source="ats") if s.show]
dead = []
for sub in subs:
try:
+30 -14
View File
@@ -6,16 +6,20 @@ website, not from a board. This module gets from the one to the other: it
fetches the site's careers page, or the pages the home page links to as
careers, and looks for a link into one of the hosts in `HOSTS`. A readable
host (one the `ats` plugin parses) becomes a `board` target the pull can
read; any other host is recorded as `ats`, so the company is kept as one the
app cannot read yet rather than dropped.
read; any other host is recorded as `ats`, with its slug where the URL
carries one, so the company is kept as one the app cannot read yet rather
than dropped.
When no page names a host, the three hosts with a cheap per-slug endpoint
(Greenhouse, Lever, Ashby) are probed with the company's own slug, because a
site that draws its jobs from an iframe or a script leaves no link to find.
Only when no page names a host at all are the hosts with a cheap per-slug
endpoint (`_PROBES`) probed with the company's own slug, because a site that
draws its jobs from an iframe or a script leaves no link to find. A page that
names an unreadable host is the answer: a probe on a common name can hit
another tenant's board.
found = discover("Pleo", "pleo.io")
found.board # "greenhouse:pleo", or "" when nothing readable
found.ats # "greenhouse", "workday", ... or "" when unknown
found.ats_slug # the name that host knows the company by, or ""
found.careers_url # the page that named the host, or the careers page
found.evidence # the URL that named it
@@ -30,7 +34,7 @@ from __future__ import annotations
import logging
import re
from collections.abc import Callable, Iterable
from dataclasses import dataclass
from dataclasses import dataclass, replace
from urllib.parse import urljoin
import httpx
@@ -112,9 +116,10 @@ _ANCHOR = re.compile(r"<a\b[^>]*href\s*=\s*[\"']([^\"'#][^\"']*)[\"'][^>]*>(.*?)
_TAGS = re.compile(r"<[^>]+>")
#: Words a pattern's slug group can catch that name no board: the hosts'
#: own path segments, and the analytics subdomain Recruitee sites embed.
#: own path segments, the analytics subdomain Recruitee sites embed, and
#: the CDN Teamtailor sites load from (cdn.teamtailor.com).
_NOT_SLUGS = frozenset({"embed", "boards", "v1", "api", "www", "jobs", "careers", "apply",
"careers-analytics"})
"careers-analytics", "cdn"})
Found = tuple[str, str, str] # (host, slug, the URL that named it)
@@ -125,6 +130,7 @@ class Discovery:
board: str = ""
ats: str = ""
ats_slug: str = ""
careers_url: str = ""
evidence: str = ""
@@ -301,14 +307,22 @@ def _discover(name: str, site: str, client: httpx.Client, fetch: Fetch,
careers_page = ""
def first_readable(found: list[Found], page: str) -> Discovery | None:
"""The first readable board on a page. Otherwise the first host named
is remembered, with the first slug any page gives that host: a page
may name teamtailor by its CDN before it links acme.teamtailor.com."""
nonlocal unreadable
for host, slug, url in found:
if host in READABLE and slug:
return Discovery(board=f"{host}:{slug}", ats=host, careers_url=page,
evidence=url)
return Discovery(board=f"{host}:{slug}", ats=host, ats_slug=slug,
careers_url=page, evidence=url)
if unreadable is None and found:
host, slug, url = found[0]
unreadable = Discovery(ats=host, careers_url=page, evidence=url)
unreadable = Discovery(ats=host, ats_slug=slug, careers_url=page, evidence=url)
if unreadable is not None and not unreadable.ats_slug:
for host, slug, url in found:
if host == unreadable.ats and slug:
unreadable = replace(unreadable, ats_slug=slug, evidence=url)
break
return None
if home and (hit := first_readable(boards_in(home), home_url)):
@@ -333,11 +347,13 @@ def _discover(name: str, site: str, client: httpx.Client, fetch: Fetch,
if hit := first_readable(boards_in(text), final):
return hit
queue[:0] = [link for link in careers_links(text, final)[:2] if link not in seen]
# A page named a host the app cannot read: that is the answer, and a
# probe could only add another tenant's board of the same name.
if unreadable is not None:
return unreadable
for slug in slug_guesses(name, site):
for host in _PROBES:
if probe_slug(client, host, slug):
return Discovery(board=f"{host}:{slug}", ats=host,
return Discovery(board=f"{host}:{slug}", ats=host, ats_slug=slug,
careers_url=careers_page, evidence=_PROBES[host].format(slug=slug))
if unreadable is not None:
return unreadable
return Discovery(careers_url=careers_page)
+153 -33
View File
@@ -1,6 +1,8 @@
"""What a pull reads, changed by hand: add a board or a search, switch one on
or off. `app companies`, `app searches` and the settings pages all call these,
so the command line and the pages cannot disagree about what "off" means.
or off, attach, replace or detach a company's board, import the seed.
`app companies`, `app searches` and the settings pages all call these, so the
command line and the pages cannot disagree about what "off" means. Every way
a company gets a board is `set_board`.
A board or search switched off has its open roles closed at once. Nothing
else would ever close them, since only a pull that reads a target closes its
@@ -9,7 +11,9 @@ roles; switched back on, the next pull reopens every role it still lists.
from __future__ import annotations
import json
import os
from pathlib import Path
from typing import TYPE_CHECKING, Any
from app.config import Config
@@ -32,55 +36,171 @@ def set_fetch(store: Store, sub_id: int, fetch: bool) -> tuple[Subscription, int
return sub, closed
def add_board(store: Store, board: str, name: str) -> tuple[Subscription, int]:
"""Follow one company's ATS board: fetch it first, and refuse if the host
is unknown, the company exists, the board does not answer or lists
nothing. Returns the subscription and how many roles the board lists."""
def set_board(store: Store, key: str, board: str, *, name: str = "",
check: bool = True, **fields: str) -> tuple[Subscription, int | None, int]:
"""Read company `key` from `board` (`host:slug`): attach it to a company
that has none, replace the one it has, or create the company from `name`
when there is no row. The board is fetched first and refused if its host
is unknown or it does not answer; a board listing zero roles is kept, since
a company between openings is still one to follow. `check=False` skips the
fetch, for the seed, whose boards were read once when it was written.
The row's `ats` and `ats_slug` become the board's host and slug, and its
`careers_url` the board's page where it was empty or the old board's.
`fields` (domain, country, city, ...) fill only columns that are empty, so
nothing a person or the seed wrote is overwritten. Row and subscription
change in one transaction (`Store.set_company_board`), and replacing a
board closes the old one's open roles. Returns the subscription, how many
roles the board lists (None unchecked) and how many roles closed."""
from plugins.ats import careers_url, fetch_board, is_known_target
board, name = board.strip(), name.strip()
if not name:
raise ValueError("a company needs a name")
key = key.strip() or company_key(name)
if not is_known_target(board):
raise ValueError(f"{board!r} is not host:slug for a known ATS host, "
"e.g. greenhouse:acme or ashby:acme")
key = company_key(name)
if store.company(key) is not None:
raise ValueError(f"{name} is already followed")
try:
rows = fetch_board(board)
except Exception as exc:
raise ValueError(f"{board} did not answer: {exc}") from exc
if not rows:
raise ValueError(f"{board} answered but lists no roles; nothing added")
host = board.partition(":")[0]
store.add_company(name, key=key, careers_url=careers_url(board))
sub = store.add_subscription("ats", board, label=name, company=name, board=host)
return sub, len(rows)
company = store.company(key)
if company is None:
if not name:
raise ValueError(f"no company {key!r}; a new company needs a name")
if company_key(name) != key:
raise ValueError(f"{name!r} is not the company {key!r}")
rows: list[Any] | None = None
if check:
try:
rows = fetch_board(board)
except Exception as exc:
raise ValueError(f"{board} did not answer: {exc}") from exc
host, _, slug = board.partition(":")
writes: dict[str, str] = {k: v for k, v in fields.items() if v}
if company is not None:
writes = {k: v for k, v in writes.items() if not getattr(company, k)}
old = store.company_subscription(key)
if not company.careers_url or (old is not None
and company.careers_url == careers_url(old.target)):
writes["careers_url"] = careers_url(board)
else:
writes.setdefault("careers_url", careers_url(board))
writes.update(ats=host, ats_slug=slug)
_company, sub, closed = store.set_company_board(key, board, host=host, name=name,
fields=writes)
assert sub is not None
return sub, None if rows is None else len(rows), closed
def detach_board(store: Store, key: str) -> int:
"""Stop reading company `key`'s board: its open roles close and the row
stays, with its `ats` and `ats_slug`, as one the app does not read.
Returns how many roles closed. Raises KeyError for an unknown company and
ValueError for one with no board."""
if store.company(key) is None:
raise KeyError(key)
sub = store.company_subscription(key)
if sub is None or not sub.show:
raise ValueError(f"{key} has no board to detach")
_company, _sub, closed = store.set_company_board(key, "")
return closed
def add_board(store: Store, board: str, name: str) -> tuple[Subscription, int]:
"""Follow one company by its ATS board: `set_board` keyed by `name`, so
a company that exists gets the board attached or replaced. Returns the
subscription and how many roles the board lists."""
name = name.strip()
if not name:
raise ValueError("a company needs a name")
sub, roles, _closed = set_board(store, company_key(name), board, name=name)
return sub, roles or 0
def add_discovered(store: Store, name: str, domain: str,
found: Discovery) -> tuple[str, int]:
"""Follow a company from what `app.jobs.discover` found: through
`add_board` when the board is readable, else as a row with its `ats` host
and careers page, so the company is kept as one the app cannot read yet.
Returns the company key and how many roles its board lists (0 with no
board). Refuses a company that exists, as `add_board` does."""
"""Follow a company from what `app.jobs.discover` found. A readable
board goes through `set_board`, which attaches it, replaces the one the
company has, or creates the company. Otherwise the row records `ats`,
`ats_slug` and the careers page, as one the app cannot read yet; a
company that already has a board keeps it and this raises, since
discovery finding nothing readable is weaker evidence than a board that
answers. Returns the company key and how many roles its board lists (0
with no board)."""
name = name.strip()
if not name:
raise ValueError("a company needs a name")
key = company_key(name)
if found.board:
_sub, roles = add_board(store, found.board, name)
store.update_company(key, domain=domain, ats=found.ats)
return key, roles
if store.company(key) is not None:
raise ValueError(f"{name} is already followed")
store.add_company(name, key=key, domain=domain, ats=found.ats,
careers_url=found.careers_url)
_sub, roles, _closed = set_board(store, key, found.board, name=name, domain=domain)
return key, roles or 0
company = store.company(key)
if company is None:
store.add_company(name, key=key, domain=domain, ats=found.ats,
ats_slug=found.ats_slug, careers_url=found.careers_url)
return key, 0
sub = store.company_subscription(key)
if sub is not None and sub.show:
raise ValueError(f"{name} is read from {sub.target} and discovery found no "
"readable board; nothing changed")
store.update_company(key, ats=found.ats, ats_slug=found.ats_slug,
careers_url=found.careers_url or company.careers_url,
**({"domain": domain} if not company.domain else {}))
return key, 0
def seed_companies(store: Store, path: str | Path) -> dict[str, Any]:
"""Import `seed/companies.json`: one `companies` row per entry, and its
`board` through `set_board` where the entry names one. An entry without
a board is the point of the list as much as one with: a company whose
careers site the app cannot read yet keeps its `ats` host, `ats_slug`
and `careers_url`, and the Companies section says so.
Re-running adds what is missing: a new entry becomes a row, and an
existing row that never had a board gets the one the seed now names, so a
seed update that gains boards reaches a database made before it. A row's
board, once it has one (detached counts), and its affinity, notes and
status are never touched. The seed's boards are not fetched again: they
were read once when the seed was written, and `app companies check`
finds the dead ones. Returns counts, and `problems`, one line per entry
that could not land whole."""
from plugins.ats import is_known_target
data = json.loads(Path(path).read_text(encoding="utf-8"))
entries = data.get("companies", [])
added = boards = skipped = 0
problems: list[str] = []
for row in entries:
name = row["name"].strip()
key = row.get("key") or company_key(name)
board = (row.get("board") or "").strip()
if board and not is_known_target(board):
problems.append(f"{key}: {board!r} is not a known board; kept without it")
board = ""
facts = {f: str(row.get(f) or "") for f in ("domain", "country", "city",
"careers_url", "ats", "ats_slug")}
exists = store.company(key) is not None
if exists and (not board or store.company_subscription(key) is not None):
skipped += 1
continue
if board:
try:
set_board(store, key, board, name=name, check=False,
**{f: v for f, v in facts.items() if f not in ("ats", "ats_slug")})
boards += 1
added += not exists
continue
except ValueError as exc:
problems.append(f"{key}: {exc}")
if exists:
skipped += 1
continue
# No board, or one that clashed: the company is still a row, and a
# clashing board's host and slug are what discovery found.
if board:
facts["ats"], _, facts["ats_slug"] = board.partition(":")
store.add_company(name, key=key, **facts)
added += 1
return {"added": added, "boards": boards, "skipped": skipped, "total": len(entries),
"problems": problems}
def search_params(provider: str, q: str, *, area: int = 0, country: str = "",
where: str = "", pages: int = 0) -> dict[str, Any]:
"""The `params` a search subscription stores, with only the keys its
+3
View File
@@ -335,6 +335,9 @@ class Company(Model):
ats: str = ""
country: str = ""
city: str = ""
#: The name `ats` knows the company by, the `acme` of acme.teamtailor.com;
#: "" where discovery found none. Kept for a host no parser reads yet.
ats_slug: str = ""
created_at: str
updated_at: str
+13 -1
View File
@@ -1200,6 +1200,17 @@ table.applications td[data-label="Saved"] { white-space: nowrap; color: var(--di
gap: .4rem; font-size: .8rem; color: var(--dim);
}
.srow[data-muted="true"], .crow[data-hidden="true"] { opacity: .6; }
/* A company row's Set board / Change editor: its own line under `.sub`,
wrapping like `.sub` does, the boxes styled as `.source-form`'s. */
.crow .board-edit {
grid-column: 1 / -1; display: flex; flex-wrap: wrap; align-items: center;
gap: .4rem; padding-top: .25rem;
}
.crow .board-edit select, .crow .board-edit input {
background: var(--bg); border: 1px solid var(--line);
border-radius: .3rem; padding: .35rem .5rem;
}
.crow .board-edit input { flex: 1 1 8rem; min-width: 0; }
.linky {
padding: 0; border: 0; background: none; font-size: .8rem;
@@ -1282,7 +1293,8 @@ table.list select { padding: .25rem .4rem; }
button, .top nav a { min-height: 2.75rem; }
.top nav a { display: inline-flex; align-items: center; justify-content: center; }
.filters input[type="search"], .filters input[type="text"], .filters select,
.agent-form input, .source-form select { min-height: 2.75rem; }
.agent-form input, .source-form select,
.crow .board-edit input, .crow .board-edit select { min-height: 2.75rem; }
.affinity button { min-width: 2.75rem; }
.check input { width: 1.25rem; height: 1.25rem; }
}
+102 -7
View File
@@ -11,6 +11,12 @@
* server's default, the enabled agent with the lowest id; with no agent at
* all there is nothing to like, and the section says so instead of drawing
* the buttons.
*
* Each row's "Set board" (no board yet) or "Change" (a board) opens a small
* editor in the row: the host select, with the options the server rendered
* into the Follow form's `#b-host`, and a slug. Saving posts to
* `/api/companies/{key}/board`, which reads the board once before keeping
* it; Detach stops reading it and keeps the company.
*/
(function () {
"use strict";
@@ -21,6 +27,9 @@
/** @type {object[]} the search agents, by id */
let agents = [];
/** The key of the row whose board editor is open, or null. One at a time. */
let editing = null;
/** The id in `#c-agent`, or null while there is no agent to pick. */
function agentId() {
const value = $("#c-agent").value;
@@ -45,7 +54,7 @@
// ---------------------------------------------------------------- the list
function subLine(company) {
function subLine(company, rerender) {
const sub = document.createElement("span");
sub.className = "sub";
if (company.board) {
@@ -57,11 +66,15 @@
// Followed, but its careers site runs on nothing the app reads yet.
const badge = document.createElement("span");
badge.className = "badge";
badge.textContent = company.ats || "own site";
badge.textContent = company.ats
? (company.ats_slug ? `${company.ats}:${company.ats_slug}` : company.ats)
: "own site";
sub.appendChild(badge);
sub.append(" · not readable yet");
} else {
sub.append("from job boards");
// Every row comes from the seed, Follow or discover: none of them
// found where this company posts its roles.
sub.append("board not found");
}
if (company.careers_url && !company.board) {
const link = document.createElement("a");
@@ -72,11 +85,92 @@
sub.append(" · ");
sub.appendChild(link);
}
sub.append(` · ${count(company.open_roles, "open role")}`);
sub.append(` · ${count(company.open_roles, "open role")} `);
const edit = document.createElement("button");
edit.type = "button";
edit.className = "linky";
edit.textContent = company.subscription_id == null ? "Set board" : "Change";
edit.setAttribute("aria-expanded", String(editing === company.key));
edit.addEventListener("click", () => {
editing = editing === company.key ? null : company.key;
rerender();
});
sub.appendChild(edit);
return sub;
}
/** The board switch, or nothing for a company known only from a search:
/** The open row's board editor: host and slug, Save, Cancel, and Detach
for a row that has a board. */
function boardEditor(company, rerender) {
const form = document.createElement("form");
form.className = "board-edit";
form.autocomplete = "off";
const host = document.createElement("select");
host.setAttribute("aria-label", `${company.name}'s board host`);
for (const option of $("#b-host").options) host.appendChild(option.cloneNode(true));
const known = [...host.options].map((o) => o.value);
const current = company.board || company.ats;
if (known.includes(current)) host.value = current;
const slug = document.createElement("input");
slug.type = "text";
slug.required = true;
slug.spellcheck = false;
slug.placeholder = "slug";
slug.value = company.ats_slug || "";
slug.setAttribute("aria-label", `${company.name}'s slug on that host`);
const save = document.createElement("button");
save.type = "submit";
save.textContent = company.subscription_id == null ? "Set board" : "Change board";
const cancel = document.createElement("button");
cancel.type = "button";
cancel.className = "linky";
cancel.textContent = "Cancel";
cancel.addEventListener("click", () => { editing = null; rerender(); });
form.append(host, slug, save, cancel);
async function done() {
editing = null;
await loadCompanies(agentId());
rerender();
}
form.addEventListener("submit", async (event) => {
event.preventDefault();
save.disabled = true;
try {
const result = await api(`/api/companies/${encodeURIComponent(company.key)}/board`,
{ method: "POST", body: { board: `${host.value}:${slug.value.trim()}` } });
const closed = result.closed ? `, ${count(result.closed, "old role")} closed` : "";
window.toast.info(`${company.name}: reading ${result.source.target}, `
+ `${count(result.found, "role")} listed${closed}.`);
await done();
} catch (err) {
save.disabled = false;
}
});
if (company.subscription_id != null) {
const detach = document.createElement("button");
detach.type = "button";
detach.className = "linky";
detach.textContent = "Detach";
detach.addEventListener("click", async () => {
detach.disabled = true;
try {
const result = await api(`/api/companies/${encodeURIComponent(company.key)}/board`,
{ method: "DELETE" });
window.toast.info(`${company.name}: board detached, `
+ `${count(result.closed, "open role")} closed.`);
await done();
} catch (err) {
detach.disabled = false;
}
});
form.append(detach);
}
return form;
}
/** The board switch, or nothing for a company with no board:
there is no subscription to switch. */
function onSwitch(company, rerender) {
if (company.subscription_id == null) return null;
@@ -109,7 +203,7 @@
case "hidden": return company.affinity === -1;
case "on": return company.on === true;
case "off": return company.on === false;
case "unreadable": return company.subscription_id == null && Boolean(company.ats || company.careers_url);
case "unreadable": return company.subscription_id == null;
default: return true;
}
}
@@ -140,7 +234,8 @@
if (agent != null) {
row.appendChild(affinityControl(company.key, company.affinity, async () => render(), agent));
}
row.appendChild(subLine(company));
row.appendChild(subLine(company, render));
if (editing === company.key) row.appendChild(boardEditor(company, render));
body.appendChild(row);
}
const open = rows.reduce((sum, c) => sum + c.open_roles, 0);
+8 -2
View File
@@ -94,7 +94,9 @@ CREATE TABLE IF NOT EXISTS role_tags (
-- ats is the host the careers site runs on, readable by the ats plugin or
-- not (workday, successfactors, ...): what discovery found, '' unknown. A
-- company with an ats and no board subscription is one the app cannot read
-- yet. country is an ISO code and city the head office's.
-- yet. country is an ISO code and city the head office's. ats_slug is the
-- name that host knows the company by (the acme of acme.teamtailor.com),
-- '' where its URL carries none, kept for the parser written later.
CREATE TABLE IF NOT EXISTS companies (
key TEXT PRIMARY KEY,
name TEXT NOT NULL,
@@ -106,7 +108,8 @@ CREATE TABLE IF NOT EXISTS companies (
updated_at TEXT NOT NULL,
ats TEXT NOT NULL DEFAULT '',
country TEXT NOT NULL DEFAULT '',
city TEXT NOT NULL DEFAULT ''
city TEXT NOT NULL DEFAULT '',
ats_slug TEXT NOT NULL DEFAULT ''
);
-- The targets desk's runner hands to plugins. newsdesk's shape, with its two
@@ -337,6 +340,9 @@ ALTER TABLE companies ADD COLUMN city TEXT NOT NULL DEFAULT '';
"""),
(8, "agents.limits: how deep into its ranking the model stages read is each agent's own", """
ALTER TABLE agents ADD COLUMN limits TEXT NOT NULL DEFAULT '{}';
"""),
(9, "companies.ats_slug: the slug of a host the app cannot read yet is kept", """
ALTER TABLE companies ADD COLUMN ats_slug TEXT NOT NULL DEFAULT '';
"""),
)
+103 -14
View File
@@ -69,7 +69,8 @@ _VOTED_SCORE = ("MIN(1.0, s.score * CASE v.vote WHEN -1 THEN 0.5 "
#: nothing, so `vote` and `voted_at` are NULL: one query shape either way.
_WITH_VOTES = "LEFT JOIN role_votes v ON v.uid = roles.uid AND v.agent_id = ?"
_COMPANY_FIELDS = ("name", "domain", "status", "careers_url", "notes", "ats", "country", "city")
_COMPANY_FIELDS = ("name", "domain", "status", "careers_url", "notes", "ats", "country", "city",
"ats_slug")
#: A company as the store serves it: the row and the asked-for agent's affinity.
_COMPANY_SELECT = """SELECT c.*, COALESCE(ac.affinity, 0) AS affinity FROM companies c
@@ -979,24 +980,31 @@ class Store:
def add_company(self, name: str, *, key: str = "", domain: str = "",
status: str = "researching", careers_url: str = "",
notes: str = "", ats: str = "", country: str = "",
city: str = "") -> Company:
city: str = "", ats_slug: str = "") -> Company:
"""Raises ValueError on a key already taken."""
name = name.strip()
key = key or company_key(name)
if not key:
raise ValueError("a company needs a name")
stamp = self.now()
try:
self._write(
"""INSERT INTO companies (key, name, domain, status, careers_url, notes,
ats, country, city, created_at, updated_at)
VALUES (?,?,?,?,?,?,?,?,?,?,?)""",
(key, name, domain, status, careers_url, notes, ats, country, city,
stamp, stamp))
with self.transaction() as conn:
self._insert_company(conn, key, name, {
"domain": domain, "status": status, "careers_url": careers_url,
"notes": notes, "ats": ats, "country": country, "city": city,
"ats_slug": ats_slug})
except sqlite3.IntegrityError as exc:
raise ValueError(f"company {key!r} already exists") from exc
return self._company(key)
def _insert_company(self, conn: sqlite3.Connection, key: str, name: str,
fields: dict[str, Any]) -> None:
stamp = self.now()
columns = ["key", "name", *fields, "created_at", "updated_at"]
conn.execute(
f"INSERT INTO companies ({', '.join(columns)}) "
f"VALUES ({', '.join('?' * len(columns))})",
(key, name, *fields.values(), stamp, stamp))
def company(self, key: str, agent_id: int | None = None) -> Company | None:
"""One company, with `affinity` as `agent_id` set it; 0 without an agent."""
row = self._row(f"{_COMPANY_SELECT} WHERE c.key=?", (agent_id, key))
@@ -1029,8 +1037,8 @@ class Store:
return self._company(key, agent_id)
def update_company(self, key: str, **fields: Any) -> Company:
"""Change any of name, domain, status, careers_url, notes. Affinity is
per agent: `set_affinity`."""
"""Change any of `_COMPANY_FIELDS`. Affinity is per agent:
`set_affinity`."""
unknown = set(fields) - set(_COMPANY_FIELDS)
if unknown:
raise ValueError(f"not a company field: {', '.join(sorted(unknown))}")
@@ -1072,13 +1080,94 @@ class Store:
"""`company_key -> ats host`, read off each company's own
subscription. `Subscription.company` is the display name it and the
company row were both added with, so `company_key` of one finds the
other; a company added by hand with no subscription is simply
absent."""
other; a company with no subscription, or a detached one (`show` 0,
see `set_company_board`), is simply absent."""
boards: dict[str, str] = {}
for sub in self.subscriptions(source="ats"):
boards.setdefault(company_key(sub.company), sub.board)
if sub.show:
boards.setdefault(company_key(sub.company), sub.board)
return boards
def company_subscription(self, key: str) -> Subscription | None:
"""Company `key`'s own `ats` subscription, detached or not; None when
it never had one. A company has at most one: `set_company_board`
rewrites it in place rather than adding a second."""
with closing(self._connect()) as conn:
return self._company_subscription(conn, key)
@staticmethod
def _company_subscription(conn: sqlite3.Connection, key: str) -> Subscription | None:
rows = conn.execute("SELECT * FROM subscriptions WHERE source = 'ats' "
"ORDER BY show DESC, id").fetchall()
return next((Subscription.from_row(r) for r in rows
if company_key(r["company"]) == key), None)
def set_company_board(self, key: str, target: str, *, host: str = "", name: str = "",
fields: dict[str, Any] | None = None,
) -> tuple[Company, Subscription | None, int]:
"""Point company `key` at the `ats` board `target`, or at none when
`target` is "", in one transaction, so a clash on either table leaves
nothing half written.
A missing company is created from `name` and `fields`; an existing one
has `fields` written over it. Its subscription is rewritten in place:
a new target closes the old target's open roles, as switching a board
off does, and "" detaches it (off, hidden, its roles closed) while the
row stays for the next board. Returns the company, its subscription
(None when there is none) and how many roles closed. Raises KeyError
for a missing company with no board, ValueError on a UNIQUE clash."""
fields = dict(fields or {})
unknown = set(fields) - set(_COMPANY_FIELDS)
if unknown:
raise ValueError(f"not a company field: {', '.join(sorted(unknown))}")
stamp = self.now()
closed = 0
try:
with self.transaction() as conn:
row = conn.execute("SELECT name FROM companies WHERE key=?", (key,)).fetchone()
if row is None:
name = fields.pop("name", "") or name.strip()
if not target:
raise KeyError(key)
if not name:
raise ValueError("a company needs a name")
self._insert_company(conn, key, name, fields)
else:
name = fields.get("name") or row["name"]
if fields:
sets = ", ".join(f"{f}=?" for f in fields)
conn.execute(f"UPDATE companies SET {sets}, updated_at=? WHERE key=?",
(*fields.values(), stamp, key))
sub = self._company_subscription(conn, key)
sub_id = sub.id if sub else None
if sub is not None and sub.target != target:
closed = conn.execute(
"UPDATE roles SET closed_at = ? "
"WHERE subscription_id = ? AND closed_at IS NULL",
(stamp, sub.id)).rowcount
if not target:
if sub is not None:
conn.execute("UPDATE subscriptions SET subscribed=0, fetch=0, show=0 "
"WHERE id=?", (sub.id,))
elif sub is not None:
conn.execute(
"""UPDATE subscriptions
SET target=?, board=?, label=?, company=?, subscribed=1,
fetch=1, show=1,
target_id=CASE WHEN target=? THEN target_id ELSE '' END
WHERE id=?""",
(target, host, name, name, target, sub.id))
else:
sub_id = conn.execute(
"""INSERT INTO subscriptions (source, target, label, company, board,
params, added_at)
VALUES ('ats',?,?,?,?,'{}',?)""",
(target, name, name, host, stamp)).lastrowid
except sqlite3.IntegrityError as exc:
raise ValueError(f"another board already has {target!r} or the label "
f"{name!r}") from exc
return (self._company(key), self.subscription(sub_id) if sub_id else None, closed)
# -------------------------------------------------------- subscriptions
def subscriptions(self, source: str | None = None,
+4 -7
View File
@@ -68,14 +68,11 @@
<h3>Follow a company</h3>
<div class="form-row">
<label>Host
{# The hosts plugins.ats reads (HOST_OPTIONS in app/web/pages.py).
companies.js copies these options into each row's Set board
control, so the page has one list of hosts. #}
<select id="b-host">
<option value="greenhouse">Greenhouse</option>
<option value="lever">Lever</option>
<option value="ashby">Ashby</option>
<option value="workable">Workable</option>
<option value="smartrecruiters">SmartRecruiters</option>
<option value="recruitee">Recruitee</option>
<option value="personio">Personio</option>
{% for value, label in hosts %}<option value="{{ value }}">{{ label }}</option>{% endfor %}
</select></label>
<label>Slug
<input type="text" id="b-slug" required placeholder="acme" spellcheck="false">
+48 -10
View File
@@ -260,11 +260,12 @@ def summarise_role(uid: str, store: StoreDep, config: ConfigDep, queue: QueueDep
class CompanyOut(BaseModel):
"""A company row plus what only the store can join in: the ats host it is
read from (empty for a company known only from a search), the id of that
read from (empty for one with no board), the id of that
board's subscription, whether it is on, and how many of its roles are
still open. `ats` is the host its careers site runs on whether or not the
app reads it, so a followed company with no `board` and an `ats` is one
the app cannot read yet; `careers_url` is where a person can."""
app reads it, and `ats_slug` its name there, so a followed company with
no `board` and an `ats` is one the app cannot read yet; `careers_url` is
where a person can."""
key: str
name: str
@@ -274,6 +275,7 @@ class CompanyOut(BaseModel):
affinity: int
open_roles: int
ats: str = ""
ats_slug: str = ""
careers_url: str = ""
country: str = ""
@@ -282,9 +284,12 @@ def _company_boards(store: Store) -> dict[str, tuple[int, str, bool]]:
"""`company_key -> (subscription id, ats host, on)`, read off each
company's own subscription. Same join as `Store.company_boards`, with the
id and switch state a company row needs to draw its own on/off control
rather than fetching `/api/sources` as well."""
rather than fetching `/api/sources` as well. A detached board (`show`
0) is no board."""
boards: dict[str, tuple[int, str, bool]] = {}
for sub in store.subscriptions(source="ats"):
if not sub.show:
continue
boards.setdefault(company_key(sub.company), (sub.id, sub.board,
bool(sub.subscribed and sub.fetch)))
return boards
@@ -302,10 +307,41 @@ def list_companies(store: StoreDep, agent: int | None = None) -> list[CompanyOut
out.append(CompanyOut(key=c.key, name=c.name, board=host,
subscription_id=sub_id, on=on, affinity=c.affinity,
open_roles=counts.get(c.key, 0), ats=c.ats,
careers_url=c.careers_url, country=c.country))
ats_slug=c.ats_slug, careers_url=c.careers_url,
country=c.country))
return out
class CompanyBoardBody(BaseModel):
board: str
@router.post("/companies/{key}/board")
def set_company_board(key: str, body: CompanyBoardBody, store: StoreDep) -> dict[str, Any]:
"""Attach a board to a company, or replace the one it has. The board is
fetched first and refused if it does not answer; replacing closes the
old board's open roles."""
if store.company(key) is None:
raise HTTPException(404, f"no such company: {key}")
try:
sub, found, closed = jobs.sources.set_board(store, key, body.board)
except ValueError as exc:
raise HTTPException(422, str(exc)) from exc
return {"source": _one_source(store, sub.id), "found": found, "closed": closed}
@router.delete("/companies/{key}/board")
def detach_company_board(key: str, store: StoreDep) -> dict[str, Any]:
"""Stop reading a company's board: its open roles close, the row stays."""
try:
closed = jobs.sources.detach_board(store, key)
except KeyError:
raise HTTPException(404, f"no such company: {key}") from None
except ValueError as exc:
raise HTTPException(409, str(exc)) from exc
return {"closed": closed}
class AffinityBody(BaseModel):
affinity: int
#: Whose affinity; the default agent when absent.
@@ -370,12 +406,13 @@ def _one_source(store: Store, sub_id: int) -> SourceOut:
@router.get("/sources")
def list_sources(store: StoreDep, config: ConfigDep) -> SourcesOut:
"""Every board, and every search not removed. A board switched off stays
listed, so it can be switched back on."""
"""Every board not detached, and every search not removed. A board
switched off stays listed, so it can be switched back on."""
counts = store.open_countries_by_subscription()
searches = [_source_out(s, counts.get(s.id, {}))
for s in store.subscriptions(source="search") if s.show]
boards = [_source_out(s, counts.get(s.id, {})) for s in store.subscriptions(source="ats")]
boards = [_source_out(s, counts.get(s.id, {}))
for s in store.subscriptions(source="ats") if s.show]
return SourcesOut(searches=searches, boards=boards,
adzuna_key=jobs.sources.has_key(config, "adzuna"))
@@ -454,8 +491,9 @@ class BoardBody(BaseModel):
@router.post("/boards", status_code=201)
def add_board(body: BoardBody, store: StoreDep) -> dict[str, Any]:
"""Follow a company's ATS board. It is fetched first, and refused if it
does not answer or lists nothing."""
"""Follow a company by its ATS board: `set_board` keyed by the name, so
an existing company gets it attached or replaced. The board is fetched
first and refused if it does not answer."""
try:
sub, found = jobs.sources.add_board(store, body.board, body.name)
except ValueError as exc:
+8 -5
View File
@@ -30,9 +30,10 @@ kept, so a bookmark still lands. A merged page's old path lands on its
section's anchor; the old Messages paths land on `#messages`, the log's
block at the end of Runs.
`FUNCTIONS` is `plugins.ats`'s, not `app`'s: the nine job functions are the
classifier's vocabulary, and this reaches `plugins/` the same way `app.cli`
does, since a plugin is app-agnostic and never reaches back into `app`. The
`FUNCTIONS` and `HOSTS` are `plugins.ats`'s, not `app`'s: the nine job
functions are the classifier's vocabulary and the hosts are what it parses,
and this reaches `plugins/` the same way `app.cli` does, since a plugin is
app-agnostic and never reaches back into `app`. The
Jobindex areas and Adzuna countries on the Sources page come from
`plugins.search` for the same reason.
"""
@@ -55,7 +56,7 @@ _ROOT = Path(__file__).resolve().parents[2]
if str(_ROOT) not in sys.path:
sys.path.insert(0, str(_ROOT))
from plugins.ats import FUNCTIONS # noqa: E402
from plugins.ats import FUNCTIONS, HOSTS # noqa: E402
from plugins.search import ADZUNA_COUNTRIES, JOBINDEX_AREAS # noqa: E402
TEMPLATE_DIR = Path(__file__).resolve().parent.parent / "templates"
@@ -118,6 +119,8 @@ REMOTE_OPTIONS = [(r, r.capitalize()) for r in REMOTE if r]
STATUS_OPTIONS = [(s, s.capitalize()) for s in STATUSES]
AREA_OPTIONS = list(JOBINDEX_AREAS.items())
COUNTRY_OPTIONS = [(c, c.upper()) for c in sorted(ADZUNA_COUNTRIES)]
#: The ATS hosts `plugins.ats` reads, for the Follow and Set board selects.
HOST_OPTIONS = list(HOSTS.items())
def asset(path: str) -> str:
@@ -183,7 +186,7 @@ def agents(request: Request) -> HTMLResponse:
@router.get("/settings/sources", response_class=HTMLResponse)
def sources(request: Request) -> HTMLResponse:
return page(request, "settings/sources.html", areas=AREA_OPTIONS,
countries=COUNTRY_OPTIONS)
countries=COUNTRY_OPTIONS, hosts=HOST_OPTIONS)
@router.get("/settings/account", response_class=HTMLResponse)
+11 -5
View File
@@ -42,11 +42,11 @@ What was decided for roledesk and the one reason for each. The longer reasoning
**Boards follow companies; searches find them.** `ats` only reads companies someone listed, so it cannot surface a role at a company nobody knew of, and the seed list came from a US-centred project: of 29,000 open roles, 31 were in Denmark. The `search` plugin runs saved queries on boards that span companies, as `search` subscriptions whose `params` are the query, and pulls them in the same task as the boards.
**The seed is a list of companies, not of boards.** The first seed was jobhunt-hq's 186 US boards, chosen because they could be read, and 31 of their 29,000 roles were in Denmark. The second is 362 European employers picked by hand, with `board` filled by `app.jobs.discover` from each company's own careers site and checked once by fetching it; 175 have one. The rest stay in the list with the `ats` host their site runs on, so that "cannot read yet" is a filter on the Companies section rather than a company that was never written down, and the next parser to write is whichever host that filter shows most. Teamtailor is first, on this list.
**The seed is a list of companies, not of boards.** The first seed was jobhunt-hq's 186 US boards, chosen because they could be read, and 31 of their 29,000 roles were in Denmark. The second is 362 European employers picked by hand, with `board` filled by `app.jobs.discover` from each company's own careers site and checked once by fetching it; 175 have one. The rest stay in the list with the `ats` host their site runs on, so that "cannot read yet" is a filter on the Companies section rather than a company that was never written down, and the next parser to write is whichever host that filter shows most. Teamtailor is first, on this list. A re-run attaches the board an entry names to a row that never had one, so a seed update that gains boards reaches an existing database; it never replaces a board, including a detached one, and never touches affinity, notes or status. Its boards go through `set_board` without the fetch, since they were read when the seed was written and `app companies check` finds the ones that died since.
**`discover` reads the site before it guesses.** A careers page that links its board is the authority, so the home page, its own careers links, then the usual paths are read for a link into a known host, and a page's own "open positions" link one level down. Only when no page names a host are Greenhouse, Lever, Ashby, SmartRecruiters, Recruitee and Personio probed with the company's slug, because a board drawn by a script leaves no link. A probe hit on a common name can be another tenant (`recruitee:google` exists, and is not Google), which is why the seed's boards were fetched and read before they landed, and why `discover --add` reports the roles it found.
**`discover` reads the site before it guesses.** A careers page that links its board is the authority, so the home page, its own careers links, then the usual paths are read for a link into a known host, and a page's own "open positions" link one level down. Only when no page names a host are Greenhouse, Lever, Ashby, SmartRecruiters, Recruitee and Personio probed with the company's slug, because a board drawn by a script leaves no link. A probe hit on a common name can be another tenant (`recruitee:google` exists, and is not Google), which is why the seed's boards were fetched and read before they landed, and why `discover --add` reports the roles it found. For the same reason a page that names only an unreadable host ends discovery with that host: probing anyway, as the first version did, could replace a known Workday site with a stranger's Greenhouse board.
**A company row carries `ats`, `country` and `city`.** `ats` is what discovery found whether or not the app reads it; a company with an `ats` and no board subscription is one it cannot read yet. `country` and `city` come from the seed for now and are the first of the company facts the feed will filter by; funding, size and industry come from a source that has them, not from a column filled by hand.
**A company row carries `ats`, `ats_slug`, `country` and `city`.** `ats` is what discovery found whether or not the app reads it; a company with an `ats` and no board subscription is one it cannot read yet. `ats_slug` is the name that host knows the company by (the `acme` of `acme.teamtailor.com`), so the parser written later has its targets without a second discovery run; with a board it is the board's slug. `country` and `city` come from the seed for now and are the first of the company facts the feed will filter by; funding, size and industry come from a source that has them, not from a column filled by hand.
**Jobindex and Adzuna, not Jobnet or Jooble.** Jobindex, Denmark's largest board, has an RSS twin for every search page, with phrase and required-word syntax and a region id; no key. Adzuna has a documented API with a free key for the UK and most of the EU, but no Denmark. Jobnet's search endpoint answers 401 outside its own site's session, and Jooble gives a key only on a manual application and answered a Cloudflare challenge. LinkedIn and Indeed have no public search API.
@@ -56,7 +56,13 @@ What was decided for roledesk and the one reason for each. The longer reasoning
**Switching a board or search off closes its open roles.** Only a pull that reads a target closes its roles, so a target no longer read would keep them open for good. Switched back on, the next pull reopens what it still lists. A batch already in flight when the switch happens reopens them again, so the pull's settle closes every switched-off target once more.
**The Sources tab and the CLI share `app.jobs.sources`.** Adding a board or a search and switching one are one function each, called by `app companies`, `app searches` and `/api/sources*`, so the two cannot disagree about what "off" does. A search or board is run once before it is saved, and refused if it does not answer or finds nothing.
**The Sources tab and the CLI share `app.jobs.sources`.** Adding a board or a search and switching one are one function each, called by `app companies`, `app searches` and `/api/sources*`, so the two cannot disagree about what "off" does. A search or board is run once before it is saved, and refused if it does not answer. A search that finds nothing is refused too, since that is usually a typo in the query; a board that lists nothing is kept, since a company between openings is still one to follow.
**Every way a company gets a board is `app.jobs.sources.set_board`.** `app companies add|set-board|discover --add|seed`, `/api/boards` and `/api/companies/{key}/board` all call it, so attaching a board to an existing row, replacing one and creating a company are one code path; the seed importer moved out of `app/cli.py` for this. The company row and its subscription change in one transaction (`Store.set_company_board`), so a board another company already has leaves no company row behind. Its host and slug become the row's `ats` and `ats_slug`; other facts only fill empty columns, so a person's or the seed's values survive.
**A company has at most one `ats` subscription, rewritten in place.** Replacing a board changes the row's target and closes the old board's open roles, as switching a board off does; detaching switches it off and hides it (`show` 0), which is how a removed search is kept. Deleting the row instead would leave a batch already in flight writing roles against a subscription id that no longer exists, and adding a second row would clash on the label, which is the company's name. A detached board is left out of the Companies and Sources lists, and the next `set_board` reuses the row.
**Discovery that finds nothing readable does not detach a board.** `discover --add` on a company that is read from a board and whose site now names only, say, Workday changes nothing and says so. A board that answers is stronger evidence than a page that names no board; `set-board` or Detach is the explicit step.
**SmartRecruiters' board list has no description.** Its `/postings` endpoint gives title, location and department; the job text is only on the per-posting endpoint, and one extra request per role would multiply a board's request count by its role count. `description` stays empty for this host.
@@ -98,7 +104,7 @@ What was decided for roledesk and the one reason for each. The longer reasoning
## The pages
**Each roledesk tab renders an empty shell; `static/roles.js` fills it and drives every action.** Roles, Applications, Agents and Companies extend the boilerplate's `base.html`; the server renders only select options (from `app.models` and `plugins.ats.FUNCTIONS`, so they cannot drift from what the store accepts). One code path for "show this" beats a server-rendered first page plus a client re-render after every filter change.
**Each roledesk tab renders an empty shell; `static/roles.js` fills it and drives every action.** Roles, Applications, Agents and Companies extend the boilerplate's `base.html`; the server renders only select options (from `app.models`, `plugins.ats.FUNCTIONS` and `plugins.ats.HOSTS`, so they cannot drift from what the store and the plugin accept). One code path for "show this" beats a server-rendered first page plus a client re-render after every filter change.
**`roles(company=, place=)` match a case-insensitive substring of what the board wrote.** The API's company and place fields are free text a person types, not a slug, so "Trade Republic" finds `trade-republic-bank` without knowing the key.
+10
View File
@@ -72,6 +72,16 @@ _BOARDS: dict[str, tuple[str, str]] = {
"https://{slug}.jobs.personio.de"),
}
#: host -> the name the host gives itself: the hosts this plugin reads, in
#: the order a select offers them. What the Sources page's host select is
#: rendered from, so it cannot offer a host `is_known_target` refuses.
HOSTS: dict[str, str] = {
"greenhouse": "Greenhouse", "lever": "Lever", "ashby": "Ashby",
"workable": "Workable", "smartrecruiters": "SmartRecruiters",
"recruitee": "Recruitee", "personio": "Personio",
}
assert set(HOSTS) == set(_BOARDS)
#: The hosts a request or a redirect may land on for a given board host — a
#: fixed API host for five of them, and the company's own subdomain for the
#: two that put the slug in the hostname. Anything else is refused, at the
+39 -16
View File
@@ -9,9 +9,9 @@ from pathlib import Path
import httpx
import pytest
from app.cli import seed_companies
from app.jobs import discover as mod
from app.jobs.discover import Discovery, boards_in, careers_links, discover, slug_guesses
from app.jobs.sources import seed_companies
from app.storage import Store
@@ -103,7 +103,7 @@ def run(site: Site, name: str = "Acme", domain: str = "acme.com") -> Discovery:
def test_a_board_named_on_the_home_page_ends_discovery_there() -> None:
site = Site({"https://acme.com/": '<a href="https://jobs.lever.co/acme">Jobs</a>'})
assert run(site) == Discovery(board="lever:acme", ats="lever",
assert run(site) == Discovery(board="lever:acme", ats="lever", ats_slug="acme",
careers_url="https://acme.com/",
evidence="jobs.lever.co/acme")
assert site.fetched == ["https://acme.com/"]
@@ -135,18 +135,42 @@ def test_a_careers_pages_own_link_is_followed_one_level_down() -> None:
"https://acme.com/careers/open-positions"]
def test_an_unreadable_host_is_kept_when_nothing_readable_turns_up() -> None:
def test_an_unreadable_host_named_by_a_page_is_the_answer_and_nothing_is_probed() -> None:
site = Site({
"https://acme.com/": "<p>welcome</p>",
"https://acme.com/careers": '<a href="https://acme.wd3.myworkdayjobs.com/x">Jobs</a>',
})
}, boards={("greenhouse", "acme")})
found = run(site)
assert found == Discovery(ats="workday", careers_url="https://acme.com/careers",
assert found == Discovery(ats="workday", ats_slug="acme",
careers_url="https://acme.com/careers",
evidence="https://acme.wd3.myworkdayjobs.com")
assert not found.readable
# Every slug was still tried on the hosts with a probe, in case a readable
# board exists that the pages do not link.
assert ("greenhouse", "acme") in site.probed
# The page named the host: a probe could only find another tenant's
# board of the same name, like the greenhouse:acme this fake web has.
assert site.probed == []
def test_a_readable_board_on_a_later_page_still_beats_an_unreadable_host() -> None:
site = Site({
"https://acme.com/": '<script src="https://cdn.teamtailor.com/x.js"></script>'
'<a href="/careers">Careers</a>',
"https://acme.com/careers": '<a href="https://jobs.lever.co/acme">Jobs</a>',
})
assert run(site).board == "lever:acme"
assert site.probed == []
def test_an_unreadable_hosts_slug_is_kept_when_a_later_link_names_it() -> None:
"""The CDN names teamtailor before any link does; the slug comes from the
first link that carries one, the acme of acme.teamtailor.com."""
site = Site({
"https://acme.com/": '<script src="https://cdn.teamtailor.com/x.js"></script>',
"https://acme.com/careers": '<a href="https://acme-jobs.teamtailor.com/jobs">Jobs</a>',
})
found = run(site)
assert (found.ats, found.ats_slug, found.board) == ("teamtailor", "acme-jobs", "")
assert found.evidence == "https://acme-jobs.teamtailor.com"
assert found.careers_url == "https://acme.com/"
def test_slug_probes_find_a_board_the_pages_never_link() -> None:
@@ -181,8 +205,7 @@ def store(tmp_path: Path) -> Store:
return Store(tmp_path / "t.db")
def test_the_seed_keeps_a_company_the_app_cannot_read(store: Store, tmp_path: Path,
capsys: pytest.CaptureFixture[str]) -> None:
def test_the_seed_keeps_a_company_the_app_cannot_read(store: Store, tmp_path: Path) -> None:
seed = tmp_path / "companies.json"
seed.write_text(json.dumps({"companies": [
{"name": "Pleo", "domain": "pleo.io", "country": "DK", "city": "Copenhagen",
@@ -192,12 +215,13 @@ def test_the_seed_keeps_a_company_the_app_cannot_read(store: Store, tmp_path: Pa
"careers_url": "https://www.novonordisk.com/careers.html"},
{"name": "Odd", "board": "nosuch:odd"},
]}))
assert seed_companies(store, str(seed)) == {"added": 3, "readable": 1, "skipped": 0,
"total": 3}
result = seed_companies(store, seed)
assert result == {"added": 3, "boards": 1, "skipped": 0, "total": 3,
"problems": ["odd: 'nosuch:odd' is not a known board; kept without it"]}
pleo = store.company("pleo")
assert pleo is not None
assert (pleo.ats, pleo.country, pleo.city, pleo.careers_url) == (
"ashby", "DK", "Copenhagen", "https://jobs.ashbyhq.com/pleo")
assert (pleo.ats, pleo.ats_slug, pleo.country, pleo.city, pleo.careers_url) == (
"ashby", "pleo", "DK", "Copenhagen", "https://jobs.ashbyhq.com/pleo")
[sub] = store.subscriptions(source="ats")
assert (sub.target, sub.company) == ("ashby:pleo", "Pleo")
novo = store.company("novo-nordisk")
@@ -206,6 +230,5 @@ def test_the_seed_keeps_a_company_the_app_cannot_read(store: Store, tmp_path: Pa
"https://www.novonordisk.com/careers.html")
odd = store.company("odd")
assert odd is not None and odd.ats == ""
assert "not a known board" in capsys.readouterr().err
# Re-running adds nothing and edits nothing.
assert seed_companies(store, str(seed))["skipped"] == 3
assert seed_companies(store, seed)["skipped"] == 3
+289
View File
@@ -0,0 +1,289 @@
"""A company's board, set by hand, by discovery or by the seed: `set_board`
and the three ways in that go through it."""
from __future__ import annotations
import json
import sys
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
import pytest
ROOT = Path(__file__).resolve().parents[1]
if str(ROOT) not in sys.path:
sys.path.insert(0, str(ROOT))
import plugins.ats as ats # noqa: E402
from desk.models import Item # noqa: E402
from app.cli import main # noqa: E402
from app.jobs import sources # noqa: E402
from app.jobs.discover import Discovery # noqa: E402
from app.storage import Store # noqa: E402
@pytest.fixture
def store(tmp_path: Path) -> Store:
return Store(tmp_path / "t.db")
class Boards:
"""A fake `fetch_board`: `listed` maps a target to how many roles it
lists; any other target does not answer. Remembers what was fetched."""
def __init__(self, **listed: int) -> None:
self.listed = {k.replace("_", ":", 1): v for k, v in listed.items()}
self.fetched: list[str] = []
def __call__(self, target: str) -> list[dict[str, Any]]:
self.fetched.append(target)
if target not in self.listed:
raise OSError("connection refused")
return [{"source_id": str(i)} for i in range(self.listed[target])]
@pytest.fixture
def boards(monkeypatch: pytest.MonkeyPatch) -> Boards:
fake = Boards(greenhouse_acme=3, ashby_acme=2, lever_quiet=0, lever_acme=1)
monkeypatch.setattr(ats, "fetch_board", fake)
return fake
def posting(source_id: str) -> Item:
item = Item(source_id=source_id, location=f"https://boards.example/acme/{source_id}",
title="Backend Engineer", body="",
published_at=datetime(2026, 9, 20, tzinfo=UTC), raw={})
item.source = "ats"
return item
# --------------------------------------------------------------- set_board
def test_set_board_creates_a_company_with_its_board(store: Store, boards: Boards) -> None:
sub, roles, closed = sources.set_board(store, "acme", "greenhouse:acme", name="Acme",
domain="acme.example")
assert (sub.target, roles, closed) == ("greenhouse:acme", 3, 0)
acme = store.company("acme")
assert acme is not None
assert (acme.name, acme.ats, acme.ats_slug, acme.domain, acme.careers_url) == (
"Acme", "greenhouse", "acme", "acme.example", "https://job-boards.greenhouse.io/acme")
def test_set_board_attaches_a_board_to_a_row_without_one(store: Store, boards: Boards) -> None:
store.add_company("Acme", domain="acme.example", ats="teamtailor", ats_slug="acme",
careers_url="https://acme.example/careers", notes="met them")
sub, roles, closed = sources.set_board(store, "acme", "ashby:acme", domain="other.example")
assert (sub.target, sub.company, roles, closed) == ("ashby:acme", "Acme", 2, 0)
acme = store.company("acme")
assert acme is not None
# The board's host and slug replace what discovery found; the rest of the
# row, domain included, is not overwritten.
assert (acme.ats, acme.ats_slug, acme.domain, acme.notes, acme.careers_url) == (
"ashby", "acme", "acme.example", "met them", "https://acme.example/careers")
def test_set_board_replaces_a_board_and_closes_its_open_roles(
store: Store, boards: Boards) -> None:
old, _, _ = sources.set_board(store, "acme", "greenhouse:acme", name="Acme")
store.upsert_role(posting("1"), old)
store.upsert_role(posting("2"), old)
new, roles, closed = sources.set_board(store, "acme", "lever:acme")
assert (new.id, new.target, roles, closed) == (old.id, "lever:acme", 1, 2)
assert store.roles() == []
acme = store.company("acme")
assert acme is not None
# careers_url was the old board's page, so it follows the board.
assert (acme.ats, acme.careers_url) == ("lever", "https://jobs.lever.co/acme")
def test_a_board_listing_nothing_is_kept(store: Store, boards: Boards) -> None:
sub, roles, _ = sources.set_board(store, "", "lever:quiet", name="Quiet")
assert (sub.target, roles) == ("lever:quiet", 0)
assert store.company("quiet") is not None
def test_a_board_that_does_not_answer_or_is_unknown_is_refused(
store: Store, boards: Boards) -> None:
with pytest.raises(ValueError, match="did not answer"):
sources.set_board(store, "", "greenhouse:gone", name="Gone")
with pytest.raises(ValueError, match="host:slug"):
sources.set_board(store, "", "teamtailor:gone", name="Gone")
with pytest.raises(ValueError, match="needs a name"):
sources.set_board(store, "nobody", "greenhouse:acme")
with pytest.raises(ValueError, match="is not the company"):
sources.set_board(store, "acme", "greenhouse:acme", name="Other")
assert store.companies() == [] and store.subscriptions() == []
assert boards.fetched == ["greenhouse:gone"] # the others refused before it
def test_a_board_another_company_has_leaves_nothing_written(
store: Store, boards: Boards) -> None:
sources.set_board(store, "", "greenhouse:acme", name="Acme")
with pytest.raises(ValueError, match="already has"):
sources.add_board(store, "greenhouse:acme", "Beta")
assert store.company("beta") is None
def test_detach_board_closes_its_roles_and_keeps_the_row(store: Store, boards: Boards) -> None:
sub, _, _ = sources.set_board(store, "", "greenhouse:acme", name="Acme")
store.upsert_role(posting("1"), sub)
assert sources.detach_board(store, "acme") == 1
acme = store.company("acme")
assert acme is not None and (acme.ats, acme.ats_slug) == ("greenhouse", "acme")
assert store.company_boards() == {}
with pytest.raises(ValueError, match="no board"):
sources.detach_board(store, "acme")
with pytest.raises(KeyError):
sources.detach_board(store, "nope")
# ----------------------------------------------------------- add_discovered
def test_add_discovered_sets_or_replaces_the_board_of_an_existing_company(
store: Store, boards: Boards) -> None:
store.add_company("Acme", ats="teamtailor")
key, roles = sources.add_discovered(store, "Acme", "acme.example",
Discovery(board="greenhouse:acme", ats="greenhouse"))
assert (key, roles) == ("acme", 3)
key, roles = sources.add_discovered(store, "Acme", "acme.example",
Discovery(board="ashby:acme", ats="ashby"))
assert roles == 2
assert store.company_subscription("acme").target == "ashby:acme" # type: ignore[union-attr]
assert store.company("acme").domain == "acme.example" # type: ignore[union-attr]
def test_add_discovered_records_an_unreadable_host_with_its_slug(
store: Store, boards: Boards) -> None:
found = Discovery(ats="teamtailor", ats_slug="acme",
careers_url="https://acme.example/careers")
assert sources.add_discovered(store, "Acme", "acme.example", found) == ("acme", 0)
store.update_company("acme", notes="keep")
newer = Discovery(ats="workday", ats_slug="acme-inc",
careers_url="https://acme.example/jobs")
assert sources.add_discovered(store, "Acme", "acme.example", newer) == ("acme", 0)
acme = store.company("acme")
assert acme is not None
assert (acme.ats, acme.ats_slug, acme.careers_url, acme.notes) == (
"workday", "acme-inc", "https://acme.example/jobs", "keep")
def test_add_discovered_keeps_a_board_when_nothing_readable_was_found(
store: Store, boards: Boards) -> None:
sources.set_board(store, "", "greenhouse:acme", name="Acme")
with pytest.raises(ValueError, match="nothing changed"):
sources.add_discovered(store, "Acme", "acme.example", Discovery(ats="workday"))
acme = store.company("acme")
assert acme is not None and acme.ats == "greenhouse"
# --------------------------------------------------------------- the seed
def _seed(tmp_path: Path, *entries: dict[str, str]) -> Path:
path = tmp_path / "companies.json"
path.write_text(json.dumps({"companies": list(entries)}))
return path
def test_a_seed_rerun_attaches_a_board_the_seed_gained(
store: Store, boards: Boards, tmp_path: Path) -> None:
first = _seed(tmp_path,
{"name": "Acme", "domain": "acme.example", "ats": "teamtailor",
"careers_url": "https://acme.example/careers"},
{"name": "Beta", "board": "lever:beta"})
assert sources.seed_companies(store, first) == {
"added": 2, "boards": 1, "skipped": 0, "total": 2, "problems": []}
agent = store.save_agent("A")
store.set_affinity(agent.id, "acme", 2)
store.update_company("acme", notes="met them", status="contacted")
# The next seed found Acme's board, and names a new board for Beta.
later = _seed(tmp_path,
{"name": "Acme", "domain": "acme.example", "board": "ashby:acme"},
{"name": "Beta", "board": "lever:beta-new"})
assert sources.seed_companies(store, later) == {
"added": 0, "boards": 1, "skipped": 1, "total": 2, "problems": []}
assert store.company_subscription("acme").target == "ashby:acme" # type: ignore[union-attr]
acme = store.company("acme", agent.id)
assert acme is not None
assert (acme.affinity, acme.notes, acme.status, acme.ats) == (
2, "met them", "contacted", "ashby")
# A board a row already has is never replaced by the seed.
assert store.company_subscription("beta").target == "lever:beta" # type: ignore[union-attr]
# The seed's boards are not fetched: they were read when it was written.
assert boards.fetched == []
def test_a_seed_rerun_leaves_a_detached_board_detached(
store: Store, boards: Boards, tmp_path: Path) -> None:
seed = _seed(tmp_path, {"name": "Acme", "board": "greenhouse:acme"})
sources.seed_companies(store, seed)
sources.detach_board(store, "acme")
assert sources.seed_companies(store, seed)["skipped"] == 1
assert store.company_boards() == {}
def test_a_seed_board_that_clashes_still_lands_the_company(
store: Store, boards: Boards, tmp_path: Path) -> None:
seed = _seed(tmp_path, {"name": "Acme", "board": "greenhouse:acme"},
{"name": "Acme Two", "board": "greenhouse:acme", "country": "DK"})
result = sources.seed_companies(store, seed)
assert (result["added"], result["boards"]) == (2, 1)
[problem] = result["problems"]
assert problem.startswith("acme-two: another board already has 'greenhouse:acme'")
two = store.company("acme-two")
assert two is not None
assert (two.ats, two.ats_slug, two.country) == ("greenhouse", "acme", "DK")
assert store.company_subscription("acme-two") is None
# -------------------------------------------------------------------- CLI
def _cli(tmp_path: Path, *argv: str) -> int:
config = tmp_path / "config.yaml"
if not config.exists():
config.write_text(f"database: {tmp_path / 't.db'}\n")
return main(["-c", str(config), "companies", *argv])
def test_cli_set_board_attaches_and_replaces(
store: Store, boards: Boards, tmp_path: Path,
capsys: pytest.CaptureFixture[str]) -> None:
store.add_company("Acme", ats="teamtailor")
assert _cli(tmp_path, "set-board", "acme", "greenhouse:acme") == 0
assert json.loads(capsys.readouterr().out) == {
"key": "acme", "board": "greenhouse:acme", "roles": 3, "closed": 0}
assert _cli(tmp_path, "set-board", "acme", "lever:acme") == 0
assert json.loads(capsys.readouterr().out)["board"] == "lever:acme"
assert _cli(tmp_path, "set-board", "nobody", "lever:acme") == 1
assert "no company 'nobody'" in capsys.readouterr().err
assert _cli(tmp_path, "detach", "acme") == 0
assert json.loads(capsys.readouterr().out) == {"key": "acme", "closed": 0}
def test_cli_discover_add_updates_an_existing_company(
store: Store, boards: Boards, tmp_path: Path, monkeypatch: pytest.MonkeyPatch,
capsys: pytest.CaptureFixture[str]) -> None:
from app.jobs import discover
store.add_company("Acme", ats="teamtailor")
monkeypatch.setattr(discover, "discover",
lambda name, domain: Discovery(board="ashby:acme", ats="ashby",
ats_slug="acme"))
assert _cli(tmp_path, "discover", "Acme", "acme.example", "--add") == 0
out = json.loads(capsys.readouterr().out)
assert (out["key"], out["board"], out["roles"]) == ("acme", "ashby:acme", 2)
assert store.company_subscription("acme").target == "ashby:acme" # type: ignore[union-attr]
def test_cli_seed_prints_counts_and_problems(
store: Store, boards: Boards, tmp_path: Path,
capsys: pytest.CaptureFixture[str]) -> None:
seed = _seed(tmp_path, {"name": "Odd", "board": "nosuch:odd"})
assert _cli(tmp_path, "seed", "--path", str(seed)) == 0
captured = capsys.readouterr()
assert json.loads(captured.out) == {"added": 1, "boards": 0, "skipped": 0, "total": 1}
assert "companies seed: odd: 'nosuch:odd' is not a known board" in captured.err
+94 -3
View File
@@ -62,7 +62,7 @@ def test_schema_creates_on_an_empty_file(tmp_path: Path) -> None:
"application_events", "agents", "scores", "state"} <= tables
assert "idx_roles_cluster" in indexes
assert store.migrations_applied == []
assert LATEST == 8
assert LATEST == 9
# A second open finds it current and applies nothing.
assert Store(path).migrations_applied == []
@@ -94,7 +94,7 @@ def test_a_v1_file_migrates_to_the_shape_a_new_file_has(tmp_path: Path) -> None:
# v1 had affinity on the company; the vote columns v4 added and v6
# drops again are not in a fresh file, so there is nothing to put back.
conn.execute("ALTER TABLE companies ADD COLUMN affinity INTEGER NOT NULL DEFAULT 0")
for column in ("ats", "country", "city"): # v7
for column in ("ats", "country", "city", "ats_slug"): # v7, v9
conn.execute(f"ALTER TABLE companies DROP COLUMN {column}")
for table in ("vectors", "role_tags", "agent_companies", "role_votes"):
conn.execute(f"DROP TABLE {table}")
@@ -107,6 +107,21 @@ def test_a_v1_file_migrates_to_the_shape_a_new_file_has(tmp_path: Path) -> None:
assert sorted(_columns(old, table)) == sorted(_columns(fresh, table))
def test_a_v8_file_gains_ats_slug_and_keeps_its_companies(tmp_path: Path) -> None:
path = tmp_path / "v8.db"
Store(path).add_company("Acme", ats="teamtailor", country="DK")
with sqlite3.connect(path) as conn:
conn.execute("ALTER TABLE companies DROP COLUMN ats_slug")
conn.execute("PRAGMA user_version = 8")
migrated = Store(path)
assert migrated.migrations_applied == [9]
acme = migrated.company("acme")
assert acme is not None
assert (acme.ats, acme.country, acme.ats_slug) == ("teamtailor", "DK", "")
def _downgrade_to_v4(path: Path) -> None:
"""A fresh file put back to the v4 shape: votes on the role, affinity on
the company, no read state, no tags, no per-agent tables."""
@@ -116,7 +131,7 @@ def _downgrade_to_v4(path: Path) -> None:
conn.execute("ALTER TABLE roles ADD COLUMN vote INTEGER CHECK (vote IN (-1, 0, 1))")
conn.execute("ALTER TABLE roles ADD COLUMN voted_at TEXT")
conn.execute("ALTER TABLE companies ADD COLUMN affinity INTEGER NOT NULL DEFAULT 0")
for column in ("ats", "country", "city"): # v7
for column in ("ats", "country", "city", "ats_slug"): # v7, v9
conn.execute(f"ALTER TABLE companies DROP COLUMN {column}")
conn.execute("ALTER TABLE agents DROP COLUMN limits") # v8
conn.execute("DROP TABLE agent_companies")
@@ -642,6 +657,82 @@ def test_open_role_counts_and_company_boards(store: Store) -> None:
assert store.company_boards() == {"acme": "greenhouse"}
def test_set_company_board_creates_the_row_and_its_board_together(store: Store) -> None:
company, sub, closed = store.set_company_board(
"acme", "greenhouse:acme", host="greenhouse", name="Acme",
fields={"ats": "greenhouse", "ats_slug": "acme", "country": "DK"})
assert (company.name, company.ats, company.ats_slug, company.country) == (
"Acme", "greenhouse", "acme", "DK")
assert sub is not None and closed == 0
assert (sub.target, sub.label, sub.company, sub.board, sub.show) == (
"greenhouse:acme", "Acme", "Acme", "greenhouse", 1)
assert store.company_subscription("acme") == sub
assert store.company_boards() == {"acme": "greenhouse"}
def test_a_board_clash_leaves_no_company_row(store: Store) -> None:
"""The subscription insert fails on UNIQUE (source, target), and the
company row written a statement earlier goes with it."""
store.set_company_board("acme", "greenhouse:acme", host="greenhouse", name="Acme")
with pytest.raises(ValueError, match="already has 'greenhouse:acme'"):
store.set_company_board("beta", "greenhouse:acme", host="greenhouse", name="Beta")
assert store.company("beta") is None
assert [s.company for s in store.subscriptions(source="ats")] == ["Acme"]
def test_a_clash_on_replace_leaves_the_old_board_and_its_roles(store: Store) -> None:
store.set_company_board("acme", "greenhouse:acme", host="greenhouse", name="Acme")
_, beta, _ = store.set_company_board("beta", "lever:beta", host="lever", name="Beta")
assert beta is not None
store.upsert_role(posting("1", company="Beta"), beta)
with pytest.raises(ValueError):
store.set_company_board("beta", "greenhouse:acme", host="greenhouse",
fields={"ats": "greenhouse"})
assert store.company_subscription("beta") == beta
assert store.company("beta").ats == "" # type: ignore[union-attr]
assert [r.source_id for r in store.roles()] == ["1"] # still open
def test_replacing_a_board_rewrites_it_in_place_and_closes_the_old_roles(
store: Store) -> None:
_, old, _ = store.set_company_board("acme", "greenhouse:acme", host="greenhouse",
name="Acme")
assert old is not None
store.update_subscription(old.id, fetch=False, target_id="gh-1")
store.upsert_role(posting("1", company="Acme"), old)
store.upsert_role(posting("2", company="Acme"), old)
company, new, closed = store.set_company_board("acme", "ashby:acme", host="ashby",
fields={"ats": "ashby"})
assert closed == 2 and store.roles() == []
assert new is not None and new.id == old.id
assert (new.target, new.board, new.target_id, new.fetch, new.subscribed) == (
"ashby:acme", "ashby", "", 1, 1)
assert company.ats == "ashby"
# The same board again closes nothing.
store.upsert_role(posting("3", company="Acme"), new)
assert store.set_company_board("acme", "ashby:acme", host="ashby")[2] == 0
assert [r.source_id for r in store.roles()] == ["3"]
def test_detaching_a_board_hides_it_closes_its_roles_and_keeps_the_row(
store: Store) -> None:
_, sub, _ = store.set_company_board("acme", "greenhouse:acme", host="greenhouse",
name="Acme")
assert sub is not None
store.upsert_role(posting("1", company="Acme"), sub)
company, detached, closed = store.set_company_board("acme", "")
assert closed == 1 and company.key == "acme"
assert detached is not None
assert (detached.subscribed, detached.fetch, detached.show) == (0, 0, 0)
assert store.company_boards() == {}
# The next board reuses the row rather than clashing on its label.
_, again, _ = store.set_company_board("acme", "lever:acme", host="lever")
assert again is not None and again.id == sub.id and again.show == 1
with pytest.raises(KeyError):
store.set_company_board("nope", "")
# ----------------------------------------------------------- subscriptions
def test_subscriptions_crud(store: Store) -> None:
+53 -4
View File
@@ -386,7 +386,7 @@ def test_companies_lists_board_and_open_role_count(
sub = store.add_subscription("ats", "greenhouse:acme", company="Acme", board="greenhouse")
store.add_company("Acme", key="acme", ats="greenhouse", country="DK")
store.upsert_role(posting("1", company="Acme"), sub)
store.add_company("Other", key="other") # known only from a search: no board
store.add_company("Other", key="other") # no board found
store.add_company("Novo", key="novo", ats="successfactors", # followed, not readable
careers_url="https://novo.example/careers")
@@ -395,7 +395,8 @@ def test_companies_lists_board_and_open_role_count(
[row] = [c for c in res.json() if c["key"] == "acme"]
assert row == {"key": "acme", "name": "Acme", "board": "greenhouse",
"subscription_id": sub.id, "on": True, "affinity": 0, "open_roles": 1,
"ats": "greenhouse", "careers_url": "", "country": "DK"}
"ats": "greenhouse", "ats_slug": "", "careers_url": "",
"country": "DK"}
[other] = [c for c in res.json() if c["key"] == "other"]
assert (other["board"], other["subscription_id"], other["on"], other["ats"]) == (
"", None, None, "")
@@ -653,7 +654,10 @@ def test_the_sources_page_has_a_section_per_plugin(client: TestClient) -> None:
# Company boards: the follow form, the agent select, the list.
assert 'id="companies-list"' in page and 'id="c-agent"' in page
assert 'id="b-host"' in page and 'id="b-slug"' in page and 'id="b-name"' in page
assert '<option value="greenhouse">' in page and '<option value="personio">' in page
# The host select is plugins.ats's HOSTS, not a list typed into the page.
from plugins.ats import HOSTS
for host, label in HOSTS.items():
assert f'<option value="{host}">{label}</option>' in page
boards = page[page.index('id="company-boards"'):page.index('id="runs"')].lower()
assert "hidden" in boards and "mute" not in boards
# Runs: the pull button, the queue's elements, and the log as its last
@@ -915,11 +919,56 @@ def test_a_board_is_followed_only_if_it_answers(
assert res.json()["found"] == 1 and res.json()["source"]["target"] == "ashby:acme"
assert store.company("acme") is not None
# Following a company that exists goes through `set_board`: the same
# board again is the same subscription, not a refusal or a second one.
again = client.post("/api/boards", json={"board": "ashby:acme", "name": "Acme"})
assert again.status_code == 422 and "already followed" in again.json()["detail"]
assert again.status_code == 201
assert again.json()["source"]["id"] == res.json()["source"]["id"]
bad = client.post("/api/boards", json={"board": "acme", "name": "Other"})
assert bad.status_code == 422 and "host:slug" in bad.json()["detail"]
def down(target: str) -> list[dict[str, str]]:
raise OSError("connection refused")
monkeypatch.setattr(ats, "fetch_board", down)
dead = client.post("/api/boards", json={"board": "ashby:other", "name": "Other"})
assert dead.status_code == 422 and "did not answer" in dead.json()["detail"]
assert store.company("other") is None
def test_a_companys_board_is_set_replaced_and_detached(
client: TestClient, store: Store, monkeypatch: pytest.MonkeyPatch) -> None:
import plugins.ats as ats
monkeypatch.setattr(ats, "fetch_board", lambda target: []) # between openings
store.add_company("Acme", key="acme", ats="teamtailor", ats_slug="acme")
res = client.post("/api/companies/acme/board", json={"board": "greenhouse:acme"})
assert res.status_code == 200
body = res.json()
assert (body["found"], body["closed"], body["source"]["target"]) == (0, 0, "greenhouse:acme")
sub_id = body["source"]["id"]
store.upsert_role(posting("1", company="Acme"), store.subscription(sub_id))
[row] = client.get("/api/companies").json()
assert (row["board"], row["subscription_id"], row["ats"], row["ats_slug"]) == (
"greenhouse", sub_id, "greenhouse", "acme")
res = client.post("/api/companies/acme/board", json={"board": "lever:acme"})
assert res.json()["closed"] == 1 and res.json()["source"]["id"] == sub_id
res = client.delete("/api/companies/acme/board")
assert res.status_code == 200 and res.json() == {"closed": 0}
[row] = client.get("/api/companies").json()
assert (row["board"], row["subscription_id"], row["ats"]) == ("", None, "lever")
assert client.get("/api/sources").json()["boards"] == []
assert client.delete("/api/companies/acme/board").status_code == 409
assert client.post("/api/companies/nope/board",
json={"board": "lever:acme"}).status_code == 404
assert client.delete("/api/companies/nope/board").status_code == 404
bad = client.post("/api/companies/acme/board", json={"board": "teamtailor:acme"})
assert bad.status_code == 422 and "host:slug" in bad.json()["detail"]
# ------------------------------------------------- roles: text and summary