Add migration 27 and read a remote role's area in the chips, the gates and the rows

roles.remote_model, roles.remote_where_model and role_remote, with a stored remote no host word gives kept as the host's word. The remote tokens filter over role_remote, remote_from and a require gate on country read a remote role through its area, extract waits while remote_where_model is NULL and writes the model's remote outright, and rows carry remote_where.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Opus 5.5 committed 2026-09-30 16:00:06 +02:00
1 parent 189cf81350
commit 57cac5f896
8 files changed
+262 -62

No files matched your search

+58 -4
View File
@@ -118,7 +118,9 @@ CREATE TABLE IF NOT EXISTS roles (
remote_host TEXT NOT NULL DEFAULT '',
country_host TEXT NOT NULL DEFAULT '',
employment TEXT NOT NULL DEFAULT '',
prep_due INTEGER NOT NULL DEFAULT 0
prep_due INTEGER NOT NULL DEFAULT 0,
remote_model TEXT NOT NULL DEFAULT '',
remote_where_model TEXT
);
-- The *_host columns hold the host's own words as a reader handed them on
-- (app.models.host_word: a bool is 'true' or 'false'), and app.prep maps
@@ -129,9 +131,12 @@ CREATE TABLE IF NOT EXISTS roles (
-- arrives and embedded while there is none; a later empty teaser does not
-- wipe it. seniority_model, seniority_stated and seniority_title are the
-- three levels behind roles.seniority: the model's (extract), the host's own
-- word mapped to app.seniority's vocabulary, and the title's. No comment sits
-- among the last columns: SQLite cannot drop a column a comment precedes,
-- which the migration tests do.
-- word mapped to app.seniority's vocabulary, and the title's. remote_model is
-- extract's remote, which wins over the host's word and the place line, and
-- remote_where_model the names it read for where a remote hire may live (a
-- JSON list, NULL never asked, [] not stated). No comment sits among the last
-- columns: SQLite cannot drop a column a comment precedes, which the
-- migration tests do.
-- The gazetteer: a place is a continent, a country (key the ISO code), a
-- region or a city, in the place parent_id names. key is unique per kind:
@@ -177,6 +182,15 @@ CREATE TABLE IF NOT EXISTS role_places (
url TEXT NOT NULL DEFAULT ''
);
-- A remote role's area, the places a remote hire may live in (app/prep/area.py),
-- in the order found. Anywhere (places.ANYWHERE) is one of them. A role with
-- no row is not remote, or remote with its area not stated.
CREATE TABLE IF NOT EXISTS role_remote (
uid TEXT NOT NULL REFERENCES roles(uid) ON DELETE CASCADE,
place_id INTEGER NOT NULL REFERENCES places(id) ON DELETE CASCADE,
PRIMARY KEY (uid, place_id)
);
-- The open vocabulary summarise reads off a posting: the role family, the
-- technologies and skills it names, and the domain. What the feed's filter
-- row is built from; see Store.set_tags and Store.tag_counts.
@@ -424,6 +438,8 @@ CREATE INDEX IF NOT EXISTS idx_roles_dup_of
CREATE INDEX IF NOT EXISTS idx_role_places_uid ON role_places(uid);
CREATE INDEX IF NOT EXISTS idx_role_places_place ON role_places(place_id, uid);
CREATE INDEX IF NOT EXISTS idx_places_parent ON places(parent_id);
-- The remote roles whose area holds a set of places (a remote: token).
CREATE INDEX IF NOT EXISTS idx_role_remote_place ON role_remote(place_id, uid);
-- The roles the next prep maps. Partial: after a prep there are none.
CREATE INDEX IF NOT EXISTS idx_roles_prep_due ON roles(prep_due) WHERE prep_due > 0;
"""
@@ -629,6 +645,10 @@ UPDATE roles SET seniority = COALESCE(NULLIF(seniority_model, ''),
# A Python step: it adds the column only where it is missing.
(26, "roles.posting_url: where a Jobindex ad's own page leads, found when its text is "
"fetched", lambda conn: _add_posting_url(conn)),
# A Python step: it adds the columns only where they are missing, and seeds
# Anywhere. The areas are written by the next prep (app.prep.stage.VERSION 3).
(27, "roles.remote_model and roles.remote_where_model, role_remote and the Anywhere "
"place: where a remote hire may live", lambda conn: _add_remote_area(conn)),
)
#: What a file created from `SCHEMA` today is already at.
@@ -878,6 +898,40 @@ def _add_posting_url(conn: sqlite3.Connection) -> None:
conn.execute("ALTER TABLE roles ADD COLUMN posting_url TEXT NOT NULL DEFAULT ''")
def _add_remote_area(conn: sqlite3.Connection) -> None:
"""Migration 27's step: `roles.remote_model` empty and
`roles.remote_where_model` NULL for every role, so `extract` reads the
roles it has read once more for the area; `role_remote`, empty until
the next prep; and Anywhere, seeded with the rest of the gazetteer.
Prep now writes `remote` outright from the model, the host's word and
the place line, where it used to keep a filled column. A board role's
`remote` without a host word becomes its `remote_host`, which a pull
keeps until the host states another word: on live 1,280 of the 3,880
open ones are given by nothing prep reads now, mostly Greenhouse, Oracle
and Workday roles an older reader mapped. A search role's `remote_host`
its reader no longer sends is an older reading of its title ("...
Remote Sensing ...", 41 roles), and is cleared for prep to read the
title again. Runs twice without harm."""
columns = {r[1] for r in conn.execute("PRAGMA table_info(roles)")}
if "remote_model" not in columns:
conn.execute("ALTER TABLE roles ADD COLUMN remote_model TEXT NOT NULL DEFAULT ''")
if "remote_where_model" not in columns:
conn.execute("ALTER TABLE roles ADD COLUMN remote_where_model TEXT")
kept = conn.execute("UPDATE roles SET remote_host = remote WHERE remote_host = '' "
"AND remote != '' AND source != 'search'").rowcount
cleared = conn.execute("UPDATE roles SET remote_host = '' WHERE source = 'search' "
"AND remote_host != '' AND json_extract(raw, '$.remote') IS NULL"
).rowcount
log.info("migration 27: remote kept as the host's word on %d roles, cleared on %d",
kept, cleared)
conn.execute("""CREATE TABLE IF NOT EXISTS role_remote (
uid TEXT NOT NULL REFERENCES roles(uid) ON DELETE CASCADE,
place_id INTEGER NOT NULL REFERENCES places(id) ON DELETE CASCADE,
PRIMARY KEY (uid, place_id))""")
gazetteer.seed(conn)
def _add_all_roles(conn: sqlite3.Connection) -> None:
"""Migration 21's step. The column; the tag for every company that is a
favourite of any agent, since favourite no longer widens what a pull