Prep stage: host words stored raw, mapped centrally in app/prep after every pull
upsert_role stores the host's own words (roles.*_host), the place texts unresolved when they change, and the company key unresolved on insert, and marks the role prep_due. app.prep maps seniority, employment, remote and country through the host tables in app/prep/hosts, resolves places and company aliases, lists unknown words in `unmapped`, remaps every open role when the tables' fingerprint changes, and runs rebuild_dupes, which moves out of the pull's settle. Migration 22 adds the columns and the table and fills the host words from raw. Cascade: pull, prep, embed, score, enrich, prep, embed, score, extract, score, summarise. `app prep [unmapped] [--remap]`, /api/prep/unmapped and a fold on the Sources page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
a66aac95cf
commit
d04ceae7e0
31 files changed
+1468
-330
No files matched your search
@@ -649,6 +649,13 @@ SENIORITIES: tuple[str, ...] = get_args(Seniority)
|
||||
#: The levels a seniority filter may name: `SENIORITIES` without the unknown.
|
||||
LEVELS: tuple[str, ...] = tuple(level for level in SENIORITIES if level)
|
||||
|
||||
#: The kind of contract a posting states, as `app.prep` maps a host's word.
|
||||
#: '' is "it does not say". `plugins/readers/_shared.py` keeps a copy for the
|
||||
#: readers' own tests, since a plugin imports nothing from `app`.
|
||||
Employment = Literal["", "full_time", "part_time", "contract", "temporary", "internship",
|
||||
"freelance"]
|
||||
EMPLOYMENTS: tuple[str, ...] = tuple(e for e in get_args(Employment) if e)
|
||||
|
||||
#: Whether a posting says it sponsors a work permit. '' is "it does not say".
|
||||
Permit = Literal["", "sponsors", "no_sponsorship"]
|
||||
PERMITS: tuple[str, ...] = get_args(Permit)
|
||||
@@ -701,6 +708,16 @@ def trusted_posted(host: str) -> bool:
|
||||
_POSTED_FLOOR = "2015-01-01T00:00:00+00:00"
|
||||
|
||||
|
||||
def host_word(value: Any) -> str:
|
||||
"""A host's word as `roles.*_host` stores it: stripped text, a bool as
|
||||
'true' or 'false', '' for nothing. What `app.prep` maps."""
|
||||
if value is None:
|
||||
return ""
|
||||
if isinstance(value, bool):
|
||||
return "true" if value else "false"
|
||||
return str(value).strip()
|
||||
|
||||
|
||||
def posted_host(raw: dict[str, Any]) -> str:
|
||||
"""The host a role's dates came from, read the way `ats` and `search`
|
||||
both put it in `Item.raw`: `board` for an ATS board (`greenhouse`,
|
||||
@@ -797,6 +814,8 @@ class Role(Model):
|
||||
seniority_model: str = ""
|
||||
seniority_stated: str = ""
|
||||
seniority_title: str = ""
|
||||
#: The contract kind, one of `EMPLOYMENTS` or ''; mapped by `app.prep`.
|
||||
employment: str = ""
|
||||
function: str = ""
|
||||
permit: Permit = ""
|
||||
#: ISO 639-1 codes the posting says are required, lowercase.
|
||||
|
||||
Reference in new issue
Block a user