Read date trust only from the readers' declarations, now that every reader declares
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
5f0f4ad5b7
commit
3adafd8950
2 files changed
+6
-67
No files matched your search
+4
-64
@@ -666,63 +666,6 @@ HAND_REASONS: tuple[str, ...] = get_args(HandReason)
|
||||
|
||||
# -------------------------------------------------------------- posting dates
|
||||
|
||||
#: Hosts whose date is a creation or first-publication timestamp, for a
|
||||
#: reader that does not declare `READS` yet (see `trusted_posted`); a reader
|
||||
#: that declares it says so with `fields.posted="trusted"`. Checked against
|
||||
#: live boards on 2026-09-29:
|
||||
#:
|
||||
#: - `greenhouse`: `plugins.readers.greenhouse.parse` already prefers
|
||||
#: `first_published` over `updated_at`. A live board (adyen) showed a job
|
||||
#: with `first_published` five months before `updated_at`, proving the
|
||||
#: first stays fixed while the second moves on an edit — the resettable
|
||||
#: field decisions.md used to warn against is `updated_at`, which this
|
||||
#: reads only when `first_published` is missing (rare: 0 of 208 on that
|
||||
#: board). The large real gaps this migration measured for Greenhouse
|
||||
#: roles (average 143 days between this date and `first_seen`) are further
|
||||
#: evidence it holds still rather than tracking every edit.
|
||||
#: - `lever` (`createdAt`), `ashby` (`publishedAt`), `personio` (`createdAt`):
|
||||
#: named and shaped as creation dates, with no "last edited" field in the
|
||||
#: feed at all to reset from, and the same large real gaps as Greenhouse's.
|
||||
#: - `smartrecruiters` (`releasedDate`), `recruitee` (`published_at`, falling
|
||||
#: back to `created_at`): `releasedDate` reads the same way; a live
|
||||
#: Recruitee board (bunq) showed `published_at` and `created_at` minutes
|
||||
#: apart while `updated_at` had moved days later on an edited posting,
|
||||
#: confirming the field this migration reads is not the one that moves.
|
||||
#: - `workable` (`published`): named as a publish date, not an edit one, but
|
||||
#: only one live board had postings to check (huggingface, 8 roles) and
|
||||
#: none showed a second, "last touched" field to compare against, so this
|
||||
#: is trusted on the name and shape alone.
|
||||
#: - `jobindex`: the RSS `pubDate` a saved search returns. The live
|
||||
#: database's 158 Jobindex-found roles have a `published_at`/`first_seen`
|
||||
#: gap of 0.4 to 31 days (median 12), which reads as real, spread-out
|
||||
#: posting ages rather than a date that jumps to "now" on every re-crawl.
|
||||
#: - `adzuna`: `created`, an aggregator's own record-creation stamp — named
|
||||
#: as creation, not modification, like the rest. No live check: this
|
||||
#: deployment carries no Adzuna key (`plugins.search`'s own docstring:
|
||||
#: "not yet run live") and no Adzuna subscription exists to fetch.
|
||||
#:
|
||||
#: - `teamtailor`: the feed's `pubDate`, and `platsbanken`: `publication_date`,
|
||||
#: both named as publication. Read on live feeds on 2026-09-29, where the
|
||||
#: dates were spread over weeks, not checked against stored roles.
|
||||
#:
|
||||
#: - `workday`: the role's `startDate`, which matched the listing's "Posted N
|
||||
#: Days Ago" on twelve tenants; `successfactors`: the role page's
|
||||
#: `datePosted`; `oracle`: `ExternalPostedStartDate`; `phenom`: the role
|
||||
#: page's JSON-LD `datePosted`, which is earlier than the listing's date
|
||||
#: where the two differ; `bamboohr`: `datePosted`; `jobylon`: the role
|
||||
#: page's JSON-LD date; `rippling`: `createdOn`. Read on live boards on
|
||||
#: 2026-09-29 by the builders of those modules, not checked against stored
|
||||
#: roles. The other hosts added that day (Eightfold, Jibe, Gem, Deel, HiBob,
|
||||
#: HR Manager) state a date nobody checked, and Pinpoint states none.
|
||||
#:
|
||||
#: A host not listed here, or a role whose stored `published_at` fails the
|
||||
#: sanity check in `posted_at`, keeps `first_seen` as its posted date.
|
||||
_UNDECLARED_TRUSTED: frozenset[str] = frozenset({
|
||||
"greenhouse", "lever", "ashby", "workable", "smartrecruiters", "recruitee",
|
||||
"personio", "teamtailor", "jobindex", "adzuna", "platsbanken",
|
||||
"workday", "successfactors", "oracle", "phenom", "bamboohr", "jobylon", "rippling",
|
||||
})
|
||||
|
||||
@lru_cache(maxsize=1)
|
||||
def _declared_posted() -> dict[str, str]:
|
||||
"""`fields.posted` of every reader that declares `READS`, by name. Read
|
||||
@@ -740,13 +683,10 @@ def _declared_posted() -> dict[str, str]:
|
||||
|
||||
def trusted_posted(host: str) -> bool:
|
||||
"""Whether `host` states a date that is safe to read as when a role was
|
||||
posted: its reader declares `fields.posted="trusted"`, or, for a reader
|
||||
without `READS` (or a search provider not yet a reader), it is in
|
||||
`_UNDECLARED_TRUSTED`."""
|
||||
declared = _declared_posted()
|
||||
if host in declared:
|
||||
return declared[host] == "trusted"
|
||||
return host in _UNDECLARED_TRUSTED
|
||||
posted: its reader declares `fields.posted="trusted"`. Which field each
|
||||
host reads, and when it was checked, is in the reader module. A role
|
||||
without a trusted date keeps `first_seen` as its posted date."""
|
||||
return _declared_posted().get(host) == "trusted"
|
||||
|
||||
|
||||
#: A `published_at` before this is a parsing slip or a placeholder, not a
|
||||
|
||||
Reference in new issue
Block a user