Merge sweeps that resume on a tuple cursor, a stalled read counted incomplete, searches closing by age only, and Lever and Oracle failing on a wrong-shaped answer (wp-rev-runner)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
commit
62cf1c738c
11 files changed
+364
-47
No files matched your search
+28
-23
@@ -39,14 +39,16 @@ the roles matching a word, which says nothing of the roles it did not list,
|
||||
so it closes by list nothing: a role of such a board closes once the board
|
||||
has not listed it for `STALE_DAYS`, the rule searches have.
|
||||
|
||||
A search closes the same way when it listed everything that matches (see
|
||||
`plugins/search.py` for how each provider knows). One that did not closes its
|
||||
roles only once no search has seen them for `STALE_DAYS`. A board or a
|
||||
search that has failed on every fetch for `STALE_DAYS` (`ats:failing:<target>`,
|
||||
`search:failing:<target>`) closes all of its roles, and its next answer
|
||||
reopens what it lists. A target that raises in its batch after its read, a
|
||||
row that does not store, is recorded as failing the same way and the batch
|
||||
goes on with the next one (`run_batch`).
|
||||
A search never closes by its list, even one stamped complete because it
|
||||
read as many results as its provider stated: a search is never a company's
|
||||
whole list, so a role that drops out of its results may still be open. Its
|
||||
roles close once no search has listed them for `STALE_DAYS`
|
||||
(`Store.close_unseen`). A board or a search that has failed on every fetch
|
||||
for `STALE_DAYS` (`ats:failing:<target>`, `search:failing:<target>`) closes
|
||||
all of its roles, and its next answer reopens what it lists. A target that
|
||||
raises in its batch after its read, a row that does not store, is recorded
|
||||
as failing the same way and the batch goes on with the next one
|
||||
(`run_batch`).
|
||||
|
||||
The settle says what needs a look in one notice (`settled`): the targets
|
||||
that did not answer or were read incomplete, and the boards that closed more
|
||||
@@ -302,11 +304,12 @@ def close_boards(config: Config, store: Store, tally: Tally, *,
|
||||
had already parsed stamped — see the module docstring. A board whose
|
||||
`ats:ok:<target>` was not written during *this* run — it failed, or its
|
||||
whole batch job never got to it — is never closed by its list, and a board
|
||||
that answered incompletely is not either. A search closes by its list only
|
||||
when that list was complete, and otherwise by `STALE_DAYS`. A board or a
|
||||
search that has failed on every fetch since `STALE_DAYS` ago closes all
|
||||
its roles (`failing_boards`). How a read that answered closes is
|
||||
`_close_read`'s.
|
||||
that answered incompletely is not either. A search never closes by its
|
||||
list, complete or not: its roles close once no search has listed them
|
||||
for `STALE_DAYS` (`Store.close_unseen`, run when a search answered). A
|
||||
board or a search that has failed on every fetch since `STALE_DAYS` ago
|
||||
closes all its roles (`failing_boards`). How a board's read that
|
||||
answered closes is `_close_read`'s.
|
||||
|
||||
The stamps are compared with this run's own marker, the one its batches
|
||||
carried, never with the stored `pull:run`, which a later run may have
|
||||
@@ -350,17 +353,18 @@ def close_boards(config: Config, store: Store, tally: Tally, *,
|
||||
unanswered.append(sub.target)
|
||||
since = failing.get(sub.target, ("", ""))[0]
|
||||
shut = store.close_subscription(sub.id) if since and since < stale else 0
|
||||
elif source == "search":
|
||||
# Never a company's whole list, complete or not: a search's
|
||||
# roles close by age alone (`close_unseen`, after the loop).
|
||||
searched += 1
|
||||
continue
|
||||
else:
|
||||
if source == "search":
|
||||
searched += 1
|
||||
else:
|
||||
answered += 1
|
||||
answered += 1
|
||||
read = _close_read(store, source, sub, stale)
|
||||
if read is None:
|
||||
if source == "ats":
|
||||
incomplete += 1
|
||||
if not store.state.get(f"ats:sweep:{sub.target}"):
|
||||
partial.append(sub.target)
|
||||
incomplete += 1
|
||||
if not store.state.get(f"ats:sweep:{sub.target}"):
|
||||
partial.append(sub.target)
|
||||
continue
|
||||
shut = read
|
||||
closed += shut
|
||||
@@ -380,8 +384,9 @@ def close_boards(config: Config, store: Store, tally: Tally, *,
|
||||
|
||||
|
||||
def _close_read(store: Store, source: str, sub: Any, stale: str) -> int | None:
|
||||
"""Close what one target's read of this run says has closed, and return
|
||||
how many; None when the read was incomplete, which closes nothing.
|
||||
"""Close what one board's read of this run says has closed, and return
|
||||
how many; None when the read was incomplete, which closes nothing. A
|
||||
search's read never comes here (see `close_boards`).
|
||||
|
||||
A read narrowed by terms closes only the roles it has not listed for
|
||||
`STALE_DAYS` (`_close_aged`). A read that finished a sweep closes what
|
||||
|
||||
Reference in new issue
Block a user