Require READS, search params and a dated robots record; record robots.txt per requested path

Every reader declares, so undeclared(), the scope() fallback, robots_page and
ROBOTS_ANY_CASE go. A search provider needs PARAMS and clean_params, a row may
carry company, places gains none. Each robots record names the paths the
reader requests, from robots.txt read again on 2026-09-30. Platsbanken states
remote, HR Manager states no place.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
WolfandClaude Fable 5.1 committed 2026-09-30 15:51:13 +02:00
1 parent 4e827665dd
commit 2349fc424b
46 files changed
+323 -100

No files matched your search

+10 -5
View File
@@ -14,9 +14,11 @@ everything for `*` except the application form. The feed is on the API host,
which has no robots.txt (a 404, which allows); it is the interface HR Manager
publishes for readers like this one, and the portal's HTML list, which the
robots.txt does shut out, is never read. The role's link is the application
form the robots.txt names as allowed, and is only shown, not fetched. The
declaration says "disallowed" all the same, and the owner decides whether the
feed is read.
form the robots.txt names as allowed, and is only shown, not fetched.
`READS.robots` records both hosts and the owner's decision.
The feed names no place (the title or the text may), so every row has
`places == []` (`places="none"`).
"""
from __future__ import annotations
@@ -34,9 +36,12 @@ from ._shared import Get, Page, is_web_url, only, parse_dt
READS = Reads(
scope="board", listing="paged", page_size=100, text="listing", complete="end",
places="one",
places="none",
fields=Fields(posted="listed"),
robots="disallowed by robots.txt; owner to decide",
robots=("read: the feed /JobPortal.svc/<customer>/PositionList/rss/ is on "
"api.hr-manager.net, whose /robots.txt answers 404 (checked 2026-09-30); the portal "
"candidate.hr-manager.net disallows all but /ApplicationInit.aspx? and is not "
"read: owner, 2026-09-29"),
)
NAME = "hrmanager"