Require READS, search params and a dated robots record; record robots.txt per requested path
Every reader declares, so undeclared(), the scope() fallback, robots_page and ROBOTS_ANY_CASE go. A search provider needs PARAMS and clean_params, a row may carry company, places gains none. Each robots record names the paths the reader requests, from robots.txt read again on 2026-09-30. Platsbanken states remote, HR Manager states no place. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
4e827665dd
commit
2349fc424b
46 files changed
+323
-100
No files matched your search
@@ -31,7 +31,8 @@ READS = Reads(
|
||||
scope="board", listing="whole", text="listing", complete="whole",
|
||||
places="many", empty="suspect",
|
||||
fields=Fields(pay="figures", posted="trusted", remote="stated", employment="stated"),
|
||||
robots="api: no robots.txt check needed (public board API), 2026-09-30",
|
||||
robots=("read: api.ashbyhq.com answers /robots.txt with 401 and no rules, so "
|
||||
"/posting-api/job-board/<slug> is not disallowed (checked 2026-09-30)"),
|
||||
)
|
||||
|
||||
_API = "api.ashbyhq.com"
|
||||
|
||||
Reference in new issue
Block a user