Require READS, search params and a dated robots record; record robots.txt per requested path
Every reader declares, so undeclared(), the scope() fallback, robots_page and ROBOTS_ANY_CASE go. A search provider needs PARAMS and clean_params, a row may carry company, places gains none. Each robots record names the paths the reader requests, from robots.txt read again on 2026-09-30. Platsbanken states remote, HR Manager states no place. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
4e827665dd
commit
2349fc424b
46 files changed
+323
-100
No files matched your search
@@ -122,9 +122,8 @@ def test_the_feed_states_no_place_and_the_robots_decision_is_the_owners() -> Non
|
||||
assert all(r["places"] == [] for r in rows)
|
||||
reads = hrmanager.READS
|
||||
assert (reads.listing, reads.page_size, reads.text, reads.places) == (
|
||||
"paged", 100, "listing", "one")
|
||||
assert reads.robots == "disallowed by robots.txt; owner to decide"
|
||||
assert not hasattr(hrmanager, "robots_page")
|
||||
"paged", 100, "listing", "none")
|
||||
assert "api.hr-manager.net" in reads.robots and "owner, 2026-09-29" in reads.robots
|
||||
|
||||
|
||||
# ------------------------------------------------------- slugs and hosts
|
||||
|
||||
Reference in new issue
Block a user