Read big boards within the agents' reach: countries and remote from the enabled agents, Europe for favourites, filtered at the source by six hosts
app/jobs/reach.py derives each board's reach once per pull and the pull hands it to the runner through the batch payload. SmartRecruiters, Eightfold and SuccessFactors filter one country a request, Phenom a list, Workday and Oracle by host ids from facets. Remote beside countries is a series of its own, since two facets intersect. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
3adafd8950
commit
3462a4abe7
17 files changed
+1324
-116
No files matched your search
@@ -30,8 +30,13 @@ module reads, and neither is a translation of a listed role.
|
||||
A path prefix names a brand of a shared site. `<host>/<prefix>/search/` lists
|
||||
only that brand on the server (Universal Robots 30 of Teradyne's 486, MiR 1),
|
||||
so the read is as small as the company; `rows` still keeps only the roles
|
||||
under `/<prefix>/job/`, in case a site does not scope. There is no server-side
|
||||
country filter in this version.
|
||||
under `/<prefix>/job/`, in case a site does not scope.
|
||||
|
||||
A country is filtered on the server by the search's own facet,
|
||||
`optionsFacetsDD_country=DK`, one country a request (`Filters(country="one")`);
|
||||
it worked on the four tenants tried in the review and on Vestas (64 of 668,
|
||||
2026-09-30). A role at several places is listed when one of them is in the
|
||||
country. There is no remote facet.
|
||||
|
||||
A role's own page carries microdata (`itemtype` schema.org JobPosting) with
|
||||
`datePosted` and the address; `detail` reads the description, the date and the
|
||||
@@ -52,7 +57,8 @@ from urllib.parse import urljoin, urlsplit
|
||||
|
||||
from desk.sources import strip_html
|
||||
|
||||
from ._contract import Ask, Fields, Place, Reads
|
||||
from ._contract import Ask, Fields, Filters, Place, Reads
|
||||
from ._countries import NAMES
|
||||
from ._shared import Get, Page, Row, is_web_url, only, parse_dt
|
||||
|
||||
NAME = "successfactors"
|
||||
@@ -79,6 +85,7 @@ KEEP_CASE = True
|
||||
|
||||
READS = Reads(
|
||||
scope="board", listing="paged", text="detail", complete="stated", places="many",
|
||||
filters=Filters(country="one"),
|
||||
fields=Fields(posted="trusted", country="stated"),
|
||||
robots="read: robots.txt allows /search/ and /job/ (checked 2026-09-29)",
|
||||
)
|
||||
@@ -122,52 +129,10 @@ _COUNTRY_LABELS = ("country", "land")
|
||||
_MONTHS = {name: number for number, name in enumerate(
|
||||
("jan", "feb", "mar", "apr", "may", "jun", "jul", "aug", "sep", "oct", "nov", "dec"), 1)}
|
||||
|
||||
#: ISO 3166-1 alpha-2 to the English name, which is what `country_of` reads:
|
||||
#: a bare "DE" or "SE" in a location line is taken for Delaware or nothing. A
|
||||
#: code that is not here stays as the site wrote it.
|
||||
_COUNTRIES = dict(entry.strip().split(" ", 1) for entry in """
|
||||
AD Andorra | AE United Arab Emirates | AF Afghanistan | AG Antigua and Barbuda | AI
|
||||
Anguilla | AL Albania | AM Armenia | AO Angola | AR Argentina | AS American Samoa | AT
|
||||
Austria | AU Australia | AW Aruba | AX Åland Islands | AZ Azerbaijan | BA Bosnia and
|
||||
Herzegovina | BB Barbados | BD Bangladesh | BE Belgium | BF Burkina Faso | BG Bulgaria
|
||||
| BH Bahrain | BI Burundi | BJ Benin | BM Bermuda | BN Brunei | BO Bolivia | BQ
|
||||
Caribbean Netherlands | BR Brazil | BS Bahamas | BT Bhutan | BW Botswana | BY Belarus
|
||||
| BZ Belize | CA Canada | CD Democratic Republic of the Congo | CF Central African
|
||||
Republic | CG Republic of the Congo | CH Switzerland | CI Côte d'Ivoire | CK Cook
|
||||
Islands | CL Chile | CM Cameroon | CN China | CO Colombia | CR Costa Rica | CU Cuba |
|
||||
CV Cabo Verde | CW Curaçao | CY Cyprus | CZ Czechia | DE Germany | DJ Djibouti | DK
|
||||
Denmark | DM Dominica | DO Dominican Republic | DZ Algeria | EC Ecuador | EE Estonia |
|
||||
EG Egypt | ER Eritrea | ES Spain | ET Ethiopia | FI Finland | FJ Fiji | FK Falkland
|
||||
Islands | FM Micronesia | FO Faroe Islands | FR France | GA Gabon | GB United Kingdom
|
||||
| GD Grenada | GE Georgia | GF French Guiana | GG Guernsey | GH Ghana | GI Gibraltar |
|
||||
GL Greenland | GM Gambia | GN Guinea | GP Guadeloupe | GQ Equatorial Guinea | GR
|
||||
Greece | GT Guatemala | GU Guam | GW Guinea-Bissau | GY Guyana | HK Hong Kong | HN
|
||||
Honduras | HR Croatia | HT Haiti | HU Hungary | ID Indonesia | IE Ireland | IL Israel
|
||||
| IM Isle of Man | IN India | IQ Iraq | IR Iran | IS Iceland | IT Italy | JE Jersey |
|
||||
JM Jamaica | JO Jordan | JP Japan | KE Kenya | KG Kyrgyzstan | KH Cambodia | KI
|
||||
Kiribati | KM Comoros | KN Saint Kitts and Nevis | KP North Korea | KR South Korea |
|
||||
KW Kuwait | KY Cayman Islands | KZ Kazakhstan | LA Laos | LB Lebanon | LC Saint Lucia
|
||||
| LI Liechtenstein | LK Sri Lanka | LR Liberia | LS Lesotho | LT Lithuania | LU
|
||||
Luxembourg | LV Latvia | LY Libya | MA Morocco | MC Monaco | MD Moldova | ME
|
||||
Montenegro | MG Madagascar | MH Marshall Islands | MK North Macedonia | ML Mali | MM
|
||||
Myanmar | MN Mongolia | MO Macao | MP Northern Mariana Islands | MQ Martinique | MR
|
||||
Mauritania | MS Montserrat | MT Malta | MU Mauritius | MV Maldives | MW Malawi | MX
|
||||
Mexico | MY Malaysia | MZ Mozambique | NA Namibia | NC New Caledonia | NE Niger | NG
|
||||
Nigeria | NI Nicaragua | NL Netherlands | NO Norway | NP Nepal | NR Nauru | NU Niue |
|
||||
NZ New Zealand | OM Oman | PA Panama | PE Peru | PF French Polynesia | PG Papua New
|
||||
Guinea | PH Philippines | PK Pakistan | PL Poland | PR Puerto Rico | PS Palestine | PT
|
||||
Portugal | PW Palau | PY Paraguay | QA Qatar | RE Réunion | RO Romania | RS Serbia |
|
||||
RU Russia | RW Rwanda | SA Saudi Arabia | SB Solomon Islands | SC Seychelles | SD
|
||||
Sudan | SE Sweden | SG Singapore | SI Slovenia | SK Slovakia | SL Sierra Leone | SM
|
||||
San Marino | SN Senegal | SO Somalia | SR Suriname | SS South Sudan | ST São Tomé and
|
||||
Príncipe | SV El Salvador | SY Syria | SZ Eswatini | TC Turks and Caicos Islands | TD
|
||||
Chad | TG Togo | TH Thailand | TJ Tajikistan | TL Timor-Leste | TM Turkmenistan | TN
|
||||
Tunisia | TO Tonga | TR Turkey | TT Trinidad and Tobago | TV Tuvalu | TW Taiwan | TZ
|
||||
Tanzania | UA Ukraine | UG Uganda | US United States | UY Uruguay | UZ Uzbekistan | VA
|
||||
Vatican City | VC Saint Vincent and the Grenadines | VE Venezuela | VG British Virgin
|
||||
Islands | VI U.S. Virgin Islands | VN Vietnam | VU Vanuatu | WS Samoa | XK Kosovo | YE
|
||||
Yemen | ZA South Africa | ZM Zambia | ZW Zimbabwe
|
||||
""".replace("\n", " ").split(" | "))
|
||||
#: ISO 3166-1 alpha-2 to the English name (`_countries.NAMES`): a bare "DE"
|
||||
#: or "SE" in a location line is taken for Delaware or nothing. A code that
|
||||
#: is not there stays as the site wrote it.
|
||||
_COUNTRIES = NAMES
|
||||
|
||||
|
||||
def _parts(slug: str) -> tuple[str, str]:
|
||||
@@ -190,9 +155,12 @@ def careers_url(slug: str) -> str:
|
||||
return f"https://{host}{prefix}/search/"
|
||||
|
||||
|
||||
def _search_url(slug: str, start: int) -> str:
|
||||
def _search_url(slug: str, start: int, country: str = "") -> str:
|
||||
"""The search page from `start`, in one country when `country` (an ISO
|
||||
code) is given."""
|
||||
host, prefix = _parts(slug)
|
||||
return f"https://{host}{prefix}/search/?q=&locationsearch=&startrow={start}"
|
||||
facet = f"&optionsFacetsDD_country={country}" if re.fullmatch(r"[A-Z]{2}", country) else ""
|
||||
return f"https://{host}{prefix}/search/?q=&locationsearch={facet}&startrow={start}"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ listing
|
||||
@@ -296,12 +264,13 @@ def _entries(text: str) -> list[dict[str, Any]]:
|
||||
|
||||
|
||||
def page(get: Get, slug: str, cursor: Any, ask: Ask = Ask()) -> Page:
|
||||
"""One search page. The cursor is `(start row, first role of the page
|
||||
"""One search page, in the one country `ask` names. The cursor is `(start row, first role of the page
|
||||
before)`: the next start row is what this page returned, and a page that
|
||||
starts with the same role as the last one is the site ignoring `startrow`,
|
||||
which ends the read (incomplete) instead of looping."""
|
||||
start, previous = cursor or (0, None)
|
||||
text = get(_search_url(slug, start)).decode("utf-8", "replace")
|
||||
country = ask.countries[0] if ask.countries else ""
|
||||
text = get(_search_url(slug, start, country)).decode("utf-8", "replace")
|
||||
if not _SEARCH_PAGE.search(text):
|
||||
raise ValueError("not a SuccessFactors search page")
|
||||
found = _entries(text)
|
||||
|
||||
Reference in new issue
Block a user