Places as a gazetteer, role_places, teaser and three seniority sources
Migration 19 adds places, place_aliases, place_includes (seeded from the dictionaries in app/models.py and app/places.py, Europe as a continent over the European countries) and role_places, one row per place of a role, resolved by app.places.Gazetteer, NULL when unknown. upsert_role writes them from raw["places"], else the place line cut at ";"; the migration builds them from each role's place column. The Place filter and the country and place gates are joins over the tree: country:DK is any place under Denmark, region:skane any town in Skane, a country gate is met by any place in its countries, and the EU fallback of the word matcher is no longer used. A place: token the gazetteer does not know still matches whole words. Store gains gazetteer, role_places, unresolved_places, add_place_alias, add_place and relink_places. roles.teaser is kept-filled and embedded while there is no description; a vector is dropped when a better text arrives. seniority_model, _stated and _title sit behind roles.seniority, which stays the first filled of them. TRUSTED_POSTED_HOSTS gives way to trusted_posted(host), which reads READS.fields.posted and falls back to the old set for undeclared readers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
4c0da58da7
commit
bbd7522163
10 files changed
+1437
-98
No files matched your search
+6
-2
@@ -2,7 +2,10 @@
|
||||
|
||||
The encoder is `desk.tools.embedding`; the table is roledesk's own `vectors`
|
||||
(see `Store.put_vectors`). What gets encoded is the title and the opening of
|
||||
the description, which is what says what a role is.
|
||||
the description, which is what says what a role is; while a role has no
|
||||
description, the teaser its host's list carried, and while it has neither,
|
||||
the title alone. `Store.upsert_role` drops the vector when a better text
|
||||
arrives, so the role is encoded again.
|
||||
|
||||
sentence-transformers is optional, as in newsdesk. Without it this task puts
|
||||
one notice on screen and does nothing; `score` then leaves `fit` out and ranks
|
||||
@@ -55,7 +58,8 @@ def wants_e5_prefix(model: str) -> bool:
|
||||
|
||||
|
||||
def document(role: Role) -> str:
|
||||
return f"{role.title}\n{role.description[:BODY_CHARS]}"
|
||||
"""The title and the description, else the teaser."""
|
||||
return f"{role.title}\n{(role.description or role.teaser)[:BODY_CHARS]}"
|
||||
|
||||
|
||||
def role_text(model: str, role: Role) -> str:
|
||||
|
||||
Reference in new issue
Block a user