Switch the embed model to multilingual-e5-small for the European corpus
Measured on the live roles against the "Copenhagen engineering" agent's intent: the English-only all-MiniLM-L6-v2 separates engineering from non-engineering roles with an AUC of 0.68 overall but only 0.74 restricted to Danish-country roles (a Danish engineering role beats the rest's non-engineering median only 68% of the time). multilingual-e5-small reaches 0.70 overall and 1.00 on Danish-country roles, for about 3x the CPU time. paraphrase-multilingual-MiniLM-L12-v2 measured worse than the English-only baseline (AUC 0.60) and was dropped. e5 needs "query: "/"passage: " prefixes; app/jobs/embed.py adds them by matching the model name so a model change can't leave a stale prefix setting behind. Added tests for the prefixing and for the two invariants the model change relies on: embed drops the old model's vectors before re-embedding, and an agent's cached intent vector is keyed by model so a model change is a cache miss, never a stale-dimension read. Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
This commit is contained in:
1 parent
0558ef19ed
commit
b560cdce2a
7 files changed
+214
-15
No files matched your search
+7
-1
@@ -68,8 +68,14 @@ summarise:
|
||||
# A role and each agent's intent as vectors, for the fit term of a score
|
||||
# (app/jobs/embed.py). Needs the `embeddings` extra; without it `embed` posts
|
||||
# a notice and scores leave fit out. Changing the model re-embeds.
|
||||
# multilingual-e5-small, not English-only MiniLM, because the corpus is now
|
||||
# European: measured against the "Copenhagen engineering" agent's intent, it
|
||||
# separates engineering from non-engineering roles better, especially among
|
||||
# Danish-country roles (see docs/decisions.md), for about 3x the CPU time. An
|
||||
# e5 model's "query: " / "passage: " prefixes are added automatically, keyed
|
||||
# off this name.
|
||||
embed:
|
||||
model: sentence-transformers/all-MiniLM-L6-v2
|
||||
model: intfloat/multilingual-e5-small
|
||||
per_run: 50000
|
||||
|
||||
# One block per plugin, keyed by the plugin's name. The ats and search plugins
|
||||
|
||||
Reference in new issue
Block a user