Accept the full re-judge after a CV or prompt edit
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
9d30582599
commit
5f1d225d21
1 file changed
+1
-1
@@ -86,7 +86,7 @@ Without the Where lines, about four in five model reads go to roles outside DK,
|
||||
|
||||
The form wording for the sizes becomes "at most N, each scoring at least X", in place of "the best N down to a score of X".
|
||||
|
||||
**Cost.** Every model read happens once per role (judge once per role and CV), so a larger cut costs mostly a one-off catch-up. With a cut of about 650 roles, that catch-up is about 16 minutes of extract and about an hour of judge on the live backend (`claude_code`, the subscription, so no per-token bill). After it, each pull reads only the roles new to the cut. The recurring cost is a re-judge: changing the CV, a prompt, or the like or dislike text changes every judgement's input, so the whole cut is judged again, about an hour for 650 roles. A changed posting text re-judges that role only.
|
||||
**Cost.** Every model read happens once per role (judge once per role and CV), so a larger cut costs mostly a one-off catch-up. With a cut of about 650 roles, that catch-up is about 16 minutes of extract and about an hour of judge on the live backend (`claude_code`, the subscription, so no per-token bill). After it, each pull reads only the roles new to the cut. The recurring cost is a re-judge: changing the CV, a prompt, or the like or dislike text changes every judgement's input, so the whole cut is judged again, about an hour for 650 roles. A changed posting text re-judges that role only. Accepted: the re-judge takes a while but is correct, since the old judgements were made against the old text.
|
||||
|
||||
**Decided: extract and judge stay two calls.** Extract needs no CV and its location and remote answers feed the rescore before judge reads anything. Judge works per agent and role, with the CV in the system prompt.
|
||||
|
||||
|
||||
Reference in new issue
Block a user