From 83e965dfeb3d3055543f78e42c5cb74891b9203b Mon Sep 17 00:00:00 2001 From: Wolf Date: Tue, 27 Oct 2026 10:48:18 +0100 Subject: [PATCH] Plan the refresh from the boilerplate and news of October 2026: pipeline, auth, llm and the generic desk modules, deploy and the toolchain, in five packages Co-Authored-By: Claude Fable 5.1 --- docs/plan-upstream-2026-10.md | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) create mode 100644 docs/plan-upstream-2026-10.md diff --git a/docs/plan-upstream-2026-10.md b/docs/plan-upstream-2026-10.md new file mode 100644 index 0000000..23b39dd --- /dev/null +++ b/docs/plan-upstream-2026-10.md @@ -0,0 +1,35 @@ +# Plan: take the boilerplate and news of October 2026 into roledesk + +Planned 2026-10-27. Folded into `decisions.md` and deleted when the packages land. The copies were last refreshed on 2026-09-29 (`pipeline/` at boilerplate a1c6f0d, `deploy/` at 1679e88, `auth/` older still, `desk/` at news a275c38). Since then the boilerplate has 88 commits and news 60 on `desk/`. None of our copies carries a local edit, so each refresh is `git archive` and then the app's adaptation. + +## What changed upstream, and what it means here + +- **pipeline/** (boilerplate 09e144d): the queue is backend only, so `POST /api/jobs` is gone and the page starts a kind through an app route; a task's `settle` and `callback` and what `context.task` names must be registered with `@callback`, since no row may name Python the app did not register; a task's step to the next is durable; a notice raised in an isolated child reaches the queue and its stderr the parent; an isolated child no longer outlives its worker; a claim landing while the queue stops hands its lease back; `/ws/jobs` frames are declared models; `FINAL`, `Tally.over`, `counters`. The table upgrades itself with `ALTER TABLE`, so the live database needs no migration of ours. Our Runs page still loads `` and ``, which the module still ships. +- **auth/** (same): the lockout is claimed before the scrypt, `APP_PASSWORD` recovery closes every session, Starlette's cookie parser, the rightmost forwarded hop in the log, `revoke_all` on a password change, a 421 for a non-loopback `Host` on an open app, API tokens as a third credential (`auth_tokens`, `Authorization: Bearer`), sockets closed with their credential. The app side adds the three hardening headers on every answer (`Hardening` in the boilerplate's `app/main.py`) and the SWAG rate limit on the passkey options route. +- **llm/** (new): the model seam, ported from news's agent tool; news's `desk/tools/agent.py` is now an adapter over it and `desk/modeltask.py` imports `llm`. Backends and names stay (`ask_json`, `backend`, `AgentError`, `Reply`); the per-model thinking rule knows Haiku and Sonnet 5.5. +- **desk/** (news 7727bb3): the generic modules moved on (`modeltask`: a reply the CLI cut off keeps whole items and counts what it lost, failures categorised; `notices`: every code the tree raises is in `BY_CODE`; `tools/agent` over `llm`; `tools/http` owns the one user agent and timeout; `config.AgentConfig.for_llm`). The source contract did not stay generic: `Item` is gone, a plugin yields an `Entry` or an `Article` with news's typed fields, media and recordings. Our two sources yield `Item` and our store reads it. +- **deploy/** (same): the release is split where the sandbox ends (the gate in bash in the distro, the image built and pushed on Windows with `wslc`), `wsl-setup`, `db.sh` reads the service from `release.env`, the image runs as `USER 1000`, pins `python:3.15-slim`, installs its extras, and the server gets a `config.example.yaml`; `backup.sh` asks the app inside the container. The boilerplate's `docs/` moved to `documentation/` (`Install Server.md`, `Install WSL.md`, the two proposals), so our five copied docs have no upstream any more. +- **Toolchain:** Python 3.15, fastapi 0.147, pydantic 2.14, starlette 1.7, uvicorn 0.54, anthropic 1.15, claude-agent-sdk 0.2.165; news runs 3.15 with torch from the CPU index. +- **The page:** the boilerplate and news both serve a React page now; the Jinja templates our Roles page was ported from are gone. Out of scope here; it is the owner's call whether roledesk follows. + +## Decisions taken for this plan + +- `desk/` stops being a whole copy of news. The generic modules (`modeltask`, `notices`, `config`, `tools/agent`, `tools/credentials`, `tools/embedding`, `tools/http`, `tools/failures`, `tools/checks`, `tools/scrape`, `tools/article`, `tools/browser`, `tools/framing`, `tools/marks`) keep news as their upstream and are refreshed file by file; `desk/sources/*` and `desk/models.py` are roledesk's own from now on, since the Plugins ADR defines our contract and news's `Entry` is a news thing. `desk/README.md` says so. +- `llm/` comes in whole from the boilerplate, under the coupling rule like `pipeline/` and `auth/`. The `agent` extra becomes `llm`. +- The model names stay as they are (Haiku 4.5); moving to 5.5 is a separate, measured change. +- The five boilerplate docs in `docs/` (`hosting`, `release`, `server`, `password-protection`, `llm-backends`) go; `docs/Install Server.md` and `docs/Install WSL.md` are copies of the boilerplate's `documentation/` files, and the two proposals are linked from the boilerplate checkout as news does. +- The toolchain step (3.15, the lock) is the planner's, last, on main, since it changes the environment every worktree tests with. + +## Work packages + +Each works in its own worktree made from main, owns the files named and nothing else, runs the full suite before committing, and reports. The planner merges in the order below. + +| WP | Model | Owns | Builds | +|---|---|---|---| +| 1 pipeline | Opus | `pipeline/**`, `app/jobs/cascade.py`, `app/jobs/pull.py`, `app/jobs/fetch_text.py`, `app/web/api.py`, `app/static/jobs.js`, `app/static/app.js`, `app/static/live.js`, `app/templates/settings/sources.html`, `tests/test_web.py`, `tests/test_stages.py`, `tests/test_pull.py`, `tests/test_fetch_text.py`, `tests/test_cli.py` | `pipeline/` from boilerplate HEAD whole; `@callback` on `cascade.step`, `pull.close_boards`, `fetch_text.settle` and whatever else a row names; `POST /api/pull` in `api.py` and the Runs page and `app.js` using it; the `/ws/jobs` frames the page reads checked against `pipeline.web.schemas`; the jobs table of a copy of the live schema upgraded by the new `Jobs` in a test; `pipeline/tests` green here | +| 2 auth | Opus | `auth/**`, `app/main.py`, `app/templates/settings/account.html`, `app/templates/base.html`, `tests/test_app_hardening.py` (new), `tests/test_pages.py` | `auth/` from boilerplate HEAD whole; `connect` call checked against the new signature; `Hardening` headers in `app/main.py`; the Account page showing the token section; `auth/tests` green here | +| 3 llm and desk | Opus, after 1 and 2 merged | `llm/**`, `desk/**`, `app/jobs/stages.py`, `app/jobs/judge.py`, `app/jobs/extract.py`, `app/jobs/summarise.py`, `app/jobs/posting.py`, `app/jobs/embed.py`, `app/config.py`, `app/web/api.py` (the model routes only), `app/jobs/cascade.py` (the `agent.backend` lines only), `pyproject.toml` (packages, extras, testpaths, mypy modules), `config.example.yaml` (`agent:`), `tests/test_architecture.py`, `tests/test_coupling.py`, `tests/test_packaging.py`, `tests/test_stages.py`, `tests/test_judge.py`, `tests/test_extract.py`, `tests/test_summarise.py` | `llm/` from boilerplate HEAD; the generic desk modules from news HEAD; `desk/models.py` and `desk/sources/*` kept, with what `modeltask` and `notices` need from news's `models.py` (`ModelTaskResult`, `JsonDict`) checked; the app's model calls on the new adapter; the `llm` extra; `llm` in the layering tests; `desk/README.md` rewritten; `llm/tests` green here | +| 4 deploy | Sonnet | `deploy/**`, `bin/**`, `docs/hosting.md`, `docs/release.md`, `docs/server.md`, `docs/password-protection.md`, `docs/llm-backends.md`, `docs/Install Server.md`, `docs/Install WSL.md`, `docs/box-docker-compose.yml`, `README.md` (the Docs list and the deploy sentences), `tests/test_deploy.py` (new), `.gitattributes` | `deploy/` from boilerplate HEAD whole, then roledesk's own facts put back: the Dockerfile without the frontend stage and copying `desk`, `plugins`, `llm`, `seed`, with the extras `passkeys`, `embeddings`, `llm`; `release.env.example` with our image name; `server/config.example.yaml` with roledesk's keys; `bin/` without `frontend.sh`; the docs swap; `.cmd` files CRLF | +| 5 planner | Fable | `CLAUDE.md`, `docs/decisions.md`, `docs/open.md`, `.python-version`, `pyproject.toml` (`requires-python`, `[tool.uv.sources]`), `uv.lock` | the merges; Python 3.15 and `uv lock --upgrade` with torch from the CPU index as news has it; `CLAUDE.md`: `llm/` under the copies rule, `desk/` as above, the UI pointer at news's React page; the fold into `decisions.md`; this file deleted | + +WP1, WP2 and WP4 run in parallel. WP3 starts when WP1 and WP2 are on main. The live server is restarted by the owner after the last merge.