Refresh deploy/ and bin/ from the boilerplate at 09e144d (the release split between the distro and Windows, wsl-setup, the server config, USER 1000, python:3.15-slim) with roledesk's facts put back: no frontend stage, desk, plugins and seed copied, the extras passkeys, embeddings and llm, roles as the table the backup and the importer require, bin/pull.sh and bin/worker.sh with their profile commands, and Install Server.md and Install WSL.md as copies in place of the five old deployment docs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
83e965dfeb
commit
a1f281a23a
40 files changed
+2226
-1287
No files matched your search
@@ -0,0 +1,213 @@
|
||||
> A copy of the boilerplate's `documentation/Install Server.md` at commit 09e144d, unedited below this line; the fix lands upstream first.
|
||||
|
||||
# Deploying a service
|
||||
|
||||
`<app>` below is your *service's* name: the hostname, the data directory, the image tag, the compose service. It is the one thing you choose. The Python package stays `app` and the environment variables stay `APP_`, here and in every project started from this repository — nothing inside the checkout is renamed, and the files these steps need are already in place under `deploy/`.
|
||||
|
||||
## What the box is
|
||||
|
||||
A single Hetzner server running Ubuntu 24.04 on x86_64, with a 37 GB root filesystem and no separate data disk. One Docker Compose project holds everything.
|
||||
|
||||
/srv/swag/docker-compose.yml every service, one file, project name `swag`
|
||||
/srv/swag/.env every secret, chmod 600, never in the compose file
|
||||
/srv/data/<app>/ every service's data, owned 1000:1000
|
||||
/etc/config/swag/ SWAG's /config: nginx confs, certificates, logs
|
||||
|
||||
A snapshot of that whole compose file, every neighbour included, is [example-docker-compose.yml](example-docker-compose.yml): what a new service block lands beside, and where the SWAG subdomain list lives.
|
||||
|
||||
`swag` is nginx, certbot and fail2ban in one container. It terminates TLS for every hostname and proxies to the others **by container name** over the project's default network. That shared network is the only reason all these services live in one compose file — nothing else about them is related.
|
||||
|
||||
Images are published to the Gitea instance on the same box, at `git.rittershofer.org/wolf/<app>`. Everything runs as uid/gid 1000 and `TZ=Europe/Brussels`.
|
||||
|
||||
Which compose file that is comes off the running container rather than out of memory:
|
||||
|
||||
```bash
|
||||
docker inspect swag --format '{{index .Config.Labels "com.docker.compose.project.config_files"}}'
|
||||
```
|
||||
|
||||
Declaring the network `external` from a second Compose project would work too, and would hardcode the first project's name into the second — a rename away from a confusing failure.
|
||||
|
||||
## The seven steps
|
||||
|
||||
**1. DNS.** `<app>.rittershofer.org` must resolve to the server *before* anything else. Confirm with `getent hosts <app>.rittershofer.org`.
|
||||
|
||||
**2. Certificate.** Add `<app>` to `SUBDOMAINS` in the swag service's environment (docker compose yaml). That list separates on commas and nothing else. SWAG runs certbot on every container start and requests **one certificate covering every name in the list**. If SWAG already holds a wildcard certificate covering the name, reuse it and skip this step entirely.
|
||||
|
||||
**3. Registry login.** On the build machine a Gitea token with package **write**; on the server, package **read**. Same command both ends, token at the password prompt:
|
||||
|
||||
```bash
|
||||
docker login git.rittershofer.org -u wolf
|
||||
```
|
||||
|
||||
Git credentials and Docker registry credentials are separate. A
|
||||
|
||||
**4. Data directory.** Root's `mkdir` produces a root-owned directory, and the container runs as 1000, so the chown is not optional:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /srv/data/<app>/home && sudo chown -R 1000:1000 /srv/data/<app>
|
||||
```
|
||||
|
||||
**5. Service block** from [../deploy/server/compose.yaml](../deploy/server/compose.yaml) into `/srv/swag/docker-compose.yml`, the variables from [../deploy/server/env.example](../deploy/server/env.example) appended to `/srv/swag/.env`, and [../deploy/server/backup.sh](../deploy/server/backup.sh) next to the compose file for the nightly cron. Copy [../deploy/server/config.example.yaml](../deploy/server/config.example.yaml) to `$APP_DATA/config.yaml` with `rp_id` and `origin` set to the hostname: it holds the keys whose defaults are wrong in a container (`server.host: 0.0.0.0`, `database: /data/app.db`, `auth.enabled: always`, `auth.secure_cookie: always`). Without it the app binds loopback inside the container and writes its database into the image's root-owned folder.
|
||||
|
||||
**6. Proxy conf** — [../deploy/server/swag/app.subdomain.conf](../deploy/server/swag/app.subdomain.conf) renamed to `<app>.subdomain.conf` in `/etc/config/swag/nginx/proxy-confs/`. Then validate, and reload only if validation passes:
|
||||
|
||||
```bash
|
||||
docker exec swag nginx -t && docker exec swag nginx -s reload
|
||||
```
|
||||
|
||||
Confirm the other sites still respond afterwards.
|
||||
|
||||
**7. Deploy.** Pull first, since `up` alone keeps the `:latest` it already has, and name the service, so nothing else in the file is recreated:
|
||||
|
||||
```bash
|
||||
cd /srv/swag && docker compose pull <app> && docker compose up -d <app>
|
||||
```
|
||||
|
||||
[../deploy/server/deploy.sh](../deploy/server/deploy.sh) is the same two commands with a check that `config.yaml` exists and sets `database` and `auth.enabled`, and a wait for the health check before it returns.
|
||||
|
||||
## What the templates already decide for you
|
||||
|
||||
Both templates are in [../deploy/server/](../deploy/server), commented in place. The decisions worth knowing before you read them:
|
||||
|
||||
**No `ports:`.** SWAG reaches the container by name on the project network. That is what keeps the app's port off the internet even if the firewall is wrong.
|
||||
|
||||
**No `networks:` either.** Being in that compose file is what puts it on the right one.
|
||||
|
||||
**`user: "1000:1000"`.** Container uid is host uid — there is no translation layer. Without this the process runs as whatever the image declares, usually root, and everything it writes into `/srv/data` lands root-owned.
|
||||
|
||||
**`mem_limit:`, not `deploy: resources: limits:`.** The latter is Swarm-only and silently ignored by `docker compose up`. 0 is unlimited, which is the honest default until the box has been watched under real load.
|
||||
|
||||
**`logging:` with a size cap.** Without it, json-file logs grow until the disk does not.
|
||||
|
||||
**`listen 443 ssl` without `http2`.** `http2 on;` is set once at `http` scope in `nginx.conf`, and the `listen ... http2` parameter is deprecated in nginx 1.25 and later.
|
||||
|
||||
**`proxy_read_timeout 86400s`** only if the app holds a WebSocket open. SWAG's `proxy.conf` already sets the Upgrade headers; what it also sets is a read timeout of a couple of minutes, which closes an idle socket on the minute. Delete those two lines if there is no socket.
|
||||
|
||||
# Publishing a release
|
||||
|
||||
The scripts are in place in `deploy/`, with `deploy/server/` beside them. The Dockerfile at `deploy/Dockerfile`, which is where `2-build-image.ps1` looks for it in the tree step 1 exports.
|
||||
|
||||
The release is split where the machines are. The checkout, its `.venv` and the tests are Linux, so the gate, step 1, is bash and runs inside the WSL distro. The distro is a sealed sandbox (Windows interop off, no drive mounts, no container engine, see [Install WSL.md, 7. Cut the distro off from Windows](Install%20WSL.md#7-cut-the-distro-off-from-windows)) and stays that way, because a sandbox that can drive containers can mount the host. Step 1 ends by exporting the production tree to `release/<folder>/<sha>/` beside the checkout, and the image is built and pushed from Windows with `wslc`, the container CLI that ships with WSL: steps 2 and 3, two short PowerShell scripts that read that tree through `\\wsl.localhost`. The distro never holds a container socket.
|
||||
|
||||
## Every release
|
||||
|
||||
From Windows, in PowerShell, once `deploy\wsl-setup.cmd` has installed the profile commands:
|
||||
|
||||
```powershell
|
||||
<folder>-0release
|
||||
```
|
||||
|
||||
where `<folder>` is the checkout folder's name (`boilerplate` here). It runs `<folder>-1test`, `<folder>-2build` and `<folder>-3push` in order and stops at the first that fails; its arguments (`--push-branch`, `--no-tests`) go to step 1. Each step is a command of its own:
|
||||
|
||||
```text
|
||||
<folder>-1test bash deploy/1-test-and-promote.sh in WSL: test local main, fast-forward local production, export it
|
||||
<folder>-2build deploy\2-build-image.ps1 on Windows: build the exported tree with wslc
|
||||
<folder>-3push deploy\3-push-image.ps1 on Windows: push the built image to the registry
|
||||
```
|
||||
|
||||
The number is the order. Each script can be run on its own and works out its paths for itself, so the working directory does not matter.
|
||||
|
||||
**Step 1 never builds; step 2 never pushes; step 3 never rebuilds or deploys.** That separation is the point of the numbering: a failed push is `3` again, not the whole thing again.
|
||||
|
||||
## What each step is actually doing
|
||||
|
||||
**1: test and promote.** Resolves `main` to a commit, checks that commit out into a throwaway worktree, and runs the project's checks there. The worktree is what makes this honest: the tests run against exactly the tree that is about to become production, with none of the working copy's uncommitted edits in the way. Worktrees have no `.venv` of their own, so the interpreter comes from the main checkout and the code from the worktree.
|
||||
|
||||
The checks are pytest, then `npm ci` and `npm run check` in `frontend/`: lint, format, the generated types, the page's tests and a build. So the machine that runs step 1 needs Node, which `frontend/install.sh` installs once. Step 1 runs the gates through `deploy/test-commit.sh`, which looks for Node from nvm and from fnm, since a non-interactive shell has not read the rc file that puts it on `PATH`. `PYTHON` in `release.env` names the interpreter when it is not `.venv/bin/python`; `--no-tests` skips the gates.
|
||||
|
||||
Promotion is a fast-forward only. If `production` has commits `main` does not, that is a release made from somewhere else, and the script stops rather than discarding it.
|
||||
|
||||
Git publication is optional: run `git push origin production` yourself, or pass `--push-branch` to step 1 or `<folder>-0release`. `production` identifies the tested candidate; the server's running image identifies the deployed release.
|
||||
|
||||
Step 1 then exports the production commit: `git archive` unpacked into `release/<folder>/<sha>/` beside the checkout (the environment variable `RELEASE_DIR` names another folder for `release/`), with `deploy/Dockerfile.dockerignore` copied to `.dockerignore` at its root, and the short sha written to `release/<folder>/production`. Older trees are removed; a tree that already exists for this sha is kept, because `wslc` can build from a stale view of a folder deleted and recreated at the same path ([microsoft/WSL#41287](https://github.com/microsoft/WSL/issues/41287)). When the folder cannot be written, step 1 says so and fails after the promote; run it again.
|
||||
|
||||
**2: build.** Reads the sha from `production` and runs `wslc build -t <IMAGE>:<sha> -t <IMAGE>:latest -f <tree>\deploy\Dockerfile <tree>` with the tree's `\\wsl.localhost` path. `-f` is a path `wslc` opens on Windows and hands to the builder on stdin, which is why the ignore rules have to be at the root of the context. The image builds the React page itself, in a Node stage from `frontend/` and its lock file, so neither Node nor a built `app/web/dist/` is needed for this step.
|
||||
|
||||
`wslc` builds an **image**. No container starts or restarts on this machine.
|
||||
|
||||
**3: push.** Re-tags `latest` onto this candidate with `wslc tag`, then `wslc push`es both. It never rebuilds, so a push that failed on connectivity is just this step again. A refused push prints the `wslc login` line to use.
|
||||
|
||||
# Proposal - Operating it
|
||||
|
||||
[../deploy/server/backup.sh](../deploy/server/backup.sh), [../deploy/server/import.sh](../deploy/server/import.sh) and [../deploy/server/deploy.sh](../deploy/server/deploy.sh) name the compose service in `SERVICE=` at their top, and `NAME` in `deploy/release.env` names it for [../deploy/db.sh](../deploy/db.sh). Each has to be `<app>`. All of them read `APP_DATA` from `/srv/swag/.env`.
|
||||
|
||||
## The nightly backup
|
||||
|
||||
`backup.sh` beside the compose file takes one snapshot through the running container. Inside it, the app's own config names the database, which is opened read-only, so a path that names no file is an error rather than a new empty database. `VACUUM INTO` writes `/data/backups/<app>-<stamp>.db` as one consistent, compacted file without blocking a writer for long. The snapshot is reopened read-only and must hold every table in `TABLES` (`jobs ticks`; a test keeps it equal to `REQUIRED_TABLES` in [../deploy/server/import_database.py](../deploy/server/import_database.py)); otherwise it is deleted and the run fails. A project that replaces `ticks` with its own first table changes both. The snapshot is gzipped under `umask 077`, so every backup is owner-only, and everything past the newest `KEEP` (14 by default) is deleted.
|
||||
|
||||
Schedule it in a crontab that can run `docker compose`:
|
||||
|
||||
```
|
||||
0 4 * * * bash /srv/swag/backup.sh >> /srv/swag/backup.log 2>&1
|
||||
```
|
||||
|
||||
The container has to be running, since the snapshot is taken through `docker compose exec`. A backup on the same disk is not an off-server copy; `db.sh pull` below makes one.
|
||||
|
||||
## Moving the database between laptop and server
|
||||
|
||||
`deploy/db.sh` runs in the checkout, with the `ssh` and `scp` of that shell, and reads `SSH_HOST`, `REMOTE_DIR` (`/srv/swag`), `REMOTE_DATA` (`/srv/data/<app>`) and `NAME` from `deploy/release.env`.
|
||||
|
||||
`bash deploy/db.sh pull` runs the server's own `backup.sh` over ssh and downloads the newest `<app>-*.db.gz` into `backups/` in the checkout. Use it as a read-only copy. Pushing it back after the server has written discards those writes, and nothing merges two copies.
|
||||
|
||||
Pushing the laptop's database onto the server replaces the server's, including the password hash, sessions, passkeys and API tokens, because the `auth_*` tables live in the same file. Set the account up against the final database, not before the push.
|
||||
|
||||
**1. Stop the app.** The import is a file swap under SQLite and `import.sh` refuses a running service:
|
||||
|
||||
```bash
|
||||
cd /srv/swag && docker compose stop <app>
|
||||
```
|
||||
|
||||
**2. Push.** From the checkout:
|
||||
|
||||
```bash
|
||||
bash deploy/db.sh push
|
||||
```
|
||||
|
||||
It snapshots the local database (the one the checkout's config names, or `LOCAL_DB`) with `VACUUM INTO`, gzips it, asks before going on unless given `--yes`, uploads it as `$REMOTE_DATA/incoming.db.gz` and `import_database.py` into `$REMOTE_DIR`, and streams `import.sh` over ssh. On the server, `import.sh` runs `import_database.py` in a one-off container of the image already there (`--pull never`). The importer decompresses into a temporary file beside the database and requires `PRAGMA integrity_check` to answer `ok` and every table in `REQUIRED_TABLES` to exist. Only then does it snapshot the current database with `VACUUM INTO` into `/data/backups/<database name>-<stamp>.db.gz`, owner-only, refuse if `wal_checkpoint(TRUNCATE)` finds another connection, delete the `-wal` and `-shm` files and move the new file into place with one `os.replace`. It prints the snapshot's path. For `/data/app.db` the snapshot is `app-<stamp>.db.gz`, which `backup.sh` prunes with its own only while the service is also called `app`.
|
||||
|
||||
**3. Start the app.**
|
||||
|
||||
```bash
|
||||
cd /srv/swag && docker compose up -d --wait <app>
|
||||
```
|
||||
|
||||
## Restoring a backup on the server
|
||||
|
||||
The same import, by hand, with a file from `$APP_DATA/backups` in place of the pushed one. `import.sh` and `import_database.py` from `deploy/server/` have to be beside the compose file; `db.sh push` leaves the importer there but streams `import.sh`.
|
||||
|
||||
**1. Stop the app.** `cd /srv/swag && docker compose stop <app>`
|
||||
|
||||
**2. Stage the backup** as the incoming file:
|
||||
|
||||
```bash
|
||||
sudo -u '#1000' cp /srv/data/<app>/backups/<app>-<stamp>.db.gz /srv/data/<app>/incoming.db.gz
|
||||
```
|
||||
|
||||
The backups are owner-only and belong to uid 1000, which the importer runs as; a copy made as root is one it cannot read.
|
||||
|
||||
**3. Import.** `bash /srv/swag/import.sh /srv/swag` checks it, snapshots the current database and replaces it, as in step 2 above.
|
||||
|
||||
**4. Start the app.** `cd /srv/swag && docker compose up -d --wait <app>`
|
||||
|
||||
## Rolling back to a pinned image
|
||||
|
||||
Every release pushes `<IMAGE>:<sha>` and `<IMAGE>:latest`, where `<sha>` is the short hash of the promoted commit. `latest` moves on every release, so it cannot name an earlier one; the compose file runs `${APP_IMAGE}:${APP_TAG:-latest}`, and `APP_TAG` in `.env` pins a commit tag.
|
||||
|
||||
**1. Find the tag.** The image's package page in Gitea lists every pushed tag with its date. A tag removed by the package cleanup rule cannot be rolled back to.
|
||||
|
||||
**2. Pin it.** Set `APP_TAG=<sha>` in `/srv/swag/.env`.
|
||||
|
||||
**3. Deploy.** `bash /srv/swag/deploy.sh` pulls the pinned image, which the weekly prune removes from the box once no container runs it, and recreates `<app>` with it.
|
||||
|
||||
**4. Unpin** with `APP_TAG=latest` and deploy again once a fixed release is out.
|
||||
|
||||
A rollback replaces the code, not the database. The queue's tables only ever gain columns, each nullable or with a default; `pipeline/table.py` adds them and never drops one. A project whose migration changed the schema in a way the older code cannot read restores a backup taken before that release, with the app stopped, as above.
|
||||
|
||||
# Pitfalls
|
||||
|
||||
**Disk.** 37 GB, and it has been full. Four causes, all recurring. `:latest` orphans the previous image on every pull and nothing cleans them up; root's crontab runs `docker image prune -af` every Sunday at 04:00, and the `-a` also removes a pinned image that is not running, so a rollback tag must be pulled again. Gitea's package store under `/srv/data/gitea/gitea/packages` keeps every pushed `:sha` and `:latest` version of every image forever: it was 11 GB on 16 October 2026 after a week of newsdesk releases at 2.6 GB each, and the prune cron never touches it. A cleanup rule per owner at `/user/settings/packages/rules` (keep the most recent n, **container** type) is applied by Gitea's `cleanup_packages` cron at midnight, which also drops blobs unreferenced for 24 hours; `/-/admin/packages` lists every version with a delete button and a Cleanup expired data button runs the same job now. Journald defaults to 10% of the filesystem and is capped here at 200 MB. VS Code Remote installs a fresh server per version under `~/.vscode-server/cli/servers` forever. [disk_usage.sh](disk_usage.sh) prints all of it, read-only.
|
||||
|
||||
**SQLite is never safe to copy live.** Stop the container, or snapshot with `VACUUM INTO` through the running one. A `cp` of a WAL-mode database is a coin toss.
|
||||
|
||||
**Secrets in `.env`, never in the compose file.** It is easy to paste a compose file into a chat window; it is harder to do that by accident with a file you had to `cat` deliberately.
|
||||
|
||||
@@ -0,0 +1,408 @@
|
||||
> A copy of the boilerplate's `documentation/Install WSL.md` at commit 09e144d, unedited below this line; the fix lands upstream first.
|
||||
|
||||
# Abstract
|
||||
|
||||
The sandbox is a WSL distro. Claude Code runs inside it from the desktop app with all permissions, because the distro cannot reach anything on Windows. Interops is turned off, no drives are mounted, and to keep it this way admin permissions are not granted.
|
||||
|
||||
Root is reached only from Windows, with `wsl -d claudebox -u root`. That is how you install and update packages later.
|
||||
|
||||
The repos live inside the distro in `/home/claude/cboxcode`. [Working from Windows](#working-from-windows) covers working with them from Windows.
|
||||
|
||||
# Install
|
||||
|
||||
## 1. Check that WSL 2 runs
|
||||
|
||||
```
|
||||
PS> wsl --status
|
||||
```
|
||||
|
||||
It should say `Default Version: 2` and nothing about virtualisation.
|
||||
|
||||
If it says virtualisation is not enabled, run `wsl.exe --install --no-distribution` once in an admin PowerShell and reboot. If it still complains, switch on Intel VT-x or AMD-V in the BIOS. This is the only step that can need admin rights.
|
||||
|
||||
If a `claudebox` from an earlier attempt exists, `wsl -l -v` lists it. `wsl --unregister claudebox` deletes it with everything inside.
|
||||
|
||||
## 2. Install Ubuntu as `claudebox`
|
||||
|
||||
```
|
||||
PS> wsl --install Ubuntu-24.04 --name claudebox --no-launch --web-download
|
||||
```
|
||||
|
||||
- `--name claudebox` names the distro, so it does not collide with an ordinary Ubuntu.
|
||||
- `--no-launch` skips Ubuntu's first-start wizard, which would create a user with sudo. We create our own user in step 5.
|
||||
- `--web-download` downloads it directly instead of through the Microsoft Store.
|
||||
|
||||
## 3. Open a root shell
|
||||
|
||||
```
|
||||
PS> wsl -d claudebox -u root
|
||||
```
|
||||
|
||||
The prompt changes to something like `root@PC:/mnt/c/...#`. You are now inside Linux as root. `C:` is still visible at this point; step 7 takes it away.
|
||||
|
||||
## 4. Update and install packages
|
||||
|
||||
```
|
||||
# apt update
|
||||
# apt upgrade -y
|
||||
# apt install -y build-essential git curl ripgrep sqlite3 jq unzip
|
||||
```
|
||||
|
||||
`apt update` fetches the package lists, `apt upgrade` installs updates, and `apt install` adds the tools: compilers for Python packages that build from source, git, curl, ripgrep for Claude's searches, sqlite3 to look into the databases, jq for JSON, unzip.
|
||||
|
||||
## 5. Create the user `claude`, without sudo
|
||||
|
||||
```
|
||||
# useradd --create-home --shell /bin/bash claude
|
||||
# groups claude
|
||||
```
|
||||
|
||||
`useradd` creates the user with a home directory and bash. It gets no password and no groups. `groups claude` must print only `claude : claude`. If `sudo` appears there, the user can become root, and `gpasswd -d claude sudo` removes it.
|
||||
|
||||
```
|
||||
# passwd -l root
|
||||
```
|
||||
|
||||
This locks root's password, so `su` cannot become root from inside either. Ubuntu ships it locked, so this only makes sure.
|
||||
|
||||
## 6. Install uv and Claude Code as `claude`
|
||||
|
||||
Switch to the user. Root can do that without a password:
|
||||
|
||||
```
|
||||
# su - claude
|
||||
```
|
||||
|
||||
The prompt ends in `$` now. Install the two tools into `~/.local/bin`, which belongs to `claude`, so they update without root:
|
||||
|
||||
```
|
||||
$ curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
$ curl -fsSL https://claude.ai/install.sh | bash
|
||||
$ echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
|
||||
```
|
||||
|
||||
The third line puts `~/.local/bin` on the path for every new shell.
|
||||
|
||||
Set the git identity, with your own name and email, and create the folder for the repos:
|
||||
|
||||
```
|
||||
$ git config --global user.name "Your Name"
|
||||
$ git config --global user.email "you@example.com"
|
||||
$ git config --global init.defaultBranch main
|
||||
$ mkdir ~/cboxcode
|
||||
```
|
||||
|
||||
Back to root:
|
||||
|
||||
```
|
||||
$ exit
|
||||
```
|
||||
|
||||
## 7. Cut the distro off from Windows
|
||||
|
||||
`/etc/wsl.conf` is the file WSL reads when the distro starts. Open it in the editor:
|
||||
|
||||
```
|
||||
# nano /etc/wsl.conf
|
||||
```
|
||||
|
||||
Delete what is there and type this:
|
||||
|
||||
```
|
||||
[boot]
|
||||
systemd=true
|
||||
|
||||
[user]
|
||||
default=claude
|
||||
|
||||
[automount]
|
||||
enabled=false
|
||||
mountFsTab=false
|
||||
|
||||
[interop]
|
||||
enabled=false
|
||||
appendWindowsPath=false
|
||||
```
|
||||
|
||||
- `[user] default=claude` makes every normal start log in as `claude`, not root.
|
||||
- `[automount] enabled=false` stops WSL from mounting `C:` and the other drives under `/mnt`. `mountFsTab=false` also ignores `/etc/fstab`, so no other way of mounting them applies.
|
||||
- `[interop] enabled=false` stops Linux from starting Windows programs. That includes `wsl.exe`, which would otherwise be a way back to root. `appendWindowsPath=false` keeps the Windows folders off the Linux path.
|
||||
- `[boot] systemd=true` is Ubuntu's default and stays.
|
||||
|
||||
Save with `Ctrl+O` then `Enter`, quit with `Ctrl+X`.
|
||||
|
||||
The first start in step 3 left an empty folder per Windows drive under `/mnt`. Nothing will be mounted on them any more, so remove them to avoid confusion. `rmdir` only removes empty folders, so it cannot delete anything:
|
||||
|
||||
```
|
||||
# rmdir /mnt/c /mnt/d
|
||||
```
|
||||
|
||||
Name every drive letter that `ls /mnt` shows. `wsl` and `wslg` belong to WSL and stay. Leave the distro:
|
||||
|
||||
```
|
||||
# exit
|
||||
```
|
||||
|
||||
WSL only reads the file on a fresh start, so stop the distro:
|
||||
|
||||
```
|
||||
PS> wsl --terminate claudebox
|
||||
```
|
||||
|
||||
## 8. Check the isolation
|
||||
|
||||
Start it the normal way, which now logs in as `claude`:
|
||||
|
||||
```
|
||||
PS> wsl -d claudebox
|
||||
```
|
||||
|
||||
Each check and what it must answer:
|
||||
|
||||
```
|
||||
$ whoami
|
||||
claude
|
||||
$ groups
|
||||
claude
|
||||
$ passwd -S claude
|
||||
claude L ...
|
||||
$ sudo -n true
|
||||
sudo: a password is required
|
||||
$ ls /mnt
|
||||
wsl wslg
|
||||
$ uv --version
|
||||
$ claude --version
|
||||
```
|
||||
|
||||
- `groups` showing only `claude` means no admin group, so sudo refuses even with a password.
|
||||
- `L` in `passwd -S` means the password is locked: no password exists, so nothing can be typed at a sudo or su prompt that works. A plain `sudo` still asks for one, and any answer fails.
|
||||
- `ls /mnt` showing no drive letters means no Windows drive is there. `mount | grep -i drvfs` printing nothing confirms it.
|
||||
- The last two print version numbers.
|
||||
|
||||
The real test that Windows programs cannot start needs one. In a second PowerShell window, copy a harmless one in from Windows, run it inside, and delete it again:
|
||||
|
||||
```
|
||||
PS> Copy-Item C:\Windows\System32\whoami.exe \\wsl.localhost\claudebox\tmp\
|
||||
PS> wsl -d claudebox -- /tmp/whoami.exe
|
||||
PS> Remove-Item \\wsl.localhost\claudebox\tmp\whoami.exe
|
||||
```
|
||||
|
||||
With interop off it prints no Windows user name, only a WSL error after a few seconds. If it prints your Windows name, step 7 did not take.
|
||||
|
||||
If any check answers differently, the step it belongs to did not take: 5 for `groups` and `passwd`, 7 for `/mnt` and `whoami.exe`.
|
||||
|
||||
```
|
||||
$ exit
|
||||
```
|
||||
|
||||
## PowerShell 7
|
||||
|
||||
Our scripts are running on powershell 7, which needs to be installed.
|
||||
|
||||
Then, set it to the default profile in the terminal app.
|
||||
|
||||
```
|
||||
PS> winget install Microsoft.PowerShell
|
||||
```
|
||||
|
||||
## The execution policy
|
||||
|
||||
Running commands in the WSL need extra permissions:
|
||||
|
||||
```
|
||||
PS> pwsh -ExecutionPolicy Bypass -File <command>
|
||||
```
|
||||
|
||||
This is one reason why we add all important commands into the profile
|
||||
|
||||
|
||||
|
||||
## PyCharm
|
||||
|
||||
### Remote execution agent failure
|
||||
|
||||
PyCharm on Windows is not able to read folders on the cut-off WSL. Its bundled plugin Remote Execution Agent copies a helper called IJent into the distro from `/mnt/c/Program Files/JetBrains/...` and reads files and runs Git through it. With the drives unmounted the copy fails: opening a file reports "unable to locate", Git reports `execvpe(.../ttyfix) failed`, and `idea.log` shows `cp: cannot stat '/mnt/c/...ijent...'`.
|
||||
|
||||
So you need to disable the plugin: Settings > Plugins > Installed, search for `Remote Execution Agent`, untick it, restart PyCharm.
|
||||
|
||||
### Git
|
||||
|
||||
Set Settings > Version Control > Git > Path to Git executable to `C:\Program Files\Git\cmd\git.exe`, so it uses Windows Git instead of trying to start the distro's.
|
||||
|
||||
### Run configurations
|
||||
|
||||
We currently don't use them.
|
||||
|
||||
## Maintenance
|
||||
|
||||
### Update
|
||||
|
||||
Packages, as root from Windows:
|
||||
|
||||
```
|
||||
PS> wsl -d claudebox -u root -- apt update
|
||||
PS> wsl -d claudebox -u root -- apt upgrade -y
|
||||
```
|
||||
|
||||
Claude Code updates itself. uv updates with the line below. The full path is needed because a command passed from PowerShell does not load `~/.bashrc`:
|
||||
|
||||
```
|
||||
PS> wsl -d claudebox -- /home/claude/.local/bin/uv self update
|
||||
```
|
||||
|
||||
### Remove WSL
|
||||
|
||||
Remove the distro with all repos in it, after pushing what matters:
|
||||
|
||||
```
|
||||
PS> wsl --unregister claudebox
|
||||
```
|
||||
|
||||
# Getting it ready for use
|
||||
|
||||
## Git
|
||||
|
||||
Tell Windows Git to trust that folder, since its files belong to the Linux user:
|
||||
|
||||
```
|
||||
PS> git config --global --add safe.directory '%(prefix)///wsl.localhost/claudebox/home/claude/cboxcode/news'
|
||||
```
|
||||
|
||||
`git config core.autocrlf false` stops Git from converting line endings in this repo. `git checkout -- .` rewrites the files Windows Git checked out with Windows line endings, so Linux Git does not report every line as changed.
|
||||
|
||||
## Working from Windows
|
||||
|
||||
The only direction that works is Windows into WSL: Windows reads and writes the checkout through `\\wsl.localhost\<distro>\...` and starts processes with `wsl.exe`. The distro never reaches back. Editing and the Git UI happen on Windows; the agents, the tests, `serve`, the jobs, `uv sync`, release step 1 and `db.sh` run in the distro, typed from Windows through the profile commands below; release steps 2 and 3 run on Windows with `wslc`, since the distro must not hold a container socket.
|
||||
|
||||
A `.venv` belongs to one OS: `bin/python` on Linux, `Scripts\python.exe` on Windows. The `.venv` in the checkout is the distro's. A Windows `uv sync` or `uv run` in that folder would replace it with a Windows one, and the distro's next `uv run` would swap it back, so run `uv` in the distro (`<folder>-sh` opens a shell there). A Windows interpreter for an editor's code insight belongs outside the checkout.
|
||||
|
||||
Run the app in the distro, not from Windows. Only one side may open a SQLite file at a time, since locking through the share is unreliable, and file change notifications through the share are unreliable too, so `--reload` started from Windows can miss edits.
|
||||
|
||||
## WSL Containers for the image
|
||||
|
||||
We don't need Docker Desktop but build images directly on WSL with `wslc`. It needs WSL 2.9.3 or later:
|
||||
|
||||
```
|
||||
PS> wsl --update
|
||||
PS> wsl --version
|
||||
PS> wslc version
|
||||
```
|
||||
|
||||
Log it in to the registry once, with a token that has package-write access, typed at its password prompt:
|
||||
|
||||
```
|
||||
PS> wslc login git.rittershofer.org -u wolf
|
||||
```
|
||||
|
||||
## Env files
|
||||
|
||||
Once per checkout, including every new repo, and again when `DISTRO` or `CHECKOUT` changes:
|
||||
|
||||
1. Copy `deploy/release.env.example` to `deploy/release.env` in the checkout (`cp` in the distro, or Explorer) and fill in `DISTRO` (`claudebox`), `CHECKOUT` (the path in the distro, `/home/claude/cboxcode/news`) and `IMAGE`. The file is gitignored.
|
||||
|
||||
## PS Profile commands
|
||||
|
||||
We want a simple way to run and deploy the app, `cd` into the repo. So we have script adding the relevant commands to the PS profile:
|
||||
|
||||
```
|
||||
PS> pwsh -ExecutionPolicy Bypass -File \\wsl.localhost\claudebox\home\claude\cboxcode\news\deploy\wsl-setup.ps1
|
||||
```
|
||||
|
||||
It creates commands into the profie
|
||||
|
||||
The prefix is the checkout folder's name, `news-` for `news` and `boilerplate-` for `boilerplate`; `-Name` on the `.ps1` sets another. The setup writes one block between `# >>> news (deploy/wsl-setup.ps1) >>>` and `# <<< news (deploy/wsl-setup.ps1) <<<` into `$PROFILE` and replaces it when run again. Blocks written by older versions of the setup, `newsdesk (deploy/wsl-setup.ps1)` and `wsl shared (deploy/wsl-setup.ps1)`, call scripts that no longer exist; delete them with `notepad $PROFILE`.
|
||||
|
||||
| Command | Runs | Does |
|
||||
|---|---|---|
|
||||
| `news-cd` | `Set-Location` | PowerShell into the checkout's folder |
|
||||
| `news-sh` | `wsl.exe` | a shell in claudebox, in the checkout |
|
||||
| `news-backend [-v]` | `bin/backend.sh` in claudebox | `uv run app serve` on :8787, until Ctrl-C |
|
||||
| `news-frontend` | `bin/frontend.sh` in claudebox | Vite with hot reload on :5173, until Ctrl-C |
|
||||
| `news-check` | `bin/check.sh` in claudebox | the gate on the working tree, without promoting |
|
||||
| `news-1test [--push-branch] [--no-tests]` | `deploy/1-test-and-promote.sh` in claudebox | test local main, fast-forward `production`, export it |
|
||||
| `news-2build` | `deploy\2-build-image.ps1` on Windows | build the exported tree with `wslc`, tags `<sha>` and `latest` |
|
||||
| `news-3push` | `deploy\3-push-image.ps1` on Windows | push both tags with `wslc` |
|
||||
| `news-0release [--push-branch] [--no-tests]` | the three above | `1test`, `2build`, `3push`, stopping at the first that fails |
|
||||
| `<folder>-db pull\|push` | `deploy/db.sh` in claudebox | only where `deploy/db.sh` exists, as in the boilerplate |
|
||||
|
||||
If the setup does not run, paste the block by hand. `New-Item -Force $PROFILE` creates the file if `Test-Path $PROFILE` says it is missing; `notepad $PROFILE` opens it. For `news`, with `DISTRO=claudebox` and `CHECKOUT=/home/claude/cboxcode/news`, the setup writes exactly this:
|
||||
|
||||
```powershell
|
||||
# >>> news (deploy/wsl-setup.ps1) >>>
|
||||
function news-cd { Set-Location '\\wsl.localhost\claudebox\home\claude\cboxcode\news' }
|
||||
function news-sh { wsl.exe -d claudebox --cd /home/claude/cboxcode/news }
|
||||
function news-backend { wsl.exe -d claudebox --cd /home/claude/cboxcode/news -- bash bin/backend.sh @args }
|
||||
function news-frontend { wsl.exe -d claudebox --cd /home/claude/cboxcode/news -- bash bin/frontend.sh @args }
|
||||
function news-check { wsl.exe -d claudebox --cd /home/claude/cboxcode/news -- bash bin/check.sh @args }
|
||||
function news-1test { wsl.exe -d claudebox --cd /home/claude/cboxcode/news -- bash deploy/1-test-and-promote.sh @args }
|
||||
function news-2build { & (Get-Process -Id $PID).Path -NoProfile -ExecutionPolicy Bypass -File '\\wsl.localhost\claudebox\home\claude\cboxcode\news\deploy\2-build-image.ps1' }
|
||||
function news-3push { & (Get-Process -Id $PID).Path -NoProfile -ExecutionPolicy Bypass -File '\\wsl.localhost\claudebox\home\claude\cboxcode\news\deploy\3-push-image.ps1' }
|
||||
function news-0release { news-1test @args; if ($LASTEXITCODE -ne 0) { return }; news-2build; if ($LASTEXITCODE -ne 0) { return }; news-3push }
|
||||
# <<< news (deploy/wsl-setup.ps1) <<<
|
||||
```
|
||||
|
||||
For another checkout, replace `news` and the path.
|
||||
|
||||
### When a command fails
|
||||
|
||||
- `wsl -l -v` lists the distros. `DISTRO` must be one of the names exactly.
|
||||
- `Test-Path \\wsl.localhost\claudebox\home\claude\cboxcode\news` must say `True`. `\\wsl$\claudebox\...` is the older spelling of the same share; if only that one answers, use it in the block.
|
||||
- `$PROFILE` prints the profile file this PowerShell loads, and `Get-Command news-*` lists the commands it defined. Commands missing in one PowerShell and present in another means the setup ran in the other one.
|
||||
- `Get-ExecutionPolicy -List` shows the policy per scope. `Restricted` or `AllSigned` where it applies means the profile does not load; a `MachinePolicy` or `UserPolicy` entry is group policy and overrides `CurrentUser`.
|
||||
- `wslc version` failing means WSL is older than 2.9.3: `wsl --update`.
|
||||
- `news-2build` saying there is no `production` file means step 1 has not exported yet: run `news-1test`.
|
||||
- If `wslc build` refuses the `\\wsl.localhost` folder as its context, which Microsoft's documentation neither promises nor rules out, copy the tree to Windows and build there: `Copy-Item -Recurse \\wsl.localhost\claudebox\home\claude\cboxcode\release\news\<sha> $env:TEMP\release-<sha>`, then the `wslc build` line of `2-build-image.ps1` with that folder.
|
||||
|
||||
# Kill and restart
|
||||
|
||||
Run everything in an **admin PowerShell** on Windows.
|
||||
|
||||
### Normal shutdown
|
||||
|
||||
`wsl --shutdown`
|
||||
|
||||
### Killing it
|
||||
|
||||
If normal shutdown hangs:
|
||||
|
||||
```
|
||||
taskkill /f /im wsl.exe
|
||||
taskkill /f /im wslhost.exe
|
||||
taskkill /f /im wslrelay.exe
|
||||
```
|
||||
|
||||
### Check the service
|
||||
|
||||
`sc queryex WslService`
|
||||
|
||||
If it says `STOP_PENDING` or `START_PENDING`, take the PID from the output and run `taskkill /PID <PID> /F`.
|
||||
|
||||
If it says `STOPPED`, start it with `sc start WslService`, then run `wsl`.
|
||||
|
||||
### **Kill a lingering VM**
|
||||
|
||||
IF it still does not work check the VM
|
||||
|
||||
```
|
||||
hcsdiag list
|
||||
hcsdiag kill <ID>
|
||||
```
|
||||
|
||||
If the old WSL VM is still listed, kill it with the ID shown. This was reported as the only fix short of a reboot in a complete-freeze case.
|
||||
|
||||
### **Restart the Hyper-V Host Compute Service**
|
||||
|
||||
```
|
||||
Restart-Service vmcompute
|
||||
```
|
||||
|
||||
A wedged vmcompute service is a common reason WSL stops working after it ran fine earlier in the same Windows session.
|
||||
|
||||
If it still hangs after all four, a reboot is the reliable fix.
|
||||
|
||||
Once it’s back up, running `wsl --update` is worth doing, since hangs like this are often fixed in newer versions.
|
||||
|
||||
# Pitfalls
|
||||
|
||||
- **Only open the distro from a normal PowerShell.** Opened from an admin one, WSL mounts the Windows drives despite `automount` being off ([microsoft/WSL#12443](https://github.com/microsoft/WSL/issues/12443)). `wsl --terminate claudebox` and a normal shell undo it.
|
||||
- **Changes to `/etc/wsl.conf` apply only after `wsl --terminate claudebox`.**
|
||||
@@ -1,175 +0,0 @@
|
||||
services:
|
||||
|
||||
mealie:
|
||||
image: ghcr.io/mealie-recipes/mealie:latest
|
||||
container_name: mealie
|
||||
restart: unless-stopped
|
||||
|
||||
# `deploy: resources: limits:` is Swarm-only and was silently ignored,
|
||||
# so this container has never actually had a memory limit. This one works.
|
||||
mem_limit: 1000m
|
||||
|
||||
# Was 9925:9000 on every interface — the internet, if the firewall is ever
|
||||
# wrong. SWAG reaches it by container name on the shared network; this is
|
||||
# only the fallback for when you are on the box itself.
|
||||
ports:
|
||||
- "127.0.0.1:9925:9000"
|
||||
|
||||
# Was a Docker-managed volume under /var/lib/docker, root-owned and
|
||||
# invisible. A plain directory next to this file, so a backup is `tar`.
|
||||
volumes:
|
||||
- /srv/data/mealie:/app/data
|
||||
|
||||
environment:
|
||||
ALLOW_SIGNUP: "false"
|
||||
PUID: 1000
|
||||
PGID: 1000
|
||||
TZ: Europe/Brussels
|
||||
MAX_WORKERS: 1
|
||||
WEB_CONCURRENCY: 1
|
||||
BASE_URL: https://mealie.rittershofer.org
|
||||
# From .env, chmod 600. Never in this file.
|
||||
OPENAI_API_KEY: ${OPENAI_API_KEY:?set OPENAI_API_KEY in .env}
|
||||
# A base, not an endpoint. The old /v1/models made it request
|
||||
# /v1/models/chat/completions.
|
||||
OPENAI_BASE_URL: https://api.openai.com/v1
|
||||
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: "20m"
|
||||
max-file: "5"
|
||||
|
||||
gitea:
|
||||
image: gitea/gitea:latest
|
||||
container_name: gitea
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
- USER_UID=1000
|
||||
- USER_GID=1000
|
||||
volumes:
|
||||
- /srv/data/gitea:/data
|
||||
- /etc/timezone:/etc/timezone:ro
|
||||
- /etc/localtime:/etc/localtime:ro
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: "20m"
|
||||
max-file: "5"
|
||||
|
||||
newsdesk:
|
||||
image: ${NEWSDESK_IMAGE:?set NEWSDESK_IMAGE in .env}:${NEWSDESK_TAG:-latest}
|
||||
container_name: newsdesk
|
||||
restart: unless-stopped
|
||||
user: "${PUID:-1000}:${PGID:-1000}"
|
||||
volumes:
|
||||
- /srv/data/newsdesk:/data
|
||||
environment:
|
||||
TZ: ${TZ:-Europe/Berlin}
|
||||
CLAUDE_CODE_OAUTH_TOKEN: ${CLAUDE_CODE_OAUTH_TOKEN:-}
|
||||
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
|
||||
NEWSDESK_PASSWORD: ${NEWSDESK_PASSWORD:-}
|
||||
mem_limit: ${NEWSDESK_MEM:-0}
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: "20m"
|
||||
max-file: "5"
|
||||
|
||||
bookly:
|
||||
image: git.rittershofer.org/wolf/bookly:${BOOKLY_TAG:-latest}
|
||||
container_name: bookly
|
||||
restart: unless-stopped
|
||||
user: "1000:1000"
|
||||
|
||||
volumes:
|
||||
- /srv/data/bookly:/data
|
||||
|
||||
environment:
|
||||
TZ: Europe/Brussels
|
||||
# Required: without it the demo would be open to anyone who finds the hostname, so Compose refuses to start.
|
||||
BOOKLY_PASSWORD: ${BOOKLY_PASSWORD:?set BOOKLY_PASSWORD in .env}
|
||||
# subscription: the Claude CLI with CLAUDE_CODE_OAUTH_TOKEN (same variable newsdesk reads).
|
||||
# api: ANTHROPIC_API_KEY, billed per token, lower latency.
|
||||
BOOKLY_BACKEND: ${BOOKLY_BACKEND:-subscription}
|
||||
CLAUDE_CODE_OAUTH_TOKEN: ${CLAUDE_CODE_OAUTH_TOKEN:-}
|
||||
ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY:-}
|
||||
|
||||
# No `ports:`. SWAG reaches :8000 by container name; nothing else can.
|
||||
|
||||
mem_limit: 0
|
||||
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: "20m"
|
||||
max-file: "5"
|
||||
|
||||
health:
|
||||
image: ${HEALTH_IMAGE:?set HEALTH_IMAGE in .env}:${HEALTH_TAG:-latest}
|
||||
container_name: health
|
||||
restart: unless-stopped
|
||||
|
||||
# Warehouse, journal, job table, speech models, tiles, backups: all of it
|
||||
# under one host directory, so a backup is one target.
|
||||
volumes:
|
||||
- ${HEALTH_DATA:-./data/health}:/data
|
||||
|
||||
user: "${PUID:-1000}:${PGID:-1000}"
|
||||
|
||||
environment:
|
||||
TZ: ${TZ:-Europe/Berlin}
|
||||
HEALTH_TIMEZONE: ${TZ:-Europe/Berlin}
|
||||
# Which model writes the evening entry. The subscription token now; add
|
||||
# ANTHROPIC_API_KEY and `docker compose up -d health` (not `restart`)
|
||||
# to move to the API. Shared with the other services in this file.
|
||||
CLAUDE_CODE_OAUTH_TOKEN: ${CLAUDE_CODE_OAUTH_TOKEN:-}
|
||||
ANTHROPIC_API_KEY: ${HEALTH_ANTHROPIC_API_KEY:-}
|
||||
# Normally empty. Set it to pin the password -- the way back in when a
|
||||
# login is lost -- then empty it and recreate to hand the password back
|
||||
# to the Settings page.
|
||||
HEALTH_PASSWORD: ${HEALTH_PASSWORD:-}
|
||||
# The speech model. `small` (0.7 GB) if turbo (1.9 GB while it runs)
|
||||
# does not fit beside everything else on the box.
|
||||
HEALTH_WHISPER_MODEL: ${HEALTH_WHISPER_MODEL:-large-v3-turbo}
|
||||
|
||||
# No `ports:` and no `networks:`: SWAG reaches it by name on this file's
|
||||
# network, and nothing else reaches it at all.
|
||||
|
||||
# `deploy: resources: limits:` is Swarm-only and ignored here. 0 is
|
||||
# unlimited until a transcription has been watched on the box.
|
||||
mem_limit: ${HEALTH_MEM:-0}
|
||||
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: "20m"
|
||||
max-file: "5"
|
||||
|
||||
swag:
|
||||
image: lscr.io/linuxserver/swag:latest
|
||||
container_name: swag
|
||||
cap_add:
|
||||
- NET_ADMIN
|
||||
environment:
|
||||
- PUID=1000
|
||||
- PGID=1000
|
||||
- TZ=Europe/Brussels
|
||||
- URL=rittershofer.org
|
||||
- VALIDATION=http
|
||||
# Commas and nothing else — SWAG splits on the comma alone, so a space
|
||||
# asks for a hostname with a leading space. `news` added for newsdesk;
|
||||
# `collabora` dropped, since nothing serves it any more.
|
||||
- SUBDOMAINS=www,mealie,git,news,play,body
|
||||
- ONLY_SUBDOMAINS=false
|
||||
volumes:
|
||||
- /etc/config/swag:/config
|
||||
ports:
|
||||
- 443:443
|
||||
- 80:80
|
||||
logging:
|
||||
driver: json-file
|
||||
options:
|
||||
max-size: "20m"
|
||||
max-file: "5"
|
||||
restart: unless-stopped
|
||||
-112
@@ -1,112 +0,0 @@
|
||||
# Deploying a service to rittershofer.org
|
||||
|
||||
The ground every service on this box shares: what the box is, the seven steps to add a service to it, the templates those steps need, and the things that have already gone wrong. [release.md](release.md) is how an image gets built and published; [server.md](server.md) is what happens once it has been.
|
||||
|
||||
`<app>` below is your *service's* name: the hostname, the data directory, the image tag, the compose service. It is the one thing you choose. The Python package stays `app` and the environment variables stay `APP_`, here and in every project started from this repository — nothing inside the checkout is renamed, and the files these steps need are already in place under `deploy/`.
|
||||
|
||||
## What the box is
|
||||
|
||||
A single Hetzner server running Ubuntu 24.04 on x86_64, with a 37 GB root filesystem and no separate data disk. One Docker Compose project holds everything.
|
||||
|
||||
/srv/swag/docker-compose.yml every service, one file, project name `swag`
|
||||
/srv/swag/.env every secret, chmod 600, never in the compose file
|
||||
/srv/data/<app>/ every service's data, owned 1000:1000
|
||||
/etc/config/swag/ SWAG's /config: nginx confs, certificates, logs
|
||||
|
||||
A snapshot of that whole compose file, every neighbour included, is [box-docker-compose.yml](box-docker-compose.yml): what a new service block lands beside, and where the SWAG subdomain list lives.
|
||||
|
||||
`swag` is nginx, certbot and fail2ban in one container. It terminates TLS for every hostname and proxies to the others **by container name** over the project's default network. That shared network is the only reason all these services live in one compose file — nothing else about them is related.
|
||||
|
||||
Images are published to the Gitea instance on the same box, at `git.rittershofer.org/wolf/<app>`. Everything runs as uid/gid 1000 and `TZ=Europe/Brussels`.
|
||||
|
||||
Which compose file that is comes off the running container rather than out of memory:
|
||||
|
||||
```bash
|
||||
docker inspect swag --format '{{index .Config.Labels "com.docker.compose.project.config_files"}}'
|
||||
```
|
||||
|
||||
Declaring the network `external` from a second Compose project would work too, and would hardcode the first project's name into the second — a rename away from a confusing failure.
|
||||
|
||||
## The seven steps
|
||||
|
||||
**1. DNS.** `<app>.rittershofer.org` must resolve to the server *before* anything else. Confirm with `getent hosts <app>.rittershofer.org`.
|
||||
|
||||
**2. Certificate.** Add `<app>` to `SUBDOMAINS` in the swag service's environment. That list separates on commas and nothing else — a space after one asks for a hostname with a leading space. SWAG runs certbot on every container start and requests **one certificate covering every name in the list**, so a name that does not answer on port 80 fails the whole request and leaves the existing names unrenewed. This is why step 1 comes first. Restart swag, then check `docker logs swag` for the issuance. If SWAG already holds a wildcard certificate covering the name, reuse it and skip this step entirely.
|
||||
|
||||
**3. Registry login.** On the build machine a Gitea token with package **write**; on the server, package **read**. Same command both ends, token at the password prompt:
|
||||
|
||||
```bash
|
||||
docker login git.rittershofer.org -u wolf
|
||||
```
|
||||
|
||||
Git credentials and Docker registry credentials are separate. A successful Git push does not log Docker in.
|
||||
|
||||
**4. Data directory.** Root's `mkdir` produces a root-owned directory, and the container runs as 1000, so the chown is not optional:
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /srv/data/<app> && sudo chown -R 1000:1000 /srv/data/<app>
|
||||
```
|
||||
|
||||
**5. Service block** from [../deploy/server/compose.yaml](../deploy/server/compose.yaml) into `/srv/swag/docker-compose.yml`, and the variables from [../deploy/server/env.example](../deploy/server/env.example) appended to `/srv/swag/.env`.
|
||||
|
||||
**6. Proxy conf** — [../deploy/server/swag/app.subdomain.conf](../deploy/server/swag/app.subdomain.conf) renamed to `<app>.subdomain.conf` in `/etc/config/swag/nginx/proxy-confs/`. SWAG's `/config` is a host directory bind-mounted into the container; read the host side off the running container rather than assuming where it is:
|
||||
|
||||
```bash
|
||||
docker inspect swag --format '{{range .Mounts}}{{if eq .Destination "/config"}}{{.Source}}{{end}}{{end}}'
|
||||
```
|
||||
|
||||
<config>/nginx/proxy-confs/ the per-site confs, where yours goes
|
||||
<config>/nginx/proxy.conf the shared defaults every site includes
|
||||
<config>/nginx/ssl.conf certificate paths and TLS settings
|
||||
<config>/nginx/resolver.conf Docker's internal DNS
|
||||
<config>/etc/letsencrypt/live/ the certificates themselves
|
||||
|
||||
Then validate, and reload only if validation passes:
|
||||
|
||||
```bash
|
||||
docker exec swag nginx -t && docker exec swag nginx -s reload
|
||||
```
|
||||
|
||||
Confirm the other sites still respond afterwards.
|
||||
|
||||
**7. Deploy.** Name the service, so nothing else in the file is recreated:
|
||||
|
||||
```bash
|
||||
cd /srv/swag && docker compose up -d <app>
|
||||
```
|
||||
|
||||
## What the templates already decide for you
|
||||
|
||||
Both templates are in [../deploy/server/](../deploy/server), commented in place. The decisions worth knowing before you read them:
|
||||
|
||||
**No `ports:`.** SWAG reaches the container by name on the project network. That is what keeps the app's port off the internet even if the firewall is wrong.
|
||||
|
||||
**No `networks:` either.** Being in that compose file is what puts it on the right one.
|
||||
|
||||
**`user: "1000:1000"`.** Container uid is host uid — there is no translation layer. Without this the process runs as whatever the image declares, usually root, and everything it writes into `/srv/data` lands root-owned.
|
||||
|
||||
**`mem_limit:`, not `deploy: resources: limits:`.** The latter is Swarm-only and silently ignored by `docker compose up`. 0 is unlimited, which is the honest default until the box has been watched under real load.
|
||||
|
||||
**`logging:` with a size cap.** Without it, json-file logs grow until the disk does not.
|
||||
|
||||
**`listen 443 ssl` without `http2`.** `http2 on;` is set once at `http` scope in `nginx.conf`, and the `listen ... http2` parameter is deprecated in nginx 1.25 and later.
|
||||
|
||||
**`proxy_read_timeout 86400s`** only if the app holds a WebSocket open. SWAG's `proxy.conf` already sets the Upgrade headers; what it also sets is a read timeout of a couple of minutes, which closes an idle socket on the minute. Delete those two lines if there is no socket.
|
||||
|
||||
## Things that have already bitten
|
||||
|
||||
**Disk.** 37 GB, and it has been full. Three causes, all recurring: `:latest` orphans the previous image on every pull and nothing cleans them up (a weekly `docker image prune -f` cron handles it — no `-a`, so pinned images survive); journald defaults to 10% of the filesystem and is capped here at 200 MB; and VS Code Remote installs a fresh server per version under `~/.vscode-server` forever.
|
||||
|
||||
**`cp -a "$SRC/."` with an unset `SRC`** copies the root filesystem. Use `"${SRC:?}"` and echo the value before running anything destructive with it.
|
||||
|
||||
**SWAG never touches your live confs**, only the `.sample` files beside them. That is deliberate and it means they drift for years. Diff them occasionally:
|
||||
|
||||
```bash
|
||||
cd /etc/config/swag/nginx && for f in *.conf site-confs/*.conf proxy-confs/*.subdomain.conf; do echo "=== $f"; diff "$f" "$f.sample"; done
|
||||
```
|
||||
|
||||
Take the samples wholesale and re-apply your own edits. `nginx.conf` and the site/proxy confs must be updated **together** — older ones include `ssl.conf` at `http` scope, newer ones per server block, and mixing the two produces `no "ssl_certificate" is defined`.
|
||||
|
||||
**SQLite is never safe to copy live.** Stop the container, or snapshot with `VACUUM INTO` through the running one. A `cp` of a WAL-mode database is a coin toss. See [server.md](server.md#the-database).
|
||||
|
||||
**Secrets in `.env`, never in the compose file.** It is easy to paste a compose file into a chat window; it is harder to do that by accident with a file you had to `cat` deliberately.
|
||||
@@ -1,241 +0,0 @@
|
||||
# Asking Claude, from a deployed app
|
||||
|
||||
One question — *ask Claude this, get JSON back* — and two ways to answer it, chosen by which credential is present in the environment. The caller asks without knowing which one ran.
|
||||
|
||||
api One call to the Claude API. Wants ANTHROPIC_API_KEY. No local
|
||||
install, no subprocess, per-token billing. The unattended path.
|
||||
claude_code The Claude Agent SDK drives the `claude` CLI, on the
|
||||
subscription it is signed in with, authenticated by
|
||||
CLAUDE_CODE_OAUTH_TOKEN. Nothing billed per token.
|
||||
|
||||
This is the reusable part of the deployment: the compose file passes both variables through, the app prefers the key when there is one, and switching from the subscription to the API is a line in `.env` and `docker compose up -d` — no rebuild, no code change.
|
||||
|
||||
## The two credentials
|
||||
|
||||
**`CLAUDE_CODE_OAUTH_TOKEN`** comes from `claude setup-token`, run interactively on a machine you are already signed in on. It lasts a year. Generating it needs browser authorization, so no deployment script can automate it — you produce it on your laptop and paste it into the server's `.env` with an editor. Then `docker compose up -d app`, because a `restart` will not pick it up.
|
||||
|
||||
**`ANTHROPIC_API_KEY`** is per-token billing from the Console. Nothing to renew, nothing to sign in to, and it is what production should run on once the thing is real.
|
||||
|
||||
Set one. If both are set, `auto` takes the key. Neither, and every call fails with a notice naming the reason, which is the correct outcome — better a batch that fails saying "no credentials" than one that waits for a human who is not there.
|
||||
|
||||
The API key and the OAuth token are not interchangeable and do not substitute for each other. `ANTHROPIC_API_KEY` in the environment does not make the CLI work, and an OAuth token does not authenticate an API call.
|
||||
|
||||
**Neither belongs in the image or in the compose file.** They live in the `.env` beside it, `chmod 600`, injected as environment variables at container creation. A compose file gets pasted into chat windows; a file you had to `cat` deliberately does not.
|
||||
|
||||
## Why there is a `backend` setting and not just detection
|
||||
|
||||
Detection answers "what can run here". The setting answers "what should". They differ in two cases worth naming: forcing `api` on a machine that also has the CLI, so a benchmark measures the thing you meant; and `off`, which makes everything needing a model fail on purpose, with a notice that says which switch did it. An app that silently degrades to no-model is worse than one that stops.
|
||||
|
||||
## What the `claude_code` path deliberately does not get
|
||||
|
||||
No tools (`tools=[]`, `allowed_tools=[]`), no filesystem settings (`setting_sources=None`, so no CLAUDE.md, no skills, no MCP servers leak into a classification call), no session to resume. One text-in, JSON-out turn — the same shape as the API call it stands in for. The CLI is built for agent work, and left to its defaults it will bring all of that with it.
|
||||
|
||||
## Thinking has to be stated, not defaulted
|
||||
|
||||
The CLI thinks unprompted; the Messages API thinks only when asked. So "say nothing about thinking" means opposite things on the two sides, and the two backends stop being interchangeable. State it explicitly to both. It is not a small difference: on a batch of twenty-five short classifications, the CLI's default cost 128 s against 17 s for the same answers.
|
||||
|
||||
Some models support manual extended thinking only and reject `adaptive`. Check before spending a call, not after.
|
||||
|
||||
## Both take the same JSON schema
|
||||
|
||||
The Agent SDK accepts the Messages API's `json_schema` output format and hands the parsed object back on `ResultMessage.structured_output`. That is what makes this two backends rather than two implementations: neither path is the degraded one, and moving between them is a config line.
|
||||
|
||||
## Reference implementation
|
||||
|
||||
Self-contained — no imports from your project, both SDKs optional. It is the shape, not the whole thing: the real one in newsdesk also streams partial answers out for a progress bar, retries, and classifies API errors by status code so a 401 and a 429 tell you different things.
|
||||
|
||||
```python
|
||||
"""Ask Claude a question, constrained to a JSON schema, get a dict back.
|
||||
|
||||
Two backends, one seam. `ANTHROPIC_API_KEY` selects the API; otherwise the
|
||||
Claude Agent SDK drives the `claude` CLI on a subscription, authenticated by
|
||||
`CLAUDE_CODE_OAUTH_TOKEN`. Both are optional dependencies:
|
||||
|
||||
uv pip install anthropic # the api backend
|
||||
uv pip install claude-agent-sdk # the claude_code backend
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
|
||||
class AgentError(Exception):
|
||||
"""The backend was there and the call still did not produce an answer."""
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class AgentConfig:
|
||||
#: "auto", "api", "claude_code", or "off" to make every call fail.
|
||||
backend: str = "auto"
|
||||
model: str = "claude-haiku-4-5-20251001"
|
||||
timeout: float = 300.0
|
||||
max_tokens: int = 8192
|
||||
#: "off" or "adaptive". Stated to both backends, never defaulted.
|
||||
thinking: str = "off"
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class Reply:
|
||||
data: dict[str, Any]
|
||||
backend: str
|
||||
model: str
|
||||
usage: dict[str, Any]
|
||||
|
||||
|
||||
def backend(settings: AgentConfig) -> str:
|
||||
"""Which backend will answer, or "" for nobody."""
|
||||
if settings.backend == "off":
|
||||
return ""
|
||||
if settings.backend in ("api", "claude_code"):
|
||||
return settings.backend
|
||||
if any(os.environ.get(name) for name in
|
||||
("ANTHROPIC_API_KEY", "ANTHROPIC_AUTH_TOKEN", "ANTHROPIC_PROFILE")):
|
||||
return "api"
|
||||
# The SDK bundles its own `claude`, so having the package is enough. Not
|
||||
# proof the call will work — the CLI may be signed out — but that surfaces
|
||||
# as an AgentError on the call, which the caller handles anyway.
|
||||
if importlib.util.find_spec("claude_agent_sdk") is not None:
|
||||
return "claude_code"
|
||||
return ""
|
||||
|
||||
|
||||
def _thinking(setting: str) -> dict[str, str]:
|
||||
return {"type": "adaptive"} if setting == "adaptive" else {"type": "disabled"}
|
||||
|
||||
|
||||
def ask_json(settings: AgentConfig, *, system: str, question: dict[str, Any],
|
||||
schema: dict[str, Any]) -> Reply:
|
||||
"""Ask `question`, constrained to `schema`, and return the parsed answer.
|
||||
|
||||
`system` should be byte-identical across the calls in a run — it is what
|
||||
the API path caches. `question` is the part that changes.
|
||||
"""
|
||||
name = backend(settings)
|
||||
if not name:
|
||||
raise AgentError("no model backend is available: set ANTHROPIC_API_KEY "
|
||||
"or install claude-agent-sdk with CLAUDE_CODE_OAUTH_TOKEN")
|
||||
if name == "api":
|
||||
return _ask_api(settings, system, question, schema)
|
||||
return _ask_claude_code(settings, system, question, schema)
|
||||
|
||||
|
||||
def _ask_api(settings: AgentConfig, system: str, question: dict[str, Any],
|
||||
schema: dict[str, Any]) -> Reply:
|
||||
try:
|
||||
import anthropic
|
||||
except ImportError as exc:
|
||||
raise AgentError("the anthropic SDK isn't installed") from exc
|
||||
|
||||
try:
|
||||
client = anthropic.Anthropic(timeout=settings.timeout)
|
||||
response = client.messages.create(
|
||||
model=settings.model,
|
||||
max_tokens=settings.max_tokens,
|
||||
# The instructions are byte-identical across every call; only the
|
||||
# payload below changes. Caching them is most of the saving.
|
||||
system=[{"type": "text", "text": system,
|
||||
"cache_control": {"type": "ephemeral"}}],
|
||||
messages=[{"role": "user", "content": json.dumps(question)}],
|
||||
thinking=_thinking(settings.thinking),
|
||||
output_config={"format": {"type": "json_schema", "schema": schema}},
|
||||
)
|
||||
except anthropic.APIError as exc:
|
||||
# Carry the status through in the text: 401 ("your key is wrong") and
|
||||
# 429 ("slow down") want different reactions, and the message body
|
||||
# alone does not always separate them.
|
||||
status = getattr(exc, "status_code", "")
|
||||
raise AgentError(f"the API call failed: {status} {exc}".strip()) from exc
|
||||
|
||||
text = next((b.text for b in response.content if b.type == "text"), "")
|
||||
return Reply(data=json.loads(text), backend="api",
|
||||
model=response.model or settings.model,
|
||||
usage=response.usage.model_dump() if response.usage else {})
|
||||
|
||||
|
||||
def _ask_claude_code(settings: AgentConfig, system: str, question: dict[str, Any],
|
||||
schema: dict[str, Any]) -> Reply:
|
||||
try:
|
||||
from claude_agent_sdk import (
|
||||
ClaudeAgentOptions, ClaudeSDKError, ResultMessage, query,
|
||||
)
|
||||
except ImportError as exc:
|
||||
raise AgentError("the Claude Agent SDK isn't installed") from exc
|
||||
|
||||
options = ClaudeAgentOptions(
|
||||
system_prompt=system,
|
||||
model=settings.model or None,
|
||||
# A classification call, not an agent: nothing to run, nothing to read,
|
||||
# and no project settings — so no CLAUDE.md, no skills and no MCP
|
||||
# servers get in front of the model on the way to an answer.
|
||||
tools=[],
|
||||
allowed_tools=[],
|
||||
permission_mode="dontAsk",
|
||||
setting_sources=None,
|
||||
thinking=_thinking(settings.thinking),
|
||||
output_format={"type": "json_schema", "schema": schema},
|
||||
# The API takes max_tokens as a parameter; the CLI takes it only as an
|
||||
# environment variable. Without this the setting is quietly API-only.
|
||||
# The SDK merges this into the inherited environment rather than
|
||||
# replacing it, so naming one variable does not cost the CLI its PATH.
|
||||
env={"CLAUDE_CODE_MAX_OUTPUT_TOKENS": str(settings.max_tokens)},
|
||||
)
|
||||
|
||||
async def ask() -> Reply:
|
||||
answer: Reply | None = None
|
||||
async for message in query(prompt=json.dumps(question), options=options):
|
||||
if not isinstance(message, ResultMessage):
|
||||
continue
|
||||
if message.is_error:
|
||||
raise AgentError("; ".join(message.errors or []) or message.result
|
||||
or f"the agent ended as {message.subtype}")
|
||||
data = message.structured_output
|
||||
if not isinstance(data, dict):
|
||||
# Ended without structured output — a limit, an interruption.
|
||||
# `result` is the raw text if there is any, worth one attempt.
|
||||
data = json.loads(message.result or "")
|
||||
answer = Reply(data=data, backend="claude_code",
|
||||
model=settings.model, usage=message.usage or {})
|
||||
if answer is None:
|
||||
raise AgentError("the agent returned no result")
|
||||
return answer
|
||||
|
||||
try:
|
||||
# asyncio.run, so this is a synchronous call like the other backend.
|
||||
# From inside a running loop, await `ask()` directly instead.
|
||||
return asyncio.run(asyncio.wait_for(ask(), settings.timeout))
|
||||
except TimeoutError as exc:
|
||||
raise AgentError(f"no answer within {settings.timeout:.0f}s") from exc
|
||||
except ClaudeSDKError as exc:
|
||||
# The SDK has its own exception hierarchy — a CLI that exited, a line
|
||||
# it could not parse, an answer that overran max_tokens — and none of
|
||||
# it is an AgentError. Left alone, those escape a caller that catches
|
||||
# AgentError per item and take down a whole run. Every failure of this
|
||||
# backend is an AgentError or none of them are.
|
||||
raise AgentError(str(exc)) from exc
|
||||
```
|
||||
|
||||
Using it:
|
||||
|
||||
```python
|
||||
settings = AgentConfig()
|
||||
schema = {
|
||||
"type": "object",
|
||||
"properties": {"tags": {"type": "array", "items": {"type": "string"}}},
|
||||
"required": ["tags"],
|
||||
}
|
||||
reply = ask_json(settings, system="Tag this article. Answer with the schema.",
|
||||
question={"title": title, "body": body}, schema=schema)
|
||||
print(reply.data["tags"], reply.backend)
|
||||
```
|
||||
|
||||
## The two failures that will actually happen
|
||||
|
||||
**A changed token in `.env` appears to do nothing.** `docker compose restart` does not update environment variables. `docker compose up -d app` recreates the container, which does.
|
||||
|
||||
**The token expired and nothing said so.** The setup token lasts a year, and the failure looks like a model call failing rather than a credential problem. Read the error text — this is why the SDK's own message is passed through intact rather than summarised into "model call failed".
|
||||
@@ -1,106 +0,0 @@
|
||||
# One password, on the open internet
|
||||
|
||||
The shape used by a single-user app that has to sit on a public hostname because that is where the browser is. No user management, no usernames, no registration: one password and the sessions it opens. Everything below is about making that enough.
|
||||
|
||||
This is the reasoning; `auth/` in this repository is the code, already written and already wired into `app/main.py`. `auth/README.md` is the ten-minute version of how to take the folder somewhere else.
|
||||
|
||||
## Locked by what it is bound to, not by a flag
|
||||
|
||||
The switch is the bind address. On `127.0.0.1` the door is already the machine's own login, and asking for a password again is theatre. On `0.0.0.0` it is reachable from elsewhere and locks itself.
|
||||
|
||||
Deriving it rather than configuring it removes the failure that looks fine: "deployed it, forgot to switch auth on". The production config then says `enabled: always` anyway — redundant with the bind address and worth saying out loud.
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
enabled: always # auto | always | never
|
||||
session_days: 30
|
||||
# `auto` marks the cookie Secure when the request arrived over HTTPS. Behind
|
||||
# a TLS-terminating proxy the app only ever sees plain HTTP and cannot work
|
||||
# that out, so behind SWAG this must be `always`.
|
||||
secure_cookie: always
|
||||
```
|
||||
|
||||
## The password itself
|
||||
|
||||
Stored as an scrypt hash, never in the config file. The first locked start either takes `APP_PASSWORD` from the environment or generates one and prints it to the log once. After that it is the app's own Account screen that changes it.
|
||||
|
||||
`APP_PASSWORD` is the way back in when a login is lost, and it is authoritative while it is set — the Account screen refuses to change what the environment pins. Set it, get in, empty it, recreate the container, then change the password in the app.
|
||||
|
||||
Two consequences worth writing down: a generated password is printed to the container log, so the retained log holds it until it rotates — rotate the password once you are in. And a password the *user* chooses is the weak spot, not the lockout: a minimum length of 8 is a very different thing from the 16 random characters the app generates.
|
||||
|
||||
## The lockout
|
||||
|
||||
Five wrong answers are free — past a typo and a stale password manager entry, short of anything systematic. After that each attempt waits, starting at 30 s and doubling to a ceiling of 15 minutes. That is about a hundred guesses a day against a secret with far more than a hundred possibilities, which is what "guessing is pointless" means in practice.
|
||||
|
||||
Three details that matter more than the numbers:
|
||||
|
||||
**Read the lockout before checking the password, not after.** Otherwise a fast attacker extends their own lockout forever and never notices, and — more to the point — an honest person hammering the button digs a deeper hole with every press.
|
||||
|
||||
**A wrong answer still costs a full scrypt derivation.** The delay an attacker meets is the one the honest user meets, and it is what lets the free-attempt count be as generous as it is.
|
||||
|
||||
**Cap the doubling.** `30s * 2**failures` with an uncapped counter eventually overflows whatever holds it, and a bot that has sent ten thousand guesses turns every future login attempt into a 500 — a denial of service delivered by arithmetic.
|
||||
|
||||
## One counter, not one per address
|
||||
|
||||
The obvious improvement is to lock out the address that is guessing, so a stranger cannot keep the owner waiting at their own door. Behind a proxy it cannot be built honestly: the only address the app sees is the one the proxy puts in a header, the peer is a container on the same Docker network, and a header is set by whoever is talking. Keyed on that, an attacker gets a fresh, empty counter per request by editing one line of their own request, while the owner still gets locked out by their own typos. Strictly worse, and it would look like protection.
|
||||
|
||||
So the address is **logged, not decided from** — at the layer that knows who connected, the decision can be made properly. See the throttling section below.
|
||||
|
||||
The honest cost of one global counter: an attacker hammering the door keeps password login locked for the owner too. A passkey login that skips the lockout is what makes that survivable.
|
||||
|
||||
## Sessions
|
||||
|
||||
One kind of session whichever door opened it, so revoking and expiry have one implementation. A row per session with its user agent and last-seen time, a screen listing them, a button to revoke one and a button to close all the others. `purge()` on startup drops the expired ones — otherwise that table grows forever.
|
||||
|
||||
The cookie: `HttpOnly` so an XSS cannot read it, `Secure` per the config above, `SameSite=Lax`, `path=/`.
|
||||
|
||||
**No CSRF token, and that is a decision rather than an omission.** A CSRF token defends a cookie-authenticated app against another origin submitting a form on your behalf. `SameSite=Lax` already refuses to send the cookie on any cross-site request that is not a top-level navigation, and if no `GET` in the app changes anything, the attack the token would stop cannot be mounted. Adding one anyway means a second secret, a second thing to rotate, and a second way for a request to fail that the UI has to explain. Where that reasoning stops holding is a browser old enough to ignore `SameSite` entirely.
|
||||
|
||||
## Passkeys
|
||||
|
||||
Here already, behind the `passkeys` extra: `uv sync --all-extras`, and without it the routes answer 501 and the Account tab simply does not draw the section.
|
||||
|
||||
A passkey and a security key are one feature: a YubiKey registers through the same flow as a phone, because to WebAuthn they are the same thing.
|
||||
|
||||
A passkey is bound to an origin, and that binding is what makes it unphishable — the browser only hands the credential back to the origin that created it. So it needs HTTPS and a real hostname, with `localhost` the one exception, which is what lets it be tried before deploying. Behind a proxy, state both explicitly:
|
||||
|
||||
```yaml
|
||||
auth:
|
||||
rp_id: app.rittershofer.org
|
||||
rp_name: app
|
||||
origin: https://app.rittershofer.org
|
||||
```
|
||||
|
||||
Left blank they are derived from the request, which behind a TLS-terminating proxy derives `http://` and rejects every ceremony as an origin mismatch. Changing `rp_id` later invalidates every key already registered against the old one.
|
||||
|
||||
**The password does not go away**, deliberately: it is the way back in if a key is lost, which is why revoking a passkey can never lock you out.
|
||||
|
||||
## Throttling a stranger who is guessing
|
||||
|
||||
The lockout already makes the secret safe. What is left is keeping the door usable for you, and keeping the traffic off the app. Two layers, in the order worth adding them:
|
||||
|
||||
**1. Rate limit in nginx.** Five lines in the proxy conf, included in [../deploy/server/swag/app.subdomain.conf](../deploy/server/swag/app.subdomain.conf). Per-address, no state to manage, no capabilities, and it cannot lock you out — it just makes requests wait. This is the part that earns its place.
|
||||
|
||||
**2. fail2ban, only if a real attack turns up.** SWAG ships it. It reads nginx's access log, where the address is the one that actually connected rather than a header, and bans in the firewall across every service on the box. A filter matching the login routes with a 401/403/429 status, and a jail pointing at `/config/log/nginx/access.log`, is about ten lines.
|
||||
|
||||
What fail2ban adds over the throttle is narrow and worth being honest about: your own password login stays usable during an attack, the ban covers the whole box rather than one site, and you get a list of who has been banned. It adds no protection of the password — that is already done. Against that it costs `NET_ADMIN` on the SWAG container, a regex that breaks silently if the log format changes, and an unban procedure for when it catches you.
|
||||
|
||||
Add it with a real log sample to test the regex against:
|
||||
|
||||
```bash
|
||||
docker exec swag fail2ban-regex /config/log/nginx/access.log /config/fail2ban/filter.d/app.conf
|
||||
```
|
||||
|
||||
If the domain is proxied through Cloudflare, nginx logs Cloudflare's addresses and a ban hits Cloudflare rather than the attacker. Check that before building anything on top of the log.
|
||||
|
||||
## What to log
|
||||
|
||||
One WARNING line per refusal, with the address first after `from`, so a filter is one regex:
|
||||
|
||||
```
|
||||
WARNING auth.router: wrong password from 203.0.113.9
|
||||
```
|
||||
|
||||
The log is the record. A second table of attempts in the database would only ever disagree with it. If the app is run at `log_level="warning"` there is no access log of its own, which makes this the whole record of who is knocking.
|
||||
|
||||
The forwarded address is the one piece of a stranger's own writing that ends up in that log, so strip it to the characters an address is made of before writing it. A line in this log is something a filter reads and a person believes, and neither should be forgeable by whoever is knocking.
|
||||
@@ -1,51 +0,0 @@
|
||||
# Building and publishing a release
|
||||
|
||||
Build on Windows, run on the server. Nothing in this half connects to the server: it tests a commit, promotes it, builds an image and pushes it to the registry. [server.md](server.md) is the other half.
|
||||
|
||||
The scripts are already in place in `deploy/`, with `deploy/server/` beside them — `db.ps1` expects that — and the Dockerfile at `deploy/Dockerfile`, which is where `2-build-image.ps1` looks for it inside the archived commit. The package is `app`; there is nothing to rename.
|
||||
|
||||
## Once
|
||||
|
||||
Copy `deploy/release.env.example` to `deploy/release.env` and fill it in. `.gitignore` already covers `deploy/release.env` and `deploy/backups/`.
|
||||
|
||||
Log Docker in with a token that has package-write access, before the first push:
|
||||
|
||||
```bash
|
||||
docker login git.rittershofer.org -u wolf
|
||||
```
|
||||
|
||||
Enter the token at Docker's password prompt, not in source files or chat. Git credentials and Docker registry credentials are separate: a successful Git push does not log Docker in.
|
||||
|
||||
## Every release
|
||||
|
||||
```powershell
|
||||
.\deploy\1-test-and-promote.ps1 # test local main, fast-forward local production
|
||||
.\deploy\2-build-image.ps1 # build production's image locally
|
||||
.\deploy\3-push-image.ps1 # push the built image to the registry
|
||||
```
|
||||
|
||||
The number is the order. Each script stops on failure, can be run on its own, and works out the repository root for itself, so the working directory does not matter. To run all three, use `.\deploy\0-release.ps1` — numbered zero so it sorts above the steps it runs.
|
||||
|
||||
**Step 1 never builds; step 2 never pushes; step 3 never rebuilds or deploys.** That separation is the point of the numbering: a failed push is `3` again, not the whole thing again.
|
||||
|
||||
## What each step is actually doing
|
||||
|
||||
**1 — test and promote.** Resolves `main` to a commit, checks that commit out into a throwaway worktree, and runs the project's checks there. The worktree is what makes this honest: the tests run against exactly the tree that is about to become production, with none of the working copy's uncommitted edits in the way. Worktrees have no `.venv` of their own, so the interpreter comes from the main checkout and the code from the worktree.
|
||||
|
||||
Promotion is a fast-forward only. If `production` has commits `main` does not, that is a release made from somewhere else, and the script stops rather than discarding it.
|
||||
|
||||
Git publication is optional: run `git push origin production` yourself, or pass `-PushBranch`. `production` identifies the tested candidate; the server's running image identifies the deployed release.
|
||||
|
||||
**2 — build.** `git archive` of the production commit, unpacked to a temporary directory, built from there. Writing the archive out rather than piping it is deliberate — a PowerShell pipeline carries text, so it would re-encode the tar stream and hand Docker a corrupted context. The build publishes two tags locally: the short commit sha and `latest`.
|
||||
|
||||
Docker builds an **image**. No container starts or restarts on this machine.
|
||||
|
||||
**3 — push.** Re-tags `latest` onto this candidate, then pushes both. It never rebuilds, so a push that failed on connectivity is just this step again.
|
||||
|
||||
## Uncommitted work is never included
|
||||
|
||||
Every step reads from git, not from the working copy. An edit you have not committed is not in the tests, not in the image, and not in the release. That is worth knowing in both directions: it cannot bite you by accident, and it will not pick up the fix you forgot to commit.
|
||||
|
||||
## Version pinning and rollback
|
||||
|
||||
`latest` moves on every publication, so it cannot identify a previous release. Commit tags can: keep them in the registry, and set `APP_TAG=<sha>` in the server's `.env` to pin or roll back. See [server.md](server.md#roll-back) for what else a rollback has to check.
|
||||
@@ -1,75 +0,0 @@
|
||||
# Server setup and operation
|
||||
|
||||
What happens once an image has been published. The release half is [release.md](release.md); the box itself and the seven steps for putting a new service on it are [hosting.md](hosting.md).
|
||||
|
||||
Deployment is server-local. No Git checkout, build tooling or SSH from the development machine is needed for it — `db.ps1` is the one exception and it moves a database, not a release.
|
||||
|
||||
## Install the files
|
||||
|
||||
`APP_DATA` decides where the data directory goes. Left out, it is `./data` beside the compose file. Set to an absolute path, it can sit under the one root every container on the box writes to — which is what makes a backup a single target rather than a hunt.
|
||||
|
||||
```bash
|
||||
sudo mkdir -p /srv/data/app/home /srv/data/app/backups
|
||||
sudo chown -R 1000:1000 /srv/data/app
|
||||
```
|
||||
|
||||
`home/` is there because `HOME=/data/home` in the Dockerfile: a CLI the app shells out to needs somewhere writable, and the image is not it.
|
||||
|
||||
Then:
|
||||
|
||||
- the `app:` block from [../deploy/server/compose.yaml](../deploy/server/compose.yaml) into the compose file that runs the proxy
|
||||
- the variables from [../deploy/server/env.example](../deploy/server/env.example) appended to the `.env` beside it, `chmod 600`
|
||||
- [../deploy/server/deploy.sh](../deploy/server/deploy.sh) and [../deploy/server/backup.sh](../deploy/server/backup.sh) next to that compose file
|
||||
- the app's own `config.yaml` into `$APP_DATA/config.yaml`
|
||||
- [../deploy/server/swag/app.subdomain.conf](../deploy/server/swag/app.subdomain.conf) into SWAG's `proxy-confs/`, renamed
|
||||
|
||||
Edit `.env` for the actual data path and UID/GID. Keep `APP_TAG=latest` for ordinary deployments or choose a published commit tag to pin.
|
||||
|
||||
## Deploy
|
||||
|
||||
```bash
|
||||
bash /srv/swag/deploy.sh
|
||||
```
|
||||
|
||||
It runs `docker compose pull app` first, and only on success runs `docker compose up -d --no-deps --pull never --wait --wait-timeout 240 app`, using that image and waiting for its `HEALTHCHECK`. Other services in the file are left alone. A failed pull stops before startup; an unhealthy startup exits nonzero. Inspect `docker compose ps` and `docker compose logs app` on failure.
|
||||
|
||||
**After changing credentials in `.env`, use `up -d`, not `restart`.** A restart does not update environment variables — the same container starts again with the environment it was created with. This is the single most common way an updated token appears not to have worked.
|
||||
|
||||
## The database
|
||||
|
||||
Never overwrite a live SQLite database with a plain file copy. It is in WAL mode, so the `.db` on disk is always missing whatever is still in the `-wal` beside it, and copying the three files while the app is running is a race you lose quietly: a database that opens fine and is short a day's writes.
|
||||
|
||||
**Backups.** `bash /srv/swag/backup.sh` takes a `VACUUM INTO` snapshot through the running container, gzips it into `$APP_DATA/backups`, and deletes everything past `KEEP` (14 by default). Schedule it with cron:
|
||||
|
||||
```
|
||||
0 4 * * * bash /srv/swag/backup.sh >> /srv/swag/backup.log 2>&1
|
||||
```
|
||||
|
||||
Keep an off-server copy, and test restoring it. A backup nobody has restored is a hypothesis.
|
||||
|
||||
**Moving a database onto the server.** `.\deploy\db.ps1 push` takes a snapshot of this machine's database, uploads it with the importer, and runs the import. The server validates the incoming file — `integrity_check` plus the tables it must have — and snapshots the live database into `/data/backups` before replacing it, atomically.
|
||||
|
||||
It does not stop the app and does not start it: **stop it yourself first** and start it afterwards. The import is refused while the app is up. Get the image the database needs into the registry and onto the server first, because older code refuses to open a newer schema.
|
||||
|
||||
If the app has accounts, importing replaces them too, so do the account setup against the final database rather than before the import.
|
||||
|
||||
**Taking one off.** `.\deploy\db.ps1 pull` runs the server's own `backup.sh` and downloads the newest snapshot into `backups\`. Work on a copy of it, never on a file you intend to push back: after a push the two copies diverge from the first write on either, and there is no merge for that.
|
||||
|
||||
## Roll back
|
||||
|
||||
Record the current image tag or digest before an update. Then set that commit tag in `.env` and run `deploy.sh` again. `latest` moves on publication, so it cannot name a previous release — this only works if commit-tagged images stay in the registry.
|
||||
|
||||
Check schema compatibility before rolling code back. If a release migrated the database incompatibly, restore a matching backup while the app is stopped. No automated rollback overwrites the database, and that is deliberate.
|
||||
|
||||
## Before relying on it
|
||||
|
||||
Verify, in this order, and write down what you saw rather than closing it from local reasoning:
|
||||
|
||||
1. HTTPS from outside, on the real hostname.
|
||||
2. Login, and a passkey if the app has them — passkeys need `rp_id`/`origin` to match the hostname exactly.
|
||||
3. The app's heaviest real operation, once, end to end.
|
||||
4. Persistence: `docker compose up -d --force-recreate app`, then check the data is still there.
|
||||
5. A restore of an off-server backup copy.
|
||||
6. Memory under that heaviest operation, before choosing `APP_MEM`.
|
||||
|
||||
The health endpoint checks the web app and its store. It does not check model credentials, and a green container says nothing about whether the API key works.
|
||||
Reference in new issue
Block a user