Files
jobboard/docs/hosting.md
T
WolfandClaude Opus 5.5 9d666d0937 Bring in pipeline, auth, deploy and docs from the boilerplate
roledesk is built on the boilerplate from here on: pipeline 2.0 (queue and
scheduler), auth (one password, sessions, passkeys), the deploy kit and its
docs, plus the coupling and packaging tests. Downstream copies, unedited.

Refreshed from boilerplate/d56949f7d0f6c0bfe5745478e09007bfad67ae87

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-23 12:29:16 +02:00

7.7 KiB

Deploying a service to rittershofer.org

The ground every service on this box shares: what the box is, the seven steps to add a service to it, the templates those steps need, and the things that have already gone wrong. release.md is how an image gets built and published; server.md is what happens once it has been.

<app> below is your service's name: the hostname, the data directory, the image tag, the compose service. It is the one thing you choose. The Python package stays app and the environment variables stay APP_, here and in every project started from this repository — nothing inside the checkout is renamed, and the files these steps need are already in place under deploy/.

What the box is

A single Hetzner server running Ubuntu 24.04 on x86_64, with a 37 GB root filesystem and no separate data disk. One Docker Compose project holds everything.

/srv/swag/docker-compose.yml   every service, one file, project name `swag`
/srv/swag/.env                 every secret, chmod 600, never in the compose file
/srv/data/<app>/               every service's data, owned 1000:1000
/etc/config/swag/              SWAG's /config: nginx confs, certificates, logs

A snapshot of that whole compose file, every neighbour included, is box-docker-compose.yml: what a new service block lands beside, and where the SWAG subdomain list lives.

swag is nginx, certbot and fail2ban in one container. It terminates TLS for every hostname and proxies to the others by container name over the project's default network. That shared network is the only reason all these services live in one compose file — nothing else about them is related.

Images are published to the Gitea instance on the same box, at git.rittershofer.org/wolf/<app>. Everything runs as uid/gid 1000 and TZ=Europe/Brussels.

Which compose file that is comes off the running container rather than out of memory:

docker inspect swag --format '{{index .Config.Labels "com.docker.compose.project.config_files"}}'

Declaring the network external from a second Compose project would work too, and would hardcode the first project's name into the second — a rename away from a confusing failure.

The seven steps

1. DNS. <app>.rittershofer.org must resolve to the server before anything else. Confirm with getent hosts <app>.rittershofer.org.

2. Certificate. Add <app> to SUBDOMAINS in the swag service's environment. That list separates on commas and nothing else — a space after one asks for a hostname with a leading space. SWAG runs certbot on every container start and requests one certificate covering every name in the list, so a name that does not answer on port 80 fails the whole request and leaves the existing names unrenewed. This is why step 1 comes first. Restart swag, then check docker logs swag for the issuance. If SWAG already holds a wildcard certificate covering the name, reuse it and skip this step entirely.

3. Registry login. On the build machine a Gitea token with package write; on the server, package read. Same command both ends, token at the password prompt:

docker login git.rittershofer.org -u wolf

Git credentials and Docker registry credentials are separate. A successful Git push does not log Docker in.

4. Data directory. Root's mkdir produces a root-owned directory, and the container runs as 1000, so the chown is not optional:

sudo mkdir -p /srv/data/<app> && sudo chown -R 1000:1000 /srv/data/<app>

5. Service block from ../deploy/server/compose.yaml into /srv/swag/docker-compose.yml, and the variables from ../deploy/server/env.example appended to /srv/swag/.env.

6. Proxy conf — ../deploy/server/swag/app.subdomain.conf renamed to <app>.subdomain.conf in /etc/config/swag/nginx/proxy-confs/. SWAG's /config is a host directory bind-mounted into the container; read the host side off the running container rather than assuming where it is:

docker inspect swag --format '{{range .Mounts}}{{if eq .Destination "/config"}}{{.Source}}{{end}}{{end}}'
<config>/nginx/proxy-confs/     the per-site confs, where yours goes
<config>/nginx/proxy.conf       the shared defaults every site includes
<config>/nginx/ssl.conf         certificate paths and TLS settings
<config>/nginx/resolver.conf    Docker's internal DNS
<config>/etc/letsencrypt/live/  the certificates themselves

Then validate, and reload only if validation passes:

docker exec swag nginx -t && docker exec swag nginx -s reload

Confirm the other sites still respond afterwards.

7. Deploy. Name the service, so nothing else in the file is recreated:

cd /srv/swag && docker compose up -d <app>

What the templates already decide for you

Both templates are in ../deploy/server/, commented in place. The decisions worth knowing before you read them:

No ports:. SWAG reaches the container by name on the project network. That is what keeps the app's port off the internet even if the firewall is wrong.

No networks: either. Being in that compose file is what puts it on the right one.

user: "1000:1000". Container uid is host uid — there is no translation layer. Without this the process runs as whatever the image declares, usually root, and everything it writes into /srv/data lands root-owned.

mem_limit:, not deploy: resources: limits:. The latter is Swarm-only and silently ignored by docker compose up. 0 is unlimited, which is the honest default until the box has been watched under real load.

logging: with a size cap. Without it, json-file logs grow until the disk does not.

listen 443 ssl without http2. http2 on; is set once at http scope in nginx.conf, and the listen ... http2 parameter is deprecated in nginx 1.25 and later.

proxy_read_timeout 86400s only if the app holds a WebSocket open. SWAG's proxy.conf already sets the Upgrade headers; what it also sets is a read timeout of a couple of minutes, which closes an idle socket on the minute. Delete those two lines if there is no socket.

Things that have already bitten

Disk. 37 GB, and it has been full. Three causes, all recurring: :latest orphans the previous image on every pull and nothing cleans them up (a weekly docker image prune -f cron handles it — no -a, so pinned images survive); journald defaults to 10% of the filesystem and is capped here at 200 MB; and VS Code Remote installs a fresh server per version under ~/.vscode-server forever.

cp -a "$SRC/." with an unset SRC copies the root filesystem. Use "${SRC:?}" and echo the value before running anything destructive with it.

SWAG never touches your live confs, only the .sample files beside them. That is deliberate and it means they drift for years. Diff them occasionally:

cd /etc/config/swag/nginx && for f in *.conf site-confs/*.conf proxy-confs/*.subdomain.conf; do echo "=== $f"; diff "$f" "$f.sample"; done

Take the samples wholesale and re-apply your own edits. nginx.conf and the site/proxy confs must be updated together — older ones include ssl.conf at http scope, newer ones per server block, and mixing the two produces no "ssl_certificate" is defined.

SQLite is never safe to copy live. Stop the container, or snapshot with VACUUM INTO through the running one. A cp of a WAL-mode database is a coin toss. See server.md.

Secrets in .env, never in the compose file. It is easy to paste a compose file into a chat window; it is harder to do that by accident with a file you had to cat deliberately.