Refresh deploy/ and bin/ from the boilerplate at 09e144d (the release split between the distro and Windows, wsl-setup, the server config, USER 1000, python:3.15-slim) with roledesk's facts put back: no frontend stage, desk, plugins and seed copied, the extras passkeys, embeddings and llm, roles as the table the backup and the importer require, bin/pull.sh and bin/worker.sh with their profile commands, and Install Server.md and Install WSL.md as copies in place of the five old deployment docs
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
83e965dfeb
commit
a1f281a23a
40 files changed
+2226
-1287
No files matched your search
+45
-21
@@ -1,52 +1,76 @@
|
||||
# Deploy
|
||||
|
||||
Everything needed to put this app on the rittershofer.org box behind SWAG: build an image on Windows, publish it to Gitea, run it as one more service in the compose file that already runs the proxy, back up its database, and give it a password that survives being on the open internet.
|
||||
Everything needed to put roledesk on a server behind SWAG ([../docs/Install Server.md](../docs/Install%20Server.md) is the box it was written for, a copy of the boilerplate's file): test it in WSL, build an image on Windows with `wslc`, publish it to Gitea, run it as one more service in the compose file that already runs the proxy, back up its database, and give it a password that survives being on the open internet.
|
||||
|
||||
Extracted from newsdesk, which is the worked example — every decision here has been run in production at least once, and the comments say why rather than what.
|
||||
This folder is a downstream copy of `../boilerplate/deploy` (commit 09e144d), refreshed with `git archive`; a fix to a shared script lands upstream first. roledesk's own facts are put back after a refresh: the Dockerfile without a frontend stage and with `desk`, `plugins`, `llm` and `seed`, the extras `passkeys`, `embeddings` and `llm`, our image name in `release.env.example`, `server/config.example.yaml` for roledesk's keys, `TABLES` and `REQUIRED_TABLES` naming `roles`, and `bin/pull.sh` and `bin/worker.sh`. `tests/test_deploy.py` pins them.
|
||||
|
||||
**It is already in place.** Nothing has to be copied and nothing has to be renamed: the package is `app`, the environment variables are `APP_`, and the files below are where the scripts expect them. `<app>` in the docs is your *service's* name — the hostname, the data directory, the image tag, the compose service — and it is the one thing you choose.
|
||||
The package is `app`, and the files below are where the scripts expect them: the `APP_` variables, the compose service `app` and the hostname `app.example.com` in the templates are placeholders for the server, and `<app>` in the docs is the service's name: the hostname, the data directory, the image tag, the compose service. The domain noun is `role`, so "job" only ever means queued work.
|
||||
|
||||
## Where each file is
|
||||
|
||||
| Here | | |
|
||||
|---|---|---|
|
||||
| `deploy/*.ps1` | the release scripts, run on Windows | |
|
||||
| `deploy/Dockerfile` | read out of the archived commit by step 2 | copies `app`, `pipeline` and `auth` |
|
||||
| `deploy/Dockerfile.dockerignore` | Docker finds it by that name | |
|
||||
| `deploy/1-test-and-promote.sh`, `test-commit.sh`, `_common.sh`, `db.sh` | the gate, the export and the database moves, run in the WSL distro | |
|
||||
| `deploy/2-build-image.ps1`, `3-push-image.ps1`, `_common.ps1` | the image build and push, run on Windows with `wslc` | the distro is a sealed sandbox and does not hold a container socket |
|
||||
| `deploy/wsl-setup.cmd`, `wsl-setup.ps1` | the profile commands, written once per checkout on Windows | |
|
||||
| `bin/backend.sh`, `check.sh`, and ours `pull.sh`, `worker.sh` | the dev commands, run in the WSL distro: the backend, the gate on the working tree, a pull, the worker | |
|
||||
| `.run/serve`, `pull`, `worker` | PyCharm run configurations, left as they are | `serve --reload` = `bin/backend.sh --reload` = `<folder>-backend --reload`; `pull` = `bin/pull.sh` = `<folder>-pull`; `worker` = `bin/worker.sh` = `<folder>-worker` |
|
||||
| `deploy/Dockerfile` | read out of the tree step 1 exports, by step 2 | no frontend stage; copies `app`, `pipeline`, `auth`, `llm`, `desk`, `plugins` and `seed` |
|
||||
| `deploy/Dockerfile.dockerignore` | copied to `.dockerignore` at the root of the exported tree | `wslc` streams the Dockerfile, so only a root ignore file applies |
|
||||
| `deploy/server/*` | committed here, then installed on the server | |
|
||||
| `deploy/server/compose.yaml` | to `/srv/swag/docker-compose.yml`, as one `services:` block | |
|
||||
| `deploy/server/env.example` | to `/srv/swag/.env`, as appended variables, `chmod 600` | |
|
||||
| `deploy/server/config.example.yaml` | to `$APP_DATA/config.yaml`, with the hostname filled in | `deploy.sh` refuses a config without `database` and `auth.enabled` |
|
||||
| `deploy/server/deploy.sh`, `deploy/server/backup.sh` | to `/srv/swag/`, next to the compose file | |
|
||||
| `deploy/server/import.sh`, `deploy/server/import_database.py` | uploaded and run by `db.sh push` | |
|
||||
| `deploy/server/swag/app.subdomain.conf` | to SWAG's `proxy-confs/`, renamed to `<app>.subdomain.conf` | |
|
||||
|
||||
`db.ps1` expects `server/` beside it, which is why `deploy/server/` is committed here as well as installed on the server.
|
||||
`db.sh` expects `server/` beside it, which is why `deploy/server/` is committed here as well as installed on the server.
|
||||
|
||||
**Keep the `.ps1` files to ASCII.** Windows PowerShell 5.1 reads a script without a byte-order mark as Windows-1252, so an em dash becomes three characters ending in a closing quote: the string ends in the middle of itself and the script fails to parse before it runs a line. Write `--`. newsdesk hit this on a comment nobody thought was code, and `tests/test_packaging.py` here holds every script in this folder to it.
|
||||
**Keep every `.ps1` to ASCII.** Windows PowerShell 5.1 reads a script without a byte-order mark as Windows-1252, so an em dash becomes three characters ending in a closing quote: the string ends in the middle of itself and the script fails to parse before it runs a line. Write `--`. newsdesk hit this on a comment nobody thought was code, and `tests/test_packaging.py` here holds every `.ps1` in this folder to it, the release scripts included.
|
||||
|
||||
`wsl-setup.sh` and `wsl-setup.ps1` are the two halves of setting up a checkout that lives in WSL. `wsl-setup.sh` runs in the distro ([../docs/Install WSL.md, Git](../docs/Install%20WSL.md#git)). `wsl-setup.ps1`, or a double-click on `deploy\wsl-setup.cmd`, writes the commands below into the PowerShell profile, with `<folder>` the checkout folder's name (`jobboard` here) or `-Name`. PowerShell 7, the execution policy, `wslc`, the hand-paste fallback and what to check when a command fails are in [../docs/Install WSL.md, PS Profile commands](../docs/Install%20WSL.md#ps-profile-commands).
|
||||
|
||||
| Command | Runs | Does |
|
||||
|---|---|---|
|
||||
| `<folder>-cd` | `Set-Location` | PowerShell into the checkout's folder |
|
||||
| `<folder>-sh` | `wsl.exe` | a shell in the distro, in the checkout |
|
||||
| `<folder>-backend [-v] [--reload]` | bin/backend.sh in WSL | `uv run app serve` on :8790 |
|
||||
| `<folder>-pull [args]` | bin/pull.sh in WSL | `uv run app job pull`, one pull in the foreground (roledesk's) |
|
||||
| `<folder>-worker [args]` | bin/worker.sh in WSL | `uv run app worker`, the queue drained with no web app (roledesk's) |
|
||||
| `<folder>-check` | bin/check.sh in WSL | the gate on the working tree, without promoting |
|
||||
| `<folder>-1test [--push-branch] [--no-tests]` | deploy/1-test-and-promote.sh in WSL | test local main, fast-forward `production`, export it |
|
||||
| `<folder>-2build` | deploy\2-build-image.ps1 on Windows | build the exported tree with `wslc` |
|
||||
| `<folder>-3push` | deploy\3-push-image.ps1 on Windows | push the built image with `wslc` |
|
||||
| `<folder>-0release [--push-branch] [--no-tests]` | the three above | steps 1, 2 and 3 in order, stopping at the first failure |
|
||||
| `<folder>-db pull\|push` | deploy/db.sh in WSL | the database, to and from the server |
|
||||
|
||||
Each command is one line in the profile: `wsl.exe -d <DISTRO> --cd <CHECKOUT> -- bash <script> @args` for the ones that run in the distro, and for steps 2 and 3 the `.ps1` with the execution policy lifted for that run. `DISTRO` and `CHECKOUT` are read from `deploy/release.env` when the setup runs, so rerun it after either changes. `-backend`, `-pull` and `-worker` stay in the foreground until Ctrl-C. The `-0release` to `-3push` names carry the number of the script's file name, so `0<Tab>` after `<folder>-` finds them.
|
||||
|
||||
`.gitignore` already covers `deploy/release.env` and `deploy/backups/`.
|
||||
|
||||
## The docs
|
||||
|
||||
They are one directory up, in [../docs/](../docs), because they are about the hosting rather than about these scripts.
|
||||
They are one directory up, in [../docs/](../docs), because they are about the hosting rather than about these scripts. `Install Server.md` and `Install WSL.md` are copies of the boilerplate's `documentation/` files; the two proposals stay in the boilerplate checkout.
|
||||
|
||||
- **[../docs/hosting.md](../docs/hosting.md)** — what the box is, the seven steps to add a service, what the templates already decide for you, and the things that have already gone wrong on it. Start here for a service that is not on the box yet.
|
||||
- **[../docs/release.md](../docs/release.md)** — the Windows side: `release.env`, the four numbered scripts, what each one refuses to do, and why the tests run in a throwaway worktree.
|
||||
- **[../docs/server.md](../docs/server.md)** — installing, deploying, backups, moving a SQLite database in either direction, rolling back, and what to verify before relying on any of it.
|
||||
- **[../docs/llm-backends.md](../docs/llm-backends.md)** — asking Claude from a deployed app: API key or subscription token, selected by which one is in the environment, with a self-contained reference implementation.
|
||||
- **[../docs/password-protection.md](../docs/password-protection.md)** — one password on a public hostname: the lockout, sessions, passkeys, and how much throttling is actually worth building. `auth/` is the implementation of it.
|
||||
- **[../docs/Install Server.md](../docs/Install%20Server.md)** — what the box is, the seven steps to add a service, what the templates already decide for you, and the things that have already gone wrong on it. Start here for a service that is not on the box yet.
|
||||
- **[../docs/Install Server.md, Publishing a release](../docs/Install%20Server.md#publishing-a-release)** — the release side: `release.env`, the numbered scripts and where each runs, what each one refuses to do, and why the tests run in a throwaway worktree.
|
||||
- **[../docs/Install Server.md, Operating it](../docs/Install%20Server.md#operating-it)** — installing, deploying, backups, moving a SQLite database in either direction, rolling back, and what to verify before relying on any of it.
|
||||
- **[Proposal LLM Backends.md](../../boilerplate/documentation/Proposal%20LLM%20Backends.md)** (boilerplate checkout) — asking Claude from a deployed app: API key or subscription token, selected by which one is in the environment. `llm/` is the implementation of it, see [../llm/README.md](../llm/README.md).
|
||||
- **[Proposal Password Protection.md](../../boilerplate/documentation/Proposal%20Password%20Protection.md)** (boilerplate checkout) — one password on a public hostname: the lockout, sessions, passkeys, and how much throttling is actually worth building. `auth/` is the implementation of it, see [../auth/README.md](../auth/README.md).
|
||||
|
||||
## A new service, in order
|
||||
|
||||
1. DNS, then certificate, then registry login, then data directory — [../docs/hosting.md](../docs/hosting.md).
|
||||
1. DNS, then certificate, then registry login, then data directory — [../docs/Install Server.md](../docs/Install%20Server.md).
|
||||
2. Copy `release.env.example` to `release.env` and fill it in. That is the only file this step touches: everything else is already committed.
|
||||
3. `.\deploy\0-release.ps1` to test, promote, build and publish.
|
||||
4. Install the server files, edit `.env`, `bash deploy.sh` — [../docs/server.md](../docs/server.md).
|
||||
5. Put the model credential in `.env` if the app needs one — [../docs/llm-backends.md](../docs/llm-backends.md).
|
||||
6. Get in, rotate the bootstrap password, register a passkey — [../docs/password-protection.md](../docs/password-protection.md).
|
||||
3. `<folder>-0release` from Windows to test, promote, build and publish. `wslc` and its registry login are on Windows, [../docs/Install Server.md, Publishing a release](../docs/Install%20Server.md#publishing-a-release).
|
||||
4. Install the server files and `config.yaml`, edit `.env`, `bash deploy.sh` — [../docs/Install Server.md, Operating it](../docs/Install%20Server.md#operating-it).
|
||||
5. Put the model credential in `.env` if the app needs one — [../llm/README.md](../llm/README.md).
|
||||
6. Get in, rotate the bootstrap password, register a passkey — [../auth/README.md](../auth/README.md).
|
||||
7. Schedule `backup.sh`, then restore one of its backups somewhere else to prove it works.
|
||||
|
||||
Step 1 of `0-release.ps1` runs `pytest` in a throwaway worktree, with `ruff` and `mypy` commented out beside it: this repository's dev group is pytest and httpx, and a gate whose tool is not installed fails every release for a reason that has nothing to do with the code. Add the tools, then uncomment the gates.
|
||||
Step 1 runs `pytest` in a throwaway worktree through `test-commit.sh`, with `ruff` and `mypy` commented out beside it, as the release has always run. The dev group has both, so uncommenting them is the owner's call: the tree has to pass them first.
|
||||
|
||||
## What this is not
|
||||
|
||||
Not a framework and not a generator: nothing here is imported or extended, it is copied and edited. That is the point — three services with three edited copies beats three services sharing one abstraction that has to be right for all of them. What this repository changes is only *where the copy comes from*: `boilerplate/deploy/` is the source of truth, and news and health hold copies of it.
|
||||
Not a framework: nothing here is imported or extended, it is copied and edited. `boilerplate/deploy/` is the source of truth, and news, health, jobboard and pocketmoney hold copies of it.
|
||||
Reference in new issue
Block a user