diff --git a/.gitea/workflows/ci.yml b/.gitea/workflows/ci.yml index eab3c9e..f007221 100644 --- a/.gitea/workflows/ci.yml +++ b/.gitea/workflows/ci.yml @@ -4,7 +4,8 @@ # A pull request: the same tests and coverage, then the firmware (from the build cache). # A branch's pushes run nothing by themselves: its pull request runs, once. # A tag v*: the tests, then the firmware built once, clean, signed and published as a -# Gitea release. +# Gitea release. The site is then rebuilt: its home page and Downloads name +# the latest release when they are built (issue #79). # Run by hand: the release of a tag that exists already (the ones from before CI). # # The job runs in a plain Python image, as scripts/ci.sh does on a developer's machine, with the @@ -153,3 +154,14 @@ jobs: GITEA_REPO: ${{ github.repository }} GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }} run: scripts/release_publish.py dist + + # The site names the latest release on its home page and lists them all on Downloads, both + # read when it is built: so it is rebuilt now (issue #79, as the Site workflow does). + - name: Refresh the site + if: steps.release.outputs.tag != '' + env: + SITE_DEPLOY_KEY: ${{ secrets.SITE_DEPLOY_KEY }} + SITE_DEPLOY_HOST: ${{ secrets.SITE_DEPLOY_HOST }} + SITE_DEPLOY_USER: ${{ secrets.SITE_DEPLOY_USER }} + SITE_DEPLOY_KNOWN_HOSTS: ${{ secrets.SITE_DEPLOY_KNOWN_HOSTS }} + run: scripts/site_refresh.sh diff --git a/.gitea/workflows/site.yml b/.gitea/workflows/site.yml index 4db97e1..d438214 100644 --- a/.gitea/workflows/site.yml +++ b/.gitea/workflows/site.yml @@ -1,5 +1,7 @@ # The project site (docs/milestones/W1.md): built with Zola to see that it builds and that its pages -# are sound. Publishing is the maintainer's: the web server pulls main and runs `zola build`. +# are sound. After a push to main it is then published: the job asks the web server, over SSH, to +# pull main and rebuild (issue #79, scripts/site_refresh.sh). The key it holds can run that one +# command there and nothing else; the server, the user and the keys are secrets, not in this file. # # It runs when the site, or a document the site is built from, changes (a pull request, or a push to # main); the firmware workflow (ci.yml) skips a change that touches only these files. A change that @@ -9,9 +11,9 @@ name: Site on: push: branches: [main] - paths: ['site/**', 'docs/**', 'README.md', 'CONTEXT.md', 'src/main.cpp', 'lib/core/src/app_keys.h', '.gitea/workflows/site.yml'] + paths: ['site/**', 'docs/**', 'README.md', 'CONTEXT.md', 'src/main.cpp', 'lib/core/src/app_keys.h', '.gitea/workflows/site.yml', 'scripts/site_refresh.sh'] pull_request: - paths: ['site/**', 'docs/**', 'README.md', 'CONTEXT.md', 'src/main.cpp', 'lib/core/src/app_keys.h', '.gitea/workflows/site.yml'] + paths: ['site/**', 'docs/**', 'README.md', 'CONTEXT.md', 'src/main.cpp', 'lib/core/src/app_keys.h', '.gitea/workflows/site.yml', 'scripts/site_refresh.sh'] jobs: build: @@ -51,3 +53,14 @@ jobs: - name: Check the pages run: python3 site/tools/check_site.py /tmp/site-out + + # Only what has been merged, and only once it has built and passed the checks above. A pull + # request never gets here, and the secrets are given to this step alone. + - name: Publish the site + if: github.event_name == 'push' && github.ref == 'refs/heads/main' + env: + SITE_DEPLOY_KEY: ${{ secrets.SITE_DEPLOY_KEY }} + SITE_DEPLOY_HOST: ${{ secrets.SITE_DEPLOY_HOST }} + SITE_DEPLOY_USER: ${{ secrets.SITE_DEPLOY_USER }} + SITE_DEPLOY_KNOWN_HOSTS: ${{ secrets.SITE_DEPLOY_KNOWN_HOSTS }} + run: scripts/site_refresh.sh diff --git a/docs/milestones/W1.md b/docs/milestones/W1.md index ceae1dd..94e2432 100644 --- a/docs/milestones/W1.md +++ b/docs/milestones/W1.md @@ -21,7 +21,7 @@ The home page was designed on a canvas in a Claude chat (a dark and a light them | Q175 | The site lives in this repository, in `site/`, so the documentation is built from `docs/`, `CONTEXT.md` and the README instead of being copied. | | Q176 | **Zola,** like the blog. The design becomes a template, its tokens CSS custom properties. Dark and light follow the visitor's setting, with a visible switch. No JavaScript except the flasher's. | | Q177 | Domain: **roro9stack.net.** The blog stays at experiments.twis.la. | -| Q178 | **Publishing is the blog's way:** the web server pulls `main` and runs `zola build`; that part is the maintainer's. Changes reach `main` through pull requests as everywhere. **CI is split:** a dedicated `site` job builds the site (`zola build`) when `site/`, `docs/`, `README.md` or `CONTEXT.md` change, and the firmware tests and builds skip a change that touches nothing else. A change that touches both runs both. | +| Q178 | **Publishing is the blog's way:** the web server pulls `main` and runs `zola build`; that part is the maintainer's. *(Since issue #79, CI asks the server to do it: see "Published by CI" below.)* Changes reach `main` through pull requests as everywhere. **CI is split:** a dedicated `site` job builds the site (`zola build`) when `site/`, `docs/`, `README.md` or `CONTEXT.md` change, and the firmware tests and builds skip a change that touches nothing else. A change that touches both runs both. | | Q179 | Phases, each its own pull request: **1.** the CI split, the home page, an Install page with the browser flasher, downloads and the changelog. **2.** a user guide page per App. **3.** how-tos and the FAQ. **4.** developer docs generated from the repository. | | Q180 | **A browser flasher** (ESP Web Tools), **without copying the firmware.** Caddy, in front of Gitea, adds `Access-Control-Allow-Origin: https://roro9stack.net` (and `Vary: Origin`) to GET and HEAD on `/twisla/roro9stack/releases/download/*` and `/api/v1/repos/twisla/roro9stack/releases*`: both are public already. The Install page asks the API for the latest release in the browser, finds the asset ending `-factory.bin`, and gives ESP Web Tools a manifest built on the spot, so it offers a new release as soon as it exists, with no rebuild. The library is **vendored** into `site/static/` (Apache-2.0), not loaded from a CDN. The file's SHA-256 is shown on the page. Chrome or Edge on a desktop only; other browsers, and visitors without JavaScript, get the `esptool` steps on the same page. | | Q181 | Docs for the latest version only. The changelog is the Gitea releases, read at build time. | @@ -125,3 +125,46 @@ Not one of the planned phases: the blog's seven roro9stack posts, imported into - **Left out on purpose:** the M0 and M1 milestone documents and `CONTEXT.md` (the glossary) describe Wi-Fi monitoring, which the site does not publish. They stay in the repository. - **The Debug Console pages were written against the source and the live console:** the protocol (the token line, the banner, the 4 KB backlog, one client, 8 queued commands, 240-byte lines, `denied` after a second) and the replies shown were checked on a Debug Build, v0.11.0-3, over Wi-Fi. Not run: `crash abort`, `crash wdt` and Safe Mode, which are described from ADR 0005 and the code. - **Found while writing it:** the README's table lacked the `gnss` commands (rows added); piping commands into `rdbg.py` returns before the replies unless the input stays open (documented, not changed); `update install` on a Debug Build needs `force` (documented). + +## Published by CI (issue #79, design round 2026-10-07) + +Q178 left publishing to the maintainer: a merge, then a command typed on the web server. It was forgotten often enough. + +| # | Decision | +|---|---| +| Q214 | **A plain ed25519 key with a forced command**, not a certificate: one line in the web server user's `authorized_keys`, `restrict,command="/full/path/to/the/refresh"`. `restrict` takes away the terminal and every forwarding. A certificate could carry the same and an expiry date, at the price of a CA to keep and a key to sign again each time: too much for one key and one command. | +| Q215 | **The CI sends no command.** The server runs the forced one whatever is asked for, so there is nothing to keep secret about it and nothing a leaked key could choose. The full path is written once, on the server (a command over SSH doesn't get the user's login `PATH`). | +| Q216 | Four secrets: `SITE_DEPLOY_KEY`, `SITE_DEPLOY_HOST` (or `host:port`), `SITE_DEPLOY_USER`, and **`SITE_DEPLOY_KNOWN_HOSTS`**, the server's host key: the job connects to that server or to nothing. None of them is in the repository, which is public. | +| Q217 | `from=""` on the same line: the key works from the runner only. | +| Q218 | **The last step of the Site workflow**, after the build and the checks, on a push to `main` only. A pull request never reaches it, and the secrets are given to that step alone. | +| Q219 | **After a release too.** The Install page asks Gitea for the latest release when it is opened, but the home page and Downloads read it when the site is built: so the release workflow refreshes the site once the release is published. | +| Q220 | A refresh that fails makes the run red, with what the server's script printed: it has to exit with an error when it fails. | +| Q221 | Two refreshes at once are the server script's to refuse or queue (`flock`). | +| Q222 | The key is a file only while the step runs, in the job's container, as the signing key is. | + +### As built + +- **`scripts/site_refresh.sh`** is what both workflows run: it writes the key and the host key to a temporary folder, connects with no configuration but its own line (`-F none`, strict host key checking, that one key, no command), and removes them. With none of the four secrets it does nothing and says so (a fork, or a repository without them); with only some it fails. +- **`scripts/site_deploy_keygen.sh`** makes the key pair once, in `~/.config/roro9stack/`, and prints the `authorized_keys` line and what goes in each secret. It never prints the private key. +- **The server's script** should start like this, for Q220 and Q221: + + ```sh + #!/bin/sh + set -e + exec 9>/tmp/rororefresh.lock + flock -w 120 9 + ``` + +### Checks (2026-10-07, against an SSH server in a throwaway container) + +| Check | Result | +|---|---| +| The refresh | The forced command runs as the server's user; the script ends with `site refresh: done` | +| The same key, asking for `id; cat /etc/passwd` | The refresh runs instead; what was asked for is only handed to it as text | +| A terminal | Refused: `PTY allocation request failed` | +| `scp` with the key | Nothing is copied | +| Another host key in the secret | `Host key verification failed`, the run fails, nothing is sent | +| The server's script exits with an error | So does the step | +| No secrets at all; only one of the four | Does nothing and says so; fails and says which are needed | + +**Not checked:** the real web server and the runner, which wait for the key to be installed: whether the runner reaches the server's SSH port is the first thing the first run will tell. Port forwarding, which `restrict` switches off, was not tried. `from=` was not tried either. diff --git a/scripts/site_deploy_keygen.sh b/scripts/site_deploy_keygen.sh new file mode 100755 index 0000000..5165c8a --- /dev/null +++ b/scripts/site_deploy_keygen.sh @@ -0,0 +1,52 @@ +#!/usr/bin/env bash +# Creates the key CI uses to ask the web server for a site refresh, once (issue #79, +# docs/milestones/W1.md), and says where each half goes. The private key stays in +# ~/.config/roro9stack/ until it is pasted into the Gitea secret; it is never committed and this +# script doesn't print it. +# +# scripts/site_deploy_keygen.sh [/full/path/to/rororefresh.sh] [the runner's address] +set -euo pipefail +KEY="${RORO_SITE_DEPLOY_KEY:-$HOME/.config/roro9stack/site-deploy-key}" +COMMAND="${1:-/full/path/to/rororefresh.sh}" +FROM="${2:-}" + +if [ -e "$KEY" ]; then + echo "A site deploy key already exists at $KEY; not overwriting it." >&2 +else + mkdir -p "$(dirname "$KEY")" + ( umask 077; ssh-keygen -q -t ed25519 -N "" -C roro9stack-ci-site-refresh -f "$KEY" ) +fi + +options="restrict,command=\"$COMMAND\"" +[ -z "$FROM" ] || options="from=\"$FROM\",$options" + +cat < Actions > Secrets: + + SITE_DEPLOY_KEY the whole of $KEY (the private key, with its BEGIN and END lines) + SITE_DEPLOY_HOST the server's address as the runner reaches it, or address:port + SITE_DEPLOY_USER that user's name + SITE_DEPLOY_KNOWN_HOSTS the server's host key, one line, from a machine you trust the network of: + ssh-keyscan -t ed25519
(or: -p
) + and compare it with the server's own: + ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub (on the server) + ssh-keyscan -t ed25519
| ssh-keygen -lf - (here) + +3. Try it, from here, with the same four values in the environment: + + SITE_DEPLOY_KEY="\$(cat $KEY)" SITE_DEPLOY_HOST=... SITE_DEPLOY_USER=... \\ + SITE_DEPLOY_KNOWN_HOSTS="\$(ssh-keyscan -t ed25519 ... 2>/dev/null)" scripts/site_refresh.sh + + (with from= set, this works from the runner's address only.) Then, to see that the key can do + nothing else: ssh -i $KEY @
id must run the refresh, not \`id\`. + +Once the secret is in Gitea, the copy at $KEY can be deleted. +TEXT diff --git a/scripts/site_refresh.sh b/scripts/site_refresh.sh new file mode 100755 index 0000000..1a7bb14 --- /dev/null +++ b/scripts/site_refresh.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +# Asks the web server to rebuild the site (issue #79, docs/milestones/W1.md). Run by CI after a push +# to main that changed the site, and after a release is published (the home page and Downloads +# name the latest release when they are built). +# +# It only connects: the server's authorized_keys line forces the one command this key may run, so +# nothing sent from here chooses what happens there. From the environment (Gitea secrets): +# SITE_DEPLOY_KEY the private key (scripts/site_deploy_keygen.sh makes it) +# SITE_DEPLOY_HOST the server, or server:port +# SITE_DEPLOY_USER the user there +# SITE_DEPLOY_KNOWN_HOSTS the server's host key, as a known_hosts line: nothing else is trusted +# With none of them set it does nothing (a fork, or before the key is installed); with only some, it fails. +set -euo pipefail + +set_count=0 +for v in SITE_DEPLOY_KEY SITE_DEPLOY_HOST SITE_DEPLOY_USER SITE_DEPLOY_KNOWN_HOSTS; do + [ -z "${!v:-}" ] || set_count=$((set_count + 1)) +done +if [ "$set_count" = 0 ]; then + echo "site refresh: no SITE_DEPLOY_* secrets here, nothing done" + exit 0 +fi +if [ "$set_count" != 4 ]; then + echo "site refresh: SITE_DEPLOY_KEY, _HOST, _USER and _KNOWN_HOSTS are needed, and only $set_count of them are set" >&2 + exit 1 +fi + +if ! command -v ssh >/dev/null; then + apt-get update -qq + apt-get install -y -qq --no-install-recommends openssh-client >/dev/null +fi + +host="$SITE_DEPLOY_HOST" port=22 +case "$host" in + *:*) port="${host##*:}" host="${host%:*}" ;; +esac + +# The key and the host key exist as files only while this runs, in a container that goes with the job. +umask 077 +tmp="$(mktemp -d)" +trap 'rm -rf "$tmp"' EXIT +printf '%s\n' "$SITE_DEPLOY_KEY" > "$tmp/key" +printf '%s\n' "$SITE_DEPLOY_KNOWN_HOSTS" > "$tmp/known_hosts" + +# -F none: no configuration but this line. -T and no command: the server's forced command runs. +ssh -F none -T -p "$port" -i "$tmp/key" \ + -o IdentitiesOnly=yes -o BatchMode=yes \ + -o StrictHostKeyChecking=yes -o UserKnownHostsFile="$tmp/known_hosts" -o GlobalKnownHostsFile=/dev/null \ + -o ConnectTimeout=20 -o ServerAliveInterval=15 -o ServerAliveCountMax=8 \ + "$SITE_DEPLOY_USER@$host" +echo "site refresh: done" diff --git a/site/content/dev/milestones/w1.md b/site/content/dev/milestones/w1.md index 6c531a8..3e55d74 100644 --- a/site/content/dev/milestones/w1.md +++ b/site/content/dev/milestones/w1.md @@ -29,7 +29,7 @@ The home page was designed on a canvas in a Claude chat (a dark and a light them | Q175 | The site lives in this repository, in `site/`, so the documentation is built from `docs/`, `CONTEXT.md` and the README instead of being copied. | | Q176 | **Zola,** like the blog. The design becomes a template, its tokens CSS custom properties. Dark and light follow the visitor's setting, with a visible switch. No JavaScript except the flasher's. | | Q177 | Domain: **roro9stack.net.** The blog stays at experiments.twis.la. | -| Q178 | **Publishing is the blog's way:** the web server pulls `main` and runs `zola build`; that part is the maintainer's. Changes reach `main` through pull requests as everywhere. **CI is split:** a dedicated `site` job builds the site (`zola build`) when `site/`, `docs/`, `README.md` or `CONTEXT.md` change, and the firmware tests and builds skip a change that touches nothing else. A change that touches both runs both. | +| Q178 | **Publishing is the blog's way:** the web server pulls `main` and runs `zola build`; that part is the maintainer's. *(Since issue #79, CI asks the server to do it: see "Published by CI" below.)* Changes reach `main` through pull requests as everywhere. **CI is split:** a dedicated `site` job builds the site (`zola build`) when `site/`, `docs/`, `README.md` or `CONTEXT.md` change, and the firmware tests and builds skip a change that touches nothing else. A change that touches both runs both. | | Q179 | Phases, each its own pull request: **1.** the CI split, the home page, an Install page with the browser flasher, downloads and the changelog. **2.** a user guide page per App. **3.** how-tos and the FAQ. **4.** developer docs generated from the repository. | | Q180 | **A browser flasher** (ESP Web Tools), **without copying the firmware.** Caddy, in front of Gitea, adds `Access-Control-Allow-Origin: https://roro9stack.net` (and `Vary: Origin`) to GET and HEAD on `/twisla/roro9stack/releases/download/*` and `/api/v1/repos/twisla/roro9stack/releases*`: both are public already. The Install page asks the API for the latest release in the browser, finds the asset ending `-factory.bin`, and gives ESP Web Tools a manifest built on the spot, so it offers a new release as soon as it exists, with no rebuild. The library is **vendored** into `site/static/` (Apache-2.0), not loaded from a CDN. The file's SHA-256 is shown on the page. Chrome or Edge on a desktop only; other browsers, and visitors without JavaScript, get the `esptool` steps on the same page. | | Q181 | Docs for the latest version only. The changelog is the Gitea releases, read at build time. | @@ -133,3 +133,46 @@ Not one of the planned phases: the blog's seven roro9stack posts, imported into - **Left out on purpose:** the M0 and M1 milestone documents and `CONTEXT.md` (the glossary) describe Wi-Fi monitoring, which the site does not publish. They stay in the repository. - **The Debug Console pages were written against the source and the live console:** the protocol (the token line, the banner, the 4 KB backlog, one client, 8 queued commands, 240-byte lines, `denied` after a second) and the replies shown were checked on a Debug Build, v0.11.0-3, over Wi-Fi. Not run: `crash abort`, `crash wdt` and Safe Mode, which are described from ADR 0005 and the code. - **Found while writing it:** the README's table lacked the `gnss` commands (rows added); piping commands into `rdbg.py` returns before the replies unless the input stays open (documented, not changed); `update install` on a Debug Build needs `force` (documented). + +## Published by CI (issue #79, design round 2026-10-07) + +Q178 left publishing to the maintainer: a merge, then a command typed on the web server. It was forgotten often enough. + +| # | Decision | +|---|---| +| Q214 | **A plain ed25519 key with a forced command**, not a certificate: one line in the web server user's `authorized_keys`, `restrict,command="/full/path/to/the/refresh"`. `restrict` takes away the terminal and every forwarding. A certificate could carry the same and an expiry date, at the price of a CA to keep and a key to sign again each time: too much for one key and one command. | +| Q215 | **The CI sends no command.** The server runs the forced one whatever is asked for, so there is nothing to keep secret about it and nothing a leaked key could choose. The full path is written once, on the server (a command over SSH doesn't get the user's login `PATH`). | +| Q216 | Four secrets: `SITE_DEPLOY_KEY`, `SITE_DEPLOY_HOST` (or `host:port`), `SITE_DEPLOY_USER`, and **`SITE_DEPLOY_KNOWN_HOSTS`**, the server's host key: the job connects to that server or to nothing. None of them is in the repository, which is public. | +| Q217 | `from=""` on the same line: the key works from the runner only. | +| Q218 | **The last step of the Site workflow**, after the build and the checks, on a push to `main` only. A pull request never reaches it, and the secrets are given to that step alone. | +| Q219 | **After a release too.** The Install page asks Gitea for the latest release when it is opened, but the home page and Downloads read it when the site is built: so the release workflow refreshes the site once the release is published. | +| Q220 | A refresh that fails makes the run red, with what the server's script printed: it has to exit with an error when it fails. | +| Q221 | Two refreshes at once are the server script's to refuse or queue (`flock`). | +| Q222 | The key is a file only while the step runs, in the job's container, as the signing key is. | + +### As built + +- **`scripts/site_refresh.sh`** is what both workflows run: it writes the key and the host key to a temporary folder, connects with no configuration but its own line (`-F none`, strict host key checking, that one key, no command), and removes them. With none of the four secrets it does nothing and says so (a fork, or a repository without them); with only some it fails. +- **`scripts/site_deploy_keygen.sh`** makes the key pair once, in `~/.config/roro9stack/`, and prints the `authorized_keys` line and what goes in each secret. It never prints the private key. +- **The server's script** should start like this, for Q220 and Q221: + + ```sh + #!/bin/sh + set -e + exec 9>/tmp/rororefresh.lock + flock -w 120 9 + ``` + +### Checks (2026-10-07, against an SSH server in a throwaway container) + +| Check | Result | +|---|---| +| The refresh | The forced command runs as the server's user; the script ends with `site refresh: done` | +| The same key, asking for `id; cat /etc/passwd` | The refresh runs instead; what was asked for is only handed to it as text | +| A terminal | Refused: `PTY allocation request failed` | +| `scp` with the key | Nothing is copied | +| Another host key in the secret | `Host key verification failed`, the run fails, nothing is sent | +| The server's script exits with an error | So does the step | +| No secrets at all; only one of the four | Does nothing and says so; fails and says which are needed | + +**Not checked:** the real web server and the runner, which wait for the key to be installed: whether the runner reaches the server's SSH port is the first thing the first run will tell. Port forwarding, which `restrict` switches off, was not tried. `from=` was not tried either.