roro9stack/site/content/dev/milestones/r1.md

+++ title = "Releases" description = "A tag is a release, built the same way every time and published where a device can find it." weight = 70

[extra] docs = true source = "docs/milestones/R1.md" tag = "R1" +++ Status: in progress. CI and signed releases on Gitea (issue #5) are in place since 2026-10-06: every tag from v0.1.0 to v0.10.0 has its release. Updates from Gitea (issue #6) is built and checked on the device, on branch gitea-updates, not merged yet. The Issues App (#4) comes after.

Goal: a tag is a release, built the same way every time and published where a device can find it.

CI and releases (issue #5)

Until now the tests, the builds, the signing and the flashing all happened on one machine, through scripts/ci.sh and scripts/flash.sh. Nothing was published.

Decisions (design round 2026-10-06)

#	Decision
Q151	A push to main: the host tests (with their coverage). A push to another branch: nothing, its pull request is what runs (a branch with an open pull request ran twice, once for each). A pull request: the same and both builds; changes reach main through pull requests. A tag v*: all of it, then a release. (First: everything on every push, which rebuilt the firmware far more often than anyone looked at it.)
Q152	CI signs. The signing key is the repository secret OTA_SIGNING_KEY; a tag push makes a complete, signed release with no manual step (ADR 0008).
Q153	Revised by Q188: there is one firmware. The Debug Build is built in CI with a token of the runner's own, to prove it compiles, and isn't published: it would hand everyone its Debug Console token.
Q154	Pull requests from forks don't start a run.
Q155	A release carries roro9stack-<version>.ota (signed), -factory.bin for USB, .elf.gz to decode crashes, and SHA256SUMS.
Q156	Its text is the tag's message, what the files are, and the commits since the tag before.
Q157	No cache service to begin with: measure first.
Q158	Reproducible builds aren't needed for signing any more (Q152); not pursued here.
Q159	The tags from before CI get their releases too, v0.1.0 to v0.10.0, built from each tag's own sources by running the workflow by hand.
Q160	Actions is switched on for the repository.
Q161	Changes reach main through pull requests, merged as "rebase, then a merge commit", the only style the repository allows: the branch's commits keep their messages, the merge commit marks the pull request, and what CI tested is what lands. No squash, no fast-forward.

As built

• One workflow, .gitea/workflows/ci.yml, one job, on the runner runner0 (label ubuntu). The job asks for a python:3.12-slim container, installs git, a compiler, openssl and PlatformIO, and runs the same scripts as a developer's machine. No Docker inside the job.
• The cache is a Docker volume, roro9stack-pio, mounted at /pio; the runner's config.yaml allows it under container.valid_volumes. A first run downloads about 1 GB and rebuilds the framework. Measured from the jobs' own start and end times: the first full run on an empty cache took 11.4 minutes (tests and both builds); the first run in a container, 8.1; a pull request now takes about 8.7 (tests, coverage and both builds), a release build alone 5.5, and a push to main (tests and coverage) 1.1. (An earlier version of this note said 17 minutes: that was the waiting time, not the job's.)
• No JavaScript actions, so the image needs no Node and nothing is fetched from GitHub: the checkout is four git commands.
• scripts/_docker.sh runs the command in place when RORO_NO_DOCKER is set (a CI job is already in a build container), and in the project's image otherwise. The Debug Build's token is made on the spot in CI and goes with the container.
• scripts/release_build.sh <checkout> <out> builds a tag's own sources with today's tools, signs, verifies against the public key in those sources, and writes the files and the release's text. scripts/release_publish.py creates the Gitea release or completes it; run twice, it replaces what's there. Both run the same on a developer's machine.
• scripts/ota_verify.py checks an Update File as a device does, on a PC.
• The job's own token (secrets.GITEA_TOKEN) is enough to create a release and upload its files.
• Old tags. v0.1.0 to v0.3.0 are from before the framework was rebuilt with our settings (ADR 0006) and can't link against a rebuilt one left in the cache: the release build puts the stock framework libraries back for them. v0.1.0 to v0.2.1 have no public key in their sources (Firmware Updates came with v0.3.0); their files are checked against today's.

How it went

• The runner's label took three tries. Registered as ubuntu://docker:ubuntu:resolute and then as ubuntu::docker://..., Gitea took the whole string for the label's name; with the first, jobs ran on the runner's host itself. The first version of the workflow was written for that (plain shell, docker run for the build) and published v0.10.0 that way. ubuntu:docker://docker.gitea.com/runner-images:ubuntu-latest is the form that works.
• Gitea 1.27's API can't cancel a run that isn't finished, only delete a finished one; switching Actions off and on for the repository doesn't either. Runs queued for a label that no longer exists stay queued until cancelled in the web UI.
• CI's image isn't byte-identical to a local build of the same tag (same size, different bytes). Not pursued (Q158).

Updates from Gitea (issue #6)

The device looks at the project's Gitea for a newer release, says so, and installs it on request, with the same signed Update Files, Probation and rollback as a push from the PC or an install from the card.

What the server gives (checked 2026-10-06)

• Its certificate is Let's Encrypt, all ECDSA: leaf git.twis.la (renewed every few months, next expiry 2026-12-14) under the intermediate YE2, Root YE and ISRG Root X2, which X1 cross-signs. Pinning the leaf would ask a question at every renewal.
• The API answers over HTTP/1.1, chunked: releases/latest is 3.2 KB (about 350 bytes of it matter), a list of ten releases is 33 KB.
• A download is a direct 200 with Content-Length and no redirect; ranges work.

Decisions (design round 2026-10-06)

#	Decision
Q162	Trust: the firmware carries ISRG Root X1 and X2 and checks the server's chain and name against them, not the framework's bundle of about 130 CAs (ADR 0009). Shared with #4. If the server moves to another CA, the next firmware comes from the PC.
Q163	The Update File's own signature stays the real guard. A hijacked connection could hide a release or offer an older signed one, never install firmware that isn't ours.
Q164	The source, git.twis.la and twisla/roro9stack, is a constant in the firmware. A fork changes it, and has its own key.
Q165	When: on request in Settings > Firmware, and once a day in the background while Wi-Fi is up and the Clock is set (certificate dates need it). A setting, Check for updates, on by default. It installs nothing by itself; it skips quietly below the memory floor and never runs during an install.
Q166	A Toast, "Update v0.11.0 available: see Settings > Firmware", once per version per boot.
Q167	The download goes straight into the inactive slot, no card needed. A truncated or tampered file is refused after 160 bytes or at its end, and the running firmware is untouched. A failed download starts over.
Q168	A version that failed (rolled back) isn't announced again by the background check until a newer one exists; it can still be installed by hand.
Q169	Older releases: a list of the last ten, newest first, the running one marked. Installing an older one asks with a stronger warning.
Q170	Enter on a release shows its version, date, size and the tag message, with Install.
Q171	Revised by Q188: there is no Debug Build, every firmware installs releases. Debug Builds check and show the latest release, but don't install it: it would replace the Debug Build and its console (Q153: Debug Builds aren't published). Their updates come from the PC.
Q172	Memory, as measured: a TLS connection peaks at about 52 KB of heap, with or without checking the certificate, so it starts with 80 KB free (Q86's 20 KB spare on top), not 55 KB. A check, list or install someone asked for makes IRC step aside and come back after; the daily check never does, and with IRC connected it waits. (First: 55 KB and nothing else. With IRC connected a check left 3 KB and a download 836 bytes.)
Q173	Left out, each with its issue: installing automatically (#52), a release channel (#53), resuming a download (#54).
Q174	Ships as v0.11.0. Tested on the device with the real signed releases; a Debug Build command pretends the device runs an older version, so v0.10.0 counts as an update.

Done when

• A check, by hand or daily, tells the right thing: up to date, newer available, no network, bad certificate, no clock, too little memory.
• A newer release installs from the Firmware page with no card and no PC, and the device restarts into it and confirms it.
• A tampered or truncated download is refused and the running firmware keeps running.
• The Older releases list shows ten, and installing one asks first.
• A release that rolled back isn't announced again.
• A Debug Build shows the latest release and doesn't install it.
• The daily check never runs below the memory floor, during an install, or without a clock.

Work breakdown

1. Model (host-tested): a streaming JSON scanner, the release list read from it, HTTP response heads and chunked bodies, URLs, which release counts as an update.
2. The connection: the root certificates, an HTTPS client, a check and a list from the Update Service's task; console commands to try them.
3. The download: an HTTPS source for the existing install path.
4. The screens: the Firmware page's release rows, the release page, Older releases, the setting, the daily check and its Toast.
5. Checks on the device, recorded here.

As built

• lib/release (host-tested): a streaming JSON scanner, the release reader built on it, HTTP heads and chunked bodies, URLs, and the decisions (which release is an update, whether to announce it, which download URLs are taken). A list of ten releases is 33 KB of JSON and costs a few hundred bytes of memory, because nothing is kept but the path.
• HttpsGet (src/platform): one GET, the answer read as a stream, redirects not followed. GiteaReleases keeps the latest and the list. The Update Service serves the requests on its own task (about 5.4 KB of its 7 KB stack at the peak) and installs through the install path that already existed, with an HTTPS source in place of the card or the TCP port.
• The daily check is scheduled from the Update Service's tick: Wi-Fi up, the Clock set, no Probation, nothing else going on, memory for a connection. The day it last succeeded is kept in flash.
• A version that failed (rolled back) is remembered as ota_failed, and isn't announced again by the daily check.
• The screens: the Firmware page's Latest release and Older releases rows, a release page with the tag's message, and the install dialog.
• Debug Builds get knobs to try what can't be tried otherwise: update pretend, probe, damage and daily.
• The first message, ... available: see Settings > Firmware, was cut at 48 bytes by the notification's own limit; it now reads v0.11.0 is out: see Settings > Firmware.

Checks on the device (2026-10-06, Debug Builds of branch gitea-updates)

Check	Result
Host tests	456 pass
Check and list against the live server	The certificate is accepted against the two embedded roots; releases/latest read; a list of ten (33 KB) streamed
Servers that must be refused	github.com, example.com, expired.badssl.com, self-signed.badssl.com, wrong.host.badssl.com, untrusted-root.badssl.com and the router: each "isn't accepted" or a TLS error
A download cut short at 800,000 bytes	Refused, "update file too short"; the running firmware untouched
One byte flipped in the signature	Refused after 160 bytes, "bad signature"; the image isn't read further
One byte flipped in the image	Downloaded in full, refused at its end, "image corrupted (hash mismatch)"
The real v0.10.0, from the console and then from the screen	Downloaded, restarted, confirmed on Probation: the slot table read v0.10.0, valid both times. The Debug Build was pushed back from the PC after each
The screens	Latest release (checking, then (current) or (new)), the release page with the tag's message, Older releases with ten rows, the install dialog (Cancel by default, Back cancels), the progress screen at 28%
IRC connected, before the hold	A check left 3 KB of heap; a full download, 836 bytes
IRC connected, with the hold	The lowest free heap during a full download: 38 KB. IRC reconnected afterwards (its counters kept growing)
The daily check	It ran by itself, announced v0.10.0 is out: see Settings > Firmware once; with IRC connected (68 KB free) it didn't run
Speed	1.9 MB in about 46 s, 40 KB/s, over the guest Wi-Fi at -65 dBm; not investigated further

Not checked: the certificate's name on its own. Connecting by IP makes the server end the handshake before it shows its certificate, so that test proved nothing; the library sets the name it verifies, and OpenSSL on the PC refused the wrong name against the same chain. A failed daily check retrying, the clock not being set, the release that failed before not being announced (host-tested, not on the device), and the hold when IRC isn't connected but Gemini holds memory.

Limits worth knowing:

• With IRC connected for days, the daily check doesn't run. It would have to take IRC down to make room. Opening Latest release does.
• A server that changes CA can't be reached until a firmware carrying the new root comes from the PC (ADR 0009).
• No resuming: a broken download starts over (#54).
• A key press during the hold: Back on the Firmware page while a check is going doesn't cancel it.

Two slips during the checks: a blind sequence of keys on the Firmware page opened the SD card's install dialog (the page keeps its selection between visits); it was cancelled with Back, nothing installed. And my port-polling while waiting for a restart took the Debug Console's only client slot, which made the first install attempt look like a failure.

One firmware: the Debug Console in every build (issue #68)

The issue asked for a token that could be set, so that Debug Builds could be published. The design round ended somewhere simpler: no Debug Builds. The console is in every firmware, off until switched on, with a token that belongs to the device (ADR 0010, which supersedes part of ADR 0004).

Decisions (design round 2026-10-06)

#	Decision
Q188	One firmware, with the Debug Console and the test commands compiled in. The cardputer-adv-debug environment, RORO_DEBUG, the +debug version and scripts/debug_flags.py go; CI builds one firmware. Revises Q153 and Q171.
Q189	Off by default, and off when the stored setting is missing or invalid. Off, nothing listens and no memory is used: the task and the 4 KB ring exist only while it's on.
Q190	Settings → Debug Console: the switch, where to connect, the token, "New token", "Type a token". It stays on across restarts and in Safe Mode. DBG in the Status Bar while it listens, bright with a client connected.
Q191	The device makes the token the first time the console is switched on: 100 bits as 20 characters of Crockford's base32, shown in groups of four. It can be replaced by one typed by hand, of at least 16 characters. Case and dashes don't count.
Q192	USB serial can set it up: debug on, debug token <value>, debug token new, over USB only; debug status and debug off from anywhere. scripts/flash.sh --debug gives a device the developer's token. The token is never printed.
Q193	Challenge and answer: the device sends 16 random bytes, the client their HMAC-SHA256 keyed by the token. Five wrong answers in a row close the console for a minute, with a Notification. Old clients and old firmwares don't talk to each other: accepted, this far from v1 and with one user.
Q194	rdbg.py takes the token from -t/--token, then $RORO_DEBUG_TOKEN, then ~/.config/roro9stack/debug-token.
Q195	The test commands are in every build (crash abort, crash wdt, update pretend, update damage, lora inject, sd fill, loop spin, wifi ip … try). update install … force goes: there's nothing left to protect.

As built

• lib/debug (host-tested, 9 tests): the token maker, tidying what a person types, HMAC-SHA256 on the project's own SHA-256 (checked against RFC 4231), the answer to a challenge (the same vector as rdbg.py's), a comparison that takes the same time whatever it's given, and the pause after five wrong answers, right across the clock's wrap.
• Two settings, DebugConsole and DebugToken, with the others: validated, and invalid stored values count as missing.
• The console's task and ring come and go with the setting. Console::openRing allocates the ring; switching off closes the socket, frees it and ends the task. tick follows the settings once a second, so a switch off and on again takes a second or two.
• Measured against the two builds it replaces: 1,881,799 bytes of flash and 54,700 of static RAM. The release build was 1,851,387 and 54,612 (so 30 KB more flash and 88 bytes more RAM); the Debug Build was 1,874,887 and 58,780 (4 KB of RAM back: the ring is no longer a static array).
• scripts/rdbg.py answers the challenge, and fetches a release's ELF from Gitea when a crash names a version that isn't in .pio/elves/.

Checks

Check	Result
Host tests	468 pass (456 before)
The firmware builds	One environment, 1,881,799 bytes of flash used
Pushed over Wi-Fi to a device running a Debug Build	Installed, restarted; the update port answers and port 2323 refuses connections: off by default
Switched on at the device (Settings → Debug Console)	A token is made and shown. Its first drawing, in bold at normal size, was misread once: it is now at twice the size, in two lines, and O, I and L are taken for 0 and 1
A login with scripts/rdbg.py	The challenge is answered; info, screenshot and the backlog work
DBG in the Status Bar	There while the console is on, bright while a client is connected (screenshots)
debug on and debug token over the console	Refused: over USB serial only
Six logins with a wrong token	Five are refused, a second apart; the sixth, and the right token after it, get locked. After a minute the right token works again. The console's own log and a Notification say so
Three crash abort in a row	Safe Mode, with the console reachable: info works, ls / says not available in Safe Mode. rdbg.py crash decodes the backtrace to runCommand at the abort() line. reboot leaves Safe Mode
An update over Wi-Fi with the console on	The setting and the token survive: the console is back by itself after the restart
debug off over the console	The client is dropped and port 2323 refuses connections; the update port still answers
Memory, console on with a client	108 KB free (the Debug Build it replaces: 104 KB). In Safe Mode: 178 KB free

Not checked: scripts/flash.sh --debug over USB (no device on USB here); "New token" and "Type a token" from the page (the code paths are the ones debug token uses, which need the cable); the Toast itself on screen (the console is closed while it shows; its Notification is in the log); free memory with the console off, which only the serial port could say (the static figures above are the evidence); fetching a release's ELF, which needs a crash on a released version.