One firmware: the Debug Console in every build, off until switched on, with the device's own token
CI / build (pull_request) Successful in 7m20s
Site / build (pull_request) Successful in 9s

There is no Debug Build any more (ADR 0010, issue #68, Q188 to Q195). The
console and the test commands are compiled into every firmware. It listens
only while Settings > Debug Console is on, which isn't the default; off,
neither its task nor its 4 KB ring exists. The token is made by the device
and shown on that page; a client proves it knows it by answering a challenge
with an HMAC, so it never crosses the network, and five wrong answers close
the console for a minute. DBG in the Status Bar while it listens.

Over USB serial only: debug on, debug token <value>, debug token new.
scripts/flash.sh --debug uses them to set a device up with the developer's
token. scripts/rdbg.py takes the token from -t, $RORO_DEBUG_TOKEN or the
file, answers the challenge, and fetches a release's ELF to decode a crash.

Gone: the cardputer-adv-debug environment, RORO_DEBUG, the +debug version,
scripts/debug_flags.py, update install ... force, and the rule that a Debug
Build doesn't install releases. Old clients and old firmwares don't talk to
each other.

Against the builds it replaces: 30 KB more flash and 88 bytes more static
RAM than the release, 4 KB less RAM than the Debug Build. 468 host tests.
Checked on the device: off by default, login, the pause after wrong tokens,
Safe Mode with the console, the setting surviving an update, debug off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EhqxQ49eCju4CzKYNjZzwT
This commit is contained in:
2026-10-06 22:59:35 +02:00
co-authored by Claude Opus 5.5
parent 1353e6a5f9
commit c68741cc46
65 changed files with 1245 additions and 374 deletions
+7 -8
View File
@@ -6,7 +6,7 @@ weight = 5
tag = "Console"
+++
Rollback (see [How an update works](/dev/build/how-an-update-works/)) protects against **new** firmware that fails Probation. These measures cover firmware that was **confirmed and then crashes**: a corrupt setting, a server that sends something unexpected, a bug that takes an hour to show. They are in **every build**, release and Debug alike; the Debug Console is what makes them convenient. The decision is [ADR 0005](/dev/decisions/0005-safe-mode-crash-reports-watchdog/).
Rollback (see [How an update works](/dev/build/how-an-update-works/)) protects against **new** firmware that fails Probation. These measures cover firmware that was **confirmed and then crashes**: a corrupt setting, a server that sends something unexpected, a bug that takes an hour to show. They are in every firmware; the Debug Console is what makes them convenient. The decision is [ADR 0005](/dev/decisions/0005-safe-mode-crash-reports-watchdog/).
## What a crash leaves behind
@@ -18,7 +18,7 @@ The `crash` command prints it again whenever you like:
```
> crash
crash: last one in v0.9.0-1-g4ab873e-dirty+debug (panic)
crash: last one in v0.9.0-1-g4ab873e-dirty (panic)
crash: task loopTask, pc 0x4037e179, cause 0, address 0x00000000
crash: reason: abort() was called at PC 0x421209b3 on core 1
crash: backtrace 0x4037e179 0x4037e141 0x4038582d 0x421209b3 ...
@@ -40,7 +40,8 @@ scripts/rdbg.py coredump my.bin # ...to a file you name
- **`crash`** runs the device's `crash`, finds the ELF by digest (or by version), and passes the backtrace to `scripts/decode_backtrace.sh`, which runs `addr2line` from the build container: one line for each frame, with function and file:line.
- **`coredump`** fetches the raw dump over the console (it works when the main loop is stuck) and runs `scripts/decode_coredump.sh`: `esp-coredump` and GDB, giving **every task's backtrace, the registers, and the crashed task's stack**.
- By hand: `scripts/decode_backtrace.sh <version|digest> <address>...`, and `scripts/decode_coredump.sh <core.bin> <version|digest>`. Both say which ELF they used.
- If no archived ELF matches, they say so: the build was made on another machine, or `.pio/` was cleaned.
- **A release's ELF is fetched for you.** When the crash names a released version (`v0.12.0`) and no local ELF matches, `rdbg.py` downloads `roro9stack-<version>.elf.gz` from the release on Gitea into `.pio/elves/`, so a crash on a firmware you did not build can be decoded. Decoding itself still runs in the build container.
- If nothing matches, the scripts say so: an unreleased build made on another machine, or `.pio/` was cleaned.
## The main loop is watched
@@ -52,21 +53,19 @@ An installed update no longer depends on the main loop either: the Update Servic
The count of starts that follow a crash (a panic or the watchdog) is kept in NVS. After **three in a row**, the firmware starts **Safe Mode** instead of everything else:
- only the **clock, Wi-Fi, the Update Service and, in a Debug Build, the Debug Console** start: no Apps, no IRC, no SD card;
- only the **clock, Wi-Fi, the Update Service and, if it is switched on, the Debug Console** start: no Apps, no IRC, no SD card;
- the screen says so, with the **address to push an update to**;
- only a few commands run (the [command reference](/dev/debug/commands/) lists them); anything else answers `not available in Safe Mode`.
Any **normal restart** (`reboot`, an update) or **a minute of uptime** resets the count. So Safe Mode is reached by crashing three times *quickly*, and a crash loop costs about half a minute before the device becomes reachable. To leave it: push a fix (`scripts/flash.sh --debug --ota`), or `reboot`.
Any **normal restart** (`reboot`, an update) or **a minute of uptime** resets the count. So Safe Mode is reached by crashing three times *quickly*, and a crash loop costs about half a minute before the device becomes reachable. To leave it: push a fix (`scripts/flash.sh --ota`), or `reboot`. Over USB, `debug on` works in Safe Mode too, if the console was off.
**Its limit:** if Wi-Fi or the Update Service is what crashes, Safe Mode cannot help, and it takes USB.
## Crash on purpose
On a Debug Build:
```
crash abort # abort(): a panic with a core dump
crash wdt # hang the main loop until the task watchdog fires
```
Use them to see a real report and decode it, to check that **the counter reaches Safe Mode**, and to prove that a Debug Build you are about to rely on will survive its own crashes. The device restarts by itself after either, and the report is there to read when the console comes back.
Use them to see a real report and decode it, to check that **the counter reaches Safe Mode**, and to prove that a build you are about to rely on will survive its own crashes. The device restarts by itself after either, and the report is there to read when the console comes back.