twisla/roro9stack
Public Access
1
0
Fork 0
Code Issues 50 Pull Requests Actions Packages Releases 15 Activity

Safe Mode, crash reports, and a watched main loop

Browse Source 
Every build now records at boot which version runs and, after a crash
restart, which one crashed (even across a Rollback). The core dump
summary (task, PC, reason, backtrace) is printed and raised as a
Notification; `crash` shows it later. After 3 crash restarts in a row
the firmware starts in Safe Mode: clock, Wi-Fi, Update Service and Debug
Console only (SafeMode, 2 host tests). A normal restart or a minute up
resets the count.

The main loop is now on the task watchdog (enableLoopWDT): Arduino only
watched core 0's idle task, so a stuck loop hung the device for good.
The Update Service restarts into an installed update by itself if the
main loop hasn't after 90 s.

Debug Builds: `coredump get` and `reset` are answered by the console's
own task; rdbg.py crash decodes the backtrace and rdbg.py coredump runs
esp-coredump, against ELFs archived by version and digest in .pio/elves.

The StorageService mutex is now made in the constructor: Safe Mode never
starts that Service, and `info` crashed on the null mutex, 29 times in a
row before the fix was pushed into Safe Mode over Wi-Fi.

Verified on the device: crash report and full core dump decoded over
Wi-Fi; Safe Mode at exactly 3 crashes, left by `reboot`; a hung loop
caught by the watchdog in 5 s; `reset` from the console task. ADR 0005.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EhqxQ49eCju4CzKYNjZzwT
This commit is contained in:
twisla
2026-10-04 03:07:56 +02:00
co-authored by Claude Opus 5.5
parent 0fb7f4e9d5
commit 14ff13f634
16 changed files with 452 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+14 -1
View File
@@ -74,6 +74,9 @@ To install from the SD card instead, copy the `.ota` file from `.pio/build/cardp
| `tasks` | FreeRTOS tasks: state, priority, lowest free stack, CPU share |
| `reboot` / `boot other` | Restart, or restart into the other app slot (a manual Rollback) |
| `log level <0-5>` | ESP-IDF log level |
| `crash` | The last crash: which firmware, why, task, PC and backtrace (from the core dump in flash) |
| `coredump erase` | Forgets the core dump |
| `crash abort` / `crash wdt` | Debug Builds: crash on purpose, or hang the main loop until the watchdog fires |
| `help` | Lists the commands |
`scripts/flash.sh` stops a running serial log first, since it would hold the port.
@@ -88,4 +91,14 @@ scripts/rdbg.py info # one command and its reply
scripts/rdbg.py -b tasks # the same, after the backlog (boot messages and so on)
```
Every command above works there too. The token is in `~/.config/roro9stack/debug-token`, made by the first build; keep developing on Debug Builds, so the firmware a Rollback returns to always has the console.
Every command above works there too, plus a few handled by the PC side or the console's own task:
```sh
scripts/rdbg.py crash # the last crash, its backtrace decoded against that exact build's ELF
scripts/rdbg.py coredump # fetch the core dump and decode it all (registers, every task) with esp-coredump
scripts/rdbg.py reset # restart at once, even if the main loop is stuck
```
Every build keeps its ELF in `.pio/elves/` (version and digest in the name) for that; `scripts/decode_backtrace.sh <version|digest> <addresses>` decodes any backtrace by hand.
After 3 crash restarts in a row the firmware starts in **Safe Mode** (ADR 0005): only Wi-Fi, Firmware Updates and the Debug Console, so a fix can be pushed as usual. `reboot` leaves it. The token is in `~/.config/roro9stack/debug-token`, made by the first build; keep developing on Debug Builds, so the firmware a Rollback returns to always has the console.