Safe Mode, crash reports, and a watched main loop

Every build now records at boot which version runs and, after a crash
restart, which one crashed (even across a Rollback). The core dump
summary (task, PC, reason, backtrace) is printed and raised as a
Notification; `crash` shows it later. After 3 crash restarts in a row
the firmware starts in Safe Mode: clock, Wi-Fi, Update Service and Debug
Console only (SafeMode, 2 host tests). A normal restart or a minute up
resets the count.

The main loop is now on the task watchdog (enableLoopWDT): Arduino only
watched core 0's idle task, so a stuck loop hung the device for good.
The Update Service restarts into an installed update by itself if the
main loop hasn't after 90 s.

Debug Builds: `coredump get` and `reset` are answered by the console's
own task; rdbg.py crash decodes the backtrace and rdbg.py coredump runs
esp-coredump, against ELFs archived by version and digest in .pio/elves.

The StorageService mutex is now made in the constructor: Safe Mode never
starts that Service, and `info` crashed on the null mutex, 29 times in a
row before the fix was pushed into Safe Mode over Wi-Fi.

Verified on the device: crash report and full core dump decoded over
Wi-Fi; Safe Mode at exactly 3 crashes, left by `reboot`; a hung loop
caught by the watchdog in 5 s; `reset` from the console task. ADR 0005.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EhqxQ49eCju4CzKYNjZzwT
This commit is contained in:
2026-10-04 03:07:56 +02:00
co-authored by Claude Opus 5.5
parent 0fb7f4e9d5
commit 14ff13f634
16 changed files with 452 additions and 21 deletions
+20
View File
@@ -5,6 +5,7 @@
#include <vector>
#include "probation.h"
#include "safe_mode.h"
#include "sha256.h"
#include "update_parser.h"
#include "version_compare.h"
@@ -284,6 +285,23 @@ void test_rollback_at_boot_after_an_unconfirmed_start() {
TEST_ASSERT_FALSE(Probation::rollBackAtBoot(false, 3)); // confirmed firmware: never
}
void test_safe_mode_after_three_crashes_in_a_row() {
int crashes = 0;
crashes = SafeMode::countAtBoot(true, crashes);
crashes = SafeMode::countAtBoot(true, crashes);
TEST_ASSERT_FALSE(SafeMode::active(crashes));
crashes = SafeMode::countAtBoot(true, crashes);
TEST_ASSERT_TRUE(SafeMode::active(crashes));
}
void test_safe_mode_count_restarts_after_a_normal_start() {
int crashes = SafeMode::countAtBoot(true, 2);
TEST_ASSERT_TRUE(SafeMode::active(crashes));
crashes = SafeMode::countAtBoot(false, crashes); // e.g. `reboot` from the Debug Console
TEST_ASSERT_EQUAL(0, crashes);
TEST_ASSERT_FALSE(SafeMode::active(crashes));
}
int main() {
UNITY_BEGIN();
RUN_TEST(test_sha256_known_vectors);
@@ -305,5 +323,7 @@ int main() {
RUN_TEST(test_probation_needs_wifi_when_it_is_configured);
RUN_TEST(test_probation_rolls_back_when_wifi_never_comes);
RUN_TEST(test_rollback_at_boot_after_an_unconfirmed_start);
RUN_TEST(test_safe_mode_after_three_crashes_in_a_row);
RUN_TEST(test_safe_mode_count_restarts_after_a_normal_start);
return UNITY_END();
}