Safe Mode, crash reports, and a watched main loop

Every build now records at boot which version runs and, after a crash restart, which one crashed (even across a Rollback). The core dump summary (task, PC, reason, backtrace) is printed and raised as a Notification; `crash` shows it later. After 3 crash restarts in a row the firmware starts in Safe Mode: clock, Wi-Fi, Update Service and Debug Console only (SafeMode, 2 host tests). A normal restart or a minute up resets the count. The main loop is now on the task watchdog (enableLoopWDT): Arduino only watched core 0's idle task, so a stuck loop hung the device for good. The Update Service restarts into an installed update by itself if the main loop hasn't after 90 s. Debug Builds: `coredump get` and `reset` are answered by the console's own task; rdbg.py crash decodes the backtrace and rdbg.py coredump runs esp-coredump, against ELFs archived by version and digest in .pio/elves. The StorageService mutex is now made in the constructor: Safe Mode never starts that Service, and `info` crashed on the null mutex, 29 times in a row before the fix was pushed into Safe Mode over Wi-Fi. Verified on the device: crash report and full core dump decoded over Wi-Fi; Safe Mode at exactly 3 crashes, left by `reboot`; a hung loop caught by the watchdog in 5 s; `reset` from the console task. ADR 0005. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01EhqxQ49eCju4CzKYNjZzwT
2026-10-04 03:07:56 +02:00
parent 0fb7f4e9d5
commit 14ff13f634
16 changed files with 452 additions and 21 deletions
@@ -0,0 +1,66 @@
+#include "platform/crash_report.h"
+
+#include <esp_core_dump.h>
+
+#include "platform/system_info.h"
+#include "safe_mode.h"
+#include "version.h"
+
+namespace roro::crash_report {
+
+int noteBoot(KeyValueStore& store) {
+    bool crashed = system_info::resetWasCrash();
+    std::string last;
+    store.getString("run_ver", last);
+    if (crashed) {
+        store.putString("crash_ver", last.empty() ? versionString() : last);
+        store.putString("crash_why", system_info::resetReason());
+    }
+    if (last != versionString()) store.putString("run_ver", versionString());
+
+    int32_t before = 0;
+    store.getInt("crash_boots", before);
+    int now = SafeMode::countAtBoot(crashed, before);
+    if (now != before) store.putInt("crash_boots", now);
+    return now;
+}
+
+void noteStable(KeyValueStore& store) {
+    int32_t n = 0;
+    if (store.getInt("crash_boots", n) && n != 0) store.putInt("crash_boots", 0);
+}
+
+void print(Print& out, KeyValueStore& store) {
+    std::string version, why;
+    store.getString("crash_ver", version);
+    store.getString("crash_why", why);
+    if (version.empty()) out.println("crash: none recorded");
+    else out.printf("crash: last one in %s (%s)\n", version.c_str(), why.c_str());
+
+    esp_core_dump_summary_t sum;
+    if (esp_core_dump_image_check() != ESP_OK || esp_core_dump_get_summary(&sum) != ESP_OK) {
+        out.println("crash: no core dump in flash");
+        return;
+    }
+    char reason[160] = "";
+    esp_core_dump_get_panic_reason(reason, sizeof reason);
+    out.printf("crash: task %s, pc 0x%08lx, cause %lu, address 0x%08lx\n", sum.exc_task, (unsigned long)sum.exc_pc,
+               (unsigned long)sum.ex_info.exc_cause, (unsigned long)sum.ex_info.exc_vaddr);
+    if (reason[0]) out.printf("crash: reason: %s\n", reason);
+    out.print("crash: backtrace");
+    for (uint32_t i = 0; i < sum.exc_bt_info.depth && i < 16; i++) out.printf(" 0x%08lx", (unsigned long)sum.exc_bt_info.bt[i]);
+    out.println(sum.exc_bt_info.corrupted ? " (corrupted)" : "");
+    out.printf("crash: elf sha256 %s\n", reinterpret_cast<const char*>(sum.app_elf_sha256));
+}
+
+std::string headline() {
+    esp_core_dump_summary_t sum;
+    std::string where = "";
+    if (esp_core_dump_image_check() == ESP_OK && esp_core_dump_get_summary(&sum) == ESP_OK)
+        where = std::string(" in ") + sum.exc_task;
+    return std::string("Restarted after a ") + system_info::resetReason() + where;
+}
+
+bool erase() { return esp_core_dump_image_erase() == ESP_OK; }
+
+}  // namespace roro::crash_report