diff --git a/CONTEXT.md b/CONTEXT.md index dfd6ef1..45a41c6 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -123,10 +123,18 @@ Data the user explicitly starts recording, such as Wi-Fi packet captures and LoR _Avoid_: dump, log **Storage Warning**: -The Notification raised once per boot when the SD card passes 80% full. Selecting it opens Storage Clean-up. +The Notification raised once per boot when the SD card passes 80% full. It points at the Storage App, where Maintenance holds Storage Clean-up. **Storage Clean-up**: -The screen where the user deletes old Logs and Captures by category and age, with a preview of the space freed. Notes are never offered for deletion. +The screen where the user deletes old Logs and Captures by category and age, with a preview of the space freed. Notes are never offered for deletion. It lives in the Storage App, under Maintenance. + +**Storage App**: +The App that browses the SD card: folders and files, a clipboard for one item at a time (copy, cut, paste), rename, delete, new folder, and a viewer for each kind of file the firmware writes. The top-level folders, `/gemini/cache` and files being written are read-only. +_Avoid_: file manager, Files, explorer + +**Maintenance**: +The part of the Storage App that deletes in bulk: the card's usage, Storage Clean-up and erasing the card. Reached through a warning. +_Avoid_: Settings > Storage **Firmware Update**: Installing a new firmware image without a USB cable: pushed over Wi-Fi from the developer's PC, or read from the SD card. diff --git a/README.md b/README.md index c9ae822..fa7cae7 100644 --- a/README.md +++ b/README.md @@ -82,6 +82,31 @@ On a page, `b` bookmarks it, `s` saves it to the SD card to read offline (a non- The LoRa Scanner (docs/milestones/M3.md) listens with the Cap's radio and **never transmits**. The Sniffer lists what it hears, newest first: time, RSSI, SNR, and for Meshtastic packets the sender and receiver (their last 4 hex digits) and hops. Enter shows a packet's details: the Meshtastic header (which is never encrypted) and a hex dump. `p` picks one of the 7 Meshtastic presets allowed in EU868 (LongFast by default), `c` starts or stops a Capture: a pcap file with LoRaTap headers in `/captures/lora/`, for Wireshark. A Capture keeps recording with the App closed; otherwise the radio sleeps when the App isn't open. Tab switches to **Sweep**: the signal strength across 863–870 MHz in 100 kHz steps, as bars with peak hold and a waterfall, with the Sniffer's frequency marked; the Sniffer is paused meanwhile and picks up where it was. The GNSS receiver on the same Cap raises the radio's noise floor by 8 dB while it runs: Settings > "Pause GNSS for LoRa" (off by default) puts it in standby while the radio listens, except during a Track. The Status Bar shows `L` while the radio listens (bright for a moment on each packet), `SW` while sweeping, and `CAP` while capturing. +## Storage + +The Storage App (docs/milestones/F1.md) shows what's on the SD card: each folder's entries with their size and date, folders first. Enter opens a folder, Back goes up; `s` sorts by name, date or size. It works on one item at a time, with a clipboard: + +| Key | Does | +|---|---| +| `c` / `x` | Copies or cuts the selected file or folder; the footer shows what `v` would paste | +| `v` | Pastes it into the folder shown. A copy next to its original is named `name (2).txt`; anything in the way is asked about first | +| `r` | Renames | +| `d` | Deletes, after saying what's inside: "Delete saved and its 42 files (1.2 MB)?" | +| `n` | Makes a folder | +| `i` | Details: type, exact size, date, and why an item is read-only if it is | + +A copy runs in the background of the card (about 400 KB a second) in short turns, so Logs and Captures keep being written; it shows its progress, Back cancels it and takes back what was copied, and each file's size is checked afterwards. Three things can't be changed: the top-level folders the firmware keeps its files in (what's inside them can), `/gemini/cache`, and any file being written right now (today's IRC Logs, a Track or a Capture being recorded). The App says why when it refuses. A folder with more than 256 entries shows the first 256 by name and says so. + +Enter on a file opens it by type; Tab switches to the same file as a hex dump or as text: + +- **Text** (`.txt`, `.log`, `.gmi`, `.csv`, and anything that looks like text): only the screen's worth is read from the card, so a file of any size opens at once. Logs open at the end. Up and Down move a line, Left and Right a page, `t` and `b` go to the top and the end. +- **Captures** (`.pcap`): the packets as the LoRa Scanner lists them; Enter shows one with its Meshtastic header and bytes. +- **Tracks** (`.gpx`): the number of points, the start, the duration and the distance. +- **Update Files** (`.ota`): the version, and whether the file would install: it's checked as an install checks it (signature and contents) without writing anything. Enter then installs it. +- **Anything else:** a hex dump. + +At the top of the card the last row, **Maintenance** (also `m`), holds the card's usage, Storage Clean-up and "Erase SD card", behind a warning: those delete for good. It replaces Settings > Storage. + ## Development aids `scripts/serial_log.sh [seconds] [command…]` records the serial output, and can send commands to the firmware first. For example, `scripts/serial_log.sh 30 short sleep:12 burst` sets short screen timeouts, waits 12 s, then sends a burst of Toasts. @@ -98,6 +123,7 @@ The LoRa Scanner (docs/milestones/M3.md) listens with the Cap's radio and **neve | `log ` | Appends a line to a test IRC Log (`/irc/dev/#test/.log`) | | `sd card` | What the SD card says it is: type, size, and its identity register (maker, name, revision, serial, date) | | `sd list` | Lists the files of each Storage Clean-up category | +| `sd fill ` | Debug Builds: makes that many small files in a folder, to test a crowded one | | `cat ` | Prints the first ~1.2 KB of a file on the SD card | | `irc start` | Starts the IRC Service (normally done by opening the IRC App) | | `irc stop` | Stops it, as `/quit` does: QUIT if connected, no more retries, and the App stays disconnected until you type | @@ -111,7 +137,8 @@ The LoRa Scanner (docs/milestones/M3.md) listens with the Cap's radio and **neve | `net` | Bytes each network service has read and written since boot | | `reboot` / `boot other` | Restart, or restart into the other app slot (a manual Rollback) | | `log level <0-5>` | ESP-IDF log level | -| `ls [folder]` / `rm ` | Lists a folder of the SD card, or deletes a file | +| `ls [folder]` / `du ` | Lists a folder of the SD card with sizes and dates, or counts the files and bytes under a path | +| `cp [-f] ` / `mv [-f] ` / `rm ` / `mkdir ` / `cancel` | What the Storage App does, with its rules: copy (folders too), move or rename, delete (a folder with what's in it), new folder. `-f` replaces a file that's in the way; a tab separates paths that hold spaces; `cancel` stops a copy or a delete | | `install ` | Update from SD with that `.ota` file, as Settings → Firmware does | | `lora probe` | Finds the radio: chip, oscillator, antenna switch, DIO1 interrupt, noise floor | | `lora status` | Radio settings, who's listening, packet and error counters, noise floor, task stack | diff --git a/docs/milestones/F1.md b/docs/milestones/F1.md index ef6a44c..46b6a4c 100644 --- a/docs/milestones/F1.md +++ b/docs/milestones/F1.md @@ -1,6 +1,6 @@ # F1 — Files and Notes -**Status:** in progress (branch `f1`): the Storage App (issue #3) is being built. Notes (#19) and the card as a USB drive (#1) come after. +**Status:** in progress (branch `f1`): the Storage App (issue #3) is built and checked on the device, not merged yet. Notes (#19) and the card as a USB drive (#1) come after. **Goal:** get at what's on the SD card from the device itself: browse it, look inside the files the firmware writes, copy, move, rename and delete, and keep notes. A side milestone, like G1 and S1; Files and Notes were M3's original second half (Q30, Q89). @@ -34,7 +34,7 @@ Until now the card could be looked at only through the Debug Console (`ls`, `get - A file can be copied, moved, renamed and deleted, and a folder too; a new folder can be made. Each destructive action asks first; a copy shows progress and can be cancelled. - The read-only rules of Q130 hold, with a reason given when something is refused. - Each viewer of Q134 opens its type, and a 1 MB text file scrolls without loading whole. -- Maintenance shows the card's usage and does what Settings > Storage did, behind its warning; Settings no longer has a Storage row; the Storage Warning opens the App's Maintenance. +- Maintenance shows the card's usage and does what Settings > Storage did, behind its warning; Settings no longer has a Storage row; the Storage Warning points at the Storage App. - A file written with only a GNSS Fix (no Wi-Fi) is dated correctly. - Free heap stays above the floors with the App open, Wi-Fi and IRC on TLS. @@ -46,3 +46,41 @@ Until now the card could be looked at only through the Debug Console (`ls`, `get 4. **Maintenance:** usage, Clean-up and Erase moved in from Settings, with the warning. 5. **Viewers:** text, hex, `.pcap`, `.gpx`, `.ota`. 6. **Checks on the device**, recorded here. + +### As built + +- **`FileOps`** (`src/services/file_ops`) does the card's work for the App and for the console alike: list, count, copy, move, delete, new folder. One operation at a time on the storage task, **in turns of about 150 ms** that queue themselves again, so Log lines and a Capture are written in between. The rules of Q130 are checked there, whoever asks. +- **A listing reads the folder straight from FatFs.** Through the Arduino `File`, every entry was looked up by name again for its size and again for its date: 329 entries took over two seconds. One pass now, and it's there before the screen has redrawn. Counting, copying and deleting still walk with `File`; they show progress and can be stopped. +- **A copy shows its progress in a box in the App**, not a Toast (Q133): it has a bar and says Back cancels. A cancelled or failed copy deletes what it had written. The copy gets today's date, like `cp`. +- **The viewers** (`src/apps/file_viewer`, models in `lib/files`): text through `TextPager`, which reads about a kilobyte around the screen and wraps at spaces, 38 columns; going back a line wraps the paragraph before again, so a file reads the same in both directions. A `.pcap`, a `.gpx` and an `.ota` are read through once by a storage job, in the same 150 ms turns. An Update File is fed to the installer's own parser with a sink that writes nothing, so "would it install" is the same answer an install gives. +- **Tab** in a viewer shows the same file as hex, or as text (not in Q134). +- **Maintenance** is the last row at the top of the card, and `m` anywhere in the App. It's the old Settings > Storage page behind a dialog. +- **The Storage Warning** was only ever a Toast; "selecting it opens Storage Clean-up" (CONTEXT.md) was never built. It now reads "SD card over 80% full: see Storage". +- **Console:** `cp`, `mv`, `mkdir` (Q138), and `rm` and `du` through the same code, so `rm` now takes folders and follows the rules; `ls` shows dates. Debug Builds: `sd fill ` makes test files. + +### Checks on the device (2026-10-06, v0.8.1-2 Debug Build) + +All in a scratch folder, `/f1test`, removed afterwards. + +| Check | Result | +|---|---| +| Host tests | 424 pass (411 before the viewers' models) | +| Browsing | Folders first, sizes and dates, the three sorts; a 300-file and a 329-file folder show "first 256 of 300" and "of 329" | +| New folder, rename, copy, cut and paste, delete | Each works on a file and on a folder; a copy next to its original is named `(2)`; a name in the way asks "Replace it?" | +| A folder of 11 files, 8.4 MB, copied | 19.4 s, 435 KB/s, the bar moving; two Log lines queued meanwhile were written | +| The same copy cancelled at 1.8 MB | "Cancelled: nothing was copied", and nothing was left behind | +| Delete | 341 files in 9.7 s; the dialog had counted them first | +| Read-only rules | `/irc`, `/gnss` (top-level folders), `/`, `/gemini/cache` and a folder made inside it, a folder into itself, a name with `:`; a Capture being recorded and the folder holding it; a folder under `/irc` while IRC runs. Each refused with its reason; the Capture could still be copied | +| Text | A 1 MB log opens at its last line at once; top, pages, lines; a file without an extension that looks like text opens as text | +| Hex | A 5 KB binary file; Tab from any other viewer | +| `.pcap` | A LoRa Capture: 3 packets as the Scanner lists them, Enter shows the Meshtastic header and bytes | +| `.gpx` | 400 points: start, 33 min 15 s, 4.68 km; Tab shows the text | +| `.ota` | A signed file: version, "intact", "older than what's running", Enter asks to install (not confirmed). A tampered one: "image corrupted (hash mismatch)" | +| Maintenance | The warning, then usage, Clean-up's categories and Erase (not run) | +| Date with GNSS only | NTP pointed at an address that doesn't answer, restart: the Clock came from the Fix, and a folder made then is dated 2026-10-06 08:39. A Track from the day before, written the same way by v0.8.1, shows "-" | +| Memory | IRC connected, the App open on 256 entries: 61 KB free (70 KB before opening). Lowest since boot 29.7 KB, during IRC's TLS handshake | +| Stacks | `storage` 3.1 KB free of 6 KB at worst, `loopTask` 1.5 KB | + +**Not checked by hand:** how the keys feel on the device itself; everything above was driven through the Debug Console's `key` command and screenshots. + +**One slip during the checks:** a scripted key sequence ran in the wrong folder and renamed `/gemini/saved` to `saved2`, then copied it to the top of the card. Both were put right at once (renamed back, the copy deleted; 7 files, 53,798 bytes, as before). diff --git a/lib/apps_model/src/settings_menu.cpp b/lib/apps_model/src/settings_menu.cpp index 176246e..b1fab38 100644 --- a/lib/apps_model/src/settings_menu.cpp +++ b/lib/apps_model/src/settings_menu.cpp @@ -23,7 +23,6 @@ const RowDef kRows[] = { {Row::Gnss, Kind::Toggle, "GNSS"}, {Row::GnssQuiet, Kind::Toggle, "Pause GNSS for LoRa"}, {Row::Coordinates, Kind::Toggle, "Coordinates"}, {Row::ProbeMacs, Kind::Toggle, "Probe MACs"}, {Row::Wifi, Kind::Page, "Wi-Fi"}, - {Row::Storage, Kind::Page, "Storage"}, {Row::Firmware, Kind::Page, "Firmware"}, {Row::About, Kind::Page, "About"}, }; diff --git a/lib/apps_model/src/settings_menu.h b/lib/apps_model/src/settings_menu.h index 176cea9..09eb0a4 100644 --- a/lib/apps_model/src/settings_menu.h +++ b/lib/apps_model/src/settings_menu.h @@ -11,7 +11,7 @@ namespace roro { // values, choice lists and validation messages. Rendering and navigation live in the App. class SettingsMenu { public: - enum class Row { LongName, ShortName, Region, Timezone, Brightness, DimTimeout, OffTimeout, Sound, Gnss, GnssQuiet, Coordinates, ProbeMacs, Wifi, Storage, Firmware, About }; + enum class Row { LongName, ShortName, Region, Timezone, Brightness, DimTimeout, OffTimeout, Sound, Gnss, GnssQuiet, Coordinates, ProbeMacs, Wifi, Firmware, About }; enum class Kind { Text, Choice, Toggle, Slider, Page }; explicit SettingsMenu(Settings& settings) : settings_(settings) {} diff --git a/lib/files/src/file_list.cpp b/lib/files/src/file_list.cpp index c8fdb42..a5b8f29 100644 --- a/lib/files/src/file_list.cpp +++ b/lib/files/src/file_list.cpp @@ -128,4 +128,10 @@ std::string rowDetail(bool folder, uint32_t size, uint32_t modified) { return sizeText(size) + " " + (stamp == "-" ? stamp : stamp.substr(0, 10)); } +std::string fitName(const std::string& name, size_t maxChars) { + if (name.size() <= maxChars || maxChars < 8) return name.substr(0, maxChars); + size_t tail = std::min(6, maxChars / 3), head = maxChars - tail - 2; + return name.substr(0, head) + ".." + name.substr(name.size() - tail); +} + } // namespace roro::files diff --git a/lib/files/src/file_list.h b/lib/files/src/file_list.h index cf9f0a7..cd7eb69 100644 --- a/lib/files/src/file_list.h +++ b/lib/files/src/file_list.h @@ -48,6 +48,8 @@ class FileList { // What a row shows on the right (Q129): "folder", or "1.2 KB 2026-10-05". A file dated before // 2020 was written before the clock was set: "-" (Q137). Local time. std::string rowDetail(bool folder, uint32_t size, uint32_t modified); -std::string formatStamp(uint32_t modified); // "2026-10-05 20:00", or "-" +std::string formatStamp(uint32_t modified); +// A name cut to `maxChars` for a row, from the middle: the start and the extension stay readable. +std::string fitName(const std::string& name, size_t maxChars); // "2026-10-05 20:00", or "-" } // namespace roro::files diff --git a/lib/files/src/file_views.cpp b/lib/files/src/file_views.cpp new file mode 100644 index 0000000..fdaa120 --- /dev/null +++ b/lib/files/src/file_views.cpp @@ -0,0 +1,143 @@ +#include "file_views.h" + +#include +#include +#include + +#include "file_list.h" +#include "track.h" + +namespace roro::files { + +std::string hexRow(uint32_t offset, const uint8_t* data, size_t len) { + char head[8]; + std::snprintf(head, sizeof head, "%05X", static_cast(offset)); + std::string row = head, text; + for (size_t i = 0; i < 8; i++) { + if (i % 2 == 0) row += ' '; + char hex[3] = " "; + if (i < len) { + std::snprintf(hex, sizeof hex, "%02x", data[i]); + text += data[i] >= 0x20 && data[i] < 0x7F ? static_cast(data[i]) : '.'; + } + row += hex; + } + return row + " " + text; +} + +namespace { +// Days since 1970-01-01 (Howard Hinnant's days_from_civil): no timegm() everywhere. +int64_t daysFromCivil(int y, int m, int d) { + y -= m <= 2; + int64_t era = (y >= 0 ? y : y - 399) / 400; + int yoe = static_cast(y - era * 400); + int doy = (153 * (m + (m > 2 ? -3 : 9)) + 2) / 5 + d - 1; + int doe = yoe * 365 + yoe / 4 - yoe / 100 + doy; + return era * 146097 + doe - 719468; +} + +bool attribute(const std::string& element, const char* name, double& out) { + size_t at = element.find(name); + if (at == std::string::npos) return false; + const char* from = element.c_str() + at + std::strlen(name); + char* end = nullptr; + out = std::strtod(from, &end); + return end != from; +} +} // namespace + +void GpxSummary::point(const std::string& element) { + double lat, lon; + if (!attribute(element, "lat=\"", lat) || !attribute(element, "lon=\"", lon)) return; + if (points_ > 0) meters_ += gnss::distanceMeters(lat_, lon_, lat, lon); + lat_ = lat; + lon_ = lon; + points_++; + size_t at = element.find("