Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5bec851a1a | ||
|
|
9dfe675db7 | ||
|
|
079de4aec7 | ||
|
|
6b71aace4f | ||
|
|
7223147f26 | ||
|
|
01a8e2a233 | ||
|
|
0498916740 | ||
|
|
298407b5cf | ||
|
|
9808013fc0 | ||
|
|
9b6457d1ec | ||
|
|
a8f267416b | ||
|
|
ed7abdcaf5 | ||
|
|
86172c0342 | ||
|
|
e00fff670f | ||
|
|
4404dd9380 | ||
|
|
f0306dd880 | ||
|
|
e3fe618c7f | ||
|
|
d7092ee6d6 | ||
|
|
63c2da8138 | ||
|
|
057af773e4 | ||
|
|
6b02cd3d5f | ||
|
|
c35bc47693 | ||
|
|
4cdb4c342f | ||
|
|
1c9f90f92e | ||
|
|
2c18762614 | ||
|
|
ae25cf0be2 | ||
|
|
834c6eb0f2 | ||
|
|
5823584bfd | ||
|
|
35f0d5959c | ||
|
|
dd4e6c31ed | ||
|
|
de8af6ed92 | ||
|
|
10c5291e15 | ||
|
|
12c88c98d3 | ||
|
|
55c9ad2eb4 | ||
|
|
0fdbb5b1ed | ||
|
|
d17d10948d | ||
|
|
7b5df713ad | ||
|
|
3863d28593 | ||
|
|
c278a06ca1 | ||
|
|
828f7ce821 | ||
|
|
ca5874fe70 | ||
|
|
07ac7ba457 | ||
|
|
942725a047 | ||
|
|
34e6714785 | ||
|
|
82023d36b3 | ||
|
|
7fe8b3d22a | ||
|
|
1c6ae0e04f | ||
|
|
74b7713553 | ||
|
|
6be05b782d | ||
|
|
1874a1b586 | ||
|
|
c68741cc46 | ||
|
|
1353e6a5f9 |
@@ -1,14 +1,32 @@
|
||||
# CI and releases (docs/milestones/R1.md).
|
||||
# A push to main: the host tests, with their coverage of lib/, and the README's badges
|
||||
# published to the branch `badges`.
|
||||
# A pull request: the same tests and coverage, then the release firmware and the Debug Build.
|
||||
# A pull request: the same tests and coverage, then the firmware (from the build cache).
|
||||
# A branch's pushes run nothing by themselves: its pull request runs, once.
|
||||
# A tag v*: all of it, then a Gitea release with the signed Update File.
|
||||
# A tag v*: the tests, then the firmware built once, clean, signed and published as a
|
||||
# Gitea release. The site is then rebuilt: its home page and Downloads name
|
||||
# the latest release when they are built (issue #79).
|
||||
# Run by hand: the release of a tag that exists already (the ones from before CI).
|
||||
#
|
||||
# The job runs in a plain Python image, as scripts/ci.sh does on a developer's machine, with the
|
||||
# toolchains in a Docker volume the runner allows (container.valid_volumes: roro9stack-pio): that
|
||||
# volume is the cache. No JavaScript actions, so the image needs no Node: the checkout is git.
|
||||
#
|
||||
# What the volume keeps between runs, and what makes each go stale (issue #74, R1.md):
|
||||
# /pio/packages, /pio/platforms toolchains and the framework: by their versions in platformio.ini
|
||||
# .../framework-arduinoespressif32-libs/.roro-sdkconfig.defaults
|
||||
# the mark that the framework is already rebuilt with our SDK settings: the
|
||||
# project's sdkconfig.defaults, which isn't in git, so that every fresh
|
||||
# checkout rebuilt the framework (260 s). The platform checks its hash
|
||||
# against platformio.ini's settings, and rebuilds if they differ. It is kept
|
||||
# inside the libraries it describes, so it goes when they are reinstalled.
|
||||
# /pio/ci/build-cache PlatformIO's build cache (SCons): objects by the signature of their
|
||||
# sources and command line. For pull requests only: a release compiles
|
||||
# its own sources from nothing.
|
||||
# /pio/ci/ccache the host tests' objects (they're built for coverage, which the build
|
||||
# cache can't keep: it would lose the .gcno files)
|
||||
# /pio/ci/venv PlatformIO and gcovr: delete the folder to upgrade them
|
||||
# To start from nothing (a slow run, about 7 minutes): delete /pio/ci and that .roro-sdkconfig.defaults file.
|
||||
name: CI
|
||||
on:
|
||||
push:
|
||||
@@ -37,13 +55,23 @@ jobs:
|
||||
env:
|
||||
PLATFORMIO_CORE_DIR: /pio
|
||||
RORO_NO_DOCKER: 1
|
||||
SDK_MARK: /pio/packages/framework-arduinoespressif32-libs/.roro-sdkconfig.defaults
|
||||
CCACHE_DIR: /pio/ci/ccache
|
||||
CCACHE_MAXSIZE: 1G
|
||||
steps:
|
||||
- name: Tools
|
||||
run: |
|
||||
apt-get update -qq
|
||||
apt-get install -y -qq --no-install-recommends git build-essential openssl >/dev/null
|
||||
pip install -q --no-cache-dir --root-user-action=ignore platformio gcovr
|
||||
pio --version; df -h /pio | tail -1; ls /pio | head
|
||||
apt-get install -y -qq --no-install-recommends git build-essential openssl ccache >/dev/null
|
||||
mkdir -p /pio/ci
|
||||
if [ ! -x /pio/ci/venv/bin/pio ]; then
|
||||
python -m venv /pio/ci/venv
|
||||
/pio/ci/venv/bin/pip install -q --no-cache-dir platformio gcovr
|
||||
fi
|
||||
ln -sf /pio/ci/venv/bin/pio /pio/ci/venv/bin/gcovr /usr/local/bin/
|
||||
pio --version; df -h /pio | tail -1; du -sh /pio/ci/* 2>/dev/null || true
|
||||
# The build cache only grows: start it again past 3 GB (a full set of objects is 160 MB, and each run adds about 40).
|
||||
if [ "$(du -sm /pio/ci/build-cache 2>/dev/null | cut -f1)" -gt 3072 ] 2>/dev/null; then rm -rf /pio/ci/build-cache; fi
|
||||
|
||||
- name: Check out
|
||||
run: |
|
||||
@@ -57,11 +85,20 @@ jobs:
|
||||
|
||||
- name: Host tests, and their coverage of lib/
|
||||
if: github.event_name != 'workflow_dispatch'
|
||||
run: scripts/coverage.sh
|
||||
run: |
|
||||
export PATH="/usr/lib/ccache:$PATH" # gcc and g++ through ccache
|
||||
scripts/coverage.sh
|
||||
ccache -s | grep -E 'Hits|Misses' | head -2
|
||||
|
||||
- name: The release firmware and the Debug Build
|
||||
if: github.event_name == 'pull_request' || github.ref_type == 'tag'
|
||||
run: scripts/ci.sh builds
|
||||
# A pull request only: a tag's firmware is built once, by the release step below.
|
||||
- name: The firmware
|
||||
if: github.event_name == 'pull_request'
|
||||
env:
|
||||
PLATFORMIO_BUILD_CACHE_DIR: /pio/ci/build-cache
|
||||
run: |
|
||||
[ ! -f "$SDK_MARK" ] || cp "$SDK_MARK" sdkconfig.defaults
|
||||
scripts/ci.sh builds
|
||||
cp sdkconfig.defaults "$SDK_MARK" # what the framework in the volume is rebuilt with, now
|
||||
|
||||
# The README's badges are files on a branch of their own, replaced at each push to main and
|
||||
# at each tag (the release badge says which tag is the latest)
|
||||
@@ -103,7 +140,12 @@ jobs:
|
||||
trap 'rm -f "$RORO_OTA_KEY"' EXIT
|
||||
printf '%s\n' "$OTA_SIGNING_KEY" > "$RORO_OTA_KEY"
|
||||
umask 022
|
||||
# The framework rebuilt with our settings is reused if it matches (the platform checks);
|
||||
# the release's own sources are compiled from nothing, with no build cache.
|
||||
[ ! -f "$SDK_MARK" ] || cp "$SDK_MARK" /tmp/release-src/sdkconfig.defaults
|
||||
scripts/release_build.sh /tmp/release-src dist
|
||||
# An old tag has no SDK settings of its own, and no sdkconfig.defaults afterwards: nothing to mark.
|
||||
[ ! -f /tmp/release-src/sdkconfig.defaults ] || [ ! -d "$(dirname "$SDK_MARK")" ] || cp /tmp/release-src/sdkconfig.defaults "$SDK_MARK"
|
||||
|
||||
- name: Publish the release
|
||||
if: steps.release.outputs.tag != ''
|
||||
@@ -112,3 +154,14 @@ jobs:
|
||||
GITEA_REPO: ${{ github.repository }}
|
||||
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||
run: scripts/release_publish.py dist
|
||||
|
||||
# The site names the latest release on its home page and lists them all on Downloads, both
|
||||
# read when it is built: so it is rebuilt now (issue #79, as the Site workflow does).
|
||||
- name: Refresh the site
|
||||
if: steps.release.outputs.tag != ''
|
||||
env:
|
||||
SITE_DEPLOY_KEY: ${{ secrets.SITE_DEPLOY_KEY }}
|
||||
SITE_DEPLOY_HOST: ${{ secrets.SITE_DEPLOY_HOST }}
|
||||
SITE_DEPLOY_USER: ${{ secrets.SITE_DEPLOY_USER }}
|
||||
SITE_DEPLOY_KNOWN_HOSTS: ${{ secrets.SITE_DEPLOY_KNOWN_HOSTS }}
|
||||
run: scripts/site_refresh.sh
|
||||
|
||||
@@ -1,17 +1,19 @@
|
||||
# The project site (docs/milestones/W1.md): built with Zola to see that it builds and that its pages
|
||||
# are sound. Publishing is the maintainer's: the web server pulls main and runs `zola build`.
|
||||
# are sound. After a push to main it is then published: the job asks the web server, over SSH, to
|
||||
# pull main and rebuild (issue #79, scripts/site_refresh.sh). The key it holds can run that one
|
||||
# command there and nothing else; the server, the user and the keys are secrets, not in this file.
|
||||
#
|
||||
# It runs when the site, or a document the site is built from, changes (a pull request, or a push to
|
||||
# main); the firmware workflow (ci.yml) skips a change that touches only these files. A change that
|
||||
# touches both runs both. src/main.cpp is here too: the site's command reference is generated from the
|
||||
# firmware's own `help` text, and this job checks that it is still current.
|
||||
# touches both runs both. src/main.cpp and lib/core/src/app_keys.h are here too: the site's command
|
||||
# reference and its key tables are generated from them, and this job checks that they are still current.
|
||||
name: Site
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
paths: ['site/**', 'docs/**', 'README.md', 'CONTEXT.md', 'src/main.cpp', '.gitea/workflows/site.yml']
|
||||
paths: ['site/**', 'docs/**', 'README.md', 'CONTEXT.md', 'src/main.cpp', 'lib/core/src/app_keys.h', '.gitea/workflows/site.yml', 'scripts/site_refresh.sh']
|
||||
pull_request:
|
||||
paths: ['site/**', 'docs/**', 'README.md', 'CONTEXT.md', 'src/main.cpp', '.gitea/workflows/site.yml']
|
||||
paths: ['site/**', 'docs/**', 'README.md', 'CONTEXT.md', 'src/main.cpp', 'lib/core/src/app_keys.h', '.gitea/workflows/site.yml', 'scripts/site_refresh.sh']
|
||||
|
||||
jobs:
|
||||
build:
|
||||
@@ -51,3 +53,14 @@ jobs:
|
||||
|
||||
- name: Check the pages
|
||||
run: python3 site/tools/check_site.py /tmp/site-out
|
||||
|
||||
# Only what has been merged, and only once it has built and passed the checks above. A pull
|
||||
# request never gets here, and the secrets are given to this step alone.
|
||||
- name: Publish the site
|
||||
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
|
||||
env:
|
||||
SITE_DEPLOY_KEY: ${{ secrets.SITE_DEPLOY_KEY }}
|
||||
SITE_DEPLOY_HOST: ${{ secrets.SITE_DEPLOY_HOST }}
|
||||
SITE_DEPLOY_USER: ${{ secrets.SITE_DEPLOY_USER }}
|
||||
SITE_DEPLOY_KNOWN_HOSTS: ${{ secrets.SITE_DEPLOY_KNOWN_HOSTS }}
|
||||
run: scripts/site_refresh.sh
|
||||
|
||||
@@ -12,3 +12,6 @@ sdkconfig.*
|
||||
|
||||
# The built site (site/config.toml sends it here)
|
||||
/public/
|
||||
|
||||
# The version, written by scripts/version.py before each build
|
||||
lib/version/src/version_generated.h
|
||||
|
||||
@@ -106,6 +106,34 @@ The regulatory band plan the device transmits under (here EU868). It sets the al
|
||||
**Duty Cycle Budget**:
|
||||
The share of airtime the Region allows this device to transmit. When it's used up, outgoing traffic waits.
|
||||
|
||||
**Shell**:
|
||||
The App that runs the console's commands on the device itself, and shows what the console prints. Trusted like the USB port, not like the network.
|
||||
_Avoid_: terminal, command line, REPL
|
||||
|
||||
**Tunnel**:
|
||||
The WireGuard connection to one server, over whatever Wi-Fi the device is on. It carries either everything or the one subnet the device's address in it belongs to. Wanted or not is the user's switch; up or not depends on Wi-Fi, the clock and the server.
|
||||
_Avoid_: VPN connection, link, session
|
||||
|
||||
**Session**:
|
||||
The one SSH connection to a shell on another machine, from login until either side ends it. It belongs to the SSH Service, not to the SSH App: it goes on while another App is in front.
|
||||
_Avoid_: connection, tunnel, terminal (the terminal is what draws it)
|
||||
|
||||
**Device Key**:
|
||||
The Ed25519 key pair the device makes for itself to log in over SSH. The private half never leaves the device and is never shown; the public half is meant to be copied to servers.
|
||||
_Avoid_: identity, SSH key file, certificate
|
||||
|
||||
**Sharing**:
|
||||
Serving the SD card as a web page to a browser on the same network, for as long as the Storage App's Share screen is open, to whoever typed the code that screen shows.
|
||||
_Avoid_: file server, web server, FTP, upload mode
|
||||
|
||||
**Screenshot**:
|
||||
The screen as a PNG in `/screenshots`, taken with Fn+p on any screen or the Shell's `screenshot`. Not the Debug Console's `screenshot`, which sends the screen to a PC.
|
||||
_Avoid_: capture (a **Capture** is radio packets), screen grab
|
||||
|
||||
**Help panel**:
|
||||
The list of the keys that work on the screen you are on, opened with Fn+h anywhere (or `?` outside Text Entry). Each App answers for its current state; no screen names keys any other way, except the first-start Setup.
|
||||
_Avoid_: hints, cheat sheet, shortcuts bar
|
||||
|
||||
**Text Entry**:
|
||||
When an App is editing text. During Text Entry, `;` `.` `,` `/` type their characters and Fn makes them arrows. Otherwise they are arrows on their own.
|
||||
_Avoid_: edit mode, insert mode
|
||||
@@ -144,7 +172,7 @@ _Avoid_: Settings > Storage
|
||||
Installing a new firmware image without a USB cable: pushed over Wi-Fi from the developer's PC, read from the SD card, or downloaded from the project's Gitea **Release**.
|
||||
|
||||
**Release**:
|
||||
A tag `v*` on the project's Gitea with a signed **Update File**, a factory image for USB, the ELF to decode crashes and checksums, built and published by CI. The device reads them to look for updates. Debug Builds aren't published.
|
||||
A tag `v*` on the project's Gitea with a signed **Update File**, a factory image for USB, the ELF to decode crashes and checksums, built and published by CI. The device reads them to look for updates.
|
||||
_Avoid_: flash, upgrade (alone)
|
||||
|
||||
**Update File**:
|
||||
@@ -160,16 +188,12 @@ Returning automatically to the previous firmware when new firmware resets or cra
|
||||
_Avoid_: revert, downgrade (a downgrade is installing an older version on purpose)
|
||||
|
||||
**Safe Mode**:
|
||||
What the firmware starts instead of everything else after 3 crash restarts in a row: Wi-Fi and Firmware Updates (and the Debug Console in a Debug Build), so it can be fixed without a cable. A normal restart leaves it.
|
||||
What the firmware starts instead of everything else after 3 crash restarts in a row: Wi-Fi and Firmware Updates (and the Debug Console if it's switched on), so it can be fixed without a cable. A normal restart leaves it.
|
||||
_Avoid_: recovery mode, failsafe
|
||||
|
||||
**Debug Build**:
|
||||
A firmware built with the remote debugging aids compiled in (`+debug` in its version). Release builds have none of them.
|
||||
_Avoid_: dev build, test build (a test build is one made to fail on purpose, such as a crashing update)
|
||||
|
||||
**Debug Console**:
|
||||
The console of a Debug Build over Wi-Fi: live log lines and the serial commands, behind a token.
|
||||
_Avoid_: telnet, remote shell
|
||||
The console over Wi-Fi, in every firmware but off until switched on in Settings: live log lines and the serial commands, for whoever holds the device's token.
|
||||
_Avoid_: telnet, remote shell, Debug Build (there is one firmware)
|
||||
|
||||
## Relationships
|
||||
|
||||
@@ -177,6 +201,7 @@ _Avoid_: telnet, remote shell
|
||||
- **Services** keep running underneath, regardless of which **App** is in the foreground.
|
||||
- The **Mesh Service** speaks one or more **Mesh Protocols** and tracks the known **Nodes**.
|
||||
- The **Wi-Fi Service** is either Connected or Monitoring, never both. Monitoring pauses the **IRC Service**, which reconnects and rejoins its **Buffers** afterwards.
|
||||
- The SSH App draws the one **Session**; the **Session** and the **IRC Service**'s connection don't fit in memory together, so each waits for the other.
|
||||
- **Services** raise **Notifications**; the **Status Bar** summarises **Service** state.
|
||||
- The **Radio Service** owns the radio; the **Mesh Service** and the LoRa Scanner use it.
|
||||
- A **Sweep** pauses the **Mesh Service**; a **Sniffer** does not.
|
||||
@@ -184,6 +209,8 @@ _Avoid_: telnet, remote shell
|
||||
- Past 90% SD usage, **Logs** stop being written; the remaining space is kept for **Captures**. Nothing is deleted without the user's confirmation.
|
||||
- A **Firmware Update** installs an **Update File**; the new firmware runs on **Probation**, and fails back by **Rollback**.
|
||||
- **Rollback** covers new firmware; **Safe Mode** covers confirmed firmware that keeps crashing.
|
||||
- A **Tunnel** rides on the **Wi-Fi Service**'s connection and ends with it; the next connection starts it afresh.
|
||||
- **Sharing** lasts as long as its screen: leaving the **Storage App** ends it.
|
||||
- A **Node** may be in several **Channels**. A **Direct Message** targets exactly one **Node**.
|
||||
|
||||
## Flagged ambiguities
|
||||
|
||||
@@ -2,12 +2,35 @@
|
||||
|
||||
[](https://git.twis.la/twisla/roro9stack/actions?workflow=ci.yml) [](#build-and-test-local-ci) [](https://git.twis.la/twisla/roro9stack/releases/latest)
|
||||
|
||||
A multi-app firmware for the **M5Stack Cardputer ADV** with the **Cap LoRa-1262**. It's a Meshtastic-compatible mesh messenger, plus Wi-Fi tools, IRC, GNSS and more. Licensed GPL-3.0.
|
||||
A multi-app firmware for the **M5Stack Cardputer ADV** with the **Cap LoRa-1262**. Licensed GPL-3.0. The user guide, the how-tos and every release are at **[roro9stack.net](https://roro9stack.net)**.
|
||||
|
||||
What it does today:
|
||||
|
||||
- **LoRa Scanner:** every packet it hears, with the Meshtastic header read; a spectrum Sweep; captures for Wireshark. It listens and never transmits: the mesh messenger is the next milestone.
|
||||
- **GNSS:** position, sky view, tracks as GPX.
|
||||
- **Gemini:** a browser, with bookmarks and pages saved for offline.
|
||||
- **IRC:** over TLS, with logs on the card.
|
||||
- **Wi-Fi Tools:** the networks around, sorted, filtered, logged.
|
||||
- **Notes:** plain text files of any size, saved by themselves.
|
||||
- **Storage:** the SD card: copy, move, rename, delete; viewers for text, hex, pictures (PNG, JPEG, BMP, GIF), tracks, captures and update files; **sharing with a phone's browser**.
|
||||
- **Shell:** the firmware's commands on the device itself, with completion, including `ping`, `nslookup`, `port`, `traceroute`, `tls` and `ifconfig`.
|
||||
- **System:** load, tasks, memory, network, battery, live.
|
||||
- **SSH:** a terminal on another machine, with a password or the device's own key.
|
||||
- **VPN:** a WireGuard tunnel.
|
||||
- **Everywhere:** Fn+h lists the keys of the screen you are on; Fn+p takes a screenshot.
|
||||
- **Updates:** signed, from the project's server, the card or a PC, with a rollback if the new firmware fails.
|
||||
- **Debug Console:** the device's console over Wi-Fi, off until switched on.
|
||||
|
||||
- Domain language: [CONTEXT.md](CONTEXT.md)
|
||||
- Decisions: [docs/adr/](docs/adr/)
|
||||
- Milestones: [docs/milestones/](docs/milestones/)
|
||||
|
||||
## On the device: one key
|
||||
|
||||
**Fn+p, on any screen, saves a screenshot** to `/screenshots` on the card (not on the page that shows the Debug Console's token; issue #83).
|
||||
|
||||
**Fn+h, on any screen, lists the keys that work there** (`?` does the same outside a text field). No screen names its keys itself (docs/milestones/U1.md). Every screen's keys are one table in `lib/core/src/app_keys.h`: the help panel shows the table of the state an App is in, and the website's key tables are generated from the same file.
|
||||
|
||||
## Requirements
|
||||
|
||||
Only **Docker** is needed. PlatformIO and the ESP32 toolchain run inside a container, and are cached in the `roro9stack-pio` Docker volume. The first build downloads about 1 GB and takes a few minutes.
|
||||
@@ -22,18 +45,18 @@ This runs the host-side unit tests (`test/`, `native` environment), then builds
|
||||
|
||||
`scripts/coverage.sh` runs the same tests with coverage counters and writes a line-by-line report to `.pio/coverage/index.html`. The badge above is its figure for `main`: the share of the lines of `lib/` that the host tests run. `lib/` is the logic that compiles on a PC; `lib/SD` (the card's driver) and `src/` (the Apps, the Services, everything that needs the device) have no host tests and aren't in that figure.
|
||||
|
||||
The framework is rebuilt with the TLS settings in `platformio.ini` (`custom_sdkconfig`, ADR 0006), so the first build after a fresh checkout takes about 4 minutes; later builds take under a minute.
|
||||
The framework is rebuilt with the TLS settings in `platformio.ini` (`custom_sdkconfig`, ADR 0006), so the first build after a fresh checkout takes about 6 minutes; later builds take about 15 seconds when little has changed. The platform knows the framework is already rebuilt by `sdkconfig.defaults` in the project folder, which it writes and git ignores: delete it and the next build rebuilds the framework. CI keeps that file, a build cache and ccache in its volume (docs/milestones/R1.md).
|
||||
|
||||
## CI and releases
|
||||
|
||||
Gitea Actions (`.gitea/workflows/ci.yml`, docs/milestones/R1.md) runs the host tests on every push to `main`, and on a pull request also builds the release firmware and the Debug Build: changes reach `main` through pull requests. Pushing a tag `v*` runs all of it and publishes a release on Gitea with:
|
||||
Gitea Actions (`.gitea/workflows/ci.yml`, docs/milestones/R1.md) runs the host tests on every push to `main`, and on a pull request also builds the firmware: changes reach `main` through pull requests. Pushing a tag `v*` runs all of it and publishes a release on Gitea with:
|
||||
|
||||
- `roro9stack-<version>.ota`, the signed Update File;
|
||||
- `roro9stack-<version>-factory.bin`, the whole flash image for a first install over USB;
|
||||
- `roro9stack-<version>.elf.gz`, to decode crash reports from that build;
|
||||
- `SHA256SUMS`.
|
||||
|
||||
CI signs with the project's key, held as a repository secret (ADR 0008). Debug Builds are built but never published: each carries its builder's Debug Console token.
|
||||
CI signs with the project's key, held as a repository secret (ADR 0008). There is one firmware: the Debug Console is in every build, switched off until its owner switches it on (ADR 0010).
|
||||
|
||||
`scripts/ota_verify.py <file.ota>` checks an Update File on a PC the way a device does. `scripts/release_build.sh` and `scripts/release_publish.py` are what the workflow runs; they work the same by hand.
|
||||
|
||||
@@ -87,7 +110,7 @@ The connection is checked against the two ISRG roots Let's Encrypt chains end in
|
||||
|
||||
**IRC steps aside.** A secure connection takes about 52 KB of memory at its peak, and IRC's own takes 40 KB of the 107 KB there is. A check or an install you ask for makes IRC disconnect for the few seconds it takes and reconnect afterwards. The daily check never does that: with IRC connected it waits for a moment when IRC isn't, so while IRC stays connected for days it doesn't run, and **Latest release** is the way to check.
|
||||
|
||||
**A Debug Build** shows the latest release but doesn't install it: releases have no Debug Console (they aren't built with one, so that nobody's console token is published), and installing one would take it away. A Debug Build is updated from the PC with `scripts/flash.sh --debug --ota`.
|
||||
**There is no separate Debug Build** any more (ADR 0010): every firmware installs releases, and the Debug Console is a setting, which an update leaves as it was.
|
||||
|
||||
## Networks without DHCP
|
||||
|
||||
@@ -132,9 +155,12 @@ The Storage App (docs/milestones/F1.md) shows what's on the SD card: each folder
|
||||
|
||||
A copy runs in the background of the card (about 400 KB a second) in short turns, so Logs and Captures keep being written; it shows its progress, Back cancels it and takes back what was copied, and each file's size is checked afterwards. Three things can't be changed: the top-level folders the firmware keeps its files in (what's inside them can), `/gemini/cache`, and any file being written right now (today's IRC Logs, a Track or a Capture being recorded). The App says why when it refuses. A folder with more than 256 entries shows the first 256 by name and says so.
|
||||
|
||||
**`w` shares the card with a browser on the same network** (issue #88): a small HTTP server and one page, for a phone with nothing to install. The screen shows the address as a QR code and a six-digit code, new each time; whoever has typed it can list, download, upload (streamed to the card under a temporary name), make folders and delete, under the Storage App's rules. It runs only while that screen is open, takes one request at a time, moves about 200 KB a second, and is not encrypted. It costs 57 KB of flash, and 13 KB of memory while it is on.
|
||||
|
||||
Enter on a file opens it by type; Tab switches to the same file as a hex dump or as text:
|
||||
|
||||
- **Text** (`.txt`, `.log`, `.gmi`, `.csv`, and anything that looks like text): only the screen's worth is read from the card, so a file of any size opens at once. Logs open at the end. Up and Down move a line, Left and Right a page, `t` and `b` go to the top and the end, `e` edits it (up to 16 KB, see Notes).
|
||||
- **Text** (`.txt`, `.log`, `.gmi`, `.csv`, and anything that looks like text): only the screen's worth is read from the card, so a file of any size opens at once. Logs open at the end. Up and Down move a line, Left and Right a page, `t` and `b` go to the top and the end, `e` edits it, whatever its size (see Notes).
|
||||
- **Pictures** (`.png`, `.jpg`, `.bmp`, `.gif`; issue #45): shrunk to fit the screen, or at their own size with Enter, the arrows then moving half a screen at a time; `i` gives the size in pixels. Dithered to the screen's 256 colours, decoded straight into the screen's buffer with no copy in memory, on the storage task so the keys keep working (12 megapixels of JPEG: 7 s). A GIF shows its first picture. A progressive JPEG or an interlaced PNG opens as hex, with the reason.
|
||||
- **Captures** (`.pcap`): the packets as the LoRa Scanner lists them; Enter shows one with its Meshtastic header and bytes.
|
||||
- **Tracks** (`.gpx`): the number of points, the start, the duration and the distance.
|
||||
- **Update Files** (`.ota`): the version, and whether the file would install: it's checked as an install checks it (signature and contents) without writing anything. Enter then installs it.
|
||||
@@ -146,11 +172,27 @@ At the top of the card the last row, **Maintenance** (also `m`), holds the card'
|
||||
|
||||
The Notes App (docs/milestones/F1.md) keeps plain text notes in `/notes` on the SD card. The list shows each note's first line and its date, newest first; `s` switches to by file name. `n` starts a note, Enter opens one, `r` renames its file, `d` deletes it after asking.
|
||||
|
||||
In the editor, type. Enter starts a line, Del deletes backwards, Fn with the arrows moves the cursor through the wrapped text, Ctrl+A and Ctrl+E go to the start and the end of the line, Tab types two spaces, and the Compose Key gives accents as everywhere. **There's no save key:** the note is written five seconds after the last key, on Back, on leaving the App, when the screen turns off and before the device powers off. The top line says "typing" or "saved". Each save writes a temporary file and then puts it in the note's place, so a power cut costs a few seconds of typing and never the note; if a save was cut short, opening the note offers its copy back.
|
||||
In the editor, type. Enter starts a line, Del deletes backwards, Fn with the arrows moves the cursor through the wrapped text, Ctrl+A and Ctrl+E go to the start and the end of the line, Ctrl with Fn+Up and Fn+Down to the start and the end of the note, Tab types two spaces, and the Compose Key gives accents as everywhere. **There's no save key:** the note is written five seconds after the last key, on Back, on leaving the App, when the screen turns off and before the device powers off. The top line says "typing" or "saved". Each save writes a temporary file and then puts it in the note's place, so a power cut costs a few seconds of typing and never the note; if a save was cut short, opening the note offers its copy back.
|
||||
|
||||
A new note has no file until something is typed; its file is then named after its first line (`shopping-list.txt`), or `note-<date>-<time>.txt`.
|
||||
|
||||
A note holds up to 16 KB while it's edited. A bigger text file opens read-only in the Storage App (editing any size is issue #47). The Storage App's text viewer has `e` to edit a file with the same editor, anywhere on the card, unless the file is read-only.
|
||||
**A note can be any size** (issue #47): the editor keeps a window of about 8 KB around the cursor in memory and the rest on the card, so a megabyte opens as fast as a line and uses the same 17 KB. Up to 64 KB a save rewrites the file. Above, the five-second save writes only what changed to a side file, `<note>.edit`, and the file itself is rewritten when the note is left, with a progress bar (about 450 KB a second). After a power cut, opening the note picks the edit up where it was saved. Saving needs room on the card for a second copy. The Storage App's text viewer has `e` to edit a file with the same editor, anywhere on the card, unless the file is read-only.
|
||||
|
||||
## SSH
|
||||
|
||||
The SSH App (docs/milestones/N1.md, issue #2) is a terminal on another machine: one session to a shell, over libssh (`ewpa/LibSSH-ESP32`) on mbedTLS. `user@host[:port]`, typed in the App or as `ssh user@host` in the Shell; up to eight hosts are remembered. **A server is trusted the first time, on its fingerprint, and a changed key is a warning** whose selected answer is Cancel. **The password is typed each time and kept nowhere;** or the device makes itself an Ed25519 key (kept in the device, never shown, no passphrase) whose public half is shown, written to `/ssh/id_ed25519.pub` and printed by `ssh status`, for a server's `authorized_keys`.
|
||||
|
||||
The terminal (`lib/term`, host-tested) understands what a shell, `less`, `top`, `nano` and `vim` send: the cursor, erasing, sixteen colours, scroll regions, the alternate screen; no mouse. Five text sizes with Ctrl and + or -, from 60 x 20 to 26 x 8, told to the far end; 100 lines of scrollback with Alt and up or down. The backtick key sends Esc. **Leaving the App doesn't end the session:** `SSH` shows in the Status Bar and the App finds it again. It costs 292 KB of flash (109 KB of it one table, for signing with the device's key) and about 50 KB of memory while a session is open, so it isn't started under 75 KB free and IRC doesn't connect while one is open.
|
||||
|
||||
## VPN
|
||||
|
||||
A WireGuard tunnel (docs/milestones/N1.md), over whatever Wi-Fi the device is on: one peer, IPv4. Copy a client's `.conf` to the card as `/vpn/wg0.conf` and import it in Settings > VPN (or `vpn import`); the configuration, private key included, is then kept in the device and never shown, and Settings offers to delete the file. A switch brings the tunnel up until the next restart; "Start with Wi-Fi" does it every time. It waits for the clock, which WireGuard needs. `VPN` shows in the Status Bar, bright once the server has answered.
|
||||
|
||||
**What goes through it is one of two things:** everything, when AllowedIPs has `0.0.0.0/0` (and then nothing leaves the device while the server is silent), or the one subnet the device's tunnel address is in. A home network behind the server needs the first: lwIP routes by an interface's subnet or by default, nothing finer, and the import says how many ranges it can't reach. With the tunnel up the Debug Console and the Update Service answer on the tunnel address too, behind their token and their signature. It costs 63 KB of flash and under 2 KB of memory while up.
|
||||
|
||||
## Shell
|
||||
|
||||
The Shell App (docs/milestones/S1.md) runs the commands below on the device's own screen and keyboard: no PC, no cable, no Wi-Fi. **It shows the replies to its own commands and nothing else**: the console knows who each line is printed for, so a listing read by another task a moment later is still the Shell's, and what USB or the Debug Console asked for is not. Ctrl+b shows everything instead. Tab completes a command word by word (`lora st` gives `lora status`) and, past it, a path on the SD card (`ls /no` gives `ls /notes/`), Fn with up and down recalls earlier lines, Alt with up and down scrolls back. **An App's name with a capital opens it** (`Notes`, `Irc`, `Wifi`, `Gnss`, `Gemini`, `Lora`, `Storage`, `Ssh`, `System`, `Settings`), from the consoles too. `rm` is Unix's, with a question: a folder needs `-r`; a file, or a folder with something in it, is asked about unless `-f` (`rm -rf`); an empty folder goes without a word. `*` and `?` in a name stand for several files (`rm /notes/*.txt` asks once, with the count; 64 at most). `clear` empties the screen and `quit` leaves. It is trusted like the USB port: `debug on` and `debug token` work from it. It uses about 7 KB of memory while it is open, and none otherwise. **For the network:** `ping`, `nslookup`, `port`, `traceroute`, `tls`, `ntp`, `ifconfig`, `arp` and `netstat` (issue #90).
|
||||
|
||||
## Development aids
|
||||
|
||||
@@ -159,16 +201,16 @@ A note holds up to 16 KB while it's edited. A bigger text file opens read-only i
|
||||
| Command | Effect |
|
||||
|---|---|
|
||||
| `burst` | Publishes 5 Notifications at once |
|
||||
| `key up\|down\|left\|right\|select\|back\|home`, or `key <char>` | Injects a key press |
|
||||
| `key up\|down\|left\|right\|select\|back\|home\|del\|tab\|space\|help\|shot`, or `key <char>` | Injects a key press (`help` is Fn+h: the keys of the screen that is showing; `shot` is Fn+p: a screenshot). `ctrl-`, `alt-` and `shift-` before it hold that key: `key ctrl-down`, `key alt-up`, `key ctrl-b` |
|
||||
| `sound on` / `sound off` | Toggles the Sound setting (beep + LED) |
|
||||
| `short` / `normal` | Screen timeouts 5 s / 10 s, or 30 s / 60 s |
|
||||
| `wifi add <ssid><TAB><password>` | Adds a Saved Network (so credentials stay out of the repo) |
|
||||
| `wifi ip <ssid> dhcp` / `wifi ip <ssid> <address>/<prefix> [gateway]` | A Saved Network's IP setting: Automatic, or Fixed. Debug Builds: add `try <seconds>` to go back to the previous setting unless `wifi ip keep` follows |
|
||||
| `wifi ip <ssid> dhcp` / `wifi ip <ssid> <address>/<prefix> [gateway]` | A Saved Network's IP setting: Automatic, or Fixed. Add `try <seconds>` to go back to the previous setting unless `wifi ip keep` follows |
|
||||
| `wifi dns <a> [b]` / `wifi dns always on\|off` / `wifi ntp <a> [b]` | DNS servers (used on Fixed networks, or always), and NTP servers |
|
||||
| `log <text>` | Appends a line to a test IRC Log (`/irc/dev/#test/<date>.log`) |
|
||||
| `sd card` | What the SD card says it is: type, size, and its identity register (maker, name, revision, serial, date) |
|
||||
| `sd list` | Lists the files of each Storage Clean-up category |
|
||||
| `sd fill <folder> <count>` | Debug Builds: makes that many small files in a folder, to test a crowded one |
|
||||
| `sd fill <folder> <count>` | Makes that many small files in a folder, to test a crowded one |
|
||||
| `cat <path>` | Prints the first ~1.2 KB of a file on the SD card |
|
||||
| `irc start` | Starts the IRC Service (normally done by opening the IRC App) |
|
||||
| `irc stop` | Stops it, as `/quit` does: QUIT if connected, no more retries, and the App stays disconnected until you type |
|
||||
@@ -177,16 +219,18 @@ A note holds up to 16 KB while it's edited. A bigger text file opens read-only i
|
||||
| `irc say <buffer> <text>` | Types into a Buffer, commands included (`irc say 0 /join #test`) |
|
||||
| `irc dump` | Prints IRC status, memory, and the last lines of each Buffer |
|
||||
| `wifi status` | Prints Wi-Fi state, network, signal, clock and free heap, then the address, gateway, DNS and NTP servers in use and where each came from |
|
||||
| `info` | Firmware, uptime, last start reason, memory, Wi-Fi, the SD card and its write faults since boot, and both app slots with their versions and OTA states |
|
||||
| `info` | Firmware, uptime, last start reason, memory, Wi-Fi, the SD card and its write faults since boot, **which App is in front**, and both app slots with their versions and OTA states |
|
||||
| `Notes`, `Irc`, `Wifi`, `Gnss`, `Gemini`, `Lora`, `Storage`, `Shell`, `System`, `Settings` | Opens that App: a capital letter is an App, not a command |
|
||||
| `tasks` | FreeRTOS tasks over the next second: state, priority, lowest free stack, share of a core, each core's load, and how many passes the main loop made |
|
||||
| `net` | Bytes each network service has read and written since boot |
|
||||
| `reboot` / `boot other` | Restart, or restart into the other app slot (a manual Rollback) |
|
||||
| `log level <0-5>` | ESP-IDF log level |
|
||||
| `ls [folder]` / `du <path>` | Lists a folder of the SD card with sizes and dates, or counts the files and bytes under a path |
|
||||
| `cp [-f] <from> <to>` / `mv [-f] <from> <to>` / `rm <path>` / `mkdir <path>` / `cancel` | What the Storage App does, with its rules: copy (folders too), move or rename, delete (a folder with what's in it), new folder. `-f` replaces a file that's in the way; a tab separates paths that hold spaces; `cancel` stops a copy or a delete |
|
||||
| `screenshot [seconds]` | The screen as a PNG in `/screenshots` on the card, now or after a pause to get to the screen you want (240 x 135, about 33 KB). Over the Debug Console a bare `screenshot` sends the screen to the PC instead |
|
||||
| `cp [-f] <from> <to>` / `mv [-f] <from> <to>` / `rm [-r] [-f] <path>` / `mkdir <path>` / `cancel` | What the Storage App does, with its rules: copy (folders too), move or rename, delete (`rm -r` for a folder and what's in it, as Unix has it), new folder. `-f` replaces a file that's in the way; `*` and `?` in the last part of a path (`ls`, `du`, `rm`, `cp`, `mv`) run the command for each name matched, 64 at most; a tab separates paths that hold spaces; `cancel` stops a copy or a delete |
|
||||
| `install <path>` | Update from SD with that `.ota` file, as Settings → Firmware does |
|
||||
| `update check` / `update list` / `update status` / `update install <tag>` | The project's releases on Gitea: look at the latest, list the last ten, say what's known, or download and install one (not on a Debug Build) |
|
||||
| `update pretend <version>` / `update probe <host>` / `update damage cut\|flip <n>` / `update daily` | Debug Builds: pretend to run another version (so a release counts as an update), see whether a server's certificate is accepted, cut or damage the next download, run the daily check again |
|
||||
| `update check` / `update list` / `update status` / `update install <tag>` | The project's releases on Gitea: look at the latest, list the last ten, say what's known, or download and install one |
|
||||
| `update pretend <version>` / `update probe <host>` / `update damage cut\|flip <n>` / `update daily` | Pretend to run another version (so a release counts as an update), see whether a server's certificate is accepted, cut or damage the next download, run the daily check again |
|
||||
| `lora probe` | Finds the radio: chip, oscillator, antenna switch, DIO1 interrupt, noise floor |
|
||||
| `lora status` | Radio settings, who's listening, packet and error counters, noise floor, task stack |
|
||||
| `lora rx on` / `lora rx off` | Listens and prints each packet on the console |
|
||||
@@ -198,26 +242,36 @@ A note holds up to 16 KB while it's edited. A bigger text file opens read-only i
|
||||
| `gnss track start` / `gnss track stop` | A Track, as `r` in the GNSS App (the reason is printed if it can't start) |
|
||||
| `gnss nmea on` / `gnss nmea off` | Prints each NMEA sentence the receiver sends, as `nmea: …` |
|
||||
| `gnss send <sentence>` | Sends a sentence to the receiver, without the `$` and the checksum (it adds them) |
|
||||
| `lora noise test [gnss\|quiet]` / `lora noise report` | Debug Builds: Sweep under one changed condition at a time to find what raises the noise floor (Wi-Fi goes off for a few seconds); then the result |
|
||||
| `lora inject <hex> [rssi] [snr]` | Debug Builds: a packet into the Scanner as if received (nothing is sent) |
|
||||
| `lora noise test [gnss\|quiet]` / `lora noise report` | Sweep under one changed condition at a time to find what raises the noise floor (Wi-Fi goes off for a few seconds); then the result |
|
||||
| `lora inject <hex> [rssi] [snr]` | A packet into the Scanner as if received (nothing is sent) |
|
||||
| `crash` | The last crash: which firmware, why, task, PC and backtrace (from the core dump in flash) |
|
||||
| `coredump erase` | Forgets the core dump |
|
||||
| `loop spin on` / `loop spin off` | Debug Builds: make the main loop spin without resting, to compare load and radio noise |
|
||||
| `crash abort` / `crash wdt` | Debug Builds: crash on purpose, or hang the main loop until the watchdog fires |
|
||||
| `loop spin on` / `loop spin off` | Make the main loop spin without resting, to compare load and radio noise |
|
||||
| `crash abort` / `crash wdt` | Crash on purpose, or hang the main loop until the watchdog fires |
|
||||
| `ping <host> [count] [size]` / `nslookup <name> [server]` / `port <host> <port>` / `traceroute <host>` / `cancel` | Network troubleshooting (issue #90): does a host answer and how fast; a name's addresses, from which DNS server and in how long; is a TCP port open, refused or silent; the routers on the way. Each runs on a task of its own and prints as it goes, one at a time; `cancel` stops it |
|
||||
| `tls <host> [port]` / `ntp [server]` | A TLS handshake that checks nothing, then the certificate said in words: who it is for, who signed it, until when, its SHA-256, and whether this device's roots and the name asked for accept it (about 52 KB of heap while it runs; refused under 70 KB free). A time server's clock against the device's, with the round trip |
|
||||
| `ifconfig` / `arp` / `netstat` | The interfaces (Wi-Fi and the VPN) with their addresses, MTU, which is the default route, and the DNS servers; the neighbours heard on the Wi-Fi; what listens and what is connected |
|
||||
| `ssh user@host[:port]` / `ssh status` / `ssh stop` | Opens the SSH App and connects (the password is asked there, never on a console); the session's state and this device's public key; end the session |
|
||||
| `vpn status` / `vpn up [seconds]` / `vpn down` / `vpn import [path]` / `vpn forget` / `vpn auto on\|off` | The WireGuard tunnel: its state, on (for that many seconds, then off by itself: for trying a configuration from afar), off, read a `.conf` from the card (`/vpn/wg0.conf`), erase it, start with Wi-Fi. No key is ever printed |
|
||||
| `debug status` / `debug off [seconds]` | The Debug Console: whether it's on, has a token and a client; switch it off. With a number of seconds, it comes back by itself after that long |
|
||||
| `debug on` / `debug token <value>` / `debug token new` | USB serial only: switch it on (making a token if there's none), give it a token of 16 to 64 characters, or make a new one. The token is never printed |
|
||||
| `help` | Lists the commands |
|
||||
|
||||
`scripts/flash.sh` stops a running serial log first, since it would hold the port.
|
||||
|
||||
### Debug Builds and the Debug Console
|
||||
### The Debug Console
|
||||
|
||||
`scripts/flash.sh --debug` (USB) or `scripts/flash.sh --debug --ota <ip>` (Wi-Fi) installs a Debug Build: the same firmware plus the Debug Console on TCP 2323 (ADR 0004). Then, with `RORO_OTA_HOST` set to the device's IP:
|
||||
Every firmware has the Debug Console (ADR 0010): the serial console over Wi-Fi, on TCP 2323. It is **off** until you switch it on in **Settings → Debug Console**, which also makes its token and shows it. With the device on USB, `scripts/flash.sh --debug` flashes it, switches the console on and gives it the token in `~/.config/roro9stack/debug-token`, so nothing has to be typed. Then, with `RORO_OTA_HOST` set to the device's IP:
|
||||
|
||||
```sh
|
||||
scripts/rdbg.py # interactive: the console backlog, live lines, and commands
|
||||
scripts/rdbg.py info # one command and its reply
|
||||
scripts/rdbg.py -b tasks # the same, after the backlog (boot messages and so on)
|
||||
scripts/rdbg.py -t K7QF-3M2X-9WBD-HT4P-6RNC info # another device's token; or $RORO_DEBUG_TOKEN
|
||||
```
|
||||
|
||||
The token never crosses the network: the device sends a challenge and `rdbg.py` answers with its HMAC. Five wrong answers in a row close the console for a minute. `debug status` and `debug off` work from anywhere; `debug on`, `debug token <value>` and `debug token new` work over USB serial only.
|
||||
|
||||
Every command above works there too, plus a few handled by the PC side or the console's own task:
|
||||
|
||||
```sh
|
||||
@@ -231,6 +285,6 @@ scripts/rdbg.py get <card path> [file] # from the SD card
|
||||
|
||||
So a Firmware Update can also go `rdbg.py put roro9stack-….ota` then `rdbg.py install /updates/roro9stack-….ota`: the Update from SD path, without touching the device.
|
||||
|
||||
Every build keeps its ELF in `.pio/elves/` (version and digest in the name) for that; `scripts/decode_backtrace.sh <version|digest> <addresses>` decodes any backtrace by hand.
|
||||
Every build keeps its ELF in `.pio/elves/` (version and digest in the name) for that, and `rdbg.py crash` fetches a release's ELF from Gitea when it isn't there; `scripts/decode_backtrace.sh <version|digest> <addresses>` decodes any backtrace by hand.
|
||||
|
||||
After 3 crash restarts in a row the firmware starts in **Safe Mode** (ADR 0005): only Wi-Fi, Firmware Updates and the Debug Console, so a fix can be pushed as usual. `reboot` leaves it. The token is in `~/.config/roro9stack/debug-token`, made by the first build; keep developing on Debug Builds, so the firmware a Rollback returns to always has the console.
|
||||
After 3 crash restarts in a row the firmware starts in **Safe Mode** (ADR 0005): only Wi-Fi, Firmware Updates and, if it's switched on, the Debug Console, so a fix can be pushed as usual. `reboot` leaves it.
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# A Debug Console over Wi-Fi, in Debug Builds only
|
||||
|
||||
**Superseded in part by [ADR 0010](0010-debug-console-in-every-build.md) (2026-10-06):** there is no Debug Build any more. The console is in every firmware, off until switched on, and its token belongs to the device, not to the build. What this record says about how the console works (the ring, commands on the main loop, binary commands on its own task, one client) still holds.
|
||||
|
||||
The goal of Firmware Updates is to manage the device without a cable, and that includes finding out what went wrong. So a **Debug Build** (`cardputer-adv-debug`, `-DRORO_DEBUG`, version suffix `+debug`) adds a **Debug Console** on TCP 2323: the serial console, over Wi-Fi. A client sends a token as its first line, then gets the last 4 KB of console output (boot messages included), every new line live, and runs the same commands as the serial port, plus a few that only make sense remotely. ESP-IDF's own log lines are teed into it.
|
||||
|
||||
It's compiled out of release builds entirely, rather than switched off by a setting. A console that runs commands is a remote control: in a release build, nothing listens.
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
# Safe Mode, crash reports and a watched main loop, in every build
|
||||
|
||||
Rollback protects against new firmware that fails Probation. It does nothing for firmware that was confirmed and crashes later: a corrupt setting, a server that sends something unexpected, a bug that takes an hour to show. Without a cable, such a device would restart forever. Three measures, in release and Debug Builds alike, keep it reachable:
|
||||
Rollback protects against new firmware that fails Probation. It does nothing for firmware that was confirmed and crashes later: a corrupt setting, a server that sends something unexpected, a bug that takes an hour to show. Without a cable, such a device would restart forever. Three measures, in every build, keep it reachable:
|
||||
|
||||
- **Safe Mode.** The firmware counts starts that follow a crash (panic or watchdog) in NVS, first thing at boot. After 3 in a row, it starts only the clock, Wi-Fi, the Update Service and, in a Debug Build, the Debug Console: no Apps, no IRC, no SD card, and a screen that says so with the address to push an update to. Any normal restart (a `reboot`, an update), or a minute of uptime, resets the count.
|
||||
- **Crash reports.** The same boot record keeps which version was running, so after a crash the firmware knows which one crashed, even when a Rollback has switched slots since. ESP-IDF already writes a core dump to its flash partition on a panic; after the restart the firmware prints its summary (task, PC, reason, backtrace) and raises a Notification. The `crash` command shows it again later. In a Debug Build, `scripts/rdbg.py crash` decodes the backtrace and `scripts/rdbg.py coredump` fetches the whole dump for `esp-coredump`, against the ELF of that exact build (`.pio/elves/`, named by version and ELF digest).
|
||||
- **Safe Mode.** The firmware counts starts that follow a crash (panic or watchdog) in NVS, first thing at boot. After 3 in a row, it starts only the clock, Wi-Fi, the Update Service and, if it's switched on, the Debug Console: no Apps, no IRC, no SD card, and a screen that says so with the address to push an update to. Any normal restart (a `reboot`, an update), or a minute of uptime, resets the count.
|
||||
- **Crash reports.** The same boot record keeps which version was running, so after a crash the firmware knows which one crashed, even when a Rollback has switched slots since. ESP-IDF already writes a core dump to its flash partition on a panic; after the restart the firmware prints its summary (task, PC, reason, backtrace) and raises a Notification. The `crash` command shows it again later. With the Debug Console, `scripts/rdbg.py crash` decodes the backtrace and `scripts/rdbg.py coredump` fetches the whole dump for `esp-coredump`, against the ELF of that exact build (`.pio/elves/`, named by version and ELF digest).
|
||||
- **The main loop is watched.** Arduino-ESP32 subscribes only core 0's idle task to the task watchdog, and the main loop runs on core 1: a stuck loop used to hang the device for good, with the screen frozen and the Debug Console unable to run commands. `enableLoopWDT()` makes a loop stuck for 5 s a panic, with a core dump, counted towards Safe Mode. And an installed update no longer depends on the main loop: the Update Service restarts into it by itself after 90 s.
|
||||
|
||||
## Consequences
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
# The Debug Console is in every build, off until its owner switches it on
|
||||
|
||||
There is **one firmware**. The Debug Console of ADR 0004 is compiled into it, and so are the commands that used to exist only in Debug Builds. It listens only while **Settings → Debug Console** is on, which is not the default, and it lets in whoever proves they hold **the device's own token**. The `cardputer-adv-debug` environment, `RORO_DEBUG`, the `+debug` version and the token compiled in from the builder's machine are gone.
|
||||
|
||||
ADR 0004 kept the console out of release builds with one argument: a console that runs commands is a remote control, so in a release nothing should listen. That argument was never whole: the Update Service has listened on TCP 3232 in every build since Firmware Updates exist, guarded by a signature. A listener that is off by default and guarded by a secret the device made itself is the same kind of trade, and what the split cost had grown:
|
||||
|
||||
- **What was tested wasn't what shipped.** Development ran on Debug Builds; releases were a different binary, built to be published and never run by the person who wrote them.
|
||||
- **Debug Builds couldn't be published**, since each carried its builder's token. So the console, `rdbg.py`, screenshots and crash decoding were for whoever built the firmware, and nobody else.
|
||||
- **Rules that existed only to protect the console:** a Debug Build never installed a release (it would have lost the console), `update install … force` to do it anyway, "keep a Debug Build in the fallback slot", versions with `+debug` that had to compare equal to their release.
|
||||
- **CI built two firmwares** on every pull request and every tag.
|
||||
|
||||
## How it works
|
||||
|
||||
- **Off means nothing is there.** With the setting off, no socket listens, the console's task doesn't exist and neither does its 4 KB ring: a device that never uses the console pays for it in flash (30 KB more than the release build was) and 88 bytes of static RAM, measured. A missing or invalid stored setting counts as off.
|
||||
- **The token is the device's.** The first time the console is switched on, the device makes one from its hardware random generator: 100 bits, as 20 characters of Crockford's base32 (`K7QF-3M2X-9WBD-HT4P-6RNC`), shown on the Debug Console page and nowhere else. It can be replaced by one typed by hand, of 16 to 64 characters. It is stored with the other settings and is never printed on a console.
|
||||
- **The token never crosses the network.** The device sends 16 random bytes; the client answers with their HMAC-SHA256 keyed by the token. Someone on the same Wi-Fi who records a login has nothing they can use for the next one.
|
||||
- **Five wrong answers in a row** close the console to everyone for a minute, with a Notification naming the address they came from. A wrong answer still costs a second.
|
||||
- **`DBG` in the Status Bar** while the console listens, bright while a client is connected.
|
||||
- **USB serial can set it up:** `debug on`, `debug token <value>`, `debug token new`. `scripts/flash.sh --debug` uses them to give a freshly flashed device the developer's token. Whoever holds the cable can flash anything anyway (ADR 0003); the console itself can only switch itself off.
|
||||
- **Safe Mode** starts the console if it's switched on: the settings are read before Safe Mode is decided.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **"Nothing listens" now rests on one stored setting**, not on absent code. That setting is typed, validated and host-tested, and its default is off; a bug that flipped it would have to come from code that already runs on the device.
|
||||
- **Whoever has the token and the network has the device:** the console, its keys, the SD card's files, a restart. Not its firmware: an update still has to be signed (ADR 0003).
|
||||
- **The stream is still plain text.** The token is safe from an eavesdropper; what the console prints, and the files it carries, are not. TLS would cost about 52 KB of the 107 KB there is, next to IRC's own connection: not for a console.
|
||||
- **An old `rdbg.py` can't talk to a new firmware**, and the reverse: the first line of the protocol changed. Accepted, with one user.
|
||||
- **A device whose settings are damaged has no console in Safe Mode**, only the signed update push. Before, a Debug Build always had it.
|
||||
- **The commands that crash, damage a download or fill a folder on purpose** are in every build. They need the cable or the token, like everything else.
|
||||
- **Releases already publish their ELF**, so `rdbg.py crash` fetches it when it isn't in `.pio/elves/`: crash reports can be decoded without having built the firmware.
|
||||
@@ -1,6 +1,6 @@
|
||||
# F1 — Files and Notes
|
||||
|
||||
**Status:** in progress. The Storage App (issue #3) shipped as **v0.9.0** on 2026-10-06. Notes (#19) shipped as **v0.10.0** the same day. The card as a USB drive (#1) comes after.
|
||||
**Status:** in progress. Shipped: the Storage App (issue #3, **v0.9.0**), Notes (#19, **v0.10.0**), notes of any size (#47, **v0.15.0**), pictures in the Storage App (#45, **v0.16.0**), sharing the card with a browser (#88, **v0.18.0**). Not started: the card as a USB drive (#1), selecting several items (#41), finding files by name (#42), opening a `.gmi` in Gemini (#43), a table view for `.csv` (#44), search, undo and copy-paste in Notes (#48, #49, #50).
|
||||
|
||||
**Goal:** get at what's on the SD card from the device itself: browse it, look inside the files the firmware writes, copy, move, rename and delete, and keep notes. A side milestone, like G1 and S1; Files and Notes were M3's original second half (Q30, Q89).
|
||||
|
||||
@@ -98,7 +98,7 @@ Plain text notes on the SD card, written on the device. Q30 settled the base: `.
|
||||
| Q141 | A **Notes** App in the Launcher. One row per note: its first line as the title, then the date. Newest first; `s` switches to by name. `n` new, Enter opens, `d` deletes after a confirmation, `r` renames the file. |
|
||||
| Q142 | A new note's file name is never typed: it comes from the first line when the note is first saved (`shopping-list.txt`), or `note-20261006-0919.txt` if that line is empty. It doesn't change afterwards unless the note is renamed. |
|
||||
| Q143 | **Autosave, no "discard changes?" prompt:** five seconds after the last key, on leaving the note or the App, and when the screen turns off. A save writes a temporary file and renames it over the note, so a power cut loses the last few seconds at most. A temporary file left behind is offered back at the next open. |
|
||||
| Q144 | The whole note is in memory while it's edited, up to **16 KB**. A bigger text file opens read-only in the Storage App's viewer. The App refuses to open below the memory floors (Q86). **Editing files of any size must come in a later release: issue #47.** |
|
||||
| Q144 | The whole note is in memory while it's edited, up to **16 KB**. A bigger text file opens read-only in the Storage App's viewer. The App refuses to open below the memory floors (Q86). *(Lifted by issue #47: see "Notes of any size" below.)* **Editing files of any size must come in a later release: issue #47.** |
|
||||
| Q145 | The editor wraps at spaces, 38 columns by 8 rows, with a line for the name and the state. Enter is a new line, Del deletes backwards, Fn+arrows move (the Text Entry rule), Ctrl+A and Ctrl+E go to the start and the end of the line, Tab types two spaces, Back saves and returns. The Compose Key works as elsewhere. |
|
||||
| Q146 | The Storage App's text viewer gets `e`: edit this file with the same editor, for a text file up to 16 KB that isn't read-only. That lifts Q135 without Apps opening each other (#43 stays). |
|
||||
| Q147 | The list is flat: the files directly in `/notes`. Sub-folders are reached through the Storage App. |
|
||||
@@ -159,3 +159,177 @@ Test notes were made in `/notes` and removed afterwards; the folder is left, emp
|
||||
**Not checked:** accents through the Compose Key and Ctrl+A / Ctrl+E (the remote `key` command can't send them; the model's tests cover both), the power button's save (it needs a hand on the device), a missing card, and how typing feels on the keyboard itself.
|
||||
|
||||
**One slip during the checks:** a key sequence sent right after a restart opened IRC instead of Notes, and the test letters went into IRC's input line. Nothing was sent: the line was cleared and the App left. IRC connected to Libera as it does when opened.
|
||||
|
||||
## Notes of any size (issue #47)
|
||||
|
||||
Q144 held the whole note in memory and stopped at 16 KB, for the first version only. This lifts it: the editor opens a text file whatever its size.
|
||||
|
||||
### Decisions (design round 2026-10-07)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q223 | **The note is the file on the card plus one window in memory.** The window is the `NoteText` of before, up to 16 KB around the cursor; the rest is a list of pieces: runs of the file, and runs of a side file. The cursor leaving the window writes it to the side file if it was changed, and loads the next. Typing never fills a note: a full window is written away and loaded smaller. |
|
||||
| Q224 | **The five-second save:** up to 64 KB it rewrites the file, as before (about 150 ms). Above, it appends the window and the list of pieces to `<note>.edit`: 8 KB or so, whatever the note's size. "saved" means "on the card" either way. |
|
||||
| Q225 | **The file itself is rewritten on leaving the note** (Back, Home, another App), with a progress bar. The screen turning off and the device powering off write the side file only: powering off never waits. |
|
||||
| Q226 | **After a power cut, opening the note picks the edit up** where it was last saved, without a question, and says so. Until then the file has the old text for anything else that reads it. |
|
||||
| Q227 | If the file was changed elsewhere meanwhile, the side file no longer fits it: it is **kept as `<note>.edit.lost`** and the editor says so. Typed text is never deleted without a word. |
|
||||
| Q228 | **No limit but the card:** a note over 16 KB needs room for a second copy to be opened for editing. No warning for a big file; the progress bar on leaving tells the cost. |
|
||||
| Q229 | A side file over 1 MB, or a list of over 256 pieces, makes the next save a rewrite. |
|
||||
| Q230 | **CRLF becomes LF** (Q148) for a long file too: in the window as it is read, and in the rest of the file as the rewrite streams it, so a saved file is never of both kinds. |
|
||||
| Q231 | **One path.** A 16 KB note is the case with no pieces: there is no second editor for small notes. |
|
||||
| Q232 | Notes, and `e` in the Storage App's viewer, which no longer says "Too big to edit". |
|
||||
|
||||
### As built
|
||||
|
||||
- **`NoteDocument`** (`lib/notes/src/note_document.h`, host-tested against a card in memory) is the list of pieces, the window's moves, the side file and the recovery. `NoteText` is unchanged but for being refilled.
|
||||
- **The window moves** when the cursor comes within 2 KB of an end of it that isn't an end of the note: it is then 4 KB on each side of the cursor. It starts where a line starts on screen whenever that can be known (after a newline, or where the window before had a line start), so the same text wraps the same from one window to the next, and never in the middle of a character. The cursor keeps its row on screen.
|
||||
- **Looking writes nothing:** a window that wasn't changed goes back as the pieces it was read from.
|
||||
- **The side file** starts with a line of text, the note's size and checksums of its first and last kilobyte, which is how a file changed elsewhere is told. After that, text that left a window, and snapshots of the list of pieces, each with its checksum. The newest snapshot that checks out is the note as last saved; anything after it is ignored.
|
||||
- **The rewrite** streams the pieces and the window into `<note>.tmp`, checks its size, then writes a mark at the end of the side file: from that mark on, the rewrite counts as done, and opening the note finishes it whatever was cut (remove the old file, rename, remove the side file). Before the mark, the note and its side file are still the truth and the temporary file is dropped.
|
||||
- **On the device** the card is reached through an adapter that keeps the file being read and the file being appended to open between calls; every call runs on the storage task while the main loop waits. The rewrite runs in steps of 64 KB with the progress drawn between them.
|
||||
- **The Notes list** doesn't show `.edit` and `.edit.lost` files, and a note's side file is deleted and renamed with it.
|
||||
- **Ctrl with Fn+Up and Fn+Down** go to the start and the end of the note.
|
||||
- **`key ctrl-down`**: the consoles' `key` command takes `ctrl-`, `alt-` and `shift-`, which these checks needed. It also lets the checks S1 couldn't make (Ctrl+b, the Alt scroll) be made.
|
||||
- **Cost:** 15 KB of flash. Memory with a note open is what it was: 17.5 KB, for 62 bytes or for 1.2 MB.
|
||||
|
||||
### Host tests (15, `test/test_note_document`)
|
||||
|
||||
A walk down 3,000 lines and back up through the windows; start and end; an edit in the middle rewritten into the file; 48 KB typed into a new note; a journal picked up after a cut; **a cut at every 997th byte of a sequence of two saves and a rewrite**, after which the note is always one of the three texts it should be, what was reported saved is there, and no stray file is left; a file changed elsewhere; CRLF; windows on text with no space and no newline, made of 2, 3 and 4-byte characters; a full card; and **36,000 random keys** (typing, deleting, moving, jumping, saving, power cuts) on six notes of 30 to 130 KB, compared with a plain string after every key.
|
||||
|
||||
### Checks on the device (2026-10-07, driven over the Debug Console)
|
||||
|
||||
Test notes were copied to `/notes` and removed afterwards; the note that was already there was not touched.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| A 36 KB note | Opens (it was refused before). Two letters at the top, 400 lines down across the windows, four more: the file fetched back is exactly that, and no other file is left |
|
||||
| A 1.2 MB note | Opens at once. Free memory 104.2 KB before, 86.7 KB with it open |
|
||||
| Its five-second save | `zz-big.txt.edit`, 4 KB; the note's file untouched |
|
||||
| Ctrl with Down, Ctrl with Up | The end and the start, as fast as any key |
|
||||
| A restart with unsaved keys | "Your unsaved changes are back", the cursor where it was, the unsaved keys gone and nothing else |
|
||||
| Leaving it | The progress bar, then one file: **1.2 MB rewritten in 2.6 s**. Fetched back: the original with what was typed at both ends, byte for byte |
|
||||
| A restart in the middle of that rewrite | The note, its side file and an empty `.tmp` remain; opening picks the edit up, leaving rewrites it, the result is right |
|
||||
| A new note | No file until typed in, then `zz-test-note.txt` from its first line |
|
||||
| `e` in the Storage App on the 1.2 MB file | The same editor; edited and rewritten |
|
||||
| The Notes list | Side files are not listed as notes |
|
||||
|
||||
**Not checked:** the power button's path (side file only), the screen turning off, a card pulled while editing, and memory with IRC connected, which wasn't connected for these checks: the editor's own use hasn't changed, and it still refuses to open without a free block of 24 KB. The real keyboard's Ctrl with Fn and the arrows. A file of tens of megabytes. Renaming or deleting a note from the Storage App leaves its side file behind.
|
||||
|
||||
**Measured against what was said:** the first build rewrote 1.2 MB in 3.5 to 4.5 s, with 2 KB blocks. With 4 KB blocks it is 2.6 s, about 450 KB a second, which is what the card gives a plain copy.
|
||||
|
||||
## Pictures in the Storage App (issue #45)
|
||||
|
||||
Q139 left images out: the firmware wrote none. Since the Shell's `screenshot` it does.
|
||||
|
||||
### Decisions (design round 2026-10-07)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q233 | **PNG, JPEG, BMP and GIF.** A GIF shows its first picture; it doesn't move. |
|
||||
| Q234 | *Revised while building.* **Our own PNG decoder**, a row at a time, with the window the file's compression asks for: 32 KB at most. The plan was the display library's, which takes 44 KB in one block: after one picture the largest free block was 43 to 47 KB, and every second PNG was refused. **The firmware's own screenshots** are read with no decoding at all: they are stored uncompressed, each byte already a colour of the screen. |
|
||||
| Q235 | **The picture is decoded once, straight into the screen's buffer, and left there.** No copy in memory (it would be up to 30 KB). `App::retainsContent()` tells the screen not to clear the App's part; `contentLost()` tells the App that it was cleared after all, or that a Toast or the help panel drawn over it has gone: then it is decoded again. |
|
||||
| Q236 | **Shrunk to fit; Enter shows it at its own size**, the arrows then moving half a screen. A picture smaller than the screen sits in the middle at its size. |
|
||||
| Q237 | **Ordered dithering** to the screen's 256 colours (a 4 x 4 pattern). A colour the screen has exactly comes out as itself wherever it lands, so a screenshot isn't touched. |
|
||||
| Q238 | Shrinking takes, for each pixel of the screen, the first of the picture's that falls on it. No averaging: there is nowhere to keep the sums. Thin lines break up; a JPEG looks better, its decoder halving it up to three times first. |
|
||||
| Q239 | **What can't be shown opens as hex, with the reason:** a progressive JPEG, an interlaced PNG, a BMP that is compressed or has 16 bits. The rotation a camera stores in the file is ignored. |
|
||||
| Q240 | *Revised while building.* **Decoding runs on the storage task while the main loop goes on.** The picture appears as it comes, and anything that needs the screen back stops the decoding first. The plan was to wait for it, behind a "Decoding..." line: 12 megapixels took longer than the watchdog allows the main loop to stand still, and the device restarted. |
|
||||
| Q241 | The Storage App: Enter on `.png`, `.jpg`, `.jpeg`, `.bmp`, `.gif`, or on a file with no known extension whose first bytes say what it is. Tab gives the hex. `i`, and opening, show the size in pixels on the last line for three seconds. |
|
||||
| Q242 | Not in this one: animation, opening a picture from the Gemini App, a slideshow. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`lib/files/src/image_file.h`** (host-tested): what a file is and how big, where each pixel lands (`ImageFrame`, `ImageMap`), the dithering, and readers for BMP and GIF that hand their pixels on as they get them. **`png_reader.h`**: the PNG decoder, with its own inflate: every colour type and bit depth, palettes with transparency. Transparent pixels are left as the background.
|
||||
- **`ImagePane`** (`src/apps/image_pane`) is the view. One decoding is a `Job` shared with the storage task; `cancel()` flags it and waits behind it in the task's queue, which is how the screen is known to be free again.
|
||||
- **JPEG is the one decoder that isn't ours:** the display library's TJpgDec, with its 3.9 KB pool. It shrinks by 2, 4 or 8 while decoding, which is why a photograph is possible at all.
|
||||
- **A decoder stops early** once the rest of the file is below the screen (a picture at its own size), and a BMP's rows that aren't shown aren't read.
|
||||
- **The note** on the last line is written over the picture; the strip under it (2.6 KB) is kept and put back, so showing it costs no decoding.
|
||||
- **A BMP is read in the order its rows are stored**, last row first: reading it top to bottom meant going back through the file for every row, a second for 135 rows.
|
||||
- **Cost:** 21 KB of flash. Nothing while no picture is shown.
|
||||
|
||||
### Measured on the device
|
||||
|
||||
| Picture | Fitted | Its own size |
|
||||
|---|---|---|
|
||||
| A screenshot of ours, 240 x 135 | 80 ms | 78 ms |
|
||||
| PNG, 800 x 600 | 855 ms | 575 ms |
|
||||
| PNG with transparency, 800 x 600 | 1,098 ms | |
|
||||
| JPEG, 800 x 600 | 305 ms | |
|
||||
| JPEG, 4000 x 3000 (2.6 MB) | 6.9 s | 7.7 s (the middle of it) |
|
||||
| GIF, 800 x 600 | 642 ms | |
|
||||
| BMP, 800 x 600 (1.4 MB) | 991 ms | |
|
||||
| BMP, 240 x 135 | 124 ms | |
|
||||
|
||||
Free memory fell to 48.8 KB at the lowest while a PNG was decoded, from 104 KB. The storage task's stack: 3.2 KB never used, of 6.
|
||||
|
||||
### Host tests (20, `test/test_image_file` and `test/test_png_reader`)
|
||||
|
||||
The pictures in them were made with Pillow, so the readers are checked against an encoder that isn't ours: GIFs plain, interlaced, transparent and long enough for the codes to reach 12 bits; BMPs of 8 and 24 bits; PNGs of every kind (colour, with alpha, palette of 8 and 4 bits with a transparent entry, greys of 1, 8 and 16 bits, grey with alpha, not compressed, and one that refers 21,600 bytes back). Every reader is also fed its file with a byte changed, at every few bytes: an answer each time, and no pixel outside the picture.
|
||||
|
||||
### Checks on the device (2026-10-07, driven over the Debug Console)
|
||||
|
||||
Test pictures were copied to a scratch folder and removed afterwards, with the test screenshot.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Colour bars as PNG, JPEG, BMP and GIF, 240 x 135 and 800 x 600 | The same picture each time, the colours in the right order |
|
||||
| A PNG with a transparent square | The square is the background |
|
||||
| A screenshot taken in the Shell | Shown; at its own size it is the screen, pixel for pixel |
|
||||
| A progressive JPEG | Hex, with "A progressive JPEG can't be shown" |
|
||||
| An animated GIF | Its first picture |
|
||||
| 12 megapixels | Arrives from the top down in 6.9 s; the size is noted when it is whole |
|
||||
| Enter, then the arrows | Its own size from the middle, then half a screen at a time |
|
||||
| Back in the middle of a decoding | The folder's listing at once |
|
||||
| Tab to hex and back, three times; the help panel, then closed | The picture again each time |
|
||||
|
||||
**Not checked:** a photograph from a real camera (the test JPEGs were made by Pillow); a Toast over a picture; a PNG with IRC connected, when there may not be the memory; the card pulled while decoding; the real keyboard.
|
||||
|
||||
### What went wrong while building it
|
||||
|
||||
**The watchdog.** The first version waited for the decoder. The main loop is watched: five seconds without a pass and the device restarts, which is what it did on the 12-megapixel test. The crash report named the decoder's line. The decoding moved to the background, which also made the picture appear as it comes.
|
||||
|
||||
**A decoder that worked once.** The library's PNG decoder showed the first picture and refused the next five: "no memory". It wants 44 KB in one piece, and after some use the largest piece is 43 to 47 KB. Writing a decoder that needs 32 KB was less work than it sounds, and unlike the library's it has tests.
|
||||
|
||||
**Two sentences still said "up to 16 KB"** about editing, in the README and the Storage guide, after issue #47 lifted that. Corrected here.
|
||||
|
||||
## Sharing the card with a browser (issue #88)
|
||||
|
||||
Files reached the card through the Debug Console's `put` and `get`, or by taking the card out. A phone has neither.
|
||||
|
||||
### Decisions (2026-10-07; the recommendation was accepted as it stood, without a round of questions)
|
||||
|
||||
- **A web page, not FTP, SFTP or WebDAV.** A browser is the only client every phone has. FTP and WebDAV need an app there; SFTP needs a whole SSH server here. WebDAV can come later on the same server, for computers.
|
||||
- **Off unless asked for:** `w` in the Storage App opens a "Share" screen, and the server runs only while that screen is open.
|
||||
- **A six-digit code on the screen**, new each time, typed in the page; the address is also a QR code, which carries the code. Five wrong codes close it for a minute (the Debug Console's `AuthGate`). A browser that got it right holds a cookie; starting again puts every browser out.
|
||||
- **Not encrypted.** A TLS server costs about 40 KB of memory a connection. The screen says so.
|
||||
- **The Storage App's rules** (`whyReadOnly`): the firmware's own folders, and files in use.
|
||||
|
||||
### As built
|
||||
|
||||
- **`WebShare`** (`src/services/web_share`): ESP-IDF's HTTP server, which is in the framework already. Seven requests: the page, the code, a listing as JSON, a download, an upload, a new folder, a delete.
|
||||
- **Every access to the card is handed to the storage task**, 8 KB at a time, from the server's own task: a download and an upload are loops of "one piece from the card, one piece to the network".
|
||||
- **An upload is the request's body**, as the browser's `PUT` sends it: no form to take apart. It goes to `<name>.part` and is renamed when the last byte has come; anything less is removed. A file that exists is refused unless the page asked, after asking the user.
|
||||
- **The page** (`web_share_page.h`) is one file of 5.4 KB with its style and script in it, served from flash.
|
||||
- **`lib/files/src/share_rules.h`** (host-tested, 5 tests): what a request names, which paths a browser may ask for (from the root, no `..`), the JSON, the code and the cookie.
|
||||
- **Cost:** 57 KB of flash, most of it the server. 13 KB of memory while sharing (108.4 KB free before, 95.4 with the screen open), given back on leaving.
|
||||
|
||||
### Checks on the device (2026-10-07 and 08)
|
||||
|
||||
A scratch folder was used and removed.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `w` | The QR code, the address and the code; `share: on` on the console |
|
||||
| The page, and a listing without the code | 200; 401 |
|
||||
| A wrong code, the right one (typed `825 132`) | 403; in |
|
||||
| Upload, 2.6 MB | 11 to 17 s (150 to 230 KB/s); downloaded again and compared: the same, byte for byte |
|
||||
| The same name again; with "replace" | 409; replaced |
|
||||
| `..` in a path; deleting `/notes`; deleting a folder that isn't empty | 400; 403 "The firmware keeps its files in /notes"; 403 |
|
||||
| Five wrong codes | The fifth and every one after: 429, the right code too. A browser that was in stays in |
|
||||
| In Chromium at a phone's width | The scanned address logs in by itself; two files uploaded, one downloaded and compared, a folder made, a file deleted after asking, a replacement after asking, the refusal shown. No sideways scroll |
|
||||
| Back | The server is gone (connection refused), memory is back |
|
||||
|
||||
**Found on the way:** the server answers one request at a time. A second request during a slow download waited until it had ended. It is said in the guide, and not changed.
|
||||
|
||||
**On a real phone** (the maintainer's, 2026-10-08): the QR code, scanned with the phone's camera, opens the page and logs in; the page lists the card, in the phone's dark theme.
|
||||
|
||||
**Not checked:** Safari. A card pulled during a transfer. Sharing with IRC connected, when memory is shorter. Home, and the screen turning off, while sharing (the code stops the server on leaving the App; only Back was tried).
|
||||
|
||||
@@ -0,0 +1,222 @@
|
||||
# N1 — Network tools
|
||||
|
||||
**Status:** in progress. The WireGuard tunnel (issue #8) shipped as **v0.19.0**. The network troubleshooting commands (issue #90) shipped in two parts: `ping`, `nslookup`, `port`, `traceroute`, `ifconfig` and `arp` as **v0.20.0**; `tls`, `ntp` and `netstat` as **v0.21.0**. The SSH client (issue #2) shipped as **v0.22.0**.
|
||||
|
||||
**Goal:** reach things from the device that aren't on the Wi-Fi it happens to be on, and keep its traffic private on a network that isn't yours.
|
||||
|
||||
## The WireGuard tunnel (issue #8)
|
||||
|
||||
A WireGuard client: the Cardputer joins a WireGuard network over whatever Wi-Fi it is on.
|
||||
|
||||
### Measured before deciding (2026-10-07)
|
||||
|
||||
The issue asked for the libraries to be measured first. `esphome/wireguard` 0.4.8 (maintained, published the same week; BSD-3-Clause) was built into a trial firmware and a tunnel brought up against a throwaway peer in a container.
|
||||
|
||||
| | Cost |
|
||||
|---|---|
|
||||
| Flash, the library | 43 KB |
|
||||
| Flash, with our service, page and commands | 63 KB |
|
||||
| Static RAM | 1.2 KB |
|
||||
| Heap with the tunnel up | 1.8 KB |
|
||||
|
||||
- **It crashes this build as shipped.** The library calls lwIP's raw functions without taking lwIP's lock, and this framework is built to check for that (`CONFIG_LWIP_CHECK_THREAD_SAFETY`): the first `netif_add` stopped the device. Every call into it is made with the lock held, on our side; the library is not changed.
|
||||
- **One address range is allowed by default;** more need `CONFIG_WIREGUARD_MAX_SRC_IPS`, set in `platformio.ini`.
|
||||
- One peer, IPv4.
|
||||
- The older `ciniml/WireGuard-ESP32` was last touched in 2021 and was not tried.
|
||||
|
||||
### Decisions (design round 2026-10-07)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q243 | **`esphome/wireguard`, pinned at 0.4.8,** with lwIP's lock taken around every call. |
|
||||
| Q244 | **Configured by importing a standard `.conf` from the card** (`/vpn/wg0.conf`), from Settings or with `vpn import`. Nothing is typed on the device. |
|
||||
| Q245 | **The private key comes in that file,** as WireGuard configurations are handed out. It is kept in the device's settings, never shown and never printed. After an import Settings **offers to delete the file**: the card comes out, and the key is in it in clear. |
|
||||
| Q246 | One tunnel, one peer. |
|
||||
| Q247 | **A switch, and "Start with Wi-Fi"** (off by default). The switch is for now: it doesn't outlast a restart. The tunnel waits for the clock, since a handshake carries the time and a server refuses one older than the last it saw; the clock is set over plain Wi-Fi first. |
|
||||
| Q248 | *Narrowed while building.* **Either everything goes through the tunnel, or one subnet does.** With `0.0.0.0/0` in AllowedIPs the tunnel is the default route. Otherwise only the subnet this device's tunnel address is in is routed: the widest allowed range that holds it. **A home network behind the server can't be reached without the full tunnel:** lwIP routes by an interface's own subnet or by default, and has no table for anything finer. The import says how many ranges it can't reach. |
|
||||
| Q249 | *Not as planned.* **With everything through the tunnel, nothing leaves while the server is silent:** the default route stays in the tunnel, which has nowhere to send. That is a kill switch, by construction and not by choice. With one subnet, packets for it go out on Wi-Fi again while the tunnel has no peer. |
|
||||
| Q250 | The file's DNS servers are used while the tunnel is up, if they can be reached through it; what was there before goes back when it stops. |
|
||||
| Q251 | **The Debug Console and the Update Service answer over the tunnel** as they do on Wi-Fi: the console still wants its token and an update its signature. |
|
||||
| Q252 | **`VPN` in the Status Bar** while the tunnel is wanted, bright once the server has answered. Settings > VPN has the state, the server, this device's address, what goes through it and how long ago the server was heard. `vpn status`, `up`, `down`, `import`, `forget`, `auto`. A Toast when it comes up and when the server stops answering. |
|
||||
| Q253 | PresharedKey, MTU and ListenPort from the file; keepalive 25 s if the file has none; the tunnel is taken down with the Wi-Fi it was on and started afresh on the next. No IPv6. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`lib/net/src/wg_config.h`** (host-tested, 6 tests): reads a `.conf` as people write them (any case, comments, CRLF, IPv6 entries left out), refuses what it can't use with the line and the field and never the key, writes it back tidy for the settings store, and says what will be routed.
|
||||
- **`VpnService`** (`src/services/vpn_service`): the tunnel is up when it is wanted, Wi-Fi is connected and the clock is set. It holds lwIP's lock around the library, adds the allowed ranges, makes the tunnel the default route for "everything", and puts the DNS servers in and out. A DHCP renewal that replaces them is noticed: the tunnel's go back in, and the renewed ones are what is restored later.
|
||||
- **The tunnel's own packets never go into the tunnel:** the library sends them on the interface that was the default when it started.
|
||||
- **Connections that came in over Wi-Fi stay on Wi-Fi** with everything routed into the tunnel: a reply leaves by the interface whose address it carries.
|
||||
- **`vpn up <seconds>`** takes the tunnel down again by itself: for trying a configuration from afar, when a wrong one could cut the connection it was sent over.
|
||||
- Settings: `VpnConfig` (the `.conf`, checked on every load) and `VpnAuto`.
|
||||
|
||||
### Checks on the device (2026-10-07, against a WireGuard peer in a container)
|
||||
|
||||
The test keys were made for the purpose and deleted. Two rounds: first with the device on a guest Wi-Fi that can't open connections to the machine the test peer ran on, so **the peer called the device** (`ListenPort`), which WireGuard allows either way round; then on a network where **the device called the peer**, as it normally would.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `vpn import`, then the file removed | "imported, through it 10.9.0.0/24"; the configuration survives a firmware update |
|
||||
| `vpn up` | Up within seconds; `VPN` bright in the Status Bar; a Toast |
|
||||
| From the peer, through the tunnel | 25 pings of 25, 1300 bytes too; the Debug Console's greeting on TCP 2323; TCP 3232 answers |
|
||||
| DNS | The file's server while up (`wifi status` says `(VPN)`), DHCP's back after `vpn down`, with no reconnection |
|
||||
| Everything through the tunnel | The device stays reachable over Wi-Fi; an update check's HTTPS to the release server is seen inside the tunnel at the peer |
|
||||
| `vpn up 100` | Down by itself after 100 s |
|
||||
| "Start with Wi-Fi", then a restart | Up by itself 40 s after the restart, once Wi-Fi and the clock were there |
|
||||
| The peer silenced | After three minutes: "no answer yet", a Toast, `VPN` dim. With everything through the tunnel, an update check then fails: nothing leaves. The peer back: up again in under half a minute, and a Toast |
|
||||
| `vpn forget` | "not set"; DNS as before |
|
||||
| **The device calling the peer**, the server given by name, with a PresharedKey and `MTU = 1280` | Up in seconds; the peer shows the device's address and port as the endpoint; pings through it |
|
||||
| One subnet: a connection the device opens to the peer's tunnel address | Seen inside the tunnel at the peer |
|
||||
| Everything: a Gemini page from a public capsule | Fetched (TLS, 1,184 bytes), and seen inside the tunnel at the peer. Free memory fell to 45.9 KB at the lowest |
|
||||
| Memory | 106.1 KB free before, 104.3 KB with the tunnel up, 106.2 KB after |
|
||||
| Settings > VPN | The four rows, the state and "heard 66 s ago", the server, the address; no key anywhere on it |
|
||||
|
||||
**Against a real server** (the maintainer's own, 2026-10-08): a configuration put on the card by the maintainer and imported in Settings; the server named by host name, on a port of its own, the device's address a /32, everything through the tunnel, "Start with Wi-Fi" on. The tunnel is up, and from another machine the device answers on its tunnel address: pings, and the Debug Console.
|
||||
|
||||
**Not checked:** from a network far from the server (the device was on the server's own network, reaching it by its public name). That the MTU is what limits a packet (larger pings were answered too, in pieces). Roaming from one Wi-Fi to another with the tunnel wanted. IRC through the tunnel. A day of uptime.
|
||||
|
||||
### What went wrong while building it
|
||||
|
||||
**The device stopped on the first try,** on lwIP's "Required to lock TCPIP core functionality!". The library was written for builds that don't check; ours does. The fix is three lines of ours, and the crash report named `netif_add` and the line that called it.
|
||||
|
||||
**Taking the tunnel down reconnected Wi-Fi.** The first version gave DHCP's DNS servers back by asking for a new lease, which is how the Wi-Fi settings do it, and which drops every connection: the Debug Console session that had typed `vpn down` among them. The servers that were there are now simply remembered and put back.
|
||||
|
||||
**"What AllowedIPs say" was more than the network stack can do.** The design round promised split tunnels by AllowedIPs. lwIP has no routing table: it can send by an interface's subnet, or by default. So it is one subnet or everything, and the import tells which.
|
||||
|
||||
## Network troubleshooting commands (issue #90)
|
||||
|
||||
With a tunnel, fixed addresses and a file server on the device, "is it the network or is it me" needed another machine to answer.
|
||||
|
||||
### Decisions (2026-10-08; built on the issue's list, without a round of questions)
|
||||
|
||||
- **The familiar names:** `ping`, `nslookup`, `traceroute`, `ifconfig`, `arp`. `port <host> <port>` for "is that TCP port open", which has no single familiar name.
|
||||
- **In this version:** those six. **Not yet:** `tls` (why a certificate fails), `ntp` (the clock's offset), `netstat` (what listens). The issue stays open for them.
|
||||
- **Commands only,** in the Shell and over both consoles; no page in an App.
|
||||
- **One line an answer, short:** a Shell line is 38 characters.
|
||||
|
||||
### As built
|
||||
|
||||
- **`lib/net/src/net_probe.h`** (host-tested, 5 tests): what was typed; the ICMP echo request and what answers it, a router's "time exceeded" included; the DNS query and its answer, with the pointers names are shortened by.
|
||||
- **`NetTools`** (`src/services/net_tools`): `ping`, `nslookup`, `port` and `traceroute` each run on a task of their own, made for the command and gone after it, printing to the console that asked (the Shell shows only its own replies). One at a time; `cancel` stops it within a fifth of a second.
|
||||
- **`ping`** and **`traceroute`** share a raw ICMP socket: a traceroute is echo requests allowed one hop, then two, then three, and the routers' complaints are the list.
|
||||
- **`nslookup`** asks one server itself, over UDP, and so can say which server answered and how long it took, which the system's resolver doesn't; and it can ask a server that isn't the configured one.
|
||||
- **`port`** is a connection attempt that is not waited for: open, refused, or five seconds of nothing.
|
||||
- **`ifconfig`** and **`arp`** read lwIP's own lists, with its lock held.
|
||||
- **Cost:** 12 KB of flash. A 6 KB task while a command runs (2.6 KB of it never used), nothing otherwise.
|
||||
|
||||
### Checks on the device (2026-10-08, with the VPN up and everything routed through it)
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `ifconfig` | `vpn 10.9.0.2/32 mtu 1420, up, default route`; `wifi ... gw ... mtu 1500, up`; the DNS server |
|
||||
| `arp` | The gateway and one other machine |
|
||||
| `ping` of a neighbour, of a name | 4 of 4 in 3 to 4 ms; 3 of 3 in about 50 ms |
|
||||
| `ping 9.9.9.9 2 1392`, then `1393` | Both back; neither back: the tunnel carries 1420 bytes exactly |
|
||||
| `nslookup` | The address, the server and the time; an alias followed; with another server; "there is no nope.invalid" |
|
||||
| `port` | `open, 52 ms`; `refused`; "no answer in 5 s"; "doesn't resolve" |
|
||||
| `traceroute 9.9.9.9` | Nine hops, the tunnel's server first, "arrived" |
|
||||
| A second command while a ping runs | "another one is running: `cancel` stops it" |
|
||||
| `cancel` | "stopped", with the count so far |
|
||||
| In the Shell | Tab completes them; the lines appear there and only there |
|
||||
|
||||
**Not checked:** without the VPN (every check went through the tunnel, or to the local network); a network that drops ICMP; the commands in Safe Mode, where they are not offered.
|
||||
|
||||
**Found on the way:** a refused connection is reported by lwIP as "reset", not "refused"; the first version called it "no route". And the header for the tested half was first given the same name as the service's, which makes a file include itself: the same mistake as an hour before, in the same way.
|
||||
|
||||
### The rest of the list: `tls`, `ntp`, `netstat` (2026-10-08)
|
||||
|
||||
- **`tls <host> [port]`** makes a handshake that checks nothing, so that a bad certificate can be looked at, and then checks it itself: against this device's roots (`ca_roots.h`, the ones the Update Service trusts) and the name asked for. It says who the certificate is for, who signed it, from when to when with the days left, the verdict with its reasons, and the SHA-256 that a Gemini pin is. It runs on a 12 KB task and isn't tried with less than 70 KB free: a handshake peaks at about 52 KB.
|
||||
- **`ntp [server]`** sends one SNTP request and compares the answer with the device's clock, allowing for half the round trip. With no server it asks the first one in Settings.
|
||||
- **`netstat`** reads lwIP's own lists: what listens, labelled where the firmware knows what it is, what is connected, and the UDP ports in use.
|
||||
- Host tests: the NTP packet and the year 2036, the offset in words, a certificate's name (an old string type that mbedTLS prints as hex included), days between dates. 7 tests in `test/test_net_probe` in all.
|
||||
|
||||
| Check on the device | Result |
|
||||
|---|---|
|
||||
| `tls git.twis.la` | 709 ms; for git.twis.la, 67 days left, "this device trusts it", the SHA-256 |
|
||||
| `tls geminiprotocol.net 1965` | "NOT trusted here: not signed by a root this device has": a capsule signs its own |
|
||||
| `tls expired.badssl.com` | "EXPIRED 4197 days ago" |
|
||||
| `tls wrong.host.badssl.com` | "NOT trusted here: not for that name" |
|
||||
| `tls` to a port that isn't TLS | "no handshake ... An invalid SSL record was received" |
|
||||
| `ntp` | The server, its stratum, 50 ms away; "this clock is right, to 0.1 s" |
|
||||
| `netstat` | The update port and the Debug Console listening, the console's own connection, the UDP ports |
|
||||
| Memory during a `tls` | 44.5 KB free at the lowest, from 104 KB |
|
||||
|
||||
**Not checked:** `tls` with IRC connected (it should refuse for lack of memory); `ntp` against a clock that is wrong; `netstat` while sharing.
|
||||
|
||||
## The SSH client (issue #2)
|
||||
|
||||
A terminal on another machine: one session to a shell, from the SSH App.
|
||||
|
||||
### Measured before deciding (2026-10-08)
|
||||
|
||||
`ewpa/LibSSH-ESP32` 5.10.0 (libssh on mbedTLS) was built into a trial firmware and a session opened against OpenSSH in a container, with a password.
|
||||
|
||||
| | Measured in the trial | As built |
|
||||
|---|---|---|
|
||||
| Flash | 120 KB | **292 KB** |
|
||||
| Static RAM | 1.2 KB | |
|
||||
| The session's task stack | 13 KB used | 13.7 KB used of 20 KB |
|
||||
| Free heap with a session open | | 49 KB of 99 KB: it costs about 50 KB, the stack included |
|
||||
| Lowest free heap during a login | | 30 KB |
|
||||
| Key exchange (curve25519, ed25519 host key) | 227 ms | |
|
||||
|
||||
- **The trial undercounted the flash.** It logged in with a password. Signing with a key of the device's own (Q255) links libssh's table of multiples of the Ed25519 base point: `ge25519.c.o` alone is 109 KB. The rest of the difference is the public-key code, the terminal and three fonts. The firmware is at 71% of its slot.
|
||||
- **libssh carries its own curve25519** (`src/external/curve25519_ref.c`) with the names libsodium uses, and libsodium is already here for WireGuard: two definitions don't link. A build script (`scripts/libssh_filter.py`) leaves libssh's copy out, and it uses libsodium's. It has to be a `pre:` script: libraries are built before any `post:` one runs.
|
||||
- A session and a TLS connection don't fit together: IRC holds 40 KB.
|
||||
|
||||
### Decisions (design round 2026-10-08)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q254 | **LibSSH-ESP32 5.10.0**, with its duplicate curve file left out of the build. |
|
||||
| Q255 | **A password, typed each time and never stored**, or **a key the device makes for itself** (Ed25519, no passphrase, kept in the settings store). Its public half is shown, written to `/ssh/id_ed25519.pub` and printed by `ssh status`. Keys made elsewhere aren't imported. |
|
||||
| Q256 | **A terminal good enough for a shell, `less`, `top`, `nano` and `vim`:** cursor movement, erasing, sixteen colours, scroll regions, the alternate screen, the cursor keys' two modes. `TERM=xterm`. No mouse. |
|
||||
| Q257 | **Five text sizes, changed with Ctrl and + or -** (the user's change to the round: the proposal was a setting). 4x6, 5x8, 6x10, 6x13 and 9x15 pixels: from 60 x 20 to 26 x 8 characters. The far end is told the new size; the choice is kept. |
|
||||
| Q258 | **100 lines of scrollback**, as text, with Alt and up or down, as in the Shell. Not on the alternate screen. |
|
||||
| Q259 | **Keys:** Ctrl with a letter; Tab; the backtick key sends Esc, as it is printed; Alt with it types a backtick; Fn with the arrow keys; Shift with those for Page Up and Down; Ctrl+Alt+q disconnects. Fn with backtick is Home, as everywhere. |
|
||||
| Q260 | **The session outlives the App's time in front.** `SSH` in the Status Bar while one is open. |
|
||||
| Q261 | **Not started under 75 KB free**, with the reason in words. |
|
||||
| Q262 | **Up to eight hosts remembered**, the last used first, once a login has succeeded. Forgetting one forgets its server's fingerprint too, unless another remembered host is the same server. |
|
||||
| Q263 | **Trust on first use,** on the SHA-256 fingerprint; sixteen servers remembered. **A changed key is a warning**, with Cancel selected. |
|
||||
| Q264 | **UTF-8 in, the fonts' Latin-1 out:** what they lack is `?`, box-drawing lines are `+ - \|`. |
|
||||
| Q265 | **`ssh user@host` in the Shell opens the App** and connects there. The password is never asked on a console. |
|
||||
| Q266 | **Not built:** port forwarding, SFTP and scp, jump hosts, agent forwarding, keys with a passphrase, more than one session. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`lib/term`** (host-tested, 12 tests in `test/test_terminal`): `Terminal`, the screen a program's output makes, with its history and the replies a program asks for; `encodeKey`, what a key sends; `SshHosts` and `SshKnownHosts`, the two lists kept in the settings store as lines of text.
|
||||
- **`SshService`** (`src/services/ssh_service`): one session on a task of its own (20 KB of stack). The task and the main loop share the terminal, the bytes to send and the state under one lock. Its questions (is this the right server? the password?) are states it waits in until the App answers; the settings store is only written from the main loop. The password and the private key are overwritten after use.
|
||||
- **`SshApp`** (`src/apps/ssh_app`): the hosts, the entry, the session, the key page. It draws the grid a run of same-coloured cells at a time, at most every 60 ms.
|
||||
- **Keys that aren't characters now say what was held with them** (`lib/input/src/key_mapper.cpp`): the arrows, Enter, Del, Tab and Back carry Shift, Ctrl and Alt. The terminal needs it for Page Up and Alt+backtick. It also makes two documented keys work from the real keyboard, which until now only worked from the Debug Console's `key` command: Ctrl with Fn and up or down in a note, and Shift+Tab in Gemini.
|
||||
- **IRC doesn't try to connect with less than 60 KB free** (`IrcService::kNeedFree`): see below.
|
||||
- Settings: `SshHosts`, `SshKnown`, `SshKey`, `SshPublic`, `SshFont`.
|
||||
|
||||
### Checks on the device (2026-10-08, against OpenSSH 9.7 in a container on the same network)
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `ssh tester@host:2222` from the Debug Console | The App opens; the fingerprint shown is the one `ssh-keygen -lf` prints on the server |
|
||||
| Trust it, a password | A shell; `stty size` says 15 48, `$TERM` is xterm |
|
||||
| `ls -la`, `top`, `vim` (insert, Esc, `:wq`) | Drawn right: `top`'s reverse-video header, `vim`'s alternate screen and what was there before coming back; the file is on the server |
|
||||
| Ctrl with + and - | `stty size` says 12 40, then 20 60; `top` redraws for it |
|
||||
| `seq 1 60`, Alt with up | The history, in grey, with how far back in the corner |
|
||||
| Fn+backtick, then the App again | The Launcher with `SSH` in the Status Bar; the session as it was |
|
||||
| `sleep 100`, Ctrl+C; Alt+backtick | Interrupted; `` echo `id -u` `` prints 1000 |
|
||||
| Ctrl+Alt+q; `exit` | "Disconnected"; "The session ended" |
|
||||
| This device's key, its public half in `authorized_keys` | "Accepted publickey" in the server's log; no password asked |
|
||||
| The server's host keys replaced | "THE SERVER'S KEY CHANGED" with the new fingerprint, Cancel selected. Cancel: "Not trusted: not connected". Replace: it connects, and doesn't ask again |
|
||||
| A wrong password | "Wrong password", and asked again; Back gives up |
|
||||
| A port nothing listens on | "Nothing listens there: the connection was refused" |
|
||||
| `ssh nobody`, `ssh a@`, a port of 99999 | Refused, each with its reason |
|
||||
| Forgetting a host | Asked, then gone from the list |
|
||||
| `irc start` with a session open | "not enough memory: close the SSH session, retrying in 5 s", and no attempt: the lowest free heap doesn't move |
|
||||
| Memory | 99 KB free before, 49 KB with a session open, 30 KB at the lowest during a login, 99 KB again after |
|
||||
| Stacks | `ssh` 6.8 KB free of 20 KB; `loopTask` 1.9 KB free, as before |
|
||||
|
||||
**Not checked:** the refusal under 75 KB free (it is one comparison, and wasn't provoked). A server on the internet, or through the VPN. Wi-Fi lost in the middle of a session. Servers other than OpenSSH. `nano`, `less`, `htop`, `tmux`. Keyboard-interactive logins (two-factor prompts). A session left open for hours.
|
||||
|
||||
### What went wrong while building it
|
||||
|
||||
- **IRC, started with a session open, took the free heap down to 236 bytes.** A test script's keys went to the Launcher instead of the terminal and opened the IRC App, which connects when opened. Its TLS handshake found no memory, failed, and tried again with its usual back-off, six times; nothing crashed and it never connected, but 236 bytes is no margin at all. IRC now looks at the free heap before each attempt and says "not enough memory: close the SSH session" instead of trying.
|
||||
- **The build script did nothing as a `post:` script:** the libraries were already built when it ran.
|
||||
- **A failed connection was first shown as an empty terminal** with its reason squeezed on the last line, and a host was remembered before anyone had logged in to it. Both changed: the reason has a page, and a host is remembered once a login succeeds.
|
||||
- **The trust question didn't fit its dialog:** the fingerprint is 50 characters. It is now split over two lines, under one line of words.
|
||||
@@ -1,6 +1,6 @@
|
||||
# R1 — Releases
|
||||
|
||||
**Status:** in progress. CI and signed releases on Gitea (issue #5) are in place since 2026-10-06: every tag from v0.1.0 to v0.10.0 has its release. Updates from Gitea (issue #6) is built and checked on the device, on branch `gitea-updates`, not merged yet. The Issues App (#4) comes after.
|
||||
**Status:** in progress. Shipped: CI and signed releases on Gitea (issue #5), a release for every tag; updates from Gitea (#6, **v0.11.0**); one firmware with the Debug Console in it (#68, **v0.12.0**); CI in about a minute (#74, **v0.13.0**). Not started: the Issues App (#4), automatic installs (#52), release channels (#53), resuming a download (#54).
|
||||
|
||||
**Goal:** a tag is a release, built the same way every time and published where a device can find it.
|
||||
|
||||
@@ -14,7 +14,7 @@ Until now the tests, the builds, the signing and the flashing all happened on on
|
||||
|---|---|
|
||||
| Q151 | A push to `main`: the host tests (with their coverage). A push to another branch: nothing, its pull request is what runs (a branch with an open pull request ran twice, once for each). A pull request: the same and both builds; changes reach `main` through pull requests. A tag `v*`: all of it, then a release. (First: everything on every push, which rebuilt the firmware far more often than anyone looked at it.) |
|
||||
| Q152 | **CI signs.** The signing key is the repository secret `OTA_SIGNING_KEY`; a tag push makes a complete, signed release with no manual step (ADR 0008). |
|
||||
| Q153 | The Debug Build is built in CI with a token of the runner's own, to prove it compiles, and **isn't published**: it would hand everyone its Debug Console token. |
|
||||
| Q153 | *Revised by Q188: there is one firmware.* The Debug Build is built in CI with a token of the runner's own, to prove it compiles, and **isn't published**: it would hand everyone its Debug Console token. |
|
||||
| Q154 | Pull requests from forks don't start a run. |
|
||||
| Q155 | A release carries `roro9stack-<version>.ota` (signed), `-factory.bin` for USB, `.elf.gz` to decode crashes, and `SHA256SUMS`. |
|
||||
| Q156 | Its text is the tag's message, what the files are, and the commits since the tag before. |
|
||||
@@ -64,7 +64,7 @@ The device looks at the project's Gitea for a newer release, says so, and instal
|
||||
| Q168 | A version that failed (rolled back) isn't announced again by the background check until a newer one exists; it can still be installed by hand. |
|
||||
| Q169 | **Older releases:** a list of the last ten, newest first, the running one marked. Installing an older one asks with a stronger warning. |
|
||||
| Q170 | Enter on a release shows its version, date, size and the tag message, with Install. |
|
||||
| Q171 | **Debug Builds** check and show the latest release, but don't install it: it would replace the Debug Build and its console (Q153: Debug Builds aren't published). Their updates come from the PC. |
|
||||
| Q171 | *Revised by Q188: there is no Debug Build, every firmware installs releases.* **Debug Builds** check and show the latest release, but don't install it: it would replace the Debug Build and its console (Q153: Debug Builds aren't published). Their updates come from the PC. |
|
||||
| Q172 | **Memory, as measured:** a TLS connection peaks at about 52 KB of heap, with or without checking the certificate, so it starts with 80 KB free (Q86's 20 KB spare on top), not 55 KB. **A check, list or install someone asked for makes IRC step aside** and come back after; the daily check never does, and with IRC connected it waits. (First: 55 KB and nothing else. With IRC connected a check left 3 KB and a download 836 bytes.) |
|
||||
| Q173 | Left out, each with its issue: installing automatically (#52), a release channel (#53), resuming a download (#54). |
|
||||
| Q174 | Ships as **v0.11.0**. Tested on the device with the real signed releases; a Debug Build command pretends the device runs an older version, so v0.10.0 counts as an update. |
|
||||
@@ -123,3 +123,111 @@ The device looks at the project's Gitea for a newer release, says so, and instal
|
||||
- **A key press during the hold:** Back on the Firmware page while a check is going doesn't cancel it.
|
||||
|
||||
**Two slips during the checks:** a blind sequence of keys on the Firmware page opened the SD card's install dialog (the page keeps its selection between visits); it was cancelled with Back, nothing installed. And my port-polling while waiting for a restart took the Debug Console's only client slot, which made the first install attempt look like a failure.
|
||||
|
||||
## One firmware: the Debug Console in every build (issue #68)
|
||||
|
||||
The issue asked for a token that could be set, so that Debug Builds could be published. The design round ended somewhere simpler: no Debug Builds. The console is in every firmware, off until switched on, with a token that belongs to the device (ADR 0010, which supersedes part of ADR 0004).
|
||||
|
||||
### Decisions (design round 2026-10-06)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q188 | **One firmware,** with the Debug Console and the test commands compiled in. The `cardputer-adv-debug` environment, `RORO_DEBUG`, the `+debug` version and `scripts/debug_flags.py` go; CI builds one firmware. Revises Q153 and Q171. |
|
||||
| Q189 | **Off by default,** and off when the stored setting is missing or invalid. Off, nothing listens and no memory is used: the task and the 4 KB ring exist only while it's on. |
|
||||
| Q190 | **Settings → Debug Console:** the switch, where to connect, the token, "New token", "Type a token". It stays on across restarts and in Safe Mode. **`DBG` in the Status Bar** while it listens, bright with a client connected. |
|
||||
| Q191 | **The device makes the token** the first time the console is switched on: 100 bits as 20 characters of Crockford's base32, shown in groups of four. It can be replaced by one typed by hand, of at least 16 characters. Case and dashes don't count. |
|
||||
| Q192 | **USB serial can set it up:** `debug on`, `debug token <value>`, `debug token new`, over USB only; `debug status` and `debug off` from anywhere. `scripts/flash.sh --debug` gives a device the developer's token. The token is never printed. |
|
||||
| Q193 | **Challenge and answer:** the device sends 16 random bytes, the client their HMAC-SHA256 keyed by the token. Five wrong answers in a row close the console for a minute, with a Notification. **Old clients and old firmwares don't talk to each other:** accepted, this far from v1 and with one user. |
|
||||
| Q194 | `rdbg.py` takes the token from **`-t`/`--token`**, then **`$RORO_DEBUG_TOKEN`**, then `~/.config/roro9stack/debug-token`. |
|
||||
| Q195 | **The test commands are in every build** (`crash abort`, `crash wdt`, `update pretend`, `update damage`, `lora inject`, `sd fill`, `loop spin`, `wifi ip … try`). `update install … force` goes: there's nothing left to protect. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`lib/debug`** (host-tested, 9 tests): the token maker, tidying what a person types, HMAC-SHA256 on the project's own SHA-256 (checked against RFC 4231), the answer to a challenge (the same vector as `rdbg.py`'s), a comparison that takes the same time whatever it's given, and the pause after five wrong answers, right across the clock's wrap.
|
||||
- **Two settings,** `DebugConsole` and `DebugToken`, with the others: validated, and invalid stored values count as missing.
|
||||
- **The console's task and ring come and go with the setting.** `Console::openRing` allocates the ring; switching off closes the socket, frees it and ends the task. `tick` follows the settings once a second, so a switch off and on again takes a second or two.
|
||||
- **Measured against the two builds it replaces:** 1,881,799 bytes of flash and 54,700 of static RAM. The release build was 1,851,387 and 54,612 (so 30 KB more flash and 88 bytes more RAM); the Debug Build was 1,874,887 and 58,780 (4 KB of RAM back: the ring is no longer a static array).
|
||||
- **The listener is asked whether it listens.** The framework's `begin()` returns nothing and fails without a word; the task now checks, says so on the console, and tries again every two seconds.
|
||||
- **`debug off <seconds>`** closes the console and brings it back by itself: the only way to try its closing and reopening from afar, since switching it on is for the device and the cable only.
|
||||
- **`scripts/rdbg.py`** answers the challenge, and fetches a release's ELF from Gitea when a crash names a version that isn't in `.pio/elves/`.
|
||||
|
||||
### Checks
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Host tests | 468 pass (456 before) |
|
||||
| The firmware builds | One environment, 1,881,799 bytes of flash used |
|
||||
| Pushed over Wi-Fi to a device running a Debug Build | Installed, restarted; the update port answers and **port 2323 refuses connections**: off by default |
|
||||
| Switched on at the device (Settings → Debug Console) | A token is made and shown. Its first drawing, in bold at normal size, was misread once: it is now at twice the size, in two lines, and `O`, `I` and `L` are taken for `0` and `1` |
|
||||
| A login with `scripts/rdbg.py` | The challenge is answered; `info`, `screenshot` and the backlog work |
|
||||
| `DBG` in the Status Bar | There while the console is on, bright while a client is connected (screenshots) |
|
||||
| `debug on` and `debug token` over the console | Refused: `over USB serial only` |
|
||||
| Six logins with a wrong token | Five are refused, a second apart; the sixth, and the right token after it, get `locked`. After a minute the right token works again. The console's own log and a Notification say so |
|
||||
| Three `crash abort` in a row | **Safe Mode**, with the console reachable: `info` works, `ls /` says `not available in Safe Mode`. `rdbg.py crash` decodes the backtrace to `runCommand` at the `abort()` line. `reboot` leaves Safe Mode |
|
||||
| An update over Wi-Fi with the console on | The setting and the token survive: the console is back by itself after the restart |
|
||||
| `debug off` over the console | The client is dropped and port 2323 refuses connections; the update port still answers |
|
||||
| `debug off 2`, 10 times in a row, then 15 | It came back each time, a second after the pause. Free heap dips by about 270 bytes for each connection the device closes and **is all back two minutes later** (107.2 KB before, 103.2 right after 15, 107.0 at two minutes): TCP holds a closed connection that long. Not a leak, though it looked like one for an hour |
|
||||
| Memory, console on with a client | 108 KB free (the Debug Build it replaces: 104 KB). In Safe Mode: 178 KB free |
|
||||
|
||||
**One false alarm:** after the tests the console "wouldn't come back on". It was off: the setting had been left off by `debug off`, and the page's "Switch it on?" dialog opens on **Cancel**, so Enter twice leaves it off. Nothing was lost.
|
||||
|
||||
**Not checked:** `scripts/flash.sh --debug` over USB (no device on USB here); "New token" and "Type a token" from the page (the code paths are the ones `debug token` uses, which need the cable); the Toast itself on screen (the console is closed while it shows; its Notification is in the log); free memory with the console off, which only the serial port could say (the static figures above are the evidence); fetching a release's ELF, which needs a crash on a released version.
|
||||
|
||||
## CI that doesn't rebuild the world (issue #74)
|
||||
|
||||
A pull request's run took over seven minutes, a tag's thirteen and a half. The goal: a firmware build under a minute.
|
||||
|
||||
### Where the time went (run 81, a pull request, 2026-10-06)
|
||||
|
||||
| Step | Time |
|
||||
|---|---|
|
||||
| Tools (apt, pip) | 15 s |
|
||||
| Check out | 2 s |
|
||||
| Host tests and coverage | 55 s |
|
||||
| **The firmware** | **358 s** |
|
||||
| of which: CMake configuring ESP-IDF | 87 s |
|
||||
| of which: compiling ESP-IDF's libraries | 171 s |
|
||||
| of which: our own build (the Arduino core, the libraries, `src/`) | 91 s |
|
||||
|
||||
A tag's run did the firmware step, then built the same commit again for the release: twice 356 s.
|
||||
|
||||
### Why the framework was rebuilt every time
|
||||
|
||||
The framework is rebuilt with our SDK settings (ADR 0006), and the rebuilt libraries stay in the toolchain volume. But the platform decides whether they match by reading **`sdkconfig.defaults` in the project folder**, whose first line carries a hash of the settings. That file is generated, and not in git. A fresh checkout has none, so the platform concluded "different settings", **reinstalled the framework and rebuilt it**: 260 seconds, at every run, to arrive at the libraries that were already there. On a developer's machine the file is simply still there from the last build, which is why nobody saw it.
|
||||
|
||||
### What changed
|
||||
|
||||
| Change | Effect |
|
||||
|---|---|
|
||||
| **The file is kept in the volume, inside the libraries it describes** (`framework-arduinoespressif32-libs/.roro-sdkconfig.defaults`), copied into the checkout before a build and back after one that passed. The platform still checks its hash against `platformio.ini`: changed settings rebuild, as they must. Kept there and not beside them, it disappears when the libraries are reinstalled, so it can't describe libraries that are gone | 358 s to 92 s |
|
||||
| **The version is no longer a `-D` on every compiler command line.** `scripts/version.py` writes `lib/version/src/version_generated.h` (not in git, written only when it changes), read by one file. Before, every commit recompiled everything, on a developer's machine too, and no cache could have helped | A rebuild with nothing changed: 77 s to 13 s, locally |
|
||||
| **PlatformIO's build cache** (`PLATFORMIO_BUILD_CACHE_DIR`, SCons's CacheDir) in the volume, for the firmware of pull requests: objects by the signature of their sources and command line | 92 s to 26 s, with a new version and one changed file |
|
||||
| **ccache for the host tests.** They are built with coverage counters, and the build cache would return objects without their `.gcno` files; ccache keeps both | 49 s to 33 s. What's left is PlatformIO starting 51 test programs |
|
||||
| **A tag builds its firmware once**, in the release step | minus 6 minutes |
|
||||
| PlatformIO and gcovr in a virtual environment in the volume | a few seconds |
|
||||
|
||||
**A release compiles its own sources from nothing:** it reuses the rebuilt framework (the platform checks the hash) but not the build cache, so no published file contains an object that came from another commit's build.
|
||||
|
||||
### Measured (a development machine, fresh copies of the tree, the same volume)
|
||||
|
||||
| | Before | After |
|
||||
|---|---|---|
|
||||
| The firmware, fresh checkout, nothing cached for it | 358 s | 82 s (it fills the cache) |
|
||||
| The firmware, fresh checkout, a new version and one file changed | 358 s | **27 s** |
|
||||
| Host tests and coverage | 49 s | 33 s |
|
||||
| Rebuilding locally with nothing changed | 77 s | 13 s |
|
||||
|
||||
### Measured on the runner (pull request #76, 2026-10-07)
|
||||
|
||||
| Run | Tools | Tests and coverage | The firmware | The whole job |
|
||||
|---|---|---|---|---|
|
||||
| Before (run 81) | 15 s | 55 s | 358 s | 434 s |
|
||||
| The first with the new workflow: no mark yet, the framework is rebuilt once more and the caches fill | 16 s | 59 s | 354 s | 431 s |
|
||||
| The next commit (only the workflow changed) | 10 s | 36 s | **51 s** | **100 s** |
|
||||
| The same commit again | 10 s | 36 s | **18 s** | **66 s** |
|
||||
|
||||
In the 51-second run, 245 objects came from the cache and 43 were compiled: `version.cpp`, as expected, and all 42 files of `src/`, which had not changed. In the run after it, all 290 came from the cache. So the objects of `src/` made by the run that rebuilt the framework were not reusable by a normal run, and those of a normal run are: the two-pass build that rebuilds the framework compiles `src/` with something different on its command line. It costs one 51-second run after each framework rebuild, which is rare; I did not look for what differs.
|
||||
|
||||
The firmware step with everything cached is 18 seconds: the libraries are downloaded and unpacked (4 s), the dependency scan (5 s), fetching 290 objects, the link and the image (the last 11 s). A pull request that changes a few files should land between that and 51 seconds.
|
||||
|
||||
**What it costs:** the build cache grows by about 40 MB a run (each linked firmware is kept) and is started again past 3 GB; ccache is held to 1 GB.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# S1 — System basics
|
||||
|
||||
**Status:** the three planned items are done: the SD driver fix in v0.6.1 (issue #21, ADR 0007), fixed IPv4 settings in v0.7.0 (issue #7), the System App in v0.8.0 (issue #11). v0.8.1 adds the resting main loop (issue #40) and the GNSS pause for the radio's noise (issue #20, still open for the 11 dB that remain). Still open in the milestone: #39, following the SD driver upstream.
|
||||
**Status:** the three planned items are done: the SD driver fix in v0.6.1 (issue #21, ADR 0007), fixed IPv4 settings in v0.7.0 (issue #7), the System App in v0.8.0 (issue #11). v0.8.1 adds the resting main loop (issue #40) and the GNSS pause for the radio's noise (issue #20, still open for the 11 dB that remain). Still open in the milestone: #39, following the SD driver upstream. The **Shell** (issue #67) shipped as **v0.14.0**; the Shell in Safe Mode (#77) is not started.
|
||||
|
||||
**Goal:** the device works on any network, the card can be trusted, and you can see what the system is doing. A side milestone, like G1.
|
||||
|
||||
@@ -136,3 +136,69 @@ M3 found the LoRa radio's noise floor about 15 dB above what the chip hears alon
|
||||
**Settings > Pause GNSS for LoRa**, off by default: while the LoRa radio listens or sweeps, the GNSS receiver waits in standby, and wakes when the radio goes back to sleep (a Fix again after about 7 s here). Never during a Track. The GNSS App says "GNSS is paused" meanwhile. `gnss quiet on|off` on the console.
|
||||
|
||||
It's off by default because GNSS on by default was decided in M2 (Q58), and from M4 the radio listens all the time: then "pause while listening" means GNSS mostly off, which is a decision about position, the clock and Tracks, for M4's design round (issue #23).
|
||||
|
||||
## The Shell (issue #67)
|
||||
|
||||
The console's commands could only be typed on a PC: over USB, or over Wi-Fi with the Debug Console. A device in a bag, or on a network that is down, could not be asked anything. The Shell is an App that runs the same commands on the device's own screen and keyboard.
|
||||
|
||||
### Decisions (design round 2026-10-07)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q204 | **An App, "Shell", in the Launcher**, in every firmware: its commands already work over USB for anyone holding the device. |
|
||||
| Q205 | **Trusted like USB serial**, not like the network: `debug on` and `debug token` work from it, as they do in Settings. The token is never shown. |
|
||||
| Q206 | *Revised the same day, after trying it.* **It shows the replies to its own commands, and only those.** The console knows who each line is printed for (`Console::Origin`): a command run from the Shell prints as the Shell's, and so does what answers it later from another task, which notes who asked and takes it back when it prints (`ls`, `tasks`, `du`, `cp`, `update check`, `sd list`, `screenshot`, `gemini get`). What USB or the Debug Console asked for, and the system's own lines, are not the Shell's. **Ctrl+b shows everything** instead. The first version kept whatever was printed in the ten seconds after a command, which was a guess, and a noisy one. |
|
||||
| Q207 | **Nothing while it's closed.** Open, a 4 KB ring of the console's and up to 4 KB of lines; both go when the App is left, with the list of commands for Tab. |
|
||||
| Q208 | Enter runs the line; Fn with up and down recalls the last 16; Alt with up and down scrolls back. **Tab completes** the command, **every word of it** *(the first only, at first)*: `lora st` gives `lora status`, `gnss track ` lists `start stop`, `key ` its eleven names. The words come from the firmware's `help` text, read as it is written, so a new command completes without a table to keep. *(Added the same day)* **past the command, a path on the SD card**: a folder keeps its slash to go on from, a file completed whole gets a space, several candidates are listed. Whatever the case typed, the name's own is taken, since the card doesn't tell them apart. After a file command the first slash is understood (`cat no` is `/no`). A name with a space in it isn't completed. |
|
||||
| Q209 | *Revised the same day.* **`rm` is Unix's, with a question.** A folder needs `-r`, here and over the consoles (where `rm <folder>` used to remove it with what was in it). In the Shell, a file, or a folder with something in it, is asked about unless `-f` (`-rf`, `-r -f`); an empty folder with `-r` goes without a word; what `rm` would refuse anyway, it refuses itself. Over the consoles nothing is asked: scripts delete as before. **`screenshot [seconds]`** saves the screen as a PNG in `/screenshots`, now or after a pause, since from the Shell "now" is the Shell. `get`, `put`, `coredump get` and `reset` answer `Debug Console only`. *(Added the same day)* **`*` and `?` in a name**, for `ls`, `du`, `rm`, `cp` and `mv`, here and over the consoles: `rm /notes/*.txt`, `cp /gnss/2026-10-0?.gpx /backup`. In the last part of the path only, any case, as the card has it. It is the same command once for each name matched, in the name's order; `cp` and `mv` then need a folder that exists to put them in. Over 64 matches is refused whole, as is none. In the Shell `rm` with a pattern asks **once**, with the count. |
|
||||
| Q210 | Commands are echoed as `> command` into the console, so a session reads the same from afar. **A token being set is not echoed.** |
|
||||
| Q211 | **Not in Safe Mode**, which starts no Apps: issue #77. |
|
||||
| Q212 | Its keys are a table in `app_keys.h`, so the help panel and the website have them; a page in the user guide. |
|
||||
| Q213 | *Added the same day.* **An App's name with a capital opens it:** `Irc`, `Wifi`, `Gnss`, `Gemini`, `Lora`, `Storage`, `Notes`, `Shell`, `System`, `Settings` (the App's id, up to its first dash). From the Shell, without going back to the Launcher, and from the consoles too. The capital says "an App": every command of the firmware is in small letters. Tab completes them. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`ShellApp`** (`src/apps/shell_app.cpp`), with its model host-tested in `lib/apps_model/src/shell_log.h`: lines arriving in pieces, the 4 KB limit, the words of every command out of the `help` text (Apps' names included), and Tab.
|
||||
- **Who a line is for:** `Console::As` marks the calling task as printing for the Shell while it lives, and `Console::origin()` lets a command that answers later carry that to wherever it prints. The console has a second ring for the Shell, which gets the lines marked so, or everything.
|
||||
- **The Shell hands its lines to the main loop**, which runs them like the consoles' commands. See below for why.
|
||||
- **`info` says which App is in front** (`app: Shell`): so that a hand driving the device from afar can look before it types.
|
||||
- **`screenshot`** writes the PNG a row at a time with no buffer: 8-bit indexed colour, the 256 colours of RGB332 as the palette, the pixels in one stored deflate block (`lib/files/src/png_rgb332.h`, 4 tests). 33,383 bytes for the 240 x 135 screen. A Toast says so once it is on the card, so the Toast is never in the picture.
|
||||
- **The `help` text lost its shorthand** (`update check | list | status` is written out), since Tab reads its words from there: `|` between two commands, `on|off` between two words, three spaces before the description. The Shell's own words (`clear`, `quit`, `key`'s names) are added in the same notation.
|
||||
- **Tab on a path** reads the folder on the storage task while the main loop waits, so it is bounded: 400 entries looked at, 24 candidates given back, and `...` after the list when there were more.
|
||||
- **A pattern** is matched in `lib/files/src/file_names.h` (`globMatch`, host-tested), and the names it matches are lined up as so many commands, which the main loop runs one after the other as each finishes. `cancel` empties the line-up. 2000 entries looked at, 64 matches at most.
|
||||
- **Cost:** 21 KB of flash, 96 bytes of static RAM. Open: 7 KB of heap (107.6 KB free before, 100.7 with it open, 106.1 after leaving).
|
||||
|
||||
### What went wrong while building it
|
||||
|
||||
**The device crashed, and the test sent a message to an IRC channel.** The first version ran a command from inside the key handler. Driven from the Debug Console, that is: the main loop, a remote command, `key select`, the App manager, the Shell, `runCommand` a second time, the file command, and `printf` under all of it. The main loop has under 2 KB of stack to spare; `rm` on a folder went past it. The crash report decoded to exactly that chain.
|
||||
|
||||
The device restarted into the Launcher, and the test script, which did not look, went on typing. Its next Enter opened IRC, which connected, and a few lines later it typed "No" into a channel and pressed Enter. One word, sent to real people, that can't be taken back.
|
||||
|
||||
Two changes came of it. The Shell now **queues** its line and the main loop runs it, at the same stack depth as a console's command. And `info` reports the App in front, which the test script now checks before every line it types.
|
||||
|
||||
### Checks on the device (2026-10-07, driven over the Debug Console with `key`)
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Open it from the Launcher, type `ls /`, Enter | The folders are listed |
|
||||
| Tab on `in` | `info install` is shown and the line stays; on `u` it becomes `update `; on `l`, `log ls lora loop` |
|
||||
| Up | The line before comes back |
|
||||
| `screenshot` | `/screenshots/20261007-104357.png`, 33,383 bytes. Fetched and decoded on the PC: 240 x 135, indexed, every chunk's CRC right, the pixels the Shell's screen |
|
||||
| Output | With a Debug Console client connecting and disconnecting for every key, the Shell shows the commands and their replies and nothing else: `info`, `ls /` (answered by the storage task), `tasks` (answered a second later) |
|
||||
| `rm` on a folder, without `-r` | Refused, the folder stays |
|
||||
| `rm -r` on an empty folder | Removed, no question |
|
||||
| `rm -r` on a folder with files | Asks; Cancel leaves it. `rm -rf` removes it without asking |
|
||||
| `rm` on a file | Asks; Delete removes it |
|
||||
| Tab on a path | `ls /no` becomes `ls /notes/`, and again, with one note in it, the note's whole name and a space. `ls /g` lists `gemini/ gnss/`. `cat no` becomes `cat /notes/`. `rm -r /CAP` becomes `rm -r /captures/`. `ls /zz` stays as it is |
|
||||
| Tab past the first word | `lora st` becomes `lora status `; `gnss tr` becomes `gnss track ` and Tab again lists `start stop`; `key ` lists its eleven names; `upd` becomes `update ` and lists `check list status install` |
|
||||
| `ls` with a pattern | `ls /gt/*.txt` the five files, `ls /gt/*2*` the one, `ls /gt3/*6?.txt` ten of seventy. `ls /g*/x`: refused, a pattern goes in the last part |
|
||||
| `cp /gt/file-000?.txt /gt2`, `du /gt2/*1*` | Five copies; one size |
|
||||
| `rm /gt2/*` in the Shell | "The 5 that match /gt2/\*": Cancel leaves the five. `rm /gt3/*6?.txt`, Delete: ten gone, sixty left. `rm -f /gt2/*`: gone without a question |
|
||||
| `rm /gt/*` where one match is a folder | The files go, the folder stays (no `-r`) |
|
||||
| No match, and seventy | `rm: error nothing matches`; `rm: error more than 64 match: a narrower pattern, please`, and all seventy still there |
|
||||
| `No`, Tab, Enter | Completes to `Notes ` and opens Notes. `Shell` from the Debug Console opens the Shell |
|
||||
| `screenshot 4`, then Home | The picture, taken four seconds later, is of the Launcher: the pause works, and the Toast isn't in it |
|
||||
| `quit` | Back to the Launcher, and the memory comes back |
|
||||
| The help panel in the Shell | Its keys, then the ones that work everywhere |
|
||||
|
||||
**Not checked:** Ctrl+b and the Alt scroll, which `key` can't press (no modifiers): the filter itself is host-tested, the key that flips it is not; the real keyboard altogether; the Toast after a screenshot, which was published but not looked at; and `mv` with a pattern and `cancel` in the middle of a line-up, which share their code with `cp` and `rm` but were not run. The main loop's lowest free stack after all of it: 1.8 KB, where it was.
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
# U1 — Look and feel
|
||||
|
||||
**Status:** in progress. Shipped: the help key (issue #69) and the key tables the website shares with it (#72), both in **v0.13.0**; the screenshot key (#83, **v0.17.0**). Not started: screen recording (#17), a Launcher of tiles (#9), themes (#10).
|
||||
|
||||
**Goal:** the interface is consistent and uncrowded: the same thing is done the same way on every screen, and the 135 pixels of height go to content.
|
||||
|
||||
## The help key (issue #69)
|
||||
|
||||
Every screen used to say something about its keys, differently: a footer of abbreviations in one place (`c x v:paste r:name d:del n:new i:info s:sort`), a line under a text field in another (`Enter: save \`: cancel`), `Tab: sky` in a corner, and nothing at all in several. About 30 such strings, each costing a line of a small screen, and none of them complete.
|
||||
|
||||
### Decisions (design round 2026-10-07)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q196 | **Fn+h, on every screen,** text fields included (Fn is held, so nothing is typed). **`?` too, outside Text Entry.** |
|
||||
| Q197 | It opens **a panel over the content area**, titled with where you are: the screen's own keys, then an "Everywhere" group (Back, Home, the arrows, the help key). The arrows scroll it; any other key closes it and is not passed on. |
|
||||
| Q198 | **Each App answers "what are your keys right now?"** for the state it is in; pages, viewers, dialogs and text fields answer for themselves, with shared lists for dialogs, lists and text entry. The lists are constants; the panel's rows exist only while it is open. |
|
||||
| Q199 | **Every hint that names a key goes,** text fields included. What stays is state: `REC 12 points`, `LOG 42`, `sort:signal`, `typing`/`saved`, what is waiting to be pasted, the Sweep's floor. Messages were reworded where they named a key ("v pastes a copy of…" is "Copied …: paste it where you like"). |
|
||||
| Q200 | **The first-start Setup keeps its hints,** and is the one place that does: someone in their first minute doesn't know the help key yet. It tells them about it on its first and last screens. |
|
||||
| Q201 | **Loud everywhere else:** the user guide opens with it, the FAQ has it first, and a device set up before this firmware gets one Toast, once: "Fn+h: the keys of any screen". |
|
||||
| Q202 | `key help` over the consoles. Generating the website's key tables from the same lists is a follow-up, not this issue. |
|
||||
| Q203 | The key and the panel first, host-tested; then one App at a time, declaring its keys and losing its hints in the same step; then every screen looked at on the device. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`Key::Help`** from the key mapper: Fn+h in both modes, `?` only outside Text Entry (`lib/input`, 3 tests).
|
||||
- **`App::help()` and `App::helpTitle()`** (`lib/core/src/app.h`), `KeyHelp` rows and `HelpModel` (`key_help.h`). The **App manager** opens the panel, appends the "Everywhere" group, and while it is open takes every key: nothing reaches the App, Home included. It closes when the App changes (5 tests).
|
||||
- **Every App declares its keys by state:** the Launcher, IRC (chat, settings, a field), Wi-Fi Tools (4 views), GNSS, Gemini (page, saved page, address, answer, dialogs), the LoRa Scanner (4 views), Storage (browse, details, a name, the viewer's 6 modes, the editor, Maintenance, busy), Notes (list, editor, a file name), System (5 views), Settings (menu, text, choice, and the Wi-Fi, Firmware and Debug Console pages with their own states), Setup and the widget demo.
|
||||
- **The hints are gone** from all of them. The footers that remain say state only.
|
||||
- **It costs** 8.5 KB of flash and 40 bytes of static RAM.
|
||||
|
||||
### Checks
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Host tests | 476 pass (468 before) |
|
||||
| On the device, `key help` and a screenshot | The Launcher and all nine Apps, and these states: Storage scrolled, System's tasks, a Settings text field, Wi-Fi Tools' networks. The panel is titled with the scope, lists the right keys, scrolls, and closes on Tab |
|
||||
| The one-time Toast | `notification: Fn+h: the keys of any screen` on the first start after the update |
|
||||
|
||||
### One source for the device and the website (issue #72)
|
||||
|
||||
The lists first lived in each App's `help()`, as code. They are now **data, in one file**: `lib/core/src/app_keys.h`, 52 constant tables, each under a comment `// id: Title`. An App's `help()` picks the table of the state it is in. `site/tools/gen_dev_docs.py` reads the same file and writes `site/data/keys.toml`; the `keys` shortcode shows a screen's tables on its guide page, and `/guide/keys/` shows all of them. The Site job fails when the data file is out of date, or when a page asks for a table that doesn't exist, and it now runs when `app_keys.h` changes. A key added to an App shows up on the website without anyone editing a page.
|
||||
|
||||
Three rows lost their second wording on the way, since a table is constant: GNSS's `Tab` and `r`, and the Scanner's `c`, now say both things they do ("record a Track, or stop it") instead of the one that applies. The guide pages keep their written tables too, where they say more than a key list can; those can still drift, and the generated ones under them are the reference.
|
||||
|
||||
**Not checked:** the real Fn+h and `?` on the keyboard (the mapper is host-tested; the device was driven with `key help`); the Setup screens, which only a device that was never set up shows, so their new text has not been seen on a screen; and the states that need something to happen first (a dialog, a copy in progress, a Gemini prompt, a packet's details): their lists were read against the key handling, not looked at.
|
||||
|
||||
## The screenshot key (issue #83)
|
||||
|
||||
A screenshot could only be taken by typing `screenshot` in the Shell, where "now" is a picture of the Shell.
|
||||
|
||||
- **Fn+p, on every screen**, text fields included, saves the screen as it is (a dialog, the help panel or a Toast if one is showing) as a PNG in `/screenshots`, the way the Shell's command does. A Toast says so once the file is written, so it is never in the picture.
|
||||
- **It never reaches an App.** `Key::Screenshot` comes out of the key mapper and is handled before the App manager, so the help panel stays open and a dialog keeps its selection.
|
||||
- **Not on Settings > Debug Console:** that page shows the token, and a picture of it is a copy of the token in a file. `App::showsSecret()` says so, and the key answers with a Toast instead.
|
||||
- **No card:** a Toast says so.
|
||||
- No setting to switch it off: Fn with a letter isn't pressed by accident.
|
||||
- It is in the "Everywhere" group of the help panel, and so in the website's key tables. `key shot` presses it over the consoles.
|
||||
|
||||
**Checked on the device** (2026-10-07, with `key shot`): in the Launcher, a 33,383-byte PNG appears in `/screenshots` and is the Launcher; with the help panel open, the picture is of the panel (which now lists Fn p) and the panel stays open; on Settings > Debug Console, no file is written; back on the Settings list, one is. The test pictures were removed.
|
||||
|
||||
**Not checked:** the real Fn+p on the keyboard (the mapper is host-tested); the two Toasts that refuse, which weren't looked at (one of them is on the page that mustn't be photographed); a device with no card.
|
||||
@@ -1,6 +1,6 @@
|
||||
# W1: Website
|
||||
|
||||
**Status:** phases 1 to 3 (home, Install and Downloads; the user guide; how-tos and the FAQ) and the devlog are live at roro9stack.net; phase 4 (the developer docs) is in a pull request. Issue #12.
|
||||
**Status:** live at roro9stack.net: the home, Install and Downloads pages, the user guide, the how-tos and the FAQ, the developer docs and the devlog (issue #12), published by CI since issue #79, with a search since issue #60. Not started: a Gemini mirror (#57), a French translation (#58), the docs of each version (#59).
|
||||
|
||||
**Goal:** a public home for the project at **roro9stack.net**, separate from the blog (stories) and from Gitea (developers): what it is, how to install it, how to use each App, and the docs.
|
||||
|
||||
@@ -21,7 +21,7 @@ The home page was designed on a canvas in a Claude chat (a dark and a light them
|
||||
| Q175 | The site lives in this repository, in `site/`, so the documentation is built from `docs/`, `CONTEXT.md` and the README instead of being copied. |
|
||||
| Q176 | **Zola,** like the blog. The design becomes a template, its tokens CSS custom properties. Dark and light follow the visitor's setting, with a visible switch. No JavaScript except the flasher's. |
|
||||
| Q177 | Domain: **roro9stack.net.** The blog stays at experiments.twis.la. |
|
||||
| Q178 | **Publishing is the blog's way:** the web server pulls `main` and runs `zola build`; that part is the maintainer's. Changes reach `main` through pull requests as everywhere. **CI is split:** a dedicated `site` job builds the site (`zola build`) when `site/`, `docs/`, `README.md` or `CONTEXT.md` change, and the firmware tests and builds skip a change that touches nothing else. A change that touches both runs both. |
|
||||
| Q178 | **Publishing is the blog's way:** the web server pulls `main` and runs `zola build`; that part is the maintainer's. *(Since issue #79, CI asks the server to do it: see "Published by CI" below.)* Changes reach `main` through pull requests as everywhere. **CI is split:** a dedicated `site` job builds the site (`zola build`) when `site/`, `docs/`, `README.md` or `CONTEXT.md` change, and the firmware tests and builds skip a change that touches nothing else. A change that touches both runs both. |
|
||||
| Q179 | Phases, each its own pull request: **1.** the CI split, the home page, an Install page with the browser flasher, downloads and the changelog. **2.** a user guide page per App. **3.** how-tos and the FAQ. **4.** developer docs generated from the repository. |
|
||||
| Q180 | **A browser flasher** (ESP Web Tools), **without copying the firmware.** Caddy, in front of Gitea, adds `Access-Control-Allow-Origin: https://roro9stack.net` (and `Vary: Origin`) to GET and HEAD on `/twisla/roro9stack/releases/download/*` and `/api/v1/repos/twisla/roro9stack/releases*`: both are public already. The Install page asks the API for the latest release in the browser, finds the asset ending `-factory.bin`, and gives ESP Web Tools a manifest built on the spot, so it offers a new release as soon as it exists, with no rebuild. The library is **vendored** into `site/static/` (Apache-2.0), not loaded from a CDN. The file's SHA-256 is shown on the page. Chrome or Edge on a desktop only; other browsers, and visitors without JavaScript, get the `esptool` steps on the same page. |
|
||||
| Q181 | Docs for the latest version only. The changelog is the Gitea releases, read at build time. |
|
||||
@@ -125,3 +125,60 @@ Not one of the planned phases: the blog's seven roro9stack posts, imported into
|
||||
- **Left out on purpose:** the M0 and M1 milestone documents and `CONTEXT.md` (the glossary) describe Wi-Fi monitoring, which the site does not publish. They stay in the repository.
|
||||
- **The Debug Console pages were written against the source and the live console:** the protocol (the token line, the banner, the 4 KB backlog, one client, 8 queued commands, 240-byte lines, `denied` after a second) and the replies shown were checked on a Debug Build, v0.11.0-3, over Wi-Fi. Not run: `crash abort`, `crash wdt` and Safe Mode, which are described from ADR 0005 and the code.
|
||||
- **Found while writing it:** the README's table lacked the `gnss` commands (rows added); piping commands into `rdbg.py` returns before the replies unless the input stays open (documented, not changed); `update install` on a Debug Build needs `force` (documented).
|
||||
|
||||
## Published by CI (issue #79, design round 2026-10-07)
|
||||
|
||||
Q178 left publishing to the maintainer: a merge, then a command typed on the web server, each time.
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q214 | **A plain ed25519 key with a forced command**, not a certificate: one line in the web server user's `authorized_keys`, `restrict,command="/full/path/to/the/refresh"`. `restrict` takes away the terminal and every forwarding. A certificate could carry the same and an expiry date, at the price of a CA to keep and a key to sign again each time: too much for one key and one command. |
|
||||
| Q215 | **The CI sends no command.** The server runs the forced one whatever is asked for, so there is nothing to keep secret about it and nothing a leaked key could choose. The full path is written once, on the server (a command over SSH doesn't get the user's login `PATH`). |
|
||||
| Q216 | Four secrets: `SITE_DEPLOY_KEY`, `SITE_DEPLOY_HOST` (or `host:port`), `SITE_DEPLOY_USER`, and **`SITE_DEPLOY_KNOWN_HOSTS`**, the server's host key: the job connects to that server or to nothing. None of them is in the repository, which is public. |
|
||||
| Q217 | `from="<the runner's address>"` on the same line: the key works from the runner only. |
|
||||
| Q218 | **The last step of the Site workflow**, after the build and the checks, on a push to `main` only. A pull request never reaches it, and the secrets are given to that step alone. |
|
||||
| Q219 | **After a release too.** The Install page asks Gitea for the latest release when it is opened, but the home page and Downloads read it when the site is built: so the release workflow refreshes the site once the release is published. |
|
||||
| Q220 | A refresh that fails makes the run red, with what the server's script printed: it has to exit with an error when it fails. |
|
||||
| Q221 | Two refreshes at once are the server script's to refuse or queue (`flock`). |
|
||||
| Q222 | The key is a file only while the step runs, in the job's container, as the signing key is. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`scripts/site_refresh.sh`** is what both workflows run: it writes the key and the host key to a temporary folder, connects with no configuration but its own line (`-F none`, strict host key checking, that one key, no command), and removes them. With none of the four secrets it does nothing and says so (a fork, or a repository without them); with only some it fails.
|
||||
- **`scripts/site_deploy_keygen.sh`** makes the key pair once, in `~/.config/roro9stack/`, and prints the `authorized_keys` line and what goes in each secret. It never prints the private key.
|
||||
- **The server's script** should start like this, for Q220 and Q221:
|
||||
|
||||
```sh
|
||||
#!/bin/sh
|
||||
set -e
|
||||
exec 9>/tmp/rororefresh.lock
|
||||
flock -w 120 9
|
||||
```
|
||||
|
||||
### Checks (2026-10-07, against an SSH server in a throwaway container)
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| The refresh | The forced command runs as the server's user; the script ends with `site refresh: done` |
|
||||
| The same key, asking for `id; cat /etc/passwd` | The refresh runs instead; what was asked for is only handed to it as text |
|
||||
| A terminal | Refused: `PTY allocation request failed` |
|
||||
| `scp` with the key | Nothing is copied |
|
||||
| Another host key in the secret | `Host key verification failed`, the run fails, nothing is sent |
|
||||
| The server's script exits with an error | So does the step |
|
||||
| No secrets at all; only one of the four | Does nothing and says so; fails and says which are needed |
|
||||
|
||||
**Not checked:** the real web server and the runner, which wait for the key to be installed: whether the runner reaches the server's SSH port is the first thing the first run will tell. Port forwarding, which `restrict` switches off, was not tried. `from=` was not tried either.
|
||||
|
||||
## Search (issue #60)
|
||||
|
||||
A search over the documentation: the user guide, the how-tos, the questions and answers, and the developer docs. Not the devlog.
|
||||
|
||||
- **The index is the search page itself** (`/search/`, `templates/search.html`): one list item for each page and for each `##` heading of it, with that part's text, written by Zola from the pages' own content when the site is built. Nothing is fetched and nothing typed leaves the browser, so the Content-Security-Policy needs nothing new, and the web server still only runs `zola build`.
|
||||
- **Without JavaScript** the page is a list of every page and heading of the documentation, each a link.
|
||||
- **With it**, `js/search.js` filters and ranks the items as you type: every word has to be in the part; a word in a heading counts for most, the words side by side for more than scattered, and the user guide, the how-tos and the FAQ come before the developer docs, the milestones last. A result links to its heading, with the text around the match.
|
||||
- **The content pages get no script for it:** the navigation has a link, and the index pages of the guide, the how-tos and the developer docs have a box that is a plain form to `/search/?q=`.
|
||||
- **Size:** about 245 items, about 310 KB of HTML, under 100 KB compressed, loaded only by who searches.
|
||||
|
||||
**Checked** in Chromium with the production Content-Security-Policy on every response, no violation: "probation" (the guide's "Probation and Rollback" first), "safe mode", "rm -r", "how big can a note" (the FAQ's question first), a word that isn't there; typing, following a result to its heading, the box on the guide's index, 390 px wide with no sideways scroll, and JavaScript off. `check_site.py` follows every link of the page, so an index entry can't point at a heading that doesn't exist.
|
||||
|
||||
**Not checked:** other browsers, and a screen reader.
|
||||
|
||||
@@ -24,7 +24,8 @@ const RowDef kRows[] = {
|
||||
{Row::Coordinates, Kind::Toggle, "Coordinates"},
|
||||
{Row::ProbeMacs, Kind::Toggle, "Probe MACs"}, {Row::Wifi, Kind::Page, "Wi-Fi"},
|
||||
{Row::CheckUpdates, Kind::Toggle, "Check for updates"}, {Row::Firmware, Kind::Page, "Firmware"},
|
||||
{Row::About, Kind::Page, "About"},
|
||||
{Row::Vpn, Kind::Page, "VPN"},
|
||||
{Row::DebugConsole, Kind::Page, "Debug Console"}, {Row::About, Kind::Page, "About"},
|
||||
};
|
||||
|
||||
const int kDimSeconds[] = {10, 15, 30, 60, 120, 300};
|
||||
@@ -86,6 +87,8 @@ std::string SettingsMenu::value(int i) const {
|
||||
case Row::Coordinates: return settings_.getBool(Setting::CoordinatesDms) ? "Deg min sec" : "Decimal";
|
||||
case Row::ProbeMacs: return settings_.getBool(Setting::ProbeMacRaw) ? "Raw" : "Pseudonymised";
|
||||
case Row::Wifi: return settings_.getBool(Setting::WifiEnabled) ? "On" : "Off";
|
||||
case Row::Vpn: return settings_.getString(Setting::VpnConfig).empty() ? "Not set" : settings_.getBool(Setting::VpnAuto) ? "With Wi-Fi" : "By hand";
|
||||
case Row::DebugConsole: return settings_.getBool(Setting::DebugConsole) ? "On" : "Off";
|
||||
default: return "";
|
||||
}
|
||||
}
|
||||
|
||||
@@ -11,7 +11,7 @@ namespace roro {
|
||||
// values, choice lists and validation messages. Rendering and navigation live in the App.
|
||||
class SettingsMenu {
|
||||
public:
|
||||
enum class Row { LongName, ShortName, Region, Timezone, Brightness, DimTimeout, OffTimeout, Sound, Gnss, GnssQuiet, Coordinates, ProbeMacs, Wifi, CheckUpdates, Firmware, About };
|
||||
enum class Row { LongName, ShortName, Region, Timezone, Brightness, DimTimeout, OffTimeout, Sound, Gnss, GnssQuiet, Coordinates, ProbeMacs, Wifi, CheckUpdates, Firmware, Vpn, DebugConsole, About };
|
||||
enum class Kind { Text, Choice, Toggle, Slider, Page };
|
||||
|
||||
explicit SettingsMenu(Settings& settings) : settings_(settings) {}
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
#include "shell_log.h"
|
||||
|
||||
#include <algorithm>
|
||||
|
||||
namespace roro {
|
||||
|
||||
void ShellLog::push(const std::string& line) {
|
||||
lines_.push_back(line);
|
||||
bytes_ += line.size() + 1;
|
||||
while (bytes_ > kMaxBytes && lines_.size() > 1) {
|
||||
bytes_ -= lines_.front().size() + 1;
|
||||
lines_.pop_front();
|
||||
}
|
||||
revision_++;
|
||||
}
|
||||
|
||||
void ShellLog::add(const std::string& line) { push(line); }
|
||||
|
||||
void ShellLog::feed(const char* data, size_t len) {
|
||||
for (size_t i = 0; i < len; i++) {
|
||||
char c = data[i];
|
||||
if (c == '\r') continue;
|
||||
if (c != '\n') {
|
||||
if (partial_.size() < 512) partial_ += c; // a line that never ends doesn't take the heap
|
||||
continue;
|
||||
}
|
||||
if (partial_.rfind("status: heap ", 0) != 0) push(partial_);
|
||||
partial_.clear();
|
||||
}
|
||||
}
|
||||
|
||||
void ShellLog::clear() {
|
||||
lines_.clear();
|
||||
partial_.clear();
|
||||
bytes_ = 0;
|
||||
revision_++;
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
bool isWord(const std::string& w) {
|
||||
if (w.empty()) return false;
|
||||
for (size_t i = 0; i < w.size(); i++) {
|
||||
bool small = w[i] >= 'a' && w[i] <= 'z', capital = w[i] >= 'A' && w[i] <= 'Z';
|
||||
if (!(small || (i == 0 && capital))) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
std::vector<std::string> split(const std::string& text, const std::string& by) {
|
||||
std::vector<std::string> out;
|
||||
size_t at = 0;
|
||||
for (;;) {
|
||||
size_t next = text.find(by, at);
|
||||
out.push_back(text.substr(at, next == std::string::npos ? std::string::npos : next - at));
|
||||
if (next == std::string::npos) return out;
|
||||
at = next + by.size();
|
||||
}
|
||||
}
|
||||
|
||||
// The words a command can have at `index`, given the `index` words before it: added to `out`.
|
||||
void nextWords(const std::string& command, const std::vector<std::string>& before, size_t index, std::vector<std::string>& out) {
|
||||
std::vector<std::string> tokens;
|
||||
for (auto& t : split(command, " "))
|
||||
if (!t.empty()) tokens.push_back(t);
|
||||
for (size_t i = 0; i < tokens.size(); i++) {
|
||||
std::vector<std::string> either = split(tokens[i], "|"); // on|off: either of them
|
||||
bool words = true;
|
||||
for (auto& w : either) words = words && isWord(w);
|
||||
if (!words) return; // an <argument>, an [option], "...": the command's words end here
|
||||
if (i == index) {
|
||||
for (auto& w : either)
|
||||
if (std::find(out.begin(), out.end(), w) == out.end()) out.push_back(w);
|
||||
return;
|
||||
}
|
||||
if (std::find(either.begin(), either.end(), before[i]) == either.end()) return; // another command
|
||||
}
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::string completeWords(const std::string& typed, const char* helpText, std::vector<std::string>& matches) {
|
||||
matches.clear();
|
||||
std::vector<std::string> words = split(typed, " ");
|
||||
for (size_t i = 0; i + 1 < words.size(); i++)
|
||||
if (words[i].empty()) return typed; // two spaces: not ours to guess
|
||||
const std::string last = words.back();
|
||||
words.pop_back();
|
||||
if (words.empty() && last.empty()) return typed;
|
||||
|
||||
std::vector<std::string> next;
|
||||
for (auto& line : split(helpText ? helpText : "", "\n")) {
|
||||
std::string commands = line.substr(0, line.find(" ")); // the description starts at three spaces
|
||||
for (auto& command : split(commands, " | ")) nextWords(command, words, words.size(), next);
|
||||
}
|
||||
for (auto& w : next)
|
||||
if (w.rfind(last, 0) == 0) matches.push_back(w);
|
||||
if (matches.empty()) return typed;
|
||||
|
||||
std::string common = matches[0];
|
||||
for (auto& m : matches) {
|
||||
size_t n = 0;
|
||||
while (n < common.size() && n < m.size() && common[n] == m[n]) n++;
|
||||
common.resize(n);
|
||||
}
|
||||
std::string head = typed.substr(0, typed.size() - last.size());
|
||||
if (matches.size() == 1) {
|
||||
matches.clear();
|
||||
return head + common + " ";
|
||||
}
|
||||
return head + common;
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
char lower(char c) { return c >= 'A' && c <= 'Z' ? static_cast<char>(c - 'A' + 'a') : c; }
|
||||
|
||||
bool startsWithNoCase(const std::string& name, const std::string& prefix) {
|
||||
if (name.size() < prefix.size()) return false;
|
||||
for (size_t i = 0; i < prefix.size(); i++)
|
||||
if (lower(name[i]) != lower(prefix[i])) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool takesAPath(const std::string& command) {
|
||||
static const char* const kCommands[] = {"ls", "du", "mkdir", "rm", "cp", "mv", "cat", "install"};
|
||||
for (auto c : kCommands)
|
||||
if (command == c) return true;
|
||||
return false;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
bool splitForPath(const std::string& typed, PathToComplete& out) {
|
||||
size_t space = typed.rfind(' ');
|
||||
if (space == std::string::npos) return false; // still the command's own word
|
||||
std::string word = typed.substr(space + 1);
|
||||
if (!word.empty() && word[0] == '-') return false; // a switch
|
||||
if (word.empty() || word[0] != '/') {
|
||||
if (!takesAPath(typed.substr(0, typed.find(' ')))) return false;
|
||||
word = "/" + word;
|
||||
}
|
||||
size_t slash = word.rfind('/');
|
||||
out.head = typed.substr(0, space + 1);
|
||||
out.folder = word.substr(0, slash + 1);
|
||||
out.prefix = word.substr(slash + 1);
|
||||
return true;
|
||||
}
|
||||
|
||||
std::string completePath(const PathToComplete& what, const std::vector<std::string>& names, std::vector<std::string>& matches) {
|
||||
matches.clear();
|
||||
for (auto& n : names)
|
||||
if (startsWithNoCase(n, what.prefix)) matches.push_back(n);
|
||||
if (matches.empty()) return what.head + what.folder + what.prefix;
|
||||
std::string common = matches[0];
|
||||
for (auto& m : matches) {
|
||||
size_t n = 0;
|
||||
while (n < common.size() && n < m.size() && lower(common[n]) == lower(m[n])) n++;
|
||||
common.resize(n);
|
||||
}
|
||||
if (matches.size() == 1) {
|
||||
bool folder = !common.empty() && common.back() == '/';
|
||||
matches.clear();
|
||||
return what.head + what.folder + common + (folder ? "" : " ");
|
||||
}
|
||||
// Several: never shorter than what was typed (cases may differ past the prefix).
|
||||
if (common.size() < what.prefix.size()) common = what.prefix;
|
||||
return what.head + what.folder + common;
|
||||
}
|
||||
|
||||
} // namespace roro
|
||||
@@ -0,0 +1,65 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <deque>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
namespace roro {
|
||||
|
||||
// What the Shell App shows (issue #67): the lines it was given, with the oldest dropped past a size.
|
||||
// Which lines it is given is the console's business: by default, only what is printed for the
|
||||
// Shell's own commands (Console::Origin).
|
||||
class ShellLog {
|
||||
public:
|
||||
static constexpr size_t kMaxBytes = 4096;
|
||||
|
||||
// Bytes as the console printed them: lines may arrive in pieces. The line with the free heap
|
||||
// every ten seconds is never kept: it would push everything else off a ten-line screen.
|
||||
void feed(const char* data, size_t len);
|
||||
// A line the Shell adds itself.
|
||||
void add(const std::string& line);
|
||||
|
||||
const std::deque<std::string>& lines() const { return lines_; }
|
||||
void clear();
|
||||
uint32_t revision() const { return revision_; } // changes when the lines do
|
||||
|
||||
private:
|
||||
void push(const std::string& line);
|
||||
|
||||
std::deque<std::string> lines_;
|
||||
std::string partial_;
|
||||
size_t bytes_ = 0;
|
||||
uint32_t revision_ = 0;
|
||||
};
|
||||
|
||||
// Tab on a command (issue #67): every word of it, read from the firmware's `help` text each time it
|
||||
// is asked, so nothing is kept in memory for it. A line of that text is commands, three spaces, then
|
||||
// what they do; commands are separated by " | ", a word like on|off is either of them, and a command
|
||||
// stops being words at its first <argument>, [option] or "...":
|
||||
// lora rx on|off | lora preset <name> the radio
|
||||
// gives "lora rx on", "lora rx off" and "lora preset". An App's name starts with a capital.
|
||||
//
|
||||
// The line with its last word completed as far as the commands that fit agree; a word completed
|
||||
// whole gets a space after it. `matches` gets the candidates when there are several. Unchanged, with
|
||||
// no matches, when no command goes on that way: then it may be a path (below).
|
||||
std::string completeWords(const std::string& typed, const char* helpText, std::vector<std::string>& matches);
|
||||
|
||||
// Tab on a later word: a path on the SD card (issue #67). What is being completed, taken apart:
|
||||
// `rm -r /notes/sh` is head "rm -r ", folder "/notes/", prefix "sh". False when the cursor is still
|
||||
// on the first word, when the word is a switch (-r), or when it isn't a path and the command doesn't
|
||||
// take one. After a file command a path may be started without its slash: `cat no` is /no.
|
||||
struct PathToComplete {
|
||||
std::string head, folder, prefix;
|
||||
};
|
||||
bool splitForPath(const std::string& typed, PathToComplete& out);
|
||||
|
||||
// `names` are the folder's entries, a folder's with a slash at its end. The line with the path
|
||||
// completed as far as the entries that start with the prefix agree, whatever their case (the card
|
||||
// doesn't tell cases apart, so the name's own case is taken). A file completed whole gets a space
|
||||
// after it; a folder keeps its slash, to go on from. `matches` gets the candidates when there are
|
||||
// several. Unchanged when nothing matches.
|
||||
std::string completePath(const PathToComplete& what, const std::vector<std::string>& names, std::vector<std::string>& matches);
|
||||
|
||||
} // namespace roro
|
||||
@@ -2,7 +2,10 @@
|
||||
|
||||
#include <cstdint>
|
||||
|
||||
#include <vector>
|
||||
|
||||
#include "key_event.h"
|
||||
#include "key_help.h"
|
||||
|
||||
namespace roro {
|
||||
|
||||
@@ -25,11 +28,28 @@ class App {
|
||||
// True while the App is editing text: the arrow keys then type ; . , / and need Fn to move.
|
||||
virtual bool textEntryActive() const { return false; }
|
||||
|
||||
// The keys that work right now, for the help panel (Fn+h): the App's own, in the state it's
|
||||
// in. Back, Home and the arrows are added for it. No screen names keys any other way.
|
||||
virtual void help(std::vector<KeyHelp>& out) const { (void)out; }
|
||||
// What the panel is titled with, when the App's name isn't enough ("Notes: editor").
|
||||
virtual const char* helpTitle() const { return nullptr; }
|
||||
|
||||
// Called every main-loop pass while in the foreground (e.g. to refresh live values).
|
||||
virtual void update(uint32_t nowMs) { (void)nowMs; }
|
||||
|
||||
virtual void draw(Canvas& canvas) = 0;
|
||||
|
||||
// True while the screen shows something a picture of it shouldn't hold: the screenshot key
|
||||
// (Fn+p, issue #83) then refuses, and says so.
|
||||
virtual bool showsSecret() const { return false; }
|
||||
|
||||
// An App whose screen is costly to draw again (a picture decoded from the card, issue #45) can
|
||||
// keep what it drew: while this is true its part of the screen isn't cleared before draw(),
|
||||
// which then draws only what changed. contentLost() says that it was cleared after all, or
|
||||
// that something drawn over it has gone: everything has to be drawn again.
|
||||
virtual bool retainsContent() const { return false; }
|
||||
virtual void contentLost() {}
|
||||
|
||||
void requestRedraw() { redraw_ = true; }
|
||||
|
||||
bool consumeRedraw() {
|
||||
|
||||
@@ -0,0 +1,501 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstddef>
|
||||
#include <vector>
|
||||
|
||||
#include "key_help.h"
|
||||
|
||||
// Every key of every screen, as data (issues #69 and #72). The help panel (Fn+h) shows the table of
|
||||
// the state an App is in; site/tools/gen_dev_docs.py reads this file to write the website's key
|
||||
// tables, so the two can't drift.
|
||||
//
|
||||
// The format is read by that script, so keep it: a comment `// id: Title`, then one table,
|
||||
// inline constexpr KeyHelp kName[] = {
|
||||
// {"keys", "what they do"},
|
||||
// };
|
||||
// with one row a line and plain string literals. `id` is what a guide page asks for.
|
||||
namespace roro::keys {
|
||||
|
||||
template <size_t N>
|
||||
inline void add(std::vector<KeyHelp>& out, const KeyHelp (&rows)[N]) {
|
||||
out.insert(out.end(), rows, rows + N);
|
||||
}
|
||||
|
||||
// everywhere: Everywhere
|
||||
inline constexpr KeyHelp kEverywhere[] = {
|
||||
{"`", "back"},
|
||||
{"Fn `", "home, the Launcher"},
|
||||
{"; . , /", "arrows (Fn+ while typing)"},
|
||||
{"Fn h ?", "these keys (? not typing)"},
|
||||
{"Fn p", "a screenshot, on the card"},
|
||||
};
|
||||
|
||||
// dialog: A question
|
||||
inline constexpr KeyHelp kDialog[] = {
|
||||
{", /", "the other answer"},
|
||||
{"Enter", "choose it"},
|
||||
{"`", "cancel"},
|
||||
};
|
||||
|
||||
// text: A text field
|
||||
inline constexpr KeyHelp kText[] = {
|
||||
{"Enter", "save"},
|
||||
{"`", "cancel"},
|
||||
{"Del", "delete backwards"},
|
||||
{"Fn , /", "move the cursor"},
|
||||
{"opt ' e", "an accent: \xC3\xA9"},
|
||||
};
|
||||
|
||||
// launcher: The Launcher
|
||||
inline constexpr KeyHelp kLauncher[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "open the App"},
|
||||
};
|
||||
|
||||
// setup: Setup, a step
|
||||
inline constexpr KeyHelp kSetup[] = {
|
||||
{"Enter", "continue"},
|
||||
{"`", "the step before"},
|
||||
};
|
||||
|
||||
// setup-choice: Setup, a choice
|
||||
inline constexpr KeyHelp kSetupChoice[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "choose it, next step"},
|
||||
{"`", "the step before"},
|
||||
};
|
||||
|
||||
// setup-text: Setup, a name
|
||||
inline constexpr KeyHelp kSetupText[] = {
|
||||
{"Enter", "next step"},
|
||||
{"`", "the step before"},
|
||||
{"Del", "delete backwards"},
|
||||
{"Fn , /", "move the cursor"},
|
||||
{"opt ' e", "an accent: \xC3\xA9"},
|
||||
};
|
||||
|
||||
// irc: IRC
|
||||
inline constexpr KeyHelp kIrc[] = {
|
||||
{"Enter", "send the line"},
|
||||
{"Tab", "the next buffer"},
|
||||
{"Alt ; .", "scroll back, forward"},
|
||||
{"Fn ; .", "lines you sent before"},
|
||||
{"Fn , /", "move the cursor"},
|
||||
{"Del", "delete backwards"},
|
||||
{"/settings", "server, nick, passwords"},
|
||||
{"/join #x", "join a channel"},
|
||||
{"/part", "leave it"},
|
||||
{"/msg nick", "a private chat"},
|
||||
{"/me", "an action"},
|
||||
{"/nick", "change your nick"},
|
||||
{"/topic", "see or set the topic"},
|
||||
{"/names", "who is there"},
|
||||
{"/quit", "disconnect, and stay so"},
|
||||
{"/raw", "a line as it is"},
|
||||
{"`", "leave: IRC stays connected"},
|
||||
};
|
||||
|
||||
// irc-settings: IRC settings
|
||||
inline constexpr KeyHelp kIrcSettings[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "edit, switch, or save"},
|
||||
{"`", "leave without saving"},
|
||||
};
|
||||
|
||||
// irc-field: IRC, a setting
|
||||
inline constexpr KeyHelp kIrcField[] = {
|
||||
{"Enter", "keep it"},
|
||||
{"`", "cancel"},
|
||||
{"Del", "delete backwards"},
|
||||
{"Fn , /", "move the cursor"},
|
||||
};
|
||||
|
||||
// wifi-tools: Wi-Fi Tools
|
||||
inline constexpr KeyHelp kWifiTools[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "open"},
|
||||
};
|
||||
|
||||
// wifi-networks: Networks nearby
|
||||
inline constexpr KeyHelp kWifiNetworks[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "track its signal"},
|
||||
{"s", "sort: signal, channel, name"},
|
||||
{"o", "open networks only"},
|
||||
{"h", "hide the hidden ones"},
|
||||
{"w", "strong ones only"},
|
||||
{"l", "log the scans to the card"},
|
||||
};
|
||||
|
||||
// wifi-tracker: Signal tracker
|
||||
inline constexpr KeyHelp kWifiTracker[] = {
|
||||
{"m", "clicks on or off"},
|
||||
};
|
||||
|
||||
// gnss: GNSS
|
||||
inline constexpr KeyHelp kGnss[] = {
|
||||
{"Tab", "the position, or the sky"},
|
||||
{"r", "record a Track, or stop it"},
|
||||
};
|
||||
|
||||
// gemini: Gemini, a page
|
||||
inline constexpr KeyHelp kGemini[] = {
|
||||
{"Tab", "the next link"},
|
||||
{"Aa Tab", "the link before"},
|
||||
{"Enter", "follow the link"},
|
||||
{"` Del", "the page before"},
|
||||
{"; .", "scroll"},
|
||||
{"Space", "a page down"},
|
||||
{", /", "sideways, in wide blocks"},
|
||||
{"g", "type an address"},
|
||||
{"b", "bookmark this page"},
|
||||
{"s", "save the page to the card"},
|
||||
{"S", "...with the pages it links to"},
|
||||
};
|
||||
|
||||
// gemini-saved: Gemini, a Saved Page
|
||||
inline constexpr KeyHelp kGeminiSaved[] = {
|
||||
{"Tab", "the next link"},
|
||||
{"Aa Tab", "the link before"},
|
||||
{"Enter", "follow the link"},
|
||||
{"` Del", "the page before"},
|
||||
{"; .", "scroll"},
|
||||
{"Space", "a page down"},
|
||||
{", /", "sideways, in wide blocks"},
|
||||
{"g", "type an address"},
|
||||
{"b", "bookmark this page"},
|
||||
{"r", "refresh this Saved Page"},
|
||||
{"d", "delete this Saved Page"},
|
||||
};
|
||||
|
||||
// gemini-address: Gemini, an address
|
||||
inline constexpr KeyHelp kGeminiAddress[] = {
|
||||
{"Enter", "go there"},
|
||||
{"`", "cancel"},
|
||||
{"Del", "delete backwards"},
|
||||
{"Fn , /", "move the cursor"},
|
||||
};
|
||||
|
||||
// gemini-answer: Gemini, an answer to a page
|
||||
inline constexpr KeyHelp kGeminiAnswer[] = {
|
||||
{"Enter", "send it"},
|
||||
{"`", "cancel"},
|
||||
{"Del", "delete backwards"},
|
||||
{"Fn , /", "move the cursor"},
|
||||
};
|
||||
|
||||
// lora: LoRa Scanner, the packets
|
||||
inline constexpr KeyHelp kLora[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "the packet's details"},
|
||||
{"p", "pick a Meshtastic preset"},
|
||||
{"c", "start a Capture, or stop it"},
|
||||
{"Tab", "the Sweep"},
|
||||
};
|
||||
|
||||
// lora-packet: LoRa Scanner, a packet
|
||||
inline constexpr KeyHelp kLoraPacket[] = {
|
||||
{"; .", "scroll"},
|
||||
{"Enter", "back to the list"},
|
||||
};
|
||||
|
||||
// lora-presets: LoRa Scanner, the presets
|
||||
inline constexpr KeyHelp kLoraPresets[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "listen with this preset"},
|
||||
};
|
||||
|
||||
// lora-sweep: LoRa Scanner, the Sweep
|
||||
inline constexpr KeyHelp kLoraSweep[] = {
|
||||
{"Tab", "the Sniffer"},
|
||||
};
|
||||
|
||||
// storage: Storage, a folder
|
||||
inline constexpr KeyHelp kStorage[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "open the folder or the file"},
|
||||
{", /", "a page up, down"},
|
||||
{"c x", "copy, cut"},
|
||||
{"v", "paste here"},
|
||||
{"r", "rename"},
|
||||
{"d Del", "delete, after asking"},
|
||||
{"n", "a new folder"},
|
||||
{"i", "details: size, date, type"},
|
||||
{"s", "sort: name, date, size"},
|
||||
{"m", "Maintenance: clean-up, erase"},
|
||||
{"w", "share with a browser"},
|
||||
{"`", "the folder above"},
|
||||
};
|
||||
|
||||
// storage-share: Storage, sharing with a browser
|
||||
inline constexpr KeyHelp kStorageShare[] = {
|
||||
{"`", "stop sharing"},
|
||||
};
|
||||
|
||||
// storage-details: Storage, an item's details
|
||||
inline constexpr KeyHelp kStorageDetails[] = {
|
||||
{"; .", "scroll"},
|
||||
{"Enter", "back to the folder"},
|
||||
};
|
||||
|
||||
// storage-name: Storage, a name
|
||||
inline constexpr KeyHelp kStorageName[] = {
|
||||
{"Enter", "rename it, or make the folder"},
|
||||
{"`", "cancel"},
|
||||
{"Del", "delete backwards"},
|
||||
{"Fn , /", "move the cursor"},
|
||||
};
|
||||
|
||||
// storage-busy: Storage, while it copies or deletes
|
||||
inline constexpr KeyHelp kStorageBusy[] = {
|
||||
{"`", "stop the copy or the delete"},
|
||||
};
|
||||
|
||||
// maintenance: Storage, Maintenance
|
||||
inline constexpr KeyHelp kMaintenance[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "open, or choose"},
|
||||
};
|
||||
|
||||
// viewer-text: A file, as text
|
||||
inline constexpr KeyHelp kViewerText[] = {
|
||||
{"; .", "a line up, down"},
|
||||
{", /", "a page up, down"},
|
||||
{"t b", "the top, the end"},
|
||||
{"e", "edit it"},
|
||||
{"Tab", "the file as hex, or back"},
|
||||
};
|
||||
|
||||
// viewer-hex: A file, as hex
|
||||
inline constexpr KeyHelp kViewerHex[] = {
|
||||
{"; .", "a line up, down"},
|
||||
{", /", "a page up, down"},
|
||||
{"t b", "the top, the end"},
|
||||
{"Tab", "the file as text, or back"},
|
||||
};
|
||||
|
||||
// viewer-pcap: A Capture
|
||||
inline constexpr KeyHelp kViewerPcap[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "the packet"},
|
||||
{", /", "a page up, down"},
|
||||
{"Tab", "the file as hex"},
|
||||
};
|
||||
|
||||
// viewer-packet: A Capture's packet
|
||||
inline constexpr KeyHelp kViewerPacket[] = {
|
||||
{"; .", "scroll"},
|
||||
{"Enter", "back to the packets"},
|
||||
};
|
||||
|
||||
// viewer-gpx: A Track
|
||||
inline constexpr KeyHelp kViewerGpx[] = {
|
||||
{"Tab", "the file as text"},
|
||||
};
|
||||
|
||||
// viewer-ota: An Update File
|
||||
inline constexpr KeyHelp kViewerOta[] = {
|
||||
{"Enter", "install it, if it's genuine"},
|
||||
{"Tab", "the file as hex"},
|
||||
};
|
||||
|
||||
// viewer-image: A picture
|
||||
inline constexpr KeyHelp kViewerImage[] = {
|
||||
{"Enter", "its own size, or all of it"},
|
||||
{"; . , /", "move around it"},
|
||||
{"i", "its size"},
|
||||
{"Tab", "the file as hex"},
|
||||
};
|
||||
|
||||
// vpn: Settings, VPN
|
||||
inline constexpr KeyHelp kVpn[] = {
|
||||
{"Enter", "switch, import, forget"},
|
||||
{"; .", "up, down"},
|
||||
};
|
||||
|
||||
// ssh: SSH, the hosts
|
||||
inline constexpr KeyHelp kSsh[] = {
|
||||
{"Enter", "connect, open"},
|
||||
{"; .", "up, down"},
|
||||
{"n", "a new connection"},
|
||||
{"d", "forget this host"},
|
||||
};
|
||||
|
||||
// ssh-terminal: SSH, the terminal
|
||||
inline constexpr KeyHelp kSshTerminal[] = {
|
||||
{"`", "Esc"},
|
||||
{"Alt `", "a backtick"},
|
||||
{"Fn ; . , /", "the arrows"},
|
||||
{"Fn Shift ; .", "Page Up, Page Down"},
|
||||
{"Ctrl a..z", "Ctrl+C and the rest"},
|
||||
{"Alt ; .", "scroll back, forward"},
|
||||
{"Ctrl + -", "larger, smaller text"},
|
||||
{"Ctrl Alt q", "disconnect"},
|
||||
{"Fn `", "leave it running"},
|
||||
};
|
||||
|
||||
// ssh-key: SSH, this device's key
|
||||
inline constexpr KeyHelp kSshKey[] = {
|
||||
{"Enter", "make a key, or a new one"},
|
||||
{"w", "write the public half to the card"},
|
||||
{"`", "back"},
|
||||
};
|
||||
|
||||
// notes: Notes, the list
|
||||
inline constexpr KeyHelp kNotes[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "open the note"},
|
||||
{", /", "a page up, down"},
|
||||
{"n", "a new note"},
|
||||
{"r", "rename its file"},
|
||||
{"d Del", "delete it"},
|
||||
{"s", "sort: newest, or by name"},
|
||||
};
|
||||
|
||||
// notes-editor: Notes, the editor
|
||||
inline constexpr KeyHelp kNotesEditor[] = {
|
||||
{"Enter", "a new line"},
|
||||
{"Del", "delete backwards"},
|
||||
{"Tab", "two spaces"},
|
||||
{"Fn ; . , /", "move the cursor"},
|
||||
{"Alt Fn ; .", "a page up, down"},
|
||||
{"Ctrl Fn ; .", "start, end of the note"},
|
||||
{"Ctrl a e", "start, end of the line"},
|
||||
{"opt ' e", "an accent: \xC3\xA9"},
|
||||
{"`", "done: it saves by itself"},
|
||||
};
|
||||
|
||||
// notes-name: Notes, a file name
|
||||
inline constexpr KeyHelp kNotesName[] = {
|
||||
{"Enter", "rename the file"},
|
||||
{"`", "cancel"},
|
||||
{"Del", "delete backwards"},
|
||||
{"Fn , /", "move the cursor"},
|
||||
};
|
||||
|
||||
// shell: Shell
|
||||
inline constexpr KeyHelp kShell[] = {
|
||||
{"Enter", "run the line"},
|
||||
{"Tab", "complete: a command, a path"},
|
||||
{"* ?", "several files: /notes/*.txt"},
|
||||
{"Fn ; .", "lines you typed before"},
|
||||
{"Alt ; .", "scroll back, forward"},
|
||||
{"Ctrl b", "your replies only, or all"},
|
||||
{"Fn , /", "move the cursor"},
|
||||
{"Del", "delete backwards"},
|
||||
{"help", "every command"},
|
||||
{"clear", "an empty screen"},
|
||||
{"Notes", "an App, by its name"},
|
||||
{"rm -rf", "delete without being asked"},
|
||||
{"quit `", "leave the Shell"},
|
||||
};
|
||||
|
||||
// system: System, any view
|
||||
inline constexpr KeyHelp kSystem[] = {
|
||||
{"Tab", "the next view"},
|
||||
{"Aa Tab", "the view before"},
|
||||
};
|
||||
|
||||
// system-tasks: System, the tasks
|
||||
inline constexpr KeyHelp kSystemTasks[] = {
|
||||
{"Tab", "the next view"},
|
||||
{"Aa Tab", "the view before"},
|
||||
{"; .", "scroll"},
|
||||
{"s", "sort: cpu, stack, name"},
|
||||
};
|
||||
|
||||
// system-system: System, the system view
|
||||
inline constexpr KeyHelp kSystemSystem[] = {
|
||||
{"Tab", "the next view"},
|
||||
{"Aa Tab", "the view before"},
|
||||
{"; .", "scroll"},
|
||||
};
|
||||
|
||||
// settings: Settings
|
||||
inline constexpr KeyHelp kSettings[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "edit, or open the page"},
|
||||
{", /", "change a switch or a slider"},
|
||||
};
|
||||
|
||||
// settings-choice: Settings, a choice
|
||||
inline constexpr KeyHelp kSettingsChoice[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "choose it"},
|
||||
};
|
||||
|
||||
// wifi: Settings, Wi-Fi
|
||||
inline constexpr KeyHelp kWifi[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "open, or change"},
|
||||
{", /", "switch Wi-Fi on or off"},
|
||||
};
|
||||
|
||||
// wifi-servers: Settings, DNS and NTP
|
||||
inline constexpr KeyHelp kWifiServers[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "edit"},
|
||||
{", /", "Always use my DNS: on, off"},
|
||||
};
|
||||
|
||||
// wifi-network: Settings, a saved network
|
||||
inline constexpr KeyHelp kWifiNetwork[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "edit, or forget"},
|
||||
{", /", "Automatic or Fixed"},
|
||||
};
|
||||
|
||||
// wifi-status: Settings, the Wi-Fi status
|
||||
inline constexpr KeyHelp kWifiStatus[] = {
|
||||
{"Enter", "back"},
|
||||
};
|
||||
|
||||
// wifi-scan: Settings, adding a network
|
||||
inline constexpr KeyHelp kWifiScan[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "choose this network"},
|
||||
};
|
||||
|
||||
// wifi-name: Settings, a hidden network's name
|
||||
inline constexpr KeyHelp kWifiName[] = {
|
||||
{"Enter", "next: the password"},
|
||||
{"`", "cancel"},
|
||||
{"Del", "delete backwards"},
|
||||
{"Fn , /", "move the cursor"},
|
||||
};
|
||||
|
||||
// firmware: Settings, Firmware
|
||||
inline constexpr KeyHelp kFirmware[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "check, open, or install"},
|
||||
{"c", "look for a newer release"},
|
||||
};
|
||||
|
||||
// firmware-release: Settings, a release
|
||||
inline constexpr KeyHelp kFirmwareRelease[] = {
|
||||
{"; .", "scroll"},
|
||||
{"Enter", "install it"},
|
||||
{"c", "check again"},
|
||||
};
|
||||
|
||||
// firmware-older: Settings, older releases
|
||||
inline constexpr KeyHelp kFirmwareOlder[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "its details"},
|
||||
{"c", "read the list again"},
|
||||
};
|
||||
|
||||
// debug-console: Settings, Debug Console
|
||||
inline constexpr KeyHelp kDebugConsole[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "switch, or open"},
|
||||
{", /", "switch the console on or off"},
|
||||
};
|
||||
|
||||
// demo: The widget demo
|
||||
inline constexpr KeyHelp kDemo[] = {
|
||||
{"; .", "up, down"},
|
||||
{"Enter", "try the widget"},
|
||||
};
|
||||
|
||||
} // namespace roro::keys
|
||||
@@ -1,5 +1,7 @@
|
||||
#include "app_manager.h"
|
||||
|
||||
#include "app_keys.h"
|
||||
|
||||
#include <cstring>
|
||||
|
||||
namespace roro {
|
||||
@@ -52,6 +54,22 @@ void AppManager::endModal() {
|
||||
}
|
||||
|
||||
void AppManager::handleKey(const KeyEvent& event) {
|
||||
if (help_.isOpen()) { // scrolls or closes; nothing reaches the App, Home included
|
||||
help_.onKey(event);
|
||||
redraw_ = true;
|
||||
return;
|
||||
}
|
||||
if (event.key == Key::Help) {
|
||||
std::vector<KeyHelp> rows;
|
||||
foreground_->help(rows);
|
||||
rows.push_back({"Everywhere", nullptr});
|
||||
keys::add(rows, keys::kEverywhere);
|
||||
const char* scope = foreground_->helpTitle();
|
||||
const char* app = foregroundTitle();
|
||||
help_.open(scope ? scope : app ? app : "Launcher", std::move(rows));
|
||||
redraw_ = true;
|
||||
return;
|
||||
}
|
||||
if (modal_) {
|
||||
foreground_->onKey(event);
|
||||
return;
|
||||
@@ -64,6 +82,19 @@ void AppManager::handleKey(const KeyEvent& event) {
|
||||
if (!consumed && event.key == Key::Back) home();
|
||||
}
|
||||
|
||||
std::string AppManager::commandFor(const char* id) {
|
||||
std::string command;
|
||||
for (const char* p = id; *p && *p != '-'; p++) command += *p;
|
||||
if (!command.empty() && command[0] >= 'a' && command[0] <= 'z') command[0] = static_cast<char>(command[0] - 'a' + 'A');
|
||||
return command;
|
||||
}
|
||||
|
||||
const AppInfo* AppManager::byCommand(const std::string& command) const {
|
||||
for (auto& info : apps_)
|
||||
if (!info.hidden && commandFor(info.id) == command) return &info;
|
||||
return nullptr;
|
||||
}
|
||||
|
||||
const char* AppManager::foregroundTitle() const {
|
||||
for (auto& info : apps_)
|
||||
if (info.app == foreground_) return info.title;
|
||||
@@ -77,6 +108,7 @@ bool AppManager::takeRedraw() {
|
||||
}
|
||||
|
||||
void AppManager::switchTo(App& app) {
|
||||
help_.close(); // an App opened from elsewhere (a Notification, a command): its keys, not the last one's
|
||||
if (&app == foreground_) return;
|
||||
foreground_->onExit();
|
||||
foreground_ = &app;
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
#pragma once
|
||||
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "app.h"
|
||||
@@ -34,9 +35,19 @@ class AppManager {
|
||||
void handleKey(const KeyEvent& event);
|
||||
void update(uint32_t nowMs) { foreground_->update(nowMs); }
|
||||
|
||||
// The command that opens an App from the consoles and the Shell (issue #67): its id with a
|
||||
// capital letter, up to the first dash. "notes" is Notes, "wifi-tools" is Wifi. The capital says
|
||||
// "an App", where every command of the firmware is in small letters.
|
||||
static std::string commandFor(const char* id);
|
||||
// The App that command names, among the ones the Launcher lists; nullptr if there's none.
|
||||
const AppInfo* byCommand(const std::string& command) const;
|
||||
|
||||
App& foreground() const { return *foreground_; }
|
||||
const char* foregroundTitle() const; // nullptr for the Launcher
|
||||
|
||||
// The help panel (Fn+h): open, it takes every key, and the App sees none of them.
|
||||
const HelpModel& help() const { return help_; }
|
||||
|
||||
// True once after the screen needs redrawing (App switch, or the App asked for it).
|
||||
bool takeRedraw();
|
||||
|
||||
@@ -47,6 +58,7 @@ class AppManager {
|
||||
App& launcher_;
|
||||
App* foreground_;
|
||||
std::vector<AppInfo> apps_;
|
||||
HelpModel help_;
|
||||
bool redraw_ = true;
|
||||
bool modal_ = false;
|
||||
};
|
||||
|
||||
@@ -16,6 +16,8 @@ enum class Key : uint8_t {
|
||||
Home,
|
||||
Tab,
|
||||
Delete,
|
||||
Help, // Fn+h anywhere, or ? outside Text Entry: the keys of this screen (issue #69)
|
||||
Screenshot, // Fn+p anywhere: the screen as a PNG on the card (issue #83). Never reaches an App
|
||||
};
|
||||
|
||||
struct KeyEvent {
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "key_event.h"
|
||||
|
||||
// The help panel (issue #69, docs/milestones/U1.md): Fn+h on any screen lists the keys that work
|
||||
// there. Every App says what its keys are in the state it's in; nothing on a screen names keys.
|
||||
namespace roro {
|
||||
|
||||
// One line of the panel: a key (or keys) and what it does. A line with no action is a heading.
|
||||
struct KeyHelp {
|
||||
const char* keys;
|
||||
const char* action;
|
||||
};
|
||||
|
||||
|
||||
// The panel itself: what it lists, and how far it's scrolled. The keys it takes while open are the
|
||||
// arrows; any other key closes it, and none reaches the App.
|
||||
class HelpModel {
|
||||
public:
|
||||
explicit HelpModel(int visibleRows = 8) : visible_(visibleRows) {}
|
||||
|
||||
void open(const std::string& title, std::vector<KeyHelp> rows) {
|
||||
title_ = title;
|
||||
rows_ = std::move(rows);
|
||||
top_ = 0;
|
||||
open_ = true;
|
||||
}
|
||||
void close() {
|
||||
open_ = false;
|
||||
rows_.clear();
|
||||
rows_.shrink_to_fit(); // nothing is kept while it's closed
|
||||
}
|
||||
bool isOpen() const { return open_; }
|
||||
|
||||
void onKey(const KeyEvent& e) {
|
||||
int last = std::max(0, static_cast<int>(rows_.size()) - visible_);
|
||||
switch (e.key) {
|
||||
case Key::Up: top_ = std::max(0, top_ - 1); break;
|
||||
case Key::Down: top_ = std::min(last, top_ + 1); break;
|
||||
case Key::Left: top_ = std::max(0, top_ - visible_); break;
|
||||
case Key::Right: top_ = std::min(last, top_ + visible_); break;
|
||||
default: close(); break;
|
||||
}
|
||||
}
|
||||
|
||||
const std::string& title() const { return title_; }
|
||||
const std::vector<KeyHelp>& rows() const { return rows_; }
|
||||
int top() const { return top_; }
|
||||
int visibleRows() const { return visible_; }
|
||||
|
||||
private:
|
||||
std::string title_;
|
||||
std::vector<KeyHelp> rows_;
|
||||
int top_ = 0;
|
||||
int visible_;
|
||||
bool open_ = false;
|
||||
};
|
||||
|
||||
} // namespace roro
|
||||
@@ -0,0 +1,117 @@
|
||||
#include "debug_auth.h"
|
||||
|
||||
#include <cstring>
|
||||
|
||||
#include "sha256.h"
|
||||
|
||||
namespace roro::debug {
|
||||
|
||||
namespace {
|
||||
const char kAlphabet[] = "0123456789ABCDEFGHJKMNPQRSTVWXYZ";
|
||||
}
|
||||
|
||||
std::string makeToken(const uint8_t random[kTokenRandom]) {
|
||||
std::string token;
|
||||
uint32_t bits = 0;
|
||||
int have = 0;
|
||||
size_t next = 0;
|
||||
while (token.size() < kTokenChars) {
|
||||
if (have < 5) {
|
||||
bits = (bits << 8) | random[next++];
|
||||
have += 8;
|
||||
}
|
||||
token += kAlphabet[(bits >> (have - 5)) & 31];
|
||||
have -= 5;
|
||||
}
|
||||
return token;
|
||||
}
|
||||
|
||||
std::string tidyToken(const std::string& typed) {
|
||||
std::string out;
|
||||
for (char c : typed) {
|
||||
if (c == '-' || c == ' ' || c == '\t' || c == '\r' || c == '\n') continue;
|
||||
if (c >= 'a' && c <= 'z') c = static_cast<char>(c - 'a' + 'A');
|
||||
// Crockford's rule for the letters his alphabet leaves out: read as the digit they look like.
|
||||
if (c == 'O') c = '0';
|
||||
if (c == 'I' || c == 'L') c = '1';
|
||||
out += c;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
bool validToken(const std::string& tidied) {
|
||||
if (tidied.size() < kMinTokenChars || tidied.size() > kMaxTokenChars) return false;
|
||||
for (char c : tidied)
|
||||
if (c <= ' ' || c > '~' || c == '-' || (c >= 'a' && c <= 'z') || c == 'O' || c == 'I' || c == 'L') return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
std::string groupToken(const std::string& token) {
|
||||
std::string out;
|
||||
for (size_t i = 0; i < token.size(); i++) {
|
||||
if (i && i % 4 == 0) out += '-';
|
||||
out += token[i];
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
void hmacSha256(const uint8_t* key, size_t keyLen, const uint8_t* message, size_t messageLen, uint8_t out[32]) {
|
||||
uint8_t block[64] = {};
|
||||
if (keyLen > sizeof block) Sha256::hash(key, keyLen, block); // a long key is hashed first
|
||||
else memcpy(block, key, keyLen);
|
||||
|
||||
uint8_t pad[64];
|
||||
for (size_t i = 0; i < sizeof pad; i++) pad[i] = block[i] ^ 0x36;
|
||||
uint8_t inner[32];
|
||||
Sha256 in;
|
||||
in.update(pad, sizeof pad);
|
||||
in.update(message, messageLen);
|
||||
in.finish(inner);
|
||||
|
||||
for (size_t i = 0; i < sizeof pad; i++) pad[i] = block[i] ^ 0x5c;
|
||||
Sha256 outer;
|
||||
outer.update(pad, sizeof pad);
|
||||
outer.update(inner, sizeof inner);
|
||||
outer.finish(out);
|
||||
}
|
||||
|
||||
std::string toHex(const uint8_t* data, size_t len) {
|
||||
static const char digits[] = "0123456789abcdef";
|
||||
std::string out;
|
||||
out.reserve(len * 2);
|
||||
for (size_t i = 0; i < len; i++) {
|
||||
out += digits[data[i] >> 4];
|
||||
out += digits[data[i] & 15];
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
std::string answerFor(const std::string& token, const uint8_t nonce[kNonceBytes]) {
|
||||
uint8_t mac[32];
|
||||
hmacSha256(reinterpret_cast<const uint8_t*>(token.data()), token.size(), nonce, kNonceBytes, mac);
|
||||
return toHex(mac, sizeof mac);
|
||||
}
|
||||
|
||||
bool sameText(const std::string& a, const std::string& b) {
|
||||
uint8_t diff = a.size() != b.size();
|
||||
for (size_t i = 0; i < b.size(); i++) diff |= static_cast<uint8_t>((i < a.size() ? a[i] : 0) ^ b[i]);
|
||||
return diff == 0;
|
||||
}
|
||||
|
||||
bool AuthGate::locked(uint32_t nowMs) {
|
||||
if (locked_ && nowMs - lockedAtMs_ >= kLockMs) { // unsigned: right across the 49-day wrap too
|
||||
locked_ = false;
|
||||
failures_ = 0;
|
||||
}
|
||||
return locked_;
|
||||
}
|
||||
|
||||
bool AuthGate::failed(uint32_t nowMs) {
|
||||
if (locked(nowMs)) return false;
|
||||
if (++failures_ < kMaxFailures) return false;
|
||||
locked_ = true;
|
||||
lockedAtMs_ = nowMs;
|
||||
return true;
|
||||
}
|
||||
|
||||
} // namespace roro::debug
|
||||
@@ -0,0 +1,59 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <string>
|
||||
|
||||
// Who may use the Debug Console (ADR 0010): a token that only the device and its owner know, proved
|
||||
// with a challenge and an answer so that it never crosses the network, and a pause after wrong answers.
|
||||
namespace roro::debug {
|
||||
|
||||
constexpr size_t kTokenChars = 20; // a token the device makes: 100 bits
|
||||
constexpr size_t kTokenRandom = 13; // the random bytes it takes
|
||||
constexpr size_t kMinTokenChars = 16; // a token typed by hand, once tidied
|
||||
constexpr size_t kMaxTokenChars = 64;
|
||||
constexpr size_t kNonceBytes = 16;
|
||||
|
||||
// A token from random bytes: 20 characters of Crockford's base32 (no I, L, O or U to misread).
|
||||
std::string makeToken(const uint8_t random[kTokenRandom]);
|
||||
|
||||
// What a person typed, as it's stored and compared: no dashes or spaces, in capitals, and with O
|
||||
// read as 0, I and L as 1. So a token can be read off the screen in groups, typed in any case,
|
||||
// and the usual misreadings don't matter.
|
||||
std::string tidyToken(const std::string& typed);
|
||||
|
||||
// A tidied token that may be stored: 16 to 64 printable ASCII characters, as tidyToken leaves them.
|
||||
bool validToken(const std::string& tidied);
|
||||
|
||||
// For the screen: K7QF-3M2X-9WBD-HT4P-6RNC.
|
||||
std::string groupToken(const std::string& token);
|
||||
|
||||
void hmacSha256(const uint8_t* key, size_t keyLen, const uint8_t* message, size_t messageLen, uint8_t out[32]);
|
||||
|
||||
std::string toHex(const uint8_t* data, size_t len);
|
||||
|
||||
// What a client must send back for a challenge: HMAC-SHA256 of the nonce, keyed by the token, in hex.
|
||||
std::string answerFor(const std::string& token, const uint8_t nonce[kNonceBytes]);
|
||||
|
||||
// Compares without stopping at the first difference, so timing says nothing about the answer.
|
||||
bool sameText(const std::string& a, const std::string& b);
|
||||
|
||||
// Five wrong answers in a row, from anyone, and nobody is listened to for a minute.
|
||||
class AuthGate {
|
||||
public:
|
||||
static constexpr int kMaxFailures = 5;
|
||||
static constexpr uint32_t kLockMs = 60000;
|
||||
|
||||
bool locked(uint32_t nowMs);
|
||||
// A wrong answer. True if it's the one that starts the pause.
|
||||
bool failed(uint32_t nowMs);
|
||||
void succeeded() { failures_ = 0; }
|
||||
int failures() const { return failures_; }
|
||||
|
||||
private:
|
||||
int failures_ = 0;
|
||||
bool locked_ = false;
|
||||
uint32_t lockedAtMs_ = 0;
|
||||
};
|
||||
|
||||
} // namespace roro::debug
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
namespace roro::files {
|
||||
|
||||
const char* const kFirmwareFolders[] = {"/irc", "/wifi", "/updates", "/gnss", "/gemini", "/captures", "/notes"};
|
||||
const char* const kFirmwareFolders[] = {"/irc", "/wifi", "/updates", "/gnss", "/gemini", "/captures", "/notes", "/screenshots"};
|
||||
const size_t kFirmwareFolderCount = sizeof kFirmwareFolders / sizeof kFirmwareFolders[0];
|
||||
|
||||
std::string parentOf(const std::string& path) {
|
||||
@@ -96,4 +96,52 @@ bool looksLikeText(const uint8_t* data, size_t len) {
|
||||
return odd * 20 <= len; // a stray control character or two is still text
|
||||
}
|
||||
|
||||
RmArgs parseRm(const std::string& args) {
|
||||
RmArgs out;
|
||||
size_t at = 0;
|
||||
while (at < args.size()) {
|
||||
while (at < args.size() && args[at] == ' ') at++;
|
||||
if (at >= args.size() || args[at] != '-') break;
|
||||
size_t end = args.find(' ', at);
|
||||
std::string flags = args.substr(at + 1, end == std::string::npos ? std::string::npos : end - at - 1);
|
||||
bool known = !flags.empty();
|
||||
for (char c : flags) known = known && (c == 'r' || c == 'R' || c == 'f');
|
||||
if (!known) break; // a name that starts with a dash
|
||||
for (char c : flags) (c == 'f' ? out.force : out.recursive) = true;
|
||||
at = end == std::string::npos ? args.size() : end;
|
||||
}
|
||||
out.path = at < args.size() ? args.substr(at) : "";
|
||||
while (!out.path.empty() && out.path.back() == ' ') out.path.pop_back();
|
||||
return out;
|
||||
}
|
||||
|
||||
bool hasGlob(const std::string& text) { return text.find_first_of("*?") != std::string::npos; }
|
||||
|
||||
bool globMatch(const std::string& pattern, const std::string& name) {
|
||||
auto lower = [](char c) { return c >= 'A' && c <= 'Z' ? static_cast<char>(c - 'A' + 'a') : c; };
|
||||
size_t p = 0, n = 0, star = std::string::npos, mark = 0;
|
||||
while (n < name.size()) {
|
||||
if (p < pattern.size() && (pattern[p] == '?' || lower(pattern[p]) == lower(name[n]))) {
|
||||
p++;
|
||||
n++;
|
||||
} else if (p < pattern.size() && pattern[p] == '*') {
|
||||
star = p++; // try it as nothing first; come back here to let it take one more
|
||||
mark = n;
|
||||
} else if (star != std::string::npos) {
|
||||
p = star + 1;
|
||||
n = ++mark;
|
||||
} else return false;
|
||||
}
|
||||
while (p < pattern.size() && pattern[p] == '*') p++;
|
||||
return p == pattern.size();
|
||||
}
|
||||
|
||||
bool splitGlob(const std::string& path, std::string& folder, std::string& pattern) {
|
||||
size_t slash = path.rfind('/');
|
||||
if (slash == std::string::npos) return false;
|
||||
pattern = path.substr(slash + 1);
|
||||
folder = slash == 0 ? "/" : path.substr(0, slash);
|
||||
return hasGlob(pattern) && !hasGlob(folder);
|
||||
}
|
||||
|
||||
} // namespace roro::files
|
||||
|
||||
@@ -39,4 +39,20 @@ FileKind kindOf(const std::string& name);
|
||||
bool opensAtEnd(const std::string& name); // logs
|
||||
bool looksLikeText(const uint8_t* data, size_t len);
|
||||
|
||||
// `rm`'s arguments, as Unix has them (issue #67): -r for a folder and what's in it, -f for no
|
||||
// question, alone or together (-rf, -fr, -r -f), then the path, which may hold spaces.
|
||||
struct RmArgs {
|
||||
bool recursive = false, force = false;
|
||||
std::string path;
|
||||
};
|
||||
RmArgs parseRm(const std::string& args);
|
||||
|
||||
// Patterns in a path (issue #67): * for any run of characters, ? for one, in the last part of the
|
||||
// path only (/notes/*.txt, not /*/a.txt). Cases aren't told apart, as on the card.
|
||||
bool hasGlob(const std::string& text);
|
||||
bool globMatch(const std::string& pattern, const std::string& name);
|
||||
// "/notes/*.txt" taken apart: the folder ("/notes", or "/" at the top) and the pattern ("*.txt").
|
||||
// False if there's no pattern in it, or if the folder has one too.
|
||||
bool splitGlob(const std::string& path, std::string& folder, std::string& pattern);
|
||||
|
||||
} // namespace roro::files
|
||||
|
||||
@@ -0,0 +1,464 @@
|
||||
#include "image_file.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <cstring>
|
||||
#include <memory>
|
||||
#include <new>
|
||||
|
||||
#include "file_names.h"
|
||||
#include "png_rgb332.h"
|
||||
|
||||
namespace roro::files {
|
||||
|
||||
namespace {
|
||||
uint32_t le16(const uint8_t* p) { return p[0] | (p[1] << 8); }
|
||||
uint32_t le32(const uint8_t* p) { return p[0] | (p[1] << 8) | (p[2] << 16) | (static_cast<uint32_t>(p[3]) << 24); }
|
||||
uint32_t be16(const uint8_t* p) { return (p[0] << 8) | p[1]; }
|
||||
uint32_t be32(const uint8_t* p) { return (static_cast<uint32_t>(p[0]) << 24) | (p[1] << 16) | (p[2] << 8) | p[3]; }
|
||||
|
||||
constexpr int kMaxSide = 16384; // more than that isn't a picture for this screen
|
||||
const uint8_t kPngSignature[] = {0x89, 'P', 'N', 'G', '\r', '\n', 0x1A, '\n'};
|
||||
|
||||
// A file read from its start on, a small block at a time.
|
||||
class Stream {
|
||||
public:
|
||||
Stream(const ImageRead& read, uint32_t size) : read_(read), size_(size) {}
|
||||
int get() {
|
||||
if (at_ >= have_) {
|
||||
if (next_ >= size_) return -1;
|
||||
have_ = read_(next_, buf_, std::min<size_t>(sizeof buf_, size_ - next_));
|
||||
at_ = 0;
|
||||
next_ += static_cast<uint32_t>(have_);
|
||||
if (!have_) return -1;
|
||||
}
|
||||
return buf_[at_++];
|
||||
}
|
||||
bool take(uint8_t* into, size_t len) {
|
||||
for (size_t i = 0; i < len; i++) {
|
||||
int c = get();
|
||||
if (c < 0) return false;
|
||||
into[i] = static_cast<uint8_t>(c);
|
||||
}
|
||||
return true;
|
||||
}
|
||||
bool skip(size_t len) {
|
||||
size_t buffered = std::min(len, have_ - at_);
|
||||
at_ += buffered;
|
||||
len -= buffered;
|
||||
if (len > size_ - next_) return false;
|
||||
next_ += static_cast<uint32_t>(len);
|
||||
return true;
|
||||
}
|
||||
|
||||
private:
|
||||
const ImageRead& read_;
|
||||
uint32_t size_, next_ = 0;
|
||||
uint8_t buf_[256];
|
||||
size_t have_ = 0, at_ = 0;
|
||||
};
|
||||
} // namespace
|
||||
|
||||
ImageKind imageKindOfName(const std::string& name) {
|
||||
std::string ext = extensionOf(name);
|
||||
if (ext == "png") return ImageKind::Png;
|
||||
if (ext == "jpg" || ext == "jpeg") return ImageKind::Jpeg;
|
||||
if (ext == "bmp") return ImageKind::Bmp;
|
||||
if (ext == "gif") return ImageKind::Gif;
|
||||
return ImageKind::None;
|
||||
}
|
||||
|
||||
ImageKind imageKindOfBytes(const uint8_t* head, size_t len) {
|
||||
if (len >= 8 && std::memcmp(head, kPngSignature, 8) == 0) return ImageKind::Png;
|
||||
if (len >= 3 && head[0] == 0xFF && head[1] == 0xD8 && head[2] == 0xFF) return ImageKind::Jpeg;
|
||||
if (len >= 6 && (std::memcmp(head, "GIF87a", 6) == 0 || std::memcmp(head, "GIF89a", 6) == 0)) return ImageKind::Gif;
|
||||
if (len >= 2 && head[0] == 'B' && head[1] == 'M') return ImageKind::Bmp;
|
||||
return ImageKind::None;
|
||||
}
|
||||
|
||||
const char* imageKindName(ImageKind kind) {
|
||||
switch (kind) {
|
||||
case ImageKind::Png: return "PNG";
|
||||
case ImageKind::Jpeg: return "JPEG";
|
||||
case ImageKind::Bmp: return "BMP";
|
||||
case ImageKind::Gif: return "GIF";
|
||||
default: return "";
|
||||
}
|
||||
}
|
||||
|
||||
std::string imageInfo(const ImageRead& read, uint32_t size, ImageInfo& out) {
|
||||
uint8_t head[32];
|
||||
size_t n = read(0, head, std::min<size_t>(sizeof head, size));
|
||||
out.kind = imageKindOfBytes(head, n);
|
||||
long w = 0, h = 0;
|
||||
switch (out.kind) {
|
||||
case ImageKind::Png:
|
||||
if (n < 24 || std::memcmp(head + 12, "IHDR", 4) != 0) return "This PNG is damaged";
|
||||
w = static_cast<long>(be32(head + 16));
|
||||
h = static_cast<long>(be32(head + 20));
|
||||
break;
|
||||
case ImageKind::Gif:
|
||||
if (n < 10) return "This GIF is damaged";
|
||||
w = static_cast<long>(le16(head + 6));
|
||||
h = static_cast<long>(le16(head + 8));
|
||||
break;
|
||||
case ImageKind::Bmp: {
|
||||
if (n < 26) return "This BMP is damaged";
|
||||
uint32_t dib = le32(head + 14);
|
||||
if (dib < 40) return "This kind of BMP can't be shown";
|
||||
w = static_cast<int32_t>(le32(head + 18));
|
||||
h = static_cast<int32_t>(le32(head + 22));
|
||||
if (h < 0) h = -h; // top row first
|
||||
break;
|
||||
}
|
||||
case ImageKind::Jpeg: {
|
||||
// Marker after marker until the one that carries the size.
|
||||
uint32_t at = 2;
|
||||
for (int guard = 0; guard < 4000; guard++) {
|
||||
uint8_t m[9];
|
||||
if (read(at, m, 4) != 4 || m[0] != 0xFF) return "This JPEG is damaged";
|
||||
uint8_t marker = m[1];
|
||||
if (marker == 0xFF) { // padding
|
||||
at++;
|
||||
continue;
|
||||
}
|
||||
if (marker == 0x01 || (marker >= 0xD0 && marker <= 0xD8)) { // no length
|
||||
at += 2;
|
||||
continue;
|
||||
}
|
||||
if (marker == 0xD9 || marker == 0xDA) return "This JPEG is damaged"; // the picture, and no size yet
|
||||
bool frame = marker >= 0xC0 && marker <= 0xCF && marker != 0xC4 && marker != 0xC8 && marker != 0xCC;
|
||||
if (frame) {
|
||||
if (read(at, m, 9) != 9) return "This JPEG is damaged";
|
||||
h = static_cast<long>(be16(m + 5));
|
||||
w = static_cast<long>(be16(m + 7));
|
||||
if (marker == 0xC2) return "A progressive JPEG can't be shown";
|
||||
if (marker != 0xC0 && marker != 0xC1) return "This kind of JPEG can't be shown";
|
||||
break;
|
||||
}
|
||||
at += 2 + be16(m + 2);
|
||||
}
|
||||
break;
|
||||
}
|
||||
default: return "Not a picture this can show";
|
||||
}
|
||||
if (w <= 0 || h <= 0) return "This picture is damaged";
|
||||
if (w > kMaxSide || h > kMaxSide) return "Too big: 16,384 pixels a side at most";
|
||||
out.width = static_cast<int>(w);
|
||||
out.height = static_cast<int>(h);
|
||||
return "";
|
||||
}
|
||||
|
||||
uint32_t screenshotPixelsAt(const ImageRead& read, uint32_t size, int width, int height) {
|
||||
if (width <= 0 || height <= 0 || size != png::Rgb332Writer::fileSize(width, height)) return 0;
|
||||
// Signature, IHDR, then a palette of 256 colours, then the one IDAT: a zlib header and a
|
||||
// single stored block.
|
||||
uint8_t ihdr[5], plte[8], idat[15];
|
||||
constexpr uint32_t kPlteAt = 8 + 25, kIdatAt = kPlteAt + 12 + 768;
|
||||
if (read(24, ihdr, 5) != 5 || ihdr[0] != 8 || ihdr[1] != 3 || ihdr[4] != 0) return 0;
|
||||
if (read(kPlteAt, plte, 8) != 8 || be32(plte) != 768 || std::memcmp(plte + 4, "PLTE", 4) != 0) return 0;
|
||||
if (read(kIdatAt, idat, 15) != 15 || std::memcmp(idat + 4, "IDAT", 4) != 0) return 0;
|
||||
uint32_t raw = static_cast<uint32_t>(width + 1) * static_cast<uint32_t>(height);
|
||||
if (idat[8] != 0x78 || idat[10] != 0x01 || le16(idat + 11) != raw || le16(idat + 13) != (raw ^ 0xFFFF)) return 0;
|
||||
return kIdatAt + 15 + 1; // past the first row's filter byte
|
||||
}
|
||||
|
||||
uint8_t rgb332Dithered(uint8_t r, uint8_t g, uint8_t b, int x, int y) {
|
||||
static const uint8_t kBayer[16] = {0, 8, 2, 10, 12, 4, 14, 6, 3, 11, 1, 9, 15, 7, 13, 5};
|
||||
int threshold = kBayer[((y & 3) << 2) | (x & 3)] * 16 + 8; // 8 to 248
|
||||
auto level = [threshold](int v, int top) {
|
||||
int nearest = (v * top + 127) / 255;
|
||||
if (nearest * 255 / top == v) return nearest; // a colour the screen has
|
||||
int low = v * top / 255, rest = v * top - low * 255;
|
||||
return rest > threshold ? low + 1 : low;
|
||||
};
|
||||
return static_cast<uint8_t>((level(r, 7) << 5) | (level(g, 7) << 2) | level(b, 3));
|
||||
}
|
||||
|
||||
bool ImageMap::at(int sx, int sy, int& tx, int& ty) const {
|
||||
if (sx < 0 || sy < 0) return false;
|
||||
if (scale >= 65536) {
|
||||
tx = sx + offX;
|
||||
ty = sy + offY;
|
||||
} else {
|
||||
uint64_t fx = static_cast<uint64_t>(sx) * scale, fy = static_cast<uint64_t>(sy) * scale;
|
||||
if ((fx & 0xFFFF) >= scale || (fy & 0xFFFF) >= scale) return false;
|
||||
tx = static_cast<int>(fx >> 16) + offX;
|
||||
ty = static_cast<int>(fy >> 16) + offY;
|
||||
}
|
||||
if (tx < 0 || ty < 0 || tx >= viewW || ty >= viewH) return false;
|
||||
tx += viewX;
|
||||
ty += viewY;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool ImageMap::rowUsed(int sy) const {
|
||||
if (sy < 0) return false;
|
||||
int ty;
|
||||
if (scale >= 65536) {
|
||||
ty = sy + offY;
|
||||
} else {
|
||||
uint64_t fy = static_cast<uint64_t>(sy) * scale;
|
||||
if ((fy & 0xFFFF) >= scale) return false;
|
||||
ty = static_cast<int>(fy >> 16) + offY;
|
||||
}
|
||||
return ty >= 0 && ty < viewH;
|
||||
}
|
||||
|
||||
bool ImageMap::below(int sy) const {
|
||||
if (sy < 0) return false;
|
||||
int ty = scale >= 65536 ? sy + offY : static_cast<int>((static_cast<uint64_t>(sy) * scale) >> 16) + offY;
|
||||
return ty >= viewH;
|
||||
}
|
||||
|
||||
ImageFrame::ImageFrame(int width, int height, int viewX, int viewY, int viewW, int viewH)
|
||||
: w_(std::max(1, width)), h_(std::max(1, height)), vx_(viewX), vy_(viewY), vw_(std::max(1, viewW)), vh_(std::max(1, viewH)) {}
|
||||
|
||||
uint32_t ImageFrame::fitScale() const {
|
||||
uint64_t sx = (static_cast<uint64_t>(vw_) << 16) / static_cast<uint64_t>(w_), sy = (static_cast<uint64_t>(vh_) << 16) / static_cast<uint64_t>(h_);
|
||||
return static_cast<uint32_t>(std::min<uint64_t>(65536, std::max<uint64_t>(1, std::min(sx, sy))));
|
||||
}
|
||||
|
||||
void ImageFrame::toggle() {
|
||||
if (!bigger()) return;
|
||||
actual_ = !actual_;
|
||||
if (actual_) { // the middle of it first
|
||||
panX_ = std::max(0, (w_ - vw_) / 2);
|
||||
panY_ = std::max(0, (h_ - vh_) / 2);
|
||||
}
|
||||
}
|
||||
|
||||
bool ImageFrame::pan(int dx, int dy) {
|
||||
if (!actual_) return false;
|
||||
int x = std::clamp(panX_ + dx * (vw_ / 2), 0, std::max(0, w_ - vw_));
|
||||
int y = std::clamp(panY_ + dy * (vh_ / 2), 0, std::max(0, h_ - vh_));
|
||||
bool moved = x != panX_ || y != panY_;
|
||||
panX_ = x;
|
||||
panY_ = y;
|
||||
return moved;
|
||||
}
|
||||
|
||||
int ImageFrame::percent() const { return actual_ ? 100 : static_cast<int>((static_cast<uint64_t>(fitScale()) * 100 + 32768) >> 16); }
|
||||
|
||||
int ImageFrame::jpegShrink() const {
|
||||
if (actual_) return 0;
|
||||
uint32_t scale = fitScale();
|
||||
int shrink = 0;
|
||||
while (shrink < 3 && (static_cast<uint64_t>(scale) << (shrink + 1)) <= 65536) shrink++;
|
||||
return shrink;
|
||||
}
|
||||
|
||||
ImageMap ImageFrame::map(int shrink) const {
|
||||
ImageMap m;
|
||||
m.viewX = vx_;
|
||||
m.viewY = vy_;
|
||||
m.viewW = vw_;
|
||||
m.viewH = vh_;
|
||||
if (actual_) {
|
||||
m.scale = 65536;
|
||||
m.offX = w_ <= vw_ ? (vw_ - w_) / 2 : -panX_;
|
||||
m.offY = h_ <= vh_ ? (vh_ - h_) / 2 : -panY_;
|
||||
return m;
|
||||
}
|
||||
uint32_t scale = fitScale();
|
||||
int tw = std::max<int>(1, static_cast<int>((static_cast<uint64_t>(w_) * scale) >> 16));
|
||||
int th = std::max<int>(1, static_cast<int>((static_cast<uint64_t>(h_) * scale) >> 16));
|
||||
m.offX = (vw_ - tw) / 2;
|
||||
m.offY = (vh_ - th) / 2;
|
||||
m.scale = static_cast<uint32_t>(std::min<uint64_t>(65536, static_cast<uint64_t>(scale) << shrink));
|
||||
return m;
|
||||
}
|
||||
|
||||
std::string readBmp(const ImageRead& read, uint32_t size, const ImagePixels& pixels, const std::function<bool(int y)>& rowNeeded) {
|
||||
uint8_t head[54];
|
||||
if (read(0, head, sizeof head) != sizeof head || head[0] != 'B' || head[1] != 'M') return "This BMP is damaged";
|
||||
uint32_t dataAt = le32(head + 10), dib = le32(head + 14), compression = le32(head + 30), colours = le32(head + 46);
|
||||
int32_t w = static_cast<int32_t>(le32(head + 18)), h = static_cast<int32_t>(le32(head + 22));
|
||||
uint32_t bits = le16(head + 28);
|
||||
bool topDown = h < 0;
|
||||
if (topDown) h = -h;
|
||||
if (dib < 40 || w <= 0 || h <= 0 || w > kMaxSide || h > kMaxSide) return "This kind of BMP can't be shown";
|
||||
if ((bits != 8 && bits != 24 && bits != 32) || (compression != 0 && !(compression == 3 && bits == 32))) return "This kind of BMP can't be shown";
|
||||
std::unique_ptr<uint8_t[]> palette;
|
||||
if (bits == 8) {
|
||||
if (!colours || colours > 256) colours = 256;
|
||||
palette.reset(new (std::nothrow) uint8_t[1024]());
|
||||
if (!palette) return "Not enough memory";
|
||||
if (read(14 + dib, palette.get(), colours * 4) != colours * 4) return "This BMP is damaged";
|
||||
}
|
||||
uint32_t bytes = bits / 8, rowSize = (static_cast<uint32_t>(w) * bytes + 3) & ~3u;
|
||||
if (static_cast<uint64_t>(dataAt) + static_cast<uint64_t>(rowSize) * static_cast<uint32_t>(h) > size) return "This BMP is cut short";
|
||||
// A row in one read when it fits, a piece of it at a time otherwise.
|
||||
constexpr int kOut = 64; // pixels handed on at once
|
||||
constexpr uint32_t kRowBuffer = 4096; // bytes
|
||||
uint32_t rowBytes = static_cast<uint32_t>(w) * bytes, bufSize = std::min(rowBytes, kRowBuffer);
|
||||
bufSize -= bufSize % bytes;
|
||||
std::unique_ptr<uint8_t[]> in(new (std::nothrow) uint8_t[bufSize]);
|
||||
if (!in) return "Not enough memory";
|
||||
uint8_t out[kOut * 3];
|
||||
// In the order the file has them, which is usually the last row first: going back through a
|
||||
// file on the card costs far more than going on (measured: a second for 135 rows).
|
||||
for (int stored = 0; stored < h; stored++) {
|
||||
int y = topDown ? stored : h - 1 - stored;
|
||||
if (rowNeeded && !rowNeeded(y)) continue;
|
||||
uint32_t rowAt = dataAt + rowSize * static_cast<uint32_t>(stored);
|
||||
for (uint32_t done = 0; done < rowBytes; done += bufSize) {
|
||||
size_t want = std::min(bufSize, rowBytes - done);
|
||||
if (read(rowAt + done, in.get(), want) != want) return "The card refused to read it";
|
||||
int first = static_cast<int>(done / bytes), count = static_cast<int>(want / bytes);
|
||||
for (int at = 0; at < count; at += kOut) {
|
||||
int n = std::min(kOut, count - at);
|
||||
for (int i = 0; i < n; i++) {
|
||||
const uint8_t* p = bits == 8 ? palette.get() + in[at + i] * 4 : in.get() + static_cast<size_t>(at + i) * bytes;
|
||||
out[i * 3] = p[2]; // stored blue, green, red
|
||||
out[i * 3 + 1] = p[1];
|
||||
out[i * 3 + 2] = p[0];
|
||||
}
|
||||
pixels(first + at, y, n, out);
|
||||
}
|
||||
}
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
namespace {
|
||||
struct GifWork {
|
||||
uint8_t palette[768];
|
||||
uint16_t prefix[4096];
|
||||
uint8_t suffix[4096], stack[4096];
|
||||
};
|
||||
} // namespace
|
||||
|
||||
std::string readGif(const ImageRead& read, uint32_t size, const ImagePixels& pixels) {
|
||||
Stream in(read, size);
|
||||
uint8_t head[13];
|
||||
if (!in.take(head, 13) || imageKindOfBytes(head, 6) != ImageKind::Gif) return "This GIF is damaged";
|
||||
int screenW = static_cast<int>(le16(head + 6)), screenH = static_cast<int>(le16(head + 8));
|
||||
std::unique_ptr<GifWork> work(new (std::nothrow) GifWork);
|
||||
if (!work) return "Not enough memory to show a GIF";
|
||||
std::memset(work->palette, 0, sizeof work->palette);
|
||||
if (head[10] & 0x80 && !in.take(work->palette, 3u << ((head[10] & 7) + 1))) return "This GIF is damaged";
|
||||
int transparent = -1;
|
||||
for (int guard = 0; guard < 100000; guard++) {
|
||||
int kind = in.get();
|
||||
if (kind == 0x21) { // an extension: only the one before a picture matters, for its transparent colour
|
||||
int label = in.get();
|
||||
for (bool first = true;; first = false) {
|
||||
int len = in.get();
|
||||
if (len < 0) return "This GIF is damaged";
|
||||
if (len == 0) break;
|
||||
uint8_t block[255];
|
||||
if (!in.take(block, static_cast<size_t>(len))) return "This GIF is damaged";
|
||||
if (label == 0xF9 && first && len >= 4) transparent = (block[0] & 1) ? block[3] : -1;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (kind != 0x2C) return kind == 0x3B ? "This GIF has no picture" : "This GIF is damaged";
|
||||
break;
|
||||
}
|
||||
uint8_t desc[9];
|
||||
if (!in.take(desc, 9)) return "This GIF is damaged";
|
||||
int left = static_cast<int>(le16(desc)), top = static_cast<int>(le16(desc + 2));
|
||||
int fw = static_cast<int>(le16(desc + 4)), fh = static_cast<int>(le16(desc + 6));
|
||||
bool interlaced = desc[8] & 0x40;
|
||||
if (desc[8] & 0x80 && !in.take(work->palette, 3u << ((desc[8] & 7) + 1))) return "This GIF is damaged";
|
||||
int minBits = in.get();
|
||||
if (fw <= 0 || fh <= 0 || minBits < 2 || minBits > 8) return "This GIF is damaged";
|
||||
|
||||
// The pixels come out in the order they are stored; an interlaced picture stores every eighth
|
||||
// row first, then the rows between, in four passes.
|
||||
static const int kStart[4] = {0, 4, 2, 1}, kStep[4] = {8, 8, 4, 2};
|
||||
int px = 0, row = 0, pass = 0, rowsDone = 0;
|
||||
uint8_t run[64 * 3];
|
||||
int runLen = 0, runX = 0;
|
||||
auto flush = [&]() {
|
||||
int y = top + row;
|
||||
if (runLen && y >= 0 && y < screenH) pixels(left + runX, y, runLen, run);
|
||||
runLen = 0;
|
||||
};
|
||||
auto put = [&](uint8_t index) {
|
||||
if (rowsDone >= fh) return;
|
||||
int x = left + px;
|
||||
if (index == transparent || x < 0 || x >= screenW) {
|
||||
flush();
|
||||
} else {
|
||||
if (!runLen) runX = px;
|
||||
std::memcpy(run + runLen * 3, work->palette + index * 3, 3);
|
||||
if (++runLen == 64) flush();
|
||||
}
|
||||
if (++px < fw) return;
|
||||
flush();
|
||||
px = 0;
|
||||
rowsDone++;
|
||||
if (!interlaced) {
|
||||
row++;
|
||||
return;
|
||||
}
|
||||
row += kStep[pass];
|
||||
while (row >= fh && pass < 3) row = kStart[++pass];
|
||||
};
|
||||
|
||||
const int clear = 1 << minBits, stop = clear + 1;
|
||||
int bits = minBits + 1, next = clear + 2, prev = -1, first = 0;
|
||||
uint32_t hold = 0;
|
||||
int held = 0, blockLeft = 0;
|
||||
bool ended = false;
|
||||
for (int i = 0; i < clear; i++) work->suffix[i] = static_cast<uint8_t>(i);
|
||||
while (rowsDone < fh && !ended) {
|
||||
while (held < bits) {
|
||||
if (!blockLeft) {
|
||||
blockLeft = in.get();
|
||||
if (blockLeft <= 0) {
|
||||
ended = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
int c = in.get();
|
||||
if (c < 0) return "This GIF is cut short";
|
||||
blockLeft--;
|
||||
hold |= static_cast<uint32_t>(c) << held;
|
||||
held += 8;
|
||||
}
|
||||
if (ended) break;
|
||||
int code = static_cast<int>(hold & ((1u << bits) - 1));
|
||||
hold >>= bits;
|
||||
held -= bits;
|
||||
if (code == clear) {
|
||||
bits = minBits + 1;
|
||||
next = clear + 2;
|
||||
prev = -1;
|
||||
continue;
|
||||
}
|
||||
if (code == stop) break;
|
||||
if (prev < 0) {
|
||||
if (code >= clear) return "This GIF is damaged";
|
||||
put(static_cast<uint8_t>(code));
|
||||
first = prev = code;
|
||||
continue;
|
||||
}
|
||||
if (code > next) return "This GIF is damaged";
|
||||
int sp = 0, walk = code;
|
||||
if (code == next) { // the string being defined: the one before, and its own first pixel again
|
||||
work->stack[sp++] = static_cast<uint8_t>(first);
|
||||
walk = prev;
|
||||
}
|
||||
while (walk >= clear && sp < 4095) {
|
||||
work->stack[sp++] = work->suffix[walk];
|
||||
walk = work->prefix[walk];
|
||||
}
|
||||
if (walk >= clear) return "This GIF is damaged";
|
||||
first = walk;
|
||||
work->stack[sp++] = static_cast<uint8_t>(walk);
|
||||
if (next < 4096) {
|
||||
work->prefix[next] = static_cast<uint16_t>(prev);
|
||||
work->suffix[next] = static_cast<uint8_t>(first);
|
||||
next++;
|
||||
if (next == (1 << bits) && bits < 12) bits++;
|
||||
}
|
||||
prev = code;
|
||||
while (sp) put(work->stack[--sp]);
|
||||
}
|
||||
flush();
|
||||
return rowsDone ? "" : "This GIF is damaged";
|
||||
}
|
||||
|
||||
} // namespace roro::files
|
||||
@@ -0,0 +1,86 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <functional>
|
||||
#include <string>
|
||||
|
||||
// Pictures for the Storage App (issue #45, F1 Q233-Q242): what a file is and how big, where each
|
||||
// of its pixels goes on the screen, and the readers for BMP and the first frame of a GIF. PNG is in
|
||||
// png_reader.h; JPEG is decoded on the device by the display library's decoder.
|
||||
// Nothing here holds a picture: every reader hands its pixels on as it gets them.
|
||||
namespace roro::files {
|
||||
|
||||
enum class ImageKind : uint8_t { None, Png, Jpeg, Bmp, Gif };
|
||||
|
||||
// Reads up to `len` bytes at `offset`; returns how many it got.
|
||||
using ImageRead = std::function<size_t(uint32_t offset, uint8_t* into, size_t len)>;
|
||||
// `count` pixels of row `y` from column `x` on, three bytes each: red, green, blue.
|
||||
using ImagePixels = std::function<void(int x, int y, int count, const uint8_t* rgb)>;
|
||||
|
||||
ImageKind imageKindOfName(const std::string& name); // by its extension
|
||||
ImageKind imageKindOfBytes(const uint8_t* head, size_t len); // by its first bytes (8 are enough)
|
||||
const char* imageKindName(ImageKind kind);
|
||||
|
||||
struct ImageInfo {
|
||||
ImageKind kind = ImageKind::None;
|
||||
int width = 0, height = 0;
|
||||
};
|
||||
// "" and `out` filled, or why the file can't be shown.
|
||||
std::string imageInfo(const ImageRead& read, uint32_t size, ImageInfo& out);
|
||||
|
||||
// A PNG the firmware's `screenshot` wrote (png_rgb332.h): its pixels are not compressed and each
|
||||
// is already a colour of the screen. Where the first row's pixels start, or 0 if it isn't one.
|
||||
// Row y's pixels are at that offset + y * (width + 1).
|
||||
uint32_t screenshotPixelsAt(const ImageRead& read, uint32_t size, int width, int height);
|
||||
|
||||
// A colour as the screen has it (RRRGGGBB), dithered by where it lands: the screen has 8 levels of
|
||||
// red and green and 4 of blue, and a photograph bands without it. A colour the screen has exactly
|
||||
// comes out as itself wherever it lands, so a screenshot isn't touched.
|
||||
uint8_t rgb332Dithered(uint8_t r, uint8_t g, uint8_t b, int x, int y);
|
||||
|
||||
// From a picture's pixels to the screen's.
|
||||
struct ImageMap {
|
||||
int viewX = 0, viewY = 0, viewW = 0, viewH = 0; // the part of the screen the picture may use
|
||||
int offX = 0, offY = 0; // the picture's corner in it (negative: scrolled)
|
||||
uint32_t scale = 65536; // screen pixels for one of the picture's, 16.16; never over 1
|
||||
|
||||
// Where the picture's pixel lands. False if it's outside the view, or if another pixel is the
|
||||
// one drawn there (shrunk, each screen pixel takes the first of the picture's that falls on it).
|
||||
bool at(int sx, int sy, int& tx, int& ty) const;
|
||||
bool rowUsed(int sy) const; // does any pixel of this row land?
|
||||
bool below(int sy) const; // this row and every one after it land under the view: nothing more to draw
|
||||
};
|
||||
|
||||
// How a picture is looked at: whole, shrunk to fit if it has to be; or at its own size, a
|
||||
// screenful at a time.
|
||||
class ImageFrame {
|
||||
public:
|
||||
ImageFrame() = default;
|
||||
ImageFrame(int width, int height, int viewX, int viewY, int viewW, int viewH);
|
||||
|
||||
bool bigger() const { return w_ > vw_ || h_ > vh_; } // than the view: there is something to zoom
|
||||
bool actual() const { return actual_; }
|
||||
void toggle();
|
||||
bool pan(int dx, int dy); // half a view a step, at its own size only; false: nothing moved
|
||||
int percent() const; // of its own size, as shown
|
||||
int jpegShrink() const; // 0 to 3: the halvings a JPEG decoder may do first, the picture still at least as big as shown
|
||||
// For pixels counted after `shrink` halvings (a JPEG's), or the picture's own.
|
||||
ImageMap map(int shrink = 0) const;
|
||||
|
||||
private:
|
||||
uint32_t fitScale() const;
|
||||
|
||||
int w_ = 0, h_ = 0, vx_ = 0, vy_ = 0, vw_ = 1, vh_ = 1, panX_ = 0, panY_ = 0;
|
||||
bool actual_ = false;
|
||||
};
|
||||
|
||||
// A BMP: 8 bits with a palette, 24 or 32 bits, not compressed. `rowNeeded` lets rows be skipped
|
||||
// without being read. "" or why it can't be shown.
|
||||
std::string readBmp(const ImageRead& read, uint32_t size, const ImagePixels& pixels, const std::function<bool(int y)>& rowNeeded = nullptr);
|
||||
|
||||
// The first picture of a GIF, interlaced or not; transparent pixels are not handed on. It needs
|
||||
// 17 KB while it runs. "" or why it can't be shown.
|
||||
std::string readGif(const ImageRead& read, uint32_t size, const ImagePixels& pixels);
|
||||
|
||||
} // namespace roro::files
|
||||
@@ -0,0 +1,354 @@
|
||||
#include "png_reader.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <cstring>
|
||||
#include <memory>
|
||||
#include <new>
|
||||
|
||||
namespace roro::files {
|
||||
|
||||
namespace {
|
||||
uint32_t be32(const uint8_t* p) { return (static_cast<uint32_t>(p[0]) << 24) | (p[1] << 16) | (p[2] << 8) | p[3]; }
|
||||
|
||||
const char* const kDamaged = "This PNG is damaged";
|
||||
const char* const kCut = "This PNG is cut short";
|
||||
const char* const kNoMemory = "Not enough memory for this PNG";
|
||||
|
||||
// A Huffman code as its lengths say: how many codes of each length, and the symbols in order.
|
||||
struct Huffman {
|
||||
uint16_t count[16];
|
||||
uint16_t symbol[288];
|
||||
|
||||
// False if the lengths don't make a code.
|
||||
bool build(const uint8_t* lengths, int n) {
|
||||
std::memset(count, 0, sizeof count);
|
||||
for (int i = 0; i < n; i++) count[lengths[i]]++;
|
||||
int left = 1;
|
||||
for (int len = 1; len < 16; len++) {
|
||||
left = (left << 1) - count[len];
|
||||
if (left < 0) return false;
|
||||
}
|
||||
uint16_t offs[16];
|
||||
offs[1] = 0;
|
||||
for (int len = 1; len < 15; len++) offs[len + 1] = static_cast<uint16_t>(offs[len] + count[len]);
|
||||
for (int i = 0; i < n; i++)
|
||||
if (lengths[i]) symbol[offs[lengths[i]]++] = static_cast<uint16_t>(i);
|
||||
return true;
|
||||
}
|
||||
};
|
||||
|
||||
// Everything one decoding holds, but the window and the two rows: on the heap, in one piece.
|
||||
struct Work {
|
||||
// The file, and the IDAT chunks as one stream of bytes.
|
||||
const ImageRead* read = nullptr;
|
||||
uint32_t size = 0, at = 0, chunkLeft = 0;
|
||||
uint8_t in[256];
|
||||
size_t inHave = 0, inAt = 0;
|
||||
bool inEnd = false;
|
||||
// Bits.
|
||||
uint32_t hold = 0;
|
||||
int held = 0;
|
||||
// The picture.
|
||||
int width = 0, height = 0, depth = 0, type = 0, channels = 0, bpp = 0;
|
||||
uint32_t rowBytes = 0;
|
||||
uint8_t palette[768], alpha[256];
|
||||
bool hasAlpha = false;
|
||||
// The window, the rows, and where the decoding is.
|
||||
std::unique_ptr<uint8_t[]> window, rows;
|
||||
uint32_t windowSize = 0, written = 0;
|
||||
uint8_t *cur = nullptr, *prev = nullptr;
|
||||
int64_t pos = -1; // in the row; -1: its filter byte comes next
|
||||
int filter = 0, y = 0;
|
||||
bool stop = false;
|
||||
const char* problem = nullptr;
|
||||
const ImagePixels* pixels = nullptr;
|
||||
const std::function<bool(int)>* rowNeeded = nullptr;
|
||||
const std::function<bool(int)>* enough = nullptr;
|
||||
Huffman lengths, distances;
|
||||
uint8_t codeLengths[320];
|
||||
|
||||
int byte() {
|
||||
if (inAt >= inHave) {
|
||||
while (!chunkLeft && !inEnd) { // the next IDAT, past this one's checksum
|
||||
uint8_t head[12];
|
||||
if (at + 12 > size || (*read)(at, head, 12) != 12) return inEnd = true, -1;
|
||||
at += 4; // the checksum
|
||||
if (std::memcmp(head + 8, "IDAT", 4) != 0) return inEnd = true, -1;
|
||||
chunkLeft = be32(head + 4);
|
||||
at += 8;
|
||||
}
|
||||
if (inEnd) return -1;
|
||||
size_t want = std::min<size_t>(sizeof in, chunkLeft);
|
||||
inHave = (*read)(at, in, want);
|
||||
inAt = 0;
|
||||
if (inHave != want) return inEnd = true, -1;
|
||||
at += static_cast<uint32_t>(want);
|
||||
chunkLeft -= static_cast<uint32_t>(want);
|
||||
}
|
||||
return in[inAt++];
|
||||
}
|
||||
int bits(int n) { // -1: no more
|
||||
while (held < n) {
|
||||
int b = byte();
|
||||
if (b < 0) return -1;
|
||||
hold |= static_cast<uint32_t>(b) << held;
|
||||
held += 8;
|
||||
}
|
||||
int v = static_cast<int>(hold & ((1u << n) - 1));
|
||||
hold >>= n;
|
||||
held -= n;
|
||||
return v;
|
||||
}
|
||||
int decode(const Huffman& h) { // -1: no more, or not a code
|
||||
int code = 0, first = 0, index = 0;
|
||||
for (int len = 1; len < 16; len++) {
|
||||
int b = bits(1);
|
||||
if (b < 0) return -1;
|
||||
code |= b;
|
||||
int n = h.count[len];
|
||||
if (code - n < first) return h.symbol[index + (code - first)];
|
||||
index += n;
|
||||
first += n;
|
||||
first <<= 1;
|
||||
code <<= 1;
|
||||
}
|
||||
return -1;
|
||||
}
|
||||
|
||||
void row();
|
||||
// One byte out of the decompression: into the window, and into the row being rebuilt.
|
||||
void out(uint8_t b) {
|
||||
window[written++ & (windowSize - 1)] = b;
|
||||
if (pos < 0) {
|
||||
if (b > 4) {
|
||||
problem = kDamaged;
|
||||
stop = true;
|
||||
}
|
||||
filter = b;
|
||||
pos = 0;
|
||||
return;
|
||||
}
|
||||
uint32_t i = static_cast<uint32_t>(pos);
|
||||
int a = i >= static_cast<uint32_t>(bpp) ? cur[i - bpp] : 0, up = prev[i], c = i >= static_cast<uint32_t>(bpp) ? prev[i - bpp] : 0, add = 0;
|
||||
switch (filter) {
|
||||
case 1: add = a; break;
|
||||
case 2: add = up; break;
|
||||
case 3: add = (a + up) >> 1; break;
|
||||
case 4: {
|
||||
int p = a + up - c, pa = std::abs(p - a), pb = std::abs(p - up), pc = std::abs(p - c);
|
||||
add = pa <= pb && pa <= pc ? a : pb <= pc ? up : c;
|
||||
break;
|
||||
}
|
||||
default: break;
|
||||
}
|
||||
cur[i] = static_cast<uint8_t>(b + add);
|
||||
if (static_cast<uint32_t>(++pos) < rowBytes) return;
|
||||
if (!rowNeeded || !*rowNeeded || (*rowNeeded)(y)) row();
|
||||
std::swap(cur, prev);
|
||||
pos = -1;
|
||||
y++;
|
||||
if (y >= height || (enough && *enough && (*enough)(y))) stop = true;
|
||||
}
|
||||
bool inflate();
|
||||
};
|
||||
|
||||
// The row as colours: runs of pixels, broken where one is transparent.
|
||||
void Work::row() {
|
||||
uint8_t run[64 * 3];
|
||||
int n = 0, from = 0;
|
||||
auto flush = [&]() {
|
||||
if (n) (*pixels)(from, y, n, run);
|
||||
n = 0;
|
||||
};
|
||||
int top = (1 << depth) - 1;
|
||||
for (int x = 0; x < width; x++) {
|
||||
uint8_t r, g, b, a = 255;
|
||||
auto sample = [&](int k) -> int { // the k-th value of this pixel, as 8 bits; an index stays an index
|
||||
if (depth == 8) return cur[x * channels + k];
|
||||
if (depth == 16) return cur[(x * channels + k) * 2];
|
||||
int bit = x * depth, v = (cur[bit >> 3] >> (8 - depth - (bit & 7))) & top;
|
||||
return type == 3 ? v : v * 255 / top;
|
||||
};
|
||||
switch (type) {
|
||||
case 0: r = g = b = static_cast<uint8_t>(sample(0)); break;
|
||||
case 2: r = static_cast<uint8_t>(sample(0)), g = static_cast<uint8_t>(sample(1)), b = static_cast<uint8_t>(sample(2)); break;
|
||||
case 3: {
|
||||
int i = sample(0);
|
||||
r = palette[i * 3], g = palette[i * 3 + 1], b = palette[i * 3 + 2];
|
||||
if (hasAlpha) a = alpha[i];
|
||||
break;
|
||||
}
|
||||
case 4: r = g = b = static_cast<uint8_t>(sample(0)), a = static_cast<uint8_t>(sample(1)); break;
|
||||
default: r = static_cast<uint8_t>(sample(0)), g = static_cast<uint8_t>(sample(1)), b = static_cast<uint8_t>(sample(2)), a = static_cast<uint8_t>(sample(3)); break;
|
||||
}
|
||||
if (a < 128) {
|
||||
flush();
|
||||
continue;
|
||||
}
|
||||
if (!n) from = x;
|
||||
run[n * 3] = r, run[n * 3 + 1] = g, run[n * 3 + 2] = b;
|
||||
if (++n == 64) flush();
|
||||
}
|
||||
flush();
|
||||
}
|
||||
|
||||
// Deflate (RFC 1951) inside a zlib stream (RFC 1950). False with `problem` set, or true when the
|
||||
// stream ended or enough rows were made.
|
||||
bool Work::inflate() {
|
||||
static const uint16_t kLenBase[29] = {3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 15, 17, 19, 23, 27, 31, 35, 43, 51, 59, 67, 83, 99, 115, 131, 163, 195, 227, 258};
|
||||
static const uint8_t kLenExtra[29] = {0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 5, 5, 5, 5, 0};
|
||||
static const uint16_t kDistBase[30] = {1, 2, 3, 4, 5, 7, 9, 13, 17, 25, 33, 49, 65, 97, 129, 193, 257, 385, 513, 769, 1025, 1537, 2049, 3073, 4097, 6145, 8193, 12289, 16385, 24577};
|
||||
static const uint8_t kDistExtra[30] = {0, 0, 0, 0, 1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, 8, 8, 9, 9, 10, 10, 11, 11, 12, 12, 13, 13};
|
||||
static const uint8_t kOrder[19] = {16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15};
|
||||
auto fail = [this](const char* what) {
|
||||
if (!problem) problem = what;
|
||||
return false;
|
||||
};
|
||||
for (bool last = false; !last && !stop;) {
|
||||
int head = bits(3);
|
||||
if (head < 0) return fail(kCut);
|
||||
last = head & 1;
|
||||
int kind = head >> 1;
|
||||
if (kind == 0) { // stored
|
||||
hold = 0;
|
||||
held = 0;
|
||||
int a = byte(), b = byte(), c = byte(), d = byte();
|
||||
if (d < 0) return fail(kCut);
|
||||
int len = a | (b << 8);
|
||||
if (len != ((c | (d << 8)) ^ 0xFFFF)) return fail(kDamaged);
|
||||
for (int i = 0; i < len && !stop; i++) {
|
||||
int v = byte();
|
||||
if (v < 0) return fail(kCut);
|
||||
out(static_cast<uint8_t>(v));
|
||||
}
|
||||
continue;
|
||||
}
|
||||
if (kind == 3) return fail(kDamaged);
|
||||
if (kind == 1) { // the code every decoder knows
|
||||
for (int i = 0; i < 288; i++) codeLengths[i] = i < 144 ? 8 : i < 256 ? 9 : i < 280 ? 7 : 8;
|
||||
lengths.build(codeLengths, 288);
|
||||
for (int i = 0; i < 30; i++) codeLengths[i] = 5;
|
||||
distances.build(codeLengths, 30);
|
||||
} else { // a code of the block's own, itself sent coded
|
||||
int nlen = bits(5), ndist = bits(5), ncode = bits(4);
|
||||
if (ncode < 0) return fail(kCut);
|
||||
nlen += 257, ndist += 1, ncode += 4;
|
||||
if (nlen > 286 || ndist > 30) return fail(kDamaged);
|
||||
uint8_t first[19] = {0};
|
||||
for (int i = 0; i < ncode; i++) {
|
||||
int v = bits(3);
|
||||
if (v < 0) return fail(kCut);
|
||||
first[kOrder[i]] = static_cast<uint8_t>(v);
|
||||
}
|
||||
if (!lengths.build(first, 19)) return fail(kDamaged);
|
||||
for (int i = 0; i < nlen + ndist;) {
|
||||
int sym = decode(lengths);
|
||||
if (sym < 0) return fail(kDamaged);
|
||||
if (sym < 16) {
|
||||
codeLengths[i++] = static_cast<uint8_t>(sym);
|
||||
continue;
|
||||
}
|
||||
int repeat, value = 0;
|
||||
if (sym == 16) {
|
||||
if (!i) return fail(kDamaged);
|
||||
value = codeLengths[i - 1];
|
||||
repeat = 3 + bits(2);
|
||||
} else if (sym == 17) {
|
||||
repeat = 3 + bits(3);
|
||||
} else {
|
||||
repeat = 11 + bits(7);
|
||||
}
|
||||
if (i + repeat > nlen + ndist) return fail(kDamaged);
|
||||
while (repeat--) codeLengths[i++] = static_cast<uint8_t>(value);
|
||||
}
|
||||
uint8_t dist[30];
|
||||
std::memcpy(dist, codeLengths + nlen, static_cast<size_t>(ndist));
|
||||
if (!lengths.build(codeLengths, nlen) || !distances.build(dist, ndist)) return fail(kDamaged);
|
||||
}
|
||||
while (!stop) {
|
||||
int sym = decode(lengths);
|
||||
if (sym < 0) return fail(inEnd ? kCut : kDamaged);
|
||||
if (sym < 256) {
|
||||
out(static_cast<uint8_t>(sym));
|
||||
continue;
|
||||
}
|
||||
if (sym == 256) break;
|
||||
sym -= 257;
|
||||
if (sym >= 29) return fail(kDamaged);
|
||||
int extra = bits(kLenExtra[sym]);
|
||||
int dsym = decode(distances);
|
||||
if (extra < 0 || dsym < 0 || dsym >= 30) return fail(inEnd ? kCut : kDamaged);
|
||||
int dextra = bits(kDistExtra[dsym]);
|
||||
if (dextra < 0) return fail(kCut);
|
||||
uint32_t len = static_cast<uint32_t>(kLenBase[sym] + extra), dist = static_cast<uint32_t>(kDistBase[dsym] + dextra);
|
||||
if (dist > written || dist > windowSize) return fail(kDamaged);
|
||||
for (uint32_t i = 0; i < len && !stop; i++) out(window[(written - dist) & (windowSize - 1)]);
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
std::string readPng(const ImageRead& read, uint32_t size, const ImagePixels& pixels, const std::function<bool(int y)>& rowNeeded,
|
||||
const std::function<bool(int y)>& enough) {
|
||||
static const uint8_t kSignature[] = {0x89, 'P', 'N', 'G', '\r', '\n', 0x1A, '\n'};
|
||||
uint8_t head[33];
|
||||
if (read(0, head, sizeof head) != sizeof head || std::memcmp(head, kSignature, 8) != 0 || std::memcmp(head + 12, "IHDR", 4) != 0) return kDamaged;
|
||||
std::unique_ptr<Work> w(new (std::nothrow) Work);
|
||||
if (!w) return kNoMemory;
|
||||
w->read = &read;
|
||||
w->size = size;
|
||||
w->pixels = &pixels;
|
||||
w->rowNeeded = &rowNeeded;
|
||||
w->enough = &enough;
|
||||
uint32_t width = be32(head + 16), height = be32(head + 20);
|
||||
w->depth = head[24];
|
||||
w->type = head[25];
|
||||
if (!width || !height || width > 16384 || height > 16384 || head[26] || head[27]) return kDamaged;
|
||||
if (head[28]) return "An interlaced PNG can't be shown";
|
||||
w->width = static_cast<int>(width);
|
||||
w->height = static_cast<int>(height);
|
||||
static const int8_t kChannels[7] = {1, 0, 3, 1, 2, 0, 4};
|
||||
int depth = w->depth, type = w->type;
|
||||
bool depthOk = depth == 8 || (depth == 16 && type != 3) || ((depth == 1 || depth == 2 || depth == 4) && (type == 0 || type == 3));
|
||||
if (type > 6 || !kChannels[type] || !depthOk) return "This kind of PNG can't be shown";
|
||||
w->channels = kChannels[type];
|
||||
w->bpp = std::max(1, w->channels * depth / 8);
|
||||
w->rowBytes = (width * static_cast<uint32_t>(w->channels * depth) + 7) / 8;
|
||||
std::memset(w->palette, 0, sizeof w->palette);
|
||||
std::memset(w->alpha, 255, sizeof w->alpha);
|
||||
|
||||
// The chunks before the picture: the palette and its transparency.
|
||||
uint32_t at = 33;
|
||||
for (int guard = 0; guard < 1000; guard++) {
|
||||
uint8_t c[8];
|
||||
if (at + 8 > size || read(at, c, 8) != 8) return kCut;
|
||||
uint32_t len = be32(c);
|
||||
if (std::memcmp(c + 4, "IDAT", 4) == 0) break;
|
||||
if (std::memcmp(c + 4, "IEND", 4) == 0 || len > size) return kDamaged;
|
||||
if (std::memcmp(c + 4, "PLTE", 4) == 0 && read(at + 8, w->palette, std::min<size_t>(len, 768)) != std::min<size_t>(len, 768)) return kCut;
|
||||
if (std::memcmp(c + 4, "tRNS", 4) == 0 && type == 3) {
|
||||
if (read(at + 8, w->alpha, std::min<size_t>(len, 256)) != std::min<size_t>(len, 256)) return kCut;
|
||||
w->hasAlpha = true;
|
||||
}
|
||||
at += 12 + len;
|
||||
}
|
||||
w->at = at - 4; // as if a chunk's checksum had just been reached: byte() steps over it to the IDAT
|
||||
|
||||
// The zlib header says how far back the data refers: the window is that big and no bigger.
|
||||
int cmf = w->byte(), flg = w->byte();
|
||||
if (flg < 0) return kCut;
|
||||
if ((cmf & 0x0F) != 8 || (cmf >> 4) > 7 || ((cmf << 8) | flg) % 31 || (flg & 0x20)) return kDamaged;
|
||||
w->windowSize = 1u << ((cmf >> 4) + 8);
|
||||
w->window.reset(new (std::nothrow) uint8_t[w->windowSize]);
|
||||
w->rows.reset(new (std::nothrow) uint8_t[static_cast<size_t>(w->rowBytes) * 2]());
|
||||
if (!w->window || !w->rows) return kNoMemory;
|
||||
w->cur = w->rows.get();
|
||||
w->prev = w->rows.get() + w->rowBytes;
|
||||
if (enough && enough(0)) return "";
|
||||
if (!w->inflate()) return w->problem ? w->problem : kDamaged;
|
||||
if (w->problem) return w->problem;
|
||||
return w->stop ? "" : kCut; // the data ended before the last row
|
||||
}
|
||||
|
||||
} // namespace roro::files
|
||||
@@ -0,0 +1,20 @@
|
||||
#pragma once
|
||||
|
||||
#include "image_file.h"
|
||||
|
||||
namespace roro::files {
|
||||
|
||||
// A PNG, decoded a row at a time (issue #45): every colour type and bit depth, not interlaced.
|
||||
// Pixels that are mostly transparent are not handed on. `rowNeeded` lets rows be left out (they
|
||||
// are still decoded: a row is stored as its difference from the one before); `enough` says that
|
||||
// from this row on nothing is wanted, and the decoding stops there.
|
||||
//
|
||||
// Memory while it runs: the window the file's compression refers back into (what its header asks
|
||||
// for, 32 KB at most), two rows of the picture, and about 3 KB. The display library's decoder
|
||||
// wanted 44 KB in one block, which this device often doesn't have.
|
||||
//
|
||||
// "" or why it can't be shown.
|
||||
std::string readPng(const ImageRead& read, uint32_t size, const ImagePixels& pixels, const std::function<bool(int y)>& rowNeeded = nullptr,
|
||||
const std::function<bool(int y)>& enough = nullptr);
|
||||
|
||||
} // namespace roro::files
|
||||
@@ -0,0 +1,118 @@
|
||||
#include "png_rgb332.h"
|
||||
|
||||
namespace roro::png {
|
||||
|
||||
uint32_t crc32(uint32_t crc, const uint8_t* data, size_t len) {
|
||||
crc = ~crc;
|
||||
for (size_t i = 0; i < len; i++) {
|
||||
crc ^= data[i];
|
||||
for (int bit = 0; bit < 8; bit++) crc = (crc >> 1) ^ (0xEDB88320u & (0u - (crc & 1)));
|
||||
}
|
||||
return ~crc;
|
||||
}
|
||||
|
||||
uint32_t adler32(uint32_t adler, const uint8_t* data, size_t len) {
|
||||
uint32_t a = adler & 0xFFFF, b = adler >> 16;
|
||||
for (size_t i = 0; i < len; i++) {
|
||||
a = (a + data[i]) % 65521;
|
||||
b = (b + a) % 65521;
|
||||
}
|
||||
return (b << 16) | a;
|
||||
}
|
||||
|
||||
namespace {
|
||||
|
||||
void be32(uint8_t* out, uint32_t v) {
|
||||
out[0] = static_cast<uint8_t>(v >> 24);
|
||||
out[1] = static_cast<uint8_t>(v >> 16);
|
||||
out[2] = static_cast<uint8_t>(v >> 8);
|
||||
out[3] = static_cast<uint8_t>(v);
|
||||
}
|
||||
|
||||
size_t rawSize(int w, int h) { return static_cast<size_t>(w + 1) * h; } // a filter byte before each row
|
||||
size_t idatSize(int w, int h) { return 2 + 5 + rawSize(w, h) + 4; } // zlib header, block header, data, adler
|
||||
|
||||
} // namespace
|
||||
|
||||
size_t Rgb332Writer::fileSize(int w, int h) {
|
||||
return 8 + (12 + 13) + (12 + 768) + (12 + idatSize(w, h)) + 12; // signature, IHDR, PLTE, IDAT, IEND
|
||||
}
|
||||
|
||||
bool Rgb332Writer::put(const uint8_t* data, size_t len, bool inIdat) {
|
||||
if (inIdat) crc_ = crc32(crc_, data, len);
|
||||
return sink_(data, len);
|
||||
}
|
||||
|
||||
bool Rgb332Writer::put32(uint32_t value, bool inIdat) {
|
||||
uint8_t b[4];
|
||||
be32(b, value);
|
||||
return put(b, 4, inIdat);
|
||||
}
|
||||
|
||||
bool Rgb332Writer::begin() {
|
||||
if (w_ <= 0 || h_ <= 0 || rawSize(w_, h_) > 65535) return false;
|
||||
static const uint8_t signature[] = {0x89, 'P', 'N', 'G', '\r', '\n', 0x1A, '\n'};
|
||||
if (!sink_(signature, sizeof signature)) return false;
|
||||
|
||||
uint8_t ihdr[4 + 13] = {'I', 'H', 'D', 'R'};
|
||||
be32(ihdr + 4, static_cast<uint32_t>(w_));
|
||||
be32(ihdr + 8, static_cast<uint32_t>(h_));
|
||||
ihdr[12] = 8; // bits a pixel
|
||||
ihdr[13] = 3; // indexed colour
|
||||
ihdr[14] = ihdr[15] = ihdr[16] = 0;
|
||||
uint8_t word[4];
|
||||
be32(word, 13);
|
||||
if (!sink_(word, 4) || !sink_(ihdr, sizeof ihdr)) return false;
|
||||
be32(word, crc32(0, ihdr, sizeof ihdr));
|
||||
if (!sink_(word, 4)) return false;
|
||||
|
||||
// The palette: every RGB332 value is its own index, as scripts/rdbg.py expands them. Sixteen
|
||||
// colours at a time: this runs on a task with a small stack.
|
||||
be32(word, 768);
|
||||
const uint8_t plteKind[] = {'P', 'L', 'T', 'E'};
|
||||
if (!sink_(word, 4) || !sink_(plteKind, 4)) return false;
|
||||
uint32_t plteCrc = crc32(0, plteKind, 4);
|
||||
for (int first = 0; first < 256; first += 16) {
|
||||
uint8_t piece[48];
|
||||
for (int i = 0; i < 16; i++) {
|
||||
int v = first + i;
|
||||
piece[i * 3] = static_cast<uint8_t>((v >> 5) * 255 / 7);
|
||||
piece[i * 3 + 1] = static_cast<uint8_t>(((v >> 2) & 7) * 255 / 7);
|
||||
piece[i * 3 + 2] = static_cast<uint8_t>((v & 3) * 255 / 3);
|
||||
}
|
||||
plteCrc = crc32(plteCrc, piece, sizeof piece);
|
||||
if (!sink_(piece, sizeof piece)) return false;
|
||||
}
|
||||
be32(word, plteCrc);
|
||||
if (!sink_(word, 4)) return false;
|
||||
|
||||
// IDAT: a zlib stream of one stored block. Its length is known, so it can be written first.
|
||||
size_t raw = rawSize(w_, h_);
|
||||
be32(word, static_cast<uint32_t>(idatSize(w_, h_)));
|
||||
if (!sink_(word, 4)) return false;
|
||||
crc_ = 0;
|
||||
const uint8_t head[] = {'I', 'D', 'A', 'T', 0x78, 0x01, 0x01, static_cast<uint8_t>(raw), static_cast<uint8_t>(raw >> 8),
|
||||
static_cast<uint8_t>(~raw), static_cast<uint8_t>(~raw >> 8)};
|
||||
return put(head, sizeof head, true);
|
||||
}
|
||||
|
||||
bool Rgb332Writer::row(const uint8_t* pixels) {
|
||||
if (rows_ >= h_) return false;
|
||||
rows_++;
|
||||
const uint8_t filter = 0; // none
|
||||
adler_ = adler32(adler_, &filter, 1);
|
||||
adler_ = adler32(adler_, pixels, static_cast<size_t>(w_));
|
||||
return put(&filter, 1, true) && put(pixels, static_cast<size_t>(w_), true);
|
||||
}
|
||||
|
||||
bool Rgb332Writer::end() {
|
||||
if (rows_ != h_) return false;
|
||||
if (!put32(adler_, true)) return false;
|
||||
uint8_t word[4];
|
||||
be32(word, crc_);
|
||||
if (!sink_(word, 4)) return false;
|
||||
static const uint8_t iend[] = {0, 0, 0, 0, 'I', 'E', 'N', 'D', 0xAE, 0x42, 0x60, 0x82};
|
||||
return sink_(iend, sizeof iend);
|
||||
}
|
||||
|
||||
} // namespace roro::png
|
||||
@@ -0,0 +1,38 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <functional>
|
||||
|
||||
// A PNG of the screen, written a row at a time with almost no memory (issue #67, Q209): 8-bit
|
||||
// indexed colour with the 256 colours of RGB332 as its palette, and the pixels stored, not
|
||||
// compressed (a "stored" deflate block), so there is nothing to compress with and nothing to buffer.
|
||||
// One block holds at most 65,535 bytes: enough for the 240 x 135 screen (32,535 with its row bytes).
|
||||
namespace roro::png {
|
||||
|
||||
uint32_t crc32(uint32_t crc, const uint8_t* data, size_t len); // running; start from 0
|
||||
uint32_t adler32(uint32_t adler, const uint8_t* data, size_t len); // running; start from 1
|
||||
|
||||
class Rgb332Writer {
|
||||
public:
|
||||
using Sink = std::function<bool(const uint8_t* data, size_t len)>; // false: writing failed
|
||||
|
||||
Rgb332Writer(int width, int height, Sink sink) : w_(width), h_(height), sink_(std::move(sink)) {}
|
||||
|
||||
// The file's size, known before a byte is written.
|
||||
static size_t fileSize(int width, int height);
|
||||
|
||||
bool begin(); // false: too big for one block, or the sink refused
|
||||
bool row(const uint8_t* pixels); // `width` bytes, RRRGGGBB each
|
||||
bool end();
|
||||
|
||||
private:
|
||||
bool put(const uint8_t* data, size_t len, bool inIdat);
|
||||
bool put32(uint32_t value, bool inIdat);
|
||||
|
||||
int w_, h_, rows_ = 0;
|
||||
Sink sink_;
|
||||
uint32_t crc_ = 0, adler_ = 1;
|
||||
};
|
||||
|
||||
} // namespace roro::png
|
||||
@@ -0,0 +1,135 @@
|
||||
#include "share_rules.h"
|
||||
|
||||
#include <cstdio>
|
||||
|
||||
namespace roro::files {
|
||||
|
||||
namespace {
|
||||
int hexDigit(char c) {
|
||||
if (c >= '0' && c <= '9') return c - '0';
|
||||
if (c >= 'a' && c <= 'f') return c - 'a' + 10;
|
||||
if (c >= 'A' && c <= 'F') return c - 'A' + 10;
|
||||
return -1;
|
||||
}
|
||||
// Whatever the two strings hold, the time taken says nothing about where they differ.
|
||||
bool sameText(const std::string& a, const std::string& b) {
|
||||
unsigned diff = static_cast<unsigned>(a.size() ^ b.size());
|
||||
for (size_t i = 0; i < a.size() && i < b.size(); i++) diff |= static_cast<unsigned char>(a[i]) ^ static_cast<unsigned char>(b[i]);
|
||||
return diff == 0;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
std::string urlDecode(const std::string& text) {
|
||||
std::string out;
|
||||
out.reserve(text.size());
|
||||
for (size_t i = 0; i < text.size(); i++) {
|
||||
int hi, lo;
|
||||
if (text[i] == '%' && i + 2 < text.size() + 0 && (hi = hexDigit(text[i + 1])) >= 0 && (lo = hexDigit(text[i + 2])) >= 0) {
|
||||
out += static_cast<char>(hi * 16 + lo);
|
||||
i += 2;
|
||||
} else {
|
||||
out += text[i];
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
bool queryParam(const std::string& query, const std::string& key, std::string& out) {
|
||||
for (size_t at = 0; at <= query.size();) {
|
||||
size_t amp = query.find('&', at);
|
||||
if (amp == std::string::npos) amp = query.size();
|
||||
size_t eq = query.find('=', at);
|
||||
if (eq != std::string::npos && eq < amp && query.compare(at, eq - at, key) == 0) {
|
||||
out = urlDecode(query.substr(eq + 1, amp - eq - 1));
|
||||
return true;
|
||||
}
|
||||
at = amp + 1;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
std::string cookieValue(const std::string& header, const std::string& name) {
|
||||
for (size_t at = 0; at < header.size();) {
|
||||
while (at < header.size() && (header[at] == ' ' || header[at] == ';')) at++;
|
||||
size_t end = header.find(';', at);
|
||||
if (end == std::string::npos) end = header.size();
|
||||
size_t eq = header.find('=', at);
|
||||
if (eq != std::string::npos && eq < end && header.compare(at, eq - at, name) == 0) return header.substr(eq + 1, end - eq - 1);
|
||||
at = end;
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
std::string checkSharePath(const std::string& path) {
|
||||
if (path.empty() || path[0] != '/') return "a path starts with /";
|
||||
if (path.size() > 255) return "that path is too long";
|
||||
if (path.size() > 1 && path.back() == '/') return "a path doesn't end with /";
|
||||
for (size_t at = 1; at < path.size();) {
|
||||
size_t end = path.find('/', at);
|
||||
if (end == std::string::npos) end = path.size();
|
||||
std::string part = path.substr(at, end - at);
|
||||
if (part.empty() || part == "." || part == "..") return "that isn't a path on the card";
|
||||
for (char c : part)
|
||||
if (static_cast<unsigned char>(c) < 0x20 || c == 0x7F || c == '\\' || c == ':' || c == '*' || c == '?' || c == '"' || c == '<' || c == '>' || c == '|')
|
||||
return "a name can't hold that character";
|
||||
at = end + 1;
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
std::string jsonString(const std::string& text) {
|
||||
std::string out = "\"";
|
||||
for (char c : text) {
|
||||
unsigned char u = static_cast<unsigned char>(c);
|
||||
if (c == '"' || c == '\\') {
|
||||
out += '\\';
|
||||
out += c;
|
||||
} else if (u < 0x20) {
|
||||
char buf[8];
|
||||
std::snprintf(buf, sizeof buf, "\\u%04x", u);
|
||||
out += buf;
|
||||
} else {
|
||||
out += c;
|
||||
}
|
||||
}
|
||||
return out + "\"";
|
||||
}
|
||||
|
||||
ShareListing::ShareListing(const std::string& path) : out_("{\"path\":" + jsonString(path) + ",\"items\":[") {}
|
||||
|
||||
void ShareListing::add(const std::string& name, uint32_t size, bool folder, int64_t modified) {
|
||||
if (count_++) out_ += ',';
|
||||
out_ += "{\"n\":" + jsonString(name) + ",\"s\":" + std::to_string(size) + ",\"d\":" + (folder ? "1" : "0") + ",\"t\":" + std::to_string(modified) + "}";
|
||||
}
|
||||
|
||||
std::string ShareListing::json(bool more) { return out_ + "],\"more\":" + (more ? "true" : "false") + "}"; }
|
||||
|
||||
void ShareAuth::begin(const uint8_t random[4]) {
|
||||
uint32_t n = (static_cast<uint32_t>(random[0]) << 24 | random[1] << 16 | random[2] << 8 | random[3]) % 1000000u;
|
||||
char buf[8];
|
||||
std::snprintf(buf, sizeof buf, "%06u", static_cast<unsigned>(n));
|
||||
code_ = buf;
|
||||
token_.clear();
|
||||
gate_ = debug::AuthGate();
|
||||
}
|
||||
|
||||
ShareAuth::Result ShareAuth::login(const std::string& code, uint32_t nowMs, const uint8_t random[16], std::string& token) {
|
||||
if (code_.empty() || gate_.locked(nowMs)) return Result::Locked;
|
||||
std::string digits;
|
||||
for (char c : code)
|
||||
if (c >= '0' && c <= '9') digits += c; // "123 456" is as good
|
||||
if (!sameText(digits, code_)) return gate_.failed(nowMs) ? Result::Locked : Result::Wrong;
|
||||
gate_.succeeded();
|
||||
static const char* const kHex = "0123456789abcdef";
|
||||
token_.clear();
|
||||
for (int i = 0; i < 16; i++) {
|
||||
token_ += kHex[random[i] >> 4];
|
||||
token_ += kHex[random[i] & 15];
|
||||
}
|
||||
token = token_;
|
||||
return Result::Ok;
|
||||
}
|
||||
|
||||
bool ShareAuth::allowed(const std::string& token) const { return !token_.empty() && sameText(token, token_); }
|
||||
|
||||
} // namespace roro::files
|
||||
@@ -0,0 +1,55 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstdint>
|
||||
#include <string>
|
||||
|
||||
#include "debug_auth.h"
|
||||
|
||||
// The parts of sharing files with a browser (issue #88) that need no network: what a request
|
||||
// asks for, whether it may, and the answers as JSON. The server itself is src/services/web_share.h.
|
||||
namespace roro::files {
|
||||
|
||||
std::string urlDecode(const std::string& text); // %41 is A; a + stays a +
|
||||
// The value of `key` in a query string ("path=%2Fnotes&replace=1"), decoded. False if it isn't there.
|
||||
bool queryParam(const std::string& query, const std::string& key, std::string& out);
|
||||
// The value of a cookie in a Cookie header ("a=1; s=abc"), or "".
|
||||
std::string cookieValue(const std::string& header, const std::string& name);
|
||||
|
||||
// A path a browser may name: from the card's root, no "..", nothing a file name can't hold.
|
||||
// "" or why not.
|
||||
std::string checkSharePath(const std::string& path);
|
||||
|
||||
std::string jsonString(const std::string& text); // with its quotes
|
||||
|
||||
// A folder's listing as the page wants it: {"path":"/notes","items":[{"n":"a.txt","s":12,"d":0,"t":1791400000}],"more":false}
|
||||
class ShareListing {
|
||||
public:
|
||||
explicit ShareListing(const std::string& path);
|
||||
void add(const std::string& name, uint32_t size, bool folder, int64_t modified);
|
||||
std::string json(bool more);
|
||||
size_t count() const { return count_; }
|
||||
|
||||
private:
|
||||
std::string out_;
|
||||
size_t count_ = 0;
|
||||
};
|
||||
|
||||
// Who may use the page: whoever typed the code the device's screen shows. The code is new each
|
||||
// time sharing starts; five wrong ones in a row close the door for a minute (as the Debug
|
||||
// Console's token does). A browser that got it right is given a token to send back as a cookie.
|
||||
// Nothing here is encrypted on the way: see the issue.
|
||||
class ShareAuth {
|
||||
public:
|
||||
enum class Result { Ok, Wrong, Locked };
|
||||
|
||||
void begin(const uint8_t random[4]); // a new code, and nobody is logged in
|
||||
const std::string& code() const { return code_; } // six digits
|
||||
Result login(const std::string& code, uint32_t nowMs, const uint8_t random[16], std::string& token);
|
||||
bool allowed(const std::string& token) const;
|
||||
|
||||
private:
|
||||
std::string code_, token_;
|
||||
debug::AuthGate gate_;
|
||||
};
|
||||
|
||||
} // namespace roro::files
|
||||
@@ -43,6 +43,15 @@ KeyEvent charEvent(uint32_t cp, const RawKeys& keys) {
|
||||
return e;
|
||||
}
|
||||
|
||||
// A key that isn't a character, with what was held: a terminal tells Alt+Enter from Enter (issue #2).
|
||||
KeyEvent keyEvent(Key key, const RawKeys& keys) {
|
||||
KeyEvent e = KeyEvent::of(key);
|
||||
e.shift = keys.shift;
|
||||
e.ctrl = keys.ctrl;
|
||||
e.alt = keys.alt;
|
||||
return e;
|
||||
}
|
||||
|
||||
} // namespace
|
||||
|
||||
std::vector<KeyEvent> KeyMapper::update(const RawKeys& keys) {
|
||||
@@ -51,9 +60,9 @@ std::vector<KeyEvent> KeyMapper::update(const RawKeys& keys) {
|
||||
if (keys.opt && !previous_.opt && keys.chars.empty()) {
|
||||
compose_ = compose_ ? 0 : kArmed; // a second opt cancels
|
||||
}
|
||||
if (keys.enter && !previous_.enter) out.push_back(KeyEvent::of(Key::Select));
|
||||
if (keys.del && !previous_.del) out.push_back(KeyEvent::of(Key::Delete));
|
||||
if (keys.tab && !previous_.tab) out.push_back(KeyEvent::of(Key::Tab));
|
||||
if (keys.enter && !previous_.enter) out.push_back(keyEvent(Key::Select, keys));
|
||||
if (keys.del && !previous_.del) out.push_back(keyEvent(Key::Delete, keys));
|
||||
if (keys.tab && !previous_.tab) out.push_back(keyEvent(Key::Tab, keys));
|
||||
|
||||
for (char c : keys.chars) {
|
||||
bool wasHeld = std::find(previous_.chars.begin(), previous_.chars.end(), c) != previous_.chars.end();
|
||||
@@ -89,23 +98,44 @@ void KeyMapper::onChar(char c, const RawKeys& keys, std::vector<KeyEvent>& out)
|
||||
|
||||
if (keys.fn || !textEntry_) {
|
||||
switch (c) {
|
||||
case ';': out.push_back(KeyEvent::of(Key::Up)); return;
|
||||
case '.': out.push_back(KeyEvent::of(Key::Down)); return;
|
||||
case ',': out.push_back(KeyEvent::of(Key::Left)); return;
|
||||
case '/': out.push_back(KeyEvent::of(Key::Right)); return;
|
||||
case ';': out.push_back(keyEvent(Key::Up, keys)); return;
|
||||
case '.': out.push_back(keyEvent(Key::Down, keys)); return;
|
||||
case ',': out.push_back(keyEvent(Key::Left, keys)); return;
|
||||
case '/': out.push_back(keyEvent(Key::Right, keys)); return;
|
||||
case '`':
|
||||
if (keys.fn) {
|
||||
out.push_back(KeyEvent::of(Key::Home));
|
||||
return;
|
||||
}
|
||||
break;
|
||||
case 'h':
|
||||
case 'H':
|
||||
if (keys.fn) { // Fn+h: help, while typing too
|
||||
out.push_back(KeyEvent::of(Key::Help));
|
||||
return;
|
||||
}
|
||||
break;
|
||||
case 'p':
|
||||
case 'P':
|
||||
if (keys.fn) { // Fn+p: a screenshot, while typing too
|
||||
out.push_back(KeyEvent::of(Key::Screenshot));
|
||||
return;
|
||||
}
|
||||
break;
|
||||
case '?':
|
||||
if (!textEntry_ && !keys.fn) { // ? alone, when it wouldn't be typed
|
||||
out.push_back(KeyEvent::of(Key::Help));
|
||||
return;
|
||||
}
|
||||
if (keys.fn) return;
|
||||
break;
|
||||
default:
|
||||
if (keys.fn) return; // other Fn combos are unassigned
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (c == '`') {
|
||||
out.push_back(KeyEvent::of(Key::Back));
|
||||
out.push_back(keyEvent(Key::Back, keys));
|
||||
return;
|
||||
}
|
||||
out.push_back(charEvent(static_cast<unsigned char>(c), keys));
|
||||
|
||||
@@ -23,7 +23,7 @@ struct RawKeys {
|
||||
|
||||
// Turns keyboard state changes into logical KeyEvents: only newly pressed keys produce events;
|
||||
// Fn + ; . , / are arrows, and so are ; . , / alone when no text is being entered; ` is Back and
|
||||
// Fn + ` is Home; the Compose Key (opt) followed by an accent and a letter types the accented
|
||||
// Fn + ` is Home; Fn + h is Help anywhere, and so is ? when no text is being entered; the Compose Key (opt) followed by an accent and a letter types the accented
|
||||
// letter (opt ' e -> é).
|
||||
class KeyMapper {
|
||||
public:
|
||||
|
||||
@@ -0,0 +1,300 @@
|
||||
#include "net_probe.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <cstring>
|
||||
|
||||
#include "ipv4.h"
|
||||
|
||||
namespace roro::net {
|
||||
|
||||
namespace {
|
||||
std::vector<std::string> words(const std::string& text) {
|
||||
std::vector<std::string> out;
|
||||
size_t at = 0;
|
||||
while (at < text.size()) {
|
||||
while (at < text.size() && text[at] == ' ') at++;
|
||||
size_t end = text.find(' ', at);
|
||||
if (end == std::string::npos) end = text.size();
|
||||
if (end > at) out.push_back(text.substr(at, end - at));
|
||||
at = end;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
bool number(const std::string& s, long& out) {
|
||||
if (s.empty() || s.size() > 6) return false;
|
||||
out = 0;
|
||||
for (char c : s) {
|
||||
if (c < '0' || c > '9') return false;
|
||||
out = out * 10 + (c - '0');
|
||||
}
|
||||
return true;
|
||||
}
|
||||
uint16_t be16(const uint8_t* p) { return static_cast<uint16_t>((p[0] << 8) | p[1]); }
|
||||
} // namespace
|
||||
|
||||
std::string parsePing(const std::string& args, PingArgs& out) {
|
||||
static const char* const kUsage = "ping <host> [count] [size]";
|
||||
auto w = words(args);
|
||||
if (w.empty() || w.size() > 3 || !validHost(w[0])) return kUsage;
|
||||
PingArgs a;
|
||||
a.host = w[0];
|
||||
long n;
|
||||
if (w.size() > 1) {
|
||||
if (!number(w[1], n) || n < 1 || n > 100) return "a count from 1 to 100";
|
||||
a.count = static_cast<int>(n);
|
||||
}
|
||||
if (w.size() > 2) {
|
||||
if (!number(w[2], n) || n > 1400) return "a size from 0 to 1400 bytes";
|
||||
a.size = static_cast<int>(n);
|
||||
}
|
||||
out = a;
|
||||
return "";
|
||||
}
|
||||
|
||||
std::string parsePort(const std::string& args, PortArgs& out) {
|
||||
static const char* const kUsage = "port <host> <port>";
|
||||
auto w = words(args);
|
||||
if (w.size() == 1) { // host:port
|
||||
size_t colon = w[0].rfind(':');
|
||||
if (colon == std::string::npos) return kUsage;
|
||||
w = {w[0].substr(0, colon), w[0].substr(colon + 1)};
|
||||
}
|
||||
long n;
|
||||
if (w.size() != 2 || !validHost(w[0])) return kUsage;
|
||||
if (!number(w[1], n) || n < 1 || n > 65535) return "a port from 1 to 65535";
|
||||
out.host = w[0];
|
||||
out.port = static_cast<uint16_t>(n);
|
||||
return "";
|
||||
}
|
||||
|
||||
std::string parseLookup(const std::string& args, LookupArgs& out) {
|
||||
static const char* const kUsage = "nslookup <name> [server's address]";
|
||||
auto w = words(args);
|
||||
uint32_t ip;
|
||||
if (w.empty() || w.size() > 2 || !validHost(w[0])) return kUsage;
|
||||
if (w.size() == 2 && !parseIpv4(w[1], ip)) return "the server as an address: 9.9.9.9";
|
||||
out.name = w[0];
|
||||
out.server = w.size() == 2 ? w[1] : "";
|
||||
return "";
|
||||
}
|
||||
|
||||
uint16_t inetChecksum(const uint8_t* data, size_t len) {
|
||||
uint32_t sum = 0;
|
||||
for (size_t i = 0; i + 1 < len; i += 2) sum += static_cast<uint32_t>((data[i] << 8) | data[i + 1]);
|
||||
if (len & 1) sum += static_cast<uint32_t>(data[len - 1] << 8);
|
||||
while (sum >> 16) sum = (sum & 0xFFFF) + (sum >> 16);
|
||||
return static_cast<uint16_t>(~sum);
|
||||
}
|
||||
|
||||
size_t buildEcho(uint8_t* out, size_t max, uint16_t id, uint16_t seq, size_t payload) {
|
||||
size_t len = 8 + payload;
|
||||
if (len > max) return 0;
|
||||
out[0] = 8; // echo request
|
||||
out[1] = 0;
|
||||
out[2] = out[3] = 0;
|
||||
out[4] = static_cast<uint8_t>(id >> 8);
|
||||
out[5] = static_cast<uint8_t>(id);
|
||||
out[6] = static_cast<uint8_t>(seq >> 8);
|
||||
out[7] = static_cast<uint8_t>(seq);
|
||||
for (size_t i = 0; i < payload; i++) out[8 + i] = static_cast<uint8_t>('a' + i % 26);
|
||||
uint16_t sum = inetChecksum(out, len);
|
||||
out[2] = static_cast<uint8_t>(sum >> 8);
|
||||
out[3] = static_cast<uint8_t>(sum);
|
||||
return len;
|
||||
}
|
||||
|
||||
IcmpAnswer parseIcmp(const uint8_t* packet, size_t len) {
|
||||
IcmpAnswer a;
|
||||
if (len < 20 || (packet[0] >> 4) != 4) return a;
|
||||
size_t header = static_cast<size_t>(packet[0] & 0x0F) * 4;
|
||||
if (header < 20 || len < header + 8 || packet[9] != 1) return a; // not ICMP
|
||||
const uint8_t* icmp = packet + header;
|
||||
size_t left = len - header;
|
||||
if (icmp[0] == 0 && icmp[1] == 0) { // echo reply
|
||||
a.kind = IcmpAnswer::Kind::Echo;
|
||||
a.id = be16(icmp + 4);
|
||||
a.seq = be16(icmp + 6);
|
||||
return a;
|
||||
}
|
||||
if (icmp[0] != 11 && icmp[0] != 3) return a;
|
||||
// Inside: the IP header of the packet it is about, and that packet's first 8 bytes.
|
||||
if (left < 8 + 20) return a;
|
||||
const uint8_t* inner = icmp + 8;
|
||||
size_t innerHeader = static_cast<size_t>(inner[0] & 0x0F) * 4;
|
||||
if ((inner[0] >> 4) != 4 || innerHeader < 20 || left < 8 + innerHeader + 8 || inner[9] != 1 || inner[innerHeader] != 8) return a;
|
||||
a.kind = icmp[0] == 11 ? IcmpAnswer::Kind::TimeExceeded : IcmpAnswer::Kind::Unreachable;
|
||||
a.id = be16(inner + innerHeader + 4);
|
||||
a.seq = be16(inner + innerHeader + 6);
|
||||
return a;
|
||||
}
|
||||
|
||||
void PingStats::add(uint32_t ms) {
|
||||
minMs = back ? std::min(minMs, ms) : ms;
|
||||
maxMs = std::max(maxMs, ms);
|
||||
sumMs += ms;
|
||||
back++;
|
||||
}
|
||||
|
||||
std::string PingStats::summary() const {
|
||||
int lost = sent ? (sent - back) * 100 / sent : 0;
|
||||
std::string s = std::to_string(back) + "/" + std::to_string(sent) + " back, " + std::to_string(lost) + "% lost"; // short: a Shell line is 38 characters
|
||||
if (back) s += ", " + std::to_string(minMs) + "/" + std::to_string(sumMs / static_cast<uint32_t>(back)) + "/" + std::to_string(maxMs) + " ms";
|
||||
return s;
|
||||
}
|
||||
|
||||
size_t buildDnsQuery(uint8_t* out, size_t max, uint16_t id, const std::string& name) {
|
||||
if (name.empty() || name.size() > 253 || 12 + name.size() + 2 + 4 > max) return 0;
|
||||
std::memset(out, 0, 12);
|
||||
out[0] = static_cast<uint8_t>(id >> 8);
|
||||
out[1] = static_cast<uint8_t>(id);
|
||||
out[2] = 0x01; // recursion wanted
|
||||
out[5] = 1; // one question
|
||||
size_t at = 12;
|
||||
for (size_t from = 0; from <= name.size();) {
|
||||
size_t dot = name.find('.', from);
|
||||
if (dot == std::string::npos) dot = name.size();
|
||||
size_t n = dot - from;
|
||||
if (n == 0 && dot == name.size()) break; // a final dot
|
||||
if (n == 0 || n > 63) return 0;
|
||||
out[at++] = static_cast<uint8_t>(n);
|
||||
std::memcpy(out + at, name.data() + from, n);
|
||||
at += n;
|
||||
from = dot + 1;
|
||||
}
|
||||
out[at++] = 0;
|
||||
out[at++] = 0;
|
||||
out[at++] = 1; // A
|
||||
out[at++] = 0;
|
||||
out[at++] = 1; // IN
|
||||
return at;
|
||||
}
|
||||
|
||||
namespace {
|
||||
// Reads a name at `at`, following the pointers DNS shortens names with. Where the name ends in
|
||||
// the message (not where a pointer led), or 0 if it is broken.
|
||||
size_t readName(const uint8_t* m, size_t len, size_t at, std::string* out) {
|
||||
size_t end = 0;
|
||||
int jumps = 0;
|
||||
while (at < len) {
|
||||
uint8_t n = m[at];
|
||||
if (n == 0) return end ? end : at + 1;
|
||||
if ((n & 0xC0) == 0xC0) {
|
||||
if (at + 1 >= len || ++jumps > 8) return 0;
|
||||
if (!end) end = at + 2;
|
||||
at = static_cast<size_t>(((n & 0x3F) << 8) | m[at + 1]);
|
||||
continue;
|
||||
}
|
||||
if (n > 63 || at + 1 + n > len) return 0;
|
||||
if (out) {
|
||||
if (!out->empty()) *out += '.';
|
||||
out->append(reinterpret_cast<const char*>(m + at + 1), n);
|
||||
}
|
||||
at += 1 + static_cast<size_t>(n);
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
bool parseDnsAnswer(const uint8_t* m, size_t len, uint16_t id, DnsAnswer& out) {
|
||||
if (len < 12 || be16(m) != id || !(m[2] & 0x80)) return false;
|
||||
DnsAnswer a;
|
||||
a.truncated = m[2] & 0x02;
|
||||
a.rcode = m[3] & 0x0F;
|
||||
int questions = be16(m + 4), answers = be16(m + 6);
|
||||
size_t at = 12;
|
||||
for (int i = 0; i < questions; i++) {
|
||||
at = readName(m, len, at, nullptr);
|
||||
if (!at || at + 4 > len) return false;
|
||||
at += 4;
|
||||
}
|
||||
for (int i = 0; i < answers; i++) {
|
||||
at = readName(m, len, at, nullptr);
|
||||
if (!at || at + 10 > len) return false;
|
||||
uint16_t type = be16(m + at), size = be16(m + at + 8);
|
||||
at += 10;
|
||||
if (at + size > len) return false;
|
||||
if (type == 1 && size == 4) a.addresses.push_back((static_cast<uint32_t>(m[at]) << 24) | (m[at + 1] << 16) | (m[at + 2] << 8) | m[at + 3]);
|
||||
if (type == 5) {
|
||||
std::string name;
|
||||
if (readName(m, len, at, &name)) a.alias = name;
|
||||
}
|
||||
at += size;
|
||||
}
|
||||
out = a;
|
||||
return true;
|
||||
}
|
||||
|
||||
void buildNtpRequest(uint8_t out[kNtpPacket]) {
|
||||
std::memset(out, 0, kNtpPacket);
|
||||
out[0] = 0x23; // no warning, version 4, a client
|
||||
}
|
||||
|
||||
bool parseNtpAnswer(const uint8_t* p, size_t len, NtpAnswer& out) {
|
||||
if (len < kNtpPacket || (p[0] & 0x07) != 4) return false; // not a server's
|
||||
if (p[1] == 0 || p[1] > 15) return false; // "kiss of death", or not synchronised
|
||||
uint32_t secs = (static_cast<uint32_t>(p[40]) << 24) | (p[41] << 16) | (p[42] << 8) | p[43];
|
||||
uint32_t frac = (static_cast<uint32_t>(p[44]) << 24) | (p[45] << 16) | (p[46] << 8) | p[47];
|
||||
if (!secs) return false;
|
||||
// NTP counts from 1900 and wraps in 2036: a small number is the era after.
|
||||
constexpr int64_t k1900To1970 = 2208988800LL;
|
||||
int64_t since1900 = secs < 0x80000000u ? static_cast<int64_t>(secs) + 4294967296LL : static_cast<int64_t>(secs);
|
||||
out.stratum = p[1];
|
||||
out.seconds = since1900 - k1900To1970;
|
||||
out.millis = static_cast<uint32_t>((static_cast<uint64_t>(frac) * 1000) >> 32);
|
||||
return true;
|
||||
}
|
||||
|
||||
std::string clockOffset(int64_t ownMs, int64_t serverMs) {
|
||||
int64_t diff = ownMs - serverMs, size = diff < 0 ? -diff : diff;
|
||||
if (size < 100) return "right, to 0.1 s";
|
||||
std::string amount = size < 10000 ? std::to_string(size / 1000) + "." + std::to_string(size % 1000 / 100) + " s"
|
||||
: size < 120000 ? std::to_string(size / 1000) + " s"
|
||||
: size < 7200000 ? std::to_string(size / 60000) + " min"
|
||||
: size < 172800000LL ? std::to_string(size / 3600000) + " h" : std::to_string(size / 86400000LL) + " days";
|
||||
return amount + (diff > 0 ? " ahead" : " behind");
|
||||
}
|
||||
|
||||
std::string certName(const std::string& dn) {
|
||||
for (const char* key : {"CN=", "O="}) {
|
||||
size_t at = 0;
|
||||
while ((at = dn.find(key, at)) != std::string::npos) {
|
||||
if (at == 0 || dn[at - 1] == ' ' || dn[at - 1] == ',') {
|
||||
size_t from = at + std::strlen(key), end = dn.find(", ", from);
|
||||
std::string name = dn.substr(from, end == std::string::npos ? std::string::npos : end - from);
|
||||
// An old kind of string comes out as "#" and hex, type and length first: read it.
|
||||
if (name.size() > 5 && name[0] == '#' && name.size() % 2 == 1) {
|
||||
std::string plain;
|
||||
for (size_t i = 5; i + 1 < name.size(); i += 2) {
|
||||
auto digit = [](char c) { return c >= '0' && c <= '9' ? c - '0' : c >= 'A' && c <= 'F' ? c - 'A' + 10 : c >= 'a' && c <= 'f' ? c - 'a' + 10 : -1; };
|
||||
int hi = digit(name[i]), lo = digit(name[i + 1]);
|
||||
if (hi < 0 || lo < 0 || hi * 16 + lo < 0x20 || hi * 16 + lo > 0x7E) return name;
|
||||
plain += static_cast<char>(hi * 16 + lo);
|
||||
}
|
||||
return plain;
|
||||
}
|
||||
return name;
|
||||
}
|
||||
at++;
|
||||
}
|
||||
}
|
||||
return dn;
|
||||
}
|
||||
|
||||
namespace {
|
||||
// Days since a fixed day long ago (the civil calendar, leap years and all).
|
||||
long dayNumber(int y, int m, int d) {
|
||||
y -= m <= 2;
|
||||
long era = (y >= 0 ? y : y - 399) / 400;
|
||||
long yoe = y - era * 400, doy = (153 * (m + (m > 2 ? -3 : 9)) + 2) / 5 + d - 1;
|
||||
return era * 146097 + yoe * 365 + yoe / 4 - yoe / 100 + doy;
|
||||
}
|
||||
} // namespace
|
||||
|
||||
int daysBetween(int y1, int m1, int d1, int y2, int m2, int d2) { return static_cast<int>(dayNumber(y2, m2, d2) - dayNumber(y1, m1, d1)); }
|
||||
|
||||
const char* portLabel(uint16_t port, bool tcp) {
|
||||
if (tcp) return port == 3232 ? "updates" : port == 2323 ? "Debug Console" : port == 80 ? "sharing" : "";
|
||||
return port == 68 ? "DHCP" : port == 123 ? "NTP" : "";
|
||||
}
|
||||
|
||||
} // namespace roro::net
|
||||
@@ -0,0 +1,98 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
// The parts of the network troubleshooting commands (issue #90) that need no network: what was
|
||||
// asked, the packets to send, and what the answers mean. The sockets are src/services/net_tools.h (a different name on purpose: two headers of one name find themselves).
|
||||
namespace roro::net {
|
||||
|
||||
// --- what was typed
|
||||
|
||||
struct PingArgs {
|
||||
std::string host;
|
||||
int count = 4; // 1 to 100
|
||||
int size = 56; // bytes of payload, 0 to 1400: with its headers a ping is 28 more
|
||||
};
|
||||
// "ping <host> [count] [size]". "" or how to ask.
|
||||
std::string parsePing(const std::string& args, PingArgs& out);
|
||||
|
||||
struct PortArgs {
|
||||
std::string host;
|
||||
uint16_t port = 0;
|
||||
};
|
||||
// "port <host> <port>", or host:port.
|
||||
std::string parsePort(const std::string& args, PortArgs& out);
|
||||
|
||||
struct LookupArgs {
|
||||
std::string name, server; // server: an address, or "" for the one in use
|
||||
};
|
||||
std::string parseLookup(const std::string& args, LookupArgs& out);
|
||||
|
||||
// --- ICMP: ping and traceroute
|
||||
|
||||
uint16_t inetChecksum(const uint8_t* data, size_t len);
|
||||
// An echo request: 8 bytes of header and `payload` bytes after it. The length written, or 0 if
|
||||
// it doesn't fit.
|
||||
size_t buildEcho(uint8_t* out, size_t max, uint16_t id, uint16_t seq, size_t payload);
|
||||
|
||||
struct IcmpAnswer {
|
||||
enum class Kind { Other, Echo, TimeExceeded, Unreachable } kind = Kind::Other;
|
||||
uint16_t id = 0, seq = 0; // of the echo request it answers
|
||||
};
|
||||
// `packet` as a raw socket hands it over: the IP header first. A router's "time exceeded" and
|
||||
// "unreachable" carry the start of the packet they are about, which is where id and seq come from.
|
||||
IcmpAnswer parseIcmp(const uint8_t* packet, size_t len);
|
||||
|
||||
// What a run of pings came to: "3/4 back, 25% lost, 12/25/41 ms" (the least, the mean, the most).
|
||||
struct PingStats {
|
||||
int sent = 0, back = 0;
|
||||
uint32_t minMs = 0, maxMs = 0, sumMs = 0;
|
||||
void add(uint32_t ms);
|
||||
std::string summary() const;
|
||||
};
|
||||
|
||||
// --- DNS: nslookup
|
||||
|
||||
// A query for the IPv4 addresses of `name`. The length written, or 0 if that isn't a name.
|
||||
size_t buildDnsQuery(uint8_t* out, size_t max, uint16_t id, const std::string& name);
|
||||
|
||||
struct DnsAnswer {
|
||||
int rcode = 0; // 0: fine, 3: no such name
|
||||
bool truncated = false; // the answer didn't fit in one packet
|
||||
std::vector<uint32_t> addresses;
|
||||
std::string alias; // the last name a CNAME led to, if any
|
||||
};
|
||||
// False if it isn't the answer to query `id`, or is cut short.
|
||||
bool parseDnsAnswer(const uint8_t* message, size_t len, uint16_t id, DnsAnswer& out);
|
||||
|
||||
// --- NTP: the clock's offset
|
||||
|
||||
constexpr size_t kNtpPacket = 48;
|
||||
void buildNtpRequest(uint8_t out[kNtpPacket]);
|
||||
struct NtpAnswer {
|
||||
int stratum = 0; // 1: a reference clock; 2 and up: that many steps from one
|
||||
int64_t seconds = 0; // the server's clock when it answered, UTC since 1970
|
||||
uint32_t millis = 0; // and the part of a second
|
||||
};
|
||||
// False if it isn't a server's answer, or says the server has no time to give.
|
||||
bool parseNtpAnswer(const uint8_t* packet, size_t len, NtpAnswer& out);
|
||||
// "0.3 s ahead", "12 s behind", "right, to 0.1 s": this clock against the server's, both in ms.
|
||||
std::string clockOffset(int64_t ownMs, int64_t serverMs);
|
||||
|
||||
// --- TLS: who a certificate is for
|
||||
|
||||
// The common name out of a certificate's subject or issuer as mbedTLS prints it
|
||||
// ("C=US, O=Let's Encrypt, CN=R11"): the CN, else the O, else all of it.
|
||||
std::string certName(const std::string& dn);
|
||||
// Whole days from one date to another (negative: the second is earlier).
|
||||
int daysBetween(int y1, int m1, int d1, int y2, int m2, int d2);
|
||||
|
||||
// --- netstat
|
||||
|
||||
// What listens on a port of this firmware, or "".
|
||||
const char* portLabel(uint16_t port, bool tcp);
|
||||
|
||||
} // namespace roro::net
|
||||
@@ -0,0 +1,217 @@
|
||||
#include "wg_config.h"
|
||||
|
||||
#include <algorithm>
|
||||
|
||||
#include "ipv4.h"
|
||||
|
||||
namespace roro::net {
|
||||
|
||||
namespace {
|
||||
std::string trim(const std::string& s) {
|
||||
size_t a = s.find_first_not_of(" \t\r"), b = s.find_last_not_of(" \t\r");
|
||||
return a == std::string::npos ? "" : s.substr(a, b - a + 1);
|
||||
}
|
||||
std::string lower(std::string s) {
|
||||
for (char& c : s)
|
||||
if (c >= 'A' && c <= 'Z') c = static_cast<char>(c + 32);
|
||||
return s;
|
||||
}
|
||||
bool number(const std::string& s, long& out, long max) {
|
||||
if (s.empty() || s.size() > 6) return false;
|
||||
out = 0;
|
||||
for (char c : s) {
|
||||
if (c < '0' || c > '9') return false;
|
||||
out = out * 10 + (c - '0');
|
||||
}
|
||||
return out <= max;
|
||||
}
|
||||
// "10.9.0.2/24", or an address alone (then /32). False for anything else, IPv6 included.
|
||||
bool range(const std::string& text, WgRange& out) {
|
||||
size_t slash = text.find('/');
|
||||
long prefix = 32;
|
||||
if (slash != std::string::npos && !number(text.substr(slash + 1), prefix, 32)) return false;
|
||||
if (!parseIpv4(text.substr(0, slash), out.address)) return false;
|
||||
out.prefix = static_cast<int>(prefix);
|
||||
return true;
|
||||
}
|
||||
template <typename Each>
|
||||
void eachItem(const std::string& list, Each each) {
|
||||
size_t at = 0;
|
||||
while (at <= list.size()) {
|
||||
size_t comma = list.find(',', at);
|
||||
if (comma == std::string::npos) comma = list.size();
|
||||
std::string item = trim(list.substr(at, comma - at));
|
||||
if (!item.empty()) each(item);
|
||||
at = comma + 1;
|
||||
}
|
||||
}
|
||||
bool inRange(uint32_t address, const WgRange& r) { return (address & maskOf(r.prefix)) == (r.address & maskOf(r.prefix)); }
|
||||
} // namespace
|
||||
|
||||
bool validWgKey(const std::string& key) {
|
||||
if (key.size() != 44 || key[43] != '=') return false;
|
||||
for (size_t i = 0; i < 43; i++) {
|
||||
char c = key[i];
|
||||
if (!((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || c == '+' || c == '/')) return false;
|
||||
}
|
||||
// 43 characters carry 258 bits: the last one's two low bits belong to no byte and are zero.
|
||||
static const std::string kLast = "AEIMQUYcgkosw048";
|
||||
return kLast.find(key[42]) != std::string::npos;
|
||||
}
|
||||
|
||||
std::string parseWgConf(const std::string& text, WgConfig& out) {
|
||||
WgConfig c;
|
||||
enum { None, Interface, Peer, OtherPeer } section = None;
|
||||
bool hasAddress = false, hasEndpoint = false, hasKeepalive = false;
|
||||
int lineNo = 0;
|
||||
std::string problem;
|
||||
auto fail = [&](const std::string& what) {
|
||||
if (problem.empty()) problem = "line " + std::to_string(lineNo) + ": " + what;
|
||||
};
|
||||
for (size_t at = 0; at <= text.size() && problem.empty();) {
|
||||
size_t end = text.find('\n', at);
|
||||
if (end == std::string::npos) end = text.size();
|
||||
std::string line = text.substr(at, end - at);
|
||||
at = end + 1;
|
||||
lineNo++;
|
||||
size_t hash = line.find_first_of("#;");
|
||||
if (hash != std::string::npos) line.resize(hash);
|
||||
line = trim(line);
|
||||
if (line.empty()) continue;
|
||||
if (line[0] == '[') {
|
||||
std::string name = lower(line);
|
||||
if (name == "[interface]") section = Interface;
|
||||
else if (name == "[peer]") section = section == Peer || section == OtherPeer ? OtherPeer : Peer;
|
||||
else fail("a section this doesn't know");
|
||||
if (section == OtherPeer) fail("a second peer: this device has one tunnel to one peer");
|
||||
continue;
|
||||
}
|
||||
size_t eq = line.find('=');
|
||||
if (eq == std::string::npos) {
|
||||
fail("not a setting");
|
||||
continue;
|
||||
}
|
||||
std::string key = lower(trim(line.substr(0, eq))), value = trim(line.substr(eq + 1));
|
||||
long n = 0;
|
||||
if (section == Interface) {
|
||||
if (key == "privatekey") {
|
||||
if (!validWgKey(value)) fail("PrivateKey isn't a key");
|
||||
c.privateKey = value;
|
||||
} else if (key == "address") {
|
||||
eachItem(value, [&](const std::string& item) {
|
||||
WgRange r;
|
||||
if (!hasAddress && range(item, r)) {
|
||||
c.address = r.address;
|
||||
c.prefix = r.prefix;
|
||||
hasAddress = true;
|
||||
}
|
||||
});
|
||||
if (!hasAddress) fail("Address has no IPv4 address");
|
||||
} else if (key == "dns") {
|
||||
int count = 0;
|
||||
eachItem(value, [&](const std::string& item) { // names and IPv6 servers are left out
|
||||
uint32_t ip;
|
||||
if (count < 2 && parseIpv4(item, ip)) c.dns[count++] = ip;
|
||||
});
|
||||
} else if (key == "mtu") {
|
||||
if (!number(value, n, 1500) || n < 576) fail("MTU must be 576 to 1500");
|
||||
c.mtu = static_cast<int>(n);
|
||||
} else if (key == "listenport") {
|
||||
if (!number(value, n, 65535)) fail("ListenPort must be a port");
|
||||
c.listenPort = static_cast<uint16_t>(n);
|
||||
} // Table, PostUp and the rest mean nothing here
|
||||
} else if (section == Peer) {
|
||||
if (key == "publickey") {
|
||||
if (!validWgKey(value)) fail("PublicKey isn't a key");
|
||||
c.peerKey = value;
|
||||
} else if (key == "presharedkey") {
|
||||
if (!validWgKey(value)) fail("PresharedKey isn't a key");
|
||||
c.presharedKey = value;
|
||||
} else if (key == "endpoint") {
|
||||
size_t colon = value.rfind(':');
|
||||
if (value.empty() || value[0] == '[') fail("an IPv6 Endpoint: IPv4 or a name only");
|
||||
else if (colon == std::string::npos || colon == 0 || !number(value.substr(colon + 1), n, 65535) || n == 0) fail("Endpoint must be host:port");
|
||||
else if (value.find_first_of(" \t,/") != std::string::npos || colon > 253) fail("Endpoint must be host:port");
|
||||
else {
|
||||
c.endpointHost = value.substr(0, colon);
|
||||
c.endpointPort = static_cast<uint16_t>(n);
|
||||
hasEndpoint = true;
|
||||
}
|
||||
} else if (key == "allowedips") {
|
||||
eachItem(value, [&](const std::string& item) {
|
||||
WgRange r;
|
||||
if (item.find(':') != std::string::npos) return; // IPv6: not routed here
|
||||
if (!range(item, r)) return fail("AllowedIPs has something that isn't an address range");
|
||||
if (c.allowedCount == WgConfig::kMaxRanges) return fail("AllowedIPs: four IPv4 ranges at most");
|
||||
r.address &= maskOf(r.prefix);
|
||||
c.allowed[c.allowedCount++] = r;
|
||||
});
|
||||
} else if (key == "persistentkeepalive") {
|
||||
if (lower(value) == "off") n = 0;
|
||||
else if (!number(value, n, 65535)) fail("PersistentKeepalive must be seconds");
|
||||
c.keepalive = static_cast<int>(n);
|
||||
hasKeepalive = true;
|
||||
}
|
||||
} else if (section == None) {
|
||||
fail("a setting before [Interface]");
|
||||
}
|
||||
}
|
||||
(void)hasKeepalive;
|
||||
if (!problem.empty()) return problem;
|
||||
if (c.privateKey.empty()) return "no PrivateKey under [Interface]";
|
||||
if (!hasAddress) return "no Address under [Interface]";
|
||||
if (c.peerKey.empty()) return "no PublicKey under [Peer]";
|
||||
if (!hasEndpoint) return "no Endpoint under [Peer]";
|
||||
if (!c.allowedCount) return "no IPv4 range in AllowedIPs";
|
||||
out = c;
|
||||
return "";
|
||||
}
|
||||
|
||||
std::string toWgConf(const WgConfig& c) {
|
||||
std::string s = "[Interface]\nPrivateKey = " + c.privateKey + "\nAddress = " + formatIpv4(c.address) + "/" + std::to_string(c.prefix) + "\n";
|
||||
if (c.dns[0]) s += "DNS = " + formatIpv4(c.dns[0]) + (c.dns[1] ? ", " + formatIpv4(c.dns[1]) : "") + "\n";
|
||||
if (c.mtu) s += "MTU = " + std::to_string(c.mtu) + "\n";
|
||||
if (c.listenPort) s += "ListenPort = " + std::to_string(c.listenPort) + "\n";
|
||||
s += "[Peer]\nPublicKey = " + c.peerKey + "\n";
|
||||
if (!c.presharedKey.empty()) s += "PresharedKey = " + c.presharedKey + "\n";
|
||||
s += "Endpoint = " + c.endpointHost + ":" + std::to_string(c.endpointPort) + "\nAllowedIPs = ";
|
||||
for (int i = 0; i < c.allowedCount; i++) s += (i ? ", " : "") + formatIpv4(c.allowed[i].address) + "/" + std::to_string(c.allowed[i].prefix);
|
||||
s += "\nPersistentKeepalive = " + std::to_string(c.keepalive) + "\n";
|
||||
return s;
|
||||
}
|
||||
|
||||
WgRouting routingOf(const WgConfig& c) {
|
||||
WgRouting r;
|
||||
for (int i = 0; i < c.allowedCount; i++)
|
||||
if (c.allowed[i].prefix == 0) r.full = true;
|
||||
if (r.full) return r;
|
||||
// The widest allowed range this device's own address is in is the interface's subnet; with
|
||||
// none, the Address line's own.
|
||||
r.prefix = c.prefix;
|
||||
bool found = false;
|
||||
for (int i = 0; i < c.allowedCount; i++)
|
||||
if (inRange(c.address, c.allowed[i]) && (!found || c.allowed[i].prefix < r.prefix)) {
|
||||
r.prefix = c.allowed[i].prefix;
|
||||
found = true;
|
||||
}
|
||||
WgRange subnet{c.address, r.prefix};
|
||||
for (int i = 0; i < c.allowedCount; i++)
|
||||
if (c.allowed[i].prefix < r.prefix || !inRange(c.allowed[i].address, subnet)) r.unreachable++;
|
||||
return r;
|
||||
}
|
||||
|
||||
bool wgReaches(const WgConfig& c, uint32_t address) {
|
||||
WgRouting r = routingOf(c);
|
||||
if (r.full) return true;
|
||||
return inRange(address, WgRange{c.address, r.prefix});
|
||||
}
|
||||
|
||||
std::string describeWgRouting(const WgConfig& c) {
|
||||
WgRouting r = routingOf(c);
|
||||
if (r.full) return "everything";
|
||||
std::string s = formatIpv4(c.address & maskOf(r.prefix)) + "/" + std::to_string(r.prefix);
|
||||
if (r.unreachable) s += ", not " + std::to_string(r.unreachable) + " other range" + (r.unreachable > 1 ? "s" : "");
|
||||
return s;
|
||||
}
|
||||
|
||||
} // namespace roro::net
|
||||
@@ -0,0 +1,53 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstdint>
|
||||
#include <string>
|
||||
|
||||
// A WireGuard tunnel's configuration (issue #8, N1 Q243-Q253): read from the standard `.conf` a
|
||||
// server's owner hands out, checked, and written back in a tidy form for the device's settings.
|
||||
// One peer, IPv4. The keys are never put in a message: errors name the line and the field.
|
||||
namespace roro::net {
|
||||
|
||||
struct WgRange {
|
||||
uint32_t address = 0;
|
||||
int prefix = 0;
|
||||
};
|
||||
|
||||
struct WgConfig {
|
||||
static constexpr int kMaxRanges = 4;
|
||||
|
||||
std::string privateKey, peerKey, presharedKey; // base64, as in the file; the last may be empty
|
||||
uint32_t address = 0; // the tunnel's address on this device
|
||||
int prefix = 32;
|
||||
uint32_t dns[2] = {0, 0};
|
||||
int mtu = 0; // 0: WireGuard's 1420
|
||||
uint16_t listenPort = 0; // 0: any; a fixed one lets the peer be the one that calls
|
||||
std::string endpointHost;
|
||||
uint16_t endpointPort = 51820;
|
||||
WgRange allowed[kMaxRanges];
|
||||
int allowedCount = 0;
|
||||
int keepalive = 25; // seconds; what the file says, or 25: this device is always behind a NAT
|
||||
};
|
||||
|
||||
// "" and `out` filled, or why the file can't be used ("line 7: ...").
|
||||
std::string parseWgConf(const std::string& text, WgConfig& out);
|
||||
// The same configuration as a `.conf` again: what the settings keep.
|
||||
std::string toWgConf(const WgConfig& config);
|
||||
bool validWgKey(const std::string& key); // 32 bytes in base64
|
||||
|
||||
// What can go through the tunnel. The network stack routes by an interface's own subnet or by
|
||||
// default, nothing finer: so either everything goes through it (AllowedIPs has 0.0.0.0/0), or the
|
||||
// one subnet this device's tunnel address is in. Ranges that are neither can't be reached, and
|
||||
// the user is told how many.
|
||||
struct WgRouting {
|
||||
bool full = false; // the tunnel is the default route
|
||||
int prefix = 32; // of the tunnel interface, when not full
|
||||
int unreachable = 0; // allowed ranges outside it
|
||||
};
|
||||
WgRouting routingOf(const WgConfig& config);
|
||||
bool wgReaches(const WgConfig& config, uint32_t address); // would a packet to this address go through it?
|
||||
|
||||
// For the screen and the console: never a key.
|
||||
std::string describeWgRouting(const WgConfig& config);
|
||||
|
||||
} // namespace roro::net
|
||||
@@ -0,0 +1,622 @@
|
||||
#include "note_document.h"
|
||||
|
||||
#include <algorithm>
|
||||
#include <cstring>
|
||||
|
||||
namespace roro::notes {
|
||||
|
||||
namespace {
|
||||
// The side file: this line, the note's size and two checksums of it (its first and last
|
||||
// kilobyte), then, in any order, text that left a window and snapshots of the list of pieces.
|
||||
// snapshot: "RSNP" cursor count { src at len }... crc32 length "PNSR" (numbers: 32 bits, low byte first)
|
||||
// The newest snapshot that checks out is the note as it was last saved. kDone at the very end:
|
||||
// the rewrite this file was for is complete in `<note>.tmp`, and only has to take the note's place.
|
||||
const char kMagic[] = "roro9stack note edits 1\n";
|
||||
constexpr size_t kMagicLen = sizeof(kMagic) - 1;
|
||||
constexpr size_t kHeaderLen = kMagicLen + 12;
|
||||
const char kSnap[] = "RSNP", kSnapEnd[] = "PNSR", kDone[] = "RDONE1\n\n";
|
||||
constexpr size_t kDoneLen = 8;
|
||||
constexpr size_t kCheck = 1024; // of each end of the note, in the header
|
||||
constexpr uint32_t kStepBytes = 64 * 1024; // a rewrite's step
|
||||
constexpr size_t kBlock = 4096;
|
||||
constexpr uint32_t kSeekNewline = 1024;
|
||||
constexpr size_t kMaxSnapshot = 12 + 9 * 4096 + 12;
|
||||
|
||||
bool continuation(int c) { return (c & 0xC0) == 0x80; }
|
||||
|
||||
uint32_t crc32(uint32_t crc, const uint8_t* data, size_t len) {
|
||||
crc = ~crc;
|
||||
for (size_t i = 0; i < len; i++) {
|
||||
crc ^= data[i];
|
||||
for (int k = 0; k < 8; k++) crc = (crc >> 1) ^ (0xEDB88320u & (0u - (crc & 1)));
|
||||
}
|
||||
return ~crc;
|
||||
}
|
||||
|
||||
void put32(std::string& s, uint32_t v) {
|
||||
for (int i = 0; i < 4; i++) s += static_cast<char>((v >> (8 * i)) & 0xFF);
|
||||
}
|
||||
|
||||
uint32_t get32(const uint8_t* p) { return p[0] | (p[1] << 8) | (p[2] << 16) | (static_cast<uint32_t>(p[3]) << 24); }
|
||||
|
||||
const uint8_t* bytes(const std::string& s) { return reinterpret_cast<const uint8_t*>(s.data()); }
|
||||
} // namespace
|
||||
|
||||
NoteDocument::NoteDocument(NoteCard& card, int cols, int rows) : card_(card), text_(cols, rows) { openNew(); }
|
||||
|
||||
void NoteDocument::reset() {
|
||||
path_.clear();
|
||||
sidePath_.clear();
|
||||
pieces_.clear();
|
||||
loaded_.clear();
|
||||
win_ = 0;
|
||||
windowLoaded_ = false;
|
||||
before_ = after_ = 0;
|
||||
droppedAtLoad_ = newlinesAtLoad_ = 0;
|
||||
flushedSinceSave_ = sidePending_ = false;
|
||||
sideSize_ = 0;
|
||||
windowSaved_.valid = false;
|
||||
rw_.active = false;
|
||||
std::vector<uint8_t>().swap(rw_.block);
|
||||
}
|
||||
|
||||
void NoteDocument::openNew() {
|
||||
reset();
|
||||
text_.buffer().clear();
|
||||
text_.refilled(0, 0);
|
||||
windowLoaded_ = true;
|
||||
loadedRevision_ = savedRevision_ = text_.revision();
|
||||
}
|
||||
|
||||
std::string NoteDocument::open(const std::string& path, std::string* told) {
|
||||
openNew();
|
||||
path_ = path;
|
||||
sidePath_ = side();
|
||||
uint32_t sideSize = 0, fileSize = 0, other = 0;
|
||||
bool hasSide = card_.size(sidePath_, sideSize);
|
||||
if (hasSide && sideIsDone(sideSize)) { // a rewrite was cut after its last write: finish it
|
||||
if (card_.size(tmp(), other)) {
|
||||
if (card_.size(path_, fileSize)) card_.remove(path_);
|
||||
card_.rename(tmp(), path_);
|
||||
}
|
||||
card_.remove(sidePath_);
|
||||
hasSide = false;
|
||||
}
|
||||
if (!card_.size(path_, fileSize)) {
|
||||
card_.done();
|
||||
openNew();
|
||||
return "The card refused to open it";
|
||||
}
|
||||
if (fileSize > NoteText::kMaxBytes && card_.freeBytes() < static_cast<uint64_t>(fileSize) + 16 * 1024) {
|
||||
card_.done();
|
||||
openNew();
|
||||
return "Not enough room on the card: saving it needs a second copy";
|
||||
}
|
||||
uint32_t cursor = 0;
|
||||
bool resumed = false;
|
||||
if (hasSide) {
|
||||
if (card_.size(tmp(), other)) card_.remove(tmp()); // a rewrite that didn't get that far
|
||||
sideSize_ = sideSize;
|
||||
Resume r = resume(fileSize, cursor);
|
||||
if (r == Resume::Ok) {
|
||||
resumed = sidePending_ = true;
|
||||
if (told) *told = "Your unsaved changes are back";
|
||||
} else {
|
||||
sideSize_ = 0;
|
||||
pieces_.clear();
|
||||
if (r == Resume::Mismatch) { // typed text is never thrown away without a word (Q227)
|
||||
std::string lost = sidePath_ + ".lost";
|
||||
card_.remove(lost);
|
||||
card_.rename(sidePath_, lost);
|
||||
if (told) *told = "The file changed: unsaved edits kept as .edit.lost";
|
||||
} else {
|
||||
card_.remove(sidePath_);
|
||||
}
|
||||
}
|
||||
}
|
||||
if (!resumed && fileSize) pieces_.push_back({0, 0, fileSize});
|
||||
windowLoaded_ = false;
|
||||
bool ok = load(cursor, cursor ? 1000 : 0, -1); // an edit picked up: its last lines above the cursor
|
||||
card_.done();
|
||||
if (!ok) {
|
||||
openNew();
|
||||
return "The card refused to read it";
|
||||
}
|
||||
savedRevision_ = text_.revision();
|
||||
return "";
|
||||
}
|
||||
|
||||
uint32_t NoteDocument::piecesBytes() const {
|
||||
uint32_t n = 0;
|
||||
for (const Piece& p : pieces_) n += p.len;
|
||||
return n;
|
||||
}
|
||||
|
||||
int NoteDocument::percent() const {
|
||||
uint32_t all = size();
|
||||
return all ? static_cast<int>(static_cast<uint64_t>(before_ + text_.top()) * 100 / all) : 0;
|
||||
}
|
||||
|
||||
size_t NoteDocument::readDoc(uint32_t at, uint8_t* into, size_t len) {
|
||||
size_t got = 0;
|
||||
uint32_t pos = 0;
|
||||
for (const Piece& p : pieces_) {
|
||||
if (got == len) break;
|
||||
if (at < pos + p.len) {
|
||||
uint32_t skip = at - pos;
|
||||
size_t n = std::min<size_t>(len - got, p.len - skip);
|
||||
size_t r = card_.read(fileOf(p), p.at + skip, into + got, n);
|
||||
got += r;
|
||||
at += static_cast<uint32_t>(r);
|
||||
if (r != n) break;
|
||||
}
|
||||
pos += p.len;
|
||||
}
|
||||
return got;
|
||||
}
|
||||
|
||||
int NoteDocument::byteAt(uint32_t at) {
|
||||
uint8_t b;
|
||||
return readDoc(at, &b, 1) == 1 ? b : -1;
|
||||
}
|
||||
|
||||
size_t NoteDocument::splitAt(uint32_t at) {
|
||||
uint32_t pos = 0;
|
||||
for (size_t i = 0; i < pieces_.size(); i++) {
|
||||
if (at == pos) return i;
|
||||
Piece& p = pieces_[i];
|
||||
if (at < pos + p.len) {
|
||||
uint32_t first = at - pos;
|
||||
Piece rest{p.src, p.at + first, p.len - first};
|
||||
p.len = first;
|
||||
pieces_.insert(pieces_.begin() + static_cast<long>(i) + 1, rest);
|
||||
return i + 1;
|
||||
}
|
||||
pos += p.len;
|
||||
}
|
||||
return pieces_.size();
|
||||
}
|
||||
|
||||
void NoteDocument::merge() {
|
||||
size_t kept = 0;
|
||||
for (size_t i = 0; i < pieces_.size(); i++) {
|
||||
const Piece p = pieces_[i];
|
||||
if (!p.len) continue;
|
||||
if (kept && pieces_[kept - 1].src == p.src && pieces_[kept - 1].at + pieces_[kept - 1].len == p.at) pieces_[kept - 1].len += p.len;
|
||||
else pieces_[kept++] = p;
|
||||
}
|
||||
pieces_.resize(kept);
|
||||
}
|
||||
|
||||
// The window dropped the CRs of the file's CRLFs when it was read. If it's put back untouched,
|
||||
// the cursor is further along in the file than in the window: by one for each line before it.
|
||||
uint32_t NoteDocument::noteCursor() const {
|
||||
size_t c = text_.cursor();
|
||||
if (windowLoaded_ && droppedAtLoad_ && text_.revision() == loadedRevision_) {
|
||||
const std::string& t = text_.text();
|
||||
if (droppedAtLoad_ == newlinesAtLoad_) c += static_cast<size_t>(std::count(t.begin(), t.begin() + static_cast<long>(c), '\n'));
|
||||
else if (!t.empty()) c += droppedAtLoad_ * c / t.size(); // a file of both kinds of line: near enough
|
||||
}
|
||||
return before_ + static_cast<uint32_t>(c);
|
||||
}
|
||||
|
||||
bool NoteDocument::headerFor(std::string& header) {
|
||||
uint32_t fileSize = 0;
|
||||
if (path_.empty() || !card_.size(path_, fileSize)) return false;
|
||||
std::vector<uint8_t> buf(kCheck);
|
||||
size_t n = std::min<size_t>(kCheck, fileSize);
|
||||
if (card_.read(path_, 0, buf.data(), n) != n) return false;
|
||||
uint32_t head = crc32(0, buf.data(), n);
|
||||
if (card_.read(path_, fileSize - static_cast<uint32_t>(n), buf.data(), n) != n) return false;
|
||||
uint32_t tail = crc32(0, buf.data(), n);
|
||||
header.assign(kMagic, kMagicLen);
|
||||
put32(header, fileSize);
|
||||
put32(header, head);
|
||||
put32(header, tail);
|
||||
return true;
|
||||
}
|
||||
|
||||
bool NoteDocument::ensureSide(std::string& why) {
|
||||
if (sideSize_) return true;
|
||||
std::string header;
|
||||
if (!headerFor(header)) {
|
||||
why = path_.empty() ? "the note has no file yet" : "the card refused to read the note";
|
||||
return false;
|
||||
}
|
||||
if (!card_.create(sidePath_) || !card_.append(sidePath_, bytes(header), header.size())) {
|
||||
card_.remove(sidePath_);
|
||||
why = "the card refused a write";
|
||||
return false;
|
||||
}
|
||||
sideSize_ = static_cast<uint32_t>(header.size());
|
||||
return true;
|
||||
}
|
||||
|
||||
bool NoteDocument::putBack(std::string& why) {
|
||||
if (!windowLoaded_) return true;
|
||||
if (text_.revision() == loadedRevision_) {
|
||||
pieces_.insert(pieces_.begin() + static_cast<long>(win_), loaded_.begin(), loaded_.end());
|
||||
} else {
|
||||
Piece p{1, 0, static_cast<uint32_t>(text_.size())};
|
||||
if (windowSaved_.valid && windowSaved_.revision == text_.revision()) {
|
||||
p.at = windowSaved_.at;
|
||||
} else if (p.len) {
|
||||
if (!ensureSide(why)) return false;
|
||||
if (!card_.append(sidePath_, bytes(text_.text()), p.len)) {
|
||||
card_.done();
|
||||
if (!card_.size(sidePath_, sideSize_)) sideSize_ = 0;
|
||||
why = "the card refused a write";
|
||||
return false;
|
||||
}
|
||||
p.at = sideSize_;
|
||||
sideSize_ += p.len;
|
||||
}
|
||||
if (p.len) pieces_.insert(pieces_.begin() + static_cast<long>(win_), p);
|
||||
flushedSinceSave_ = sidePending_ = true;
|
||||
}
|
||||
windowLoaded_ = false;
|
||||
loaded_.clear();
|
||||
windowSaved_.valid = false;
|
||||
before_ = after_ = 0;
|
||||
merge();
|
||||
return true;
|
||||
}
|
||||
|
||||
// The window's start is where a line starts on screen whenever that can be known: after a
|
||||
// newline, or where the window before had a line start. Otherwise the same text could wrap
|
||||
// differently from one window to the next.
|
||||
bool NoteDocument::load(uint32_t cursor, int row, int64_t startHint) {
|
||||
uint32_t total = piecesBytes();
|
||||
cursor = std::min(cursor, total);
|
||||
uint32_t s = 0, e = total;
|
||||
if (total > NoteText::kMaxBytes - kEdge) {
|
||||
uint32_t c = cursor > kHalf ? cursor - kHalf : 0;
|
||||
if (c == 0) {
|
||||
s = 0;
|
||||
} else if (startHint >= 0 && startHint <= static_cast<int64_t>(c)) {
|
||||
s = static_cast<uint32_t>(startHint);
|
||||
} else {
|
||||
uint8_t buf[128];
|
||||
uint32_t at = c, limit = std::min(c + kSeekNewline, cursor);
|
||||
bool found = false;
|
||||
while (at < limit && !found) {
|
||||
size_t n = readDoc(at, buf, std::min<size_t>(sizeof buf, limit - at));
|
||||
if (!n) break;
|
||||
for (size_t i = 0; i < n && !found; i++)
|
||||
if (buf[i] == '\n') {
|
||||
s = at + static_cast<uint32_t>(i) + 1;
|
||||
found = true;
|
||||
}
|
||||
at += static_cast<uint32_t>(n);
|
||||
}
|
||||
if (!found) {
|
||||
s = c;
|
||||
for (int k = 0; k < 3 && s < cursor && continuation(byteAt(s)); k++) s++;
|
||||
}
|
||||
}
|
||||
e = std::min(total, cursor + kHalf);
|
||||
for (int k = 0; k < 3 && e < total && continuation(byteAt(e)); k++) e++;
|
||||
if (e < total && e > 0 && byteAt(e) == '\n' && byteAt(e - 1) == '\r') e++;
|
||||
e = std::min<uint32_t>(e, s + NoteText::kMaxBytes);
|
||||
}
|
||||
size_t i0 = splitAt(s), i1 = splitAt(e);
|
||||
loaded_.assign(pieces_.begin() + static_cast<long>(i0), pieces_.begin() + static_cast<long>(i1));
|
||||
pieces_.erase(pieces_.begin() + static_cast<long>(i0), pieces_.begin() + static_cast<long>(i1));
|
||||
win_ = i0;
|
||||
before_ = s;
|
||||
after_ = total - e;
|
||||
std::string& b = text_.buffer();
|
||||
b.resize(e - s);
|
||||
size_t got = 0;
|
||||
bool ok = true;
|
||||
for (const Piece& p : loaded_) {
|
||||
size_t n = card_.read(fileOf(p), p.at, reinterpret_cast<uint8_t*>(&b[got]), p.len);
|
||||
got += n;
|
||||
if (n != p.len) {
|
||||
ok = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!ok) { // the note is whole in its pieces: stand on an empty window where the cursor was
|
||||
pieces_.insert(pieces_.begin() + static_cast<long>(i0), loaded_.begin(), loaded_.end());
|
||||
loaded_.clear();
|
||||
win_ = splitAt(cursor);
|
||||
before_ = cursor;
|
||||
after_ = total - cursor;
|
||||
s = cursor;
|
||||
b.clear();
|
||||
}
|
||||
droppedAtLoad_ = text_.refilled(cursor - s, row);
|
||||
newlinesAtLoad_ = static_cast<size_t>(std::count(b.begin(), b.end(), '\n'));
|
||||
loadedRevision_ = text_.revision();
|
||||
windowLoaded_ = true;
|
||||
windowSaved_.valid = false;
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool NoteDocument::wantsMove() const {
|
||||
if (!windowLoaded_) return true;
|
||||
size_t n = text_.size(), c = text_.cursor();
|
||||
if (n + kSpare >= NoteText::kMaxBytes) return true;
|
||||
if (before_ && c < kEdge) return true;
|
||||
return after_ && n - c < kEdge;
|
||||
}
|
||||
|
||||
bool NoteDocument::move(std::string& why) {
|
||||
uint32_t cursor = noteCursor();
|
||||
int row = text_.cursorRow();
|
||||
int64_t hint = -1;
|
||||
if (windowLoaded_ && !droppedAtLoad_ && cursor > kHalf) {
|
||||
uint32_t c = cursor - kHalf;
|
||||
if (c >= before_ && c < before_ + text_.size()) hint = static_cast<int64_t>(before_) + static_cast<int64_t>(text_.startOfLine(c - before_));
|
||||
}
|
||||
if (!putBack(why)) return false;
|
||||
bool ok = load(cursor, row, hint);
|
||||
card_.done();
|
||||
if (!ok) why = "the card refused to read";
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool NoteDocument::jump(uint32_t to, std::string& why) {
|
||||
to = std::min(to, size());
|
||||
if (windowLoaded_ && to == 0 && !before_) return text_.toStart(), true;
|
||||
if (windowLoaded_ && to == size() && !after_) return text_.toEnd(), true;
|
||||
if (!putBack(why)) return false;
|
||||
bool ok = load(to, to ? 1000 : 0, -1);
|
||||
card_.done();
|
||||
if (!ok) why = "the card refused to read";
|
||||
return ok;
|
||||
}
|
||||
|
||||
bool NoteDocument::wantsRewrite() const { return size() <= kWholeLimit || sideSize_ > kSideLimit || pieces_.size() > kManyPieces; }
|
||||
|
||||
bool NoteDocument::journal(std::string& why) {
|
||||
if (path_.empty()) {
|
||||
why = "the note has no file yet";
|
||||
return false;
|
||||
}
|
||||
bool modified = text_.revision() != loadedRevision_;
|
||||
Piece w{1, 0, static_cast<uint32_t>(text_.size())};
|
||||
uint32_t cursor = noteCursor();
|
||||
if (!ensureSide(why)) return false;
|
||||
bool wrote = true;
|
||||
if (modified && w.len) {
|
||||
if (windowSaved_.valid && windowSaved_.revision == text_.revision()) {
|
||||
w.at = windowSaved_.at;
|
||||
} else if ((wrote = card_.append(sidePath_, bytes(text_.text()), w.len))) {
|
||||
w.at = sideSize_;
|
||||
sideSize_ += w.len;
|
||||
windowSaved_.valid = true;
|
||||
windowSaved_.at = w.at;
|
||||
windowSaved_.len = w.len;
|
||||
windowSaved_.revision = text_.revision();
|
||||
}
|
||||
}
|
||||
if (wrote) {
|
||||
std::vector<Piece> all(pieces_.begin(), pieces_.begin() + static_cast<long>(win_));
|
||||
if (!modified) all.insert(all.end(), loaded_.begin(), loaded_.end());
|
||||
else if (w.len) all.push_back(w);
|
||||
all.insert(all.end(), pieces_.begin() + static_cast<long>(win_), pieces_.end());
|
||||
std::string rec(kSnap, 4);
|
||||
put32(rec, cursor);
|
||||
put32(rec, static_cast<uint32_t>(all.size()));
|
||||
for (const Piece& p : all) {
|
||||
rec += static_cast<char>(p.src);
|
||||
put32(rec, p.at);
|
||||
put32(rec, p.len);
|
||||
}
|
||||
put32(rec, crc32(0, bytes(rec), rec.size()));
|
||||
put32(rec, static_cast<uint32_t>(rec.size()) + 8);
|
||||
rec.append(kSnapEnd, 4);
|
||||
wrote = card_.append(sidePath_, bytes(rec), rec.size());
|
||||
if (wrote) sideSize_ += static_cast<uint32_t>(rec.size());
|
||||
}
|
||||
card_.done();
|
||||
if (!wrote) {
|
||||
windowSaved_.valid = false;
|
||||
if (!card_.size(sidePath_, sideSize_)) sideSize_ = 0;
|
||||
why = "the card refused a write";
|
||||
return false;
|
||||
}
|
||||
savedRevision_ = text_.revision();
|
||||
flushedSinceSave_ = false;
|
||||
sidePending_ = true;
|
||||
return true;
|
||||
}
|
||||
|
||||
bool NoteDocument::rewriteStart(std::string& why) {
|
||||
if (path_.empty()) {
|
||||
why = "the note has no file yet";
|
||||
return false;
|
||||
}
|
||||
if (card_.freeBytes() < static_cast<uint64_t>(size()) + 16 * 1024) {
|
||||
why = "the card is full";
|
||||
return false;
|
||||
}
|
||||
if (!card_.create(tmp())) {
|
||||
why = "the card refused to open a file";
|
||||
return false;
|
||||
}
|
||||
rw_.active = true;
|
||||
rw_.stage = 0;
|
||||
rw_.index = 0;
|
||||
rw_.offset = rw_.done = rw_.wrote = rw_.wroteBefore = rw_.wroteAfter = 0;
|
||||
rw_.total = size();
|
||||
rw_.revision = text_.revision();
|
||||
rw_.carry = false;
|
||||
rw_.block.resize(kBlock);
|
||||
return true;
|
||||
}
|
||||
|
||||
int NoteDocument::rewriteStep(std::string& why) {
|
||||
if (!rw_.active) return -1;
|
||||
auto failed = [&](const char* what) {
|
||||
card_.done();
|
||||
card_.remove(tmp());
|
||||
rw_.active = false;
|
||||
std::vector<uint8_t>().swap(rw_.block);
|
||||
why = what;
|
||||
return -1;
|
||||
};
|
||||
auto out = [&](const uint8_t* data, size_t len) {
|
||||
if (!len) return true;
|
||||
if (!card_.append(tmp(), data, len)) return false;
|
||||
rw_.wrote += static_cast<uint32_t>(len);
|
||||
if (rw_.stage == 0) rw_.wroteBefore += static_cast<uint32_t>(len);
|
||||
if (rw_.stage == 2) rw_.wroteAfter += static_cast<uint32_t>(len);
|
||||
return true;
|
||||
};
|
||||
const uint8_t cr = '\r';
|
||||
uint32_t budget = kStepBytes;
|
||||
while (budget > 0 && rw_.stage < 3) {
|
||||
if (rw_.stage == 1) {
|
||||
uint32_t left = static_cast<uint32_t>(text_.size()) - rw_.offset;
|
||||
if (!left) {
|
||||
rw_.stage = 2;
|
||||
rw_.index = win_;
|
||||
rw_.offset = 0;
|
||||
continue;
|
||||
}
|
||||
uint32_t n = std::min(left, budget);
|
||||
if (!out(bytes(text_.text()) + rw_.offset, n)) return failed("the card refused a write");
|
||||
rw_.offset += n;
|
||||
rw_.done += n;
|
||||
budget -= n;
|
||||
continue;
|
||||
}
|
||||
size_t end = rw_.stage == 0 ? win_ : pieces_.size();
|
||||
bool pieceOver = rw_.index < end && rw_.offset >= pieces_[rw_.index].len;
|
||||
if (rw_.index >= end || pieceOver) {
|
||||
if (rw_.carry && !out(&cr, 1)) return failed("the card refused a write"); // a CR that ended its piece stays
|
||||
rw_.carry = false;
|
||||
rw_.offset = 0;
|
||||
if (pieceOver) rw_.index++;
|
||||
else rw_.stage++;
|
||||
continue;
|
||||
}
|
||||
const Piece& p = pieces_[rw_.index];
|
||||
uint32_t n = std::min<uint32_t>(std::min<uint32_t>(p.len - rw_.offset, budget), static_cast<uint32_t>(rw_.block.size()));
|
||||
uint8_t* b = rw_.block.data();
|
||||
if (card_.read(fileOf(p), p.at + rw_.offset, b, n) != n) return failed("the card refused to read the note");
|
||||
size_t m = n;
|
||||
if (p.src == 0) { // CRLF becomes LF (Q148, Q230), in the file's own text: what was typed has none
|
||||
if (rw_.carry && b[0] != '\n' && !out(&cr, 1)) return failed("the card refused a write");
|
||||
rw_.carry = false;
|
||||
m = 0;
|
||||
for (uint32_t i = 0; i < n; i++) {
|
||||
if (b[i] == '\r') {
|
||||
if (i + 1 == n) {
|
||||
rw_.carry = true;
|
||||
continue;
|
||||
}
|
||||
if (b[i + 1] == '\n') continue;
|
||||
}
|
||||
b[m++] = b[i];
|
||||
}
|
||||
}
|
||||
if (!out(b, m)) return failed("the card refused a write");
|
||||
rw_.offset += n;
|
||||
rw_.done += n;
|
||||
budget -= n;
|
||||
}
|
||||
if (rw_.stage < 3) return std::min(99, static_cast<int>(static_cast<uint64_t>(rw_.done) * 100 / std::max<uint32_t>(1, rw_.total)));
|
||||
|
||||
// All of it is in the temporary file. From the mark in the side file on, the rewrite counts as
|
||||
// done: whatever is cut after that, opening the note finishes it.
|
||||
card_.done();
|
||||
uint32_t written = 0, old = 0;
|
||||
if (!card_.size(tmp(), written) || written != rw_.wrote) return failed("the card refused a write");
|
||||
if (sideSize_) {
|
||||
if (!card_.append(sidePath_, reinterpret_cast<const uint8_t*>(kDone), kDoneLen)) return failed("the card refused a write");
|
||||
sideSize_ += kDoneLen;
|
||||
card_.done();
|
||||
}
|
||||
rw_.active = false;
|
||||
std::vector<uint8_t>().swap(rw_.block);
|
||||
// FAT can't rename onto a file. Between these two lines only the temporary file exists: the
|
||||
// Notes list puts such a file back under its name.
|
||||
if ((card_.size(path_, old) && !card_.remove(path_)) || !card_.rename(tmp(), path_)) {
|
||||
card_.done();
|
||||
why = "the card refused to replace the note";
|
||||
return -1;
|
||||
}
|
||||
if (sideSize_) card_.remove(sidePath_);
|
||||
card_.done();
|
||||
sideSize_ = 0;
|
||||
uint32_t window = static_cast<uint32_t>(text_.size());
|
||||
pieces_.clear();
|
||||
loaded_.clear();
|
||||
if (rw_.wroteBefore) pieces_.push_back({0, 0, rw_.wroteBefore});
|
||||
win_ = pieces_.size();
|
||||
if (rw_.wroteAfter) pieces_.push_back({0, rw_.wroteBefore + window, rw_.wroteAfter});
|
||||
if (window) loaded_.push_back({0, rw_.wroteBefore, window});
|
||||
before_ = rw_.wroteBefore;
|
||||
after_ = rw_.wroteAfter;
|
||||
loadedRevision_ = savedRevision_ = rw_.revision;
|
||||
droppedAtLoad_ = 0;
|
||||
windowLoaded_ = true;
|
||||
flushedSinceSave_ = sidePending_ = false;
|
||||
windowSaved_.valid = false;
|
||||
return 100;
|
||||
}
|
||||
|
||||
bool NoteDocument::sideIsDone(uint32_t sideSize) {
|
||||
uint8_t tail[kDoneLen];
|
||||
return sideSize >= kHeaderLen + kDoneLen && card_.read(sidePath_, sideSize - kDoneLen, tail, kDoneLen) == kDoneLen &&
|
||||
std::memcmp(tail, kDone, kDoneLen) == 0;
|
||||
}
|
||||
|
||||
NoteDocument::Resume NoteDocument::resume(uint32_t fileSize, uint32_t& cursor) {
|
||||
uint8_t header[kHeaderLen];
|
||||
if (sideSize_ < kHeaderLen || card_.read(sidePath_, 0, header, kHeaderLen) != kHeaderLen || std::memcmp(header, kMagic, kMagicLen) != 0)
|
||||
return Resume::Nothing;
|
||||
|
||||
// The newest snapshot that checks out, looking back from the end: after it there may be text
|
||||
// that left a window, or a write the power cut short.
|
||||
auto snapshotEndingAt = [&](uint32_t end) {
|
||||
uint8_t lenBytes[4];
|
||||
if (end < kHeaderLen + 24 || card_.read(sidePath_, end - 8, lenBytes, 4) != 4) return false;
|
||||
uint32_t len = get32(lenBytes);
|
||||
if (len < 24 || len > kMaxSnapshot || len > end - kHeaderLen || (len - 24) % 9) return false;
|
||||
std::string rec(len, '\0');
|
||||
if (card_.read(sidePath_, end - len, reinterpret_cast<uint8_t*>(&rec[0]), len) != len) return false;
|
||||
const uint8_t* r = bytes(rec);
|
||||
if (std::memcmp(r, kSnap, 4) != 0 || get32(r + len - 12) != crc32(0, r, len - 12)) return false;
|
||||
uint32_t count = get32(r + 8);
|
||||
if (count != (len - 24) / 9) return false;
|
||||
std::vector<Piece> list;
|
||||
list.reserve(count);
|
||||
for (uint32_t i = 0; i < count; i++) {
|
||||
const uint8_t* q = r + 12 + 9 * i;
|
||||
Piece p{q[0], get32(q + 1), get32(q + 5)};
|
||||
uint64_t stop = static_cast<uint64_t>(p.at) + p.len;
|
||||
if (p.src > 1 || !p.len) return false;
|
||||
if (p.src == 1 && (p.at < kHeaderLen || stop > end - len)) return false;
|
||||
list.push_back(p);
|
||||
}
|
||||
pieces_.swap(list);
|
||||
cursor = get32(r + 4);
|
||||
return true;
|
||||
};
|
||||
bool found = false;
|
||||
std::vector<uint8_t> buf(1024 + 3);
|
||||
for (uint32_t end = sideSize_; end > kHeaderLen && !found;) {
|
||||
uint32_t a = end > 1024 + kHeaderLen ? end - 1024 : static_cast<uint32_t>(kHeaderLen);
|
||||
size_t n = card_.read(sidePath_, a, buf.data(), std::min<size_t>(buf.size(), sideSize_ - a));
|
||||
for (size_t i = n >= 4 ? n - 4 + 1 : 0; i-- > 0 && !found;)
|
||||
if (std::memcmp(buf.data() + i, kSnapEnd, 4) == 0) found = snapshotEndingAt(a + static_cast<uint32_t>(i) + 4);
|
||||
end = a;
|
||||
}
|
||||
if (!found) return Resume::Nothing;
|
||||
std::string expect;
|
||||
if (!headerFor(expect) || std::memcmp(header, expect.data(), kHeaderLen) != 0) {
|
||||
pieces_.clear();
|
||||
return Resume::Mismatch;
|
||||
}
|
||||
for (const Piece& p : pieces_)
|
||||
if (p.src == 0 && static_cast<uint64_t>(p.at) + p.len > fileSize) return pieces_.clear(), Resume::Mismatch;
|
||||
merge();
|
||||
return Resume::Ok;
|
||||
}
|
||||
|
||||
} // namespace roro::notes
|
||||
@@ -0,0 +1,130 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
#include "note_text.h"
|
||||
|
||||
namespace roro::notes {
|
||||
|
||||
// The card, as a note needs it. On the device every call is made on the storage task.
|
||||
class NoteCard {
|
||||
public:
|
||||
virtual ~NoteCard() = default;
|
||||
virtual bool size(const std::string& path, uint32_t& size) = 0; // false: no such file
|
||||
virtual size_t read(const std::string& path, uint32_t at, uint8_t* into, size_t len) = 0;
|
||||
virtual bool create(const std::string& path) = 0; // an empty file, in place of what was there
|
||||
virtual bool append(const std::string& path, const uint8_t* data, size_t len) = 0;
|
||||
virtual bool remove(const std::string& path) = 0;
|
||||
virtual bool rename(const std::string& from, const std::string& to) = 0;
|
||||
virtual uint64_t freeBytes() = 0;
|
||||
virtual void done() {} // what was appended is on the card now (files kept open are closed)
|
||||
};
|
||||
|
||||
// A text file of any size, edited (issue #47, F1 Q223-Q232). The file stays on the card; what is
|
||||
// in memory is one window of it, a NoteText of up to 16 KB around the cursor, and a list of pieces
|
||||
// saying what the rest is made of: runs of bytes of the file, and runs of the side file
|
||||
// `<note>.edit`, where a window that was changed is written when the cursor leaves it.
|
||||
//
|
||||
// the note = pieces before the window + the window + pieces after it
|
||||
//
|
||||
// Saving comes in two kinds. `journal` appends the window and the list of pieces to the side
|
||||
// file: quick whatever the note's size, and enough to pick the edit up after a power cut.
|
||||
// `rewrite` streams the whole note into `<note>.tmp` and puts it in the note's place: the file is
|
||||
// then the note again, and the side file goes. A note of up to 64 KB is always rewritten.
|
||||
//
|
||||
// Every method marked [card] reads or writes the card.
|
||||
class NoteDocument {
|
||||
public:
|
||||
static constexpr uint32_t kHalf = 4096; // loaded on each side of the cursor
|
||||
static constexpr uint32_t kEdge = 2048; // this near an end of the window, it moves
|
||||
static constexpr uint32_t kSpare = 256; // this near full, it moves
|
||||
static constexpr uint32_t kWholeLimit = 64 * 1024; // up to here a save is a rewrite
|
||||
static constexpr uint32_t kSideLimit = 1024 * 1024; // a side file this big asks for a rewrite
|
||||
static constexpr size_t kManyPieces = 256; // and so does a list this long
|
||||
|
||||
NoteDocument(NoteCard& card, int cols, int rows);
|
||||
|
||||
// [card] "" or why not. `told`: something the user should read (an edit picked up, or set aside).
|
||||
std::string open(const std::string& path, std::string* told = nullptr);
|
||||
void openNew(); // nothing on the card until the first rewrite
|
||||
const std::string& path() const { return path_; }
|
||||
void setPath(const std::string& path) { // a new note's, before its first rewrite
|
||||
path_ = path;
|
||||
sidePath_ = path.empty() ? "" : side();
|
||||
}
|
||||
|
||||
NoteText& text() { return text_; }
|
||||
const NoteText& text() const { return text_; }
|
||||
uint32_t size() const { return before_ + static_cast<uint32_t>(text_.size()) + after_; }
|
||||
uint32_t cursor() const { return before_ + static_cast<uint32_t>(text_.cursor()); } // in the note
|
||||
int percent() const;
|
||||
bool windowed() const { return before_ || after_; } // the note is more than its window
|
||||
|
||||
// After each key: the cursor is near an end of the window that isn't an end of the note, or
|
||||
// the window is nearly full.
|
||||
bool wantsMove() const;
|
||||
bool move(std::string& why); // [card] the window, around the cursor
|
||||
bool jump(uint32_t to, std::string& why); // [card] the cursor, anywhere in the note
|
||||
|
||||
bool dirty() const { return text_.revision() != savedRevision_ || flushedSinceSave_; } // the card doesn't have it
|
||||
bool filePending() const { return sidePending_; } // saved, but in the side file: a rewrite is owed
|
||||
bool wantsRewrite() const; // the next save should be a rewrite
|
||||
bool journal(std::string& why); // [card]
|
||||
bool rewriteStart(std::string& why); // [card]
|
||||
int rewriteStep(std::string& why); // [card] percent done; 100: the file is the note; -1: failed
|
||||
bool rewriting() const { return rw_.active; }
|
||||
|
||||
private:
|
||||
struct Piece {
|
||||
uint8_t src; // 0: the note's file, 1: the side file
|
||||
uint32_t at, len;
|
||||
};
|
||||
enum class Resume { Ok, Mismatch, Nothing };
|
||||
|
||||
std::string side() const { return path_ + ".edit"; }
|
||||
std::string tmp() const { return path_ + ".tmp"; }
|
||||
const std::string& fileOf(const Piece& p) const { return p.src ? sidePath_ : path_; }
|
||||
uint32_t piecesBytes() const;
|
||||
size_t readDoc(uint32_t at, uint8_t* into, size_t len); // from the pieces: the window is put back first
|
||||
int byteAt(uint32_t at);
|
||||
size_t splitAt(uint32_t at); // the index of the piece that starts there
|
||||
void merge();
|
||||
uint32_t noteCursor() const; // where the cursor is among the pieces once the window is put back
|
||||
bool putBack(std::string& why);
|
||||
bool load(uint32_t cursor, int row, int64_t startHint); // false: the card refused, and the window is empty
|
||||
bool ensureSide(std::string& why);
|
||||
bool headerFor(std::string& header);
|
||||
Resume resume(uint32_t fileSize, uint32_t& cursor);
|
||||
bool sideIsDone(uint32_t sideSize);
|
||||
void reset();
|
||||
|
||||
NoteCard& card_;
|
||||
NoteText text_;
|
||||
std::string path_, sidePath_;
|
||||
std::vector<Piece> pieces_; // without the window while it's loaded
|
||||
std::vector<Piece> loaded_; // what the window was read from
|
||||
size_t win_ = 0; // the window sits before pieces_[win_]
|
||||
bool windowLoaded_ = false;
|
||||
uint32_t before_ = 0, after_ = 0;
|
||||
uint32_t loadedRevision_ = 0, savedRevision_ = 0;
|
||||
size_t droppedAtLoad_ = 0, newlinesAtLoad_ = 0;
|
||||
bool flushedSinceSave_ = false, sidePending_ = false;
|
||||
uint32_t sideSize_ = 0; // 0: no side file
|
||||
struct {
|
||||
bool valid = false;
|
||||
uint32_t at = 0, len = 0, revision = 0;
|
||||
} windowSaved_; // the window as the side file already has it
|
||||
struct {
|
||||
bool active = false;
|
||||
int stage = 0; // 0: pieces before, 1: the window, 2: pieces after
|
||||
size_t index = 0;
|
||||
uint32_t offset = 0, done = 0, total = 0, wrote = 0, wroteBefore = 0, wroteAfter = 0, revision = 0;
|
||||
bool carry = false; // a CR at the end of a block, waiting to see what follows
|
||||
std::vector<uint8_t> block;
|
||||
} rw_;
|
||||
};
|
||||
|
||||
} // namespace roro::notes
|
||||
@@ -16,11 +16,24 @@ NoteText::NoteText(int cols, int rows, std::string&& text) : cols_(std::max(1, c
|
||||
text_.reserve(kMaxBytes);
|
||||
}
|
||||
|
||||
void NoteText::dropCarriageReturns() {
|
||||
size_t kept = 0;
|
||||
for (size_t i = 0; i < text_.size(); i++)
|
||||
if (!(text_[i] == '\r' && i + 1 < text_.size() && text_[i + 1] == '\n')) text_[kept++] = text_[i];
|
||||
size_t NoteText::dropCarriageReturns(size_t* follow) {
|
||||
size_t kept = 0, size = text_.size(), place = follow ? *follow : 0;
|
||||
for (size_t i = 0; i < size; i++) {
|
||||
if (follow && i == place) *follow = kept;
|
||||
if (!(text_[i] == '\r' && i + 1 < size && text_[i + 1] == '\n')) text_[kept++] = text_[i];
|
||||
}
|
||||
if (follow && place >= size) *follow = kept;
|
||||
text_.resize(kept);
|
||||
return size - kept;
|
||||
}
|
||||
|
||||
size_t NoteText::refilled(size_t cursor, int row) {
|
||||
size_t dropped = dropCarriageReturns(&cursor);
|
||||
cursor_ = std::min(cursor, text_.size());
|
||||
while (cursor_ > 0 && cursor_ < text_.size() && continuation(text_[cursor_])) cursor_--;
|
||||
top_ = lineOf(cursor_);
|
||||
for (int i = 0; i < row && top_ > 0; i++) top_ = lineOf(top_ - 1);
|
||||
return dropped;
|
||||
}
|
||||
|
||||
bool NoteText::setText(const std::string& text) {
|
||||
|
||||
@@ -7,8 +7,9 @@
|
||||
|
||||
namespace roro::notes {
|
||||
|
||||
// The text of a note while it's edited (F1, Q144, Q145): UTF-8 held whole in memory, a cursor, and
|
||||
// the part of it on screen. Lines wrap at spaces, `cols` characters wide; a line owns the space or
|
||||
// The text of a note while it's edited (F1, Q144, Q145): UTF-8 held in memory, a cursor, and the
|
||||
// part of it on screen. Up to 16 KB: a longer note is edited through NoteDocument (note_document.h),
|
||||
// which keeps this as its window on the file. Lines wrap at spaces, `cols` characters wide; a line owns the space or
|
||||
// the newline it ends with, so every byte of the text belongs to exactly one line. No index of
|
||||
// lines is kept (a note of newlines alone would need twice its size): where a line starts is
|
||||
// worked out from the start of its paragraph, which is never far.
|
||||
@@ -27,8 +28,17 @@ class NoteText {
|
||||
bool setText(const std::string& text);
|
||||
const std::string& text() const { return text_; }
|
||||
size_t cursor() const { return cursor_; }
|
||||
size_t size() const { return text_.size(); }
|
||||
uint32_t revision() const { return revision_; } // changes with every edit: is it saved?
|
||||
|
||||
// For a window on a longer text (issue #47): the caller refills the buffer, then says where
|
||||
// the cursor is in it and which row of the screen it should be on. CRLF becomes LF as in
|
||||
// setText; returns how many CRs went. The revision doesn't change: nothing was edited.
|
||||
std::string& buffer() { return text_; }
|
||||
size_t refilled(size_t cursor, int row);
|
||||
size_t startOfLine(size_t pos) const { return lineOf(pos); }
|
||||
size_t top() const { return top_; }
|
||||
|
||||
bool insert(uint32_t codePoint); // false: the note is full
|
||||
bool insertText(const std::string& s); // all of it or nothing
|
||||
void backspace();
|
||||
@@ -61,7 +71,7 @@ class NoteText {
|
||||
bool hasLineAfter(size_t start) const;
|
||||
void moved(bool keepGoal = false);
|
||||
void follow(); // scrolls so the cursor is on screen
|
||||
void dropCarriageReturns();
|
||||
size_t dropCarriageReturns(size_t* follow = nullptr); // how many; `follow` is a place in the text, kept on its character
|
||||
|
||||
int cols_, rows_;
|
||||
std::string text_;
|
||||
|
||||
@@ -13,12 +13,6 @@ constexpr const char* kGiteaRepo = "twisla/roro9stack";
|
||||
|
||||
inline bool versionNewer(const std::string& a, const std::string& b) { return versionOlder(b, a); }
|
||||
|
||||
// "v0.10.0+debug": the Debug Build says so wherever the version shows (scripts/version.py).
|
||||
inline bool isDebugBuild(const std::string& version) {
|
||||
static const std::string tail = "+debug";
|
||||
return version.size() >= tail.size() && version.compare(version.size() - tail.size(), tail.size(), tail) == 0;
|
||||
}
|
||||
|
||||
// Is `release` a newer one than what runs?
|
||||
inline bool isNewer(const Release& release, const std::string& running) {
|
||||
return release.usable() && versionNewer(release.tag, running);
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
#include "settings.h"
|
||||
|
||||
#include "debug_auth.h"
|
||||
#include "ipv4.h"
|
||||
#include "wg_config.h"
|
||||
|
||||
namespace roro {
|
||||
|
||||
@@ -41,6 +43,16 @@ const Definition kDefinitions[] = {
|
||||
{"ntp2", Kind::String, 0, "time.cloudflare.com", 0, 63},
|
||||
{"gnss_quiet", Kind::Bool, 0, nullptr, 0, 1}, // off: GNSS stays on, as decided in M2 (Q58)
|
||||
{"check_updates", Kind::Bool, 1, nullptr, 0, 1}, // on: it only looks, and says so (R1, Q165)
|
||||
{"debug_on", Kind::Bool, 0, nullptr, 0, 1}, // off: nothing listens until the owner says so (Q189)
|
||||
{"debug_token", Kind::String, 0, "", 0, 64}, // empty, or a valid token
|
||||
{"help_told", Kind::Bool, 0, nullptr, 0, 1},
|
||||
{"vpn_config", Kind::String, 0, "", 0, 900}, // empty, or a .conf that parses
|
||||
{"vpn_auto", Kind::Bool, 0, nullptr, 0, 1},
|
||||
{"ssh_hosts", Kind::String, 0, "", 0, 800},
|
||||
{"ssh_known", Kind::String, 0, "", 0, 2400},
|
||||
{"ssh_key", Kind::String, 0, "", 0, 800},
|
||||
{"ssh_public", Kind::String, 0, "", 0, 200},
|
||||
{"ssh_font", Kind::Int, 1, nullptr, 0, 4},
|
||||
};
|
||||
static_assert(sizeof(kDefinitions) / sizeof(kDefinitions[0]) == static_cast<size_t>(Setting::Count),
|
||||
"every Setting needs a definition");
|
||||
@@ -98,6 +110,11 @@ bool Settings::validString(Setting s, const std::string& value) const {
|
||||
if (s == Setting::Dns2) return value.empty() || net::parseIpv4(value, address);
|
||||
if (s == Setting::Ntp1) return net::validHost(value);
|
||||
if (s == Setting::Ntp2) return value.empty() || net::validHost(value);
|
||||
if (s == Setting::DebugToken) return value.empty() || debug::validToken(value);
|
||||
if (s == Setting::VpnConfig) {
|
||||
net::WgConfig config;
|
||||
return value.empty() || net::parseWgConf(value, config).empty();
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
|
||||
@@ -31,6 +31,16 @@ enum class Setting : uint8_t {
|
||||
Ntp2, // string: the second, or empty
|
||||
GnssQuietForLora, // bool: put the GNSS receiver in standby while the LoRa radio listens (issue #20)
|
||||
CheckUpdates, // bool: look for a newer release on Gitea once a day (R1, Q165)
|
||||
DebugConsole, // bool: the Debug Console listens on Wi-Fi (ADR 0010, Q189: off unless switched on)
|
||||
DebugToken, // string: its token, tidied (debug_auth.h); empty until the console is first switched on
|
||||
HelpTold, // bool: this device has been told about the help key once (issue #69, Q201)
|
||||
VpnConfig, // string: the WireGuard tunnel as a .conf (wg_config.h), private key included: never shown (issue #8)
|
||||
VpnAuto, // bool: the tunnel starts whenever Wi-Fi is connected (Q247: off unless switched on)
|
||||
SshHosts, // string: the SSH App's saved hosts, one user@host[:port] a line (issue #2, ssh_hosts.h)
|
||||
SshKnown, // string: the fingerprint each server showed first, one "host:port fingerprint" a line
|
||||
SshKey, // string: this device's own SSH private key, as OpenSSH writes one: never shown
|
||||
SshPublic, // string: its public half, the line to put in a server's authorized_keys
|
||||
SshFont, // int: the terminal's font, 0 (smallest) to 4
|
||||
Count
|
||||
};
|
||||
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
#include "ssh_hosts.h"
|
||||
|
||||
#include <algorithm>
|
||||
|
||||
namespace roro::term {
|
||||
|
||||
namespace {
|
||||
std::vector<std::string> linesOf(const std::string& text) {
|
||||
std::vector<std::string> out;
|
||||
for (size_t at = 0; at < text.size();) {
|
||||
size_t end = text.find('\n', at);
|
||||
if (end == std::string::npos) end = text.size();
|
||||
if (end > at) out.push_back(text.substr(at, end - at));
|
||||
at = end + 1;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
bool hostChars(const std::string& s) {
|
||||
if (s.empty() || s.size() > 253) return false;
|
||||
for (char c : s)
|
||||
if (!((c >= 'a' && c <= 'z') || (c >= 'A' && c <= 'Z') || (c >= '0' && c <= '9') || c == '.' || c == '-')) return false;
|
||||
return s.front() != '.' && s.front() != '-';
|
||||
}
|
||||
} // namespace
|
||||
|
||||
std::string SshTarget::text() const { return user + "@" + host + (port == 22 ? "" : ":" + std::to_string(port)); }
|
||||
std::string SshTarget::hostPort() const { return host + ":" + std::to_string(port); }
|
||||
|
||||
std::string parseSshTarget(const std::string& text, SshTarget& out) {
|
||||
size_t at = text.find('@');
|
||||
if (at == std::string::npos || at == 0) return "user@host, please";
|
||||
SshTarget t;
|
||||
t.user = text.substr(0, at);
|
||||
std::string rest = text.substr(at + 1);
|
||||
if (t.user.size() > 32 || t.user.find_first_of(" @:/") != std::string::npos) return "that isn't a user name";
|
||||
size_t colon = rest.rfind(':');
|
||||
if (colon != std::string::npos) {
|
||||
std::string port = rest.substr(colon + 1);
|
||||
long n = 0;
|
||||
if (port.empty() || port.size() > 5) return "a port from 1 to 65535";
|
||||
for (char c : port) {
|
||||
if (c < '0' || c > '9') return "a port from 1 to 65535";
|
||||
n = n * 10 + (c - '0');
|
||||
}
|
||||
if (n < 1 || n > 65535) return "a port from 1 to 65535";
|
||||
t.port = static_cast<uint16_t>(n);
|
||||
rest.resize(colon);
|
||||
}
|
||||
if (!hostChars(rest)) return "that isn't a host";
|
||||
t.host = rest;
|
||||
out = t;
|
||||
return "";
|
||||
}
|
||||
|
||||
SshHosts::SshHosts(const std::string& stored) {
|
||||
for (auto& line : linesOf(stored)) {
|
||||
SshTarget t;
|
||||
if (hosts_.size() < kMax && parseSshTarget(line, t).empty()) hosts_.push_back(t.text());
|
||||
}
|
||||
}
|
||||
|
||||
void SshHosts::used(const SshTarget& target) {
|
||||
std::string text = target.text();
|
||||
hosts_.erase(std::remove(hosts_.begin(), hosts_.end(), text), hosts_.end());
|
||||
hosts_.insert(hosts_.begin(), text);
|
||||
if (hosts_.size() > kMax) hosts_.resize(kMax);
|
||||
}
|
||||
|
||||
void SshHosts::remove(size_t index) {
|
||||
if (index < hosts_.size()) hosts_.erase(hosts_.begin() + static_cast<long>(index));
|
||||
}
|
||||
|
||||
std::string SshHosts::stored() const {
|
||||
std::string s;
|
||||
for (auto& h : hosts_) s += h + "\n";
|
||||
return s;
|
||||
}
|
||||
|
||||
SshKnownHosts::SshKnownHosts(const std::string& stored) {
|
||||
for (auto& line : linesOf(stored)) {
|
||||
size_t space = line.find(' ');
|
||||
if (space != std::string::npos && space > 0 && space + 1 < line.size() && known_.size() < kMax) known_.emplace_back(line.substr(0, space), line.substr(space + 1));
|
||||
}
|
||||
}
|
||||
|
||||
std::string SshKnownHosts::fingerprintOf(const std::string& hostPort) const {
|
||||
for (auto& k : known_)
|
||||
if (k.first == hostPort) return k.second;
|
||||
return "";
|
||||
}
|
||||
|
||||
void SshKnownHosts::remember(const std::string& hostPort, const std::string& fingerprint) {
|
||||
known_.erase(std::remove_if(known_.begin(), known_.end(), [&](const std::pair<std::string, std::string>& k) { return k.first == hostPort; }), known_.end());
|
||||
known_.emplace_back(hostPort, fingerprint);
|
||||
if (known_.size() > kMax) known_.erase(known_.begin());
|
||||
}
|
||||
|
||||
void SshKnownHosts::forget(const std::string& hostPort) {
|
||||
known_.erase(std::remove_if(known_.begin(), known_.end(), [&](const std::pair<std::string, std::string>& k) { return k.first == hostPort; }), known_.end());
|
||||
}
|
||||
|
||||
std::string SshKnownHosts::stored() const {
|
||||
std::string s;
|
||||
for (auto& k : known_) s += k.first + " " + k.second + "\n";
|
||||
return s;
|
||||
}
|
||||
|
||||
} // namespace roro::term
|
||||
@@ -0,0 +1,49 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstdint>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
// Who the SSH App connects to, and which servers it has met (issue #2, N1 Q262 and Q263): kept
|
||||
// in the device's settings as a few lines of text.
|
||||
namespace roro::term {
|
||||
|
||||
struct SshTarget {
|
||||
std::string user, host;
|
||||
uint16_t port = 22;
|
||||
std::string text() const; // user@host, with :port when it isn't 22
|
||||
std::string hostPort() const; // host:port, always: what a host key is remembered under
|
||||
};
|
||||
// "user@host", "user@host:2222". "" or what is wrong with it.
|
||||
std::string parseSshTarget(const std::string& text, SshTarget& out);
|
||||
|
||||
// Up to eight, the last used first; one a line.
|
||||
class SshHosts {
|
||||
public:
|
||||
static constexpr size_t kMax = 8;
|
||||
explicit SshHosts(const std::string& stored = "");
|
||||
const std::vector<std::string>& list() const { return hosts_; }
|
||||
void used(const SshTarget& target); // to the front, added if it's new
|
||||
void remove(size_t index);
|
||||
std::string stored() const;
|
||||
|
||||
private:
|
||||
std::vector<std::string> hosts_;
|
||||
};
|
||||
|
||||
// The fingerprint each server showed the first time: "host:port SHA256:...", one a line, sixteen
|
||||
// at most (the oldest goes).
|
||||
class SshKnownHosts {
|
||||
public:
|
||||
static constexpr size_t kMax = 16;
|
||||
explicit SshKnownHosts(const std::string& stored = "");
|
||||
std::string fingerprintOf(const std::string& hostPort) const; // "" if never met
|
||||
void remember(const std::string& hostPort, const std::string& fingerprint);
|
||||
void forget(const std::string& hostPort);
|
||||
std::string stored() const;
|
||||
|
||||
private:
|
||||
std::vector<std::pair<std::string, std::string>> known_;
|
||||
};
|
||||
|
||||
} // namespace roro::term
|
||||
@@ -0,0 +1,495 @@
|
||||
#include "terminal.h"
|
||||
|
||||
#include <algorithm>
|
||||
|
||||
namespace roro::term {
|
||||
|
||||
namespace {
|
||||
// A character the screen's font has, for one it may not.
|
||||
uint8_t glyphFor(uint32_t cp) {
|
||||
if (cp >= 0x20 && cp <= 0x7E) return static_cast<uint8_t>(cp);
|
||||
if (cp >= 0xA0 && cp <= 0xFF) return static_cast<uint8_t>(cp);
|
||||
if (cp >= 0x2500 && cp <= 0x257F) { // box drawing
|
||||
switch (cp) {
|
||||
case 0x2500: case 0x2501: case 0x2504: case 0x2505: case 0x2508: case 0x2509: case 0x254C: case 0x254D: case 0x2550: return '-';
|
||||
case 0x2502: case 0x2503: case 0x2506: case 0x2507: case 0x250A: case 0x250B: case 0x254E: case 0x254F: case 0x2551: return '|';
|
||||
default: return '+';
|
||||
}
|
||||
}
|
||||
switch (cp) {
|
||||
case 0x2018: case 0x2019: return '\'';
|
||||
case 0x201C: case 0x201D: return '"';
|
||||
case 0x2010: case 0x2011: case 0x2012: case 0x2013: case 0x2014: return '-';
|
||||
case 0x2022: case 0x25CF: return '*';
|
||||
case 0x2026: return '.';
|
||||
case 0x2190: return '<';
|
||||
case 0x2192: return '>';
|
||||
case 0x2191: return '^';
|
||||
case 0x2193: return 'v';
|
||||
case 0x2588: case 0x2593: case 0x2592: case 0x2591: return '#';
|
||||
default: return '?';
|
||||
}
|
||||
}
|
||||
// The DEC "special graphics" set: lines drawn with the letters j to x.
|
||||
uint8_t lineGlyph(uint8_t c) {
|
||||
switch (c) {
|
||||
case 'q': return '-';
|
||||
case 'x': return '|';
|
||||
case 'j': case 'k': case 'l': case 'm': case 'n': case 't': case 'u': case 'v': case 'w': return '+';
|
||||
case '`': return '*';
|
||||
case 'a': return '#';
|
||||
case '~': return '*';
|
||||
default: return c;
|
||||
}
|
||||
}
|
||||
// One of 256 colours, or a colour given as red, green and blue, as the nearest of the sixteen.
|
||||
int nearest16(int r, int g, int b) {
|
||||
int most = std::max(r, std::max(g, b));
|
||||
if (most < 48) return 0;
|
||||
int half = most / 2;
|
||||
int colour = (r > half ? 1 : 0) | (g > half ? 2 : 0) | (b > half ? 4 : 0);
|
||||
if (colour == 7 && most < 200) return most < 110 ? 8 : 7;
|
||||
return most > 170 ? colour | 8 : colour;
|
||||
}
|
||||
int from256(int n) {
|
||||
if (n < 16) return n;
|
||||
if (n >= 232) return nearest16(8 + (n - 232) * 10, 8 + (n - 232) * 10, 8 + (n - 232) * 10);
|
||||
n -= 16;
|
||||
static const int kSteps[6] = {0, 95, 135, 175, 215, 255};
|
||||
return nearest16(kSteps[n / 36], kSteps[(n / 6) % 6], kSteps[n % 6]);
|
||||
}
|
||||
} // namespace
|
||||
|
||||
void colourRgb(int index, uint8_t& r, uint8_t& g, uint8_t& b) {
|
||||
static const uint8_t kTable[16][3] = {{0, 0, 0}, {205, 49, 49}, {13, 188, 121}, {229, 229, 16}, {36, 114, 200}, {188, 63, 188},
|
||||
{17, 168, 205}, {204, 204, 204}, {102, 102, 102}, {241, 76, 76}, {35, 209, 139}, {245, 245, 67},
|
||||
{59, 142, 234}, {214, 112, 214}, {41, 184, 219}, {255, 255, 255}};
|
||||
r = kTable[index & 15][0];
|
||||
g = kTable[index & 15][1];
|
||||
b = kTable[index & 15][2];
|
||||
}
|
||||
|
||||
Terminal::Terminal(int cols, int rows, int historyLines)
|
||||
: cols_(std::max(2, cols)), rows_(std::max(2, rows)), historyMax_(std::max(0, historyLines)), mainGrid_(static_cast<size_t>(cols_ * rows_)),
|
||||
altGrid_(static_cast<size_t>(cols_ * rows_)), bottom_(rows_ - 1) {}
|
||||
|
||||
Cell Terminal::blank() const {
|
||||
Cell c;
|
||||
c.attr = static_cast<uint8_t>((bg_ << 4) | 7);
|
||||
return c;
|
||||
}
|
||||
|
||||
bool Terminal::takeBell() {
|
||||
bool b = bell_;
|
||||
bell_ = false;
|
||||
return b;
|
||||
}
|
||||
|
||||
std::string Terminal::rowText(int row) const {
|
||||
std::string s;
|
||||
for (int c = 0; c < cols_; c++) s += static_cast<char>(cell(row, c).ch);
|
||||
size_t end = s.find_last_not_of(' ');
|
||||
s.resize(end == std::string::npos ? 0 : end + 1);
|
||||
return s;
|
||||
}
|
||||
|
||||
int Terminal::param(size_t i, int fallback) const { return i < params_.size() && params_[i] > 0 ? params_[i] : fallback; }
|
||||
|
||||
void Terminal::moveTo(int row, int col) {
|
||||
row_ = std::clamp(row, 0, rows_ - 1);
|
||||
col_ = std::clamp(col, 0, cols_ - 1);
|
||||
wrapPending_ = false;
|
||||
}
|
||||
|
||||
void Terminal::eraseCells(int row, int from, int to) {
|
||||
Cell b = blank();
|
||||
for (int c = std::max(0, from); c <= std::min(cols_ - 1, to); c++) grid()[static_cast<size_t>(row * cols_ + c)] = b;
|
||||
}
|
||||
|
||||
void Terminal::scrollUp(int top, int bottom, int n, bool toHistory) {
|
||||
n = std::min(n, bottom - top + 1);
|
||||
auto& g = grid();
|
||||
for (int i = 0; i < n; i++) {
|
||||
if (toHistory && !alt_ && top == 0 && historyMax_ > 0) { // off the top of the real screen: kept, as text
|
||||
history_.push_back(rowText(0));
|
||||
if (static_cast<int>(history_.size()) > historyMax_) history_.pop_front();
|
||||
}
|
||||
std::move(g.begin() + (top + 1) * cols_, g.begin() + (bottom + 1) * cols_, g.begin() + top * cols_);
|
||||
eraseCells(bottom, 0, cols_ - 1);
|
||||
}
|
||||
}
|
||||
|
||||
void Terminal::scrollDown(int top, int bottom, int n) {
|
||||
n = std::min(n, bottom - top + 1);
|
||||
auto& g = grid();
|
||||
for (int i = 0; i < n; i++) {
|
||||
std::move_backward(g.begin() + top * cols_, g.begin() + bottom * cols_, g.begin() + (bottom + 1) * cols_);
|
||||
eraseCells(top, 0, cols_ - 1);
|
||||
}
|
||||
}
|
||||
|
||||
void Terminal::lineFeed() {
|
||||
wrapPending_ = false;
|
||||
if (row_ == bottom_) scrollUp(top_, bottom_, 1);
|
||||
else if (row_ < rows_ - 1) row_++;
|
||||
}
|
||||
|
||||
void Terminal::reverseIndex() {
|
||||
wrapPending_ = false;
|
||||
if (row_ == top_) scrollDown(top_, bottom_, 1);
|
||||
else if (row_ > 0) row_--;
|
||||
}
|
||||
|
||||
void Terminal::put(uint32_t cp) {
|
||||
uint8_t ch = lineDrawing_ && cp < 0x80 ? lineGlyph(static_cast<uint8_t>(cp)) : glyphFor(cp);
|
||||
if (wrapPending_) {
|
||||
col_ = 0;
|
||||
lineFeed();
|
||||
}
|
||||
uint8_t fg = static_cast<uint8_t>(bold_ && fg_ < 8 ? fg_ | 8 : fg_), bg = bg_;
|
||||
if (inverse_) std::swap(fg, bg);
|
||||
Cell& c = grid()[static_cast<size_t>(row_ * cols_ + col_)];
|
||||
c.ch = ch;
|
||||
c.attr = static_cast<uint8_t>((bg << 4) | (fg & 15));
|
||||
if (col_ == cols_ - 1) wrapPending_ = autoWrap_;
|
||||
else col_++;
|
||||
}
|
||||
|
||||
void Terminal::control(uint8_t c) {
|
||||
switch (c) {
|
||||
case 0x07: bell_ = true; break;
|
||||
case 0x08:
|
||||
if (col_ > 0) col_--;
|
||||
wrapPending_ = false;
|
||||
break;
|
||||
case 0x09: moveTo(row_, std::min(cols_ - 1, (col_ / 8 + 1) * 8)); break;
|
||||
case 0x0A: case 0x0B: case 0x0C: lineFeed(); break;
|
||||
case 0x0D:
|
||||
col_ = 0;
|
||||
wrapPending_ = false;
|
||||
break;
|
||||
default: break;
|
||||
}
|
||||
}
|
||||
|
||||
void Terminal::reset() {
|
||||
alt_ = false;
|
||||
Cell b;
|
||||
std::fill(mainGrid_.begin(), mainGrid_.end(), b);
|
||||
std::fill(altGrid_.begin(), altGrid_.end(), b);
|
||||
row_ = col_ = top_ = 0;
|
||||
bottom_ = rows_ - 1;
|
||||
fg_ = 7;
|
||||
bg_ = 0;
|
||||
bold_ = inverse_ = wrapPending_ = appCursor_ = lineDrawing_ = false;
|
||||
autoWrap_ = cursorShown_ = true;
|
||||
}
|
||||
|
||||
void Terminal::escape(uint8_t c) {
|
||||
state_ = State::Ground;
|
||||
switch (c) {
|
||||
case '[':
|
||||
state_ = State::Csi;
|
||||
params_.clear();
|
||||
paramStarted_ = private_ = false;
|
||||
break;
|
||||
case ']': case 'P': case '^': case '_': state_ = State::Osc; break; // a title, or something this doesn't read: skipped to its end
|
||||
case '(': case ')': case '*': case '+': state_ = State::Charset; break;
|
||||
case '7':
|
||||
savedRow_ = row_;
|
||||
savedCol_ = col_;
|
||||
savedAttr_ = static_cast<uint8_t>((bg_ << 4) | fg_);
|
||||
break;
|
||||
case '8':
|
||||
moveTo(savedRow_, savedCol_);
|
||||
fg_ = savedAttr_ & 15;
|
||||
bg_ = savedAttr_ >> 4;
|
||||
break;
|
||||
case 'D': lineFeed(); break;
|
||||
case 'E':
|
||||
col_ = 0;
|
||||
lineFeed();
|
||||
break;
|
||||
case 'M': reverseIndex(); break;
|
||||
case 'c': reset(); break;
|
||||
default: break; // = and >, the keypad's modes, among others
|
||||
}
|
||||
}
|
||||
|
||||
void Terminal::sgr() {
|
||||
if (params_.empty()) params_.push_back(0);
|
||||
for (size_t i = 0; i < params_.size(); i++) {
|
||||
int p = params_[i];
|
||||
if (p == 0) {
|
||||
fg_ = 7;
|
||||
bg_ = 0;
|
||||
bold_ = inverse_ = false;
|
||||
} else if (p == 1) bold_ = true;
|
||||
else if (p == 7) inverse_ = true;
|
||||
else if (p == 22) bold_ = false;
|
||||
else if (p == 27) inverse_ = false;
|
||||
else if (p >= 30 && p <= 37) fg_ = static_cast<uint8_t>(p - 30);
|
||||
else if (p == 39) fg_ = 7;
|
||||
else if (p >= 40 && p <= 47) bg_ = static_cast<uint8_t>(p - 40);
|
||||
else if (p == 49) bg_ = 0;
|
||||
else if (p >= 90 && p <= 97) fg_ = static_cast<uint8_t>(p - 90 + 8);
|
||||
else if (p >= 100 && p <= 107) bg_ = static_cast<uint8_t>(p - 100 + 8);
|
||||
else if ((p == 38 || p == 48) && i + 1 < params_.size()) {
|
||||
int colour = -1;
|
||||
if (params_[i + 1] == 5 && i + 2 < params_.size()) {
|
||||
colour = from256(params_[i + 2] & 255);
|
||||
i += 2;
|
||||
} else if (params_[i + 1] == 2 && i + 4 < params_.size()) {
|
||||
colour = nearest16(params_[i + 2] & 255, params_[i + 3] & 255, params_[i + 4] & 255);
|
||||
i += 4;
|
||||
} else {
|
||||
break;
|
||||
}
|
||||
(p == 38 ? fg_ : bg_) = static_cast<uint8_t>(colour);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void Terminal::mode(bool on) {
|
||||
for (int p : params_) {
|
||||
if (!private_) continue;
|
||||
switch (p) {
|
||||
case 1: appCursor_ = on; break;
|
||||
case 7: autoWrap_ = on; break;
|
||||
case 25: cursorShown_ = on; break;
|
||||
case 47: case 1047: case 1049:
|
||||
if (on == alt_) break;
|
||||
if (on && p == 1049) {
|
||||
savedRow_ = row_;
|
||||
savedCol_ = col_;
|
||||
}
|
||||
alt_ = on;
|
||||
if (on) std::fill(altGrid_.begin(), altGrid_.end(), Cell());
|
||||
top_ = 0;
|
||||
bottom_ = rows_ - 1;
|
||||
if (!on && p == 1049) moveTo(savedRow_, savedCol_);
|
||||
else if (on) moveTo(0, 0);
|
||||
break;
|
||||
default: break; // the mouse, bracketed paste and the rest
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void Terminal::csi(uint8_t final) {
|
||||
int n = param(0, 1);
|
||||
switch (final) {
|
||||
case 'A': moveTo(std::max(row_ - n, row_ >= top_ ? top_ : 0), col_); break;
|
||||
case 'B': case 'e': moveTo(std::min(row_ + n, row_ <= bottom_ ? bottom_ : rows_ - 1), col_); break;
|
||||
case 'C': case 'a': moveTo(row_, col_ + n); break;
|
||||
case 'D': moveTo(row_, col_ - n); break;
|
||||
case 'E': moveTo(row_ + n, 0); break;
|
||||
case 'F': moveTo(row_ - n, 0); break;
|
||||
case 'G': case '`': moveTo(row_, n - 1); break;
|
||||
case 'd': moveTo(n - 1, col_); break;
|
||||
case 'H': case 'f': moveTo(param(0, 1) - 1, param(1, 1) - 1); break;
|
||||
case 'J': {
|
||||
int what = params_.empty() ? 0 : params_[0];
|
||||
if (what == 0) {
|
||||
eraseCells(row_, col_, cols_ - 1);
|
||||
for (int r = row_ + 1; r < rows_; r++) eraseCells(r, 0, cols_ - 1);
|
||||
} else if (what == 1) {
|
||||
for (int r = 0; r < row_; r++) eraseCells(r, 0, cols_ - 1);
|
||||
eraseCells(row_, 0, col_);
|
||||
} else {
|
||||
for (int r = 0; r < rows_; r++) eraseCells(r, 0, cols_ - 1);
|
||||
}
|
||||
break;
|
||||
}
|
||||
case 'K': {
|
||||
int what = params_.empty() ? 0 : params_[0];
|
||||
eraseCells(row_, what == 0 ? col_ : 0, what == 1 ? col_ : cols_ - 1);
|
||||
break;
|
||||
}
|
||||
case 'L':
|
||||
if (row_ >= top_ && row_ <= bottom_) scrollDown(row_, bottom_, n);
|
||||
break;
|
||||
case 'M':
|
||||
if (row_ >= top_ && row_ <= bottom_) scrollUp(row_, bottom_, n, false); // deleted, not scrolled away: not history
|
||||
break;
|
||||
case 'P': {
|
||||
n = std::min(n, cols_ - col_);
|
||||
auto row = grid().begin() + row_ * cols_;
|
||||
std::move(row + col_ + n, row + cols_, row + col_);
|
||||
eraseCells(row_, cols_ - n, cols_ - 1);
|
||||
break;
|
||||
}
|
||||
case '@': {
|
||||
n = std::min(n, cols_ - col_);
|
||||
auto row = grid().begin() + row_ * cols_;
|
||||
std::move_backward(row + col_, row + cols_ - n, row + cols_);
|
||||
eraseCells(row_, col_, col_ + n - 1);
|
||||
break;
|
||||
}
|
||||
case 'X': eraseCells(row_, col_, col_ + n - 1); break;
|
||||
case 'S': scrollUp(top_, bottom_, n); break;
|
||||
case 'T': scrollDown(top_, bottom_, n); break;
|
||||
case 'm': sgr(); break;
|
||||
case 'r': {
|
||||
int top = param(0, 1) - 1, bottom = param(1, rows_) - 1;
|
||||
if (top < bottom && bottom < rows_) {
|
||||
top_ = top;
|
||||
bottom_ = bottom;
|
||||
} else {
|
||||
top_ = 0;
|
||||
bottom_ = rows_ - 1;
|
||||
}
|
||||
moveTo(0, 0);
|
||||
break;
|
||||
}
|
||||
case 's':
|
||||
savedRow_ = row_;
|
||||
savedCol_ = col_;
|
||||
break;
|
||||
case 'u': moveTo(savedRow_, savedCol_); break;
|
||||
case 'h': mode(true); break;
|
||||
case 'l': mode(false); break;
|
||||
case 'n':
|
||||
if (!reply) break;
|
||||
if (param(0, 0) == 6) reply("\x1b[" + std::to_string(row_ + 1) + ";" + std::to_string(col_ + 1) + "R");
|
||||
else if (param(0, 0) == 5) reply("\x1b[0n");
|
||||
break;
|
||||
case 'c':
|
||||
if (reply && !private_) reply("\x1b[?6c"); // "a VT102"
|
||||
break;
|
||||
default: break;
|
||||
}
|
||||
}
|
||||
|
||||
void Terminal::feed(const uint8_t* data, size_t len) {
|
||||
for (size_t i = 0; i < len; i++) {
|
||||
uint8_t c = data[i];
|
||||
if (state_ == State::Osc || state_ == State::OscEsc) { // skipped: to a bell, or to ESC backslash
|
||||
if (c == 0x07 || (state_ == State::OscEsc && c == '\\')) state_ = State::Ground;
|
||||
else state_ = c == 0x1B ? State::OscEsc : State::Osc;
|
||||
continue;
|
||||
}
|
||||
if (c == 0x1B) {
|
||||
state_ = State::Esc;
|
||||
utf8Left_ = 0;
|
||||
continue;
|
||||
}
|
||||
if (c < 0x20) { // these act wherever they come, in the middle of a sequence too
|
||||
if (c == 0x0E) lineDrawing_ = true;
|
||||
else if (c == 0x0F) lineDrawing_ = false;
|
||||
else control(c);
|
||||
continue;
|
||||
}
|
||||
switch (state_) {
|
||||
case State::Esc: escape(c); break;
|
||||
case State::Charset:
|
||||
lineDrawing_ = c == '0';
|
||||
state_ = State::Ground;
|
||||
break;
|
||||
case State::Csi:
|
||||
if (c >= '0' && c <= '9') {
|
||||
if (!paramStarted_) {
|
||||
if (params_.size() < 16) params_.push_back(0);
|
||||
paramStarted_ = true;
|
||||
}
|
||||
if (!params_.empty() && params_.back() < 10000) params_.back() = params_.back() * 10 + (c - '0');
|
||||
} else if (c == ';' || c == ':') {
|
||||
if (!paramStarted_ && params_.size() < 16) params_.push_back(0);
|
||||
paramStarted_ = false;
|
||||
} else if (c == '?' || c == '>' || c == '=' || c == '<') {
|
||||
private_ = true;
|
||||
} else if (c >= 0x40 && c <= 0x7E) {
|
||||
state_ = State::Ground;
|
||||
csi(c);
|
||||
} // anything else is an in-between byte: passed over
|
||||
break;
|
||||
default:
|
||||
if (c == 0x7F) break;
|
||||
if (c < 0x80) {
|
||||
utf8Left_ = 0;
|
||||
put(c);
|
||||
} else if (c >= 0xC0) { // the first byte of a longer character
|
||||
utf8Left_ = c >= 0xF0 ? 3 : c >= 0xE0 ? 2 : 1;
|
||||
utf8_ = c & (c >= 0xF0 ? 0x07 : c >= 0xE0 ? 0x0F : 0x1F);
|
||||
} else if (utf8Left_ > 0) {
|
||||
utf8_ = (utf8_ << 6) | (c & 0x3F);
|
||||
if (--utf8Left_ == 0) put(utf8_);
|
||||
} else {
|
||||
put('?'); // a byte that belongs to nothing
|
||||
}
|
||||
break;
|
||||
}
|
||||
}
|
||||
revision_++;
|
||||
}
|
||||
|
||||
void Terminal::resize(int cols, int rows) {
|
||||
cols = std::max(2, cols);
|
||||
rows = std::max(2, rows);
|
||||
if (cols == cols_ && rows == rows_) return;
|
||||
// The cursor's line stays on screen: what is above it goes to the history if it has to.
|
||||
int shift = alt_ ? 0 : std::max(0, row_ - (rows - 1));
|
||||
if (shift) {
|
||||
int oldTop = top_, oldBottom = bottom_;
|
||||
top_ = 0;
|
||||
bottom_ = rows_ - 1;
|
||||
scrollUp(0, rows_ - 1, shift);
|
||||
top_ = oldTop;
|
||||
bottom_ = oldBottom;
|
||||
}
|
||||
auto copy = [&](std::vector<Cell>& g) {
|
||||
std::vector<Cell> fresh(static_cast<size_t>(cols * rows));
|
||||
for (int r = 0; r < std::min(rows, rows_); r++)
|
||||
for (int c = 0; c < std::min(cols, cols_); c++) fresh[static_cast<size_t>(r * cols + c)] = g[static_cast<size_t>(r * cols_ + c)];
|
||||
g.swap(fresh);
|
||||
};
|
||||
copy(mainGrid_);
|
||||
copy(altGrid_);
|
||||
cols_ = cols;
|
||||
rows_ = rows;
|
||||
top_ = 0;
|
||||
bottom_ = rows_ - 1;
|
||||
moveTo(row_ - shift, col_);
|
||||
savedRow_ = std::min(savedRow_, rows_ - 1);
|
||||
savedCol_ = std::min(savedCol_, cols_ - 1);
|
||||
revision_++;
|
||||
}
|
||||
|
||||
std::string encodeKey(TermKey key, uint32_t ch, bool ctrl, bool alt, bool appCursorKeys) {
|
||||
std::string out;
|
||||
auto arrow = [&](char letter) { return std::string(appCursorKeys ? "\x1bO" : "\x1b[") + letter; };
|
||||
switch (key) {
|
||||
case TermKey::Up: out = arrow('A'); break;
|
||||
case TermKey::Down: out = arrow('B'); break;
|
||||
case TermKey::Right: out = arrow('C'); break;
|
||||
case TermKey::Left: out = arrow('D'); break;
|
||||
case TermKey::Enter: out = "\r"; break;
|
||||
case TermKey::Backspace: out = "\x7f"; break;
|
||||
case TermKey::Tab: out = "\t"; break;
|
||||
case TermKey::Escape: out = "\x1b"; break;
|
||||
case TermKey::PageUp: out = "\x1b[5~"; break;
|
||||
case TermKey::PageDown: out = "\x1b[6~"; break;
|
||||
case TermKey::Char:
|
||||
if (ctrl) {
|
||||
uint32_t c = ch >= 'a' && ch <= 'z' ? ch - 32 : ch;
|
||||
if (c >= '@' && c <= '_') out = std::string(1, static_cast<char>(c - '@'));
|
||||
else if (c == ' ' || c == '2') out = std::string(1, '\0');
|
||||
else if (c == '?' || c == '8') out = "\x7f";
|
||||
else if (c == '3') out = "\x1b";
|
||||
else if (c == '4') out = "\x1c";
|
||||
else if (c == '5') out = "\x1d";
|
||||
else if (c == '6') out = "\x1e";
|
||||
else if (c == '7' || c == '/') out = "\x1f";
|
||||
break;
|
||||
}
|
||||
if (ch < 0x80) out = std::string(1, static_cast<char>(ch));
|
||||
else if (ch < 0x800) out = {static_cast<char>(0xC0 | (ch >> 6)), static_cast<char>(0x80 | (ch & 0x3F))};
|
||||
else if (ch < 0x10000) out = {static_cast<char>(0xE0 | (ch >> 12)), static_cast<char>(0x80 | ((ch >> 6) & 0x3F)), static_cast<char>(0x80 | (ch & 0x3F))};
|
||||
else out = {static_cast<char>(0xF0 | (ch >> 18)), static_cast<char>(0x80 | ((ch >> 12) & 0x3F)), static_cast<char>(0x80 | ((ch >> 6) & 0x3F)),
|
||||
static_cast<char>(0x80 | (ch & 0x3F))};
|
||||
break;
|
||||
}
|
||||
if (alt && !out.empty() && key != TermKey::Escape) out.insert(0, 1, '\x1b');
|
||||
return out;
|
||||
}
|
||||
|
||||
} // namespace roro::term
|
||||
@@ -0,0 +1,96 @@
|
||||
#pragma once
|
||||
|
||||
#include <cstddef>
|
||||
#include <cstdint>
|
||||
#include <deque>
|
||||
#include <functional>
|
||||
#include <string>
|
||||
#include <vector>
|
||||
|
||||
// A terminal's screen (issue #2, N1 Q256): what a remote program's output makes of a grid of
|
||||
// characters. It understands what a shell, `less`, `top`, `nano` and plain `vim` send: the cursor,
|
||||
// erasing, sixteen colours, scroll regions, the alternate screen. No mouse. Characters are kept
|
||||
// as the screen's font has them (Latin-1); what it lacks becomes `?`, and box-drawing lines
|
||||
// become + - |.
|
||||
namespace roro::term {
|
||||
|
||||
struct Cell {
|
||||
uint8_t ch = ' ';
|
||||
uint8_t attr = 0x07; // low four bits: the colour of the character, high four: of what is behind it
|
||||
};
|
||||
|
||||
class Terminal {
|
||||
public:
|
||||
Terminal(int cols, int rows, int historyLines);
|
||||
|
||||
void feed(const uint8_t* data, size_t len);
|
||||
// A new size: what is on screen stays where it is, from the top left; if the cursor would fall
|
||||
// off the bottom, the top lines go to the history.
|
||||
void resize(int cols, int rows);
|
||||
// What the program asked to be told (where the cursor is, what kind of terminal this is).
|
||||
std::function<void(const std::string&)> reply;
|
||||
|
||||
int cols() const { return cols_; }
|
||||
int rows() const { return rows_; }
|
||||
const Cell& cell(int row, int col) const { return grid()[static_cast<size_t>(row * cols_ + col)]; }
|
||||
int cursorRow() const { return row_; }
|
||||
int cursorCol() const { return col_; }
|
||||
bool cursorVisible() const { return cursorShown_; }
|
||||
bool appCursorKeys() const { return appCursor_; } // the arrows are sent another way (vim, less)
|
||||
bool altScreen() const { return alt_; }
|
||||
uint32_t revision() const { return revision_; } // changes whenever the screen may have
|
||||
bool takeBell();
|
||||
|
||||
// Lines that scrolled off the top of the main screen, oldest first; their text only.
|
||||
int historyCount() const { return static_cast<int>(history_.size()); }
|
||||
const std::string& historyLine(int i) const { return history_[static_cast<size_t>(i)]; }
|
||||
|
||||
std::string rowText(int row) const; // without trailing spaces
|
||||
|
||||
private:
|
||||
enum class State { Ground, Esc, Csi, Osc, OscEsc, Charset };
|
||||
|
||||
std::vector<Cell>& grid() { return alt_ ? altGrid_ : mainGrid_; }
|
||||
const std::vector<Cell>& grid() const { return alt_ ? altGrid_ : mainGrid_; }
|
||||
Cell blank() const;
|
||||
void put(uint32_t codePoint);
|
||||
void control(uint8_t c);
|
||||
void escape(uint8_t c);
|
||||
void csi(uint8_t final);
|
||||
void sgr();
|
||||
void mode(bool on);
|
||||
void lineFeed();
|
||||
void reverseIndex();
|
||||
void scrollUp(int top, int bottom, int n, bool toHistory = true);
|
||||
void scrollDown(int top, int bottom, int n);
|
||||
void eraseCells(int row, int from, int to);
|
||||
void moveTo(int row, int col);
|
||||
void reset();
|
||||
int param(size_t i, int fallback) const;
|
||||
|
||||
int cols_, rows_, historyMax_;
|
||||
std::vector<Cell> mainGrid_, altGrid_;
|
||||
std::deque<std::string> history_;
|
||||
bool alt_ = false;
|
||||
int row_ = 0, col_ = 0, top_ = 0, bottom_ = 0;
|
||||
int savedRow_ = 0, savedCol_ = 0;
|
||||
uint8_t savedAttr_ = 0x07;
|
||||
bool wrapPending_ = false, autoWrap_ = true, cursorShown_ = true, appCursor_ = false, lineDrawing_ = false, bell_ = false;
|
||||
uint8_t fg_ = 7, bg_ = 0;
|
||||
bool bold_ = false, inverse_ = false;
|
||||
State state_ = State::Ground;
|
||||
std::vector<int> params_;
|
||||
bool paramStarted_ = false, private_ = false;
|
||||
uint32_t utf8_ = 0;
|
||||
int utf8Left_ = 0;
|
||||
uint32_t revision_ = 0;
|
||||
};
|
||||
|
||||
// What a key sends to the remote program.
|
||||
enum class TermKey { Char, Up, Down, Left, Right, Enter, Backspace, Tab, Escape, PageUp, PageDown };
|
||||
std::string encodeKey(TermKey key, uint32_t ch, bool ctrl, bool alt, bool appCursorKeys);
|
||||
|
||||
// One of the terminal's sixteen colours as red, green and blue.
|
||||
void colourRgb(int index, uint8_t& r, uint8_t& g, uint8_t& b);
|
||||
|
||||
} // namespace roro::term
|
||||
@@ -0,0 +1,15 @@
|
||||
#include "version.h"
|
||||
|
||||
// Written by scripts/version.py before each build; not in git.
|
||||
#if __has_include("version_generated.h")
|
||||
#include "version_generated.h"
|
||||
#endif
|
||||
#ifndef RORO_VERSION
|
||||
#define RORO_VERSION "unknown"
|
||||
#endif
|
||||
|
||||
namespace roro {
|
||||
|
||||
const char* versionString() { return RORO_VERSION; }
|
||||
|
||||
} // namespace roro
|
||||
@@ -1,14 +1,12 @@
|
||||
#pragma once
|
||||
|
||||
#ifndef RORO_VERSION
|
||||
#define RORO_VERSION "unknown"
|
||||
#endif
|
||||
|
||||
namespace roro {
|
||||
|
||||
constexpr const char* kProductName = "roro9stack";
|
||||
|
||||
// "roro9stack v0.1.0" — used on the boot screen and in About.
|
||||
inline const char* versionString() { return RORO_VERSION; }
|
||||
// "v0.1.0", from `git describe` (scripts/version.py): used on the boot screen and in About.
|
||||
// A function in one file, not a macro on every compiler command line: a new commit then recompiles
|
||||
// that one file, and everything else comes from the build cache (issue #74).
|
||||
const char* versionString();
|
||||
|
||||
} // namespace roro
|
||||
|
||||
@@ -9,6 +9,9 @@ extra_scripts = pre:scripts/version.py
|
||||
test_framework = unity
|
||||
|
||||
[env:cardputer-adv]
|
||||
extra_scripts =
|
||||
${env.extra_scripts}
|
||||
pre:scripts/libssh_filter.py
|
||||
platform = https://github.com/pioarduino/platform-espressif32/releases/download/55.03.312/platform-espressif32.zip
|
||||
board = m5stack-stamps3
|
||||
framework = arduino
|
||||
@@ -17,9 +20,12 @@ monitor_speed = 115200
|
||||
build_flags =
|
||||
-DARDUINO_USB_CDC_ON_BOOT=1
|
||||
-DARDUINO_USB_MODE=1
|
||||
-DCONFIG_WIREGUARD_MAX_SRC_IPS=4
|
||||
lib_deps =
|
||||
m5stack/M5Cardputer @ 1.1.1
|
||||
jgromes/RadioLib @ 7.8.1
|
||||
esphome/wireguard @ 0.4.8
|
||||
ewpa/LibSSH-ESP32 @ 5.10.0
|
||||
test_ignore = *
|
||||
; Smaller TLS buffers (M2): the framework is rebuilt with these settings (pioarduino "hybrid
|
||||
; compile"). Receive stays 16 KB (servers send full TLS records); send drops to 4 KB (IRC lines are
|
||||
@@ -32,15 +38,6 @@ custom_sdkconfig =
|
||||
CONFIG_MBEDTLS_DYNAMIC_FREE_CONFIG_DATA=y
|
||||
CONFIG_MBEDTLS_DYNAMIC_FREE_CA_CERT=y
|
||||
|
||||
; Debug Build: the same firmware plus the Debug Console on TCP 2323 (see ADR 0004). The token comes
|
||||
; from ~/.config/roro9stack/debug-token, passed in by scripts/_docker.sh; it's never committed.
|
||||
[env:cardputer-adv-debug]
|
||||
extends = env:cardputer-adv
|
||||
extra_scripts = pre:scripts/version.py, pre:scripts/debug_flags.py
|
||||
build_flags =
|
||||
${env:cardputer-adv.build_flags}
|
||||
-DRORO_DEBUG
|
||||
|
||||
; Host-side unit tests for pure logic (no hardware).
|
||||
[env:native]
|
||||
platform = native
|
||||
|
||||
@@ -6,7 +6,9 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
|
||||
[ -n "${RORO_NO_DOCKER:-}" ] || docker build -q -t "$IMAGE" "$ROOT/docker" >/dev/null
|
||||
|
||||
# The Debug Console token (ADR 0004): made once, kept with the OTA key, never committed.
|
||||
# The Debug Console token of the developer's device (ADR 0010): made once, kept with the OTA key, never
|
||||
# committed and never compiled in. scripts/flash.sh --debug gives it to a device over USB, and
|
||||
# scripts/rdbg.py answers the device's challenge with it.
|
||||
DEBUG_TOKEN_FILE="$HOME/.config/roro9stack/debug-token"
|
||||
if [ ! -s "$DEBUG_TOKEN_FILE" ]; then
|
||||
mkdir -p "$(dirname "$DEBUG_TOKEN_FILE")"
|
||||
|
||||
@@ -7,8 +7,8 @@ DOCKER_EXTRA=()
|
||||
|
||||
case "${1:-all}" in
|
||||
tests) STEPS='pio test -e native' ;;
|
||||
builds) STEPS='pio run -e cardputer-adv -e cardputer-adv-debug' ;;
|
||||
all) STEPS='pio test -e native && pio run -e cardputer-adv -e cardputer-adv-debug' ;;
|
||||
builds) STEPS='pio run -e cardputer-adv' ;;
|
||||
all) STEPS='pio test -e native && pio run -e cardputer-adv' ;;
|
||||
*) echo "Usage: scripts/ci.sh [tests|builds]" >&2; exit 1 ;;
|
||||
esac
|
||||
run_in_container bash -c 'git config --global --add safe.directory "$PWD" && '"$STEPS"
|
||||
|
||||
@@ -1,12 +0,0 @@
|
||||
# Debug Builds only: compiles in the Debug Console token from $RORO_DEBUG_TOKEN (set by _docker.sh
|
||||
# from ~/.config/roro9stack/debug-token). Refuses to build without one rather than use a default.
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
Import("env") # noqa: F821 (provided by PlatformIO)
|
||||
|
||||
token = os.environ.get("RORO_DEBUG_TOKEN", "").strip()
|
||||
if not re.fullmatch(r"[0-9a-f]{32}", token):
|
||||
sys.exit("debug build: RORO_DEBUG_TOKEN is missing; build through scripts/ci.sh or scripts/flash.sh --debug")
|
||||
env.Append(CPPDEFINES=[("RORO_DEBUG_TOKEN", '\\"%s\\"' % token)]) # noqa: F821
|
||||
@@ -1,17 +1,21 @@
|
||||
#!/usr/bin/env bash
|
||||
# Flash the firmware over USB, then open the serial monitor.
|
||||
# Usage: scripts/flash.sh [--debug] [port] USB (default: the first Espressif device found)
|
||||
# scripts/flash.sh [--debug] --ota [host] Wi-Fi: build, sign and push a Firmware Update
|
||||
# (host: the device's IP from Settings > About, or $RORO_OTA_HOST)
|
||||
# --debug builds the Debug Build (cardputer-adv-debug): the Debug Console on TCP 2323, see ADR 0004.
|
||||
# scripts/flash.sh --ota [host] Wi-Fi: build, sign and push a Firmware Update
|
||||
# (host: the device's IP from Settings > Firmware, or $RORO_OTA_HOST)
|
||||
# --debug (USB only): once flashed, switches the Debug Console on and gives it the token in
|
||||
# ~/.config/roro9stack/debug-token, over the serial port (ADR 0010, Q192), so scripts/rdbg.py works
|
||||
# at once. The settings survive later updates: it is needed once for a device, not at each flash.
|
||||
set -euo pipefail
|
||||
ENV=cardputer-adv
|
||||
PROVISION=
|
||||
if [ "${1:-}" = "--debug" ]; then
|
||||
ENV=cardputer-adv-debug
|
||||
PROVISION=1
|
||||
shift
|
||||
fi
|
||||
|
||||
if [ "${1:-}" = "--ota" ]; then
|
||||
[ -z "$PROVISION" ] || echo "flash.sh: --debug does nothing over Wi-Fi: the console's setting stays as it is on the device" >&2
|
||||
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
||||
HOST="${2:-${RORO_OTA_HOST:-}}"
|
||||
[ -n "$HOST" ] || { echo "Usage: scripts/flash.sh --ota <device IP> (or set RORO_OTA_HOST)" >&2; exit 1; }
|
||||
@@ -19,7 +23,6 @@ if [ "${1:-}" = "--ota" ]; then
|
||||
DOCKER_EXTRA=()
|
||||
run_in_container bash -c "git config --global --add safe.directory /work && pio run -e $ENV" | tail -3
|
||||
VERSION="$(git -C "$ROOT" describe --tags --always --dirty)"
|
||||
[ "$ENV" = cardputer-adv-debug ] && VERSION="$VERSION+debug" # as scripts/version.py names it
|
||||
OUT="$ROOT/.pio/build/$ENV/roro9stack-$VERSION.ota"
|
||||
"$ROOT/scripts/make_ota.py" "$ROOT/.pio/build/$ENV/firmware.bin" "$VERSION" "$OUT"
|
||||
exec "$ROOT/scripts/ota_push.py" "$OUT" "$HOST"
|
||||
@@ -30,4 +33,10 @@ source "$(dirname "$0")/_docker.sh"
|
||||
docker rm -f roro9stack-serial >/dev/null 2>&1 || true # a serial log would hold the port
|
||||
DOCKER_EXTRA=(--device "$PORT" --group-add "$(stat -c %g "$PORT")" -it)
|
||||
|
||||
run_in_container bash -c "git config --global --add safe.directory /work && pio run -e $ENV -t upload --upload-port $PORT && pio device monitor -p $PORT -b 115200"
|
||||
# The token is read from the environment inside the container (_docker.sh passes it), so it is on no
|
||||
# command line of the host; serial_log.py prints what the device says, not what it sends.
|
||||
SETUP=""
|
||||
# serial_log.py finds the port by its name, and follows it when the device re-enumerates after the upload.
|
||||
[ -z "$PROVISION" ] || DOCKER_EXTRA=(--group-add "$(stat -c %g "$PORT")" --privileged -v /dev:/dev -it)
|
||||
[ -z "$PROVISION" ] || SETUP='&& /pio/penv/bin/python scripts/serial_log.py 8 sleep:4 "debug token $RORO_DEBUG_TOKEN" "debug on"'
|
||||
run_in_container bash -c "git config --global --add safe.directory /work && pio run -e $ENV -t upload --upload-port $PORT $SETUP && pio device monitor -p $PORT -b 115200"
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
# libssh ships its own copy of curve25519 (src/external/curve25519_ref.c), whose two functions,
|
||||
# crypto_scalarmult and crypto_scalarmult_base, have the names libsodium's have: and libsodium is
|
||||
# already in the firmware, for WireGuard. Two definitions don't link. libssh's copy is left out
|
||||
# of the build and it uses libsodium's, which is the same function (issue #2, docs/milestones/N1.md).
|
||||
#
|
||||
# A `pre:` script: the libraries are built by the platform's own script, which runs before any `post:` one.
|
||||
Import("env") # noqa: F821 (provided by PlatformIO)
|
||||
|
||||
env.AddBuildMiddleware(lambda env, node: None, "*LibSSH-ESP32*curve25519_ref.c") # noqa: F821
|
||||
@@ -1,10 +1,12 @@
|
||||
#!/usr/bin/env python3
|
||||
"""The Debug Console of a Debug Build, over Wi-Fi (TCP 2323, see ADR 0004).
|
||||
"""The Debug Console, over Wi-Fi (TCP 2323, see ADR 0010). Switch it on first: Settings > Debug Console.
|
||||
|
||||
Usage: scripts/rdbg.py [-H host] [-b] [command ...]
|
||||
Usage: scripts/rdbg.py [-H host] [-t token] [-b] [command ...]
|
||||
no command interactive: type commands, see every console line live; Ctrl-D or `quit` leaves
|
||||
command runs it and prints what follows, until the console has been quiet for a moment
|
||||
-H host the device's IP (Settings > Firmware), default $RORO_OTA_HOST
|
||||
-H host the device's IP (Settings > Debug Console), default $RORO_OTA_HOST
|
||||
-t token the device's token (also --token), as Settings > Debug Console shows it: dashes and
|
||||
case don't matter. Otherwise $RORO_DEBUG_TOKEN, otherwise ~/.config/roro9stack/debug-token
|
||||
-b also print the backlog the device sends on connecting (boot messages and so on)
|
||||
|
||||
Commands handled here as well as on the device:
|
||||
@@ -14,9 +16,11 @@ Commands handled here as well as on the device:
|
||||
put <file> [card path] copy a file to the SD card (default /updates/<name>), checked with SHA-256
|
||||
screenshot [file.png] what the screen shows (default screen-<date>.png), at 2x
|
||||
reset restart at once, even if the main loop is stuck
|
||||
The token is read from ~/.config/roro9stack/debug-token (made by the first build).
|
||||
The token never crosses the network: the device sends a challenge, and this answers with its HMAC.
|
||||
"""
|
||||
import gzip
|
||||
import hashlib
|
||||
import hmac
|
||||
import os
|
||||
import re
|
||||
import select
|
||||
@@ -26,9 +30,53 @@ import zlib
|
||||
import socket
|
||||
import sys
|
||||
import time
|
||||
import urllib.request
|
||||
|
||||
PORT = 2323
|
||||
TOKEN_FILE = os.path.expanduser("~/.config/roro9stack/debug-token")
|
||||
RELEASES = "https://git.twis.la/twisla/roro9stack/releases/download"
|
||||
|
||||
|
||||
def tidy_token(typed):
|
||||
"""As the device stores it (lib/debug/src/debug_auth.cpp): no dashes or spaces, in capitals."""
|
||||
tidy = "".join(c for c in typed if c not in "- \t\r\n").upper()
|
||||
return tidy.replace("O", "0").replace("I", "1").replace("L", "1") # read as the digits they look like
|
||||
|
||||
|
||||
def answer_for(token, nonce):
|
||||
"""What the device expects back for a challenge: HMAC-SHA256 of the nonce, keyed by the token."""
|
||||
return hmac.new(token.encode(), nonce, hashlib.sha256).hexdigest()
|
||||
|
||||
|
||||
def find_token(given):
|
||||
token = given or os.environ.get("RORO_DEBUG_TOKEN")
|
||||
if not token and os.path.exists(TOKEN_FILE):
|
||||
token = open(TOKEN_FILE).read()
|
||||
token = tidy_token(token or "")
|
||||
if not token:
|
||||
sys.exit("No token: pass -t <token>, set RORO_DEBUG_TOKEN, or put it in " + TOKEN_FILE +
|
||||
"\n(the device shows it in Settings > Debug Console)")
|
||||
return token
|
||||
|
||||
|
||||
def log_in(sock, token):
|
||||
"""Answers the device's challenge; returns the banner line, or exits saying what went wrong."""
|
||||
first = read_until(sock, b"\n", 10)
|
||||
if first is None:
|
||||
sys.exit("device: nothing came back. Is the console switched on (Settings > Debug Console)?")
|
||||
line = first.decode(errors="replace").strip()
|
||||
if line.startswith("locked"):
|
||||
sys.exit("device: closed for a minute after too many wrong tokens")
|
||||
challenge = re.search(r"challenge ([0-9a-f]{32})$", line)
|
||||
if not challenge:
|
||||
sys.exit("device: no challenge (an older firmware?): " + line[:60])
|
||||
sock.sendall((answer_for(token, bytes.fromhex(challenge.group(1))) + "\n").encode())
|
||||
banner = read_until(sock, b"\n", 15)
|
||||
if banner is None:
|
||||
sys.exit("device: no answer")
|
||||
if banner.startswith(b"denied"):
|
||||
sys.exit("device: wrong token (Settings > Debug Console shows the right one)")
|
||||
return banner
|
||||
|
||||
|
||||
def read_until(sock, marker, timeout):
|
||||
@@ -102,17 +150,40 @@ def crash_firmware(reply):
|
||||
return version.group(1) if version else None
|
||||
|
||||
|
||||
def have_elf(reply):
|
||||
"""Makes sure .pio/elves/ holds the ELF of the firmware that crashed: a release's is on Gitea."""
|
||||
folder = os.path.join(os.path.dirname(SCRIPTS), ".pio", "elves")
|
||||
key = crash_firmware(reply)
|
||||
version = re.search(r"last one in (\S+)", reply)
|
||||
version = version.group(1) if version else None
|
||||
names = os.listdir(folder) if os.path.isdir(folder) else []
|
||||
if not key or any(key in n for n in names) or not version or not re.fullmatch(r"v\d+\.\d+\.\d+", version):
|
||||
return # there already, or not a release: only tags are published
|
||||
url = f"{RELEASES}/{version}/roro9stack-{version}.elf.gz"
|
||||
try:
|
||||
elf = gzip.decompress(urllib.request.urlopen(url, timeout=60).read())
|
||||
except Exception as e:
|
||||
return print(f"(no ELF for {version} here, and none fetched from {url}: {e})")
|
||||
digest = hashlib.sha256(elf).hexdigest()[:16]
|
||||
os.makedirs(folder, exist_ok=True)
|
||||
path = os.path.join(folder, f"{version}.{digest}.elf") # as scripts/version.py names them
|
||||
open(path, "wb").write(elf)
|
||||
print(f"(fetched the ELF of {version} from its release: {os.path.relpath(path)})")
|
||||
|
||||
|
||||
def crash(sock):
|
||||
reply = run(sock, "crash")
|
||||
trace = re.search(r"backtrace((?: 0x[0-9a-f]+)+)", reply)
|
||||
key = crash_firmware(reply)
|
||||
if trace and key:
|
||||
have_elf(reply)
|
||||
print()
|
||||
subprocess.call([os.path.join(SCRIPTS, "decode_backtrace.sh"), key] + trace.group(1).split())
|
||||
|
||||
|
||||
def coredump(sock, path):
|
||||
info = run(sock, "crash", out=None)
|
||||
have_elf(info)
|
||||
sock.sendall(b"coredump get\n")
|
||||
# One buffer throughout: the header, the size and the first bytes often share a packet.
|
||||
buf = b""
|
||||
@@ -237,25 +308,27 @@ def interactive(sock):
|
||||
|
||||
def main():
|
||||
args = sys.argv[1:]
|
||||
host, backlog = os.environ.get("RORO_OTA_HOST"), False
|
||||
host, backlog, token = os.environ.get("RORO_OTA_HOST"), False, None
|
||||
while args and args[0].startswith("-"):
|
||||
flag = args.pop(0)
|
||||
if flag == "-H" and args:
|
||||
host = args.pop(0)
|
||||
elif flag in ("-t", "--token") and args:
|
||||
token = args.pop(0)
|
||||
elif flag == "-b":
|
||||
backlog = True
|
||||
else:
|
||||
sys.exit(__doc__)
|
||||
if not host:
|
||||
sys.exit("No device: pass -H <ip> or set RORO_OTA_HOST\n\n" + __doc__)
|
||||
token = open(TOKEN_FILE).read().strip()
|
||||
token = find_token(token)
|
||||
|
||||
with socket.create_connection((host, PORT), timeout=10) as sock:
|
||||
sock.sendall((token + "\n").encode())
|
||||
# The device may still be finishing a previous client: wait for this connection's banner.
|
||||
banner = read_until(sock, b"Backlog follows.\n", 15)
|
||||
if banner is None:
|
||||
sys.exit("device: no banner (wrong token, or another client is connected)")
|
||||
try:
|
||||
sock = socket.create_connection((host, PORT), timeout=10)
|
||||
except OSError as e:
|
||||
sys.exit(f"device: can't connect to {host}:{PORT} ({e}). Is the console switched on (Settings > Debug Console)?")
|
||||
with sock:
|
||||
banner = log_in(sock, token)
|
||||
show = sys.stdout if backlog or not args else None
|
||||
if show:
|
||||
show.write(banner.decode(errors="replace"))
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
#!/usr/bin/env bash
|
||||
# Creates the key CI uses to ask the web server for a site refresh, once (issue #79,
|
||||
# docs/milestones/W1.md), and says where each half goes. The private key stays in
|
||||
# ~/.config/roro9stack/ until it is pasted into the Gitea secret; it is never committed and this
|
||||
# script doesn't print it.
|
||||
#
|
||||
# scripts/site_deploy_keygen.sh [/full/path/to/rororefresh.sh] [the runner's address]
|
||||
set -euo pipefail
|
||||
KEY="${RORO_SITE_DEPLOY_KEY:-$HOME/.config/roro9stack/site-deploy-key}"
|
||||
COMMAND="${1:-/full/path/to/rororefresh.sh}"
|
||||
FROM="${2:-}"
|
||||
|
||||
if [ -e "$KEY" ]; then
|
||||
echo "A site deploy key already exists at $KEY; not overwriting it." >&2
|
||||
else
|
||||
mkdir -p "$(dirname "$KEY")"
|
||||
( umask 077; ssh-keygen -q -t ed25519 -N "" -C roro9stack-ci-site-refresh -f "$KEY" )
|
||||
fi
|
||||
|
||||
options="restrict,command=\"$COMMAND\""
|
||||
[ -z "$FROM" ] || options="from=\"$FROM\",$options"
|
||||
|
||||
cat <<TEXT
|
||||
|
||||
1. On the web server, as the user that runs the refresh, add this one line to ~/.ssh/authorized_keys:
|
||||
|
||||
$options $(cat "$KEY.pub")
|
||||
|
||||
restrict: no terminal, no forwarding of any kind. command=: whatever the client asks for, this
|
||||
runs instead.$([ -n "$FROM" ] || printf '\n Give the runner'"'"'s address as the second argument to add from="...": the key then works from there only.')
|
||||
|
||||
2. In Gitea, the repository's Settings > Actions > Secrets:
|
||||
|
||||
SITE_DEPLOY_KEY the whole of $KEY (the private key, with its BEGIN and END lines)
|
||||
SITE_DEPLOY_HOST the server's address as the runner reaches it, or address:port
|
||||
SITE_DEPLOY_USER that user's name
|
||||
SITE_DEPLOY_KNOWN_HOSTS the server's host key, one line, from a machine you trust the network of:
|
||||
ssh-keyscan -t ed25519 <address> (or: -p <port> <address>)
|
||||
and compare it with the server's own:
|
||||
ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub (on the server)
|
||||
ssh-keyscan -t ed25519 <address> | ssh-keygen -lf - (here)
|
||||
|
||||
3. Try it, from here, with the same four values in the environment:
|
||||
|
||||
SITE_DEPLOY_KEY="\$(cat $KEY)" SITE_DEPLOY_HOST=... SITE_DEPLOY_USER=... \\
|
||||
SITE_DEPLOY_KNOWN_HOSTS="\$(ssh-keyscan -t ed25519 ... 2>/dev/null)" scripts/site_refresh.sh
|
||||
|
||||
(with from= set, this works from the runner's address only.) Then, to see that the key can do
|
||||
nothing else: ssh -i $KEY <user>@<address> id must run the refresh, not \`id\`.
|
||||
|
||||
Once the secret is in Gitea, the copy at $KEY can be deleted.
|
||||
TEXT
|
||||
@@ -0,0 +1,51 @@
|
||||
#!/usr/bin/env bash
|
||||
# Asks the web server to rebuild the site (issue #79, docs/milestones/W1.md). Run by CI after a push
|
||||
# to main that changed the site, and after a release is published (the home page and Downloads
|
||||
# name the latest release when they are built).
|
||||
#
|
||||
# It only connects: the server's authorized_keys line forces the one command this key may run, so
|
||||
# nothing sent from here chooses what happens there. From the environment (Gitea secrets):
|
||||
# SITE_DEPLOY_KEY the private key (scripts/site_deploy_keygen.sh makes it)
|
||||
# SITE_DEPLOY_HOST the server, or server:port
|
||||
# SITE_DEPLOY_USER the user there
|
||||
# SITE_DEPLOY_KNOWN_HOSTS the server's host key, as a known_hosts line: nothing else is trusted
|
||||
# With none of them set it does nothing (a fork, or before the key is installed); with only some, it fails.
|
||||
set -euo pipefail
|
||||
|
||||
set_count=0
|
||||
for v in SITE_DEPLOY_KEY SITE_DEPLOY_HOST SITE_DEPLOY_USER SITE_DEPLOY_KNOWN_HOSTS; do
|
||||
[ -z "${!v:-}" ] || set_count=$((set_count + 1))
|
||||
done
|
||||
if [ "$set_count" = 0 ]; then
|
||||
echo "site refresh: no SITE_DEPLOY_* secrets here, nothing done"
|
||||
exit 0
|
||||
fi
|
||||
if [ "$set_count" != 4 ]; then
|
||||
echo "site refresh: SITE_DEPLOY_KEY, _HOST, _USER and _KNOWN_HOSTS are needed, and only $set_count of them are set" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if ! command -v ssh >/dev/null; then
|
||||
apt-get update -qq
|
||||
apt-get install -y -qq --no-install-recommends openssh-client >/dev/null
|
||||
fi
|
||||
|
||||
host="$SITE_DEPLOY_HOST" port=22
|
||||
case "$host" in
|
||||
*:*) port="${host##*:}" host="${host%:*}" ;;
|
||||
esac
|
||||
|
||||
# The key and the host key exist as files only while this runs, in a container that goes with the job.
|
||||
umask 077
|
||||
tmp="$(mktemp -d)"
|
||||
trap 'rm -rf "$tmp"' EXIT
|
||||
printf '%s\n' "$SITE_DEPLOY_KEY" > "$tmp/key"
|
||||
printf '%s\n' "$SITE_DEPLOY_KNOWN_HOSTS" > "$tmp/known_hosts"
|
||||
|
||||
# -F none: no configuration but this line. -T and no command: the server's forced command runs.
|
||||
ssh -F none -T -p "$port" -i "$tmp/key" \
|
||||
-o IdentitiesOnly=yes -o BatchMode=yes \
|
||||
-o StrictHostKeyChecking=yes -o UserKnownHostsFile="$tmp/known_hosts" -o GlobalKnownHostsFile=/dev/null \
|
||||
-o ConnectTimeout=20 -o ServerAliveInterval=15 -o ServerAliveCountMax=8 \
|
||||
"$SITE_DEPLOY_USER@$host"
|
||||
echo "site refresh: done"
|
||||
@@ -10,10 +10,15 @@ try:
|
||||
except Exception:
|
||||
version = "unknown"
|
||||
|
||||
if env["PIOENV"].endswith("-debug"): # noqa: F821
|
||||
version += "+debug" # a Debug Build says so wherever the version shows
|
||||
# The version goes into one generated header, read by one file (lib/version/src/version.cpp). As a -D
|
||||
# on every command line it made each new commit recompile everything, and no build cache could help
|
||||
# (issue #74). Written only when it changes, so an unchanged version rebuilds nothing.
|
||||
import os
|
||||
|
||||
env.Append(CPPDEFINES=[("RORO_VERSION", '\\"%s\\"' % version)]) # noqa: F821
|
||||
_header = os.path.join(env.subst("$PROJECT_DIR"), "lib", "version", "src", "version_generated.h") # noqa: F821
|
||||
_text = '#define RORO_VERSION "%s"\n' % version
|
||||
if not os.path.exists(_header) or open(_header).read() != _text:
|
||||
open(_header, "w").write(_text)
|
||||
|
||||
|
||||
# Keep every build's ELF, named by version and the first 16 hex digits of its SHA-256 (the core dump
|
||||
|
||||
@@ -8,6 +8,6 @@ sort_by = "weight"
|
||||
eyebrow = "Developer docs"
|
||||
+++
|
||||
|
||||
The firmware is open source (GPL-3.0) and lives on [Gitea](https://git.twis.la/twisla/roro9stack). It is built for one device, the M5Stack Cardputer ADV with the Cap LoRa-1262, and it is built to be **worked on without touching the device**: install a build over Wi-Fi, read its console, press its keys, take screenshots of it, copy files to and from its SD card, and fetch its crash dumps, all from a PC on the same network. The first section is about exactly that.
|
||||
The firmware is open source (GPL-3.0) and lives on [Gitea](https://git.twis.la/twisla/roro9stack). It is built for one device, the M5Stack Cardputer ADV with the Cap LoRa-1262, and it is built to be **worked on without touching the device**: install a build over Wi-Fi, read its console, press its keys, take screenshots of it, copy files to and from its SD card, and fetch its crash dumps, all from a PC on the same network. The first section is about exactly that. The same commands also run on the device itself, in the [Shell](/guide/shell/).
|
||||
|
||||
Some of these pages are written by hand. The rest are **generated from the repository's own documents** (the decisions, the milestone plans, the README and the firmware's own `help` text), so they are never out of date: each says which file it comes from.
|
||||
|
||||
@@ -10,4 +10,4 @@ weight = 2
|
||||
eyebrow = "Developer docs"
|
||||
+++
|
||||
|
||||
The firmware is built, tested and released **in Docker**, so that a clean machine, your machine and the CI runner all get the same result. These pages say how. For the Debug Console, which is how you work on a device once it is flashed, see [Debug Builds and the Debug Console](/dev/debug/).
|
||||
The firmware is built, tested and released **in Docker**, so that a clean machine, your machine and the CI runner all get the same result. These pages say how. For the Debug Console, which is how you work on a device once it is flashed, see [The Debug Console](/dev/debug/).
|
||||
|
||||
@@ -22,17 +22,17 @@ This runs the host-side unit tests (`test/`, `native` environment), then builds
|
||||
|
||||
`scripts/coverage.sh` runs the same tests with coverage counters and writes a line-by-line report to `.pio/coverage/index.html`. The badge above is its figure for `main`: the share of the lines of `lib/` that the host tests run. `lib/` is the logic that compiles on a PC; `lib/SD` (the card's driver) and `src/` (the Apps, the Services, everything that needs the device) have no host tests and aren't in that figure.
|
||||
|
||||
The framework is rebuilt with the TLS settings in `platformio.ini` (`custom_sdkconfig`, ADR 0006), so the first build after a fresh checkout takes about 4 minutes; later builds take under a minute.
|
||||
The framework is rebuilt with the TLS settings in `platformio.ini` (`custom_sdkconfig`, ADR 0006), so the first build after a fresh checkout takes about 6 minutes; later builds take about 15 seconds when little has changed. The platform knows the framework is already rebuilt by `sdkconfig.defaults` in the project folder, which it writes and git ignores: delete it and the next build rebuilds the framework. CI keeps that file, a build cache and ccache in its volume (docs/milestones/R1.md).
|
||||
|
||||
## CI and releases
|
||||
|
||||
Gitea Actions (`.gitea/workflows/ci.yml`, docs/milestones/R1.md) runs the host tests on every push to `main`, and on a pull request also builds the release firmware and the Debug Build: changes reach `main` through pull requests. Pushing a tag `v*` runs all of it and publishes a release on Gitea with:
|
||||
Gitea Actions (`.gitea/workflows/ci.yml`, docs/milestones/R1.md) runs the host tests on every push to `main`, and on a pull request also builds the firmware: changes reach `main` through pull requests. Pushing a tag `v*` runs all of it and publishes a release on Gitea with:
|
||||
|
||||
- `roro9stack-<version>.ota`, the signed Update File;
|
||||
- `roro9stack-<version>-factory.bin`, the whole flash image for a first install over USB;
|
||||
- `roro9stack-<version>.elf.gz`, to decode crash reports from that build;
|
||||
- `SHA256SUMS`.
|
||||
|
||||
CI signs with the project's key, held as a repository secret (ADR 0008). Debug Builds are built but never published: each carries its builder's Debug Console token.
|
||||
CI signs with the project's key, held as a repository secret (ADR 0008). There is one firmware: the Debug Console is in every build, switched off until its owner switches it on (ADR 0010).
|
||||
|
||||
`scripts/ota_verify.py <file.ota>` checks an Update File on a PC the way a device does. `scripts/release_build.sh` and `scripts/release_publish.py` are what the workflow runs; they work the same by hand.
|
||||
|
||||
@@ -54,4 +54,4 @@ The connection is checked against the two ISRG roots Let's Encrypt chains end in
|
||||
|
||||
**IRC steps aside.** A secure connection takes about 52 KB of memory at its peak, and IRC's own takes 40 KB of the 107 KB there is. A check or an install you ask for makes IRC disconnect for the few seconds it takes and reconnect afterwards. The daily check never does that: with IRC connected it waits for a moment when IRC isn't, so while IRC stays connected for days it doesn't run, and **Latest release** is the way to check.
|
||||
|
||||
**A Debug Build** shows the latest release but doesn't install it: releases have no Debug Console (they aren't built with one, so that nobody's console token is published), and installing one would take it away. A Debug Build is updated from the PC with `scripts/flash.sh --debug --ota`.
|
||||
**There is no separate Debug Build** any more (ADR 0010): every firmware installs releases, and the Debug Console is a setting, which an update leaves as it was.
|
||||
|
||||
@@ -24,7 +24,7 @@ scripts/ota_keygen.sh # once: creates the key p
|
||||
scripts/make_ota.py firmware.bin v0.12.0 out.ota # wraps and signs an image (the key: $RORO_OTA_KEY or the default path)
|
||||
scripts/ota_verify.py out.ota # checks a file the way a device does, on the PC
|
||||
scripts/ota_push.py out.ota 10.39.39.12 # pushes it to the Update Service, TCP 3232
|
||||
scripts/flash.sh --ota 10.39.39.12 # builds, signs and pushes in one go (add --debug for a Debug Build)
|
||||
scripts/flash.sh --ota 10.39.39.12 # builds, signs and pushes in one go
|
||||
```
|
||||
|
||||
`ota_push.py` prints the device's answer (`OK …`), or says the device **refused the update and hung up**, with the reason on the device's screen: the header is checked first, so a refused file stops mid-transfer.
|
||||
|
||||
@@ -10,7 +10,7 @@
|
||||
<text x="30" y="75" font-size="11" class="wi-dim">builds, signs, pushes</text>
|
||||
<rect class="wi-box" x="16" y="112" width="220" height="56" rx="6"/>
|
||||
<text x="30" y="135" font-size="12" font-weight="600">rdbg.py put</text>
|
||||
<text x="30" y="155" font-size="11" class="wi-dim">Debug Build, Wi-Fi 2323</text>
|
||||
<text x="30" y="155" font-size="11" class="wi-dim">Debug Console, Wi-Fi 2323</text>
|
||||
<rect class="wi-box" x="16" y="192" width="220" height="56" rx="6"/>
|
||||
<text x="30" y="215" font-size="12" font-weight="600">sd_put.sh</text>
|
||||
<text x="30" y="235" font-size="11" class="wi-dim">USB serial, card stays in</text>
|
||||
|
||||
|
Before Width: | Height: | Size: 4.3 KiB After Width: | Height: | Size: 4.3 KiB |
@@ -1,5 +1,5 @@
|
||||
+++
|
||||
title = "Debug Builds and the Debug Console"
|
||||
title = "The Debug Console"
|
||||
description = "A firmware you can drive from your desk: its console over Wi-Fi, its screen as a PNG, its SD card, its crash dumps, and a way back when an update goes wrong."
|
||||
template = "guide-index.html"
|
||||
page_template = "guide-page.html"
|
||||
@@ -10,13 +10,13 @@ weight = 1
|
||||
eyebrow = "Developer docs"
|
||||
+++
|
||||
|
||||
A **Debug Build** is the same firmware plus a **Debug Console**: the serial console, over Wi-Fi, behind a token. It is the most useful thing in the project. With it you can:
|
||||
The **Debug Console** is the serial console, over Wi-Fi, for whoever holds the device's token. It is in **every firmware**, switched off until you switch it on, and it is the most useful thing in the project. With it you can:
|
||||
|
||||
- **see everything the device prints**, boot messages included, without a cable;
|
||||
- **run every serial command** from your desk;
|
||||
- **press keys** and **take screenshots**, so a UI change can be tested and looked at remotely;
|
||||
- **copy files** to and from the SD card;
|
||||
- **read crash reports and fetch core dumps**, decoded against the exact build that crashed;
|
||||
- **update the firmware** over the same Wi-Fi, and know that a bad update rolls back to a build that still has the console.
|
||||
- **update the firmware** over the same Wi-Fi, and know that a bad update rolls back by itself.
|
||||
|
||||
Start with [Debug Builds](/dev/debug/debug-builds/) to put one on a device, then [the Debug Console](/dev/debug/console/). The rest are what you can do with it.
|
||||
Start with [Switch the console on](/dev/debug/switch-it-on/), then [the Debug Console](/dev/debug/console/). The rest are what you can do with it.
|
||||
|
||||
@@ -10,7 +10,7 @@ tag = "Reference"
|
||||
+++
|
||||
## What `help` prints
|
||||
|
||||
The firmware's own list, read from `src/main.cpp`. These work in every build, over USB serial:
|
||||
The firmware's own list, read from `src/main.cpp`. Every firmware has all of them, over USB serial and over the Debug Console. The ones marked *Debug Console only* exist only over Wi-Fi, where [`rdbg.py`](/dev/debug/files-and-screens/) speaks them, and the ones marked *USB serial only* only over the cable:
|
||||
|
||||
```
|
||||
info firmware, uptime, memory, Wi-Fi, app slots
|
||||
@@ -19,29 +19,33 @@ net bytes each network service has read and written since boot
|
||||
reboot restart
|
||||
boot other restart into the other app slot (manual Rollback)
|
||||
log level <0-5> ESP-IDF log level (0 none ... 5 verbose)
|
||||
ls [folder] | du <path> | mkdir <path> | rm <path> | cp [-f] <from> <to> | mv [-f] <from> <to> | cancel the SD card, with the Storage App's rules
|
||||
ls [folder] | du <path> | mkdir <path> | rm [-r] [-f] <path> | cp [-f] <from> <to> | mv [-f] <from> <to> | cancel the SD card, with the Storage App's rules (rm -r for a folder; -f: the Shell doesn't ask; * and ? in a name: /notes/*.txt)
|
||||
screenshot [seconds] the screen as a PNG in /screenshots on the card, now or after a pause
|
||||
lora probe | lora status | lora rx on|off | lora preset <name> the LoRa radio, receive only
|
||||
lora capture start|stop a LoRa Capture to /captures/lora (pcap, LoRaTap)
|
||||
lora sweep on [from MHz] [to MHz] [step kHz] | off | dump RSSI across a band (863 870 100)
|
||||
lora sweep on [from MHz] [to MHz] [step kHz] | lora sweep off | lora sweep dump RSSI across a band (863 870 100)
|
||||
lora custom <MHz> <BW kHz> <SF> <CR 5-8> <sync hex> [preamble] e.g. 868.1 125 7 5 34 8 (LoRaWAN)
|
||||
gnss quiet on|off pause the GNSS receiver while the LoRa radio listens (it costs the radio 8 dB)
|
||||
gnss status | gnss restart | gnss track start|stop | gnss nmea on|off | gnss send <sentence without $ and checksum>
|
||||
crash the last crash: firmware, reason, task, backtrace
|
||||
coredump erase forget the core dump in flash
|
||||
key <name|char> press a key: up down left right select back home del tab space, or one character
|
||||
key <name|char> press a key: up down left right select back home del tab space help shot, or one character; ctrl- alt- shift- before it (key ctrl-down)
|
||||
wifi status | wifi add <ssid><TAB><password>
|
||||
wifi ip <ssid> dhcp | wifi ip <ssid> <address>/<prefix> [gateway] a Saved Network's IP setting
|
||||
wifi dns <a> [b] | wifi dns always on|off | wifi ntp <a> [b] DNS and NTP servers
|
||||
gemini get <url> fetch a Gemini page and report header, size, certificate, heap
|
||||
irc start | irc stop | irc dump | irc say <buffer> <text>
|
||||
install <path.ota> Update from SD
|
||||
update check | list | status | install <tag> the project's releases on Gitea
|
||||
update check | update list | update status | update install <tag> the project's releases on Gitea
|
||||
sd card | sd list | cat <path> | log <text> | burst | sound on|off | short | normal
|
||||
```
|
||||
|
||||
A **Debug Build** adds these (the last ones, marked *Debug Console only*, exist only over Wi-Fi, where [`rdbg.py`](/dev/debug/files-and-screens/) speaks them):
|
||||
|
||||
```
|
||||
Irc | Wifi | Gnss | Gemini | Lora | Storage | Notes | Shell | System | Settings open that App: a capital letter is an App, not a command
|
||||
ping <host> [count] [size] | nslookup <name> [server] | port <host> <port> | traceroute <host> | cancel is it there, does its name resolve, is its port open, which way; one at a time
|
||||
tls <host> [port] | ntp [server] a TLS handshake: who the certificate is for, by whom, until when, and whether this device trusts it; a time server's clock against this one
|
||||
ifconfig | arp | netstat the interfaces (Wi-Fi and the VPN), their addresses, the default route and the DNS servers; the neighbours heard; what listens and what is connected
|
||||
ssh user@host[:port] | ssh status | ssh stop a terminal on another machine, in the SSH App; the password is asked there, never here
|
||||
vpn status | vpn up [seconds] | vpn down | vpn import [path] | vpn forget | vpn auto on|off the WireGuard tunnel (Settings > VPN); import reads /vpn/wg0.conf; with seconds, it goes down by itself
|
||||
debug status | debug off [seconds] the Debug Console over Wi-Fi (Settings > Debug Console); with seconds, it comes back
|
||||
debug on | debug token <16 to 64 characters> | debug token new (USB serial only) switch it on, set its token
|
||||
crash abort|wdt crash on purpose (to test crash reports and Safe Mode)
|
||||
wifi ip ... try <seconds> | wifi ip keep a trial IP setting: back to the previous one unless kept
|
||||
loop spin on|off make the main loop spin without resting, to compare load and radio noise
|
||||
@@ -50,11 +54,11 @@ lora inject <hex> [rssi] [snr] a packet into the LoRa Scanner as if received (
|
||||
sd fill <folder> <count> makes that many small files there, to test a crowded folder
|
||||
coredump get (Debug Console only) send the raw core dump: use scripts/rdbg.py coredump
|
||||
reset (Debug Console only) restart at once, even if the main loop is stuck
|
||||
get <path> | put <path> <size> <sha256> | screenshot (Debug Console only) binary, see rdbg.py
|
||||
get <path> | put <path> <size> <sha256> | screenshot (Debug Console only) binary, see rdbg.py: there, a bare `screenshot` sends the screen instead of saving it
|
||||
quit close the Debug Console connection
|
||||
```
|
||||
|
||||
In **Safe Mode** (see [Crashes and Safe Mode](/dev/debug/crashes/)) only a few run: `help`, `info`, `tasks`, `net`, `reboot`, `boot other`, `wifi status`, and anything starting with `log level`, `crash`, `coredump`, `wifi add`. Anything else answers `not available in Safe Mode`.
|
||||
In **Safe Mode** (see [Crashes and Safe Mode](/dev/debug/crashes/)) only a few run: `help`, `info`, `tasks`, `net`, `reboot`, `boot other`, `wifi status`, and anything starting with `log level`, `crash`, `coredump`, `wifi add`, `debug`. Anything else answers `not available in Safe Mode`.
|
||||
|
||||
## What they do
|
||||
|
||||
@@ -63,16 +67,16 @@ In **Safe Mode** (see [Crashes and Safe Mode](/dev/debug/crashes/)) only a few r
|
||||
| Command | Effect |
|
||||
|---|---|
|
||||
| `burst` | Publishes 5 Notifications at once |
|
||||
| `key up\|down\|left\|right\|select\|back\|home`, or `key <char>` | Injects a key press |
|
||||
| `key up\|down\|left\|right\|select\|back\|home\|del\|tab\|space\|help\|shot`, or `key <char>` | Injects a key press (`help` is Fn+h: the keys of the screen that is showing; `shot` is Fn+p: a screenshot). `ctrl-`, `alt-` and `shift-` before it hold that key: `key ctrl-down`, `key alt-up`, `key ctrl-b` |
|
||||
| `sound on` / `sound off` | Toggles the Sound setting (beep + LED) |
|
||||
| `short` / `normal` | Screen timeouts 5 s / 10 s, or 30 s / 60 s |
|
||||
| `wifi add <ssid><TAB><password>` | Adds a Saved Network (so credentials stay out of the repo) |
|
||||
| `wifi ip <ssid> dhcp` / `wifi ip <ssid> <address>/<prefix> [gateway]` | A Saved Network's IP setting: Automatic, or Fixed. Debug Builds: add `try <seconds>` to go back to the previous setting unless `wifi ip keep` follows |
|
||||
| `wifi ip <ssid> dhcp` / `wifi ip <ssid> <address>/<prefix> [gateway]` | A Saved Network's IP setting: Automatic, or Fixed. Add `try <seconds>` to go back to the previous setting unless `wifi ip keep` follows |
|
||||
| `wifi dns <a> [b]` / `wifi dns always on\|off` / `wifi ntp <a> [b]` | DNS servers (used on Fixed networks, or always), and NTP servers |
|
||||
| `log <text>` | Appends a line to a test IRC Log (`/irc/dev/#test/<date>.log`) |
|
||||
| `sd card` | What the SD card says it is: type, size, and its identity register (maker, name, revision, serial, date) |
|
||||
| `sd list` | Lists the files of each Storage Clean-up category |
|
||||
| `sd fill <folder> <count>` | Debug Builds: makes that many small files in a folder, to test a crowded one |
|
||||
| `sd fill <folder> <count>` | Makes that many small files in a folder, to test a crowded one |
|
||||
| `cat <path>` | Prints the first ~1.2 KB of a file on the SD card |
|
||||
| `irc start` | Starts the IRC Service (normally done by opening the IRC App) |
|
||||
| `irc stop` | Stops it, as `/quit` does: QUIT if connected, no more retries, and the App stays disconnected until you type |
|
||||
@@ -81,16 +85,18 @@ In **Safe Mode** (see [Crashes and Safe Mode](/dev/debug/crashes/)) only a few r
|
||||
| `irc say <buffer> <text>` | Types into a Buffer, commands included (`irc say 0 /join #test`) |
|
||||
| `irc dump` | Prints IRC status, memory, and the last lines of each Buffer |
|
||||
| `wifi status` | Prints Wi-Fi state, network, signal, clock and free heap, then the address, gateway, DNS and NTP servers in use and where each came from |
|
||||
| `info` | Firmware, uptime, last start reason, memory, Wi-Fi, the SD card and its write faults since boot, and both app slots with their versions and OTA states |
|
||||
| `info` | Firmware, uptime, last start reason, memory, Wi-Fi, the SD card and its write faults since boot, **which App is in front**, and both app slots with their versions and OTA states |
|
||||
| `Notes`, `Irc`, `Wifi`, `Gnss`, `Gemini`, `Lora`, `Storage`, `Shell`, `System`, `Settings` | Opens that App: a capital letter is an App, not a command |
|
||||
| `tasks` | FreeRTOS tasks over the next second: state, priority, lowest free stack, share of a core, each core's load, and how many passes the main loop made |
|
||||
| `net` | Bytes each network service has read and written since boot |
|
||||
| `reboot` / `boot other` | Restart, or restart into the other app slot (a manual Rollback) |
|
||||
| `log level <0-5>` | ESP-IDF log level |
|
||||
| `ls [folder]` / `du <path>` | Lists a folder of the SD card with sizes and dates, or counts the files and bytes under a path |
|
||||
| `cp [-f] <from> <to>` / `mv [-f] <from> <to>` / `rm <path>` / `mkdir <path>` / `cancel` | What the Storage App does, with its rules: copy (folders too), move or rename, delete (a folder with what's in it), new folder. `-f` replaces a file that's in the way; a tab separates paths that hold spaces; `cancel` stops a copy or a delete |
|
||||
| `screenshot [seconds]` | The screen as a PNG in `/screenshots` on the card, now or after a pause to get to the screen you want (240 x 135, about 33 KB). Over the Debug Console a bare `screenshot` sends the screen to the PC instead |
|
||||
| `cp [-f] <from> <to>` / `mv [-f] <from> <to>` / `rm [-r] [-f] <path>` / `mkdir <path>` / `cancel` | What the Storage App does, with its rules: copy (folders too), move or rename, delete (`rm -r` for a folder and what's in it, as Unix has it), new folder. `-f` replaces a file that's in the way; `*` and `?` in the last part of a path (`ls`, `du`, `rm`, `cp`, `mv`) run the command for each name matched, 64 at most; a tab separates paths that hold spaces; `cancel` stops a copy or a delete |
|
||||
| `install <path>` | Update from SD with that `.ota` file, as Settings → Firmware does |
|
||||
| `update check` / `update list` / `update status` / `update install <tag>` | The project's releases on Gitea: look at the latest, list the last ten, say what's known, or download and install one (not on a Debug Build) |
|
||||
| `update pretend <version>` / `update probe <host>` / `update damage cut\|flip <n>` / `update daily` | Debug Builds: pretend to run another version (so a release counts as an update), see whether a server's certificate is accepted, cut or damage the next download, run the daily check again |
|
||||
| `update check` / `update list` / `update status` / `update install <tag>` | The project's releases on Gitea: look at the latest, list the last ten, say what's known, or download and install one |
|
||||
| `update pretend <version>` / `update probe <host>` / `update damage cut\|flip <n>` / `update daily` | Pretend to run another version (so a release counts as an update), see whether a server's certificate is accepted, cut or damage the next download, run the daily check again |
|
||||
| `lora probe` | Finds the radio: chip, oscillator, antenna switch, DIO1 interrupt, noise floor |
|
||||
| `lora status` | Radio settings, who's listening, packet and error counters, noise floor, task stack |
|
||||
| `lora rx on` / `lora rx off` | Listens and prints each packet on the console |
|
||||
@@ -102,12 +108,19 @@ In **Safe Mode** (see [Crashes and Safe Mode](/dev/debug/crashes/)) only a few r
|
||||
| `gnss track start` / `gnss track stop` | A Track, as `r` in the GNSS App (the reason is printed if it can't start) |
|
||||
| `gnss nmea on` / `gnss nmea off` | Prints each NMEA sentence the receiver sends, as `nmea: …` |
|
||||
| `gnss send <sentence>` | Sends a sentence to the receiver, without the `$` and the checksum (it adds them) |
|
||||
| `lora noise test [gnss\|quiet]` / `lora noise report` | Debug Builds: Sweep under one changed condition at a time to find what raises the noise floor (Wi-Fi goes off for a few seconds); then the result |
|
||||
| `lora inject <hex> [rssi] [snr]` | Debug Builds: a packet into the Scanner as if received (nothing is sent) |
|
||||
| `lora noise test [gnss\|quiet]` / `lora noise report` | Sweep under one changed condition at a time to find what raises the noise floor (Wi-Fi goes off for a few seconds); then the result |
|
||||
| `lora inject <hex> [rssi] [snr]` | A packet into the Scanner as if received (nothing is sent) |
|
||||
| `crash` | The last crash: which firmware, why, task, PC and backtrace (from the core dump in flash) |
|
||||
| `coredump erase` | Forgets the core dump |
|
||||
| `loop spin on` / `loop spin off` | Debug Builds: make the main loop spin without resting, to compare load and radio noise |
|
||||
| `crash abort` / `crash wdt` | Debug Builds: crash on purpose, or hang the main loop until the watchdog fires |
|
||||
| `loop spin on` / `loop spin off` | Make the main loop spin without resting, to compare load and radio noise |
|
||||
| `crash abort` / `crash wdt` | Crash on purpose, or hang the main loop until the watchdog fires |
|
||||
| `ping <host> [count] [size]` / `nslookup <name> [server]` / `port <host> <port>` / `traceroute <host>` / `cancel` | Network troubleshooting (issue #90): does a host answer and how fast; a name's addresses, from which DNS server and in how long; is a TCP port open, refused or silent; the routers on the way. Each runs on a task of its own and prints as it goes, one at a time; `cancel` stops it |
|
||||
| `tls <host> [port]` / `ntp [server]` | A TLS handshake that checks nothing, then the certificate said in words: who it is for, who signed it, until when, its SHA-256, and whether this device's roots and the name asked for accept it (about 52 KB of heap while it runs; refused under 70 KB free). A time server's clock against the device's, with the round trip |
|
||||
| `ifconfig` / `arp` / `netstat` | The interfaces (Wi-Fi and the VPN) with their addresses, MTU, which is the default route, and the DNS servers; the neighbours heard on the Wi-Fi; what listens and what is connected |
|
||||
| `ssh user@host[:port]` / `ssh status` / `ssh stop` | Opens the SSH App and connects (the password is asked there, never on a console); the session's state and this device's public key; end the session |
|
||||
| `vpn status` / `vpn up [seconds]` / `vpn down` / `vpn import [path]` / `vpn forget` / `vpn auto on\|off` | The WireGuard tunnel: its state, on (for that many seconds, then off by itself: for trying a configuration from afar), off, read a `.conf` from the card (`/vpn/wg0.conf`), erase it, start with Wi-Fi. No key is ever printed |
|
||||
| `debug status` / `debug off [seconds]` | The Debug Console: whether it's on, has a token and a client; switch it off. With a number of seconds, it comes back by itself after that long |
|
||||
| `debug on` / `debug token <value>` / `debug token new` | USB serial only: switch it on (making a token if there's none), give it a token of 16 to 64 characters, or make a new one. The token is never printed |
|
||||
| `help` | Lists the commands |
|
||||
|
||||
`scripts/flash.sh` stops a running serial log first, since it would hold the port.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
+++
|
||||
title = "The Debug Console"
|
||||
description = "Connect to a Debug Build over Wi-Fi: the protocol, what you get when you connect, how commands run, and what the console can and cannot do."
|
||||
description = "Connect to the console over Wi-Fi: the protocol, what you get when you connect, how commands run, and what the console can and cannot do."
|
||||
weight = 2
|
||||
[extra]
|
||||
tag = "Console"
|
||||
@@ -17,16 +17,16 @@ scripts/rdbg.py info # one command, and its reply
|
||||
scripts/rdbg.py -b tasks # the same, with the backlog shown first
|
||||
```
|
||||
|
||||
`scripts/rdbg.py` is a Python script with no dependencies. A one-shot command prints the reply and what follows, until the console has been quiet for a moment (1.5 seconds). It reads the token from `~/.config/roro9stack/debug-token` and talks to **TCP 2323** on the device's address. In interactive mode, <kbd>Ctrl</kbd>+<kbd>D</kbd> or `quit` leaves. Piped input works too, but `rdbg.py` leaves **as soon as its input ends**, before the replies arrive: keep the input open for a few seconds, as in `(printf 'info\ntasks\n'; sleep 3) | scripts/rdbg.py`. For one command, the one-shot form above is simpler.
|
||||
`scripts/rdbg.py` is a Python script with no dependencies. A one-shot command prints the reply and what follows, until the console has been quiet for a moment (1.5 seconds). It takes the token from `-t`, `$RORO_DEBUG_TOKEN` or `~/.config/roro9stack/debug-token` ([Switch the console on](/dev/debug/switch-it-on/)) and talks to **TCP 2323** on the device's address. In interactive mode, <kbd>Ctrl</kbd>+<kbd>D</kbd> or `quit` leaves. Piped input works too, but `rdbg.py` leaves **as soon as its input ends**, before the replies arrive: keep the input open for a few seconds, as in `(printf 'info\ntasks\n'; sleep 3) | scripts/rdbg.py`. For one command, the one-shot form above is simpler.
|
||||
|
||||
The device listens **only while Wi-Fi is connected**, and the console follows Wi-Fi: it stops listening when Wi-Fi drops and starts again when it is back.
|
||||
The device listens **only while the console is switched on and Wi-Fi is connected**, and the console follows Wi-Fi: it stops listening when Wi-Fi drops and starts again when it is back.
|
||||
|
||||
## What you get
|
||||
|
||||
On connecting, in order:
|
||||
|
||||
1. a **banner**: `roro9stack v0.11.0-3-g12006bd+debug debug console. 'help' lists the commands. Backlog follows.`
|
||||
2. the **backlog**: the last **4 KB** of console output, oldest first, **boot messages included** (a ring buffer in RAM);
|
||||
1. a **banner**: `roro9stack v0.12.0 debug console. 'help' lists the commands. Backlog follows.`
|
||||
2. the **backlog**: the last **4 KB** of console output, oldest first (a ring buffer in RAM, kept while the console is switched on: boot messages included, if it was on at boot);
|
||||
3. then **every new line, live**: everything the firmware prints, and ESP-IDF's own log lines, which are copied into the same stream (they still reach the USB port too);
|
||||
4. the **replies to your commands**, in the same stream, each preceded by the `> command` line when it runs.
|
||||
|
||||
@@ -49,12 +49,17 @@ It is a plain line protocol, easy to speak from anything. This is what `rdbg.py`
|
||||
| Step | Detail |
|
||||
|---|---|
|
||||
| Connect | TCP 2323. **One client at a time**: a second one waits until the first leaves |
|
||||
| Authenticate | Send the token and `\n` within **10 seconds**. The comparison takes the same time whatever you send |
|
||||
| Refused | After about a second the device sends `denied\n` and hangs up, and prints `debug: refused a client from <ip>` on its own console. No quick retries |
|
||||
| Challenge | The device sends one line: `roro9stack debug console, challenge <32 hex digits>`. Those are 16 random bytes, new each time |
|
||||
| Answer | Within **10 seconds**, send the **HMAC-SHA256 of those 16 bytes, keyed by the token**, as 64 hex digits and `\n`. The token is the tidied one: capitals, no dashes |
|
||||
| Accepted | The banner line, then the backlog |
|
||||
| Refused | After about a second the device sends `denied\n` and hangs up, and prints `debug: refused a client from <ip>` on its own console |
|
||||
| Locked | **Five wrong answers in a row** close the console to everyone for 60 seconds: it answers `locked\n` and hangs up, and a Toast on the device names the address they came from |
|
||||
| Commands | One line each, up to 240 bytes. The device **queues at most 8**; past that, `debug: busy, command dropped` |
|
||||
| Leave | `quit` or `exit` closes the connection |
|
||||
| Binary commands | `get`, `put`, `screenshot`, `coredump get`: a text header line, then raw bytes (see [files and screens](/dev/debug/files-and-screens/)) |
|
||||
|
||||
**The token never crosses the network.** Someone on the same Wi-Fi who records a login gets a challenge and its answer, which are no use for the next challenge. The comparison on the device takes the same time whatever it is given.
|
||||
|
||||
A command that never runs has not been dropped by the network: the main loop is busy or stuck. That is what the next section is about.
|
||||
|
||||
## How commands run
|
||||
@@ -69,22 +74,26 @@ A command that never runs has not been dropped by the network: the main loop is
|
||||
|
||||
## Security
|
||||
|
||||
- The token is checked **before anything else**, and a wrong one costs a second.
|
||||
- Anyone on the same network **with the token** can read the console, press keys and restart the device. The console never prints stored secrets (Wi-Fi and IRC passwords), but the IRC traffic it shows is readable.
|
||||
- The stream is **plain text**: fine on a home network, not across the internet. Do not forward port 2323.
|
||||
- **Release builds have no console at all.** Nothing listens.
|
||||
- **Off by default.** Nothing listens until the console is switched on at the device, or over USB.
|
||||
- The login is a **challenge and an answer**: the token itself is never sent, and five wrong answers close the console for a minute.
|
||||
- Anyone on the same network **with the token** can read the console, press keys, copy the SD card's files and restart the device. The console never prints stored secrets (the token, Wi-Fi and IRC passwords), but the IRC traffic it shows is readable.
|
||||
- They cannot change the firmware: an update still has to be **signed**.
|
||||
- The stream after the login is **plain text**: fine on a home network, not across the internet. Do not forward port 2323.
|
||||
|
||||
The decision is [ADR 0004](/dev/decisions/0004-debug-console-in-debug-builds/).
|
||||
The decisions are [ADR 0010](/dev/decisions/0010-debug-console-in-every-build/) and, for how the console works inside, [ADR 0004](/dev/decisions/0004-debug-console-in-debug-builds/).
|
||||
|
||||
## Without `rdbg.py`
|
||||
|
||||
Anything that can open a TCP connection works. The token and each command are just lines:
|
||||
Anything that can open a TCP connection and compute an HMAC works:
|
||||
|
||||
```python
|
||||
import socket
|
||||
import hashlib, hmac, socket
|
||||
token = "K7QF3M2X9WBDHT4P6RNC" # tidied: capitals, no dashes
|
||||
s = socket.create_connection(("10.39.39.12", 2323))
|
||||
s.sendall(b"<token>\n") # then read the banner line
|
||||
s.sendall(b"info\n") # read until the stream goes quiet
|
||||
challenge = s.makefile().readline().split()[-1] # "... challenge 0f1e2d..."
|
||||
answer = hmac.new(token.encode(), bytes.fromhex(challenge), hashlib.sha256).hexdigest()
|
||||
s.sendall((answer + "\n").encode()) # then read the banner line
|
||||
s.sendall(b"info\n") # and what follows, until the stream goes quiet
|
||||
```
|
||||
|
||||
`scripts/rdbg.py` adds the parts that need work on the PC side: decoding crash backtraces, saving files and screenshots, and checking checksums.
|
||||
|
||||
@@ -6,7 +6,7 @@ weight = 5
|
||||
tag = "Console"
|
||||
+++
|
||||
|
||||
Rollback (see [How an update works](/dev/build/how-an-update-works/)) protects against **new** firmware that fails Probation. These measures cover firmware that was **confirmed and then crashes**: a corrupt setting, a server that sends something unexpected, a bug that takes an hour to show. They are in **every build**, release and Debug alike; the Debug Console is what makes them convenient. The decision is [ADR 0005](/dev/decisions/0005-safe-mode-crash-reports-watchdog/).
|
||||
Rollback (see [How an update works](/dev/build/how-an-update-works/)) protects against **new** firmware that fails Probation. These measures cover firmware that was **confirmed and then crashes**: a corrupt setting, a server that sends something unexpected, a bug that takes an hour to show. They are in every firmware; the Debug Console is what makes them convenient. The decision is [ADR 0005](/dev/decisions/0005-safe-mode-crash-reports-watchdog/).
|
||||
|
||||
## What a crash leaves behind
|
||||
|
||||
@@ -18,7 +18,7 @@ The `crash` command prints it again whenever you like:
|
||||
|
||||
```
|
||||
> crash
|
||||
crash: last one in v0.9.0-1-g4ab873e-dirty+debug (panic)
|
||||
crash: last one in v0.9.0-1-g4ab873e-dirty (panic)
|
||||
crash: task loopTask, pc 0x4037e179, cause 0, address 0x00000000
|
||||
crash: reason: abort() was called at PC 0x421209b3 on core 1
|
||||
crash: backtrace 0x4037e179 0x4037e141 0x4038582d 0x421209b3 ...
|
||||
@@ -40,7 +40,8 @@ scripts/rdbg.py coredump my.bin # ...to a file you name
|
||||
- **`crash`** runs the device's `crash`, finds the ELF by digest (or by version), and passes the backtrace to `scripts/decode_backtrace.sh`, which runs `addr2line` from the build container: one line for each frame, with function and file:line.
|
||||
- **`coredump`** fetches the raw dump over the console (it works when the main loop is stuck) and runs `scripts/decode_coredump.sh`: `esp-coredump` and GDB, giving **every task's backtrace, the registers, and the crashed task's stack**.
|
||||
- By hand: `scripts/decode_backtrace.sh <version|digest> <address>...`, and `scripts/decode_coredump.sh <core.bin> <version|digest>`. Both say which ELF they used.
|
||||
- If no archived ELF matches, they say so: the build was made on another machine, or `.pio/` was cleaned.
|
||||
- **A release's ELF is fetched for you.** When the crash names a released version (`v0.12.0`) and no local ELF matches, `rdbg.py` downloads `roro9stack-<version>.elf.gz` from the release on Gitea into `.pio/elves/`, so a crash on a firmware you did not build can be decoded. Decoding itself still runs in the build container.
|
||||
- If nothing matches, the scripts say so: an unreleased build made on another machine, or `.pio/` was cleaned.
|
||||
|
||||
## The main loop is watched
|
||||
|
||||
@@ -52,21 +53,19 @@ An installed update no longer depends on the main loop either: the Update Servic
|
||||
|
||||
The count of starts that follow a crash (a panic or the watchdog) is kept in NVS. After **three in a row**, the firmware starts **Safe Mode** instead of everything else:
|
||||
|
||||
- only the **clock, Wi-Fi, the Update Service and, in a Debug Build, the Debug Console** start: no Apps, no IRC, no SD card;
|
||||
- only the **clock, Wi-Fi, the Update Service and, if it is switched on, the Debug Console** start: no Apps, no IRC, no SD card;
|
||||
- the screen says so, with the **address to push an update to**;
|
||||
- only a few commands run (the [command reference](/dev/debug/commands/) lists them); anything else answers `not available in Safe Mode`.
|
||||
|
||||
Any **normal restart** (`reboot`, an update) or **a minute of uptime** resets the count. So Safe Mode is reached by crashing three times *quickly*, and a crash loop costs about half a minute before the device becomes reachable. To leave it: push a fix (`scripts/flash.sh --debug --ota`), or `reboot`.
|
||||
Any **normal restart** (`reboot`, an update) or **a minute of uptime** resets the count. So Safe Mode is reached by crashing three times *quickly*, and a crash loop costs about half a minute before the device becomes reachable. To leave it: push a fix (`scripts/flash.sh --ota`), or `reboot`. Over USB, `debug on` works in Safe Mode too, if the console was off.
|
||||
|
||||
**Its limit:** if Wi-Fi or the Update Service is what crashes, Safe Mode cannot help, and it takes USB.
|
||||
|
||||
## Crash on purpose
|
||||
|
||||
On a Debug Build:
|
||||
|
||||
```
|
||||
crash abort # abort(): a panic with a core dump
|
||||
crash wdt # hang the main loop until the task watchdog fires
|
||||
```
|
||||
|
||||
Use them to see a real report and decode it, to check that **the counter reaches Safe Mode**, and to prove that a Debug Build you are about to rely on will survive its own crashes. The device restarts by itself after either, and the report is there to read when the console comes back.
|
||||
Use them to see a real report and decode it, to check that **the counter reaches Safe Mode**, and to prove that a build you are about to rely on will survive its own crashes. The device restarts by itself after either, and the report is there to read when the console comes back.
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
+++
|
||||
title = "Debug Builds"
|
||||
description = "What a Debug Build is, how to build and flash one, how its token works, and why you should keep one in the fallback slot."
|
||||
weight = 1
|
||||
[extra]
|
||||
tag = "Start here"
|
||||
+++
|
||||
|
||||
## What it is
|
||||
|
||||
A Debug Build is built from the same source as a release, with the `cardputer-adv-debug` environment (it `extends` `cardputer-adv` in `platformio.ini`) and `-DRORO_DEBUG`. Differences:
|
||||
|
||||
- the **Debug Console** on TCP **2323** (next page);
|
||||
- extra commands, only meant for testing: crash on purpose, fake an installed version, damage a download, inject a LoRa packet, fill a folder with files (see the [command reference](/dev/debug/commands/));
|
||||
- the version ends in **`+debug`** (`scripts/version.py`) wherever the version shows: Settings, `info`, the Update Service. A `+debug` version compares **equal** to its release counterpart, so going between the two is never refused as a downgrade.
|
||||
|
||||
**It is compiled out of release builds, not switched off by a setting.** A console that runs commands, presses keys and reboots the device is a remote control; in a release build nothing listens and the code is not there.
|
||||
|
||||
## Build and flash one
|
||||
|
||||
Everything runs in Docker (see [Build, test and release](/dev/build/build-and-test/)). Over USB:
|
||||
|
||||
```sh
|
||||
scripts/flash.sh --debug # builds cardputer-adv-debug, uploads, opens the serial monitor
|
||||
```
|
||||
|
||||
Once a Debug Build is on the device, every later one can go over Wi-Fi, with no cable:
|
||||
|
||||
```sh
|
||||
export RORO_OTA_HOST=10.39.39.12 # the address Settings > Firmware shows
|
||||
scripts/flash.sh --debug --ota # builds, signs and pushes; the device installs and restarts
|
||||
```
|
||||
|
||||
The device's address is on **Settings → Firmware** ("Push to", which also gives port 3232, the update port). The Debug Console is on the same address, port 2323. See [Flash and update](/dev/build/flash/) for the update side.
|
||||
|
||||
## The token
|
||||
|
||||
The console asks for a secret first. It is **128 random bits**, made the first time any build runs (`scripts/_docker.sh`), kept in `~/.config/roro9stack/debug-token` next to the update-signing key, and passed into the build container as `RORO_DEBUG_TOKEN`. `scripts/debug_flags.py` compiles it into the firmware, and **refuses to build a Debug Build without one**, rather than fall back on a default.
|
||||
|
||||
- It is **never committed.** Releases have no console, so nothing of it is published: CI builds a Debug Build on a pull request to prove it still compiles, but never publishes it, because each one carries its builder's token.
|
||||
- It guards against **the network**, not against someone who holds the device: anyone with USB access can flash anything anyway ([ADR 0003](/dev/decisions/0003-own-signature-check-not-secure-boot/)).
|
||||
- To change it, delete the file and build again; the new token goes into the next Debug Build you flash.
|
||||
|
||||
## Keep a Debug Build in the other slot
|
||||
|
||||
The device has two app slots, so an update never overwrites the running firmware. If a new firmware crashes during [Probation](/dev/build/how-an-update-works/), the device goes **back to the previous one**, whatever that is. As long as you develop on Debug Builds, **the firmware a crash falls back to has the console too**, so a bad update never costs you remote access. Pushing a *release* build over a Debug Build leaves the Debug Build in the other slot until the next update overwrites it.
|
||||
|
||||
So the habit is: develop on Debug Builds, release by tag, and think twice before pushing a release over the only Debug Build you have.
|
||||
|
||||
## Releases and Debug Builds
|
||||
|
||||
- A Debug Build shows the latest release in Settings → Firmware but **never installs it**: that would replace the console with a release that has none. Update a Debug Build from the PC with `scripts/flash.sh --debug --ota`.
|
||||
- A Debug Build still checks and lists releases, which is useful for testing the update path: `update pretend` makes a released version count as newer (see [Drive the UI](/dev/debug/drive-the-ui/)).
|
||||
- **Safe Mode** (after 3 crashes in a row) keeps the Debug Console running, so a crash loop is something you fix remotely: see [Crashes and Safe Mode](/dev/debug/crashes/).
|
||||
|
||||
The reasoning is in [ADR 0004](/dev/decisions/0004-debug-console-in-debug-builds/).
|
||||
@@ -11,7 +11,7 @@ Everything the keyboard can do, a command can do, and everything on the screen c
|
||||
## Keys
|
||||
|
||||
```
|
||||
key up|down|left|right|select|back|home|del|tab|space
|
||||
key up|down|left|right|select|back|home|del|tab|space|help|shot
|
||||
key a # any single character: it is typed
|
||||
```
|
||||
|
||||
@@ -20,7 +20,19 @@ Two things to know before you use them:
|
||||
1. **A name `key` does not know is `select`.** `key sleect` presses Enter. A single character is typed as that character; anything else that is not a known name is treated as Enter. Check what you type.
|
||||
2. **A key that wakes a dark screen only wakes it.** The device's power policy swallows the key press that turns the screen back on, as it does for the real keyboard: the first `key` after the screen went off does nothing else. Send `key back` (harmless) first, or keep the screen on with the `normal` and `short` commands below.
|
||||
|
||||
`Fn` combinations, modifiers and the compose key have no command: the arrows are `key up|down|left|right`, and `key back` is the back key (`` ` `` on the device). Text is typed one character at a time.
|
||||
**Ctrl, Alt and Shift** go before the name: `key ctrl-down`, `key alt-up`, `key ctrl-b`, `key shift-alt-down`. `Fn` combinations and the compose key have no command: the arrows are `key up|down|left|right` (what `Fn` with `;` `.` `,` `/` gives on the device), and `key back` is the back key (`` ` `` on the device). Text is typed one character at a time.
|
||||
|
||||
**`key help` opens the help panel** (<kbd>Fn</kbd>+<kbd>h</kbd> on the device): the keys of the screen that is showing. A screenshot of it is the quickest way to learn what a screen accepts, and it is how every screen's list was checked. Any key but the arrows closes it.
|
||||
|
||||
## Open an App by its name
|
||||
|
||||
```
|
||||
Notes # an App's name, with a capital: opens it
|
||||
Shell # Irc Wifi Gnss Gemini Lora Storage Notes Shell System Settings
|
||||
info # ...and `app: Notes` says which App is in front
|
||||
```
|
||||
|
||||
Far better than `key home`, some `key down` and `key select`: it doesn't depend on where the Launcher's selection was.
|
||||
|
||||
## Look before you press
|
||||
|
||||
@@ -34,6 +46,7 @@ scripts/rdbg.py key select
|
||||
scripts/rdbg.py screenshot b.png # look again before the next destructive step
|
||||
```
|
||||
|
||||
- **Check the App in front before typing anything.** `info` prints `app: <name>`. A crash restarts the device into the Launcher, and a script that goes on typing is typing somewhere else: one of this project's own test scripts sent a word to an IRC channel that way.
|
||||
- Prefer **reading a state** to assuming it: `info`, `ls <folder>`, `cat <file>`, `irc dump`, `gnss status`, `lora status`, `wifi status`, `update status`.
|
||||
- Test on a **scratch folder** on the card, not on your real files.
|
||||
- For anything that deletes (`rm`, a delete dialog), `ls` first and `ls` after.
|
||||
@@ -59,12 +72,12 @@ Each of these puts something in, **without** the outside world:
|
||||
| `gnss send <sentence>` | Sends an NMEA sentence **to** the GNSS receiver (the checksum is added), to configure it |
|
||||
| `gemini get <url>` | Fetches a page and reports header, size, certificate and heap use, without the App |
|
||||
| `gemini trust <host> <port> <sha256>` | Pins a certificate by hand |
|
||||
| `sd fill <folder> <count>` | **Debug Build.** Makes that many small files in a folder, to test a crowded one (the Storage App shows the first 256) |
|
||||
| `sd fill <folder> <count>` | Makes that many small files in a folder, to test a crowded one (the Storage App shows the first 256) |
|
||||
| `wifi add <ssid><TAB><password>` | Adds a Saved Network, so credentials stay out of the repository |
|
||||
|
||||
## Test the update path
|
||||
|
||||
An update that goes wrong is the case you most want to rehearse, and a Debug Build can make it go wrong **on purpose**:
|
||||
An update that goes wrong is the case you most want to rehearse, and the firmware can make it go wrong **on purpose**:
|
||||
|
||||
```
|
||||
update status # what the device runs, what failed here before, the daily check, heap
|
||||
@@ -72,15 +85,14 @@ update check | update list # look at the server: the latest release, or
|
||||
update pretend v0.9.0 # take the running version to be v0.9.0: the latest release now counts as "new"
|
||||
update damage cut 50000 # the next download is cut after 50000 bytes
|
||||
update damage flip 100000 # ...or has the byte at offset 100000 damaged
|
||||
update install v0.11.0 force # try the install
|
||||
update install v0.11.0 # try the install
|
||||
update probe git.twis.la # is that server's certificate accepted? (the two ISRG roots only)
|
||||
update daily # forget today's daily check: it runs again at the next tick
|
||||
update pretend off # back to the real version
|
||||
```
|
||||
|
||||
- **`force` is needed on a Debug Build.** A plain `update install <tag>` answers `Debug Build: update from the PC`, because installing a release would replace the console with a build that has none. `force` is accepted only on a Debug Build, and only from the console.
|
||||
- **A damaged download must be refused cleanly:** the Update Service checks the signature after the first 160 bytes and the image hash at the end, so a cut or a flipped byte must end in a refusal with **nothing switched**. Check `info` afterwards: both slots, and `update: confirmed`.
|
||||
- **An undamaged `force` install really installs** the release, into the other slot. The Debug Build stays where it was until the next update overwrites it, and a Rollback returns to it, but think before you do it.
|
||||
- **An undamaged install really installs** the release, into the other slot, and the device restarts into it. Your build stays in the slot it was in until the next update overwrites it, and the console's setting and token are untouched: the release has the console too.
|
||||
- The server is read by the device itself, with Wi-Fi up; the download is one TLS connection, about 52 KB of heap at its peak, so IRC steps aside (see [the memory limit](/howto/not-enough-memory/)). `status: heap` in the stream shows it happen.
|
||||
|
||||
For crashes during Probation, see [Crashes and Safe Mode](/dev/debug/crashes/).
|
||||
@@ -94,7 +106,7 @@ wifi ip MyNet 10.39.39.50/24 10.39.39.1 try 60 # use this address for 60 s..
|
||||
wifi ip keep # ...and keep it, if you could still reach the device
|
||||
```
|
||||
|
||||
If you do not send `wifi ip keep` in time, the device goes back to the **previous** setting by itself, and the console comes back with it. (A Debug Build command: `try` is not in release builds.)
|
||||
If you do not send `wifi ip keep` in time, the device goes back to the **previous** setting by itself, and the console comes back with it.
|
||||
|
||||
## Measure
|
||||
|
||||
@@ -104,7 +116,7 @@ tasks # each task over the next second: state, priority, least free stack,
|
||||
net # bytes each network service has read and written since start
|
||||
```
|
||||
|
||||
`tasks` takes a second to answer. A low number in the `stack` column is a risk (the System App shows it in the warning colour under 512 bytes). `loop spin on|off` (Debug Build) makes the main loop spin without resting, to compare load and radio noise. And the `status: heap` line every 10 seconds in the stream is the cheapest memory trace there is: watch it while you do the thing you suspect.
|
||||
`tasks` takes a second to answer. A low number in the `stack` column is a risk (the System App shows it in the warning colour under 512 bytes). `loop spin on|off` makes the main loop spin without resting, to compare load and radio noise. And the `status: heap` line every 10 seconds in the stream is the cheapest memory trace there is: watch it while you do the thing you suspect.
|
||||
|
||||
```
|
||||
task st pri stack cpu% core
|
||||
@@ -120,4 +132,4 @@ The [System App](/guide/system/) shows the same, live, on the device.
|
||||
|
||||
## Radio experiments
|
||||
|
||||
`lora preset <name>` and `lora custom <MHz> <BW kHz> <SF> <CR> <sync hex> [preamble]` change what the receiver listens to (receive only: the radio never transmits), `lora sweep on [from] [to] [step]` takes a survey, and `lora noise test [gnss|quiet]` (Debug Build) runs a Sweep under one changed condition at a time, with Wi-Fi off for a moment, to find what raises the noise floor. `lora probe` finds the radio and reports its chip, oscillator, antenna switch, interrupt line and noise floor. Details in the [command reference](/dev/debug/commands/).
|
||||
`lora preset <name>` and `lora custom <MHz> <BW kHz> <SF> <CR> <sync hex> [preamble]` change what the receiver listens to (receive only: the radio never transmits), `lora sweep on [from] [to] [step]` takes a survey, and `lora noise test [gnss|quiet]` runs a Sweep under one changed condition at a time, with Wi-Fi off for a moment, to find what raises the noise floor. `lora probe` finds the radio and reports its chip, oscillator, antenna switch, interrupt line and noise floor. Details in the [command reference](/dev/debug/commands/).
|
||||
|
||||
@@ -8,6 +8,8 @@ tag = "Console"
|
||||
|
||||
These are the **binary commands**: a text header line, then raw bytes. The console task answers them itself, so they keep working when the main loop is stuck. `scripts/rdbg.py` handles each one on the PC side; the protocol is given too, for your own tools.
|
||||
|
||||
**Without a PC,** the device does both by itself now: <kbd>Fn</kbd> + <kbd>p</kbd> saves a screenshot to the card ([how-to](/howto/screenshot/)), and <kbd>w</kbd> in the Storage App serves the card to a browser ([how-to](/howto/phone-files/)). What follows is the scripted way, with checksums.
|
||||
|
||||
## `get`: card to PC
|
||||
|
||||
```sh
|
||||
@@ -45,6 +47,7 @@ The device sends `screenshot: rgb332 <width> <height>` and then **one byte per p
|
||||
|
||||
- It is read **as it stands**, while the UI may be drawing, so it can **tear**. It is for looking at, not for pixel-exact comparison.
|
||||
- It is the real thing: the screenshots on this site, in the [user guide](/guide/) and the [devlog](/devlog/), were taken this way.
|
||||
- **With a number, it saves to the card instead:** `screenshot 5` (or `screenshot 0`) writes a PNG to `/screenshots` on the SD card after that many seconds, as the [Shell](/guide/shell/) does. A bare `screenshot` over the console is the binary one above.
|
||||
- Its main use is in a loop: send a key, wait a moment, take a screenshot, look. See [Drive the UI](/dev/debug/drive-the-ui/).
|
||||
|
||||
## `coredump get`: the crash dump
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
+++
|
||||
title = "Switch the console on"
|
||||
description = "The Debug Console is in every firmware, and off. How to switch it on, where its token comes from, and how to set a device up without typing anything."
|
||||
weight = 1
|
||||
[extra]
|
||||
tag = "Start here"
|
||||
+++
|
||||
|
||||
## One firmware
|
||||
|
||||
There is no special build. **Every roro9stack firmware has the Debug Console**, the same one, and the commands made for testing (crash on purpose, fake an installed version, damage a download, inject a LoRa packet). It is **off** until its owner switches it on.
|
||||
|
||||
**Off means nothing is there:** no socket listens, the console's task does not exist, and neither does its 4 KB buffer. A device that never uses it pays 30 KB of flash and 88 bytes of memory.
|
||||
|
||||
Before version 0.12 this was a separate *Debug Build* with a token compiled in from the builder's machine, which is why it could not be published. [ADR 0010](/dev/decisions/0010-debug-console-in-every-build/) says why that changed.
|
||||
|
||||
## On the device
|
||||
|
||||
**Settings → Debug Console:**
|
||||
|
||||
| Row | Does |
|
||||
|---|---|
|
||||
| **Debug Console** | The switch. Switching it **on** asks first (the question opens on *Cancel*: move to *Switch on*), and makes a token if there is none |
|
||||
| **Connect to** | The address and port: `10.39.39.12:2323` |
|
||||
| **New token** | Makes another one. The old one stops working, and whoever is connected is cut off |
|
||||
| **Type a token** | One of your own, of 16 to 64 characters |
|
||||
|
||||
Under the rows, the **token**, in large type on two lines, in groups of four: `K7QF-3M2X-9WBD-HT4P-6RNC`. This page is the only place it is ever shown. The Status Bar shows **`DBG`** while the console listens, and brighter while someone is connected.
|
||||
|
||||
The setting **stays** across restarts and updates, and in Safe Mode.
|
||||
|
||||
## The token
|
||||
|
||||
- **The device makes it,** from its hardware random generator, the first time the console is switched on: 100 bits, written as 20 characters without the letters that get misread (no I, L, O or U).
|
||||
- **Dashes and case do not count,** and an `O`, `I` or `L` is taken for the `0` or `1` it was. Type it as you read it.
|
||||
- **One you type** must have at least 16 characters. A short one would be the weakest part of the whole thing.
|
||||
- It is **never printed** on a console, and it **never crosses the network** ([the Debug Console](/dev/debug/console/) says how).
|
||||
- It guards against **the network**, not against someone who holds the device: anyone with USB access can flash anything anyway ([ADR 0003](/dev/decisions/0003-own-signature-check-not-secure-boot/)).
|
||||
|
||||
## Give `rdbg.py` the token
|
||||
|
||||
`scripts/rdbg.py` looks for it in this order:
|
||||
|
||||
```sh
|
||||
scripts/rdbg.py -t K7QF-3M2X-9WBD-HT4P-6RNC info # 1. on the command line (also --token)
|
||||
RORO_DEBUG_TOKEN=K7QF-3M2X-9WBD-HT4P-6RNC scripts/rdbg.py info # 2. in the environment
|
||||
echo K7QF-3M2X-9WBD-HT4P-6RNC > ~/.config/roro9stack/debug-token # 3. in a file, for the device you use every day
|
||||
```
|
||||
|
||||
## Without typing: over USB
|
||||
|
||||
With the device on a cable, the console can be set up from the PC:
|
||||
|
||||
```sh
|
||||
scripts/flash.sh --debug # flashes over USB, then switches the console on and gives it your token
|
||||
```
|
||||
|
||||
It sends two commands over the serial port, which you can also type there yourself:
|
||||
|
||||
```
|
||||
debug on # switch it on (a token is made if there is none)
|
||||
debug token <value> # give it this token: 16 to 64 characters
|
||||
debug token new # make a new one
|
||||
debug status # on or off, token set or not, a client or not (the token itself is never shown)
|
||||
debug off # switch it off
|
||||
debug off 30 # ...for 30 seconds: it comes back by itself
|
||||
```
|
||||
|
||||
`debug on` and `debug token` work **over USB serial only**: the console cannot be used to open itself wider. `debug status` and `debug off` work from anywhere, and a screenshot taken over the console while this page is open shows the token, to someone who already had it. Your token file is made by the first build (`scripts/_docker.sh`), 32 hex digits, and is never committed.
|
||||
|
||||
Since the setting survives updates, this is needed **once for a device**, not at each flash. Later builds go over Wi-Fi with `scripts/flash.sh --ota`.
|
||||
|
||||
## Should it be on?
|
||||
|
||||
On your own network, on a device you are working on: yes, that is what it is for. Remember what it gives to whoever has the token **and** is on the same network: the console, the keys, the files on the SD card, a restart. It does not give them the firmware: an update still has to be signed.
|
||||
|
||||
On a network you share with strangers, switch it off, or at least know that what the console prints is not encrypted. The token is safe there; the conversation is not.
|
||||
@@ -1,6 +1,6 @@
|
||||
+++
|
||||
title = "A Debug Console over Wi-Fi, in Debug Builds only"
|
||||
description = "The goal of Firmware Updates is to manage the device without a cable, and that includes finding out what went wrong. So a Debug Build (cardputer-adv-debug, -DRORO_DEBUG, version suffix +debug) adds a Debug Console on TCP 2323…"
|
||||
description = "Superseded in part by ADR 0010 (2026-10-06): there is no Debug Build any more. The console is in every firmware, off until switched on, and its token belongs to the device, not to the build. What this record says about how the…"
|
||||
weight = 4
|
||||
|
||||
[extra]
|
||||
@@ -8,6 +8,8 @@ docs = true
|
||||
source = "docs/adr/0004-debug-console-in-debug-builds.md"
|
||||
tag = "ADR 0004"
|
||||
+++
|
||||
**Superseded in part by [ADR 0010](/dev/decisions/0010-debug-console-in-every-build/) (2026-10-06):** there is no Debug Build any more. The console is in every firmware, off until switched on, and its token belongs to the device, not to the build. What this record says about how the console works (the ring, commands on the main loop, binary commands on its own task, one client) still holds.
|
||||
|
||||
The goal of Firmware Updates is to manage the device without a cable, and that includes finding out what went wrong. So a **Debug Build** (`cardputer-adv-debug`, `-DRORO_DEBUG`, version suffix `+debug`) adds a **Debug Console** on TCP 2323: the serial console, over Wi-Fi. A client sends a token as its first line, then gets the last 4 KB of console output (boot messages included), every new line live, and runs the same commands as the serial port, plus a few that only make sense remotely. ESP-IDF's own log lines are teed into it.
|
||||
|
||||
It's compiled out of release builds entirely, rather than switched off by a setting. A console that runs commands is a remote control: in a release build, nothing listens.
|
||||
|
||||
@@ -8,10 +8,10 @@ docs = true
|
||||
source = "docs/adr/0005-safe-mode-crash-reports-watchdog.md"
|
||||
tag = "ADR 0005"
|
||||
+++
|
||||
Rollback protects against new firmware that fails Probation. It does nothing for firmware that was confirmed and crashes later: a corrupt setting, a server that sends something unexpected, a bug that takes an hour to show. Without a cable, such a device would restart forever. Three measures, in release and Debug Builds alike, keep it reachable:
|
||||
Rollback protects against new firmware that fails Probation. It does nothing for firmware that was confirmed and crashes later: a corrupt setting, a server that sends something unexpected, a bug that takes an hour to show. Without a cable, such a device would restart forever. Three measures, in every build, keep it reachable:
|
||||
|
||||
- **Safe Mode.** The firmware counts starts that follow a crash (panic or watchdog) in NVS, first thing at boot. After 3 in a row, it starts only the clock, Wi-Fi, the Update Service and, in a Debug Build, the Debug Console: no Apps, no IRC, no SD card, and a screen that says so with the address to push an update to. Any normal restart (a `reboot`, an update), or a minute of uptime, resets the count.
|
||||
- **Crash reports.** The same boot record keeps which version was running, so after a crash the firmware knows which one crashed, even when a Rollback has switched slots since. ESP-IDF already writes a core dump to its flash partition on a panic; after the restart the firmware prints its summary (task, PC, reason, backtrace) and raises a Notification. The `crash` command shows it again later. In a Debug Build, `scripts/rdbg.py crash` decodes the backtrace and `scripts/rdbg.py coredump` fetches the whole dump for `esp-coredump`, against the ELF of that exact build (`.pio/elves/`, named by version and ELF digest).
|
||||
- **Safe Mode.** The firmware counts starts that follow a crash (panic or watchdog) in NVS, first thing at boot. After 3 in a row, it starts only the clock, Wi-Fi, the Update Service and, if it's switched on, the Debug Console: no Apps, no IRC, no SD card, and a screen that says so with the address to push an update to. Any normal restart (a `reboot`, an update), or a minute of uptime, resets the count.
|
||||
- **Crash reports.** The same boot record keeps which version was running, so after a crash the firmware knows which one crashed, even when a Rollback has switched slots since. ESP-IDF already writes a core dump to its flash partition on a panic; after the restart the firmware prints its summary (task, PC, reason, backtrace) and raises a Notification. The `crash` command shows it again later. With the Debug Console, `scripts/rdbg.py crash` decodes the backtrace and `scripts/rdbg.py coredump` fetches the whole dump for `esp-coredump`, against the ELF of that exact build (`.pio/elves/`, named by version and ELF digest).
|
||||
- **The main loop is watched.** Arduino-ESP32 subscribes only core 0's idle task to the task watchdog, and the main loop runs on core 1: a stuck loop used to hang the device for good, with the screen frozen and the Debug Console unable to run commands. `enableLoopWDT()` makes a loop stuck for 5 s a panic, with a core dump, counted towards Safe Mode. And an installed update no longer depends on the main loop: the Update Service restarts into it by itself after 90 s.
|
||||
|
||||
## Consequences
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
+++
|
||||
title = "The Debug Console is in every build, off until its owner switches it on"
|
||||
description = "There is one firmware. The Debug Console of ADR 0004 is compiled into it, and so are the commands that used to exist only in Debug Builds. It listens only while Settings → Debug Console is on, which is not the default, and it…"
|
||||
weight = 10
|
||||
|
||||
[extra]
|
||||
docs = true
|
||||
source = "docs/adr/0010-debug-console-in-every-build.md"
|
||||
tag = "ADR 0010"
|
||||
+++
|
||||
There is **one firmware**. The Debug Console of ADR 0004 is compiled into it, and so are the commands that used to exist only in Debug Builds. It listens only while **Settings → Debug Console** is on, which is not the default, and it lets in whoever proves they hold **the device's own token**. The `cardputer-adv-debug` environment, `RORO_DEBUG`, the `+debug` version and the token compiled in from the builder's machine are gone.
|
||||
|
||||
ADR 0004 kept the console out of release builds with one argument: a console that runs commands is a remote control, so in a release nothing should listen. That argument was never whole: the Update Service has listened on TCP 3232 in every build since Firmware Updates exist, guarded by a signature. A listener that is off by default and guarded by a secret the device made itself is the same kind of trade, and what the split cost had grown:
|
||||
|
||||
- **What was tested wasn't what shipped.** Development ran on Debug Builds; releases were a different binary, built to be published and never run by the person who wrote them.
|
||||
- **Debug Builds couldn't be published**, since each carried its builder's token. So the console, `rdbg.py`, screenshots and crash decoding were for whoever built the firmware, and nobody else.
|
||||
- **Rules that existed only to protect the console:** a Debug Build never installed a release (it would have lost the console), `update install … force` to do it anyway, "keep a Debug Build in the fallback slot", versions with `+debug` that had to compare equal to their release.
|
||||
- **CI built two firmwares** on every pull request and every tag.
|
||||
|
||||
## How it works
|
||||
|
||||
- **Off means nothing is there.** With the setting off, no socket listens, the console's task doesn't exist and neither does its 4 KB ring: a device that never uses the console pays for it in flash (30 KB more than the release build was) and 88 bytes of static RAM, measured. A missing or invalid stored setting counts as off.
|
||||
- **The token is the device's.** The first time the console is switched on, the device makes one from its hardware random generator: 100 bits, as 20 characters of Crockford's base32 (`K7QF-3M2X-9WBD-HT4P-6RNC`), shown on the Debug Console page and nowhere else. It can be replaced by one typed by hand, of 16 to 64 characters. It is stored with the other settings and is never printed on a console.
|
||||
- **The token never crosses the network.** The device sends 16 random bytes; the client answers with their HMAC-SHA256 keyed by the token. Someone on the same Wi-Fi who records a login has nothing they can use for the next one.
|
||||
- **Five wrong answers in a row** close the console to everyone for a minute, with a Notification naming the address they came from. A wrong answer still costs a second.
|
||||
- **`DBG` in the Status Bar** while the console listens, bright while a client is connected.
|
||||
- **USB serial can set it up:** `debug on`, `debug token <value>`, `debug token new`. `scripts/flash.sh --debug` uses them to give a freshly flashed device the developer's token. Whoever holds the cable can flash anything anyway (ADR 0003); the console itself can only switch itself off.
|
||||
- **Safe Mode** starts the console if it's switched on: the settings are read before Safe Mode is decided.
|
||||
|
||||
## Consequences
|
||||
|
||||
- **"Nothing listens" now rests on one stored setting**, not on absent code. That setting is typed, validated and host-tested, and its default is off; a bug that flipped it would have to come from code that already runs on the device.
|
||||
- **Whoever has the token and the network has the device:** the console, its keys, the SD card's files, a restart. Not its firmware: an update still has to be signed (ADR 0003).
|
||||
- **The stream is still plain text.** The token is safe from an eavesdropper; what the console prints, and the files it carries, are not. TLS would cost about 52 KB of the 107 KB there is, next to IRC's own connection: not for a console.
|
||||
- **An old `rdbg.py` can't talk to a new firmware**, and the reverse: the first line of the protocol changed. Accepted, with one user.
|
||||
- **A device whose settings are damaged has no console in Safe Mode**, only the signed update push. Before, a Debug Build always had it.
|
||||
- **The commands that crash, damage a download or fill a folder on purpose** are in every build. They need the cable or the token, like everything else.
|
||||
- **Releases already publish their ELF**, so `rdbg.py crash` fetches it when it isn't in `.pio/elves/`: crash reports can be decoded without having built the firmware.
|
||||
@@ -8,7 +8,7 @@ docs = true
|
||||
source = "docs/milestones/F1.md"
|
||||
tag = "F1"
|
||||
+++
|
||||
**Status:** in progress. The Storage App (issue #3) shipped as **v0.9.0** on 2026-10-06. Notes (#19) shipped as **v0.10.0** the same day. The card as a USB drive (#1) comes after.
|
||||
**Status:** in progress. Shipped: the Storage App (issue #3, **v0.9.0**), Notes (#19, **v0.10.0**), notes of any size (#47, **v0.15.0**), pictures in the Storage App (#45, **v0.16.0**), sharing the card with a browser (#88, **v0.18.0**). Not started: the card as a USB drive (#1), selecting several items (#41), finding files by name (#42), opening a `.gmi` in Gemini (#43), a table view for `.csv` (#44), search, undo and copy-paste in Notes (#48, #49, #50).
|
||||
|
||||
**Goal:** get at what's on the SD card from the device itself: browse it, look inside the files the firmware writes, copy, move, rename and delete, and keep notes. A side milestone, like G1 and S1; Files and Notes were M3's original second half (Q30, Q89).
|
||||
|
||||
@@ -106,7 +106,7 @@ Plain text notes on the SD card, written on the device. Q30 settled the base: `.
|
||||
| Q141 | A **Notes** App in the Launcher. One row per note: its first line as the title, then the date. Newest first; `s` switches to by name. `n` new, Enter opens, `d` deletes after a confirmation, `r` renames the file. |
|
||||
| Q142 | A new note's file name is never typed: it comes from the first line when the note is first saved (`shopping-list.txt`), or `note-20261006-0919.txt` if that line is empty. It doesn't change afterwards unless the note is renamed. |
|
||||
| Q143 | **Autosave, no "discard changes?" prompt:** five seconds after the last key, on leaving the note or the App, and when the screen turns off. A save writes a temporary file and renames it over the note, so a power cut loses the last few seconds at most. A temporary file left behind is offered back at the next open. |
|
||||
| Q144 | The whole note is in memory while it's edited, up to **16 KB**. A bigger text file opens read-only in the Storage App's viewer. The App refuses to open below the memory floors (Q86). **Editing files of any size must come in a later release: issue #47.** |
|
||||
| Q144 | The whole note is in memory while it's edited, up to **16 KB**. A bigger text file opens read-only in the Storage App's viewer. The App refuses to open below the memory floors (Q86). *(Lifted by issue #47: see "Notes of any size" below.)* **Editing files of any size must come in a later release: issue #47.** |
|
||||
| Q145 | The editor wraps at spaces, 38 columns by 8 rows, with a line for the name and the state. Enter is a new line, Del deletes backwards, Fn+arrows move (the Text Entry rule), Ctrl+A and Ctrl+E go to the start and the end of the line, Tab types two spaces, Back saves and returns. The Compose Key works as elsewhere. |
|
||||
| Q146 | The Storage App's text viewer gets `e`: edit this file with the same editor, for a text file up to 16 KB that isn't read-only. That lifts Q135 without Apps opening each other (#43 stays). |
|
||||
| Q147 | The list is flat: the files directly in `/notes`. Sub-folders are reached through the Storage App. |
|
||||
@@ -167,3 +167,177 @@ Test notes were made in `/notes` and removed afterwards; the folder is left, emp
|
||||
**Not checked:** accents through the Compose Key and Ctrl+A / Ctrl+E (the remote `key` command can't send them; the model's tests cover both), the power button's save (it needs a hand on the device), a missing card, and how typing feels on the keyboard itself.
|
||||
|
||||
**One slip during the checks:** a key sequence sent right after a restart opened IRC instead of Notes, and the test letters went into IRC's input line. Nothing was sent: the line was cleared and the App left. IRC connected to Libera as it does when opened.
|
||||
|
||||
## Notes of any size (issue #47)
|
||||
|
||||
Q144 held the whole note in memory and stopped at 16 KB, for the first version only. This lifts it: the editor opens a text file whatever its size.
|
||||
|
||||
### Decisions (design round 2026-10-07)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q223 | **The note is the file on the card plus one window in memory.** The window is the `NoteText` of before, up to 16 KB around the cursor; the rest is a list of pieces: runs of the file, and runs of a side file. The cursor leaving the window writes it to the side file if it was changed, and loads the next. Typing never fills a note: a full window is written away and loaded smaller. |
|
||||
| Q224 | **The five-second save:** up to 64 KB it rewrites the file, as before (about 150 ms). Above, it appends the window and the list of pieces to `<note>.edit`: 8 KB or so, whatever the note's size. "saved" means "on the card" either way. |
|
||||
| Q225 | **The file itself is rewritten on leaving the note** (Back, Home, another App), with a progress bar. The screen turning off and the device powering off write the side file only: powering off never waits. |
|
||||
| Q226 | **After a power cut, opening the note picks the edit up** where it was last saved, without a question, and says so. Until then the file has the old text for anything else that reads it. |
|
||||
| Q227 | If the file was changed elsewhere meanwhile, the side file no longer fits it: it is **kept as `<note>.edit.lost`** and the editor says so. Typed text is never deleted without a word. |
|
||||
| Q228 | **No limit but the card:** a note over 16 KB needs room for a second copy to be opened for editing. No warning for a big file; the progress bar on leaving tells the cost. |
|
||||
| Q229 | A side file over 1 MB, or a list of over 256 pieces, makes the next save a rewrite. |
|
||||
| Q230 | **CRLF becomes LF** (Q148) for a long file too: in the window as it is read, and in the rest of the file as the rewrite streams it, so a saved file is never of both kinds. |
|
||||
| Q231 | **One path.** A 16 KB note is the case with no pieces: there is no second editor for small notes. |
|
||||
| Q232 | Notes, and `e` in the Storage App's viewer, which no longer says "Too big to edit". |
|
||||
|
||||
### As built
|
||||
|
||||
- **`NoteDocument`** (`lib/notes/src/note_document.h`, host-tested against a card in memory) is the list of pieces, the window's moves, the side file and the recovery. `NoteText` is unchanged but for being refilled.
|
||||
- **The window moves** when the cursor comes within 2 KB of an end of it that isn't an end of the note: it is then 4 KB on each side of the cursor. It starts where a line starts on screen whenever that can be known (after a newline, or where the window before had a line start), so the same text wraps the same from one window to the next, and never in the middle of a character. The cursor keeps its row on screen.
|
||||
- **Looking writes nothing:** a window that wasn't changed goes back as the pieces it was read from.
|
||||
- **The side file** starts with a line of text, the note's size and checksums of its first and last kilobyte, which is how a file changed elsewhere is told. After that, text that left a window, and snapshots of the list of pieces, each with its checksum. The newest snapshot that checks out is the note as last saved; anything after it is ignored.
|
||||
- **The rewrite** streams the pieces and the window into `<note>.tmp`, checks its size, then writes a mark at the end of the side file: from that mark on, the rewrite counts as done, and opening the note finishes it whatever was cut (remove the old file, rename, remove the side file). Before the mark, the note and its side file are still the truth and the temporary file is dropped.
|
||||
- **On the device** the card is reached through an adapter that keeps the file being read and the file being appended to open between calls; every call runs on the storage task while the main loop waits. The rewrite runs in steps of 64 KB with the progress drawn between them.
|
||||
- **The Notes list** doesn't show `.edit` and `.edit.lost` files, and a note's side file is deleted and renamed with it.
|
||||
- **Ctrl with Fn+Up and Fn+Down** go to the start and the end of the note.
|
||||
- **`key ctrl-down`**: the consoles' `key` command takes `ctrl-`, `alt-` and `shift-`, which these checks needed. It also lets the checks S1 couldn't make (Ctrl+b, the Alt scroll) be made.
|
||||
- **Cost:** 15 KB of flash. Memory with a note open is what it was: 17.5 KB, for 62 bytes or for 1.2 MB.
|
||||
|
||||
### Host tests (15, `test/test_note_document`)
|
||||
|
||||
A walk down 3,000 lines and back up through the windows; start and end; an edit in the middle rewritten into the file; 48 KB typed into a new note; a journal picked up after a cut; **a cut at every 997th byte of a sequence of two saves and a rewrite**, after which the note is always one of the three texts it should be, what was reported saved is there, and no stray file is left; a file changed elsewhere; CRLF; windows on text with no space and no newline, made of 2, 3 and 4-byte characters; a full card; and **36,000 random keys** (typing, deleting, moving, jumping, saving, power cuts) on six notes of 30 to 130 KB, compared with a plain string after every key.
|
||||
|
||||
### Checks on the device (2026-10-07, driven over the Debug Console)
|
||||
|
||||
Test notes were copied to `/notes` and removed afterwards; the note that was already there was not touched.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| A 36 KB note | Opens (it was refused before). Two letters at the top, 400 lines down across the windows, four more: the file fetched back is exactly that, and no other file is left |
|
||||
| A 1.2 MB note | Opens at once. Free memory 104.2 KB before, 86.7 KB with it open |
|
||||
| Its five-second save | `zz-big.txt.edit`, 4 KB; the note's file untouched |
|
||||
| Ctrl with Down, Ctrl with Up | The end and the start, as fast as any key |
|
||||
| A restart with unsaved keys | "Your unsaved changes are back", the cursor where it was, the unsaved keys gone and nothing else |
|
||||
| Leaving it | The progress bar, then one file: **1.2 MB rewritten in 2.6 s**. Fetched back: the original with what was typed at both ends, byte for byte |
|
||||
| A restart in the middle of that rewrite | The note, its side file and an empty `.tmp` remain; opening picks the edit up, leaving rewrites it, the result is right |
|
||||
| A new note | No file until typed in, then `zz-test-note.txt` from its first line |
|
||||
| `e` in the Storage App on the 1.2 MB file | The same editor; edited and rewritten |
|
||||
| The Notes list | Side files are not listed as notes |
|
||||
|
||||
**Not checked:** the power button's path (side file only), the screen turning off, a card pulled while editing, and memory with IRC connected, which wasn't connected for these checks: the editor's own use hasn't changed, and it still refuses to open without a free block of 24 KB. The real keyboard's Ctrl with Fn and the arrows. A file of tens of megabytes. Renaming or deleting a note from the Storage App leaves its side file behind.
|
||||
|
||||
**Measured against what was said:** the first build rewrote 1.2 MB in 3.5 to 4.5 s, with 2 KB blocks. With 4 KB blocks it is 2.6 s, about 450 KB a second, which is what the card gives a plain copy.
|
||||
|
||||
## Pictures in the Storage App (issue #45)
|
||||
|
||||
Q139 left images out: the firmware wrote none. Since the Shell's `screenshot` it does.
|
||||
|
||||
### Decisions (design round 2026-10-07)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q233 | **PNG, JPEG, BMP and GIF.** A GIF shows its first picture; it doesn't move. |
|
||||
| Q234 | *Revised while building.* **Our own PNG decoder**, a row at a time, with the window the file's compression asks for: 32 KB at most. The plan was the display library's, which takes 44 KB in one block: after one picture the largest free block was 43 to 47 KB, and every second PNG was refused. **The firmware's own screenshots** are read with no decoding at all: they are stored uncompressed, each byte already a colour of the screen. |
|
||||
| Q235 | **The picture is decoded once, straight into the screen's buffer, and left there.** No copy in memory (it would be up to 30 KB). `App::retainsContent()` tells the screen not to clear the App's part; `contentLost()` tells the App that it was cleared after all, or that a Toast or the help panel drawn over it has gone: then it is decoded again. |
|
||||
| Q236 | **Shrunk to fit; Enter shows it at its own size**, the arrows then moving half a screen. A picture smaller than the screen sits in the middle at its size. |
|
||||
| Q237 | **Ordered dithering** to the screen's 256 colours (a 4 x 4 pattern). A colour the screen has exactly comes out as itself wherever it lands, so a screenshot isn't touched. |
|
||||
| Q238 | Shrinking takes, for each pixel of the screen, the first of the picture's that falls on it. No averaging: there is nowhere to keep the sums. Thin lines break up; a JPEG looks better, its decoder halving it up to three times first. |
|
||||
| Q239 | **What can't be shown opens as hex, with the reason:** a progressive JPEG, an interlaced PNG, a BMP that is compressed or has 16 bits. The rotation a camera stores in the file is ignored. |
|
||||
| Q240 | *Revised while building.* **Decoding runs on the storage task while the main loop goes on.** The picture appears as it comes, and anything that needs the screen back stops the decoding first. The plan was to wait for it, behind a "Decoding..." line: 12 megapixels took longer than the watchdog allows the main loop to stand still, and the device restarted. |
|
||||
| Q241 | The Storage App: Enter on `.png`, `.jpg`, `.jpeg`, `.bmp`, `.gif`, or on a file with no known extension whose first bytes say what it is. Tab gives the hex. `i`, and opening, show the size in pixels on the last line for three seconds. |
|
||||
| Q242 | Not in this one: animation, opening a picture from the Gemini App, a slideshow. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`lib/files/src/image_file.h`** (host-tested): what a file is and how big, where each pixel lands (`ImageFrame`, `ImageMap`), the dithering, and readers for BMP and GIF that hand their pixels on as they get them. **`png_reader.h`**: the PNG decoder, with its own inflate: every colour type and bit depth, palettes with transparency. Transparent pixels are left as the background.
|
||||
- **`ImagePane`** (`src/apps/image_pane`) is the view. One decoding is a `Job` shared with the storage task; `cancel()` flags it and waits behind it in the task's queue, which is how the screen is known to be free again.
|
||||
- **JPEG is the one decoder that isn't ours:** the display library's TJpgDec, with its 3.9 KB pool. It shrinks by 2, 4 or 8 while decoding, which is why a photograph is possible at all.
|
||||
- **A decoder stops early** once the rest of the file is below the screen (a picture at its own size), and a BMP's rows that aren't shown aren't read.
|
||||
- **The note** on the last line is written over the picture; the strip under it (2.6 KB) is kept and put back, so showing it costs no decoding.
|
||||
- **A BMP is read in the order its rows are stored**, last row first: reading it top to bottom meant going back through the file for every row, a second for 135 rows.
|
||||
- **Cost:** 21 KB of flash. Nothing while no picture is shown.
|
||||
|
||||
### Measured on the device
|
||||
|
||||
| Picture | Fitted | Its own size |
|
||||
|---|---|---|
|
||||
| A screenshot of ours, 240 x 135 | 80 ms | 78 ms |
|
||||
| PNG, 800 x 600 | 855 ms | 575 ms |
|
||||
| PNG with transparency, 800 x 600 | 1,098 ms | |
|
||||
| JPEG, 800 x 600 | 305 ms | |
|
||||
| JPEG, 4000 x 3000 (2.6 MB) | 6.9 s | 7.7 s (the middle of it) |
|
||||
| GIF, 800 x 600 | 642 ms | |
|
||||
| BMP, 800 x 600 (1.4 MB) | 991 ms | |
|
||||
| BMP, 240 x 135 | 124 ms | |
|
||||
|
||||
Free memory fell to 48.8 KB at the lowest while a PNG was decoded, from 104 KB. The storage task's stack: 3.2 KB never used, of 6.
|
||||
|
||||
### Host tests (20, `test/test_image_file` and `test/test_png_reader`)
|
||||
|
||||
The pictures in them were made with Pillow, so the readers are checked against an encoder that isn't ours: GIFs plain, interlaced, transparent and long enough for the codes to reach 12 bits; BMPs of 8 and 24 bits; PNGs of every kind (colour, with alpha, palette of 8 and 4 bits with a transparent entry, greys of 1, 8 and 16 bits, grey with alpha, not compressed, and one that refers 21,600 bytes back). Every reader is also fed its file with a byte changed, at every few bytes: an answer each time, and no pixel outside the picture.
|
||||
|
||||
### Checks on the device (2026-10-07, driven over the Debug Console)
|
||||
|
||||
Test pictures were copied to a scratch folder and removed afterwards, with the test screenshot.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Colour bars as PNG, JPEG, BMP and GIF, 240 x 135 and 800 x 600 | The same picture each time, the colours in the right order |
|
||||
| A PNG with a transparent square | The square is the background |
|
||||
| A screenshot taken in the Shell | Shown; at its own size it is the screen, pixel for pixel |
|
||||
| A progressive JPEG | Hex, with "A progressive JPEG can't be shown" |
|
||||
| An animated GIF | Its first picture |
|
||||
| 12 megapixels | Arrives from the top down in 6.9 s; the size is noted when it is whole |
|
||||
| Enter, then the arrows | Its own size from the middle, then half a screen at a time |
|
||||
| Back in the middle of a decoding | The folder's listing at once |
|
||||
| Tab to hex and back, three times; the help panel, then closed | The picture again each time |
|
||||
|
||||
**Not checked:** a photograph from a real camera (the test JPEGs were made by Pillow); a Toast over a picture; a PNG with IRC connected, when there may not be the memory; the card pulled while decoding; the real keyboard.
|
||||
|
||||
### What went wrong while building it
|
||||
|
||||
**The watchdog.** The first version waited for the decoder. The main loop is watched: five seconds without a pass and the device restarts, which is what it did on the 12-megapixel test. The crash report named the decoder's line. The decoding moved to the background, which also made the picture appear as it comes.
|
||||
|
||||
**A decoder that worked once.** The library's PNG decoder showed the first picture and refused the next five: "no memory". It wants 44 KB in one piece, and after some use the largest piece is 43 to 47 KB. Writing a decoder that needs 32 KB was less work than it sounds, and unlike the library's it has tests.
|
||||
|
||||
**Two sentences still said "up to 16 KB"** about editing, in the README and the Storage guide, after issue #47 lifted that. Corrected here.
|
||||
|
||||
## Sharing the card with a browser (issue #88)
|
||||
|
||||
Files reached the card through the Debug Console's `put` and `get`, or by taking the card out. A phone has neither.
|
||||
|
||||
### Decisions (2026-10-07; the recommendation was accepted as it stood, without a round of questions)
|
||||
|
||||
- **A web page, not FTP, SFTP or WebDAV.** A browser is the only client every phone has. FTP and WebDAV need an app there; SFTP needs a whole SSH server here. WebDAV can come later on the same server, for computers.
|
||||
- **Off unless asked for:** `w` in the Storage App opens a "Share" screen, and the server runs only while that screen is open.
|
||||
- **A six-digit code on the screen**, new each time, typed in the page; the address is also a QR code, which carries the code. Five wrong codes close it for a minute (the Debug Console's `AuthGate`). A browser that got it right holds a cookie; starting again puts every browser out.
|
||||
- **Not encrypted.** A TLS server costs about 40 KB of memory a connection. The screen says so.
|
||||
- **The Storage App's rules** (`whyReadOnly`): the firmware's own folders, and files in use.
|
||||
|
||||
### As built
|
||||
|
||||
- **`WebShare`** (`src/services/web_share`): ESP-IDF's HTTP server, which is in the framework already. Seven requests: the page, the code, a listing as JSON, a download, an upload, a new folder, a delete.
|
||||
- **Every access to the card is handed to the storage task**, 8 KB at a time, from the server's own task: a download and an upload are loops of "one piece from the card, one piece to the network".
|
||||
- **An upload is the request's body**, as the browser's `PUT` sends it: no form to take apart. It goes to `<name>.part` and is renamed when the last byte has come; anything less is removed. A file that exists is refused unless the page asked, after asking the user.
|
||||
- **The page** (`web_share_page.h`) is one file of 5.4 KB with its style and script in it, served from flash.
|
||||
- **`lib/files/src/share_rules.h`** (host-tested, 5 tests): what a request names, which paths a browser may ask for (from the root, no `..`), the JSON, the code and the cookie.
|
||||
- **Cost:** 57 KB of flash, most of it the server. 13 KB of memory while sharing (108.4 KB free before, 95.4 with the screen open), given back on leaving.
|
||||
|
||||
### Checks on the device (2026-10-07 and 08)
|
||||
|
||||
A scratch folder was used and removed.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `w` | The QR code, the address and the code; `share: on` on the console |
|
||||
| The page, and a listing without the code | 200; 401 |
|
||||
| A wrong code, the right one (typed `825 132`) | 403; in |
|
||||
| Upload, 2.6 MB | 11 to 17 s (150 to 230 KB/s); downloaded again and compared: the same, byte for byte |
|
||||
| The same name again; with "replace" | 409; replaced |
|
||||
| `..` in a path; deleting `/notes`; deleting a folder that isn't empty | 400; 403 "The firmware keeps its files in /notes"; 403 |
|
||||
| Five wrong codes | The fifth and every one after: 429, the right code too. A browser that was in stays in |
|
||||
| In Chromium at a phone's width | The scanned address logs in by itself; two files uploaded, one downloaded and compared, a folder made, a file deleted after asking, a replacement after asking, the refusal shown. No sideways scroll |
|
||||
| Back | The server is gone (connection refused), memory is back |
|
||||
|
||||
**Found on the way:** the server answers one request at a time. A second request during a slow download waited until it had ended. It is said in the guide, and not changed.
|
||||
|
||||
**On a real phone** (the maintainer's, 2026-10-08): the QR code, scanned with the phone's camera, opens the page and logs in; the page lists the card, in the phone's dark theme.
|
||||
|
||||
**Not checked:** Safari. A card pulled during a transfer. Sharing with IRC connected, when memory is shorter. Home, and the screen turning off, while sharing (the code stops the server on leaving the App; only Back was tried).
|
||||
|
||||
@@ -0,0 +1,230 @@
|
||||
+++
|
||||
title = "Network tools"
|
||||
description = "Reach things from the device that aren't on the Wi-Fi it happens to be on, and keep its traffic private on a network that isn't yours."
|
||||
weight = 100
|
||||
|
||||
[extra]
|
||||
docs = true
|
||||
source = "docs/milestones/N1.md"
|
||||
tag = "N1"
|
||||
+++
|
||||
**Status:** in progress. The WireGuard tunnel (issue #8) shipped as **v0.19.0**. The network troubleshooting commands (issue #90) shipped in two parts: `ping`, `nslookup`, `port`, `traceroute`, `ifconfig` and `arp` as **v0.20.0**; `tls`, `ntp` and `netstat` as **v0.21.0**. The SSH client (issue #2) shipped as **v0.22.0**.
|
||||
|
||||
**Goal:** reach things from the device that aren't on the Wi-Fi it happens to be on, and keep its traffic private on a network that isn't yours.
|
||||
|
||||
## The WireGuard tunnel (issue #8)
|
||||
|
||||
A WireGuard client: the Cardputer joins a WireGuard network over whatever Wi-Fi it is on.
|
||||
|
||||
### Measured before deciding (2026-10-07)
|
||||
|
||||
The issue asked for the libraries to be measured first. `esphome/wireguard` 0.4.8 (maintained, published the same week; BSD-3-Clause) was built into a trial firmware and a tunnel brought up against a throwaway peer in a container.
|
||||
|
||||
| | Cost |
|
||||
|---|---|
|
||||
| Flash, the library | 43 KB |
|
||||
| Flash, with our service, page and commands | 63 KB |
|
||||
| Static RAM | 1.2 KB |
|
||||
| Heap with the tunnel up | 1.8 KB |
|
||||
|
||||
- **It crashes this build as shipped.** The library calls lwIP's raw functions without taking lwIP's lock, and this framework is built to check for that (`CONFIG_LWIP_CHECK_THREAD_SAFETY`): the first `netif_add` stopped the device. Every call into it is made with the lock held, on our side; the library is not changed.
|
||||
- **One address range is allowed by default;** more need `CONFIG_WIREGUARD_MAX_SRC_IPS`, set in `platformio.ini`.
|
||||
- One peer, IPv4.
|
||||
- The older `ciniml/WireGuard-ESP32` was last touched in 2021 and was not tried.
|
||||
|
||||
### Decisions (design round 2026-10-07)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q243 | **`esphome/wireguard`, pinned at 0.4.8,** with lwIP's lock taken around every call. |
|
||||
| Q244 | **Configured by importing a standard `.conf` from the card** (`/vpn/wg0.conf`), from Settings or with `vpn import`. Nothing is typed on the device. |
|
||||
| Q245 | **The private key comes in that file,** as WireGuard configurations are handed out. It is kept in the device's settings, never shown and never printed. After an import Settings **offers to delete the file**: the card comes out, and the key is in it in clear. |
|
||||
| Q246 | One tunnel, one peer. |
|
||||
| Q247 | **A switch, and "Start with Wi-Fi"** (off by default). The switch is for now: it doesn't outlast a restart. The tunnel waits for the clock, since a handshake carries the time and a server refuses one older than the last it saw; the clock is set over plain Wi-Fi first. |
|
||||
| Q248 | *Narrowed while building.* **Either everything goes through the tunnel, or one subnet does.** With `0.0.0.0/0` in AllowedIPs the tunnel is the default route. Otherwise only the subnet this device's tunnel address is in is routed: the widest allowed range that holds it. **A home network behind the server can't be reached without the full tunnel:** lwIP routes by an interface's own subnet or by default, and has no table for anything finer. The import says how many ranges it can't reach. |
|
||||
| Q249 | *Not as planned.* **With everything through the tunnel, nothing leaves while the server is silent:** the default route stays in the tunnel, which has nowhere to send. That is a kill switch, by construction and not by choice. With one subnet, packets for it go out on Wi-Fi again while the tunnel has no peer. |
|
||||
| Q250 | The file's DNS servers are used while the tunnel is up, if they can be reached through it; what was there before goes back when it stops. |
|
||||
| Q251 | **The Debug Console and the Update Service answer over the tunnel** as they do on Wi-Fi: the console still wants its token and an update its signature. |
|
||||
| Q252 | **`VPN` in the Status Bar** while the tunnel is wanted, bright once the server has answered. Settings > VPN has the state, the server, this device's address, what goes through it and how long ago the server was heard. `vpn status`, `up`, `down`, `import`, `forget`, `auto`. A Toast when it comes up and when the server stops answering. |
|
||||
| Q253 | PresharedKey, MTU and ListenPort from the file; keepalive 25 s if the file has none; the tunnel is taken down with the Wi-Fi it was on and started afresh on the next. No IPv6. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`lib/net/src/wg_config.h`** (host-tested, 6 tests): reads a `.conf` as people write them (any case, comments, CRLF, IPv6 entries left out), refuses what it can't use with the line and the field and never the key, writes it back tidy for the settings store, and says what will be routed.
|
||||
- **`VpnService`** (`src/services/vpn_service`): the tunnel is up when it is wanted, Wi-Fi is connected and the clock is set. It holds lwIP's lock around the library, adds the allowed ranges, makes the tunnel the default route for "everything", and puts the DNS servers in and out. A DHCP renewal that replaces them is noticed: the tunnel's go back in, and the renewed ones are what is restored later.
|
||||
- **The tunnel's own packets never go into the tunnel:** the library sends them on the interface that was the default when it started.
|
||||
- **Connections that came in over Wi-Fi stay on Wi-Fi** with everything routed into the tunnel: a reply leaves by the interface whose address it carries.
|
||||
- **`vpn up <seconds>`** takes the tunnel down again by itself: for trying a configuration from afar, when a wrong one could cut the connection it was sent over.
|
||||
- Settings: `VpnConfig` (the `.conf`, checked on every load) and `VpnAuto`.
|
||||
|
||||
### Checks on the device (2026-10-07, against a WireGuard peer in a container)
|
||||
|
||||
The test keys were made for the purpose and deleted. Two rounds: first with the device on a guest Wi-Fi that can't open connections to the machine the test peer ran on, so **the peer called the device** (`ListenPort`), which WireGuard allows either way round; then on a network where **the device called the peer**, as it normally would.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `vpn import`, then the file removed | "imported, through it 10.9.0.0/24"; the configuration survives a firmware update |
|
||||
| `vpn up` | Up within seconds; `VPN` bright in the Status Bar; a Toast |
|
||||
| From the peer, through the tunnel | 25 pings of 25, 1300 bytes too; the Debug Console's greeting on TCP 2323; TCP 3232 answers |
|
||||
| DNS | The file's server while up (`wifi status` says `(VPN)`), DHCP's back after `vpn down`, with no reconnection |
|
||||
| Everything through the tunnel | The device stays reachable over Wi-Fi; an update check's HTTPS to the release server is seen inside the tunnel at the peer |
|
||||
| `vpn up 100` | Down by itself after 100 s |
|
||||
| "Start with Wi-Fi", then a restart | Up by itself 40 s after the restart, once Wi-Fi and the clock were there |
|
||||
| The peer silenced | After three minutes: "no answer yet", a Toast, `VPN` dim. With everything through the tunnel, an update check then fails: nothing leaves. The peer back: up again in under half a minute, and a Toast |
|
||||
| `vpn forget` | "not set"; DNS as before |
|
||||
| **The device calling the peer**, the server given by name, with a PresharedKey and `MTU = 1280` | Up in seconds; the peer shows the device's address and port as the endpoint; pings through it |
|
||||
| One subnet: a connection the device opens to the peer's tunnel address | Seen inside the tunnel at the peer |
|
||||
| Everything: a Gemini page from a public capsule | Fetched (TLS, 1,184 bytes), and seen inside the tunnel at the peer. Free memory fell to 45.9 KB at the lowest |
|
||||
| Memory | 106.1 KB free before, 104.3 KB with the tunnel up, 106.2 KB after |
|
||||
| Settings > VPN | The four rows, the state and "heard 66 s ago", the server, the address; no key anywhere on it |
|
||||
|
||||
**Against a real server** (the maintainer's own, 2026-10-08): a configuration put on the card by the maintainer and imported in Settings; the server named by host name, on a port of its own, the device's address a /32, everything through the tunnel, "Start with Wi-Fi" on. The tunnel is up, and from another machine the device answers on its tunnel address: pings, and the Debug Console.
|
||||
|
||||
**Not checked:** from a network far from the server (the device was on the server's own network, reaching it by its public name). That the MTU is what limits a packet (larger pings were answered too, in pieces). Roaming from one Wi-Fi to another with the tunnel wanted. IRC through the tunnel. A day of uptime.
|
||||
|
||||
### What went wrong while building it
|
||||
|
||||
**The device stopped on the first try,** on lwIP's "Required to lock TCPIP core functionality!". The library was written for builds that don't check; ours does. The fix is three lines of ours, and the crash report named `netif_add` and the line that called it.
|
||||
|
||||
**Taking the tunnel down reconnected Wi-Fi.** The first version gave DHCP's DNS servers back by asking for a new lease, which is how the Wi-Fi settings do it, and which drops every connection: the Debug Console session that had typed `vpn down` among them. The servers that were there are now simply remembered and put back.
|
||||
|
||||
**"What AllowedIPs say" was more than the network stack can do.** The design round promised split tunnels by AllowedIPs. lwIP has no routing table: it can send by an interface's subnet, or by default. So it is one subnet or everything, and the import tells which.
|
||||
|
||||
## Network troubleshooting commands (issue #90)
|
||||
|
||||
With a tunnel, fixed addresses and a file server on the device, "is it the network or is it me" needed another machine to answer.
|
||||
|
||||
### Decisions (2026-10-08; built on the issue's list, without a round of questions)
|
||||
|
||||
- **The familiar names:** `ping`, `nslookup`, `traceroute`, `ifconfig`, `arp`. `port <host> <port>` for "is that TCP port open", which has no single familiar name.
|
||||
- **In this version:** those six. **Not yet:** `tls` (why a certificate fails), `ntp` (the clock's offset), `netstat` (what listens). The issue stays open for them.
|
||||
- **Commands only,** in the Shell and over both consoles; no page in an App.
|
||||
- **One line an answer, short:** a Shell line is 38 characters.
|
||||
|
||||
### As built
|
||||
|
||||
- **`lib/net/src/net_probe.h`** (host-tested, 5 tests): what was typed; the ICMP echo request and what answers it, a router's "time exceeded" included; the DNS query and its answer, with the pointers names are shortened by.
|
||||
- **`NetTools`** (`src/services/net_tools`): `ping`, `nslookup`, `port` and `traceroute` each run on a task of their own, made for the command and gone after it, printing to the console that asked (the Shell shows only its own replies). One at a time; `cancel` stops it within a fifth of a second.
|
||||
- **`ping`** and **`traceroute`** share a raw ICMP socket: a traceroute is echo requests allowed one hop, then two, then three, and the routers' complaints are the list.
|
||||
- **`nslookup`** asks one server itself, over UDP, and so can say which server answered and how long it took, which the system's resolver doesn't; and it can ask a server that isn't the configured one.
|
||||
- **`port`** is a connection attempt that is not waited for: open, refused, or five seconds of nothing.
|
||||
- **`ifconfig`** and **`arp`** read lwIP's own lists, with its lock held.
|
||||
- **Cost:** 12 KB of flash. A 6 KB task while a command runs (2.6 KB of it never used), nothing otherwise.
|
||||
|
||||
### Checks on the device (2026-10-08, with the VPN up and everything routed through it)
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `ifconfig` | `vpn 10.9.0.2/32 mtu 1420, up, default route`; `wifi ... gw ... mtu 1500, up`; the DNS server |
|
||||
| `arp` | The gateway and one other machine |
|
||||
| `ping` of a neighbour, of a name | 4 of 4 in 3 to 4 ms; 3 of 3 in about 50 ms |
|
||||
| `ping 9.9.9.9 2 1392`, then `1393` | Both back; neither back: the tunnel carries 1420 bytes exactly |
|
||||
| `nslookup` | The address, the server and the time; an alias followed; with another server; "there is no nope.invalid" |
|
||||
| `port` | `open, 52 ms`; `refused`; "no answer in 5 s"; "doesn't resolve" |
|
||||
| `traceroute 9.9.9.9` | Nine hops, the tunnel's server first, "arrived" |
|
||||
| A second command while a ping runs | "another one is running: `cancel` stops it" |
|
||||
| `cancel` | "stopped", with the count so far |
|
||||
| In the Shell | Tab completes them; the lines appear there and only there |
|
||||
|
||||
**Not checked:** without the VPN (every check went through the tunnel, or to the local network); a network that drops ICMP; the commands in Safe Mode, where they are not offered.
|
||||
|
||||
**Found on the way:** a refused connection is reported by lwIP as "reset", not "refused"; the first version called it "no route". And the header for the tested half was first given the same name as the service's, which makes a file include itself: the same mistake as an hour before, in the same way.
|
||||
|
||||
### The rest of the list: `tls`, `ntp`, `netstat` (2026-10-08)
|
||||
|
||||
- **`tls <host> [port]`** makes a handshake that checks nothing, so that a bad certificate can be looked at, and then checks it itself: against this device's roots (`ca_roots.h`, the ones the Update Service trusts) and the name asked for. It says who the certificate is for, who signed it, from when to when with the days left, the verdict with its reasons, and the SHA-256 that a Gemini pin is. It runs on a 12 KB task and isn't tried with less than 70 KB free: a handshake peaks at about 52 KB.
|
||||
- **`ntp [server]`** sends one SNTP request and compares the answer with the device's clock, allowing for half the round trip. With no server it asks the first one in Settings.
|
||||
- **`netstat`** reads lwIP's own lists: what listens, labelled where the firmware knows what it is, what is connected, and the UDP ports in use.
|
||||
- Host tests: the NTP packet and the year 2036, the offset in words, a certificate's name (an old string type that mbedTLS prints as hex included), days between dates. 7 tests in `test/test_net_probe` in all.
|
||||
|
||||
| Check on the device | Result |
|
||||
|---|---|
|
||||
| `tls git.twis.la` | 709 ms; for git.twis.la, 67 days left, "this device trusts it", the SHA-256 |
|
||||
| `tls geminiprotocol.net 1965` | "NOT trusted here: not signed by a root this device has": a capsule signs its own |
|
||||
| `tls expired.badssl.com` | "EXPIRED 4197 days ago" |
|
||||
| `tls wrong.host.badssl.com` | "NOT trusted here: not for that name" |
|
||||
| `tls` to a port that isn't TLS | "no handshake ... An invalid SSL record was received" |
|
||||
| `ntp` | The server, its stratum, 50 ms away; "this clock is right, to 0.1 s" |
|
||||
| `netstat` | The update port and the Debug Console listening, the console's own connection, the UDP ports |
|
||||
| Memory during a `tls` | 44.5 KB free at the lowest, from 104 KB |
|
||||
|
||||
**Not checked:** `tls` with IRC connected (it should refuse for lack of memory); `ntp` against a clock that is wrong; `netstat` while sharing.
|
||||
|
||||
## The SSH client (issue #2)
|
||||
|
||||
A terminal on another machine: one session to a shell, from the SSH App.
|
||||
|
||||
### Measured before deciding (2026-10-08)
|
||||
|
||||
`ewpa/LibSSH-ESP32` 5.10.0 (libssh on mbedTLS) was built into a trial firmware and a session opened against OpenSSH in a container, with a password.
|
||||
|
||||
| | Measured in the trial | As built |
|
||||
|---|---|---|
|
||||
| Flash | 120 KB | **292 KB** |
|
||||
| Static RAM | 1.2 KB | |
|
||||
| The session's task stack | 13 KB used | 13.7 KB used of 20 KB |
|
||||
| Free heap with a session open | | 49 KB of 99 KB: it costs about 50 KB, the stack included |
|
||||
| Lowest free heap during a login | | 30 KB |
|
||||
| Key exchange (curve25519, ed25519 host key) | 227 ms | |
|
||||
|
||||
- **The trial undercounted the flash.** It logged in with a password. Signing with a key of the device's own (Q255) links libssh's table of multiples of the Ed25519 base point: `ge25519.c.o` alone is 109 KB. The rest of the difference is the public-key code, the terminal and three fonts. The firmware is at 71% of its slot.
|
||||
- **libssh carries its own curve25519** (`src/external/curve25519_ref.c`) with the names libsodium uses, and libsodium is already here for WireGuard: two definitions don't link. A build script (`scripts/libssh_filter.py`) leaves libssh's copy out, and it uses libsodium's. It has to be a `pre:` script: libraries are built before any `post:` one runs.
|
||||
- A session and a TLS connection don't fit together: IRC holds 40 KB.
|
||||
|
||||
### Decisions (design round 2026-10-08)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q254 | **LibSSH-ESP32 5.10.0**, with its duplicate curve file left out of the build. |
|
||||
| Q255 | **A password, typed each time and never stored**, or **a key the device makes for itself** (Ed25519, no passphrase, kept in the settings store). Its public half is shown, written to `/ssh/id_ed25519.pub` and printed by `ssh status`. Keys made elsewhere aren't imported. |
|
||||
| Q256 | **A terminal good enough for a shell, `less`, `top`, `nano` and `vim`:** cursor movement, erasing, sixteen colours, scroll regions, the alternate screen, the cursor keys' two modes. `TERM=xterm`. No mouse. |
|
||||
| Q257 | **Five text sizes, changed with Ctrl and + or -** (the user's change to the round: the proposal was a setting). 4x6, 5x8, 6x10, 6x13 and 9x15 pixels: from 60 x 20 to 26 x 8 characters. The far end is told the new size; the choice is kept. |
|
||||
| Q258 | **100 lines of scrollback**, as text, with Alt and up or down, as in the Shell. Not on the alternate screen. |
|
||||
| Q259 | **Keys:** Ctrl with a letter; Tab; the backtick key sends Esc, as it is printed; Alt with it types a backtick; Fn with the arrow keys; Shift with those for Page Up and Down; Ctrl+Alt+q disconnects. Fn with backtick is Home, as everywhere. |
|
||||
| Q260 | **The session outlives the App's time in front.** `SSH` in the Status Bar while one is open. |
|
||||
| Q261 | **Not started under 75 KB free**, with the reason in words. |
|
||||
| Q262 | **Up to eight hosts remembered**, the last used first, once a login has succeeded. Forgetting one forgets its server's fingerprint too, unless another remembered host is the same server. |
|
||||
| Q263 | **Trust on first use,** on the SHA-256 fingerprint; sixteen servers remembered. **A changed key is a warning**, with Cancel selected. |
|
||||
| Q264 | **UTF-8 in, the fonts' Latin-1 out:** what they lack is `?`, box-drawing lines are `+ - \|`. |
|
||||
| Q265 | **`ssh user@host` in the Shell opens the App** and connects there. The password is never asked on a console. |
|
||||
| Q266 | **Not built:** port forwarding, SFTP and scp, jump hosts, agent forwarding, keys with a passphrase, more than one session. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`lib/term`** (host-tested, 12 tests in `test/test_terminal`): `Terminal`, the screen a program's output makes, with its history and the replies a program asks for; `encodeKey`, what a key sends; `SshHosts` and `SshKnownHosts`, the two lists kept in the settings store as lines of text.
|
||||
- **`SshService`** (`src/services/ssh_service`): one session on a task of its own (20 KB of stack). The task and the main loop share the terminal, the bytes to send and the state under one lock. Its questions (is this the right server? the password?) are states it waits in until the App answers; the settings store is only written from the main loop. The password and the private key are overwritten after use.
|
||||
- **`SshApp`** (`src/apps/ssh_app`): the hosts, the entry, the session, the key page. It draws the grid a run of same-coloured cells at a time, at most every 60 ms.
|
||||
- **Keys that aren't characters now say what was held with them** (`lib/input/src/key_mapper.cpp`): the arrows, Enter, Del, Tab and Back carry Shift, Ctrl and Alt. The terminal needs it for Page Up and Alt+backtick. It also makes two documented keys work from the real keyboard, which until now only worked from the Debug Console's `key` command: Ctrl with Fn and up or down in a note, and Shift+Tab in Gemini.
|
||||
- **IRC doesn't try to connect with less than 60 KB free** (`IrcService::kNeedFree`): see below.
|
||||
- Settings: `SshHosts`, `SshKnown`, `SshKey`, `SshPublic`, `SshFont`.
|
||||
|
||||
### Checks on the device (2026-10-08, against OpenSSH 9.7 in a container on the same network)
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| `ssh tester@host:2222` from the Debug Console | The App opens; the fingerprint shown is the one `ssh-keygen -lf` prints on the server |
|
||||
| Trust it, a password | A shell; `stty size` says 15 48, `$TERM` is xterm |
|
||||
| `ls -la`, `top`, `vim` (insert, Esc, `:wq`) | Drawn right: `top`'s reverse-video header, `vim`'s alternate screen and what was there before coming back; the file is on the server |
|
||||
| Ctrl with + and - | `stty size` says 12 40, then 20 60; `top` redraws for it |
|
||||
| `seq 1 60`, Alt with up | The history, in grey, with how far back in the corner |
|
||||
| Fn+backtick, then the App again | The Launcher with `SSH` in the Status Bar; the session as it was |
|
||||
| `sleep 100`, Ctrl+C; Alt+backtick | Interrupted; `` echo `id -u` `` prints 1000 |
|
||||
| Ctrl+Alt+q; `exit` | "Disconnected"; "The session ended" |
|
||||
| This device's key, its public half in `authorized_keys` | "Accepted publickey" in the server's log; no password asked |
|
||||
| The server's host keys replaced | "THE SERVER'S KEY CHANGED" with the new fingerprint, Cancel selected. Cancel: "Not trusted: not connected". Replace: it connects, and doesn't ask again |
|
||||
| A wrong password | "Wrong password", and asked again; Back gives up |
|
||||
| A port nothing listens on | "Nothing listens there: the connection was refused" |
|
||||
| `ssh nobody`, `ssh a@`, a port of 99999 | Refused, each with its reason |
|
||||
| Forgetting a host | Asked, then gone from the list |
|
||||
| `irc start` with a session open | "not enough memory: close the SSH session, retrying in 5 s", and no attempt: the lowest free heap doesn't move |
|
||||
| Memory | 99 KB free before, 49 KB with a session open, 30 KB at the lowest during a login, 99 KB again after |
|
||||
| Stacks | `ssh` 6.8 KB free of 20 KB; `loopTask` 1.9 KB free, as before |
|
||||
|
||||
**Not checked:** the refusal under 75 KB free (it is one comparison, and wasn't provoked). A server on the internet, or through the VPN. Wi-Fi lost in the middle of a session. Servers other than OpenSSH. `nano`, `less`, `htop`, `tmux`. Keyboard-interactive logins (two-factor prompts). A session left open for hours.
|
||||
|
||||
### What went wrong while building it
|
||||
|
||||
- **IRC, started with a session open, took the free heap down to 236 bytes.** A test script's keys went to the Launcher instead of the terminal and opened the IRC App, which connects when opened. Its TLS handshake found no memory, failed, and tried again with its usual back-off, six times; nothing crashed and it never connected, but 236 bytes is no margin at all. IRC now looks at the free heap before each attempt and says "not enough memory: close the SSH session" instead of trying.
|
||||
- **The build script did nothing as a `post:` script:** the libraries were already built when it ran.
|
||||
- **A failed connection was first shown as an empty terminal** with its reason squeezed on the last line, and a host was remembered before anyone had logged in to it. Both changed: the reason has a page, and a host is remembered once a login succeeds.
|
||||
- **The trust question didn't fit its dialog:** the fingerprint is 50 characters. It is now split over two lines, under one line of words.
|
||||
@@ -8,7 +8,7 @@ docs = true
|
||||
source = "docs/milestones/R1.md"
|
||||
tag = "R1"
|
||||
+++
|
||||
**Status:** in progress. CI and signed releases on Gitea (issue #5) are in place since 2026-10-06: every tag from v0.1.0 to v0.10.0 has its release. Updates from Gitea (issue #6) is built and checked on the device, on branch `gitea-updates`, not merged yet. The Issues App (#4) comes after.
|
||||
**Status:** in progress. Shipped: CI and signed releases on Gitea (issue #5), a release for every tag; updates from Gitea (#6, **v0.11.0**); one firmware with the Debug Console in it (#68, **v0.12.0**); CI in about a minute (#74, **v0.13.0**). Not started: the Issues App (#4), automatic installs (#52), release channels (#53), resuming a download (#54).
|
||||
|
||||
**Goal:** a tag is a release, built the same way every time and published where a device can find it.
|
||||
|
||||
@@ -22,7 +22,7 @@ Until now the tests, the builds, the signing and the flashing all happened on on
|
||||
|---|---|
|
||||
| Q151 | A push to `main`: the host tests (with their coverage). A push to another branch: nothing, its pull request is what runs (a branch with an open pull request ran twice, once for each). A pull request: the same and both builds; changes reach `main` through pull requests. A tag `v*`: all of it, then a release. (First: everything on every push, which rebuilt the firmware far more often than anyone looked at it.) |
|
||||
| Q152 | **CI signs.** The signing key is the repository secret `OTA_SIGNING_KEY`; a tag push makes a complete, signed release with no manual step (ADR 0008). |
|
||||
| Q153 | The Debug Build is built in CI with a token of the runner's own, to prove it compiles, and **isn't published**: it would hand everyone its Debug Console token. |
|
||||
| Q153 | *Revised by Q188: there is one firmware.* The Debug Build is built in CI with a token of the runner's own, to prove it compiles, and **isn't published**: it would hand everyone its Debug Console token. |
|
||||
| Q154 | Pull requests from forks don't start a run. |
|
||||
| Q155 | A release carries `roro9stack-<version>.ota` (signed), `-factory.bin` for USB, `.elf.gz` to decode crashes, and `SHA256SUMS`. |
|
||||
| Q156 | Its text is the tag's message, what the files are, and the commits since the tag before. |
|
||||
@@ -72,7 +72,7 @@ The device looks at the project's Gitea for a newer release, says so, and instal
|
||||
| Q168 | A version that failed (rolled back) isn't announced again by the background check until a newer one exists; it can still be installed by hand. |
|
||||
| Q169 | **Older releases:** a list of the last ten, newest first, the running one marked. Installing an older one asks with a stronger warning. |
|
||||
| Q170 | Enter on a release shows its version, date, size and the tag message, with Install. |
|
||||
| Q171 | **Debug Builds** check and show the latest release, but don't install it: it would replace the Debug Build and its console (Q153: Debug Builds aren't published). Their updates come from the PC. |
|
||||
| Q171 | *Revised by Q188: there is no Debug Build, every firmware installs releases.* **Debug Builds** check and show the latest release, but don't install it: it would replace the Debug Build and its console (Q153: Debug Builds aren't published). Their updates come from the PC. |
|
||||
| Q172 | **Memory, as measured:** a TLS connection peaks at about 52 KB of heap, with or without checking the certificate, so it starts with 80 KB free (Q86's 20 KB spare on top), not 55 KB. **A check, list or install someone asked for makes IRC step aside** and come back after; the daily check never does, and with IRC connected it waits. (First: 55 KB and nothing else. With IRC connected a check left 3 KB and a download 836 bytes.) |
|
||||
| Q173 | Left out, each with its issue: installing automatically (#52), a release channel (#53), resuming a download (#54). |
|
||||
| Q174 | Ships as **v0.11.0**. Tested on the device with the real signed releases; a Debug Build command pretends the device runs an older version, so v0.10.0 counts as an update. |
|
||||
@@ -131,3 +131,111 @@ The device looks at the project's Gitea for a newer release, says so, and instal
|
||||
- **A key press during the hold:** Back on the Firmware page while a check is going doesn't cancel it.
|
||||
|
||||
**Two slips during the checks:** a blind sequence of keys on the Firmware page opened the SD card's install dialog (the page keeps its selection between visits); it was cancelled with Back, nothing installed. And my port-polling while waiting for a restart took the Debug Console's only client slot, which made the first install attempt look like a failure.
|
||||
|
||||
## One firmware: the Debug Console in every build (issue #68)
|
||||
|
||||
The issue asked for a token that could be set, so that Debug Builds could be published. The design round ended somewhere simpler: no Debug Builds. The console is in every firmware, off until switched on, with a token that belongs to the device (ADR 0010, which supersedes part of ADR 0004).
|
||||
|
||||
### Decisions (design round 2026-10-06)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q188 | **One firmware,** with the Debug Console and the test commands compiled in. The `cardputer-adv-debug` environment, `RORO_DEBUG`, the `+debug` version and `scripts/debug_flags.py` go; CI builds one firmware. Revises Q153 and Q171. |
|
||||
| Q189 | **Off by default,** and off when the stored setting is missing or invalid. Off, nothing listens and no memory is used: the task and the 4 KB ring exist only while it's on. |
|
||||
| Q190 | **Settings → Debug Console:** the switch, where to connect, the token, "New token", "Type a token". It stays on across restarts and in Safe Mode. **`DBG` in the Status Bar** while it listens, bright with a client connected. |
|
||||
| Q191 | **The device makes the token** the first time the console is switched on: 100 bits as 20 characters of Crockford's base32, shown in groups of four. It can be replaced by one typed by hand, of at least 16 characters. Case and dashes don't count. |
|
||||
| Q192 | **USB serial can set it up:** `debug on`, `debug token <value>`, `debug token new`, over USB only; `debug status` and `debug off` from anywhere. `scripts/flash.sh --debug` gives a device the developer's token. The token is never printed. |
|
||||
| Q193 | **Challenge and answer:** the device sends 16 random bytes, the client their HMAC-SHA256 keyed by the token. Five wrong answers in a row close the console for a minute, with a Notification. **Old clients and old firmwares don't talk to each other:** accepted, this far from v1 and with one user. |
|
||||
| Q194 | `rdbg.py` takes the token from **`-t`/`--token`**, then **`$RORO_DEBUG_TOKEN`**, then `~/.config/roro9stack/debug-token`. |
|
||||
| Q195 | **The test commands are in every build** (`crash abort`, `crash wdt`, `update pretend`, `update damage`, `lora inject`, `sd fill`, `loop spin`, `wifi ip … try`). `update install … force` goes: there's nothing left to protect. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`lib/debug`** (host-tested, 9 tests): the token maker, tidying what a person types, HMAC-SHA256 on the project's own SHA-256 (checked against RFC 4231), the answer to a challenge (the same vector as `rdbg.py`'s), a comparison that takes the same time whatever it's given, and the pause after five wrong answers, right across the clock's wrap.
|
||||
- **Two settings,** `DebugConsole` and `DebugToken`, with the others: validated, and invalid stored values count as missing.
|
||||
- **The console's task and ring come and go with the setting.** `Console::openRing` allocates the ring; switching off closes the socket, frees it and ends the task. `tick` follows the settings once a second, so a switch off and on again takes a second or two.
|
||||
- **Measured against the two builds it replaces:** 1,881,799 bytes of flash and 54,700 of static RAM. The release build was 1,851,387 and 54,612 (so 30 KB more flash and 88 bytes more RAM); the Debug Build was 1,874,887 and 58,780 (4 KB of RAM back: the ring is no longer a static array).
|
||||
- **The listener is asked whether it listens.** The framework's `begin()` returns nothing and fails without a word; the task now checks, says so on the console, and tries again every two seconds.
|
||||
- **`debug off <seconds>`** closes the console and brings it back by itself: the only way to try its closing and reopening from afar, since switching it on is for the device and the cable only.
|
||||
- **`scripts/rdbg.py`** answers the challenge, and fetches a release's ELF from Gitea when a crash names a version that isn't in `.pio/elves/`.
|
||||
|
||||
### Checks
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Host tests | 468 pass (456 before) |
|
||||
| The firmware builds | One environment, 1,881,799 bytes of flash used |
|
||||
| Pushed over Wi-Fi to a device running a Debug Build | Installed, restarted; the update port answers and **port 2323 refuses connections**: off by default |
|
||||
| Switched on at the device (Settings → Debug Console) | A token is made and shown. Its first drawing, in bold at normal size, was misread once: it is now at twice the size, in two lines, and `O`, `I` and `L` are taken for `0` and `1` |
|
||||
| A login with `scripts/rdbg.py` | The challenge is answered; `info`, `screenshot` and the backlog work |
|
||||
| `DBG` in the Status Bar | There while the console is on, bright while a client is connected (screenshots) |
|
||||
| `debug on` and `debug token` over the console | Refused: `over USB serial only` |
|
||||
| Six logins with a wrong token | Five are refused, a second apart; the sixth, and the right token after it, get `locked`. After a minute the right token works again. The console's own log and a Notification say so |
|
||||
| Three `crash abort` in a row | **Safe Mode**, with the console reachable: `info` works, `ls /` says `not available in Safe Mode`. `rdbg.py crash` decodes the backtrace to `runCommand` at the `abort()` line. `reboot` leaves Safe Mode |
|
||||
| An update over Wi-Fi with the console on | The setting and the token survive: the console is back by itself after the restart |
|
||||
| `debug off` over the console | The client is dropped and port 2323 refuses connections; the update port still answers |
|
||||
| `debug off 2`, 10 times in a row, then 15 | It came back each time, a second after the pause. Free heap dips by about 270 bytes for each connection the device closes and **is all back two minutes later** (107.2 KB before, 103.2 right after 15, 107.0 at two minutes): TCP holds a closed connection that long. Not a leak, though it looked like one for an hour |
|
||||
| Memory, console on with a client | 108 KB free (the Debug Build it replaces: 104 KB). In Safe Mode: 178 KB free |
|
||||
|
||||
**One false alarm:** after the tests the console "wouldn't come back on". It was off: the setting had been left off by `debug off`, and the page's "Switch it on?" dialog opens on **Cancel**, so Enter twice leaves it off. Nothing was lost.
|
||||
|
||||
**Not checked:** `scripts/flash.sh --debug` over USB (no device on USB here); "New token" and "Type a token" from the page (the code paths are the ones `debug token` uses, which need the cable); the Toast itself on screen (the console is closed while it shows; its Notification is in the log); free memory with the console off, which only the serial port could say (the static figures above are the evidence); fetching a release's ELF, which needs a crash on a released version.
|
||||
|
||||
## CI that doesn't rebuild the world (issue #74)
|
||||
|
||||
A pull request's run took over seven minutes, a tag's thirteen and a half. The goal: a firmware build under a minute.
|
||||
|
||||
### Where the time went (run 81, a pull request, 2026-10-06)
|
||||
|
||||
| Step | Time |
|
||||
|---|---|
|
||||
| Tools (apt, pip) | 15 s |
|
||||
| Check out | 2 s |
|
||||
| Host tests and coverage | 55 s |
|
||||
| **The firmware** | **358 s** |
|
||||
| of which: CMake configuring ESP-IDF | 87 s |
|
||||
| of which: compiling ESP-IDF's libraries | 171 s |
|
||||
| of which: our own build (the Arduino core, the libraries, `src/`) | 91 s |
|
||||
|
||||
A tag's run did the firmware step, then built the same commit again for the release: twice 356 s.
|
||||
|
||||
### Why the framework was rebuilt every time
|
||||
|
||||
The framework is rebuilt with our SDK settings (ADR 0006), and the rebuilt libraries stay in the toolchain volume. But the platform decides whether they match by reading **`sdkconfig.defaults` in the project folder**, whose first line carries a hash of the settings. That file is generated, and not in git. A fresh checkout has none, so the platform concluded "different settings", **reinstalled the framework and rebuilt it**: 260 seconds, at every run, to arrive at the libraries that were already there. On a developer's machine the file is simply still there from the last build, which is why nobody saw it.
|
||||
|
||||
### What changed
|
||||
|
||||
| Change | Effect |
|
||||
|---|---|
|
||||
| **The file is kept in the volume, inside the libraries it describes** (`framework-arduinoespressif32-libs/.roro-sdkconfig.defaults`), copied into the checkout before a build and back after one that passed. The platform still checks its hash against `platformio.ini`: changed settings rebuild, as they must. Kept there and not beside them, it disappears when the libraries are reinstalled, so it can't describe libraries that are gone | 358 s to 92 s |
|
||||
| **The version is no longer a `-D` on every compiler command line.** `scripts/version.py` writes `lib/version/src/version_generated.h` (not in git, written only when it changes), read by one file. Before, every commit recompiled everything, on a developer's machine too, and no cache could have helped | A rebuild with nothing changed: 77 s to 13 s, locally |
|
||||
| **PlatformIO's build cache** (`PLATFORMIO_BUILD_CACHE_DIR`, SCons's CacheDir) in the volume, for the firmware of pull requests: objects by the signature of their sources and command line | 92 s to 26 s, with a new version and one changed file |
|
||||
| **ccache for the host tests.** They are built with coverage counters, and the build cache would return objects without their `.gcno` files; ccache keeps both | 49 s to 33 s. What's left is PlatformIO starting 51 test programs |
|
||||
| **A tag builds its firmware once**, in the release step | minus 6 minutes |
|
||||
| PlatformIO and gcovr in a virtual environment in the volume | a few seconds |
|
||||
|
||||
**A release compiles its own sources from nothing:** it reuses the rebuilt framework (the platform checks the hash) but not the build cache, so no published file contains an object that came from another commit's build.
|
||||
|
||||
### Measured (a development machine, fresh copies of the tree, the same volume)
|
||||
|
||||
| | Before | After |
|
||||
|---|---|---|
|
||||
| The firmware, fresh checkout, nothing cached for it | 358 s | 82 s (it fills the cache) |
|
||||
| The firmware, fresh checkout, a new version and one file changed | 358 s | **27 s** |
|
||||
| Host tests and coverage | 49 s | 33 s |
|
||||
| Rebuilding locally with nothing changed | 77 s | 13 s |
|
||||
|
||||
### Measured on the runner (pull request #76, 2026-10-07)
|
||||
|
||||
| Run | Tools | Tests and coverage | The firmware | The whole job |
|
||||
|---|---|---|---|---|
|
||||
| Before (run 81) | 15 s | 55 s | 358 s | 434 s |
|
||||
| The first with the new workflow: no mark yet, the framework is rebuilt once more and the caches fill | 16 s | 59 s | 354 s | 431 s |
|
||||
| The next commit (only the workflow changed) | 10 s | 36 s | **51 s** | **100 s** |
|
||||
| The same commit again | 10 s | 36 s | **18 s** | **66 s** |
|
||||
|
||||
In the 51-second run, 245 objects came from the cache and 43 were compiled: `version.cpp`, as expected, and all 42 files of `src/`, which had not changed. In the run after it, all 290 came from the cache. So the objects of `src/` made by the run that rebuilt the framework were not reusable by a normal run, and those of a normal run are: the two-pass build that rebuilds the framework compiles `src/` with something different on its command line. It costs one 51-second run after each framework rebuild, which is rare; I did not look for what differs.
|
||||
|
||||
The firmware step with everything cached is 18 seconds: the libraries are downloaded and unpacked (4 s), the dependency scan (5 s), fetching 290 objects, the link and the image (the last 11 s). A pull request that changes a few files should land between that and 51 seconds.
|
||||
|
||||
**What it costs:** the build cache grows by about 40 MB a run (each linked firmware is kept) and is started again past 3 GB; ccache is held to 1 GB.
|
||||
|
||||
@@ -8,7 +8,7 @@ docs = true
|
||||
source = "docs/milestones/S1.md"
|
||||
tag = "S1"
|
||||
+++
|
||||
**Status:** the three planned items are done: the SD driver fix in v0.6.1 (issue #21, ADR 0007), fixed IPv4 settings in v0.7.0 (issue #7), the System App in v0.8.0 (issue #11). v0.8.1 adds the resting main loop (issue #40) and the GNSS pause for the radio's noise (issue #20, still open for the 11 dB that remain). Still open in the milestone: #39, following the SD driver upstream.
|
||||
**Status:** the three planned items are done: the SD driver fix in v0.6.1 (issue #21, ADR 0007), fixed IPv4 settings in v0.7.0 (issue #7), the System App in v0.8.0 (issue #11). v0.8.1 adds the resting main loop (issue #40) and the GNSS pause for the radio's noise (issue #20, still open for the 11 dB that remain). Still open in the milestone: #39, following the SD driver upstream. The **Shell** (issue #67) shipped as **v0.14.0**; the Shell in Safe Mode (#77) is not started.
|
||||
|
||||
**Goal:** the device works on any network, the card can be trusted, and you can see what the system is doing. A side milestone, like G1.
|
||||
|
||||
@@ -144,3 +144,69 @@ M3 found the LoRa radio's noise floor about 15 dB above what the chip hears alon
|
||||
**Settings > Pause GNSS for LoRa**, off by default: while the LoRa radio listens or sweeps, the GNSS receiver waits in standby, and wakes when the radio goes back to sleep (a Fix again after about 7 s here). Never during a Track. The GNSS App says "GNSS is paused" meanwhile. `gnss quiet on|off` on the console.
|
||||
|
||||
It's off by default because GNSS on by default was decided in M2 (Q58), and from M4 the radio listens all the time: then "pause while listening" means GNSS mostly off, which is a decision about position, the clock and Tracks, for M4's design round (issue #23).
|
||||
|
||||
## The Shell (issue #67)
|
||||
|
||||
The console's commands could only be typed on a PC: over USB, or over Wi-Fi with the Debug Console. A device in a bag, or on a network that is down, could not be asked anything. The Shell is an App that runs the same commands on the device's own screen and keyboard.
|
||||
|
||||
### Decisions (design round 2026-10-07)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q204 | **An App, "Shell", in the Launcher**, in every firmware: its commands already work over USB for anyone holding the device. |
|
||||
| Q205 | **Trusted like USB serial**, not like the network: `debug on` and `debug token` work from it, as they do in Settings. The token is never shown. |
|
||||
| Q206 | *Revised the same day, after trying it.* **It shows the replies to its own commands, and only those.** The console knows who each line is printed for (`Console::Origin`): a command run from the Shell prints as the Shell's, and so does what answers it later from another task, which notes who asked and takes it back when it prints (`ls`, `tasks`, `du`, `cp`, `update check`, `sd list`, `screenshot`, `gemini get`). What USB or the Debug Console asked for, and the system's own lines, are not the Shell's. **Ctrl+b shows everything** instead. The first version kept whatever was printed in the ten seconds after a command, which was a guess, and a noisy one. |
|
||||
| Q207 | **Nothing while it's closed.** Open, a 4 KB ring of the console's and up to 4 KB of lines; both go when the App is left, with the list of commands for Tab. |
|
||||
| Q208 | Enter runs the line; Fn with up and down recalls the last 16; Alt with up and down scrolls back. **Tab completes** the command, **every word of it** *(the first only, at first)*: `lora st` gives `lora status`, `gnss track ` lists `start stop`, `key ` its eleven names. The words come from the firmware's `help` text, read as it is written, so a new command completes without a table to keep. *(Added the same day)* **past the command, a path on the SD card**: a folder keeps its slash to go on from, a file completed whole gets a space, several candidates are listed. Whatever the case typed, the name's own is taken, since the card doesn't tell them apart. After a file command the first slash is understood (`cat no` is `/no`). A name with a space in it isn't completed. |
|
||||
| Q209 | *Revised the same day.* **`rm` is Unix's, with a question.** A folder needs `-r`, here and over the consoles (where `rm <folder>` used to remove it with what was in it). In the Shell, a file, or a folder with something in it, is asked about unless `-f` (`-rf`, `-r -f`); an empty folder with `-r` goes without a word; what `rm` would refuse anyway, it refuses itself. Over the consoles nothing is asked: scripts delete as before. **`screenshot [seconds]`** saves the screen as a PNG in `/screenshots`, now or after a pause, since from the Shell "now" is the Shell. `get`, `put`, `coredump get` and `reset` answer `Debug Console only`. *(Added the same day)* **`*` and `?` in a name**, for `ls`, `du`, `rm`, `cp` and `mv`, here and over the consoles: `rm /notes/*.txt`, `cp /gnss/2026-10-0?.gpx /backup`. In the last part of the path only, any case, as the card has it. It is the same command once for each name matched, in the name's order; `cp` and `mv` then need a folder that exists to put them in. Over 64 matches is refused whole, as is none. In the Shell `rm` with a pattern asks **once**, with the count. |
|
||||
| Q210 | Commands are echoed as `> command` into the console, so a session reads the same from afar. **A token being set is not echoed.** |
|
||||
| Q211 | **Not in Safe Mode**, which starts no Apps: issue #77. |
|
||||
| Q212 | Its keys are a table in `app_keys.h`, so the help panel and the website have them; a page in the user guide. |
|
||||
| Q213 | *Added the same day.* **An App's name with a capital opens it:** `Irc`, `Wifi`, `Gnss`, `Gemini`, `Lora`, `Storage`, `Notes`, `Shell`, `System`, `Settings` (the App's id, up to its first dash). From the Shell, without going back to the Launcher, and from the consoles too. The capital says "an App": every command of the firmware is in small letters. Tab completes them. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`ShellApp`** (`src/apps/shell_app.cpp`), with its model host-tested in `lib/apps_model/src/shell_log.h`: lines arriving in pieces, the 4 KB limit, the words of every command out of the `help` text (Apps' names included), and Tab.
|
||||
- **Who a line is for:** `Console::As` marks the calling task as printing for the Shell while it lives, and `Console::origin()` lets a command that answers later carry that to wherever it prints. The console has a second ring for the Shell, which gets the lines marked so, or everything.
|
||||
- **The Shell hands its lines to the main loop**, which runs them like the consoles' commands. See below for why.
|
||||
- **`info` says which App is in front** (`app: Shell`): so that a hand driving the device from afar can look before it types.
|
||||
- **`screenshot`** writes the PNG a row at a time with no buffer: 8-bit indexed colour, the 256 colours of RGB332 as the palette, the pixels in one stored deflate block (`lib/files/src/png_rgb332.h`, 4 tests). 33,383 bytes for the 240 x 135 screen. A Toast says so once it is on the card, so the Toast is never in the picture.
|
||||
- **The `help` text lost its shorthand** (`update check | list | status` is written out), since Tab reads its words from there: `|` between two commands, `on|off` between two words, three spaces before the description. The Shell's own words (`clear`, `quit`, `key`'s names) are added in the same notation.
|
||||
- **Tab on a path** reads the folder on the storage task while the main loop waits, so it is bounded: 400 entries looked at, 24 candidates given back, and `...` after the list when there were more.
|
||||
- **A pattern** is matched in `lib/files/src/file_names.h` (`globMatch`, host-tested), and the names it matches are lined up as so many commands, which the main loop runs one after the other as each finishes. `cancel` empties the line-up. 2000 entries looked at, 64 matches at most.
|
||||
- **Cost:** 21 KB of flash, 96 bytes of static RAM. Open: 7 KB of heap (107.6 KB free before, 100.7 with it open, 106.1 after leaving).
|
||||
|
||||
### What went wrong while building it
|
||||
|
||||
**The device crashed, and the test sent a message to an IRC channel.** The first version ran a command from inside the key handler. Driven from the Debug Console, that is: the main loop, a remote command, `key select`, the App manager, the Shell, `runCommand` a second time, the file command, and `printf` under all of it. The main loop has under 2 KB of stack to spare; `rm` on a folder went past it. The crash report decoded to exactly that chain.
|
||||
|
||||
The device restarted into the Launcher, and the test script, which did not look, went on typing. Its next Enter opened IRC, which connected, and a few lines later it typed "No" into a channel and pressed Enter. One word, sent to real people, that can't be taken back.
|
||||
|
||||
Two changes came of it. The Shell now **queues** its line and the main loop runs it, at the same stack depth as a console's command. And `info` reports the App in front, which the test script now checks before every line it types.
|
||||
|
||||
### Checks on the device (2026-10-07, driven over the Debug Console with `key`)
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Open it from the Launcher, type `ls /`, Enter | The folders are listed |
|
||||
| Tab on `in` | `info install` is shown and the line stays; on `u` it becomes `update `; on `l`, `log ls lora loop` |
|
||||
| Up | The line before comes back |
|
||||
| `screenshot` | `/screenshots/20261007-104357.png`, 33,383 bytes. Fetched and decoded on the PC: 240 x 135, indexed, every chunk's CRC right, the pixels the Shell's screen |
|
||||
| Output | With a Debug Console client connecting and disconnecting for every key, the Shell shows the commands and their replies and nothing else: `info`, `ls /` (answered by the storage task), `tasks` (answered a second later) |
|
||||
| `rm` on a folder, without `-r` | Refused, the folder stays |
|
||||
| `rm -r` on an empty folder | Removed, no question |
|
||||
| `rm -r` on a folder with files | Asks; Cancel leaves it. `rm -rf` removes it without asking |
|
||||
| `rm` on a file | Asks; Delete removes it |
|
||||
| Tab on a path | `ls /no` becomes `ls /notes/`, and again, with one note in it, the note's whole name and a space. `ls /g` lists `gemini/ gnss/`. `cat no` becomes `cat /notes/`. `rm -r /CAP` becomes `rm -r /captures/`. `ls /zz` stays as it is |
|
||||
| Tab past the first word | `lora st` becomes `lora status `; `gnss tr` becomes `gnss track ` and Tab again lists `start stop`; `key ` lists its eleven names; `upd` becomes `update ` and lists `check list status install` |
|
||||
| `ls` with a pattern | `ls /gt/*.txt` the five files, `ls /gt/*2*` the one, `ls /gt3/*6?.txt` ten of seventy. `ls /g*/x`: refused, a pattern goes in the last part |
|
||||
| `cp /gt/file-000?.txt /gt2`, `du /gt2/*1*` | Five copies; one size |
|
||||
| `rm /gt2/*` in the Shell | "The 5 that match /gt2/\*": Cancel leaves the five. `rm /gt3/*6?.txt`, Delete: ten gone, sixty left. `rm -f /gt2/*`: gone without a question |
|
||||
| `rm /gt/*` where one match is a folder | The files go, the folder stays (no `-r`) |
|
||||
| No match, and seventy | `rm: error nothing matches`; `rm: error more than 64 match: a narrower pattern, please`, and all seventy still there |
|
||||
| `No`, Tab, Enter | Completes to `Notes ` and opens Notes. `Shell` from the Debug Console opens the Shell |
|
||||
| `screenshot 4`, then Home | The picture, taken four seconds later, is of the Launcher: the pause works, and the Toast isn't in it |
|
||||
| `quit` | Back to the Launcher, and the memory comes back |
|
||||
| The help panel in the Shell | Its keys, then the ones that work everywhere |
|
||||
|
||||
**Not checked:** Ctrl+b and the Alt scroll, which `key` can't press (no modifiers): the filter itself is host-tested, the key that flips it is not; the real keyboard altogether; the Toast after a screenshot, which was published but not looked at; and `mv` with a pattern and `cancel` in the middle of a line-up, which share their code with `cp` and `rm` but were not run. The main loop's lowest free stack after all of it: 1.8 KB, where it was.
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
+++
|
||||
title = "Look and feel"
|
||||
description = "The interface is consistent and uncrowded: the same thing is done the same way on every screen, and the 135 pixels of height go to content."
|
||||
weight = 90
|
||||
|
||||
[extra]
|
||||
docs = true
|
||||
source = "docs/milestones/U1.md"
|
||||
tag = "U1"
|
||||
+++
|
||||
**Status:** in progress. Shipped: the help key (issue #69) and the key tables the website shares with it (#72), both in **v0.13.0**; the screenshot key (#83, **v0.17.0**). Not started: screen recording (#17), a Launcher of tiles (#9), themes (#10).
|
||||
|
||||
**Goal:** the interface is consistent and uncrowded: the same thing is done the same way on every screen, and the 135 pixels of height go to content.
|
||||
|
||||
## The help key (issue #69)
|
||||
|
||||
Every screen used to say something about its keys, differently: a footer of abbreviations in one place (`c x v:paste r:name d:del n:new i:info s:sort`), a line under a text field in another (`Enter: save \`: cancel`), `Tab: sky` in a corner, and nothing at all in several. About 30 such strings, each costing a line of a small screen, and none of them complete.
|
||||
|
||||
### Decisions (design round 2026-10-07)
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q196 | **Fn+h, on every screen,** text fields included (Fn is held, so nothing is typed). **`?` too, outside Text Entry.** |
|
||||
| Q197 | It opens **a panel over the content area**, titled with where you are: the screen's own keys, then an "Everywhere" group (Back, Home, the arrows, the help key). The arrows scroll it; any other key closes it and is not passed on. |
|
||||
| Q198 | **Each App answers "what are your keys right now?"** for the state it is in; pages, viewers, dialogs and text fields answer for themselves, with shared lists for dialogs, lists and text entry. The lists are constants; the panel's rows exist only while it is open. |
|
||||
| Q199 | **Every hint that names a key goes,** text fields included. What stays is state: `REC 12 points`, `LOG 42`, `sort:signal`, `typing`/`saved`, what is waiting to be pasted, the Sweep's floor. Messages were reworded where they named a key ("v pastes a copy of…" is "Copied …: paste it where you like"). |
|
||||
| Q200 | **The first-start Setup keeps its hints,** and is the one place that does: someone in their first minute doesn't know the help key yet. It tells them about it on its first and last screens. |
|
||||
| Q201 | **Loud everywhere else:** the user guide opens with it, the FAQ has it first, and a device set up before this firmware gets one Toast, once: "Fn+h: the keys of any screen". |
|
||||
| Q202 | `key help` over the consoles. Generating the website's key tables from the same lists is a follow-up, not this issue. |
|
||||
| Q203 | The key and the panel first, host-tested; then one App at a time, declaring its keys and losing its hints in the same step; then every screen looked at on the device. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`Key::Help`** from the key mapper: Fn+h in both modes, `?` only outside Text Entry (`lib/input`, 3 tests).
|
||||
- **`App::help()` and `App::helpTitle()`** (`lib/core/src/app.h`), `KeyHelp` rows and `HelpModel` (`key_help.h`). The **App manager** opens the panel, appends the "Everywhere" group, and while it is open takes every key: nothing reaches the App, Home included. It closes when the App changes (5 tests).
|
||||
- **Every App declares its keys by state:** the Launcher, IRC (chat, settings, a field), Wi-Fi Tools (4 views), GNSS, Gemini (page, saved page, address, answer, dialogs), the LoRa Scanner (4 views), Storage (browse, details, a name, the viewer's 6 modes, the editor, Maintenance, busy), Notes (list, editor, a file name), System (5 views), Settings (menu, text, choice, and the Wi-Fi, Firmware and Debug Console pages with their own states), Setup and the widget demo.
|
||||
- **The hints are gone** from all of them. The footers that remain say state only.
|
||||
- **It costs** 8.5 KB of flash and 40 bytes of static RAM.
|
||||
|
||||
### Checks
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| Host tests | 476 pass (468 before) |
|
||||
| On the device, `key help` and a screenshot | The Launcher and all nine Apps, and these states: Storage scrolled, System's tasks, a Settings text field, Wi-Fi Tools' networks. The panel is titled with the scope, lists the right keys, scrolls, and closes on Tab |
|
||||
| The one-time Toast | `notification: Fn+h: the keys of any screen` on the first start after the update |
|
||||
|
||||
### One source for the device and the website (issue #72)
|
||||
|
||||
The lists first lived in each App's `help()`, as code. They are now **data, in one file**: `lib/core/src/app_keys.h`, 52 constant tables, each under a comment `// id: Title`. An App's `help()` picks the table of the state it is in. `site/tools/gen_dev_docs.py` reads the same file and writes `site/data/keys.toml`; the `keys` shortcode shows a screen's tables on its guide page, and `/guide/keys/` shows all of them. The Site job fails when the data file is out of date, or when a page asks for a table that doesn't exist, and it now runs when `app_keys.h` changes. A key added to an App shows up on the website without anyone editing a page.
|
||||
|
||||
Three rows lost their second wording on the way, since a table is constant: GNSS's `Tab` and `r`, and the Scanner's `c`, now say both things they do ("record a Track, or stop it") instead of the one that applies. The guide pages keep their written tables too, where they say more than a key list can; those can still drift, and the generated ones under them are the reference.
|
||||
|
||||
**Not checked:** the real Fn+h and `?` on the keyboard (the mapper is host-tested; the device was driven with `key help`); the Setup screens, which only a device that was never set up shows, so their new text has not been seen on a screen; and the states that need something to happen first (a dialog, a copy in progress, a Gemini prompt, a packet's details): their lists were read against the key handling, not looked at.
|
||||
|
||||
## The screenshot key (issue #83)
|
||||
|
||||
A screenshot could only be taken by typing `screenshot` in the Shell, where "now" is a picture of the Shell.
|
||||
|
||||
- **Fn+p, on every screen**, text fields included, saves the screen as it is (a dialog, the help panel or a Toast if one is showing) as a PNG in `/screenshots`, the way the Shell's command does. A Toast says so once the file is written, so it is never in the picture.
|
||||
- **It never reaches an App.** `Key::Screenshot` comes out of the key mapper and is handled before the App manager, so the help panel stays open and a dialog keeps its selection.
|
||||
- **Not on Settings > Debug Console:** that page shows the token, and a picture of it is a copy of the token in a file. `App::showsSecret()` says so, and the key answers with a Toast instead.
|
||||
- **No card:** a Toast says so.
|
||||
- No setting to switch it off: Fn with a letter isn't pressed by accident.
|
||||
- It is in the "Everywhere" group of the help panel, and so in the website's key tables. `key shot` presses it over the consoles.
|
||||
|
||||
**Checked on the device** (2026-10-07, with `key shot`): in the Launcher, a 33,383-byte PNG appears in `/screenshots` and is the Launcher; with the help panel open, the picture is of the panel (which now lists Fn p) and the panel stays open; on Settings > Debug Console, no file is written; back on the Settings list, one is. The test pictures were removed.
|
||||
|
||||
**Not checked:** the real Fn+p on the keyboard (the mapper is host-tested); the two Toasts that refuse, which weren't looked at (one of them is on the page that mustn't be photographed); a device with no card.
|
||||
@@ -8,7 +8,7 @@ docs = true
|
||||
source = "docs/milestones/W1.md"
|
||||
tag = "W1"
|
||||
+++
|
||||
**Status:** phases 1 to 3 (home, Install and Downloads; the user guide; how-tos and the FAQ) and the devlog are live at roro9stack.net; phase 4 (the developer docs) is in a pull request. Issue #12.
|
||||
**Status:** live at roro9stack.net: the home, Install and Downloads pages, the user guide, the how-tos and the FAQ, the developer docs and the devlog (issue #12), published by CI since issue #79, with a search since issue #60. Not started: a Gemini mirror (#57), a French translation (#58), the docs of each version (#59).
|
||||
|
||||
**Goal:** a public home for the project at **roro9stack.net**, separate from the blog (stories) and from Gitea (developers): what it is, how to install it, how to use each App, and the docs.
|
||||
|
||||
@@ -29,7 +29,7 @@ The home page was designed on a canvas in a Claude chat (a dark and a light them
|
||||
| Q175 | The site lives in this repository, in `site/`, so the documentation is built from `docs/`, `CONTEXT.md` and the README instead of being copied. |
|
||||
| Q176 | **Zola,** like the blog. The design becomes a template, its tokens CSS custom properties. Dark and light follow the visitor's setting, with a visible switch. No JavaScript except the flasher's. |
|
||||
| Q177 | Domain: **roro9stack.net.** The blog stays at experiments.twis.la. |
|
||||
| Q178 | **Publishing is the blog's way:** the web server pulls `main` and runs `zola build`; that part is the maintainer's. Changes reach `main` through pull requests as everywhere. **CI is split:** a dedicated `site` job builds the site (`zola build`) when `site/`, `docs/`, `README.md` or `CONTEXT.md` change, and the firmware tests and builds skip a change that touches nothing else. A change that touches both runs both. |
|
||||
| Q178 | **Publishing is the blog's way:** the web server pulls `main` and runs `zola build`; that part is the maintainer's. *(Since issue #79, CI asks the server to do it: see "Published by CI" below.)* Changes reach `main` through pull requests as everywhere. **CI is split:** a dedicated `site` job builds the site (`zola build`) when `site/`, `docs/`, `README.md` or `CONTEXT.md` change, and the firmware tests and builds skip a change that touches nothing else. A change that touches both runs both. |
|
||||
| Q179 | Phases, each its own pull request: **1.** the CI split, the home page, an Install page with the browser flasher, downloads and the changelog. **2.** a user guide page per App. **3.** how-tos and the FAQ. **4.** developer docs generated from the repository. |
|
||||
| Q180 | **A browser flasher** (ESP Web Tools), **without copying the firmware.** Caddy, in front of Gitea, adds `Access-Control-Allow-Origin: https://roro9stack.net` (and `Vary: Origin`) to GET and HEAD on `/twisla/roro9stack/releases/download/*` and `/api/v1/repos/twisla/roro9stack/releases*`: both are public already. The Install page asks the API for the latest release in the browser, finds the asset ending `-factory.bin`, and gives ESP Web Tools a manifest built on the spot, so it offers a new release as soon as it exists, with no rebuild. The library is **vendored** into `site/static/` (Apache-2.0), not loaded from a CDN. The file's SHA-256 is shown on the page. Chrome or Edge on a desktop only; other browsers, and visitors without JavaScript, get the `esptool` steps on the same page. |
|
||||
| Q181 | Docs for the latest version only. The changelog is the Gitea releases, read at build time. |
|
||||
@@ -133,3 +133,60 @@ Not one of the planned phases: the blog's seven roro9stack posts, imported into
|
||||
- **Left out on purpose:** the M0 and M1 milestone documents and `CONTEXT.md` (the glossary) describe Wi-Fi monitoring, which the site does not publish. They stay in the repository.
|
||||
- **The Debug Console pages were written against the source and the live console:** the protocol (the token line, the banner, the 4 KB backlog, one client, 8 queued commands, 240-byte lines, `denied` after a second) and the replies shown were checked on a Debug Build, v0.11.0-3, over Wi-Fi. Not run: `crash abort`, `crash wdt` and Safe Mode, which are described from ADR 0005 and the code.
|
||||
- **Found while writing it:** the README's table lacked the `gnss` commands (rows added); piping commands into `rdbg.py` returns before the replies unless the input stays open (documented, not changed); `update install` on a Debug Build needs `force` (documented).
|
||||
|
||||
## Published by CI (issue #79, design round 2026-10-07)
|
||||
|
||||
Q178 left publishing to the maintainer: a merge, then a command typed on the web server, each time.
|
||||
|
||||
| # | Decision |
|
||||
|---|---|
|
||||
| Q214 | **A plain ed25519 key with a forced command**, not a certificate: one line in the web server user's `authorized_keys`, `restrict,command="/full/path/to/the/refresh"`. `restrict` takes away the terminal and every forwarding. A certificate could carry the same and an expiry date, at the price of a CA to keep and a key to sign again each time: too much for one key and one command. |
|
||||
| Q215 | **The CI sends no command.** The server runs the forced one whatever is asked for, so there is nothing to keep secret about it and nothing a leaked key could choose. The full path is written once, on the server (a command over SSH doesn't get the user's login `PATH`). |
|
||||
| Q216 | Four secrets: `SITE_DEPLOY_KEY`, `SITE_DEPLOY_HOST` (or `host:port`), `SITE_DEPLOY_USER`, and **`SITE_DEPLOY_KNOWN_HOSTS`**, the server's host key: the job connects to that server or to nothing. None of them is in the repository, which is public. |
|
||||
| Q217 | `from="<the runner's address>"` on the same line: the key works from the runner only. |
|
||||
| Q218 | **The last step of the Site workflow**, after the build and the checks, on a push to `main` only. A pull request never reaches it, and the secrets are given to that step alone. |
|
||||
| Q219 | **After a release too.** The Install page asks Gitea for the latest release when it is opened, but the home page and Downloads read it when the site is built: so the release workflow refreshes the site once the release is published. |
|
||||
| Q220 | A refresh that fails makes the run red, with what the server's script printed: it has to exit with an error when it fails. |
|
||||
| Q221 | Two refreshes at once are the server script's to refuse or queue (`flock`). |
|
||||
| Q222 | The key is a file only while the step runs, in the job's container, as the signing key is. |
|
||||
|
||||
### As built
|
||||
|
||||
- **`scripts/site_refresh.sh`** is what both workflows run: it writes the key and the host key to a temporary folder, connects with no configuration but its own line (`-F none`, strict host key checking, that one key, no command), and removes them. With none of the four secrets it does nothing and says so (a fork, or a repository without them); with only some it fails.
|
||||
- **`scripts/site_deploy_keygen.sh`** makes the key pair once, in `~/.config/roro9stack/`, and prints the `authorized_keys` line and what goes in each secret. It never prints the private key.
|
||||
- **The server's script** should start like this, for Q220 and Q221:
|
||||
|
||||
```sh
|
||||
#!/bin/sh
|
||||
set -e
|
||||
exec 9>/tmp/rororefresh.lock
|
||||
flock -w 120 9
|
||||
```
|
||||
|
||||
### Checks (2026-10-07, against an SSH server in a throwaway container)
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| The refresh | The forced command runs as the server's user; the script ends with `site refresh: done` |
|
||||
| The same key, asking for `id; cat /etc/passwd` | The refresh runs instead; what was asked for is only handed to it as text |
|
||||
| A terminal | Refused: `PTY allocation request failed` |
|
||||
| `scp` with the key | Nothing is copied |
|
||||
| Another host key in the secret | `Host key verification failed`, the run fails, nothing is sent |
|
||||
| The server's script exits with an error | So does the step |
|
||||
| No secrets at all; only one of the four | Does nothing and says so; fails and says which are needed |
|
||||
|
||||
**Not checked:** the real web server and the runner, which wait for the key to be installed: whether the runner reaches the server's SSH port is the first thing the first run will tell. Port forwarding, which `restrict` switches off, was not tried. `from=` was not tried either.
|
||||
|
||||
## Search (issue #60)
|
||||
|
||||
A search over the documentation: the user guide, the how-tos, the questions and answers, and the developer docs. Not the devlog.
|
||||
|
||||
- **The index is the search page itself** (`/search/`, `templates/search.html`): one list item for each page and for each `##` heading of it, with that part's text, written by Zola from the pages' own content when the site is built. Nothing is fetched and nothing typed leaves the browser, so the Content-Security-Policy needs nothing new, and the web server still only runs `zola build`.
|
||||
- **Without JavaScript** the page is a list of every page and heading of the documentation, each a link.
|
||||
- **With it**, `js/search.js` filters and ranks the items as you type: every word has to be in the part; a word in a heading counts for most, the words side by side for more than scattered, and the user guide, the how-tos and the FAQ come before the developer docs, the milestones last. A result links to its heading, with the text around the match.
|
||||
- **The content pages get no script for it:** the navigation has a link, and the index pages of the guide, the how-tos and the developer docs have a box that is a plain form to `/search/?q=`.
|
||||
- **Size:** about 245 items, about 310 KB of HTML, under 100 KB compressed, loaded only by who searches.
|
||||
|
||||
**Checked** in Chromium with the production Content-Security-Policy on every response, no violation: "probation" (the guide's "Probation and Rollback" first), "safe mode", "rm -r", "how big can a note" (the FAQ's question first), a word that isn't there; typing, following a result to its heading, the box on the guide's index, 390 px wide with no sideways scroll, and JavaScript off. `check_site.py` follows every link of the page, so an index entry can't point at a heading that doesn't exist.
|
||||
|
||||
**Not checked:** other browsers, and a screen reader.
|
||||
|
||||
|
After Width: | Height: | Size: 2.4 KiB |
@@ -0,0 +1,266 @@
|
||||
+++
|
||||
title = '''It was off'''
|
||||
description = '''roro9stack gets a website, and writing down how its Debug Console works shows that only one person could ever use it. So the Debug Build is retired: one firmware, with the console in it, off until its owner switches it on. Then an evening of chasing a console that wouldn't come back and a memory leak, neither of which existed.'''
|
||||
date = 2026-10-07T00:30:00+02:00
|
||||
|
||||
[extra]
|
||||
topics = '''ESP32-S3 · Debugging · Security'''
|
||||
read_label = '''Read what was off →'''
|
||||
uid = '''<b>debug:</b> on, token set, client connected'''
|
||||
dek = "A day of writing for [roro9stack](/devlog/roro9stack/), my firmware for the M5Stack Cardputer: a website, a user guide, and developer docs about the thing I like best in it, a console over Wi-Fi. Documenting it properly made one fact hard to miss: nobody else could have it. Fixing that deleted more than it added, and then cost me an evening looking for two bugs that turned out to be a switch in the Off position and TCP minding its own business."
|
||||
byline = '''designed by interrogation, round twelve: fourteen questions thrown away, eight kept'''
|
||||
|
||||
[extra.sign]
|
||||
label = "Tokens leaked by the feature built to protect them"
|
||||
note = "In a screenshot, taken over the console, of the one page that shows the token."
|
||||
count = "1"
|
||||
tone = "red"
|
||||
|
||||
[[extra.cast]]
|
||||
name = "The Debug Build"
|
||||
role = "retired, v0.3.0 to v0.11.0"
|
||||
text = "The same firmware plus a console over Wi-Fi, with its builder's token compiled in. Which is why it could never be published, and why the firmware I tested was never the one I released."
|
||||
|
||||
[[extra.cast]]
|
||||
name = "Port 3232"
|
||||
role = "the Update Service, in every build since v0.3.0"
|
||||
text = "Listens on the network in release builds, guarded by a signature. The counter-example to \"in a release, nothing listens\" that had been sitting there all along."
|
||||
|
||||
[[extra.cast]]
|
||||
name = "The token"
|
||||
role = "100 bits, 20 characters"
|
||||
text = "Made by the device, shown on one page of Settings and nowhere else. Read off a 240-pixel screen by a human, which is where the trouble started."
|
||||
|
||||
[[extra.cast]]
|
||||
name = "The dialog"
|
||||
role = "\"Switch it on?\""
|
||||
text = "Opens with Cancel selected, as a question about a remote control should. Enter, Enter: still off. Works exactly as designed."
|
||||
|
||||
[[extra.cast]]
|
||||
name = "TIME_WAIT"
|
||||
role = "two minutes, per closed connection"
|
||||
text = "What TCP does with a connection it has closed, in case a late packet turns up. About 270 bytes each. Looks exactly like a leak if you only watch for one minute."
|
||||
+++
|
||||
|
||||
## TL;DR
|
||||
|
||||
- **roro9stack has a website:** [roro9stack.net](/), with an Install page that flashes a Cardputer from the browser, a [user guide](/guide/), [how-tos](/howto/), an [FAQ](/faq/), [developer docs](/dev/) generated from the repository, and this devlog, which moved here.
|
||||
- **The Debug Build is gone.** There is one firmware (**v0.12.0**), and the Debug Console is in it: **off** until you switch it on in Settings, with a token the device makes itself.
|
||||
- **The token never crosses the network.** The device sends a challenge, the client answers with an HMAC. Five wrong answers close the console for a minute.
|
||||
- **It costs** 30 KB of flash and 88 bytes of RAM over the old release build. Off, nothing listens and nothing is allocated.
|
||||
- Then I spent an evening on **a console that wouldn't come back on** (it was off) and **a memory leak of 230 bytes per reopening** (it was TCP). Both produced a real fix on the way, and one design change I had to undo.
|
||||
- 468 host tests, 12 more than last time. The decision is [ADR 0010](/dev/decisions/0010-debug-console-in-every-build/).
|
||||
|
||||
## The cast
|
||||
|
||||
{{ cast() }}
|
||||
|
||||
## A site, in four phases
|
||||
|
||||
The [last post](/devlog/roro9stack-f1-r1/) ended with a device that installs its own releases. That makes it something another person could use, and another person needs somewhere to start that isn't a Gitea README. So the first half of the day was a website: a home page, an Install page, then a guide to each App, how-tos, an FAQ, and developer docs.
|
||||
|
||||
Three things from that half are worth keeping.
|
||||
|
||||
**Flashing from the browser, without a copy of the firmware.** The Install page uses ESP Web Tools and Web Serial. The obvious way is to copy the firmware image next to the page; then every release needs a rebuild of the site. Instead the page asks Gitea's API for the latest release, in the browser, and hands the flasher a manifest it builds on the spot. That only needed one header: Caddy, in front of Gitea, adds `Access-Control-Allow-Origin: https://roro9stack.net` to the release downloads and to the releases API, which are public anyway. A new release shows up on the page the moment it exists.
|
||||
|
||||
**"Generated from the repository" met Zola.** The developer docs were to be built from `docs/`, the README and the firmware's own `help` text, not copied by hand. Zola refuses to read a file outside its own folder, and it resolves symlinks before deciding, so that door is closed too. So a small script writes those pages, they are committed, and CI fails when one is out of date. The command reference is the part I like: it is parsed from the `kHelp` string in `main.cpp`, so the site can't describe a command the firmware doesn't have.
|
||||
|
||||
**The posts you're reading had `<style>` in them.** This devlog came over from my blog, seventeen diagrams included, each an inline SVG with its own `<style>` block. The site's Content-Security-Policy is `style-src 'self'`, which refuses exactly that. The rules moved into a stylesheet and the `style="…"` attributes became classes. Tested in a real browser with the production policy on every response: no violations.
|
||||
|
||||
The site says only what the firmware does today, which meant writing the same sentence several times: the mesh messenger isn't built. The radio listens. It doesn't talk yet.
|
||||
|
||||
## Documenting a feature only I could use
|
||||
|
||||
The part of the developer docs I cared most about is the Debug Console: the serial console over Wi-Fi, with key presses, screenshots, file transfer and crash dumps. I wrote six pages on it, checked the protocol against the live device, and by the end had described, in detail, a feature with this property:
|
||||
|
||||
> A Debug Build carries its builder's token, so Debug Builds are never published.
|
||||
|
||||
Everything in those pages was for whoever builds the firmware. That's me.
|
||||
|
||||
The issue I filed was the obvious patch: make the token configurable, so that Debug Builds can be published. The design round for it had fourteen questions. Where does a published Debug Build go, so that devices in the field don't install it by accident (v0.11.0 takes the last `.ota` it finds in a release)? What is its tag called, so that CI doesn't start itself again? How does a device that runs one get the next?
|
||||
|
||||
Then one question from the other side of the table replaced all fourteen: *why have Debug Builds at all?*
|
||||
|
||||
## The argument that was never whole
|
||||
|
||||
[ADR 0004](/dev/decisions/0004-debug-console-in-debug-builds/) kept the console out of release builds with one sentence, which I was rather proud of:
|
||||
|
||||
> A console that runs commands is a remote control: in a release build, nothing listens.
|
||||
|
||||
Except something does. The Update Service has listened on TCP 3232 in every build since v0.3.0, guarded by a signature. "Nothing listens" was never true; "nothing listens without a lock on it" was. A console that is off by default, behind a secret the device made itself, is the same trade.
|
||||
|
||||
And the split had been charging rent the whole time:
|
||||
|
||||
- **What I tested wasn't what I shipped.** I lived on Debug Builds. Releases were a different binary that nobody ran before it was published.
|
||||
- **Rules that only protected the console.** A Debug Build refused to install a release (it would have lost its console), so there was `update install … force` to do it anyway, and advice about which firmware to keep in the other slot.
|
||||
- **Versions ending in `+debug`** that had to compare equal to their release.
|
||||
- **Two firmwares built by CI** on every pull request and every tag.
|
||||
|
||||
One firmware deletes all four.
|
||||
|
||||
## What off means
|
||||
|
||||
The new argument only holds if "off" is as good as "not compiled in". So:
|
||||
|
||||
- **Off is the default,** and what a missing or damaged setting falls back to.
|
||||
- **Off, nothing exists:** no socket, no task, and no 4 KB buffer, which used to be a static array and is now allocated when the console is switched on.
|
||||
- **On needs someone at the device,** or on the USB cable. Over the console itself, `debug on` and `debug token` answer `over USB serial only`: it can switch itself off, not open itself wider.
|
||||
|
||||
{% table() %}
|
||||
| Build | Flash | Static RAM |
|
||||
|---|---|---|
|
||||
| The release build, before | 1,851,387 | 54,612 |
|
||||
| The Debug Build, before | 1,874,887 | 58,780 |
|
||||
| **The one firmware** | **1,881,799** | **54,700** |
|
||||
{% end %}
|
||||
|
||||
Thirty kilobytes of flash, and 88 bytes of RAM, for a console in every device. The Debug Build's extra 4 KB of RAM was the buffer, sitting there whether anyone connected or not.
|
||||
|
||||
## A token you can read, and one you can't steal
|
||||
|
||||
The token used to be 32 hex digits in a file on my laptop, compiled in. Now the device makes it, the first time the console is switched on: 100 bits from the hardware generator, as twenty characters of Crockford's base32, the alphabet that leaves out I, L, O and U because people misread them.
|
||||
|
||||
The old login sent the token as the first line of a plain TCP connection. That was fine for a secret that lived on one laptop and one device on one network. It's not fine for a secret in every device, on whatever Wi-Fi its owner uses: mine is called `knbg-guests`, and anyone with the Wi-Fi password can watch it.
|
||||
|
||||
So the token no longer travels:
|
||||
|
||||
{% code(caption="The whole login. A recorded answer is no use for the next challenge.") %}
|
||||
```
|
||||
device: roro9stack debug console, challenge 3f9a…c1 (16 random bytes)
|
||||
client: HMAC-SHA256(key = token, message = those bytes), in hex
|
||||
device: roro9stack v0.12.0 debug console. 'help' lists the commands. Backlog follows.
|
||||
```
|
||||
{% end %}
|
||||
|
||||
HMAC is twenty lines on top of the SHA-256 the firmware already had for update files, checked against the RFC's vectors and against the same vector as the Python client. Five wrong answers in a row and the console answers `locked` to everyone for a minute:
|
||||
|
||||
{% code(caption="Six tries with a wrong token, then the right one.") %}
|
||||
```
|
||||
try 1 at 1s: wrong token
|
||||
try 2 at 3s: wrong token
|
||||
try 3 at 4s: wrong token
|
||||
try 4 at 5s: wrong token
|
||||
try 5 at 6s: wrong token
|
||||
try 6 at 6s: closed for a minute after too many wrong tokens
|
||||
right token: closed for a minute after too many wrong tokens
|
||||
```
|
||||
{% end %}
|
||||
|
||||
This breaks every old client. I'm the only user, and v1 is a long way off; this is exactly when to break things.
|
||||
|
||||
## Three ways it went wrong
|
||||
|
||||
### I couldn't read my own token
|
||||
|
||||
First login, on the real device: `wrong token`. The firmware was right. I had typed one character wrong, reading a 20-character token drawn in bold at the normal size, where 8 and B, 5 and S, 2 and Z are a matter of opinion.
|
||||
|
||||
The token is now drawn at twice the size, in regular weight, on two lines. And what I should have done from the start: Crockford's rule says that if someone types an O, an I or an L, they meant 0 or 1, because the alphabet doesn't have those letters. Both ends now apply it.
|
||||
|
||||
### The screenshot
|
||||
|
||||
To check the new layout I did what I always do: took a screenshot over the console.
|
||||
|
||||
{{ figure(src="token.png", alt="The Cardputer's Settings, Debug Console page at 2x: Debug Console On, Connect to 10.39.39.12:2323, New token, Type a token, and below them a token in large type on two lines, T49R-HQXB-NDJX then BNSB-XSWD. The Status Bar shows DBG in blue.", width=480, height=270, caption="The page that shows the token \"and nowhere else\", in a file on my laptop. This token was replaced within the hour.") }}
|
||||
|
||||
The documentation I had written an hour earlier says the token is shown on one page and never printed anywhere. And there it is, in a PNG. No escalation: whoever can take a screenshot already has the token. But a secret now sits in a file because of a habit, and the sign at the top of this post is for that. The docs now say it in so many words: a screenshot of that page is a copy of the token.
|
||||
|
||||
### The console that wouldn't come back
|
||||
|
||||
The last test was `debug off`, sent over the console: the client is dropped, the port refuses connections. Good. I switched it back on at the device, made a new token, and tried to log in.
|
||||
|
||||
Connection refused.
|
||||
|
||||
The device was up. Its update port answered. And it stayed refused across another firmware push and a restart. So, I reasoned, the console fails to come back once it has been closed, and I went looking:
|
||||
|
||||
- **The framework's `begin()` returns nothing.** `NetworkServer::begin()` can fail at `socket()`, `bind()` or `listen()` and says nothing either way; my code set `listening = true` and never asked. A real hole. Fixed: it asks, complains on the console, and tries again every two seconds.
|
||||
- **I couldn't reproduce it from my desk.** Switching the console on is for the device and the cable only, by design, so the one path I wanted to test was the one I had made unreachable. Hence `debug off <seconds>`: the console closes and comes back by itself after the pause. It can't open anything wider (same state, same token), and it makes closing and reopening something a script can do twenty times.
|
||||
|
||||
Then I went and looked at the device's screen.
|
||||
|
||||
It said **Off**.
|
||||
|
||||
The "Switch it on?" dialog opens with **Cancel** selected. Enter on the row, Enter on the dialog: nothing changes, and the page says so in plain letters that I hadn't read. I had filed a bug against a switch for being in the position I left it in.
|
||||
|
||||
### The leak that was TCP
|
||||
|
||||
With `debug off <seconds>` I could finally hammer the path. It came back every time. And free memory fell by about 230 bytes per cycle.
|
||||
|
||||
A leak in code I had just written, in a firmware where 836 bytes was [once all that was left](/devlog/roro9stack-f1-r1/). I had a suspect: the console's task is deleted when the console goes off and made again when it comes back. Twenty plain logins leaked nothing, so it wasn't connections. I changed the design: the task stays, asleep, while the console is off. That breaks the promise that off means nothing is there, but a leak is worse.
|
||||
|
||||
It leaked exactly as much as before.
|
||||
|
||||
So I did what I should have done first, and measured for longer:
|
||||
|
||||
{% table() %}
|
||||
| | Free heap |
|
||||
|---|---|
|
||||
| Before | 107,248 |
|
||||
| Right after 15 closings and reopenings | 103,180 |
|
||||
| 60 s later | 105,008 |
|
||||
| 120 s later | 107,004 |
|
||||
| 240 s later | 106,924 |
|
||||
{% end %}
|
||||
|
||||
All of it comes back. When the device closes a connection, TCP keeps it for two minutes in case a late packet arrives, and each one holds about 270 bytes. Fifteen cycles in a minute and a half look like a leak; fifteen cycles and a cup of tea look like nothing at all.
|
||||
|
||||
The design change went back out. Off means the task is gone too, as decided.
|
||||
|
||||
## What held
|
||||
|
||||
{{ figure(src="dbg.png", alt="The Cardputer's Launcher at 2x, listing IRC, Wi-Fi Tools, GNSS, Gemini, LoRa Scanner, Storage, Notes, System and Settings. The Status Bar shows DBG in blue between the GNSS and Wi-Fi indicators.", width=480, height=270, caption="`DBG` in the Status Bar: there while the console listens, bright while someone is connected. Taken over the console, so: bright.") }}
|
||||
|
||||
Checked on the device, not just in tests:
|
||||
|
||||
- **Off by default.** Pushed over a Debug Build, the new firmware came up with port 2323 refusing connections.
|
||||
- **The setting survives updates.** Four pushes later the console came back by itself each time.
|
||||
- **Safe Mode has the console.** Three `crash abort` in a row, and the device started in Safe Mode with 178 KB free and the console reachable. `ls /` answered `not available in Safe Mode`, and the crash decoded to the line of `main.cpp` with `abort()` on it.
|
||||
- **25 closings and reopenings,** each back a second after its pause.
|
||||
- **More memory than before.** 108 KB free with the console on and a client connected, against 104 KB on the Debug Build.
|
||||
|
||||
And what I didn't check: `scripts/flash.sh --debug`, which now sets a device up over USB, with no cable attached to the VM that night; typing a token of my own on the device; and the notification on the screen during a lockout, which nobody connected can see, the console being closed.
|
||||
|
||||
## By the numbers
|
||||
|
||||
{% table() %}
|
||||
| | |
|
||||
|---|---|
|
||||
| Design questions written, then thrown away | 14 |
|
||||
| Design questions kept | 8 |
|
||||
| Host tests | 468 |
|
||||
| Bits in a token | 100 |
|
||||
| Wrong answers before the console closes | 5 |
|
||||
| Flash it costs every device | 30,412 bytes |
|
||||
| RAM it costs every device, off | 88 bytes |
|
||||
| Bytes TCP holds for a closed connection, for two minutes | about 270 |
|
||||
| Pages on the website | 66 |
|
||||
| Bugs found in the console that evening | 1 (the silent `begin()`) |
|
||||
| Bugs I thought I'd found | 3 |
|
||||
{% end %}
|
||||
|
||||
## Where it stands
|
||||
|
||||
{% steps() %}
|
||||
1. ~~M0 and M1: the skeleton, Wi-Fi, IRC, Wi-Fi Tools.~~ v0.1.0 to v0.2.1, [the first post](/devlog/roro9stack/).
|
||||
|
||||
2. ~~Updates and debugging over the air.~~ v0.3.0, [Look, no cables](/devlog/roro9stack-ota/).
|
||||
|
||||
3. ~~M2: GNSS.~~ v0.4.0, [Seventeen satellites](/devlog/roro9stack-gnss/).
|
||||
|
||||
4. ~~G1: Gemini.~~ v0.5.0, [A browser in the RAM IRC left over](/devlog/roro9stack-gemini/).
|
||||
|
||||
5. ~~M3: the LoRa radio, listening.~~ v0.6.0, [The loudest thing it hears is itself](/devlog/roro9stack-lora/).
|
||||
|
||||
6. ~~S1: the card, fixed addresses, the System App.~~ v0.6.1 to v0.8.1, [One byte too early](/devlog/roro9stack-s1/).
|
||||
|
||||
7. ~~F1 and the start of R1: files, notes, signed releases, updates from Gitea.~~ v0.9.0 to v0.11.0, [836 bytes](/devlog/roro9stack-f1-r1/).
|
||||
|
||||
8. ~~W1: the website.~~ You're on it.
|
||||
|
||||
9. ~~One firmware, with the Debug Console in it.~~ v0.12.0, this post.
|
||||
|
||||
10. Next: notes of any size, a shell on the device itself for the same commands, and one help key everywhere instead of a hint line on every screen. And M4, the mesh, which still wants a second node.
|
||||
{% end %}
|
||||
|
||||
{% signoff() %}
|
||||
I wrote a feature whose whole point is that it's off until you switch it on, and then spent an evening debugging it for being off. The switch works. So does TCP. The only thing that leaked was the token, and I did that myself.
|
||||
{% end %}
|
||||
|
After Width: | Height: | Size: 4.7 KiB |
|
After Width: | Height: | Size: 132 KiB |
@@ -0,0 +1,188 @@
|
||||
+++
|
||||
title = '''Press w'''
|
||||
description = '''I asked whether roro9stack should get an FTP, SFTP or WebDAV server, to move files to and from my phone. The answer was none of them: a web page. Less than an hour later I pressed one key on the Cardputer, opened my phone's browser, and my SD card was in it. It works, and I'm still grinning.'''
|
||||
date = 2026-10-08T00:30:00+02:00
|
||||
|
||||
[extra]
|
||||
topics = '''ESP32-S3 · HTTP · Files'''
|
||||
read_label = '''Read how the card got into my phone →'''
|
||||
uid = '''<b>share:</b> on at http://172.16.42.25/'''
|
||||
dek = "A short one, written straight after it worked, because I'm too pleased to wait. [roro9stack](/devlog/roro9stack/), my firmware for the M5Stack Cardputer, can now hand its SD card to any browser on the same Wi-Fi: no cable, no app, no computer. One key, one code, done."
|
||||
byline = '''one question asked, the wrong three answers offered, a fourth taken'''
|
||||
|
||||
[extra.sign]
|
||||
label = "Apps installed on the phone to make this work"
|
||||
note = "A browser was already there."
|
||||
count = "0"
|
||||
|
||||
[[extra.cast]]
|
||||
name = "The question"
|
||||
role = "\"FTP, SFTP or WebDAV?\""
|
||||
text = "Three ways to serve files, all of which want an app on the phone. I'd have picked one and been mildly unhappy with it for months."
|
||||
|
||||
[[extra.cast]]
|
||||
name = "The w key"
|
||||
role = "in the Storage App"
|
||||
text = "Starts a web server and shows where it is. Back stops it. That is the entire user interface on the device."
|
||||
|
||||
[[extra.cast]]
|
||||
name = "The code"
|
||||
role = "six digits, new every time"
|
||||
text = "On the device's screen and nowhere else. Type it in the page and you're in. Five wrong tries and the door stays shut for a minute."
|
||||
|
||||
[[extra.cast]]
|
||||
name = "The page"
|
||||
role = "5.4 KB, one file"
|
||||
text = "A list of what's on the card, an Upload button, a New folder button, and a Delete next to every row. Served from the firmware's own flash."
|
||||
|
||||
[[extra.cast]]
|
||||
name = "The storage task"
|
||||
role = "the only one allowed to touch the card"
|
||||
text = "Every byte in either direction goes through it, 8 KB at a time. It was there long before this and didn't need to learn anything new."
|
||||
+++
|
||||
|
||||
## TL;DR
|
||||
|
||||
- **Press `w` in the Storage App** (**v0.18.0**), scan the QR code with a phone on the same Wi-Fi, and the SD card is a web page: list, download, upload, new folder, delete.
|
||||
- **Nothing to install**, on the phone or anywhere else. That was the whole point.
|
||||
- **It runs only while that screen is open.** A six-digit code, new each time, keeps the rest of the network out.
|
||||
- **It is not encrypted,** and the screen says so. Fine at home; think first elsewhere.
|
||||
- About **200 KB a second**, one request at a time, 57 KB of flash, 13 KB of memory while it's on.
|
||||
- Also since [the last post](/devlog/roro9stack-shell/): the device shows **pictures** (v0.16.0), **Fn+p takes a screenshot** anywhere (v0.17.0), and this site has a **search**.
|
||||
|
||||
## It works!
|
||||
|
||||
I'll skip the build-up. I pressed `w`. This came up:
|
||||
|
||||
{{ figure(src="share.png", alt="The Cardputer's screen at 2x: a large QR code on the left; on the right, In a browser, on this network: 172.16.42.25/, Code 825 132 in large blue digits, Nothing asked yet, Not encrypted, and backtick stops sharing.", width=480, height=270, caption="The whole feature, as the device sees it. The code in this picture stopped being valid the moment I pressed Back.") }}
|
||||
|
||||
I pointed my phone's camera at the QR code, and there was my SD card, in the browser. No address to type, no code to type. **It works. That's fucking awesome.**
|
||||
|
||||
{{ figure(src="phone.png", alt="A phone's browser in dark mode at the address 172.16.42.25, at 00:15: roro9stack: the SD card, a link SD card, a bright blue Upload files button and a New folder button, then rows captures, gemini, gnss, irc, notes, screenshots, updates and wifi, each with a Delete button.", width=462, height=1001, caption="My phone, at a quarter past midnight. Its browser, my card, nothing else.") }}
|
||||
|
||||
{{ figure(src="device-photo.jpg", alt="A photograph of the Cardputer ADV on a wooden table. Its small screen shows the QR code, the address, the code 997 216, and 2.5 MB in, 3.2 MB out. Below the screen, the whole keyboard.", width=900, height=864, landscape=true, caption="And the other end of it, for scale. The whole server is in there, behind a screen smaller than the QR code on most posters.") }}
|
||||
|
||||
Files, from my phone, to a computer the size of a biscuit and back. Over Wi-Fi. With nothing installed on either end that wasn't there this morning.
|
||||
|
||||
Most of this devlog is about things that took a week of measuring and still bit me. This one I can explain to anybody in a sentence: it's a web page with your files on it.
|
||||
|
||||
## The question I asked, and the one I should have
|
||||
|
||||
Until tonight a file reached the card in one of two ways: the Debug Console's `put` command, which wants a PC, a Python script and a token, or pulling the card out. My phone can do neither. So I asked the obvious question: FTP, SFTP or WebDAV?
|
||||
|
||||
The answer was a table, and the table was unkind to all three:
|
||||
|
||||
{% table() %}
|
||||
| | On the phone | On the device |
|
||||
|---|---|---|
|
||||
| FTP | needs an app; passwords in clear | easy |
|
||||
| SFTP | needs an app | a whole SSH server: hundreds of KB, and a key exchange this chip would feel |
|
||||
| WebDAV | needs an app, on iOS and Android both | fine, but see the first column |
|
||||
| **A web page** | **any browser** | a small HTTP server |
|
||||
{% end %}
|
||||
|
||||
I had been choosing a protocol. What I wanted was to move a file with my thumb. Every phone made in the last fifteen years has exactly one file-transfer client that needs no setup, and it's the browser.
|
||||
|
||||
## What's in it
|
||||
|
||||
**On the device, almost nothing.** `w` starts the server and draws a screen. Back stops it. The server is the one that ships inside ESP-IDF, the framework the firmware is built on, so there was no library to choose. Seven requests: the page, the code, a listing, a download, an upload, a new folder, a delete.
|
||||
|
||||
**The QR code carries the code.** Scan it and you're in without typing; type the address by hand and the page asks for the six digits. The code is made fresh from the hardware random generator each time `w` is pressed, and pressing Back throws every browser out.
|
||||
|
||||
**An upload is just the request's body.** The browser sends the file as it is with a `PUT`, so there is no form to pick apart on a device with 100 KB of free memory. It lands on the card as `photo.jpg.part`, 8 KB at a time, and is renamed when the last byte has arrived. A transfer that dies halfway leaves nothing behind. If the name is taken, the page asks before replacing it, and the device refuses until it has.
|
||||
|
||||
**The Storage App's rules still hold.** The page can't delete the folders the firmware keeps its own files in, and it says why:
|
||||
|
||||
{{ figure(src="page.png", alt="The page in a browser at a phone's width: roro9stack: the SD card, a link SD card, buttons Upload files and New folder, then rows a-web, captures, gemini, gnss, irc, notes, screenshots, updates and wifi, each with a Delete button. Below, in orange: The firmware keeps its files in /notes.", width=390, height=630, caption="The page, at a phone's width, just after it was asked to delete `/notes`. It said no, in the device's own words.") }}
|
||||
|
||||
**Nobody gets to walk out of the card.** `/a-web/../wifi` is refused before anything looks at the disk, by a function with a test that tries a dozen ways of asking.
|
||||
|
||||
## What it isn't
|
||||
|
||||
**Encrypted.** A TLS server costs this device about 40 KB of memory per connection, and it has around 100 KB on a good day. So the files and the code cross the Wi-Fi in clear. On my own network I don't mind. On a hotel's, I'd think about it. The device's screen says "Not encrypted." in plain words every time, because a limit you have to read the docs to find is a trap.
|
||||
|
||||
**Fast.** 200 KB a second, give or take. A 2.6 MB photo takes eleven to seventeen seconds going up. I tried bigger pieces and smaller ones; the numbers moved around more between two runs of the same setting than between settings, so it's 8 KB and I stopped fiddling.
|
||||
|
||||
**Able to do two things at once.** The server answers one request at a time. Start a big download and the page waits until it's done. I found that by asking for a listing in the middle of a download and watching it time out. It's written in the guide and left as it is.
|
||||
|
||||
## What went wrong, for about four minutes each
|
||||
|
||||
**A file that included itself.** The part that can be tested on a PC lived in `web_share.h`. So did the service, in another folder. The service's header said `#include "web_share.h"`, meaning the other one, and the compiler quite reasonably gave it itself. The testable half is now called `share_rules.h`.
|
||||
|
||||
**The scanned address did nothing, sometimes.** If the page was already open and you then went to the same address with the code after the `#`, nothing happened: to a browser that isn't a new page, so the script never ran again. One line, `onhashchange`. Found by a browser test, not by me, which is the right way round.
|
||||
|
||||
That's the list. Two.
|
||||
|
||||
## What I checked, and what I didn't
|
||||
|
||||
Checked, before I went anywhere near my phone:
|
||||
|
||||
- Every request and every refusal, from a PC: wrong code, right code, a path with `..` in it, deleting a protected folder, deleting a folder that isn't empty, uploading over a file that exists.
|
||||
- **2.6 MB up, then down again, compared byte for byte.** The same.
|
||||
- The page in a real browser at a phone's width: uploads, a download, a new folder, a delete, a replace.
|
||||
- Five wrong codes: shut for a minute, for the right code too. A browser that was already in stays in.
|
||||
- Back: the server is gone and the memory comes back.
|
||||
|
||||
And then on my actual phone, where it works, which is the sentence this post exists for.
|
||||
|
||||
Not checked: Safari. Pulling the card out mid-transfer. Sharing while IRC is connected, when memory is tighter. What happens if the screen turns off while it's sharing.
|
||||
|
||||
## Also since last time
|
||||
|
||||
The day didn't stop at the [last post](/devlog/roro9stack-shell/):
|
||||
|
||||
- **Pictures.** The Storage App opens PNG, JPEG, BMP and GIF files (**v0.16.0**). The interesting part was the PNG decoder: the one in the display library wanted 44 KB in a single block, got it once, and refused the next five pictures. The firmware has its own now, which needs 32.
|
||||
- **Fn+p** takes a screenshot on any screen (**v0.17.0**), except the one that shows the Debug Console's token, where it politely declines.
|
||||
- **This site has a [search](/search/).**
|
||||
- **A WireGuard tunnel** is sitting in a pull request. It works against a test server on my own network and is waiting to meet a real one.
|
||||
|
||||
## By the numbers
|
||||
|
||||
{% table() %}
|
||||
| | |
|
||||
|---|---|
|
||||
| Keys to press on the device | 1 |
|
||||
| Apps to install on the phone | 0 |
|
||||
| Digits in the code | 6 |
|
||||
| Wrong codes before it shuts for a minute | 5 |
|
||||
| The page | 5.4 KB |
|
||||
| Flash | 57 KB |
|
||||
| Memory while sharing | 13 KB |
|
||||
| Speed | about 200 KB/s |
|
||||
| Requests at a time | 1 |
|
||||
| Bytes that differed after 2.6 MB went up and came back | 0 |
|
||||
| Host tests | 533 |
|
||||
| Things that went wrong | 2 |
|
||||
{% end %}
|
||||
|
||||
## Where it stands
|
||||
|
||||
{% steps() %}
|
||||
1. ~~M0 and M1: the skeleton, Wi-Fi, IRC, Wi-Fi Tools.~~ v0.1.0 to v0.2.1, [the first post](/devlog/roro9stack/).
|
||||
|
||||
2. ~~Updates and debugging over the air.~~ v0.3.0, [Look, no cables](/devlog/roro9stack-ota/).
|
||||
|
||||
3. ~~M2: GNSS.~~ v0.4.0, [Seventeen satellites](/devlog/roro9stack-gnss/).
|
||||
|
||||
4. ~~G1: Gemini.~~ v0.5.0, [A browser in the RAM IRC left over](/devlog/roro9stack-gemini/).
|
||||
|
||||
5. ~~M3: the LoRa radio, listening.~~ v0.6.0, [The loudest thing it hears is itself](/devlog/roro9stack-lora/).
|
||||
|
||||
6. ~~S1: the card, fixed addresses, the System App.~~ v0.6.1 to v0.8.1, [One byte too early](/devlog/roro9stack-s1/).
|
||||
|
||||
7. ~~F1 and the start of R1: files, notes, signed releases, updates from Gitea.~~ v0.9.0 to v0.11.0, [836 bytes](/devlog/roro9stack-f1-r1/).
|
||||
|
||||
8. ~~W1: the website, and one firmware with the Debug Console in it.~~ v0.12.0, [It was off](/devlog/roro9stack-console/).
|
||||
|
||||
9. ~~A help key, the Shell, notes of any size.~~ v0.13.0 to v0.15.0, [It said "No"](/devlog/roro9stack-shell/).
|
||||
|
||||
10. ~~Pictures, and a screenshot key.~~ v0.16.0 and v0.17.0.
|
||||
|
||||
11. ~~The card in a phone's browser.~~ v0.18.0, this post.
|
||||
|
||||
12. Next: the WireGuard tunnel, once it has talked to a real server. And M4, the mesh, which still wants a second node.
|
||||
{% end %}
|
||||
|
||||
{% signoff() %}
|
||||
I asked which of three servers to build and got told to build none of them. Then I pressed a key, picked up my phone, and my files were on it. Most of what this firmware does took days of careful measuring to get right. This took less than an evening, and it works, and I'm going to enjoy that at least until the next thing breaks.
|
||||
{% end %}
|
||||
|
After Width: | Height: | Size: 35 KiB |
|
After Width: | Height: | Size: 71 KiB |
|
After Width: | Height: | Size: 5.6 KiB |
|
After Width: | Height: | Size: 3.0 KiB |
|
After Width: | Height: | Size: 5.0 KiB |
@@ -0,0 +1,264 @@
|
||||
+++
|
||||
title = '''It said "No"'''
|
||||
description = '''The last post listed three things as next for roro9stack: one help key, a shell on the device, notes of any size. All three shipped in a day, with a CI that stopped rebuilding the world and a website that publishes itself. On the way, a test script kept typing after the device had crashed, and sent one word to an IRC channel full of people.'''
|
||||
date = 2026-10-07T18:00:00+02:00
|
||||
|
||||
[extra]
|
||||
topics = '''ESP32-S3 · Testing · Editors'''
|
||||
read_label = '''Read who it said it to →'''
|
||||
uid = '''<b>app:</b> Shell <b>heap:</b> 104 KB free'''
|
||||
dek = "Three releases of [roro9stack](/devlog/roro9stack/) in one day: v0.13.0, v0.14.0 and v0.15.0. A help key that replaces every hint line, the firmware's console on the device's own screen, and an editor that opens a megabyte in the memory it used for a shopping list. Most of what went wrong was me being wrong about what the device had done. One thing was the device doing exactly what my script told it to, in the wrong App."
|
||||
byline = '''designed by interrogation, rounds thirteen to sixteen: thirty-seven questions, two of them answered twice'''
|
||||
|
||||
[extra.sign]
|
||||
label = "Messages sent to real people by a test script"
|
||||
note = "One word, in an IRC channel, after a crash the script didn't notice."
|
||||
count = "1"
|
||||
tone = "red"
|
||||
|
||||
[[extra.cast]]
|
||||
name = "Fn+h"
|
||||
role = "the help key, on every screen"
|
||||
text = "Lists the keys that work where you are. Every screen had a line at the bottom doing that, differently and never completely. Those lines are gone."
|
||||
|
||||
[[extra.cast]]
|
||||
name = "The Shell"
|
||||
role = "an App, since v0.14.0"
|
||||
text = "The commands I had been typing from a PC over Wi-Fi, on the device's own keyboard. It took more than one try to decide what it should show, and one crash to decide where its commands run."
|
||||
|
||||
[[extra.cast]]
|
||||
name = "The test script"
|
||||
role = "types keys over the Debug Console"
|
||||
text = "Tireless, exact, and with no idea what is on the screen. It typed the right letters. The App under them had changed."
|
||||
|
||||
[[extra.cast]]
|
||||
name = "The window"
|
||||
role = "8 KB of a note, around the cursor"
|
||||
text = "All of a note that is in memory. The rest stays on the card, described by a short list. It moves when the cursor nears its edge, and nobody is meant to notice."
|
||||
|
||||
[[extra.cast]]
|
||||
name = "<note>.edit"
|
||||
role = "the side file"
|
||||
text = "Where a long note's changes wait, four kilobytes at a time, until the note is left and rewritten. Also what a power cut leaves behind, on purpose."
|
||||
+++
|
||||
|
||||
## TL;DR
|
||||
|
||||
- The [last post](/devlog/roro9stack-console/) ended with three things as "next". **All three are in**: a help key (**v0.13.0**), a shell on the device (**v0.14.0**), notes of any size (**v0.15.0**).
|
||||
- **Fn+h lists the keys of the screen you're on**, and every hint line is gone. The same lists make the key tables on this website.
|
||||
- **CI went from over seven minutes to about one** for a pull request. It had been rebuilding the whole framework at every run because of one file that isn't in git.
|
||||
- **The Shell** runs the firmware's commands on the device: Tab completes every word and paths on the card, `rm` behaves like Unix's and asks first, `*` and `?` work.
|
||||
- **The editor opens any file.** A 1.2 MB note uses the same 17.5 KB as a 62-byte one, saves in 4 KB pieces, and is rewritten in 2.6 s when you leave it. A power cut at any byte leaves the note or the last save, never something in between.
|
||||
- **This site publishes itself** when a change is merged, through an SSH key that can do exactly one thing.
|
||||
- A test script **sent the word "No" to an IRC channel**. That one can't be fixed, only prevented.
|
||||
- 507 host tests, 39 more than last time.
|
||||
|
||||
## The cast
|
||||
|
||||
{{ cast() }}
|
||||
|
||||
## One key instead of a hint line on every screen
|
||||
|
||||
Every screen had a line at the bottom: `Enter open d delete r rename`. Each was written by hand, each was different, and none had room for everything. The screen is 240 pixels wide.
|
||||
|
||||
So: **Fn+h, everywhere**, and `?` wherever you aren't typing text. It opens a panel over the App, titled with where you are, listing that screen's keys and then the ones that work everywhere. Any other key closes it.
|
||||
|
||||
{{ figure(src="help.png", alt="The Cardputer's screen at 2x: a panel titled Keys: Shell, listing Enter run the line, Tab complete the command, Fn semicolon and period lines you typed before, Alt semicolon and period scroll back and forward, Ctrl b, Fn comma and slash move the cursor, Del delete backwards, help every command.", width=480, height=270, caption="The Shell's keys, from the first build that had a Shell. The line that runs off the edge was reworded the same afternoon.") }}
|
||||
|
||||
The hint lines went, all of them, with one exception: the first-start Setup keeps its own, because someone in their first minute doesn't know the help key exists. It tells them on its first and last screens.
|
||||
|
||||
The lists started as code inside each App. They are now **data in one file**, 52 small tables, and the same script that builds the [developer docs](/dev/) reads that file and writes the key tables in the [user guide](/guide/). CI fails if the site's copy is out of date, so the guide can't list a key the firmware doesn't have.
|
||||
|
||||
## The framework that was rebuilt every time
|
||||
|
||||
A pull request took over seven minutes to check, and a release thirteen and a half. For a firmware that builds in 77 seconds on my machine.
|
||||
|
||||
Where the time went, for one pull request:
|
||||
|
||||
{% table() %}
|
||||
| Step | Time |
|
||||
|---|---|
|
||||
| Tools | 15 s |
|
||||
| Host tests and coverage | 55 s |
|
||||
| **The firmware** | **358 s** |
|
||||
| of which: configuring ESP-IDF | 87 s |
|
||||
| of which: compiling ESP-IDF's libraries | 171 s |
|
||||
| of which: our own code | 91 s |
|
||||
{% end %}
|
||||
|
||||
roro9stack rebuilds the Arduino framework with its own settings, for [smaller TLS buffers](/dev/decisions/0006-framework-rebuilt-for-smaller-tls-buffers/). The rebuilt libraries were sitting in the runner's cache the whole time. But the build system decides whether they still match by reading a file in the project folder, and that file is generated: it isn't in git. Every fresh checkout had no such file, so every run concluded the libraries were stale and rebuilt them. 260 seconds, each time, to produce what was already there.
|
||||
|
||||
The fix is to keep that file with the libraries it describes. Two more things came out of looking:
|
||||
|
||||
- **The version was a `-D` flag on every compiler command line.** Every commit changes the version, so every commit recompiled every file, on my machine too, and no cache could ever have helped. It's now one generated header that one file includes.
|
||||
- **A release built the firmware twice**: once to check it, once to sign it.
|
||||
|
||||
With a build cache for pull requests on top: **27 seconds** for the firmware with one file changed, and about a minute for the whole run on the real runner. A release takes under three minutes, and still compiles its own sources from nothing: no published file contains an object built for another commit.
|
||||
|
||||
## A shell, and what it should show
|
||||
|
||||
The Debug Console's commands are the tool I use most, and they needed a PC. The Shell is an App that runs them on the device.
|
||||
|
||||
The first version was an afternoon's work and wrong in three ways.
|
||||
|
||||
**It showed too much.** The firmware prints all the time: IRC connecting, a packet heard, whatever a PC on the USB port is asking for. Version one showed everything printed in the ten seconds after a command, on the theory that the reply would be in there somewhere. It was, among everything else. The fix was to stop guessing: the console now knows **who each line is for**. A command run from the Shell prints as the Shell's, and so does an answer that arrives a second later from another task, which notes who asked. Ctrl+b shows everything, for when that's what you want.
|
||||
|
||||
**`rm` was dangerous in a new way.** Over the console, `rm <folder>` had always removed the folder and everything in it, which is fine for a script and less fine for a thumb on a small keyboard. It's now Unix's: a folder needs `-r`, and in the Shell it asks unless you say `-f`.
|
||||
|
||||
**Tab did one word.** It now follows the firmware's own `help` text, word by word: `lora st` becomes `lora status`, `gnss track ` lists `start stop`. The words are read from the help text as it is written, so a new command completes without anyone maintaining a table. Past the command, it completes paths on the SD card. And `*` and `?` work in file names.
|
||||
|
||||
{{ figure(src="rm.png", alt="The Cardputer's screen at 2x: a dialog titled Delete? reading The 5 that match /gt2/*. It can't be undone. with two buttons, Cancel selected and Delete.", width=480, height=270, caption="`rm /gt2/*` in the Shell: one question for all five, with Cancel selected. `/gt2` is a scratch folder, made for the purpose.") }}
|
||||
|
||||
One more addition, small and my favourite: **an App's name with a capital letter opens it.** `Notes`, `Irc`, `Storage`. Every command is lowercase, so the capital is the whole syntax.
|
||||
|
||||
## It said "No"
|
||||
|
||||
The Shell's first version ran each command from inside the key handler. I test the UI by sending key presses over the Debug Console, so the call chain was: the main loop, a remote command, a key, the App manager, the Shell, the command interpreter *a second time*, the file command, and `printf` under all of it. The main loop has under 2 KB of stack to spare. `rm` on a folder went past it.
|
||||
|
||||
The device crashed, and restarted, as it should. It came back up in the Launcher.
|
||||
|
||||
My test script didn't know. It had a list of keys to send and it sent them. Its next Enter, meant for the Shell, landed in the Launcher and opened the first App in the list. That is IRC, which connected, as it's configured to, and joined its channels.
|
||||
|
||||
A few lines later the script reached its test of the capital-letter feature: type `No`, press Tab to complete it to `Notes`, press Enter. Tab completes nothing in IRC. Enter sends.
|
||||
|
||||
One word, to a channel of real people, from my nick. Not harmful, not explainable either, and not something any commit can take back.
|
||||
|
||||
Two things changed that afternoon:
|
||||
|
||||
- **The Shell hands its line to the main loop**, which runs it at the same depth as any console command. The crash is gone, and the crash report had decoded to exactly that chain of calls.
|
||||
- **`info` reports the App in front**, and the test helper checks it before every line it types. A script that survives a restart is typing somewhere else, and now it stops.
|
||||
|
||||
The rule I'd had since the day before was "take a screenshot before any key that deletes something". It was the right rule for the wrong failure. Typing is also an action.
|
||||
|
||||
## A megabyte in 17 KB
|
||||
|
||||
The Notes editor held the whole note in memory and stopped at 16 KB. That was a limit for the first version only; [F1's notes](/dev/milestones/f1/) say so in bold.
|
||||
|
||||
The device has no spare memory to throw at this, so the design is the old one from editors that ran on less: **the note is the file on the card, plus one window in memory.** The window is about 8 KB around the cursor. Everything else is a list of pieces: "bytes 0 to 40,000 of the file", "then 9,000 bytes of what was typed". When the cursor nears the window's edge, the window is written away if it changed, and the next one is loaded.
|
||||
|
||||
What makes it usable is what it writes, and when:
|
||||
|
||||
{% table() %}
|
||||
| | Up to 64 KB | Above |
|
||||
|---|---|---|
|
||||
| The save, five seconds after the last key | The whole file, as before | What changed, appended to `<note>.edit`: about 4 KB |
|
||||
| Leaving the note | Nothing more to do | The file is rewritten, with a progress bar |
|
||||
| After a power cut | The note as last saved | The note opens with the saved changes back |
|
||||
{% end %}
|
||||
|
||||
{{ figure(src="saving.png", alt="The Cardputer's screen at 2x, all black with Saving in blue, the file name zz-big.txt, a progress bar a little over half full, and 60%.", width=480, height=270, caption="Leaving a 1.2 MB note. This takes 2.6 seconds, which is long enough to deserve a bar and short enough that I had to race the screenshot.") }}
|
||||
|
||||
Memory with a note open is 17.5 KB, for a note of 62 bytes or of 1.2 MB. Going to the end of the megabyte takes as long as any other key.
|
||||
|
||||
### The part that has to be right
|
||||
|
||||
An editor that loses text is worse than no editor, and this one now has a side file, a temporary file and the note itself, any of which can be half written when the power goes. So the rewrite ends with a mark: once the complete new file is on the card, one small write to the side file says "done". Before that mark, the old note and its side file are the truth. After it, the new file is, and whatever was interrupted is finished the next time the note is opened.
|
||||
|
||||
That is a claim, and it's the kind I don't trust until something has tried to break it. Two tests do:
|
||||
|
||||
- **A power cut at every 997th byte** of two saves and a rewrite. After each, the note has to be one of exactly three texts, the one that was reported as saved has to be there, and no stray file may be left.
|
||||
- **36,000 random keys** on six notes, typing, deleting, moving, jumping, saving and cutting the power, compared with a plain string after every key.
|
||||
|
||||
They pass. But the first run had three failures, and they're worth a line each, because only one of them was the editor's:
|
||||
|
||||
1. I had worked out by hand where the cursor should be after a recovery, and got it wrong by four.
|
||||
2. I had assumed windows would break between groups of three characters in my test text. They break between characters, which is all they promise.
|
||||
3. **A file replaced by a shorter one lost its pending edits without a word.** The design says they are set aside as `.edit.lost` and the editor tells you. The code checked the pieces against the new file's length first, found them out of range, concluded there was nothing valid to keep, and deleted them. A real bug, in exactly the path that exists to never delete typed text.
|
||||
|
||||
Two of three were the test being wrong. The third is why the tests exist.
|
||||
|
||||
{{ figure(src="back.png", alt="The Cardputer's Notes editor at 2x, showing zz-big.txt, 1.1 MB, saved. The first line reads YTOP line 000000, followed by line 000001 to line 000007. At the bottom, in orange: Your unsaved changes are back.", width=480, height=270, caption="After a restart in the middle of a rewrite. The `Y` was typed, saved to the side file, and the device was reset while it was writing the megabyte. It's there.") }}
|
||||
|
||||
### What I had promised, and what I measured
|
||||
|
||||
I'd said the rewrite would take about two and a half seconds a megabyte. The first build took 3.5 to 4.5 seconds for 1.2 MB. It was copying in 2 KB blocks. With 4 KB blocks it takes 2.6, about 450 KB a second, which is what this card gives a plain copy.
|
||||
|
||||
## A site that publishes itself
|
||||
|
||||
Until this morning, publishing this site meant logging into the web server and running a script, by hand, after every merge.
|
||||
|
||||
Now CI does it, after a merge and after a release. The interesting part is what the key in CI is allowed to do, which is one thing:
|
||||
|
||||
{% code(caption="One line of `authorized_keys` on the web server. Whatever the client asks for, this runs instead.") %}
|
||||
```
|
||||
from="<the runner>",restrict,command="/path/to/rororefresh.sh" ssh-ed25519 AAAA… roro9stack-ci
|
||||
```
|
||||
{% end %}
|
||||
|
||||
My first plan had the command as a secret in CI, next to the key. With a forced command there is nothing to keep secret: CI connects and sends no command at all, and a stolen key can refresh the website and do nothing else. The server's address, the user, the key and the server's host key are secrets; the repository is public and none of them is in it.
|
||||
|
||||
Checked against a throwaway SSH server before the real one: asking for `id; cat /etc/passwd` runs the refresh. No terminal. `scp` copies nothing. A different host key stops the run.
|
||||
|
||||
On its first real run, the live page changed fifteen seconds after the run started. This post got here that way.
|
||||
|
||||
## The device was right
|
||||
|
||||
A pattern from the day, three times over.
|
||||
|
||||
**The keys that vanished.** Twice, the first key my script sent after a quiet minute did nothing. The [F1 notes](/dev/milestones/f1/) describe that exact bug, fixed. I wrote it up as a possible regression. It isn't: a key that wakes a dark screen only wakes it, same as on the real keyboard. That's documented, on a page of this site, which I wrote.
|
||||
|
||||
**The letters in the wrong place.** In the editor test I typed two letters at the top of a note, moved 400 lines down, typed four more, fetched the file and compared it with what I expected. It differed: `liMID ne 000400` where I expected `MID line 000400`. The file matched the screen exactly. Moving down keeps the cursor's column, as it should, and I had typed two letters first.
|
||||
|
||||
**The "No".** The device did what it was sent. Every key arrived, in order.
|
||||
|
||||
Three times the instrument was right and the reading was wrong. The remote `key` command now takes `ctrl-`, `alt-` and `shift-`, by the way, because checking the editor's jump to the end of a note needed Ctrl, and "the remote `key` command can't send that" had been in the not-checked list of every milestone since the editor existed.
|
||||
|
||||
## What I didn't check
|
||||
|
||||
- **The real keyboard**, for Fn+h, `?`, Ctrl+b and the Alt scroll. Everything was driven by remote keys.
|
||||
- **The power button's path** in the editor, which writes the side file only, and the screen turning off.
|
||||
- **Memory with IRC connected** and a long note open. After the morning, I didn't connect IRC.
|
||||
- **A file of tens of megabytes.**
|
||||
- Renaming or deleting a note from the Storage App leaves its side file behind. The Notes App handles both.
|
||||
|
||||
## By the numbers
|
||||
|
||||
{% table() %}
|
||||
| | |
|
||||
|---|---|
|
||||
| Releases | 3 |
|
||||
| Design questions | 37 |
|
||||
| Host tests | 507 |
|
||||
| A pull request's CI run, before and after | over 7 min, about 1 |
|
||||
| Seconds spent rebuilding what was already built, per run | 260 |
|
||||
| Flash the Shell costs | 21 KB |
|
||||
| Flash notes of any size cost | 15 KB |
|
||||
| Memory with a note open, 62 bytes or 1.2 MB | 17.5 KB |
|
||||
| Bytes a long note's save writes | about 4,000 |
|
||||
| Random keys the editor was compared against a string for | 36,000 |
|
||||
| Bugs those tests found in the editor | 1 |
|
||||
| Bugs they found in my arithmetic | 2 |
|
||||
| Words sent to an IRC channel | 1 |
|
||||
{% end %}
|
||||
|
||||
## Where it stands
|
||||
|
||||
{% steps() %}
|
||||
1. ~~M0 and M1: the skeleton, Wi-Fi, IRC, Wi-Fi Tools.~~ v0.1.0 to v0.2.1, [the first post](/devlog/roro9stack/).
|
||||
|
||||
2. ~~Updates and debugging over the air.~~ v0.3.0, [Look, no cables](/devlog/roro9stack-ota/).
|
||||
|
||||
3. ~~M2: GNSS.~~ v0.4.0, [Seventeen satellites](/devlog/roro9stack-gnss/).
|
||||
|
||||
4. ~~G1: Gemini.~~ v0.5.0, [A browser in the RAM IRC left over](/devlog/roro9stack-gemini/).
|
||||
|
||||
5. ~~M3: the LoRa radio, listening.~~ v0.6.0, [The loudest thing it hears is itself](/devlog/roro9stack-lora/).
|
||||
|
||||
6. ~~S1: the card, fixed addresses, the System App.~~ v0.6.1 to v0.8.1, [One byte too early](/devlog/roro9stack-s1/).
|
||||
|
||||
7. ~~F1 and the start of R1: files, notes, signed releases, updates from Gitea.~~ v0.9.0 to v0.11.0, [836 bytes](/devlog/roro9stack-f1-r1/).
|
||||
|
||||
8. ~~W1: the website, and one firmware with the Debug Console in it.~~ v0.12.0, [It was off](/devlog/roro9stack-console/).
|
||||
|
||||
9. ~~One help key, and CI in a minute.~~ v0.13.0, this post.
|
||||
|
||||
10. ~~The Shell.~~ v0.14.0, this post.
|
||||
|
||||
11. ~~Notes of any size, and a site that publishes itself.~~ v0.15.0, this post.
|
||||
|
||||
12. Next: M4, the mesh, which still wants a second node. And the editor, now that it opens anything, plainly lacks undo.
|
||||
{% end %}
|
||||
|
||||
{% signoff() %}
|
||||
The last post promised three things and this one delivers them, which would be a tidy story if a script of mine hadn't said "No" to a room of strangers halfway through. The device did nothing wrong all day. It crashed where I had written a crash, restarted as designed, and typed what it was sent.
|
||||
{% end %}
|
||||