Site: the developer docs (phase 4), with the Debug Builds and the Debug Console first
CI / build (pull_request) Successful in 8m38s
Site / build (pull_request) Successful in 9s

/dev/ has Debug Builds and the Debug Console (builds and the token, the
console and its protocol, files and screenshots, driving the UI, crashes and
Safe Mode, the command reference), Build, test and release (including how an
update works), the architecture decisions and the milestone plans.

Generated from the repository by site/tools/gen_dev_docs.py: the ADRs, the
milestones, the README's sections, and the command reference, read from the
firmware's own `help` text. The pages are committed (Zola cannot read outside
its folder); the Site workflow checks they are current, and now also runs
when src/main.cpp changes. M0, M1 and CONTEXT.md are not published.
README: the gnss commands that the table lacked.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EhqxQ49eCju4CzKYNjZzwT
This commit is contained in:
2026-10-06 21:25:33 +02:00
co-authored by Claude Sonnet 5.5
parent 8f95bee744
commit 3b4100dc6a
43 changed files with 2284 additions and 6 deletions
+13
View File
@@ -0,0 +1,13 @@
+++
title = "Build, test and release"
description = "Building the firmware, running the tests, flashing a device, and what happens between a commit and a release."
template = "guide-index.html"
page_template = "guide-page.html"
sort_by = "weight"
weight = 2
[extra]
eyebrow = "Developer docs"
+++
The firmware is built, tested and released **in Docker**, so that a clean machine, your machine and the CI runner all get the same result. These pages say how. For the Debug Console, which is how you work on a device once it is flashed, see [Debug Builds and the Debug Console](/dev/debug/).
+38
View File
@@ -0,0 +1,38 @@
+++
title = "Build, test and release"
description = "Docker is the only tool you need. How the firmware is built, how the host tests run, and what CI does on a pull request and on a tag."
weight = 1
[extra]
docs = true
source = "README.md"
tag = "Build"
+++
## Requirements
Only **Docker** is needed. PlatformIO and the ESP32 toolchain run inside a container, and are cached in the `roro9stack-pio` Docker volume. The first build downloads about 1 GB and takes a few minutes.
## Build and test (local CI)
```sh
scripts/ci.sh
```
This runs the host-side unit tests (`test/`, `native` environment), then builds the firmware. The output is `.pio/build/cardputer-adv/firmware.factory.bin`.
`scripts/coverage.sh` runs the same tests with coverage counters and writes a line-by-line report to `.pio/coverage/index.html`. The badge above is its figure for `main`: the share of the lines of `lib/` that the host tests run. `lib/` is the logic that compiles on a PC; `lib/SD` (the card's driver) and `src/` (the Apps, the Services, everything that needs the device) have no host tests and aren't in that figure.
The framework is rebuilt with the TLS settings in `platformio.ini` (`custom_sdkconfig`, ADR 0006), so the first build after a fresh checkout takes about 4 minutes; later builds take under a minute.
## CI and releases
Gitea Actions (`.gitea/workflows/ci.yml`, docs/milestones/R1.md) runs the host tests on every push to `main`, and on a pull request also builds the release firmware and the Debug Build: changes reach `main` through pull requests. Pushing a tag `v*` runs all of it and publishes a release on Gitea with:
- `roro9stack-<version>.ota`, the signed Update File;
- `roro9stack-<version>-factory.bin`, the whole flash image for a first install over USB;
- `roro9stack-<version>.elf.gz`, to decode crash reports from that build;
- `SHA256SUMS`.
CI signs with the project's key, held as a repository secret (ADR 0008). Debug Builds are built but never published: each carries its builder's Debug Console token.
`scripts/ota_verify.py <file.ota>` checks an Update File on a PC the way a device does. `scripts/release_build.sh` and `scripts/release_publish.py` are what the workflow runs; they work the same by hand.
+57
View File
@@ -0,0 +1,57 @@
+++
title = "Flash and update"
description = "Put the firmware on a Cardputer over USB, then update it over Wi-Fi or from the SD card, and from the project's releases."
weight = 2
[extra]
docs = true
source = "README.md"
tag = "Flash"
+++
## Flash
1. Connect the Cardputer by USB-C.
2. Run:
```sh
scripts/flash.sh # auto-detects the port; or: scripts/flash.sh /dev/ttyACM1
```
This uploads the firmware, then opens the serial monitor. Quit the monitor with `Ctrl+C`.
**If the upload can't connect,** put the device in download mode: hold **G0** (the button next to the screen) while plugging in USB, or while pressing reset. Then retry.
**If you get "permission denied" on the port,** your user needs access to the serial device. Run this once, then log out and back in:
```sh
sudo usermod -aG dialout "$USER"
```
## Firmware Updates over Wi-Fi (OTA)
Once the Cardputer runs an OTA-capable firmware (flashed once over USB), updates can go over Wi-Fi:
```sh
scripts/ota_keygen.sh # once: creates the signing key (see ADR 0003)
scripts/flash.sh --ota 10.39.39.12 # build, sign and push; or set RORO_OTA_HOST
```
The device shows the push address in **Settings → Firmware**. It installs a correctly signed update right away, restarts (waiting up to 60 s if you're typing), and runs the new firmware on **Probation**. If the new firmware crashes, or can't reconnect Wi-Fi within 3 minutes, it rolls back to the previous one and says so.
To install from the SD card instead, copy the `.ota` file from `.pio/build/cardputer-adv/` into `/updates` on the card, then use **Settings → Firmware**. With the Cardputer on USB, the card can stay in: `scripts/sd_put.sh <file.ota>` sends it over the serial console into `/updates` (about 30 s for 1.6 MB, checked with SHA-256 before it's renamed into place; `SD_PUT_DEBUG=1` shows the console while it runs).
**The private key** lives in `~/.config/roro9stack/ota-key.pem` and must never be committed. If it's lost, generate a new pair and flash once over USB. (CI signs releases with a copy kept as a repository secret, ADR 0008.)
### Updates from Gitea
With no PC and no card, the device can install the project's releases itself (docs/milestones/R1.md). In **Settings → Firmware**:
- **Latest release** checks the server (Enter, or `c`) and says `v0.11.0 (new)` or `(current)`. Enter again opens the release: its version, date, size and the tag's message, with **Install** when it's newer. The download goes straight into the inactive slot, so no card is needed; the signature is checked after the first 160 bytes, before anything is written, and the image's hash at the end. The new firmware then runs on Probation as for any update.
- **Older releases** lists the last ten, newest first. Opening an older one offers to go back to it, with a different question.
- **Settings → Check for updates** (on by default): once a day, with Wi-Fi up and the clock set, the device looks at the latest release and says `v0.11.0 is out: see Settings > Firmware`, once per version. It installs nothing by itself, and doesn't announce a version that already failed and rolled back on this device.
The connection is checked against the two ISRG roots Let's Encrypt chains end in (ADR 0009), not the usual bundle of about 130 authorities. Whatever the connection, the Update File's own signature is what decides what gets installed.
**IRC steps aside.** A secure connection takes about 52 KB of memory at its peak, and IRC's own takes 40 KB of the 107 KB there is. A check or an install you ask for makes IRC disconnect for the few seconds it takes and reconnect afterwards. The daily check never does that: with IRC connected it waits for a moment when IRC isn't, so while IRC stays connected for days it doesn't run, and **Latest release** is the way to check.
**A Debug Build** shows the latest release but doesn't install it: releases have no Debug Console (they aren't built with one, so that nobody's console token is published), and installing one would take it away. A Debug Build is updated from the PC with `scripts/flash.sh --debug --ota`.
@@ -0,0 +1,63 @@
+++
title = "How an update works"
description = "The signed Update File, the four ways to get one onto a device, Probation and Rollback, and the check that sits in front of all of them."
weight = 3
[extra]
tag = "Updates"
diagrams = true
+++
A **Firmware Update** installs one signed file, an **Update File** (`.ota`). Every way of delivering it ends at the same gate, and a new firmware must prove itself before it is kept. The decisions are [ADR 0003](/dev/decisions/0003-own-signature-check-not-secure-boot/) (the signature), [ADR 0005](/dev/decisions/0005-safe-mode-crash-reports-watchdog/) (when the new firmware crashes) and [ADR 0008](/dev/decisions/0008-ci-signs-releases/) (who signs releases).
## The Update File
{{ diagram(src="update-file.svg", min_width=580, caption="A 160-byte header, then the image. Bytes 0 to 79 are signed.") }}
- A **160-byte header**: the magic `RORO-OTA`, the format, the header size, the image size, the image's **SHA-256** and the version (bytes 0 to 79, the signed part), then the signature's length, the **signature** and reserved bytes.
- The signature is **ECDSA P-256 over the SHA-256 of bytes 0 to 79**, checked by the firmware against a **public key compiled into it** (`keys/ota-public.pem`, committed), **before anything is written**.
- Then the **image**, hashed while it is written; at the end the hash must equal the one in the header.
Make, check and push one:
```sh
scripts/ota_keygen.sh # once: creates the key pair. The private key goes to ~/.config/roro9stack/ota-key.pem (never committed); the public key to keys/ and the firmware source
scripts/make_ota.py firmware.bin v0.12.0 out.ota # wraps and signs an image (the key: $RORO_OTA_KEY or the default path)
scripts/ota_verify.py out.ota # checks a file the way a device does, on the PC
scripts/ota_push.py out.ota 10.39.39.12 # pushes it to the Update Service, TCP 3232
scripts/flash.sh --ota 10.39.39.12 # builds, signs and pushes in one go (add --debug for a Debug Build)
```
`ota_push.py` prints the device's answer (`OK …`), or says the device **refused the update and hung up**, with the reason on the device's screen: the header is checked first, so a refused file stops mid-transfer.
## Four ways in, one gate
{{ diagram(src="ways-in.svg", min_width=580, caption="Every path ends at the same Update Service and the same signature check.") }}
1. **Push over Wi-Fi** to TCP 3232: `scripts/flash.sh --ota`. The device always listens while Wi-Fi is connected.
2. **From the SD card**: a `.ota` in `/updates`, installed from Settings → Firmware, from the Storage App, or with the `install <path>` command. Put it there by hand, with `scripts/rdbg.py put` over Wi-Fi, or with `scripts/sd_put.sh` over USB.
3. **From the project's releases** on Gitea, which the device downloads itself over TLS (Settings → Firmware, or `update check` / `update install <tag>`): see [Flash and update](/dev/build/flash/).
All three feed the same parser: the **header and signature are checked first**, the image is written to the **other app slot** while it is hashed, and the slot becomes the next boot **only if the hash matches**. Anything wrong ends in a Toast and **nothing changes**. A downgrade is allowed, and shows "older than the installed version".
The device then restarts (waiting up to 60 seconds if you are typing) into **Probation**.
## Probation and Rollback
{{ diagram(src="probation.svg", min_width=620, caption="The life of an update: install, restart, Probation, confirmed. A crash or a restart before the last step sends the device back to the previous firmware.") }}
A new image is **not trusted** at first:
1. **Install:** the image goes to the other slot and the OTA data marks it *new*.
2. **Restart:** the bootloader turns *new* into *pending verify* and boots it.
3. **Probation:** the new firmware must boot, draw its UI, start its Services, run **30 seconds without a crash**, and **reconnect Wi-Fi within 3 minutes** if one is configured.
4. **Confirmed:** it marks itself valid and a Toast says `Updated`.
If it crashes or restarts first, the bootloader marks the image *aborted* and boots the **previous firmware** again, which says the update failed. Meanwhile that previous firmware stayed in its slot: it is the way back.
**Two lines of defence.** The bootloader's rollback is the first. The firmware counts its own boots on Probation, very first thing in `setup()`, and reverts itself on the second unconfirmed start, as a second line. And Arduino-ESP32 normally marks an image valid *before* `setup()` runs, which once hid the bootloader's rollback entirely; the firmware overrides `verifyRollbackLater()` so an image stays pending until Probation confirms it.
## Who signs releases
A tag `v*` is built, signed and published by Gitea Actions, with the signing key held as a repository secret as well as on the maintainer's machine ([ADR 0008](/dev/decisions/0008-ci-signs-releases/) says what that costs and what limits it). The release step checks the signed file against the public key in the sources it built, so a wrong secret stops the release instead of publishing a file no device accepts.
**If the private key is ever lost,** the next update has to go over USB, carrying a new public key. Someone with USB access can always flash anything: only Wi-Fi and SD card updates are guarded, by design (ADR 0003).
@@ -0,0 +1,55 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 760 316" role="img" aria-label="The life of an update. One: install, the image goes to app1 and the OTA data marks it NEW. Two: restart, the bootloader turns NEW into PENDING_VERIFY and boots app1. Three: Probation, 30 seconds up, a frame drawn, and Wi-Fi within 3 minutes if it is configured. Four: confirmed, the firmware marks itself VALID and a Toast says Updated. If it crashes or restarts before step four, the bootloader turns PENDING_VERIFY into ABORTED and boots app0 again, which says the update failed. Meanwhile app0 kept the previous firmware: the way back. The trap, under step two: Arduino's initArduino marks the image VALID before setup runs, unless verifyRollbackLater returns true.">
<defs>
<marker id="pb-arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path d="M0,0 L10,5 L0,10 z" fill="currentColor"/></marker>
<marker id="pb-arrow-ok" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="f-green" d="M0,0 L10,5 L0,10 z"/></marker>
<marker id="pb-arrow-bad" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="f-red" d="M0,0 L10,5 L0,10 z"/></marker>
</defs>
<g font-family="JetBrains Mono, ui-monospace, monospace" fill="currentColor">
<rect class="pb-box" x="16" y="40" width="170" height="80" rx="6"/>
<text x="28" y="62" font-size="12" font-weight="600">1 install</text>
<text x="28" y="84" font-size="11">image → app1</text>
<text x="28" y="102" font-size="11" class="pb-dim">otadata: NEW</text>
<rect class="pb-box" x="206" y="40" width="170" height="80" rx="6"/>
<text x="218" y="62" font-size="12" font-weight="600">2 restart</text>
<text x="218" y="84" font-size="11">bootloader:</text>
<text x="218" y="102" font-size="11" class="pb-dim">NEW → PENDING_VERIFY</text>
<rect class="pb-hot" x="396" y="40" width="170" height="80" rx="6"/>
<text x="408" y="62" font-size="12" font-weight="600">3 Probation</text>
<text x="408" y="84" font-size="11">30 s up, a frame</text>
<text x="408" y="102" font-size="11" class="pb-dim">Wi-Fi within 3 min</text>
<rect class="pb-ok" x="586" y="40" width="158" height="80" rx="6"/>
<text x="598" y="62" font-size="12" font-weight="600">4 confirmed</text>
<text x="598" y="84" font-size="11">→ VALID</text>
<text x="598" y="102" font-size="11" class="pb-dim">Toast: Updated to…</text>
<path class="pb-line" d="M186 80 H202" marker-end="url(#pb-arrow)"/>
<path class="pb-line" d="M376 80 H392" marker-end="url(#pb-arrow)"/>
<path class="pb-line-ok" d="M566 80 H582" marker-end="url(#pb-arrow-ok)"/>
<rect class="pb-bad" x="396" y="196" width="348" height="76" rx="6"/>
<text x="408" y="218" font-size="12" font-weight="600">crash or restart before 4</text>
<text x="408" y="240" font-size="11">bootloader: PENDING_VERIFY → ABORTED</text>
<text x="408" y="258" font-size="11" class="pb-dim">boots app0: "Update to … failed"</text>
<path class="pb-line-bad" d="M481 120 V192" marker-end="url(#pb-arrow-bad)"/>
<text x="408" y="294" font-size="10" class="pb-dim">(second line: bootGuard() in setup() rolls back</text>
<text x="408" y="308" font-size="10" class="pb-dim"> a second unconfirmed start by itself)</text>
<text x="16" y="216" font-size="11" class="pb-dim">app0 keeps the</text>
<text x="16" y="232" font-size="11" class="pb-dim">previous firmware:</text>
<text x="16" y="248" font-size="11" class="pb-dim">the way back</text>
<rect class="pb-trap" x="206" y="176" width="170" height="122" rx="6"/>
<text class="f-red" x="218" y="198" font-size="12" font-weight="600">the trap</text>
<text x="218" y="220" font-size="11">initArduino()</text>
<text x="218" y="238" font-size="11">marks it VALID</text>
<text x="218" y="256" font-size="11">before setup(),</text>
<text x="218" y="274" font-size="11" class="pb-dim">unless verify-</text>
<text x="218" y="290" font-size="11" class="pb-dim">RollbackLater()</text>
<path class="pb-line-bad" d="M291 176 V124" marker-end="url(#pb-arrow-bad)"/>
<text x="16" y="22" font-size="13" font-weight="600">The life of an update</text>
</g>
</svg>

After

Width:  |  Height:  |  Size: 4.3 KiB

@@ -0,0 +1,43 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 760 196" role="img" aria-label="The Update File: a 160-byte header, then the firmware image. Bytes 0 to 79 are signed: the magic RORO-OTA, the format, the header size, the image size, the image's SHA-256 and the version. Then the signature's length, the signature itself, and reserved bytes up to 160. The signature is ECDSA P-256 over the SHA-256 of bytes 0 to 79, checked before anything is written. The image follows, hashed while it is written, and must match the hash in bytes 16 to 47.">
<g font-family="JetBrains Mono, ui-monospace, monospace" fill="currentColor">
<text x="16" y="22" font-size="13" font-weight="600">Update File (.ota)</text>
<text x="16" y="40" font-size="11" class="uf-dim">a 160-byte header, little-endian, then the image</text>
<rect class="uf-signed" x="16" y="72" width="380" height="40" rx="4"/>
<rect class="uf-cell" x="16" y="72" width="64" height="40"/>
<rect class="uf-cell" x="80" y="72" width="40" height="40"/>
<rect class="uf-cell" x="120" y="72" width="40" height="40"/>
<rect class="uf-cell" x="160" y="72" width="52" height="40"/>
<rect class="uf-cell" x="212" y="72" width="100" height="40"/>
<rect class="uf-cell" x="312" y="72" width="84" height="40"/>
<rect class="uf-cell" x="396" y="72" width="40" height="40"/>
<rect class="uf-cell" x="436" y="72" width="92" height="40"/>
<rect class="uf-cell" x="528" y="72" width="32" height="40"/>
<rect class="uf-image" x="560" y="72" width="184" height="40"/>
<g font-size="10" text-anchor="middle">
<text x="48" y="96">RORO-OTA</text>
<text x="100" y="96">fmt</text>
<text x="140" y="96">hdr</text>
<text x="186" y="96">size</text>
<text x="262" y="96">image SHA-256</text>
<text x="354" y="96">version</text>
<text x="416" y="96">len</text>
<text x="482" y="96">signature</text>
<text x="544" y="96">0…</text>
<text x="652" y="96">the image, ~1.6 MB</text>
</g>
<g font-size="10" class="uf-dim" text-anchor="middle">
<text x="16" y="64">0</text><text x="80" y="64">8</text><text x="120" y="64">10</text><text x="160" y="64">12</text>
<text x="212" y="64">16</text><text x="312" y="64">48</text><text x="396" y="64">80</text><text x="436" y="64">82</text>
<text x="528" y="64">154</text><text x="560" y="64">160</text>
</g>
<path d="M16 120 V128 H396 V120" fill="none" stroke="var(--accent)" stroke-width="1.5"/>
<text x="206" y="150" font-size="11" text-anchor="middle" class="uf-hot">signed: ECDSA P-256 over SHA-256(bytes 0–79)</text>
<text x="206" y="168" font-size="11" text-anchor="middle" class="uf-dim">checked before a single byte is written</text>
<path d="M560 120 V128 H744 V120" fill="none" stroke="currentColor" stroke-opacity=".5" stroke-width="1.5"/>
<text x="652" y="150" font-size="11" text-anchor="middle" class="uf-dim">hashed while it's written;</text>
<text x="652" y="168" font-size="11" text-anchor="middle" class="uf-dim">must match bytes 16–47</text>
</g>
</svg>

After

Width:  |  Height:  |  Size: 3.0 KiB

@@ -0,0 +1,53 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 760 352" role="img" aria-label="Four ways in, one gate. scripts/flash.sh --ota signs the firmware and pushes it over Wi-Fi to TCP port 3232, straight to the Update Service. rdbg.py put over Wi-Fi, sd_put.sh over USB serial, or the card by hand all put an Update File in /updates on the SD card, where Settings, Firmware, or the install command picks it up. The Update Service checks the header and signature first, writes the image to the other app slot while hashing it, and makes it the next boot only if the hash matches. Then it restarts into Probation. If anything is wrong, a Toast says so and nothing changes.">
<defs>
<marker id="wi-arrow" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path d="M0,0 L10,5 L0,10 z" fill="currentColor"/></marker>
<marker id="wi-arrow-hot" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="f-accent" d="M0,0 L10,5 L0,10 z"/></marker>
<marker id="wi-arrow-bad" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="7" markerHeight="7" orient="auto-start-reverse"><path class="f-red" d="M0,0 L10,5 L0,10 z"/></marker>
</defs>
<g font-family="JetBrains Mono, ui-monospace, monospace" fill="currentColor">
<rect class="wi-box" x="16" y="32" width="220" height="56" rx="6"/>
<text x="30" y="55" font-size="12" font-weight="600">flash.sh --ota</text>
<text x="30" y="75" font-size="11" class="wi-dim">builds, signs, pushes</text>
<rect class="wi-box" x="16" y="112" width="220" height="56" rx="6"/>
<text x="30" y="135" font-size="12" font-weight="600">rdbg.py put</text>
<text x="30" y="155" font-size="11" class="wi-dim">Debug Build, Wi-Fi 2323</text>
<rect class="wi-box" x="16" y="192" width="220" height="56" rx="6"/>
<text x="30" y="215" font-size="12" font-weight="600">sd_put.sh</text>
<text x="30" y="235" font-size="11" class="wi-dim">USB serial, card stays in</text>
<rect class="wi-box" x="16" y="272" width="220" height="56" rx="6" stroke-dasharray="4 3"/>
<text x="30" y="295" font-size="12" font-weight="600">the card, by hand</text>
<text x="30" y="315" font-size="11" class="wi-dim">the 1990s way</text>
<rect class="wi-panel" x="288" y="176" width="200" height="88" rx="8"/>
<text x="302" y="200" font-size="12" font-weight="600">SD card</text>
<text x="302" y="219" font-size="11">/updates/*.ota</text>
<text x="302" y="238" font-size="11" class="wi-dim">Settings → Firmware,</text>
<text x="302" y="254" font-size="11" class="wi-dim">or install &lt;path&gt;</text>
<rect class="wi-hot" x="536" y="32" width="208" height="158" rx="8"/>
<text x="550" y="56" font-size="12" font-weight="600">Update Service</text>
<text x="550" y="80" font-size="11">1 header, signature:</text>
<text x="550" y="96" font-size="11" class="wi-dim"> checked first</text>
<text x="550" y="118" font-size="11">2 image → other slot,</text>
<text x="550" y="134" font-size="11" class="wi-dim"> hashed on the way</text>
<text x="550" y="156" font-size="11">3 hash matches:</text>
<text x="550" y="172" font-size="11" class="wi-dim"> boot it next</text>
<rect class="wi-box" x="576" y="216" width="168" height="44" rx="6"/>
<text x="660" y="243" font-size="12" text-anchor="middle">restart → Probation</text>
<rect class="wi-bad" x="536" y="280" width="208" height="56" rx="6"/>
<text x="550" y="303" font-size="11">anything wrong: a Toast,</text>
<text x="550" y="321" font-size="11">and nothing changes</text>
<path class="wi-line-hot" d="M236 60 H532" marker-end="url(#wi-arrow-hot)"/>
<text class="f-accent" x="300" y="52" font-size="11">Wi-Fi · TCP 3232</text>
<path class="wi-line" d="M236 140 H262 V198 H284" marker-end="url(#wi-arrow)"/>
<path class="wi-line" d="M236 220 H284" marker-end="url(#wi-arrow)"/>
<path class="wi-line" d="M236 300 H262 V242 H284" marker-end="url(#wi-arrow)"/>
<path class="wi-line" d="M488 220 H512 V150 H532" marker-end="url(#wi-arrow)"/>
<text x="492" y="238" font-size="10" class="wi-dim">storage</text>
<text x="492" y="250" font-size="10" class="wi-dim">task</text>
<path class="wi-line" d="M660 190 V212" marker-end="url(#wi-arrow)"/>
<path class="wi-line-bad" d="M556 190 V276" marker-end="url(#wi-arrow-bad)"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 4.3 KiB