CI: stop rebuilding the framework at every run; a build cache, ccache, and the version out of the compiler flags (#74)

A pull request's firmware step took 358 s: 260 of them rebuilding the
framework that was already in the volume, because the platform decides by
sdkconfig.defaults in the project folder, which is generated and not in git,
so no fresh checkout had it. It is now kept in the volume, inside the
libraries it describes, and copied into the checkout; the platform still
checks its hash against platformio.ini.

The version was a -D on every command line: each commit recompiled
everything, and no cache could help. scripts/version.py now writes one
generated header, read by one file.

PlatformIO's build cache in the volume for pull requests' firmware, ccache
for the host tests (built for coverage, which the build cache can't keep),
the tools in a venv in the volume, and a tag builds its firmware once.
A release still compiles its own sources from nothing.

Measured on fresh copies of the tree: the firmware step 358 s to 27 s (a new
version and one changed file), tests and coverage 49 s to 33 s, a local
rebuild with nothing changed 77 s to 13 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EhqxQ49eCju4CzKYNjZzwT

.gitea/workflows/ci.yml

@@ -1,14 +1,30 @@
 # CI and releases (docs/milestones/R1.md).
 #   A push to main:    the host tests, with their coverage of lib/, and the README's badges
 #                      published to the branch `badges`.
-#   A pull request:    the same tests and coverage, then the firmware.
+#   A pull request:    the same tests and coverage, then the firmware (from the build cache).
 #                      A branch's pushes run nothing by themselves: its pull request runs, once.
-#   A tag v*:          all of it, then a Gitea release with the signed Update File.
+#   A tag v*:          the tests, then the firmware built once, clean, signed and published as a
+#                      Gitea release.
 #   Run by hand:       the release of a tag that exists already (the ones from before CI).
 #
 # The job runs in a plain Python image, as scripts/ci.sh does on a developer's machine, with the
 # toolchains in a Docker volume the runner allows (container.valid_volumes: roro9stack-pio): that
 # volume is the cache. No JavaScript actions, so the image needs no Node: the checkout is git.
+#
+# What the volume keeps between runs, and what makes each go stale (issue #74, R1.md):
+#   /pio/packages, /pio/platforms   toolchains and the framework: by their versions in platformio.ini
+#   .../framework-arduinoespressif32-libs/.roro-sdkconfig.defaults
+#                                   the mark that the framework is already rebuilt with our SDK settings: the
+#                                   project's sdkconfig.defaults, which isn't in git, so that every fresh
+#                                   checkout rebuilt the framework (260 s). The platform checks its hash
+#                                   against platformio.ini's settings, and rebuilds if they differ. It is kept
+#                                   inside the libraries it describes, so it goes when they are reinstalled.
+#   /pio/ci/build-cache             PlatformIO's build cache (SCons): objects by the signature of their
+#                                   sources and command line. For pull requests only: a release compiles
+#                                   its own sources from nothing.
+#   /pio/ci/ccache                  the host tests' objects (they're built for coverage, which the build
+#                                   cache can't keep: it would lose the .gcno files)
+#   /pio/ci/venv                    PlatformIO and gcovr: delete the folder to upgrade them
 name: CI
 on:
   push:
@@ -37,13 +53,23 @@ jobs:
     env:
       PLATFORMIO_CORE_DIR: /pio
       RORO_NO_DOCKER: 1
+      SDK_MARK: /pio/packages/framework-arduinoespressif32-libs/.roro-sdkconfig.defaults
+      CCACHE_DIR: /pio/ci/ccache
+      CCACHE_MAXSIZE: 1G
     steps:
       - name: Tools
         run: |
           apt-get update -qq
-          apt-get install -y -qq --no-install-recommends git build-essential openssl >/dev/null
-          pip install -q --no-cache-dir --root-user-action=ignore platformio gcovr
-          pio --version; df -h /pio | tail -1; ls /pio | head
+          apt-get install -y -qq --no-install-recommends git build-essential openssl ccache >/dev/null
+          mkdir -p /pio/ci
+          if [ ! -x /pio/ci/venv/bin/pio ]; then
+            python -m venv /pio/ci/venv
+            /pio/ci/venv/bin/pip install -q --no-cache-dir platformio gcovr
+          fi
+          ln -sf /pio/ci/venv/bin/pio /pio/ci/venv/bin/gcovr /usr/local/bin/
+          pio --version; df -h /pio | tail -1; du -sh /pio/ci/* 2>/dev/null || true
+          # The build cache only grows: start it again past 3 GB (a full set of objects is 160 MB, and each run adds about 40).
+          if [ "$(du -sm /pio/ci/build-cache 2>/dev/null | cut -f1)" -gt 3072 ] 2>/dev/null; then rm -rf /pio/ci/build-cache; fi
 
       - name: Check out
         run: |
@@ -57,11 +83,20 @@ jobs:
 
       - name: Host tests, and their coverage of lib/
         if: github.event_name != 'workflow_dispatch'
-        run: scripts/coverage.sh
+        run: |
+          export PATH="/usr/lib/ccache:$PATH"  # gcc and g++ through ccache
+          scripts/coverage.sh
+          ccache -s | grep -E 'Hits|Misses' | head -2
 
+      # A pull request only: a tag's firmware is built once, by the release step below.
       - name: The firmware
-        if: github.event_name == 'pull_request' || github.ref_type == 'tag'
-        run: scripts/ci.sh builds
+        if: github.event_name == 'pull_request'
+        env:
+          PLATFORMIO_BUILD_CACHE_DIR: /pio/ci/build-cache
+        run: |
+          [ ! -f "$SDK_MARK" ] || cp "$SDK_MARK" sdkconfig.defaults
+          scripts/ci.sh builds
+          cp sdkconfig.defaults "$SDK_MARK"  # what the framework in the volume is rebuilt with, now
 
       # The README's badges are files on a branch of their own, replaced at each push to main and
       # at each tag (the release badge says which tag is the latest)
@@ -103,7 +138,12 @@ jobs:
           trap 'rm -f "$RORO_OTA_KEY"' EXIT
           printf '%s\n' "$OTA_SIGNING_KEY" > "$RORO_OTA_KEY"
           umask 022
+          # The framework rebuilt with our settings is reused if it matches (the platform checks);
+          # the release's own sources are compiled from nothing, with no build cache.
+          [ ! -f "$SDK_MARK" ] || cp "$SDK_MARK" /tmp/release-src/sdkconfig.defaults
           scripts/release_build.sh /tmp/release-src dist
+          # An old tag has no SDK settings of its own, and no sdkconfig.defaults afterwards: nothing to mark.
+          [ ! -f /tmp/release-src/sdkconfig.defaults ] || [ ! -d "$(dirname "$SDK_MARK")" ] || cp /tmp/release-src/sdkconfig.defaults "$SDK_MARK"
 
       - name: Publish the release
         if: steps.release.outputs.tag != ''