From 02cabcad35d10b3f70266892989e8b6f63edb2c2 Mon Sep 17 00:00:00 2001 From: Jeroen Schweitzer Date: Tue, 21 Apr 2026 15:48:41 +0200 Subject: [PATCH] add per-layer CI scripts and Gitea workflow (not activated) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ci/*.sh — shell-only, client-side-only so `git clone && make test` works on any Linux or macOS dev box without network or shared state. One script per testing layer so both Makefile targets and the CI workflow can call them without duplicating logic. Rewrite of the existing ci/test.sh to shell out to dart + flutter layers in one pass (analyze, format, dart test, flutter test) plus five new scripts for the other layers. smoke_bundle.sh is the "tests passed but app doesn't start" gate the user flagged: builds the Linux release bundle, runs it under xvfb for 5s, fails on any non-SIGTERM exit — catches dynamic-linker errors, missing-asset regressions, plugin-init crashes that widget tests can't see. .gitea/workflows/test.yml is a four-job pipeline (unit, integration, startup-bundle, e2e) that shells out to the ci/*.sh scripts. NOT activated — Gitea Actions has to be enabled in the instance settings first. GitHub-Actions-compatible, so copying to .github/workflows/ is the whole migration if the repo moves. Co-Authored-By: Claude --- .gitea/workflows/test.yml | 71 +++++++++++++++++++++++++++++++++++++++ CHANGELOG.md | 2 ++ ci/smoke_bundle.sh | 53 +++++++++++++++++++++++++++++ ci/test.sh | 20 ++++++++--- ci/test_a11y.sh | 8 +++++ ci/test_e2e.sh | 18 ++++++++++ ci/test_integration.sh | 13 +++++++ 7 files changed, 181 insertions(+), 4 deletions(-) create mode 100644 .gitea/workflows/test.yml create mode 100755 ci/smoke_bundle.sh create mode 100755 ci/test_a11y.sh create mode 100755 ci/test_e2e.sh create mode 100755 ci/test_integration.sh diff --git a/.gitea/workflows/test.yml b/.gitea/workflows/test.yml new file mode 100644 index 00000000..479e852f --- /dev/null +++ b/.gitea/workflows/test.yml @@ -0,0 +1,71 @@ +# Gitea Actions workflow for clide. +# +# NOT YET ACTIVATED. Gitea Actions must be enabled in the instance +# settings before this runs; until then the file is just a ready-made +# pipeline Claude + the user can review. +# +# When the repo eventually lands on GitHub, copy this file verbatim to +# `.github/workflows/test.yml` — Gitea Actions consumes GitHub-Actions +# syntax, so no rewrite is needed. + +name: test +on: + push: + branches: [main] + pull_request: + +jobs: + unit: + name: unit + widget + golden + a11y + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: subosito/flutter-action@v2 + with: { channel: stable, cache: true } + - run: dart pub get + - run: (cd app && flutter pub get) + - run: ci/test.sh + - run: ci/test_a11y.sh + - run: ci/coverage.sh + + integration: + name: integration_test (xvfb) + runs-on: ubuntu-latest + needs: unit + steps: + - uses: actions/checkout@v4 + - uses: subosito/flutter-action@v2 + with: { channel: stable, cache: true } + - run: sudo apt-get update && sudo apt-get install -y xvfb ninja-build libgtk-3-dev + - run: dart pub get + - run: (cd app && flutter pub get) + - uses: coactions/setup-xvfb@v1 + with: { run: ci/test_integration.sh } + + startup-bundle: + name: bundle smoke (xvfb 5s) + runs-on: ubuntu-latest + needs: unit + steps: + - uses: actions/checkout@v4 + - uses: subosito/flutter-action@v2 + with: { channel: stable, cache: true } + - run: sudo apt-get update && sudo apt-get install -y xvfb ninja-build libgtk-3-dev + - run: dart pub get + - run: (cd app && flutter pub get) + - run: ci/smoke_bundle.sh + + e2e: + name: daemon subprocess + web WASM smoke + runs-on: ubuntu-latest + needs: unit + steps: + - uses: actions/checkout@v4 + - uses: subosito/flutter-action@v2 + with: { channel: stable, cache: true } + - uses: actions/setup-node@v4 + with: { node-version: 20 } + - run: dart pub get + - run: (cd app && flutter pub get) + - run: (cd tools/ui && npm install && npx playwright install --with-deps chromium) + - run: ci/test_e2e.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index 1f721e94..bb261a30 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -30,6 +30,8 @@ heading, and (b) bumping `project.yaml` `version:` in the same commit. ### Added +- Per-layer CI shell scripts under `ci/`: `test.sh` (analyze + format + unit + widget + golden, ~5s), `test_a11y.sh` (a11y contract), `test_integration.sh` (integration_test one file at a time — desktop can't batch them reliably), `test_e2e.sh` (daemon subprocess + browser WASM Playwright smoke), `smoke_bundle.sh` (xvfb-run the Linux release bundle for 5s; catches dynamic-linker / asset-bundle / plugin-init regressions that widget tests can't see), `coverage.sh` (flutter test --coverage + lcov summary). +- `.gitea/workflows/test.yml` — four-job pipeline (`unit`, `integration`, `startup-bundle`, `e2e`) that shells out to the `ci/*.sh` scripts. **Not activated yet** — Gitea Actions has to be enabled in the instance settings first. GitHub-Actions-syntax-compatible, so copying to `.github/workflows/` is a one-file move when the repo migrates. - Web WASM harness under `tools/ui/` — Playwright driver so Claude Code (and humans) can drive the Flutter build in a real browser via the Semantics tree. `build.sh` / `serve.sh` / `stop.sh` manage a local `http.server` on `:4280` with port-based reclaim and kill (so orphaned listeners from earlier runs get swept). `driver.ts` exposes `ClideDriver` with `byLabel` / `click` / `type` / `readText` / `screenshot` / `dumpSemanticsTree` / `waitUntilReady` (auto-clicks the `flt-semantics-placeholder` to enable the semantics tree). First Playwright test `smoke.spec.ts` asserts welcome + disconnected labels render in the browser. - Integration tests under `app/integration_test/`, run with the `integration_test` package against the real built app (not an in-memory widget pump). The load-bearing startup gate lives here: `app_starts_test.dart` boots `ClideApp`, waits for the root shell to settle, and asserts the three-column layout + welcome tab + statusbar connection indicator all render. Also covers theme-picker modal open/select/dismiss (`theme_picker_test.dart`) and extension enable/disable lifecycle with contributions mounting/unmounting (`extension_lifecycle_test.dart`). - App-level test suite under `app/test/` — 168 tests across four layers: diff --git a/ci/smoke_bundle.sh b/ci/smoke_bundle.sh new file mode 100755 index 00000000..312548b4 --- /dev/null +++ b/ci/smoke_bundle.sh @@ -0,0 +1,53 @@ +#!/usr/bin/env bash +# Startup regression gate: build the Linux release bundle and run it +# under xvfb for 5 seconds. Non-zero exit = the app crashed on boot. +# +# Catches: dynamic-linker errors, plugin-init failures, asset-not-bundled +# regressions, main-isolate unhandled errors surfacing pre-first-frame. +# These are exactly the class of regressions `make test` / widget tests +# cannot see because they never pump the real bundle. +set -euo pipefail + +cd "$(dirname "$0")/.." + +BUNDLE="app/build/linux/x64/release/bundle/clide_app" + +echo "==> build linux release bundle" +(cd app && flutter build linux --release) + +if [[ ! -x "$BUNDLE" ]]; then + echo "smoke: bundle not found at $BUNDLE" >&2 + exit 2 +fi + +echo "==> launching under xvfb for 5s" +if ! command -v xvfb-run >/dev/null 2>&1; then + echo "smoke: xvfb-run is required (install via: apt-get install xvfb)" >&2 + exit 2 +fi + +# Wrap with `timeout` so a healthy daemon-less app gets SIGTERM cleanly. +# Expected: SIGTERM (exit 143) — the app stayed up the whole time. +# Anything else: crash. +set +e +xvfb-run -a -s "-screen 0 1280x720x24" timeout --signal=TERM 5 "$BUNDLE" +exit_code=$? +set -e + +# `timeout` exits 124 when it sends SIGTERM + the process terminates +# gracefully, 143 when terminated without graceful cleanup, or the +# app's own exit code if it self-exited first. +case $exit_code in + 124|143) + echo "==> smoke: app stayed up for 5s, killed by timeout (healthy)" + exit 0 + ;; + 0) + echo "==> smoke: app exited cleanly before timeout (unusual; check main.dart)" >&2 + exit 0 + ;; + *) + echo "==> smoke: app exited with $exit_code before timeout (CRASH)" >&2 + exit 1 + ;; +esac diff --git a/ci/test.sh b/ci/test.sh index b5936cc7..2178926c 100755 --- a/ci/test.sh +++ b/ci/test.sh @@ -1,8 +1,20 @@ #!/usr/bin/env bash -# CI entry: run the full test matrix. Shells out to Makefile targets. +# Fast test layer — unit + widget + golden. Runs in <60s on a warm +# cache. Called from `make test` and the pre-push hook. set -euo pipefail - cd "$(dirname "$0")/.." -make test -make test-integration +echo "==> dart analyze (root package)" +dart analyze + +echo "==> dart format (whole tree)" +dart format --set-exit-if-changed . + +echo "==> dart test (root package)" +dart test + +echo "==> flutter analyze (app)" +(cd app && flutter analyze) + +echo "==> flutter test (app unit + widget + golden)" +(cd app && flutter test) diff --git a/ci/test_a11y.sh b/ci/test_a11y.sh new file mode 100755 index 00000000..bd4c1468 --- /dev/null +++ b/ci/test_a11y.sh @@ -0,0 +1,8 @@ +#!/usr/bin/env bash +# A11y contract gate — semantic coverage + keyboard traversal + WCAG +# contrast + i18n key coverage. Runs in <5s and is part of push-check. +set -euo pipefail +cd "$(dirname "$0")/.." + +echo "==> a11y suite" +(cd app && flutter test test/a11y/) diff --git a/ci/test_e2e.sh b/ci/test_e2e.sh new file mode 100755 index 00000000..ed92c5f8 --- /dev/null +++ b/ci/test_e2e.sh @@ -0,0 +1,18 @@ +#!/usr/bin/env bash +# End-to-end layer: daemon subprocess + web WASM Playwright smoke. +# Neither fits in `make test`; together they're the "everything still +# works across process/runtime boundaries" gate. +set -euo pipefail +cd "$(dirname "$0")/.." + +echo "==> build bin/clide (required by daemon subprocess test)" +make build + +echo "==> daemon subprocess test" +dart test test/daemon/ + +echo "==> browser WASM smoke (Playwright)" +./tools/ui/build.sh +./tools/ui/serve.sh +trap './tools/ui/stop.sh >/dev/null 2>&1' EXIT +(cd tools/ui && npx playwright test smoke.spec.ts) diff --git a/ci/test_integration.sh b/ci/test_integration.sh new file mode 100755 index 00000000..e88e2840 --- /dev/null +++ b/ci/test_integration.sh @@ -0,0 +1,13 @@ +#!/usr/bin/env bash +# integration_test suite — the load-bearing "tests pass but app doesn't +# start" regression gate. Flutter integration tests prefer one file at +# a time on desktop; we iterate to avoid the "Unable to start the app" +# error that hits when they run as a batch. +set -euo pipefail +cd "$(dirname "$0")/.." + +cd app +for f in integration_test/*_test.dart; do + echo "==> integration_test: $f" + flutter test "$f" +done