Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
50 most recent check-ins
|
2026-08-06
| ||
| 18:58 | Jenkinsfile: run every containerisable stage through bin/ci Routes junit/rove/parser/web/cli through the container stack now that it is green end-to-end on the node. Android XR stays on the host -- multi-GB SDK plus licence acceptance for one of eighteen suites -- and is marked skip when that toolchain is absent, so a node without it gives an honest skip, not a failure. Replaced the Schema stage with Build CI image: each bin/ci invocation now migrates and seeds its own throwaway database, so there is nothing for a shared schema stage to do. The build is layer-cached; only robotarme.asd, package-lock.json or ci/ changes cost a real rebuild (~9 min cold, almost all of it preloading the 21 Lisp systems). Removed the environment{} database block entirely. Those values now live in ci/compose.yaml pointing at the throwaway postgres; keeping them here too would be two sources of truth for one fact -- and until the load-env-defaults fix a committed .env silently overrode the whole block anyway, so a build could connect somewhere other than it was told and never say so. disableConcurrentBuilds stays, but the comment now states the ACTUAL remaining reason: server and Playwright live inside the runner so port 8000 no longer collides and each build has its own database, but bin/ci uses a fixed COMPOSE_PROJECT_NAME, so two builds would share one stack and tear it down under each other. Lift it by making the project name per-build, not by assuming the old conflicts are gone. Corrected the header prerequisite: the agent needs a container runtime and a compose implementation, nothing else. Re-validated with jenkins-sim.p: 9 stages, 0 errors, PREDICTED SUCCESS. FSD R31 -> Mostly done; the only thing left is pointing a controller at the repo. Leaf check-in: f5cc22a051 user: dkords tags: trunk | |
| 18:54 | CI stack fully green in containers; root cause of the 49 CSP failures found ALL SUITES GREEN on red5buntu: rove 1066, /console 27, xr-themes 13, CLI 8, parser 22, JUnit emitter 8, and the rest -- 1140 assertions, 2 honest skips. ROOT CAUSE of the 49 "Refused to evaluate a string as JavaScript" failures: NOT the Content-Security-Policy, and not a code defect. My own rsync was flattening `ci/Dockerfile` to the repo ROOT on the node, so every build kept using a stale ci/Dockerfile pinned to Playwright 1.47.0. That base ships chromium-1134, whose page.waitForFunction still evaluated in the main world where the app's CSP correctly refuses eval. package.json says "^1.47.0" but the caret floats and the developer machine had resolved to 1.59.1 / chromium-1217 -- so dev and CI were running browsers two years apart. With the correct base, 9 of the 10 failing suites passed immediately. The image now installs from package-lock.json via `npm ci`, so it cannot drift from what the developer machine resolved, and the build ASSERTS that the browser the installed playwright wants actually exists in the image -- the mismatch fails the build loudly instead of surfacing as 49 confusing EvalErrors. B-2 / B-3 were a genuinely WRONG ASSERTION, not a race, and the buffer dump proved it: the tail showed "1 of 1 devices" with only the matching row, and the API returned exactly one device. #tui-output holds the whole session scrollback and the TUI re-emits it, so `clear` does not durably remove earlier lines and a whole-buffer includes() also matched the echo of `devices register <name>-rpi` from earlier in the same test. Assertions about what a command did NOT print are now scoped to that command's own output via outputSince(). The server was innocent throughout -- rove's "hardware filter (exact match)" and "paginated device list respects page + per_page + search + sort" are green in the same container. Two more environment assumptions: - node --test changed its DEFAULT reporter from tap (Node 20, dev) to spec (Node 24, image), so the count parse produced an EMPTY number -- the summary read "pass assertions". Now pins --test-reporter=tap. - bin/test-cli-upload-source-url.sh hardcoded ~/quicklisp; HOME is /tmp in the container. Honours QUICKLISP_HOME. Verified BOTH ways: dev still green (/console 27) and container green. Fixes are environment-independent, not container-specific. check-in: 617fc53672 user: dkords tags: trunk | |
| 09:59 | CI: fixture gap, explicit skips, and two more host assumptions FIXTURE GAP. Three suites defaulted to `splatlab`, an account hand-made during the splat-viewer work that exists on exactly one machine. It is NOT in *seed-users* and MUST NOT BE ADDED: bin/seed calls mito:delete-dao on seed users and deletes their personal orgs, so seeding splatlab would destroy the headset scans it owns in org 9030. Verified in src/migrations.lisp before touching anything. - tests/cli.test.mjs now defaults to admin_alice/testpassword123, which is seeded everywhere and owns Acme Corp. That suite only lists devices, so it needs nothing splatlab had. - scan-lab and mobile-splat KEEP splatlab as the default, because on a developer machine it exists and gives them real coverage (a genuine captured room to render). What changes is the failure mode: when the account or the done-scan is absent, they now print "SKIP: <reason>" and exit 3 instead of failing. bin/test honours that: an explicit SKIP: sets status=skip and prints the reason BEFORE the no-summary branch, which would otherwise read a deliberate skip as a suite that died. A crash still fails — verified all three states are distinguished (skip / crash / pass) rather than assumed. The reason is printed every run, because a silently skipped suite is one you have stopped running, and this repo has been bitten by that twice. TWO MORE HOST ASSUMPTIONS, same family as /etc/sbclrc and CL_SOURCE_REGISTRY: - bin/test-cli-upload-source-url.sh hardcoded ~/quicklisp/setup.lisp. HOME is /tmp in the container, so that path does not exist and phase 1 failed with an unhelpful exit 1. Now honours QUICKLISP_HOME with the old path as fallback. - An rsync typo taught me the image had been building against a STALE Dockerfile: `rsync ci/Dockerfile host:~/repo/` flattens to ~/repo/Dockerfile rather than ~/repo/ci/Dockerfile, so ci/ kept the old Playwright pin. The tell was chromium-1134 in the image where 1.59.1 should ship a much newer build — worth recording because the symptom looked like the pin not working rather than the file not arriving. Still open and being measured: 49 Playwright failures are all the app's own Content-Security-Policy refusing eval() from page.waitForFunction. The same tests pass on the developer machine against the same CSP, so the difference is environmental rather than a code defect; the corrected-base rebuild is running now to find out whether the Playwright/Chromium pairing explains it. check-in: 210aceeb07 user: dkords tags: trunk | |
| 09:46 | CI stack: six real failures found by actually building it on the node Every one of these was invisible to reasoning and obvious to a build. 1. GLIBC. SBCL's prebuilt Linux binaries link against GLIBC_2.38. The Playwright jammy base is Ubuntu 22.04 (glibc 2.35), so install.sh succeeded and `sbcl --version` immediately died with "version `GLIBC_2.38' not found". Switched the base to v1.47.0-noble (24.04, glibc 2.39) -- same pinned Playwright. 2. Build context. The context is the repo root and Docker ships it to the daemon before the build starts: 5.0 GB / 50k files here, because contributor/ (2.3 G), vendor/ (1.5 G) and mapper/ (1.2 G) carry artifacts the image has no use for. Added .dockerignore as an ALLOWLIST -- ignore everything, re-admit only robotarme.asd and ci/preload-deps.lisp -- so a new large directory added later is excluded by default rather than silently inflating every build. Does not affect what the suite sees: the working tree arrives through the bind mount, not the image. 3. Quicklisp had to be WRITABLE, not merely readable. Symptom was "Permission denied" on an archive path that did not exist, which reads like a read-permission bug and is not one: quicklisp was trying to CREATE the archive while installing a system the preload had missed. 4. Three systems load LAZILY and no declared dependency reaches them -- lack-app-file, lack-middleware-static, trivial-rfc-1123. Caveman2 builds the Clack app at load time and pulls the static-file middleware chain, which ql:quickload's declared-graph walk cannot see. The failure was "Component trivial-rfc-1123 not found" from inside a nested load, even though (ql:quickload :trivial-rfc-1123) alone works fine. The list is DERIVED, not guessed: quickload the declared deps, snapshot asdf:map-systems, quickload the project, diff. Exactly three. 5. BASH 5.2 CHANGED PATTERN-SUBSTITUTION SEMANTICS, and it silently corrupted the JUnit XML. An unescaped `&` in the replacement of ${var//x/y} now means "the text that matched", sed-style. So ${name//</<} emitted `<lt;` and ${name//\"/"} emitted `"quot;` -- an unparseable document. bash 3.2 on macOS treats it literally, so the identical code passed on the dev machine and failed in the container. Replaced with an explicit character loop, which has no such ambiguity; only names containing a metacharacter pay for it. tests/junit-emit.test.sh caught this, which is the entire reason it exists. 6. The Playwright base ships the BROWSERS, not the npm package, so every one of the twelve Playwright suites died with ERR_MODULE_NOT_FOUND. Installed at /opt/pw and symlinked to /node_modules: it cannot live in /work, which the bind mount hides at run time, and npm refuses to run in / directly (`Tracker "idealTree" already exists`). Node's resolver walks up, so /work/tests/*.mjs finds it. bin/ci test now migrates and seeds inside the invocation, because each run gets a brand-new empty postgres -- there is no schema to inherit the way there is on a developer machine. Verified in-container so far: rove 1066 assertions green, Node parser 22 green, JUnit emitter 8 green. Playwright + CLI results pending the current run. check-in: 4fd3d2e574 user: dkords tags: trunk | |
| 08:34 | CI stack in containers: postgres + a runner carrying the whole toolchain Owner was right and I argued too long. The decisive point I undervalued: this sidesteps the hard part of R32 entirely. Nix-packaging Playwright's browsers means version-matching them to the npm package, which is the fiddliest dependency in this repo; mcr.microsoft.com/playwright gives it away free, along with Node 20 and Chromium's GL/font/X library set. ci/Dockerfile -- Playwright v1.47.0-jammy base, plus the two things it lacks: SBCL 2.5.9 from the official binary release (Ubuntu 22.04 ships 2.1.11 from 2021, which is exactly what the CI node has and why it could not run the suite), and Quicklisp with BOTH dists. Both dists matters: dexador and lack resolve from ultralisp, so an image with only the default dist builds a different dependency graph than the developer machine -- the same works-on-my-machine this is meant to end. ci/preload-deps.lisp bakes the dependency set into a layer, READING :depends-on out of robotarme.asd rather than hardcoding a list, so the image cannot drift from the .asd. Only the .asd is COPYed, never src/, so a code change does not invalidate the layer. Verified against the real .asd: extracts 21 systems (20 from robotarme + rove), correctly dropping the project's own systems. ci/compose.yaml -- postgres:17-alpine with PGDATA on tmpfs and fsync off (the data is discarded; measured 41 MB RAM for an empty cluster), healthcheck so the runner cannot race it, and a runner that mounts the checkout rather than baking it. NOTHING is published to the host: the app server and Playwright both live inside the runner, so the suite talks to 127.0.0.1:8000 within the container. That also removes the port-8000 collision between builds, which is the constraint behind disableConcurrentBuilds. bin/ci wraps it: build | test [subset] | shell | down. `compose down -v` runs on trap EXIT so an interrupted run leaves no containers, network or volumes. It accepts docker compose, docker-compose or podman-compose, so the host can end up with rootful docker, rootless docker or podman without changing anything. THREE HIDDEN HOST DEPENDENCIES found while writing this, all of the "works on the machine that happened to run it" family: - bin/test's rove runner invokes sbcl WITHOUT loading setup.lisp; it works only because a developer's ~/.sbclrc loads quicklisp. Fixed with a system-wide /etc/sbclrc (sysinit, runs before userinit, HOME-independent). - ASDF finds robotarme only because the repo happens to sit under ~/common-lisp/, an ASDF default search path. Fixed with CL_SOURCE_REGISTRY. - ASDF writes FASLs to $HOME/.cache/common-lisp. Preloading as root under /root would leave them unreadable by the unprivileged runtime user and every CI run would recompile all 21 systems. HOME is set before the preload step so build and run share it. Pre-validated without a container runtime (this Mac has none): both image tags resolve (playwright v1.47.0-jammy 200, postgres:17-alpine 200), the SBCL 2.5.9 tarball URL resolves 200, bin/ci and the extractor pass syntax and behaviour checks. NOT yet run. Needs docker access on red5buntu -- the user is not in the docker group there. Android XR is deliberately excluded from the image: it needs a multi-GB SDK plus licence acceptance for one of eighteen suites, so that stage stays on the host until it earns its own image. check-in: 5724cce3e3 user: dkords tags: trunk | |
| 08:04 | Ephemeral test database, and fix .env silently overriding the environment Validated against red5buntu (the `nix` CI node) rather than assumed. Findings changed the design twice. WHY NOT A CONTAINER. The node has NO postgres at all, so something must supply the binary; Nix already does (postgresql_17 from cache.nixos.org, 8 MB, ~4 s cold and instant warm). Once the binary is free a container is a SECOND, separately-maintained source of a dependency we already have -- plus an image to store, a runtime to keep working, and, with the Docker actually installed there, either root or docker-group membership, which is root-equivalent on the host. podman is not installed on that node; Docker 29.1.3 is, and it is root-gated. Measured on the node, PGDATA on its existing 3.2 G tmpfs: initdb 0.33s | start 0.11s | createdb 0.03s | TOTAL 0.47s 41 MB PGDATA (RAM, zero disk), 65 MB postgres RSS, 1.3% of the tmpfs bin/with-ephemeral-db (new) wraps any command: picks a tmpfs dir, picks a free port in 55432-55532 (never 5432 -- that is what a dev box already runs), initdb, starts with fsync/full_page_writes/synchronous_commit all off since the data is discarded, migrates, seeds, runs, and destroys on trap EXIT. It re-execs itself under `nix shell` when the host has no usable server, so the caller never has to remember. Two things testing caught that reasoning had not. Probing `initdb` alone is not enough: Homebrew's client-only libpq ships initdb and pg_ctl but no server, and even when a `postgres` binary IS on PATH from a different prefix, initdb refuses because it looks for the server NEXT TO ITSELF -- so the guard checks that adjacency, which is initdb's own test. And the socket directory has to be created AFTER initdb, which rejects a non-empty data directory. THE .ENV BUG, which blocked all of this and would have broken CI on its own. cl-dotenv:load-env does (setf (uiop:getenv name) value) unconditionally, so: 1. .env BEAT the real environment -- inverting the convention every other ecosystem uses. The Jenkinsfile's `environment { POSTGRES_* }` block was being silently overridden by whatever .env sat in the agent workspace, so a build would connect somewhere other than where it was told to and nothing would report it. Proven: POSTGRES_PORT=55432 became 5432. 2. A missing .env was FATAL, because config.lisp calls it as a top-level form. .env is deliberately untracked, so `ql:quickload :robotarme` failed outright on every fresh checkout -- including the CI node's. robotarme.config:load-env-defaults now treats .env as DEFAULTS ONLY (a variable already set in the environment is left alone) and tolerates an absent file. All 6 call sites across src/ and tests/ converted. Verified: rove 1066 assertions green BOTH against a fully ephemeral database (fresh initdb + migrate + seed, env supplies config) and against the host database (.env supplies config). No leftover PGDATA, no stray postgres, host database untouched at 748 users. NOT yet wired into the Jenkinsfile: `cli`, `scan-lab` and `mobile-splat` still depend on a hand-made `splatlab` account that is not in *seed-users*, and scan-lab needs a `done` scan whose GLB lives in the untracked 3.5 GB storage/. Those fail on any fresh database regardless of how it is provisioned, and that gap is next. check-in: 499e975563 user: dkords tags: trunk | |
| 06:26 | Jenkinsfile: pin the build to the `nix` agent label Was `agent any`. The stages assume a specific toolchain, so a build landing on an agent without it fails in ways that read as test failures rather than a missing dependency. Also corrected the prerequisite comment: the label names the intent, it does not enforce it. The nix/probe flake covers SBCL, Postgres, OpenSSL, zlib, fossil and curl -- not Node, Playwright browsers, the JDK or the Android SDK -- so until R32 promotes a root flake covering the whole set, the agent still has to supply those by hand. Re-validated with jenkins-sim.p: 9 stages, 0 errors, PREDICTED SUCCESS. check-in: 03b23ad2bc user: dkords tags: trunk | |
| 06:08 | CI hardening (FSD R31): JUnit XML, the orphan suite wired in, rove teardown Three gaps that all had the same shape -- the harness could report success without having earned it. 1. JUnit XML (bin/lib/junit.sh, new). bin/test printed human summaries, so Jenkins could only learn a stage's exit code: no per-test history and no way to notice a suite whose test COUNT collapsed. This repo has shipped that bug twice (mobile-splat green while running zero tests; the rove counter blind to described assertions). Every suite now emits test-results/junit/<slug>.xml -- 17 files, 1195 testcases. Gradle's Android XR suite is deliberately excluded because it already emits real XML. The emitter enforces three things a naive one would not: a suite whose status is fail always contributes at least one failing testcase (even if it died before printing any); a suite that produced nothing renders as 1 test, never 0, because "0 tests" reads as green everywhere; and marks are matched anywhere in a line, not at column 1 -- the test server writes to the same stdout and its chatter shares lines with rove's marks, which silently dropped 21 of 1065 assertions on the first attempt. tests/junit-emit.test.sh (new, 8 assertions) asserts exactly those guarantees, plus XML escaping and well-formedness. It runs as its own first stage, since a broken reporter makes every later stage meaningless. junit_init only wipes results for a full run: CI invokes bin/test once per stage, so an unconditional wipe would have each stage delete the previous one's XML and Jenkins would publish only the last. 2. bin/test-cli-upload-source-url.sh wired in as an 18th suite. It existed and passed for months with nothing running it, and it covers the one upload mode nothing else does -- server-side source_url fetch, the path the SSRF guard protects. Converted to the ✓/✗ + "N passed" shape the harness already parses, and it now DELETES the release it creates: one row per run, nightly, is how the last pile accumulated. 3. Rove teardown. The suite registered ~22 users per run and abandoned all of them with their personal orgs -- the reason bin/purge-test-data exists and the dev DB reached 726 users. Recording happens in http-post, the single choke point every registration passes through, so a test added later is cleaned up without its author knowing teardown exists. Teardown is the last deftest rather than a hook because rove has no after-suite hook and a silent teardown is indistinguishable from one that never ran; its assertion is "the rows are gone", so a teardown that stops working fails the build instead of quietly resuming the leak. Two things the first version got wrong, both found by measuring rather than assuming: `organization` has no owner column (membership is the only link, so an org is ours only when bool_and says EVERY member is one of this run's users -- an org containing a real account is left alone), and the org-id query reads organization_member, which the delete sequence empties two statements earlier, so the ids must be materialized first or the orgs are orphaned rather than deleted. Also removes personal orgs whose membership some test already deleted, matched by the server's "<username>'s Org" convention -- that was the last row leaking per run. Verified: full ./bin/test green -- 18 suites, 1258 assertions. DB delta across a full run is now users=0 orgs=0 (was +22 users, +22 orgs). All 17 XML documents parse and their headers match their element counts. Jenkinsfile re-validated with jenkins-sim.p (9 stages, PREDICTED SUCCESS). check-in: 7e928e200b user: dkords tags: trunk | |
| 04:25 | FSD: correct R31/R32 status — both said "Not started" and both had moved R31 now Partial, not Not started: the Jenkinsfile exists, has 8 stages, and was validated offline with jenkins-sim.p including a negative test. The entry still said "No Jenkinsfile exists." Recorded what actually remains -- no controller runs it, no JUnit XML from bin/test for the rove/Playwright/CLI stages, and bin/test-cli-upload-source-url.sh still unwired. R32 now Partial: the probe exists and proved the Nix/Quicklisp split works (1065/0 inside the devshell). Also records that the probe's original SBCL 2.6.6 finding was DISPROVED yesterday -- confounded experiment, real cause was trivial-mimes' file(1) flag spelling -- so no SBCL pin is needed. Removed ESP-IDF from R32's scope: nothing in bin/test or the Jenkinsfile builds firmware, which lives in ProjectMagNET. check-in: 098c962348 user: dkords tags: trunk | |
| 04:01 | Fix multipart uploads breaking wherever file(1) is not Apple's (CLI 0.33.1) Diagnosed the rove failures that this week's Nix probe blamed on SBCL 2.6.6. SBCL was innocent. The experiment was confounded: `nix develop` changes the compiler AND PATH together, and only PATH mattered. Cause: dexador asks trivial-mimes for the MIME type of any pathname passed as a multipart part. trivial-mimes shells out to file(1) with no error handling, choosing the flag at COMPILE time -- #+darwin "-bI", #-darwin "-bi". "-I" is Apple's spelling. In a Nix devshell on macOS the platform is still darwin but PATH resolves to upstream file-5.48, where -I is invalid and exits 1. So #+darwin was true and wrong simultaneously, and every upload died with an unactionable SUBPROCESS-ERROR that cascaded through the release tests. This was a latent CLI defect, not just a test artifact: `robotarme releases upload --file` failed identically, and `brew install file` would trigger it without Nix anywhere. Fix at all three multipart call sites: pass (pathname :content-type "...") instead of a bare pathname, so trivial-mimes is never consulted, plus an explicit Content-Type: multipart/form-data header -- dexador sniffs multipart-vs-urlencoded by looking for a bare pathname among the values and would otherwise URL-encode the list. Dexador consumes that header and appends its own "; boundary=", so it is not duplicated. The server ignores the part's content type and trusts artifact_type, so probing a firmware blob bought nothing in exchange for a subprocess per upload. Verified: SBCL 2.6.6 (nixpkgs, devshell) 1065 assertions, 0 failures SBCL 2.5.9 (Homebrew, bin/test) 1065 assertions, 0 failures Live `releases upload` of the same 4 KB file in both environments produced the identical server-side sha256, so the wire format is unchanged. Both probe releases were deleted afterwards. nix/probe/README.md rewritten: it asserted "the only variable is the compiler", which was false. SBCL 2.6.6 needs no pin; what still gates promoting the flake to the repo root is toolchain coverage (no Node/Playwright/JDK/Android SDK). docs/lisp-dev-setup.md gains the general rule -- the Nix/Quicklisp split is clean except where a CL library shells out, and #+darwin is a claim about the kernel, never about which binaries are on PATH. check-in: be72dcd8fe user: dkords tags: trunk | |
|
2026-08-05
| ||
| 23:31 | nix/probe — R32 feasibility probe: devShell works, but SBCL 2.6.6 breaks the suite Answering "can you actually run a flake devShell for the Lisp setup" by running one rather than reasoning about it. Two results, and the second is the one that matters. THE MECHANISM WORKS. `nix develop` gives a shell with SBCL, PostgreSQL client, fossil 2.28 and OpenSSL from the store, and the EXISTING USER QUICKLISP loads inside it — `ql:quickload :robotarme` returns LOADED OK. So the split recorded in docs/lisp-dev-setup.md is viable in practice, not just in principle: Nix supplies the system, Quicklisp stays the single Lisp dependency manager, and no CL library needs Nix-packaging. No second road into Lisp. THE SUITE IS NOT CLEAN ON NIXPKGS' SBCL. nixpkgs ships 2.6.6; we develop on Homebrew 2.5.9. SBCL 2.5.9 (Homebrew) 0 failures SBCL 2.6.6 (nixpkgs) 1032 assertions, multiple failures Identical invocation, same machine, same Quicklisp, same database — the only variable is the compiler, which I checked precisely because "it fails under Nix" is the kind of claim that is usually an environment difference. It is not. The failures cluster in the release upload / signed-download path: (= STATUS 200), (EQ (GETF DATA :|ok|) T), (SEARCH "/download?expires=" URL), plus several bare "Raise an error while testing". Not diagnosed further yet, and deliberately NOT adopted as the project flake. Committing a flake.nix at the repo root that produces a failing test suite would be worse than having none — it would look like infrastructure while quietly meaning "tests fail here". Options recorded in nix/probe/README.md, in order: diagnose the 2.6.6 failures (best — the suite should pass on a current compiler, and "our code only works on one SBCL patch release" is a defect in its own right); pin 2.5.9 via an older nixpkgs input or overlay (reproducible now, freezes us on a compiler nobody chose); or Nix only the non-Lisp tooling (half the benefit, and it reintroduces the two-roads ambiguity the split was meant to remove). FSD R32 updated with the probe result and the blocker, so the requirement now carries what is known rather than an assumption that it is straightforward. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 042b013d81 user: dkords tags: trunk | |
| 22:50 | docs: R32 drops ESP-IDF (it was wrong); record the Lisp dev-environment rules Owner asked whether the server really needs ESP-IDF to be verified. It does not, and listing it in R32 yesterday was my error. Checked rather than argued: nothing in bin/test or the Jenkinsfile references idf.py, and the C6 firmware lives in ProjectMagNET — a separate git repository. Bundling a ~2 GB cross-compiler toolchain into the server's build environment to verify a server that never touches it is exactly the accretion R32 exists to prevent. When the WaveC6LED gets CI it wants its own pipeline and its own flake, next to the code it builds. What this repo genuinely needs is now stated: SBCL + Quicklisp, Node + Playwright browsers, Android SDK + a JDK pinned to 17-21, and PostgreSQL. The Android SDK IS in scope, because androidxr/ lives in this repo and ./bin/test androidxr runs its JVM suite — that is the distinction the earlier draft blurred by listing everything that happened to be installed on one laptop. NEW docs/lisp-dev-setup.md, from the owner's gist. The three "don'ts" are kept prominent because they are the part that costs time: .emacs and .emacs.d/init.el both existing means the second silently never loads; enabling many packages at once makes interactions unattributable; and don't install your road into Lisp via multiple different ways in the same environment. That third one has architectural consequences for R32, so the doc spells them out rather than leaving a maxim. Nix packaging CL libraries ALONGSIDE Quicklisp would be precisely the multiple-roads trap — two managers owning one dependency graph, resolving differently, disagreeing as "works on my machine". So the split is Nix supplies the SYSTEM (SBCL, Postgres, Node, Playwright browsers, JDK, Android SDK, fossil) and Quicklisp remains the single Lisp dependency manager. Every bin/* entry point already loads $QUICKLISP_HOME/setup.lisp; that stays true, and robotarme.asd gets no Nix-specific conditionals, because a build that behaves differently under Nix is a build with two personalities. Also in the Jenkinsfile: a junit step for the Android XR stage. I had written "no junit anywhere" and that was too broad — Gradle emits 13 JUnit XML files under app/build/test-results/, so that one stage can have real per-test trends. The rove/Playwright/CLI runners still print human summaries only, and declaring a glob for them would show a permanently green "0 tests" trend, which is worse than no trend. Verified while answering: Nix is already installed here (Determinate 3.21.1, flakes enabled) and nixpkgs#fossil is packaged at 2.28 — asked the local nix rather than recalled. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 7381d842b5 user: dkords tags: trunk | |
| 20:35 | Jenkinsfile — CI pipeline (FSD R31), validated with the pop11 jenkins simulator Eight stages: Checkout, Schema (bin/migrate), Lisp/rove, Node parser, Android XR JVM, Web+XR Playwright, CLI end-to-end, i18n audit. Per-stage timeouts plus a 60-minute pipeline cap, disableConcurrentBuilds, buildDiscarder, nightly cron, and post.unsuccessful mail. disableConcurrentBuilds is not boilerplate here: the suites bind port 8000 and share one Postgres database, so two builds at once would fight over both and fail in ways that look like flaky tests rather than contention. SCOPE STATED HONESTLY IN THE FILE: this runs everything needing no physical hardware. The WaveC6LED device suites stay manual because they need the board on a USB port, and pretending otherwise would produce a green build that says nothing about the firmware. Also noted: the agent must already provide SBCL + Quicklisp, Node + Playwright browsers, a JDK in 17..21 and Postgres — a set reproduced by hand today, which makes this pipeline only as reproducible as whoever built the agent. That is FSD R32 (Nix), and it is a prerequisite rather than a parallel nicety. No junit step, deliberately, with the reason in the file: our runners print human summaries rather than JUnit XML. Declaring a junit glob that matches nothing would give a permanently green "0 tests" trend, which is worse than no trend. Emitting XML from bin/test is the obvious follow-up. VALIDATED with ~/pop11-tools/jenkins-sim.p (parser + structural validator + declarative-semantics simulator): 0 errors, 0 warnings, and a simulated failure of the Playwright stage correctly marks the build FAILURE, skips the remaining stages, still runs post.always, and fires post.unsuccessful to a real address. TWO FIXES TO THE SHARED TOOL, both in ~/pop11-tools (outside either repo): 1. It was hardwired to git, so on this Fossil repo `git ls-files` failed for every path and it would have reported EVERY referenced file as untracked — a wall of false failures that trains you to ignore the output. Now detects git/fossil/none once per run. Fossil note recorded at the site: `fossil finfo -s` exits 0 for tracked AND untracked files and is useless as a predicate; `fossil ls` is the reliable test. 2. My own fossil support then had a path-normalisation bug that made it silently USELESS: `fossil extras` prints paths without a leading "./", so a grep -Fx for "./bin/x" never matched "bin/x", git_ignored() returned true for every untracked file, and the "referenced but NOT committed" check passed on everything. Caught only because I pointed a test Jenkinsfile at a deliberately untracked script and it did NOT fail. A validator that cannot fail is worse than no validator, and this session has now produced three of those — the bin/test summary grep, the mobile-splat suite, and this. Also worth knowing for anyone editing a Jenkinsfile here: the parser is line-oriented and does not see `steps { sh '...' }` written on one line. My first test file was formatted that way and parsed as zero sh steps, which read as a passing validation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: bf3b49437a user: dkords tags: trunk | |
| 20:06 | CLI 0.33.0 — fix --version/--help/no-args crashing without a TTY; add e2e suite The CLI had 144 functions and 10 rove tests, all unit-level (urlencode, arg-shape, sort-cycle). Nothing ever ran the binary. The first three things tried by hand were all broken, which is the argument for the new suite in one line. --version and --help are DOCUMENTED GLOBAL FLAGS in cli-grammar.md §3 and nothing handled them. They fell through to the interactive welcome screen, which runs `stty cbreak -echo` and signalled an unhandled SUBPROCESS-ERROR when stdout is not a TTY — a Lisp backtrace, and the wrapper still exited 0, so a scripted caller could neither read the failure nor detect it. Every CI invocation is non-TTY, so the first command a pipeline would run was also the first to crash. That is a decent explanation for why the CLI has never been in CI. Now: --version prints `robotarme <semver>` (and a JSON object under --json), --help prints plain non-ANSI usage aimed at someone piping to grep, and the interactive fallthrough REFUSES to start without a TTY, with an actionable message and exit 1 instead of a backtrace. isatty via sb-unix rather than pulling in sb-posix for one predicate. New tests/cli.test.mjs, wired into bin/test as its own `cli` target and into `all`. It runs the binary with PIPED stdio deliberately — that is the condition that was broken, so a harness that allocated a TTY would have missed all of it. Eight tests covering the contract rather than every subcommand, because each invocation costs ~4 s of SBCL start: exit codes, JSON shape, TTY independence, that --json carries no ANSI escapes (one colour code makes JSON.parse fail and the failure reads as a server bug), and that --server is actually honoured — pointed at a refused port, since if the flag were ignored the test would otherwise pass against the real server. Two of the eight are pure regression guards for the above: no-args must not produce a backtrace, and unauthenticated --json must be a structured error envelope rather than a crash. Full suite now: 1065 rove + 22 parser + 63 JVM + 89 Playwright + 8 CLI, green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: e0b980c4ef user: dkords tags: trunk | |
| 19:39 | docs: refresh the FSD status matrix; add R31 (CI/Jenkins) and R32 (Nix toolchain) The matrix had drifted far enough to be misleading for planning — it understated what ships, which is the direction that causes work to be re-scoped or repeated. Corrected against evidence in the tree, not memory: - R1 was "Partial — deployment rules engine still pending". That note was stale from Sprint 2; robotarme.deployments:evaluate-rules has been in since the group work and R16 already claimed it done. Now Done. - R2 "Device availability/check-in API — Not started" is the largest error. POST /api/devices/check-in shipped, its return path apply-result shipped in 0.57.0, and both are exercised by REAL HARDWARE rather than only tests: a Waveshare ESP32-C6 running ESPIDFORTH checks in unattended, fetches, verifies and applies. - R17 "Arbitrary firmware OTA — Not started" was done end-to-end on 2026-08-03: deploy, pull, Ed25519 + SHA256 verify, apply with rollback, report. - R19 "Rules engine — Not started" exists and drives group membership, the deployment resolver and the XR rule builder. - R18 promoted to Partial with the honest remaining gap: target_hardware cannot express a full build tuple, which Jetson needs because a TensorRT engine is not portable across (SoC, JetPack, TensorRT). - R5 kept as Not started, but noted that the one backup taken so far was a manual pg_dump — "not started" was reading as "unknown" rather than "nothing exists". TWO NEW REQUIREMENTS, both flagged by the owner: R31 CI on Jenkins. bin/test runs everything locally and nothing runs it on a push. There is no Jenkinsfile. Until there is, "all suites green" means "green on the machine that happened to run it" — and v0.61.0 is the evidence that local-only discipline does not hold: two suites sat written, passing and un-run for weeks, and a third was reporting PASS while executing zero tests. Noted that rove/parser/JVM are hardware-free stages while Playwright needs a server, a Postgres and headless Chromium with SwiftShader, and that the WaveC6LED suites stay manual because they need the board. R32 Nix packaging of the build/tool environment, as a PREREQUISITE for R31 rather than a parallel nicety. Staging a build currently needs SBCL + Quicklisp, Node + Playwright browsers, Android SDK + a JDK pinned to 17-21 (Gradle rejects anything newer with a bare version string as the entire error message), ESP-IDF 5.1.1, and Postgres — a set reproduced by hand and written down nowhere as a unit. CI is only meaningful if the agent and a developer machine agree on it. Carries forward the platform caveat already recorded in the AI-fleet research: strong on Linux, which CI runs; weaker on macOS, which development runs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: e43872dbe6 user: dkords tags: trunk | |
| 19:10 | server 0.61.0 — test automation: a suite that never ran was reporting PASS Refocusing on automation surfaced three problems, and the middle one is the serious one. TWO SUITES EXISTED AND WERE NEVER RUN. tests/scan-lab.test.mjs (8 tests) and tests/mobile-splat.test.mjs (5) were written, passing, and absent from bin/test. A test nobody runs is not a test; it is a file that happens to contain assertions, and it rots silently while reading like coverage. Both are wired in now — `web` and `all` — taking the suite to 1065 rove + 22 parser + 63 JVM + 89 Playwright, all green. A SUITE THAT DIED BEFORE RUNNING WAS REPORTED AS PASSING. Every Playwright runner detected failure by grepping the output for "N failed". A suite that exits early — missing env, import error, browser launch failure — prints no summary at all, matches neither "passed" nor "failed", and fell through to the success branch. mobile-splat demonstrated it immediately on being wired in: it exited 2 with "set ROBOTARME_SCAN and ROBOTARME_UPLOAD_TOKEN", executed zero tests, and the summary said green. That flaw was in all twelve runners, not just the new ones, and it is the worst kind of test bug because it is invisible in exactly the situation you most want a signal. An empty summary is now a hard failure that prints the suite's last few lines, so the reason is on screen rather than inferred. MOBILE-SPLAT NOW SELF-PROVISIONS. Requiring hand-set environment variables is why it was never wired in to begin with; a suite that needs manual setup is a suite that will not be run. It now logs in like every other Playwright suite here, finds a done scan that actually HAS gaussians (a mesh-only scan would fail for a reason that is not the bug), and mints its own upload token. The env vars remain as overrides. Two things that bit while doing it, both now commented at the site: CSRF ROTATES ON LOGIN, so the token from /api/session is stale the moment the session is authenticated and every later mutating call 403s — the login response carries the new one. And the mint failure now includes the server's actual response rather than a bare "could not mint", which is what turned that into a one-minute fix. Still missing, and the next real automation gap: the CLI has 144 functions and 10 rove tests, all unit-level (urlencode, arg-shape, sort-cycle). Nothing actually RUNS ./bin/robotarme against a live server. bin/test-cli-upload-source- url.sh exists and is also not wired in. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: e0001eb6f1 user: dkords tags: trunk | |
| 09:58 | server 0.60.0 — scan lab: orient previews from the recorded up axis; add thumbnails Owner: scan lab previews render upside down, and the list has no thumbnail. ORIENTATION. The pipelines emit geometry in the SOURCE CAMERA FRAME and only RECORD which axis is up — mapper/sharp-room-model.py detects it per scan and writes it to boundaries.json, and build_glb() recenters but never reorients. three.js is Y-up, so anything else renders upside down or on its side. Neither loadMesh nor loadSplat applied any correction. The fix is NOT a flip, and that is the point. The axis genuinely varies per scan — across this server's storage it is -z for 20, -y for 7 and +y for 3. A blind 180 degree rotation would straighten the -y scans and leave the -z majority on their side, which is the same bug with different victims. The viewer now reads GET /api/scans/:id/boundaries and rotates about X by the amount that carries the recorded axis to +Y: +y 0, -y pi, -z pi/2, +z -pi/2. Scans with no boundaries file fall back to -y, which is the generator's own documented default, and the info line says "up=-y (assumed)" so the guess is visible rather than silent. In the splat path the rotation is applied BEFORE framing: camera placement is derived from the bounding box, so orienting afterwards would point the camera at where the model used to be. Related, NOT fixed here and worth its own look: the native headset viewer hardcodes R = diag(1,-1,-1) — a 180 degree rotation about X — which assumes up is always -y. On the 20 -z scans it is wrong the same way the web viewer was. THUMBNAILS. The list was a plain table with no image column; /api/scans/:id/photo already existed and was simply never used. Rows now show the SOURCE PHOTO rather than a render: it is what the operator actually pointed the camera at, needs no GPU work, and is the fastest way to tell two scans apart in a list of thirty. loading=lazy so a long list does not fetch every JPEG at once, and a scan that has no photo yet hides the element instead of showing a broken-image glyph in every row, which reads as "the page is broken" rather than "this one is still processing". Tests: two new Playwright cases, 8/8 green against the live server. The thumbnail test asserts the image actually DECODED (naturalWidth > 0), not merely that an <img> exists — a 404 or a wrong path would still produce the element, which is exactly the failure being guarded. The orientation test asserts the rotation matches THAT scan's recorded axis rather than a constant, so it cannot pass by accident on a fleet where one axis dominates. The test seam exposes the model through GETTERS: `current` and `splats.mesh` are replaced on every load, so capturing them once would hand the test a stale object from the previous scan and quietly assert nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 72bc11cca1 user: dkords tags: trunk | |
|
2026-08-03
| ||
| 22:57 | docs: AI fleet thought experiment — widen to three segments; Jetson is the wedge Owner widened the framing: constant churn of new research and models, site- specific configuration at deployment, and users across home lab, private enterprise AND public product deployments — on the observation that every Nvidia Jetson is a Linux system with models on it, shipping inside somebody's product. That last point changes the conclusions, so the doc gained four sections. THREE SEGMENTS, and the public-product one is demanding. A vendor shipping units with a model inside needs exactly what this week built: signed artifacts, verified-before-apply, atomic rollback, and a truthful applied-version report per unit — because "we cannot tell which of our 40,000 units is running the withdrawn model" is a recall conversation. It also inverts consent. In a home lab the operator IS the owner; in a shipped product the operator is the vendor and the owner is a member of the public who may not know a model exists. That is a disclosure question before it is an engineering one, and it should be settled deliberately rather than inherited from how firmware updates happen to work. CHURN, NOT SIZE, IS THE HARD PART. The first draft treated 40 GB weights as the challenge; size is merely expensive. Three things follow that have no firmware analogue. Supersession is not linear — a newer 3B at q8 can beat an older 8B at q4 on the target hardware — so a profile cannot say "latest" and needs an eval-ranked candidate list, re-derived as the catalog moves. Models get WITHDRAWN for licensing or safety, which firmware essentially never is, so withdrawal has to propagate as an ACTIVE "stop serving this" instruction rather than the absence of a recommendation, actionable before a replacement exists. And qualification is continuous labour: anyone can copy a deployment tool in a quarter, but a maintained catalog of what is new, qualified on which hardware, superseded, or carrying CVEs compounds. That is what a management PLAN actually sells. JETSON HAS A BUILD-MATRIX PROBLEM worth knowing before anything is designed. A TensorRT engine is built for a specific (GPU SKU, TensorRT, JetPack) tuple and is not portable the way a GGUF is. So the artifact is either engines compiled per tuple — multiplying storage and CI but keeping devices dumb — or an on-device build step that keeps the catalog small and makes apply slow and failure-prone. Genuine fork, does not arise for Ollama at all, decide before building. It also means target_hardware must express the full tuple, extending the note already in customer #7. Happy consequence: on Jetson, Nix is at its strongest and its macOS weakness does not apply, so starting there lands the Nix backend and the fleet story together. SITE CONFIG NEEDS THREE LAYERS, not two: base profile (ours) + site overlay (the customer's — system prompts, egress rules, data residency are their policy and we should deliver rather than author them) + per-device resolution. The audit trail must record all three for a device at a time, or "why is this unit running that?" is unanswerable — and in the product segment that question arrives with a lawyer attached. Revisit condition updated accordingly: the likely first user is a Jetson product vendor rather than a home lab. Enumerable hardware family, customer #7 already lives there, and requirements severe enough to be worth paying for. A home lab is the easiest to imagine and the hardest to charge. Two decisions added to the preconditions: the Jetson build fork above, and whether the catalog is a product we maintain or a feed we consume — the latter being the difference between selling a tool and selling a plan, and the only decision in the document that is strategic rather than technical. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 7858fe6761 user: dkords tags: trunk | |
| 22:31 | docs: thought experiment — managing AI service fleets in homes / small offices Owner raised it while testing D4: a household or small office runs 1-10 machines, several capable of local inference (Ollama, llama.cpp, vLLM), and keeping their QUALITY LEVEL and configuration consistent is as hard as keeping firmware current on a device fleet. Could RobotARme manage that, possibly with Nix, sold as a management plan? Written up now so the reasoning is not re-derived later. REVISIT, nothing committed, no engagement sought. Three conclusions worth the disk space: THE INVERSION. RobotARme deploys "release X to group Y" — the release is concrete, the group selects recipients. AI hosts need "class Y should serve quality tier B", with the server RESOLVING what that means per device from its reported capability. Same database shape (device_group + filter_rules + priority), opposite direction of causality. That resolver is the product: it is what turns "I have five machines" into "all five run the best thing they can, and I know it". A 64 GB Mac and a 12 GB GPU box should NOT get the same artifact, so "the same release everywhere" is the wrong goal here. TWO ARTIFACT TIERS, NEVER CONFLATED. Runtime+config is small, ours, signable, atomically rollback-able. Weights are 4-70 GB, third-party, immutable, and rollback is best-effort because you cannot keep two 70 GB models on a 512 GB laptop — better to say so up front than discover it during an incident. Ollama already uses sha256 blobs with manifests on top, which is the shape to meet rather than fight. PINS, NOT DISTRIBUTION — and that is a legal constraint, not an engineering preference. Llama/Gemma terms bind redistribution, so a plane that ships pins and fetch instructions largely avoids obligations a plane that hosts weights takes on. Related: our signature can attest "we chose this digest", never "this model is safe". Weights are opaque and unreviewable; conflating those two claims would misrepresent what our verification means. On Nix, the honest read: excellent for the Linux subset and appliance cases, but macOS is the awkward one (nix-darwin gives no atomic system switch) and unified-memory Macs are among the best home inference boxes — the machines you most want to manage are the ones Nix manages least well. So keep the profile format platform-neutral with per-platform backends, and let Nix be a strong backend rather than a hard dependency. Two threads this would force, both already deferred elsewhere: - Telemetry must be PERSISTED. check-in accepts a telemetry object and drops it by documented design, deferring the schema until customer #5 specifies one. This use case ends that deferral, and the table should be designed once for both rather than twice. - Device-as-cache. Five machines x 40 GB over a domestic uplink means one device serves the others — a role RobotARme has no concept of, every device being a leaf today. Same shape as customer #7's open "do we host the binary cache or proxy theirs" question, so if it gets built, both benefit. Filed as a prospective in docs/validation-customers.md rather than a customer, next to #7 (NixOS on Jetson), which shares most of the hard parts. Plausible sequencing is #7 first as the curated appliance, then generalise outward — starting with a heterogeneous household is starting with the hardest case. Revisit condition is deliberately concrete: a named user with three or more machines who feels this today. Everything above is speculation until then, and the speculation is cheap precisely because nothing is being built. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 8d6b471f33 user: dkords tags: trunk | |
| 20:37 | docs: D4 complete — the OTA loop closes on hardware wavec6led-ota-plan.md marks D4 done and records two corrections the phase forced. The plan specified Ed25519 "via mbedTLS mbedtls_pk_verify", inherited from docs/prototype-demo-plan.md and repeated without checking. mbedTLS ships Curve25519 (X25519 key exchange) and NO EdDSA implementation at all. TweetNaCl is vendored on the device instead. Recorded rather than quietly swapped, because a plan that silently fixes its own bad calls teaches nothing on the next read — same treatment as the ST7789T correction in §2.3. Risk #2 resolved, and it did NOT force the ESPIDFORTH repo split. The stub has no EVALUATE, but forth_core.h already exported forth_eval(), so bundles run through C. Savepoint/restore turned out to be three integers, because the dictionary is append-only and find_word searches backward — redefinition shadows, so truncation genuinely reverts. That plus an error counter is ~30 lines added beside the core, not surgery on it. §6 stays a proposal. Test plan gains §5A.4 (W25-W36) for verify/apply/report, weighted toward the tests that prove the system says NO: flipping one bit of the org public key must give SIGNATURE BAD; a signature-valid-but-broken bundle must roll back and the word defined on the line BEFORE the failure must be gone afterwards; and current_version must stay at the last good release, because a failed update that makes a device look upgraded is the most damaging possible lie in a fleet view. HTML regenerated: 334 tests. Note for anyone reproducing: the server must be running current code. The instance up during this work predated /api/devices/apply-result and returned 404 to the device's report — the loop looked broken when only the server was stale. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: c33c3318df user: dkords tags: trunk | |
| 18:47 | server 0.59.0 — a bodiless request declaring a content type must not 500; fix bin/test's undercount Both found while bringing up the WaveC6LED's signature fetch (D3). THE 500. A release signature URL returned 200 to curl and 500 to the device. The trigger is `Content-Type: application/json` TOGETHER WITH `Content-Length: 0` on a GET — neither header alone does it, which is why it took a while to isolate: every obvious single-variable test came back 200. esp_http_client emits exactly that pair once any header is set on a GET, so the board hit it on its first attempt to fetch a signature. Fixed on the server, not only on the device, because a well-formed request must never 500 and the next client that sets a default content type will trip the same wire. New make-bodiless-content-type-middleware strips CONTENT-TYPE when CONTENT-LENGTH is absent or zero. That is semantically right rather than a workaround: a request with no entity has no content type to describe. Not read-json-body's fault, incidentally — it already guards an empty body; the throw was in the framework's parameter parsing, which is why the handler looked innocent. Regression test asserts all three combinations against /api/version (chosen because it is unauthenticated and always 200, so any non-200 is the middleware layer rather than the route): content type alone fine, zero length alone fine, and the PAIR now fine. BIN/TEST WAS UNDERCOUNTING, and misleadingly. It reported `grep -c "Expect"`, but rove prints "Expect (FORM) to be true" ONLY for an assertion with no description — give (ok ... "why this matters") a description and it prints that instead. So every well-documented assertion was invisible to the counter, and adding tests left the reported total unchanged. That reads as "my new tests did not run", and it sent me chasing a stale FASL cache that was never stale. Now counts actual pass marks: 1065, where it had been reporting 390. That number was wrong in a bad direction — it rewarded terse assertions and hid thorough ones. Tests: rove green, 1065 assertions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 27f39bdc15 user: dkords tags: trunk | |
| 07:55 | server 0.58.0 / androidxr 0.33.1 — absent multipart field became the string "NIL"; 0 KB on the headset Both found by the owner looking at the aurora release he had just deployed to the WaveC6LED. THE "NIL" IS THE SERIOUS ONE, and it was never about notes. %multipart-field: (let ((v (cdr (assoc name params :test #'string=)))) (cond ((stringp v) v) ((listp v) (princ-to-string (first v))) ; absent field lands here (t nil))) NIL IS A LIST IN COMMON LISP. An absent part made ASSOC return NIL, so (cdr nil) was NIL, so the listp branch matched, and (princ-to-string (first nil)) produced the STRING "NIL" — which was written to the database and then displayed, correctly, by every surface. All five multipart fields had it, so target_hardware was affected too, not just notes. The serializer was innocent the whole time: release-to-plist already does (or notes ""). That is exactly why the bug survived — correct display code faithfully showing bad data reads as a display bug, and points the investigation at the wrong file. Now the helper tests for the entry's PRESENCE before looking at its value, because absent and present-but-empty are different answers and only the caller knows which default it wants. One row in the database carried the bad value (release 2362, the only one of 37); repaired in place. Tests, both halves. A unit test pinning %multipart-field directly — absent yields NIL, present-but-empty stays "", a list part yields its first element, and a list whose first element is NIL does not become "NIL". Plus an end-to-end upload with the notes and target_hardware parts OMITTED, asserting what reaches the DATABASE. The multipart helper gained the ability to omit optional parts at all; it previously always sent every field, which is precisely why no existing test could reproduce this. Noted in the unit test why a regression here is silent: "NIL" is a perfectly good string, so every caller's (or x "") happily preserves it and nothing type-checks. 0 KB ON THE HEADSET. FleetScreens.kt formatted sizes as "%.0f KB" with no bytes branch, so a 231-byte Forth bundle floored to "0 KB" — indistinguishable from an empty or failed upload, on the one screen where you decide whether to deploy it. The web and WebXR formatters both had a sub-1KB branch; only the Android client did not. Now shows bytes below 1 KB and one decimal place above. Test plan: new section 5.3 (apply-result, D17-D24) and a new section 5A for the WaveC6LED itself (W1-W24) covering D0 bring-up, D1 provisioning and D2 check-in — including the unattended 60 s poll, which is the actual milestone, and the regressions above. HTML regenerated: 322 tests. Tests: rove 390 assertions green, up from 388. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: d7d516aec6 user: dkords tags: trunk | |
|
2026-08-02
| ||
| 21:05 | wavec6led-ota-plan: correct §2.3 — the hardware disproved it The plan header still read "plan only, nothing below is built" after S1 and D0 had both shipped, and §2.3 still recommended esp_lcd's built-in esp_lcd_panel_st7789. That recommendation is wrong for this board and the plan should say so rather than quietly lose the claim in a later commit. The glass is an ST7789T variant; Waveshare's own ESP-IDF demo ships a separate Vernon_ST7789T driver with a different init sequence. What made this expensive is that the generic driver brings up SOMETHING — enough to look like the remaining fault must be orientation — so it invites an unbounded hunt through MADCTL combinations that cannot converge. Kept the original wording alongside the correction rather than overwriting it, because a plan that silently rewrites its own bad calls teaches nothing on the next read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: e37364dae4 user: dkords tags: trunk | |
| 21:04 | wavec6led-ota-plan: D0 complete, verified on hardware Forth REPL over USB-serial-JTAG, ST7789 status screen with readable text, RGB LED on GPIO8. 288 KB free heap with all three up — the figure D1 re-measures against when WiFi comes on, which is the plan's main memory risk. Firmware lives in ProjectMagNET as reference-designs/MagNET_OTA_WaveC6LED. Two findings from the bring-up changed the design and are recorded in that reference design's README: The display needs Waveshare's own ST7789T driver, not esp_lcd's generic ST7789 — different init sequence entirely (porch, power, gamma, plus 0x21 inversion), so no amount of orientation fiddling against the generic driver converges. Vendored in from their ESP-IDF demo, along with BGR colour order, mirror(true,false) and a 12 MHz clock. esp_lcd_panel_io_tx_color() is asynchronous, and a reused scratch buffer therefore corrupts text while leaving solid fills perfect — because every band of a rectangle fill holds the same colour, so clobbering it mid-transfer writes identical bytes. Perfect layout with shattered text impersonates a font bug convincingly enough to cost an audit of the glyph table, the renderer and the panel orientation, none of which were at fault. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 3fd8d96540 user: dkords tags: trunk | |
| 07:33 | server 0.57.0 — POST /api/devices/apply-result, closing the OTA loop's return path S1 of docs/wavec6led-ota-plan.md, and the last server-side gap before firmware. docs/prototype-demo-plan.md specified this endpoint from the start; it was never built. Until now the console could show what a device was TOLD to run and never what it actually ran — a deployment was a broadcast, not a loop. Bearer-authed with the same dvc_ token as check-in, so a device that can check in can report with no further provisioning. Four new columns on device (last_apply_release_id / _ok / _at / _error), all surfaced on the device JSON so XR, web and TUI can show APPLIED rather than DEPLOYED. Four decisions worth their reasoning: NOT rate-limited, deliberately. Check-in and apply-result are opposite problems. Check-in is a poll a device could spin on, so it carries a 5s cooldown. An apply-result is a one-shot report of work already DONE, and dropping it loses the only record that the work happened — a device retrying a failed report is behaving correctly, and a 429 would punish it for that. error/log is TRUNCATED at 1000 chars, not rejected. The device most likely to send something enormous is the one whose bundle just blew up, which is exactly the diagnosis we most want to keep. Rejecting the oversized report would discard it. Truncation is marked in the stored text so a reader knows it was cut. Cross-tenant release_id is refused with 404 rather than recorded. Without the org check a device could stamp its row with another tenant's release id, and the endpoint would answer which ids exist. ok accepts "true"/"1"/1 as well as a JSON boolean. The device end of this is a Forth word assembling JSON by hand; being strict here fails in the field for a reason that is invisible from inside a 200-line C shim. A successful apply also advances device.current_version to the release's, so the next check-in resolves to noop even if the device forgets to send current_release_id. last_apply_ok serializes as null when a device has never reported, which is genuinely different from having reported a failure. Found while testing: the CSRF middleware exempts /api/devices/check-in BY NAME, so the twin endpoint had to join the list. Without it a device sending a missing or malformed Authorization header gets "CSRF token mismatch" instead of "Invalid device token" — sending a firmware author hunting for a cookie they will never have. Noted in the plan, because every device-facing route in R3/R4 will need the same exemption. Also in the plan: S2 (server advertises _robotarme._tcp for the mDNS transport — new work, nothing in src/*.lisp speaks mDNS today, and it must advertise the LAN-reachable address rather than the bind address, the same trap %lan-reachable-p exists to catch) and S3 (device-authenticated envelope, only if we decide the BLE and serial relays must not see the dvc_ token). Tests: rove 388 assertions green, 11 new. Migration applied; the four columns are present on device. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: a39bc32771 user: dkords tags: trunk | |
| 06:55 | wavec6led-ota-plan: transport becomes an interface — IP, mDNS, BLE relay, USB serial Owner wants four transports eventually: HTTP/HTTPS now, an mDNS-discovered variant, a pure BLE version proxied through a mobile app, and a USB serial version in the style of the meshcore web flasher (which RobotARme has no equivalent of today). So transport stops being a decision in this plan and becomes an interface. The four are not four peers, though, and building them as four code paths would be most of a wasted fortnight. They collapse to ONE logical protocol, TWO transports and ONE discovery mechanism: - mDNS is DISCOVERY, not transport. It answers "what is the base URL"; the bytes still move over HTTP. Filing it as a transport duplicates the entire IP path in order to change one string. - BLE and USB serial are the SAME transport over different pipes. In both the device has no IP and cannot speak HTTP; it emits the same framed request/response and something else relays it. They share framing, request encoding and reassembly, and differ only in read/write. So: magnet_transport_t vtable, transport_ip (base URL from NVS or mDNS) and transport_relay (BLE GATT or serial), with the proxy doing a mechanical translation to HTTP on the far side. The relay framing carries an explicit chunk header rather than trusting the pipe's MTU, because BLE negotiates ~180-500 bytes and serial is effectively unbounded — the device should not have to care. Payload is the same JSON the endpoints already take, so the proxy needs no protocol knowledge and nothing to keep in sync as endpoints change. Flagged rather than glossed: for BLE and serial the relay handles the dvc_ bearer token in CLEAR TEXT, because it is forwarding HTTP requests. The phone app or web page becomes a credential-handling component. Written up with two answers — accept it for the demo since the proxy is operator-controlled, or a device-authenticated envelope where the device HMACs the payload with its token and the proxy relays an opaque blob it cannot replay. The second is the right long-term shape and is a SERVER change, not a firmware one; the framing carries a version byte so it can express it later. What already exists, verified rather than assumed: - craw_ble_provision is a working GATT service with a real state machine, so the relay characteristic extends a proven pattern rather than starting cold. - The contributor Flutter app ALREADY depends on flutter_blue_plus ^1.34.5 — it is already a phone app that talks to RobotARme and already carries a BLE stack, so it hosts the BLE proxy rather than us writing a new app. - The server advertises NO mDNS: nothing in src/*.lisp mentions mdns/_tcp/bonjour. Server-side advertisement is new Lisp work. - RobotARme has no serial feature at all. WebSerial is Chromium-only, so that path is a convenience, never the only way to provision a device. Ordering: R0 plain HTTP to close the loop, R1 HTTPS (mbedTLS is linked in D3 for Ed25519 anyway, so the marginal cost collapses and that is where the clear-text window closes), then R2 mDNS, R3 BLE, R4 serial. R2-R4 are additive and each is independently shippable ONCE THE VTABLE EXISTS — which is the whole reason to build it in D2 instead of calling esp_http_client directly and retrofitting. Added that as a risk in its own right, because it is the highest-leverage line in the plan and the easiest to quietly drop under deadline: R0 works fine without it. Also noted BLE relay throughput — a bundle over GATT at ~180 B a frame is minutes, which is fine for Forth source measured in kilobytes and would not be fine if bundles ever became app images. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 61144cc839 user: dkords tags: trunk | |
| 06:46 | docs/wavec6led-ota-plan.md — plan the device-side OTA client on the Waveshare ESP32-C6 Every operator-facing surface ships, but no device has ever pulled an update because the check-in client does not exist. Test target is the Waveshare ESP32-C6-LCD-1.47 ("WaveC6LED"), picked over the M5Stamp because its screen and RGB LED make provisioning visible rather than inferred from a log. Plan only. Nothing built. Everything in the findings section was checked on this machine rather than recalled, and four checks changed the shape of the plan: 1. ESPIDFORTH's ESP-IDF engine is a STUB — forth_core.cpp is 1,333 lines of core ANS words via add_primitive(), with the real ESP32forth v7.0.8.0 sitting unported beside it as ESP32forth.ino. Anything written against the stub's internals gets rewritten when that swap lands. So the OTA supervisor is a native C component with a thin Forth vocabulary over it: the engine swap then touches one registration shim, not the logic. 2. There is no HTTP client in the Forth build at all. components/forth/optional/http-client.h wraps Arduino HTTPClient and forth_core.cpp never references it — dead Arduino-era code that survived the ESP-IDF port. A device currently cannot make an HTTP request from Forth. 3. WiFi is compiled OUT on the C6 target (CONFIG_ESP_WIFI_ENABLED=n, "conserve memory"). Networking on this chip is unproven, not a config flip to assume. 4. M5GFX has no esp32c6 platform layer — esp32/c3/p4/s2/s3 only. Panel_ST7789 exists, right controller, wrong platform underneath. Use IDF's native esp_lcd, which is already in the link line. Also ran a trial build rather than trusting the README's "ESP32-C6: Builds": it reaches 907/909, forth_core.cpp compiles clean for RISC-V and libforth.a links. The only failure is `undefined reference to app_main`, because the root CMakeLists has no EXTRA_COMPONENT_DIRS and there is no main/ — PlatformIO injects src/ as the main component and plain idf.py does not. One line of CMake, not a porting problem. Good news from the same link line: esp_http_client, mbedtls, esp_lcd and nvs_flash are all already present in IDF 5.1.1. Server-side gap found while reading the contract: POST /api/devices/apply-result is specified in docs/prototype-demo-plan.md and does not exist. check-in, download and signature are all built and well tested, so a device can be told to update and can fetch the bundle, but there is no way for it to report whether the update worked. Deploy is a broadcast, not a loop. That is Lisp work in this repo, independent of any firmware, and it is phase S1 so it can land first. Phases D0-D4 each end with something visible on the 172x320 panel, finishing with the version string CHANGING on screen — the proof the exercise is for. Two constraints written down because they are easy to design past: the vendor warns that sustained full backlight permanently shadows the panel, and this board is meant to display status indefinitely, so attention-getting uses the RGB LED and the backlight stays at <=50%; and 4 MB flash is ample here ONLY because Forth bundles are source rather than app images, so nobody should add a dual ota_0/ota_1 partition layout that would not fit. Transport is left open with a recommendation: plain HTTP to close the loop in D2, HTTPS in D3, since mbedTLS is being linked for Ed25519 by then anyway and the marginal cost of TLS drops once it is. Splitting ESPIDFORTH into its own repo is written up as a proposal but NOT proposed yet — the plan adds components beside the Forth core and touches it in exactly one place, deliberately, to keep the split optional. The trigger is D2 or D4 forcing real surgery on forth_core.cpp. Specs pulled and parsed with the pop11 skill (popcurl fetch, then a small HTML-to-text + keyword-grep helper); the pinout table in the plan comes from that parse. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 0162fdc0c4 user: dkords tags: trunk | |
| 05:16 | remove the Caveman2 skeleton's db/schema.sql, and the README line that used it db/schema.sql has sat untracked since 25 Jan, surfacing in `fossil extras` on every commit. It was never part of this project: one SQLite `message` table with INTEGER PRIMARY KEY AUTOINCREMENT, left over from the Caveman2 skeleton, and nothing in the Postgres/Mito schema resembles it. Never tracked in fossil either (`fossil finfo -s` reports unknown). Deleting the file alone would have left README pointing at it, so the setup section goes with it. That block also still offered the sqlite path first, under a note saying sqlite is not supported — the remaining instructions are the Postgres ones that actually work, plus ./bin/migrate to apply the schema, which is what creates the tables now. The db/ directory is gone with it; it held nothing else. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: c4bc2c6281 user: dkords tags: trunk | |
| 05:00 | bin/purge-test-data — clear 3,500+ leaked test fixtures from the dev database Asked whether the 2,424 device tokens and 438 API tokens left alone by the PRNG remediation were just test devices, and whether we could tell. Both yes. They are test-suite droppings. Most rove tests register a user through the real flow — which also mints a personal org — and never tear it down. A minority DO clean up, so the pattern exists; it just is not the habit. Measured after this commit's own test run: one `bin/test rove` leaves behind 22 users, 22 orgs and 12 devices. Multiply by months of runs. DISTINGUISHING THEM. Email domain is nearly a clean split — test.local, t.local, x.local, t.l, test.com carry 3,527 accounts; robotarme.local is the 7 bin/seed fixtures; two @yahoo.com accounts are real. Device names corroborate it independently: alpha-N/beta-N/gamma-N (1,414), json-ok-N (394), mpfile-N (382), checkin-dev (104). Nothing resembling hardware. But "nearly" is the whole problem, so the tool is an ALLOWLIST, not a denylist. A denylist that misses deletes a real account with no undo. Two accounts prove the point: splatlab sits on example.com and owns org 9030 — the 19 headset scans — and pipboy2026 would fall to any "trailing digits are test data" rule. Nothing is purged unless it fails EVERY test for being real, and orgs with SCANS ON DISK are protected by reading the filesystem rather than the DB, so that rule holds even when every heuristic above is wrong about the owner. Also swept 127 devices whose org row had already been removed by some earlier partial cleanup. The org-scoped deletes cannot reach them — their org_id matches nothing — so they would have survived and stayed unreachable. Not harmless: device auth matches on token hash alone with no org join, so an org-less device can still authenticate. Deletion is set-based and FK-ordered in one transaction. That earned itself immediately: the first --apply hit `column "exists" does not exist` and rolled back with nothing deleted. The cause is worth writing down — those three sweep statements go STRAIGHT to execute-sql, so the "~" line continuations I use everywhere else were never processed by format, stayed in the SQL, and Postgres read "~" as its regex-match operator. What it did NOT protect was the filesystem: 361 release payloads had already been unlinked before the transaction, leaving rows pointing at nothing. A rollback restores rows, never bytes, so the file deletion now runs AFTER the commit. Result, verified: 3,552 users -> 10, 3,541 orgs -> 18, 2,424 devices -> 3, 442 API tokens -> 0, 3,523 signing keys -> 18, 367 releases -> 0. Database 16 MB. All 19 scans in org 9030 intact, storage/scans untouched at 3.5 GB. Server healthy on 0.56.0 throughout (db_ok true). pg_dump taken and pg_restore -l verified BEFORE applying: ~/robotarme-backups/robotarmedb-2026-08-01-pre-purge.dump docs/next-steps.md records the root cause: this is a mop, not a fix. Without teardown the pile rebuilds at 22 users a run. The cheap version is a with-test-user macro that deletes what it created in an unwind-protect, adopted as tests get touched — worth doing before the dev DB is slow enough that the pagination and rate-limit tests start measuring the wrong thing. Tests: rove 377 assertions green, before and after the purge. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 00d5c856c8 user: dkords tags: trunk | |
| 04:10 | androidxr 0.33.0 — control arm P3 (yaw, scale, snapping); weak-PRNG credential remediation CONTROL ARM P3 (docs/splat-viewer-controls-spec.md §5). The yaw ring and the scale grip become live handles, and both snap. The scene transform stops being flip-and-translate here. Yaw and scale apply about a pivot Q that belongs to the SCENE, not the world: world = s * Ryaw(yaw) * (R*model + T - Q) + Q Q sits ON THE GROUND under the scan rather than at the bbox centre, so growing a dollhouse pushes the roof up instead of sinking half the floor through the reference plane. Yaw is invariant to the pivot's height, so one pivot does both jobs. The depth-sort inverse in the frame loop now takes the translation from SplatModel's own matrix instead of restating the anchor algebra. That second copy is exactly what went stale when the scene gained a user offset, and with yaw and scale in the mix it would have gone stale again. Verified by extracting both blocks verbatim into a standalone harness: model -> world -> model round-trips across yaw x scale x offset x no-plane; the bbox centre stays on the anchor under yaw and scale; the scene rests on the plane at every scale; the floor stays pinned as scale grows. Also pinned the SIGN of the yaw drag, because Ryaw(theta) carries bearing phi to phi - theta, so the ring maps hand bearing to yaw with a MINUS. Get that backwards and the scan counter-rotates under your hand, which reads as broken rather than inverted, and no compiler would have said a word. Geometry constants are now shared between the drawing and the hit test. A handle drawn from one set of numbers and grabbed from another is the silent-failure mode §4.5 exists to prevent. The ring gets a TIGHTER grab radius than the axis tips (25 mm vs 35 mm). It lies in the ground plane at 45 mm and the X/Z arms run straight through it, so at a common radius a finger reaching for the X tip crosses the ring's catchment first. Handles now compete on distance relative to their OWN radius. Snapping is a nudge, not a jail: 15 degree yaw detents engaged within 3 degrees, and height snaps to resting-on-the-plane within 2 cm — getting a scan to sit exactly on the floor by hand is impossible at 8x gain. Scale is exponential (clamped 0.02-1.0) so a centimetre of travel feels the same at 1:20 as at 1:1. Added a yaw indicator tick on the ring: a rotated scan otherwise gives no clue how far it turned, and the detents are invisible. The ground disc now tracks sceneScale, or a shrunken dollhouse sits in the middle of a 2.4 m disc reading as the floor of a room that is no longer there. WEAK-PRNG CREDENTIAL REMEDIATION — the half 0.56.0 explicitly did not do. New bin/rotate-weak-credentials. Dry-run by default, and it REFUSES to run on a tree that still has the bug: rotating with a weak generator would re-mint weak secrets and report success, which is worse than not running at all. The blast radius was far narrower than the earlier note claimed, and checking rather than assuming is the whole finding. Read against the pre-fix tree (fossil cat -r 5e00e4209f): weak TOTP secrets (random 256) and backup codes (random alen) -> 1 enrolment cleared; a shared secret cannot be re-keyed server-side, so the user re-enrols weak seed device tokens, byte-identical on every bin/seed run -> 3 fixtures rotated weak invite/upload tokens, device + user codes, pair codes, channel ids, CSRF -> in-memory only, cleared by the restart onto 0.56.0 SOUND device tokens (dvc_) and user API tokens (rba_): db.lisp %random-bytes has always been ironclad:make-random-salt, the OS RNG SOUND org Ed25519 signing keys: ironclad:generate-key-pair So 2,424 device tokens and 438 API tokens were left alone. Rotating them would have broken every fleet, including an active validation customer, to fix a generator that was never broken. Operational half: a pre-fix server process was still running on the loopback bind and still minting weak ephemera three hours after the fixed 0.56.0 build was already serving on 0.0.0.0. Rotating the database is half the job; the vulnerable process has to actually be gone. It is now. Tests: rove 377 assertions green. androidxr 0.33.0 builds, installs and launches clean on the Aura (no FATAL, no XR_ERROR). The P3 handles themselves still need the headset worn and a signed-in session to exercise — adb cannot reach the viewer — so the maths is proven by the harness, not by the device. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 0e22138e0c user: dkords tags: trunk | |
|
2026-08-01
| ||
| 03:47 | androidxr 0.32.1 — frame-cost instrumentation; correct a stale 72 fps claim The splat viewer measured 12.2 fps at 550k gaussians, against a 72 fps figure quoted in the code as a tuning anchor. Measured rather than reasoned about it. Added a per-2s cost line next to the fps log: re-sorts requested and completed, staging uploads, memcpy ms, overlay vertex count, and the SESSION STATE. It immediately killed my first hypothesis — I expected constant re-sorts driving a ~30 MB staging copy per frame, and the counters read req=0 done=0 uploads=0 copy=0.0ms with 564 overlay verts. The CPU side is entirely idle; the whole frame is the splat draw. Then A/B'd against the last pre-control-arm native build (5e1f409a79), same device, same launch path, both at FOCUSED: pre-control-arm 14.4 fps current 12.2 fps So the 72 fps was never a FOCUSED number — it was recorded while the session sat at VISIBLE (state 4), which does not do the compositing work FOCUSED does. There is no 6x regression. The control arm costs ~2 fps, which at 72 Hz is one vsync interval: the frame was already sitting right on the 5-interval boundary (69 ms) and the arm tipped it to 6 (83 ms). Splat count is not the dominant lever either. At FOCUSED: 550k -> 12.2, 275k -> 14.4, 150k -> 19.2. A 3.7x smaller cloud bought 1.6x, so the cost is per-pixel (overdraw), not per-splat — apparent size, opacity and resolution are the levers that matter. Caveat recorded with the numbers: scene placement follows head pose at launch, so screen coverage and hence overdraw vary somewhat between runs; these are the right order of magnitude, not precise figures. Both stale claims are corrected in the comments where they were being used to justify decisions, with the session-state caveat spelled out — a VISIBLE fps and a FOCUSED fps are not comparable, and quoting one as the other is what sent this chase off in the first place. Instrumentation kept: it costs a handful of counters per frame, and it earned its place by falsifying the obvious wrong answer in one run. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 66cd87f599 user: dkords tags: trunk | |
| 03:08 | androidxr 0.32.0 — audit fixes: config restore, PLY bounds, arm-delay ordering Three findings from an audit of the androidxr side. 1. MapperProbe left hand tracking OFF for the rest of the session. Its config ladder calls session.configure() directly and NONE of the four rungs sets handTracking. configure() REPLACES, so running the probe killed the back-of-hand button with nothing to connect it to — the exact bug class XrPerception exists to prevent, and which already bit StatusHud twice. The ladder stays bare on purpose (it measures what the runtime accepts, so mixing in the superset would change what is being measured); instead the probe now restores the superset in a finally, covering cancellation and early returns too. 2. LoadPly could read out of bounds on a malformed file. The gate checked five offsets (x, f_dc_0, opacity, scale_0, rot_0) and left the other nine unvalidated, so a ply declaring "x" without "y", or "rot_0" without "rot_1..3", left those at -1 and rf() did memcpy(buf.data() - 1, 4) — a heap under-read. The vertex count was also taken from the header unchecked, so reserve(count/2) on garbage was an unbounded allocation. Every offset is now range-checked against the stride, and count is bounded. Not externally reachable: the native viewer reads a fixed internalDataPath/splat_current.ply, not an Intent extra, so the exported activity is not an injection path. This is about failing cleanly on a bad pipeline output instead of crashing a viewer that has historically taken the compositor with it. Verified by extracting the real guard into a harness: the OLD code ACCEPTED missing-y/z, missing-rot_1..3, a negative count and a 2e9 count; the new one rejects all four, still accepts a real 148,176-gaussian scan, and still accepts the multi-element layout whose stride handling was the 2026-07-14 fix. 3. The arming delays shipped yesterday were order-dependent. HandMenu.show() no-ops when a loop is already running, and DisposableEffect's addObserver synchronously replays ON_RESUME when the Activity is already RESUMED — which can beat the LaunchedEffect coroutine. So first entry took the RETURN delay and ARM_DELAY_ENTER_MS was dead code. Both callers now go through one entry point that decides from state we own, making it order-independent. Also: HandMenu clears loopJob on early exit, so a failed start no longer latches until hide() — but ONLY when the job is still the current one. hide() cancels asynchronously, so a cancelled loop's finally can run after a fresh show() has installed its replacement; clearing that handle blindly would let the next show() start a SECOND concurrent loop, and two loops means two tap handlers. Audited and found sound, for the record: the async re-sort state machine (strict ping-pong — the worker never sorts while main copies), TokenStore (Keystore-backed EncryptedSharedPreferences), no trust-manager or hostname-verifier overrides, no cleartext config, exported activities take no extrinsic input, and i18n (the six literal Text() calls are brand name, version, a glyph and numeric readouts). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: e8461400bb user: dkords tags: trunk | |
|
2026-07-31
| ||
| 04:43 | androidxr 0.31.0 — fix XR_ERROR_HANDLE_INVALID plane spam; latch plane tracking Owner 2026-07-30: continuous spam on the scan lab screen — XR_ERROR_HANDLE_INVALID in oxr_xrGetTrackablePlaneANDROID: (trackableTracker == NULL) — and the tabletop model not loading. Decompiled arcore-1.0.0-alpha15 rather than guess. Plane.subscribe ends in stateIn(session.coroutineScope, SharingStarted.Eagerly, ...): the flow it returns is shared EAGERLY in the SESSION's scope, starts whether or not anyone collects, and NO CALLER CAN CANCEL IT. It also throws IllegalStateException("Config.PlaneTrackingMode is set to DISABLED.") when plane tracking is off. That invalidates the reference-counted begin/endPlanes design added earlier today. It assumed a subscription could be released. It cannot — so switching plane tracking back off destroyed the trackable tracker underneath a still-running collector, and the runtime spammed forever. Each new subscribe added another permanent collector, which is why the spam arrived from a growing set of threads. Four call sites were doing this: the scan sweep, the dollhouse placement, and the mapper probe, each subscribing afresh every run. Now: plane tracking LATCHES on at first use and stays on for that Session's life, with ONE cached flow per session behind XrPerception.planes(session, who). Nothing calls Plane.subscribe directly. The latch is keyed to the Session object rather than a global boolean — a Session dies on Activity recreate, and a stale latch would enable planes for a fresh session that never asked, reviving the original "Failed to get scene plane" spam from the other direction. The gate also POLLS (12 x 150 ms) instead of subscribing once. configure() is not synchronous, so the old subscribe-immediately-after-enabling raced it, threw, and returned null — reporting "no surface found" on a device looking straight at a table. Same lesson as the full-space return path: gates poll, they don't one-shot. Verified: builds and installs clean, launches with no crash and no HANDLE_INVALID lines. That last is WEAK evidence — the spam only occurs inside the SCAN section, which adb cannot reach without a signed-in session, so the fix needs on-device confirmation. The leak and the latch are certain; whether they were also the whole cause of the tabletop not loading is likely but unproven. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 296a3b2e29 user: dkords tags: trunk | |
| 04:11 | androidxr 0.30.0 — stop the SCAN trigger auto-firing on return from the splat viewer Owner 2026-07-30: coming back to the scan lab, the back-of-hand SCAN button fires by itself. Two causes, and the timing one is the lesser of them. ON_RESUME re-shows the menu, which starts a fresh handLockLoop with lastTapNs = 0 — but the press test is a proximity LEVEL check, not an edge. You leave the splat viewer by PINCHING its back button, so your hands come back still together, right where the trigger materialises, and the very first poll counts as a press. Holding them there re-fired every 700 ms. So the real fix is a release edge: the fingertip must be seen CLEAR of the button before any press counts, and again after each one. A fingertip that isn't tracked at all counts as clear. This kills the whole class of bug — without it, arriving with hands together fires the instant the button arms, however long you wait first. The arming delay is the belt to that braces, and is what was asked for: 7 s hidden on RETURN, 1.5 s on first entry to the section (you came here to scan). Hidden rather than merely inert, so the button's arrival is itself the signal that it is now live. One behaviour change worth knowing: resting a fingertip on the button used to repeat-fire at 700 ms intervals; now it is one press per approach, like a physical button. Verified: builds, installs and launches clean on the Aura with no crash. The gesture change itself is NOT verified on-device — the hand menu only exists inside the SCAN section, which needs a signed-in session and hands, so it needs the headset on. Also installs contributor 0.6.0+9 to the phone (splat WebView viewer). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 735403409b user: dkords tags: trunk | |
| 03:45 | server 0.56.0 — SECURITY: every security token now comes from a CSPRNG Every token generator used CL `random`, and `*random-state*` was never seeded anywhere in the codebase. SBCL's initial random state is a constant, so the whole sequence replayed on every process start. Found when a throwaway test server handed back an upload token a previous process had already minted. Reproduced directly — three fresh processes: (92 246 238 121 44 223 5 225) (92 246 238 121 44 223 5 225) (92 246 238 121 44 223 5 225) Restart the server and the same first TOTP secret, the same first device code and the same first upload token came back. Two defects, not one. Unseeded is the visible half; the other is that SBCL's `random` is MT19937, whose state is recoverable from observed output, so seeding alone would have left tokens predictable to anyone who collected enough of them. Both die by never using `random` for anything security-bearing. New src/secure-random.lisp: /dev/urandom first, ironclad (OS-seeded, verified distinct across processes) as fallback, and a hard error if neither is available. NOTHING falls back to `random` — a silent fallback would quietly restore the bug, and a token that merely looks random is worse than a loud failure. secure-below rejection-samples instead of taking a modulo: our alphabets drop ambiguous characters, so their sizes are arbitrary and modulo bias is real. Converted: TOTP secrets (totp.lisp, enrolled at web.lisp:664), 2FA backup codes, CSRF tokens, device codes, device user codes, pair codes, channel ids, org invite tokens, upload tokens, and the seed-device tokens. That last one had a comment reasoning the tokens were 'thrown away' — which only means 'unknown' if the bytes were unpredictable, and they were not: anyone could re-run bin/seed and recompute them. The release-download temp filename moves over too. It was never a token, but a guessable name in a world-writable /tmp invites a pre-created symlink. Regression test pins *random-state* to a fixed seed and asserts no generator's output repeats — any generator still on `random` would produce identical output twice, which IS the bug. A control assertion proves the pin really does make `random` deterministic, so the guard cannot silently stop guarding. NOT fixed by this commit: tokens ALREADY ISSUED under the weak PRNG are still weak. Existing 2FA enrolments and device tokens need rotating. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 953470e2ff user: dkords tags: trunk | |
| 01:38 | server 0.55.0 / contributor 0.6.0+9 — fix the phone's blank splat view The phone's twin viewer showed a BLANK PAGE. Root cause: it rendered room.glb through model-viewer, and that .glb is not a mesh. Its payload is a glTF mode 0 (POINTS) cloud of 30k vertices plus a couple of 8-vertex translucent marker boxes. model-viewer's GLTFLoader draws points at PointsMaterial{size:1, sizeAttenuation:false} — one device pixel each — so on a 3x-density phone it is indistinguishable from nothing. Every cheap check passed, which is why this hid: HTTP 200, content-type model/gltf-binary, structurally valid glTF. The pre-flight added on 2026-07-29 for exactly this class of bug could not see it either. Web and the headset were never affected because neither touches the .glb — both consume the gaussians, and the phone was the only client pointed at the point-cloud proxy. Fix: the phone now loads a served viewer page running the SAME splat-viewer.js the web console runs, against the same gaussians. One renderer across three surfaces instead of a third implementation to keep in sync. Phone-tuned: 150k splat cap and pixel ratio capped at 2, because fill rate is the binding constraint on an overdraw-heavy scene. New GET /api/contribute/scans/:id/ply (token-authed, same rules as /model) and GET /contribute/viewer/:id. The page reads the scan id and token from its OWN query string rather than server-side interpolation, so an attacker-supplied token never reaches the generated HTML — no injection seam to get wrong. The MESH tab now names what it found ('30,000 points + 12 triangles (no mesh surface — use SPLAT)'). Ranking triangles first read as '12 triangles', true and completely misleading about why the view is sparse. Localized en+ja. page-i18n.js learns ?lang=, since a WebView reports the SYSTEM locale via navigator.language and the app's language setting is independent of the phone's; the host now passes its own choice down. tests/mobile-splat.test.mjs asserts the thing that actually broke: that gaussians land ON SCREEN, at a 412x915 DPR-3 mobile viewport. It measures via onScreenFraction, not readPixels — with preserveDrawingBuffer=false readPixels reads a cleared buffer after swap and reports blank even on a successful draw. 5/5 pass, 94.8% of splats in frustum. Verified against a throwaway server on an isolated storage root so live scans were never touched. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 5e00e4209f user: dkords tags: trunk | |
| 00:22 | androidxr 0.29.0 — splat control arm P2: pinch, hover, grab, drag The arm from P1 becomes usable. Pinch a handle, drag, the scan moves. PINCH, not touch (spec 4.3). A 16 mm arrowhead at 0.45 m is too small to poke reliably, and fingertip-proximity already gave us accidental triggers on the hand menu. Pinch has hysteresis — engages under 25 mm, releases above 35 mm — so a trembling hand cannot chatter the grab on and off. Thumb tip costs no new tracking: the default joint set was already fetched in full every frame and we simply ignored that joint. Grab fires on the pinch EDGE and only when a handle is within 35 mm, so pinching in empty space does nothing. Axis drags project the hand delta onto the axis; the centre ball free-translates in the ground plane (height stays the Y handle's job). Gain 8 — the arm is a remote control for a scene metres away, so a few cm of hand travel must move it a metre. That constant is the one thing that still needs tuning with the headset on. The offset is added to SplatModel's world-space translation column, which is why drag direction matches the arrow despite the diag(1,-1,-1) flip, and the depth-sort inverse model = R*(world - T) is updated to match on all three components. If that ever drifts, the splats sort against the wrong eye and the cloud tears. No interaction without FOCUS: at VISIBLE OpenXR hands us no joints, so there is nothing to hit-test and the arm draws its disabled grey. Verified on the Aura at 72 fps (550k gaussians x 2 eyes) — cold-launched via adb it sits at VISIBLE, exactly that path, and stays stable. The pinch loop itself is unverified until the headset is worn. Also installs contributor 0.5.0+8 to the phone (swipe-to-delete gallery). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 3946f774c2 user: dkords tags: trunk | |
|
2026-07-30
| ||
| 23:42 | Splat viewer control arm P0+P1: reference plane, ground disc, X/Y/Z gizmo geometry androidxr 0.26.0 -> 0.28.0. First two phases of docs/splat-viewer-controls-spec.md. No interaction yet — this is placement + geometry, so the maths and the vertex budget are proven before any grabbing goes in. P0 — PLACEMENT ON A PLANE (owner: "placing it in a plane"): * A reference plane is latched ONCE from the first head pose (floor = head − 1.4 m, matching ModelViewerEngine's fallback). Latched, not per-frame, or the scene bobs as you move your head. * The scan now RESTS on that plane instead of floating at eye level. Watch the sign: the model→world transform flips Y, so the model-space TOP of the bbox (hi.y) becomes the world-space FLOOR — Ty = planeY + hiY. Getting that backwards buries the scan underground. * The eyeModel inverse used by the depth sort was updated to match. If that ever drifts from SplatModel, splats sort against the wrong eye and the cloud tears. * A faint cyan grid disc (1.2 m, 32-segment ring + 8 fading spokes) is drawn on the plane under the scan — without a ground reference the eye has nothing to judge position or rotation against. Drawn as BANDS, not lines: a 1-px line foreshortens to nothing at the grazing angle you actually view a floor from. * The world-space vertex count is now computed OUTSIDE the hand-tracking branch, so the disc still draws when your hands are down (it would otherwise vanish whenever you lowered them, and look like a new bug). P1 — THE ARM (geometry only): * Latched 0.45 m ahead at chest height using the head's YAW-ONLY forward, so looking up or down does not tilt the gizmo. Near-field proxy by design: the scan is ~8.5 m wide and ~5-6 m away, so its own handles would be unreachable. * X/Y/Z prisms with pyramid tips (conventional red/green/blue — worth breaking the Wipeout palette for, it is instantly readable), an octahedral centre ball, and a yaw ring in the ground plane. Solid shapes only — the "1/5 of a red circle" lesson from the hand button. * The centre ball BREATHES at 0.5 Hz so an idle arm reads as live, not a frozen decal. * DISABLED STATE IS IMPLEMENTED, and it is the point. Our recurring failure mode is SILENCE — the hand menu, the sphere and the BACK arrow each "just didn't appear" and each cost a diagnostic session. When the session is not FOCUSED OpenXR sends no hands at all, so the arm greys out and wears a red slash: it says "I am ignoring you" instead of looking broken. DEVIATION from spec §4.5: that asked for the text "NO INPUT — FOCUS LOST", but the native viewer has no text renderer (only 7-segment FPS digits), so it is a glyph. Real text there needs a small bitmap font, done once and reused. Verified on the XR emulator (the Aura keeps dropping off USB): [xr] session state -> 5 (FOCUSED — input live) [splat] reference plane at y=-1.40 (head 0.00 − 1.4) [arm] control arm latched at (0.00 -0.35 -0.45) [splat] 60.0 fps (550000 gaussians x 2 eyes) Frame cost unchanged; ~900 of the 4096 overlay vertices used (arm ~510, disc ~240, HUD ~120). LEGIBILITY IS STILL UNVERIFIED — whether a 6 cm arm reads clearly at 0.45 m against passthrough needs real hardware. Spec §8 records two follow-ons the owner raised: the same XYZ tool for the FULL-SCALE mesh viewer (which currently occludes everything with no way to move it — but that is SceneCore, so only the interaction design transfers, not the geometry), and web-parity view controls (wireframe; for gaussians the useful equivalent is an opacity scrub, a uniform in splat.frag). Both imply the arm needs room for MODE BUTTONS, not just spatial handles. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 06267c31f7 user: dkords tags: trunk | |
| 22:37 | TTS worker (20x faster readout), scan delete on all surfaces, glitchout transition, focus hardening Server 0.53.0 -> 0.54.0, androidxr 0.22.0 -> 0.26.0, contributor 0.4.2 -> 0.5.0. TTS: ~7.8 s PER READOUT -> ~0.6 s. Measured, not guessed: of the 7.8 s, ~7.4 s was importing torch and loading the Kokoro weights in a FRESH PROCESS every time; the audio itself is ~0.4 s. So the device-auth voice arrived long after the code was already on screen ("we need to pre-warm the TTS, it takes too long to get going"). * mapper/tts-daemon.{sh,py}: one long-lived synthesiser, line protocol (READY / {"text","out"} / OK|ERR). Restarts itself if it dies and falls back to the old one-shot script, so it can never be worse than before. GOTCHA FOUND IN THE SMOKE TEST: kokoro prints warnings to STDOUT, which desynchronised the protocol — every reply came back off by one. The worker now redirects all its noise to stderr so stdout carries protocol only. * POST /api/device/request also kicks synthesis off in the background at MINT time, so it overlaps with the user reading the code. The existing mutex means a client asking mid-synthesis waits on the in-flight run instead of starting a second one. Measured end-to-end: first 6.0 s (worker start), then 624/611 ms, cached 0 ms. DELETE A SCAN — now on all three surfaces (owner ask), each matched to its interaction model, all confirming first because it removes the record AND files: * Android XR: two-tap confirm (DELETE -> CONFIRM DELETE?) — a spatial mis-tap is easy. Hides the model first if you delete the one on display. * Mobile: swipe-left + dialog. Web scan lab: per-row button + confirm(). * DELETE /api/contribute/scans/:id is Bearer-header ONLY (no ?token=), so a URL can never delete by being clicked or embedded. 9 assertions. GLITCHOUT TRANSITION (owner: "surprise me glitchy"). A datamosh, not a fade: the headline arrives corrupted and resolves LEFT-TO-RIGHT like a link negotiating, over RGB channel split, horizontal slice displacement, scanlines and a CRT sweep. Four beats — ACQUIRE / RESOLVE / LOCK / DISSOLVE — with the app's own audio cues riding the same clock so it sounds like RobotARme. * Device auth: "ENTERING ORGANIZATION NETWORK" between approval and the command centre, as a real AuthUiState.EnteringNetwork so a recomposition can't skip it. * Scan -> splat: "LOADING SPLAT". Download time varies, so the animation HOLDS at its LOCK beat until the data lands, then dissolves — and the handoff to the native viewer is deferred until after the dissolve so it is never cut short. * Corruption is seeded ⇒ reproducible in a bug report. FOCUS IS A FIRST-CLASS CONCERN NOW. Field report: "back arrow doesn't show" in the splat viewer. It was not the menu, the hands, the flags or the permission — all correct. SystemUI had ANR'd and its dialog held input focus, so our session sat at VISIBLE and never reached FOCUSED. OpenXR delivers hand joints and keys ONLY to a focused session, so every input path died at once while the viewer happily rendered at 30-45 fps. The freeze watchdog could not help: frames were still arriving. * Session state is now logged in words ("VISIBLE — NO INPUT: hands/keys go elsewhere") instead of a bare integer. * The watchdog treats 20 s unfocused-while-running as stuck and leaves, so the user is never trapped in a view that cannot be exited. PLANE-TRACKING SPAM — my own regression. Putting planeTracking into the XrPerception superset meant EVERY component turned it on and left it on, and the runtime logged "Failed to get scene plane" continuously. Only the scan sweep and the viewer's surface search need planes, both briefly, so they are now reference-counted (begin/endPlanes) and off the rest of the time — the superset still guarantees hand tracking is never clobbered. LANGUAGE SETTING (owner: "add to settings so I can try it easily"): English / 日本語 in XR Settings, persisted, swapping the bundled catalog and recreating the activity. Startup honours the override instead of always following the device. TEST HYGIENE — caught while answering "is this org sprawl from your testing?". It was not (the 24 orgs date from 07-08..07-24, weeks earlier), but my delete test WAS writing into the real storage/scans/ and left two empty org dirs. Those are removed and the test now uses a temp storage root — set globally, not LET bound, because the DELETE runs on a Hunchentoot worker thread where a dynamic rebinding is invisible. Also: docs/splat-viewer-controls-spec.md — the X/Y/Z control arm design. Near-field proxy arm at 0.45 m driving the distant scene (handles on an 8.5 m scene 4.9 m away are unreachable), pinch to grab, yaw-only rotation, ~510 verts against the overlay's 4096 budget, ~3.5 d in 5 shippable phases. Tests: rove 377 green; flutter analyze clean; androidxr builds. Verified on device: TTS timings above, 0.26.0 installed, focus diagnosis taken from live logcat + dumpsys. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: 5e1f409a79 user: dkords tags: trunk | |
| 10:14 | Phone gallery parity + scan delete/recovery; splat viewer stability & flicker fixes Server 0.49.2 -> 0.53.0, androidxr 0.15.0 -> 0.22.0, contributor 0.3.0 -> 0.4.2. SPLAT VIEWER — three field-reported failures, each root-caused from evidence rather than guessed: * FOVEATION HARD-DISABLED. App-driven XR_FB_foveation_vulkan crashed the SYSTEM COMPOSITOR (SpaceFlinger), taking the OS down. Bundling the Khronos validation layer gave the actual reason: the Aura runtime OWNS foveation (it injects its own VkPhysicalDeviceFragmentDensityMapFeaturesEXT and a static density model), so our FDM render pass violated VUID-VkFramebufferCreateInfo-renderPass-02553 and submitted draws with the density map in UNDEFINED layout (VUID-vkCmdDraw-None-09600). Left wired-but-off with the diagnosis recorded; the Settings toggle is gone. Prefix-LOD already delivers the perf. * FLICKER/JUDDER. MaybeReSort std::sorted 700k splats + vkQueueWaitIdle ON the frame loop every 0.2 m of head travel: a 30-50 ms hitch plus a whole-cloud alpha reorder. Now a worker thread sorts and the main thread does an async fenced copy into an inactive double-buffered VBO (device-local kept; host- visible was tried and re-fetches every frame on Adreno). Frame loop never blocks; threshold tightened 0.20 -> 0.12 m. * UNRECOVERABLE FREEZE. The loop waited forever (xrWaitSwapchainImage INFINITE, vkWaitForFences UINT64_MAX) on android_main, so a wedged GPU meant no BACK key, frozen fps, no way out and no resource release. Waits are now bounded (100 ms x10 / 2 s) and a watchdog THREAD escalates: finish() at 8 s without a frame, _exit(0) at 14 s to guarantee the GPU comes back. Also: loader init once per process with the APPLICATION context as a GLOBAL JNI ref (a per-activity local ref went stale, so the 2nd splat launch aborted in xrEnumerateInstanceExtensionProperties); orderly exit via xrRequestExitSession; scene auto-centres on its bbox; overdraw tuned (2.8 sigma quad, 64 px clamp, 550k budget) 49 -> locked 72 fps; BACK button now rides EITHER hand. ML/DA3 SPLAT FLICKER — root cause found by measurement, not inspection: da3-gaussians.py baked a CONSTANT opacity 0.95 while scale = z/fx tiles the surface, so ~13 near-opaque same-size blobs stack per pixel and the front-most one paints ~95% of it. Any order churn (re-sort, sub-pixel jitter, or the two eyes disagreeing) swung the pixel ~28%. New --opacity flag, default 0.30: still ~99% opaque in aggregate, ~2.5x less swing. bin/splat-reopacity retunes existing PLYs for A/B without re-running the pipeline. SCAN DELETE + FAILURE RECOVERY: * DELETE /api/contribute/scans/:id (token-authed, org-scoped). Bearer header ONLY - no ?token= - so a URL can never delete by being clicked or embedded. * bin/scan-regenerate recovers failed scans from the retained input photo. Ran on the live store: 10 failed -> 5 recovered. Three distinct causes: HEIC stored as photo.jpg (SHARP burned 5.7 GB RSS before dying); a da3 pipeline broken at the time (0.03 s / 2 MB exits - same photos reconstruct fine now); and 4-byte SOI+EOI stubs with no pixels (unrecoverable). * PREVENTION: uploads are sniffed by magic bytes at both entry points - HEIC is converted (original kept), empty/undecodable is refused with HTTP 400 and a readable reason, instead of failing minutes later in a worker. * Pipeline failures now PERSIST their error tail and rehydration restores it; previously a restart erased why anything failed (meta.json read "{}"). PHONE CONTRIBUTOR - TWIN GALLERY PARITY (owner-chosen scope: full org gallery): GET /api/contribute/scans and .../photo, org-scoped BY THE SERVER from the token. New Gallery tab with thumbnail, #id . SOURCE, GLB/PLY/SOG format labels, status, and tap-through to the 3D viewer; phone-native pull-to-refresh instead of the headset's 4-per-page paging. NOTE: a photographed QR can browse (and now delete) for the token's 900 s TTL. Viewer no longer shows a BLANK SCREEN on an expired token - it pre-flights the URL and reports expired / not-ready / failed with a retry. applicationId io.iotone.responsiveiot2026 -> io.iotone.robotarme (Gradle namespace, Kotlin package, iOS bundle ids, README). ANDROID XR UI: * XrPerception - ONE superset perception config for the whole app. StatusHud configured bare deviceTracking, and since configure() REPLACES rather than merges, the hand button died the moment a scan started and only came back by leaving the screen. All four call sites now go through it; the hand-menu loop also re-configures and retries instead of giving up permanently. * Hand button: 3D sphere with a flat-panel fallback so SOMETHING always renders (the sphere threw "Cannot set scale relative to ActivitySpace with a null parent" - parent must be set BEFORE scale). Every branch logs now; class-only exception logging is what hid this for two rounds. * FULL SCALE / TABLETOP restored to text, and the row is a FlowRow - a Row does not wrap, so the toggle was silently CLIPPED off the panel edge and looked deleted. Gallery rows get weight(1f) for the same reason. * Device-auth code readout endpoint made PUBLIC: the code is shown BEFORE login, so requiring a session 401'd every request and nothing was ever spoken. Tooling: bin/contributor-build (pins JDK 17-21; Flutter prefers Android Studio Preview's Java 25, which Gradle rejects with a bare "25.0.2"), bin/splat-test, bin/splat-reopacity, bin/scan-regenerate. Tests: rove 377 green (new: photo normalisation, contributor gallery + delete, code-audio); flutter analyze clean; androidxr builds. Verified on device where possible - splat viewer renders 700k gaussians at a locked 72 fps via bin/splat-test, and the upload gate was confirmed live by POSTing a 4-byte stub (HTTP 400). Foveation stays off; hand-menu placement still needs field eyes. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> check-in: c665f2e2cd user: dkords tags: trunk | |
|
2026-07-26
| ||
| 06:42 | Attempt to stabilize FOV (removed now), add better test framework, add docs guide check-in: f59f857236 user: dkords tags: trunk | |
|
2026-07-25
| ||
| 21:37 | Add TTS for sign in, attempt to improve back of hand menu, add user settings. check-in: 9862139f7b user: dkords tags: trunk | |
| 01:44 | Splat perf P0 (67fps on Aura) + prefix-LOD + hand menus + session recovery Splat renderer performance (docs/3dgs-lod-research.md, docs/splat-quality-plan.md): - Importance-prefix LOD: mapper/splat-importance-sort.py orders every pipeline PLY by sigmoid(opacity) x splat-area DESCENDING (multi-element-safe; marker comment robotarme_importance_sorted). Every prefix of the file is now a valid LOD level: both renderers take the TOP-N instead of a uniform subsample (research-verified: importance pruning at 90% IMPROVES quality - PUP 3D-GS). SHARP's tail splats score ~1e-9; the old uniform subsample kept invisible noise while discarding content. Unsorted files keep the old fallback. - Foveated rendering: full XR_FB_foveation_vulkan chain (4 XR exts + VK_EXT_fragment_density_map + FDM render-pass attachment), HIGH static profile, every stage guarded to degrade unfoveated. - MEASURED ON AURA: 67 fps @ 700k gaussians x 2 eyes, up from 22-28 - pinned at the device's effective refresh. FDM 60x38 per eye. Web splat viewer hardening (Playwright E2E, tests/scan-lab.test.mjs, 6 tests): - FIX blank canvas: camera framing parked the eye 0.4m from a 2.7m cloud so 380k drawn triangles all landed off-screen (proved via NDC projection of sample splats; renderer.info showed the draw succeeding). - FIX vanishing splats on orbit: InstancedBufferAttribute kept data.pos BY REFERENCE, so the in-place re-sort permuted its own source - each camera move scrambled the master further. Geometry now gets .slice() copies; regression test asserts master checksum unchanged after 6 moves. - Re-sort: comparator Array.sort (~200ms @ 294k) -> O(n) counting sort over 16-bit quantized depth in pre-allocated typed arrays: 4.7ms median. Perf + valid-permutation regression tests. - onScreenFraction() sanity API; SplatViewer consumes importance-sorted PLYs. FPS HUD on every surface that renders models/splats (lower-left, owner ask): - Web scan-lab: DOM overlay, EMA of rAF deltas (Playwright-verified). - Native viewer: 7-segment digits via a new overlay Vulkan pipeline (overlay.vert/frag, per-image mapped vertex buffers), fed by the fps counter. - Compose tabletop viewer: withFrameNanos EMA readout. Contextual back-of-hand menu (docs/hand-menu-design.md): right hand carries, left hand taps - the system hold/pinch/back gesture is hard to execute. - Native splat viewer: XR_EXT_hand_tracking; red BACK plate + white arrow drawn in world space above MIDDLE_METACARPAL; LEFT INDEX_TIP within ~1.5cm triggers the clean-exit path (700ms debounce, hover brightens). - Compose SCAN screen: HandMenu.kt - hand-locked PanelEntity (Hand.right API javap-verified vs arcore alpha15) with a red SCAN trigger; audio submit cue. - android.permission.HAND_TRACKING in manifest + requested pre-handoff. - Config discipline: session.configure() REPLACES the config set, so every configure now includes handTracking=BOTH - a mid-scan configure would have silently killed the menu (nosebar head-tracking bug class). - FULL SCALE / TABLETOP text button (wrapped illegibly) -> Canvas-drawn corner-bracket maximize/restore icon; localized label kept as a11y text. Session-loss recovery (androidxr 0.13.x): - SessionExpiryInterceptor: cookie-401 AND CSRF-mismatch-403 (a dead session 403s on mutations - the CSRF middleware runs before auth) fire sessionLost; permission 403s and Bearer 401s deliberately do NOT. peekBody keeps the response intact. 4 MockWebServer tests. - AuthViewModel bounces to login with a 'session ended - sign in again' banner (en+ja) ONLY from Authenticated state; FleetViewModel is untouched so re-login returns to the same screen + data. - Gallery thumbnail dup fix: produceState kept the PREVIOUS scan's bitmap when a paged row slot was reused for a different scan - re-resolve for the current id + key(scan.id) row identity. Scan Lab uploads now send source=web (server accepts a source param on POST /api/scans) so web scans no longer masquerade as 'glasses'; scan-lab header shows the signed-in account/org (org isolation was why web uploads 'missing' from the glasses). Pipelines + research: - SAM 3D Objects evaluated and PARKED (docs/sam3d-objects-plan.md): commercial SAM License verified, but Linux+NVIDIA>=32GB only - scaffold kept de-registered as mapper/process-scan-sam3d.sh.future until lab hardware. - Deep research captured: docs/multiview-reconstruction-research.md (COLMAP->Brush Mac-local; MapAnything/VGGT lab-GPU; DUSt3R family license-blocked) and docs/3dgs-lod-research.md (19 verified claims). - da3 model-path fix: resolve DA3_CLI/DA3_MODEL to absolute BEFORE cd'ing into the scan dir (relative overrides resolved against the wrong dir). Versions: server 0.49.1, androidxr 0.15.0. Tests: rove green, androidxr JVM green (incl. 4 new session tests), Playwright scan-lab 6/6, flutter analyze clean. On-device: foveation+prefix-LOD measured; hand menus await field test. Co-authored-by: Claude Fable 5 <noreply@anthropic.com> check-in: 4f73b5642e user: dkords tags: trunk | |
|
2026-07-21
| ||
| 07:00 | Get webxr / xr parity done. check-in: 122e3b2903 user: dkords tags: trunk | |
|
2026-07-20
| ||
| 07:48 | Major cleanup on digital twin scanning work Start on new design alternative. check-in: 789998eaff user: dkords tags: trunk | |
|
2026-07-16
| ||
| 14:00 | Splat viewer fixes + Digital Twin Gallery G2 (server tokens, QR invite, Flutter contributor) Splat renderer (androidxr 0.9.1-0.10.0): - FIX GPU hang: SHARP writes a MULTI-ELEMENT ply (element vertex, then extrinsic/intrinsic/image_size/frame). The header parser summed EVERY element's properties into one stride (72B vs 56B), so every record after the first was misread -> garbage covariances -> screen-filling quads -> Adreno fault + blacked-out headset. Now only the vertex element's properties define the stride. Verified: 1,163,658 gaussians, extent 5.04 x 3.27 x 4.47 m (was 49x49x49 = garbage clamped by the range filter). - Real EWA gaussian splatting (splat.vert/frag): per-gaussian instance, quad sized to the 3D covariance projected through the perspective Jacobian. SHARP emits ~1 gaussian per input pixel at ~1.7mm median scale, so drawing centers as points could never work - they alias into a moire lattice. - Premultiplied OVER + load-time back-to-front sort (additive would saturate to white with near-opaque gaussians tiling a surface). - Defence in depth: 96px splat radius clamp; MAX_SPLATS=700k subsample cap. - Placement: keep capture-relative positions (do NOT recentre - it is a single-photo shell; recentring put the user behind the surface) + a SPLAT_PUSH_M knob. - Jitter: device-local VBO via staging (was host-visible, re-fetched ~36MB per frame per eye) + per-image fences (was blocking right after submit, serialising CPU/GPU) + FPS logging. Nosebar: 180-degree yaw so it faces the viewer (it inherited head rotation and rendered backwards once head-locking actually started working). Digital Twin Gallery G2 (docs/androidxr-twin-gallery-design.md): - Server: org-scoped short-lived upload tokens; POST /api/upload-tokens (session-authed mint) + POST /api/contribute (Bearer, no session - CSRF middleware already exempts Authorization requests); scan 'source' attribution end-to-end; %server-base-url computes a PHONE-REACHABLE LAN URL (the headset's own 127.0.0.1 adb-reverse address is useless to a phone, so the server supplies it). +4 rove tests (375 assertions). - AndroidXR: 'INVITE PHONE' QR (zxing-core) - mints a token and encodes {server,token,org} with a live expiry countdown + audio cues. en+ja. - contributor/: Flutter phone app on the IoTone responsive starter - capture/pick a photo -> multipart upload to /api/contribute; QR scan or manual connect; persisted endpoint. analyze clean, debug APK builds. KNOWN BUG (next up): after scan -> view splat -> back, the SCAN screen menus stop responding (Full-Space managed<->unmanaged round-trip leaves the SceneCore panel input dead; we also never pause the model-viewer/nosebar entities + their per-frame coroutines across the handoff). Tests: rove 375, androidxr 59, flutter analyze clean. check-in: 3a0d170106 user: dkords tags: trunk | |
|
2026-07-14
| ||
| 18:49 | Get splats visualizing without crash check-in: c8e0c054a0 user: dkords tags: trunk | |
|
2026-07-12
| ||
| 13:09 | get scan working, better model handling check-in: 6f6909d602 user: dkords tags: trunk | |
|
2026-07-10
| ||
| 02:03 | Add front facing room scanner for data capture, splat/sog viz check-in: fdc34bf1fe user: dkords tags: trunk | |
|
2026-07-06
| ||
| 01:04 | Deep research on supporting new room mapping feature for androidxr and futher improve test plan androidxr check-in: 5a2e910416 user: dkords tags: trunk | |