Vendor a patched Perspective build with column-axis expand/collapse

Perspective's column axis cannot be collapsed. The row axis has had it
forever — GROUP BY ROLLUP holds every level and view.set_depth() hides the
deeper ones — but nothing equivalent is exposed for split_by, so a Year over
Month pivot can only ever be shown fully expanded.

The engine already implements it. t_ctx2 is symmetric: set_depth(t_header,
depth), open(t_header, idx) and close(t_header, idx) each have a real
HEADER_COLUMN branch on m_ctraversal mirroring m_rtraversal, and
t_view_config carries m_column_pivot_depth which server.cpp already applies.
None of it is reachable: set_column_pivot_depth() is never called, so the
depth stays -1, and View<t_ctx2>::expand/collapse hardcode HEADER_ROW. The
patch is 193 lines of wiring across the protobuf, the Rust client and the
datagrid — no new engine logic.

Two capabilities result, mirroring the row axis:

- split_by_depth in ViewConfig, the split_by counterpart to group_by_depth
- expand_column()/collapse_column(), addressed by column traversal index
  exactly as the row methods are addressed by row index

which together give the Excel behaviour — one year folded to its subtotal
while its siblings stay expanded — that no combination of existing config
could produce. Verified in this app against fc_cash_9: clicking a Year
header goes from 27 columns to 15, totals reconciling at every level.

Vendored rather than aliased
- The previous approach pointed vite at a local checkout, which built only on
  one laptop and left package.json claiming npm 5.2.0 while the build used
  something else. The four packages are now committed as npm tarballs and
  package.json names them, so the declaration is true and `pf.sh deploy`
  works unchanged — npm install expands them like any registry package.
- Packed with `pnpm pack`, not `npm pack`: Perspective is a pnpm workspace and
  cross-package deps are `workspace:^`, which npm rejects outright. pnpm
  rewrites those to real version ranges at pack time.
- All four move together. Perspective couples loader, package versions, data
  format and apache-arrow; a partial vendor reintroduces exactly that drift.

Cost, stated plainly: 12MB of opaque binaries in git that do not delta, a
fork to maintain, and a second engine build for anyone changing it.
rebuild-perspective.sh makes that repeatable and PROVENANCE.txt records the
commit each tarball came from, because a committed .tgz otherwise has no
recoverable source. README.md says how to delete all of it once upstream
ships the feature.

This also moves the app from 5.2.0 to 5.4.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoxNi8cFsQLPUSw3obb5NH

ui/package-lock.json

@@ -8,10 +8,10 @@
       "name": "ui",
       "version": "0.0.0",
       "dependencies": {
-        "@perspective-dev/client": "5.2.0",
-        "@perspective-dev/server": "5.2.0",
-        "@perspective-dev/viewer": "5.2.0",
-        "@perspective-dev/viewer-datagrid": "5.2.0",
+        "@perspective-dev/client": "file:./vendor/perspective-dev-client-5.4.0.tgz",
+        "@perspective-dev/server": "file:./vendor/perspective-dev-server-5.4.0.tgz",
+        "@perspective-dev/viewer": "file:./vendor/perspective-dev-viewer-5.4.0.tgz",
+        "@perspective-dev/viewer-datagrid": "file:./vendor/perspective-dev-viewer-datagrid-5.4.0.tgz",
         "react": "^19.2.5",
         "react-dom": "^19.2.5"
       },
@@ -554,43 +554,43 @@
       }
     },
     "node_modules/@perspective-dev/client": {
-      "version": "5.2.0",
-      "resolved": "https://registry.npmjs.org/@perspective-dev/client/-/client-5.2.0.tgz",
-      "integrity": "sha512-zkJmJFwdw0wMREoJt8gUJyCsflzi7s7Yc8ZtUCOLE9XB6fdyxJmDB99cxO3Q+hno/57kLsz8VNoz8XSm6PZNrg==",
+      "version": "5.4.0",
+      "resolved": "file:vendor/perspective-dev-client-5.4.0.tgz",
+      "integrity": "sha512-l9xCJ0W42wm9fLlwhoU838KyTE131qQTS686uQb/VcsLs30kCVjkoermCCUs0YHCDwDTJhly1y9DfAZDbkg9Ng==",
       "license": "Apache-2.0",
       "dependencies": {
-        "@perspective-dev/server": "",
+        "@perspective-dev/server": "^5.4.0",
         "pro_self_extracting_wasm": "0.0.9",
         "stoppable": "=1.1.0",
         "ws": "^8.17.0"
       }
     },
     "node_modules/@perspective-dev/server": {
-      "version": "5.2.0",
-      "resolved": "https://registry.npmjs.org/@perspective-dev/server/-/server-5.2.0.tgz",
-      "integrity": "sha512-WRBiokT2/BYM8Ipe7Dg1/2UYNb01Vsox79KBfFVI800VmwdId0GdqI95zufN5ZoFffeBfri1sr3S3AShBRFXKA==",
+      "version": "5.4.0",
+      "resolved": "file:vendor/perspective-dev-server-5.4.0.tgz",
+      "integrity": "sha512-iTseRJB6TL6D9xjaMKMhh2NEKMIi9JR881J+GyQflHIQXK43fDlsIWtByUAoyZzZ7uA9KNZJZicirvONxIpLuw==",
       "license": "Apache-2.0"
     },
     "node_modules/@perspective-dev/viewer": {
-      "version": "5.2.0",
-      "resolved": "https://registry.npmjs.org/@perspective-dev/viewer/-/viewer-5.2.0.tgz",
-      "integrity": "sha512-IIyqdPduofzzZ/QWrteq29IadJQR8IapTRz7U6XbrtBBm7eyiFsIBKQV0wFfcqWg4TRnhuvCAUvBRieuE0R8Eg==",
+      "version": "5.4.0",
+      "resolved": "file:vendor/perspective-dev-viewer-5.4.0.tgz",
+      "integrity": "sha512-7D6jNn7tDZ3W84MsydplWAJbqqpXIz5OlsHx5YG4sHvJ5q8ngZQvP+oZdxLQXL4hIbaxpskMld1gJbGltpcvUw==",
       "license": "Apache-2.0",
       "dependencies": {
-        "@perspective-dev/client": "",
+        "@perspective-dev/client": "^5.4.0",
         "pro_self_extracting_wasm": "0.0.9",
         "regular-layout": "=0.6.1"
       }
     },
     "node_modules/@perspective-dev/viewer-datagrid": {
-      "version": "5.2.0",
-      "resolved": "https://registry.npmjs.org/@perspective-dev/viewer-datagrid/-/viewer-datagrid-5.2.0.tgz",
-      "integrity": "sha512-v/SR/35YfyKfivO21nglJFiyG1iE5G8vMwIwWI6fQuwjW764pmNrm/zcrNWNY70x5XHkPRM94aY6LjkLsLkO2g==",
+      "version": "5.4.0",
+      "resolved": "file:vendor/perspective-dev-viewer-datagrid-5.4.0.tgz",
+      "integrity": "sha512-7ITOmrIh1ZpiQbUR/KmHYck1sLA4cpNgpVMI/qJjQc9k/uypkT1vJmKYxv4MKR24TmEkOouBLNYiO+ItFKs8vg==",
       "license": "Apache-2.0",
       "dependencies": {
-        "@perspective-dev/client": "",
-        "@perspective-dev/viewer": "",
-        "regular-table": "=0.8.6"
+        "@perspective-dev/client": "^5.4.0",
+        "@perspective-dev/viewer": "^5.4.0",
+        "regular-table": "=0.9.1"
       }
     },
     "node_modules/@rolldown/binding-android-arm64": {
@@ -2948,9 +2948,9 @@
       "license": "Apache-2.0"
     },
     "node_modules/regular-table": {
-      "version": "0.8.6",
-      "resolved": "https://registry.npmjs.org/regular-table/-/regular-table-0.8.6.tgz",
-      "integrity": "sha512-jzJzu9WLtqwMgbf5ak/VdNm/YLHUDnDSCHD5SOksnHrHLuDN6l5i2stYfe7BjsHbQV1Gx9369Ma40nTBCgOFvA==",
+      "version": "0.9.1",
+      "resolved": "https://registry.npmjs.org/regular-table/-/regular-table-0.9.1.tgz",
+      "integrity": "sha512-3I/2I3NEhmyScCevW/r1AE9hyUkqrPRMDsgo/nDnGXhpXVZOxUddJQGBmEMLgyc7OSaQSzl9BtXllnMqw1MwCA==",
       "license": "Apache-2.0",
       "engines": {
         "node": ">=16"

ui/package.json

@@ -10,10 +10,10 @@
     "preview": "vite preview"
   },
   "dependencies": {
-    "@perspective-dev/client": "5.2.0",
-    "@perspective-dev/server": "5.2.0",
-    "@perspective-dev/viewer": "5.2.0",
-    "@perspective-dev/viewer-datagrid": "5.2.0",
+    "@perspective-dev/client": "file:./vendor/perspective-dev-client-5.4.0.tgz",
+    "@perspective-dev/server": "file:./vendor/perspective-dev-server-5.4.0.tgz",
+    "@perspective-dev/viewer": "file:./vendor/perspective-dev-viewer-5.4.0.tgz",
+    "@perspective-dev/viewer-datagrid": "file:./vendor/perspective-dev-viewer-datagrid-5.4.0.tgz",
     "react": "^19.2.5",
     "react-dom": "^19.2.5"
   },

ui/vendor/PROVENANCE.txt

@@ -0,0 +1,7 @@
+Built from https://github.com/fleetside72/perspective
+  branch  column-axis-expand-collapse
+  commit  2e3901d652650a33eaf19c2ddf049f7e525ea95b
+  based on v5.4.0
+  built   2026-09-14T02:40:11Z on r710.hptrow.me
+
+Regenerate with ui/vendor/rebuild-perspective.sh

ui/vendor/README.md

@@ -0,0 +1,50 @@
+# Vendored Perspective
+
+pf_app runs a **patched build of Perspective**. Upstream's C++ engine has always
+implemented column-axis expand/collapse — `t_ctx2::set_depth(HEADER_COLUMN, …)`
+and `open`/`close(HEADER_COLUMN, idx)` are fully written — but nothing above C++
+could reach it: `set_column_pivot_depth()` was never called, and
+`View<t_ctx2>::expand/collapse` hardcoded `HEADER_ROW`. The patch is wiring, not
+new engine logic.
+
+It buys two things the released packages cannot do at all:
+
+- `split_by_depth` in `ViewConfig`, the `split_by` counterpart to `group_by_depth`
+- `expand_column()` / `collapse_column()`, so one column branch can fold to its
+  subtotal while its siblings stay expanded — the Excel behaviour
+
+**Source:** https://github.com/fleetside72/perspective, branch
+`column-axis-expand-collapse`. See `PROVENANCE.txt` for the exact commit these
+tarballs were built from.
+
+## Why tarballs and not npm
+
+The feature is not released upstream. Until it is, the four packages are built
+from the fork and committed here as npm tarballs. `npm install` expands them
+exactly as it expands anything from the registry — no special tooling, and
+`pf.sh deploy` works unchanged. A deploy machine needs node and nothing else:
+no emscripten, no cmake, no protoc, no Rust.
+
+All four move together, never a subset. Perspective couples loader, package
+versions, data format and `apache-arrow`; vendoring a partial set reintroduces
+exactly the drift that causes trouble.
+
+## Changing the engine
+
+    ./rebuild-perspective.sh          # builds the fork, repacks, rewrites PROVENANCE.txt
+    cd .. && npm install
+    git add vendor && git commit
+
+Push the fork first — the script warns if the source tree is dirty, because a
+tarball built from uncommitted code has no recoverable source.
+
+The build itself needs cmake >= 3.29.5, protoc >= 22 (its version silently
+selects which protobuf source tree gets cloned), pnpm, and the Rust nightly the
+repo pins. Roughly 40 minutes cold. Only ever on a machine changing the engine.
+
+## Getting rid of this
+
+This is a fork, with the maintenance that implies. The exit is upstream taking
+the change — the patch is small and additive, and the engine work is already
+theirs. When a release ships it, delete this directory and put normal version
+ranges back in `ui/package.json`.

ui/vendor/rebuild-perspective.sh

@@ -0,0 +1,102 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# ---------------------------------------------------------------------------
+# rebuild-perspective.sh — rebuild the patched Perspective and re-vendor it
+#
+# pf_app runs a patched build of Perspective that exposes the column axis
+# expand/collapse the engine already implements (split_by_depth, and
+# expand_column/collapse_column). Upstream does not ship this yet, so the
+# built packages are vendored into this directory as npm tarballs.
+#
+# Source of truth:  https://github.com/fleetside72/perspective
+#                   branch column-axis-expand-collapse
+#
+# This script exists because vendored binaries are opaque: once the .tgz files
+# are committed, nothing in the repo records how to regenerate them. Run this
+# after changing the fork, then commit the resulting tarballs.
+#
+# Only needed on a machine that is changing the engine. Deploys just run
+# `npm install`, which expands the committed tarballs - see ../README in this
+# directory.
+# ---------------------------------------------------------------------------
+
+PSP="${PSP_DIR:-$HOME/perspective}"
+BRANCH="${PSP_BRANCH:-column-axis-expand-collapse}"
+VENDOR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+# The four packages that must move together. Perspective's own docs are
+# emphatic that loader, packages, data format and apache-arrow are one unit;
+# vendoring a subset would reintroduce exactly the drift that causes trouble.
+PACKAGES=(
+    "rust/perspective-js"
+    "rust/perspective-server"
+    "rust/perspective-viewer"
+    "packages/viewer-datagrid"
+)
+
+info() { echo -e "\033[0;34m==>\033[0m $*"; }
+ok()   { echo -e "\033[0;32m  ✓\033[0m $*"; }
+die()  { echo -e "\033[0;31m  ✗\033[0m $*" >&2; exit 1; }
+
+# -- preflight --------------------------------------------------------------
+[[ -d "$PSP" ]] || die "No Perspective checkout at $PSP.
+    git clone https://github.com/fleetside72/perspective.git $PSP
+    cd $PSP && git checkout $BRANCH
+    Set PSP_DIR to use a different path."
+
+command -v pnpm >/dev/null || die "pnpm not found. Perspective builds with pnpm, not npm."
+command -v protoc >/dev/null || die "protoc not found.
+    Its VERSION selects which protobuf source tree the build clones, and a
+    version below 22 pulls a layout the build cannot consume. Needs >= 22
+    (33.2 known good). Distro packages are usually far too old."
+
+cmake_ver=$(cmake --version 2>/dev/null | head -1 | grep -oE '[0-9]+\.[0-9]+(\.[0-9]+)?') || die "cmake not found"
+cmake_major=${cmake_ver%%.*}; cmake_minor=$(echo "$cmake_ver" | cut -d. -f2)
+if (( cmake_major < 3 || (cmake_major == 3 && cmake_minor < 29) )); then
+    die "cmake $cmake_ver is too old; Perspective needs >= 3.29.5.
+    A user-level install works: pip3 install --user 'cmake>=3.29.5'"
+fi
+
+info "Perspective checkout: $PSP"
+git -C "$PSP" rev-parse --abbrev-ref HEAD | grep -qx "$BRANCH" \
+    || echo "  ! on branch $(git -C "$PSP" rev-parse --abbrev-ref HEAD), expected $BRANCH"
+commit=$(git -C "$PSP" rev-parse --short HEAD)
+dirty=$(git -C "$PSP" status --porcelain | wc -l)
+echo "  commit $commit$([[ $dirty -gt 0 ]] && echo " (+$dirty uncommitted files)")"
+
+# -- build ------------------------------------------------------------------
+# `metadata` first: it generates docs/expression_gen.md, which perspective-client
+# includes at compile time. Building a scope without it fails on the missing file.
+info "Building (this takes ~40 minutes cold, a few minutes warm)…"
+( cd "$PSP" && PSP_ONCE=1 PACKAGE="metadata,server,client,viewer,viewer-datagrid" pnpm run build )
+ok "build complete"
+
+# -- pack -------------------------------------------------------------------
+info "Packing tarballs into $VENDOR"
+rm -f "$VENDOR"/*.tgz
+for p in "${PACKAGES[@]}"; do
+    ( cd "$PSP/$p" && npm pack --pack-destination "$VENDOR" >/dev/null )
+    ok "$(basename "$p")"
+done
+
+# -- record provenance ------------------------------------------------------
+# A committed .tgz is an opaque binary; without this the tie back to source is
+# only in someone's memory.
+cat > "$VENDOR/PROVENANCE.txt" <<EOF
+Built from https://github.com/fleetside72/perspective
+  branch  $BRANCH
+  commit  $(git -C "$PSP" rev-parse HEAD)
+  based on $(git -C "$PSP" describe --tags --abbrev=0 2>/dev/null || echo 'unknown')
+  built   $(date -u +%Y-%m-%dT%H:%M:%SZ) on $(hostname)
+  dirty   $dirty uncommitted file(s) in the source tree at build time
+
+Regenerate with ui/vendor/rebuild-perspective.sh
+EOF
+
+echo
+ls -la "$VENDOR"/*.tgz | awk '{printf "  %-52s %5.1f MB\n", $NF, $5/1048576}'
+echo
+ok "Done. Now: cd ui && npm install && git add vendor && git commit"
+[[ $dirty -gt 0 ]] && echo -e "\033[1;33m  !\033[0m source tree had uncommitted changes — push them to the fork first"
+exit 0