Document both axes collapsing through ViewConfig

§Row depth and the observer shim and §Column hierarchy described two
different mechanisms and an asymmetry between the axes that no longer
exists: rows were imperative view state chased by an observer shim, and
columns collapsed by truncating split_by. Both are a depth in ViewConfig
now, so the two sections are one.

Records what matters for reading the code: that the depths are 1-based
against server.cpp's `set_depth(depth - 1)`, that this needed the engine
patch in ui/vendor because apply_update dropped the fields, and that
Session::update_view_config returning early on "no change" is why nothing
happened at all rather than happening wrongly.

Keeps a short account of what was deleted -- the shim, the focus listeners,
the retry loop, splitFull, collapsingRef, the prefix test -- because someone
will otherwise wonder why the surrounding code looks so plain, and because
split_full is still read for layouts saved under the old scheme.

Per-node expansion moves to its own subsection with the reason it is hard
(set-only API, no getter) and what fixing it would take. The stale claim that
the selection is cleared on every column change is gone: with a depth the
dimensions a slice was cut from are all still present.

Also: observerShim.js dropped from the project layout, and the tech stack
now says 5.4.0 from the patched vendored build rather than 5.2.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Paul Trowbridge 2026-09-17 12:38:18 -04:00
parent 607dac221d
commit 4d2af589b9

135
CLAUDE.md
View File

@ -14,7 +14,7 @@ Data transport architecture options: `pf_perspective_options.md`
- **Backend:** Node.js / Express (`server.js`) - **Backend:** Node.js / Express (`server.js`)
- **Database:** PostgreSQL — isolated `pf` schema - **Database:** PostgreSQL — isolated `pf` schema
- **Frontend:** React + Vite + Tailwind CSS in `ui/`; built output lands in `public/app/` - **Frontend:** React + Vite + Tailwind CSS in `ui/`; built output lands in `public/app/`
- **Pivot:** [Perspective](https://github.com/perspective-dev/perspective) (`@perspective-dev/*` distribution, **not** FINOS `@finos/perspective`) 5.2.0, **bundled inline via the `/inline` entrypoints — never from a CDN** (the 4.x CDN bundle resolves its server WASM to an unversioned path and silently pulls whatever is newest). See `PERSPECTIVE.md`. - **Pivot:** [Perspective](https://github.com/perspective-dev/perspective) (`@perspective-dev/*` distribution, **not** FINOS `@finos/perspective`) 5.4.0 from a **patched build vendored in `ui/vendor`** — see its README for what is patched and which host-tool versions the rebuild needs; **bundled inline via the `/inline` entrypoints — never from a CDN** (the 4.x CDN bundle resolves its server WASM to an unversioned path and silently pulls whatever is newest). See `PERSPECTIVE.md`.
- **Dev:** `npm run dev` (nodemon) in root; `npm run build` in `ui/` - **Dev:** `npm run dev` (nodemon) in root; `npm run build` in `ui/`
--- ---
@ -38,7 +38,6 @@ setup_sql/
01_schema.sql pf schema DDL — run once to install 01_schema.sql pf schema DDL — run once to install
02_auth.sql pf.app_user + pf.session 02_auth.sql pf.app_user + pf.session
ui/src/ ui/src/
observerShim.js Wraps window.Intersection/ResizeObserver; MUST be main.jsx's first import
auth.jsx AuthProvider/useAuth; wraps fetch so any 401 returns to login auth.jsx AuthProvider/useAuth; wraps fetch so any 401 returns to login
views/ views/
Login.jsx Sign-in form Login.jsx Sign-in form
@ -154,37 +153,80 @@ gets written.
--- ---
## Row depth and the observer shim ## Axis depth (collapse / expand)
`set_depth()` and per-node expand/collapse live on the **view**, not in `ViewConfig`. Both axes collapse the same way: a **depth in `ViewConfig`**, set through
So every view rebuild starts fully expanded, and nothing in Perspective restores it — `restore()`.
the app has to re-apply depth itself.
The engine has no notion of focus or tab visibility. It re-renders because - **Rows**`group_by_depth`, driven by the `EXPAND 0 1 2 3` buttons via
`IntersectionObserver` or `ResizeObserver` told it the element is visible again, and `applyDepth()`
*that* is what discards the view. `ui/src/observerShim.js` wraps both constructors so - **Columns**`split_by_depth`, driven by the `COLUMNS` buttons via
the re-apply fires on the actual callback rather than on a window `focus` guess. `applySplitDepth()`
**It must stay the first import in `main.jsx`.** Perspective's viewer captures the Both are **1-based**: they count the levels to show, where the imperative
constructors at module-evaluation time (`var it=window.ResizeObserver; var `view.set_depth()` counts the boundary below them. `server.cpp` does
st=window.IntersectionObserver`), so any import that reaches `perspective-viewer` `ctx1->set_depth(row_pivot_depth - 1)` and
first leaves the shim with nothing to intercept. `focus`/`visibilitychange`/`pageshow` `ctx2->set_depth(HEADER_COLUMN, column_pivot_depth - 1)`, so the toolbar sends
remain as a backstop for when that happens. `d + 1` and subtracts one again when reading a layout back.
Two things learned the hard way, both easy to repeat: Because a depth lives in the config, it survives every view rebuild, rides into
the persisted and named layouts through `viewer.save()`, and needs nothing
re-applied afterwards.
- Around a rebuild the viewer throws `No table set` from `getView()` while **This required patching the engine.** `ViewConfig::apply_update` applied ten
`getTable()` resolves happily. Gating on the table does not help; fields and neither depth, so a depth sent through `restore()` was accepted,
`applyDepthWhenReady()` retries `applyDepth` until it stops throwing. deserialized and dropped — and since `Session::update_view_config` returns early
- `getView()` returns a **fresh wrapper object every call**, so object identity when `apply_update` reports no change, no view was rebuilt at all. See
cannot be used to detect a rebuild. Two consecutive calls always look different. `ui/vendor/0001-apply-depth-fields-on-config-update.patch`; the vendored build
carries it.
Tracing lives behind `localStorage.pf_debug = '1'`, prefixed `[pf-depth]` and ### What this replaced
`[pf-obs]`.
Worth knowing, because a lot of machinery existed to work around it and is now
gone:
- Row depth used to be imperative — `getView()` then `view.set_depth()` — which
put it on an object the viewer discards whenever it re-renders. Restoring it
meant guessing when that had happened: an `observerShim.js` patching
`window.IntersectionObserver` and `window.ResizeObserver`, focus and
visibility listeners, a retry loop for `getView()` throwing `No table set`
while `getTable()` resolved, and a flag tracking whether the viewer had "gone
away". That guesswork produced three distinct visible faults — the tree fully
expanding on refocus, snapping on any reflow, and snapping when Perspective's
settings sidebar opened.
- Column collapse used to restore a **truncated `split_by`**. The discarded
levels therefore had to be remembered separately (`splitFull`, persisted as
`split_full`), our own collapse had to be told apart from the user rearranging
the pivot (`collapsingRef` plus a prefix test), and the selection was cleared
on every collapse because the axis was changing shape.
`split_full` is still *read* on load, for layouts saved under the old scheme.
### Still not solved: per-node expansion
Expanding one specific branch is view state with no config representation, and
the API has **no getter**:
```
expand(row_index: number): Promise<number>
collapse(row_index: number): Promise<number>
```
It can be set but not read, so it cannot be captured and replayed — this is why
it is lost on every rebuild, and why no client-side fix has worked. The honest
route is a `ViewConfig` field carrying expanded row *paths* (indices shift as
the tree opens), applied in `server.cpp` where the depths are. Bigger than the
depth patch: it needs a way to enumerate expanded nodes in the C++ traversal, a
proto field, and the apply step.
The fork's own `header_click.ts` / `expand_column` / `collapse_column` is the
column-axis equivalent and has the same limitation.
**Limitation that remains either way:** depth is whole-axis. Excel can collapse
2025 while 2026 stays expanded; a depth collapses every group at that level
together. `columns` selects which *measures* appear, not individual split
combinations.
**Not solved:** per-node `+/` is still lost on rebuild. The view exposes
`expand(row)` / `collapse(row)` with no getter, so the state cannot be read back —
it would have to be shadowed by intercepting the grid's own calls.
## Slice mechanics ## Slice mechanics
@ -202,43 +244,6 @@ Turning a region back into slices re-derives, per cell, the same filters Perspec
--- ---
## Column hierarchy (collapse / expand)
The two pivot axes collapse by completely different mechanisms, and the asymmetry is a
Perspective constraint, not a choice:
- **Rows.** The `GROUP BY ROLLUP` view holds every level at once; `view.set_depth()` — which
lives on the view, not the config — hides the deeper ones. That is what the `EXPAND 0 1 2 3`
buttons drive, via `applyDepth()`.
- **Columns.** There is no equivalent. `expand()` / `collapse()` take a **row index**,
`ViewConfig` has `group_by_depth` but no `split_by_depth`, and `split_rollup_mode`
(`'flat' | 'rollup'`) only chooses whether subtotal column groups are *emitted* — it is a
view shape, not an interaction. So `applySplitDepth(n)` collapses by restoring a
**truncated `split_by`**, which rebuilds the view.
Three things follow from the rebuild, and each is handled:
1. The full hierarchy has to be remembered separately — once collapsed, `viewer.save()`
only reports the short `split_by`. `splitFullRef` / `splitFull` hold it, and it is
persisted into the saved layout as `split_full` so a reload while collapsed can still
expand back. `adoptSplit()` is the single place it is set.
2. `perspective-config-update` fires for our own restore as well as the user rearranging
the pivot. `collapsingRef` distinguishes them — without it, a collapse would overwrite
the full hierarchy with the truncated one and the deeper levels would be unreachable.
3. Row depth lives on the discarded view, so `applyDepth(expandDepthRef.current)` is
re-applied afterwards — the same wart as the refocus re-apply.
The selection is cleared on every change: slices name the split_by dimensions they were
cut from, and the highlight is keyed on grid coordinates. Neither survives a column axis
that just changed shape.
**Limitation:** this is whole-axis, not per-branch. Excel can collapse 2025 while 2026
stays expanded; truncating `split_by` collapses every column group at that level together.
Per-branch is not reachable — `columns` selects which *measures* appear, not individual
split combinations.
---
## Operation SQL patterns ## Operation SQL patterns
All three operations follow the same structure: insert a `pf.log` row in a CTE, then insert forecast rows referencing its id. `{{where_clause}}` is built from the slice; `{{exclude_clause}}` blocks `exclude_iters` rows. All three operations follow the same structure: insert a `pf.log` row in a CTE, then insert forecast rows referencing its id. `{{where_clause}}` is built from the slice; `{{exclude_clause}}` blocks `exclude_iters` rows.
@ -298,8 +303,8 @@ Theme state lives in `ui/src/theme.jsx` — a React context (`ThemeContext`) wit
remaining lever is **dynamic grain** — group by the fields the current pivot remaining lever is **dynamic grain** — group by the fields the current pivot
actually uses rather than every `in_grain` column; see `pf_spec.md` actually uses rather than every `in_grain` column; see `pf_spec.md`
§Display-grain pre-aggregation, "the dynamic variant" §Display-grain pre-aggregation, "the dynamic variant"
- Per-node row expand/collapse is lost whenever the view rebuilds; see §Row depth and - Per-node expand/collapse is lost whenever the view rebuilds — set-only API, no
the observer shim getter; see §Axis depth
- Default pivot layout should be configurable per source (currently hardcodes first 2 dimensions) - Default pivot layout should be configurable per source (currently hardcodes first 2 dimensions)
- Source/version selection persists in `localStorage` (`pf_sourceId` / `pf_versionId`, - Source/version selection persists in `localStorage` (`pf_sourceId` / `pf_versionId`,
`App.jsx`). It is re-validated against the live list whenever that list changes, so a `App.jsx`). It is re-validated against the live list whenever that list changes, so a