diff --git a/CLAUDE.md b/CLAUDE.md index e3f111b..443e1b7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -14,7 +14,7 @@ Data transport architecture options: `pf_perspective_options.md` - **Backend:** Node.js / Express (`server.js`) - **Database:** PostgreSQL — isolated `pf` schema - **Frontend:** React + Vite + Tailwind CSS in `ui/`; built output lands in `public/app/` -- **Pivot:** [Perspective](https://github.com/perspective-dev/perspective) (`@perspective-dev/*` distribution, **not** FINOS `@finos/perspective`) 5.2.0, **bundled inline via the `/inline` entrypoints — never from a CDN** (the 4.x CDN bundle resolves its server WASM to an unversioned path and silently pulls whatever is newest). See `PERSPECTIVE.md`. +- **Pivot:** [Perspective](https://github.com/perspective-dev/perspective) (`@perspective-dev/*` distribution, **not** FINOS `@finos/perspective`) 5.4.0 from a **patched build vendored in `ui/vendor`** — see its README for what is patched and which host-tool versions the rebuild needs; **bundled inline via the `/inline` entrypoints — never from a CDN** (the 4.x CDN bundle resolves its server WASM to an unversioned path and silently pulls whatever is newest). See `PERSPECTIVE.md`. - **Dev:** `npm run dev` (nodemon) in root; `npm run build` in `ui/` --- @@ -38,7 +38,6 @@ setup_sql/ 01_schema.sql pf schema DDL — run once to install 02_auth.sql pf.app_user + pf.session ui/src/ - observerShim.js Wraps window.Intersection/ResizeObserver; MUST be main.jsx's first import auth.jsx AuthProvider/useAuth; wraps fetch so any 401 returns to login views/ Login.jsx Sign-in form @@ -154,37 +153,80 @@ gets written. --- -## Row depth and the observer shim +## Axis depth (collapse / expand) -`set_depth()` and per-node expand/collapse live on the **view**, not in `ViewConfig`. -So every view rebuild starts fully expanded, and nothing in Perspective restores it — -the app has to re-apply depth itself. +Both axes collapse the same way: a **depth in `ViewConfig`**, set through +`restore()`. -The engine has no notion of focus or tab visibility. It re-renders because -`IntersectionObserver` or `ResizeObserver` told it the element is visible again, and -*that* is what discards the view. `ui/src/observerShim.js` wraps both constructors so -the re-apply fires on the actual callback rather than on a window `focus` guess. +- **Rows** — `group_by_depth`, driven by the `EXPAND 0 1 2 3` buttons via + `applyDepth()` +- **Columns** — `split_by_depth`, driven by the `COLUMNS` buttons via + `applySplitDepth()` -**It must stay the first import in `main.jsx`.** Perspective's viewer captures the -constructors at module-evaluation time (`var it=window.ResizeObserver; var -st=window.IntersectionObserver`), so any import that reaches `perspective-viewer` -first leaves the shim with nothing to intercept. `focus`/`visibilitychange`/`pageshow` -remain as a backstop for when that happens. +Both are **1-based**: they count the levels to show, where the imperative +`view.set_depth()` counts the boundary below them. `server.cpp` does +`ctx1->set_depth(row_pivot_depth - 1)` and +`ctx2->set_depth(HEADER_COLUMN, column_pivot_depth - 1)`, so the toolbar sends +`d + 1` and subtracts one again when reading a layout back. -Two things learned the hard way, both easy to repeat: +Because a depth lives in the config, it survives every view rebuild, rides into +the persisted and named layouts through `viewer.save()`, and needs nothing +re-applied afterwards. -- Around a rebuild the viewer throws `No table set` from `getView()` while - `getTable()` resolves happily. Gating on the table does not help; - `applyDepthWhenReady()` retries `applyDepth` until it stops throwing. -- `getView()` returns a **fresh wrapper object every call**, so object identity - cannot be used to detect a rebuild. Two consecutive calls always look different. +**This required patching the engine.** `ViewConfig::apply_update` applied ten +fields and neither depth, so a depth sent through `restore()` was accepted, +deserialized and dropped — and since `Session::update_view_config` returns early +when `apply_update` reports no change, no view was rebuilt at all. See +`ui/vendor/0001-apply-depth-fields-on-config-update.patch`; the vendored build +carries it. -Tracing lives behind `localStorage.pf_debug = '1'`, prefixed `[pf-depth]` and -`[pf-obs]`. +### What this replaced + +Worth knowing, because a lot of machinery existed to work around it and is now +gone: + +- Row depth used to be imperative — `getView()` then `view.set_depth()` — which + put it on an object the viewer discards whenever it re-renders. Restoring it + meant guessing when that had happened: an `observerShim.js` patching + `window.IntersectionObserver` and `window.ResizeObserver`, focus and + visibility listeners, a retry loop for `getView()` throwing `No table set` + while `getTable()` resolved, and a flag tracking whether the viewer had "gone + away". That guesswork produced three distinct visible faults — the tree fully + expanding on refocus, snapping on any reflow, and snapping when Perspective's + settings sidebar opened. +- Column collapse used to restore a **truncated `split_by`**. The discarded + levels therefore had to be remembered separately (`splitFull`, persisted as + `split_full`), our own collapse had to be told apart from the user rearranging + the pivot (`collapsingRef` plus a prefix test), and the selection was cleared + on every collapse because the axis was changing shape. + +`split_full` is still *read* on load, for layouts saved under the old scheme. + +### Still not solved: per-node expansion + +Expanding one specific branch is view state with no config representation, and +the API has **no getter**: + +``` +expand(row_index: number): Promise +collapse(row_index: number): Promise +``` + +It can be set but not read, so it cannot be captured and replayed — this is why +it is lost on every rebuild, and why no client-side fix has worked. The honest +route is a `ViewConfig` field carrying expanded row *paths* (indices shift as +the tree opens), applied in `server.cpp` where the depths are. Bigger than the +depth patch: it needs a way to enumerate expanded nodes in the C++ traversal, a +proto field, and the apply step. + +The fork's own `header_click.ts` / `expand_column` / `collapse_column` is the +column-axis equivalent and has the same limitation. + +**Limitation that remains either way:** depth is whole-axis. Excel can collapse +2025 while 2026 stays expanded; a depth collapses every group at that level +together. `columns` selects which *measures* appear, not individual split +combinations. -**Not solved:** per-node `+/−` is still lost on rebuild. The view exposes -`expand(row)` / `collapse(row)` with no getter, so the state cannot be read back — -it would have to be shadowed by intercepting the grid's own calls. ## Slice mechanics @@ -202,43 +244,6 @@ Turning a region back into slices re-derives, per cell, the same filters Perspec --- -## Column hierarchy (collapse / expand) - -The two pivot axes collapse by completely different mechanisms, and the asymmetry is a -Perspective constraint, not a choice: - -- **Rows.** The `GROUP BY ROLLUP` view holds every level at once; `view.set_depth()` — which - lives on the view, not the config — hides the deeper ones. That is what the `EXPAND 0 1 2 3` - buttons drive, via `applyDepth()`. -- **Columns.** There is no equivalent. `expand()` / `collapse()` take a **row index**, - `ViewConfig` has `group_by_depth` but no `split_by_depth`, and `split_rollup_mode` - (`'flat' | 'rollup'`) only chooses whether subtotal column groups are *emitted* — it is a - view shape, not an interaction. So `applySplitDepth(n)` collapses by restoring a - **truncated `split_by`**, which rebuilds the view. - -Three things follow from the rebuild, and each is handled: - -1. The full hierarchy has to be remembered separately — once collapsed, `viewer.save()` - only reports the short `split_by`. `splitFullRef` / `splitFull` hold it, and it is - persisted into the saved layout as `split_full` so a reload while collapsed can still - expand back. `adoptSplit()` is the single place it is set. -2. `perspective-config-update` fires for our own restore as well as the user rearranging - the pivot. `collapsingRef` distinguishes them — without it, a collapse would overwrite - the full hierarchy with the truncated one and the deeper levels would be unreachable. -3. Row depth lives on the discarded view, so `applyDepth(expandDepthRef.current)` is - re-applied afterwards — the same wart as the refocus re-apply. - -The selection is cleared on every change: slices name the split_by dimensions they were -cut from, and the highlight is keyed on grid coordinates. Neither survives a column axis -that just changed shape. - -**Limitation:** this is whole-axis, not per-branch. Excel can collapse 2025 while 2026 -stays expanded; truncating `split_by` collapses every column group at that level together. -Per-branch is not reachable — `columns` selects which *measures* appear, not individual -split combinations. - ---- - ## Operation SQL patterns All three operations follow the same structure: insert a `pf.log` row in a CTE, then insert forecast rows referencing its id. `{{where_clause}}` is built from the slice; `{{exclude_clause}}` blocks `exclude_iters` rows. @@ -298,8 +303,8 @@ Theme state lives in `ui/src/theme.jsx` — a React context (`ThemeContext`) wit remaining lever is **dynamic grain** — group by the fields the current pivot actually uses rather than every `in_grain` column; see `pf_spec.md` → §Display-grain pre-aggregation, "the dynamic variant" -- Per-node row expand/collapse is lost whenever the view rebuilds; see §Row depth and - the observer shim +- Per-node expand/collapse is lost whenever the view rebuilds — set-only API, no + getter; see §Axis depth - Default pivot layout should be configurable per source (currently hardcodes first 2 dimensions) - Source/version selection persists in `localStorage` (`pf_sourceId` / `pf_versionId`, `App.jsx`). It is re-validated against the live list whenever that list changes, so a