pt/pf_app
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Compare commits

merge into: pt:master
Branches Tags
pt:master
pt:feature/perspective-column-collapse
pt:spike/column-branch-collapse
pt:feature/column-collapse
pt:feature/migrations
pt:feature/static-grain
pt:spike/duckdb-virtual-server
...
pull from: pt:feature/perspective-column-collapse
Branches Tags
pt:feature/perspective-column-collapse
pt:spike/column-branch-collapse
pt:feature/column-collapse
pt:master
pt:feature/migrations
pt:feature/static-grain
pt:spike/duckdb-virtual-server

3 Commits
master ... feature/pe

Author SHA1 Message Date
Paul Trowbridge
146961cc17 Point pf.sh at the DB_* env vars the server actually reads 
server.js builds its pg pool from DB_HOST/DB_PORT/DB_NAME/DB_USER/
DB_PASSWORD, and that is what install.sh writes, but pf.sh had been
written against a DATABASE_URL/PF_USER scheme that nothing consumes.
The consequences were real: `status` always reported the database
unreachable, and `config` rewrote .env with cat >, replacing a working
connection with keys the server ignores.

Connect through a run_psql helper built from the DB_* vars, and have
config prompt for those six keys instead. Each prompt defaults to the
current value so entering through changes nothing, the password prompt
is hidden and keeps the stored one when left blank, and any other key
already in .env is carried over rather than dropped.

PF_USER goes away with it — pf_user reaches the server in the request
body, never from the environment.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
 2026-09-14 22:08:49 -04:00
Paul Trowbridge
b1eb68a475 Vendor a patched Perspective build with column-axis expand/collapse 
Perspective's column axis cannot be collapsed. The row axis has had it
forever — GROUP BY ROLLUP holds every level and view.set_depth() hides the
deeper ones — but nothing equivalent is exposed for split_by, so a Year over
Month pivot can only ever be shown fully expanded.

The engine already implements it. t_ctx2 is symmetric: set_depth(t_header,
depth), open(t_header, idx) and close(t_header, idx) each have a real
HEADER_COLUMN branch on m_ctraversal mirroring m_rtraversal, and
t_view_config carries m_column_pivot_depth which server.cpp already applies.
None of it is reachable: set_column_pivot_depth() is never called, so the
depth stays -1, and View<t_ctx2>::expand/collapse hardcode HEADER_ROW. The
patch is 193 lines of wiring across the protobuf, the Rust client and the
datagrid — no new engine logic.

Two capabilities result, mirroring the row axis:

- split_by_depth in ViewConfig, the split_by counterpart to group_by_depth
- expand_column()/collapse_column(), addressed by column traversal index
  exactly as the row methods are addressed by row index

which together give the Excel behaviour — one year folded to its subtotal
while its siblings stay expanded — that no combination of existing config
could produce. Verified in this app against fc_cash_9: clicking a Year
header goes from 27 columns to 15, totals reconciling at every level.

Vendored rather than aliased
- The previous approach pointed vite at a local checkout, which built only on
  one laptop and left package.json claiming npm 5.2.0 while the build used
  something else. The four packages are now committed as npm tarballs and
  package.json names them, so the declaration is true and `pf.sh deploy`
  works unchanged — npm install expands them like any registry package.
- Packed with `pnpm pack`, not `npm pack`: Perspective is a pnpm workspace and
  cross-package deps are `workspace:^`, which npm rejects outright. pnpm
  rewrites those to real version ranges at pack time.
- All four move together. Perspective couples loader, package versions, data
  format and apache-arrow; a partial vendor reintroduces exactly that drift.

Cost, stated plainly: 12MB of opaque binaries in git that do not delta, a
fork to maintain, and a second engine build for anyone changing it.
rebuild-perspective.sh makes that repeatable and PROVENANCE.txt records the
commit each tarball came from, because a committed .tgz otherwise has no
recoverable source. README.md says how to delete all of it once upstream
ships the feature.

This also moves the app from 5.2.0 to 5.4.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoxNi8cFsQLPUSw3obb5NH
 2026-09-13 22:42:05 -04:00
Paul Trowbridge
4b9296abc1 Collapse the column hierarchy from the toolbar 
The row axis has had Expand 0/1/2/3 since the pivot landed; the column
axis had nothing. With a year over month split there was no way to step
back to whole years short of dragging split_by apart in the settings
panel and putting it back afterwards.

Perspective gives the two axes nothing in common here. Rows collapse
because the GROUP BY ROLLUP view holds every level at once and
view.set_depth() hides the deeper ones. For columns there is no
equivalent: expand()/collapse() take a row index, ViewConfig has
group_by_depth but no split_by_depth, and split_rollup_mode only chooses
whether subtotal column groups are emitted — a view shape, not an
interaction. So applySplitDepth() collapses by restoring a truncated
split_by, which rebuilds the view.

Three consequences of that rebuild, each handled:

- Once collapsed, viewer.save() only reports the short split_by, so the
  full hierarchy is held separately (splitFullRef) and persisted into the
  layout as split_full. Without it, collapsing would be a one-way door:
  reload while collapsed and the deeper levels are gone. adoptSplit() is
  the single place it is set.
- perspective-config-update fires for our own restore as well as for the
  user rearranging the pivot, and the two mean opposite things — one must
  adopt the new hierarchy, the other must not. collapsingRef separates
  them.
- Row depth lives on the discarded view, so it is re-applied afterwards.

The selection is cleared on each change: slices name the split_by
dimensions they were cut from, and the highlight is keyed on grid
coordinates. Neither survives a column axis that just changed shape.

Buttons are named for the level they show — Total, then one per split_by
column — rather than numbered like Expand, since the levels are named and
a number would say nothing about what you are collapsing to.

Whole-axis, not per-branch: Excel can collapse 2025 while 2026 stays
expanded, and this cannot. `columns` selects which measures appear, not
individual split combinations, so there is no way to hide one branch's
leaves while keeping another's.

Verified in the browser against cash/test with split_by Year x Reason:
Reason -> Total -> Year -> Reason all render the expected column sets and
the right button highlights; and a reload while collapsed to Year comes
back collapsed with Reason still offered.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoxNi8cFsQLPUSw3obb5NH
 2026-09-12 09:15:15 -04:00
12 changed files with 394 additions and 57 deletions
Show Stats Expand all files Collapse all files

37
CLAUDE.md
View File

@ -95,6 +95,43 @@ Turning a region back into slices re-derives, per cell, the same filters Perspec
--- ---
## Column hierarchy (collapse / expand)
The two pivot axes collapse by completely different mechanisms, and the asymmetry is a
Perspective constraint, not a choice:
- **Rows.** The `GROUP BY ROLLUP` view holds every level at once; `view.set_depth()` — which
lives on the view, not the config — hides the deeper ones. That is what the `EXPAND 0 1 2 3`
buttons drive, via `applyDepth()`.
- **Columns.** There is no equivalent. `expand()` / `collapse()` take a **row index**,
`ViewConfig` has `group_by_depth` but no `split_by_depth`, and `split_rollup_mode`
(`'flat' | 'rollup'`) only chooses whether subtotal column groups are *emitted* — it is a
view shape, not an interaction. So `applySplitDepth(n)` collapses by restoring a
**truncated `split_by`**, which rebuilds the view.
Three things follow from the rebuild, and each is handled:
1. The full hierarchy has to be remembered separately — once collapsed, `viewer.save()`
only reports the short `split_by`. `splitFullRef` / `splitFull` hold it, and it is
persisted into the saved layout as `split_full` so a reload while collapsed can still
expand back. `adoptSplit()` is the single place it is set.
2. `perspective-config-update` fires for our own restore as well as the user rearranging
the pivot. `collapsingRef` distinguishes them — without it, a collapse would overwrite
the full hierarchy with the truncated one and the deeper levels would be unreachable.
3. Row depth lives on the discarded view, so `applyDepth(expandDepthRef.current)` is
re-applied afterwards — the same wart as the refocus re-apply.
The selection is cleared on every change: slices name the split_by dimensions they were
cut from, and the highlight is keyed on grid coordinates. Neither survives a column axis
that just changed shape.
**Limitation:** this is whole-axis, not per-branch. Excel can collapse 2025 while 2026
stays expanded; truncating `split_by` collapses every column group at that level together.
Per-branch is not reachable — `columns` selects which *measures* appear, not individual
split combinations.
---
## Operation SQL patterns ## Operation SQL patterns
All three operations follow the same structure: insert a `pf.log` row in a CTE, then insert forecast rows referencing its id. `{{where_clause}}` is built from the slice; `{{exclude_clause}}` blocks `exclude_iters` rows. All three operations follow the same structure: insert a `pf.log` row in a CTE, then insert forecast rows referencing its id. `{{where_clause}}` is built from the slice; `{{exclude_clause}}` blocks `exclude_iters` rows.

99
pf.sh
View File

@ -67,13 +67,33 @@ require_service() {
service_installed || die "systemd service not installed. Run: ./pf.sh install-service" service_installed || die "systemd service not installed. Run: ./pf.sh install-service"
} }
# The keys cmd_config manages; anything else in .env is left alone.
ENV_KEYS=(DB_HOST DB_PORT DB_NAME DB_USER DB_PASSWORD PORT)
env_get() {
[[ -f "$ENV_FILE" ]] || return 0
grep -E "^$1=" "$ENV_FILE" | tail -1 | cut -d= -f2- | tr -d '"' || true
}
# psql against the DB_* connection in .env; extra args are passed through.
run_psql() {
PGPASSWORD="${DB_PASSWORD:-}" psql \
-h "${DB_HOST:-localhost}" \
-p "${DB_PORT:-5432}" \
-U "${DB_USER}" \
-d "${DB_NAME}" \
"$@"
}
db_ping() { db_ping() {
load_env load_env
local url="${DATABASE_URL:-}" if [[ -z "${DB_NAME:-}" || -z "${DB_USER:-}" ]]; then
[[ -z "$url" ]] && { warn "DATABASE_URL not set in .env"; return 1; } warn "DB_NAME / DB_USER not set in .env"
return 1
fi
# Use psql if available for a real connectivity check # Use psql if available for a real connectivity check
if command -v psql &>/dev/null; then if command -v psql &>/dev/null; then
psql "$url" -c "SELECT 1" &>/dev/null && return 0 || return 1 run_psql -tAc "SELECT 1" &>/dev/null && return 0 || return 1
else else
warn "psql not in PATH — skipping live DB check" warn "psql not in PATH — skipping live DB check"
return 0 return 0
@ -168,8 +188,7 @@ cmd_logs() {
cmd_db_setup() { cmd_db_setup() {
require_env; load_env require_env; load_env
local url="${DATABASE_URL:-}" [[ -n "${DB_NAME:-}" && -n "${DB_USER:-}" ]] || die "DB_NAME / DB_USER not set in .env — run: ./pf.sh config"
[[ -z "$url" ]] && die "DATABASE_URL not set in .env"
command -v psql &>/dev/null || die "psql not found — install postgresql-client" command -v psql &>/dev/null || die "psql not found — install postgresql-client"
echo echo
@ -178,7 +197,7 @@ cmd_db_setup() {
read -rp " Continue? [y/N] " confirm read -rp " Continue? [y/N] " confirm
[[ "$confirm" =~ ^[Yy]$ ]] || { echo "Aborted."; return; } [[ "$confirm" =~ ^[Yy]$ ]] || { echo "Aborted."; return; }
psql "$url" -f "${APP_DIR}/setup_sql/01_schema.sql" run_psql -v ON_ERROR_STOP=1 -f "${APP_DIR}/setup_sql/01_schema.sql"
success "Schema applied." success "Schema applied."
} }
@ -188,38 +207,62 @@ cmd_config() {
echo " File: $ENV_FILE" echo " File: $ENV_FILE"
echo echo
local current_url="" local cur_host cur_port cur_name cur_user cur_pass cur_app_port
local current_port="" cur_host=$(env_get DB_HOST)
local current_user="" cur_port=$(env_get DB_PORT)
cur_name=$(env_get DB_NAME)
cur_user=$(env_get DB_USER)
cur_pass=$(env_get DB_PASSWORD)
cur_app_port=$(env_get PORT)
if [[ -f "$ENV_FILE" ]]; then local input
current_url=$(grep -E '^DATABASE_URL=' "$ENV_FILE" | cut -d= -f2- | tr -d '"' || true) read -rp " DB_HOST [${cur_host:-localhost}]: " input
current_port=$(grep -E '^PORT=' "$ENV_FILE" | cut -d= -f2- | tr -d '"' || true) local host="${input:-${cur_host:-localhost}}"
current_user=$(grep -E '^PF_USER=' "$ENV_FILE" | cut -d= -f2- | tr -d '"' || true)
read -rp " DB_PORT [${cur_port:-5432}]: " input
local port="${input:-${cur_port:-5432}}"
read -rp " DB_NAME [${cur_name:-not set}]: " input
local name="${input:-$cur_name}"
[[ -z "$name" ]] && die "DB_NAME is required."
read -rp " DB_USER [${cur_user:-$USER}]: " input
local user="${input:-${cur_user:-$USER}}"
if [[ -n "$cur_pass" ]]; then
read -rsp " DB_PASSWORD [keep existing]: " input; echo
else
read -rsp " DB_PASSWORD: " input; echo
fi fi
local pass="${input:-$cur_pass}"
read -rp " DATABASE_URL [${current_url:-not set}]: " input_url read -rp " PORT (app) [${cur_app_port:-3010}]: " input
local url="${input_url:-$current_url}" local app_port="${input:-${cur_app_port:-3010}}"
[[ -z "$url" ]] && die "DATABASE_URL is required."
read -rp " PORT [${current_port:-3010}]: " input_port # Rewrite the managed keys, carrying over any other lines already in .env.
local port="${input_port:-${current_port:-3010}}" local tmp
tmp=$(mktemp)
read -rp " PF_USER [${current_user:-$USER}]: " input_user cat > "$tmp" <<EOF
local pf_user="${input_user:-${current_user:-$USER}}" DB_HOST=${host}
DB_PORT=${port}
cat > "$ENV_FILE" <<EOF DB_NAME=${name}
DATABASE_URL=${url} DB_USER=${user}
PORT=${port} DB_PASSWORD=${pass}
PF_USER=${pf_user} PORT=${app_port}
EOF EOF
if [[ -f "$ENV_FILE" ]]; then
local managed
managed=$(IFS='|'; echo "${ENV_KEYS[*]}")
grep -vE "^(${managed})=" "$ENV_FILE" | grep -vE '^[[:space:]]*$' >> "$tmp" || true
fi
mv "$tmp" "$ENV_FILE"
chmod 600 "$ENV_FILE" chmod 600 "$ENV_FILE"
success ".env written." success ".env written."
if db_ping; then if db_ping; then
success "Database connection verified." success "Database connection verified."
else else
warn "Could not reach the database — double-check DATABASE_URL." warn "Could not reach the database — double-check the DB_* settings."
fi fi
} }
@ -303,7 +346,7 @@ interactive_menu() {
echo " 5) status service + DB + git info" echo " 5) status service + DB + git info"
echo " 6) logs tail journald logs" echo " 6) logs tail journald logs"
echo " 7) db-setup apply setup_sql/01_schema.sql" echo " 7) db-setup apply setup_sql/01_schema.sql"
echo " 8) config set DATABASE_URL / PORT / PF_USER" echo " 8) config set DB connection + app PORT"
echo " 9) install-service create systemd unit file" echo " 9) install-service create systemd unit file"
echo " 10) uninstall-service remove systemd unit file" echo " 10) uninstall-service remove systemd unit file"
echo " q) quit" echo " q) quit"

48
ui/package-lock.json generated
View File

@ -8,10 +8,10 @@
"name": "ui", "name": "ui",
"version": "0.0.0", "version": "0.0.0",
"dependencies": { "dependencies": {
"@perspective-dev/client": "5.2.0", "@perspective-dev/client": "file:./vendor/perspective-dev-client-5.4.0.tgz",
"@perspective-dev/server": "5.2.0", "@perspective-dev/server": "file:./vendor/perspective-dev-server-5.4.0.tgz",
"@perspective-dev/viewer": "5.2.0", "@perspective-dev/viewer": "file:./vendor/perspective-dev-viewer-5.4.0.tgz",
"@perspective-dev/viewer-datagrid": "5.2.0", "@perspective-dev/viewer-datagrid": "file:./vendor/perspective-dev-viewer-datagrid-5.4.0.tgz",
"react": "^19.2.5", "react": "^19.2.5",
"react-dom": "^19.2.5" "react-dom": "^19.2.5"
}, },
@ -554,43 +554,43 @@
} }
}, },
"node_modules/@perspective-dev/client": { "node_modules/@perspective-dev/client": {
"version": "5.2.0", "version": "5.4.0",
"resolved": "https://registry.npmjs.org/@perspective-dev/client/-/client-5.2.0.tgz", "resolved": "file:vendor/perspective-dev-client-5.4.0.tgz",
"integrity": "sha512-zkJmJFwdw0wMREoJt8gUJyCsflzi7s7Yc8ZtUCOLE9XB6fdyxJmDB99cxO3Q+hno/57kLsz8VNoz8XSm6PZNrg==", "integrity": "sha512-l9xCJ0W42wm9fLlwhoU838KyTE131qQTS686uQb/VcsLs30kCVjkoermCCUs0YHCDwDTJhly1y9DfAZDbkg9Ng==",
"license": "Apache-2.0", "license": "Apache-2.0",
"dependencies": { "dependencies": {
"@perspective-dev/server": "", "@perspective-dev/server": "^5.4.0",
"pro_self_extracting_wasm": "0.0.9", "pro_self_extracting_wasm": "0.0.9",
"stoppable": "=1.1.0", "stoppable": "=1.1.0",
"ws": "^8.17.0" "ws": "^8.17.0"
} }
}, },
"node_modules/@perspective-dev/server": { "node_modules/@perspective-dev/server": {
"version": "5.2.0", "version": "5.4.0",
"resolved": "https://registry.npmjs.org/@perspective-dev/server/-/server-5.2.0.tgz", "resolved": "file:vendor/perspective-dev-server-5.4.0.tgz",
"integrity": "sha512-WRBiokT2/BYM8Ipe7Dg1/2UYNb01Vsox79KBfFVI800VmwdId0GdqI95zufN5ZoFffeBfri1sr3S3AShBRFXKA==", "integrity": "sha512-iTseRJB6TL6D9xjaMKMhh2NEKMIi9JR881J+GyQflHIQXK43fDlsIWtByUAoyZzZ7uA9KNZJZicirvONxIpLuw==",
"license": "Apache-2.0" "license": "Apache-2.0"
}, },
"node_modules/@perspective-dev/viewer": { "node_modules/@perspective-dev/viewer": {
"version": "5.2.0", "version": "5.4.0",
"resolved": "https://registry.npmjs.org/@perspective-dev/viewer/-/viewer-5.2.0.tgz", "resolved": "file:vendor/perspective-dev-viewer-5.4.0.tgz",
"integrity": "sha512-IIyqdPduofzzZ/QWrteq29IadJQR8IapTRz7U6XbrtBBm7eyiFsIBKQV0wFfcqWg4TRnhuvCAUvBRieuE0R8Eg==", "integrity": "sha512-7D6jNn7tDZ3W84MsydplWAJbqqpXIz5OlsHx5YG4sHvJ5q8ngZQvP+oZdxLQXL4hIbaxpskMld1gJbGltpcvUw==",
"license": "Apache-2.0", "license": "Apache-2.0",
"dependencies": { "dependencies": {
"@perspective-dev/client": "", "@perspective-dev/client": "^5.4.0",
"pro_self_extracting_wasm": "0.0.9", "pro_self_extracting_wasm": "0.0.9",
"regular-layout": "=0.6.1" "regular-layout": "=0.6.1"
} }
}, },
"node_modules/@perspective-dev/viewer-datagrid": { "node_modules/@perspective-dev/viewer-datagrid": {
"version": "5.2.0", "version": "5.4.0",
"resolved": "https://registry.npmjs.org/@perspective-dev/viewer-datagrid/-/viewer-datagrid-5.2.0.tgz", "resolved": "file:vendor/perspective-dev-viewer-datagrid-5.4.0.tgz",
"integrity": "sha512-v/SR/35YfyKfivO21nglJFiyG1iE5G8vMwIwWI6fQuwjW764pmNrm/zcrNWNY70x5XHkPRM94aY6LjkLsLkO2g==", "integrity": "sha512-7ITOmrIh1ZpiQbUR/KmHYck1sLA4cpNgpVMI/qJjQc9k/uypkT1vJmKYxv4MKR24TmEkOouBLNYiO+ItFKs8vg==",
"license": "Apache-2.0", "license": "Apache-2.0",
"dependencies": { "dependencies": {
"@perspective-dev/client": "", "@perspective-dev/client": "^5.4.0",
"@perspective-dev/viewer": "", "@perspective-dev/viewer": "^5.4.0",
"regular-table": "=0.8.6" "regular-table": "=0.9.1"
} }
}, },
"node_modules/@rolldown/binding-android-arm64": { "node_modules/@rolldown/binding-android-arm64": {
@ -2948,9 +2948,9 @@
"license": "Apache-2.0" "license": "Apache-2.0"
}, },
"node_modules/regular-table": { "node_modules/regular-table": {
"version": "0.8.6", "version": "0.9.1",
"resolved": "https://registry.npmjs.org/regular-table/-/regular-table-0.8.6.tgz", "resolved": "https://registry.npmjs.org/regular-table/-/regular-table-0.9.1.tgz",
"integrity": "sha512-jzJzu9WLtqwMgbf5ak/VdNm/YLHUDnDSCHD5SOksnHrHLuDN6l5i2stYfe7BjsHbQV1Gx9369Ma40nTBCgOFvA==", "integrity": "sha512-3I/2I3NEhmyScCevW/r1AE9hyUkqrPRMDsgo/nDnGXhpXVZOxUddJQGBmEMLgyc7OSaQSzl9BtXllnMqw1MwCA==",
"license": "Apache-2.0", "license": "Apache-2.0",
"engines": { "engines": {
"node": ">=16" "node": ">=16"

8
ui/package.json
View File

@ -10,10 +10,10 @@
"preview": "vite preview" "preview": "vite preview"
}, },
"dependencies": { "dependencies": {
"@perspective-dev/client": "5.2.0", "@perspective-dev/client": "file:./vendor/perspective-dev-client-5.4.0.tgz",
"@perspective-dev/server": "5.2.0", "@perspective-dev/server": "file:./vendor/perspective-dev-server-5.4.0.tgz",
"@perspective-dev/viewer": "5.2.0", "@perspective-dev/viewer": "file:./vendor/perspective-dev-viewer-5.4.0.tgz",
"@perspective-dev/viewer-datagrid": "5.2.0", "@perspective-dev/viewer-datagrid": "file:./vendor/perspective-dev-viewer-datagrid-5.4.0.tgz",
"react": "^19.2.5", "react": "^19.2.5",
"react-dom": "^19.2.5" "react-dom": "^19.2.5"
}, },

100
ui/src/views/Forecast.jsx
View File

@ -23,6 +23,8 @@ function cleanLayout(cfg, validCols) {
if (c.columns) c.columns = c.columns.filter(col => col == null || ok(col)) if (c.columns) c.columns = c.columns.filter(col => col == null || ok(col))
if (c.group_by) c.group_by = c.group_by.filter(ok) if (c.group_by) c.group_by = c.group_by.filter(ok)
if (c.split_by) c.split_by = c.split_by.filter(ok) if (c.split_by) c.split_by = c.split_by.filter(ok)
// the uncollapsed column hierarchy travels with the layout (see applySplitDepth)
if (c.split_full) c.split_full = c.split_full.filter(ok)
if (c.sort) c.sort = c.sort.filter(([col]) => ok(col)) if (c.sort) c.sort = c.sort.filter(([col]) => ok(col))
if (c.filter) c.filter = c.filter.filter(([col]) => ok(col)) if (c.filter) c.filter = c.filter.filter(([col]) => ok(col))
return c return c
@ -42,6 +44,12 @@ export default function Forecast({ sources = [], sourceId, versions = [], versio
const [saveAsName, setSaveAsName] = useState('') const [saveAsName, setSaveAsName] = useState('')
// operation panel — a selection is a LIST of slices; one entry is the common case // operation panel — a selection is a LIST of slices; one entry is the common case
// The column hierarchy and how many of its levels are showing. Mirrored into
// splitFullRef for handlers registered once; held as state so the toolbar
// re-renders when either changes.
const [splitFull, setSplitFull] = useState([])
const [splitDepth, setSplitDepth] = useState(null)
const [slices, setSlices] = useState([]) const [slices, setSlices] = useState([])
const [applyMode, setApplyMode] = useState('prorate') // 'prorate' | 'each' const [applyMode, setApplyMode] = useState('prorate') // 'prorate' | 'each'
const [activeOp, setActiveOp] = useState('scale') const [activeOp, setActiveOp] = useState('scale')
@ -167,6 +175,12 @@ export default function Forecast({ sources = [], sourceId, versions = [], versio
const tableRef = useRef(null) const tableRef = useRef(null)
const colMetaRef = useRef([]) const colMetaRef = useRef([])
const expandDepthRef = useRef(null) const expandDepthRef = useRef(null)
// The column axis has no set_depth() — collapsing it means restoring a shorter
// split_by, so the full hierarchy has to be remembered separately to expand again.
const splitFullRef = useRef([])
// set while our own restore is in flight, so the config-update listener can tell
// a collapse from the user rearranging split_by themselves
const collapsingRef = useRef(false)
const initIdRef = useRef(0) const initIdRef = useRef(0)
const modifierRef = useRef(false) const modifierRef = useRef(false)
// the datagrid plugin element, for reading cell coordinates and driving its // the datagrid plugin element, for reading cell coordinates and driving its
@ -536,6 +550,7 @@ export default function Forecast({ sources = [], sourceId, versions = [], versio
setLoadProgress(null) setLoadProgress(null)
setSlices([]) setSlices([])
expandDepthRef.current = null expandDepthRef.current = null
adoptSplit([], 0)
try { try {
const [dataResult, meta] = await Promise.all([ const [dataResult, meta] = await Promise.all([
fetch(`/api/versions/${vid}/data`).then(async r => { fetch(`/api/versions/${vid}/data`).then(async r => {
@ -616,6 +631,9 @@ export default function Forecast({ sources = [], sourceId, versions = [], versio
const { table: _t, ...rest } = cleanLayout(JSON.parse(saved), validCols) const { table: _t, ...rest } = cleanLayout(JSON.parse(saved), validCols)
const cfg = { ...rest, plugin_config: { ...(rest.plugin_config || {}), edit_mode: 'SELECT_REGION' } } const cfg = { ...rest, plugin_config: { ...(rest.plugin_config || {}), edit_mode: 'SELECT_REGION' } }
await viewer.restore(cfg) await viewer.restore(cfg)
// split_full outlives split_by: a layout saved while collapsed still knows
// the levels it was collapsed from
adoptSplit(cfg.split_full?.length ? cfg.split_full : cfg.split_by, (cfg.split_by || []).length)
if (cfg.expand_depth != null) await applyDepth(cfg.expand_depth) if (cfg.expand_depth != null) await applyDepth(cfg.expand_depth)
} else { } else {
const sourceDefault = sources.find(s => String(s.id) === String(sid))?.default_layout const sourceDefault = sources.find(s => String(s.id) === String(sid))?.default_layout
@ -633,12 +651,19 @@ export default function Forecast({ sources = [], sourceId, versions = [], versio
} }
} }
await viewer.restore(cfg) await viewer.restore(cfg)
adoptSplit(cfg.split_full?.length ? cfg.split_full : cfg.split_by, (cfg.split_by || []).length)
} }
// auto-persist viewer state (formatting, columns, etc.) to the last-used cache // auto-persist viewer state (formatting, columns, etc.) to the last-used cache
if (viewer._pspUpdate) viewer.removeEventListener('perspective-config-update', viewer._pspUpdate) if (viewer._pspUpdate) viewer.removeEventListener('perspective-config-update', viewer._pspUpdate)
viewer._pspUpdate = async () => { viewer._pspUpdate = async () => {
try { try {
// A split_by change that is not ours is the user rearranging the pivot, and
// it redefines the hierarchy. Ours is a collapse, and must not overwrite it.
if (!collapsingRef.current) {
const live = await viewer.save()
adoptSplit(live.split_by || [], (live.split_by || []).length)
}
const cfg = await captureConfig() const cfg = await captureConfig()
if (cfg) await persistLayout(vid, cfg) if (cfg) await persistLayout(vid, cfg)
} catch {} } catch {}
@ -690,6 +715,54 @@ export default function Forecast({ sources = [], sourceId, versions = [], versio
} }
} }
// Record the column hierarchy as the uncollapsed truth. Called whenever a layout
// arrives or the user rearranges split_by themselves — but never for our own
// collapse, which would otherwise overwrite the full list with the short one.
function adoptSplit(full, depth) {
const list = Array.isArray(full) ? full : []
splitFullRef.current = list
setSplitFull(list)
setSplitDepth(depth == null ? list.length : Math.min(depth, list.length))
}
// Collapse or expand the column axis to `n` split_by levels.
//
// The row axis gets this for free: its GROUP BY ROLLUP view holds every level at
// once and view.set_depth() hides the deeper ones. The column axis has no
// equivalent — there is no split_by_depth in ViewConfig and expand()/collapse()
// take a row index — so collapsing means restoring a truncated split_by, which
// rebuilds the view. Two consequences fall out of that: the row depth has to be
// re-applied afterwards (it lives on the discarded view), and it is whole-axis,
// not per-branch — every column group collapses to the same level together.
async function applySplitDepth(n) {
const viewer = viewerRef.current
const full = splitFullRef.current
if (!viewer || !full.length) return
const depth = Math.max(0, Math.min(n, full.length))
collapsingRef.current = true
try {
await viewer.restore({ split_by: full.slice(0, depth) })
setSplitDepth(depth)
// restore() rebuilt the view, so the row depth that lived on the old one is gone
if (expandDepthRef.current != null) await applyDepth(expandDepthRef.current)
} catch (err) {
console.error('[applySplitDepth]', err)
flash(err.message || String(err), 'error')
return
} finally {
collapsingRef.current = false
}
// a slice names the split_by dimensions it was cut from, and the highlight is
// keyed on grid coordinates — neither survives a column axis that just changed
setSlices([])
try {
const cfg = await captureConfig()
if (cfg) await persistLayout(versionId, cfg)
} catch (err) {
console.error('[applySplitDepth persist]', err)
}
}
async function applyDepth(d) { async function applyDepth(d) {
const viewer = viewerRef.current const viewer = viewerRef.current
if (!viewer) return if (!viewer) return
@ -704,7 +777,7 @@ export default function Forecast({ sources = [], sourceId, versions = [], versio
const viewer = viewerRef.current const viewer = viewerRef.current
if (!viewer) return null if (!viewer) return null
const cfg = await viewer.save() const cfg = await viewer.save()
return { ...cfg, expand_depth: expandDepthRef.current } return { ...cfg, expand_depth: expandDepthRef.current, split_full: splitFullRef.current }
} }
async function persistLayout(vid, cfg) { async function persistLayout(vid, cfg) {
@ -762,6 +835,7 @@ export default function Forecast({ sources = [], sourceId, versions = [], versio
const cfg = cleanLayout(layout.config, validCols) const cfg = cleanLayout(layout.config, validCols)
cfg.plugin_config = { ...(cfg.plugin_config || {}), edit_mode: 'SELECT_REGION' } cfg.plugin_config = { ...(cfg.plugin_config || {}), edit_mode: 'SELECT_REGION' }
await viewer.restore(cfg) await viewer.restore(cfg)
adoptSplit(cfg.split_full?.length ? cfg.split_full : cfg.split_by, (cfg.split_by || []).length)
if (cfg.expand_depth != null) await applyDepth(cfg.expand_depth) if (cfg.expand_depth != null) await applyDepth(cfg.expand_depth)
setActiveLayoutId(layout.id) setActiveLayoutId(layout.id)
await persistLayout(versionId, cfg) await persistLayout(versionId, cfg)
@ -1087,6 +1161,30 @@ export default function Forecast({ sources = [], sourceId, versions = [], versio
))} ))}
</div> </div>
{splitFull.length > 0 && (
<>
<div className="w-px h-4 bg-gray-200 shrink-0" />
{/* Column hierarchy group — the split_by equivalent of Expand */}
<div className="flex items-center gap-1.5">
<span className="text-gray-400 uppercase tracking-wide" style={{fontSize:'10px'}}>Columns</span>
{Array.from({ length: splitFull.length + 1 }, (_, n) => {
const label = n === 0 ? 'Total' : splitFull[n - 1]
return (
<button key={n} onClick={() => applySplitDepth(n)}
title={n === 0
? 'Collapse the columns to a single total'
: `Show columns down to ${splitFull.slice(0, n).join(' › ')}`}
className={`border rounded px-1.5 py-0.5 transition-colors max-w-[9rem] truncate
${splitDepth === n ? 'border-blue-300 text-blue-600 bg-blue-50' : 'border-gray-200 text-gray-500 hover:border-gray-400'}`}>
{label}
</button>
)
})}
</div>
</>
)}
<div className="w-px h-4 bg-gray-200 shrink-0" /> <div className="w-px h-4 bg-gray-200 shrink-0" />
{/* Data group */} {/* Data group */}

7
ui/vendor/PROVENANCE.txt vendored Normal file
View File

@ -0,0 +1,7 @@
Built from https://github.com/fleetside72/perspective
branch column-axis-expand-collapse
commit 2e3901d652650a33eaf19c2ddf049f7e525ea95b
based on v5.4.0
built 2026-09-14T02:40:11Z on r710.hptrow.me
Regenerate with ui/vendor/rebuild-perspective.sh

50
ui/vendor/README.md vendored Normal file
View File

@ -0,0 +1,50 @@
# Vendored Perspective
pf_app runs a **patched build of Perspective**. Upstream's C++ engine has always
implemented column-axis expand/collapse — `t_ctx2::set_depth(HEADER_COLUMN, …)`
and `open`/`close(HEADER_COLUMN, idx)` are fully written — but nothing above C++
could reach it: `set_column_pivot_depth()` was never called, and
`View<t_ctx2>::expand/collapse` hardcoded `HEADER_ROW`. The patch is wiring, not
new engine logic.
It buys two things the released packages cannot do at all:
- `split_by_depth` in `ViewConfig`, the `split_by` counterpart to `group_by_depth`
- `expand_column()` / `collapse_column()`, so one column branch can fold to its
subtotal while its siblings stay expanded — the Excel behaviour
**Source:** https://github.com/fleetside72/perspective, branch
`column-axis-expand-collapse`. See `PROVENANCE.txt` for the exact commit these
tarballs were built from.
## Why tarballs and not npm
The feature is not released upstream. Until it is, the four packages are built
from the fork and committed here as npm tarballs. `npm install` expands them
exactly as it expands anything from the registry — no special tooling, and
`pf.sh deploy` works unchanged. A deploy machine needs node and nothing else:
no emscripten, no cmake, no protoc, no Rust.
All four move together, never a subset. Perspective couples loader, package
versions, data format and `apache-arrow`; vendoring a partial set reintroduces
exactly the drift that causes trouble.
## Changing the engine
./rebuild-perspective.sh # builds the fork, repacks, rewrites PROVENANCE.txt
cd .. && npm install
git add vendor && git commit
Push the fork first — the script warns if the source tree is dirty, because a
tarball built from uncommitted code has no recoverable source.
The build itself needs cmake >= 3.29.5, protoc >= 22 (its version silently
selects which protobuf source tree gets cloned), pnpm, and the Rust nightly the
repo pins. Roughly 40 minutes cold. Only ever on a machine changing the engine.
## Getting rid of this
This is a fork, with the maintenance that implies. The exit is upstream taking
the change — the patch is small and additive, and the engine work is already
theirs. When a release ships it, delete this directory and put normal version
ranges back in `ui/package.json`.

BIN
ui/vendor/perspective-dev-client-5.4.0.tgz vendored Normal file
View File

Binary file not shown.

BIN
ui/vendor/perspective-dev-server-5.4.0.tgz vendored Normal file
View File

Binary file not shown.

BIN
ui/vendor/perspective-dev-viewer-5.4.0.tgz vendored Normal file
View File

Binary file not shown.

BIN
ui/vendor/perspective-dev-viewer-datagrid-5.4.0.tgz vendored Normal file
View File

Binary file not shown.

102
ui/vendor/rebuild-perspective.sh vendored Executable file
View File

@ -0,0 +1,102 @@
#!/usr/bin/env bash
set -euo pipefail
# ---------------------------------------------------------------------------
# rebuild-perspective.sh — rebuild the patched Perspective and re-vendor it
#
# pf_app runs a patched build of Perspective that exposes the column axis
# expand/collapse the engine already implements (split_by_depth, and
# expand_column/collapse_column). Upstream does not ship this yet, so the
# built packages are vendored into this directory as npm tarballs.
#
# Source of truth: https://github.com/fleetside72/perspective
# branch column-axis-expand-collapse
#
# This script exists because vendored binaries are opaque: once the .tgz files
# are committed, nothing in the repo records how to regenerate them. Run this
# after changing the fork, then commit the resulting tarballs.
#
# Only needed on a machine that is changing the engine. Deploys just run
# `npm install`, which expands the committed tarballs - see ../README in this
# directory.
# ---------------------------------------------------------------------------
PSP="${PSP_DIR:-$HOME/perspective}"
BRANCH="${PSP_BRANCH:-column-axis-expand-collapse}"
VENDOR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# The four packages that must move together. Perspective's own docs are
# emphatic that loader, packages, data format and apache-arrow are one unit;
# vendoring a subset would reintroduce exactly the drift that causes trouble.
PACKAGES=(
"rust/perspective-js"
"rust/perspective-server"
"rust/perspective-viewer"
"packages/viewer-datagrid"
)
info() { echo -e "\033[0;34m==>\033[0m $*"; }
ok() { echo -e "\033[0;32m ✓\033[0m $*"; }
die() { echo -e "\033[0;31m ✗\033[0m $*" >&2; exit 1; }
# -- preflight --------------------------------------------------------------
[[ -d "$PSP" ]] || die "No Perspective checkout at $PSP.
git clone https://github.com/fleetside72/perspective.git $PSP
cd $PSP && git checkout $BRANCH
Set PSP_DIR to use a different path."
command -v pnpm >/dev/null || die "pnpm not found. Perspective builds with pnpm, not npm."
command -v protoc >/dev/null || die "protoc not found.
Its VERSION selects which protobuf source tree the build clones, and a
version below 22 pulls a layout the build cannot consume. Needs >= 22
(33.2 known good). Distro packages are usually far too old."
cmake_ver=$(cmake --version 2>/dev/null | head -1 | grep -oE '[0-9]+\.[0-9]+(\.[0-9]+)?') || die "cmake not found"
cmake_major=${cmake_ver%%.*}; cmake_minor=$(echo "$cmake_ver" | cut -d. -f2)
if (( cmake_major < 3 || (cmake_major == 3 && cmake_minor < 29) )); then
die "cmake $cmake_ver is too old; Perspective needs >= 3.29.5.
A user-level install works: pip3 install --user 'cmake>=3.29.5'"
fi
info "Perspective checkout: $PSP"
git -C "$PSP" rev-parse --abbrev-ref HEAD | grep -qx "$BRANCH" \
|| echo " ! on branch $(git -C "$PSP" rev-parse --abbrev-ref HEAD), expected $BRANCH"
commit=$(git -C "$PSP" rev-parse --short HEAD)
dirty=$(git -C "$PSP" status --porcelain | wc -l)
echo " commit $commit$([[ $dirty -gt 0 ]] && echo " (+$dirty uncommitted files)")"
# -- build ------------------------------------------------------------------
# `metadata` first: it generates docs/expression_gen.md, which perspective-client
# includes at compile time. Building a scope without it fails on the missing file.
info "Building (this takes ~40 minutes cold, a few minutes warm)…"
( cd "$PSP" && PSP_ONCE=1 PACKAGE="metadata,server,client,viewer,viewer-datagrid" pnpm run build )
ok "build complete"
# -- pack -------------------------------------------------------------------
info "Packing tarballs into $VENDOR"
rm -f "$VENDOR"/*.tgz
for p in "${PACKAGES[@]}"; do
( cd "$PSP/$p" && npm pack --pack-destination "$VENDOR" >/dev/null )
ok "$(basename "$p")"
done
# -- record provenance ------------------------------------------------------
# A committed .tgz is an opaque binary; without this the tie back to source is
# only in someone's memory.
cat > "$VENDOR/PROVENANCE.txt" <<EOF
Built from https://github.com/fleetside72/perspective
branch $BRANCH
commit $(git -C "$PSP" rev-parse HEAD)
based on $(git -C "$PSP" describe --tags --abbrev=0 2>/dev/null || echo 'unknown')
built $(date -u +%Y-%m-%dT%H:%M:%SZ) on $(hostname)
dirty $dirty uncommitted file(s) in the source tree at build time
Regenerate with ui/vendor/rebuild-perspective.sh
EOF
echo
ls -la "$VENDOR"/*.tgz | awk '{printf " %-52s %5.1f MB\n", $NF, $5/1048576}'
echo
ok "Done. Now: cd ui && npm install && git add vendor && git commit"
[[ $dirty -gt 0 ]] && echo -e "\033[1;33m !\033[0m source tree had uncommitted changes — push them to the fork first"
exit 0