# Perspective — lessons and rules What we have learned the hard way about configuring, feeding and upgrading Perspective, written for any app that embeds `` (**pf_app**, **dataflow**). The pf_app specifics — the vendored 5.4.0 fork, its patch, and how to rebuild it — live in `ui/vendor/README.md`; how pf_app uses the viewer day to day is in `CLAUDE.md` and `pf_spec.md` → §The pivot. > **Distribution:** we use the **`@perspective-dev/*`** packages > () — **not** FINOS/OpenJS > `@finos/perspective`. Same engine lineage, but a separate npm scope, release > cadence and "Pro" theme set (`Pro Dark` / `Pro Light`). Don't mix the scopes. --- ## Perspective is one locked unit These move together and are pinned together. Bumping one without the others is the source of nearly every Perspective bug we have hit: 1. **Loader** — how the JS/WASM gets into the page (npm `/inline`, never a CDN) 2. **Package versions** — `client`, `server`, `viewer`, `viewer-datagrid` (and `viewer-d3fc` where charts are used) 3. **Data format** — Arrow IPC vs JSON rows 4. **`apache-arrow` on the server**, when delivering Arrow — it must write an IPC format the client WASM reads **Pin `@perspective-dev/server` explicitly.** `@perspective-dev/client` declares it as `"@perspective-dev/server": ""` — an empty range, which npm resolves to `latest`. Installing an older client pulls whatever server is newest and the WASM fails to link (`Import #8 "env" "psp_opfs_load": function import requires a callable`). Five packages, not four. --- ## Load inline, never from a CDN ```js import perspective from '@perspective-dev/client/inline' import '@perspective-dev/viewer/inline' import '@perspective-dev/viewer-datagrid' import '@perspective-dev/viewer/themes' ``` The `/inline` entrypoints bundle the WASM into the Vite build: no runtime fetch, works behind a firewall, reproducible from the lockfile. The themes CSS is imported in JS for the same reason, not ``ed from `index.html`. **This is not hypothetical — a CDN load took pf_app down on 2026-08-10.** The 4.x CDN bundle resolves its server WASM with `new URL("../../../server/dist/wasm/perspective-server.wasm", import.meta.url)`, which from `.../client@4.4.0/dist/cdn/` lands on `.../npm/@perspective-dev/server/dist/wasm/perspective-server.wasm` — **no version**, so jsdelivr serves `@latest`. The moment a 5.x server published, every page load linked a 5.x WASM against a 4.4.0 client and threw the `LinkError` above. Nobody changed anything. The security framing matters as much as the outage: an unversioned URL means users execute whatever that package publishes next, unreviewed. `/inline` has no runtime fetch to hijack. --- ## Charts versus inline bundling (apps that need d3fc) pf_app uses the datagrid only and does not carry `viewer-d3fc`. For an app that does, the constraint as of 2026-08: `viewer-d3fc` published nothing past 4.4.1, while `/inline` and `/themes` exist only from 4.5.x. **A 4.5.x viewer with a 4.4.1 d3fc does not work** — every chart plugin throws `get_static_config is not a function` (reproduced in isolation, and seen in dataflow's install, where every chart type but the datagrid fails). So it is inline bundling **or** d3fc charts, not both: go coherent 4.4.1 for charts (still unverified that a bundled 4.4.1 suite renders them — check before relying on it), or 5.x without them. Re-check when d3fc publishes a newer version. --- ## Arrow delivery: three coupled constraints Arrow IPC is the right format for large, numeric, incrementally-updated tables; JSON rows are simpler and fine for small read-only ones. Delivering Arrow couples three things: - **Numeric type parsers, server-side.** pg returns `bigint` / `numeric` as strings; coerce them to numbers before encoding, or `apache-arrow` infers `Dictionary` instead of `Int` / `Float64`. pf_app: the type parsers in `server.js` (oids 20 and 1700). - **One record batch.** Per-batch encoding builds independent dictionaries, and the WASM crashes on dictionary-replacement messages. Accumulate all rows and emit a single batch — pf_app does `tableToIPC(tableFromArrays(columns), 'stream')`. The client's "stream" is therefore a chunked download of one batch; nothing renders progressively. - **`apache-arrow` versus the client WASM.** The client is built against an older arrow major than pf_app's server pins; the pair was verified working, but any arrow bump has to be gated on the ingest smoke test below. --- ## Expression columns are row-level: ratios need a weighted mean Perspective evaluates every expression against each **raw row**, then feeds the result into the column's aggregate; there is no post-aggregate expression stage. So `price: '"revenue" / "qty"'` under the default `sum` **sums the per-row ratios** — wrong at every grouped level, with or without `split_by`. `avg` gives the average of ratios, which is also wrong. Weight the ratio by its own denominator — `sum(price_i × qty_i) / sum(qty_i)` is exactly `sum(revenue) / sum(qty)`, at every level of both axes: ```js expressions: { price: '"revenue" / "qty"' }, aggregates: { price: ['weighted mean', ['qty']] } // note the NESTED array ``` **The nested array matters.** `Aggregate = string | [string, Array]`, so the weight column goes in its own array. The flat `['weighted mean', 'qty']` is rejected with `data did not match any variant of untagged enum Aggregate`, which reads like the feature is missing. Being engine-side, it stays correct across incremental `table.update()`. - A **sum of a measure** (including `if(...)` subtotal expressions) → default `sum`. - A **ratio, rate, price or per-unit figure** → must carry `['weighted mean', ['']]`. **Type inference can silently break expressions.** `if("Year" == '2026', "Amount", 0)` returns 0 on every row if `Year` was *inferred* as integer from numeric-looking strings, with no error, and neither literal form fixes it. Give period and year columns an explicit `string` type at table creation. --- ## Theming The viewer theme is a `ViewerConfig` field: `viewer.restore({ theme: 'Pro Dark' })`. **`setAttribute('theme', …)` does nothing in 5.x** — the element observes no such attribute. Apply it on load and whenever the app toggle changes. `viewer.save()` includes `theme`, so strip it from stored layouts and ignore it when comparing configs — otherwise restoring a layout reasserts its author's theme. --- ## Layouts: guard every restore against schema drift Before `viewer.restore(cfg)`, filter the saved `columns` / `group_by` / `split_by` / `sort` / `filter` against the columns the current table actually has (plus its `expressions`); a stale layout naming a dropped column otherwise throws. **`aggregates` needs the same guard** — once a layout carries a weighted mean, a dropped column aborts the whole restore (`Could not get dtype for column 'X'`). An aggregate names a target column and, in the multi-arg form, a weight column; validate both and drop the *entry*, never the layout, since `restore()` is all-or-nothing. pf_app's `cleanLayout()` does all of this; dataflow's does not yet guard `aggregates`. --- ## Auto-pause deletes the view — turn it off for a static pivot `` auto-pauses by default (an `IntersectionObserver` on itself plus the document's `visibilitychange`). Pausing is not a paint optimisation: `session.set_pause(true)` runs `view_sub.take().delete()`, so the view is destroyed, and becoming visible again rebuilds it with a full traversal — a multi-second stall per tab switch on a large table, and any per-node expand/collapse is lost with it. ```js await viewer.load(table) try { if (viewer.setAutoPause) await viewer.setAutoPause(false) } catch {} ``` Leave it on only where a live stream feeds a table nobody needs kept current while away. Long-standing viewer behaviour, not a regression. --- ## Upgrade smoke test Run on any change to a Perspective package or `apache-arrow` (for pf_app, after rebuilding the vendored tarballs per `ui/vendor/README.md`): 1. All five packages (and `apache-arrow`) pinned exact and matching; lockfile committed. 2. `vite build` — no unresolved imports. 3. Arrow apps: load a real dataset; `worker.table(buffer)` ingests without a dictionary error, and a numeric column comes through as `Float64` / `Int`. 4. Charts, where used: open a d3fc plugin and confirm it renders. 5. Toggle dark/light; the viewer re-themes. 6. Save a layout, reload, confirm it restores; drop a column and confirm the cleaned restore does not throw.