Add PERSPECTIVE.md config/deploy reference; fix CLAUDE.md distribution link
Document the @perspective-dev distribution (not FINOS @finos/perspective): loader (npm /inline vs CDN), the version trilemma (inline needs 4.5.x, viewer-d3fc caps at 4.4.1, charts need 4.4.1 — can't have all three), Arrow vs JSON delivery constraints, deploy pattern, and an upgrade smoke test. Correct CLAUDE.md's stale perspective.finos.org link to the actual @perspective-dev repo. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
parent
e425c32134
commit
64f3cc58e8
@ -15,7 +15,7 @@ UX mockup: `pf_ux_mockup.md`
|
|||||||
- **Backend:** Node.js / Express (`server.js`)
|
- **Backend:** Node.js / Express (`server.js`)
|
||||||
- **Database:** PostgreSQL — isolated `pf` schema
|
- **Database:** PostgreSQL — isolated `pf` schema
|
||||||
- **Frontend:** React + Vite + Tailwind CSS in `ui/`; built output lands in `public/app/`
|
- **Frontend:** React + Vite + Tailwind CSS in `ui/`; built output lands in `public/app/`
|
||||||
- **Pivot:** [Perspective](https://perspective.finos.org/) 4.4.0 loaded from CDN at runtime
|
- **Pivot:** [Perspective](https://github.com/perspective-dev/perspective) (`@perspective-dev/*` distribution, **not** FINOS `@finos/perspective`) 4.4.0 loaded from CDN at runtime — see `PERSPECTIVE.md` for config/deploy guidance
|
||||||
- **Dev:** `npm run dev` (nodemon) in root; `npm run build` in `ui/`
|
- **Dev:** `npm run dev` (nodemon) in root; `npm run build` in `ui/`
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
199
PERSPECTIVE.md
Normal file
199
PERSPECTIVE.md
Normal file
@ -0,0 +1,199 @@
|
|||||||
|
# Perspective — configuration & deployment reference
|
||||||
|
|
||||||
|
Canonical guide for how Perspective should be configured, fed, and shipped across our
|
||||||
|
apps (**pf_app**, **dataflow**). Both embed the same `<perspective-viewer>` web component
|
||||||
|
but made different early choices; this doc defines the target state and the rationale, so
|
||||||
|
the two converge instead of drifting.
|
||||||
|
|
||||||
|
> **Distribution note:** we use the **`@perspective-dev/*`** packages
|
||||||
|
> (repo: <https://github.com/perspective-dev/perspective>, home:
|
||||||
|
> <https://perspective-dev.github.io>) — **not** the FINOS/OpenJS `@finos/perspective`
|
||||||
|
> packages from <https://perspective.finos.org/>. Same engine lineage, but a separate
|
||||||
|
> npm scope, release cadence, and "Pro" theme set (`Pro Dark`/`Pro Light`). This is why
|
||||||
|
> `viewer-d3fc` versions on a different schedule than `viewer`/`client` (§2), and why the
|
||||||
|
> client's bundled `apache-arrow` (17.x) can lag the server's (§3). Don't mix the two
|
||||||
|
> scopes.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Core principle: Perspective is one locked unit
|
||||||
|
|
||||||
|
These four things must move together and be pinned together. Bumping one without the
|
||||||
|
others is the source of nearly every Perspective bug we've hit:
|
||||||
|
|
||||||
|
1. **Loader** — how the JS/WASM gets into the page (npm-inline vs CDN)
|
||||||
|
2. **Package version** — `client` / `viewer` / `viewer-datagrid` / `viewer-d3fc`
|
||||||
|
3. **Data format** — Arrow IPC vs JSON rows
|
||||||
|
4. **`apache-arrow` version** (server-side, only if using Arrow) — must speak an IPC
|
||||||
|
format the client WASM understands
|
||||||
|
|
||||||
|
Treat a Perspective upgrade as a coordinated change to all four, gated by the smoke test
|
||||||
|
at the bottom of this doc. Never let viewer/client drift ahead of `viewer-d3fc`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. Loading — npm `/inline`, pinned exact (not CDN)
|
||||||
|
|
||||||
|
```js
|
||||||
|
import perspective from '@perspective-dev/client/inline'
|
||||||
|
import '@perspective-dev/viewer/inline'
|
||||||
|
import '@perspective-dev/viewer-datagrid'
|
||||||
|
import '@perspective-dev/viewer-d3fc'
|
||||||
|
import '@perspective-dev/viewer/themes'
|
||||||
|
```
|
||||||
|
|
||||||
|
- The `/inline` entrypoints bundle the WASM into the Vite build — **no runtime network
|
||||||
|
fetch, works offline / behind a firewall, reproducible from the lockfile.**
|
||||||
|
- **Do not load from a CDN at runtime.** It's convenient for a prototype (smaller build,
|
||||||
|
one-line version bumps) but in production it means: app breaks if the CDN is
|
||||||
|
unreachable, version isn't captured in `package-lock.json`, slower cold start, and you
|
||||||
|
pull executable WASM from a third party on every load. (pf_app currently does this in
|
||||||
|
`ui/src/views/Forecast.jsx` — migrating off it is the main open item.)
|
||||||
|
- The themes CSS is imported in JS (`@perspective-dev/viewer/themes`), **not** via a
|
||||||
|
`<link>` in `index.html` — so it's bundled and versioned too.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. Version policy — a real trilemma (read carefully)
|
||||||
|
|
||||||
|
The version choice is constrained by two hard facts about the `@perspective-dev` packages
|
||||||
|
**(verified against installed metadata, 2026-06)**:
|
||||||
|
|
||||||
|
- **`viewer-d3fc` caps at 4.4.1** — npm publishes no 4.5.x. The d3fc charts (Bar / Line /
|
||||||
|
Treemap / Heatmap / etc.) live only in this package.
|
||||||
|
- **The `/inline` and `/themes` entrypoints are 4.5.x-only** — `@perspective-dev/client/inline`,
|
||||||
|
`@perspective-dev/viewer/inline`, and `@perspective-dev/viewer/themes` do **not** exist
|
||||||
|
in 4.4.1's `exports` map. Bundling inline WASM requires 4.5.x.
|
||||||
|
|
||||||
|
So you can have at most **two** of these three:
|
||||||
|
|
||||||
|
| Want | Requires |
|
||||||
|
|---|---|
|
||||||
|
| Inline WASM bundling (`/inline`, `/themes`) | **4.5.x** viewer/client |
|
||||||
|
| One coherent single-version suite | **4.4.1** everything (d3fc ceiling) |
|
||||||
|
| d3fc chart plugins | **4.4.1** viewer-d3fc |
|
||||||
|
|
||||||
|
There is **no** version where all three hold. Pick by what the app needs:
|
||||||
|
|
||||||
|
- **Inline-bundled + charts** (dataflow's case) → `^4.5.1` viewer/client/datagrid **+
|
||||||
|
`^4.4.1` viewer-d3fc`. This is a deliberate, necessary mixed-version pair, *not* an
|
||||||
|
accident — it's the only combo that keeps both. Accept it; pin the lockfile and gate
|
||||||
|
bumps on the smoke test (§7). Do **not** "fix" it by pinning everything to 4.4.1 — the
|
||||||
|
build breaks (`"./inline" is not exported`).
|
||||||
|
- **Coherent single suite, no inline** (e.g. CDN or `.`-entry loading) → pin all four to
|
||||||
|
**4.4.1 exact**. Charts work; you give up `/inline` bundling.
|
||||||
|
|
||||||
|
Whatever you pick, **commit the lockfile** so the resolved set can't drift on
|
||||||
|
`npm install`. Re-evaluate the whole policy only when `viewer-d3fc` ships a 4.5.x (then a
|
||||||
|
fully-coherent inline-capable 4.5.x suite becomes possible).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. Data delivery — match the format to the workload
|
||||||
|
|
||||||
|
| Workload | Format | Why |
|
||||||
|
|---|---|---|
|
||||||
|
| Large (100k+ rows), numeric-heavy, writes/incremental updates | **Arrow IPC** | Compact columnar binary, near-zero-copy ingest, carries types (no string coercion). Powers pf_app's 500k-row path. |
|
||||||
|
| Small (≤100k), read-only, click-to-inspect | **JSON rows** | Simpler, no encoding step, no dictionary pitfalls. dataflow's model. |
|
||||||
|
|
||||||
|
### Arrow constraints (read before choosing it)
|
||||||
|
|
||||||
|
If you deliver Arrow, three pieces are coupled and must stay aligned:
|
||||||
|
|
||||||
|
- **Numeric type parsers, server-side.** pg returns `bigint`/`numeric` as *strings*; you
|
||||||
|
must coerce them to JS numbers before encoding, or `apache-arrow` infers
|
||||||
|
`Dictionary<Utf8>` instead of `Int`/`Float64`. See `server.js` type parsers (oid 20,
|
||||||
|
1700).
|
||||||
|
- **Single record batch.** Per-batch Arrow builds independent dictionaries; the
|
||||||
|
Perspective WASM crashes on dictionary-replacement messages. The server must
|
||||||
|
accumulate all rows and emit **one** batch (`tableToIPC(tableFromJSON(allRows),
|
||||||
|
'stream')`). Consequence: the client "stream" is just a chunked *download* of one
|
||||||
|
batch — nothing renders progressively, and the server holds the full result set in
|
||||||
|
memory per request.
|
||||||
|
- **`apache-arrow` pinned to match the client WASM.** Pin it exact in the server
|
||||||
|
`package.json` and treat it as part of the locked unit (principle above). Note the
|
||||||
|
`@perspective-dev/client` build is tested against **`apache-arrow@17.0.0`**, while
|
||||||
|
pf_app's server currently pins **`^21.1.0`** — a real IPC version gap. Prefer aligning
|
||||||
|
the server toward the arrow major the client was built against, or at minimum make
|
||||||
|
Arrow ingestion (smoke test §7) the gate on any arrow bump.
|
||||||
|
|
||||||
|
JSON avoids all three but pays in payload size and parse cost, and pushes type handling
|
||||||
|
to client-side heuristics — acceptable only at small scale.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. Theming
|
||||||
|
|
||||||
|
- One toggle drives both app CSS and the viewer:
|
||||||
|
`viewer.setAttribute('theme', dark ? 'Pro Dark' : 'Pro Light')`.
|
||||||
|
- Apply it on initial load **and** in an effect keyed on the dark flag, so the viewer
|
||||||
|
re-themes when the toggle fires (not just on mount).
|
||||||
|
- App-level dark mode is plain CSS custom properties + a `.dark` class on `<html>`; the
|
||||||
|
viewer theme name is the only Perspective-specific piece.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. Layout persistence
|
||||||
|
|
||||||
|
- Persist the full viewer config (`await viewer.save()`, incl. `plugin_config`) to
|
||||||
|
`localStorage`, keyed per source/version. Restore with `viewer.restore(cfg)`.
|
||||||
|
- **Guard restores against schema drift.** Before restoring, filter the saved config's
|
||||||
|
`columns`/`group_by`/`split_by`/`sort`/`filter` against the columns that actually
|
||||||
|
exist in the current dataset (plus any `expressions`). dataflow's `cleanLayout()` is
|
||||||
|
the reference implementation; a stale layout referencing a dropped column otherwise
|
||||||
|
throws on restore.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. Build & deploy (target)
|
||||||
|
|
||||||
|
- **Build:** `vite build` emits a static bundle (pf_app → `public/app`, via
|
||||||
|
`outDir: '../public/app'`). The API server serves it statically (`express.static`).
|
||||||
|
- **Process:** run the Node API under **systemd** with `Restart=always` and an
|
||||||
|
`EnvironmentFile=.env`; front it with **nginx** (reverse proxy + TLS via certbot).
|
||||||
|
dataflow's `dataflow.service` + `deploy.sh` are the reference; **pf_app has no deploy
|
||||||
|
automation yet** and should adopt the same pattern.
|
||||||
|
- **`deploy.sh`** should be idempotent: first run installs (db/schema, UI build, nginx,
|
||||||
|
systemd unit); later runs rebuild the UI and restart the service.
|
||||||
|
- Keep secrets in `.env` (db creds), loaded by both the app (`dotenv`) and the systemd
|
||||||
|
unit — never commit it.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. Upgrade checklist / smoke test
|
||||||
|
|
||||||
|
Run this whenever bumping **any** Perspective package or `apache-arrow`:
|
||||||
|
|
||||||
|
1. Confirm `viewer-d3fc` publishes the target version
|
||||||
|
(`npm view @perspective-dev/viewer-d3fc versions`). If not, **don't bump** the others.
|
||||||
|
2. Pin all four packages + `apache-arrow` to exact, matching versions; `npm install`;
|
||||||
|
commit the lockfile.
|
||||||
|
3. `vite build` — no unresolved imports.
|
||||||
|
4. **Arrow apps:** load a real dataset and confirm `worker.table(buffer)` ingests
|
||||||
|
without a WASM dictionary error; verify a numeric column is `Float64`/`Int`, not a
|
||||||
|
string/dictionary.
|
||||||
|
5. Open a d3fc **chart** plugin (not just datagrid) and confirm it renders.
|
||||||
|
6. Toggle dark/light; confirm the viewer re-themes.
|
||||||
|
7. Save a layout, reload, confirm it restores; then drop a column and confirm the
|
||||||
|
cleaned restore doesn't throw.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Per-project state (2026-06)
|
||||||
|
|
||||||
|
| | pf_app | dataflow | Target |
|
||||||
|
|---|---|---|---|
|
||||||
|
| Loader | CDN (runtime) | npm `/inline` | **npm `/inline`** |
|
||||||
|
| Version | 4.4.0 (CDN URLs) | 4.5.1 viewer/client + 4.4.1 d3fc | depends on loader (§2) |
|
||||||
|
| Data | Arrow IPC (single batch) | JSON (≤100k) | per workload (§3) |
|
||||||
|
| `apache-arrow` | `^21.1.0` (client built vs 17) | n/a | pin exact, match WASM |
|
||||||
|
| Deploy | none | systemd + nginx + `deploy.sh` | **systemd + nginx + `deploy.sh`** |
|
||||||
|
|
||||||
|
**dataflow's 4.5.1/4.4.1 pair is correct** — it's the only combo giving both inline
|
||||||
|
bundling and d3fc charts (§2). Leave it; just keep the lockfile committed.
|
||||||
|
|
||||||
|
**Open items:**
|
||||||
|
- pf_app → move off CDN. Note this forces the §2 choice: going npm-`/inline` means
|
||||||
|
4.5.x viewer/client + 4.4.1 d3fc (same pair as dataflow); or stay coherent at 4.4.x and
|
||||||
|
load via the `.` entry instead of `/inline`. Either way, pin + commit the lockfile, and
|
||||||
|
add deploy automation (systemd + nginx + `deploy.sh`).
|
||||||
Loading…
Reference in New Issue
Block a user