diff --git a/SPEC.md b/SPEC.md index f5d4f1b..67d2920 100644 --- a/SPEC.md +++ b/SPEC.md @@ -218,7 +218,24 @@ Built with React + Vite + Tailwind CSS. Compiled output goes to `public/`. The s - **Records** — Paginated table showing the `dfv.{source}` view. Server-side sorting (column validated against `information_schema.columns`, interpolated with `quote_ident`). Dates are formatted `YYYY-MM-DD` for correct lexicographic sort. Regex filters can be added per column. If the view cast fails (e.g. a field typed as `date` contains text), the error is shown inline rather than a blank page. -- **Pivot** — Interactive pivot/crosstab powered by [Perspective](https://perspective.finos.org/) (loaded from CDN at runtime). Loads all rows from the source view into an in-browser Perspective worker and renders a `` web component. Supports grouping, splitting, filtering, sorting, and charting interactively. Layout (group_by, split_by, filters, plugin) is saved per source to `localStorage` and can be reset. Clicking a cell opens an inspector panel on the right showing the exact cell coordinates (group_by › split_by values), the clicked metric value, any active user filters, and the underlying raw rows that contribute to that cell. Row matching applies the `perspective-click` event's filter array directly against the in-memory rows, skipping filters for Perspective-computed columns (e.g. derived date parts like Month or YearDate) that don't exist in the source data. +- **Pivot** — Interactive pivot/crosstab powered by [Perspective](https://perspective.finos.org/) (`@perspective-dev` v4.4.0, loaded from CDN at runtime). Loads all rows from the source view into an in-browser Perspective worker and renders a `` web component. Supports grouping, splitting, filtering, sorting, and charting interactively. + + **Toolbar (above the viewer):** + - Named layouts — saved per source in the `pivot_layouts` DB table. Each chip recalls the full viewer state including group_by, split_by, filters, expressions, selection mode, and expand depth. A blue **Save** button overwrites the active layout in place; **+ Save as…** saves to a new name. The × on each chip deletes it. + - **depth: 0 1 2 3** — collapses or expands all grouped rows to the specified hierarchy level. Implemented via `view.set_depth(d)` + `plugin.draw(view)` (the only working mechanism found in v4.4.0 — `plugin_config.expand_depth` and `viewer.flush()` alone have no effect). + - The Perspective built-in **selection mode button** (Read-Only / Select Row / Select Column / Select Region) defaults to **Select Region** on fresh load, set directly via `plugin.restore({ edit_mode: 'SELECT_REGION' })` after the viewer loads. + + **Cell inspector (right panel):** + - Opens when a cell is clicked and a `group_by` hierarchy is active. If there is no `group_by`, the click is ignored — without coordinate filters the query would return the full dataset. + - Row filtering uses a temporary Perspective view (`table.view({ filter: eventFilters, expressions: config.expressions })`) so that computed/expression columns in `split_by` are evaluated correctly. Falls back to JS-side filtering if the view query fails. + - Shows cell coordinates (group_by › split_by values), the clicked metric with value, any user-set filters, and a table of matching raw rows. + - Number formatting rounds to 2 decimal places by default; a −/+ control in the inspector header adjusts precision (0–8). + + **Layout persistence:** + - `localStorage` key `psp_layout_{source}` saves the last viewer state on each named layout save. + - Named layouts store `{ ...viewer.save(), plugin_config: plugin.save(), expand_depth }` as JSONB in `pivot_layouts`. On recall, viewer config, plugin config (edit mode), and expand depth are all restored independently. + + See `docs/perspective-pivot.md` for the full technical reference on controlling Perspective programmatically. - **Log** — Global import log across all sources. Same expandable key detail and delete capability as the Import page, plus a source name column. diff --git a/docs/perspective-pivot.md b/docs/perspective-pivot.md new file mode 100644 index 0000000..013408b --- /dev/null +++ b/docs/perspective-pivot.md @@ -0,0 +1,285 @@ +# Perspective Pivot — Technical Reference + +Version tested: `@perspective-dev` v4.4.0 (client, viewer, viewer-datagrid, viewer-d3fc), loaded from CDN. + +This document captures everything learned about controlling Perspective programmatically. The official docs are incomplete for some of these APIs — treat this as a ground-truth supplement. + +--- + +## Loading from CDN + +```js +const [{ default: perspective }] = await Promise.all([ + import('https://cdn.jsdelivr.net/npm/@perspective-dev/client@4.4.0/dist/cdn/perspective.js'), + import('https://cdn.jsdelivr.net/npm/@perspective-dev/viewer@4.4.0/dist/cdn/perspective-viewer.js'), + import('https://cdn.jsdelivr.net/npm/@perspective-dev/viewer-datagrid@4.4.0/dist/cdn/perspective-viewer-datagrid.js'), + import('https://cdn.jsdelivr.net/npm/@perspective-dev/viewer-d3fc@4.4.0/dist/cdn/perspective-viewer-d3fc.js'), +]) +``` + +Stylesheet: +```html + +``` + +--- + +## Core Objects + +``` +perspective — the module default export + .worker() — creates a Web Worker instance + +worker + .table(rows, opts) — creates a named Table; returns the Table object + .open_table(name) — re-opens a previously created named table + +table + .view(config) — creates a View (filtered/grouped projection) + .update(rows) — incremental row upsert/insert + +view + .to_json() — returns rows as array of objects + .set_depth(n) — sets expansion depth for all grouped rows (see below) + .delete() — frees the view; always call when done + +viewer (the DOM element) + .load(worker) — attaches the worker to the viewer + .save() — returns full viewer config as plain object + .restore(config) — applies a config object to the viewer + .flush() — forces viewer to synchronize (limited effect on plugin state) + .getPlugin() — returns the active plugin element (e.g. datagrid) + .getView() — returns the current View object + .toggleConfig() — shows/hides the settings panel + +plugin (datagrid element, from viewer.getPlugin()) + .save() — returns plugin-specific state: { columns, scroll_lock, edit_mode } + .restore(config) — applies plugin-specific state + .draw(view) — redraws the plugin against the given View +``` + +--- + +## viewer.save() — Config Shape + +```js +{ + table: "source_name", + plugin: "datagrid", // or "d3_y_bar", etc. + plugin_config: { ... }, // NOT reliably populated — use plugin.save() instead + group_by: ["field1"], + split_by: ["field2"], + columns: ["Amount"], + filter: [["field", "op", "value"]], + sort: [["field", "asc"]], + expressions: { "ExprName": "// formula\n..." }, + settings: false, // whether the config panel is open +} +``` + +**Important:** `plugin_config` in `viewer.save()` is NOT reliably populated in v4.4.0. Use `plugin.save()` separately to capture plugin state. + +--- + +## plugin.save() — Plugin State Shape (datagrid) + +```js +{ + columns: {}, // per-column formatting overrides + scroll_lock: false, + edit_mode: "SELECT_REGION" // see valid values below +} +``` + +--- + +## Selection Modes (edit_mode) + +Valid values for the datagrid plugin's `edit_mode` field: + +| Value | Button label | Behavior | +|---|---|---| +| `READ_ONLY` | Read-Only | No selection highlight | +| `SELECT_ROW` | Select Row | Highlights full rows | +| `SELECT_COLUMN` | Select Column | Highlights full columns | +| `SELECT_REGION` | Select Region | Highlights clicked cell region | +| `EDIT` | Edit | Enables cell editing | + +The built-in button in the viewer toolbar cycles through these in order. + +**Setting the default:** +```js +// After viewer.restore(...), set it directly on the plugin: +const plugin = await viewer.getPlugin() +await plugin.restore({ edit_mode: 'SELECT_REGION' }) +``` + +Setting via `viewer.restore({ plugin_config: { edit_mode: ... } })` does NOT reliably work in v4.4.0. + +--- + +## Expand/Collapse Row Depth + +Controls how many levels of the `group_by` hierarchy are expanded. This is the only working mechanism found in v4.4.0: + +```js +const view = await viewer.getView() +await view.set_depth(depth) // 0 = collapse all, 1 = expand one level, etc. +const plugin = await viewer.getPlugin() +await plugin.draw(view) // required — viewer does not redraw automatically +``` + +**What does NOT work:** +- `viewer.restore({ plugin_config: { expand_depth: d } })` — silently ignored +- `view.set_depth(d)` alone — view state changes but display doesn't update +- `view.set_depth(d)` + `viewer.flush()` — still no visual update +- `plugin.restore({ expand_depth: d })` — "Unknown" field, ignored + +**The `plugin.draw(view)` call is required** to make the datagrid re-render after `set_depth`. + +--- + +## Saving and Restoring Full State + +To capture complete state (viewer + plugin + expand depth): + +```js +async function captureConfig(viewer, expandDepth) { + const plugin = await viewer.getPlugin() + const [viewerConfig, pluginConfig] = await Promise.all([viewer.save(), plugin.save()]) + return { ...viewerConfig, plugin_config: pluginConfig, expand_depth: expandDepth } +} +``` + +To restore: +```js +async function restoreConfig(viewer, config, applyDepth) { + await viewer.restore(config) + if (config.plugin_config) { + const plugin = await viewer.getPlugin() + await plugin.restore(config.plugin_config) + } + if (config.expand_depth != null) { + await applyDepth(viewer, config.expand_depth) + } + await viewer.flush() +} + +async function applyDepth(viewer, depth) { + const view = await viewer.getView() + await view.set_depth(depth) + const plugin = await viewer.getPlugin() + await plugin.draw(view) +} +``` + +--- + +## The perspective-click Event + +Fires when the user clicks a cell. The event detail: + +```js +viewer.addEventListener('perspective-click', async (e) => { + const { row, column_names, config } = e.detail + // row — aggregated values for the clicked cell (keyed by "split|metric" format) + // column_names — array of metric column names clicked + // config — { filter: [[field, op, value], ...] } + // filter includes: + // - group_by coordinate filters (field == value, one per group_by level) + // - split_by coordinate filters (field == value, one per split_by field) + // - user-set filters (any op) +}) +``` + +`__ROW_PATH__` in `row` contains the group_by path as an array. + +**The `config.filter` array is the reliable way to get cell coordinates.** Do not try to zip `__ROW_PATH__` with `group_by` — the filter approach handles all cases including partial paths. + +--- + +## Filtering Rows for a Clicked Cell + +The click event's `filter` array can be applied to the underlying table via a new View, which correctly evaluates expression/computed columns (unlike filtering raw JS rows): + +```js +const config = await viewer.save() +const view = await table.view({ + filter: eventFilters, + expressions: config.expressions || [], +}) +const rows = await view.to_json() +await view.delete() + +// Strip expression columns from results (they're computed, not source fields) +const exprNames = new Set(Object.keys(config.expressions || {})) +const clean = rows.map(r => + Object.fromEntries(Object.entries(r).filter(([k]) => !exprNames.has(k))) +) +``` + +**Why not filter raw JS rows?** Expression columns (computed in Perspective) don't exist in the source data. `filterRowsByConfig` on raw rows will skip those filters, returning all rows for the group rather than the specific cell. + +**Guard against no group_by:** Without `group_by`, the filter array has no coordinate filters and the view query returns the entire table (slow). Check first: + +```js +const config = await viewer.save() +if ((config.group_by || []).length === 0) return // no hierarchy — skip inspector +``` + +--- + +## Viewer Methods (full list, v4.4.0) + +From `Object.getOwnPropertyNames(Object.getPrototypeOf(viewer))`: + +`constructor`, `__destroy_into_raw`, `free`, `__get_model`, `connectedCallback`, `copy`, `delete`, `download`, `eject`, `export`, `flush`, `getAllPlugins`, `getClient`, `getEditPort`, `getPlugin`, `getRenderStats`, `getSelection`, `getTable`, `getView`, `getViewConfig`, `load`, `openColumnSettings`, `reset`, `resetError`, `resetThemes`, `resize`, `restore`, `restyleElement`, `save`, `setAutoPause`, `setAutoSize`, `setSelection`, `setThrottle`, `toggleColumnSettings`, `toggleConfig` + +## Plugin Methods (datagrid, full list) + +From `Object.getOwnPropertyNames(Object.getPrototypeOf(plugin))`: + +`constructor`, `connectedCallback`, `disconnectedCallback`, `activate`, `name`, `category`, `select_mode`, `min_config_columns`, `config_column_names`, `group_rollups`, `priority`, `can_render_column_styles`, `column_style_controls`, `draw`, `update`, `render`, `resize`, `clear`, `save`, `restore`, `restyle`, `delete` + +## View Methods (full list) + +From `Object.getOwnPropertyNames(Object.getPrototypeOf(view))` — includes `set_depth`, `expand`, `collapse`, `to_json`, `to_csv`, `to_arrow`, `schema`, `num_rows`, `num_columns`, `delete`, and others. + +--- + +## settings Panel + +The `settings` key in `viewer.restore()` controls whether the config panel (gear icon) is open: + +```js +// Hide on load: +await viewer.restore({ table: "name", settings: false, plugin_config: DEFAULT_PLUGIN_CONFIG }) + +// Toggle programmatically: +viewer.toggleConfig() +``` + +The settings state is saved by `viewer.save()` and restored on `viewer.restore()`, so it persists across layout saves automatically. + +--- + +## Incremental Updates + +To update the table data without a full reload: + +```js +table.update(newRows) // upserts by index (or by index_col if specified at table creation) +``` + +The viewer re-renders automatically after `table.update()`. + +--- + +## Common Pitfalls + +- **`plugin_config` in `viewer.restore()` is unreliable.** Always set plugin state via `plugin.restore()` separately after `viewer.restore()`. +- **`view.set_depth()` requires `plugin.draw(view)`.** The viewer won't redraw automatically. +- **Expression columns don't exist in raw data.** Filter via a Perspective View (`table.view({ filter, expressions })`), not against raw JS rows. +- **Always `await view.delete()`** after using a temporary view, or you'll leak worker memory. +- **Named tables:** `worker.table(rows, { name: 'foo' })` — the name is used by the viewer's `table` config key. Re-open with `worker.open_table('foo')`.