dataflow/docs/perspective.md

Perspective

Everything about the Perspective pivot in dataflow: which packages and versions are pinned and why, and a ground-truth reference for the parts of the API the official docs don't cover.

Shared rationale across projects lives in the canonical guide at /home/pt/pf_app/PERSPECTIVE.md (loading, version policy, Arrow constraints, deploy pattern, upgrade smoke test). This file records what's specific to dataflow.

---

Distribution: these are the @perspective-dev/* packages (repo github.com/perspective-dev/perspective), not FINOS @finos/perspective. Same engine, separate npm scope and release schedule — don't mix the two.

---

Current state

• Loader: npm /inline (ui/src/pages/Pivot.jsx) — bundled WASM, offline-capable. ✅ This is the target loader; pf_app should adopt it.
• Data: JSON rows via api.getViewData(source, 100000, 0), capped at 100k. ✅ Correct for dataflow's read-only, click-to-inspect model. No need to move to Arrow unless view sizes grow well past 100k.
• Deploy: manage.py + dataflow.service (systemd) + nginx. ✅ Reference pattern for the org; pf_app should copy it.
• Charts: viewer-d3fc is imported, so the chart plugins are available in the UI. Default plugin config is datagrid-only ({ edit_mode: 'SELECT_REGION' }).
• Layout safety: cleanLayout() filters saved configs against valid columns before restore — the reference implementation; keep it.

The version pair is correct — do NOT "fix" it to 4.4.1

ui/package.json pins viewer/client/datagrid at ^4.5.1 and viewer-d3fc at ^4.4.1. This looks like a skew but is deliberate and necessary — it's the only combination that keeps both of dataflow's hard requirements:

• Inline WASM bundling. Pivot.jsx imports @perspective-dev/client/inline, @perspective-dev/viewer/inline, and @perspective-dev/viewer/themes. Those export paths exist only in 4.5.x — they are absent from 4.4.1's exports map.
• d3fc chart plugins. viewer-d3fc is published only up to 4.4.1.

Verified the hard way: pinning all four to 4.4.1 and rebuilding fails with "./inline" is not exported … from @perspective-dev/client. So the 4.5.1/4.4.1 pair stays. Don't touch it.

What to actually do:

• Keep the versions as-is; commit package-lock.json so the resolved set can't drift on npm install. (Optionally tighten the carets to exact 4.5.1/4.4.1 to make that explicit.)
• Treat any Perspective bump as gated by the canonical smoke test (§7): a d3fc chart renders, dark/light re-themes, and save→reload→drop-column layout restore.
• Revisit only when viewer-d3fc ships a 4.5.x — then a fully-coherent inline-capable 4.5.x suite becomes possible and the pair can collapse to one version.

---

API Reference

Packages: @perspective-dev client/viewer/viewer-datagrid at v4.5.1, viewer-d3fc at v4.4.1 — installed via npm. API notes that reference v4.4.0 behaviour have not been re-verified at 4.5.1 but are believed to still apply. The official docs are incomplete for some of these APIs — treat this as a ground-truth supplement.

Loading via npm

import perspective from '@perspective-dev/client/inline'
import '@perspective-dev/viewer/inline'
import '@perspective-dev/viewer-datagrid'
import '@perspective-dev/viewer-d3fc'
import '@perspective-dev/viewer/themes'

The inline builds embed WebAssembly directly into the JS bundle — no separate .wasm file to serve. viewer-datagrid and viewer-d3fc have no inline variant; they import normally. viewer-d3fc is currently at v4.4.1 (no v4.5.x release yet); its chart plugins register but may not appear in the viewer due to an API change in v4.5.x's registerPlugin.

---

Core Objects

perspective            — the module default export
  .worker()            — creates a Web Worker instance

worker
  .table(rows, opts)   — creates a named Table; returns the Table object
  .open_table(name)    — re-opens a previously created named table

table
  .view(config)        — creates a View (filtered/grouped projection)
  .update(rows)        — incremental row upsert/insert

view
  .to_json()           — returns rows as array of objects
  .set_depth(n)        — sets expansion depth for all grouped rows (see below)
  .delete()            — frees the view; always call when done

viewer (the <perspective-viewer> DOM element)
  .load(worker)        — attaches the worker to the viewer
  .save()              — returns full viewer config as plain object
  .restore(config)     — applies a config object to the viewer
  .flush()             — forces viewer to synchronize (limited effect on plugin state)
  .getPlugin()         — returns the active plugin element (e.g. datagrid)
  .getView()           — returns the current View object
  .toggleConfig()      — shows/hides the settings panel

plugin (datagrid element, from viewer.getPlugin())
  .save()              — returns plugin-specific state: { columns, scroll_lock, edit_mode }
  .restore(config)     — applies plugin-specific state
  .draw(view)          — redraws the plugin against the given View

---

viewer.save() — Config Shape

{
  table: "source_name",
  plugin: "datagrid",           // or "d3_y_bar", etc.
  plugin_config: { ... },       // NOT reliably populated — use plugin.save() instead
  group_by: ["field1"],
  split_by: ["field2"],
  columns: ["Amount"],
  filter: [["field", "op", "value"]],
  sort: [["field", "asc"]],
  expressions: { "ExprName": "// formula\n..." },
  settings: false,              // whether the config panel is open
}

Important: plugin_config in viewer.save() is NOT reliably populated in v4.4.0. Use plugin.save() separately to capture plugin state.

---

plugin.save() — Plugin State Shape (datagrid)

{
  columns: {},          // per-column formatting overrides
  scroll_lock: false,
  edit_mode: "SELECT_REGION"   // see valid values below
}

---

Selection Modes (edit_mode)

Valid values for the datagrid plugin's edit_mode field:

Value	Button label	Behavior
READ_ONLY	Read-Only	No selection highlight
SELECT_ROW	Select Row	Highlights full rows
SELECT_COLUMN	Select Column	Highlights full columns
SELECT_REGION	Select Region	Highlights clicked cell region
EDIT	Edit	Enables cell editing

The built-in button in the viewer toolbar cycles through these in order.

Setting the default:

// After viewer.restore(...), set it directly on the plugin:
const plugin = await viewer.getPlugin()
await plugin.restore({ edit_mode: 'SELECT_REGION' })

Setting via viewer.restore({ plugin_config: { edit_mode: ... } }) does NOT reliably work in v4.4.0.

---

Expand/Collapse Row Depth

Controls how many levels of the group_by hierarchy are expanded. This is the only working mechanism found in v4.4.0:

const view = await viewer.getView()
await view.set_depth(depth)       // 0 = collapse all, 1 = expand one level, etc.
const plugin = await viewer.getPlugin()
await plugin.draw(view)           // required — viewer does not redraw automatically

What does NOT work:

• viewer.restore({ plugin_config: { expand_depth: d } }) — silently ignored
• view.set_depth(d) alone — view state changes but display doesn't update
• view.set_depth(d) + viewer.flush() — still no visual update
• plugin.restore({ expand_depth: d }) — "Unknown" field, ignored

The plugin.draw(view) call is required to make the datagrid re-render after set_depth.

---

Saving and Restoring Full State

To capture complete state (viewer + plugin + expand depth):

async function captureConfig(viewer, expandDepth) {
  const plugin = await viewer.getPlugin()
  const [viewerConfig, pluginConfig] = await Promise.all([viewer.save(), plugin.save()])
  return { ...viewerConfig, plugin_config: pluginConfig, expand_depth: expandDepth }
}

To restore:

async function restoreConfig(viewer, config, applyDepth) {
  await viewer.restore(config)
  if (config.plugin_config) {
    const plugin = await viewer.getPlugin()
    await plugin.restore(config.plugin_config)
  }
  if (config.expand_depth != null) {
    await applyDepth(viewer, config.expand_depth)
  }
  await viewer.flush()
}

async function applyDepth(viewer, depth) {
  const view = await viewer.getView()
  await view.set_depth(depth)
  const plugin = await viewer.getPlugin()
  await plugin.draw(view)
}

---

The perspective-click Event

Fires when the user clicks a cell. The event detail:

viewer.addEventListener('perspective-click', async (e) => {
  const { row, column_names, config } = e.detail
  // row          — aggregated values for the clicked cell (keyed by "split|metric" format)
  // column_names — array of metric column names clicked
  // config       — { filter: [[field, op, value], ...] }
  //   filter includes:
  //     - group_by coordinate filters  (field == value, one per group_by level)
  //     - split_by coordinate filters  (field == value, one per split_by field)
  //     - user-set filters             (any op)
})

__ROW_PATH__ in row contains the group_by path as an array.

The config.filter array is the reliable way to get cell coordinates. Do not try to zip __ROW_PATH__ with group_by — the filter approach handles all cases including partial paths.

---

Filtering Rows for a Clicked Cell

The click event's filter array can be applied to the underlying table via a new View, which correctly evaluates expression/computed columns (unlike filtering raw JS rows):

const config = await viewer.save()
const view = await table.view({
  filter: eventFilters,
  expressions: config.expressions || [],
})
const rows = await view.to_json()
await view.delete()

// Strip expression columns from results (they're computed, not source fields)
const exprNames = new Set(Object.keys(config.expressions || {}))
const clean = rows.map(r =>
  Object.fromEntries(Object.entries(r).filter(([k]) => !exprNames.has(k)))
)

Why not filter raw JS rows? Expression columns (computed in Perspective) don't exist in the source data. filterRowsByConfig on raw rows will skip those filters, returning all rows for the group rather than the specific cell.

Guard against no group_by: Without group_by, the filter array has no coordinate filters and the view query returns the entire table (slow). Check first:

const config = await viewer.save()
if ((config.group_by || []).length === 0) return  // no hierarchy — skip inspector

---

Viewer Methods (full list, v4.4.0)

From Object.getOwnPropertyNames(Object.getPrototypeOf(viewer)):

constructor, __destroy_into_raw, free, __get_model, connectedCallback, copy, delete, download, eject, export, flush, getAllPlugins, getClient, getEditPort, getPlugin, getRenderStats, getSelection, getTable, getView, getViewConfig, load, openColumnSettings, reset, resetError, resetThemes, resize, restore, restyleElement, save, setAutoPause, setAutoSize, setSelection, setThrottle, toggleColumnSettings, toggleConfig

Plugin Methods (datagrid, full list)

From Object.getOwnPropertyNames(Object.getPrototypeOf(plugin)):

constructor, connectedCallback, disconnectedCallback, activate, name, category, select_mode, min_config_columns, config_column_names, group_rollups, priority, can_render_column_styles, column_style_controls, draw, update, render, resize, clear, save, restore, restyle, delete

View Methods (full list)

From Object.getOwnPropertyNames(Object.getPrototypeOf(view)) — includes set_depth, expand, collapse, to_json, to_csv, to_arrow, schema, num_rows, num_columns, delete, and others.

---

settings Panel

The settings key in viewer.restore() controls whether the config panel (gear icon) is open:

// Hide on load:
await viewer.restore({ table: "name", settings: false, plugin_config: DEFAULT_PLUGIN_CONFIG })

// Toggle programmatically:
viewer.toggleConfig()

The settings state is saved by viewer.save() and restored on viewer.restore(), so it persists across layout saves automatically.

---

Incremental Updates

To update the table data without a full reload:

table.update(newRows)  // upserts by index (or by index_col if specified at table creation)

The viewer re-renders automatically after table.update().

---

Common Pitfalls

• plugin_config in viewer.restore() is unreliable. Always set plugin state via plugin.restore() separately after viewer.restore().
• view.set_depth() requires plugin.draw(view). The viewer won't redraw automatically.
• Expression columns don't exist in raw data. Filter via a Perspective View (table.view({ filter, expressions })), not against raw JS rows.
• Always await view.delete() after using a temporary view, or you'll leak worker memory.
• Named tables: worker.table(rows, { name: 'foo' }) — the name is used by the viewer's table config key. Re-open with worker.open_table('foo').