Working measurement spike for Option C: aggregation runs in a server-side
DuckDB instead of shipping every row to the browser. Opt-in via
?engine=duck; the default client-side WASM path is untouched.
- routes/perspective.js: POST /perspective/init materializes a version's
forecast table into a persistent in-process DuckDB (ATTACH postgres,
so 535k rows land in ~1.5s without streaming through Node); POST
/perspective/sql executes one Perspective-generated statement, with
Arrow IPC output for viewport reads.
- ui/src/duckServerHandler.js: VirtualServerHandler that delegates SQL
generation to Perspective's own GenericSQLVirtualServerModel and POSTs
the result to the server. Adapted from @perspective-dev/client's
virtual_servers/duckdb.ts — same SQL model, different transport.
- Forecast.jsx: DUCK_MODE branch replaces the heavy /data fetch with
/perspective/init, opens the hosted table over a message port, and
shows a latency overlay. applyDepth routes through viewer.restore()
since the SQL server cannot handle view.set_depth().
Preserved for a future faster-load effort. Findings and the reason this
is not the shipped path (no interactive tree expand/collapse) are in
pf_perspective_options.md on master.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Server streams rows from a pg cursor in 10k-row batches, building Arrow
record batches incrementally and piping them as chunked HTTP response —
Node.js heap stays bounded regardless of dataset size.
Client fetches as arrayBuffer() and loads directly into Perspective worker
(native Arrow path, no JSON deserialization). X-Row-Count header drives
a non-blocking banner for datasets >= 500k rows. validCols now derived
from col_meta rather than from row keys.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ui/: React + Vite + Tailwind app (Setup, Baseline, Forecast views, collapsible sidebar, status bar, canvas timeline)
- server.js: serve built UI from public/app/
- package.json: add build script (cd ui && npm run build)
- routes/sources.js: default new col_meta role to 'dimension' instead of 'ignore'
- .gitignore: exclude public/app/ build output
- pf_spec.md: update tech stack, nav, frontend section, and project status to reflect current implementation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
