Document the outcome of the Option C spike (server-side DuckDB as a
Perspective virtual server) and the resulting architecture decision.
pf_perspective_options.md:
- Spike findings: latency is excellent (~12ms round trip, 5-29ms
aggregation, ~1.5-1.9s to materialize 534,902 rows) but Perspective's
GenericSQLVirtualServerModel ignores group_by_depth and has no
ViewConfig field for per-node expansion state, so interactive
drill-down is not achievable. This affects options B, C and D alike
since they share that SQL model.
- Decision: the real lever is grain, not transport. Pre-aggregating to
display grain collapses 534,902 -> 4,642 rows (~115x) on osm_stack
while keeping the native Perspective engine, so expand/collapse/
depth/sort/filter continue to work.
- Two candidate designs (Path A live virtual server vs Path B
pre-aggregated extract) with the deciding question: do real cuts ever
exceed the browser's leaf-row ceiling?
pf_spec.md:
- Concrete Path B design: pf.col_meta.in_grain, GET /api/versions/:id/agg,
synthetic pf_gkey index, and the append-deltas write/undo model that
mirrors the prior Excel pivot-cache workflow.
Drop pf_ux_mockup.md — an ASCII mockup of UI that is now built; the
views in ui/src/views are the current reference.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Captures current /data path (with bug history that forced single-batch
encoding), and four candidate redesigns: optimize the existing encoder,
DuckDB-WASM with Parquet, server-side DuckDB virtual server, and the
hybrid read-from-WASM/write-via-deltas variant. Each option weighed
against the forecasting write path, not just initial load. Intended as
a decision record so context survives a lost conversation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
