Captures current /data path (with bug history that forced single-batch
encoding), and four candidate redesigns: optimize the existing encoder,
DuckDB-WASM with Parquet, server-side DuckDB virtual server, and the
hybrid read-from-WASM/write-via-deltas variant. Each option weighed
against the forecasting write path, not just initial load. Intended as
a decision record so context survives a lost conversation.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Perspective table is now created with index: 'pf_id'. Delete endpoints
return the pf_ids they removed; the client calls table.remove(pf_ids)
in undoEntry. Avoids the full /data refetch that dominated undo time.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
/data now joins pf.log to surface note text and operation type as
pf_note/pf_op so users can pivot/bridge by assumption. Joining at
fetch time avoids storing notes per row and keeps edits live.
/api/tables joined pg_class by name only with namespace filtered in
a separate LEFT JOIN, which cross-producted table names that exist in
multiple schemas. Restructured so namespace participates in the join.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
pg now returns bigint/numeric as JS numbers so Arrow infers Int/Float64
instead of Dictionary<Utf8>. /data accumulates rows and emits a single
record batch to avoid dictionary REPLACEMENT messages that crash
Perspective's WASM reader. Forecast view streams the response body and
shows received/total bytes while loading. Drops stale public/ static
middleware that was shadowing the React build at /.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Server streams rows from a pg cursor in 10k-row batches, building Arrow
record batches incrementally and piping them as chunked HTTP response —
Node.js heap stays bounded regardless of dataset size.
Client fetches as arrayBuffer() and loads directly into Perspective worker
(native Arrow path, no JSON deserialization). X-Row-Count header drives
a non-blocking banner for datasets >= 500k rows. validCols now derived
from col_meta rather than from row keys.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Re-applies fetchCurrentTotals dimension-only filter (prevents Perspective errors
from split_by/expression columns in the filter), toolbar three-group reorganization
(Layout | Expand | Data with dividers), always-visible Save as…, msg in toolbar,
resizable panel, change log modal with undo and inline note editing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Expression columns (bucket, computed) are defined in cfg.expressions and
are valid pivot axes, but weren't in validCols (raw table columns), so
they were filtered out of group_by/split_by on every layout restore.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GET /api/versions/:id/log — log entries with row counts via JOIN
- DELETE /api/log/:logid — undo in a transaction (delete fc rows + log entry)
- PATCH /api/log/:logid — update note text
- History button opens a modal: op badge, slice, editable note, row count, Undo per entry
- Undo triggers full Perspective table reload via initViewer
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix perspective-click handler to use event filter triples instead of
__ROW_PATH__ — Perspective encodes row position as [col,'==',val] in
detail.config.filter
- buildWhere now skips unrecognised slice keys (e.g. pf_iter) instead of
throwing, so only dimension columns reach the WHERE clause
- Add draggable resize handle on the operation panel (160–480px)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Baseline.jsx: merge Reference section into Add Segment form with baseline/reference toggle; segment rows now clickable to expand stored WHERE clause + timeline; date filter inputs use type="date" for date-role columns
- Timeline.jsx: add type prop ('baseline'|'reference'); reference band uses purple; single-band height shrinks to 52px; canvas uses requestAnimationFrame to fix offsetWidth=0 on mount
- operations.js: reference route now accepts where_clause like baseline (drops date_from/date_to)
- sql_generator.js: reference SQL template uses {{filter_clause}} instead of hardcoded BETWEEN
Note: existing sources need Generate SQL re-run to pick up the new reference template.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ui/: React + Vite + Tailwind app (Setup, Baseline, Forecast views, collapsible sidebar, status bar, canvas timeline)
- server.js: serve built UI from public/app/
- package.json: add build script (cd ui && npm run build)
- routes/sources.js: default new col_meta role to 'dimension' instead of 'ignore'
- .gitignore: exclude public/app/ build output
- pf_spec.md: update tech stack, nav, frontend section, and project status to reflect current implementation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- HTML mockup with collapsible side nav, 3-step flow (Setup/Baseline/Forecast)
- Canvas-based timeline preview in baseline segment form
- Table peek modal, status bar, help popovers
- Spec updated: 3-step mental model, AG Grid replacement note
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- restore() must include table: tableName so viewer knows which table
- Replace cloneNode listener reset with named _pspClickHandler pattern
- Always inject table name into saved layouts on restore
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sources/Versions/Log grids still use AG Grid — only the Forecast
pivot was replaced with Perspective.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Swap AG Grid CSS/JS for Perspective CDN imports (4.4.0)
- Replace #pivot-grid div with <perspective-viewer> web component
- Add loadPerspective() singleton, initPerspectiveViewer(data)
- Build default layout from col_meta (dims → group_by, date → split_by)
- Extract slice from perspective-click event.detail.config.filter
(only == filters on role=dimension columns feed the operation panel)
- table.update() appends operation result rows without full reload
- Save/reset layout buttons per version in localStorage
- Remove expand/collapse buttons (Perspective handles natively)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Sources page: left column with stacked DB tables + registered sources panels,
right column as full-height column mapping workbench
- Add compact table search, column search, table preview button, delete source button
- Rename fc_table system columns to pf_ prefix (pf_id, pf_iter, pf_logid,
pf_created_at) to avoid collisions with source table columns like 'id'
- Remove 'filter' col_meta role — any non-ignore column usable in baseline filters
- Replace structured filter row builder with free-form SQL WHERE clause textarea
and clickable column chips for insertion; fully flexible AND/OR logic
- Baseline segment cards now display raw WHERE clause text + offset
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Switches from side-by-side panels to stacked layout so the filter
builder has full width and input fields are no longer cramped.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New Baseline nav view replaces the simple Load Baseline modal
- Baseline loads are now additive; each segment is independently undoable
- Filter builder: any date/filter-role column, full operator set
- Timeline preview shows source → projected period bars for date BETWEEN filters
- Clear Baseline action deletes all baseline rows and log entries
- DELETE /api/versions/:id/baseline route
- buildFilterClause() added to sql_generator
- filter role added to col_meta editor
- Reminder: re-run generate-sql for each source after this change
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removes the redundant date_from/date_to/date_col fields from the request
body. Period selection is now expressed as a filter condition in the
filters array like any other condition. SQL pattern updated to match
(single {{filter_clause}} token instead of date_range + filter split).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Period selection (date range, season, etc.) is now expressed as a
filter condition like any other — no separate date range section.
Preview uses a timeline/number-line bar instead of month chips.
Documents the unified filter builder approach.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers architecture, data model, API routes, SQL patterns, and UI design.
Includes baseline workbench design with multi-segment additive loads,
filter role for col_meta, and date offset for projecting actuals into
the forecast period.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The baseline operation now accepts a date_offset interval (e.g. "1 year",
"6 months") and applies it to every date when inserting rows, shifting
historical actuals into the target forecast period.
SQL: {date_col} + '{{date_offset}}'::interval)::date at insert time.
Route: defaults to '0 days' if omitted so existing calls are unaffected.
UI: year/month spinners with a live before→after month chip preview so
the projected landing period is visible before submitting.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Exposes the stored params (e.g. date_from/date_to for baseline/reference)
so the date range used in each operation is visible in the log.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the small inline form with a centred modal dialog. When both
dates are selected, a live chip list shows every month covered (up to
36 months) so it is immediately clear what periods will be loaded.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Instead of re-fetching all forecast data after scale/recode/clone/reference,
the routes now return the inserted rows directly. The frontend uses ag-Grid's
applyTransaction to add only the new rows, eliminating the full reload round-trip.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
