pf_app/CLAUDE.md

Pivot Forecast — CLAUDE.md

What this app is

A web app for building named forecast scenarios against any PostgreSQL table. The workflow: load historical actuals as a baseline (optionally date-shifted into the forecast period), then apply incremental adjustments (scale, recode, clone) to build a plan. All changes are append-only, fully audited, and reversible by log entry.

Full spec: pf_spec.md Data transport architecture options: pf_perspective_options.md UX mockup: pf_ux_mockup.md Open work: todo.md

---

Tech stack

• Backend: Node.js / Express (server.js), runs on port 3010
• Database: PostgreSQL — isolated pf schema
• Frontend: React + Vite + Tailwind CSS in ui/; built output lands in public/app/
• Pivot: Perspective 4.4.0 loaded from CDN at runtime
• Dev: npm run dev (nodemon) in root; npm run build in ui/

---

Project layout

server.js               Express entry point; pg pool; type parsers for bigint/numeric
routes/
  tables.js             GET /api/tables, /api/tables/:schema/:tname/preview
  sources.js            Source registration, col_meta, SQL generation
  versions.js           Version CRUD, baseline/reference load, data stream
  operations.js         scale, recode, clone, undo — the core forecast ops
  log.js                GET /api/versions/:id/log, DELETE /api/log/:logid
lib/
  sql_generator.js      buildFilterClause, token substitution helpers
  utils.js
setup_sql/
  01_schema.sql         pf schema DDL — run once to install
ui/src/
  views/
    Setup.jsx           DB browser, source registration, col_meta editor
    Baseline.jsx        Version management, baseline workbench, reference load
    Forecast.jsx        Perspective pivot + operation panel (Scale/Recode/Clone)
    Sidebar.jsx         3-step collapsible nav
    StatusBar.jsx       Source · version · row count · status
    Timeline.jsx        Date-range preview bar for baseline segments

---

Database schema (pf)

• pf.source — registered source tables
• pf.col_meta — column roles: dimension | value | units | date | filter | ignore; is_key marks dimensions used in slice WHERE clauses
• pf.version — named forecast scenarios; exclude_iters (default ["reference"]) blocks those iter values from all operations
• pf.fc_{tname}_{version_id} — one forecast table per version; contains both operational rows (iter = baseline|scale|recode|clone) and reference rows (iter = reference)
• pf.log — audit log; every write gets one entry; slice + params stored as jsonb
• pf.sql — generated SQL templates per source/operation; tokens substituted at request time

Key token substitution tokens

{{fc_table}}, {{where_clause}}, {{exclude_clause}}, {{logid}}, {{pf_user}}, {{value_incr}}, {{units_incr}}, {{pct}}, {{set_clause}}, {{scale_factor}}, {{date_offset}}, {{filter_clause}}

---

Core data flow

Initial load (Forecast view)

GET /api/versions/:id/data → Arrow IPC binary stream → worker.table(buffer) in Perspective WASM

Why one batch (not streaming): pg returns bigint/numeric as strings by default — type parsers in server.js coerce them to numbers. Per-batch Arrow encoding creates independent dictionaries that cause Perspective WASM to crash on dictionary replacement messages. Server accumulates all rows, emits one record batch.

Forecast operations

POST to /api/versions/:id/{scale|recode|clone} → SQL executed with RETURNING * → new rows returned as JSON → pspTable.update(rows) — no full reload.

Undo

DELETE /api/log/:logid → removes rows by logid → full Perspective reload (known wart).

---

Slice mechanics

When the user clicks a pivot cell, perspective-click fires. The handler in Forecast.jsx extracts [col, '==', value] filters from detail.config.filter — only role = dimension columns are kept as the slice. This slice populates the operation panel and is sent as the slice object in all operation POST bodies.

Limitation: computed columns created by Perspective's split_by (e.g. Month, YearDate) don't map back to raw rows — only native dimension columns work for slice extraction.

---

Operation SQL patterns

All three operations follow the same structure: insert a pf.log row in a CTE, then insert forecast rows referencing its id. {{where_clause}} is built from the slice; {{exclude_clause}} blocks exclude_iters rows.

• Scale — distributes value_incr/units_incr proportionally across rows in the slice using window functions
• Recode — inserts negative rows (zero out original) + positive rows with {{set_clause}} dimension overrides; both share the same logid
• Clone — copies the slice with {{set_clause}} overrides and {{scale_factor}} multiplier; original untouched

build_where() validates every slice key against col_meta (only role = dimension allowed). Values are escaped but not parameterized — consistent with existing patterns, debuggable in pg logs.

---

Known issues / active work (see todo.md for detail)

• Operation panel (Scale/Recode/Clone) wiring to API is a stub — needs completion
• Status bar is hardcoded — needs to reflect actual selected source/version
• Load progress bar is jittery — needs throttle (~10 updates/sec)
• Default pivot layout should be configurable per source (currently hardcodes first 2 dimensions)
• Source/version selection doesn't persist across page reload
• Col_meta / version schema drift: if col_meta roles change after a version's forecast table is created, SQL and DDL go out of sync — workaround is to delete and recreate the version

Deferred (not in v1)

Baseline replay (replay: true returns 501), approval workflow, territory filtering, export, version comparison, multi-DB connections.