Commit Graph

94 Commits

Author SHA1 Message Date
442c38d3c4 Add PERSPECTIVE.md documenting the @perspective-dev version pairing
Records why the 4.5.1 viewer/client + 4.4.1 d3fc pairing is deliberate,
not a skew to "fix": the /inline and /themes entrypoints exist only in
4.5.x, while viewer-d3fc caps at 4.4.1, so this is the only combination
that keeps both inline WASM bundling and the d3fc charts. Verified by
build failure when pinning all four to 4.4.1. Points to the canonical
guide in pf_app for shared rationale.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-15 23:53:31 -04:00
efa65d8409 Update all docs to reflect current state
- perspective-pivot.md: npm install pattern, v4.5.1/v4.4.1 versions
- README.md: Node 18+, port 3020, add stacks routes, fix project structure
- SPEC.md: add stacks/status routes, pages, SQL functions; update Perspective version
- ui/README.md: replace Vite boilerplate with project-specific content
- Remove docs/refactor-transformed-split.md (completed work)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 23:51:00 -04:00
0ece53e7be Fix pg deprecation warning: set search_path via connection options
Replace pool.on('connect') query with connection-level options parameter.
Avoids calling client.query() during handshake, which pg will remove in v9.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 23:36:27 -04:00
317791341c Bump major dependencies: express 5, csv-parse 6, dotenv 17, multer 2
All APIs compatible with existing code. Added quiet:true to dotenv config
to suppress the new startup log message added in dotenv 17.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 23:35:15 -04:00
60924d03b5 Update patch/minor dependencies; skip major version bumps
UI: vite 8.0.16, react 19.2.7, react-router-dom 7.17.0, tailwindcss 4.3.1,
@vitejs/plugin-react 6.0.2, eslint-plugin-react-hooks 7.1.1, sql-formatter 15.8.1
API: pg 8.21.0
Skipped: eslint 10, express 5, csv-parse 6, dotenv 17, multer 2 (majors)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 23:04:25 -04:00
0c3cee4945 Migrate Perspective from CDN to npm; upgrade to 4.5.1
Replace runtime CDN imports with static ESM imports from npm packages.
Uses @perspective-dev/client and viewer inline builds (WASM embedded).
Bumps all packages to 4.5.1; d3fc stays at 4.4.1 (no 4.5.x release yet).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-12 23:00:23 -04:00
89a70bdf7e Split transformed column; add override management; show all override keys in panel
- transformed now stores only rule additions (not merged data+overrides)
- View dynamically computes data || transformed || overrides at query time
- New DB functions: set/clear/bulk_set_record_overrides
- Records panel now includes source-wide override keys so party/reason etc.
  appear even on records that don't have them set yet

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-23 11:00:24 -04:00
1baadaca61 Lift stack state to App; merge Records panel; fix Pivot theme on load
- Stack selection lifted to App.jsx: stacks fetched on login, selectedStack
  state shared via StatusBar (pills) and Pivot (view switching); Stacks page
  calls onStacksChange to keep list fresh
- Pivot: derive selectedView/viewType from props, remove local stack state;
  toolbar replaced with dedicated layouts sub-bar (h-9, layouts only)
- Records panel: merge read-only and override sections into single field list;
  known cols seeded from record's transformed fields; rule-derived fields
  (transformed minus data) will be editable in follow-up refactor
- Pivot theme: setAttribute moved to after flush() so restore() can't reset it

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-03 10:35:34 -04:00
9e0fa4aa7e Add collapsable sidebar with icons; move source picker to status bar
- New Sidebar component (modelled on pf_app): collapses 200px→48px via
  hamburger toggle, persists state to df_sidebar in localStorage; each
  nav item has an SVG icon with label that fades out when collapsed;
  user avatar + sign-out at bottom
- New StatusBar component: source picker + dark-mode toggle across the
  top of the content area
- Fix Pivot theme: setAttribute('theme') moved to after flush() so
  viewer.restore() can no longer reset it back to light

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 23:14:49 -04:00
738e1919ce Add light/dark mode with Perspective theme sync
Port light/dark mode from pf_app: ThemeProvider context, CSS custom
properties (Pro Dark palette), dark overrides for Tailwind classes, and
Perspective viewer theme sync in Pivot. Toggle button in sidebar header.
Improve toggle icons to Feather-style stroke SVGs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 22:59:24 -04:00
1791bf0f0a Store stack pivot layouts in DB; drop pivot_layouts FK
pivot_layouts.source_name had a FK to sources(name) preventing stack names
from being used as layout keys. Dropped the FK so any view name works.

- database/migrate_pivot_layouts_drop_fk.sql: drop the FK constraint
- api/routes/stacks.js: add GET/POST/DELETE /:name/layouts routes
- ui/src/api.js: add getStackPivotLayouts / saveStackPivotLayout / deleteStackPivotLayout
- ui/src/pages/Pivot.jsx: use DB for stack layouts instead of localStorage;
  collapse source/stack branches into saveLayout/deleteLayout helpers
- CLAUDE.md: document pivot layout persistence pattern

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 15:19:58 -04:00
bef3d6d89c CLAUDE.md: add UI section covering Pivot inspector patterns
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 10:39:25 -04:00
b52f5c930e Pivot inspector: toggle, resize, sort, totals, filter fix
- Click cell to open inspector pane; click same cell again to close (toggle).
  Uses __ROW_PATH__ + column_names as key so it works on both sources and stacks.
  Removes event listener on view change to prevent listener accumulation.
- Drag handle on left edge of inspector pane for resizing (min 240px)
- Removed redundant cell-coordinates block; breadcrumb now inline in header
- Sortable columns: click header to sort asc/desc with ▲/▼ indicator
- Totals row: sums all-numeric columns, sticky at bottom
- Derive missing split_by filters from column_names when Perspective omits
  them from detail.config.filter (fixes over-broad results on split_by views)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-02 10:37:47 -04:00
f373c85c16 Fix false-positive stale view warnings
Rules/mappings changes don't affect view SQL (views read from
transformed, shaped by config.fields) — remove those triggers.
Replace with a BEFORE UPDATE trigger on sources that only clears
view_generated_at when config actually changes.

Stack sources trigger now skips no-op upserts: the live SQL preview
calls upsertStackSource on every edit, which was unconditionally
clearing view_generated_at even when nothing changed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 12:51:32 -04:00
e5b95e7112 Add bulk override: DB function, API route, UI select bar
- bulk_set_record_overrides() DB function merges overrides into multiple
  records at once using a CTE with RETURNING for accurate count
- POST /records/bulk-overrides calls the function (consistent with rest
  of API — no raw SQL in routes)
- UI: regex input on loaded rows selects rows for bulk override; labeled
  "Bulk select:" / "DB query:" to distinguish from server-side filters

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 10:55:59 -04:00
814dcb7af1 Fix Records override save/clear: reload grid, allow blank overrides to suppress mappings
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 12:28:35 -04:00
d3a423c6ad Records override panel: read-only transformed view + Mappings-style override cols
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 11:13:05 -04:00
9ab2052f2b Separate mapping changes from view-stale: show Reprocess prompt instead of Generate
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 10:28:32 -04:00
ca266f2839 Fix autocomplete dropdown clipped by overflow container; use fixed positioning
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 09:59:14 -04:00
5951cbbba3 Redesign Records override panel: table layout, add-new-field support
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-25 09:55:23 -04:00
7a7fd01285 Fix cleanLayout stripping expression columns from pivot layout save/restore
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-20 20:39:02 -04:00
99b7b7d721 Add migration scripts for dataflow/dcard reimport
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 21:35:28 -04:00
9e6d184bd8 Stacks: source ordering via seq field with drag-to-reorder
- Add seq column to stack_sources; existing rows seeded by insertion order
- New sources auto-assigned max(seq)+1 so they always append to the end
- get_stack and generate_stack_view now order by seq instead of source_name
- Add reorder_stack_sources() function and PUT /:name/sources/reorder endpoint
- Source cards have drag handles matching the output columns grid behavior

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 17:11:44 -04:00
4e477420ad Fix lit() truncating decimals; restore last selected stack on load
- lit() was calling Math.trunc() on numbers, dropping decimals from balance_offset and any other numeric SQL params
- Stacks page now saves last selected stack to localStorage and restores it on load

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 16:38:02 -04:00
a89bd36f40 Stacks: calibrate modal redesign, layout column cleanup, SQL preview sync
- Calibrate modal now auto-fetches computed sum and shows live reconciliation table (data sum, known balance, plug) without requiring a button click
- as_of_date is now optional in calibrate — omitting it sums all transactions
- SQL preview syncs current UI state to DB before fetching so preview is always accurate
- Pivot cleanLayout strips stale columns from saved layouts when switching stack views

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 15:36:34 -04:00
95e63679ef Stacks: live SQL preview, side-by-side layout, cascade stale detection
- Two-column layout: config left, SQL panel right (equal halves)
- SQL panel shows formatted SQL (sql-formatter, 4-space indent)
- Live preview: SQL updates 400ms after any field/source/mapping change
- Run button executes edited SQL directly via new exec-sql endpoint
- generate_stack_view gains p_dry_run mode for preview without executing
- CASCADE drop detects dependent stacks, marks them stale in DB and status bar
- net_balance moved to last column in generated view
- Backfill 458 missing dcard rows and 123 missing chase rows from TPS migration bug

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 10:36:27 -04:00
7c63a2ac29 Status bar, stale tracking, Pivot stack selector, stack view fixes
- Add get_status() SQL and /api/status route; load stale state on login
- Replace polling with immediate client-side stale tracking via callbacks
- Amber status bar with per-item Generate buttons for sources and stacks
- Pivot: add stack selector to view any dfv.stack view via Perspective
- Stack views: DROP CASCADE, add id to source views, per-source balance columns
- net_balance = sum(all amounts) + total_offset guarantees chase+dcard=net per row
- CLAUDE.md: document correct dedup spec (within-batch duplicates always allowed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-19 09:43:10 -04:00
f941c5ae4a Stacks: per-source amount/date fields, mapping grid UI, dfv view generation with source balance CTEs
- SQL: upsert_stack_source gains amount_field/date_field params; calibrate_balance queries dfv.{source} directly (no stack view needed); generate_stack_view builds per-source CTEs with source_balance, outer net_balance; information_schema check for missing columns
- API: pass amount_field/date_field through upsert route; calibrate accepts source_name
- UI: mapping grid table (rows=fields, cols=sources); per-source amount/date/sign in Sources section; auto-populate output columns on first source config; horizontal stack chips above full-width config panel; calibration auto-saves before opening, editable offset input

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 21:17:12 -04:00
f63a0ec0e5 Stacks UI: reorder flow, balance dropdowns, current balance display, calibration editable offset
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 16:20:49 -04:00
ef6c6bbbb8 Add Stacks feature: multi-source union with running balance and calibration
- database/queries/stacks.sql: tables, functions for create/update/delete/calibrate/generate view
- api/routes/stacks.js: REST endpoints for stacks and stack sources
- api/server.js: register stacks router
- ui/src/api.js: stacks API methods
- ui/src/App.jsx: Stacks page route and nav entry
- ui/src/pages/Stacks.jsx: full UI for stack management, source mapping, calibration

Note: SQL deployment pending fix for balance_offset column and calibrate_balance signature

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-18 15:48:42 -04:00
fd67bb03af Records: fix override panel not opening
Open panel immediately on row click (panelOpen state), then load full
record async. Previously the panel condition depended on selectedRecord
or panelLoading both of which are set after async work, so if id was
missing or the API call failed the panel never appeared.

Also shows a message if id is missing (view needs regeneration).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 21:11:28 -04:00
c9b830b286 Add per-record overrides that survive reprocess
Schema:
- records.overrides JSONB column (ALTER TABLE, already applied)
- apply_transformations merges overrides on top: data || rules || overrides
- generate_source_view always includes id and _overridden columns
- set_record_overrides(id, overrides): stores and immediately merges into transformed
- clear_record_overrides(id): clears overrides then reprocesses record

API:
- PUT  /records/:id/overrides — set overrides
- DELETE /records/:id/overrides — clear and reprocess

UI (Records page):
- Rows are clickable; overridden rows highlighted amber
- Side panel shows all transformed fields as editable inputs
- Overridden fields highlighted amber with pencil indicator
- Save stores overrides; Clear removes them and restores computed values
- id and _overridden hidden from table display

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 21:02:54 -04:00
24675feb49 Mappings: clear regex filter after Save all
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 20:33:54 -04:00
dc32060c42 Add global Remap page for bulk output value replacement
- SQL: search_mapping_outputs(search) — distinct (col, val, count) groups
         get_mappings_by_output_field(col, val) — individual mappings
         remap_output_field(col, from, to) — bulk UPDATE via jsonb_set
- API: GET /mappings/outputs?search=, GET /mappings/outputs/:col/:val,
       POST /mappings/remap-field
- UI: Remap page — search output values, click to select, edit the
  replacement value, see all affected mappings, apply globally
- Nav: Remap added between Mappings and Records

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 20:22:52 -04:00
bda59c7675 Docs: update Pivot spec section and add Perspective technical reference
SPEC.md: rewrite Pivot page description to cover named layouts, depth
control, selection mode, inspector filtering, and layout persistence.

docs/perspective-pivot.md: new file documenting all discovered Perspective
v4.4.0 APIs — viewer/plugin/view methods, selection modes, set_depth
mechanism, perspective-click event shape, full state save/restore pattern,
and common pitfalls.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 08:58:36 -04:00
420bc1bbe8 Pivot: round numbers in inspector to 2 decimals with adjustable precision
formatVal now rounds numeric values using toLocaleString with
configurable decimal places (default 2, range 0-8). Adds -/+ controls
in the inspector header to adjust on the fly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 08:54:34 -04:00
8d3cc24094 Pivot: skip inspector query when no group_by hierarchy is active
Without group_by there are no coordinate filters, so the view query
would return the full dataset and hang. Early-return on click if
config.group_by is empty.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 08:52:09 -04:00
6e9cdd82ea Pivot: widen detail pane from w-80 to w-96
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 08:43:47 -04:00
ed07dde492 Pivot: default settings panel to hidden on fresh load
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 08:42:07 -04:00
7c07434049 Pivot: save/restore edit mode and expand depth in named layouts
- Default selection mode is now SELECT_REGION
- plugin.save()/restore() used to capture and apply edit mode
- expand_depth tracked in ref and included in layout config
- applyExpandDepth helper restores depth on layout recall and page load
- Save button overwrites active layout in place (no re-typing name)
- captureConfig() helper shared by save-over and save-as flows

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 08:40:24 -04:00
b88795b015 Clean up expand depth control into proper toolbar UI
Replace debug test buttons with a minimal 'depth: 0 1 2 3' control
in the pivot toolbar right side.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 08:05:47 -04:00
3a172e2456 Find working expand depth control: view.set_depth + plugin.draw
After testing plugin_config.expand_depth (no effect) and view.set_depth
+ flush() (no effect), confirmed that view.set_depth(d) followed by
plugin.draw(view) correctly collapses/expands all rows to depth d.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 08:05:10 -04:00
0b8c2935d7 Add expand_depth test buttons to Pivot toolbar
Temporary UI for testing programmatic row expansion control via
plugin_config.expand_depth in Perspective viewer.restore().

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 07:55:37 -04:00
3723778cbb Pivot: named layouts saved in DB per source
- pivot_layouts table (source_name, layout_name, config JSONB)
- list/save/delete SQL functions and API routes
- Pivot toolbar above viewer: layout chips, save-as inline input,
  delete per layout, reset to default
- Applying a named layout also updates localStorage working state
- Layouts reload on source change

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 07:31:46 -04:00
23fa14f22c Pivot: move save layout button to top-left
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-15 07:28:52 -04:00
c98efe58d1 Pivot: show all row metrics in inspector, highlight clicked cell
Always display all non-null metric columns from the clicked row.
When a specific cell can be identified (split_by in use, cell mode),
highlight that row in blue/bold. Fixes row mode showing only one value.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 23:02:04 -04:00
ec0cc73f31 SPEC: add Pivot and Log pages, update file structure
Document the Perspective-based pivot viewer, cell inspector
behavior, layout persistence, and row matching approach.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 22:54:40 -04:00
fb9ff8720a Pivot: use event filters for row matching, skip computed columns
Replace __ROW_PATH__ zip approach with direct application of
perspective-click event filters against raw rows. Fields not
present in the raw data (Perspective computed columns like Month,
YearDate) are skipped. Also removes debug console.log calls.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 22:51:48 -04:00
1587d06967 Pivot: add debug logging for cell click investigation
Temporary logs to inspect perspective-click event detail.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 22:41:55 -04:00
f7d73ad821 Pivot: clean up click inspector upper pane display
Show row path prominently, filter to non-null metric values,
use group_by › split_by as section header.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 22:31:56 -04:00