Sources with a `simplefin` block in their config can pull transactions
straight from the bridge instead of taking a CSV upload. Only the fetch
differs — dedupe, logging, and transformation reuse the import path.
The access URL is the whole credential, so it lives in .env rather than
the database that manage.py offers to reset. Claiming a setup token is
exposed as an endpoint because the token is single-use and easy to burn.
The bridge answers 200 with a populated `errors` array when a bank is
failing, which would otherwise read as a successful empty pull — those
errors ride along in the sync response and show on the Import page.
Pending transactions are skipped by default: they get a new id once they
post, which would import the same charge twice under two keys. Sources
should use ['id'] as constraint_fields — the transaction id makes
overlapping pulls free while keeping genuinely repeated charges distinct.
Verified against a stubbed bridge response, not a live account.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G2HFeU5neCKagTnmA6o9Tu
get_source_fields only unioned schema fields, raw data keys, rule
output_field and mapping output keys. Keys that live solely in
records.transformed or records.overrides — a manual override such as
dcard's "Note", or a transformed key whose rule was since deleted — never
appeared on the source page, so there was no way to add them to the view.
Read both columns off the records directly.
Records showed no id column, leaving no handle to identify a row. Split
the hidden-column set: HIDDEN_COLS still keeps id out of the override
editor, GRID_HIDDEN_COLS hides only _overridden, and gridCols() pins id
first in the grid and the filter dropdown.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BDFU3ueEbCYGE37oDgeu2j
The page loaded unsorted, so the most recent rows were rarely on screen. On
source change it now looks up the source's first date-typed field and sorts
descending on it, falling back to unsorted when a source has no date field.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Vite scaffolding, unmodified since the UI was added. Everything in it that this
project produces is already covered by the root .gitignore.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Architecture, file structure, the manage.py menu, and the API reference were
each documented in two or three of README.md, SPEC.md, and CLAUDE.md — the same
drift trap the SQL just had.
SPEC.md, examples/GETTING_STARTED.md, and ui/README.md move into docs/.
PERSPECTIVE.md and docs/perspective-pivot.md merge into docs/perspective.md,
version rationale first, then the API reference. README.md becomes an entry
point that links out, and CLAUDE.md keeps only working rules and non-obvious
behaviour, pointing at docs/spec.md for the rest. examples/ keeps just the
sample CSV the tutorial loads.
Corrections found while consolidating:
- the spec's API table was missing 20 routes — every override endpoint, most of
/api/stacks, the mapping remap routes, /health. Rebuilt from the route files
- the tutorial used port 3000 (default is 3020) and never mentioned Basic auth,
so every curl in it would have 401'd
- the tutorial and the spec each hand-listed the SQL deploy order; both now
point at manage.py, which is where the order actually lives
- CLAUDE.md described deduplication as an MD5 hash (it is a plain JSONB object),
claimed 5 tables and 4 functions, and told you to run a setup.sh that has not
existed for some time
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The caret ranges in ui/package.json (^4.5.1 viewer/client/datagrid, ^4.4.1
viewer-d3fc) let npm install resolve a newer Perspective on a fresh machine.
That pairing is deliberate and load-bearing — 4.4.1 lacks the /inline export
paths Pivot.jsx imports, and viewer-d3fc has no 4.5.x — so the resolved tree
needs pinning, not just the ranges.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Drop deploy.sh, scripts/setup-service.sh, and migrate/ — all superseded by
manage.py. Remove leftover Vite template assets (App.css, hero.png, react.svg,
vite.svg), none of which are imported.
Hoist the database/queries/ file list in manage.py to a module-level
QUERY_FILES constant so configure and deploy-functions share one definition,
and add the stacks/status query files that were missing from it.
Swap npm start/dev so start runs node and dev runs nodemon.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Replace runtime CDN imports with static ESM imports from npm packages.
Uses @perspective-dev/client and viewer inline builds (WASM embedded).
Bumps all packages to 4.5.1; d3fc stays at 4.4.1 (no 4.5.x release yet).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- transformed now stores only rule additions (not merged data+overrides)
- View dynamically computes data || transformed || overrides at query time
- New DB functions: set/clear/bulk_set_record_overrides
- Records panel now includes source-wide override keys so party/reason etc.
appear even on records that don't have them set yet
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Stack selection lifted to App.jsx: stacks fetched on login, selectedStack
state shared via StatusBar (pills) and Pivot (view switching); Stacks page
calls onStacksChange to keep list fresh
- Pivot: derive selectedView/viewType from props, remove local stack state;
toolbar replaced with dedicated layouts sub-bar (h-9, layouts only)
- Records panel: merge read-only and override sections into single field list;
known cols seeded from record's transformed fields; rule-derived fields
(transformed minus data) will be editable in follow-up refactor
- Pivot theme: setAttribute moved to after flush() so restore() can't reset it
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- New Sidebar component (modelled on pf_app): collapses 200px→48px via
hamburger toggle, persists state to df_sidebar in localStorage; each
nav item has an SVG icon with label that fades out when collapsed;
user avatar + sign-out at bottom
- New StatusBar component: source picker + dark-mode toggle across the
top of the content area
- Fix Pivot theme: setAttribute('theme') moved to after flush() so
viewer.restore() can no longer reset it back to light
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Port light/dark mode from pf_app: ThemeProvider context, CSS custom
properties (Pro Dark palette), dark overrides for Tailwind classes, and
Perspective viewer theme sync in Pivot. Toggle button in sidebar header.
Improve toggle icons to Feather-style stroke SVGs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
pivot_layouts.source_name had a FK to sources(name) preventing stack names
from being used as layout keys. Dropped the FK so any view name works.
- database/migrate_pivot_layouts_drop_fk.sql: drop the FK constraint
- api/routes/stacks.js: add GET/POST/DELETE /:name/layouts routes
- ui/src/api.js: add getStackPivotLayouts / saveStackPivotLayout / deleteStackPivotLayout
- ui/src/pages/Pivot.jsx: use DB for stack layouts instead of localStorage;
collapse source/stack branches into saveLayout/deleteLayout helpers
- CLAUDE.md: document pivot layout persistence pattern
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Click cell to open inspector pane; click same cell again to close (toggle).
Uses __ROW_PATH__ + column_names as key so it works on both sources and stacks.
Removes event listener on view change to prevent listener accumulation.
- Drag handle on left edge of inspector pane for resizing (min 240px)
- Removed redundant cell-coordinates block; breadcrumb now inline in header
- Sortable columns: click header to sort asc/desc with ▲/▼ indicator
- Totals row: sums all-numeric columns, sticky at bottom
- Derive missing split_by filters from column_names when Perspective omits
them from detail.config.filter (fixes over-broad results on split_by views)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- bulk_set_record_overrides() DB function merges overrides into multiple
records at once using a CTE with RETURNING for accurate count
- POST /records/bulk-overrides calls the function (consistent with rest
of API — no raw SQL in routes)
- UI: regex input on loaded rows selects rows for bulk override; labeled
"Bulk select:" / "DB query:" to distinguish from server-side filters
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add seq column to stack_sources; existing rows seeded by insertion order
- New sources auto-assigned max(seq)+1 so they always append to the end
- get_stack and generate_stack_view now order by seq instead of source_name
- Add reorder_stack_sources() function and PUT /:name/sources/reorder endpoint
- Source cards have drag handles matching the output columns grid behavior
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- lit() was calling Math.trunc() on numbers, dropping decimals from balance_offset and any other numeric SQL params
- Stacks page now saves last selected stack to localStorage and restores it on load
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Calibrate modal now auto-fetches computed sum and shows live reconciliation table (data sum, known balance, plug) without requiring a button click
- as_of_date is now optional in calibrate — omitting it sums all transactions
- SQL preview syncs current UI state to DB before fetching so preview is always accurate
- Pivot cleanLayout strips stale columns from saved layouts when switching stack views
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Two-column layout: config left, SQL panel right (equal halves)
- SQL panel shows formatted SQL (sql-formatter, 4-space indent)
- Live preview: SQL updates 400ms after any field/source/mapping change
- Run button executes edited SQL directly via new exec-sql endpoint
- generate_stack_view gains p_dry_run mode for preview without executing
- CASCADE drop detects dependent stacks, marks them stale in DB and status bar
- net_balance moved to last column in generated view
- Backfill 458 missing dcard rows and 123 missing chase rows from TPS migration bug
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add get_status() SQL and /api/status route; load stale state on login
- Replace polling with immediate client-side stale tracking via callbacks
- Amber status bar with per-item Generate buttons for sources and stacks
- Pivot: add stack selector to view any dfv.stack view via Perspective
- Stack views: DROP CASCADE, add id to source views, per-source balance columns
- net_balance = sum(all amounts) + total_offset guarantees chase+dcard=net per row
- CLAUDE.md: document correct dedup spec (within-batch duplicates always allowed)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Open panel immediately on row click (panelOpen state), then load full
record async. Previously the panel condition depended on selectedRecord
or panelLoading both of which are set after async work, so if id was
missing or the API call failed the panel never appeared.
Also shows a message if id is missing (view needs regeneration).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Schema:
- records.overrides JSONB column (ALTER TABLE, already applied)
- apply_transformations merges overrides on top: data || rules || overrides
- generate_source_view always includes id and _overridden columns
- set_record_overrides(id, overrides): stores and immediately merges into transformed
- clear_record_overrides(id): clears overrides then reprocesses record
API:
- PUT /records/:id/overrides — set overrides
- DELETE /records/:id/overrides — clear and reprocess
UI (Records page):
- Rows are clickable; overridden rows highlighted amber
- Side panel shows all transformed fields as editable inputs
- Overridden fields highlighted amber with pencil indicator
- Save stores overrides; Clear removes them and restores computed values
- id and _overridden hidden from table display
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- SQL: search_mapping_outputs(search) — distinct (col, val, count) groups
get_mappings_by_output_field(col, val) — individual mappings
remap_output_field(col, from, to) — bulk UPDATE via jsonb_set
- API: GET /mappings/outputs?search=, GET /mappings/outputs/:col/:val,
POST /mappings/remap-field
- UI: Remap page — search output values, click to select, edit the
replacement value, see all affected mappings, apply globally
- Nav: Remap added between Mappings and Records
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
formatVal now rounds numeric values using toLocaleString with
configurable decimal places (default 2, range 0-8). Adds -/+ controls
in the inspector header to adjust on the fly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Without group_by there are no coordinate filters, so the view query
would return the full dataset and hang. Early-return on click if
config.group_by is empty.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Default selection mode is now SELECT_REGION
- plugin.save()/restore() used to capture and apply edit mode
- expand_depth tracked in ref and included in layout config
- applyExpandDepth helper restores depth on layout recall and page load
- Save button overwrites active layout in place (no re-typing name)
- captureConfig() helper shared by save-over and save-as flows
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace debug test buttons with a minimal 'depth: 0 1 2 3' control
in the pivot toolbar right side.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After testing plugin_config.expand_depth (no effect) and view.set_depth
+ flush() (no effect), confirmed that view.set_depth(d) followed by
plugin.draw(view) correctly collapses/expands all rows to depth d.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Temporary UI for testing programmatic row expansion control via
plugin_config.expand_depth in Perspective viewer.restore().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- pivot_layouts table (source_name, layout_name, config JSONB)
- list/save/delete SQL functions and API routes
- Pivot toolbar above viewer: layout chips, save-as inline input,
delete per layout, reset to default
- Applying a named layout also updates localStorage working state
- Layouts reload on source change
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Always display all non-null metric columns from the clicked row.
When a specific cell can be identified (split_by in use, cell mode),
highlight that row in blue/bold. Fixes row mode showing only one value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace __ROW_PATH__ zip approach with direct application of
perspective-click event filters against raw rows. Fields not
present in the raw data (Perspective computed columns like Month,
YearDate) are skipped. Also removes debug console.log calls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Show row path prominently, filter to non-null metric values,
use group_by › split_by as section header.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
