Field discovery now samples an account's actual transactions instead of
assuming a shape. flatten() passes through every scalar the bridge sends
rather than whitelisting eleven keys, so institution-specific fields turn
up on their own, and inferFields() — extracted from the CSV suggest route
so both paths share it — unions keys across the sample because API feeds
omit optional fields entirely.
Three bugs the live bridge exposed:
- posted=0 on pending transactions became 1970-01-01; falsy epochs are
now "no date", with date falling back to transacted_at and posted_date
kept separate.
- days=0 omitted start-date, which returns only the few most recent
transactions rather than everything — 4 instead of 89. A start-date is
always sent now, clamped to 89 days (the bridge hard-caps at 90).
- Sampling asked for more than 45 days, and the bridge's advisory notice
about that surfaced in the UI as an error. Samples use 44 days; the
threshold is exclusive.
The Sources page can now link an account: a picker in both the create
dialog and the detail panel, populated on demand, which fills the field
table from the sample and defaults the constraint field to the
transaction id with an explanation of why.
manage.py option 10 claims a setup token and writes the access URL to
.env, replacing the throwaway script.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G2HFeU5neCKagTnmA6o9Tu
Sources with a `simplefin` block in their config can pull transactions
straight from the bridge instead of taking a CSV upload. Only the fetch
differs — dedupe, logging, and transformation reuse the import path.
The access URL is the whole credential, so it lives in .env rather than
the database that manage.py offers to reset. Claiming a setup token is
exposed as an endpoint because the token is single-use and easy to burn.
The bridge answers 200 with a populated `errors` array when a bank is
failing, which would otherwise read as a successful empty pull — those
errors ride along in the sync response and show on the Import page.
Pending transactions are skipped by default: they get a new id once they
post, which would import the same charge twice under two keys. Sources
should use ['id'] as constraint_fields — the transaction id makes
overlapping pulls free while keeping genuinely repeated charges distinct.
Verified against a stubbed bridge response, not a live account.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01G2HFeU5neCKagTnmA6o9Tu
- Add get_status() SQL and /api/status route; load stale state on login
- Replace polling with immediate client-side stale tracking via callbacks
- Amber status bar with per-item Generate buttons for sources and stacks
- Pivot: add stack selector to view any dfv.stack view via Perspective
- Stack views: DROP CASCADE, add id to source views, per-source balance columns
- net_balance = sum(all amounts) + total_offset guarantees chase+dcard=net per row
- CLAUDE.md: document correct dedup spec (within-batch duplicates always allowed)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- pivot_layouts table (source_name, layout_name, config JSONB)
- list/save/delete SQL functions and API routes
- Pivot toolbar above viewer: layout chips, save-as inline input,
delete per layout, reset to default
- Applying a named layout also updates localStorage working state
- Layouts reload on source change
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix stale import_records in sources.sql that referenced deleted generate_constraint_key
- Auto-transform after import, auto-generate view after create
- New source form matches existing source layout (In view, Seq, type dropdown)
- Sample data table (50 rows) shown below field config in both new and existing source views
- Import sample CSV on create (checked by default)
- Sortable column headers on field table
- Choose CSV styled as a button showing filename
- + button in sidebar opens new source form
- Records tab shows error message when view cast fails instead of blank
- Pivot page with Perspective viewer, per-source saved layouts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Rename dedup_key/dedup_fields → constraint_key/constraint_fields everywhere
(schema, functions, routes, UI, migration script, docs)
- Change constraint_key from MD5 TEXT hash to readable JSONB object
- Drop unique constraint on (source_name, constraint_key); dedup is now
enforced at import time via CTE, allowing intra-file duplicate rows
- Add import_id FK (ON DELETE CASCADE) so deleting a log entry removes its records
- Add info JSONB to import_log with inserted_keys and excluded_keys arrays
- Add get_import_log, get_all_import_logs, delete_import SQL functions
- Auto-apply transformations immediately after import
- Import UI: expandable key detail, checkbox selection, delete with confirm,
import ID column, transform result display
- New Log page: global import log across all sources
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add import_id column to records (links each record to its import batch)
- import_records() now stores readable dedup field values (not hashes) in
info.inserted_keys / info.excluded_keys, and stamps import_id on insert
- delete_import() simplified to delete log row; ON DELETE CASCADE removes records
- Add get_import_log() and get_all_import_logs() DB functions
- Add DELETE /api/sources/:name/import-log/:id endpoint
- Add GET /api/sources/import-log global log endpoint
- Import route now auto-applies transformations to new records after import
- Import page: show ID column, expandable key detail, checkbox delete
- New Log page: global view of all imports across sources
- Update README API reference and workflow
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add database/queries/{sources,rules,mappings,records}.sql — one file per
route, all business logic in PostgreSQL functions
- Replace parameterized queries in all four route files with lit()/jsonLit()
literal interpolation for debuggability
- Add api/lib/sql.js with lit(), jsonLit(), arr() helpers
- Fix get_view_data to use json_agg (preserves column order) with subquery
(guarantees sort order is respected before aggregation)
- Fix jsonLit() for JSONB params so plain strings become valid JSON
- Update manage.py option 3 to deploy database/queries/ instead of functions.sql
- Add SPEC.md covering architecture, philosophy, and manage.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Clicking a column header reloads from page 1 with ORDER BY col ASC/DESC
NULLS LAST passed to the view query. Sort column is validated against
information_schema.columns to prevent injection. Pagination preserves
the active sort across prev/next.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Support multi-capture-group regex: mappings.input_value changed to JSONB,
regexp_match() result stored as scalar or array JSONB in transformed column
- Computed expression fields in generated views: {fieldname} refs substituted
with (transformed->>'fieldname')::numeric for arithmetic in view columns
- Fix generate_source_view to DROP VIEW before CREATE (avoids column drop error)
- Collapsible rule cards that open directly to inline edit form
- Debounced live regex preview (extract + replace) with popout modal for 50 rows
- Records page now shows dfv.<source> view output instead of raw records
- Unified field table in Sources: single table with In view, Seq, expression columns
- Fix "Rule already exists" error when editing by passing rule.id directly to submit
- Fix Sources page clearing on F5 by watching sourceObj?.name in useEffect dep
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- POST /api/sources/suggest: derive source definition from CSV upload
- GET /api/sources/:name/import-log: query import history
- GET /api/rules/:id/test: test rule pattern against real records
- rules: add function_type (extract/replace) and flags columns
- get_unmapped_values: include up to 3 sample records per value
- npm start now uses nodemon for auto-reload
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
