Fast-path single-pass CTE when no chaining is needed

If all rules share one sequence value, skip the loop and temp table
and use the original single-pass CTE that the planner can fully
optimize. The loop path only runs when multiple distinct sequence
values exist (i.e. chaining is actually being used).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

.env.example

@@ -8,10 +8,3 @@ DB_PASSWORD=your_password_here
 # API Configuration
 API_PORT=3000
 NODE_ENV=development
-
-# SimpleFIN (optional — only needed for API-based bank feeds)
-# The access URL is the credential: claim a setup token once via
-# POST /api/sources/simplefin-claim and paste the result here.
-# A source picks its bridge with config.simplefin.access_url_env;
-# SIMPLEFIN_ACCESS_URL is the default.
-SIMPLEFIN_ACCESS_URL=https://user:pass@bridge.simplefin.org/simplefin

.gitignore

@@ -2,10 +2,10 @@
 .env
 
 # Dependencies
-# Lockfiles ARE tracked — they pin the Perspective 4.5.1/4.4.1 pairing that the
-# caret ranges in ui/package.json would otherwise let drift. See docs/perspective.md.
 node_modules/
 ui/node_modules/
+package-lock.json
+ui/package-lock.json
 
 # UI build output (generated — run `cd ui && npm run build`)
 public/
@@ -28,5 +28,8 @@ Thumbs.db
 *.swp
 *.swo
 
-# Scratch data exports
+# Uploads
-/*.tsv
+uploads/*
+!uploads/.gitkeep
+
+*.tsv

CLAUDE.md

@@ -2,122 +2,207 @@
 
 This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
 
-Dataflow imports CSV data, extracts structure from it with regex rules, maps the extracted
+## Overview
-values to standardized output, and serves the result over a REST API and React UI. It is a
-**simple system by design** — don't over-engineer it.
 
-**Read [docs/spec.md](docs/spec.md) for architecture, schema, data flow, the full API, and
+Dataflow is a simple data transformation tool for importing, cleaning, and standardizing data from various sources. Built with PostgreSQL and Node.js/Express, it emphasizes clarity and simplicity over complexity.
-`manage.py`.** This file covers only what you need to work in the repo without breaking
-something — the rules and non-obvious behaviours that aren't visible from the code.
 
-## Where things live
+## Core Concepts
 
-Both the API routes and the SQL are one file per resource: `api/routes/rules.js` and
+1. **Sources** - Define data sources and deduplication rules (which fields make a record unique)
-`database/rules.sql` are two halves of the same feature. Two SQL files are shared engines
+2. **Import** - Load CSV data, automatically deduplicating based on source rules
-rather than per-route — `import.sql` (CSV import, audit trail) and `transform.sql` (the
+3. **Rules** - Extract information using regex patterns (e.g., extract merchant from transaction description)
-rule/mapping engine, including the `jsonb_concat_obj` aggregate).
+4. **Mappings** - Map extracted values to standardized output (e.g., "WALMART" → {"vendor": "Walmart", "category": "Groceries"})
+5. **Transform** - Apply rules and mappings to create clean, enriched data
 
-`manage.py`'s `QUERY_FILES` list is the deploy order and the authoritative file list.
+## Architecture
 
-## Rules that matter
+### Database Schema (`database/schema.sql`)
 
-**`database/*.sql` is the source of truth for every database function. Never edit a function
+**5 simple tables:**
-directly in the database.** A live edit that isn't written back to the file is silently
+- `sources` - Source definitions with `constraint_fields` array
-reverted the next time anyone runs "Redeploy SQL functions". This has already happened once:
+- `records` - Imported data with `data` (raw) and `transformed` (enriched) JSONB columns
-five functions drifted and sat wrong in the repo for months — see the git history of the
+- `rules` - Regex extraction rules with `field`, `pattern`, `output_field`
-deleted `database/functions.sql`.
+- `mappings` - Input/output value mappings
+- `import_log` - Audit trail
 
-**Always run `npm run build` from `ui/` after any change to `ui/src/`.** The Express server
+**Key design:**
-serves the built output in `public/`; source changes are invisible until you rebuild.
+- JSONB for flexible data storage
+- Deduplication via MD5 hash of specified fields
+- Simple, flat structure (no complex relationships)
 
-**Never use `ON CONFLICT (constraint_key)`.** See deduplication below — there is no unique
+### Database Functions (`database/functions.sql`)
-constraint, and adding one would drop legitimate transactions.
 
-## The three data layers
+**4 focused functions:**
+- `import_records(source_name, data)` - Import with deduplication
+- `apply_transformations(source_name, record_ids)` - Apply rules and mappings
+- `get_unmapped_values(source_name, rule_name)` - Find values needing mappings
+- `reprocess_records(source_name)` - Re-transform all records
 
-Each row in `records` keeps its data in three JSONB columns:
+**Design principle:** Each function does ONE thing. No nested CTEs, no duplication.
 
-- `data` — raw imported values, never modified
+### API Server (`api/server.js` + `api/routes/`)
-- `transformed` — rule and mapping output only (the delta)
-- `overrides` — manual edits, highest precedence
 
-Readers merge them as `data || transformed || overrides`. Keeping them separate is what lets
+**RESTful endpoints:**
-`reprocess_records` re-run the rules without clobbering a manual edit. Anything that writes
+- `/api/sources` - CRUD sources, import CSV, trigger transformations
-overrides into `transformed` is a bug — that was the pre-May-2026 behaviour.
+- `/api/rules` - CRUD transformation rules
+- `/api/mappings` - CRUD value mappings, view unmapped values
+- `/api/records` - Query and search transformed data
 
-## Deduplication
+**Route files:**
+- `routes/sources.js` - Source management and CSV import
+- `routes/rules.js` - Rule management
+- `routes/mappings.js` - Mapping management + unmapped values
+- `routes/records.js` - Record queries and search
 
-- `constraint_key` is a JSONB object of the constraint field values — readable, no hashing
+## Common Development Tasks
-- Dedup is enforced at import time in a CTE. There is **no unique DB constraint** on it
-- **The constraint key is cross-batch re-import protection, not record uniqueness**
-- Within one import batch, all rows insert even when constraint keys collide. Banks
-  legitimately send identical-looking transactions — 11 separate Cedar Point charges on the
-  same day are 11 real rows
-- On re-import of an overlapping date range, rows whose key already exists are skipped, so
-  re-running a month-to-date export the next day doesn't double-count
-- Deleting an import log entry cascades to every record in that batch (`import_id` FK)
 
-## Error handling
+### Running the Application
 
-API routes use `try/catch` and pass errors to `next(err)`; `server.js` has a global handler.
+```bash
-Database functions return JSON with a `success` boolean.
+# Setup (first time only)
+./setup.sh
 
-## Light / dark mode
+# Start development server with auto-reload
+npm run dev
 
-Theme state lives in `ui/src/theme.jsx` — a React context (`ThemeContext`) with a
+# Start production server
-`ThemeProvider` that wraps the app in `main.jsx`.
+npm start
 
-- **Storage key:** `df_dark` in `localStorage`; falls back to `window.matchMedia('(prefers-color-scheme: dark)')` on first visit
+# Test API
-- **Toggle:** button at the foot of the sidebar (`Sidebar.jsx`), and in `BottomNav.jsx` on mobile; the effect writes `localStorage` and toggles the `.dark` class on `<html>`
+curl http://localhost:3000/health
-- **CSS:** `ui/src/index.css` declares semantic tokens under `@theme` (`bg-surface`, `text-ink`, `text-muted`, `border-line`, `text-danger`, …) that resolve to CSS custom properties redefined by `.dark`. **Write components against the tokens, never against literal shades like `bg-white` or `text-gray-400`** — the old per-utility `.dark .bg-white { … }` overrides are gone and must not come back
+```
-- **Palette:** dark mode uses Perspective's "Pro Dark" colours (`--bg-primary: #242526`, panels `#2a2c2f`, gridlines `#3b3f46`, text `#c5c9d0`)
-- **Perspective viewer:** `Pivot.jsx` calls `viewer.setAttribute('theme', dark ? 'Pro Dark' : 'Pro Light')` on initial load and in a `useEffect([dark])` so the viewer stays in sync when the toggle fires
-- **Consuming the theme:** `import useTheme from '../theme.jsx'` then `const { dark, setDark } = useTheme()`
 
-## Pivot inspector panel
+### Database Changes
 
-Clicking a data cell opens a right-hand inspector panel showing the underlying transactions
+When modifying schema:
-for that cell. See [docs/perspective.md](docs/perspective.md) for the Perspective API itself.
+1. Edit `database/schema.sql`
+2. Drop and recreate schema: `psql -d dataflow -f database/schema.sql`
+3. Redeploy functions: `psql -d dataflow -f database/functions.sql`
 
-- **Toggle**: clicking the same cell again closes the panel. The toggle key is `JSON.stringify({ p: row.__ROW_PATH__, c: column_names })` — stable across source and stack views.
+For production, write migration scripts instead of dropping schema.
-- **Listener cleanup**: the `perspective-click` handler is stored in `perspClickHandlerRef` and removed via `removeEventListener` on effect cleanup. Without this, switching views accumulates duplicate listeners that fire multiple times per click.
-- **split_by filter derivation**: `detail.config.filter` from the click event may omit split_by column constraints. They are derived from `column_names` positionally (`column_names[i]` matches `config.split_by[i]`) and appended to the filter before querying.
-- **Row filtering**: a temporary `table.view({ filter, expressions })` is used so Perspective evaluates expression/computed columns correctly. Falls back to JS-side `filterRowsByConfig` on error (which skips filters for fields not in raw data).
-- The panel is resizable via a drag handle on its left edge (`paneWidth` state, min 240px).
-- The transaction table is sortable (click header) and shows column totals for all-numeric columns.
 
-## Pivot layout persistence
+### Adding a New API Endpoint
 
-Named layouts are stored in `dataflow.pivot_layouts` for both sources and stacks. The
+1. Add route to appropriate file in `api/routes/`
-`source_name` column holds either a source name or a stack name — the FK to `sources(name)`
+2. Follow existing patterns (async/await, error handling via `next()`)
-was dropped to allow this. Source layouts use `/api/sources/:name/layouts`; stack layouts use
+3. Use parameterized queries to prevent SQL injection
-`/api/stacks/:name/layouts`. Both call the same DB functions (`list_pivot_layouts`,
+4. Return consistent JSON format
-`save_pivot_layout`, `delete_pivot_layout`). `localStorage` still remembers the *last active
-layout* for a view (the `psp_layout_<name>` key), but the definitions live in the DB so they
-persist across machines.
 
-## Adding features
+### Testing
 
-- One function, one job; keep functions under 100 lines
+Manual testing workflow:
-- Write clear SQL, not clever SQL
+1. Create a source: `POST /api/sources`
-- Add the SQL function to the matching `database/*.sql` file, then the route that calls it
+2. Create rules: `POST /api/rules`
-- Update `docs/spec.md` when you add or change an endpoint
+3. Import data: `POST /api/sources/:name/import`
+4. Apply transformations: `POST /api/sources/:name/transform`
+5. View results: `GET /api/records/source/:name`
+
+See `examples/GETTING_STARTED.md` for complete curl examples.
+
+## Design Principles
+
+1. **Simple over clever** - Straightforward code beats optimization
+2. **Explicit over implicit** - No magic, no hidden triggers
+3. **Clear naming** - `data` not `rec`, `transformed` not `allj`
+4. **One function, one job** - No 250-line functions
+5. **JSONB for flexibility** - Handle varying schemas without migrations
+
+## Common Patterns
+
+### Import Flow
+```
+CSV file → parse → import_records() → records table (data column)
+```
+
+### Transformation Flow
+```
+records.data → apply_transformations() →
+  - Apply each rule (regex extraction)
+  - Look up mappings
+  - Merge into records.transformed
+```
+
+### Deduplication
+- `constraint_key` is a JSONB object of the constraint field values (readable, no hashing)
+- Dedup is enforced at import time via CTE — no unique DB constraint
+- Intra-file duplicate rows are allowed (bank may send identical rows); they all insert
+- On re-import, all rows whose constraint_key already exists in the DB are skipped
+- Deleting an import log entry cascades to all records from that batch (import_id FK)
+
+### Error Handling
+- API routes use `try/catch` and pass errors to `next(err)`
+- Server.js has global error handler
+- Database functions return JSON with `success` boolean
+
+## File Structure
+
+```
+dataflow/
+├── database/
+│   ├── schema.sql         # Table definitions
+│   └── functions.sql      # Import/transform functions
+├── api/
+│   ├── server.js          # Express server
+│   └── routes/            # API endpoints
+│       ├── sources.js
+│       ├── rules.js
+│       ├── mappings.js
+│       └── records.js
+├── examples/
+│   ├── GETTING_STARTED.md # Tutorial
+│   └── bank_transactions.csv
+├── .env.example           # Config template
+├── package.json
+└── README.md
+```
+
+## Comparison to Legacy TPS System
+
+This project replaces an older system (in `/opt/tps`) that had:
+- 2,150 lines of complex SQL with heavy duplication
+- 5 nearly-identical 200+ line functions
+- Confusing names and deep nested CTEs
+- Complex trigger-based processing
+
+Dataflow achieves the same functionality with:
+- ~400 lines of simple SQL
+- 4 focused functions
+- Clear names and linear logic
+- Explicit API-triggered processing
+
+The simplification makes it easy to understand, modify, and maintain.
 
 ## Troubleshooting
 
-**Database connection fails** — check `.env` credentials, that PostgreSQL is running, and
+**Database connection fails:**
-that the search path resolves to the `dataflow` schema.
+- Check `.env` file exists and has correct credentials
+- Verify PostgreSQL is running: `psql -U postgres -l`
+- Check search path is set: Should default to `dataflow` schema
 
-**Import succeeds but transformation does nothing** — check rules exist for that source
+**Import succeeds but transformation fails:**
-(`SELECT * FROM dataflow.rules WHERE source_name = '…'`), that `field` matches an actual key
+- Check rules exist: `SELECT * FROM dataflow.rules WHERE source_name = 'xxx'`
-in `data`, and test the pattern with `GET /api/rules/preview`.
+- Verify field names match CSV columns
+- Test regex pattern manually
+- Check for SQL errors in logs
 
-**Everything is marked duplicate** — `constraint_fields` probably don't match the real field
+**All records marked as duplicates:**
-names, or the batch was already imported.
+- Verify `constraint_fields` match actual field names in data
+- Check if data was already imported
+- Use different source name for testing
 
-## History
+## Adding New Features
 
-This replaces an older system still in `/opt/tps` — 2,150 lines of SQL with five
+When adding features, follow these principles:
-nearly-identical 200-line functions and trigger-based processing. Dataflow is a clean
+- Add ONE function that does ONE thing
-rewrite, not a refactor. Some function bodies still carry `mirrors TPS …` comments pointing
+- Keep functions under 100 lines if possible
-at their counterpart there.
+- Write clear SQL, not clever SQL
+- Add API endpoint that calls the function
+- Document in README.md and update examples
+
+## Notes for Claude
+
+- This is a **simple** system by design - don't over-engineer it
+- Keep functions focused and linear
+- Use JSONB for flexibility, not as a crutch for bad design
+- When confused, read the examples/GETTING_STARTED.md walkthrough
+- The old TPS system is in `/opt/tps` - this is a clean rewrite, not a refactor

README.md

@@ -2,71 +2,198 @@
 
 A simple data transformation tool for importing, cleaning, and standardizing data from various sources.
 
-Point it at a messy CSV — bank transactions, product lists, anything repetitive — and it will
+## What It Does
-deduplicate on import, pull structure out with regex rules, map the extracted values to clean
-output, and serve the result through a web UI and REST API.
 
-## How it works
+Dataflow helps you:
+1. **Import** CSV data with automatic deduplication
+2. **Transform** data using regex rules to extract meaningful information
+3. **Map** extracted values to standardized output
+4. **Query** the transformed data via a web UI or REST API
 
-1. **Sources** define where data comes from and which fields make a record unique
+Perfect for cleaning up messy data like bank transactions, product lists, or any repetitive data that needs normalization.
-2. **Rules** extract information with regex (`extract` or `replace` mode) —
-   e.g. pull the merchant out of a transaction description
-3. **Mappings** turn extracted values into clean output —
-   `"DISCOUNT DRUG MART 32"` → `{"vendor": "Discount Drug Mart", "category": "Healthcare"}`
-4. **Records** are then queryable, pivotable, and exportable
 
-Each record keeps three layers: `data` (raw import), `transformed` (rule and mapping output),
+## Core Concepts
-and `overrides` (manual edits). Reads merge them in that order, so re-running the rules never
-clobbers something you typed by hand.
 
-## Stack
+### 1. Sources
+Define where data comes from and how to deduplicate it.
 
-PostgreSQL with JSONB storage, a Node.js/Express API, and a React SPA served from `public/`.
+**Example:** Bank transactions deduplicated by date + amount + description
-HTTP Basic auth, configured in `.env`.
 
-## Getting started
+### 2. Rules
+Extract information using regex patterns (`extract` or `replace` modes).
 
-Requires PostgreSQL 12+, Node.js 18+, and Python 3.
+**Example:** Extract merchant name from transaction description
 
+### 3. Mappings
+Map extracted values to clean, standardized output.
+
+**Example:** "DISCOUNT DRUG MART 32" → `{"vendor": "Discount Drug Mart", "category": "Healthcare"}`
+
+## Architecture
+
+- **Database:** PostgreSQL with JSONB for flexible data storage
+- **API:** Node.js/Express REST API
+- **UI:** React SPA served from `public/`
+- **Auth:** HTTP Basic auth (configured in `.env`)
+
+## Design Principles
+
+- **Simple & Clear** - Easy to understand what's happening
+- **Explicit** - No hidden magic or complex triggers
+- **Flexible** - Handle varying data formats without schema changes
+
+## Getting Started
+
+### Prerequisites
+- PostgreSQL 12+
+- Node.js 16+
+- Python 3 (for `manage.py`)
+
+### Installation
+
+1. Install Node dependencies:
 ```bash
 npm install
-python3 manage.py     # interactive setup: .env, database, schema, functions, UI, service
 ```
 
-The UI is then at `http://localhost:3020` and the API at `http://localhost:3020/api`
+2. Run the management script to configure and deploy everything:
-(port set by `API_PORT` in `.env`).
+```bash
+python3 manage.py
+```
 
-For a walkthrough that creates a source, adds rules and mappings, and imports the sample
+For development with auto-reload:
-CSV in `examples/`, see **[docs/getting-started.md](docs/getting-started.md)**.
+```bash
+npm run dev
+```
 
-## Documentation
+The UI is available at `http://localhost:3000`. The API is at `http://localhost:3000/api`.
 
-| | |
+## Management Script (`manage.py`)
-|---|---|
-| **[docs/getting-started.md](docs/getting-started.md)** | Tutorial — build a working pipeline from scratch with curl |
-| **[docs/spec.md](docs/spec.md)** | Full reference — architecture, schema, data flow, API, `manage.py` |
-| **[docs/ui.md](docs/ui.md)** | Frontend: React + Vite build, key packages |
-| **[docs/perspective.md](docs/perspective.md)** | Pivot table: pinned versions and API reference |
 
-## Project structure
+`manage.py` is an interactive tool for configuring, deploying, and managing the service. Run it and choose from the numbered menu:
+
+```
+python3 manage.py
+```
+
+| # | Action |
+|---|--------|
+| 1 | **Database configuration** — create/update `.env`, optionally create the PostgreSQL user/database, and deploy schema + functions |
+| 2 | Redeploy schema only (`database/schema.sql`) — drops and recreates all tables |
+| 3 | Redeploy SQL functions only (`database/queries/`) |
+| 4 | Build UI (`ui/` → `public/`) |
+| 5 | Set up nginx reverse proxy (HTTP or HTTPS via certbot) |
+| 6 | Install systemd service unit (`dataflow.service`) |
+| 7 | Start / restart `dataflow.service` |
+| 8 | Stop `dataflow.service` |
+| 9 | Set login credentials (`LOGIN_USER` / `LOGIN_PASSWORD_HASH` in `.env`) |
+
+The status screen at the top of the menu shows the current state of each component (database connection, schema, UI build, service, nginx).
+
+**Typical first-time setup:** run options 1 → 4 → 9 → 6 → 7 (→ 5 if you want nginx).
+
+## API Reference
+
+All `/api` routes require HTTP Basic authentication.
+
+### Sources — `/api/sources`
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/sources` | List all sources |
+| POST | `/api/sources` | Create a source |
+| GET | `/api/sources/:name` | Get a source |
+| PUT | `/api/sources/:name` | Update a source |
+| DELETE | `/api/sources/:name` | Delete a source |
+| POST | `/api/sources/suggest` | Suggest source definition from CSV upload |
+| POST | `/api/sources/:name/import` | Import CSV data and auto-apply transformations to new records |
+| GET | `/api/sources/:name/import-log` | View import history (includes `inserted_keys` / `excluded_keys` in `info`) |
+| DELETE | `/api/sources/:name/import-log/:id` | Delete an import batch and all its records |
+| POST | `/api/sources/:name/transform` | Apply rules and mappings to any untransformed records |
+| POST | `/api/sources/:name/reprocess` | Re-transform all records |
+| GET | `/api/sources/:name/fields` | List all known field names |
+| GET | `/api/sources/:name/stats` | Get record and mapping counts |
+| POST | `/api/sources/:name/view` | Generate output view |
+| GET | `/api/sources/:name/view-data` | Query output view (paginated, sortable) |
+
+### Rules — `/api/rules`
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/rules/source/:source_name` | List rules for a source |
+| POST | `/api/rules` | Create a rule |
+| GET | `/api/rules/:id` | Get a rule |
+| PUT | `/api/rules/:id` | Update a rule |
+| DELETE | `/api/rules/:id` | Delete a rule |
+| GET | `/api/rules/preview` | Preview a pattern against real records (ad-hoc) |
+| GET | `/api/rules/:id/test` | Test a saved rule against real records |
+
+### Mappings — `/api/mappings`
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/mappings/source/:source_name` | List mappings |
+| POST | `/api/mappings` | Create a mapping |
+| POST | `/api/mappings/bulk` | Bulk create/update mappings |
+| GET | `/api/mappings/:id` | Get a mapping |
+| PUT | `/api/mappings/:id` | Update a mapping |
+| DELETE | `/api/mappings/:id` | Delete a mapping |
+| GET | `/api/mappings/source/:source_name/unmapped` | Get values with no mapping yet |
+| GET | `/api/mappings/source/:source_name/all-values` | All extracted values with counts |
+| GET | `/api/mappings/source/:source_name/counts` | Record counts for existing mappings |
+| GET | `/api/mappings/source/:source_name/export.tsv` | Export values as TSV |
+| POST | `/api/mappings/source/:source_name/import-csv` | Import mappings from TSV |
+
+### Records — `/api/records`
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/records/source/:source_name` | List records (paginated) |
+| GET | `/api/records/:id` | Get a single record |
+| POST | `/api/records/search` | Search records |
+| DELETE | `/api/records/:id` | Delete a record |
+| DELETE | `/api/records/source/:source_name/all` | Delete all records for a source |
+
+## Typical Workflow
+
+```
+1. Create a source (POST /api/sources)
+2. Create transformation rules (POST /api/rules)
+3. Import CSV data (POST /api/sources/:name/import) — transformations applied automatically to new records
+4. Preview rules against real data (GET /api/rules/preview)
+5. Review unmapped values (GET /api/mappings/source/:name/unmapped)
+6. Add mappings (POST /api/mappings or bulk import via TSV)
+7. Reprocess to apply new mappings (POST /api/sources/:name/reprocess)
+8. Query results (GET /api/sources/:name/view-data)
+```
+
+See `examples/GETTING_STARTED.md` for a complete walkthrough with curl examples.
+
+## Project Structure
 
 ```
 dataflow/
-├── manage.py           # interactive setup / deploy / uninstall
+├── database/
-├── database/           # schema.sql + one .sql file per API route
+│   ├── schema.sql         # Table definitions
-├── api/                # Express server, routes, auth middleware
+│   └── functions.sql      # Import/transform/query functions
-├── ui/                 # React source (built to public/)
+├── api/
-├── public/             # built UI, served as static files
+│   ├── server.js          # Express server
-├── docs/
+│   ├── middleware/
-└── examples/           # sample CSV for the tutorial
+│   │   └── auth.js        # Basic auth middleware
+│   ├── lib/
+│   │   └── sql.js         # SQL literal helpers
+│   └── routes/
+│       ├── sources.js
+│       ├── rules.js
+│       ├── mappings.js
+│       └── records.js
+├── public/                # Built React UI (served as static files)
+├── examples/
+│   ├── GETTING_STARTED.md
+│   └── bank_transactions.csv
+└── .env.example
 ```
 
-Both the API routes and the SQL are organized one file per resource, so `api/routes/rules.js`
-and `database/rules.sql` are the two halves of the same feature.
-
-`database/*.sql` is the source of truth for every database function — never edit one directly
-in the database, or the next redeploy will silently revert it.
-
 ## License
 
 MIT

SPEC.md

@@ -18,7 +18,7 @@ SQL functions are the single source of truth for business logic. The API layer i
 Database calls in the route files use fully formed SQL strings with values interpolated directly (not parameterized). This makes every query copy-pasteable into psql for debugging. A small `lit()` helper in `api/lib/sql.js` handles quoting and escaping. This is an intentional trade-off: the tool is internal, and debuggability is worth more than the marginal injection protection parameterization provides over what `lit()` already does.
 
 ### One SQL file per route
-SQL is organized in `database/` with one file per route (`sources.sql`, `rules.sql`, `mappings.sql`, `records.sql`, `stacks.sql`, `status.sql`) plus `import.sql` and `transform.sql` for the import/transform engine. This makes it easy to find the SQL behind any API endpoint — look at the route file to find the function name, then look at the matching query file for the implementation.
+SQL is organized in `database/queries/` with one file per route (`sources.sql`, `rules.sql`, `mappings.sql`, `records.sql`). This makes it easy to find the SQL behind any API endpoint — look at the route file to find the function name, then look at the matching query file for the implementation.
 
 ### Explicit over implicit
 Nothing happens automatically. Transformations are triggered by the user. Views are generated on demand. There are no database triggers, no background workers, no scheduled jobs.
@@ -34,52 +34,36 @@ Raw imported records and transformed records are stored as JSONB. This avoids sc
 manage.py              — interactive CLI for setup, deployment, and management
 database/
   schema.sql           — table definitions (run once or to reset)
-  sources.sql          — all SQL for /api/sources
+  queries/
-  rules.sql            — all SQL for /api/rules
+    sources.sql        — all SQL for /api/sources
-  mappings.sql         — all SQL for /api/mappings
+    rules.sql          — all SQL for /api/rules
-  records.sql          — all SQL for /api/records
+    mappings.sql       — all SQL for /api/mappings
-  stacks.sql           — all SQL for /api/stacks
+    records.sql        — all SQL for /api/records
-  status.sql           — all SQL for /api/status
-  import.sql           — CSV import and the import audit trail
-  transform.sql        — the rule/mapping engine
 api/
   server.js            — Express server, mounts routes, auth middleware
   middleware/
     auth.js            — Basic Auth enforcement on all /api routes
   lib/
     sql.js             — lit() and arr() helpers for SQL literal building
-    simplefin.js       — SimpleFIN Bridge client for bank transaction pulls
   routes/
     sources.js         — HTTP handlers for source management
     rules.js           — HTTP handlers for rule management
     mappings.js        — HTTP handlers for mapping management
     records.js         — HTTP handlers for record queries
-    stacks.js          — HTTP handlers for stack management
-    status.js          — HTTP handler for deployment status
 ui/
   src/
     api.js             — fetch wrapper, credential management
-    App.jsx            — root: login gate, routing, stale/reprocess banners
+    App.jsx            — root: login gate, sidebar, source selector, routing
-    index.css          — semantic colour tokens for light and dark
     pages/
       Login.jsx        — username/password form
-      SourceList.jsx   — source list and the create dialog
+      Sources.jsx      — source CRUD, field config, view generation
-      SourceDetail.jsx — one source: connection, fields, view, maintenance
+      Import.jsx       — CSV upload and import log
-      Bridge.jsx       — SimpleFIN accounts, balances, and subtotals
-      ImportHub.jsx    — all sources with sync / upload actions
-      Import.jsx       — CSV upload, SimpleFIN sync, and import log
       Rules.jsx        — rule CRUD with live pattern preview
       Mappings.jsx     — mapping table with TSV import/export
       Records.jsx      — paginated, sortable view of transformed records
       Pivot.jsx        — interactive pivot table with cell inspector
-      Stacks.jsx       — multi-source union views with running balance
-      Remap.jsx        — bulk remap of an output field value across mappings
       Log.jsx          — global import log across all sources
-    components/        — Sidebar, BottomNav, navItems, SourceTabs, Section, SampleTable
-    theme.jsx          — light/dark context provider
 public/                — compiled UI (output of npm run build in ui/)
-docs/                  — this file, tutorial, UI and Perspective references
-examples/              — bank_transactions.csv, the tutorial's sample data
 ```
 
 ---
@@ -117,45 +101,6 @@ CSV file → parse in Node.js → import_records(source, data)
   → apply_transformations() runs automatically on new records
 ```
 
-### SimpleFIN sync (API-based bank feeds)
-```
-POST /api/sources/:name/sync → api/lib/simplefin.js
-  → GET {access_url}/accounts?account=…&start-date=… (Basic auth)
-  → drop pending, flatten transactions, fold in account context
-  → import_records(source, data) — identical path to a CSV import from here on
-```
-An alternative to CSV upload for sources that read from a bank API. Only the
-fetching differs: dedup, logging, and transformation are the same code.
-
-- **Authentication.** A SimpleFIN access URL *is* the credential — it carries
-  its own username and password (`https://user:pass@bridge.simplefin.org/simplefin`).
-  You claim it once from a setup token (`POST /api/sources/simplefin-claim`,
-  which consumes the token) and store it in `.env`, one variable per bridge. It
-  is deliberately **not** stored in the database, which `manage.py` offers to reset.
-- **Source config.** A source opts in by having `simplefin` in its `config` JSONB:
-  `{"simplefin": {"account_id": "ACT-…", "access_url_env": "SIMPLEFIN_ACCESS_URL",
-  "days": 10}}`. Only `account_id` is required. `GET /api/sources/simplefin-accounts`
-  lists the accounts behind a bridge so you can find the id.
-- **`constraint_fields` should be `['id']`.** SimpleFIN assigns each transaction a
-  stable id, which makes overlapping pulls free and — unlike date + amount +
-  description — keeps genuinely repeated charges as separate records.
-- **Pending transactions are skipped** (`?include_pending=true` overrides). A
-  pending transaction gets a different id once it posts, so importing it would
-  produce a duplicate under a different key a day or two later.
-- **Bridge errors are surfaced, not swallowed.** SimpleFIN returns HTTP 200 with
-  an `errors` array when an institution is failing. Those errors ride along in
-  the sync response so a broken connection doesn't read as a successful empty
-  pull; the Import page shows them in orange above the counts.
-- **Refresh cadence and the 90-day wall.** The bridge polls banks roughly daily
-  and transactions can take a few days to appear, so the pull asks for a rolling
-  window (`days`, default 10) rather than tracking a cursor. The window has hard
-  limits: SimpleFIN caps any range at 90 days and advises staying under 45, so
-  `MAX_DAYS` is 89 and `days=0` or anything larger clamps to it. A `start-date`
-  is **always** sent — omitting it does not mean "everything available", it
-  returns only the few most recent transactions.
-- **Cron.** A daily pull is just the endpoint:
-  `curl -sS -u user:pass -X POST http://localhost:3000/api/sources/NAME/sync`
-
 ### Transform
 ```
 apply_transformations(source) — pure SQL CTE
@@ -176,7 +121,7 @@ The transform is fully set-based — no row-by-row loops. All records for a sour
 
 ## SQL Functions
 
-Each route file has a matching SQL file in `database/`; `import.sql` and `transform.sql` hold the engine shared by several routes.
+Each file in `database/queries/` maps 1-to-1 with a route file.
 
 **sources.sql**
 `list_sources`, `get_source`, `create_source`, `update_source`, `delete_source`, `get_import_log`, `get_source_stats`, `get_source_fields`, `get_view_data` (plpgsql — dynamic sort via EXECUTE + quote_ident), `import_records`, `jsonb_merge` + `jsonb_concat_obj` aggregate, `apply_transformations`, `reprocess_records`, `generate_source_view`
@@ -190,12 +135,6 @@ Each route file has a matching SQL file in `database/`; `import.sql` and `transf
 **records.sql**
 `list_records`, `get_record`, `search_records` (JSONB containment on data and transformed), `delete_record`, `delete_source_records`
 
-**stacks.sql**
-`list_stacks`, `get_stack`, `create_stack`, `update_stack`, `delete_stack`, `get_stack_view_data` (union of source views with field mapping and running balance), `list_pivot_layouts`, `save_pivot_layout`, `delete_pivot_layout`
-
-**status.sql**
-`get_status` — returns deployment state (schema version, function presence, service status)
-
 ---
 
 ## API
@@ -206,107 +145,43 @@ All routes are under `/api`. Every route requires HTTP Basic Auth. The `GET /hea
 
 **Route summary:**
 
-### Sources — `api/routes/sources.js`
-
 | Method | Path | Description |
 |--------|------|-------------|
 | GET | /api/sources | List all sources |
 | POST | /api/sources | Create source |
 | GET | /api/sources/:name | Get source |
-| PUT | /api/sources/:name | Update source (constraint_fields, config, global_picklist) |
+| PUT | /api/sources/:name | Update source (constraint_fields, config) |
-| DELETE | /api/sources/:name | Delete source and all its data |
+| DELETE | /api/sources/:name | Delete source and all data |
-| POST | /api/sources/suggest | Suggest source config from an uploaded CSV |
+| POST | /api/sources/suggest | Suggest source config from CSV upload |
-| POST | /api/sources/:name/import | Import CSV; transformations are applied to the new records |
+| POST | /api/sources/:name/import | Import CSV records |
-| POST | /api/sources/:name/sync | Pull transactions from SimpleFIN and import them (`?days=`, `?include_pending=`) |
+| GET | /api/sources/:name/import-log | Import history |
-| GET | /api/sources/simplefin-accounts | List accounts behind a bridge (`?access_url_env=`) |
-| POST | /api/sources/simplefin-claim | Exchange a setup token for a permanent access URL |
-| GET | /api/sources/import-log | Import history across all sources |
-| GET | /api/sources/:name/import-log | Import history for one source |
-| DELETE | /api/sources/:name/import-log/:id | Delete an import batch and every record in it |
-| POST | /api/sources/:name/transform | Apply transformations to untransformed records only |
-| POST | /api/sources/:name/reprocess | Reapply transformations to all records |
 | GET | /api/sources/:name/stats | Record counts |
-| GET | /api/sources/:name/fields | All known field names and their origins |
+| GET | /api/sources/:name/fields | All known field names and origins |
-| GET | /api/sources/:name/override-keys | Distinct field names used in overrides for this source |
+| GET | /api/sources/:name/view-data | Paginated, sortable view data |
-| POST | /api/sources/:name/view | Generate/refresh the `dfv` view |
+| POST | /api/sources/:name/transform | Apply transformations (new records only) |
-| GET | /api/sources/:name/view-data | Paginated, sortable, filterable view data |
+| POST | /api/sources/:name/reprocess | Reapply transformations to all records |
-| GET | /api/sources/:name/layouts | List saved pivot layouts |
+| POST | /api/sources/:name/view | Generate dfv view |
-| POST | /api/sources/:name/layouts | Save a pivot layout |
-| DELETE | /api/sources/:name/layouts/:id | Delete a pivot layout |
-
-### Rules — `api/routes/rules.js`
-
-| Method | Path | Description |
-|--------|------|-------------|
 | GET | /api/rules/source/:name | List rules for a source |
-| GET | /api/rules/:id | Get a rule |
+| GET | /api/rules/preview | Preview pattern against live records |
+| GET | /api/rules/:id/test | Test saved rule against live records |
 | POST | /api/rules | Create rule |
 | PUT | /api/rules/:id | Update rule |
 | DELETE | /api/rules/:id | Delete rule |
-| GET | /api/rules/preview | Preview an ad-hoc pattern against live records |
-| GET | /api/rules/:id/test | Test a saved rule against live records |
-
-### Mappings — `api/routes/mappings.js`
-
-| Method | Path | Description |
-|--------|------|-------------|
 | GET | /api/mappings/source/:name | List mappings |
-| GET | /api/mappings/:id | Get a mapping |
+| GET | /api/mappings/source/:name/all-values | All extracted values (mapped + unmapped) |
+| GET | /api/mappings/source/:name/unmapped | Only unmapped extracted values |
+| GET | /api/mappings/source/:name/counts | Record counts per mapping |
+| GET | /api/mappings/source/:name/export.tsv | Export mappings as TSV |
+| POST | /api/mappings/source/:name/import-csv | Import/update mappings from TSV |
 | POST | /api/mappings | Create mapping |
 | POST | /api/mappings/bulk | Upsert multiple mappings |
 | PUT | /api/mappings/:id | Update mapping |
 | DELETE | /api/mappings/:id | Delete mapping |
-| GET | /api/mappings/source/:name/all-values | All extracted values (mapped + unmapped) with counts |
+| GET | /api/records/source/:name | List raw records |
-| GET | /api/mappings/source/:name/unmapped | Only values with no mapping yet |
+| GET | /api/records/:id | Get single record |
-| GET | /api/mappings/source/:name/counts | Record counts per mapping |
-| GET | /api/mappings/source/:name/export.tsv | Export extracted values as TSV |
-| POST | /api/mappings/source/:name/import-csv | Import/update mappings from an uploaded TSV |
-| GET | /api/mappings/global-values | Output values across all `global_picklist` sources (autocomplete) |
-| GET | /api/mappings/outputs | Search output field values across all mappings |
-| GET | /api/mappings/outputs/:col/:val | Mappings carrying a specific output field value |
-| POST | /api/mappings/remap-field | Replace an output field value across all mappings |
-
-### Records — `api/routes/records.js`
-
-| Method | Path | Description |
-|--------|------|-------------|
-| GET | /api/records/source/:name | List records (paginated) |
-| GET | /api/records/:id | Get a single record |
 | POST | /api/records/search | Search by JSONB containment |
 | DELETE | /api/records/:id | Delete record |
 | DELETE | /api/records/source/:name/all | Delete all records for a source |
-| PUT | /api/records/:id/overrides | Set manual overrides on a record |
-| DELETE | /api/records/:id/overrides | Clear a record's overrides |
-| POST | /api/records/bulk-overrides | Apply the same overrides to many records |
-
-### Stacks — `api/routes/stacks.js`
-
-| Method | Path | Description |
-|--------|------|-------------|
-| GET | /api/stacks | List all stacks |
-| GET | /api/stacks/:name | Get a stack with its sources |
-| POST | /api/stacks | Create stack |
-| PUT | /api/stacks/:name | Update stack |
-| DELETE | /api/stacks/:name | Delete stack |
-| PUT | /api/stacks/:name/sources/:source | Add or update a source in the stack |
-| DELETE | /api/stacks/:name/sources/:source | Remove a source from the stack |
-| PUT | /api/stacks/:name/sources/reorder | Reorder the stack's sources |
-| GET | /api/stacks/:name/view-sql | Preview the SQL that would build the view (dry run) |
-| POST | /api/stacks/:name/view | Generate/refresh the `dfv` view |
-| POST | /api/stacks/:name/exec-sql | Execute user-edited SQL for the view |
-| GET | /api/stacks/:name/view-data | Paginated stacked data with running balance |
-| GET | /api/stacks/:name/balance | Current running balance from the generated view |
-| POST | /api/stacks/:name/calibrate | Set the balance offset from a known balance at a date |
-| GET | /api/stacks/:name/layouts | List saved pivot layouts |
-| POST | /api/stacks/:name/layouts | Save a pivot layout |
-| DELETE | /api/stacks/:name/layouts/:id | Delete a pivot layout |
-
-### Status — `api/routes/status.js`
-
-| Method | Path | Description |
-|--------|------|-------------|
-| GET | /api/status | Deployment status |
-| GET | /health | Health check (no auth) |
 
 ---
 
@@ -331,43 +206,11 @@ Built with React + Vite + Tailwind CSS. Compiled output goes to `public/`. The s
 3. On 401 response, credentials are cleared and the login screen is shown.
 4. `localStorage` persists the selected source name across sessions.
 
-**Navigation.** The selected source is a route parameter, not global state: `/sources`
-lists sources, `/sources/:name` owns one, and Import, Rules, Mappings, Records, and Pivot
-are tabs beneath it (`components/SourceTabs.jsx`). The sidebar holds only top-level
-destinations — Sources, Import, Bridge, Remap, Stacks, Log — defined once in
-`components/navItems.jsx` and rendered by `Sidebar.jsx` on desktop and `BottomNav.jsx`
-below the `md:` breakpoint. Out-of-sync and reprocess banners render above every page from
-`App.jsx`.
-
-**Colour.** Components use semantic tokens (`bg-surface`, `text-ink`, `text-muted`,
-`border-line`, `text-danger`) declared in `index.css` under `@theme`, which resolve to CSS
-variables redefined by `.dark`. There are no per-utility `.dark` override rules; a new
-component gets both themes for free.
-
-**Bundle.** `Pivot` is loaded with `React.lazy`, keeping Perspective (~4.7 MB gzipped) out
-of the initial download and in a chunk fetched only when a pivot is opened.
-
 **Pages:**
 
-- **Sources** (`SourceList.jsx`) — Lists every source with its constraint fields and a
+- **Sources** — View and edit source configuration. Shows all known field names and their origins (raw data, schema, rules, mappings). Checkboxes control which fields are constraint fields and which appear in the output view. Supports CSV upload to auto-detect fields.
-  badge for bank feeds; clicking opens it. "New source" opens the create dialog, which can
-  seed fields from a CSV sample or from a linked SimpleFIN account.
 
-- **Source detail** (`SourceDetail.jsx`) — The Setup tab, grouped into titled panels:
+- **Import** — Upload a CSV to import records into the selected source. Transformations run automatically on new records. Shows import log with inserted/duplicate counts, expandable key detail, checkbox selection, and delete with confirmation.
-  Connection (link/unlink a SimpleFIN account, warns when constraint fields aren't `id`),
-  Fields and view (all known field names and their origins, with checkboxes for constraint
-  fields and view columns), Sample rows, Maintenance (reprocess), and Delete source.
-
-- **Bridge** (`Bridge.jsx`) — Every account behind the SimpleFIN credential with balances,
-  the source each maps to, and subtotals split into banking versus retirement (recognised by
-  keyword on the account and institution names). Queries SimpleFIN only when Refresh is
-  pressed. Balances render in accounting style with negatives in parentheses.
-
-- **Import** (`ImportHub.jsx`) — Top-level entry point for the frequent job. Lists every
-  source with record counts and last import date, a Sync button for bank feeds, and an
-  upload link for CSV sources.
-
-- **Source › Import tab** — Upload a CSV to import records into the selected source. Transformations run automatically on new records. Shows import log with inserted/duplicate counts, expandable key detail, checkbox selection, and delete with confirmation. Sources with `config.simplefin.account_id` also get a Sync panel — a window selector (10/30/45 days or a full backfill) and a "Sync now" button that pulls from the bank API through the same import path.
 
 - **Rules** — Create and manage regex rules. Live preview fires automatically (debounced 500ms) as pattern/field/flags are edited, showing match results against real records. Rules can be enabled/disabled by toggle.
 
@@ -375,11 +218,11 @@ of the initial download and in a chunk fetched only when a pivot is opened.
 
 - **Records** — Paginated table showing the `dfv.{source}` view. Server-side sorting (column validated against `information_schema.columns`, interpolated with `quote_ident`). Dates are formatted `YYYY-MM-DD` for correct lexicographic sort. Regex filters can be added per column. If the view cast fails (e.g. a field typed as `date` contains text), the error is shown inline rather than a blank page.
 
-- **Pivot** — Interactive pivot/crosstab powered by [Perspective](https://perspective.finos.org/) (`@perspective-dev` client/viewer/datagrid v4.5.1, viewer-d3fc v4.4.1 — installed via npm). Loads all rows from the source view into an in-browser Perspective worker and renders a `<perspective-viewer>` web component. Supports grouping, splitting, filtering, sorting, and charting interactively.
+- **Pivot** — Interactive pivot/crosstab powered by [Perspective](https://perspective.finos.org/) (`@perspective-dev` v4.4.0, loaded from CDN at runtime). Loads all rows from the source view into an in-browser Perspective worker and renders a `<perspective-viewer>` web component. Supports grouping, splitting, filtering, sorting, and charting interactively.
 
   **Toolbar (above the viewer):**
   - Named layouts — saved per source in the `pivot_layouts` DB table. Each chip recalls the full viewer state including group_by, split_by, filters, expressions, selection mode, and expand depth. A blue **Save** button overwrites the active layout in place; **+ Save as…** saves to a new name. The × on each chip deletes it.
-  - **depth: 0 1 2 3** — collapses or expands all grouped rows to the specified hierarchy level. Implemented via `view.set_depth(d)` + `plugin.draw(view)` (the only working mechanism found — `plugin_config.expand_depth` and `viewer.flush()` alone have no effect).
+  - **depth: 0 1 2 3** — collapses or expands all grouped rows to the specified hierarchy level. Implemented via `view.set_depth(d)` + `plugin.draw(view)` (the only working mechanism found in v4.4.0 — `plugin_config.expand_depth` and `viewer.flush()` alone have no effect).
   - The Perspective built-in **selection mode button** (Read-Only / Select Row / Select Column / Select Region) defaults to **Select Region** on fresh load, set directly via `plugin.restore({ edit_mode: 'SELECT_REGION' })` after the viewer loads.
 
   **Cell inspector (right panel):**
@@ -392,10 +235,7 @@ of the initial download and in a chunk fetched only when a pivot is opened.
   - `localStorage` key `psp_layout_{source}` saves the last viewer state on each named layout save.
   - Named layouts store `{ ...viewer.save(), plugin_config: plugin.save(), expand_depth }` as JSONB in `pivot_layouts`. On recall, viewer config, plugin config (edit mode), and expand depth are all restored independently.
 
-  See `docs/perspective.md` for the full technical reference on controlling Perspective programmatically.
+  See `docs/perspective-pivot.md` for the full technical reference on controlling Perspective programmatically.
-
-- **Stacks** — Named unions of multiple sources, each chip linking to its pivot at
-  `/stacks/:name/pivot`. Each stack defines a field mapping (how source fields map to common output columns), an amount field, a date field, and an optional balance offset. The view-data endpoint unions the underlying source views and computes a running balance sorted by date. The Pivot page supports stacks as well as individual sources, with layouts stored in the same `pivot_layouts` table.
 
 - **Log** — Global import log across all sources. Same expandable key detail and delete capability as the Import page, plus a source name column.
 
@@ -420,7 +260,7 @@ Shows current status on every screen:
 
 2. **Redeploy schema** — Runs `database/schema.sql` against the configured database. Warns that this drops all data. Requires explicit confirmation.
 
-3. **Redeploy SQL functions** — Runs the function files in `database/` in dependency order: `sources.sql`, `rules.sql`, `mappings.sql`, `records.sql`, `import.sql`, `transform.sql`, `stacks.sql`, `status.sql`. Safe to run at any time without data loss.
+3. **Redeploy SQL functions** — Runs all four files in `database/queries/` in order: `sources.sql`, `rules.sql`, `mappings.sql`, `records.sql`. Safe to run at any time without data loss.
 
 4. **Build UI** — Runs `npm run build` in `ui/`, outputting to `public/`.
 
@@ -434,10 +274,6 @@ Shows current status on every screen:
 
 9. **Set login credentials** — Prompts for username and password, bcrypt-hashes the password via `node -e "require('bcrypt')..."`, and writes `LOGIN_USER` and `LOGIN_PASSWORD_HASH` to `.env`. Requires Node.js and bcrypt npm package to be installed.
 
-10. **Claim SimpleFIN setup token** — Prompts for a setup token (hidden input), exchanges it for a permanent access URL via `api/lib/simplefin.js`, and writes `SIMPLEFIN_ACCESS_URL` to `.env`. Warns and confirms before replacing an existing URL. Setup tokens are single-use, so a failed claim generally means a new token is needed. Prints only the bridge host, never the embedded credentials.
-
-11. **Uninstall** — Reverses everything the other options install, in reverse order: stops/disables/removes the systemd unit, removes the nginx site and reloads nginx, drops the database and its user (prompts for admin credentials), then deletes `.env`, `public/`, and `node_modules`. Lists exactly what it found before doing anything and requires typing `delete` to proceed. The repository itself is left in place.
-
 **Key behaviors:**
 - All commands that will be run are printed before the user is asked to confirm.
 - Actions that require sudo prompt transparently — `sudo` is not run with `-n`, so it uses cached credentials or prompts as normal.
@@ -459,8 +295,6 @@ API_PORT             Port the Express server listens on (default 3020)
 NODE_ENV             development | production
 LOGIN_USER           Username for Basic Auth
 LOGIN_PASSWORD_HASH  bcrypt hash of the password
-
-SIMPLEFIN_ACCESS_URL Default SimpleFIN bridge URL; per-source override via config.simplefin.access_url_env
 ```
 
 ---
@@ -487,18 +321,13 @@ The server binds to `0.0.0.0` on `API_PORT` and serves both the API and the comp
 
 ## Deploying SQL Changes
 
-Any time SQL functions are modified, run `python3 manage.py` and choose "Redeploy SQL
+Any time SQL functions are modified:
-functions only". It runs every function file in dependency order — the list lives in
-`QUERY_FILES` in `manage.py`, which is the one place the order is defined.
-
-To deploy a single file by hand:
 ```bash
-PGPASSWORD=<pass> psql -h <host> -U <user> -d <db> -v ON_ERROR_STOP=1 -f database/rules.sql
+PGPASSWORD=<pass> psql -h <host> -U <user> -d <db> -f database/queries/sources.sql
+PGPASSWORD=<pass> psql -h <host> -U <user> -d <db> -f database/queries/rules.sql
+PGPASSWORD=<pass> psql -h <host> -U <user> -d <db> -f database/queries/mappings.sql
+PGPASSWORD=<pass> psql -h <host> -U <user> -d <db> -f database/queries/records.sql
 ```
-
+Then restart the server. Function deployment is safe to repeat — all functions use `CREATE OR REPLACE`.
-Deployment is safe to repeat — every function uses `CREATE OR REPLACE`.
-
-**The files are the source of truth.** Editing a function directly in the database, without
-writing the change back to its file, means the next redeploy silently reverts it.
 
 Schema changes (`schema.sql`) drop and recreate the schema, deleting all data. In production, write migration scripts instead.

api/lib/fields.js

@@ -1,42 +0,0 @@
-/**
- * Field inference
- * Derives a field list and column types from sample records, so a source can be
- * configured from real data rather than an assumed shape. Used by the CSV
- * suggest endpoint and by bank-feed sampling.
- */
-
-const ISO_DATE_RE = /^\d{4}-\d{2}-\d{2}(T[\d:.Z+-]+)?$/;
-
-/**
- * @param records  array of flat objects
- * @param limit    how many rows to look at
- * @returns { fields: [{name, type}], sampleRows }
- *
- * Keys are unioned across the sample, not read off the first row — API feeds
- * omit optional keys entirely on records that lack them.
- */
-function inferFields(records, limit = 50) {
-    const sampleRows = records.slice(0, limit);
-
-    const keys = [];
-    for (const row of sampleRows) {
-        for (const key of Object.keys(row)) {
-            if (!keys.includes(key)) keys.push(key);
-        }
-    }
-
-    const fields = keys.map(key => {
-        const vals = sampleRows.map(r => r[key]).filter(v => v !== '' && v != null);
-        let type = 'text';
-        if (vals.length > 0 && vals.every(v => !isNaN(parseFloat(v)) && isFinite(v) && String(v).charAt(0) !== '0')) {
-            type = 'numeric';
-        } else if (vals.length > 0 && vals.every(v => ISO_DATE_RE.test(String(v)))) {
-            type = 'date';
-        }
-        return { name: key, type };
-    });
-
-    return { fields, sampleRows };
-}
-
-module.exports = { inferFields };

api/lib/simplefin.js

@@ -1,219 +0,0 @@
-/**
- * SimpleFIN Bridge client
- *
- * Read-only access to linked bank accounts. Auth is a single "access URL" with
- * credentials embedded (https://user:pass@bridge.simplefin.org/simplefin),
- * obtained once by claiming a setup token. No certificates, no refresh flow —
- * the URL is the credential, so it lives in .env and never in the database.
- *
- * Protocol: https://www.simplefin.org/protocol.html
- */
-
-const REQUEST_TIMEOUT = 30000;
-
-// The bridge refreshes from banks roughly daily and transactions can land a few
-// days late, so pulls overlap by design and lean on constraint_key for dedupe.
-const DEFAULT_DAYS = 10;
-
-// The bridge hard-caps ranges at 90 days and reports the capping in `errors`;
-// 89 stays just inside it so a routine backfill doesn't look like data loss. It
-// also advises staying under 45 days, which it says on anything longer — that
-// notice is passed through, since it is a real hint about future behaviour.
-// Omitting start-date entirely is not "everything": it returns only the most
-// recent handful of transactions, so a start-date is always sent.
-const MAX_DAYS = 89;
-
-class SimpleFinError extends Error {
-    constructor(message, status) {
-        super(message);
-        this.name = 'SimpleFinError';
-        this.status = status;
-    }
-}
-
-// Resolve an access URL from the environment. Sources name their own variable
-// via config.simplefin.access_url_env so several bridges can coexist.
-function getAccessUrl(urlEnv) {
-    const name = urlEnv || 'SIMPLEFIN_ACCESS_URL';
-    const value = process.env[name];
-    if (!value) {
-        throw new SimpleFinError(`SimpleFIN access URL not found — set ${name} in .env`, 500);
-    }
-
-    let parsed;
-    try {
-        parsed = new URL(value);
-    } catch {
-        throw new SimpleFinError(`${name} is not a valid URL`, 500);
-    }
-    if (!parsed.username) {
-        throw new SimpleFinError(`${name} has no credentials — expected https://user:pass@host/path`, 500);
-    }
-    return parsed;
-}
-
-async function get(accessUrl, path, params) {
-    // Credentials travel in the Authorization header, not the request line.
-    const target = new URL(accessUrl.pathname + path, accessUrl.origin);
-    for (const [k, v] of Object.entries(params || {})) {
-        if (v !== undefined && v !== null) target.searchParams.append(k, String(v));
-    }
-    const auth = Buffer.from(`${decodeURIComponent(accessUrl.username)}:${decodeURIComponent(accessUrl.password)}`).toString('base64');
-
-    let res;
-    try {
-        res = await fetch(target, {
-            headers: { Authorization: `Basic ${auth}`, Accept: 'application/json' },
-            signal: AbortSignal.timeout(REQUEST_TIMEOUT),
-        });
-    } catch (err) {
-        throw new SimpleFinError(`SimpleFIN request failed: ${err.message}`, 502);
-    }
-
-    if (res.status === 401 || res.status === 403) {
-        throw new SimpleFinError(
-            'SimpleFIN rejected the access URL — it may have been revoked; claim a new setup token', res.status);
-    }
-    if (!res.ok) {
-        throw new SimpleFinError(`SimpleFIN returned ${res.status}: ${(await res.text()).slice(0, 200)}`, res.status);
-    }
-
-    try {
-        return await res.json();
-    } catch {
-        throw new SimpleFinError('SimpleFIN returned a non-JSON response', 502);
-    }
-}
-
-/**
- * Exchange a setup token for a permanent access URL. Run once per bridge; the
- * returned URL goes in .env. The token is base64 of a one-shot claim URL and is
- * consumed by this call.
- */
-async function claimSetupToken(setupToken) {
-    let claimUrl;
-    try {
-        claimUrl = Buffer.from(String(setupToken).trim(), 'base64').toString('utf8');
-        new URL(claimUrl);
-    } catch {
-        throw new SimpleFinError('Setup token is not valid base64 of a claim URL', 400);
-    }
-
-    let res;
-    try {
-        res = await fetch(claimUrl, { method: 'POST', signal: AbortSignal.timeout(REQUEST_TIMEOUT) });
-    } catch (err) {
-        throw new SimpleFinError(`Claim request failed: ${err.message}`, 502);
-    }
-    if (!res.ok) {
-        throw new SimpleFinError(
-            `Claim returned ${res.status} — setup tokens can only be claimed once`, res.status);
-    }
-    return (await res.text()).trim();
-}
-
-// Epoch seconds → YYYY-MM-DD, so dates sort and compare as plain text the way
-// CSV-imported dates already do. Pending transactions carry posted=0 rather than
-// omitting it, so falsy epochs are "no date", not 1970-01-01.
-function toDate(epochSeconds) {
-    if (!epochSeconds) return null;
-    return new Date(epochSeconds * 1000).toISOString().slice(0, 10);
-}
-
-// Flatten a SimpleFIN transaction into the shallow map the rule engine expects.
-//
-// Every scalar the bridge sends is passed through rather than whitelisted, so
-// institution-specific fields (mcc, and whatever a given bank adds) show up in
-// the data and in field discovery without a code change here. Only the epoch
-// timestamps are reshaped, and account context is folded in so records stay
-// self-describing.
-function flatten(txn, account) {
-    const out = {};
-
-    for (const [key, value] of Object.entries(txn)) {
-        if (value === null || value === undefined) continue;
-        if (typeof value === 'object') continue;  // nested objects handled below
-        out[key] = value;
-    }
-
-    // `extra` is free-form and institution-specific; hoist its scalars under a
-    // prefix so they can't collide with the documented fields
-    for (const [key, value] of Object.entries(txn.extra || {})) {
-        if (value !== null && value !== undefined && typeof value !== 'object') {
-            out[`extra_${key}`] = value;
-        }
-    }
-
-    const posted = toDate(txn.posted);
-    const transacted = toDate(txn.transacted_at);
-
-    delete out.posted;  // replaced by the YYYY-MM-DD forms below
-
-    // A pending transaction has no posted date yet; fall back to when it was
-    // transacted so every record has something usable to sort and filter on
-    out.date = posted || transacted;
-    out.posted_date = posted;
-    out.transacted_at = transacted;
-    out.pending = txn.pending ? 'true' : 'false';
-
-    out.account_id = account.id;
-    out.account_name = account.name;
-    out.organization = account.org?.name || account.org?.domain;
-
-    return out;
-}
-
-async function listAccounts(urlEnv) {
-    const data = await get(getAccessUrl(urlEnv), '/accounts', { 'balances-only': 1 });
-    return {
-        errors: data.errors || [],
-        accounts: (data.accounts || []).map(a => ({
-            id: a.id,
-            name: a.name,
-            organization: a.org?.name || a.org?.domain,
-            currency: a.currency,
-            balance: a.balance,
-            available_balance: a['available-balance'],
-            balance_date: toDate(a['balance-date']),
-        })),
-    };
-}
-
-/**
- * Fetch transactions for one account.
- *
- * days           — how far back to ask for; 0 or more than MAX_DAYS means the full 90
- * includePending — pending transactions get a new id once they post, so they are excluded by default
- */
-async function fetchTransactions({ accountId, accessUrlEnv, days, includePending = false }) {
-    days = Number.isFinite(days) ? days : DEFAULT_DAYS;
-    if (days <= 0 || days > MAX_DAYS) days = MAX_DAYS;
-
-    const params = {
-        account: accountId,
-        'start-date': Math.floor(Date.now() / 1000) - days * 86400,
-    };
-    if (includePending) params.pending = 1;
-
-    const data = await get(getAccessUrl(accessUrlEnv), '/accounts', params);
-
-    // The bridge reports per-institution problems here and still returns 200 —
-    // one bank being down must not look like a successful empty pull.
-    const errors = data.errors || [];
-    const account = (data.accounts || []).find(a => a.id === accountId);
-    if (!account) {
-        const detail = errors.length ? ` — bridge reported: ${errors.join('; ')}` : '';
-        throw new SimpleFinError(`Account ${accountId} not returned by SimpleFIN${detail}`, 502);
-    }
-
-    const txns = account.transactions || [];
-    const kept = includePending ? txns : txns.filter(t => !t.pending);
-
-    return {
-        fetched: txns.length,
-        errors,
-        records: kept.map(t => flatten(t, account)),
-    };
-}
-
-module.exports = { listAccounts, fetchTransactions, claimSetupToken, SimpleFinError, DEFAULT_DAYS, MAX_DAYS };

api/lib/sql.js

@@ -8,7 +8,7 @@
 function lit(val) {
     if (val === null || val === undefined) return 'NULL';
     if (typeof val === 'boolean') return val ? 'TRUE' : 'FALSE';
-    if (typeof val === 'number') return String(val);
+    if (typeof val === 'number') return String(Math.trunc(val));
     if (typeof val === 'object') return `'${JSON.stringify(val).replace(/'/g, "''")}'`;
     return `'${String(val).replace(/'/g, "''")}'`;
 }

api/routes/records.js

@@ -49,33 +49,17 @@ module.exports = (pool) => {
         }
     });
 
-    // Set overrides for all selected records
+    // Set overrides for a record and immediately merge into transformed
-    router.post('/bulk-overrides', async (req, res, next) => {
-        try {
-            const { source_name, record_ids, overrides } = req.body;
-            if (!source_name || !Array.isArray(record_ids) || record_ids.length === 0 || !overrides || typeof overrides !== 'object')
-                return res.status(400).json({ error: 'source_name, record_ids array, and overrides object required' });
-            const idList = record_ids.map(id => parseInt(id)).join(',');
-            const result = await pool.query(
-                `SELECT bulk_set_record_overrides(${lit(source_name)}, ARRAY[${idList}]::int[], ${lit(overrides)}) as result`
-            );
-            res.json(result.rows[0].result);
-        } catch (err) {
-            next(err);
-        }
-    });
-
-    // Set overrides for a record
     router.put('/:id/overrides', async (req, res, next) => {
         try {
             const { overrides } = req.body;
             if (!overrides || typeof overrides !== 'object')
                 return res.status(400).json({ error: 'overrides object required' });
             const result = await pool.query(
-                `SELECT set_record_overrides(${lit(parseInt(req.params.id))}, ${lit(overrides)}) as rec`
+                `SELECT * FROM set_record_overrides(${lit(parseInt(req.params.id))}, ${lit(overrides)})`
             );
-            if (!result.rows[0].rec) return res.status(404).json({ error: 'Record not found' });
+            if (result.rows.length === 0) return res.status(404).json({ error: 'Record not found' });
-            res.json(result.rows[0].rec);
+            res.json(result.rows[0]);
         } catch (err) {
             next(err);
         }
@@ -84,13 +68,13 @@ module.exports = (pool) => {
     // Clear overrides and reprocess that record to restore computed values
     router.delete('/:id/overrides', async (req, res, next) => {
         try {
-            const result = await pool.query(
+            const rec = await pool.query(
-                `SELECT clear_record_overrides(${lit(parseInt(req.params.id))}) as rec`
+                `SELECT * FROM clear_record_overrides(${lit(parseInt(req.params.id))})`
             );
-            if (!result.rows[0].rec) return res.status(404).json({ error: 'Record not found' });
+            if (rec.rows.length === 0) return res.status(404).json({ error: 'Record not found' });
-            const { source_name } = result.rows[0].rec;
+            // Reprocess this record so transformed reflects rules/mappings without overrides
             await pool.query(
-                `SELECT apply_transformations(${lit(source_name)}, ARRAY[${lit(parseInt(req.params.id))}::int], true)`
+                `SELECT apply_transformations(${lit(rec.rows[0].source_name)}, ARRAY[${lit(parseInt(req.params.id))}::int], true)`
             );
             const updated = await pool.query(`SELECT * FROM get_record(${lit(parseInt(req.params.id))})`);
             res.json(updated.rows[0]);

api/routes/sources.js

@@ -7,8 +7,6 @@ const express = require('express');
 const multer = require('multer');
 const { parse } = require('csv-parse/sync');
 const { lit, arr } = require('../lib/sql');
-const simplefin = require('../lib/simplefin');
-const { inferFields } = require('../lib/fields');
 
 const upload = multer({ storage: multer.memoryStorage() });
 
@@ -25,56 +23,6 @@ module.exports = (pool) => {
         }
     });
 
-    // SimpleFIN helpers. Declared before /:name so they aren't shadowed by it.
-
-    // List the accounts behind a bridge — used to find the account_id for a source
-    router.get('/simplefin-accounts', async (req, res, next) => {
-        try {
-            res.json(await simplefin.listAccounts(req.query.access_url_env));
-        } catch (err) {
-            if (err instanceof simplefin.SimpleFinError) return res.status(err.status || 502).json({ error: err.message });
-            next(err);
-        }
-    });
-
-    // Sample an account's real transactions and infer its field list — the API
-    // equivalent of uploading a CSV to /suggest. Whatever the account actually
-    // returns is what gets offered, so investment or loan accounts describe
-    // themselves rather than being forced into a checking-account shape.
-    router.get('/simplefin-sample', async (req, res, next) => {
-        try {
-            const { account_id, access_url_env } = req.query;
-            if (!account_id) return res.status(400).json({ error: 'account_id is required' });
-
-            const { fetched, records, errors } = await simplefin.fetchTransactions({
-                accountId: account_id,
-                accessUrlEnv: access_url_env,
-                // The bridge advises staying under 45 days and warns at exactly
-                // 45, so 44 is the largest quiet sample window
-                days: req.query.days !== undefined ? parseInt(req.query.days) : 44,
-                includePending: true,  // widen the sample; pending rows can carry extra keys
-            });
-
-            const { fields, sampleRows } = inferFields(records);
-            res.json({ fields, sampleRows, fetched, errors });
-        } catch (err) {
-            if (err instanceof simplefin.SimpleFinError) return res.status(err.status || 502).json({ error: err.message });
-            next(err);
-        }
-    });
-
-    // Exchange a one-shot setup token for the permanent access URL to put in .env
-    router.post('/simplefin-claim', async (req, res, next) => {
-        try {
-            const { setup_token } = req.body || {};
-            if (!setup_token) return res.status(400).json({ error: 'setup_token is required' });
-            res.json({ access_url: await simplefin.claimSetupToken(setup_token) });
-        } catch (err) {
-            if (err instanceof simplefin.SimpleFinError) return res.status(err.status || 502).json({ error: err.message });
-            next(err);
-        }
-    });
-
     // List all sources
     router.get('/', async (req, res, next) => {
         try {
@@ -104,7 +52,21 @@ module.exports = (pool) => {
             const records = parse(req.file.buffer, { columns: true, skip_empty_lines: true, trim: true });
             if (records.length === 0) return res.status(400).json({ error: 'CSV file is empty' });
 
-            const { fields, sampleRows } = inferFields(records);
+            const ISO_DATE_RE = /^\d{4}-\d{2}-\d{2}(T[\d:.Z+-]+)?$/;
+            const sample = records[0];
+            const sampleRows = records.slice(0, 50);
+
+            const fields = Object.keys(sample).map(key => {
+                const vals = sampleRows.map(r => r[key]).filter(v => v !== '' && v != null);
+                let type = 'text';
+                if (vals.length > 0 && vals.every(v => !isNaN(parseFloat(v)) && isFinite(v) && String(v).charAt(0) !== '0')) {
+                    type = 'numeric';
+                } else if (vals.length > 0 && vals.every(v => ISO_DATE_RE.test(String(v)))) {
+                    type = 'date';
+                }
+                return { name: key, type };
+            });
+
             res.json({ name: '', constraint_fields: [], fields, sampleRows });
         } catch (err) {
             next(err);
@@ -177,51 +139,6 @@ module.exports = (pool) => {
         }
     });
 
-    // Pull transactions from SimpleFIN and import them, same as a CSV upload.
-    // Safe to re-run: overlapping transactions are skipped by constraint key.
-    router.post('/:name/sync', async (req, res, next) => {
-        try {
-            const sourceResult = await pool.query(`SELECT * FROM get_source(${lit(req.params.name)})`);
-            const source = sourceResult.rows[0];
-            if (!source || !source.name) return res.status(404).json({ error: 'Source not found' });
-
-            const cfg = (source.config || {}).simplefin;
-            if (!cfg || !cfg.account_id) {
-                return res.status(400).json({
-                    error: `Source "${req.params.name}" has no simplefin.account_id in its config`
-                });
-            }
-
-            const opts = { ...req.query, ...req.body };
-            const { fetched, errors, records } = await simplefin.fetchTransactions({
-                accountId: cfg.account_id,
-                accessUrlEnv: cfg.access_url_env,
-                days: opts.days !== undefined ? parseInt(opts.days) : cfg.days,
-                includePending: opts.include_pending === true || opts.include_pending === 'true',
-            });
-
-            if (records.length === 0) {
-                return res.json({ success: true, fetched, errors, imported: 0, duplicates: 0 });
-            }
-
-            const importResult = await pool.query(
-                `SELECT import_records(${lit(req.params.name)}, ${lit(records)}) as result`
-            );
-            const importData = importResult.rows[0].result;
-
-            if (!importData.success) return res.json({ ...importData, fetched, errors });
-
-            const transformResult = await pool.query(
-                `SELECT apply_transformations(${lit(req.params.name)}) as result`
-            );
-
-            res.json({ ...importData, fetched, errors, transform: transformResult.rows[0].result });
-        } catch (err) {
-            if (err instanceof simplefin.SimpleFinError) return res.status(err.status || 502).json({ error: err.message });
-            next(err);
-        }
-    });
-
     // Get import log
     router.get('/:name/import-log', async (req, res, next) => {
         try {
@@ -270,11 +187,7 @@ module.exports = (pool) => {
     router.post('/:name/view', async (req, res, next) => {
         try {
             const result = await pool.query(`SELECT generate_source_view(${lit(req.params.name)}) as result`);
-            const data = result.rows[0].result;
+            res.json(result.rows[0].result);
-            if (data && data.success) {
-                await pool.query(`UPDATE dataflow.sources SET view_generated_at = NOW() WHERE name = ${lit(req.params.name)}`);
-            }
-            res.json(data);
         } catch (err) {
             next(err);
         }
@@ -317,19 +230,6 @@ module.exports = (pool) => {
         }
     });
 
-    // Override keys — distinct field names used in overrides across all records for this source
-    router.get('/:name/override-keys', async (req, res, next) => {
-        try {
-            const result = await pool.query(
-                `SELECT DISTINCT jsonb_object_keys(overrides) AS key
-                 FROM dataflow.records
-                 WHERE source_name = ${lit(req.params.name)} AND overrides IS NOT NULL
-                 ORDER BY key`
-            );
-            res.json(result.rows.map(r => r.key));
-        } catch (err) { next(err); }
-    });
-
     // Pivot layouts
     router.get('/:name/layouts', async (req, res, next) => {
         try {

api/routes/stacks.js

@@ -1,201 +0,0 @@
-/**
- * Stacks Routes
- * Named unions of multiple sources with field mappings and running balance
- */
-
-const express = require('express');
-const { lit, arr } = require('../lib/sql');
-
-module.exports = (pool) => {
-    const router = express.Router();
-
-    // List all stacks
-    router.get('/', async (req, res, next) => {
-        try {
-            const result = await pool.query('SELECT * FROM list_stacks()');
-            res.json(result.rows);
-        } catch (err) { next(err); }
-    });
-
-    // Get single stack with sources
-    router.get('/:name', async (req, res, next) => {
-        try {
-            const result = await pool.query(`SELECT * FROM get_stack(${lit(req.params.name)})`);
-            if (!result.rows.length) return res.status(404).json({ error: 'Stack not found' });
-            res.json(result.rows[0]);
-        } catch (err) { next(err); }
-    });
-
-    // Create stack
-    router.post('/', async (req, res, next) => {
-        try {
-            const { name, label, fields, amount_field, date_field, balance_offset } = req.body;
-            if (!name) return res.status(400).json({ error: 'name is required' });
-            const result = await pool.query(
-                `SELECT * FROM create_stack(${lit(name)}, ${lit(label || null)}, ${lit(JSON.stringify(fields || []))}, ${lit(amount_field || null)}, ${lit(date_field || null)}, ${lit(balance_offset ?? 0)})`
-            );
-            res.status(201).json(result.rows[0]);
-        } catch (err) {
-            if (err.code === '23505') return res.status(409).json({ error: 'Stack already exists' });
-            next(err);
-        }
-    });
-
-    // Update stack
-    router.put('/:name', async (req, res, next) => {
-        try {
-            const { label, fields, amount_field, date_field, balance_offset } = req.body;
-            const n = v => v !== undefined ? lit(v) : 'NULL';
-            const f = v => v !== undefined ? lit(JSON.stringify(v)) : 'NULL';
-            const result = await pool.query(
-                `SELECT * FROM update_stack(${lit(req.params.name)}, ${n(label)}, ${f(fields)}, ${n(amount_field)}, ${n(date_field)}, ${n(balance_offset)})`
-            );
-            if (!result.rows.length) return res.status(404).json({ error: 'Stack not found' });
-            res.json(result.rows[0]);
-        } catch (err) { next(err); }
-    });
-
-    // Delete stack
-    router.delete('/:name', async (req, res, next) => {
-        try {
-            const result = await pool.query(`SELECT * FROM delete_stack(${lit(req.params.name)})`);
-            if (!result.rows.length) return res.status(404).json({ error: 'Stack not found' });
-            res.json({ success: true, deleted: req.params.name });
-        } catch (err) { next(err); }
-    });
-
-    // Add or update a source in a stack
-    router.put('/:name/sources/:source', async (req, res, next) => {
-        try {
-            const { field_map, amount_sign, balance_offset, amount_field, date_field } = req.body;
-            const n = v => v != null ? lit(v) : 'NULL';
-            const result = await pool.query(
-                `SELECT * FROM upsert_stack_source(${lit(req.params.name)}, ${lit(req.params.source)}, ${lit(JSON.stringify(field_map || {}))}, ${lit(amount_sign ?? 1)}, ${lit(balance_offset ?? 0)}, ${n(amount_field)}, ${n(date_field)})`
-            );
-            res.json(result.rows[0]);
-        } catch (err) {
-            if (err.code === '23503') return res.status(404).json({ error: 'Stack or source not found' });
-            next(err);
-        }
-    });
-
-    // Remove a source from a stack
-    router.delete('/:name/sources/:source', async (req, res, next) => {
-        try {
-            const result = await pool.query(
-                `SELECT * FROM remove_stack_source(${lit(req.params.name)}, ${lit(req.params.source)})`
-            );
-            if (!result.rows.length) return res.status(404).json({ error: 'Source not in stack' });
-            res.json({ success: true, removed: req.params.source });
-        } catch (err) { next(err); }
-    });
-
-    // Reorder sources within a stack
-    router.put('/:name/sources/reorder', async (req, res, next) => {
-        try {
-            const { source_names } = req.body;
-            if (!Array.isArray(source_names)) return res.status(400).json({ error: 'source_names array required' });
-            await pool.query(`SELECT reorder_stack_sources(${lit(req.params.name)}, ${arr(source_names)})`);
-            res.json({ success: true });
-        } catch (err) { next(err); }
-    });
-
-    // Get current running balance from the generated view
-    router.get('/:name/balance', async (req, res, next) => {
-        try {
-            const result = await pool.query(`SELECT get_stack_balance(${lit(req.params.name)}) AS result`);
-            res.json(result.rows[0].result);
-        } catch (err) { next(err); }
-    });
-
-    // Preview the SQL that would be generated (dry run — does not create the view)
-    router.get('/:name/view-sql', async (req, res, next) => {
-        try {
-            const result = await pool.query(`SELECT generate_stack_view(${lit(req.params.name)}, true) AS result`);
-            res.json(result.rows[0].result);
-        } catch (err) { next(err); }
-    });
-
-    // Generate / refresh the dfv view
-    router.post('/:name/view', async (req, res, next) => {
-        try {
-            const result = await pool.query(`SELECT generate_stack_view(${lit(req.params.name)}) AS result`);
-            const data = result.rows[0].result;
-            if (data && data.success) {
-                await pool.query(`UPDATE dataflow.stacks SET view_generated_at = NOW() WHERE name = ${lit(req.params.name)}`);
-            }
-            res.json(data);
-        } catch (err) { next(err); }
-    });
-
-    // Execute custom SQL for the view (user-edited SQL)
-    router.post('/:name/exec-sql', async (req, res, next) => {
-        try {
-            const { sql } = req.body;
-            if (!sql) return res.status(400).json({ success: false, error: 'sql is required' });
-            await pool.query(`DROP VIEW IF EXISTS dfv.${req.params.name} CASCADE`);
-            await pool.query(sql);
-            await pool.query(`UPDATE dataflow.stacks SET view_generated_at = NOW() WHERE name = ${lit(req.params.name)}`);
-            // Detect stacks whose views were dropped by CASCADE
-            const staleResult = await pool.query(`
-                SELECT array_agg(name) AS names FROM dataflow.stacks
-                WHERE name != ${lit(req.params.name)}
-                  AND view_generated_at IS NOT NULL
-                  AND NOT EXISTS (
-                      SELECT 1 FROM pg_views WHERE schemaname = 'dfv' AND viewname = name
-                  )
-            `);
-            const cascadeStale = staleResult.rows[0].names || [];
-            if (cascadeStale.length) {
-                await pool.query(`UPDATE dataflow.stacks SET view_generated_at = NULL WHERE name = ANY($1)`, [cascadeStale]);
-            }
-            res.json({ success: true, cascade_stale: cascadeStale });
-        } catch (err) {
-            res.json({ success: false, error: err.message });
-        }
-    });
-
-    // Calibrate balance offset given a known good balance at a specific date
-    router.post('/:name/calibrate', async (req, res, next) => {
-        try {
-            const { as_of_date, known_balance, source_name } = req.body;
-            if (known_balance === undefined) {
-                return res.status(400).json({ error: 'known_balance is required' });
-            }
-            const dateExpr = as_of_date ? `${lit(as_of_date)}::date` : 'NULL';
-            const result = await pool.query(
-                `SELECT calibrate_balance(${lit(req.params.name)}, ${source_name ? lit(source_name) : 'NULL'}, ${dateExpr}, ${lit(known_balance)}::numeric) AS result`
-            );
-            res.json(result.rows[0].result);
-        } catch (err) { next(err); }
-    });
-
-    // Pivot layouts (same DB table as sources; FK was dropped to allow stack names)
-    router.get('/:name/layouts', async (req, res, next) => {
-        try {
-            const result = await pool.query(`SELECT * FROM list_pivot_layouts(${lit(req.params.name)})`);
-            res.json(result.rows);
-        } catch (err) { next(err); }
-    });
-
-    router.post('/:name/layouts', async (req, res, next) => {
-        try {
-            const { layout_name, config } = req.body;
-            if (!layout_name || !config) return res.status(400).json({ error: 'layout_name and config required' });
-            const result = await pool.query(
-                `SELECT * FROM save_pivot_layout(${lit(req.params.name)}, ${lit(layout_name)}, ${lit(config)})`
-            );
-            res.json(result.rows[0]);
-        } catch (err) { next(err); }
-    });
-
-    router.delete('/:name/layouts/:id', async (req, res, next) => {
-        try {
-            const result = await pool.query(`SELECT * FROM delete_pivot_layout(${lit(parseInt(req.params.id))})`);
-            if (result.rows.length === 0) return res.status(404).json({ error: 'Layout not found' });
-            res.json({ success: true });
-        } catch (err) { next(err); }
-    });
-
-    return router;
-};

api/routes/status.js

@@ -1,14 +0,0 @@
-const express = require('express');
-
-module.exports = (pool) => {
-    const router = express.Router();
-
-    router.get('/', async (req, res, next) => {
-        try {
-            const result = await pool.query('SELECT get_status() AS result');
-            res.json(result.rows[0].result);
-        } catch (err) { next(err); }
-    });
-
-    return router;
-};

api/server.js

@@ -3,7 +3,7 @@
  * Simple REST API for data transformation
  */
 
-require('dotenv').config({ quiet: true });
+require('dotenv').config();
 const express = require('express');
 const { Pool } = require('pg');
 
@@ -16,8 +16,7 @@ const pool = new Pool({
     port: process.env.DB_PORT,
     database: process.env.DB_NAME,
     user: process.env.DB_USER,
-    password: process.env.DB_PASSWORD,
+    password: process.env.DB_PASSWORD
-    options: '-c search_path=dataflow,public'
 });
 
 // Middleware
@@ -32,6 +31,11 @@ app.use('/api', auth);
 const path = require('path');
 app.use(express.static(path.join(__dirname, '../public')));
 
+// Set search path for all queries
+pool.on('connect', (client) => {
+    client.query('SET search_path TO dataflow, public');
+});
+
 // Test database connection
 pool.query('SELECT NOW()', (err, res) => {
     if (err) {
@@ -50,16 +54,12 @@ const sourcesRoutes = require('./routes/sources');
 const rulesRoutes = require('./routes/rules');
 const mappingsRoutes = require('./routes/mappings');
 const recordsRoutes = require('./routes/records');
-const stacksRoutes = require('./routes/stacks');
-const statusRoutes = require('./routes/status');
 
 // Mount routes
 app.use('/api/sources', sourcesRoutes(pool));
 app.use('/api/rules', rulesRoutes(pool));
 app.use('/api/mappings', mappingsRoutes(pool));
 app.use('/api/records', recordsRoutes(pool));
-app.use('/api/stacks', stacksRoutes(pool));
-app.use('/api/status', statusRoutes(pool));
 
 // Health check
 app.get('/health', (req, res) => {

database/functions.sql

@@ -0,0 +1,621 @@
+--
+-- Dataflow Functions
+-- Simple, clear functions for import and transformation
+--
+
+SET search_path TO dataflow, public;
+
+------------------------------------------------------
+-- Function: import_records
+-- Import data with automatic deduplication
+------------------------------------------------------
+CREATE OR REPLACE FUNCTION import_records(
+    p_source_name TEXT,
+    p_data JSONB  -- Array of records
+) RETURNS JSON AS $$
+DECLARE
+    v_constraint_fields TEXT[];
+    v_inserted          INTEGER;
+    v_duplicates        INTEGER;
+    v_log_id            INTEGER;
+BEGIN
+    SELECT constraint_fields INTO v_constraint_fields
+    FROM dataflow.sources
+    WHERE name = p_source_name;
+
+    IF v_constraint_fields IS NULL THEN
+        RETURN json_build_object(
+            'success', false,
+            'error', 'Source not found: ' || p_source_name
+        );
+    END IF;
+
+    WITH
+    -- All incoming records with their constraint keys
+    pending AS (
+        SELECT
+            rec.value      AS data,
+            rec.ordinality AS seq,
+            (SELECT jsonb_object_agg(f, rec.value->>f)
+             FROM unnest(v_constraint_fields) AS f) AS constraint_key
+        FROM jsonb_array_elements(p_data) WITH ORDINALITY AS rec
+    ),
+    -- Keys already in the database (excluded)
+    existing AS (
+        SELECT DISTINCT r.constraint_key
+        FROM dataflow.records r
+        INNER JOIN pending p ON p.constraint_key = r.constraint_key
+        WHERE r.source_name = p_source_name
+    ),
+    -- Rows whose constraint key is not yet in the database
+    new_records AS (
+        SELECT p.data, p.constraint_key, p.seq
+        FROM pending p
+        WHERE NOT EXISTS (SELECT 1 FROM existing e WHERE e.constraint_key = p.constraint_key)
+    ),
+    -- Write the log entry
+    log_entry AS (
+        INSERT INTO dataflow.import_log (source_name, records_imported, records_duplicate, info)
+        VALUES (
+            p_source_name,
+            (SELECT count(*) FROM new_records),
+            (SELECT count(*) FROM pending) - (SELECT count(*) FROM new_records),
+            jsonb_build_object(
+                'total',          jsonb_array_length(p_data),
+                'inserted_keys',  (SELECT jsonb_agg(constraint_key ORDER BY constraint_key) FROM new_records),
+                'excluded_keys',  (SELECT jsonb_agg(constraint_key) FROM existing)
+            )
+        )
+        RETURNING id, records_imported, records_duplicate
+    ),
+    -- Insert new records
+    inserted AS (
+        INSERT INTO dataflow.records (source_name, data, constraint_key, import_id)
+        SELECT p_source_name, nr.data, nr.constraint_key, (SELECT id FROM log_entry)
+        FROM new_records nr
+        ORDER BY nr.seq
+        RETURNING id
+    )
+    SELECT le.id, le.records_imported, le.records_duplicate
+    INTO   v_log_id, v_inserted, v_duplicates
+    FROM   log_entry le;
+
+    RETURN json_build_object(
+        'success',    true,
+        'imported',   v_inserted,
+        'duplicates', v_duplicates,
+        'log_id',     v_log_id
+    );
+END;
+$$ LANGUAGE plpgsql;
+
+COMMENT ON FUNCTION import_records IS 'Import records with automatic deduplication';
+
+------------------------------------------------------
+-- Function: get_import_log
+-- Return import history for a source
+------------------------------------------------------
+CREATE OR REPLACE FUNCTION get_import_log(p_source_name TEXT)
+RETURNS TABLE (
+    id              INTEGER,
+    source_name     TEXT,
+    records_imported INTEGER,
+    records_duplicate INTEGER,
+    imported_at     TIMESTAMPTZ,
+    info            JSONB
+) AS $$
+    SELECT id, source_name, records_imported, records_duplicate, imported_at, info
+    FROM dataflow.import_log
+    WHERE source_name = p_source_name
+    ORDER BY imported_at DESC;
+$$ LANGUAGE sql;
+
+COMMENT ON FUNCTION get_import_log IS 'Return import history for a source, newest first, including inserted/excluded key lists';
+
+------------------------------------------------------
+-- Function: get_all_import_logs
+-- Return import history across all sources
+------------------------------------------------------
+CREATE OR REPLACE FUNCTION get_all_import_logs()
+RETURNS TABLE (
+    id               INTEGER,
+    source_name      TEXT,
+    records_imported INTEGER,
+    records_duplicate INTEGER,
+    imported_at      TIMESTAMPTZ,
+    info             JSONB
+) AS $$
+    SELECT id, source_name, records_imported, records_duplicate, imported_at, info
+    FROM dataflow.import_log
+    ORDER BY imported_at DESC;
+$$ LANGUAGE sql;
+
+COMMENT ON FUNCTION get_all_import_logs IS 'Return import history across all sources, newest first';
+
+------------------------------------------------------
+-- Function: delete_import
+-- Delete all records from a specific import and remove the log entry
+------------------------------------------------------
+CREATE OR REPLACE FUNCTION delete_import(p_log_id INTEGER)
+RETURNS JSON AS $$
+DECLARE
+    v_deleted INTEGER;
+BEGIN
+    IF NOT EXISTS (SELECT 1 FROM dataflow.import_log WHERE id = p_log_id) THEN
+        RETURN json_build_object('success', false, 'error', 'Import log entry not found');
+    END IF;
+
+    SELECT count(*) INTO v_deleted FROM dataflow.records WHERE import_id = p_log_id;
+
+    -- Cascade handles deleting records via FK ON DELETE CASCADE
+    DELETE FROM dataflow.import_log WHERE id = p_log_id;
+
+    RETURN json_build_object(
+        'success',         true,
+        'records_deleted', v_deleted,
+        'log_id',          p_log_id
+    );
+END;
+$$ LANGUAGE plpgsql;
+
+COMMENT ON FUNCTION delete_import IS 'Delete all records belonging to an import batch and remove the log entry';
+
+------------------------------------------------------
+-- Aggregate: jsonb_concat_obj
+-- Merge JSONB objects across rows (later rows win on key conflicts)
+-- Usage: jsonb_concat_obj(col ORDER BY sequence)
+------------------------------------------------------
+CREATE OR REPLACE FUNCTION dataflow.jsonb_merge(a JSONB, b JSONB)
+RETURNS JSONB AS $$
+    SELECT COALESCE(a, '{}') || COALESCE(b, '{}')
+$$ LANGUAGE sql IMMUTABLE;
+
+DROP AGGREGATE IF EXISTS dataflow.jsonb_concat_obj(JSONB);
+CREATE AGGREGATE dataflow.jsonb_concat_obj(JSONB) (
+    sfunc    = dataflow.jsonb_merge,
+    stype    = JSONB,
+    initcond = '{}'
+);
+
+------------------------------------------------------
+-- Function: apply_transformations
+-- Apply all transformation rules to records (set-based)
+------------------------------------------------------
+DROP FUNCTION IF EXISTS apply_transformations(TEXT, INTEGER[]);
+CREATE OR REPLACE FUNCTION apply_transformations(
+    p_source_name TEXT,
+    p_record_ids  INTEGER[] DEFAULT NULL,  -- NULL = all eligible records
+    p_overwrite   BOOLEAN   DEFAULT FALSE  -- FALSE = skip already-transformed, TRUE = overwrite all
+) RETURNS JSON AS $$
+DECLARE
+    v_seq        INT;
+    v_seq_count  INT;
+    v_count      INT := 0;
+BEGIN
+    -- Fast path: if all rules share one sequence value, no chaining is needed —
+    -- use the original single-pass CTE which the planner can fully optimize.
+    SELECT count(DISTINCT sequence) INTO v_seq_count
+    FROM dataflow.rules
+    WHERE source_name = p_source_name AND enabled = true;
+
+    IF v_seq_count <= 1 THEN
+        WITH
+        qualifying AS (
+            SELECT id, data
+            FROM dataflow.records
+            WHERE source_name = p_source_name
+              AND (p_overwrite OR transformed IS NULL)
+              AND (p_record_ids IS NULL OR id = ANY(p_record_ids))
+        ),
+        rx AS (
+            SELECT
+                q.id,
+                r.name          AS rule_name,
+                r.sequence,
+                r.output_field,
+                r.retain,
+                r.function_type,
+                COALESCE(mt.rn, rp.rn, 1) AS result_number,
+                CASE WHEN array_length(mt.mt, 1) = 1 THEN to_jsonb(mt.mt[1]) ELSE to_jsonb(mt.mt) END AS match_val,
+                to_jsonb(rp.rp) AS replace_val
+            FROM dataflow.rules r
+            INNER JOIN qualifying q ON q.data ? r.field
+            LEFT JOIN LATERAL regexp_matches(q.data ->> r.field, r.pattern, r.flags)
+                WITH ORDINALITY AS mt(mt, rn) ON r.function_type = 'extract'
+            LEFT JOIN LATERAL regexp_replace(q.data ->> r.field, r.pattern, r.replace_value, r.flags)
+                WITH ORDINALITY AS rp(rp, rn) ON r.function_type = 'replace'
+            WHERE r.source_name = p_source_name
+              AND r.enabled = true
+              AND (r.function_type != 'extract' OR mt.mt IS NOT NULL)
+        ),
+        agg_matches AS (
+            SELECT
+                id, rule_name, sequence, output_field, retain, function_type,
+                CASE function_type
+                    WHEN 'replace' THEN jsonb_agg(replace_val) -> 0
+                    ELSE
+                        CASE WHEN max(result_number) = 1
+                            THEN jsonb_agg(match_val ORDER BY result_number) -> 0
+                            ELSE jsonb_agg(match_val ORDER BY result_number)
+                        END
+                END AS extracted
+            FROM rx
+            GROUP BY id, rule_name, sequence, output_field, retain, function_type
+        ),
+        linked AS (
+            SELECT a.id, a.sequence, a.output_field, a.retain, a.extracted, m.output AS mapped
+            FROM agg_matches a
+            LEFT JOIN dataflow.mappings m ON
+                m.source_name = p_source_name
+                AND m.rule_name = a.rule_name
+                AND m.input_value = a.extracted
+            WHERE a.extracted IS NOT NULL
+        ),
+        rule_output AS (
+            SELECT id, sequence,
+                CASE
+                    WHEN mapped IS NOT NULL THEN
+                        mapped || CASE WHEN retain THEN jsonb_build_object(output_field, extracted) ELSE '{}'::jsonb END
+                    ELSE
+                        jsonb_build_object(output_field, extracted)
+                END AS output
+            FROM linked
+        ),
+        record_additions AS (
+            SELECT id, dataflow.jsonb_concat_obj(output ORDER BY sequence) AS additions
+            FROM rule_output
+            GROUP BY id
+        ),
+        updated AS (
+            UPDATE dataflow.records rec
+            SET transformed    = rec.data || COALESCE(ra.additions, '{}'::jsonb) || COALESCE(rec.overrides, '{}'::jsonb),
+                transformed_at = CURRENT_TIMESTAMP
+            FROM qualifying q
+            LEFT JOIN record_additions ra ON ra.id = q.id
+            WHERE rec.id = q.id
+            RETURNING rec.id
+        )
+        SELECT count(*) INTO v_count FROM updated;
+
+        RETURN json_build_object('success', true, 'transformed', v_count);
+    END IF;
+
+    -- Chaining path: multiple sequence groups — process in order so each group
+    -- can read fields written by earlier groups.
+    CREATE TEMP TABLE _xform_acc ON COMMIT DROP AS
+    SELECT id, data, '{}'::jsonb AS additions
+    FROM dataflow.records
+    WHERE source_name = p_source_name
+      AND (p_overwrite OR transformed IS NULL)
+      AND (p_record_ids IS NULL OR id = ANY(p_record_ids));
+
+    FOR v_seq IN
+        SELECT DISTINCT sequence
+        FROM dataflow.rules
+        WHERE source_name = p_source_name AND enabled = true
+        ORDER BY sequence
+    LOOP
+        WITH
+        current AS (
+            SELECT id, data || additions AS current_data
+            FROM _xform_acc
+        ),
+        rx AS (
+            SELECT
+                c.id,
+                r.name          AS rule_name,
+                r.sequence,
+                r.output_field,
+                r.retain,
+                r.function_type,
+                COALESCE(mt.rn, rp.rn, 1) AS result_number,
+                CASE WHEN array_length(mt.mt, 1) = 1 THEN to_jsonb(mt.mt[1]) ELSE to_jsonb(mt.mt) END AS match_val,
+                to_jsonb(rp.rp) AS replace_val
+            FROM dataflow.rules r
+            INNER JOIN current c ON (c.current_data ? r.field)
+            LEFT JOIN LATERAL regexp_matches(c.current_data ->> r.field, r.pattern, r.flags)
+                WITH ORDINALITY AS mt(mt, rn) ON r.function_type = 'extract'
+            LEFT JOIN LATERAL regexp_replace(c.current_data ->> r.field, r.pattern, r.replace_value, r.flags)
+                WITH ORDINALITY AS rp(rp, rn) ON r.function_type = 'replace'
+            WHERE r.source_name = p_source_name
+              AND r.sequence = v_seq
+              AND r.enabled = true
+              AND (r.function_type != 'extract' OR mt.mt IS NOT NULL)
+        ),
+        agg_matches AS (
+            SELECT
+                id, rule_name, sequence, output_field, retain, function_type,
+                CASE function_type
+                    WHEN 'replace' THEN jsonb_agg(replace_val) -> 0
+                    ELSE
+                        CASE WHEN max(result_number) = 1
+                            THEN jsonb_agg(match_val ORDER BY result_number) -> 0
+                            ELSE jsonb_agg(match_val ORDER BY result_number)
+                        END
+                END AS extracted
+            FROM rx
+            GROUP BY id, rule_name, sequence, output_field, retain, function_type
+        ),
+        linked AS (
+            SELECT a.id, a.sequence, a.output_field, a.retain, a.extracted, m.output AS mapped
+            FROM agg_matches a
+            LEFT JOIN dataflow.mappings m ON
+                m.source_name = p_source_name
+                AND m.rule_name = a.rule_name
+                AND m.input_value = a.extracted
+            WHERE a.extracted IS NOT NULL
+        ),
+        rule_output AS (
+            SELECT id, sequence,
+                CASE
+                    WHEN mapped IS NOT NULL THEN
+                        mapped || CASE WHEN retain THEN jsonb_build_object(output_field, extracted) ELSE '{}'::jsonb END
+                    ELSE
+                        jsonb_build_object(output_field, extracted)
+                END AS output
+            FROM linked
+        ),
+        seq_additions AS (
+            SELECT id, dataflow.jsonb_concat_obj(output ORDER BY sequence) AS seq_adds
+            FROM rule_output
+            GROUP BY id
+        )
+        UPDATE _xform_acc acc
+        SET additions = acc.additions || COALESCE(sa.seq_adds, '{}'::jsonb)
+        FROM seq_additions sa
+        WHERE acc.id = sa.id;
+    END LOOP;
+
+    WITH updated AS (
+        UPDATE dataflow.records rec
+        SET transformed    = rec.data || acc.additions || COALESCE(rec.overrides, '{}'::jsonb),
+            transformed_at = CURRENT_TIMESTAMP
+        FROM _xform_acc acc
+        WHERE rec.id = acc.id
+        RETURNING rec.id
+    )
+    SELECT count(*) INTO v_count FROM updated;
+
+    RETURN json_build_object('success', true, 'transformed', v_count);
+END;
+$$ LANGUAGE plpgsql;
+
+COMMENT ON FUNCTION apply_transformations IS 'Apply transformation rules and mappings to records. Single-sequence sources use a fast single-pass CTE; multi-sequence sources use a loop so rules at sequence N can read outputs from sequence < N (chaining).';
+
+------------------------------------------------------
+-- Function: get_all_values
+-- All extracted values (mapped + unmapped) with counts and mapping output
+------------------------------------------------------
+DROP FUNCTION IF EXISTS get_all_values(TEXT, TEXT);
+CREATE FUNCTION get_all_values(
+    p_source_name TEXT,
+    p_rule_name TEXT DEFAULT NULL
+) RETURNS TABLE (
+    rule_name TEXT,
+    output_field TEXT,
+    source_field TEXT,
+    extracted_value JSONB,
+    record_count BIGINT,
+    sample JSONB,
+    mapping_id INTEGER,
+    output JSONB,
+    is_mapped BOOLEAN
+) AS $$
+BEGIN
+    RETURN QUERY
+    WITH extracted AS (
+        SELECT
+            r.name AS rule_name,
+            r.output_field,
+            r.field AS source_field,
+            rec.transformed->r.output_field AS extracted_value,
+            rec.data AS record_data,
+            row_number() OVER (
+                PARTITION BY r.name, rec.transformed->r.output_field
+                ORDER BY rec.id
+            ) AS rn
+        FROM dataflow.records rec
+        CROSS JOIN dataflow.rules r
+        WHERE
+            rec.source_name = p_source_name
+            AND r.source_name = p_source_name
+            AND rec.transformed IS NOT NULL
+            AND rec.transformed ? r.output_field
+            AND (p_rule_name IS NULL OR r.name = p_rule_name)
+            AND rec.data ? r.field
+    ),
+    aggregated AS (
+        SELECT
+            e.rule_name,
+            e.output_field,
+            e.source_field,
+            e.extracted_value,
+            count(*) AS record_count,
+            jsonb_agg(e.record_data ORDER BY e.rn) FILTER (WHERE e.rn <= 5) AS sample
+        FROM extracted e
+        GROUP BY e.rule_name, e.output_field, e.source_field, e.extracted_value
+    )
+    SELECT
+        a.rule_name,
+        a.output_field,
+        a.source_field,
+        a.extracted_value,
+        a.record_count,
+        a.sample,
+        m.id AS mapping_id,
+        m.output,
+        (m.id IS NOT NULL) AS is_mapped
+    FROM aggregated a
+    LEFT JOIN dataflow.mappings m ON
+        m.source_name = p_source_name
+        AND m.rule_name = a.rule_name
+        AND m.input_value = a.extracted_value
+    ORDER BY a.record_count DESC;
+END;
+$$ LANGUAGE plpgsql;
+
+COMMENT ON FUNCTION get_all_values IS 'All extracted values with record counts and mapping output (single query for All tab)';
+
+------------------------------------------------------
+-- Function: get_unmapped_values
+-- Find extracted values that need mappings
+------------------------------------------------------
+DROP FUNCTION IF EXISTS get_unmapped_values(TEXT, TEXT);
+CREATE FUNCTION get_unmapped_values(
+    p_source_name TEXT,
+    p_rule_name TEXT DEFAULT NULL
+) RETURNS TABLE (
+    rule_name TEXT,
+    output_field TEXT,
+    source_field TEXT,
+    extracted_value JSONB,
+    record_count BIGINT,
+    sample JSONB
+) AS $$
+BEGIN
+    RETURN QUERY
+    WITH extracted AS (
+        SELECT
+            r.name AS rule_name,
+            r.output_field,
+            r.field AS source_field,
+            rec.transformed->r.output_field AS extracted_value,
+            rec.data AS record_data,
+            row_number() OVER (
+                PARTITION BY r.name, rec.transformed->r.output_field
+                ORDER BY rec.id
+            ) AS rn
+        FROM
+            dataflow.records rec
+            CROSS JOIN dataflow.rules r
+        WHERE
+            rec.source_name = p_source_name
+            AND r.source_name = p_source_name
+            AND rec.transformed IS NOT NULL
+            AND rec.transformed ? r.output_field
+            AND (p_rule_name IS NULL OR r.name = p_rule_name)
+            AND rec.data ? r.field
+    )
+    SELECT
+        e.rule_name,
+        e.output_field,
+        e.source_field,
+        e.extracted_value,
+        count(*) AS record_count,
+        jsonb_agg(e.record_data ORDER BY e.rn) FILTER (WHERE e.rn <= 5) AS sample
+    FROM extracted e
+    WHERE NOT EXISTS (
+        SELECT 1 FROM dataflow.mappings m
+        WHERE m.source_name = p_source_name
+          AND m.rule_name = e.rule_name
+          AND m.input_value = e.extracted_value
+    )
+    GROUP BY e.rule_name, e.output_field, e.source_field, e.extracted_value
+    ORDER BY count(*) DESC;
+END;
+$$ LANGUAGE plpgsql;
+
+COMMENT ON FUNCTION get_unmapped_values IS 'Find extracted values that need mappings defined';
+
+------------------------------------------------------
+-- Function: reprocess_records
+-- Clear and reapply transformations
+------------------------------------------------------
+CREATE OR REPLACE FUNCTION reprocess_records(p_source_name TEXT)
+RETURNS JSON AS $$
+    -- Overwrite all records directly — no clear step, mirrors TPS srce_map_overwrite
+    SELECT dataflow.apply_transformations(p_source_name, NULL, TRUE)
+$$ LANGUAGE sql;
+
+COMMENT ON FUNCTION reprocess_records IS 'Clear and reapply all transformations for a source';
+
+------------------------------------------------------
+-- Function: generate_source_view
+-- Build a typed flat view in dfv schema
+------------------------------------------------------
+CREATE OR REPLACE FUNCTION generate_source_view(p_source_name TEXT)
+RETURNS JSON AS $$
+DECLARE
+    v_config  JSONB;
+    v_fields  JSONB;
+    v_field   JSONB;
+    v_cols    TEXT := '';
+    v_sql     TEXT;
+    v_view    TEXT;
+BEGIN
+    SELECT config INTO v_config
+    FROM dataflow.sources
+    WHERE name = p_source_name;
+
+    IF v_config IS NULL OR NOT (v_config ? 'fields') OR jsonb_array_length(v_config->'fields') = 0 THEN
+        RETURN json_build_object('success', false, 'error', 'No schema fields defined for this source');
+    END IF;
+
+    v_fields := v_config->'fields';
+
+    FOR v_field IN SELECT * FROM jsonb_array_elements(v_fields)
+    LOOP
+        IF v_cols != '' THEN v_cols := v_cols || ', '; END IF;
+
+        IF v_field->>'expression' IS NOT NULL THEN
+            -- Computed expression: substitute {fieldname} refs with (transformed->>'fieldname')::type
+            -- e.g. "{Amount} * {sign}" → "(transformed->>'Amount')::numeric * (transformed->>'sign')::numeric"
+            DECLARE
+                v_expr    TEXT := v_field->>'expression';
+                v_ref     TEXT;
+                v_cast    TEXT := COALESCE(NULLIF(v_field->>'type', ''), 'numeric');
+            BEGIN
+                WHILE v_expr ~ '\{[^}]+\}' LOOP
+                    v_ref  := substring(v_expr FROM '\{([^}]+)\}');
+                    v_expr := replace(v_expr, '{' || v_ref || '}',
+                                      format('(transformed->>%L)::numeric', v_ref));
+                END LOOP;
+                v_cols := v_cols || format('%s AS %I', v_expr, v_field->>'name');
+            END;
+        ELSE
+            CASE v_field->>'type'
+                WHEN 'date' THEN
+                    v_cols := v_cols || format('(transformed->>%L)::date AS %I',
+                        v_field->>'name', v_field->>'name');
+                WHEN 'numeric' THEN
+                    v_cols := v_cols || format('(transformed->>%L)::numeric AS %I',
+                        v_field->>'name', v_field->>'name');
+                ELSE
+                    v_cols := v_cols || format('transformed->>%L AS %I',
+                        v_field->>'name', v_field->>'name');
+            END CASE;
+        END IF;
+    END LOOP;
+
+    CREATE SCHEMA IF NOT EXISTS dfv;
+
+    v_view := 'dfv.' || quote_ident(p_source_name);
+
+    EXECUTE format('DROP VIEW IF EXISTS %s', v_view);
+
+    v_sql := format(
+        'CREATE VIEW %s AS SELECT %s FROM dataflow.records WHERE source_name = %L AND transformed IS NOT NULL',
+        v_view, v_cols, p_source_name
+    );
+
+    EXECUTE v_sql;
+
+    RETURN json_build_object('success', true, 'view', v_view, 'sql', v_sql);
+END;
+$$ LANGUAGE plpgsql;
+
+COMMENT ON FUNCTION generate_source_view IS 'Generate a typed flat view in dfv schema from source config.fields';
+
+------------------------------------------------------
+-- Summary
+------------------------------------------------------
+-- Functions: 4 simple, focused functions
+--   1. import_records - Import with deduplication
+--   2. apply_transformations - Apply rules and mappings
+--   3. get_unmapped_values - Find values needing mappings
+--   4. reprocess_records - Re-transform all records
+--
+-- Each function does ONE thing clearly
+-- No complex nested CTEs
+-- Easy to understand and debug
+------------------------------------------------------

database/import.sql

@@ -1,159 +0,0 @@
---
--- Import queries
--- CSV import and the import audit trail; SQL for the import/log endpoints in
--- api/routes/sources.js
---
-
-SET search_path TO dataflow, public;
-
--- ── Import ────────────────────────────────────────────────────────────────────
-
--- Import records, skipping any whose constraint key already exists in the table.
---
--- Dedup is enforced here, in the CTE — there is no unique constraint on
--- constraint_key and ON CONFLICT must never be used. Within one batch every row
--- inserts even if two rows share a constraint key, because banks legitimately send
--- identical-looking transactions (same date, description, amount) on the same day.
--- The key exists only to stop a re-imported overlapping date range from
--- double-counting rows already in the table.
-CREATE OR REPLACE FUNCTION import_records(
-    p_source_name TEXT,
-    p_data JSONB  -- Array of records
-) RETURNS JSON AS $$
-DECLARE
-    v_constraint_fields TEXT[];
-    v_inserted          INTEGER;
-    v_duplicates        INTEGER;
-    v_log_id            INTEGER;
-BEGIN
-    SELECT constraint_fields INTO v_constraint_fields
-    FROM dataflow.sources
-    WHERE name = p_source_name;
-
-    IF v_constraint_fields IS NULL THEN
-        RETURN json_build_object(
-            'success', false,
-            'error', 'Source not found: ' || p_source_name
-        );
-    END IF;
-
-    WITH
-    -- All incoming records with their constraint keys
-    pending AS (
-        SELECT
-            rec.value      AS data,
-            rec.ordinality AS seq,
-            (SELECT jsonb_object_agg(f, rec.value->>f)
-             FROM unnest(v_constraint_fields) AS f) AS constraint_key
-        FROM jsonb_array_elements(p_data) WITH ORDINALITY AS rec
-    ),
-    -- Keys already in the database (excluded)
-    existing AS (
-        SELECT DISTINCT r.constraint_key
-        FROM dataflow.records r
-        INNER JOIN pending p ON p.constraint_key = r.constraint_key
-        WHERE r.source_name = p_source_name
-    ),
-    -- Rows whose constraint key is not yet in the database
-    new_records AS (
-        SELECT p.data, p.constraint_key, p.seq
-        FROM pending p
-        WHERE NOT EXISTS (SELECT 1 FROM existing e WHERE e.constraint_key = p.constraint_key)
-    ),
-    -- Write the log entry
-    log_entry AS (
-        INSERT INTO dataflow.import_log (source_name, records_imported, records_duplicate, info)
-        VALUES (
-            p_source_name,
-            (SELECT count(*) FROM new_records),
-            (SELECT count(*) FROM pending) - (SELECT count(*) FROM new_records),
-            jsonb_build_object(
-                'total',          jsonb_array_length(p_data),
-                'inserted_keys',  (SELECT jsonb_agg(constraint_key ORDER BY constraint_key) FROM new_records),
-                'excluded_keys',  (SELECT jsonb_agg(constraint_key) FROM existing)
-            )
-        )
-        RETURNING id, records_imported, records_duplicate
-    ),
-    -- Insert new records
-    inserted AS (
-        INSERT INTO dataflow.records (source_name, data, constraint_key, import_id)
-        SELECT p_source_name, nr.data, nr.constraint_key, (SELECT id FROM log_entry)
-        FROM new_records nr
-        ORDER BY nr.seq
-        RETURNING id
-    )
-    SELECT le.id, le.records_imported, le.records_duplicate
-    INTO   v_log_id, v_inserted, v_duplicates
-    FROM   log_entry le;
-
-    RETURN json_build_object(
-        'success',    true,
-        'imported',   v_inserted,
-        'duplicates', v_duplicates,
-        'log_id',     v_log_id
-    );
-END;
-$$ LANGUAGE plpgsql;
-
-COMMENT ON FUNCTION import_records IS 'Import records with automatic deduplication';
-
--- ── Audit trail ───────────────────────────────────────────────────────────────
-
-CREATE OR REPLACE FUNCTION get_import_log(p_source_name TEXT)
-RETURNS TABLE (
-    id              INTEGER,
-    source_name     TEXT,
-    records_imported INTEGER,
-    records_duplicate INTEGER,
-    imported_at     TIMESTAMPTZ,
-    info            JSONB
-) AS $$
-    SELECT id, source_name, records_imported, records_duplicate, imported_at, info
-    FROM dataflow.import_log
-    WHERE source_name = p_source_name
-    ORDER BY imported_at DESC;
-$$ LANGUAGE sql;
-
-COMMENT ON FUNCTION get_import_log IS 'Return import history for a source, newest first, including inserted/excluded key lists';
-
-CREATE OR REPLACE FUNCTION get_all_import_logs()
-RETURNS TABLE (
-    id               INTEGER,
-    source_name      TEXT,
-    records_imported INTEGER,
-    records_duplicate INTEGER,
-    imported_at      TIMESTAMPTZ,
-    info             JSONB
-) AS $$
-    SELECT id, source_name, records_imported, records_duplicate, imported_at, info
-    FROM dataflow.import_log
-    ORDER BY imported_at DESC;
-$$ LANGUAGE sql;
-
-COMMENT ON FUNCTION get_all_import_logs IS 'Return import history across all sources, newest first';
-
--- Records are removed by the import_id FK's ON DELETE CASCADE
-CREATE OR REPLACE FUNCTION delete_import(p_log_id INTEGER)
-RETURNS JSON AS $$
-DECLARE
-    v_deleted INTEGER;
-BEGIN
-    IF NOT EXISTS (SELECT 1 FROM dataflow.import_log WHERE id = p_log_id) THEN
-        RETURN json_build_object('success', false, 'error', 'Import log entry not found');
-    END IF;
-
-    SELECT count(*) INTO v_deleted FROM dataflow.records WHERE import_id = p_log_id;
-
-    -- Cascade handles deleting records via FK ON DELETE CASCADE
-    DELETE FROM dataflow.import_log WHERE id = p_log_id;
-
-    RETURN json_build_object(
-        'success',         true,
-        'records_deleted', v_deleted,
-        'log_id',          p_log_id
-    );
-END;
-$$ LANGUAGE plpgsql;
-
-COMMENT ON FUNCTION delete_import IS 'Delete all records belonging to an import batch and remove the log entry';

database/migrate_input_value_jsonb.sql

@@ -0,0 +1,22 @@
+--
+-- Migration: Change mappings.input_value from TEXT to JSONB
+-- Allows multi-capture-group regex results to be used as mapping keys
+--
+
+SET search_path TO dataflow, public;
+
+-- Drop dependent constraint and index first
+ALTER TABLE dataflow.mappings DROP CONSTRAINT mappings_source_name_rule_name_input_value_key;
+DROP INDEX IF EXISTS dataflow.idx_mappings_input;
+
+-- Convert column: existing TEXT values become JSONB strings e.g. "MEIJER"
+ALTER TABLE dataflow.mappings
+    ALTER COLUMN input_value TYPE JSONB
+    USING to_jsonb(input_value);
+
+-- Recreate constraint and index
+ALTER TABLE dataflow.mappings
+    ADD CONSTRAINT mappings_source_name_rule_name_input_value_key
+    UNIQUE (source_name, rule_name, input_value);
+
+CREATE INDEX idx_mappings_input ON dataflow.mappings(source_name, rule_name, input_value);

database/migrate_tps.sql

@@ -0,0 +1,121 @@
+--
+-- TPS → Dataflow Migration
+--
+-- Migrates sources, rules, mappings, and records from the TPS system.
+-- Run against the dataflow database:
+--   PGPASSWORD=dataflow psql -U dataflow -d dataflow -h localhost -f database/migrate_tps.sql
+--
+-- Existing rows are skipped (ON CONFLICT DO NOTHING) so the script is safe to re-run.
+-- NOTE: dcard already configured in dataflow will NOT be overwritten.
+--
+
+SET search_path TO dataflow, public;
+
+CREATE EXTENSION IF NOT EXISTS dblink;
+
+-- Connection string to the TPS database
+\set tps_conn 'host=192.168.1.110 dbname=ubm user=api password=gyaswddh1983'
+
+\echo ''
+\echo '=== 1. Sources ==='
+
+INSERT INTO dataflow.sources (name, constraint_fields, config)
+SELECT
+    srce AS name,
+    -- Strip {} wrappers from constraint paths → constraint field names
+    ARRAY(
+        SELECT regexp_replace(c, '^\{|\}$', '', 'g')
+        FROM jsonb_array_elements_text(defn->'constraint') AS c
+    ) AS constraint_fields,
+    -- Build config.fields from the first schema (index 0 = "mapped" for dcard, "default" for others)
+    jsonb_build_object('fields',
+        (SELECT jsonb_agg(
+            jsonb_build_object(
+                'name', regexp_replace(col->>'path', '^\{|\}$', '', 'g'),
+                'type', COALESCE(NULLIF(col->>'type', ''), 'text')
+            ) ORDER BY ord
+        )
+        FROM jsonb_array_elements(defn->'schemas'->0->'columns')
+             WITH ORDINALITY AS t(col, ord)
+        )
+    ) AS config
+FROM dblink(:'tps_conn',
+    'SELECT srce, defn FROM tps.srce'
+) AS t(srce TEXT, defn JSONB)
+ON CONFLICT (name) DO NOTHING;
+
+SELECT name, constraint_fields, jsonb_array_length(config->'fields') AS field_count
+FROM dataflow.sources ORDER BY name;
+
+\echo ''
+\echo '=== 2. Rules ==='
+
+INSERT INTO dataflow.rules
+    (source_name, name, field, pattern, output_field, function_type, flags, replace_value, sequence, enabled, retain)
+SELECT
+    srce                                                                AS source_name,
+    target                                                              AS name,
+    -- Strip {} from the input field key
+    regexp_replace(regex->'regex'->'defn'->0->>'key', '^\{|\}$', '', 'g') AS field,
+    regex->'regex'->'defn'->0->>'regex'                                AS pattern,
+    regex->'regex'->'defn'->0->>'field'                                AS output_field,
+    COALESCE(NULLIF(regex->'regex'->>'function', ''), 'extract')       AS function_type,
+    COALESCE(regex->'regex'->'defn'->0->>'flag', '')                   AS flags,
+    ''                                                                  AS replace_value,
+    seq                                                                 AS sequence,
+    true                                                                AS enabled,
+    (regex->'regex'->'defn'->0->>'retain') = 'y'                       AS retain
+FROM dblink(:'tps_conn',
+    'SELECT srce, target, seq, regex FROM tps.map_rm'
+) AS t(srce TEXT, target TEXT, seq INT, regex JSONB)
+ON CONFLICT (source_name, name) DO NOTHING;
+
+SELECT source_name, name, field, pattern, output_field, sequence
+FROM dataflow.rules ORDER BY source_name, sequence;
+
+\echo ''
+\echo '=== 3. Mappings ==='
+
+INSERT INTO dataflow.mappings (source_name, rule_name, input_value, output)
+SELECT
+    srce        AS source_name,
+    target      AS rule_name,
+    -- retval is {"f20": "<extracted string>"} — pull out the value as JSONB
+    (SELECT value FROM jsonb_each(retval) LIMIT 1) AS input_value,
+    map         AS output
+FROM dblink(:'tps_conn',
+    'SELECT srce, target, retval, map FROM tps.map_rv'
+) AS t(srce TEXT, target TEXT, retval JSONB, map JSONB)
+ON CONFLICT (source_name, rule_name, input_value) DO NOTHING;
+
+SELECT source_name, rule_name, COUNT(*) AS mapping_count
+FROM dataflow.mappings GROUP BY source_name, rule_name ORDER BY source_name, rule_name;
+
+\echo ''
+\echo '=== 4. Records ==='
+\echo '    (13 000+ rows — may take a moment)'
+
+INSERT INTO dataflow.records (source_name, data, constraint_key, transformed, imported_at, transformed_at)
+SELECT
+    t.srce                                                  AS source_name,
+    t.rec                                                   AS data,
+    (SELECT jsonb_object_agg(f, t.rec->>f) FROM unnest(s.constraint_fields) AS f) AS constraint_key,
+    t.allj                                                  AS transformed,
+    CURRENT_TIMESTAMP                                       AS imported_at,
+    CASE WHEN t.allj IS NOT NULL THEN CURRENT_TIMESTAMP END AS transformed_at
+FROM dblink(:'tps_conn',
+    'SELECT srce, rec, allj FROM tps.trans'
+) AS t(srce TEXT, rec JSONB, allj JSONB)
+JOIN dataflow.sources s ON s.name = t.srce
+ON CONFLICT (source_name, constraint_key) DO NOTHING;
+
+SELECT source_name, COUNT(*) AS records, COUNT(transformed) AS transformed
+FROM dataflow.records GROUP BY source_name ORDER BY source_name;
+
+\echo ''
+\echo '=== Migration complete ==='
+SELECT
+    (SELECT COUNT(*) FROM dataflow.sources)  AS sources,
+    (SELECT COUNT(*) FROM dataflow.rules)    AS rules,
+    (SELECT COUNT(*) FROM dataflow.mappings) AS mappings,
+    (SELECT COUNT(*) FROM dataflow.records)  AS records;

database/queries/records.sql

@@ -41,45 +41,23 @@ $$ LANGUAGE sql STABLE;
 
 -- ── Overrides ─────────────────────────────────────────────────────────────────
 
--- Store manual overrides. Overrides stay in their own column — never merged into
+-- Store manual overrides and immediately merge into transformed
--- transformed — so reprocessing rules cannot clobber a manual edit.
+CREATE OR REPLACE FUNCTION set_record_overrides(p_id INT, p_overrides JSONB)
-DROP FUNCTION IF EXISTS set_record_overrides(INTEGER, JSONB);
+RETURNS dataflow.records AS $$
-CREATE OR REPLACE FUNCTION set_record_overrides(p_id INTEGER, p_overrides JSONB)
+    UPDATE dataflow.records
-RETURNS JSON AS $$
+    SET overrides  = CASE WHEN p_overrides = '{}'::jsonb THEN NULL ELSE p_overrides END,
-    WITH updated AS (
+        transformed = COALESCE(transformed, data) || COALESCE(p_overrides, '{}'::jsonb)
-        UPDATE dataflow.records
+    WHERE id = p_id
-        SET overrides = CASE WHEN p_overrides = '{}'::jsonb THEN NULL ELSE p_overrides END
+    RETURNING *;
-        WHERE id = p_id
-        RETURNING *
-    )
-    SELECT row_to_json(updated) FROM updated;
 $$ LANGUAGE sql;
 
--- Merge overrides into multiple records at once; returns actual updated count
+-- Clear overrides; caller should reprocess to restore computed transformed value
-DROP FUNCTION IF EXISTS bulk_set_record_overrides(TEXT, INTEGER[], JSONB);
+CREATE OR REPLACE FUNCTION clear_record_overrides(p_id INT)
-CREATE OR REPLACE FUNCTION bulk_set_record_overrides(p_source_name TEXT, p_ids INTEGER[], p_overrides JSONB)
+RETURNS dataflow.records AS $$
-RETURNS JSON AS $$
+    UPDATE dataflow.records
-    WITH updated AS (
+    SET overrides = NULL
-        UPDATE dataflow.records
+    WHERE id = p_id
-        SET overrides = COALESCE(overrides, '{}'::jsonb) || p_overrides
+    RETURNING *;
-        WHERE id = ANY(p_ids)
-          AND source_name = p_source_name
-        RETURNING id
-    )
-    SELECT json_build_object('updated', count(*)) FROM updated;
-$$ LANGUAGE sql;
-
--- Clear overrides; the computed values in transformed are untouched
-DROP FUNCTION IF EXISTS clear_record_overrides(INTEGER);
-CREATE OR REPLACE FUNCTION clear_record_overrides(p_id INTEGER)
-RETURNS JSON AS $$
-    WITH updated AS (
-        UPDATE dataflow.records
-        SET overrides = NULL
-        WHERE id = p_id
-        RETURNING *
-    )
-    SELECT row_to_json(updated) FROM updated;
 $$ LANGUAGE sql;
 
 -- ── Delete ────────────────────────────────────────────────────────────────────

database/queries/sources.sql

@@ -40,6 +40,8 @@ RETURNS TEXT AS $$
     DELETE FROM dataflow.sources WHERE name = p_name RETURNING name;
 $$ LANGUAGE sql;
 
+-- ── Import log ────────────────────────────────────────────────────────────────
+
 -- ── Stats ─────────────────────────────────────────────────────────────────────
 
 CREATE OR REPLACE FUNCTION get_source_stats(p_source_name TEXT)
@@ -65,16 +67,6 @@ RETURNS TABLE (key TEXT, origins TEXT[]) AS $$
         SELECT jsonb_object_keys(data) AS key, 'raw'     AS origin
         FROM dataflow.records WHERE source_name = p_source_name
         UNION ALL
-        -- transformed/overrides are read straight off the records so that keys with
-        -- no surviving rule or mapping (e.g. a manual override) still get listed
-        SELECT jsonb_object_keys(transformed) AS key, 'transformed' AS origin
-        FROM dataflow.records
-        WHERE source_name = p_source_name AND transformed IS NOT NULL
-        UNION ALL
-        SELECT jsonb_object_keys(overrides) AS key, 'override' AS origin
-        FROM dataflow.records
-        WHERE source_name = p_source_name AND overrides IS NOT NULL
-        UNION ALL
         SELECT output_field AS key, 'rule: ' || name     AS origin
         FROM dataflow.rules WHERE source_name = p_source_name
         UNION ALL
@@ -169,7 +161,6 @@ BEGIN
         RETURN json_build_object('success', false, 'error', 'No schema fields defined for this source');
     END IF;
 
-    -- Columns read from r, the merged data || transformed || overrides object
     FOR v_field IN SELECT * FROM jsonb_array_elements(v_config->'fields') LOOP
         IF v_cols != '' THEN v_cols := v_cols || ', '; END IF;
 
@@ -180,27 +171,24 @@ BEGIN
             BEGIN
                 WHILE v_expr ~ '\{[^}]+\}' LOOP
                     v_ref  := substring(v_expr FROM '\{([^}]+)\}');
-                    v_expr := replace(v_expr, '{' || v_ref || '}', format('(r->>%L)::numeric', v_ref));
+                    v_expr := replace(v_expr, '{' || v_ref || '}', format('(transformed->>%L)::numeric', v_ref));
                 END LOOP;
                 v_cols := v_cols || format('%s AS %I', v_expr, v_field->>'name');
             END;
         ELSE
             CASE v_field->>'type'
-                WHEN 'date'    THEN v_cols := v_cols || format('(r->>%L)::date    AS %I', v_field->>'name', v_field->>'name');
+                WHEN 'date'    THEN v_cols := v_cols || format('(transformed->>%L)::date    AS %I', v_field->>'name', v_field->>'name');
-                WHEN 'numeric' THEN v_cols := v_cols || format('(r->>%L)::numeric AS %I', v_field->>'name', v_field->>'name');
+                WHEN 'numeric' THEN v_cols := v_cols || format('(transformed->>%L)::numeric AS %I', v_field->>'name', v_field->>'name');
-                ELSE                v_cols := v_cols || format('r->>%L             AS %I', v_field->>'name', v_field->>'name');
+                ELSE                v_cols := v_cols || format('transformed->>%L             AS %I', v_field->>'name', v_field->>'name');
             END CASE;
         END IF;
     END LOOP;
 
     CREATE SCHEMA IF NOT EXISTS dfv;
     v_view := 'dfv.' || quote_ident(p_source_name);
-    EXECUTE format('DROP VIEW IF EXISTS %s CASCADE', v_view);
+    EXECUTE format('DROP VIEW IF EXISTS %s', v_view);
     v_sql := format(
-        'CREATE VIEW %s AS SELECT id, _overridden, %s FROM ('
+        'CREATE VIEW %s AS SELECT id, overrides IS NOT NULL AS _overridden, %s FROM dataflow.records WHERE source_name = %L AND transformed IS NOT NULL',
-        || 'SELECT id, overrides IS NOT NULL AS _overridden, '
-        || 'data || COALESCE(transformed, ''{}''::jsonb) || COALESCE(overrides, ''{}''::jsonb) AS r '
-        || 'FROM dataflow.records WHERE source_name = %L AND transformed IS NOT NULL) rec',
         v_view, v_cols, p_source_name
     );
     EXECUTE v_sql;

database/schema.sql

@@ -37,27 +37,26 @@ CREATE TABLE records (
     -- Data
     data JSONB NOT NULL,                    -- Original imported data
     constraint_key JSONB,                   -- Fields that uniquely identify this record (set on import)
-    transformed JSONB,                      -- Rule/mapping output fields only (delta, not raw data)
+    transformed JSONB,                      -- Data after transformations applied
-    overrides JSONB,                        -- Manual user overrides (highest precedence)
 
     -- Metadata
-    import_id INTEGER REFERENCES import_log(id) ON DELETE CASCADE,
+    import_id INTEGER REFERENCES import_log(id) ON DELETE CASCADE,  -- Which import batch this came from
     imported_at TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP,
-    transformed_at TIMESTAMPTZ
+    transformed_at TIMESTAMPTZ,
+
+
 );
 
 COMMENT ON TABLE records IS 'Imported records with raw and transformed data';
-COMMENT ON COLUMN records.data IS 'Original data as imported — never mutated after import';
+COMMENT ON COLUMN records.data IS 'Original data as imported';
 COMMENT ON COLUMN records.constraint_key IS 'JSONB object of constraint field values — uniquely identifies this record within its source';
-COMMENT ON COLUMN records.transformed IS 'Rule/mapping output fields only (delta); merge as data || transformed || overrides for final values';
+COMMENT ON COLUMN records.transformed IS 'Data after applying transformation rules';
-COMMENT ON COLUMN records.overrides IS 'Manual user overrides; highest precedence in data || transformed || overrides merge';
 
 -- Indexes
 CREATE INDEX idx_records_source ON records(source_name);
 CREATE INDEX idx_records_constraint ON records USING gin(constraint_key);
 CREATE INDEX idx_records_data ON records USING gin(data);
 CREATE INDEX idx_records_transformed ON records USING gin(transformed);
-CREATE INDEX idx_records_overrides ON records USING gin(overrides) WHERE overrides IS NOT NULL;
 
 ------------------------------------------------------
 -- Table: rules

database/stacks.sql

@@ -1,472 +0,0 @@
---
--- Stacks queries
--- All SQL for api/routes/stacks.js
---
-
-SET search_path TO dataflow, public;
-
-------------------------------------------------------
--- Tables
-------------------------------------------------------
-
-CREATE TABLE IF NOT EXISTS dataflow.stacks (
-    name            TEXT PRIMARY KEY,
-    label           TEXT,
-    -- Ordered canonical field definitions: [{name, label, type}]
-    -- type: 'text' | 'numeric' | 'date'
-    fields          JSONB NOT NULL DEFAULT '[]',
-    -- Running balance config
-    amount_field    TEXT,           -- canonical field to sum for running balance
-    date_field      TEXT,           -- canonical field to order by
-    balance_offset  NUMERIC DEFAULT 0,  -- added to running sum (calibration)
-    created_at      TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP
-);
-
-CREATE TABLE IF NOT EXISTS dataflow.stack_sources (
-    id              SERIAL PRIMARY KEY,
-    stack_name      TEXT NOT NULL REFERENCES dataflow.stacks(name) ON DELETE CASCADE,
-    source_name     TEXT NOT NULL REFERENCES dataflow.sources(name) ON DELETE CASCADE,
-    -- Maps other canonical field names → source view column names (not amount/date — those are explicit)
-    field_map       JSONB NOT NULL DEFAULT '{}',
-    -- Which column in dfv.{source} is the amount, and its sign (+1/-1)
-    amount_field    TEXT,
-    amount_sign     INTEGER NOT NULL DEFAULT 1,
-    -- Which column in dfv.{source} is the date
-    date_field      TEXT,
-    -- Calibration offset added to this source's running balance
-    balance_offset  NUMERIC NOT NULL DEFAULT 0,
-    UNIQUE (stack_name, source_name)
-);
-
--- Migrations: add columns that may be missing from earlier deploys
-ALTER TABLE dataflow.stack_sources ADD COLUMN IF NOT EXISTS balance_offset NUMERIC NOT NULL DEFAULT 0;
-ALTER TABLE dataflow.stack_sources ADD COLUMN IF NOT EXISTS amount_field TEXT;
-ALTER TABLE dataflow.stack_sources ADD COLUMN IF NOT EXISTS date_field TEXT;
-ALTER TABLE dataflow.stack_sources ADD COLUMN IF NOT EXISTS seq INTEGER NOT NULL DEFAULT 0;
-
--- Seed seq from insertion order for existing rows
-UPDATE dataflow.stack_sources ss
-SET seq = sub.rn
-FROM (
-    SELECT id, ROW_NUMBER() OVER (PARTITION BY stack_name ORDER BY id) AS rn
-    FROM dataflow.stack_sources WHERE seq = 0
-) sub
-WHERE ss.id = sub.id AND ss.seq = 0;
-
--- Drop old signatures before recreating
-DROP FUNCTION IF EXISTS calibrate_balance(TEXT, DATE, NUMERIC);
-DROP FUNCTION IF EXISTS upsert_stack_source(TEXT, TEXT, JSONB, INTEGER, NUMERIC);
-DROP FUNCTION IF EXISTS generate_stack_view(TEXT);
-
-------------------------------------------------------
--- Function: list_stacks
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION list_stacks()
-RETURNS TABLE (
-    name            TEXT,
-    label           TEXT,
-    fields          JSONB,
-    amount_field    TEXT,
-    date_field      TEXT,
-    balance_offset  NUMERIC,
-    source_count    BIGINT,
-    created_at      TIMESTAMPTZ
-) AS $$
-    SELECT
-        s.name, s.label, s.fields,
-        s.amount_field, s.date_field, s.balance_offset,
-        count(ss.id) AS source_count,
-        s.created_at
-    FROM dataflow.stacks s
-    LEFT JOIN dataflow.stack_sources ss ON ss.stack_name = s.name
-    GROUP BY s.name, s.label, s.fields, s.amount_field, s.date_field, s.balance_offset, s.created_at
-    ORDER BY s.name;
-$$ LANGUAGE sql STABLE;
-
-------------------------------------------------------
--- Function: get_stack
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION get_stack(p_name TEXT)
-RETURNS TABLE (
-    name            TEXT,
-    label           TEXT,
-    fields          JSONB,
-    amount_field    TEXT,
-    date_field      TEXT,
-    balance_offset  NUMERIC,
-    created_at      TIMESTAMPTZ,
-    sources         JSONB
-) AS $$
-    SELECT
-        s.name, s.label, s.fields,
-        s.amount_field, s.date_field, s.balance_offset,
-        s.created_at,
-        COALESCE(jsonb_agg(
-            jsonb_build_object(
-                'id',             ss.id,
-                'source_name',    ss.source_name,
-                'field_map',      ss.field_map,
-                'amount_field',   ss.amount_field,
-                'amount_sign',    ss.amount_sign,
-                'date_field',     ss.date_field,
-                'balance_offset', ss.balance_offset,
-                'seq',            ss.seq
-            ) ORDER BY ss.seq, ss.id
-        ) FILTER (WHERE ss.id IS NOT NULL), '[]')
-    FROM dataflow.stacks s
-    LEFT JOIN dataflow.stack_sources ss ON ss.stack_name = s.name
-    WHERE s.name = p_name
-    GROUP BY s.name, s.label, s.fields, s.amount_field, s.date_field, s.balance_offset, s.created_at;
-$$ LANGUAGE sql STABLE;
-
-------------------------------------------------------
--- Function: create_stack
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION create_stack(
-    p_name          TEXT,
-    p_label         TEXT DEFAULT NULL,
-    p_fields        JSONB DEFAULT '[]',
-    p_amount_field  TEXT DEFAULT NULL,
-    p_date_field    TEXT DEFAULT NULL,
-    p_balance_offset NUMERIC DEFAULT 0
-) RETURNS dataflow.stacks AS $$
-    INSERT INTO dataflow.stacks (name, label, fields, amount_field, date_field, balance_offset)
-    VALUES (p_name, p_label, p_fields, p_amount_field, p_date_field, p_balance_offset)
-    RETURNING *;
-$$ LANGUAGE sql;
-
-------------------------------------------------------
--- Function: update_stack
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION update_stack(
-    p_name           TEXT,
-    p_label          TEXT DEFAULT NULL,
-    p_fields         JSONB DEFAULT NULL,
-    p_amount_field   TEXT DEFAULT NULL,
-    p_date_field     TEXT DEFAULT NULL,
-    p_balance_offset NUMERIC DEFAULT NULL
-) RETURNS dataflow.stacks AS $$
-    UPDATE dataflow.stacks SET
-        label          = COALESCE(p_label, label),
-        fields         = COALESCE(p_fields, fields),
-        amount_field   = COALESCE(p_amount_field, amount_field),
-        date_field     = COALESCE(p_date_field, date_field),
-        balance_offset = COALESCE(p_balance_offset, balance_offset)
-    WHERE name = p_name
-    RETURNING *;
-$$ LANGUAGE sql;
-
-------------------------------------------------------
--- Function: delete_stack
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION delete_stack(p_name TEXT)
-RETURNS TABLE (name TEXT) AS $$
-    DELETE FROM dataflow.stacks WHERE name = p_name RETURNING name;
-$$ LANGUAGE sql;
-
-------------------------------------------------------
--- Function: upsert_stack_source
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION upsert_stack_source(
-    p_stack_name      TEXT,
-    p_source_name     TEXT,
-    p_field_map       JSONB DEFAULT '{}',
-    p_amount_sign     INTEGER DEFAULT 1,
-    p_balance_offset  NUMERIC DEFAULT 0,
-    p_amount_field    TEXT DEFAULT NULL,
-    p_date_field      TEXT DEFAULT NULL
-) RETURNS dataflow.stack_sources AS $$
-    INSERT INTO dataflow.stack_sources (stack_name, source_name, field_map, amount_sign, balance_offset, amount_field, date_field, seq)
-    VALUES (
-        p_stack_name, p_source_name, p_field_map, p_amount_sign, p_balance_offset, p_amount_field, p_date_field,
-        (SELECT COALESCE(MAX(seq), 0) + 1 FROM dataflow.stack_sources WHERE stack_name = p_stack_name)
-    )
-    ON CONFLICT (stack_name, source_name) DO UPDATE SET
-        field_map       = EXCLUDED.field_map,
-        amount_sign     = EXCLUDED.amount_sign,
-        balance_offset  = EXCLUDED.balance_offset,
-        amount_field    = EXCLUDED.amount_field,
-        date_field      = EXCLUDED.date_field
-    RETURNING *;
-$$ LANGUAGE sql;
-
-------------------------------------------------------
--- Function: remove_stack_source
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION remove_stack_source(p_stack_name TEXT, p_source_name TEXT)
-RETURNS TABLE (source_name TEXT) AS $$
-    DELETE FROM dataflow.stack_sources
-    WHERE stack_name = p_stack_name AND source_name = p_source_name
-    RETURNING source_name;
-$$ LANGUAGE sql;
-
-------------------------------------------------------
--- Function: calibrate_balance
--- Queries dfv.{source} directly using per-source amount/date fields.
--- No stack view required.
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION calibrate_balance(
-    p_stack_name    TEXT,
-    p_source_name   TEXT,
-    p_as_of_date    DATE,
-    p_known_balance NUMERIC
-) RETURNS JSON AS $$
-DECLARE
-    v_src       dataflow.stack_sources%ROWTYPE;
-    v_running   NUMERIC;
-    v_sql       TEXT;
-BEGIN
-    SELECT * INTO v_src
-    FROM dataflow.stack_sources
-    WHERE stack_name = p_stack_name AND source_name = p_source_name;
-
-    IF NOT FOUND THEN
-        RETURN json_build_object('success', false, 'error', 'Source not in stack');
-    END IF;
-    IF v_src.amount_field IS NULL OR v_src.date_field IS NULL THEN
-        RETURN json_build_object('success', false, 'error', 'Set amount and date fields on this source first');
-    END IF;
-
-    BEGIN
-        IF p_as_of_date IS NULL THEN
-            v_sql := format(
-                'SELECT COALESCE(SUM(%I * %s), 0) FROM dfv.%I',
-                v_src.amount_field, v_src.amount_sign, p_source_name
-            );
-        ELSE
-            v_sql := format(
-                'SELECT COALESCE(SUM(%I * %s), 0) FROM dfv.%I WHERE %I <= %L::date',
-                v_src.amount_field, v_src.amount_sign, p_source_name, v_src.date_field, p_as_of_date
-            );
-        END IF;
-        EXECUTE v_sql INTO v_running;
-    EXCEPTION WHEN undefined_table THEN
-        RETURN json_build_object('success', false, 'error', 'Source view not found — generate the source view first');
-    END;
-
-    RETURN json_build_object(
-        'success',          true,
-        'source',           p_source_name,
-        'as_of_date',       p_as_of_date,
-        'known_balance',    p_known_balance,
-        'computed_sum',     v_running,
-        'suggested_offset', p_known_balance - v_running
-    );
-END;
-$$ LANGUAGE plpgsql STABLE;
-
-------------------------------------------------------
--- Function: generate_stack_view
--- Builds a WITH ... UNION ALL view in dfv schema from existing dfv source views.
--- Each source CTE applies amount_sign and computes a per-source running balance.
--- Outer SELECT adds net_balance across all sources.
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION generate_stack_view(p_stack_name TEXT, p_dry_run BOOLEAN DEFAULT false)
-RETURNS JSON AS $$
-DECLARE
-    v_stack      dataflow.stacks%ROWTYPE;
-    v_src        dataflow.stack_sources%ROWTYPE;
-    v_field      JSONB;
-    v_ctes       TEXT[] := '{}';
-    v_cte_names  TEXT[] := '{}';
-    v_select     TEXT;
-    v_col        TEXT;
-    v_src_field  TEXT;
-    v_amt_src    TEXT;
-    v_date_src   TEXT;
-    v_view          TEXT;
-    v_sql           TEXT;
-    v_has_bal       BOOLEAN;
-    v_canon_cols    TEXT;
-    v_src_bal_cols  TEXT;
-    v_total_offset  NUMERIC := 0;
-    v_cascade_stale TEXT[];
-BEGIN
-    SELECT * INTO v_stack FROM dataflow.stacks WHERE name = p_stack_name;
-    IF NOT FOUND THEN
-        RETURN json_build_object('success', false, 'error', 'Stack not found');
-    END IF;
-
-    v_has_bal := v_stack.amount_field IS NOT NULL AND v_stack.date_field IS NOT NULL;
-
-    -- Build one CTE per source querying dfv.{source} directly
-    FOR v_src IN
-        SELECT * FROM dataflow.stack_sources WHERE stack_name = p_stack_name ORDER BY seq, id
-    LOOP
-        v_select := format('SELECT %L AS _source, id AS _id', v_src.source_name);
-
-        FOR v_field IN SELECT * FROM jsonb_array_elements(v_stack.fields)
-        LOOP
-            v_col := v_field->>'name';
-
-            IF v_has_bal AND v_col = v_stack.amount_field THEN
-                -- Use per-source amount_field with sign applied
-                IF v_src.amount_field IS NULL THEN
-                    v_select := v_select || format(', NULL::%s AS %I', v_field->>'type', v_col);
-                ELSE
-                    v_select := v_select || format(', %I * %s AS %I', v_src.amount_field, v_src.amount_sign, v_col);
-                END IF;
-            ELSIF v_has_bal AND v_col = v_stack.date_field THEN
-                -- Use per-source date_field
-                IF v_src.date_field IS NULL THEN
-                    v_select := v_select || format(', NULL::date AS %I', v_col);
-                ELSE
-                    v_select := v_select || format(', %I AS %I', v_src.date_field, v_col);
-                END IF;
-            ELSE
-                -- Other canonical fields: use field_map or same name, NULL if column doesn't exist
-                v_src_field := COALESCE(v_src.field_map->>v_col, v_col);
-                IF EXISTS (
-                    SELECT 1 FROM information_schema.columns
-                    WHERE table_schema = 'dfv'
-                      AND table_name = v_src.source_name
-                      AND column_name = v_src_field
-                ) THEN
-                    v_select := v_select || format(', %I AS %I', v_src_field, v_col);
-                ELSE
-                    v_select := v_select || format(', NULL::text AS %I', v_col);
-                END IF;
-            END IF;
-        END LOOP;
-
-        v_select := v_select || format(' FROM dfv.%I', v_src.source_name);
-
-        v_ctes      := v_ctes      || format('%I AS (%s)', v_src.source_name, v_select);
-        v_cte_names := v_cte_names || quote_ident(v_src.source_name);
-
-        -- Accumulate carried-forward source balance column and total offset
-        IF v_has_bal THEN
-            IF v_src_bal_cols IS NOT NULL THEN v_src_bal_cols := v_src_bal_cols || ', '; END IF;
-            v_src_bal_cols := COALESCE(v_src_bal_cols, '') || format(
-                'SUM(CASE WHEN _source = %L THEN %I END) OVER (ORDER BY %I ASC, _id ASC) + %s AS %I',
-                v_src.source_name, v_stack.amount_field, v_stack.date_field,
-                v_src.balance_offset, v_src.source_name || '_balance'
-            );
-            v_total_offset := v_total_offset + v_src.balance_offset;
-        END IF;
-    END LOOP;
-
-    IF array_length(v_ctes, 1) IS NULL THEN
-        RETURN json_build_object('success', false, 'error', 'Stack has no sources');
-    END IF;
-
-    v_view := 'dfv.' || quote_ident(p_stack_name);
-
-    v_canon_cols := (
-        SELECT string_agg(quote_ident(f->>'name'), ', ')
-        FROM jsonb_array_elements(v_stack.fields) f
-    );
-
-    IF v_has_bal THEN
-        v_sql := format(
-            'CREATE VIEW %s AS '
-            'WITH %s, _stacked AS (SELECT * FROM %s) '
-            'SELECT _source, _id, %s, '
-            '%s, '
-            'SUM(%I) OVER (ORDER BY %I ASC, _id ASC) + %s AS net_balance '
-            'FROM _stacked ORDER BY %I DESC, _id DESC',
-            v_view,
-            array_to_string(v_ctes, ', '),
-            array_to_string(v_cte_names, ' UNION ALL SELECT * FROM '),
-            v_canon_cols,
-            v_src_bal_cols,
-            v_stack.amount_field,
-            v_stack.date_field,
-            v_total_offset,
-            v_stack.date_field
-        );
-    ELSE
-        v_sql := format(
-            'CREATE VIEW %s AS '
-            'WITH %s, _stacked AS (SELECT * FROM %s) '
-            'SELECT _source, _id, %s FROM _stacked',
-            v_view,
-            array_to_string(v_ctes, ', '),
-            array_to_string(v_cte_names, ' UNION ALL SELECT * FROM '),
-            v_canon_cols
-        );
-    END IF;
-
-    IF NOT p_dry_run THEN
-        CREATE SCHEMA IF NOT EXISTS dfv;
-        EXECUTE format('DROP VIEW IF EXISTS %s CASCADE', v_view);
-        EXECUTE v_sql;
-
-        -- Detect stacks whose views were dropped by CASCADE and mark them stale
-        SELECT array_agg(s.name) INTO v_cascade_stale
-        FROM dataflow.stacks s
-        WHERE s.name != p_stack_name
-          AND s.view_generated_at IS NOT NULL
-          AND NOT EXISTS (
-              SELECT 1 FROM pg_views v
-              WHERE v.schemaname = 'dfv' AND v.viewname = s.name
-          );
-
-        UPDATE dataflow.stacks SET view_generated_at = NULL
-        WHERE name = ANY(v_cascade_stale);
-    END IF;
-
-    RETURN json_build_object(
-        'success', true,
-        'view', v_view,
-        'sql', v_sql,
-        'cascade_stale', COALESCE(to_json(v_cascade_stale), '[]'::json)
-    );
-END;
-$$ LANGUAGE plpgsql;
-
-------------------------------------------------------
--- Function: get_stack_balance
--- Returns the current running balance (last row of the generated view)
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION get_stack_balance(p_stack_name TEXT)
-RETURNS JSON AS $$
-DECLARE
-    v_stack     dataflow.stacks%ROWTYPE;
-    v_balance   NUMERIC;
-    v_view      TEXT;
-    v_sql       TEXT;
-BEGIN
-    SELECT * INTO v_stack FROM dataflow.stacks WHERE name = p_stack_name;
-    IF NOT FOUND THEN
-        RETURN json_build_object('success', false, 'error', 'Stack not found');
-    END IF;
-    IF v_stack.amount_field IS NULL OR v_stack.date_field IS NULL THEN
-        RETURN json_build_object('success', false, 'error', 'amount_field and date_field must be set');
-    END IF;
-
-    v_view := 'dfv.' || quote_ident(p_stack_name);
-
-    BEGIN
-        v_sql := format(
-            'SELECT net_balance FROM %s ORDER BY %I DESC, _id DESC LIMIT 1',
-            v_view, v_stack.date_field
-        );
-        EXECUTE v_sql INTO v_balance;
-    EXCEPTION WHEN undefined_table THEN
-        RETURN json_build_object('success', false, 'error', 'View not generated yet — click Generate first');
-    END;
-
-    RETURN json_build_object('success', true, 'balance', v_balance);
-END;
-$$ LANGUAGE plpgsql STABLE;
-
-COMMENT ON FUNCTION generate_stack_view(TEXT, BOOLEAN) IS 'Generate a UNION ALL view in dfv schema combining multiple sources with optional running balance; p_dry_run=true returns SQL without executing';
-COMMENT ON FUNCTION calibrate_balance IS 'Given a known good balance at a date, compute the offset to add to balance_offset';
-COMMENT ON FUNCTION get_stack_balance IS 'Return the current running balance (last row) from the generated dfv view';
-
-------------------------------------------------------
--- Function: reorder_stack_sources
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION reorder_stack_sources(p_stack_name TEXT, p_source_names TEXT[])
-RETURNS VOID AS $$
-DECLARE
-    i INTEGER;
-BEGIN
-    FOR i IN 1..array_length(p_source_names, 1) LOOP
-        UPDATE dataflow.stack_sources
-        SET seq = i
-        WHERE stack_name = p_stack_name AND source_name = p_source_names[i];
-    END LOOP;
-END;
-$$ LANGUAGE plpgsql;

database/status.sql

@@ -1,86 +0,0 @@
---
--- Status tracking: view_generated_at on sources and stacks
--- Cleared by triggers when definitions change; set by API when views are generated.
---
-
-SET search_path TO dataflow, public;
-
--- Add view_generated_at columns
-ALTER TABLE dataflow.sources ADD COLUMN IF NOT EXISTS view_generated_at TIMESTAMPTZ;
-ALTER TABLE dataflow.stacks  ADD COLUMN IF NOT EXISTS view_generated_at TIMESTAMPTZ;
-
-------------------------------------------------------
--- Trigger: clear source view_generated_at when config (field definitions) changes
--- Rules and mappings affect transformed data, not view structure — no trigger needed there
-------------------------------------------------------
-DROP TRIGGER IF EXISTS trg_rules_changed ON dataflow.rules;
-DROP TRIGGER IF EXISTS trg_mappings_changed ON dataflow.mappings;
-DROP FUNCTION IF EXISTS dataflow.rules_changed();
-DROP FUNCTION IF EXISTS dataflow.mappings_changed();
-
-CREATE OR REPLACE FUNCTION dataflow.source_config_changed()
-RETURNS TRIGGER AS $$
-BEGIN
-    IF NEW.config IS DISTINCT FROM OLD.config THEN
-        NEW.view_generated_at := NULL;
-    END IF;
-    RETURN NEW;
-END;
-$$ LANGUAGE plpgsql;
-
-DROP TRIGGER IF EXISTS trg_source_config_changed ON dataflow.sources;
-CREATE TRIGGER trg_source_config_changed
-BEFORE UPDATE ON dataflow.sources
-FOR EACH ROW EXECUTE FUNCTION dataflow.source_config_changed();
-
-------------------------------------------------------
--- Trigger: clear stack view_generated_at when sources change
--- On UPDATE, skip if all view-relevant columns are unchanged (upsert no-ops should not mark stale)
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION dataflow.stack_sources_changed()
-RETURNS TRIGGER AS $$
-BEGIN
-    IF TG_OP = 'UPDATE' THEN
-        IF NEW.field_map        IS NOT DISTINCT FROM OLD.field_map        AND
-           NEW.amount_sign      IS NOT DISTINCT FROM OLD.amount_sign      AND
-           NEW.balance_offset   IS NOT DISTINCT FROM OLD.balance_offset   AND
-           NEW.amount_field     IS NOT DISTINCT FROM OLD.amount_field     AND
-           NEW.date_field       IS NOT DISTINCT FROM OLD.date_field       AND
-           NEW.seq              IS NOT DISTINCT FROM OLD.seq THEN
-            RETURN NULL;
-        END IF;
-    END IF;
-    UPDATE dataflow.stacks SET view_generated_at = NULL
-    WHERE name = COALESCE(NEW.stack_name, OLD.stack_name);
-    RETURN NULL;
-END;
-$$ LANGUAGE plpgsql;
-
-DROP TRIGGER IF EXISTS trg_stack_sources_changed ON dataflow.stack_sources;
-CREATE TRIGGER trg_stack_sources_changed
-AFTER INSERT OR UPDATE OR DELETE ON dataflow.stack_sources
-FOR EACH ROW EXECUTE FUNCTION dataflow.stack_sources_changed();
-
-------------------------------------------------------
--- Function: get_status
--- Returns sources and stacks whose view is stale (null or never generated)
-------------------------------------------------------
-CREATE OR REPLACE FUNCTION get_status()
-RETURNS JSON AS $$
-DECLARE
-    v_sources JSON;
-    v_stacks  JSON;
-BEGIN
-    SELECT COALESCE(json_agg(json_build_object('name', name, 'view_generated_at', view_generated_at) ORDER BY name), '[]'::json)
-    INTO v_sources
-    FROM dataflow.sources
-    WHERE view_generated_at IS NULL;
-
-    SELECT COALESCE(json_agg(json_build_object('name', name, 'view_generated_at', view_generated_at) ORDER BY name), '[]'::json)
-    INTO v_stacks
-    FROM dataflow.stacks
-    WHERE view_generated_at IS NULL;
-
-    RETURN json_build_object('stale_sources', v_sources, 'stale_stacks', v_stacks);
-END;
-$$ LANGUAGE plpgsql STABLE;

database/transform.sql

@@ -1,156 +0,0 @@
---
--- Transform queries
--- The rule/mapping engine; SQL for the transform endpoints in api/routes/sources.js,
--- api/routes/rules.js and api/routes/records.js
---
--- Order matters within this file: the aggregate is used by apply_transformations,
--- which in turn is called by reprocess_records.
---
-
-SET search_path TO dataflow, public;
-
--- ── Merge aggregate ───────────────────────────────────────────────────────────
-
--- Merge JSONB objects across rows (later rows win on key conflicts)
--- Usage: jsonb_concat_obj(col ORDER BY sequence)
-CREATE OR REPLACE FUNCTION dataflow.jsonb_merge(a JSONB, b JSONB)
-RETURNS JSONB AS $$
-    SELECT COALESCE(a, '{}') || COALESCE(b, '{}')
-$$ LANGUAGE sql IMMUTABLE;
-
-DROP AGGREGATE IF EXISTS dataflow.jsonb_concat_obj(JSONB);
-CREATE AGGREGATE dataflow.jsonb_concat_obj(JSONB) (
-    sfunc    = dataflow.jsonb_merge,
-    stype    = JSONB,
-    initcond = '{}'
-);
-
--- ── Apply rules and mappings ──────────────────────────────────────────────────
-
--- Writes only the rule/mapping output into records.transformed. Raw values stay in
--- data and manual edits stay in overrides; readers merge the three layers.
-DROP FUNCTION IF EXISTS apply_transformations(TEXT, INTEGER[]);
-CREATE OR REPLACE FUNCTION apply_transformations(
-    p_source_name TEXT,
-    p_record_ids INTEGER[] DEFAULT NULL,  -- NULL = all eligible records
-    p_overwrite   BOOLEAN  DEFAULT FALSE  -- FALSE = skip already-transformed, TRUE = overwrite all
-) RETURNS JSON AS $$
-WITH
--- All records to process
-qualifying AS (
-    SELECT id, data
-    FROM dataflow.records
-    WHERE source_name = p_source_name
-      AND (p_overwrite OR transformed IS NULL)
-      AND (p_record_ids IS NULL OR id = ANY(p_record_ids))
-),
--- Mirror TPS rx: fan out one row per regex match, drive from rules → records
-rx AS (
-    SELECT
-        q.id,
-        r.name          AS rule_name,
-        r.sequence,
-        r.output_field,
-        r.retain,
-        r.function_type,
-        COALESCE(mt.rn, rp.rn, 1) AS result_number,
-        -- extract: build map_val and retain_val per match (mirrors TPS)
-        CASE WHEN array_length(mt.mt, 1) = 1 THEN to_jsonb(mt.mt[1]) ELSE to_jsonb(mt.mt) END AS match_val,
-        to_jsonb(rp.rp) AS replace_val
-    FROM dataflow.rules r
-    INNER JOIN qualifying q ON q.data ? r.field
-    LEFT JOIN LATERAL regexp_matches(q.data ->> r.field, r.pattern, r.flags)
-        WITH ORDINALITY AS mt(mt, rn) ON r.function_type = 'extract'
-    LEFT JOIN LATERAL regexp_replace(q.data ->> r.field, r.pattern, r.replace_value, r.flags)
-        WITH ORDINALITY AS rp(rp, rn) ON r.function_type = 'replace'
-    WHERE r.source_name = p_source_name
-      AND r.enabled = true
-),
--- Aggregate match rows back into one value per (record, rule) — mirrors TPS agg_to_target_items
-agg_matches AS (
-    SELECT
-        id,
-        rule_name,
-        sequence,
-        output_field,
-        retain,
-        function_type,
-        CASE function_type
-            WHEN 'replace' THEN jsonb_agg(replace_val) -> 0
-            ELSE
-                CASE WHEN max(result_number) = 1
-                    THEN jsonb_agg(match_val ORDER BY result_number) -> 0
-                    ELSE jsonb_agg(match_val ORDER BY result_number)
-                END
-        END AS extracted
-    FROM rx
-    GROUP BY id, rule_name, sequence, output_field, retain, function_type
-),
--- Join with mappings to find mapped output — mirrors TPS link_map
-linked AS (
-    SELECT
-        a.id,
-        a.sequence,
-        a.output_field,
-        a.retain,
-        a.extracted,
-        m.output AS mapped
-    FROM agg_matches a
-    LEFT JOIN dataflow.mappings m ON
-        m.source_name = p_source_name
-        AND m.rule_name = a.rule_name
-        AND m.input_value = a.extracted
-    WHERE a.extracted IS NOT NULL
-),
--- Build per-rule output JSONB:
---   mapped  → use mapping output; also write output_field if retain = true
---   no map  → write extracted value to output_field
-rule_output AS (
-    SELECT
-        id,
-        sequence,
-        CASE
-            WHEN mapped IS NOT NULL THEN
-                mapped ||
-                CASE WHEN retain
-                     THEN jsonb_build_object(output_field, extracted)
-                     ELSE '{}'::jsonb
-                END
-            ELSE
-                jsonb_build_object(output_field, extracted)
-        END AS output
-    FROM linked
-),
--- Merge all rule outputs per record in sequence order — mirrors TPS agg_to_id
-record_additions AS (
-    SELECT
-        id,
-        dataflow.jsonb_concat_obj(output ORDER BY sequence) AS additions
-    FROM rule_output
-    GROUP BY id
-),
--- Update all qualifying records; records with no rule matches get an empty object
-updated AS (
-    UPDATE dataflow.records rec
-    SET transformed    = COALESCE(ra.additions, '{}'::jsonb),
-        transformed_at = CURRENT_TIMESTAMP
-    FROM qualifying q
-    LEFT JOIN record_additions ra ON ra.id = q.id
-    WHERE rec.id = q.id
-    RETURNING rec.id
-)
-SELECT json_build_object('success', true, 'transformed', count(*))
-FROM updated
-$$ LANGUAGE sql;
-
-COMMENT ON FUNCTION apply_transformations IS 'Apply transformation rules and mappings to records (set-based CTE)';
-
--- ── Reprocess ─────────────────────────────────────────────────────────────────
-
-CREATE OR REPLACE FUNCTION reprocess_records(p_source_name TEXT)
-RETURNS JSON AS $$
-    -- Overwrite all records directly — no clear step, mirrors TPS srce_map_overwrite
-    SELECT dataflow.apply_transformations(p_source_name, NULL, TRUE)
-$$ LANGUAGE sql;
-
-COMMENT ON FUNCTION reprocess_records IS 'Reapply all transformations for a source, overwriting existing values';

deploy.sh

@@ -0,0 +1,409 @@
+#!/bin/bash
+#
+# Dataflow Deploy Script
+# First run: full install (database, schema, UI, nginx, systemd)
+# Subsequent runs: update functions, UI, and restart service
+#
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+cd "$SCRIPT_DIR"
+
+# ── Helpers ───────────────────────────────────────────────────────────────────
+
+BOLD='\033[1m'
+DIM='\033[2m'
+GREEN='\033[0;32m'
+YELLOW='\033[0;33m'
+RED='\033[0;31m'
+RESET='\033[0m'
+
+section() { echo ""; echo -e "${BOLD}── $1 ──${RESET}"; }
+step()    { printf "  %-42s" "$1..."; }
+ok()      { echo -e "${GREEN}✓${RESET}"; }
+fail()    { echo -e "${RED}✗ $1${RESET}"; exit 1; }
+info()    { echo -e "  ${DIM}$1${RESET}"; }
+warn()    { echo -e "  ${YELLOW}$1${RESET}"; }
+
+confirm() {
+    # confirm "Section name" — prints section header, asks to proceed
+    # Returns 0 to proceed, 1 to skip
+    section "$1"
+    read -p "  Proceed? [Y/n]: " _yn
+    [[ ! "$_yn" =~ ^[Nn]$ ]]
+}
+
+# ── Mode detection ────────────────────────────────────────────────────────────
+
+echo ""
+echo "  This script covers:"
+echo "    1. Database  — create user/database, deploy schema + functions"
+echo "    2. UI        — build from source"
+echo "    3. Nginx     — reverse proxy config + SSL via certbot"
+echo "    4. Service   — install and enable systemd unit"
+echo "    5. Restart   — start or restart the API server"
+echo ""
+echo "  Each step will be confirmed before running."
+echo "  Press Ctrl+C at any time to abort."
+
+section "Mode"
+if [ ! -f .env ]; then
+    echo "  No .env found — first-time install"
+    MODE=install
+else
+    echo "  .env found — update mode"
+    MODE=update
+    export $(cat .env | grep -v '^#' | xargs)
+fi
+
+# ── Phase 1: Collect all config ───────────────────────────────────────────────
+
+if [ "$MODE" = "install" ]; then
+
+    section "PostgreSQL Admin"
+    info "Needed to create the app user and database"
+    read -p "  Admin username [postgres]: " ADMIN_USER; ADMIN_USER=${ADMIN_USER:-postgres}
+    read -s -p "  Admin password: " ADMIN_PASS; echo ""
+
+    section "Application Database"
+    read -p "  Host         [localhost]: " DB_HOST;     DB_HOST=${DB_HOST:-localhost}
+    read -p "  Port         [5432]:      " DB_PORT;     DB_PORT=${DB_PORT:-5432}
+    read -p "  Database     [dataflow]:  " DB_NAME;     DB_NAME=${DB_NAME:-dataflow}
+    read -p "  App user     [dataflow]:  " DB_USER;     DB_USER=${DB_USER:-dataflow}
+    read -s -p "  App password:             " DB_PASSWORD; echo ""
+
+    section "API"
+    read -p "  Port         [3020]:      " API_PORT;    API_PORT=${API_PORT:-3020}
+    read -p "  Environment  [production]:" NODE_ENV;    NODE_ENV=${NODE_ENV:-production}
+
+    section "Nginx"
+    read -p "  Set up nginx reverse proxy? [Y/n]: " _yn
+    if [[ ! "$_yn" =~ ^[Nn]$ ]]; then
+        read -p "  Domain (e.g. dataflow.example.com): " NGINX_DOMAIN
+    fi
+
+    DO_DEPS=y; DO_DB=y; DO_SCHEMA=y; DO_FN=y; DO_BUILD=y; DO_SERVICE=y; DO_RESTART=y
+
+else
+
+    section "Database"
+    echo "  Current: ${DB_USER}@${DB_HOST}:${DB_PORT}/${DB_NAME}"
+    read -p "  Change target? [y/N]: " _yn
+    if [[ "$_yn" =~ ^[Yy]$ ]]; then
+        read -p "  Host     [${DB_HOST}]: " _in; DB_HOST=${_in:-$DB_HOST}
+        read -p "  Port     [${DB_PORT}]: " _in; DB_PORT=${_in:-$DB_PORT}
+        read -p "  Database [${DB_NAME}]: " _in; DB_NAME=${_in:-$DB_NAME}
+        read -p "  User     [${DB_USER}]: " _in; DB_USER=${_in:-$DB_USER}
+        read -s -p "  Password (blank = keep): " _in; echo ""
+        if [ -n "$_in" ]; then DB_PASSWORD=$_in; fi
+        CHANGE_DB=y
+    fi
+
+    section "Select steps to run"
+    read -p "  Redeploy SQL functions? [Y/n]: " DO_FN;      DO_FN=${DO_FN:-y}
+    read -p "  Rebuild UI?             [Y/n]: " DO_BUILD;    DO_BUILD=${DO_BUILD:-y}
+    read -p "  Set up / update nginx?  [y/N]: " DO_NGINX
+    read -p "  Restart API service?    [Y/n]: " DO_RESTART;  DO_RESTART=${DO_RESTART:-y}
+
+    if [[ "$DO_NGINX" =~ ^[Yy]$ ]]; then
+        read -p "  Nginx domain: " NGINX_DOMAIN
+    fi
+fi
+
+# ── Phase 2: Plan summary ─────────────────────────────────────────────────────
+
+section "Plan"
+echo "  Mode:     $( [ "$MODE" = "install" ] && echo "First-time install" || echo "Update" )"
+echo "  Database: ${DB_USER}@${DB_HOST}:${DB_PORT}/${DB_NAME}"
+echo "  API port: ${API_PORT:-3020}"
+echo ""
+echo "  Steps:"
+
+if [ "$MODE" = "install" ]; then
+    echo "    • Install Node.js dependencies"
+    echo "    • Test admin connection and create DB user/database"
+    echo "    • Deploy schema and SQL functions"
+    [ -n "$NGINX_DOMAIN" ] && echo "    • Configure nginx → $NGINX_DOMAIN" \
+                           || echo "    • Nginx — skipped (no domain provided)"
+    echo "    • Build UI"
+    echo "    • Install systemd service"
+    echo "    • Start service"
+else
+    [ "${CHANGE_DB}" = "y" ]              && echo "    • Update .env with new database target"
+    [[ ! "$DO_FN" =~ ^[Nn]$ ]]           && echo "    • Redeploy SQL functions" \
+                                          || echo "    • SQL functions — skipped"
+    [[ ! "$DO_BUILD" =~ ^[Nn]$ ]]        && echo "    • Rebuild UI" \
+                                          || echo "    • UI build — skipped"
+    [ -n "$NGINX_DOMAIN" ]               && echo "    • Configure nginx → $NGINX_DOMAIN" \
+                                          || echo "    • Nginx — skipped"
+    [[ ! "$DO_RESTART" =~ ^[Nn]$ ]]      && echo "    • Restart API service" \
+                                          || echo "    • Service restart — skipped"
+fi
+
+echo ""
+read -p "  Continue? [Y/n]: " _yn
+[[ "$_yn" =~ ^[Nn]$ ]] && echo "  Aborted." && exit 0
+
+# ── Phase 3: Execute ──────────────────────────────────────────────────────────
+
+if [ "$MODE" = "install" ]; then
+
+    # Dependencies
+    if confirm "Dependencies"; then
+        step "API (npm install)"
+        npm install --omit=dev -q && ok || fail "npm install failed"
+
+        step "UI (npm install)"
+        cd ui && npm install -q && cd .. && ok || fail "ui npm install failed"
+    else
+        info "skipped"
+    fi
+
+    # PostgreSQL
+    if confirm "PostgreSQL"; then
+        step "Testing admin connection"
+        export PGPASSWORD="$ADMIN_PASS"
+        psql -U "$ADMIN_USER" -h "$DB_HOST" -p "$DB_PORT" -d postgres -c '\q' 2>/dev/null && ok \
+            || fail "Cannot connect as $ADMIN_USER"
+
+        step "Creating user '$DB_USER'"
+        if psql -U "$ADMIN_USER" -h "$DB_HOST" -p "$DB_PORT" -d postgres -tAc \
+            "SELECT 1 FROM pg_roles WHERE rolname='$DB_USER'" 2>/dev/null | grep -q 1; then
+            echo -e "${DIM}already exists${RESET}"
+        else
+            psql -U "$ADMIN_USER" -h "$DB_HOST" -p "$DB_PORT" -d postgres \
+                -c "CREATE USER $DB_USER WITH PASSWORD '$DB_PASSWORD';" > /dev/null && ok \
+                || fail "Could not create user"
+        fi
+
+        step "Creating database '$DB_NAME'"
+        if psql -U "$ADMIN_USER" -h "$DB_HOST" -p "$DB_PORT" -lqt \
+            | cut -d'|' -f1 | grep -qw "$DB_NAME"; then
+            echo -e "${DIM}already exists${RESET}"
+        else
+            psql -U "$ADMIN_USER" -h "$DB_HOST" -p "$DB_PORT" -d postgres \
+                -c "CREATE DATABASE $DB_NAME OWNER $DB_USER;" > /dev/null && ok \
+                || fail "Could not create database"
+        fi
+        unset PGPASSWORD
+
+        step "Writing .env"
+        cat > .env << ENVEOF
+# Database Configuration
+DB_HOST=$DB_HOST
+DB_PORT=$DB_PORT
+DB_NAME=$DB_NAME
+DB_USER=$DB_USER
+DB_PASSWORD=$DB_PASSWORD
+
+# API Configuration
+API_PORT=$API_PORT
+NODE_ENV=$NODE_ENV
+ENVEOF
+        ok
+
+        export PGPASSWORD="$DB_PASSWORD"
+
+        step "Deploying schema"
+        psql -U "$DB_USER" -h "$DB_HOST" -p "$DB_PORT" -d "$DB_NAME" -f database/schema.sql -q && ok \
+            || fail "Schema deploy failed"
+
+        step "Deploying functions"
+        psql -U "$DB_USER" -h "$DB_HOST" -p "$DB_PORT" -d "$DB_NAME" -f database/functions.sql -q && ok \
+            || fail "Functions deploy failed"
+    else
+        info "skipped"
+    fi
+
+else
+
+    # Update: save .env if DB target changed
+    section ".env"
+    if [ "${CHANGE_DB}" = "y" ]; then
+        step "Writing updated .env"
+        cat > .env << ENVEOF
+# Database Configuration
+DB_HOST=$DB_HOST
+DB_PORT=$DB_PORT
+DB_NAME=$DB_NAME
+DB_USER=$DB_USER
+DB_PASSWORD=$DB_PASSWORD
+
+# API Configuration
+API_PORT=${API_PORT:-3020}
+NODE_ENV=${NODE_ENV:-production}
+ENVEOF
+        ok
+    else
+        info "no changes"
+    fi
+
+    # Test connection
+    section "Database Connection"
+    step "Testing"
+    export PGPASSWORD="$DB_PASSWORD"
+    psql -U "$DB_USER" -h "$DB_HOST" -p "$DB_PORT" -d "$DB_NAME" -c '\q' 2>/dev/null && ok \
+        || fail "Cannot connect — check credentials"
+
+    # SQL functions
+    if [[ ! "$DO_FN" =~ ^[Nn]$ ]]; then
+        if confirm "SQL Functions"; then
+            step "Deploying functions"
+            psql -U "$DB_USER" -h "$DB_HOST" -p "$DB_PORT" -d "$DB_NAME" -f database/functions.sql -q && ok \
+                || fail "Functions deploy failed"
+        else
+            info "skipped"
+        fi
+    else
+        section "SQL Functions"
+        info "skipped"
+    fi
+fi
+
+# ── UI ────────────────────────────────────────────────────────────────────────
+
+if [ "$MODE" = "install" ] || [[ ! "$DO_BUILD" =~ ^[Nn]$ ]]; then
+    if confirm "UI Build"; then
+        step "Building"
+        cd ui && npm run build > /dev/null 2>&1 && cd .. && ok \
+            || fail "UI build failed"
+    else
+        info "skipped"
+    fi
+else
+    section "UI Build"
+    info "skipped"
+fi
+
+# ── Nginx ─────────────────────────────────────────────────────────────────────
+
+if [ -n "$NGINX_DOMAIN" ]; then
+    if confirm "Nginx ($NGINX_DOMAIN)"; then
+        CONF_NAME=$(echo "$NGINX_DOMAIN" | cut -d. -f1)
+        CONF_PATH="/etc/nginx/sites-enabled/$CONF_NAME"
+        CERT_PATH="/etc/letsencrypt/live/$NGINX_DOMAIN/fullchain.pem"
+        TMP_CONF=$(mktemp)
+
+        if [ -f "$CERT_PATH" ]; then
+            cat > "$TMP_CONF" << NGINXEOF
+server {
+    listen 80;
+    listen [::]:80;
+    server_name $NGINX_DOMAIN;
+    location / { return 301 https://\$host\$request_uri; }
+}
+
+server {
+    listen 443 ssl http2;
+    listen [::]:443 ssl http2;
+    server_name $NGINX_DOMAIN;
+
+    ssl_certificate     $CERT_PATH;
+    ssl_certificate_key /etc/letsencrypt/live/$NGINX_DOMAIN/privkey.pem;
+    ssl_protocols       TLSv1.2 TLSv1.3;
+    ssl_ciphers         HIGH:!MEDIUM:!LOW:!aNULL:!NULL:!SHA;
+    ssl_prefer_server_ciphers on;
+    ssl_session_cache   shared:SSL:10m;
+
+    keepalive_timeout    70;
+    sendfile             on;
+    client_max_body_size 80m;
+
+    location / {
+        proxy_pass http://localhost:${API_PORT:-3020};
+    }
+}
+NGINXEOF
+        else
+            cat > "$TMP_CONF" << NGINXEOF
+server {
+    listen 80;
+    listen [::]:80;
+    server_name $NGINX_DOMAIN;
+    location / {
+        proxy_pass http://localhost:${API_PORT:-3020};
+    }
+}
+NGINXEOF
+        fi
+
+        step "Writing /etc/nginx/sites-enabled/$CONF_NAME"
+        sudo cp "$TMP_CONF" "$CONF_PATH" && rm "$TMP_CONF" && ok \
+            || fail "Could not write nginx config (check sudo)"
+
+        step "Testing nginx config"
+        sudo nginx -t > /dev/null 2>&1 && ok \
+            || fail "nginx config invalid — run: sudo nginx -t"
+
+        step "Reloading nginx"
+        sudo systemctl reload nginx && ok \
+            || fail "nginx reload failed"
+
+        if [ ! -f "$CERT_PATH" ]; then
+            warn "No SSL cert found for $NGINX_DOMAIN"
+            read -p "  Run certbot now? [Y/n]: " _yn
+            if [[ ! "$_yn" =~ ^[Nn]$ ]]; then
+                step "Running certbot"
+                sudo certbot --nginx -d "$NGINX_DOMAIN" --non-interactive --agree-tos \
+                    --redirect -m "admin@$NGINX_DOMAIN" > /dev/null 2>&1 && ok \
+                    || fail "certbot failed — run manually: sudo certbot --nginx -d $NGINX_DOMAIN"
+            fi
+        fi
+    else
+        info "skipped"
+    fi
+else
+    section "Nginx"
+    info "skipped"
+fi
+
+# ── Systemd service ───────────────────────────────────────────────────────────
+
+SERVICE_FILE="/etc/systemd/system/dataflow.service"
+
+section "Systemd Service"
+if [ -f "$SERVICE_FILE" ]; then
+    info "already installed"
+else
+    read -p "  Not installed. Install now? (requires sudo) [y/N]: " _yn
+    if [[ "$_yn" =~ ^[Yy]$ ]]; then
+        step "Installing service"
+        sudo cp "$SCRIPT_DIR/dataflow.service" "$SERVICE_FILE" && ok \
+            || fail "Could not install service"
+
+        step "Enabling on boot"
+        sudo systemctl daemon-reload && sudo systemctl enable dataflow > /dev/null 2>&1 && ok \
+            || fail "Could not enable service"
+    else
+        info "skipped"
+    fi
+fi
+
+# ── API Server ────────────────────────────────────────────────────────────────
+
+if [ "$MODE" = "install" ] || [[ ! "$DO_RESTART" =~ ^[Nn]$ ]]; then
+    if confirm "API Server"; then
+        if [ -f "$SERVICE_FILE" ]; then
+            step "Restarting dataflow service"
+            sudo systemctl restart dataflow && sleep 1
+            systemctl is-active --quiet dataflow && ok \
+                || fail "Service failed — check: journalctl -u dataflow -n 30"
+        else
+            info "systemd service not installed — start manually: node api/server.js"
+        fi
+    else
+        info "skipped"
+    fi
+else
+    section "API Server"
+    info "skipped"
+fi
+
+# ── Done ──────────────────────────────────────────────────────────────────────
+
+section "Done"
+echo "  API:  http://localhost:${API_PORT:-3020}"
+[ -n "$NGINX_DOMAIN" ] && echo "  Web:  https://$NGINX_DOMAIN"
+echo "  Logs: journalctl -u dataflow -f"
+echo ""

docs/perspective-pivot.md

@@ -1,79 +1,27 @@
-# Perspective
+# Perspective Pivot — Technical Reference
 
-Everything about the Perspective pivot in dataflow: which packages and versions are
+Version tested: `@perspective-dev` v4.4.0 (client, viewer, viewer-datagrid, viewer-d3fc), loaded from CDN.
-pinned and why, and a ground-truth reference for the parts of the API the official docs
-don't cover.
 
-Shared rationale across projects lives in the canonical guide at
+This document captures everything learned about controlling Perspective programmatically. The official docs are incomplete for some of these APIs — treat this as a ground-truth supplement.
-`/home/pt/pf_app/PERSPECTIVE.md` (loading, version policy, Arrow constraints, deploy
-pattern, upgrade smoke test). This file records what's specific to dataflow.
 
 ---
 
-> **Distribution:** these are the **`@perspective-dev/*`** packages (repo
+## Loading from CDN
-> github.com/perspective-dev/perspective), **not** FINOS `@finos/perspective`. Same
-> engine, separate npm scope and release schedule — don't mix the two.
-
----
-
-## Current state
-
-- **Loader:** npm `/inline` (`ui/src/pages/Pivot.jsx`) — bundled WASM, offline-capable. ✅
-  This is the target loader; pf_app should adopt it.
-- **Data:** JSON rows via `api.getViewData(source, 100000, 0)`, capped at 100k. ✅
-  Correct for dataflow's read-only, click-to-inspect model. No need to move to Arrow
-  unless view sizes grow well past 100k.
-- **Deploy:** `manage.py` + `dataflow.service` (systemd) + nginx. ✅ Reference pattern
-  for the org; pf_app should copy it.
-- **Charts:** `viewer-d3fc` is imported, so the chart plugins are available in the UI.
-  Default plugin config is datagrid-only (`{ edit_mode: 'SELECT_REGION' }`).
-- **Layout safety:** `cleanLayout()` filters saved configs against valid columns before
-  restore — the reference implementation; keep it.
-
-## The version pair is correct — do NOT "fix" it to 4.4.1
-
-`ui/package.json` pins **viewer/client/datagrid at `^4.5.1`** and **`viewer-d3fc` at
-`^4.4.1`**. This looks like a skew but is **deliberate and necessary** — it's the only
-combination that keeps both of dataflow's hard requirements:
-
-- **Inline WASM bundling.** `Pivot.jsx` imports `@perspective-dev/client/inline`,
-  `@perspective-dev/viewer/inline`, and `@perspective-dev/viewer/themes`. Those export
-  paths **exist only in 4.5.x** — they are absent from 4.4.1's `exports` map.
-- **d3fc chart plugins.** `viewer-d3fc` is published only up to **4.4.1**.
-
-Verified the hard way: pinning all four to 4.4.1 and rebuilding fails with
-`"./inline" is not exported … from @perspective-dev/client`. So the 4.5.1/4.4.1 pair
-stays. Don't touch it.
-
-**What to actually do:**
-- Keep the versions as-is; **commit `package-lock.json`** so the resolved set can't drift
-  on `npm install`. (Optionally tighten the carets to exact `4.5.1`/`4.4.1` to make that
-  explicit.)
-- Treat any Perspective bump as gated by the canonical smoke test (§7): a d3fc **chart**
-  renders, dark/light re-themes, and save→reload→drop-column layout restore.
-- Revisit only when `viewer-d3fc` ships a 4.5.x — then a fully-coherent inline-capable
-  4.5.x suite becomes possible and the pair can collapse to one version.
-
----
-
-# API Reference
-
-Packages: `@perspective-dev` client/viewer/viewer-datagrid at **v4.5.1**, viewer-d3fc at
-**v4.4.1** — installed via npm. API notes that reference v4.4.0 behaviour have not been
-re-verified at 4.5.1 but are believed to still apply. The official docs are incomplete
-for some of these APIs — treat this as a ground-truth supplement.
-
-## Loading via npm
 
 ```js
-import perspective from '@perspective-dev/client/inline'
+const [{ default: perspective }] = await Promise.all([
-import '@perspective-dev/viewer/inline'
+  import('https://cdn.jsdelivr.net/npm/@perspective-dev/client@4.4.0/dist/cdn/perspective.js'),
-import '@perspective-dev/viewer-datagrid'
+  import('https://cdn.jsdelivr.net/npm/@perspective-dev/viewer@4.4.0/dist/cdn/perspective-viewer.js'),
-import '@perspective-dev/viewer-d3fc'
+  import('https://cdn.jsdelivr.net/npm/@perspective-dev/viewer-datagrid@4.4.0/dist/cdn/perspective-viewer-datagrid.js'),
-import '@perspective-dev/viewer/themes'
+  import('https://cdn.jsdelivr.net/npm/@perspective-dev/viewer-d3fc@4.4.0/dist/cdn/perspective-viewer-d3fc.js'),
+])
 ```
 
-The `inline` builds embed WebAssembly directly into the JS bundle — no separate `.wasm` file to serve. viewer-datagrid and viewer-d3fc have no inline variant; they import normally. viewer-d3fc is currently at v4.4.1 (no v4.5.x release yet); its chart plugins register but may not appear in the viewer due to an API change in v4.5.x's `registerPlugin`.
+Stylesheet:
+```html
+<link rel="stylesheet" crossorigin="anonymous"
+  href="https://cdn.jsdelivr.net/npm/@perspective-dev/viewer/dist/css/themes.css" />
+```
 
 ---
 

docs/ui.md

@@ -1,25 +0,0 @@
-# Dataflow UI
-
-React + Vite + Tailwind CSS frontend for Dataflow.
-
-## Development
-
-```bash
-npm install
-npm run dev      # dev server on :5173, proxies /api to :3020
-```
-
-## Build
-
-```bash
-npm run build    # outputs to ../public/
-```
-
-The Express server serves `../public/` as static files — no separate web server needed in production.
-
-## Key packages
-
-- `react` / `react-router-dom` — SPA routing
-- `@perspective-dev/client`, `viewer`, `viewer-datagrid`, `viewer-d3fc` — pivot table (npm, inline WASM builds)
-- `tailwindcss` — utility CSS
-- `sql-formatter` — SQL display formatting

examples/GETTING_STARTED.md

@@ -4,40 +4,32 @@ This guide walks through a complete example using bank transaction data.
 
 ## Prerequisites
 
-PostgreSQL running, Node.js 18+, and Python 3.
+1. PostgreSQL database running
+2. Database created: `CREATE DATABASE dataflow;`
+3. `.env` file configured (copy from `.env.example`)
 
-## Step 1: Configure and Deploy
+## Step 1: Deploy Database Schema
 
 ```bash
 cd /opt/dataflow
-npm install
+psql -U postgres -d dataflow -f database/schema.sql
-python3 manage.py
+psql -U postgres -d dataflow -f database/functions.sql
 ```
 
-Choose option 1. It writes `.env`, creates the database and user if they don't exist,
+You should see tables created without errors.
-then deploys `database/schema.sql` and the SQL function files in dependency order.
 
 ## Step 2: Start the API Server
 
 ```bash
+npm install
 npm start
 ```
 
-The server starts on the port set by `API_PORT` in `.env` (3020 by default).
+The server should start on port 3000 (or your configured port).
-
-Every `/api` route requires HTTP Basic auth using the credentials set by `manage.py`
-option 9. The examples below omit it for readability — add `-u username:password` to each
-curl, or export it once:
-
-```bash
-alias dfcurl='curl -u username:password'
-```
-
-`GET /health` is the one route that needs no auth.
 
 Test it:
 ```bash
-curl http://localhost:3020/health
+curl http://localhost:3000/health
 # Should return: {"status":"ok","timestamp":"..."}
 ```
 
@@ -46,7 +38,7 @@ curl http://localhost:3020/health
 A source defines where data comes from and how to deduplicate it.
 
 ```bash
-curl -X POST http://localhost:3020/api/sources \
+curl -X POST http://localhost:3000/api/sources \
   -H "Content-Type: application/json" \
   -d '{
     "name": "bank_transactions",
@@ -63,7 +55,7 @@ Rules extract meaningful data using regex patterns.
 ### Rule 1: Extract merchant name (first part of description)
 
 ```bash
-curl -X POST http://localhost:3020/api/rules \
+curl -X POST http://localhost:3000/api/rules \
   -H "Content-Type: application/json" \
   -d '{
     "source_name": "bank_transactions",
@@ -78,7 +70,7 @@ curl -X POST http://localhost:3020/api/rules \
 ### Rule 2: Extract location (city + state pattern)
 
 ```bash
-curl -X POST http://localhost:3020/api/rules \
+curl -X POST http://localhost:3000/api/rules \
   -H "Content-Type: application/json" \
   -d '{
     "source_name": "bank_transactions",
@@ -95,7 +87,7 @@ curl -X POST http://localhost:3020/api/rules \
 Import the example CSV file:
 
 ```bash
-curl -X POST http://localhost:3020/api/sources/bank_transactions/import \
+curl -X POST http://localhost:3000/api/sources/bank_transactions/import \
   -F "file=@examples/bank_transactions.csv"
 ```
 
@@ -112,7 +104,7 @@ Response:
 ## Step 6: View Imported Records
 
 ```bash
-curl http://localhost:3020/api/records/source/bank_transactions?limit=5
+curl http://localhost:3000/api/records/source/bank_transactions?limit=5
 ```
 
 You'll see the raw imported data. Note that `transformed` is `null` - we haven't applied transformations yet!
@@ -120,7 +112,7 @@ You'll see the raw imported data. Note that `transformed` is `null` - we haven't
 ## Step 7: Apply Transformations
 
 ```bash
-curl -X POST http://localhost:3020/api/sources/bank_transactions/transform
+curl -X POST http://localhost:3000/api/sources/bank_transactions/transform
 ```
 
 Response:
@@ -133,7 +125,7 @@ Response:
 
 Now check the records again:
 ```bash
-curl http://localhost:3020/api/records/source/bank_transactions?limit=2
+curl http://localhost:3000/api/records/source/bank_transactions?limit=2
 ```
 
 You'll see the `transformed` field now contains the original data plus extracted fields like `merchant` and `location`.
@@ -141,7 +133,7 @@ You'll see the `transformed` field now contains the original data plus extracted
 ## Step 8: View Extracted Values That Need Mapping
 
 ```bash
-curl http://localhost:3020/api/mappings/source/bank_transactions/unmapped
+curl http://localhost:3000/api/mappings/source/bank_transactions/unmapped
 ```
 
 Response shows extracted merchant names that aren't mapped yet:
@@ -159,7 +151,7 @@ Response shows extracted merchant names that aren't mapped yet:
 Map extracted values to clean, standardized output:
 
 ```bash
-curl -X POST http://localhost:3020/api/mappings \
+curl -X POST http://localhost:3000/api/mappings \
   -H "Content-Type: application/json" \
   -d '{
     "source_name": "bank_transactions",
@@ -171,7 +163,7 @@ curl -X POST http://localhost:3020/api/mappings \
     }
   }'
 
-curl -X POST http://localhost:3020/api/mappings \
+curl -X POST http://localhost:3000/api/mappings \
   -H "Content-Type: application/json" \
   -d '{
     "source_name": "bank_transactions",
@@ -183,7 +175,7 @@ curl -X POST http://localhost:3020/api/mappings \
     }
   }'
 
-curl -X POST http://localhost:3020/api/mappings \
+curl -X POST http://localhost:3000/api/mappings \
   -H "Content-Type: application/json" \
   -d '{
     "source_name": "bank_transactions",
@@ -201,13 +193,13 @@ curl -X POST http://localhost:3020/api/mappings \
 Clear and reapply transformations to pick up the new mappings:
 
 ```bash
-curl -X POST http://localhost:3020/api/sources/bank_transactions/reprocess
+curl -X POST http://localhost:3000/api/sources/bank_transactions/reprocess
 ```
 
 ## Step 11: View Final Results
 
 ```bash
-curl http://localhost:3020/api/records/source/bank_transactions?limit=5
+curl http://localhost:3000/api/records/source/bank_transactions?limit=5
 ```
 
 Now the `transformed` field contains:
@@ -242,7 +234,7 @@ Example result:
 Try importing the same file again:
 
 ```bash
-curl -X POST http://localhost:3020/api/sources/bank_transactions/import \
+curl -X POST http://localhost:3000/api/sources/bank_transactions/import \
   -F "file=@examples/bank_transactions.csv"
 ```
 
@@ -280,19 +272,19 @@ You've now:
 
 ```bash
 # View all sources
-curl http://localhost:3020/api/sources
+curl http://localhost:3000/api/sources
 
 # View source statistics
-curl http://localhost:3020/api/sources/bank_transactions/stats
+curl http://localhost:3000/api/sources/bank_transactions/stats
 
 # View all rules for a source
-curl http://localhost:3020/api/rules/source/bank_transactions
+curl http://localhost:3000/api/rules/source/bank_transactions
 
 # View all mappings for a source
-curl http://localhost:3020/api/mappings/source/bank_transactions
+curl http://localhost:3000/api/mappings/source/bank_transactions
 
 # Search for specific records
-curl -X POST http://localhost:3020/api/records/search \
+curl -X POST http://localhost:3000/api/records/search \
   -H "Content-Type: application/json" \
   -d '{
     "source_name": "bank_transactions",
@@ -309,11 +301,11 @@ curl -X POST http://localhost:3020/api/records/search \
 - Check logs for error messages
 
 **Import fails:**
-- Verify source exists: `curl http://localhost:3020/api/sources`
+- Verify source exists: `curl http://localhost:3000/api/sources`
 - Check CSV format matches expectations
 - Ensure constraint_fields match CSV column names
 
 **Transformations not working:**
-- Check rules exist: `curl http://localhost:3020/api/rules/source/bank_transactions`
+- Check rules exist: `curl http://localhost:3000/api/rules/source/bank_transactions`
 - Test regex pattern manually
 - Check records have the specified field

manage.py

@@ -18,20 +18,6 @@ SERVICE_FILE = Path('/etc/systemd/system/dataflow.service')
 SERVICE_SRC  = ROOT / 'dataflow.service'
 NGINX_DIR    = Path('/etc/nginx/sites-enabled')
 
-# Deployed in order — stacks.sql creates tables that reference sources, and
-# transform.sql defines an aggregate its own functions depend on
-QUERIES_DIR  = ROOT / 'database'
-QUERY_FILES  = [
-    QUERIES_DIR / 'sources.sql',
-    QUERIES_DIR / 'rules.sql',
-    QUERIES_DIR / 'mappings.sql',
-    QUERIES_DIR / 'records.sql',
-    QUERIES_DIR / 'import.sql',
-    QUERIES_DIR / 'transform.sql',
-    QUERIES_DIR / 'stacks.sql',
-    QUERIES_DIR / 'status.sql',
-]
-
 # ── Terminal helpers ──────────────────────────────────────────────────────────
 
 BOLD   = '\033[1m'
@@ -167,31 +153,23 @@ def ui_build_time():
         return datetime.fromtimestamp(ts).strftime('%Y-%m-%d %H:%M')
     return None
 
-def nginx_conf_path(port):
+def nginx_domain(port):
-    """Path of the nginx site proxying to our port, if any."""
+    """Find nginx site proxying to our port."""
     if not NGINX_DIR.exists():
         return None
     for f in NGINX_DIR.iterdir():
         try:
-            if f':{port}' in f.read_text():
+            text = f.read_text()
-                return f
+            if f':{port}' in text:
+                for line in text.splitlines():
+                    if 'server_name' in line:
+                        parts = line.split()
+                        if len(parts) >= 2:
+                            return parts[1].rstrip(';')
         except Exception:
             pass
     return None
 
-
-def nginx_domain(port):
-    """server_name of the nginx site proxying to our port."""
-    conf = nginx_conf_path(port)
-    if not conf:
-        return None
-    for line in conf.read_text().splitlines():
-        if 'server_name' in line:
-            parts = line.split()
-            if len(parts) >= 2:
-                return parts[1].rstrip(';')
-    return None
-
 def sudo_run(args, **kwargs):
     return subprocess.run(['sudo'] + args, **kwargs)
 
@@ -354,8 +332,13 @@ def action_configure(cfg):
 
     db_location = f'database "{new_cfg["DB_NAME"]}" on {new_cfg["DB_HOST"]}:{new_cfg["DB_PORT"]}'
     schema_file  = ROOT / 'database' / 'schema.sql'
-    queries_dir  = QUERIES_DIR
+    queries_dir  = ROOT / 'database' / 'queries'
-    query_files  = QUERY_FILES
+    query_files  = [
+        queries_dir / 'sources.sql',
+        queries_dir / 'rules.sql',
+        queries_dir / 'mappings.sql',
+        queries_dir / 'records.sql',
+    ]
 
     # Offer schema deployment
     print()
@@ -444,14 +427,19 @@ def action_deploy_schema(cfg):
 
 
 def action_deploy_functions(cfg):
-    header('Deploy SQL functions (database/*.sql)')
+    header('Deploy SQL functions (database/queries/)')
     if not cfg:
         err(f'{ENV_FILE} not found — run option 1 to configure the database connection first')
         return
 
     db_location = f'database "{cfg["DB_NAME"]}" on {cfg["DB_HOST"]}:{cfg["DB_PORT"]}'
-    queries_dir = QUERIES_DIR
+    queries_dir = ROOT / 'database' / 'queries'
-    query_files = QUERY_FILES
+    query_files = [
+        queries_dir / 'sources.sql',
+        queries_dir / 'rules.sql',
+        queries_dir / 'mappings.sql',
+        queries_dir / 'records.sql',
+    ]
 
     print(f'  Source files: {queries_dir}/')
     for f in query_files:
@@ -740,116 +728,6 @@ def action_stop_service():
     ok('dataflow.service stopped')
 
 
-def action_uninstall(cfg):
-    """Reverse everything this script installs, outside the repo itself."""
-    header('Uninstall dataflow')
-
-    port      = cfg.get('API_PORT', '3020') if cfg else '3020'
-    db_name   = cfg.get('DB_NAME', 'dataflow') if cfg else 'dataflow'
-    db_user   = cfg.get('DB_USER', 'dataflow') if cfg else 'dataflow'
-    conf_path = nginx_conf_path(port)
-
-    # Everything that exists right now, in reverse install order
-    targets = []
-    if service_installed():
-        targets.append(f'systemd service   {SERVICE_FILE}' +
-                       ('  (running)' if service_running() else ''))
-    if conf_path:
-        targets.append(f'nginx site        {conf_path}')
-    if cfg and can_connect(cfg):
-        targets.append(f'database          "{db_name}" on {cfg["DB_HOST"]}:{cfg["DB_PORT"]}  (ALL DATA)')
-        targets.append(f'database user     {db_user}')
-    if ENV_FILE.exists():
-        targets.append(f'config            {ENV_FILE}')
-    if (ROOT / 'public').exists():
-        targets.append(f'built UI          {ROOT / "public"}')
-    if (ROOT / 'node_modules').exists():
-        targets.append(f'dependencies      {ROOT / "node_modules"}')
-
-    if not targets:
-        info('Nothing installed to remove.')
-        return cfg
-
-    print('  This will permanently remove:')
-    for t in targets:
-        print(f'    {t}')
-    print()
-    info(f'The repository itself ({ROOT}) is left alone — delete it manually if you want it gone.')
-    print()
-
-    if input("  Type 'delete' to confirm: ").strip() != 'delete':
-        info('Cancelled — no changes made')
-        return cfg
-
-    # ── Service ───────────────────────────────────────────────────────────────
-    if service_installed():
-        print()
-        print('  Removing systemd service...')
-        sudo_run(['systemctl', 'stop', 'dataflow'])
-        sudo_run(['systemctl', 'disable', 'dataflow'])
-        r = sudo_run(['rm', '-f', str(SERVICE_FILE)])
-        if r.returncode != 0:
-            err(f'Could not remove {SERVICE_FILE} — check sudo permissions')
-        else:
-            sudo_run(['systemctl', 'daemon-reload'])
-            ok(f'Service stopped, disabled, and {SERVICE_FILE} removed')
-
-    # ── nginx ─────────────────────────────────────────────────────────────────
-    if conf_path:
-        print()
-        print('  Removing nginx site...')
-        r = sudo_run(['rm', '-f', str(conf_path)])
-        if r.returncode != 0:
-            err(f'Could not remove {conf_path} — check sudo permissions')
-        elif sudo_run(['nginx', '-t'], capture_output=True).returncode != 0:
-            err('nginx config test failed after removal — not reloading; check nginx manually')
-        else:
-            sudo_run(['systemctl', 'reload', 'nginx'])
-            ok(f'{conf_path} removed and nginx reloaded')
-
-    # ── Database ──────────────────────────────────────────────────────────────
-    if cfg and can_connect(cfg):
-        print()
-        print(f'  Dropping the database requires PostgreSQL admin credentials.')
-        admin = {
-            'user':     prompt('PostgreSQL admin username', 'postgres'),
-            'password': prompt('PostgreSQL admin password', secret=True),
-            'host':     cfg['DB_HOST'],
-            'port':     cfg['DB_PORT'],
-        }
-        r = psql_admin(admin, 'SELECT 1')
-        if r.returncode != 0:
-            err(f'Cannot connect as admin — database and user left in place\n{r.stderr.strip()}')
-        else:
-            r = psql_admin(admin, f'DROP DATABASE IF EXISTS {db_name}')
-            if r.returncode != 0:
-                err(f'Could not drop database "{db_name}"\n{r.stderr.strip()}')
-            else:
-                ok(f'Database "{db_name}" dropped')
-                r = psql_admin(admin, f'DROP USER IF EXISTS {db_user}')
-                if r.returncode != 0:
-                    err(f'Could not drop user {db_user}\n{r.stderr.strip()}')
-                else:
-                    ok(f'User {db_user} dropped')
-
-    # ── Generated files ───────────────────────────────────────────────────────
-    print()
-    for path, label in [(ENV_FILE, 'config'),
-                        (ROOT / 'public', 'built UI'),
-                        (ROOT / 'node_modules', 'dependencies')]:
-        if not path.exists():
-            continue
-        if path.is_dir():
-            shutil.rmtree(path, ignore_errors=True)
-        else:
-            path.unlink()
-        ok(f'Removed {label} ({path})')
-
-    print()
-    ok('Uninstall complete')
-    return None
-
-
 def action_set_login_credentials(cfg):
     header('Set login credentials (LOGIN_USER / LOGIN_PASSWORD_HASH in .env)')
 
@@ -902,72 +780,18 @@ def action_set_login_credentials(cfg):
     info('Restart the service for changes to take effect (option 7).')
 
 
-def action_claim_simplefin(cfg):
-    header('Claim a SimpleFIN setup token')
-    print('  A setup token is single-use. Claiming it returns the permanent access')
-    print('  URL, which is written to .env as SIMPLEFIN_ACCESS_URL.')
-    print()
-
-    if not ENV_FILE.exists():
-        err(f'{ENV_FILE} does not exist — run the database configuration dialog first')
-        return
-
-    if cfg and cfg.get('SIMPLEFIN_ACCESS_URL'):
-        warn('SIMPLEFIN_ACCESS_URL is already set — claiming again will replace it')
-        if not confirm('Replace the existing access URL?', default_yes=False):
-            info('Cancelled — no changes made')
-            return
-
-    token = prompt('Setup token', secret=True)
-    if not token:
-        info('Cancelled — no changes made')
-        return
-
-    print('  Claiming token with SimpleFIN...')
-    r = subprocess.run(
-        ['node', '-e',
-         "require('./api/lib/simplefin.js').claimSetupToken(process.argv[1])"
-         ".then(u=>process.stdout.write(u),e=>{process.stderr.write(e.message);process.exit(1)})",
-         token],
-        capture_output=True, text=True, cwd=ROOT
-    )
-    if r.returncode != 0 or not r.stdout:
-        err(f'Claim failed — the token may already have been used\n  {r.stderr.strip()}')
-        return
-
-    access_url = r.stdout.strip()
-
-    # Update .env
-    env_text = ENV_FILE.read_text()
-    key = 'SIMPLEFIN_ACCESS_URL'
-    if f'{key}=' in env_text:
-        import re
-        env_text = re.sub(rf'^{key}=.*$', f'{key}={access_url}', env_text, flags=re.MULTILINE)
-    else:
-        env_text = env_text.rstrip('\n') + f'\n{key}={access_url}\n'
-    ENV_FILE.write_text(env_text)
-
-    # Show the host only — the URL embeds its own credentials
-    host = access_url.split('@')[-1] if '@' in access_url else access_url
-    ok(f'{key} written to {ENV_FILE}')
-    info(f'Bridge: {host}')
-    info('Restart the service for changes to take effect (option 7).')
-
-
 # ── Main menu ─────────────────────────────────────────────────────────────────
 
 MENU = [
     ('Database configuration and deployment dialog  (.env)',       action_configure),
     ('Redeploy "dataflow" schema only  (database/schema.sql)',    action_deploy_schema),
-    ('Redeploy SQL functions only      (database/*.sql)',         action_deploy_functions),
+    ('Redeploy SQL functions only      (database/queries/)',      action_deploy_functions),
     ('Build UI                  (ui/ → public/)',           action_build_ui),
     ('Set up nginx reverse proxy',                         action_setup_nginx),
     ('Install dataflow systemd service unit',               action_install_service),
     ('Start / restart dataflow.service',                    action_restart_service),
     ('Stop dataflow.service',                               action_stop_service),
     ('Set login credentials',                               action_set_login_credentials),
-    ('Claim SimpleFIN setup token  (.env)',                 action_claim_simplefin),
-    ('Uninstall  (service, nginx, database, .env, build)',  action_uninstall),
 ]
 
 def main():
@@ -980,11 +804,14 @@ def main():
         show_status(cfg)
 
         db_target = f'into "{cfg["DB_NAME"]}" on {cfg["DB_HOST"]}' if cfg else '(not configured)'
-        DB_ACTIONS = {action_deploy_schema, action_deploy_functions}
+        DB_ACTIONS = {
+            'Deploy "dataflow" schema  (database/schema.sql)',
+            'Deploy SQL functions      (database/functions.sql)',
+        }
 
         print(bold('Actions'))
-        for i, (label, fn) in enumerate(MENU, 1):
+        for i, (label, _) in enumerate(MENU, 1):
-            suffix = f'  {dim(db_target)}' if fn in DB_ACTIONS else ''
+            suffix = f'  {dim(db_target)}' if label in DB_ACTIONS else ''
             print(f'  {cyan(str(i))}. {label}{suffix}')
         print(f'  {cyan("q")}. Quit')
         print()
@@ -1000,12 +827,13 @@ def main():
             if 0 <= idx < len(MENU):
                 label, fn = MENU[idx]
                 import inspect
-                # cfg is reloaded from .env at the top of every loop, so a return
+                sig = inspect.signature(fn)
-                # value is only ever informational
+                if len(sig.parameters) == 0:
-                if len(inspect.signature(fn).parameters) == 0:
+                    result = fn()
-                    fn()
+                elif len(sig.parameters) == 1:
-                else:
+                    result = fn(cfg)
-                    fn(cfg)
+                if label.startswith('Configure') and result is not None:
+                    cfg = result
                 pause()
             else:
                 warn('Invalid choice — enter a number from the list above')

package.json

@@ -4,8 +4,8 @@
   "description": "Simple data transformation tool for ingesting, mapping, and transforming data",
   "main": "api/server.js",
   "scripts": {
-    "start": "node api/server.js",
+    "start": "nodemon api/server.js",
-    "dev": "nodemon api/server.js",
+    "dev": "node api/server.js",
     "test": "echo \"Tests coming soon\" && exit 0"
   },
   "keywords": [
@@ -18,11 +18,11 @@
   "license": "MIT",
   "dependencies": {
     "bcrypt": "^6.0.0",
-    "csv-parse": "^6.2.1",
+    "csv-parse": "^5.5.2",
-    "dotenv": "^17.4.2",
+    "dotenv": "^16.3.1",
-    "express": "^5.2.1",
+    "express": "^4.18.2",
-    "multer": "^2.1.1",
+    "multer": "^1.4.5-lts.1",
-    "pg": "^8.21.0"
+    "pg": "^8.11.3"
   },
   "devDependencies": {
     "nodemon": "^3.0.1"

scripts/setup-service.sh

@@ -0,0 +1,38 @@
+#!/bin/bash
+set -e
+
+SERVICE_NAME="dataflow"
+SERVICE_FILE="/etc/systemd/system/${SERVICE_NAME}.service"
+WORKDIR=$(pwd)
+
+echo "Creating systemd service file..."
+sudo tee "$SERVICE_FILE" > /dev/null <<EOF
+[Unit]
+Description=Dataflow API Server
+After=postgresql.service
+Wants=postgresql.service
+
+[Service]
+Type=simple
+User=www-data
+WorkingDirectory=${WORKDIR}
+Environment=NODE_ENV=production
+ExecStart=/usr/bin/node api/server.js
+Restart=always
+RestartSec=5
+
+[Install]
+WantedBy=multi-user.target
+EOF
+
+echo "Reloading systemd daemon..."
+sudo systemctl daemon-reload
+
+echo "Enabling dataflow service..."
+sudo systemctl enable "$SERVICE_NAME"
+
+echo "Starting dataflow service..."
+sudo systemctl start "$SERVICE_NAME"
+
+echo "Done! Service status:"
+sudo systemctl status "$SERVICE_NAME" --no-pager

ui/.gitignore

@@ -0,0 +1,24 @@
+# Logs
+logs
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+pnpm-debug.log*
+lerna-debug.log*
+
+node_modules
+dist
+dist-ssr
+*.local
+
+# Editor directories and files
+.vscode/*
+!.vscode/extensions.json
+.idea
+.DS_Store
+*.suo
+*.ntvs*
+*.njsproj
+*.sln
+*.sw?

ui/README.md

@@ -0,0 +1,16 @@
+# React + Vite
+
+This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
+
+Currently, two official plugins are available:
+
+- [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Oxc](https://oxc.rs)
+- [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/)
+
+## React Compiler
+
+The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation).
+
+## Expanding the ESLint configuration
+
+If you are developing a production application, we recommend using TypeScript with type-aware lint rules enabled. Check out the [TS template](https://github.com/vitejs/vite/tree/main/packages/create-vite/template-react-ts) for information on how to integrate TypeScript and [`typescript-eslint`](https://typescript-eslint.io) in your project.

ui/index.html

@@ -4,7 +4,7 @@
     <meta charset="UTF-8" />
     <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
-    <title>Dataflow</title>
+    <title>ui</title>
   </head>
   <body>
     <div id="root"></div>

ui/package.json

@@ -10,26 +10,21 @@
     "preview": "vite preview"
   },
   "dependencies": {
-    "@perspective-dev/client": "^4.5.1",
+    "react": "^19.2.4",
-    "@perspective-dev/viewer": "^4.5.1",
+    "react-dom": "^19.2.4",
-    "@perspective-dev/viewer-d3fc": "^4.4.1",
+    "react-router-dom": "^7.13.2"
-    "@perspective-dev/viewer-datagrid": "^4.5.1",
-    "react": "^19.2.7",
-    "react-dom": "^19.2.7",
-    "react-router-dom": "^7.17.0",
-    "sql-formatter": "^15.8.1"
   },
   "devDependencies": {
     "@eslint/js": "^9.39.4",
-    "@tailwindcss/vite": "^4.3.1",
+    "@tailwindcss/vite": "^4.2.2",
-    "@types/react": "^19.2.17",
+    "@types/react": "^19.2.14",
     "@types/react-dom": "^19.2.3",
-    "@vitejs/plugin-react": "^6.0.2",
+    "@vitejs/plugin-react": "^6.0.1",
     "eslint": "^9.39.4",
-    "eslint-plugin-react-hooks": "^7.1.1",
+    "eslint-plugin-react-hooks": "^7.0.1",
     "eslint-plugin-react-refresh": "^0.5.2",
-    "globals": "^17.6.0",
+    "globals": "^17.4.0",
-    "tailwindcss": "^4.3.1",
+    "tailwindcss": "^4.2.2",
-    "vite": "^8.0.16"
+    "vite": "^8.0.1"
   }
 }

ui/src/App.css

@@ -0,0 +1 @@
+/* App-level styles — layout handled by Tailwind */

ui/src/App.jsx

@@ -1,57 +1,44 @@
-import { useState, useEffect, createElement, lazy, Suspense } from 'react'
+import { useState, useEffect } from 'react'
-import { BrowserRouter, Routes, Route, Navigate, useParams } from 'react-router-dom'
+import { BrowserRouter, Routes, Route, NavLink, Navigate } from 'react-router-dom'
 import { api, setCredentials, clearCredentials } from './api'
-import Sidebar from './components/Sidebar.jsx'
-import BottomNav from './components/BottomNav.jsx'
-import SourceTabs from './components/SourceTabs.jsx'
 import Login from './pages/Login'
-import SourceList from './pages/SourceList'
+import Sources from './pages/Sources'
-import SourceDetail from './pages/SourceDetail'
-import Bridge from './pages/Bridge'
-import ImportHub from './pages/ImportHub'
 import Import from './pages/Import'
 import Rules from './pages/Rules'
 import Mappings from './pages/Mappings'
 import Records from './pages/Records'
 import Log from './pages/Log'
-const Pivot = lazy(() => import('./pages/Pivot'))
+import Pivot from './pages/Pivot'
 import Remap from './pages/Remap'
-import Stacks from './pages/Stacks'
 
-// Source-scoped pages still take a `source` prop; this reads it off the URL so
+const NAV = [
-// they didn't all need rewriting when selection moved out of the status bar.
+  { to: '/sources', label: 'Sources' },
-function ScopedToSource({ component, ...props }) {
+  { to: '/import', label: 'Import' },
-  const { name } = useParams()
+  { to: '/rules', label: 'Rules' },
-  return createElement(component, { source: name, ...props })
+  { to: '/mappings', label: 'Mappings' },
-}
+  { to: '/remap', label: 'Remap' },
-
+  { to: '/records', label: 'Records' },
-// Pivot doubles as the stack viewer; a stack in the URL takes precedence there
+  { to: '/pivot', label: 'Pivot' },
-function StackPivot() {
+  { to: '/log', label: 'Log' },
-  const { name } = useParams()
+]
-  return <Pivot source={name} selectedStack={name} setSelectedStack={() => {}} />
-}
 
 export default function App() {
   const [authed, setAuthed] = useState(false)
   const [loginUser, setLoginUser] = useState('')
   const [sources, setSources] = useState([])
   const [source, setSource] = useState(() => localStorage.getItem('selectedSource') || '')
-  const [sidebarExpanded, setSidebarExpanded] = useState(() => localStorage.getItem('df_sidebar') !== 'collapsed')
+  const [sidebarOpen, setSidebarOpen] = useState(false)
-  // Sets of names whose dfv view is out of sync with current definitions
-  const [staleSources, setStaleSources] = useState(new Set())
-  const [staleStacks, setStaleStacks] = useState(new Set())
-  const [reprocessSources, setReprocessSources] = useState(new Set())
-  const [generating, setGenerating] = useState({}) // { 'source:name': true }
 
   async function handleLogin(user, pass) {
     setCredentials(user, pass)
-    const s = await api.getSources()
+    await api.getSources().then(s => {
-    sessionStorage.setItem('df_user', user)
+      sessionStorage.setItem('df_user', user)
-    sessionStorage.setItem('df_pass', pass)
+      sessionStorage.setItem('df_pass', pass)
-    setSources(s)
+      setSources(s)
-    if (!source && s.length > 0) setSource(s[0].name)
+      if (!source && s.length > 0) setSource(s[0].name)
-    setAuthed(true)
+      setAuthed(true)
-    setLoginUser(user)
+      setLoginUser(user)
+    })
   }
 
   function handleLogout() {
@@ -61,57 +48,6 @@ export default function App() {
     setAuthed(false)
     setLoginUser('')
     setSources([])
-    setStaleSources(new Set())
-    setStaleStacks(new Set())
-    setReprocessSources(new Set())
-  }
-
-  // Load initial stale state from DB once on login
-  useEffect(() => {
-    if (!authed) return
-    api.getStatus().then(s => {
-      setStaleSources(new Set((s.stale_sources || []).map(x => x.name)))
-      setStaleStacks(new Set((s.stale_stacks || []).map(x => x.name)))
-    }).catch(() => {})
-  }, [authed])
-
-  function markSourceStale(name) {
-    setStaleSources(prev => new Set([...prev, name]))
-  }
-  function markNeedsReprocess(name) {
-    setReprocessSources(prev => new Set([...prev, name]))
-  }
-  async function handleReprocessSource(name) {
-    setGenerating(g => ({ ...g, [`rp:${name}`]: true }))
-    try {
-      await api.reprocess(name)
-      setReprocessSources(prev => { const n = new Set(prev); n.delete(name); return n })
-    } catch (e) { alert(e.message) }
-    finally { setGenerating(g => { const n = { ...g }; delete n[`rp:${name}`]; return n }) }
-  }
-  function markStackStale(name) {
-    setStaleStacks(prev => new Set([...prev, name]))
-  }
-  function clearStackStale(name) {
-    setStaleStacks(prev => { const n = new Set(prev); n.delete(name); return n })
-  }
-
-  async function handleGenerateSource(name) {
-    setGenerating(g => ({ ...g, [`src:${name}`]: true }))
-    try {
-      await api.generateView(name)
-      setStaleSources(prev => { const n = new Set(prev); n.delete(name); return n })
-    } catch (e) { alert(e.message) }
-    finally { setGenerating(g => { const n = { ...g }; delete n[`src:${name}`]; return n }) }
-  }
-
-  async function handleGenerateStack(name) {
-    setGenerating(g => ({ ...g, [`stk:${name}`]: true }))
-    try {
-      await api.generateStackView(name)
-      setStaleStacks(prev => { const n = new Set(prev); n.delete(name); return n })
-    } catch (e) { alert(e.message) }
-    finally { setGenerating(g => { const n = { ...g }; delete n[`stk:${name}`]; return n }) }
   }
 
   // On mount, restore session if credentials are saved
@@ -125,104 +61,99 @@ export default function App() {
     if (source) localStorage.setItem('selectedSource', source)
   }, [source])
 
-  useEffect(() => {
-    localStorage.setItem('df_sidebar', sidebarExpanded ? 'expanded' : 'collapsed')
-  }, [sidebarExpanded])
-
   if (!authed) return <Login onLogin={handleLogin} />
 
+  const sidebar = (
+    <div className="flex flex-col h-full">
+      {/* Header */}
+      <div className="px-4 py-3 border-b border-gray-200">
+        <div className="flex items-center justify-between">
+          <span className="text-sm font-semibold text-gray-800 tracking-wide uppercase">Dataflow</span>
+          <button onClick={() => setSidebarOpen(false)} className="md:hidden text-gray-400 hover:text-gray-600 leading-none" title="Close">✕</button>
+        </div>
+        <div className="flex items-center justify-between mt-1">
+          <span className="text-xs text-gray-400">{loginUser}</span>
+          <button onClick={handleLogout} className="text-xs text-gray-400 hover:text-red-500" title="Sign out">Sign out</button>
+        </div>
+      </div>
+
+      {/* Source selector */}
+      <div className="px-3 py-3 border-b border-gray-200">
+        <div className="flex items-center justify-between mb-1">
+          <label className="text-xs text-gray-500">Source</label>
+          <NavLink to="/sources?new=1" className="text-xs text-blue-400 hover:text-blue-600 leading-none" title="New source" onClick={() => setSidebarOpen(false)}>+</NavLink>
+        </div>
+        <select
+          className="w-full text-sm border border-gray-200 rounded px-2 py-1 bg-white focus:outline-none focus:border-blue-400"
+          value={source}
+          onChange={e => setSource(e.target.value)}
+        >
+          {sources.length === 0 && <option value="">—</option>}
+          {sources.map(s => <option key={s.name} value={s.name}>{s.name}</option>)}
+        </select>
+      </div>
+
+      {/* Nav */}
+      <nav className="flex-1 py-2">
+        {NAV.map(({ to, label }) => (
+          <NavLink
+            key={to}
+            to={to}
+            onClick={() => setSidebarOpen(false)}
+            className={({ isActive }) =>
+              `block px-4 py-2 text-sm ${isActive
+                ? 'bg-blue-50 text-blue-700 font-medium'
+                : 'text-gray-600 hover:bg-gray-50'}`
+            }
+          >
+            {label}
+          </NavLink>
+        ))}
+      </nav>
+    </div>
+  )
+
   return (
     <BrowserRouter>
-      <div className="flex h-screen">
+      <div className="flex h-screen bg-gray-50">
 
-        <div className="hidden md:flex">
+        {/* Mobile overlay */}
-        <Sidebar
+        {sidebarOpen && (
-          expanded={sidebarExpanded}
+          <div className="fixed inset-0 z-20 bg-black/30 md:hidden" onClick={() => setSidebarOpen(false)} />
-          setExpanded={setSidebarExpanded}
+        )}
-          loginUser={loginUser}
+
-          onLogout={handleLogout}
+        {/* Sidebar — fixed on mobile, static on desktop */}
-          sources={sources}
+        <div className={`
-        />
+          fixed inset-y-0 left-0 z-30 w-44 bg-white border-r border-gray-200 transform transition-transform duration-200
+          md:static md:translate-x-0 md:z-auto md:transition-none
+          ${sidebarOpen ? 'translate-x-0' : '-translate-x-full'}
+        `}>
+          {sidebar}
         </div>
 
         {/* Main */}
-        <div className="flex-1 overflow-hidden flex flex-col min-w-0">
+        <div className="flex-1 overflow-auto flex flex-col min-w-0">
-          {(staleSources.size > 0 || staleStacks.size > 0) && (
+          {/* Mobile top bar */}
-            <div className="bg-warn-soft border-b border-warn-line px-4 py-1.5 text-xs text-warn flex flex-wrap items-center gap-x-3 gap-y-1">
+          <div className="md:hidden flex items-center px-3 py-2 bg-white border-b border-gray-200">
-              <span className="font-medium">View out of sync:</span>
+            <button onClick={() => setSidebarOpen(true)} className="text-gray-500 hover:text-gray-700 mr-3 text-lg leading-none">☰</button>
-              {[...staleSources].map(name => (
+            <span className="text-sm font-semibold text-gray-800 tracking-wide uppercase">Dataflow</span>
-                <span key={name} className="flex items-center gap-1">
+          </div>
-                  {name}
-                  <button
-                    onClick={() => handleGenerateSource(name)}
-                    disabled={generating[`src:${name}`]}
-                    className="px-1.5 py-0.5 rounded bg-warn-line hover:bg-warn-line disabled:opacity-50 font-medium"
-                  >
-                    {generating[`src:${name}`] ? '…' : 'Generate'}
-                  </button>
-                </span>
-              ))}
-              {staleSources.size > 0 && staleStacks.size > 0 && <span className="text-warn">|</span>}
-              {[...staleStacks].map(name => (
-                <span key={name} className="flex items-center gap-1">
-                  stack: {name}
-                  <button
-                    onClick={() => handleGenerateStack(name)}
-                    disabled={generating[`stk:${name}`]}
-                    className="px-1.5 py-0.5 rounded bg-warn-line hover:bg-warn-line disabled:opacity-50 font-medium"
-                  >
-                    {generating[`stk:${name}`] ? '…' : 'Generate'}
-                  </button>
-                </span>
-              ))}
-            </div>
-          )}
-          {reprocessSources.size > 0 && (
-            <div className="bg-accent-soft border-b border-accent-line px-4 py-1.5 text-xs text-accent flex flex-wrap items-center gap-x-3 gap-y-1">
-              <span className="font-medium">Mappings updated:</span>
-              {[...reprocessSources].map(name => (
-                <span key={name} className="flex items-center gap-1">
-                  {name}
-                  <button
-                    onClick={() => handleReprocessSource(name)}
-                    disabled={generating[`rp:${name}`]}
-                    className="px-1.5 py-0.5 rounded bg-blue-200 hover:bg-blue-300 disabled:opacity-50 font-medium"
-                  >
-                    {generating[`rp:${name}`] ? '…' : 'Reprocess'}
-                  </button>
-                </span>
-              ))}
-            </div>
-          )}
 
-          <div className="flex-1 overflow-auto pb-14 md:pb-0">
+          <div className="flex-1 overflow-auto">
-            <Suspense fallback={<div className="p-6 text-sm text-muted">Loading…</div>}>
             <Routes>
               <Route path="/" element={<Navigate to="/sources" replace />} />
-
+              <Route path="/sources" element={<Sources source={source} sources={sources} setSources={setSources} setSource={setSource} />} />
-              <Route path="/sources" element={<SourceList sources={sources} setSources={setSources} setSource={setSource} />} />
+              <Route path="/import" element={<Import source={source} />} />
-              <Route path="/sources/:name" element={<SourceTabs sources={sources} />}>
+              <Route path="/rules" element={<Rules source={source} />} />
-                <Route index element={<Navigate to="records" replace />} />
+              <Route path="/mappings" element={<Mappings source={source} />} />
-                <Route path="setup" element={<SourceDetail sources={sources} setSources={setSources} />} />
-                <Route path="import" element={<ScopedToSource component={Import} />} />
-                <Route path="rules" element={<ScopedToSource component={Rules} onStale={markSourceStale} />} />
-                <Route path="mappings" element={<ScopedToSource component={Mappings} onNeedsReprocess={markNeedsReprocess} />} />
-                <Route path="records" element={<ScopedToSource component={Records} />} />
-                <Route path="pivot" element={<ScopedToSource component={Pivot} />} />
-              </Route>
-
-              <Route path="/import" element={<ImportHub sources={sources} />} />
-              <Route path="/bridge" element={<Bridge sources={sources} />} />
-              <Route path="/stacks" element={<Stacks sources={sources} onStackStale={markStackStale} onStackViewGenerated={clearStackStale} />} />
-              <Route path="/stacks/:name/pivot" element={<StackPivot />} />
               <Route path="/remap" element={<Remap />} />
+              <Route path="/records" element={<Records source={source} />} />
+              <Route path="/pivot" element={<Pivot source={source} />} />
               <Route path="/log" element={<Log />} />
             </Routes>
-            </Suspense>
           </div>
         </div>
       </div>
-      <BottomNav />
     </BrowserRouter>
   )
 }

ui/src/api.js

@@ -66,18 +66,6 @@ export const api = {
     fd.append('file', file)
     return request('POST', `/sources/${name}/import`, fd, true)
   },
-  syncSimpleFin: (name, opts = {}) => {
-    const params = new URLSearchParams(opts)
-    return request('POST', `/sources/${name}/sync${params.toString() ? `?${params}` : ''}`)
-  },
-  getSimpleFinSample: (accountId, days) => {
-    const params = new URLSearchParams({ account_id: accountId })
-    if (days !== undefined) params.set('days', days)
-    return request('GET', `/sources/simplefin-sample?${params}`)
-  },
-  getSimpleFinAccounts: (accessUrlEnv) =>
-    request('GET', `/sources/simplefin-accounts${accessUrlEnv ? `?access_url_env=${encodeURIComponent(accessUrlEnv)}` : ''}`),
-  claimSimpleFinToken: (setup_token) => request('POST', '/sources/simplefin-claim', { setup_token }),
   transform: (name) => request('POST', `/sources/${name}/transform`),
   reprocess: (name) => request('POST', `/sources/${name}/reprocess`),
   generateView: (name) => request('POST', `/sources/${name}/view`),
@@ -120,40 +108,15 @@ export const api = {
   getMappingsByOutputField: (col, val) => request('GET', `/mappings/outputs/${encodeURIComponent(col)}/${encodeURIComponent(val)}`),
   remapOutputField: (col, from_val, to_val) => request('POST', '/mappings/remap-field', { col, from_val, to_val }),
 
-  // Pivot layouts (sources)
+  // Pivot layouts
   getPivotLayouts: (source) => request('GET', `/sources/${source}/layouts`),
   savePivotLayout: (source, layout_name, config) => request('POST', `/sources/${source}/layouts`, { layout_name, config }),
   deletePivotLayout: (source, id) => request('DELETE', `/sources/${source}/layouts/${id}`),
 
-  // Pivot layouts (stacks)
-  getStackPivotLayouts: (name) => request('GET', `/stacks/${name}/layouts`),
-  saveStackPivotLayout: (name, layout_name, config) => request('POST', `/stacks/${name}/layouts`, { layout_name, config }),
-  deleteStackPivotLayout: (name, id) => request('DELETE', `/stacks/${name}/layouts/${id}`),
-
-  // Stacks
-  getStacks: () => request('GET', '/stacks'),
-  getStack: (name) => request('GET', `/stacks/${name}`),
-  createStack: (body) => request('POST', '/stacks', body),
-  updateStack: (name, body) => request('PUT', `/stacks/${name}`, body),
-  deleteStack: (name) => request('DELETE', `/stacks/${name}`),
-  upsertStackSource: (name, source, body) => request('PUT', `/stacks/${name}/sources/${source}`, body),
-  reorderStackSources: (name, source_names) => request('PUT', `/stacks/${name}/sources/reorder`, { source_names }),
-  removeStackSource: (name, source) => request('DELETE', `/stacks/${name}/sources/${source}`),
-  previewStackSql: (name) => request('GET', `/stacks/${name}/view-sql`),
-  generateStackView: (name) => request('POST', `/stacks/${name}/view`),
-  execStackSql: (name, sql) => request('POST', `/stacks/${name}/exec-sql`, { sql }),
-  getStackBalance: (name) => request('GET', `/stacks/${name}/balance`),
-  calibrateBalance: (name, source, body) => request('POST', `/stacks/${name}/calibrate`, { ...body, source_name: source || null }),
-
-  // Status
-  getStatus: () => request('GET', '/status'),
-
   // Records
   getRecords: (source, limit = 100, offset = 0) =>
     request('GET', `/records/source/${source}?limit=${limit}&offset=${offset}`),
   getRecord: (id) => request('GET', `/records/${id}`),
-  getOverrideKeys: (source) => request('GET', `/sources/${source}/override-keys`),
-  setBulkRecordOverrides: (source, recordIds, overrides) => request('POST', `/records/bulk-overrides`, { source_name: source, record_ids: recordIds, overrides }),
   setRecordOverrides: (id, overrides) => request('PUT', `/records/${id}/overrides`, { overrides }),
   clearRecordOverrides: (id) => request('DELETE', `/records/${id}/overrides`),
 }

ui/src/assets/react.svg

@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" aria-hidden="true" role="img" class="iconify iconify--logos" width="35.93" height="32" preserveAspectRatio="xMidYMid meet" viewBox="0 0 256 228"><path fill="#00D8FF" d="M210.483 73.824a171.49 171.49 0 0 0-8.24-2.597c.465-1.9.893-3.777 1.273-5.621c6.238-30.281 2.16-54.676-11.769-62.708c-13.355-7.7-35.196.329-57.254 19.526a171.23 171.23 0 0 0-6.375 5.848a155.866 155.866 0 0 0-4.241-3.917C100.759 3.829 77.587-4.822 63.673 3.233C50.33 10.957 46.379 33.89 51.995 62.588a170.974 170.974 0 0 0 1.892 8.48c-3.28.932-6.445 1.924-9.474 2.98C17.309 83.498 0 98.307 0 113.668c0 15.865 18.582 31.778 46.812 41.427a145.52 145.52 0 0 0 6.921 2.165a167.467 167.467 0 0 0-2.01 9.138c-5.354 28.2-1.173 50.591 12.134 58.266c13.744 7.926 36.812-.22 59.273-19.855a145.567 145.567 0 0 0 5.342-4.923a168.064 168.064 0 0 0 6.92 6.314c21.758 18.722 43.246 26.282 56.54 18.586c13.731-7.949 18.194-32.003 12.4-61.268a145.016 145.016 0 0 0-1.535-6.842c1.62-.48 3.21-.974 4.76-1.488c29.348-9.723 48.443-25.443 48.443-41.52c0-15.417-17.868-30.326-45.517-39.844Zm-6.365 70.984c-1.4.463-2.836.91-4.3 1.345c-3.24-10.257-7.612-21.163-12.963-32.432c5.106-11 9.31-21.767 12.459-31.957c2.619.758 5.16 1.557 7.61 2.4c23.69 8.156 38.14 20.213 38.14 29.504c0 9.896-15.606 22.743-40.946 31.14Zm-10.514 20.834c2.562 12.94 2.927 24.64 1.23 33.787c-1.524 8.219-4.59 13.698-8.382 15.893c-8.067 4.67-25.32-1.4-43.927-17.412a156.726 156.726 0 0 1-6.437-5.87c7.214-7.889 14.423-17.06 21.459-27.246c12.376-1.098 24.068-2.894 34.671-5.345a134.17 134.17 0 0 1 1.386 6.193ZM87.276 214.515c-7.882 2.783-14.16 2.863-17.955.675c-8.075-4.657-11.432-22.636-6.853-46.752a156.923 156.923 0 0 1 1.869-8.499c10.486 2.32 22.093 3.988 34.498 4.994c7.084 9.967 14.501 19.128 21.976 27.15a134.668 134.668 0 0 1-4.877 4.492c-9.933 8.682-19.886 14.842-28.658 17.94ZM50.35 144.747c-12.483-4.267-22.792-9.812-29.858-15.863c-6.35-5.437-9.555-10.836-9.555-15.216c0-9.322 13.897-21.212 37.076-29.293c2.813-.98 5.757-1.905 8.812-2.773c3.204 10.42 7.406 21.315 12.477 32.332c-5.137 11.18-9.399 22.249-12.634 32.792a134.718 134.718 0 0 1-6.318-1.979Zm12.378-84.26c-4.811-24.587-1.616-43.134 6.425-47.789c8.564-4.958 27.502 2.111 47.463 19.835a144.318 144.318 0 0 1 3.841 3.545c-7.438 7.987-14.787 17.08-21.808 26.988c-12.04 1.116-23.565 2.908-34.161 5.309a160.342 160.342 0 0 1-1.76-7.887Zm110.427 27.268a347.8 347.8 0 0 0-7.785-12.803c8.168 1.033 15.994 2.404 23.343 4.08c-2.206 7.072-4.956 14.465-8.193 22.045a381.151 381.151 0 0 0-7.365-13.322Zm-45.032-43.861c5.044 5.465 10.096 11.566 15.065 18.186a322.04 322.04 0 0 0-30.257-.006c4.974-6.559 10.069-12.652 15.192-18.18ZM82.802 87.83a323.167 323.167 0 0 0-7.227 13.238c-3.184-7.553-5.909-14.98-8.134-22.152c7.304-1.634 15.093-2.97 23.209-3.984a321.524 321.524 0 0 0-7.848 12.897Zm8.081 65.352c-8.385-.936-16.291-2.203-23.593-3.793c2.26-7.3 5.045-14.885 8.298-22.6a321.187 321.187 0 0 0 7.257 13.246c2.594 4.48 5.28 8.868 8.038 13.147Zm37.542 31.03c-5.184-5.592-10.354-11.779-15.403-18.433c4.902.192 9.899.29 14.978.29c5.218 0 10.376-.117 15.453-.343c-4.985 6.774-10.018 12.97-15.028 18.486Zm52.198-57.817c3.422 7.8 6.306 15.345 8.596 22.52c-7.422 1.694-15.436 3.058-23.88 4.071a382.417 382.417 0 0 0 7.859-13.026a347.403 347.403 0 0 0 7.425-13.565Zm-16.898 8.101a358.557 358.557 0 0 1-12.281 19.815a329.4 329.4 0 0 1-23.444.823c-7.967 0-15.716-.248-23.178-.732a310.202 310.202 0 0 1-12.513-19.846h.001a307.41 307.41 0 0 1-10.923-20.627a310.278 310.278 0 0 1 10.89-20.637l-.001.001a307.318 307.318 0 0 1 12.413-19.761c7.613-.576 15.42-.876 23.31-.876H128c7.926 0 15.743.303 23.354.883a329.357 329.357 0 0 1 12.335 19.695a358.489 358.489 0 0 1 11.036 20.54a329.472 329.472 0 0 1-11 20.722Zm22.56-122.124c8.572 4.944 11.906 24.881 6.52 51.026c-.344 1.668-.73 3.367-1.15 5.09c-10.622-2.452-22.155-4.275-34.23-5.408c-7.034-10.017-14.323-19.124-21.64-27.008a160.789 160.789 0 0 1 5.888-5.4c18.9-16.447 36.564-22.941 44.612-18.3ZM128 90.808c12.625 0 22.86 10.235 22.86 22.86s-10.235 22.86-22.86 22.86s-22.86-10.235-22.86-22.86s10.235-22.86 22.86-22.86Z"></path></svg>

ui/src/components/BottomNav.jsx

@@ -1,40 +0,0 @@
-import { NavLink } from 'react-router-dom'
-import useTheme from '../theme.jsx'
-import { NAV } from './navItems.jsx'
-
-// Phone-sized replacement for the sidebar: same destinations, thumb-reachable.
-// Hidden from md: upward, where the sidebar takes over.
-export default function BottomNav() {
-  const { dark, setDark } = useTheme()
-
-  return (
-    <nav className="md:hidden fixed bottom-0 inset-x-0 z-20 bg-surface border-t border-line flex justify-around items-stretch h-14 pb-[env(safe-area-inset-bottom)]">
-      {NAV.map(({ to, label, icon }) => (
-        <NavLink
-          key={to}
-          to={to}
-          className={({ isActive }) =>
-            `flex-1 flex flex-col items-center justify-center gap-0.5 text-[10px] ${
-              isActive ? 'text-accent' : 'text-muted'
-            }`
-          }
-        >
-          <span>{icon}</span>
-          {label}
-        </NavLink>
-      ))}
-      <button
-        onClick={() => setDark(d => !d)}
-        className="flex-1 flex flex-col items-center justify-center gap-0.5 text-[10px] text-muted"
-        title={dark ? 'Light mode' : 'Dark mode'}
-      >
-        <svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
-          {dark
-            ? <><circle cx="12" cy="12" r="4"/><line x1="12" y1="2" x2="12" y2="5"/><line x1="12" y1="19" x2="12" y2="22"/><line x1="2" y1="12" x2="5" y2="12"/><line x1="19" y1="12" x2="22" y2="12"/></>
-            : <path d="M21 12.79A9 9 0 1 1 11.21 3 7 7 0 0 0 21 12.79z"/>}
-        </svg>
-        Theme
-      </button>
-    </nav>
-  )
-}

ui/src/components/SampleTable.jsx

@@ -1,28 +0,0 @@
-// Compact preview of raw record values — used by the source detail page and by
-// the create dialog when a CSV or bank feed is sampled
-export default function SampleTable({ rows }) {
-  if (!rows || rows.length === 0) return null
-  const cols = Object.keys(rows[0])
-  return (
-    <div className="overflow-auto border border-line-soft rounded bg-raised max-h-36">
-      <table className="text-xs w-full">
-        <thead>
-          <tr className="text-left text-muted border-b border-line-soft bg-raised sticky top-0">
-            {cols.map(c => <th key={c} className="px-2 py-1 font-medium whitespace-nowrap">{c}</th>)}
-          </tr>
-        </thead>
-        <tbody>
-          {rows.map((row, i) => (
-            <tr key={i} className="border-t border-line-soft">
-              {cols.map(c => (
-                <td key={c} className="px-2 py-1 whitespace-nowrap text-ink-soft max-w-32 truncate font-mono">
-                  {row[c] == null ? <span className="text-muted">—</span> : String(row[c])}
-                </td>
-              ))}
-            </tr>
-          ))}
-        </tbody>
-      </table>
-    </div>
-  )
-}

ui/src/components/Section.jsx

@@ -1,13 +0,0 @@
-// One titled panel per job, so setup, schema, and destructive actions read as
-// separate things rather than one flat form
-export default function Section({ title, description, children }) {
-  return (
-    <section className="bg-surface border border-line rounded p-4">
-      <h2 className="text-sm font-semibold text-ink-soft">{title}</h2>
-      {description
-        ? <p className="text-xs text-muted mt-0.5 mb-3">{description}</p>
-        : <div className="mb-3" />}
-      {children}
-    </section>
-  )
-}

ui/src/components/Sidebar.jsx

@@ -1,165 +0,0 @@
-import { Fragment, useMemo } from 'react'
-import { NavLink } from 'react-router-dom'
-import useTheme from '../theme.jsx'
-import { NAV } from './navItems.jsx'
-
-// Same distinction the Import page makes: a source is either on a bank feed or
-// it gets CSVs uploaded to it.
-const feedIcon = (
-  <svg width="12" height="12" viewBox="0 0 16 16" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round">
-    <path d="M4 10.5a4.5 4.5 0 0 1 4.5 4.5"/>
-    <path d="M4 6a9 9 0 0 1 9 9"/>
-    <circle cx="4.2" cy="14.8" r="1.2" fill="currentColor" stroke="none"/>
-  </svg>
-)
-
-const csvIcon = (
-  <svg width="12" height="12" viewBox="0 0 16 16" fill="none" stroke="currentColor" strokeWidth="1.4" strokeLinecap="round" strokeLinejoin="round">
-    <path d="M4 1.5h5l3 3v10H4z"/>
-    <polyline points="9,1.5 9,4.5 12,4.5"/>
-  </svg>
-)
-
-export default function Sidebar({ expanded, setExpanded, loginUser, onLogout, sources = [] }) {
-  const { dark, setDark } = useTheme()
-
-  // Bank feeds first, then CSV sources, alphabetical within each group
-  const navSources = useMemo(() => (
-    sources
-      .map(s => ({ ...s, isFeed: !!s.config?.simplefin?.account_id }))
-      .sort((a, b) =>
-        (b.isFeed - a.isFeed) || a.name.localeCompare(b.name, undefined, { sensitivity: 'base' })
-      )
-  ), [sources])
-
-  return (
-    <div
-      className="bg-surface border-r border-line flex flex-col shrink-0 overflow-hidden transition-all duration-150"
-      style={{ width: expanded ? 200 : 48 }}
-    >
-      {/* Header */}
-      <div className="h-12 flex items-center px-3 border-b border-line-soft gap-2 shrink-0">
-        <button
-          onClick={() => setExpanded(e => !e)}
-          className="w-8 h-8 flex items-center justify-center rounded hover:bg-raised text-muted shrink-0"
-          title="Toggle sidebar"
-        >
-          <svg width="16" height="16" viewBox="0 0 16 16" fill="none">
-            <rect x="1" y="3" width="14" height="1.5" rx="0.75" fill="currentColor"/>
-            <rect x="1" y="7.25" width="14" height="1.5" rx="0.75" fill="currentColor"/>
-            <rect x="1" y="11.5" width="14" height="1.5" rx="0.75" fill="currentColor"/>
-          </svg>
-        </button>
-        <span
-          className="text-xs font-semibold text-ink-soft tracking-wide uppercase whitespace-nowrap transition-opacity duration-100"
-          style={{ opacity: expanded ? 1 : 0, pointerEvents: expanded ? 'auto' : 'none' }}
-        >
-          Dataflow
-        </span>
-      </div>
-
-      {/* Nav */}
-      <nav className="flex flex-col gap-0.5 p-2 flex-1 overflow-y-auto">
-        {NAV.map(({ to, label, icon }) => (
-          <Fragment key={to}>
-            <NavLink
-              to={to}
-              end={to === '/sources'}
-              title={!expanded ? label : undefined}
-              className={({ isActive }) =>
-                `flex items-center gap-3 px-2 py-2 rounded w-full transition-colors ${
-                  isActive
-                    ? 'bg-accent-soft text-accent'
-                    : 'text-muted hover:bg-raised hover:text-ink'
-                }`
-              }
-            >
-              <span className="shrink-0">{icon}</span>
-              <span
-                className="text-sm whitespace-nowrap transition-opacity duration-100"
-                style={{ opacity: expanded ? 1 : 0, pointerEvents: expanded ? 'auto' : 'none', width: expanded ? 'auto' : 0, overflow: 'hidden' }}
-              >
-                {label}
-              </span>
-            </NavLink>
-
-            {/* Jump straight to a source. Collapsed there is no room for names,
-                so the shortcut list only exists when the sidebar is open. */}
-            {to === '/sources' && expanded && navSources.map(({ isFeed, ...s }) => {
-              return (
-                <NavLink
-                  key={s.name}
-                  to={`/sources/${encodeURIComponent(s.name)}`}
-                  title={`${s.name} — ${isFeed ? 'bank feed' : 'CSV'}`}
-                  className={({ isActive }) =>
-                    `flex items-center gap-2 ml-4 pl-2 pr-2 py-1 rounded border-l border-line-soft transition-colors ${
-                      isActive
-                        ? 'bg-accent-soft text-accent'
-                        : 'text-muted hover:bg-raised hover:text-ink'
-                    }`
-                  }
-                >
-                  <span className="shrink-0 opacity-70">{isFeed ? feedIcon : csvIcon}</span>
-                  <span className="text-xs truncate">{s.name}</span>
-                </NavLink>
-              )
-            })}
-          </Fragment>
-        ))}
-      </nav>
-
-      {/* Theme */}
-      <div className="border-t border-line-soft px-3 py-2 shrink-0">
-        <button
-          onClick={() => setDark(d => !d)}
-          title={dark ? 'Switch to light mode' : 'Switch to dark mode'}
-          className="flex items-center gap-2.5 w-full rounded px-1 py-1 text-muted hover:bg-raised hover:text-ink"
-        >
-          <span className="shrink-0">
-            {dark ? (
-              <svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
-                <circle cx="12" cy="12" r="4"/>
-                <line x1="12" y1="2" x2="12" y2="5"/><line x1="12" y1="19" x2="12" y2="22"/>
-                <line x1="4.93" y1="4.93" x2="7.05" y2="7.05"/><line x1="16.95" y1="16.95" x2="19.07" y2="19.07"/>
-                <line x1="2" y1="12" x2="5" y2="12"/><line x1="19" y1="12" x2="22" y2="12"/>
-                <line x1="4.93" y1="19.07" x2="7.05" y2="16.95"/><line x1="16.95" y1="7.05" x2="19.07" y2="4.93"/>
-              </svg>
-            ) : (
-              <svg width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
-                <path d="M21 12.79A9 9 0 1 1 11.21 3 7 7 0 0 0 21 12.79z"/>
-              </svg>
-            )}
-          </span>
-          <span
-            className="text-sm whitespace-nowrap transition-opacity duration-100"
-            style={{ opacity: expanded ? 1 : 0, pointerEvents: expanded ? 'auto' : 'none', width: expanded ? 'auto' : 0, overflow: 'hidden' }}
-          >
-            {dark ? 'Light mode' : 'Dark mode'}
-          </span>
-        </button>
-      </div>
-
-      {/* User / logout */}
-      <div className="border-t border-line-soft px-3 py-2.5 flex items-center gap-2 shrink-0 overflow-hidden">
-        <div
-          className="w-6 h-6 rounded-full bg-raised text-muted flex items-center justify-center shrink-0 text-xs font-medium"
-          title={!expanded ? loginUser : undefined}
-        >
-          {loginUser ? loginUser[0].toUpperCase() : '?'}
-        </div>
-        <div
-          className="flex-1 flex items-center justify-between min-w-0 transition-opacity duration-100"
-          style={{ opacity: expanded ? 1 : 0, pointerEvents: expanded ? 'auto' : 'none', width: expanded ? 'auto' : 0, overflow: 'hidden' }}
-        >
-          <span className="text-xs text-muted truncate">{loginUser}</span>
-          <button
-            onClick={onLogout}
-            className="text-xs text-muted hover:text-danger ml-2 shrink-0"
-          >
-            Sign out
-          </button>
-        </div>
-      </div>
-    </div>
-  )
-}

ui/src/components/SourceTabs.jsx

@@ -1,60 +0,0 @@
-import { NavLink, Outlet, useParams, Link } from 'react-router-dom'
-
-// Everything scoped to one source lives under /sources/:name, so the source is
-// in the URL rather than in a global selector.
-// Records is the tab you want nine times out of ten, so it leads and is what
-// /sources/:name redirects to; Setup is the rare one and sits at the end.
-const TABS = [
-  { to: 'records', label: 'Records' },
-  { to: 'import', label: 'Import' },
-  { to: 'rules', label: 'Rules' },
-  { to: 'mappings', label: 'Mappings' },
-  { to: 'pivot', label: 'Pivot' },
-  { to: 'setup', label: 'Setup' },
-]
-
-export default function SourceTabs({ sources }) {
-  const { name } = useParams()
-  const sourceObj = sources.find(s => s.name === name)
-  const base = `/sources/${encodeURIComponent(name)}`
-
-  return (
-    <div className="flex flex-col h-full min-h-0">
-      <div className="px-4 sm:px-6 pt-4 sm:pt-5 shrink-0">
-        <div className="flex items-center gap-3">
-          <Link to="/sources" className="text-xs text-muted hover:text-ink-soft">Sources</Link>
-          <span className="text-muted text-xs">/</span>
-          <h1 className="text-xl font-semibold text-ink">{name}</h1>
-          {sourceObj?.config?.simplefin?.account_id && (
-            <span className="text-xs bg-accent-soft text-accent border border-accent-line rounded px-1.5 py-0.5">
-              bank feed
-            </span>
-          )}
-        </div>
-
-        <nav className="flex gap-1 mt-3 border-b border-line overflow-x-auto whitespace-nowrap">
-          {TABS.map(({ to, label, end }) => (
-            <NavLink
-              key={label}
-              to={to ? `${base}/${to}` : base}
-              end={end}
-              className={({ isActive }) =>
-                `text-sm px-3 py-1.5 -mb-px border-b-2 ${
-                  isActive
-                    ? 'border-accent text-accent font-medium'
-                    : 'border-transparent text-muted hover:text-ink-soft'
-                }`
-              }
-            >
-              {label}
-            </NavLink>
-          ))}
-        </nav>
-      </div>
-
-      <div className="flex-1 overflow-auto min-h-0">
-        <Outlet />
-      </div>
-    </div>
-  )
-}

ui/src/components/navItems.jsx

@@ -1,70 +0,0 @@
-// Top-level destinations, shared by the desktop sidebar and the mobile bar
-export const NAV = [
-  {
-    to: '/sources',
-    label: 'Sources',
-    icon: (
-      <svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
-        <ellipse cx="10" cy="5.5" rx="7" ry="2.5"/>
-        <path d="M3 5.5v9c0 1.4 3.1 2.5 7 2.5s7-1.1 7-2.5v-9"/>
-        <path d="M3 10.5c0 1.4 3.1 2.5 7 2.5s7-1.1 7-2.5"/>
-      </svg>
-    ),
-  },
-  {
-    to: '/import',
-    label: 'Import',
-    icon: (
-      <svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
-        <line x1="10" y1="3" x2="10" y2="14"/>
-        <polyline points="6,10 10,14 14,10"/>
-        <line x1="3" y1="18" x2="17" y2="18"/>
-      </svg>
-    ),
-  },
-  {
-    to: '/bridge',
-    label: 'Bridge',
-    icon: (
-      <svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
-        <path d="M2 13a8 8 0 0 1 16 0"/>
-        <line x1="2" y1="13" x2="18" y2="13"/>
-        <line x1="7" y1="13" x2="7" y2="9"/>
-        <line x1="13" y1="13" x2="13" y2="9"/>
-      </svg>
-    ),
-  },
-  {
-    to: '/remap',
-    label: 'Remap',
-    icon: (
-      <svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
-        <polyline points="2,8 2,4 6,4"/>
-        <path d="M2 4a8 8 0 0 1 14 2"/>
-        <polyline points="18,12 18,16 14,16"/>
-        <path d="M18 16a8 8 0 0 1-14-2"/>
-      </svg>
-    ),
-  },
-  {
-    to: '/stacks',
-    label: 'Stacks',
-    icon: (
-      <svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
-        <polygon points="10,2 18,6 10,10 2,6"/>
-        <polyline points="2,10 10,14 18,10"/>
-        <polyline points="2,14 10,18 18,14"/>
-      </svg>
-    ),
-  },
-  {
-    to: '/log',
-    label: 'Log',
-    icon: (
-      <svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
-        <circle cx="10" cy="10" r="8"/>
-        <polyline points="10,5 10,10 14,12"/>
-      </svg>
-    ),
-  },
-]

ui/src/index.css

@@ -1,107 +1,6 @@
 @import "tailwindcss";
 
-/*
-  Semantic colour tokens.
-
-  Components name the *role* of a colour (bg-surface, text-muted, border-line)
-  rather than a literal shade, so light and dark are two sets of variable values
-  instead of two sets of rules. Adding a component no longer means adding a
-  matching `.dark` override.
-
-  The raw palette below is the only place actual colours appear.
-*/
-
-@theme {
-  --color-canvas:      var(--bg-primary);
-  --color-surface:     var(--bg-secondary);
-  --color-raised:      var(--bg-tertiary);
-
-  --color-ink:         var(--text-primary);
-  --color-ink-soft:    var(--text-secondary);
-  --color-muted:       var(--text-muted);
-
-  --color-line:        var(--border-color);
-  --color-line-soft:   var(--border-light);
-
-  --color-accent:      var(--accent-text);
-  --color-accent-soft: var(--accent-bg);
-  --color-accent-line: var(--accent-line);
-
-  --color-ok:          var(--ok-text);
-  --color-ok-soft:     var(--ok-bg);
-  --color-warn:        var(--warn-text);
-  --color-warn-soft:   var(--warn-bg);
-  --color-warn-line:   var(--warn-line);
-  --color-danger:      var(--danger-text);
-  --color-danger-soft: var(--danger-bg);
-  --color-danger-line: var(--danger-line);
-}
-
-:root, .light {
-  --bg-primary: #f3f4f6;
-  --bg-secondary: #ffffff;
-  --bg-tertiary: #f9fafb;
-  --text-primary: #1f2937;
-  --text-secondary: #374151;
-  --text-muted: #9ca3af;
-  --border-color: #e5e7eb;
-  --border-light: #f3f4f6;
-  --accent-bg: #eff6ff;
-  --accent-text: #1d4ed8;
-  --accent-line: #bfdbfe;
-
-  --ok-text: #059669;
-  --ok-bg: #ecfdf5;
-  --warn-text: #b45309;
-  --warn-bg: #fffbeb;
-  --warn-line: #fde68a;
-  --danger-text: #ef4444;
-  --danger-bg: #fef2f2;
-  --danger-line: #fecaca;
-}
-
-/* Dark palette tuned to Perspective's "Pro Dark" theme:
-   bg #242526, tooltip #2a2c2f, gridline #3b3f46, inactive #61656e,
-   inactive border #4c505b, active #2770a9, legend #c5c9d0. */
-.dark {
-  --bg-primary: #242526;
-  --bg-secondary: #2a2c2f;
-  --bg-tertiary: #3b3f46;
-  --text-primary: #ffffff;
-  --text-secondary: #c5c9d0;
-  --text-muted: #61656e;
-  --border-color: #4c505b;
-  --border-light: #3b3f46;
-  --accent-bg: rgba(39, 113, 170, 0.32);
-  --accent-text: #4778c2;
-  --accent-line: #2770a9;
-
-  /* Status accents desaturated to sit on Pro Dark's neutral background */
-  --ok-text: #6ee7b7;
-  --ok-bg: #1a3d2c;
-  --warn-text: #f5c66f;
-  --warn-bg: #3a2e14;
-  --warn-line: #5a4a26;
-  --danger-text: #ff9485;
-  --danger-bg: #3d1f1f;
-  --danger-line: #6b3030;
-}
-
 body {
   margin: 0;
   font-family: system-ui, -apple-system, sans-serif;
-  background-color: var(--bg-primary);
-  color: var(--text-primary);
 }
-
-/* Bare border utilities have no colour of their own */
-.border, .border-t, .border-b, .border-l, .border-r { border-color: var(--border-color); }
-
-/* Form controls don't inherit the surface token on their own */
-input, select, textarea {
-  background-color: var(--bg-secondary);
-  color: var(--text-primary);
-  border-color: var(--border-color);
-}
-
-::selection { background-color: var(--accent-bg); color: var(--text-primary); }

ui/src/main.jsx

@@ -1,13 +1,10 @@
 import { StrictMode } from 'react'
 import { createRoot } from 'react-dom/client'
-import { ThemeProvider } from './theme.jsx'
 import './index.css'
 import App from './App.jsx'
 
 createRoot(document.getElementById('root')).render(
   <StrictMode>
-    <ThemeProvider>
+    <App />
-      <App />
-    </ThemeProvider>
   </StrictMode>,
 )

ui/src/pages/Bridge.jsx

@@ -1,176 +0,0 @@
-import { useState } from 'react'
-import { Link } from 'react-router-dom'
-import { api } from '../api'
-import Section from '../components/Section.jsx'
-
-// Accounting style: aligned to 2 decimals, negatives in parentheses
-function money(value) {
-  const n = parseFloat(value)
-  if (!isFinite(n)) return '—'
-  const abs = Math.abs(n).toLocaleString(undefined, { minimumFractionDigits: 2, maximumFractionDigits: 2 })
-  return n < 0 ? `(${abs})` : abs
-}
-
-// SimpleFIN doesn't report an account type, so retirement accounts are
-// recognised from the institution and account names
-const RETIREMENT_RE = /401\(?k\)?|403\(?b\)?|\bira\b|retirement|pension|profit sharing/i
-const isRetirement = (a) => RETIREMENT_RE.test(`${a.name} ${a.organization || ''}`)
-
-// One SimpleFIN bridge covers every linked bank account, so connection state is
-// a bridge-level concern rather than something to hunt for source by source.
-export default function Bridge({ sources }) {
-  const [accounts, setAccounts] = useState(null)
-  const [errors, setErrors] = useState([])
-  const [loading, setLoading] = useState(false)
-  const [error, setError] = useState('')
-
-  async function load() {
-    setLoading(true)
-    setError('')
-    try {
-      const res = await api.getSimpleFinAccounts()
-      setAccounts(res.accounts || [])
-      setErrors(res.errors || [])
-    } catch (err) {
-      setError(err.message)
-    } finally {
-      setLoading(false)
-    }
-  }
-
-  // Which source, if any, pulls from each account
-  const sourceFor = (accountId) =>
-    sources.find(s => s.config?.simplefin?.account_id === accountId)
-
-  const sum = (list) => list.reduce((t, a) => t + (parseFloat(a.balance) || 0), 0)
-  const banking = (accounts || []).filter(a => !isRetirement(a))
-  const retirement = (accounts || []).filter(isRetirement)
-  // Banking first, then retirement, so each subtotal sits under its own rows
-  const ordered = [...banking, ...retirement]
-
-  return (
-    <div className="p-4 sm:p-6 max-w-5xl space-y-4">
-      <div className="flex items-center justify-between mb-2">
-        <h1 className="text-xl font-semibold text-ink">Bridge</h1>
-        <button
-          onClick={load}
-          disabled={loading}
-          className="text-sm border border-line rounded px-3 py-1.5 text-ink-soft hover:bg-raised hover:border-line disabled:opacity-50"
-        >
-          {loading ? 'Refreshing…' : 'Refresh'}
-        </button>
-      </div>
-
-      {error && (
-        <Section title="Not connected" description="Claim a setup token with manage.py option 10, then restart the service.">
-          <p className="text-xs text-danger">{error}</p>
-        </Section>
-      )}
-
-      {errors.length > 0 && (
-        <div className="bg-warn-soft border border-warn-line rounded p-3 text-xs text-warn space-y-1">
-          {errors.map((e, i) => <div key={i}>{e}</div>)}
-        </div>
-      )}
-
-      {!accounts && !loading && !error && (
-        <p className="text-sm text-muted">Click Refresh to load balances from SimpleFIN.</p>
-      )}
-
-      {accounts && (
-        <Section
-          title="SimpleFIN accounts"
-          description="Every account behind the bridge credential, and which source pulls from it."
-        >
-          <table className="w-full text-xs hidden sm:table">
-            <thead>
-              <tr className="text-left text-muted border-b border-line-soft">
-                <th className="pb-1 font-medium">Account</th>
-                <th className="pb-1 font-medium">Institution</th>
-                <th className="pb-1 font-medium text-right">Balance</th>
-                <th className="pb-1 pl-4 font-medium">As of</th>
-                <th className="pb-1 font-medium">Source</th>
-              </tr>
-            </thead>
-            <tbody>
-              {ordered.map(a => {
-                const src = sourceFor(a.id)
-                return (
-                  <tr key={a.id} className="border-t border-line-soft">
-                    <td className="py-1.5 text-ink-soft">{a.name}</td>
-                    <td className="py-1.5 text-muted">{a.organization}</td>
-                    <td className="py-1.5 text-right font-mono text-ink-soft">{money(a.balance)}</td>
-                    <td className="py-1.5 pl-4 text-muted">{a.balance_date}</td>
-                    <td className="py-1.5">
-                      {src
-                        ? <Link to={`/sources/${encodeURIComponent(src.name)}`} className="text-accent hover:text-accent">{src.name}</Link>
-                        : <span className="text-muted">not linked</span>}
-                    </td>
-                  </tr>
-                )
-              })}
-            </tbody>
-            <tfoot>
-              {banking.length > 0 && (
-                <tr className="border-t border-line">
-                  <td className="pt-2 text-muted" colSpan={2}>Banking and cards</td>
-                  <td className="pt-2 text-right font-mono text-ink-soft">{money(sum(banking))}</td>
-                  <td colSpan={2}></td>
-                </tr>
-              )}
-              {retirement.length > 0 && (
-                <tr>
-                  <td className="pt-1 text-muted" colSpan={2}>Retirement</td>
-                  <td className="pt-1 text-right font-mono text-ink-soft">{money(sum(retirement))}</td>
-                  <td colSpan={2}></td>
-                </tr>
-              )}
-              <tr className="border-t border-line">
-                <td className="pt-2 text-ink-soft font-medium" colSpan={2}>Total</td>
-                <td className="pt-2 text-right font-mono font-medium text-ink">{money(sum(accounts))}</td>
-                <td colSpan={2}></td>
-              </tr>
-            </tfoot>
-          </table>
-          {/* Stacked cards for narrow screens */}
-          <div className="sm:hidden divide-y divide-line-soft">
-            {ordered.map(a => {
-              const src = sourceFor(a.id)
-              return (
-                <div key={a.id} className="py-2">
-                  <div className="flex justify-between gap-2">
-                    <span className="text-xs text-ink-soft">{a.name}</span>
-                    <span className="text-xs font-mono text-ink">{money(a.balance)}</span>
-                  </div>
-                  <div className="flex justify-between gap-2 text-xs text-muted">
-                    <span>{a.organization}</span>
-                    <span>{a.balance_date}</span>
-                  </div>
-                  <div className="text-xs mt-0.5">
-                    {src
-                      ? <Link to={`/sources/${encodeURIComponent(src.name)}`} className="text-accent">{src.name}</Link>
-                      : <span className="text-muted">not linked</span>}
-                  </div>
-                </div>
-              )
-            })}
-            <div className="pt-2 flex justify-between text-xs">
-              <span className="text-muted">Banking and cards</span>
-              <span className="font-mono text-ink-soft">{money(sum(banking))}</span>
-            </div>
-            <div className="pt-1 flex justify-between text-xs border-0">
-              <span className="text-muted">Retirement</span>
-              <span className="font-mono text-ink-soft">{money(sum(retirement))}</span>
-            </div>
-            <div className="pt-2 flex justify-between text-xs font-medium">
-              <span className="text-ink-soft">Total</span>
-              <span className="font-mono text-ink">{money(sum(accounts))}</span>
-            </div>
-          </div>
-
-          {accounts.length === 0 && <p className="text-xs text-muted">No accounts returned.</p>}
-        </Section>
-      )}
-    </div>
-  )
-}

ui/src/pages/Import.jsx

@@ -6,7 +6,7 @@ function KeyList({ keys, label, color }) {
   return (
     <div className="mb-2">
       <div className={`text-xs font-medium mb-1 ${color}`}>{label} ({keys.length})</div>
-      <div className="max-h-32 overflow-y-auto bg-raised rounded p-2 font-mono text-xs text-muted space-y-0.5">
+      <div className="max-h-32 overflow-y-auto bg-gray-50 rounded p-2 font-mono text-xs text-gray-500 space-y-0.5">
         {keys.map((k, i) => (
           <div key={i}>
             {typeof k === 'object' && k !== null
@@ -28,19 +28,19 @@ function LogRow({ entry, selected, onToggle }) {
 
   return (
     <>
-      <tr className={`border-b border-line-soft ${selected ? 'bg-danger-soft' : ''}`}>
+      <tr className={`border-b border-gray-50 ${selected ? 'bg-red-50' : ''}`}>
         <td className="py-1.5 pr-2">
           <input type="checkbox" checked={selected} onChange={onToggle} className="cursor-pointer" />
         </td>
-        <td className="py-1.5 text-xs text-muted font-mono">{entry.id}</td>
+        <td className="py-1.5 text-xs text-gray-400 font-mono">{entry.id}</td>
-        <td className="py-1.5 text-muted">{new Date(entry.imported_at).toLocaleString()}</td>
+        <td className="py-1.5 text-gray-500">{new Date(entry.imported_at).toLocaleString()}</td>
-        <td className="py-1.5 text-ink">{entry.records_imported}</td>
+        <td className="py-1.5 text-gray-800">{entry.records_imported}</td>
-        <td className="py-1.5 text-muted">{entry.records_duplicate}</td>
+        <td className="py-1.5 text-gray-400">{entry.records_duplicate}</td>
         <td className="py-1.5">
           {hasKeys && (
             <button
               onClick={() => setExpanded(e => !e)}
-              className="text-xs text-accent hover:text-accent"
+              className="text-xs text-blue-400 hover:text-blue-600"
             >
               {expanded ? '▲ hide' : '▼ keys'}
             </button>
@@ -48,10 +48,10 @@ function LogRow({ entry, selected, onToggle }) {
         </td>
       </tr>
       {expanded && (
-        <tr className={selected ? 'bg-danger-soft' : 'bg-raised'}>
+        <tr className={selected ? 'bg-red-50' : 'bg-gray-50'}>
           <td colSpan={6} className="px-4 py-3">
-            <KeyList keys={insertedKeys} label="Inserted" color="text-ok" />
+            <KeyList keys={insertedKeys} label="Inserted" color="text-green-600" />
-            <KeyList keys={excludedKeys} label="Excluded" color="text-muted" />
+            <KeyList keys={excludedKeys} label="Excluded" color="text-gray-500" />
           </td>
         </tr>
       )}
@@ -67,15 +67,12 @@ export default function Import({ source }) {
   const [error, setError] = useState('')
   const [dragOver, setDragOver] = useState(false)
   const [selected, setSelected] = useState(new Set())
-  const [simplefin, setSimplefin] = useState(null)
-  const [days, setDays] = useState('10')
   const fileRef = useRef()
 
   useEffect(() => {
     if (!source) return
     api.getStats(source).then(setStats).catch(() => {})
     api.getImportLog(source).then(setLog).catch(() => {})
-    api.getSource(source).then(s => setSimplefin(s.config?.simplefin || null)).catch(() => setSimplefin(null))
     setSelected(new Set())
   }, [source])
 
@@ -96,23 +93,6 @@ export default function Import({ source }) {
     }
   }
 
-  async function handleSync() {
-    if (!source) return
-    setLoading(true)
-    setError('')
-    setResult(null)
-    try {
-      const res = await api.syncSimpleFin(source, { days })
-      setResult(res)
-      api.getStats(source).then(setStats)
-      api.getImportLog(source).then(setLog)
-    } catch (err) {
-      setError(err.message)
-    } finally {
-      setLoading(false)
-    }
-  }
-
   async function handleTransform() {
     if (!source) return
     setLoading(true)
@@ -168,11 +148,11 @@ export default function Import({ source }) {
     }
   }
 
-  if (!source) return <div className="p-4 sm:p-6 text-sm text-muted">Select a source first.</div>
+  if (!source) return <div className="p-6 text-sm text-gray-400">Select a source first.</div>
 
   return (
-    <div className="p-4 sm:p-6 max-w-2xl">
+    <div className="p-6 max-w-2xl">
-      <h1 className="text-xl font-semibold text-ink mb-6">Import — {source}</h1>
+      <h1 className="text-xl font-semibold text-gray-800 mb-6">Import — {source}</h1>
 
       {/* Stats */}
       {stats && (
@@ -182,42 +162,18 @@ export default function Import({ source }) {
             { label: 'Transformed', value: stats.transformed_records },
             { label: 'Pending', value: stats.pending_records },
           ].map(({ label, value }) => (
-            <div key={label} className="bg-surface border border-line rounded px-4 py-3 flex-1 text-center">
+            <div key={label} className="bg-white border border-gray-200 rounded px-4 py-3 flex-1 text-center">
-              <div className="text-2xl font-semibold text-ink">{value}</div>
+              <div className="text-2xl font-semibold text-gray-800">{value}</div>
-              <div className="text-xs text-muted mt-0.5">{label}</div>
+              <div className="text-xs text-gray-400 mt-0.5">{label}</div>
             </div>
           ))}
         </div>
       )}
 
-      {/* SimpleFIN sync — only for sources with a bridge account in their config */}
-      {simplefin?.account_id && (
-        <div className="bg-surface border border-line rounded p-4 mb-4 flex items-center gap-3">
-          <div className="flex-1 min-w-0">
-            <div className="text-sm font-medium text-ink-soft">SimpleFIN</div>
-            <div className="text-xs text-muted font-mono truncate">{simplefin.account_id}</div>
-          </div>
-          <select
-            value={days}
-            onChange={e => setDays(e.target.value)}
-            className="text-sm border border-line rounded px-2 py-1.5 bg-surface text-ink-soft"
-          >
-            <option value="10">Last 10 days</option>
-            <option value="30">Last 30 days</option>
-            <option value="45">Last 45 days</option>
-            <option value="89">Backfill (bridge maximum)</option>
-          </select>
-          <button onClick={handleSync} disabled={loading}
-            className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700 disabled:opacity-50">
-            {loading ? 'Syncing…' : 'Sync now'}
-          </button>
-        </div>
-      )}
-
       {/* Drop zone */}
       <div
         className={`border-2 border-dashed rounded-lg p-8 text-center mb-4 cursor-pointer transition-colors ${
-          dragOver ? 'border-accent bg-accent-soft' : 'border-line hover:border-line'
+          dragOver ? 'border-blue-400 bg-blue-50' : 'border-gray-200 hover:border-gray-300'
         }`}
         onDragOver={e => { e.preventDefault(); setDragOver(true) }}
         onDragLeave={() => setDragOver(false)}
@@ -232,22 +188,22 @@ export default function Import({ source }) {
           onChange={e => handleImport(e.target.files[0])}
         />
         {loading
-          ? <p className="text-sm text-muted">Importing…</p>
+          ? <p className="text-sm text-gray-500">Importing…</p>
-          : <p className="text-sm text-muted">Drop a CSV file here, or click to browse</p>
+          : <p className="text-sm text-gray-400">Drop a CSV file here, or click to browse</p>
         }
       </div>
 
-      {error && <p className="text-sm text-danger mb-3">{error}</p>}
+      {error && <p className="text-sm text-red-500 mb-3">{error}</p>}
 
       {result && (
-        <div className={`border rounded p-4 mb-4 text-sm ${result.success === false ? 'bg-danger-soft border-danger-line' : 'bg-surface border-line'}`}>
+        <div className={`border rounded p-4 mb-4 text-sm ${result.success === false ? 'bg-red-50 border-red-200' : 'bg-white border-gray-200'}`}>
           {result.success === false ? (
             <>
-              <p className="text-danger font-medium mb-2">{result.error}</p>
+              <p className="text-red-600 font-medium mb-2">{result.error}</p>
               {result.duplicate_rows && (
                 <div>
-                  <p className="text-xs text-danger mb-1">Offending rows:</p>
+                  <p className="text-xs text-red-500 mb-1">Offending rows:</p>
-                  <div className="max-h-48 overflow-y-auto bg-surface rounded border border-danger-line p-2 font-mono text-xs text-danger space-y-0.5">
+                  <div className="max-h-48 overflow-y-auto bg-white rounded border border-red-100 p-2 font-mono text-xs text-red-700 space-y-0.5">
                     {result.duplicate_rows.map((row, i) => (
                       <div key={i}>
                         {Object.entries(row).map(([f, v]) => `${f}: ${v}`).join(' · ')}
@@ -259,29 +215,18 @@ export default function Import({ source }) {
             </>
           ) : result.imported !== undefined ? (
             <>
-              {result.errors?.length > 0 && (
+              <span className="text-green-600 font-medium">{result.imported} imported</span>
-                <div className="mb-2 text-xs text-warn">
+              <span className="text-gray-400 mx-2">·</span>
-                  {result.errors.map((e, i) => <div key={i}>Bridge: {e}</div>)}
+              <span className="text-gray-500">{result.duplicates} duplicates skipped</span>
-                </div>
-              )}
-              {result.fetched !== undefined && (
-                <>
-                  <span className="text-muted">{result.fetched} fetched</span>
-                  <span className="text-muted mx-2">·</span>
-                </>
-              )}
-              <span className="text-ok font-medium">{result.imported} imported</span>
-              <span className="text-muted mx-2">·</span>
-              <span className="text-muted">{result.duplicates} duplicates skipped</span>
               {result.transform && (
                 <>
-                  <span className="text-muted mx-2">·</span>
+                  <span className="text-gray-400 mx-2">·</span>
-                  <span className="text-muted">{result.transform.transformed} transformed</span>
+                  <span className="text-gray-500">{result.transform.transformed} transformed</span>
                 </>
               )}
             </>
           ) : (
-            <span className="text-ok font-medium">{result.transformed} records transformed</span>
+            <span className="text-green-600 font-medium">{result.transformed} records transformed</span>
           )}
         </div>
       )}
@@ -306,7 +251,7 @@ export default function Import({ source }) {
       {log.length > 0 && (
         <div>
           <div className="flex items-center justify-between mb-2">
-            <h2 className="text-sm font-semibold text-ink-soft">Import history</h2>
+            <h2 className="text-sm font-semibold text-gray-700">Import history</h2>
             {selected.size > 0 && (
               <button
                 onClick={handleDeleteSelected}
@@ -319,7 +264,7 @@ export default function Import({ source }) {
           </div>
           <table className="w-full text-sm">
             <thead>
-              <tr className="text-left text-xs text-muted border-b border-line-soft">
+              <tr className="text-left text-xs text-gray-400 border-b border-gray-100">
                 <th className="pb-1 w-6"></th>
                 <th className="pb-1 font-medium w-12">ID</th>
                 <th className="pb-1 font-medium">Date</th>

ui/src/pages/ImportHub.jsx

@@ -1,129 +0,0 @@
-import { useState, useEffect } from 'react'
-import { Link } from 'react-router-dom'
-import { api } from '../api'
-
-// Importing is the frequent job; configuring a source is the rare one. This is
-// the top-level entry point for the frequent one — every source in one place,
-// with a sync button for anything on a bank feed.
-export default function ImportHub({ sources }) {
-  const [stats, setStats] = useState({})       // name -> stats
-  const [lastImport, setLastImport] = useState({})  // name -> ISO timestamp
-  const [busy, setBusy] = useState('')
-  const [results, setResults] = useState({})   // name -> message
-  const [errors, setErrors] = useState({})     // name -> message
-
-  useEffect(() => {
-    let cancelled = false
-    Promise.all(sources.map(s =>
-      api.getStats(s.name).then(st => [s.name, st]).catch(() => [s.name, null])
-    )).then(pairs => {
-      if (!cancelled) setStats(Object.fromEntries(pairs))
-    })
-    api.getAllImportLog().then(log => {
-      if (cancelled) return
-      const latest = {}
-      for (const entry of log) {
-        if (!latest[entry.source_name] || entry.imported_at > latest[entry.source_name]) {
-          latest[entry.source_name] = entry.imported_at
-        }
-      }
-      setLastImport(latest)
-    }).catch(() => {})
-    return () => { cancelled = true }
-  }, [sources])
-
-  async function sync(name) {
-    setBusy(name)
-    setErrors(e => ({ ...e, [name]: '' }))
-    setResults(r => ({ ...r, [name]: '' }))
-    try {
-      const res = await api.syncSimpleFin(name, { days: 10 })
-      setResults(r => ({
-        ...r,
-        [name]: `${res.imported} imported, ${res.duplicates} already had` +
-                (res.errors?.length ? ` — ${res.errors.join('; ')}` : ''),
-      }))
-      api.getStats(name).then(st => setStats(s => ({ ...s, [name]: st }))).catch(() => {})
-      api.getAllImportLog().then(log => {
-        const entry = log.find(l => l.source_name === name)
-        if (entry) setLastImport(l => ({ ...l, [name]: entry.imported_at }))
-      }).catch(() => {})
-    } catch (err) {
-      setErrors(e => ({ ...e, [name]: err.message }))
-    } finally {
-      setBusy('')
-    }
-  }
-
-  const feeds = sources.filter(s => s.config?.simplefin?.account_id)
-  const manual = sources.filter(s => !s.config?.simplefin?.account_id)
-
-  function Row({ s, isFeed }) {
-    const st = stats[s.name]
-    const when = lastImport[s.name]
-    return (
-      <div className="px-4 py-3 flex items-center gap-3 flex-wrap">
-        <div className="flex-1 min-w-40">
-          <Link to={`/sources/${encodeURIComponent(s.name)}/import`}
-            className="text-sm font-medium text-ink hover:text-accent">
-            {s.name}
-          </Link>
-          <div className="text-xs text-muted">
-            {st ? `${st.total_records} records` : '—'}
-            {st && Number(st.pending_records) > 0 && ` · ${st.pending_records} untransformed`}
-            {when && ` · last import ${new Date(when).toLocaleDateString()}`}
-          </div>
-        </div>
-
-        {results[s.name] && <span className="text-xs text-ok">{results[s.name]}</span>}
-        {errors[s.name] && <span className="text-xs text-danger">{errors[s.name]}</span>}
-
-        {isFeed ? (
-          <button
-            onClick={() => sync(s.name)}
-            disabled={busy === s.name}
-            className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700 disabled:opacity-50"
-          >
-            {busy === s.name ? 'Syncing…' : 'Sync'}
-          </button>
-        ) : (
-          <Link to={`/sources/${encodeURIComponent(s.name)}/import`}
-            className="text-sm border border-line rounded px-3 py-1.5 text-ink-soft hover:bg-raised hover:border-line">
-            Upload CSV
-          </Link>
-        )}
-      </div>
-    )
-  }
-
-  return (
-    <div className="p-4 sm:p-6 max-w-4xl space-y-6">
-      <h1 className="text-xl font-semibold text-ink">Import</h1>
-
-      {feeds.length > 0 && (
-        <div>
-          <h2 className="text-sm font-semibold text-ink-soft mb-2">Bank feeds</h2>
-          <div className="bg-surface border border-line rounded divide-y divide-line-soft">
-            {feeds.map(s => <Row key={s.name} s={s} isFeed />)}
-          </div>
-          <p className="text-xs text-muted mt-1">Syncs pull the last 10 days; use a source&rsquo;s Import tab to backfill further.</p>
-        </div>
-      )}
-
-      {manual.length > 0 && (
-        <div>
-          <h2 className="text-sm font-semibold text-ink-soft mb-2">CSV sources</h2>
-          <div className="bg-surface border border-line rounded divide-y divide-line-soft">
-            {manual.map(s => <Row key={s.name} s={s} isFeed={false} />)}
-          </div>
-        </div>
-      )}
-
-      {sources.length === 0 && <p className="text-sm text-muted">No sources yet.</p>}
-
-      <Link to="/log" className="inline-block text-xs text-accent hover:text-accent">
-        Full import history →
-      </Link>
-    </div>
-  )
-}

ui/src/pages/Log.jsx

@@ -6,7 +6,7 @@ function KeyList({ keys, label, color }) {
   return (
     <div className="mb-2">
       <div className={`text-xs font-medium mb-1 ${color}`}>{label} ({keys.length})</div>
-      <div className="max-h-32 overflow-y-auto bg-raised rounded p-2 font-mono text-xs text-muted space-y-0.5">
+      <div className="max-h-32 overflow-y-auto bg-gray-50 rounded p-2 font-mono text-xs text-gray-500 space-y-0.5">
         {keys.map((k, i) => (
           <div key={i}>
             {typeof k === 'object' && k !== null
@@ -28,17 +28,17 @@ function LogRow({ entry }) {
 
   return (
     <>
-      <tr className="border-b border-line-soft hover:bg-raised">
+      <tr className="border-b border-gray-50 hover:bg-gray-50">
-        <td className="py-1.5 text-xs text-muted font-mono pr-3">{entry.id}</td>
+        <td className="py-1.5 text-xs text-gray-400 font-mono pr-3">{entry.id}</td>
-        <td className="py-1.5 text-ink-soft pr-3">{entry.source_name}</td>
+        <td className="py-1.5 text-gray-700 pr-3">{entry.source_name}</td>
-        <td className="py-1.5 text-muted pr-3">{new Date(entry.imported_at).toLocaleString()}</td>
+        <td className="py-1.5 text-gray-500 pr-3">{new Date(entry.imported_at).toLocaleString()}</td>
-        <td className="py-1.5 text-ink pr-3">{entry.records_imported}</td>
+        <td className="py-1.5 text-gray-800 pr-3">{entry.records_imported}</td>
-        <td className="py-1.5 text-muted pr-3">{entry.records_duplicate}</td>
+        <td className="py-1.5 text-gray-400 pr-3">{entry.records_duplicate}</td>
         <td className="py-1.5">
           {hasKeys && (
             <button
               onClick={() => setExpanded(e => !e)}
-              className="text-xs text-accent hover:text-accent"
+              className="text-xs text-blue-400 hover:text-blue-600"
             >
               {expanded ? '▲ hide' : '▼ keys'}
             </button>
@@ -46,10 +46,10 @@ function LogRow({ entry }) {
         </td>
       </tr>
       {expanded && (
-        <tr className="bg-raised">
+        <tr className="bg-gray-50">
           <td colSpan={6} className="px-4 py-3">
-            <KeyList keys={insertedKeys} label="Inserted" color="text-ok" />
+            <KeyList keys={insertedKeys} label="Inserted" color="text-green-600" />
-            <KeyList keys={excludedKeys} label="Excluded" color="text-muted" />
+            <KeyList keys={excludedKeys} label="Excluded" color="text-gray-500" />
           </td>
         </tr>
       )}
@@ -70,18 +70,18 @@ export default function Log() {
 
   return (
     <div className="p-6">
-      <h1 className="text-xl font-semibold text-ink mb-6">Import Log</h1>
+      <h1 className="text-xl font-semibold text-gray-800 mb-6">Import Log</h1>
 
-      {loading && <p className="text-sm text-muted">Loading…</p>}
+      {loading && <p className="text-sm text-gray-400">Loading…</p>}
 
       {!loading && log.length === 0 && (
-        <p className="text-sm text-muted">No imports yet.</p>
+        <p className="text-sm text-gray-400">No imports yet.</p>
       )}
 
       {log.length > 0 && (
         <table className="w-full text-sm">
           <thead>
-            <tr className="text-left text-xs text-muted border-b border-line-soft">
+            <tr className="text-left text-xs text-gray-400 border-b border-gray-100">
               <th className="pb-1 font-medium pr-3">ID</th>
               <th className="pb-1 font-medium pr-3">Source</th>
               <th className="pb-1 font-medium pr-3">Date</th>

ui/src/pages/Login.jsx

@@ -20,32 +20,32 @@ export default function Login({ onLogin }) {
   }
 
   return (
-    <div className="flex items-center justify-center h-screen bg-raised">
+    <div className="flex items-center justify-center h-screen bg-gray-50">
-      <div className="bg-surface border border-line rounded-lg p-8 w-80 shadow-sm">
+      <div className="bg-white border border-gray-200 rounded-lg p-8 w-80 shadow-sm">
-        <h1 className="text-lg font-semibold text-ink mb-6">Dataflow</h1>
+        <h1 className="text-lg font-semibold text-gray-800 mb-6">Dataflow</h1>
         <form onSubmit={handleSubmit} className="space-y-4">
           <div>
-            <label className="block text-xs text-muted mb-1">Username</label>
+            <label className="block text-xs text-gray-500 mb-1">Username</label>
             <input
               type="text"
               autoFocus
               value={user}
               onChange={e => setUser(e.target.value)}
-              className="w-full border border-line rounded px-3 py-2 text-sm focus:outline-none focus:border-accent"
+              className="w-full border border-gray-200 rounded px-3 py-2 text-sm focus:outline-none focus:border-blue-400"
               required
             />
           </div>
           <div>
-            <label className="block text-xs text-muted mb-1">Password</label>
+            <label className="block text-xs text-gray-500 mb-1">Password</label>
             <input
               type="password"
               value={pass}
               onChange={e => setPass(e.target.value)}
-              className="w-full border border-line rounded px-3 py-2 text-sm focus:outline-none focus:border-accent"
+              className="w-full border border-gray-200 rounded px-3 py-2 text-sm focus:outline-none focus:border-blue-400"
               required
             />
           </div>
-          {error && <p className="text-xs text-danger">{error}</p>}
+          {error && <p className="text-xs text-red-500">{error}</p>}
           <button
             type="submit"
             disabled={loading}

ui/src/pages/Mappings.jsx

@@ -4,7 +4,6 @@ import { api, authHeaders } from '../api'
 function AutocompleteInput({ value, onChange, onEnter, suggestions = [], className, placeholder }) {
   const [open, setOpen] = useState(false)
   const [highlighted, setHighlighted] = useState(0)
-  const [dropPos, setDropPos] = useState(null)
   const inputRef = useRef()
   const listRef = useRef()
 
@@ -13,10 +12,6 @@ function AutocompleteInput({ value, onChange, onEnter, suggestions = [], classNa
     : suggestions
 
   function openList() {
-    if (inputRef.current) {
-      const r = inputRef.current.getBoundingClientRect()
-      setDropPos({ top: r.bottom + 2, left: r.left, minWidth: r.width })
-    }
     setOpen(true)
     setHighlighted(0)
   }
@@ -47,6 +42,7 @@ function AutocompleteInput({ value, onChange, onEnter, suggestions = [], classNa
     if (e.key === 'Enter') onEnter?.()
   }
 
+  // Scroll highlighted item into view
   useEffect(() => {
     if (!open || !listRef.current) return
     const item = listRef.current.children[highlighted]
@@ -64,17 +60,16 @@ function AutocompleteInput({ value, onChange, onEnter, suggestions = [], classNa
         onKeyDown={handleKeyDown}
         onBlur={e => { if (!listRef.current?.contains(e.relatedTarget)) setOpen(false) }}
       />
-      {open && filtered.length > 0 && dropPos && (
+      {open && filtered.length > 0 && (
         <div
           ref={listRef}
-          style={{ position: 'fixed', top: dropPos.top, left: dropPos.left, minWidth: dropPos.minWidth, zIndex: 9999 }}
+          className="absolute z-50 left-0 top-full mt-0.5 bg-white border border-gray-200 rounded shadow-lg max-h-48 overflow-y-auto min-w-full"
-          className="bg-surface border border-line rounded shadow-lg max-h-48 overflow-y-auto"
         >
           {filtered.map((s, i) => (
             <div
               key={s}
               className={`px-2 py-1 text-xs cursor-pointer whitespace-nowrap ${
-                i === highlighted ? 'bg-accent-soft text-accent' : 'text-ink-soft hover:bg-raised'
+                i === highlighted ? 'bg-blue-50 text-blue-700' : 'text-gray-700 hover:bg-gray-50'
               }`}
               onMouseDown={e => { e.preventDefault(); select(s) }}
             >
@@ -100,16 +95,16 @@ function SortHeader({ col, label, sortBy, onSort, className = '' }) {
   const active = sortBy?.col === col
   return (
     <th
-      className={`px-3 py-2 font-medium cursor-pointer select-none hover:text-ink-soft ${className}`}
+      className={`px-3 py-2 font-medium cursor-pointer select-none hover:text-gray-600 ${className}`}
       onClick={() => onSort(col)}
     >
       {label}
-      <span className="ml-1 text-muted">{active ? (sortBy.dir === 'asc' ? '↑' : '↓') : '↕'}</span>
+      <span className="ml-1 text-gray-300">{active ? (sortBy.dir === 'asc' ? '↑' : '↓') : '↕'}</span>
     </th>
   )
 }
 
-export default function Mappings({ source, onNeedsReprocess }) {
+export default function Mappings({ source }) {
   const [rules, setRules] = useState([])
   const [selectedRule, setSelectedRule] = useState('')
   const [allValues, setAllValues] = useState([])
@@ -268,7 +263,6 @@ export default function Mappings({ source, onNeedsReprocess }) {
           valueKey(x.extracted_value) === k ? { ...x, is_mapped: true, mapping_id: created.id, output } : x
         ))
       }
-      onNeedsReprocess?.(source)
       setDrafts(d => { const n = { ...d }; delete n[k]; return n })
     } catch (err) {
       alert(err.message)
@@ -315,7 +309,6 @@ export default function Mappings({ source, onNeedsReprocess }) {
         setSaving(s => ({ ...s, [k]: false }))
       }
     }))
-    onNeedsReprocess?.(source)
     setSelected(new Set())
     setBulkDraft({})
   }
@@ -324,7 +317,6 @@ export default function Mappings({ source, onNeedsReprocess }) {
     if (!row.mapping_id) return
     try {
       await api.deleteMapping(row.mapping_id)
-      onNeedsReprocess?.(source)
       setAllValues(av => av.map(x =>
         valueKey(x.extracted_value) === valueKey(row.extracted_value)
           ? { ...x, is_mapped: false, mapping_id: null, output: null }
@@ -354,18 +346,18 @@ export default function Mappings({ source, onNeedsReprocess }) {
     }
   }
 
-  if (!source) return <div className="p-4 sm:p-6 text-sm text-muted">Select a source first.</div>
+  if (!source) return <div className="p-6 text-sm text-gray-400">Select a source first.</div>
 
   const displayRows = sortedRows(filteredRows)
 
   return (
     <div>
       {/* Sticky control bar */}
-      <div className="sticky top-0 z-10 bg-surface border-b border-line px-6 py-3 flex items-center gap-3 flex-wrap">
+      <div className="sticky top-0 z-10 bg-white border-b border-gray-200 px-6 py-3 flex items-center gap-3 flex-wrap">
-        <span className="text-sm font-medium text-ink-soft">{source}</span>
+        <span className="text-sm font-medium text-gray-700">{source}</span>
 
         <select
-          className="text-sm border border-line rounded px-2 py-1.5 focus:outline-none focus:border-accent"
+          className="text-sm border border-gray-200 rounded px-2 py-1.5 focus:outline-none focus:border-blue-400"
           value={selectedRule}
           onChange={e => setSelectedRule(e.target.value)}
         >
@@ -374,7 +366,7 @@ export default function Mappings({ source, onNeedsReprocess }) {
         </select>
 
         {selectedRule && (
-          <div className="flex bg-raised rounded p-0.5">
+          <div className="flex bg-gray-100 rounded p-0.5">
             {[
               { key: 'all', label: `All (${allValues.length})` },
               { key: 'unmapped', label: `Unmapped (${unmappedCount})` },
@@ -382,7 +374,7 @@ export default function Mappings({ source, onNeedsReprocess }) {
             ].map(({ key, label }) => (
               <button key={key} onClick={() => setFilter(key)}
                 className={`text-xs px-3 py-1 rounded transition-colors ${
-                  filter === key ? 'bg-surface text-ink shadow-sm' : 'text-muted'
+                  filter === key ? 'bg-white text-gray-800 shadow-sm' : 'text-gray-500'
                 }`}>
                 {label}
               </button>
@@ -393,15 +385,15 @@ export default function Mappings({ source, onNeedsReprocess }) {
         {selectedRule && (
           <div className="relative">
             <input
-              className={`text-xs font-mono border rounded px-2 py-1.5 w-44 focus:outline-none focus:border-accent ${
+              className={`text-xs font-mono border rounded px-2 py-1.5 w-44 focus:outline-none focus:border-blue-400 ${
-                rowFilterError ? 'border-danger-line bg-danger-soft' : rowFilter ? 'border-accent-line' : 'border-line'
+                rowFilterError ? 'border-red-400 bg-red-50' : rowFilter ? 'border-blue-300' : 'border-gray-200'
               }`}
               placeholder="filter regex…"
               value={rowFilter}
               onChange={e => setRowFilter(e.target.value)}
             />
             {rowFilter && !rowFilterError && (
-              <span className="absolute right-2 top-1/2 -translate-y-1/2 text-xs text-muted">
+              <span className="absolute right-2 top-1/2 -translate-y-1/2 text-xs text-gray-400">
                 {filteredRows.length}
               </span>
             )}
@@ -435,12 +427,12 @@ export default function Mappings({ source, onNeedsReprocess }) {
                   alert(err.message)
                 }
               }}
-              className="text-sm px-3 py-1.5 border border-line rounded hover:bg-raised text-ink-soft"
+              className="text-sm px-3 py-1.5 border border-gray-200 rounded hover:bg-gray-50 text-gray-600"
             >
               Export TSV
             </button>
           )}
-          <label className={`text-sm px-3 py-1.5 border border-line rounded cursor-pointer hover:bg-raised text-ink-soft ${importing ? 'opacity-50 pointer-events-none' : ''}`}>
+          <label className={`text-sm px-3 py-1.5 border border-gray-200 rounded cursor-pointer hover:bg-gray-50 text-gray-600 ${importing ? 'opacity-50 pointer-events-none' : ''}`}>
             {importing ? 'Importing…' : 'Import TSV'}
             <input type="file" accept=".tsv,.txt" className="hidden" onChange={handleImportCSV} />
           </label>
@@ -450,24 +442,24 @@ export default function Mappings({ source, onNeedsReprocess }) {
       {/* Content */}
       <div className="p-6">
         {!selectedRule && (
-          <p className="text-sm text-muted">Select a rule to view mappings.</p>
+          <p className="text-sm text-gray-400">Select a rule to view mappings.</p>
         )}
         {selectedRule && loading && (
-          <p className="text-sm text-muted">Loading…</p>
+          <p className="text-sm text-gray-400">Loading…</p>
         )}
         {selectedRule && !loading && allValues.length === 0 && (
-          <p className="text-sm text-muted">No extracted values for this rule. Run a transform first.</p>
+          <p className="text-sm text-gray-400">No extracted values for this rule. Run a transform first.</p>
         )}
         {selectedRule && !loading && allValues.length > 0 && (
           <div className="overflow-x-auto">
             {/* Bulk assign bar */}
             {selected.size > 0 && (
-              <div className="flex items-center gap-2 mb-2 p-2 bg-accent-soft border border-accent-line rounded flex-wrap">
+              <div className="flex items-center gap-2 mb-2 p-2 bg-blue-50 border border-blue-200 rounded flex-wrap">
-                <span className="text-xs text-accent font-medium whitespace-nowrap">{selected.size} selected</span>
+                <span className="text-xs text-blue-700 font-medium whitespace-nowrap">{selected.size} selected</span>
                 {cols.map(col => (
                   <AutocompleteInput
                     key={col}
-                    className="border border-accent-line rounded px-2 py-1 text-xs min-w-24 focus:outline-none focus:border-accent bg-surface"
+                    className="border border-blue-300 rounded px-2 py-1 text-xs min-w-24 focus:outline-none focus:border-blue-500 bg-white"
                     placeholder={col}
                     value={bulkDraft[col] || ''}
                     onChange={v => setBulkDraft(d => ({ ...d, [col]: v }))}
@@ -483,15 +475,15 @@ export default function Mappings({ source, onNeedsReprocess }) {
                 </button>
                 <button
                   onClick={() => { setSelected(new Set()); setBulkDraft({}) }}
-                  className="text-xs text-accent hover:text-accent"
+                  className="text-xs text-blue-400 hover:text-blue-600"
                 >
                   cancel
                 </button>
               </div>
             )}
-            <table className="w-full text-xs bg-surface border border-line rounded">
+            <table className="w-full text-xs bg-white border border-gray-200 rounded">
               <thead>
-                <tr className="text-left text-muted border-b border-line-soft bg-raised">
+                <tr className="text-left text-gray-400 border-b border-gray-100 bg-gray-50">
                   <th className="px-2 py-2 w-6">
                     <input
                       type="checkbox"
@@ -511,7 +503,7 @@ export default function Mappings({ source, onNeedsReprocess }) {
                   {extraCols.map((col, idx) => (
                     <th key={`extra-${idx}`} className="px-3 py-2 font-medium">
                       <input
-                        className="border border-line rounded px-1 py-0.5 w-24 focus:outline-none focus:border-accent font-normal"
+                        className="border border-gray-200 rounded px-1 py-0.5 w-24 focus:outline-none focus:border-blue-400 font-normal"
                         value={col}
                         placeholder="new key"
                         onChange={e => setExtraCols(ec => { const c = [...ec]; c[idx] = e.target.value; return c })}
@@ -521,7 +513,7 @@ export default function Mappings({ source, onNeedsReprocess }) {
                   <th className="px-2 py-2">
                     <button
                       onClick={() => setExtraCols(ec => [...ec, ''])}
-                      className="text-muted hover:text-ink-soft font-medium"
+                      className="text-gray-400 hover:text-gray-700 font-medium"
                       title="Add column"
                     >+</button>
                   </th>
@@ -536,7 +528,7 @@ export default function Mappings({ source, onNeedsReprocess }) {
                   const isSaving = saving[k]
                   const isSelected = selected.has(k)
                   const hasDraft = !!(drafts[k] && Object.keys(drafts[k]).length > 0)
-                  const rowBg = isSelected ? 'bg-accent-soft' : hasDraft ? 'bg-accent-soft' : row.is_mapped ? '' : 'bg-warn-soft'
+                  const rowBg = isSelected ? 'bg-blue-50' : hasDraft ? 'bg-blue-50' : row.is_mapped ? '' : 'bg-yellow-50'
 
                   function handleRowClick(e) {
                     if (e.target.closest('input,button,a,select')) return
@@ -571,7 +563,7 @@ export default function Mappings({ source, onNeedsReprocess }) {
                         key={k}
                         ref={el => rowRefs.current[k] = el}
                         tabIndex={0}
-                        className={`border-t border-line-soft hover:bg-raised cursor-pointer outline-none ${rowBg}`}
+                        className={`border-t border-gray-50 hover:bg-gray-50 cursor-pointer outline-none ${rowBg}`}
                         onClick={handleRowClick}
                         onKeyDown={handleRowKeyDown}
                       >
@@ -586,13 +578,13 @@ export default function Mappings({ source, onNeedsReprocess }) {
                             }}
                           />
                         </td>
-                        <td className="px-3 py-1.5 font-mono text-ink whitespace-nowrap">{displayValue(row.extracted_value)}</td>
+                        <td className="px-3 py-1.5 font-mono text-gray-800 whitespace-nowrap">{displayValue(row.extracted_value)}</td>
-                        <td className="px-3 py-1.5 text-right text-muted">{row.record_count}</td>
+                        <td className="px-3 py-1.5 text-right text-gray-400">{row.record_count}</td>
                         {cols.map(col => (
                           <td key={col} className="px-3 py-1.5">
                             <AutocompleteInput
-                              className={`border rounded px-2 py-1 w-full min-w-24 focus:outline-none focus:border-accent ${
+                              className={`border rounded px-2 py-1 w-full min-w-24 focus:outline-none focus:border-blue-400 ${
-                                hasDraft ? 'border-accent-line' : row.is_mapped ? 'border-line' : 'border-warn-line'
+                                hasDraft ? 'border-blue-300' : row.is_mapped ? 'border-gray-200' : 'border-yellow-300'
                               }`}
                               value={cellVal(col)}
                               onChange={v => setCellValue(row.extracted_value, col, v)}
@@ -605,7 +597,7 @@ export default function Mappings({ source, onNeedsReprocess }) {
                         <td className="px-3 py-1.5 whitespace-nowrap">
                           {samples.length > 0 && (
                             <button
-                              className="text-accent hover:text-accent"
+                              className="text-blue-400 hover:text-blue-600"
                               onClick={() => setSampleOpen(s => ({ ...s, [k]: !s[k] }))}
                             >
                               {sampleOpen[k] ? 'hide' : 'show'}
@@ -624,7 +616,7 @@ export default function Mappings({ source, onNeedsReprocess }) {
                             {row.is_mapped && (
                               <button
                                 onClick={() => deleteRow(row)}
-                                className="text-danger hover:text-danger text-base leading-none"
+                                className="text-red-400 hover:text-red-600 text-base leading-none"
                                 title="Remove mapping"
                               >×</button>
                             )}
@@ -634,21 +626,21 @@ export default function Mappings({ source, onNeedsReprocess }) {
                       {sampleOpen[k] && (() => {
                         const sampleCols = [...new Set(samples.flatMap(r => Object.keys(r)))]
                         return (
-                          <tr key={`${k}-sample`} className="border-t border-line-soft bg-raised">
+                          <tr key={`${k}-sample`} className="border-t border-gray-50 bg-gray-50">
                             <td colSpan={3 + cols.length + 4} className="px-3 py-2">
-                              <table className="w-full text-xs border border-line-soft rounded bg-surface">
+                              <table className="w-full text-xs border border-gray-100 rounded bg-white">
                                 <thead>
-                                  <tr className="bg-raised border-b border-line-soft">
+                                  <tr className="bg-gray-50 border-b border-gray-100">
                                     {sampleCols.map(c => (
-                                      <th key={c} className="px-2 py-1 text-left font-medium text-muted whitespace-nowrap">{c}</th>
+                                      <th key={c} className="px-2 py-1 text-left font-medium text-gray-400 whitespace-nowrap">{c}</th>
                                     ))}
                                   </tr>
                                 </thead>
                                 <tbody>
                                   {samples.map((rec, i) => (
-                                    <tr key={i} className="border-t border-line-soft">
+                                    <tr key={i} className="border-t border-gray-50">
                                       {sampleCols.map(c => (
-                                        <td key={c} className="px-2 py-1 font-mono text-ink-soft whitespace-nowrap">
+                                        <td key={c} className="px-2 py-1 font-mono text-gray-600 whitespace-nowrap">
                                           {rec[c] != null ? String(rec[c]) : ''}
                                         </td>
                                       ))}

ui/src/pages/Pivot.jsx

@@ -1,19 +1,33 @@
 import { useEffect, useRef, useState, useCallback } from 'react'
 import { api } from '../api'
-import useTheme from '../theme.jsx'
-import perspective from '@perspective-dev/client/inline'
-import '@perspective-dev/viewer/inline'
-import '@perspective-dev/viewer-datagrid'
-import '@perspective-dev/viewer-d3fc'
-import '@perspective-dev/viewer/themes'
 
 async function fetchAllRows(source) {
   const res = await api.getViewData(source, 100000, 0)
   return res.rows || []
 }
 
+let perspectivePromise = null
+
 function loadPerspective() {
-  return Promise.resolve(perspective)
+  if (perspectivePromise) return perspectivePromise
+  perspectivePromise = (async () => {
+    if (!document.getElementById('psp-theme')) {
+      const link = document.createElement('link')
+      link.id = 'psp-theme'
+      link.rel = 'stylesheet'
+      link.crossOrigin = 'anonymous'
+      link.href = 'https://cdn.jsdelivr.net/npm/@perspective-dev/viewer/dist/css/themes.css'
+      document.head.appendChild(link)
+    }
+    const [{ default: perspective }] = await Promise.all([
+      import(/* @vite-ignore */ 'https://cdn.jsdelivr.net/npm/@perspective-dev/client@4.4.0/dist/cdn/perspective.js'),
+      import(/* @vite-ignore */ 'https://cdn.jsdelivr.net/npm/@perspective-dev/viewer@4.4.0/dist/cdn/perspective-viewer.js'),
+      import(/* @vite-ignore */ 'https://cdn.jsdelivr.net/npm/@perspective-dev/viewer-datagrid@4.4.0/dist/cdn/perspective-viewer-datagrid.js'),
+      import(/* @vite-ignore */ 'https://cdn.jsdelivr.net/npm/@perspective-dev/viewer-d3fc@4.4.0/dist/cdn/perspective-viewer-d3fc.js'),
+    ])
+    return perspective
+  })()
+  return perspectivePromise
 }
 
 function formatVal(v, decimals = 2) {
@@ -66,32 +80,19 @@ const LAYOUT_KEY = (source) => `psp_layout_${source}`
 const DEFAULT_PLUGIN_CONFIG = { edit_mode: 'SELECT_REGION' }
 
 
-export default function Pivot({ source, selectedStack, setSelectedStack }) {
+export default function Pivot({ source }) {
-  const { dark } = useTheme()
   const viewerRef = useRef()
   const workerRef = useRef()
   const tableRef = useRef()
   const allRowsRef = useRef([])
   const expandDepthRef = useRef(null)
-  const lastClickKeyRef = useRef(null)
-  const perspClickHandlerRef = useRef(null)
   const [status, setStatus] = useState('idle')
   const [error, setError] = useState('')
   const [inspectedRows, setInspectedRows] = useState(null)
   const [clickDetail, setClickDetail] = useState(null)
   const [decimals, setDecimals] = useState(2)
-  const [paneWidth, setPaneWidth] = useState(384)
-  const [sortCol, setSortCol] = useState(null)
-  const [sortDir, setSortDir] = useState('asc')
 
-  const selectedView = selectedStack ?? source
+  // Named layouts
-  const viewType = selectedStack ? 'stack' : 'source'
-
-  useEffect(() => {
-    if (viewerRef.current) viewerRef.current.setAttribute('theme', dark ? 'Pro Dark' : 'Pro Light')
-  }, [dark])
-
-  // Named layouts — stacks use localStorage only (no server FK to sources)
   const [layouts, setLayouts] = useState([])
   const [activeLayoutId, setActiveLayoutId] = useState(null)
   const [saveAsName, setSaveAsName] = useState('')
@@ -104,21 +105,18 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
   }
 
   const loadLayouts = useCallback(async () => {
-    if (!selectedView) return
+    if (!source) return
     try {
-      const rows = viewType === 'source'
+      const rows = await api.getPivotLayouts(source)
-        ? await api.getPivotLayouts(selectedView)
-        : await api.getStackPivotLayouts(selectedView)
       setLayouts(rows)
     } catch {}
-  }, [selectedView])
+  }, [source])
 
   useEffect(() => {
-    if (!selectedView) return
+    if (!source) return
     let cancelled = false
     setInspectedRows(null)
     setClickDetail(null)
-    lastClickKeyRef.current = null
     setActiveLayoutId(null)
     setShowSaveAs(false)
     allRowsRef.current = []
@@ -131,7 +129,7 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
       try {
         const [perspective, rows] = await Promise.all([
           loadPerspective(),
-          fetchAllRows(selectedView),
+          fetchAllRows(source),
         ])
         if (cancelled) return
         if (!rows.length) { setStatus('noview'); return }
@@ -144,27 +142,13 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
         if (cancelled) { worker.terminate(); return }
         workerRef.current = worker
 
-        const table = await worker.table(rows, { name: selectedView })
+        const table = await worker.table(rows, { name: source })
         if (cancelled) return
         tableRef.current = table
 
         const viewer = viewerRef.current
-        const validCols = new Set(Object.keys(rows[0] || {}))
 
-        function cleanLayout(cfg) {
+        viewer.addEventListener('perspective-click', async (e) => {
-          if (!cfg) return cfg
-          const clean = { ...cfg }
-          const exprNames = new Set(Object.keys(clean.expressions || {}))
-          const valid = (c) => validCols.has(c) || exprNames.has(c)
-          if (clean.columns) clean.columns = clean.columns.filter(c => c == null || valid(c))
-          if (clean.group_by) clean.group_by = clean.group_by.filter(valid)
-          if (clean.split_by) clean.split_by = clean.split_by.filter(valid)
-          if (clean.sort) clean.sort = clean.sort.filter(([c]) => valid(c))
-          if (clean.filter) clean.filter = clean.filter.filter(([c]) => valid(c))
-          return clean
-        }
-
-        perspClickHandlerRef.current = async (e) => {
           const detail = e.detail || {}
           const { row, column_names } = detail
           if (!row) return
@@ -176,39 +160,14 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
           const hasHierarchy = (config.group_by || []).length > 0
           if (!hasHierarchy) return
 
-          // column_names encodes the full column path: [split_val_1, ..., split_val_N, measure]
+          setClickDetail({ row, config, column_names, eventFilters })
-          // positionally matching config.split_by. Perspective may omit split_by coordinate
-          // filters from detail.config.filter, so derive any missing ones from column_names.
-          const splitByFields = config.split_by || []
-          const coveredByEvent = new Set(eventFilters.filter(([, op]) => op === '==').map(([f]) => f))
-          const derivedSplitFilters = splitByFields
-            .map((field, i) => {
-              if (coveredByEvent.has(field)) return null
-              const val = Array.isArray(column_names) && column_names[i] != null
-                ? String(column_names[i]) : null
-              return val != null ? [field, '==', val] : null
-            })
-            .filter(Boolean)
-          const allFilters = [...eventFilters, ...derivedSplitFilters]
-
-          // Same cell clicked again — toggle the pane closed.
-          // Key on row path + column names (from the raw event) rather than derived
-          // filters, which can vary between clicks on stack/expression views.
-          const clickKey = JSON.stringify({ p: row['__ROW_PATH__'], c: column_names })
-          if (lastClickKeyRef.current === clickKey) {
-            lastClickKeyRef.current = null
-            setInspectedRows(null)
-            setClickDetail(null)
-            return
-          }
-          lastClickKeyRef.current = clickKey
-
-          setClickDetail({ row, config, column_names, eventFilters: allFilters })
 
+          // Use a Perspective view with the event filters + expressions so computed
+          // columns (split_by) are evaluated and filtered correctly
           try {
             const view = await tableRef.current.view({
-              filter: allFilters,
+              filter: eventFilters,
-              expressions: config.expressions || {},
+              expressions: config.expressions || [],
             })
             const data = await view.to_json()
             await view.delete()
@@ -218,28 +177,25 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
               Object.fromEntries(Object.entries(r).filter(([k]) => !exprNames.has(k)))
             )
             setInspectedRows(cleaned)
-          } catch (err) {
+          } catch {
-            console.warn('Perspective inspector view failed, falling back to JS filter:', err)
+            setInspectedRows(filterRowsByConfig(allRowsRef.current, eventFilters))
-            setInspectedRows(filterRowsByConfig(allRowsRef.current, allFilters))
           }
-        }
+        })
-        viewer.addEventListener('perspective-click', perspClickHandlerRef.current)
 
         await viewer.load(worker)
 
         const plugin = await viewer.getPlugin()
-        const savedLayout = localStorage.getItem(LAYOUT_KEY(selectedView))
+        const savedLayout = localStorage.getItem(LAYOUT_KEY(source))
         if (savedLayout) {
-          const parsed = cleanLayout(JSON.parse(savedLayout))
+          const parsed = JSON.parse(savedLayout)
           await viewer.restore(parsed)
           await plugin.restore(parsed.plugin_config || DEFAULT_PLUGIN_CONFIG)
           if (parsed.expand_depth != null) await applyExpandDepth(viewer, parsed.expand_depth)
         } else {
-          await viewer.restore({ table: selectedView, settings: false, plugin_config: DEFAULT_PLUGIN_CONFIG })
+          await viewer.restore({ table: source, settings: false, plugin_config: DEFAULT_PLUGIN_CONFIG })
           await plugin.restore(DEFAULT_PLUGIN_CONFIG)
         }
         await viewer.flush()
-        viewer.setAttribute('theme', dark ? 'Pro Dark' : 'Pro Light')
 
         setStatus('ready')
       } catch (err) {
@@ -248,14 +204,8 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
     }
 
     init()
-    return () => {
+    return () => { cancelled = true }
-      cancelled = true
+  }, [source])
-      if (perspClickHandlerRef.current && viewerRef.current) {
-        viewerRef.current.removeEventListener('perspective-click', perspClickHandlerRef.current)
-        perspClickHandlerRef.current = null
-      }
-    }
-  }, [selectedView])
 
   async function applyExpandDepth(viewer, depth) {
     if (depth == null) return
@@ -269,35 +219,15 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
   async function applyLayout(layout) {
     const viewer = viewerRef.current
     if (!viewer) return
-    try {
+    await viewer.restore(layout.config)
-      const validCols = new Set(Object.keys(allRowsRef.current[0] || {}))
+    if (layout.config.plugin_config) {
-      function cleanLayout(cfg) {
+      const plugin = await viewer.getPlugin()
-        if (!cfg) return cfg
+      await plugin.restore(layout.config.plugin_config)
-        const clean = { ...cfg }
-        const exprNames = new Set(Object.keys(clean.expressions || {}))
-        const valid = (c) => validCols.has(c) || exprNames.has(c)
-        if (clean.columns) clean.columns = clean.columns.filter(c => c == null || valid(c))
-        if (clean.group_by) clean.group_by = clean.group_by.filter(valid)
-        if (clean.split_by) clean.split_by = clean.split_by.filter(valid)
-        if (clean.sort) clean.sort = clean.sort.filter(([c]) => valid(c))
-        if (clean.filter) clean.filter = clean.filter.filter(([c]) => valid(c))
-        return clean
-      }
-      const cleaned = cleanLayout(layout.config)
-      await viewer.restore(cleaned)
-      if (cleaned.plugin_config) {
-        const plugin = await viewer.getPlugin()
-        await plugin.restore(cleaned.plugin_config)
-      }
-      await applyExpandDepth(viewer, cleaned.expand_depth ?? null)
-      setActiveLayoutId(layout.id)
-      localStorage.setItem(LAYOUT_KEY(selectedView), JSON.stringify(cleaned))
-    } catch {
-      // Layout references columns that no longer exist — remove it
-      localStorage.removeItem(LAYOUT_KEY(selectedView))
-      setActiveLayoutId(null)
-      await viewer.restore({ table: selectedView, settings: false })
     }
+    await applyExpandDepth(viewer, layout.config.expand_depth ?? null)
+    setActiveLayoutId(layout.id)
+    // also persist to localStorage so it survives refresh
+    localStorage.setItem(LAYOUT_KEY(source), JSON.stringify(layout.config))
   }
 
   async function captureConfig() {
@@ -308,24 +238,16 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
     return { ...viewerConfig, plugin_config: pluginConfig, expand_depth: expandDepthRef.current }
   }
 
-  const saveLayout = (name, config) => viewType === 'source'
-    ? api.savePivotLayout(selectedView, name, config)
-    : api.saveStackPivotLayout(selectedView, name, config)
-
-  const deleteLayout = (id) => viewType === 'source'
-    ? api.deletePivotLayout(selectedView, id)
-    : api.deleteStackPivotLayout(selectedView, id)
-
   async function handleSaveOver() {
     const layout = layouts.find(l => l.id === activeLayoutId)
     if (!layout) return
     const config = await captureConfig()
     if (!config) return
     try {
-      const saved = await saveLayout(layout.layout_name, config)
+      const saved = await api.savePivotLayout(source, layout.layout_name, config)
-      setActiveLayoutId(saved.id)
+      localStorage.setItem(LAYOUT_KEY(source), JSON.stringify(config))
-      localStorage.setItem(LAYOUT_KEY(selectedView), JSON.stringify(config))
       await loadLayouts()
+      setActiveLayoutId(saved.id)
       flashMsg('Saved!')
     } catch (err) {
       flashMsg(err.message)
@@ -338,8 +260,8 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
     const config = await captureConfig()
     if (!config) return
     try {
-      const saved = await saveLayout(name, config)
+      const saved = await api.savePivotLayout(source, name, config)
-      localStorage.setItem(LAYOUT_KEY(selectedView), JSON.stringify(config))
+      localStorage.setItem(LAYOUT_KEY(source), JSON.stringify(config))
       await loadLayouts()
       setActiveLayoutId(saved.id)
       setShowSaveAs(false)
@@ -353,7 +275,7 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
   async function handleDelete(layout, e) {
     e.stopPropagation()
     try {
-      await deleteLayout(layout.id)
+      await api.deletePivotLayout(source, layout.id)
       if (activeLayoutId === layout.id) setActiveLayoutId(null)
       await loadLayouts()
       flashMsg('Deleted')
@@ -365,33 +287,15 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
   function handleResetToDefault() {
     const viewer = viewerRef.current
     if (!viewer) return
-    localStorage.removeItem(LAYOUT_KEY(selectedView))
+    localStorage.removeItem(LAYOUT_KEY(source))
     setActiveLayoutId(null)
-    viewer.restore({ table: selectedView, settings: true, plugin_config: DEFAULT_PLUGIN_CONFIG })
+    viewer.restore({ table: source, settings: true, plugin_config: DEFAULT_PLUGIN_CONFIG })
   }
 
-  if (!source) return <div className="p-4 sm:p-6 text-sm text-muted">Select a source first.</div>
+  if (!source) return <div className="p-6 text-sm text-gray-400">Select a source first.</div>
 
   const cols = inspectedRows?.length ? Object.keys(inspectedRows[0]) : []
 
-  const sortedRows = sortCol == null || !inspectedRows ? inspectedRows : [...inspectedRows].sort((a, b) => {
-    const av = a[sortCol], bv = b[sortCol]
-    if (av == null && bv == null) return 0
-    if (av == null) return 1
-    if (bv == null) return -1
-    const num = typeof av === 'number' && typeof bv === 'number'
-    const cmp = num ? av - bv : String(av).localeCompare(String(bv))
-    return sortDir === 'asc' ? cmp : -cmp
-  })
-
-  const totals = cols.reduce((acc, c) => {
-    const vals = (inspectedRows || []).map(r => r[c])
-    if (vals.length > 0 && vals.every(v => v == null || typeof v === 'number')) {
-      acc[c] = vals.reduce((s, v) => s + (v ?? 0), 0)
-    }
-    return acc
-  }, {})
-
   const groupBy = clickDetail?.config?.group_by || []
   const splitBy = clickDetail?.config?.split_by || []
   const coordFields = new Set([...groupBy, ...splitBy])
@@ -401,39 +305,36 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
       .map(([f, , v]) => [f, v])
   )
   const cellCoords = [...groupBy, ...splitBy].map(f => coordMap[f]).filter(Boolean)
-  // column_names = [split_val_1, ..., split_val_N, measure_name] — use positional split_by length
+  const splitVals = splitBy.map(f => coordMap[f]).filter(Boolean)
-  // to separate split values from measure names; fall back to coordMap when ambiguous
+  const metrics = clickDetail?.column_names || []
-  const colNames = clickDetail?.column_names || []
+  const cellKey = splitVals.length > 0 && metrics.length > 0
-  const splitVals = splitBy.map((f, i) =>
-    coordMap[f] ?? (colNames[i] != null ? String(colNames[i]) : null)
-  ).filter(Boolean)
-  const metrics = splitBy.length > 0 ? colNames.slice(splitBy.length) : colNames
-  const cellKey = metrics.length > 0
     ? [...splitVals, ...metrics].join('|')
     : null
 
   return (
     <div className="w-full h-full flex flex-col">
 
-      {/* Layouts sub-bar */}
+      {/* Layout toolbar */}
-      <div className="flex items-center gap-2 px-3 h-9 bg-surface border-b border-line shrink-0 text-xs">
+      <div className="flex items-center gap-2 px-3 py-1.5 bg-white border-b border-gray-200 flex-shrink-0">
+        <span className="text-xs text-gray-400 uppercase tracking-wide mr-1">Layouts</span>
+
         {layouts.map(l => (
           <div key={l.id}
             onClick={() => applyLayout(l)}
-            className={`flex items-center gap-1 rounded px-2 py-0.5 cursor-pointer border transition-colors
+            className={`flex items-center gap-1 text-xs rounded px-2 py-0.5 cursor-pointer border transition-colors
               ${activeLayoutId === l.id
-                ? 'bg-accent-soft border-accent-line text-accent'
+                ? 'bg-blue-50 border-blue-300 text-blue-700'
-                : 'bg-surface border-line text-ink-soft hover:border-line'}`}>
+                : 'bg-white border-gray-200 text-gray-600 hover:border-gray-400'}`}>
             {l.layout_name}
             <button
               onClick={(e) => handleDelete(l, e)}
-              className="text-muted hover:text-danger leading-none ml-0.5 text-sm">×</button>
+              className="text-gray-300 hover:text-red-400 leading-none ml-0.5 text-sm">×</button>
           </div>
         ))}
 
         {activeLayoutId !== null && !showSaveAs && (
           <button onClick={handleSaveOver}
-            className="text-accent hover:text-accent border border-accent-line rounded px-2 py-0.5">
+            className="text-xs text-blue-500 hover:text-blue-700 border border-blue-200 rounded px-2 py-0.5">
             Save
           </button>
         )}
@@ -446,27 +347,30 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
               onChange={e => setSaveAsName(e.target.value)}
               onKeyDown={e => { if (e.key === 'Enter') handleSaveAs(); if (e.key === 'Escape') { setShowSaveAs(false); setSaveAsName('') } }}
               placeholder="Layout name…"
-              className="border border-line rounded px-2 py-0.5 w-36 focus:outline-none focus:border-accent"
+              className="text-xs border border-gray-300 rounded px-2 py-0.5 w-36 focus:outline-none focus:border-blue-400"
             />
-            <button onClick={handleSaveAs} className="text-accent hover:text-accent px-1">Save</button>
+            <button onClick={handleSaveAs} className="text-xs text-blue-600 hover:text-blue-800 px-1">Save</button>
-            <button onClick={() => { setShowSaveAs(false); setSaveAsName('') }} className="text-muted hover:text-ink-soft px-1">Cancel</button>
+            <button onClick={() => { setShowSaveAs(false); setSaveAsName('') }} className="text-xs text-gray-400 hover:text-gray-600 px-1">Cancel</button>
           </div>
         ) : (
           <button
             onClick={() => setShowSaveAs(true)}
-            className="text-muted hover:text-ink-soft border border-dashed border-line rounded px-2 py-0.5">
+            className="text-xs text-gray-400 hover:text-gray-600 border border-dashed border-gray-200 rounded px-2 py-0.5">
             + Save as…
           </button>
         )}
 
         {activeLayoutId !== null && (
-          <button onClick={handleResetToDefault} className="text-muted hover:text-muted ml-1">reset</button>
+          <button onClick={handleResetToDefault}
+            className="text-xs text-gray-300 hover:text-gray-500 ml-1">
+            reset
+          </button>
         )}
 
-        {layoutMsg && <span className="text-ok ml-1">{layoutMsg}</span>}
+        {layoutMsg && <span className="text-xs text-green-600 ml-1">{layoutMsg}</span>}
 
         <div className="ml-auto flex items-center gap-1">
-          <span className="text-muted">depth:</span>
+          <span className="text-xs text-gray-400">depth:</span>
           {[0, 1, 2, 3].map(d => (
             <button key={d} onClick={async () => {
               const v = viewerRef.current; if (!v) return
@@ -475,7 +379,7 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
               const p = await v.getPlugin()
               await p.draw(view)
               expandDepthRef.current = d
-            }} className="border border-line rounded px-1.5 py-0.5 text-muted hover:border-line">
+            }} className="text-xs border border-gray-200 rounded px-1.5 py-0.5 text-gray-500 hover:border-gray-400">
               {d}
             </button>
           ))}
@@ -486,18 +390,18 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
       <div className="relative flex-1 flex min-h-0">
         <div className="relative flex-1">
           {status === 'loading' && (
-            <div className="absolute inset-0 flex items-center justify-center z-10 bg-raised">
+            <div className="absolute inset-0 flex items-center justify-center z-10 bg-gray-50">
-              <p className="text-sm text-muted">Loading…</p>
+              <p className="text-sm text-gray-400">Loading…</p>
             </div>
           )}
           {status === 'error' && (
-            <div className="absolute inset-0 flex items-center justify-center z-10 bg-raised">
+            <div className="absolute inset-0 flex items-center justify-center z-10 bg-gray-50">
-              <p className="text-sm text-danger">Error: {error}</p>
+              <p className="text-sm text-red-500">Error: {error}</p>
             </div>
           )}
           {status === 'noview' && (
-            <div className="absolute inset-0 flex items-center justify-center z-10 bg-raised">
+            <div className="absolute inset-0 flex items-center justify-center z-10 bg-gray-50">
-              <p className="text-sm text-muted">No view data — generate a view and transform records first.</p>
+              <p className="text-sm text-gray-400">No view data — generate a view and transform records first.</p>
             </div>
           )}
           <perspective-viewer
@@ -507,61 +411,56 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
         </div>
 
         {inspectedRows && clickDetail && (
-          <div
+          <div className="w-96 border-l border-gray-200 bg-white flex flex-col overflow-hidden flex-shrink-0">
-            style={{ width: paneWidth }}
+            <div className="flex items-center justify-between px-3 py-2 border-b border-gray-100">
-            className="relative border-l border-line bg-surface flex flex-col overflow-hidden flex-shrink-0"
+              <span className="text-xs font-semibold text-gray-600 uppercase tracking-wide">
-          >
+                {inspectedRows.length} row{inspectedRows.length !== 1 ? 's' : ''}
-            {/* Drag-to-resize handle on left edge */}
+              </span>
-            <div
+              <div className="flex items-center gap-2">
-              className="absolute left-0 top-0 bottom-0 w-1 cursor-col-resize hover:bg-blue-300 z-10"
-              onMouseDown={(e) => {
-                e.preventDefault()
-                const startX = e.clientX
-                const startW = paneWidth
-                const onMove = (me) => setPaneWidth(Math.max(240, startW + startX - me.clientX))
-                const onUp = () => {
-                  document.removeEventListener('mousemove', onMove)
-                  document.removeEventListener('mouseup', onUp)
-                }
-                document.addEventListener('mousemove', onMove)
-                document.addEventListener('mouseup', onUp)
-              }}
-            />
-
-            {/* Header: breadcrumb + row count + controls */}
-            <div className="flex items-center justify-between pl-3 pr-2 py-2 border-b border-line-soft flex-shrink-0">
-              <div className="flex items-center gap-2 min-w-0">
-                {cellCoords.length > 0 && (
-                  <span className="text-xs text-ink-soft font-mono font-semibold truncate">
-                    {cellCoords.join(' › ')}
-                  </span>
-                )}
-                <span className="text-xs text-muted flex-shrink-0">
-                  {inspectedRows.length} row{inspectedRows.length !== 1 ? 's' : ''}
-                </span>
-              </div>
-              <div className="flex items-center gap-2 flex-shrink-0">
                 <div className="flex items-center gap-0.5">
                   <button onClick={() => setDecimals(d => Math.max(0, d - 1))}
-                    className="text-xs text-muted hover:text-ink-soft w-4 text-center">−</button>
+                    className="text-xs text-gray-400 hover:text-gray-600 w-4 text-center">−</button>
-                  <span className="text-xs text-muted w-4 text-center">{decimals}</span>
+                  <span className="text-xs text-gray-400 w-4 text-center">{decimals}</span>
                   <button onClick={() => setDecimals(d => Math.min(8, d + 1))}
-                    className="text-xs text-muted hover:text-ink-soft w-4 text-center">+</button>
+                    className="text-xs text-gray-400 hover:text-gray-600 w-4 text-center">+</button>
                 </div>
-                <button onClick={() => { setInspectedRows(null); setClickDetail(null); lastClickKeyRef.current = null }}
+                <button onClick={() => { setInspectedRows(null); setClickDetail(null) }}
-                  className="text-muted hover:text-muted leading-none text-lg">×</button>
+                  className="text-gray-300 hover:text-gray-500 leading-none text-lg">×</button>
               </div>
             </div>
-
             <div className="flex-1 overflow-y-auto">
-              {/* User-set filters (only shown when active) */}
+
+              {/* Cell coordinates */}
+              <div className="px-3 py-2 border-b border-gray-100">
+                <div className="text-xs text-gray-400 uppercase tracking-wide mb-1">
+                  {[...groupBy, ...splitBy].join(' › ') || clickDetail.column_names?.join(', ') || 'Cell'}
+                </div>
+                {cellCoords.length > 0 && (
+                  <div className="text-xs text-gray-700 font-mono font-semibold">
+                    {cellCoords.join(' › ')}
+                  </div>
+                )}
+                {Object.entries(clickDetail.row)
+                  .filter(([k, v]) => k !== '__ROW_PATH__' && v != null)
+                  .map(([k, v]) => {
+                    const isSelected = cellKey != null && k === cellKey
+                    return (
+                      <div key={k} className={`flex justify-between py-0.5 gap-2 ${isSelected ? 'font-semibold' : ''}`}>
+                        <span className={`text-xs font-mono shrink-0 ${isSelected ? 'text-gray-700' : 'text-gray-400'}`}>{k}</span>
+                        <span className={`text-xs font-mono text-right ${isSelected ? 'text-blue-600' : 'text-gray-700'}`}>{formatVal(v, decimals)}</span>
+                      </div>
+                    )
+                  })}
+              </div>
+
+              {/* User-set filters */}
               {(() => {
                 const userFilters = (clickDetail.eventFilters || []).filter(([f]) => !coordFields.has(f))
                 return userFilters.length > 0 ? (
-                  <div className="px-3 py-2 border-b border-line-soft">
+                  <div className="px-3 py-2 border-b border-gray-100">
-                    <div className="text-xs text-muted uppercase tracking-wide mb-1">Filters</div>
+                    <div className="text-xs text-gray-400 uppercase tracking-wide mb-1">Filters</div>
                     {userFilters.map((f, i) => (
-                      <div key={i} className="text-xs text-muted py-0.5 font-mono">{f.join(' ')}</div>
+                      <div key={i} className="text-xs text-gray-500 py-0.5 font-mono">{f.join(' ')}</div>
                     ))}
                   </div>
                 ) : null
@@ -572,44 +471,26 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
                 <div className="overflow-auto">
                   <table className="w-full text-xs">
                     <thead>
-                      <tr className="text-left text-muted border-b border-line-soft bg-raised sticky top-0">
+                      <tr className="text-left text-gray-400 border-b border-gray-100 bg-gray-50 sticky top-0">
-                        {cols.map(c => {
+                        {cols.map(c => (
-                          const active = sortCol === c
+                          <th key={c} className="px-2 py-1 font-medium whitespace-nowrap">{c}</th>
-                          return (
+                        ))}
-                            <th key={c}
-                              onClick={() => { if (active) setSortDir(d => d === 'asc' ? 'desc' : 'asc'); else { setSortCol(c); setSortDir('asc') } }}
-                              className="px-2 py-1 font-medium whitespace-nowrap cursor-pointer select-none hover:text-ink-soft">
-                              {c}{active ? (sortDir === 'asc' ? ' ▲' : ' ▼') : ''}
-                            </th>
-                          )
-                        })}
                       </tr>
                     </thead>
                     <tbody>
-                      {sortedRows.map((row, i) => (
+                      {inspectedRows.map((row, i) => (
-                        <tr key={i} className="border-t border-line-soft hover:bg-raised">
+                        <tr key={i} className="border-t border-gray-50 hover:bg-gray-50">
                           {cols.map(c => {
                             const f = formatVal(row[c], decimals)
                             return (
-                              <td key={c} className="px-2 py-1 font-mono whitespace-nowrap text-ink-soft max-w-40 truncate">
+                              <td key={c} className="px-2 py-1 font-mono whitespace-nowrap text-gray-700 max-w-40 truncate">
-                                {f == null ? <span className="text-muted">—</span> : f}
+                                {f == null ? <span className="text-gray-300">—</span> : f}
                               </td>
                             )
                           })}
                         </tr>
                       ))}
                     </tbody>
-                    {Object.keys(totals).length > 0 && (
-                      <tfoot>
-                        <tr className="border-t-2 border-line bg-raised font-semibold text-ink-soft sticky bottom-0">
-                          {cols.map(c => (
-                            <td key={c} className="px-2 py-1 font-mono whitespace-nowrap text-right">
-                              {totals[c] != null ? formatVal(totals[c], decimals) : ''}
-                            </td>
-                          ))}
-                        </tr>
-                      </tfoot>
-                    )}
                   </table>
                 </div>
               )}

ui/src/pages/Records.jsx

@@ -1,77 +1,8 @@
 import { useState, useEffect, useRef } from 'react'
 import { api } from '../api'
 
-function AutocompleteInput({ value, onChange, onEnter, suggestions = [], className, placeholder }) {
-  const [open, setOpen] = useState(false)
-  const [highlighted, setHighlighted] = useState(0)
-  const [dropPos, setDropPos] = useState(null)
-  const inputRef = useRef()
-  const listRef = useRef()
-  const filtered = value
-    ? suggestions.filter(s => s.toLowerCase().includes(value.toLowerCase()))
-    : suggestions
-
-  function openList() {
-    if (inputRef.current) {
-      const r = inputRef.current.getBoundingClientRect()
-      setDropPos({ top: r.bottom + 2, left: r.left, minWidth: r.width })
-    }
-    setOpen(true)
-    setHighlighted(0)
-  }
-
-  function select(val) { onChange(val); setOpen(false); inputRef.current?.focus() }
-
-  function handleKeyDown(e) {
-    if (e.altKey && e.key === 'ArrowDown') { e.preventDefault(); openList(); return }
-    if (open && filtered.length > 0) {
-      if (e.key === 'Tab')       { e.preventDefault(); setHighlighted(h => (h + 1) % filtered.length); return }
-      if (e.key === 'ArrowDown') { e.preventDefault(); setHighlighted(h => Math.min(h + 1, filtered.length - 1)); return }
-      if (e.key === 'ArrowUp')   { e.preventDefault(); setHighlighted(h => Math.max(h - 1, 0)); return }
-      if (e.key === 'Enter')     { e.preventDefault(); select(filtered[highlighted]); return }
-      if (e.key === 'Escape')    { setOpen(false); return }
-    }
-    if (e.key === 'Enter') onEnter?.()
-  }
-
-  useEffect(() => {
-    if (!open || !listRef.current) return
-    listRef.current.children[highlighted]?.scrollIntoView({ block: 'nearest' })
-  }, [highlighted, open])
-
-  return (
-    <div className="relative">
-      <input ref={inputRef} className={className} value={value} placeholder={placeholder}
-        onChange={e => { onChange(e.target.value); if (e.target.value) openList() }}
-        onKeyDown={handleKeyDown}
-        onBlur={e => { if (!listRef.current?.contains(e.relatedTarget)) setOpen(false) }}
-      />
-      {open && filtered.length > 0 && dropPos && (
-        <div ref={listRef}
-          style={{ position: 'fixed', top: dropPos.top, left: dropPos.left, minWidth: dropPos.minWidth, zIndex: 9999 }}
-          className="bg-surface border border-line rounded shadow-lg max-h-40 overflow-y-auto">
-          {filtered.map((s, i) => (
-            <div key={s}
-              className={`px-2 py-1 text-xs cursor-pointer whitespace-nowrap ${i === highlighted ? 'bg-accent-soft text-accent' : 'text-ink-soft hover:bg-raised'}`}
-              onMouseDown={e => { e.preventDefault(); select(s) }}>{s}</div>
-          ))}
-        </div>
-      )}
-    </div>
-  )
-}
-
 const DATE_RE = /^\d{4}-\d{2}-\d{2}(T[\d:.Z+-]+)?$/
-// Hidden everywhere a field can be edited as an override — you can't override an id
 const HIDDEN_COLS = new Set(['id', '_overridden'])
-// The grid and its filters keep id: it's the only handle on a specific row
-const GRID_HIDDEN_COLS = new Set(['_overridden'])
-
-// id first, then the rest in view order
-function gridCols(keys) {
-  const visible = keys.filter(c => !GRID_HIDDEN_COLS.has(c))
-  return visible.includes('id') ? ['id', ...visible.filter(c => c !== 'id')] : visible
-}
 
 function formatVal(val) {
   if (val === null || val === undefined) return null
@@ -96,26 +27,18 @@ export default function Records({ source }) {
   const [loading, setLoading] = useState(false)
   const [viewError, setViewError] = useState(null)
   const [sort, setSort] = useState({ col: null, dir: 'asc' })
-  const [filters, setFilters] = useState([])  // DB sort/filter queries
+  const [filters, setFilters] = useState([])
-  const [rowFilter, setRowFilter] = useState('')  // regex filter for selecting rows
+  const debounceRef = useRef(null)
-  const [selected, setSelected] = useState(new Set())  // row IDs selected for bulk override
-  const [bulkDraft, setBulkDraft] = useState({})  // bulk override values
   const LIMIT = 100
 
-  // Override cols — loaded from DB once per source, extended by user via +
-  const [overrideCols, setOverrideCols] = useState([])   // keys seen in overrides across all records
-  const [extraCols, setExtraCols] = useState([])         // new cols added this session via +
-  const [globalValues, setGlobalValues] = useState({})   // picklist suggestions
-
   // Override panel
   const [panelOpen, setPanelOpen] = useState(false)
-  const [selectedRow, setSelectedRow] = useState(null)
+  const [selectedRow, setSelectedRow] = useState(null)          // raw view row (has id)
-  const [selectedRecord, setSelectedRecord] = useState(null)
+  const [selectedRecord, setSelectedRecord] = useState(null)    // full record from API
-  const [overrideDraft, setOverrideDraft] = useState({})
+  const [overrideDraft, setOverrideDraft] = useState({})        // { field: newValue }
   const [panelLoading, setPanelLoading] = useState(false)
   const [panelSaving, setPanelSaving] = useState(false)
   const [panelMsg, setPanelMsg] = useState(null)
-  const debounceRef = useRef(null)
 
   useEffect(() => {
     if (!source) return
@@ -126,38 +49,9 @@ export default function Records({ source }) {
     setSelectedRecord(null)
     setSelectedRow(null)
     setPanelOpen(false)
-    setOverrideCols([])
+    load(0, null, 'asc', [])
-    setExtraCols([])
-    // Default to newest first on the source's first date field, if it has one
-    api.getSource(source)
-      .then(src => (src?.config?.fields || []).find(f => f.type === 'date')?.name || null)
-      .catch(() => null)
-      .then(dateCol => {
-        if (dateCol) setSort({ col: dateCol, dir: 'desc' })
-        load(0, dateCol, 'desc', [])
-      })
-    api.getOverrideKeys(source).then(setOverrideCols).catch(() => {})
-    api.getGlobalValues().then(setGlobalValues).catch(() => {})
-    setSelected(new Set())
-    setBulkDraft({})
-    setRowFilter('')
   }, [source])
 
-  // Auto-select all rows matching the regex filter when it changes
-  useEffect(() => {
-    if (!rowFilter) return
-    let re = null
-    try { re = new RegExp(rowFilter, 'i') } catch { return }
-    const matches = rows.filter(r => {
-      for (const col of displayCols) {
-        const val = r[col]
-        if (val != null && re.test(String(val))) return true
-      }
-      return false
-    })
-    setSelected(new Set(matches.map(r => r.id)))
-  }, [rowFilter, rows])
-
   async function load(off, col, dir, filt) {
     setLoading(true)
     try {
@@ -188,15 +82,13 @@ export default function Records({ source }) {
   }
 
   function addFilter() {
-    // id is filterable but a poor default — start on the first data column
+    const visCols = cols.filter(c => !HIDDEN_COLS.has(c))
-    const visCols = gridCols(cols).filter(c => c !== 'id')
     setFilters(f => [...f, { col: visCols[0] || '', pattern: '' }])
   }
 
   function removeFilter(i) {
     const next = filters.filter((_, idx) => idx !== i)
     setFilters(next)
-    setSelected(new Set())
     setOffset(0)
     load(0, sort.col, sort.dir, next)
   }
@@ -204,15 +96,15 @@ export default function Records({ source }) {
   function updateFilter(i, key, val) {
     const next = filters.map((f, idx) => idx === i ? { ...f, [key]: val } : f)
     setFilters(next)
-    setSelected(new Set())
     setOffset(0)
     triggerLoad(0, sort.col, sort.dir, next)
   }
 
-  function prev() { const o = Math.max(0, offset - LIMIT); setOffset(o); setSelected(new Set()); load(o, sort.col, sort.dir, filters) }
+  function prev() { const o = Math.max(0, offset - LIMIT); setOffset(o); load(o, sort.col, sort.dir, filters) }
-  function next() { const o = offset + LIMIT; setOffset(o); setSelected(new Set()); load(o, sort.col, sort.dir, filters) }
+  function next() { const o = offset + LIMIT; setOffset(o); load(o, sort.col, sort.dir, filters) }
 
   async function openPanel(row) {
+    // Open panel immediately, then load full record async
     setPanelOpen(true)
     setSelectedRow(row)
     setSelectedRecord(null)
@@ -250,15 +142,15 @@ export default function Records({ source }) {
     setPanelSaving(true)
     setPanelMsg(null)
     try {
-      const toSave = { ...overrideDraft }
+      const updated = await api.setRecordOverrides(selectedRecord.id, overrideDraft)
-      const updated = await api.setRecordOverrides(selectedRecord.id, toSave)
       setSelectedRecord(updated)
       setOverrideDraft(updated.overrides || {})
-      // Merge any new cols from extraCols into overrideCols
-      setOverrideCols(prev => [...new Set([...prev, ...extraCols.filter(c => c.trim())])])
-      setExtraCols([])
       setPanelMsg({ text: 'Saved.', ok: true })
-      load(offset, sort.col, sort.dir, filters)
+      // Refresh the row in the table
+      setRows(rs => rs.map(r => r.id === updated.id
+        ? { ...r, _overridden: updated.overrides != null }
+        : r
+      ))
     } catch (err) {
       setPanelMsg({ text: err.message, ok: false })
     } finally {
@@ -274,8 +166,8 @@ export default function Records({ source }) {
       const updated = await api.clearRecordOverrides(selectedRecord.id)
       setSelectedRecord(updated)
       setOverrideDraft({})
-      setPanelMsg({ text: 'Cleared.', ok: true })
+      setPanelMsg({ text: 'Overrides cleared. Transformed values restored.', ok: true })
-      load(offset, sort.col, sort.dir, filters)
+      setRows(rs => rs.map(r => r.id === updated.id ? { ...r, _overridden: false } : r))
     } catch (err) {
       setPanelMsg({ text: err.message, ok: false })
     } finally {
@@ -283,164 +175,104 @@ export default function Records({ source }) {
     }
   }
 
-  if (!source) return <div className="p-4 sm:p-6 text-sm text-muted">Select a source first.</div>
+  if (!source) return <div className="p-6 text-sm text-gray-400">Select a source first.</div>
 
-  const displayCols = gridCols(rows.length > 0 ? Object.keys(rows[0]) : cols)
+  const displayCols = (rows.length > 0 ? Object.keys(rows[0]) : cols).filter(c => !HIDDEN_COLS.has(c))
-  const visCols = gridCols(cols)
+  const visCols = cols.filter(c => !HIDDEN_COLS.has(c))
 
-  // For bulk bar: only established override keys
+  // Fields available for override: keys from transformed
-  const allOverrideCols = [...new Set([...overrideCols, ...extraCols])]
+  const transformedFields = selectedRecord?.transformed
-
+    ? Object.keys(selectedRecord.transformed)
-  const savedOverrides = selectedRecord?.overrides || {}
+    : []
-  const isDirty = Object.values(overrideDraft).some(v => String(v).trim())
-    || extraCols.some(c => c.trim())
-    || Object.keys(savedOverrides).some(k => !String(overrideDraft[k] ?? '').trim())
 
   return (
     <div className="flex h-full min-h-0 overflow-hidden">
       <div className="flex-1 overflow-auto p-6 min-w-0">
         <div className="flex items-center justify-between mb-4">
-          <h1 className="text-xl font-semibold text-ink">Records — {source}</h1>
+          <h1 className="text-xl font-semibold text-gray-800">Records — {source}</h1>
           {exists && rows.length > 0 && (
-            <span className="text-xs text-muted font-mono">dfv.{source}</span>
+            <span className="text-xs text-gray-400 font-mono">dfv.{source}</span>
           )}
         </div>
 
         {/* Filter bar */}
         {exists !== false && visCols.length > 0 && (
           <div className="mb-4 flex flex-wrap gap-2 items-center">
-            <span className="text-xs text-muted font-medium mr-1">DB query:</span>
             {filters.map((f, i) => (
-              <div key={i} className="flex items-center gap-1 bg-surface border border-line rounded px-2 py-1">
+              <div key={i} className="flex items-center gap-1 bg-white border border-gray-200 rounded px-2 py-1">
                 <select
-                  className="text-xs text-ink-soft border-0 focus:outline-none bg-transparent"
+                  className="text-xs text-gray-600 border-0 focus:outline-none bg-transparent"
                   value={f.col}
                   onChange={e => updateFilter(i, 'col', e.target.value)}
                 >
                   {visCols.map(c => <option key={c} value={c}>{c}</option>)}
                 </select>
-                <span className="text-xs text-muted mx-0.5">~*</span>
+                <span className="text-xs text-gray-300 mx-0.5">~*</span>
                 <input
                   className="text-xs font-mono border-0 focus:outline-none w-36 bg-transparent"
                   placeholder="regex…"
                   value={f.pattern}
                   onChange={e => updateFilter(i, 'pattern', e.target.value)}
                 />
-                <button onClick={() => removeFilter(i)} className="text-muted hover:text-muted ml-1 leading-none">×</button>
+                <button
+                  onClick={() => removeFilter(i)}
+                  className="text-gray-300 hover:text-gray-500 ml-1 leading-none"
+                >×</button>
               </div>
             ))}
-            <button onClick={addFilter}
+            <button
-              className="text-xs text-muted hover:text-ink-soft border border-dashed border-line rounded px-2 py-1">
+              onClick={addFilter}
+              className="text-xs text-gray-400 hover:text-gray-600 border border-dashed border-gray-200 rounded px-2 py-1"
+            >
               + filter
             </button>
             {filters.length > 0 && (
-              <button onClick={() => { setFilters([]); setOffset(0); load(0, sort.col, sort.dir, []) }}
+              <button
-                className="text-xs text-muted hover:text-danger">clear</button>
+                onClick={() => { setFilters([]); setOffset(0); load(0, sort.col, sort.dir, []) }}
+                className="text-xs text-gray-400 hover:text-red-500"
+              >
+                clear
+              </button>
             )}
           </div>
         )}
 
-        {/* Bulk select + override bar */}
+        {loading && <p className="text-sm text-gray-400">Loading…</p>}
-        {exists && visCols.length > 0 && (
+
-          <div className="mb-4 flex flex-wrap gap-2 items-center">
+        {!loading && viewError && (
-            <span className="text-xs text-muted font-medium mr-1">Bulk select:</span>
+          <p className="text-sm text-red-500">View error: {viewError} — check field types in Sources.</p>
-            <input
-              className={`text-xs font-mono border rounded px-2 py-1.5 w-44 focus:outline-none focus:border-accent ${
-                rowFilter ? 'border-accent-line' : 'border-line'
-              }`}
-              placeholder="regex on loaded rows…"
-              value={rowFilter}
-              onChange={e => setRowFilter(e.target.value)}
-            />
-            {rowFilter && (
-              <span className="text-xs text-muted">{selected.size} of {rows.length} rows selected</span>
-            )}
-            {selected.size > 0 && (
-              <div className="flex items-center gap-2 ml-4 p-2 bg-accent-soft border border-accent-line rounded flex-wrap">
-                {allOverrideCols.map(col => (
-                  <AutocompleteInput
-                    key={col}
-                    className="border border-accent-line rounded px-2 py-1 text-xs min-w-24 focus:outline-none focus:border-accent bg-surface"
-                    placeholder={col}
-                    value={bulkDraft[col] || ''}
-                    onChange={v => setBulkDraft(d => ({ ...d, [col]: v }))}
-                    suggestions={[...(globalValues[col] || [])].sort()}
-                  />
-                ))}
-                <button
-                  onClick={async () => {
-                    const overrides = Object.fromEntries(
-                      Object.entries(bulkDraft).filter(([, v]) => v.trim())
-                    )
-                    if (Object.keys(overrides).length === 0) return
-                    if (selected.size === 0) return
-                    setPanelSaving(true)
-                    setPanelMsg(null)
-                    try {
-                      const res = await api.setBulkRecordOverrides(source, [...selected], overrides)
-                      setSelected(new Set())
-                      setBulkDraft({})
-                      setPanelMsg({ text: `Updated ${res.updated} records.`, ok: true })
-                      load(offset, sort.col, sort.dir, filters)
-                    } catch (err) {
-                      setPanelMsg({ text: err.message, ok: false })
-                    } finally {
-                      setPanelSaving(false)
-                    }
-                  }}
-                  disabled={panelSaving || selected.size === 0 || Object.values(bulkDraft).every(v => !v.trim())}
-                  className="text-xs bg-blue-600 text-white px-3 py-1 rounded hover:bg-blue-700 disabled:opacity-40 whitespace-nowrap"
-                >
-                  Apply to {selected.size}
-                </button>
-                <button
-                  onClick={() => { setSelected(new Set()); setBulkDraft({}); setRowFilter('') }}
-                  className="text-xs text-accent hover:text-accent"
-                >
-                  cancel
-                </button>
-              </div>
-            )}
-          </div>
         )}
 
-        {loading && <p className="text-sm text-muted">Loading…</p>}
-        {!loading && viewError && <p className="text-sm text-danger">View error: {viewError} — check field types in Sources.</p>}
         {!loading && exists === false && (
-          <p className="text-sm text-muted">
+          <p className="text-sm text-gray-400">
-            No view generated yet. Go to <span className="font-medium text-ink-soft">Sources</span>, check fields as <span className="font-medium text-ink-soft">In view</span>, then click <span className="font-medium text-ink-soft">Generate view</span>.
+            No view generated yet. Go to <span className="font-medium text-gray-600">Sources</span>, check fields as <span className="font-medium text-gray-600">In view</span>, then click <span className="font-medium text-gray-600">Generate view</span>.
           </p>
         )}
+
         {!loading && exists && rows.length === 0 && (
-          <p className="text-sm text-muted">
+          <p className="text-sm text-gray-400">
             {filters.some(f => f.col && f.pattern) ? 'No records match the current filters.' : 'View exists but no transformed records yet. Import data and run a transform first.'}
           </p>
         )}
 
         {!loading && exists && rows.length > 0 && (
           <>
-            <div className="bg-surface border border-line rounded overflow-auto mb-4">
+            <div className="bg-white border border-gray-200 rounded overflow-auto mb-4">
               <table className="w-full text-sm">
                 <thead>
-                  <tr className="text-left text-xs text-muted border-b border-line-soft bg-raised">
+                  <tr className="text-left text-xs text-gray-400 border-b border-gray-100 bg-gray-50">
-                    <th className="px-2 py-2 w-8">
-                      <input
-                        type="checkbox"
-                        className="cursor-pointer"
-                        checked={rows.length > 0 && rows.every(r => selected.has(r.id))}
-                        onChange={e => {
-                          if (e.target.checked) setSelected(new Set(rows.map(r => r.id)))
-                          else setSelected(new Set())
-                        }}
-                      />
-                    </th>
                     {displayCols.map(col => {
                       const active = sort.col === col
                       return (
-                        <th key={col} onClick={() => toggleSort(col)}
+                        <th
-                          className="px-3 py-2 font-medium whitespace-nowrap cursor-pointer select-none hover:text-ink-soft">
+                          key={col}
+                          onClick={() => toggleSort(col)}
+                          className="px-3 py-2 font-medium whitespace-nowrap cursor-pointer select-none hover:text-gray-600"
+                        >
                           {col}
-                          <span className="ml-1 text-muted">{active ? (sort.dir === 'asc' ? '▲' : '▼') : '⇅'}</span>
+                          <span className="ml-1 text-gray-300">
+                            {active ? (sort.dir === 'asc' ? '▲' : '▼') : '⇅'}
+                          </span>
                         </th>
                       )
                     })}
@@ -449,28 +281,19 @@ export default function Records({ source }) {
                 <tbody>
                   {rows.map((row, i) => {
                     const isOverridden = row._overridden
-                    const isRowSelected = selected.has(row.id)
+                    const isSelected = selectedRow?.id != null && selectedRow.id === row.id
-                    const isPanelSelected = selectedRow?.id != null && selectedRow.id === row.id
                     return (
-                      <tr key={i} onClick={() => openPanel(row)}
+                      <tr
-                        className={`border-t border-line-soft cursor-pointer transition-colors
+                        key={i}
-                          ${isPanelSelected ? 'bg-accent-soft' : isRowSelected ? 'bg-accent-soft' : isOverridden ? 'bg-warn-soft hover:bg-warn-soft' : 'hover:bg-raised'}`}>
+                        onClick={() => openPanel(row)}
-                        <td className="px-2 py-2">
+                        className={`border-t border-gray-50 cursor-pointer transition-colors
-                          <input
+                          ${isSelected ? 'bg-blue-50' : isOverridden ? 'bg-amber-50 hover:bg-amber-100' : 'hover:bg-gray-50'}`}
-                            type="checkbox"
+                      >
-                            className="cursor-pointer"
-                            checked={isRowSelected}
-                            onChange={e => {
-                              e.stopPropagation()
-                              setSelected(s => { const n = new Set(s); n.has(row.id) ? n.delete(row.id) : n.add(row.id); return n })
-                            }}
-                          />
-                        </td>
                         {displayCols.map((col, j) => {
                           const formatted = formatVal(row[col])
                           return (
-                            <td key={j} className="px-3 py-2 text-xs text-ink-soft whitespace-nowrap max-w-48 truncate">
+                            <td key={j} className="px-3 py-2 text-xs text-gray-600 whitespace-nowrap max-w-48 truncate">
-                              {formatted === null ? <span className="text-muted">—</span> : formatted}
+                              {formatted === null ? <span className="text-gray-300">—</span> : formatted}
                             </td>
                           )
                         })}
@@ -481,169 +304,80 @@ export default function Records({ source }) {
               </table>
             </div>
 
-            <div className="flex items-center gap-3 text-sm text-muted">
+            <div className="flex items-center gap-3 text-sm text-gray-500">
               <button onClick={prev} disabled={offset === 0}
-                className="px-3 py-1 border border-line rounded hover:bg-raised disabled:opacity-40">← Prev</button>
+                className="px-3 py-1 border border-gray-200 rounded hover:bg-gray-50 disabled:opacity-40">
+                ← Prev
+              </button>
               <span>{offset + 1}–{offset + rows.length}</span>
               <button onClick={next} disabled={rows.length < LIMIT}
-                className="px-3 py-1 border border-line rounded hover:bg-raised disabled:opacity-40">Next →</button>
+                className="px-3 py-1 border border-gray-200 rounded hover:bg-gray-50 disabled:opacity-40">
+                Next →
+              </button>
             </div>
           </>
         )}
       </div>
 
-      {/* Panel */}
+      {/* Override panel */}
       {panelOpen && (
-        <div className="w-80 border-l border-line bg-surface flex flex-col overflow-hidden flex-shrink-0">
+        <div className="w-80 border-l border-gray-200 bg-white flex flex-col overflow-hidden flex-shrink-0">
-          <div className="flex items-center justify-between px-3 py-2 border-b border-line-soft">
+          <div className="flex items-center justify-between px-3 py-2 border-b border-gray-100">
-            <span className="text-xs font-semibold text-ink-soft uppercase tracking-wide">Record</span>
+            <span className="text-xs font-semibold text-gray-600 uppercase tracking-wide">Override</span>
-            <button onClick={closePanel} className="text-muted hover:text-muted leading-none text-lg">×</button>
+            <button onClick={closePanel} className="text-gray-300 hover:text-gray-500 leading-none text-lg">×</button>
           </div>
 
-          {panelLoading && <p className="text-xs text-muted p-3">Loading…</p>}
+          {panelLoading && <p className="text-xs text-gray-400 p-3">Loading…</p>}
 
           {selectedRecord && !panelLoading && (
-            <div className="flex-1 overflow-y-auto flex flex-col min-h-0">
+            <div className="flex-1 overflow-y-auto p-3 flex flex-col gap-3">
               {panelMsg && (
-                <div className={`text-xs px-3 py-2 border-b border-line-soft ${panelMsg.ok ? 'text-ok' : 'text-danger'}`}>
+                <div className={`text-xs ${panelMsg.ok ? 'text-green-600' : 'text-red-500'}`}>
                   {panelMsg.text}
                 </div>
               )}
 
-              {/* Raw fields — read only */}
+              <div className="text-xs text-gray-400">
-              <div className="border-b border-line-soft">
+                Click any field to override its value. Overrides survive reprocess.
-                <div className="px-3 py-1.5 bg-raised border-b border-line-soft">
-                  <span className="text-xs font-medium text-muted uppercase tracking-wide">Raw</span>
-                </div>
-                {Object.entries(selectedRecord.data || {}).map(([field, val]) => (
-                  <div key={field} className="flex items-baseline gap-2 px-3 py-1 border-t border-line-soft first:border-t-0">
-                    <span className="text-xs font-mono text-muted w-28 shrink-0 truncate">{field}</span>
-                    <span className="text-xs font-mono text-muted truncate">{formatVal(val) ?? <span className="text-muted">—</span>}</span>
-                  </div>
-                ))}
               </div>
 
-              {/* Transformed fields — read only delta */}
+              <div className="flex flex-col gap-1">
-              <div className="border-b border-line-soft">
+                {transformedFields.map(field => {
-                <div className="px-3 py-1.5 bg-raised border-b border-line-soft">
+                  const currentVal = selectedRecord.transformed[field]
-                  <span className="text-xs font-medium text-muted uppercase tracking-wide">Transformed</span>
+                  const isOverridden = field in overrideDraft
-                </div>
+                  return (
-                {Object.entries(selectedRecord.transformed || {}).filter(([k]) => !HIDDEN_COLS.has(k)).length === 0
+                    <div key={field} className={`rounded px-2 py-1.5 ${isOverridden ? 'bg-amber-50 border border-amber-200' : 'bg-gray-50'}`}>
-                  ? <div className="px-3 py-2 text-xs text-muted">No rule output yet.</div>
+                      <div className={`text-xs font-mono mb-0.5 ${isOverridden ? 'text-amber-700' : 'text-gray-400'}`}>
-                  : Object.entries(selectedRecord.transformed || {}).filter(([k]) => !HIDDEN_COLS.has(k)).map(([field, val]) => (
+                        {field}
-                    <div key={field} className="flex items-baseline gap-2 px-3 py-1 border-t border-line-soft first:border-t-0">
+                        {isOverridden && <span className="ml-1 text-amber-500">✎</span>}
-                      <span className="text-xs font-mono text-muted w-28 shrink-0 truncate">{field}</span>
+                      </div>
-                      <span className="text-xs font-mono text-accent truncate">{formatVal(val) ?? <span className="text-muted">—</span>}</span>
+                      <input
+                        className={`w-full text-xs font-mono bg-transparent border-0 focus:outline-none focus:ring-0 ${isOverridden ? 'text-amber-800' : 'text-gray-700'}`}
+                        value={isOverridden ? overrideDraft[field] : (currentVal ?? '')}
+                        onChange={e => setOverrideDraft(d => ({ ...d, [field]: e.target.value }))}
+                        onFocus={() => {
+                          if (!(field in overrideDraft)) {
+                            setOverrideDraft(d => ({ ...d, [field]: String(currentVal ?? '') }))
+                          }
+                        }}
+                      />
                     </div>
-                  ))
+                  )
-                }
+                })}
               </div>
 
-              {/* Overrides — editable */}
+              <div className="flex gap-2 pt-1">
-              <div className="flex-1 border-b border-line-soft">
-                <div className="flex items-center justify-between px-3 py-1.5 bg-raised border-b border-line-soft">
-                  <span className="text-xs font-medium text-muted uppercase tracking-wide">Overrides</span>
-                  <button
-                    onClick={() => setExtraCols(ec => [...ec, ''])}
-                    className="text-muted hover:text-ink-soft font-medium text-sm leading-none"
-                    title="Add field">+</button>
-                </div>
-                <table className="w-full text-xs">
-                  <tbody>
-                    {[...new Set([
-                      ...Object.keys(selectedRecord.transformed || {}),
-                      ...Object.keys(selectedRecord.overrides || {}),
-                      ...overrideCols
-                    ])].filter(k => !HIDDEN_COLS.has(k)).map(col => {
-                      const override = overrideDraft[col] ?? ''
-                      const placeholder = formatVal(selectedRecord.transformed?.[col]) ?? ''
-                      const suggestions = [...(globalValues[col] || [])].sort()
-                      return (
-                        <tr key={col} className="border-t border-line-soft">
-                          <td className="px-3 py-1.5 w-28 shrink-0">
-                            <span className="font-mono text-muted truncate block">{col}</span>
-                          </td>
-                          <td className="px-1 py-1.5">
-                            <AutocompleteInput
-                              className={`w-full text-xs font-mono px-2 py-0.5 rounded border focus:outline-none ${
-                                override ? 'border-warn-line bg-warn-soft text-warn' : 'border-line text-ink-soft'
-                              }`}
-                              value={override}
-                              placeholder={placeholder}
-                              onChange={v => setOverrideDraft(d => ({ ...d, [col]: v }))}
-                              onEnter={handleSaveOverrides}
-                              suggestions={suggestions}
-                            />
-                          </td>
-                          <td className="pr-2 text-center w-6">
-                            {override && (
-                              <button
-                                onClick={() => setOverrideDraft(d => { const n = { ...d }; delete n[col]; return n })}
-                                className="text-muted hover:text-danger leading-none text-base">×</button>
-                            )}
-                          </td>
-                        </tr>
-                      )
-                    })}
-                    {extraCols.map((col, i) => {
-                      const val = overrideDraft[col] ?? ''
-                      const suggestions = [...(globalValues[col] || [])].sort()
-                      return (
-                        <tr key={`extra-${i}`} className="border-t border-line-soft">
-                          <td className="px-3 py-1.5 w-28 shrink-0">
-                            <input
-                              className="w-full text-xs font-mono border border-line rounded px-1 py-0.5 focus:outline-none focus:border-accent"
-                              value={col}
-                              placeholder="field name"
-                              onChange={e => {
-                                const newName = e.target.value
-                                setExtraCols(ec => { const c = [...ec]; c[i] = newName; return c })
-                                if (val) setOverrideDraft(d => {
-                                  const n = { ...d }
-                                  delete n[col]
-                                  if (newName) n[newName] = val
-                                  return n
-                                })
-                              }}
-                            />
-                          </td>
-                          <td className="px-1 py-1.5">
-                            <AutocompleteInput
-                              className={`w-full text-xs font-mono px-2 py-0.5 rounded border focus:outline-none ${
-                                val ? 'border-warn-line bg-warn-soft text-warn' : 'border-line text-ink-soft'
-                              }`}
-                              value={val}
-                              onChange={v => setOverrideDraft(d => ({ ...d, [col]: v }))}
-                              onEnter={handleSaveOverrides}
-                              suggestions={suggestions}
-                            />
-                          </td>
-                          <td className="pr-2 text-center w-6">
-                            {val && (
-                              <button
-                                onClick={() => setOverrideDraft(d => { const n = { ...d }; delete n[col]; return n })}
-                                className="text-muted hover:text-danger leading-none text-base">×</button>
-                            )}
-                          </td>
-                        </tr>
-                      )
-                    })}
-                  </tbody>
-                </table>
-              </div>
-
-              <div className="flex gap-2 px-3 py-2 border-t border-line-soft shrink-0">
                 <button
                   onClick={handleSaveOverrides}
-                  disabled={panelSaving || !isDirty}
+                  disabled={panelSaving || Object.keys(overrideDraft).length === 0}
                   className="flex-1 text-xs bg-blue-600 text-white rounded px-3 py-1.5 hover:bg-blue-700 disabled:opacity-40">
-                  {panelSaving ? 'Saving…' : 'Save'}
+                  {panelSaving ? 'Saving…' : 'Save overrides'}
                 </button>
-                {selectedRecord.overrides && Object.keys(selectedRecord.overrides).length > 0 && (
+                {selectedRecord.overrides && (
                   <button
                     onClick={handleClearOverrides}
                     disabled={panelSaving}
-                    className="text-xs border border-line rounded px-3 py-1.5 text-muted hover:border-danger-line hover:text-danger disabled:opacity-40">
+                    className="text-xs border border-gray-200 rounded px-3 py-1.5 text-gray-500 hover:border-red-300 hover:text-red-500 disabled:opacity-40">
                     Clear
                   </button>
                 )}

ui/src/pages/Remap.jsx

@@ -73,8 +73,8 @@ export default function Remap() {
   }
 
   return (
-    <div className="p-4 sm:p-6 max-w-4xl">
+    <div className="p-6 max-w-4xl">
-      <h1 className="text-base font-semibold text-ink mb-4">Remap Output Values</h1>
+      <h1 className="text-base font-semibold text-gray-800 mb-4">Remap Output Values</h1>
 
       {/* Search */}
       <form onSubmit={handleSearch} className="flex items-center gap-2 mb-5">
@@ -83,7 +83,7 @@ export default function Remap() {
           value={search}
           onChange={e => setSearch(e.target.value)}
           placeholder="Search output values…"
-          className="text-sm border border-line rounded px-3 py-1.5 w-72 focus:outline-none focus:border-accent"
+          className="text-sm border border-gray-300 rounded px-3 py-1.5 w-72 focus:outline-none focus:border-blue-400"
         />
         <button type="submit" disabled={searching}
           className="text-sm bg-blue-600 text-white rounded px-3 py-1.5 hover:bg-blue-700 disabled:opacity-50">
@@ -95,15 +95,15 @@ export default function Remap() {
       {results !== null && (
         <div className="mb-6">
           {results.length === 0 ? (
-            <p className="text-sm text-muted">No matching output values found.</p>
+            <p className="text-sm text-gray-400">No matching output values found.</p>
           ) : (
             <>
-              <div className="text-xs text-muted uppercase tracking-wide mb-1">
+              <div className="text-xs text-gray-400 uppercase tracking-wide mb-1">
                 {results.length} result{results.length !== 1 ? 's' : ''} — click one to remap
               </div>
-              <table className="w-full text-sm border border-line rounded overflow-hidden">
+              <table className="w-full text-sm border border-gray-200 rounded overflow-hidden">
                 <thead>
-                  <tr className="bg-raised text-left text-xs text-muted uppercase tracking-wide">
+                  <tr className="bg-gray-50 text-left text-xs text-gray-400 uppercase tracking-wide">
                     <th className="px-3 py-2">Field</th>
                     <th className="px-3 py-2">Value</th>
                     <th className="px-3 py-2 text-right">Mappings</th>
@@ -115,11 +115,11 @@ export default function Remap() {
                     return (
                       <tr key={i}
                         onClick={() => handleSelect(r)}
-                        className={`border-t border-line-soft cursor-pointer transition-colors
+                        className={`border-t border-gray-100 cursor-pointer transition-colors
-                          ${isActive ? 'bg-accent-soft' : 'hover:bg-raised'}`}>
+                          ${isActive ? 'bg-blue-50' : 'hover:bg-gray-50'}`}>
-                        <td className="px-3 py-2 font-mono text-muted">{r.col}</td>
+                        <td className="px-3 py-2 font-mono text-gray-500">{r.col}</td>
-                        <td className="px-3 py-2 font-mono text-ink">{r.val}</td>
+                        <td className="px-3 py-2 font-mono text-gray-800">{r.val}</td>
-                        <td className="px-3 py-2 text-right text-muted">{r.mapping_count}</td>
+                        <td className="px-3 py-2 text-right text-gray-400">{r.mapping_count}</td>
                       </tr>
                     )
                   })}
@@ -132,25 +132,25 @@ export default function Remap() {
 
       {/* Remap panel */}
       {selected && (
-        <div className="border border-line rounded p-4 mb-6 bg-surface">
+        <div className="border border-gray-200 rounded p-4 mb-6 bg-white">
-          <div className="text-xs text-muted uppercase tracking-wide mb-3">
+          <div className="text-xs text-gray-400 uppercase tracking-wide mb-3">
-            Remap <span className="font-mono text-ink-soft">{selected.col}</span>
+            Remap <span className="font-mono text-gray-600">{selected.col}</span>
           </div>
           <div className="flex items-center gap-3 mb-4">
             <div className="flex-1">
-              <div className="text-xs text-muted mb-1">From</div>
+              <div className="text-xs text-gray-400 mb-1">From</div>
-              <div className="text-sm font-mono bg-raised border border-line rounded px-3 py-1.5 text-ink-soft">
+              <div className="text-sm font-mono bg-gray-50 border border-gray-200 rounded px-3 py-1.5 text-gray-700">
                 {selected.val}
               </div>
             </div>
-            <div className="text-muted mt-4">→</div>
+            <div className="text-gray-300 mt-4">→</div>
             <div className="flex-1">
-              <div className="text-xs text-muted mb-1">To</div>
+              <div className="text-xs text-gray-400 mb-1">To</div>
               <input
                 value={toVal}
                 onChange={e => setToVal(e.target.value)}
                 onKeyDown={e => e.key === 'Enter' && handleApply()}
-                className="w-full text-sm font-mono border border-line rounded px-3 py-1.5 focus:outline-none focus:border-accent"
+                className="w-full text-sm font-mono border border-gray-300 rounded px-3 py-1.5 focus:outline-none focus:border-blue-400"
               />
             </div>
             <div className="mt-4">
@@ -164,22 +164,22 @@ export default function Remap() {
           </div>
 
           {msg && (
-            <div className={`text-sm mb-3 ${msg.ok ? 'text-ok' : 'text-danger'}`}>
+            <div className={`text-sm mb-3 ${msg.ok ? 'text-green-600' : 'text-red-500'}`}>
               {msg.text}
             </div>
           )}
 
           {/* Affected mappings */}
           {loadingMatches ? (
-            <p className="text-xs text-muted">Loading…</p>
+            <p className="text-xs text-gray-400">Loading…</p>
           ) : matches && matches.length > 0 && (
             <div>
-              <div className="text-xs text-muted uppercase tracking-wide mb-1">
+              <div className="text-xs text-gray-400 uppercase tracking-wide mb-1">
                 Affected mappings
               </div>
-              <table className="w-full text-xs border border-line-soft rounded overflow-hidden">
+              <table className="w-full text-xs border border-gray-100 rounded overflow-hidden">
                 <thead>
-                  <tr className="bg-raised text-left text-muted">
+                  <tr className="bg-gray-50 text-left text-gray-400">
                     <th className="px-2 py-1">Source</th>
                     <th className="px-2 py-1">Rule</th>
                     <th className="px-2 py-1">Input</th>
@@ -188,15 +188,15 @@ export default function Remap() {
                 </thead>
                 <tbody>
                   {matches.map(m => (
-                    <tr key={m.id} className="border-t border-line-soft">
+                    <tr key={m.id} className="border-t border-gray-50">
-                      <td className="px-2 py-1 font-mono text-muted">{m.source_name}</td>
+                      <td className="px-2 py-1 font-mono text-gray-500">{m.source_name}</td>
-                      <td className="px-2 py-1 font-mono text-muted">{m.rule_name}</td>
+                      <td className="px-2 py-1 font-mono text-gray-500">{m.rule_name}</td>
-                      <td className="px-2 py-1 font-mono text-ink-soft">
+                      <td className="px-2 py-1 font-mono text-gray-700">
                         {typeof m.input_value === 'string' ? m.input_value : JSON.stringify(m.input_value)}
                       </td>
-                      <td className="px-2 py-1 font-mono text-ink-soft">
+                      <td className="px-2 py-1 font-mono text-gray-700">
                         {Object.entries(m.output).map(([k, v]) => (
-                          <span key={k} className={k === selected.col ? 'text-accent font-semibold' : ''}>
+                          <span key={k} className={k === selected.col ? 'text-blue-600 font-semibold' : ''}>
                             {k}: {v}{' '}
                           </span>
                         ))}

ui/src/pages/Rules.jsx

@@ -7,27 +7,27 @@ function PreviewModal({ rows, onClose }) {
   const matched = rows.filter(r => r.extracted_value != null).length
   return (
     <div className="fixed inset-0 bg-black/40 flex items-center justify-center z-50" onClick={onClose}>
-      <div className="bg-surface rounded-lg shadow-xl w-3/4 max-w-3xl max-h-[80vh] flex flex-col"
+      <div className="bg-white rounded-lg shadow-xl w-3/4 max-w-3xl max-h-[80vh] flex flex-col"
         onClick={e => e.stopPropagation()}>
-        <div className="flex items-center justify-between px-5 py-3 border-b border-line-soft">
+        <div className="flex items-center justify-between px-5 py-3 border-b border-gray-100">
-          <span className="text-sm font-medium text-ink-soft">
+          <span className="text-sm font-medium text-gray-700">
-            Pattern results — <span className="text-muted font-normal">{matched}/{rows.length} matched</span>
+            Pattern results — <span className="text-gray-500 font-normal">{matched}/{rows.length} matched</span>
           </span>
-          <button onClick={onClose} className="text-muted hover:text-ink-soft text-lg leading-none">✕</button>
+          <button onClick={onClose} className="text-gray-400 hover:text-gray-600 text-lg leading-none">✕</button>
         </div>
         <div className="overflow-auto flex-1 px-5 py-3">
           <table className="w-full text-xs">
             <thead>
-              <tr className="text-left text-muted border-b border-line-soft">
+              <tr className="text-left text-gray-400 border-b border-gray-100">
                 <th className="pb-2 font-medium w-1/2 pr-4">Raw value</th>
                 <th className="pb-2 font-medium">Result</th>
               </tr>
             </thead>
             <tbody>
               {rows.map((r, i) => (
-                <tr key={i} className="border-t border-line-soft">
+                <tr key={i} className="border-t border-gray-50">
-                  <td className="py-1 font-mono text-muted pr-4 break-all">{r.raw_value}</td>
+                  <td className="py-1 font-mono text-gray-400 pr-4 break-all">{r.raw_value}</td>
-                  <td className={`py-1 font-mono break-all ${r.extracted_value != null ? 'text-ink' : 'text-muted'}`}>
+                  <td className={`py-1 font-mono break-all ${r.extracted_value != null ? 'text-gray-800' : 'text-gray-300'}`}>
                     {r.extracted_value != null
                       ? (Array.isArray(r.extracted_value) ? r.extracted_value.join(' · ') : String(r.extracted_value))
                       : '—'}
@@ -42,7 +42,7 @@ function PreviewModal({ rows, onClose }) {
   )
 }
 
-function FormPanel({ form, setForm, editing, error, loading, fields, source, onSubmit, onCancel }) {
+function FormPanel({ form, setForm, editing, error, loading, fields, rules, source, onSubmit, onCancel }) {
   const [preview, setPreview] = useState([])
   const [previewing, setPreviewing] = useState(false)
   const [modalOpen, setModalOpen] = useState(false)
@@ -67,68 +67,81 @@ function FormPanel({ form, setForm, editing, error, loading, fields, source, onS
   }, [form.field, form.pattern, form.flags, form.function_type, form.replace_value, source])
 
   return (
-    <div className="bg-surface border border-line rounded p-4 mb-4">
+    <div className="bg-white border border-gray-200 rounded p-4 mb-4">
-      <h2 className="text-sm font-semibold text-ink-soft mb-3">{editing ? 'Edit rule' : 'New rule'}</h2>
+      <h2 className="text-sm font-semibold text-gray-700 mb-3">{editing ? 'Edit rule' : 'New rule'}</h2>
       <form onSubmit={onSubmit} className="space-y-3">
         <div className="grid grid-cols-2 gap-3">
           <div>
-            <label className="text-xs text-muted block mb-1">Rule name</label>
+            <label className="text-xs text-gray-500 block mb-1">Rule name</label>
             <input
-              className="w-full border border-line rounded px-3 py-1.5 text-sm focus:outline-none focus:border-accent"
+              className="w-full border border-gray-200 rounded px-3 py-1.5 text-sm focus:outline-none focus:border-blue-400"
               value={form.name} onChange={e => setForm(f => ({ ...f, name: e.target.value }))}
               placeholder="e.g. First 20"
             />
           </div>
           <div>
-            <label className="text-xs text-muted block mb-1">Sequence</label>
+            <label className="text-xs text-gray-500 block mb-1">Sequence</label>
             <input
               type="number"
-              className="w-full border border-line rounded px-3 py-1.5 text-sm focus:outline-none focus:border-accent"
+              className="w-full border border-gray-200 rounded px-3 py-1.5 text-sm focus:outline-none focus:border-blue-400"
               value={form.sequence} onChange={e => setForm(f => ({ ...f, sequence: parseInt(e.target.value) || 0 }))}
             />
           </div>
         </div>
         <div className="grid grid-cols-2 gap-3">
           <div>
-            <label className="text-xs text-muted block mb-1">Input field</label>
+            <label className="text-xs text-gray-500 block mb-1">Input field</label>
-            {fields.length > 0 ? (
+            {fields.length > 0 ? (() => {
-              <select
+              // Output fields from rules at a lower sequence — available as chained inputs
-                className="w-full border border-line rounded px-3 py-1.5 text-sm focus:outline-none focus:border-accent"
+              const chainedFields = [...new Set(
-                value={form.field} onChange={e => setForm(f => ({ ...f, field: e.target.value }))}
+                (rules || [])
-              >
+                  .filter(r => r.sequence < form.sequence && r.output_field && (!editing || r.id !== editing))
-                <option value="">— select field —</option>
+                  .map(r => r.output_field)
-                {fields.map(f => <option key={f} value={f}>{f}</option>)}
+              )].filter(f => !fields.includes(f))
-              </select>
+              return (
-            ) : (
+                <select
+                  className="w-full border border-gray-200 rounded px-3 py-1.5 text-sm focus:outline-none focus:border-blue-400"
+                  value={form.field} onChange={e => setForm(f => ({ ...f, field: e.target.value }))}
+                >
+                  <option value="">— select field —</option>
+                  {fields.map(f => <option key={f} value={f}>{f}</option>)}
+                  {chainedFields.length > 0 && (
+                    <optgroup label="from earlier rules">
+                      {chainedFields.map(f => <option key={f} value={f}>{f}</option>)}
+                    </optgroup>
+                  )}
+                </select>
+              )
+            })() : (
               <input
-                className="w-full border border-line rounded px-3 py-1.5 text-sm focus:outline-none focus:border-accent"
+                className="w-full border border-gray-200 rounded px-3 py-1.5 text-sm focus:outline-none focus:border-blue-400"
                 value={form.field} onChange={e => setForm(f => ({ ...f, field: e.target.value }))}
                 placeholder="e.g. description"
               />
             )}
           </div>
           <div>
-            <label className="text-xs text-muted block mb-1">Output field</label>
+            <label className="text-xs text-gray-500 block mb-1">Output field</label>
             <input
-              className="w-full border border-line rounded px-3 py-1.5 text-sm focus:outline-none focus:border-accent"
+              className="w-full border border-gray-200 rounded px-3 py-1.5 text-sm focus:outline-none focus:border-blue-400"
               value={form.output_field} onChange={e => setForm(f => ({ ...f, output_field: e.target.value }))}
               placeholder="e.g. merchant"
             />
           </div>
         </div>
         <div>
-          <label className="text-xs text-muted block mb-1">Pattern (regex)</label>
+          <label className="text-xs text-gray-500 block mb-1">Pattern (regex)</label>
           <input
-            className="w-full border border-line rounded px-3 py-1.5 text-sm font-mono focus:outline-none focus:border-accent"
+            className="w-full border border-gray-200 rounded px-3 py-1.5 text-sm font-mono focus:outline-none focus:border-blue-400"
             value={form.pattern} onChange={e => setForm(f => ({ ...f, pattern: e.target.value }))}
             placeholder="e.g. .{1,20}"
           />
         </div>
         <div className="grid grid-cols-2 gap-3">
           <div>
-            <label className="text-xs text-muted block mb-1">Function</label>
+            <label className="text-xs text-gray-500 block mb-1">Function</label>
             <select
-              className="w-full border border-line rounded px-3 py-1.5 text-sm focus:outline-none focus:border-accent"
+              className="w-full border border-gray-200 rounded px-3 py-1.5 text-sm focus:outline-none focus:border-blue-400"
               value={form.function_type} onChange={e => setForm(f => ({ ...f, function_type: e.target.value }))}
             >
               <option value="extract">extract</option>
@@ -136,16 +149,16 @@ function FormPanel({ form, setForm, editing, error, loading, fields, source, onS
             </select>
           </div>
           <div>
-            <label className="text-xs text-muted block mb-1">Flags</label>
+            <label className="text-xs text-gray-500 block mb-1">Flags</label>
             <input
-              className="w-full border border-line rounded px-3 py-1.5 text-sm font-mono focus:outline-none focus:border-accent"
+              className="w-full border border-gray-200 rounded px-3 py-1.5 text-sm font-mono focus:outline-none focus:border-blue-400"
               value={form.flags} onChange={e => setForm(f => ({ ...f, flags: e.target.value }))}
               placeholder="e.g. i"
             />
           </div>
         </div>
         {form.function_type === 'extract' && (
-          <label className="flex items-center gap-2 text-xs text-ink-soft cursor-pointer select-none">
+          <label className="flex items-center gap-2 text-xs text-gray-600 cursor-pointer select-none">
             <input
               type="checkbox"
               checked={!!form.retain}
@@ -156,9 +169,9 @@ function FormPanel({ form, setForm, editing, error, loading, fields, source, onS
         )}
         {form.function_type === 'replace' && (
           <div>
-            <label className="text-xs text-muted block mb-1">Replacement string</label>
+            <label className="text-xs text-gray-500 block mb-1">Replacement string</label>
             <input
-              className="w-full border border-line rounded px-3 py-1.5 text-sm font-mono focus:outline-none focus:border-accent"
+              className="w-full border border-gray-200 rounded px-3 py-1.5 text-sm font-mono focus:outline-none focus:border-blue-400"
               value={form.replace_value} onChange={e => setForm(f => ({ ...f, replace_value: e.target.value }))}
               placeholder="e.g. leave blank to delete the match"
             />
@@ -166,23 +179,23 @@ function FormPanel({ form, setForm, editing, error, loading, fields, source, onS
         )}
         {/* Live preview */}
         {(preview.length > 0 || previewing) && (
-          <div className="border border-line-soft rounded p-2 bg-raised">
+          <div className="border border-gray-100 rounded p-2 bg-gray-50">
             <div className="flex items-center justify-between mb-1">
-              <p className="text-xs text-muted">
+              <p className="text-xs text-gray-400">
                 {previewing ? 'Testing…' : `${preview.filter(r => r.extracted_value != null).length}/${preview.length} matched`}
               </p>
               {!previewing && preview.length > 0 && (
                 <button type="button" onClick={() => setModalOpen(true)}
-                  className="text-xs text-accent hover:text-accent">expand</button>
+                  className="text-xs text-blue-400 hover:text-blue-600">expand</button>
               )}
             </div>
             {!previewing && (
               <table className="w-full text-xs">
                 <tbody>
                   {preview.slice(0, 5).map((r, i) => (
-                    <tr key={i} className="border-t border-line-soft first:border-0">
+                    <tr key={i} className="border-t border-gray-100 first:border-0">
-                      <td className="py-0.5 font-mono text-muted truncate max-w-0 w-1/2 pr-3">{r.raw_value}</td>
+                      <td className="py-0.5 font-mono text-gray-400 truncate max-w-0 w-1/2 pr-3">{r.raw_value}</td>
-                      <td className={`py-0.5 font-mono truncate ${r.extracted_value != null ? 'text-ink' : 'text-muted'}`}>
+                      <td className={`py-0.5 font-mono truncate ${r.extracted_value != null ? 'text-gray-800' : 'text-gray-300'}`}>
                         {r.extracted_value != null
                           ? (Array.isArray(r.extracted_value) ? r.extracted_value.join(' · ') : String(r.extracted_value))
                           : '—'}
@@ -197,14 +210,14 @@ function FormPanel({ form, setForm, editing, error, loading, fields, source, onS
 
         {modalOpen && <PreviewModal rows={preview} onClose={() => setModalOpen(false)} />}
 
-        {error && <p className="text-xs text-danger">{error}</p>}
+        {error && <p className="text-xs text-red-500">{error}</p>}
         <div className="flex gap-2">
           <button type="submit" disabled={loading}
             className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700 disabled:opacity-50">
             {loading ? 'Saving…' : (editing ? 'Save' : 'Create')}
           </button>
           <button type="button" onClick={onCancel}
-            className="text-sm text-muted px-3 py-1.5 rounded hover:bg-raised">
+            className="text-sm text-gray-500 px-3 py-1.5 rounded hover:bg-gray-100">
             Cancel
           </button>
         </div>
@@ -213,7 +226,7 @@ function FormPanel({ form, setForm, editing, error, loading, fields, source, onS
   )
 }
 
-export default function Rules({ source, onStale }) {
+export default function Rules({ source }) {
   const [rules, setRules] = useState([])
   const [creating, setCreating] = useState(false)
   const [editing, setEditing] = useState(null)
@@ -266,7 +279,6 @@ export default function Rules({ source, onStale }) {
       } else {
         await api.createRule({ ...form, source_name: source })
       }
-      onStale?.(source)
       const updated = await api.getRules(source)
       setRules(updated)
       setCreating(false)
@@ -283,7 +295,6 @@ export default function Rules({ source, onStale }) {
     if (!confirm('Delete this rule and all its mappings?')) return
     try {
       await api.deleteRule(id)
-      onStale?.(source)
       setRules(r => r.filter(x => x.id !== id))
       setTestResults(t => { const n = { ...t }; delete n[id]; return n })
     } catch (err) {
@@ -303,19 +314,18 @@ export default function Rules({ source, onStale }) {
   async function handleToggle(rule) {
     try {
       await api.updateRule(rule.id, { enabled: !rule.enabled })
-      onStale?.(source)
       setRules(r => r.map(x => x.id === rule.id ? { ...x, enabled: !x.enabled } : x))
     } catch (err) {
       alert(err.message)
     }
   }
 
-  if (!source) return <div className="p-4 sm:p-6 text-sm text-muted">Select a source first.</div>
+  if (!source) return <div className="p-6 text-sm text-gray-400">Select a source first.</div>
 
   return (
-    <div className="p-4 sm:p-6 max-w-3xl">
+    <div className="p-6 max-w-3xl">
       <div className="flex items-center justify-between mb-6">
-        <h1 className="text-xl font-semibold text-ink">Rules — {source}</h1>
+        <h1 className="text-xl font-semibold text-gray-800">Rules — {source}</h1>
         <button onClick={startCreate}
           className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700">
           New rule
@@ -325,24 +335,24 @@ export default function Rules({ source, onStale }) {
       {creating && (
         <FormPanel
           form={form} setForm={setForm} editing={false}
-          error={error} loading={loading} fields={fields} source={source}
+          error={error} loading={loading} fields={fields} rules={rules} source={source}
           onSubmit={handleSubmit}
           onCancel={() => { setCreating(false); setError('') }}
         />
       )}
 
       {rules.length === 0 && !creating && (
-        <p className="text-sm text-muted">No rules yet. Add a regex rule to start extracting values.</p>
+        <p className="text-sm text-gray-400">No rules yet. Add a regex rule to start extracting values.</p>
       )}
 
       <div className="space-y-2">
         {rules.map(rule => {
           const isExpanded = expanded === rule.id
           return (
-            <div key={rule.id} className="bg-surface border border-line rounded">
+            <div key={rule.id} className="bg-white border border-gray-200 rounded">
               {/* Header — always visible, click to expand/collapse */}
               <div
-                className="flex items-center gap-3 px-4 py-3 cursor-pointer hover:bg-raised select-none"
+                className="flex items-center gap-3 px-4 py-3 cursor-pointer hover:bg-gray-50 select-none"
                 onClick={() => {
                   if (isExpanded) { setExpanded(null); setEditing(null) }
                   else { setExpanded(rule.id); startEdit(rule) }
@@ -350,38 +360,38 @@ export default function Rules({ source, onStale }) {
               >
                 <button
                   onClick={e => { e.stopPropagation(); handleToggle(rule) }}
-                  className={`w-8 h-4 rounded-full flex-shrink-0 transition-colors ${rule.enabled ? 'bg-blue-500' : 'bg-raised'}`}
+                  className={`w-8 h-4 rounded-full flex-shrink-0 transition-colors ${rule.enabled ? 'bg-blue-500' : 'bg-gray-200'}`}
                   title={rule.enabled ? 'Disable' : 'Enable'}
                 />
                 <div className="flex-1 min-w-0">
-                  <span className="font-medium text-ink text-sm">{rule.name}</span>
+                  <span className="font-medium text-gray-800 text-sm">{rule.name}</span>
-                  <span className="text-muted text-xs ml-2">seq {rule.sequence}</span>
+                  <span className="text-gray-400 text-xs ml-2">seq {rule.sequence}</span>
                   {!isExpanded && (
-                    <div className="text-xs text-muted mt-0.5 truncate">
+                    <div className="text-xs text-gray-400 mt-0.5 truncate">
                       <span className="font-mono">{rule.field}</span>
                       <span className="mx-1">→</span>
-                      <span className="font-mono bg-raised px-1 rounded">{rule.pattern}</span>
+                      <span className="font-mono bg-gray-50 px-1 rounded">{rule.pattern}</span>
-                      {rule.flags && <span className="text-accent ml-1">/{rule.flags}</span>}
+                      {rule.flags && <span className="text-blue-400 ml-1">/{rule.flags}</span>}
                       <span className="mx-1">→</span>
                       <span className="font-mono">{rule.output_field}</span>
-                      {rule.function_type === 'replace' && <span className="ml-1 text-warn">(replace)</span>}
+                      {rule.function_type === 'replace' && <span className="ml-1 text-orange-400">(replace)</span>}
                     </div>
                   )}
                 </div>
-                <span className="text-xs text-muted flex-shrink-0">{isExpanded ? '▲' : '▼'}</span>
+                <span className="text-xs text-gray-300 flex-shrink-0">{isExpanded ? '▲' : '▼'}</span>
               </div>
 
               {/* Expanded content */}
               {isExpanded && (
-                <div className="border-t border-line-soft">
+                <div className="border-t border-gray-100">
                   <div className="px-4 pt-3 pb-1 flex justify-end">
                     <button onClick={e => { e.stopPropagation(); handleDelete(rule.id) }}
-                      className="text-xs text-danger hover:text-danger">Delete</button>
+                      className="text-xs text-red-400 hover:text-red-600">Delete</button>
                   </div>
                   <div className="px-4 pb-4">
                     <FormPanel
-                      form={form} setForm={setForm} editing={true}
+                      form={form} setForm={setForm} editing={rule.id}
-                      error={error} loading={loading} fields={fields} source={source}
+                      error={error} loading={loading} fields={fields} rules={rules} source={source}
                       onSubmit={e => handleSubmit(e, rule.id)}
                       onCancel={() => { setEditing(null); setExpanded(null) }}
                     />

ui/src/pages/SourceDetail.jsx

@@ -1,424 +0,0 @@
-import { useState, useEffect } from 'react'
-import { useParams, useNavigate } from 'react-router-dom'
-import { api } from '../api'
-import Section from '../components/Section.jsx'
-import SampleTable from '../components/SampleTable.jsx'
-
-const FIELD_TYPES = ['text', 'numeric', 'date']
-
-export default function SourceDetail({ sources, setSources }) {
-  const { name: source } = useParams()
-  const navigate = useNavigate()
-  const [constraintFields, setConstraintFields] = useState('')
-  const [globalPicklist, setGlobalPicklist] = useState(true)
-  const [schemaFields, setSchemaFields] = useState([])
-  const [stats, setStats] = useState(null)
-  const [sampleRows, setSampleRows] = useState([])
-  const [saving, setSaving] = useState(false)
-  const [reprocessing, setReprocessing] = useState(false)
-  const [generating, setGenerating] = useState(false)
-  const [result, setResult] = useState('')
-  const [error, setError] = useState('')
-  const [viewName, setViewName] = useState('')
-  const [availableFields, setAvailableFields] = useState([])
-  const [fieldSort, setFieldSort] = useState({ col: 'key', dir: 'asc' })
-  const [bridgeAccounts, setBridgeAccounts] = useState(null)
-  const [bridgeLoading, setBridgeLoading] = useState(false)
-  const [bridgeError, setBridgeError] = useState('')
-
-  const sourceObj = sources.find(s => s.name === source)
-
-  useEffect(() => {
-    if (!sourceObj) return
-    setConstraintFields(sourceObj.constraint_fields?.join(', ') || '')
-    setGlobalPicklist(sourceObj.global_picklist !== false)
-    setSchemaFields((sourceObj.config?.fields || []).map((f, i) => ({ seq: i + 1, ...f })))
-    setViewName(sourceObj.config?.fields?.length ? `dfv.${sourceObj.name}` : '')
-    setResult('')
-    setError('')
-    setStats(null)
-    setAvailableFields([])
-    setSampleRows([])
-    setBridgeAccounts(null)
-    setBridgeError('')
-    api.getStats(sourceObj.name).then(setStats).catch(() => {})
-    api.getFields(sourceObj.name).then(setAvailableFields).catch(() => {})
-    api.getRecords(sourceObj.name, 50).then(rows => setSampleRows(rows.map(r => r.data).filter(Boolean))).catch(() => {})
-  }, [source, sourceObj?.name])
-
-
-  async function handleSave(e) {
-    e.preventDefault()
-    setSaving(true)
-    setError('')
-    try {
-      const constraint_fields = constraintFields.split(',').map(s => s.trim()).filter(Boolean)
-      const fields = [...schemaFields.filter(f => f.name)].sort((a, b) => (a.seq ?? 0) - (b.seq ?? 0))
-      const config = { ...(sourceObj.config || {}), fields }
-      await api.updateSource(sourceObj.name, { constraint_fields, config, global_picklist: globalPicklist })
-      if (fields.length > 0) {
-        const res = await api.generateView(sourceObj.name)
-        if (res.success) setViewName(res.view)
-      }
-      const updated = await api.getSources()
-      setSources(updated)
-      setResult('Saved.')
-    } catch (err) {
-      setError(err.message)
-    } finally {
-      setSaving(false)
-    }
-  }
-
-  async function handleGenerateView() {
-    setGenerating(true)
-    setResult('')
-    setError('')
-    try {
-      const constraint_fields = constraintFields.split(',').map(s => s.trim()).filter(Boolean)
-      const fields = [...schemaFields.filter(f => f.name)].sort((a, b) => (a.seq ?? 0) - (b.seq ?? 0))
-      const config = { ...(sourceObj.config || {}), fields }
-      await api.updateSource(sourceObj.name, { constraint_fields, config, global_picklist: globalPicklist })
-      const res = await api.generateView(sourceObj.name)
-      if (res.success) {
-        setViewName(res.view)
-        setResult(`View created: ${res.view}`)
-      } else {
-        setError(res.error)
-      }
-    } catch (err) {
-      setError(err.message)
-    } finally {
-      setGenerating(false)
-    }
-  }
-
-  async function handleReprocess() {
-    if (!confirm(`Reprocess all records for "${sourceObj.name}"? This will clear and reapply all transformations.`)) return
-    setReprocessing(true)
-    setResult('')
-    setError('')
-    try {
-      const res = await api.reprocess(sourceObj.name)
-      setResult(`Reprocessed ${res.transformed} records.`)
-      api.getStats(sourceObj.name).then(setStats).catch(() => {})
-    } catch (err) {
-      setError(err.message)
-    } finally {
-      setReprocessing(false)
-    }
-  }
-
-  async function handleDelete() {
-    if (!confirm(`Delete source "${sourceObj.name}" and all its data?`)) return
-    try {
-      await api.deleteSource(sourceObj.name)
-      setSources(await api.getSources())
-      navigate('/sources')
-    } catch (err) {
-      alert(err.message)
-    }
-  }
-
-  // The bridge is an external call, so accounts are fetched on demand rather
-  // than on every visit to this page.
-  async function loadBridgeAccounts() {
-    setBridgeLoading(true)
-    setBridgeError('')
-    try {
-      const res = await api.getSimpleFinAccounts()
-      setBridgeAccounts(res.accounts || [])
-      if (res.errors?.length) setBridgeError(res.errors.join('; '))
-    } catch (err) {
-      setBridgeError(err.message)
-    } finally {
-      setBridgeLoading(false)
-    }
-  }
-
-  // Writes only config — constraint_fields and global_picklist are left NULL so
-  // update_source keeps whatever is already stored.
-  async function handleLinkAccount(accountId) {
-    setError('')
-    setResult('')
-    try {
-      const config = { ...(sourceObj.config || {}) }
-      if (accountId) {
-        config.simplefin = { ...(config.simplefin || {}), account_id: accountId }
-      } else {
-        delete config.simplefin
-      }
-      await api.updateSource(sourceObj.name, { config })
-      setSources(await api.getSources())
-      setResult(accountId ? 'Account linked.' : 'Account unlinked.')
-    } catch (err) {
-      setError(err.message)
-    }
-  }
-
-  if (!sourceObj) return <div className="p-4 sm:p-6 text-sm text-muted">Source not found.</div>
-
-  return (
-    <div className="p-4 sm:p-6 max-w-5xl space-y-4">
-          {stats && (
-            <div className="flex gap-4 text-xs">
-              <span className="text-muted"><span className="font-medium text-ink">{stats.total_records}</span> total</span>
-              <span className="text-muted"><span className="font-medium text-ink">{stats.transformed_records}</span> transformed</span>
-              <span className="text-muted"><span className="font-medium text-ink">{stats.pending_records}</span> pending</span>
-            </div>
-          )}
-
-          {/* Bank feed — link this source to a SimpleFIN account */}
-          <Section
-            title="Connection"
-            description="Where this source gets its data. Unlinked sources are filled by CSV upload on the Import page."
-          >
-            <div className="flex items-center gap-3 flex-wrap">
-              {bridgeAccounts === null ? (
-                <>
-                  <span className="text-xs text-muted font-mono">
-                    {sourceObj.config?.simplefin?.account_id || 'not linked'}
-                  </span>
-                  <button
-                    onClick={loadBridgeAccounts}
-                    disabled={bridgeLoading}
-                    className="text-xs border border-line rounded px-2 py-1 text-ink-soft hover:bg-raised hover:border-line disabled:opacity-50"
-                  >
-                    {bridgeLoading ? 'Loading…' : sourceObj.config?.simplefin?.account_id ? 'Change' : 'Link SimpleFIN account'}
-                  </button>
-                </>
-              ) : (
-                <select
-                  value={sourceObj.config?.simplefin?.account_id || ''}
-                  onChange={e => handleLinkAccount(e.target.value)}
-                  className="text-xs border border-line rounded px-2 py-1 bg-surface text-ink-soft"
-                >
-                  <option value="">Not linked</option>
-                  {bridgeAccounts.map(a => (
-                    <option key={a.id} value={a.id}>
-                      {a.name}{a.organization ? ` — ${a.organization}` : ''}{a.balance ? ` (${a.balance})` : ''}
-                    </option>
-                  ))}
-                </select>
-              )}
-            </div>
-
-            {bridgeError && <p className="text-xs text-warn mt-1">{bridgeError}</p>}
-
-            {/* Dedupe depends on the transaction id being the constraint key */}
-            {sourceObj.config?.simplefin?.account_id
-              && sourceObj.constraint_fields?.join(',') !== 'id' && (
-              <p className="text-xs text-warn mt-1">
-                Constraint fields are “{sourceObj.constraint_fields?.join(', ') || 'none'}” — a
-                bank feed should use “id” so re-syncs don’t duplicate rows.
-              </p>
-            )}
-          </Section>
-
-          {/* Unified field table */}
-          {availableFields.length > 0 && (
-            <Section
-              title="Fields and view"
-              description="Every field seen in this source's records. Tick which identify a row for deduplication, and which become columns in the generated view."
-            >
-              <table className="w-full text-xs">
-                <thead>
-                  <tr className="text-left text-muted border-b border-line-soft">
-                    {[
-                      { col: 'key', label: 'Key' },
-                      { col: 'origin', label: 'Origin' },
-                      { col: 'type', label: 'Type' },
-                      { col: 'constraint', label: 'Constraint', center: true },
-                      { col: 'inview', label: 'In view', center: true },
-                      { col: 'seq', label: 'Seq', center: true },
-                    ].map(({ col, label, center }) => (
-                      <th
-                        key={col}
-                        onClick={() => setFieldSort(s => ({ col, dir: s.col === col && s.dir === 'asc' ? 'desc' : 'asc' }))}
-                        className={`pb-1 font-medium cursor-pointer select-none hover:text-ink-soft ${center ? 'text-center' : ''}`}
-                      >
-                        {label}
-                        <span className="ml-1 text-muted">
-                          {fieldSort.col === col ? (fieldSort.dir === 'asc' ? '▲' : '▼') : '⇅'}
-                        </span>
-                      </th>
-                    ))}
-                  </tr>
-                </thead>
-                <tbody>
-                  {[...availableFields].sort((a, b) => {
-                    const constraintList = constraintFields.split(',').map(s => s.trim())
-                    const aSchema = schemaFields.find(sf => sf.name === a.key)
-                    const bSchema = schemaFields.find(sf => sf.name === b.key)
-                    let av, bv
-                    if (fieldSort.col === 'key') { av = a.key; bv = b.key }
-                    else if (fieldSort.col === 'origin') { av = a.origins.join(','); bv = b.origins.join(',') }
-                    else if (fieldSort.col === 'type') { av = aSchema?.type || ''; bv = bSchema?.type || '' }
-                    else if (fieldSort.col === 'constraint') { av = constraintList.includes(a.key) ? 0 : 1; bv = constraintList.includes(b.key) ? 0 : 1 }
-                    else if (fieldSort.col === 'inview') { av = aSchema ? 0 : 1; bv = bSchema ? 0 : 1 }
-                    else if (fieldSort.col === 'seq') { av = aSchema?.seq ?? 999; bv = bSchema?.seq ?? 999 }
-                    if (av < bv) return fieldSort.dir === 'asc' ? -1 : 1
-                    if (av > bv) return fieldSort.dir === 'asc' ? 1 : -1
-                    return 0
-                  }).map(f => {
-                    const isRaw = f.origins.includes('raw')
-                    const constraintChecked = constraintFields.split(',').map(s => s.trim()).includes(f.key)
-                    const schemaEntry = schemaFields.find(sf => sf.name === f.key)
-                    const inView = !!schemaEntry
-                    return (
-                      <tr key={f.key} className="border-t border-line-soft">
-                        <td className="py-1 font-mono text-ink-soft">{f.key}</td>
-                        <td className="py-1 text-muted">{f.origins.join(', ')}</td>
-                        <td className="py-1">
-                          {inView && (
-                            <div className="flex gap-1 items-center">
-                              <select
-                                className="border border-line rounded px-1 py-0.5 text-xs focus:outline-none focus:border-accent"
-                                value={schemaEntry.type}
-                                onChange={e => setSchemaFields(sf =>
-                                  sf.map(s => s.name === f.key ? { ...s, type: e.target.value } : s)
-                                )}
-                              >
-                                {FIELD_TYPES.map(t => <option key={t} value={t}>{t}</option>)}
-                              </select>
-                              <input
-                                className="border border-line rounded px-1 py-0.5 text-xs font-mono w-32 focus:outline-none focus:border-accent"
-                                value={schemaEntry.expression || ''}
-                                placeholder="{field} * {sign}"
-                                onChange={e => setSchemaFields(sf =>
-                                  sf.map(s => s.name === f.key ? { ...s, expression: e.target.value || undefined } : s)
-                                )}
-                              />
-                            </div>
-                          )}
-                        </td>
-                        <td className="py-1 text-center">
-                          {isRaw && (
-                            <input
-                              type="checkbox"
-                              checked={constraintChecked}
-                              onChange={e => {
-                                const current = constraintFields.split(',').map(s => s.trim()).filter(Boolean)
-                                const next = e.target.checked
-                                  ? [...current, f.key]
-                                  : current.filter(k => k !== f.key)
-                                setConstraintFields(next.join(', '))
-                              }}
-                            />
-                          )}
-                        </td>
-                        <td className="py-1 text-center">
-                          <input
-                            type="checkbox"
-                            checked={inView}
-                            onChange={e => {
-                              if (e.target.checked) {
-                                const nextSeq = schemaFields.length > 0
-                                  ? Math.max(...schemaFields.map(s => s.seq ?? 0)) + 1
-                                  : 1
-                                setSchemaFields(sf => [...sf, { name: f.key, type: 'text', seq: nextSeq }])
-                              } else {
-                                setSchemaFields(sf => sf.filter(s => s.name !== f.key))
-                              }
-                            }}
-                          />
-                        </td>
-                        <td className="py-1 text-center">
-                          {inView && (
-                            <input
-                              type="number"
-                              className="w-12 border border-line rounded px-1 py-0.5 text-xs text-center focus:outline-none focus:border-accent"
-                              value={schemaEntry.seq ?? ''}
-                              onChange={e => setSchemaFields(sf =>
-                                sf.map(s => s.name === f.key ? { ...s, seq: parseInt(e.target.value) || 0 } : s)
-                              )}
-                            />
-                          )}
-                        </td>
-                      </tr>
-                    )
-                  })}
-                </tbody>
-              </table>
-
-              <div className="flex items-center gap-3 pt-3 mt-2 border-t border-line-soft flex-wrap">
-                <label className="flex items-center gap-1.5 text-xs text-muted cursor-pointer">
-                  <input type="checkbox" checked={globalPicklist} onChange={e => setGlobalPicklist(e.target.checked)} />
-                  Global picklist
-                </label>
-                <form onSubmit={handleSave}>
-                  <button type="submit" disabled={saving}
-                    className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700 disabled:opacity-50">
-                    {saving ? 'Saving…' : 'Save'}
-                  </button>
-                </form>
-                {schemaFields.length > 0 && (
-                  <>
-                    <button
-                      onClick={handleGenerateView}
-                      disabled={generating}
-                      className="text-xs bg-green-600 text-white px-2 py-1.5 rounded hover:bg-green-700 disabled:opacity-50"
-                    >
-                      {generating ? 'Generating…' : 'Generate view'}
-                    </button>
-                    {viewName && (
-                      <code className="text-xs bg-raised px-2 py-1 rounded text-ink-soft">{viewName}</code>
-                    )}
-                  </>
-                )}
-              </div>
-            </Section>
-          )}
-
-          {/* Save button when no fields loaded yet */}
-          {availableFields.length === 0 && (
-            <Section
-              title="Fields and view"
-              description="No fields yet — they are discovered from imported records. Import or sync data first."
-            >
-              <div className="flex items-center gap-3">
-                <label className="flex items-center gap-1.5 text-xs text-muted cursor-pointer">
-                  <input type="checkbox" checked={globalPicklist} onChange={e => setGlobalPicklist(e.target.checked)} />
-                  Global picklist
-                </label>
-                <form onSubmit={handleSave}>
-                  <button type="submit" disabled={saving}
-                    className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700 disabled:opacity-50">
-                    {saving ? 'Saving…' : 'Save'}
-                  </button>
-                </form>
-              </div>
-            </Section>
-          )}
-
-          {sampleRows.length > 0 && (
-            <Section title="Sample rows" description="The most recent imported records, as stored.">
-              <SampleTable rows={sampleRows} />
-            </Section>
-          )}
-
-          <Section title="Maintenance">
-            <div className="flex items-center gap-3">
-              <button
-                onClick={handleReprocess}
-                disabled={reprocessing}
-                className="text-sm bg-orange-500 text-white px-3 py-1.5 rounded hover:bg-orange-600 disabled:opacity-50"
-              >
-                {reprocessing ? 'Reprocessing…' : 'Reprocess all records'}
-              </button>
-              <span className="text-xs text-muted">Clears and reruns all transformation rules</span>
-            </div>
-          </Section>
-
-          {result && <p className="text-xs text-ok">{result}</p>}
-          {error && <p className="text-xs text-danger">{error}</p>}
-
-          <Section title="Delete source" description="Removes the source and every record, rule, and mapping belonging to it.">
-            <button onClick={handleDelete}
-              className="text-sm border border-danger-line text-danger px-3 py-1.5 rounded hover:bg-danger-soft hover:border-danger-line">
-              Delete source
-            </button>
-          </Section>
-    </div>
-  )
-}

ui/src/pages/SourceList.jsx

@@ -1,388 +0,0 @@
-import { useState, useRef } from 'react'
-import { useNavigate } from 'react-router-dom'
-import { api } from '../api'
-import SampleTable from '../components/SampleTable.jsx'
-
-const FIELD_TYPES = ['text', 'numeric', 'date']
-
-// Ticked into the view by default when a bank feed sample contains them
-const FEED_DEFAULT_VIEW = ['date', 'description', 'payee', 'amount']
-
-export default function SourceList({ sources, setSources, setSource }) {
-  const navigate = useNavigate()
-  const [creating, setCreating] = useState(false)
-  const [form, setForm] = useState({ name: '', constraint_fields: '', fields: [], schema: [], importSample: true })
-  const [createError, setCreateError] = useState('')
-  const [createLoading, setCreateLoading] = useState(false)
-  const [csvFileName, setCsvFileName] = useState('')
-  const [bridgeAccounts, setBridgeAccounts] = useState(null)
-  const [bridgeLoading, setBridgeLoading] = useState(false)
-  const [bridgeError, setBridgeError] = useState('')
-  const [sampleInfo, setSampleInfo] = useState(null)
-  const fileRef = useRef()
-
-  async function loadBridgeAccounts() {
-    setBridgeLoading(true)
-    setBridgeError('')
-    try {
-      const res = await api.getSimpleFinAccounts()
-      setBridgeAccounts(res.accounts || [])
-      if (res.errors?.length) setBridgeError(res.errors.join('; '))
-    } catch (err) {
-      setBridgeError(err.message)
-    } finally {
-      setBridgeLoading(false)
-    }
-  }
-
-  // Writes only config — constraint_fields and global_picklist are left NULL so
-  // update_source keeps whatever is already stored.
-  async function handleSelectFeedAccount(accountId) {
-    if (!accountId) {
-      // Clearing the feed only resets fields we populated, not a loaded CSV
-      setForm(f => csvFileName ? { ...f, simplefin_account_id: '' } : {
-        ...f, simplefin_account_id: '', fields: [], schema: [], constraint_fields: '', sampleRows: [],
-      })
-      setSampleInfo(null)
-      return
-    }
-
-    setForm(f => ({ ...f, simplefin_account_id: accountId }))
-    setBridgeLoading(true)
-    setBridgeError('')
-    try {
-      const res = await api.getSimpleFinSample(accountId)
-      const names = res.fields.map(f => f.name)
-      setSampleInfo({ fetched: res.fetched, fields: res.fields.length })
-      if (res.errors?.length) setBridgeError(res.errors.join('; '))
-      setForm(f => ({
-        ...f,
-        fields: res.fields,
-        sampleRows: res.sampleRows || [],
-        schema: FEED_DEFAULT_VIEW.filter(n => names.includes(n)).map((name, i) => ({
-          name, type: res.fields.find(sf => sf.name === name).type, seq: i + 1,
-        })),
-        // Only default the constraint if the sample actually has an id
-        constraint_fields: f.constraint_fields || (names.includes('id') ? 'id' : ''),
-      }))
-    } catch (err) {
-      setBridgeError(err.message)
-    } finally {
-      setBridgeLoading(false)
-    }
-  }
-
-  async function handleSuggest(e) {
-    const file = e.target.files[0]
-    if (!file) return
-    setCsvFileName(file.name)
-    try {
-      const suggestion = await api.suggestSource(file)
-      setForm(f => ({
-        ...f,
-        fields: suggestion.fields,
-        constraint_fields: '',
-        schema: suggestion.fields.map(f => ({ name: f.name, type: f.type, seq: suggestion.fields.indexOf(f) + 1 })),
-        sampleRows: suggestion.sampleRows || []
-      }))
-    } catch (err) {
-      setCreateError(err.message)
-    }
-  }
-
-  async function handleCreate(e) {
-    e.preventDefault()
-    setCreateError('')
-    const constraintArr = form.constraint_fields.split(',').map(s => s.trim()).filter(Boolean)
-    if (!form.name || constraintArr.length === 0) {
-      setCreateError('Name and at least one constraint field required')
-      return
-    }
-    setCreateLoading(true)
-    try {
-      const config = form.schema.length > 0 ? { fields: form.schema } : {}
-      if (form.simplefin_account_id) {
-        config.simplefin = { account_id: form.simplefin_account_id }
-      }
-      await api.createSource({ name: form.name, constraint_fields: constraintArr, config, global_picklist: form.global_picklist !== false })
-      if (form.schema.length > 0) {
-        await api.generateView(form.name)
-      }
-      if (form.importSample && fileRef.current?.files[0]) {
-        await api.importCSV(form.name, fileRef.current.files[0])
-      }
-      const updated = await api.getSources()
-      setSources(updated)
-      setSource(form.name)
-      setForm({ name: '', constraint_fields: '', fields: [], schema: [], importSample: true, simplefin_account_id: '' })
-      setCreating(false)
-    } catch (err) {
-      setCreateError(err.message)
-    } finally {
-      setCreateLoading(false)
-    }
-  }
-
-  return (
-    <div className="p-4 sm:p-6 max-w-5xl">
-      <div className="flex items-center justify-between mb-6">
-        <h1 className="text-xl font-semibold text-ink">Sources</h1>
-        {!creating && (
-          <button
-            onClick={() => { setCreating(true); setCreateError('') }}
-            className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700"
-          >
-            New source
-          </button>
-        )}
-      </div>
-
-      {!creating && sources.length === 0 && (
-        <p className="text-sm text-muted">No sources yet. Create one to get started.</p>
-      )}
-
-      {!creating && sources.length > 0 && (
-        <div className="bg-surface border border-line rounded divide-y divide-line-soft">
-          {sources.map(s => (
-            <button
-              key={s.name}
-              onClick={() => { setSource(s.name); navigate(`/sources/${encodeURIComponent(s.name)}`) }}
-              className="w-full text-left px-4 py-3 hover:bg-raised flex items-center gap-3"
-            >
-              <span className="text-sm font-medium text-ink flex-1">{s.name}</span>
-              {s.config?.simplefin?.account_id && (
-                <span className="text-xs bg-accent-soft text-accent border border-accent-line rounded px-1.5 py-0.5">
-                  bank feed
-                </span>
-              )}
-              <span className="text-xs text-muted">
-                {(s.constraint_fields || []).join(', ') || 'no constraint'}
-              </span>
-              <span className="text-muted">›</span>
-            </button>
-          ))}
-        </div>
-      )}
-
-      {creating && (
-        <div className="bg-surface border border-line rounded p-4">
-          <h2 className="text-sm font-semibold text-ink-soft mb-3">New source</h2>
-
-          <div className="mb-4">
-            <input type="file" accept=".csv" ref={fileRef} onChange={handleSuggest} className="hidden" />
-            <button
-              type="button"
-              onClick={() => fileRef.current?.click()}
-              className="text-sm border border-line rounded px-3 py-1.5 text-ink-soft hover:bg-raised hover:border-line"
-            >
-              {csvFileName || 'Choose CSV…'}
-            </button>
-          </div>
-
-          <form onSubmit={handleCreate} className="space-y-3">
-            <div>
-              <label className="text-xs text-muted block mb-1">Source name</label>
-              <input
-                className="w-full border border-line rounded px-3 py-1.5 text-sm focus:outline-none focus:border-accent"
-                value={form.name}
-                onChange={e => setForm(f => ({ ...f, name: e.target.value }))}
-                placeholder="e.g. chase, dcard"
-              />
-            </div>
-
-            {/* Bank feed — optional; picking an account defaults the constraint
-                field to the transaction id, which is what dedupe needs */}
-            <div>
-              <label className="text-xs text-muted block mb-1">Bank feed (optional)</label>
-              <div className="flex items-center gap-3 flex-wrap">
-                {bridgeAccounts === null ? (
-                  <button
-                    type="button"
-                    onClick={loadBridgeAccounts}
-                    disabled={bridgeLoading}
-                    className="text-sm border border-line rounded px-3 py-1.5 text-ink-soft hover:bg-raised hover:border-line disabled:opacity-50"
-                  >
-                    {bridgeLoading ? 'Loading…' : 'Link SimpleFIN account…'}
-                  </button>
-                ) : (
-                  <select
-                    value={form.simplefin_account_id || ''}
-                    onChange={e => handleSelectFeedAccount(e.target.value)}
-                    className="text-sm border border-line rounded px-3 py-1.5 bg-surface text-ink-soft"
-                  >
-                    <option value="">No bank feed — CSV import</option>
-                    {bridgeAccounts.map(a => (
-                      <option key={a.id} value={a.id}>
-                        {a.name}{a.organization ? ` — ${a.organization}` : ''}{a.balance ? ` (${a.balance})` : ''}
-                      </option>
-                    ))}
-                  </select>
-                )}
-              </div>
-              {bridgeError && <p className="text-xs text-warn mt-1">{bridgeError}</p>}
-
-              {form.simplefin_account_id && sampleInfo && (
-                <div className="mt-2 bg-accent-soft border border-accent-line rounded p-3 text-xs text-ink-soft space-y-1">
-                  <p>
-                    Read {sampleInfo.fetched} transaction{sampleInfo.fetched === 1 ? '' : 's'} from this
-                    account and found {sampleInfo.fields} field{sampleInfo.fields === 1 ? '' : 's'}.
-                    The table below lists what this account actually returns — fields it never sends
-                    won&rsquo;t appear.
-                  </p>
-                  {form.constraint_fields === 'id' && (
-                    <p>
-                      <span className="font-mono text-ink-soft">id</span> is checked as the constraint
-                      field because it is SimpleFIN&rsquo;s own transaction identifier. Syncs pull an
-                      overlapping window of days, so the same transaction arrives more than once —
-                      matching on <span className="font-mono text-ink-soft">id</span> skips the repeats
-                      while still keeping genuinely separate charges that share a date, amount, and
-                      description.
-                    </p>
-                  )}
-                  {sampleInfo.fetched === 0 && (
-                    <p className="text-warn">
-                      No transactions came back, so there was nothing to infer fields from. Sync first,
-                      then set the fields up here.
-                    </p>
-                  )}
-                </div>
-              )}
-            </div>
-
-            {form.fields.length > 0 && (
-              <div className="pt-2 border-t border-line-soft space-y-2">
-                <table className="w-full text-xs">
-                  <thead>
-                    <tr className="text-left text-muted border-b border-line-soft">
-                      <th className="pb-1 font-medium">Key</th>
-                      <th className="pb-1 font-medium">Type</th>
-                      <th className="pb-1 font-medium text-center">Constraint</th>
-                      <th className="pb-1 font-medium text-center">In view</th>
-                      <th className="pb-1 font-medium text-center">Seq</th>
-                    </tr>
-                  </thead>
-                  <tbody>
-                    {form.fields.map(f => {
-                      const schemaEntry = form.schema.find(s => s.name === f.name)
-                      const inView = !!schemaEntry
-                      const currentType = schemaEntry?.type || f.type
-                      return (
-                        <tr key={f.name} className="border-t border-line-soft">
-                          <td className="py-1 font-mono text-ink-soft">{f.name}</td>
-                          <td className="py-1">
-                            {inView && (
-                              <select
-                                className="border border-line rounded px-1 py-0.5 text-xs focus:outline-none focus:border-accent"
-                                value={currentType}
-                                onChange={e => setForm(ff => ({
-                                  ...ff,
-                                  schema: ff.schema.map(s => s.name === f.name ? { ...s, type: e.target.value } : s)
-                                }))}
-                              >
-                                {FIELD_TYPES.map(t => <option key={t} value={t}>{t}</option>)}
-                              </select>
-                            )}
-                          </td>
-                          <td className="py-1 text-center">
-                            <input
-                              type="checkbox"
-                              checked={form.constraint_fields.split(',').map(s => s.trim()).includes(f.name)}
-                              onChange={e => {
-                                const current = form.constraint_fields.split(',').map(s => s.trim()).filter(Boolean)
-                                const next = e.target.checked
-                                  ? [...current, f.name]
-                                  : current.filter(n => n !== f.name)
-                                setForm(ff => ({ ...ff, constraint_fields: next.join(', ') }))
-                              }}
-                            />
-                          </td>
-                          <td className="py-1 text-center">
-                            <input
-                              type="checkbox"
-                              checked={inView}
-                              onChange={e => {
-                                if (e.target.checked) {
-                                  const nextSeq = form.schema.length > 0
-                                    ? Math.max(...form.schema.map(s => s.seq ?? 0)) + 1
-                                    : 1
-                                  setForm(ff => ({ ...ff, schema: [...ff.schema, { name: f.name, type: f.type, seq: nextSeq }] }))
-                                } else {
-                                  setForm(ff => ({ ...ff, schema: ff.schema.filter(s => s.name !== f.name) }))
-                                }
-                              }}
-                            />
-                          </td>
-                          <td className="py-1 text-center">
-                            {inView && (
-                              <input
-                                type="number"
-                                className="w-12 border border-line rounded px-1 py-0.5 text-xs text-center focus:outline-none focus:border-accent"
-                                value={schemaEntry.seq ?? ''}
-                                onChange={e => setForm(ff => ({
-                                  ...ff,
-                                  schema: ff.schema.map(s => s.name === f.name ? { ...s, seq: parseInt(e.target.value) || 0 } : s)
-                                }))}
-                              />
-                            )}
-                          </td>
-                        </tr>
-                      )
-                    })}
-                  </tbody>
-                </table>
-                <SampleTable rows={form.sampleRows || []} />
-              </div>
-            )}
-
-            {form.fields.length === 0 && (
-              <div>
-                <label className="text-xs text-muted block mb-1">Constraint fields (comma-separated)</label>
-                <input
-                  className="w-full border border-line rounded px-3 py-1.5 text-sm focus:outline-none focus:border-accent"
-                  value={form.constraint_fields}
-                  onChange={e => setForm(f => ({ ...f, constraint_fields: e.target.value }))}
-                  placeholder="e.g. date, amount, description"
-                />
-              </div>
-            )}
-
-            <div className="flex gap-4">
-              <label className="flex items-center gap-1.5 text-xs text-muted cursor-pointer">
-                <input
-                  type="checkbox"
-                  checked={form.global_picklist !== false}
-                  onChange={e => setForm(f => ({ ...f, global_picklist: e.target.checked }))}
-                />
-                Global picklist
-              </label>
-              {form.fields.length > 0 && (
-                <label className="flex items-center gap-1.5 text-xs text-muted cursor-pointer">
-                  <input
-                    type="checkbox"
-                    checked={form.importSample !== false}
-                    onChange={e => setForm(f => ({ ...f, importSample: e.target.checked }))}
-                  />
-                  Import sample data
-                </label>
-              )}
-            </div>
-
-            {createError && <p className="text-xs text-danger">{createError}</p>}
-
-            <div className="flex gap-2">
-              <button type="submit" disabled={createLoading}
-                className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700 disabled:opacity-50">
-                {createLoading ? 'Creating…' : 'Create'}
-              </button>
-              <button type="button"
-                onClick={() => { setCreating(false); setCreateError(''); setForm({ name: '', constraint_fields: '', fields: [], schema: [] }) }}
-                className="text-sm text-muted px-3 py-1.5 rounded hover:bg-raised">
-                Cancel
-              </button>
-            </div>
-          </form>
-        </div>
-      )}
-    </div>
-  )
-}

ui/src/pages/Sources.jsx

@@ -0,0 +1,590 @@
+import { useState, useEffect, useRef } from 'react'
+import { useSearchParams } from 'react-router-dom'
+import { api } from '../api'
+
+const FIELD_TYPES = ['text', 'numeric', 'date']
+
+function SampleTable({ rows }) {
+  if (!rows || rows.length === 0) return null
+  const cols = Object.keys(rows[0])
+  return (
+    <div className="overflow-auto border border-gray-100 rounded bg-gray-50 max-h-36">
+      <table className="text-xs w-full">
+        <thead>
+          <tr className="text-left text-gray-400 border-b border-gray-100 bg-gray-50 sticky top-0">
+            {cols.map(c => <th key={c} className="px-2 py-1 font-medium whitespace-nowrap">{c}</th>)}
+          </tr>
+        </thead>
+        <tbody>
+          {rows.map((row, i) => (
+            <tr key={i} className="border-t border-gray-100">
+              {cols.map(c => (
+                <td key={c} className="px-2 py-1 whitespace-nowrap text-gray-600 max-w-32 truncate font-mono">
+                  {row[c] == null ? <span className="text-gray-300">—</span> : String(row[c])}
+                </td>
+              ))}
+            </tr>
+          ))}
+        </tbody>
+      </table>
+    </div>
+  )
+}
+
+export default function Sources({ source, sources, setSources, setSource }) {
+  const [constraintFields, setConstraintFields] = useState('')
+  const [globalPicklist, setGlobalPicklist] = useState(true)
+  const [schemaFields, setSchemaFields] = useState([])
+  const [stats, setStats] = useState(null)
+  const [sampleRows, setSampleRows] = useState([])
+  const [saving, setSaving] = useState(false)
+  const [reprocessing, setReprocessing] = useState(false)
+  const [generating, setGenerating] = useState(false)
+  const [result, setResult] = useState('')
+  const [error, setError] = useState('')
+  const [viewName, setViewName] = useState('')
+  const [availableFields, setAvailableFields] = useState([])
+  const [fieldSort, setFieldSort] = useState({ col: 'key', dir: 'asc' })
+  const [creating, setCreating] = useState(false)
+  const [form, setForm] = useState({ name: '', constraint_fields: '', fields: [], schema: [], importSample: true })
+  const [createError, setCreateError] = useState('')
+  const [createLoading, setCreateLoading] = useState(false)
+  const [csvFileName, setCsvFileName] = useState('')
+  const fileRef = useRef()
+
+  const [searchParams, setSearchParams] = useSearchParams()
+  const sourceObj = sources.find(s => s.name === source)
+
+  useEffect(() => {
+    if (searchParams.get('new') === '1') {
+      setCreating(true)
+      setSearchParams({})
+    }
+  }, [searchParams])
+
+  useEffect(() => {
+    if (!sourceObj) return
+    setConstraintFields(sourceObj.constraint_fields?.join(', ') || '')
+    setGlobalPicklist(sourceObj.global_picklist !== false)
+    setSchemaFields((sourceObj.config?.fields || []).map((f, i) => ({ seq: i + 1, ...f })))
+    setViewName(sourceObj.config?.fields?.length ? `dfv.${sourceObj.name}` : '')
+    setResult('')
+    setError('')
+    setStats(null)
+    setAvailableFields([])
+    setSampleRows([])
+    api.getStats(sourceObj.name).then(setStats).catch(() => {})
+    api.getFields(sourceObj.name).then(setAvailableFields).catch(() => {})
+    api.getRecords(sourceObj.name, 50).then(rows => setSampleRows(rows.map(r => r.data).filter(Boolean))).catch(() => {})
+  }, [source, sourceObj?.name])
+
+  async function handleSave(e) {
+    e.preventDefault()
+    setSaving(true)
+    setError('')
+    try {
+      const constraint_fields = constraintFields.split(',').map(s => s.trim()).filter(Boolean)
+      const fields = [...schemaFields.filter(f => f.name)].sort((a, b) => (a.seq ?? 0) - (b.seq ?? 0))
+      const config = { ...(sourceObj.config || {}), fields }
+      await api.updateSource(sourceObj.name, { constraint_fields, config, global_picklist: globalPicklist })
+      if (fields.length > 0) {
+        const res = await api.generateView(sourceObj.name)
+        if (res.success) setViewName(res.view)
+      }
+      const updated = await api.getSources()
+      setSources(updated)
+      setResult('Saved.')
+    } catch (err) {
+      setError(err.message)
+    } finally {
+      setSaving(false)
+    }
+  }
+
+  async function handleGenerateView() {
+    setGenerating(true)
+    setResult('')
+    setError('')
+    try {
+      const constraint_fields = constraintFields.split(',').map(s => s.trim()).filter(Boolean)
+      const fields = [...schemaFields.filter(f => f.name)].sort((a, b) => (a.seq ?? 0) - (b.seq ?? 0))
+      const config = { ...(sourceObj.config || {}), fields }
+      await api.updateSource(sourceObj.name, { constraint_fields, config, global_picklist: globalPicklist })
+      const res = await api.generateView(sourceObj.name)
+      if (res.success) {
+        setViewName(res.view)
+        setResult(`View created: ${res.view}`)
+      } else {
+        setError(res.error)
+      }
+    } catch (err) {
+      setError(err.message)
+    } finally {
+      setGenerating(false)
+    }
+  }
+
+  async function handleReprocess() {
+    if (!confirm(`Reprocess all records for "${sourceObj.name}"? This will clear and reapply all transformations.`)) return
+    setReprocessing(true)
+    setResult('')
+    setError('')
+    try {
+      const res = await api.reprocess(sourceObj.name)
+      setResult(`Reprocessed ${res.transformed} records.`)
+      api.getStats(sourceObj.name).then(setStats).catch(() => {})
+    } catch (err) {
+      setError(err.message)
+    } finally {
+      setReprocessing(false)
+    }
+  }
+
+  async function handleDelete() {
+    if (!confirm(`Delete source "${sourceObj.name}" and all its data?`)) return
+    try {
+      await api.deleteSource(sourceObj.name)
+      const updated = await api.getSources()
+      setSources(updated)
+      if (updated.length > 0) setSource(updated[0].name)
+      else setSource('')
+    } catch (err) {
+      alert(err.message)
+    }
+  }
+
+  async function handleSuggest(e) {
+    const file = e.target.files[0]
+    if (!file) return
+    setCsvFileName(file.name)
+    try {
+      const suggestion = await api.suggestSource(file)
+      setForm(f => ({
+        ...f,
+        fields: suggestion.fields,
+        constraint_fields: '',
+        schema: suggestion.fields.map(f => ({ name: f.name, type: f.type, seq: suggestion.fields.indexOf(f) + 1 })),
+        sampleRows: suggestion.sampleRows || []
+      }))
+    } catch (err) {
+      setCreateError(err.message)
+    }
+  }
+
+  async function handleCreate(e) {
+    e.preventDefault()
+    setCreateError('')
+    const constraintArr = form.constraint_fields.split(',').map(s => s.trim()).filter(Boolean)
+    if (!form.name || constraintArr.length === 0) {
+      setCreateError('Name and at least one constraint field required')
+      return
+    }
+    setCreateLoading(true)
+    try {
+      const config = form.schema.length > 0 ? { fields: form.schema } : {}
+      await api.createSource({ name: form.name, constraint_fields: constraintArr, config, global_picklist: form.global_picklist !== false })
+      if (form.schema.length > 0) {
+        await api.generateView(form.name)
+      }
+      if (form.importSample && fileRef.current?.files[0]) {
+        await api.importCSV(form.name, fileRef.current.files[0])
+      }
+      const updated = await api.getSources()
+      setSources(updated)
+      setSource(form.name)
+      setForm({ name: '', constraint_fields: '', fields: [], schema: [], importSample: true })
+      setCreating(false)
+    } catch (err) {
+      setCreateError(err.message)
+    } finally {
+      setCreateLoading(false)
+    }
+  }
+
+  return (
+    <div className="p-6 max-w-5xl">
+      <div className="flex items-center justify-between mb-6">
+        <h1 className="text-xl font-semibold text-gray-800">
+          {sourceObj ? sourceObj.name : 'Sources'}
+        </h1>
+        <button
+          onClick={() => { setCreating(true); setCreateError('') }}
+          className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700"
+        >
+          New source
+        </button>
+      </div>
+
+      {/* No source selected */}
+      {!sourceObj && !creating && (
+        <p className="text-sm text-gray-400">No sources yet. Create one to get started.</p>
+      )}
+
+      {/* Source detail */}
+      {sourceObj && !creating && (
+        <div className="space-y-4">
+          {/* Stats */}
+          {stats && (
+            <div className="flex gap-4 text-xs">
+              <span className="text-gray-500"><span className="font-medium text-gray-800">{stats.total_records}</span> total</span>
+              <span className="text-gray-500"><span className="font-medium text-gray-800">{stats.transformed_records}</span> transformed</span>
+              <span className="text-gray-500"><span className="font-medium text-gray-800">{stats.pending_records}</span> pending</span>
+            </div>
+          )}
+
+          {/* Unified field table */}
+          {availableFields.length > 0 && (
+            <div className="pt-2 border-t border-gray-100 space-y-2">
+              <table className="w-full text-xs">
+                <thead>
+                  <tr className="text-left text-gray-400 border-b border-gray-100">
+                    {[
+                      { col: 'key', label: 'Key' },
+                      { col: 'origin', label: 'Origin' },
+                      { col: 'type', label: 'Type' },
+                      { col: 'constraint', label: 'Constraint', center: true },
+                      { col: 'inview', label: 'In view', center: true },
+                      { col: 'seq', label: 'Seq', center: true },
+                    ].map(({ col, label, center }) => (
+                      <th
+                        key={col}
+                        onClick={() => setFieldSort(s => ({ col, dir: s.col === col && s.dir === 'asc' ? 'desc' : 'asc' }))}
+                        className={`pb-1 font-medium cursor-pointer select-none hover:text-gray-600 ${center ? 'text-center' : ''}`}
+                      >
+                        {label}
+                        <span className="ml-1 text-gray-300">
+                          {fieldSort.col === col ? (fieldSort.dir === 'asc' ? '▲' : '▼') : '⇅'}
+                        </span>
+                      </th>
+                    ))}
+                  </tr>
+                </thead>
+                <tbody>
+                  {[...availableFields].sort((a, b) => {
+                    const constraintList = constraintFields.split(',').map(s => s.trim())
+                    const aSchema = schemaFields.find(sf => sf.name === a.key)
+                    const bSchema = schemaFields.find(sf => sf.name === b.key)
+                    let av, bv
+                    if (fieldSort.col === 'key') { av = a.key; bv = b.key }
+                    else if (fieldSort.col === 'origin') { av = a.origins.join(','); bv = b.origins.join(',') }
+                    else if (fieldSort.col === 'type') { av = aSchema?.type || ''; bv = bSchema?.type || '' }
+                    else if (fieldSort.col === 'constraint') { av = constraintList.includes(a.key) ? 0 : 1; bv = constraintList.includes(b.key) ? 0 : 1 }
+                    else if (fieldSort.col === 'inview') { av = aSchema ? 0 : 1; bv = bSchema ? 0 : 1 }
+                    else if (fieldSort.col === 'seq') { av = aSchema?.seq ?? 999; bv = bSchema?.seq ?? 999 }
+                    if (av < bv) return fieldSort.dir === 'asc' ? -1 : 1
+                    if (av > bv) return fieldSort.dir === 'asc' ? 1 : -1
+                    return 0
+                  }).map(f => {
+                    const isRaw = f.origins.includes('raw')
+                    const constraintChecked = constraintFields.split(',').map(s => s.trim()).includes(f.key)
+                    const schemaEntry = schemaFields.find(sf => sf.name === f.key)
+                    const inView = !!schemaEntry
+                    return (
+                      <tr key={f.key} className="border-t border-gray-50">
+                        <td className="py-1 font-mono text-gray-700">{f.key}</td>
+                        <td className="py-1 text-gray-400">{f.origins.join(', ')}</td>
+                        <td className="py-1">
+                          {inView && (
+                            <div className="flex gap-1 items-center">
+                              <select
+                                className="border border-gray-200 rounded px-1 py-0.5 text-xs focus:outline-none focus:border-blue-400"
+                                value={schemaEntry.type}
+                                onChange={e => setSchemaFields(sf =>
+                                  sf.map(s => s.name === f.key ? { ...s, type: e.target.value } : s)
+                                )}
+                              >
+                                {FIELD_TYPES.map(t => <option key={t} value={t}>{t}</option>)}
+                              </select>
+                              <input
+                                className="border border-gray-200 rounded px-1 py-0.5 text-xs font-mono w-32 focus:outline-none focus:border-blue-400"
+                                value={schemaEntry.expression || ''}
+                                placeholder="{field} * {sign}"
+                                onChange={e => setSchemaFields(sf =>
+                                  sf.map(s => s.name === f.key ? { ...s, expression: e.target.value || undefined } : s)
+                                )}
+                              />
+                            </div>
+                          )}
+                        </td>
+                        <td className="py-1 text-center">
+                          {isRaw && (
+                            <input
+                              type="checkbox"
+                              checked={constraintChecked}
+                              onChange={e => {
+                                const current = constraintFields.split(',').map(s => s.trim()).filter(Boolean)
+                                const next = e.target.checked
+                                  ? [...current, f.key]
+                                  : current.filter(k => k !== f.key)
+                                setConstraintFields(next.join(', '))
+                              }}
+                            />
+                          )}
+                        </td>
+                        <td className="py-1 text-center">
+                          <input
+                            type="checkbox"
+                            checked={inView}
+                            onChange={e => {
+                              if (e.target.checked) {
+                                const nextSeq = schemaFields.length > 0
+                                  ? Math.max(...schemaFields.map(s => s.seq ?? 0)) + 1
+                                  : 1
+                                setSchemaFields(sf => [...sf, { name: f.key, type: 'text', seq: nextSeq }])
+                              } else {
+                                setSchemaFields(sf => sf.filter(s => s.name !== f.key))
+                              }
+                            }}
+                          />
+                        </td>
+                        <td className="py-1 text-center">
+                          {inView && (
+                            <input
+                              type="number"
+                              className="w-12 border border-gray-200 rounded px-1 py-0.5 text-xs text-center focus:outline-none focus:border-blue-400"
+                              value={schemaEntry.seq ?? ''}
+                              onChange={e => setSchemaFields(sf =>
+                                sf.map(s => s.name === f.key ? { ...s, seq: parseInt(e.target.value) || 0 } : s)
+                              )}
+                            />
+                          )}
+                        </td>
+                      </tr>
+                    )
+                  })}
+                </tbody>
+              </table>
+
+              <div className="flex items-center gap-3 pt-1 flex-wrap">
+                <label className="flex items-center gap-1.5 text-xs text-gray-500 cursor-pointer">
+                  <input type="checkbox" checked={globalPicklist} onChange={e => setGlobalPicklist(e.target.checked)} />
+                  Global picklist
+                </label>
+                <form onSubmit={handleSave}>
+                  <button type="submit" disabled={saving}
+                    className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700 disabled:opacity-50">
+                    {saving ? 'Saving…' : 'Save'}
+                  </button>
+                </form>
+                {schemaFields.length > 0 && (
+                  <>
+                    <button
+                      onClick={handleGenerateView}
+                      disabled={generating}
+                      className="text-xs bg-green-600 text-white px-2 py-1.5 rounded hover:bg-green-700 disabled:opacity-50"
+                    >
+                      {generating ? 'Generating…' : 'Generate view'}
+                    </button>
+                    {viewName && (
+                      <code className="text-xs bg-gray-100 px-2 py-1 rounded text-gray-600">{viewName}</code>
+                    )}
+                  </>
+                )}
+              </div>
+              <SampleTable rows={sampleRows} />
+            </div>
+          )}
+
+          {/* Save button when no fields loaded yet */}
+          {availableFields.length === 0 && (
+            <div className="flex items-center gap-3">
+              <label className="flex items-center gap-1.5 text-xs text-gray-500 cursor-pointer">
+                <input type="checkbox" checked={globalPicklist} onChange={e => setGlobalPicklist(e.target.checked)} />
+                Global picklist
+              </label>
+              <form onSubmit={handleSave}>
+                <button type="submit" disabled={saving}
+                  className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700 disabled:opacity-50">
+                  {saving ? 'Saving…' : 'Save'}
+                </button>
+              </form>
+            </div>
+          )}
+
+          {/* Reprocess */}
+          <div className="flex items-center gap-3 pt-2 border-t border-gray-100">
+            <button
+              onClick={handleReprocess}
+              disabled={reprocessing}
+              className="text-sm bg-orange-500 text-white px-3 py-1.5 rounded hover:bg-orange-600 disabled:opacity-50"
+            >
+              {reprocessing ? 'Reprocessing…' : 'Reprocess all records'}
+            </button>
+            <span className="text-xs text-gray-400">Clears and reruns all transformation rules</span>
+          </div>
+
+          {result && <p className="text-xs text-green-600">{result}</p>}
+          {error && <p className="text-xs text-red-500">{error}</p>}
+
+          <div className="pt-2 border-t border-gray-100">
+            <button onClick={handleDelete} className="text-xs text-red-400 hover:text-red-600">
+              Delete source
+            </button>
+          </div>
+        </div>
+      )}
+
+      {/* Create form */}
+      {creating && (
+        <div className="bg-white border border-gray-200 rounded p-4">
+          <h2 className="text-sm font-semibold text-gray-700 mb-3">New source</h2>
+
+          <div className="mb-4">
+            <input type="file" accept=".csv" ref={fileRef} onChange={handleSuggest} className="hidden" />
+            <button
+              type="button"
+              onClick={() => fileRef.current?.click()}
+              className="text-sm border border-gray-300 rounded px-3 py-1.5 text-gray-600 hover:bg-gray-50 hover:border-gray-400"
+            >
+              {csvFileName || 'Choose CSV…'}
+            </button>
+          </div>
+
+          <form onSubmit={handleCreate} className="space-y-3">
+            <div>
+              <label className="text-xs text-gray-500 block mb-1">Source name</label>
+              <input
+                className="w-full border border-gray-200 rounded px-3 py-1.5 text-sm focus:outline-none focus:border-blue-400"
+                value={form.name}
+                onChange={e => setForm(f => ({ ...f, name: e.target.value }))}
+                placeholder="e.g. chase, dcard"
+              />
+            </div>
+
+            {form.fields.length > 0 && (
+              <div className="pt-2 border-t border-gray-100 space-y-2">
+                <table className="w-full text-xs">
+                  <thead>
+                    <tr className="text-left text-gray-400 border-b border-gray-100">
+                      <th className="pb-1 font-medium">Key</th>
+                      <th className="pb-1 font-medium">Type</th>
+                      <th className="pb-1 font-medium text-center">Constraint</th>
+                      <th className="pb-1 font-medium text-center">In view</th>
+                      <th className="pb-1 font-medium text-center">Seq</th>
+                    </tr>
+                  </thead>
+                  <tbody>
+                    {form.fields.map(f => {
+                      const schemaEntry = form.schema.find(s => s.name === f.name)
+                      const inView = !!schemaEntry
+                      const currentType = schemaEntry?.type || f.type
+                      return (
+                        <tr key={f.name} className="border-t border-gray-50">
+                          <td className="py-1 font-mono text-gray-700">{f.name}</td>
+                          <td className="py-1">
+                            {inView && (
+                              <select
+                                className="border border-gray-200 rounded px-1 py-0.5 text-xs focus:outline-none focus:border-blue-400"
+                                value={currentType}
+                                onChange={e => setForm(ff => ({
+                                  ...ff,
+                                  schema: ff.schema.map(s => s.name === f.name ? { ...s, type: e.target.value } : s)
+                                }))}
+                              >
+                                {FIELD_TYPES.map(t => <option key={t} value={t}>{t}</option>)}
+                              </select>
+                            )}
+                          </td>
+                          <td className="py-1 text-center">
+                            <input
+                              type="checkbox"
+                              checked={form.constraint_fields.split(',').map(s => s.trim()).includes(f.name)}
+                              onChange={e => {
+                                const current = form.constraint_fields.split(',').map(s => s.trim()).filter(Boolean)
+                                const next = e.target.checked
+                                  ? [...current, f.name]
+                                  : current.filter(n => n !== f.name)
+                                setForm(ff => ({ ...ff, constraint_fields: next.join(', ') }))
+                              }}
+                            />
+                          </td>
+                          <td className="py-1 text-center">
+                            <input
+                              type="checkbox"
+                              checked={inView}
+                              onChange={e => {
+                                if (e.target.checked) {
+                                  const nextSeq = form.schema.length > 0
+                                    ? Math.max(...form.schema.map(s => s.seq ?? 0)) + 1
+                                    : 1
+                                  setForm(ff => ({ ...ff, schema: [...ff.schema, { name: f.name, type: f.type, seq: nextSeq }] }))
+                                } else {
+                                  setForm(ff => ({ ...ff, schema: ff.schema.filter(s => s.name !== f.name) }))
+                                }
+                              }}
+                            />
+                          </td>
+                          <td className="py-1 text-center">
+                            {inView && (
+                              <input
+                                type="number"
+                                className="w-12 border border-gray-200 rounded px-1 py-0.5 text-xs text-center focus:outline-none focus:border-blue-400"
+                                value={schemaEntry.seq ?? ''}
+                                onChange={e => setForm(ff => ({
+                                  ...ff,
+                                  schema: ff.schema.map(s => s.name === f.name ? { ...s, seq: parseInt(e.target.value) || 0 } : s)
+                                }))}
+                              />
+                            )}
+                          </td>
+                        </tr>
+                      )
+                    })}
+                  </tbody>
+                </table>
+                <SampleTable rows={form.sampleRows || []} />
+              </div>
+            )}
+
+            {form.fields.length === 0 && (
+              <div>
+                <label className="text-xs text-gray-500 block mb-1">Constraint fields (comma-separated)</label>
+                <input
+                  className="w-full border border-gray-200 rounded px-3 py-1.5 text-sm focus:outline-none focus:border-blue-400"
+                  value={form.constraint_fields}
+                  onChange={e => setForm(f => ({ ...f, constraint_fields: e.target.value }))}
+                  placeholder="e.g. date, amount, description"
+                />
+              </div>
+            )}
+
+            <div className="flex gap-4">
+              <label className="flex items-center gap-1.5 text-xs text-gray-500 cursor-pointer">
+                <input
+                  type="checkbox"
+                  checked={form.global_picklist !== false}
+                  onChange={e => setForm(f => ({ ...f, global_picklist: e.target.checked }))}
+                />
+                Global picklist
+              </label>
+              {form.fields.length > 0 && (
+                <label className="flex items-center gap-1.5 text-xs text-gray-500 cursor-pointer">
+                  <input
+                    type="checkbox"
+                    checked={form.importSample !== false}
+                    onChange={e => setForm(f => ({ ...f, importSample: e.target.checked }))}
+                  />
+                  Import sample data
+                </label>
+              )}
+            </div>
+
+            {createError && <p className="text-xs text-red-500">{createError}</p>}
+
+            <div className="flex gap-2">
+              <button type="submit" disabled={createLoading}
+                className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700 disabled:opacity-50">
+                {createLoading ? 'Creating…' : 'Create'}
+              </button>
+              <button type="button"
+                onClick={() => { setCreating(false); setCreateError(''); setForm({ name: '', constraint_fields: '', fields: [], schema: [] }) }}
+                className="text-sm text-gray-500 px-3 py-1.5 rounded hover:bg-gray-100">
+                Cancel
+              </button>
+            </div>
+          </form>
+        </div>
+      )}
+    </div>
+  )
+}

ui/src/pages/Stacks.jsx

@@ -1,834 +0,0 @@
-import { Link } from 'react-router-dom'
-import { useState, useEffect, useRef } from 'react'
-import { api } from '../api'
-import { format as formatSql } from 'sql-formatter'
-
-function prettySql(sql) {
-  try {
-    return formatSql(sql, { language: 'postgresql', tabWidth: 4, keywordCase: 'upper' })
-  } catch {
-    return sql
-  }
-}
-
-const FIELD_TYPES = ['text', 'numeric', 'date']
-
-// ── Calibrate modal ────────────────────────────────────────────────────────────
-
-function fmt(n) {
-  return Number(n).toLocaleString(undefined, { minimumFractionDigits: 2, maximumFractionDigits: 2 })
-}
-
-function CalibrateModal({ stack, sourceName, currentOffset, onClose, onApply }) {
-  const [asOf, setAsOf] = useState('')
-  const [known, setKnown] = useState('')
-  const [computed, setComputed] = useState(null)  // raw sum from DB (no offset)
-  const [loading, setLoading] = useState(false)
-  const [error, setError] = useState('')
-  const [applyOffset, setApplyOffset] = useState('')
-  const debounceRef = useRef(null)
-
-  const knownNum = parseFloat(known)
-  const hasKnown = known !== '' && !isNaN(knownNum)
-  const plug = hasKnown && computed !== null ? knownNum - computed : null
-
-  // Auto-fetch computed sum on mount (all transactions) and whenever date changes
-  useEffect(() => {
-    clearTimeout(debounceRef.current)
-    debounceRef.current = setTimeout(async () => {
-      setLoading(true); setError('')
-      try {
-        const r = await api.calibrateBalance(stack.name, sourceName, { as_of_date: asOf || null, known_balance: 0 })
-        if (r.success) setComputed(Number(r.computed_sum))
-        else setError(r.error)
-      } catch (e) { setError(e.message) }
-      finally { setLoading(false) }
-    }, asOf ? 400 : 0)
-    return () => clearTimeout(debounceRef.current)
-  }, [asOf])
-
-  // Keep applyOffset in sync with plug
-  useEffect(() => {
-    if (plug !== null) setApplyOffset(plug.toFixed(2))
-  }, [plug])
-
-  return (
-    <div className="fixed inset-0 bg-black/40 flex items-center justify-center z-50" onMouseDown={e => { if (e.target === e.currentTarget) onClose() }}>
-      <div className="bg-surface rounded-lg shadow-xl w-[420px] p-5" onClick={e => e.stopPropagation()}>
-        <div className="flex items-center justify-between mb-4">
-          <span className="text-sm font-semibold text-ink-soft">Calibrate — {sourceName}</span>
-          <button onClick={onClose} className="text-muted hover:text-ink-soft">✕</button>
-        </div>
-
-        {/* Date */}
-        <div className="mb-4">
-          <label className="text-xs text-muted block mb-1">As-of date</label>
-          <input type="date" className="w-full border border-line rounded px-3 py-1.5 text-sm focus:outline-none focus:border-accent"
-            value={asOf} onChange={e => setAsOf(e.target.value)} />
-        </div>
-
-        {/* Reconciliation table */}
-        <div className="bg-raised rounded border border-line mb-4 text-sm">
-          <div className="flex items-center justify-between px-3 py-2 border-b border-line">
-            <span className="text-muted text-xs">Data sum at date</span>
-            <span className="font-mono text-ink-soft">
-              {loading ? <span className="text-muted">…</span> : computed !== null ? fmt(computed) : <span className="text-muted">—</span>}
-            </span>
-          </div>
-          <div className="flex items-center justify-between px-3 py-2 border-b border-line">
-            <span className="text-muted text-xs">Known balance</span>
-            <input
-              type="number" step="0.01"
-              className="font-mono text-right bg-transparent border-0 focus:outline-none w-36 text-sm text-ink-soft placeholder-gray-300"
-              placeholder="enter balance"
-              value={known} onChange={e => setKnown(e.target.value)}
-            />
-          </div>
-          <div className="flex items-center justify-between px-3 py-2 border-b border-line">
-            <span className="text-muted text-xs">Current offset</span>
-            <span className="font-mono text-muted">{fmt(currentOffset ?? 0)}</span>
-          </div>
-          <div className="flex items-center justify-between px-3 py-2 font-medium">
-            <span className="text-ink-soft text-xs">Plug (offset needed)</span>
-            <span className={`font-mono ${plug !== null ? 'text-accent' : 'text-muted'}`}>
-              {plug !== null ? fmt(plug) : '—'}
-            </span>
-          </div>
-        </div>
-
-        {error && <p className="text-xs text-danger mb-3">{error}</p>}
-
-        {/* Apply */}
-        <div className="flex gap-2 items-center">
-          <input type="number" step="0.01"
-            className="flex-1 border border-line rounded px-3 py-1.5 text-sm font-mono focus:outline-none focus:border-accent"
-            placeholder="offset to apply"
-            value={applyOffset} onChange={e => setApplyOffset(e.target.value)} />
-          <button onClick={() => onApply(parseFloat(applyOffset))} disabled={applyOffset === '' || isNaN(parseFloat(applyOffset))}
-            className="text-sm bg-green-600 text-white px-4 py-1.5 rounded hover:bg-green-700 disabled:opacity-40">
-            Apply
-          </button>
-        </div>
-      </div>
-    </div>
-  )
-}
-
-// ── Stack panel ────────────────────────────────────────────────────────────────
-
-function StackPanel({ stack, sources, onUpdated, onStale, onViewGenerated, onSqlGenerated }) {
-  const members = stack.sources || []
-
-  const [label, setLabel] = useState(stack.label || '')
-  const [fields, setFields] = useState(stack.fields || [])
-  const [newField, setNewField] = useState({ name: '', type: 'text' })
-  const [addingSrc, setAddingSrc] = useState('')
-
-  // Per-source config: sign, offset, amount_field, date_field, field_map
-  const [srcCfg, setSrcCfg] = useState(() =>
-    Object.fromEntries(members.map(m => [m.source_name, {
-      sign:         m.amount_sign ?? 1,
-      offset:       m.balance_offset ?? 0,
-      amount_field: m.amount_field || '',
-      date_field:   m.date_field || '',
-      field_map:    { ...(m.field_map || {}) },
-    }]))
-  )
-
-  // Available columns from each source's dfv view
-  const [srcFields, setSrcFields] = useState({})
-
-  // Drag-to-reorder state
-  const [dragIdx, setDragIdx] = useState(null)
-  const [dragOverIdx, setDragOverIdx] = useState(null)
-  const [srcDragIdx, setSrcDragIdx] = useState(null)
-  const [srcDragOverIdx, setSrcDragOverIdx] = useState(null)
-
-  // Calibrate
-  const [calibratingSource, setCalibratingSource] = useState(null)
-
-  // View / balance
-  const [viewResult, setViewResult] = useState(null)
-  const [netBalance, setNetBalance] = useState(null)
-  const [balanceError, setBalanceError] = useState('')
-
-  const [saving, setSaving] = useState(false)
-  const [mappingsDirty, setMappingsDirty] = useState(false)
-  const [error, setError] = useState('')
-
-  // Fetch source columns whenever members change
-  useEffect(() => {
-    members.forEach(m => {
-      api.getFields(m.source_name)
-        .then(f => setSrcFields(prev => ({ ...prev, [m.source_name]: f.map(x => x.key) })))
-        .catch(() => {})
-    })
-  }, [members.map(m => m.source_name).join(',')])
-
-  // Live SQL preview — debounced; syncs current UI state to DB first so preview is accurate
-  const previewTimer = useRef(null)
-  useEffect(() => {
-    clearTimeout(previewTimer.current)
-    previewTimer.current = setTimeout(async () => {
-      try {
-        for (const m of members) {
-          const cfg = srcCfg[m.source_name] || {}
-          await api.upsertStackSource(stack.name, m.source_name, {
-            field_map:      cfg.field_map    || {},
-            amount_sign:    cfg.sign         ?? 1,
-            balance_offset: cfg.offset       ?? 0,
-            amount_field:   cfg.amount_field || null,
-            date_field:     cfg.date_field   || null,
-          })
-        }
-        await api.updateStack(stack.name, {
-          fields,
-          amount_field: amountCanonical || null,
-          date_field:   dateCanonical   || null,
-        })
-        const r = await api.previewStackSql(stack.name)
-        if (r.success) onSqlGenerated?.(r.sql)
-      } catch {}
-    }, 600)
-    return () => clearTimeout(previewTimer.current)
-  }, [
-    JSON.stringify(fields),
-    JSON.stringify(srcCfg),
-    members.map(m => m.source_name).join(','),
-  ])
-
-  // Auto-detect canonical amount/date field from field types
-  const amountCanonical = fields.find(f => f.type === 'numeric')?.name || stack.amount_field
-  const dateCanonical   = fields.find(f => f.type === 'date')?.name   || stack.date_field
-
-  // ── Label ──
-  async function saveLabel() {
-    setSaving(true); setError('')
-    try { await api.updateStack(stack.name, { label }); onUpdated() }
-    catch (e) { setError(e.message) }
-    finally { setSaving(false) }
-  }
-
-  // ── Fields ──
-  async function addField() {
-    if (!newField.name) return
-    const updated = [...fields, { name: newField.name, type: newField.type }]
-    setFields(updated)
-    setNewField({ name: '', type: 'text' })
-    await api.updateStack(stack.name, { fields: updated })
-    onUpdated()
-  }
-
-  async function removeField(name) {
-    const updated = fields.filter(f => f.name !== name)
-    setFields(updated)
-    await api.updateStack(stack.name, { fields: updated })
-    onUpdated()
-  }
-
-  // ── Drag reorder ──
-  function handleDragStart(e, idx) {
-    setDragIdx(idx)
-    e.dataTransfer.effectAllowed = 'move'
-  }
-
-  function handleDragOver(e, idx) {
-    e.preventDefault()
-    setDragOverIdx(idx)
-  }
-
-  async function handleDrop(e, toIdx) {
-    e.preventDefault()
-    if (dragIdx === null || dragIdx === toIdx) { setDragIdx(null); setDragOverIdx(null); return }
-    const updated = [...fields]
-    const [moved] = updated.splice(dragIdx, 1)
-    updated.splice(toIdx, 0, moved)
-    setFields(updated)
-    setDragIdx(null); setDragOverIdx(null)
-    await api.updateStack(stack.name, { fields: updated })
-    onUpdated()
-  }
-
-  // ── Source drag-to-reorder ──
-  function handleSrcDragStart(e, idx) {
-    setSrcDragIdx(idx)
-    e.dataTransfer.effectAllowed = 'move'
-  }
-
-  function handleSrcDragOver(e, idx) {
-    e.preventDefault()
-    setSrcDragOverIdx(idx)
-  }
-
-  async function handleSrcDrop(e, toIdx) {
-    e.preventDefault()
-    if (srcDragIdx === null || srcDragIdx === toIdx) { setSrcDragIdx(null); setSrcDragOverIdx(null); return }
-    const updated = [...members]
-    const [moved] = updated.splice(srcDragIdx, 1)
-    updated.splice(toIdx, 0, moved)
-    setSrcDragIdx(null); setSrcDragOverIdx(null)
-    await api.reorderStackSources(stack.name, updated.map(m => m.source_name))
-    onUpdated()
-  }
-
-  // ── Mapping grid ──
-  function getMappingValue(srcName, canonicalName) {
-    const cfg = srcCfg[srcName] || {}
-    if (canonicalName === amountCanonical) return cfg.amount_field || ''
-    if (canonicalName === dateCanonical)   return cfg.date_field   || ''
-    return cfg.field_map?.[canonicalName] || ''
-  }
-
-  function setMappingValue(srcName, canonicalName, value) {
-    setSrcCfg(prev => {
-      const cfg = { ...prev[srcName] }
-      if (canonicalName === amountCanonical)      cfg.amount_field = value
-      else if (canonicalName === dateCanonical)   cfg.date_field   = value
-      else cfg.field_map = { ...cfg.field_map, [canonicalName]: value }
-      return { ...prev, [srcName]: cfg }
-    })
-    setMappingsDirty(true)
-  }
-
-  function setSrcSign(srcName, sign) {
-    setSrcCfg(prev => ({ ...prev, [srcName]: { ...prev[srcName], sign } }))
-    setMappingsDirty(true)
-  }
-
-  function setSrcOffset(srcName, offset) {
-    setSrcCfg(prev => ({ ...prev, [srcName]: { ...prev[srcName], offset } }))
-    setMappingsDirty(true)
-  }
-
-  async function saveMappings() {
-    setSaving(true); setError('')
-    try {
-      for (const m of members) {
-        const cfg = srcCfg[m.source_name] || {}
-        await api.upsertStackSource(stack.name, m.source_name, {
-          field_map:    cfg.field_map    || {},
-          amount_sign:  cfg.sign         ?? 1,
-          balance_offset: cfg.offset     ?? 0,
-          amount_field: cfg.amount_field || null,
-          date_field:   cfg.date_field   || null,
-        })
-      }
-      // Persist the auto-detected canonical field names on the stack
-      await api.updateStack(stack.name, {
-        amount_field: amountCanonical || null,
-        date_field:   dateCanonical   || null,
-      })
-      setMappingsDirty(false)
-      onStale?.(stack.name)
-      onUpdated()
-    } catch (e) { setError(e.message) }
-    finally { setSaving(false) }
-  }
-
-  // ── Sources ──
-  async function addSource() {
-    if (!addingSrc) return
-    await api.upsertStackSource(stack.name, addingSrc, { field_map: {}, amount_sign: 1 })
-    setSrcCfg(prev => ({ ...prev, [addingSrc]: { sign: 1, offset: 0, amount_field: '', date_field: '', field_map: {} } }))
-    // Load fields immediately so dropdowns are ready
-    try {
-      const f = await api.getFields(addingSrc)
-      setSrcFields(prev => ({ ...prev, [addingSrc]: f.map(x => x.key) }))
-    } catch (e) {}
-    setAddingSrc('')
-    onStale?.(stack.name)
-    onUpdated()
-  }
-
-  function handleSrcAmountField(srcName, value) {
-    setSrcCfg(prev => ({ ...prev, [srcName]: { ...prev[srcName], amount_field: value } }))
-    // Update column type to numeric if a column with this name exists
-    setFields(prev => prev.map(f => f.name === value ? { ...f, type: 'numeric' } : f))
-    setMappingsDirty(true)
-    maybeAutoPopulate(srcName, value, srcCfg[srcName]?.date_field)
-  }
-
-  function handleSrcDateField(srcName, value) {
-    setSrcCfg(prev => ({ ...prev, [srcName]: { ...prev[srcName], date_field: value } }))
-    setFields(prev => prev.map(f => f.name === value ? { ...f, type: 'date' } : f))
-    setMappingsDirty(true)
-    maybeAutoPopulate(srcName, srcCfg[srcName]?.amount_field, value)
-  }
-
-  function maybeAutoPopulate(srcName, amtField, dtField) {
-    if (!amtField || !dtField) return
-    if (fields.length > 0) return  // don't overwrite existing columns
-    const sourceFields = srcFields[srcName] || []
-    if (sourceFields.length === 0) return
-    const newFields = sourceFields.map(sf => ({
-      name: sf,
-      type: sf === amtField ? 'numeric' : sf === dtField ? 'date' : 'text',
-    }))
-    setFields(newFields)
-    api.updateStack(stack.name, { fields: newFields, amount_field: amtField, date_field: dtField })
-  }
-
-  async function removeSource(src) {
-    await api.removeStackSource(stack.name, src)
-    setSrcCfg(prev => { const n = { ...prev }; delete n[src]; return n })
-    onStale?.(stack.name)
-    onUpdated()
-  }
-
-  async function handleCalibrate(srcName) {
-    // Save this source's current config before opening modal
-    const cfg = srcCfg[srcName] || {}
-    await api.upsertStackSource(stack.name, srcName, {
-      field_map:     cfg.field_map    || {},
-      amount_sign:   cfg.sign         ?? 1,
-      balance_offset: cfg.offset      ?? 0,
-      amount_field:  cfg.amount_field || null,
-      date_field:    cfg.date_field   || null,
-    })
-    setCalibratingSource(srcName)
-  }
-
-  async function applyCalibration(srcName, offset) {
-    const cfg = srcCfg[srcName] || {}
-    await api.upsertStackSource(stack.name, srcName, {
-      field_map:     cfg.field_map    || {},
-      amount_sign:   cfg.sign         ?? 1,
-      balance_offset: offset,
-      amount_field:  cfg.amount_field || null,
-      date_field:    cfg.date_field   || null,
-    })
-    setSrcCfg(prev => ({ ...prev, [srcName]: { ...prev[srcName], offset } }))
-    setCalibratingSource(null)
-    onStale?.(stack.name)
-    onUpdated()
-  }
-
-  // ── View ──
-  async function generateView() {
-    setViewResult(null); setNetBalance(null); setBalanceError('')
-    try {
-      const r = await api.generateStackView(stack.name)
-      setViewResult(r)
-      if (r.success) {
-        fetchBalance()
-        onViewGenerated?.(stack.name)
-        onSqlGenerated?.(r.sql || '')
-        ;(r.cascade_stale || []).forEach(n => onStale?.(n))
-      }
-    } catch (e) { setError(e.message) }
-  }
-
-  async function fetchBalance() {
-    setBalanceError('')
-    try {
-      const r = await api.getStackBalance(stack.name)
-      if (r.success) setNetBalance(r.balance)
-      else setBalanceError(r.error)
-    } catch (e) { setBalanceError(e.message) }
-  }
-
-  const availableSources = sources.filter(s => !members.find(m => m.source_name === s.name))
-  if (addingSrc === '' && availableSources.length === 1) setAddingSrc(availableSources[0].name)
-
-  return (
-    <div className="space-y-5">
-
-      {/* Label */}
-      <div className="bg-surface border border-line rounded p-4">
-        <h3 className="text-sm font-semibold text-ink-soft mb-3">Configuration</h3>
-        <div className="flex gap-3 items-end">
-          <div className="flex-1">
-            <label className="text-xs text-muted block mb-1">Label <span className="text-muted">(optional)</span></label>
-            <input className="w-full border border-line rounded px-3 py-1.5 text-sm focus:outline-none focus:border-accent"
-              value={label} onChange={e => setLabel(e.target.value)}
-              onKeyDown={e => e.key === 'Enter' && saveLabel()} />
-          </div>
-          <button onClick={saveLabel} disabled={saving}
-            className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700 disabled:opacity-50">
-            {saving ? 'Saving…' : 'Save'}
-          </button>
-        </div>
-        {error && <p className="text-xs text-danger mt-2">{error}</p>}
-      </div>
-
-      {/* Sources */}
-      <div className="bg-surface border border-line rounded p-4">
-        <h3 className="text-sm font-semibold text-ink-soft mb-1">Sources</h3>
-        <p className="text-xs text-muted mb-3">Each source contributes rows to the combined view. Set the sign to flip the direction of amounts (e.g. credit card charges are positive in the source but should subtract from your balance). The offset adjusts the running balance — use Calibrate to compute it from a known good balance.</p>
-        <div className="space-y-2 mb-3">
-          {members.map((m, idx) => {
-            const cfg = srcCfg[m.source_name] || {}
-            const sf = srcFields[m.source_name] || []
-            const canCalibrate = !!cfg.amount_field && !!cfg.date_field
-            return (
-              <div key={m.source_name}
-                draggable
-                onDragStart={e => handleSrcDragStart(e, idx)}
-                onDragOver={e => handleSrcDragOver(e, idx)}
-                onDrop={e => handleSrcDrop(e, idx)}
-                onDragEnd={() => { setSrcDragIdx(null); setSrcDragOverIdx(null) }}
-                className={`border border-line-soft rounded px-3 py-2 text-xs space-y-2 ${srcDragOverIdx === idx && srcDragIdx !== idx ? 'bg-accent-soft' : ''}`}>
-                <div className="flex items-center gap-2">
-                  <span className="text-muted cursor-grab select-none">⠿</span>
-                  <span className="font-medium text-ink-soft flex-1">{m.source_name}</span>
-                  <button onClick={() => removeSource(m.source_name)} className="text-danger hover:text-danger">Remove</button>
-                </div>
-                <div className="grid grid-cols-2 gap-x-4 gap-y-1.5">
-                  <div>
-                    <label className="text-muted block mb-0.5">Amount field</label>
-                    <select value={cfg.amount_field || ''}
-                      onChange={e => handleSrcAmountField(m.source_name, e.target.value)}
-                      className="w-full border border-line rounded px-1.5 py-0.5 focus:outline-none focus:border-accent">
-                      <option value="">— select —</option>
-                      {sf.map(f => <option key={f} value={f}>{f}</option>)}
-                    </select>
-                  </div>
-                  <div>
-                    <label className="text-muted block mb-0.5">Sign</label>
-                    <select value={cfg.sign ?? 1}
-                      onChange={e => { setSrcSign(m.source_name, parseInt(e.target.value)); setMappingsDirty(true) }}
-                      className="w-full border border-line rounded px-1.5 py-0.5 focus:outline-none focus:border-accent">
-                      <option value={1}>+1 (as-is)</option>
-                      <option value={-1}>−1 (flip)</option>
-                    </select>
-                  </div>
-                  <div>
-                    <label className="text-muted block mb-0.5">Date field</label>
-                    <select value={cfg.date_field || ''}
-                      onChange={e => handleSrcDateField(m.source_name, e.target.value)}
-                      className="w-full border border-line rounded px-1.5 py-0.5 focus:outline-none focus:border-accent">
-                      <option value="">— select —</option>
-                      {sf.map(f => <option key={f} value={f}>{f}</option>)}
-                    </select>
-                  </div>
-                  <div>
-                    <label className="text-muted block mb-0.5">Balance offset</label>
-                    <div className="flex items-center gap-1">
-                      <input type="number" step="0.01" value={cfg.offset ?? 0}
-                        onChange={e => { setSrcOffset(m.source_name, parseFloat(e.target.value) || 0); setMappingsDirty(true) }}
-                        className="flex-1 border border-line rounded px-1.5 py-0.5 font-mono focus:outline-none focus:border-accent" />
-                      <button onClick={() => handleCalibrate(m.source_name)}
-                        disabled={!canCalibrate}
-                        title={!canCalibrate ? 'Set amount and date fields first' : 'Calibrate balance'}
-                        className="text-accent hover:text-accent underline disabled:opacity-40 disabled:cursor-not-allowed disabled:no-underline">
-                        Calibrate
-                      </button>
-                    </div>
-                  </div>
-                </div>
-              </div>
-            )
-          })}
-          {members.length === 0 && <p className="text-xs text-muted">No sources added yet.</p>}
-        </div>
-        {availableSources.length > 0 && (
-          <div className="flex gap-2">
-            <select className="flex-1 border border-line rounded px-2 py-1 text-sm focus:outline-none focus:border-accent"
-              value={addingSrc} onChange={e => setAddingSrc(e.target.value)}>
-              <option value="">— add source —</option>
-              {availableSources.map(s => <option key={s.name} value={s.name}>{s.name}</option>)}
-            </select>
-            <button onClick={addSource} disabled={!addingSrc}
-              className="text-sm bg-raised px-3 py-1 rounded hover:bg-raised text-ink-soft disabled:opacity-40">Add</button>
-          </div>
-        )}
-      </div>
-
-      {/* Output columns mapping grid */}
-      <div className="bg-surface border border-line rounded p-4">
-        <h3 className="text-sm font-semibold text-ink-soft mb-1">Output columns</h3>
-        <p className="text-xs text-muted mb-3">
-          Each row is a column in the combined view. Each source column shows which field from that source maps to it.
-          The first <span className="text-accent">numeric</span> field drives the running balance; the first <span className="text-ok">date</span> field drives the ordering.
-          Both <span className="font-mono">source_balance</span> (per-source) and <span className="font-mono">net_balance</span> (combined) are always included in the generated view.
-          Drag rows to reorder.
-        </p>
-
-        {members.length === 0 ? (
-          <p className="text-xs text-muted mb-3">Add sources above first.</p>
-        ) : (
-          <div className="overflow-x-auto mb-3">
-            <table className="w-full text-xs border-collapse">
-              <thead>
-                <tr className="border-b border-line">
-                  <th className="w-5 pb-2"></th>
-                  <th className="text-left text-muted font-normal pb-2 pr-4">Column</th>
-                  <th className="text-left text-muted font-normal pb-2 pr-4">Type</th>
-                  {members.map(m => (
-                    <th key={m.source_name} className="text-left text-muted font-normal pb-2 pr-3 min-w-36">{m.source_name}</th>
-                  ))}
-                  <th className="w-5 pb-2"></th>
-                </tr>
-              </thead>
-              <tbody>
-                {fields.map((f, idx) => {
-                  const isAmount = f.name === amountCanonical
-                  const isDate   = f.name === dateCanonical
-                  return (
-                    <tr key={f.name}
-                      draggable
-                      onDragStart={e => handleDragStart(e, idx)}
-                      onDragOver={e => handleDragOver(e, idx)}
-                      onDrop={e => handleDrop(e, idx)}
-                      onDragEnd={() => { setDragIdx(null); setDragOverIdx(null) }}
-                      className={`border-b border-line-soft ${dragOverIdx === idx && dragIdx !== idx ? 'bg-accent-soft' : ''}`}>
-                      <td className="py-1.5 pr-1 text-muted cursor-grab select-none">⠿</td>
-                      <td className="py-1.5 pr-4 font-mono text-ink-soft whitespace-nowrap">
-                        {f.name}
-                        {isAmount && <span className="ml-1.5 text-accent font-sans font-normal">amount</span>}
-                        {isDate   && <span className="ml-1.5 text-ok font-sans font-normal">date</span>}
-                      </td>
-                      <td className="py-1.5 pr-4 text-muted">{f.type}</td>
-                      {members.map(m => (
-                        <td key={m.source_name} className="py-1.5 pr-3">
-                          <div className="flex items-center gap-1">
-                            <select
-                              value={getMappingValue(m.source_name, f.name)}
-                              onChange={e => setMappingValue(m.source_name, f.name, e.target.value)}
-                              className="border border-line rounded px-1.5 py-0.5 focus:outline-none focus:border-accent min-w-0 flex-1">
-                              <option value="">— same name —</option>
-                              {(srcFields[m.source_name] || []).map(sf => (
-                                <option key={sf} value={sf}>{sf}</option>
-                              ))}
-                            </select>
-                          </div>
-                        </td>
-                      ))}
-                      <td className="py-1.5">
-                        <button onClick={() => removeField(f.name)} className="text-danger hover:text-danger">✕</button>
-                      </td>
-                    </tr>
-                  )
-                })}
-                {fields.length === 0 && (
-                  <tr><td colSpan={3 + members.length} className="py-3 text-muted text-center">No columns defined yet — add one below.</td></tr>
-                )}
-              </tbody>
-            </table>
-          </div>
-        )}
-
-        {/* Add field */}
-        <div className="flex gap-2 mb-3">
-          <input className="flex-1 border border-line rounded px-2 py-1 text-sm focus:outline-none focus:border-accent"
-            placeholder="column name" value={newField.name}
-            onChange={e => setNewField(f => ({ ...f, name: e.target.value }))}
-            onKeyDown={e => e.key === 'Enter' && addField()} />
-          <select className="border border-line rounded px-2 py-1 text-sm focus:outline-none focus:border-accent"
-            value={newField.type} onChange={e => setNewField(f => ({ ...f, type: e.target.value }))}>
-            {FIELD_TYPES.map(t => <option key={t} value={t}>{t}</option>)}
-          </select>
-          <button onClick={addField} className="text-sm bg-raised px-3 py-1 rounded hover:bg-raised text-ink-soft">Add</button>
-        </div>
-
-        {mappingsDirty && (
-          <button onClick={saveMappings} disabled={saving}
-            className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700 disabled:opacity-50">
-            {saving ? 'Saving…' : 'Save mappings'}
-          </button>
-        )}
-      </div>
-
-      {/* Generate view + balance */}
-      <div className="bg-surface border border-line rounded p-4">
-        <div className="flex items-center justify-between mb-3">
-          <h3 className="text-sm font-semibold text-ink-soft">View</h3>
-          <div className="flex gap-2">
-            <button onClick={fetchBalance}
-              className="text-sm bg-raised text-ink-soft px-3 py-1.5 rounded hover:bg-raised">
-              Refresh balance
-            </button>
-            <button onClick={generateView}
-              className="text-sm bg-blue-600 text-white px-3 py-1.5 rounded hover:bg-blue-700">
-              Generate / refresh
-            </button>
-          </div>
-        </div>
-        {netBalance !== null && (
-          <div className="mb-3 flex items-center gap-3">
-            <span className="text-xs text-muted">Current net balance</span>
-            <span className="text-lg font-mono font-semibold text-ink">
-              {Number(netBalance).toLocaleString(undefined, { minimumFractionDigits: 2 })}
-            </span>
-          </div>
-        )}
-        {balanceError && <p className="text-xs text-muted mb-3">{balanceError}</p>}
-        {viewResult && !viewResult.success && (
-          <p className="text-xs text-danger">{viewResult.error}</p>
-        )}
-        {viewResult && viewResult.success && (
-          <p className="text-xs text-ok">View created: <span className="font-mono">{viewResult.view}</span></p>
-        )}
-      </div>
-
-      {calibratingSource && (
-        <CalibrateModal
-          stack={stack}
-          sourceName={calibratingSource}
-          currentOffset={srcCfg[calibratingSource]?.offset ?? 0}
-          onClose={() => setCalibratingSource(null)}
-          onApply={offset => applyCalibration(calibratingSource, offset)}
-        />
-      )}
-    </div>
-  )
-}
-
-// ── Main page ──────────────────────────────────────────────────────────────────
-
-export default function Stacks({ sources, onStackStale, onStackViewGenerated, onStacksChange }) {
-  const [stacks, setStacks] = useState([])
-  const [selected, setSelected] = useState(null)
-  const [stackDetail, setStackDetail] = useState(null)
-  const [creating, setCreating] = useState(false)
-  const [newName, setNewName] = useState('')
-  const [error, setError] = useState('')
-  const [sqlDraft, setSqlDraft] = useState('')
-  const [sqlRunning, setSqlRunning] = useState(false)
-  const [sqlResult, setSqlResult] = useState(null)
-
-  async function load() {
-    const s = await api.getStacks()
-    setStacks(s)
-    return s
-  }
-
-  async function loadDetail(name) {
-    const s = await api.getStack(name)
-    setStackDetail(s)
-    setSelected(name)
-    localStorage.setItem('stacks_last_selected', name)
-    setSqlDraft('')
-    setSqlResult(null)
-  }
-
-  useEffect(() => {
-    load().then(s => {
-      const last = localStorage.getItem('stacks_last_selected')
-      if (last && s.find(x => x.name === last)) loadDetail(last)
-    })
-  }, [])
-  useEffect(() => { if (selected) loadDetail(selected) }, [selected])
-
-  async function createStack() {
-    if (!newName) return
-    setError('')
-    try {
-      await api.createStack({ name: newName, fields: [] })
-      setNewName(''); setCreating(false)
-      await load()
-      onStacksChange?.()
-      loadDetail(newName)
-    } catch (e) { setError(e.message) }
-  }
-
-  async function deleteStack(name) {
-    if (!confirm(`Delete stack "${name}"?`)) return
-    await api.deleteStack(name)
-    if (selected === name) { setSelected(null); setStackDetail(null); setSqlDraft(''); setSqlResult(null) }
-    load()
-    onStacksChange?.()
-  }
-
-  async function runSql() {
-    if (!sqlDraft.trim() || !selected) return
-    setSqlRunning(true); setSqlResult(null)
-    try {
-      const r = await api.execStackSql(selected, sqlDraft)
-      setSqlResult(r)
-      if (r.success) {
-        onStackViewGenerated?.(selected)
-        ;(r.cascade_stale || []).forEach(n => onStackStale?.(n))
-      }
-    } catch (e) { setSqlResult({ success: false, error: e.message }) }
-    finally { setSqlRunning(false) }
-  }
-
-  return (
-    <div className="p-6">
-      {/* Stack list — horizontal row of cards */}
-      <div className="flex items-center gap-2 mb-5 flex-wrap">
-        <h1 className="text-sm font-semibold text-ink mr-1">Stacks</h1>
-        {stacks.map(s => (
-          <div key={s.name}
-            onClick={() => loadDetail(s.name)}
-            className={`flex items-center gap-2 px-3 py-1.5 rounded border cursor-pointer text-xs group transition-colors ${selected === s.name ? 'border-accent-line bg-accent-soft text-accent' : 'border-line bg-surface text-ink-soft hover:border-line hover:bg-raised'}`}>
-            <span className="font-medium">{s.label || s.name}</span>
-            <span className="text-muted">{s.source_count}s</span>
-            <Link to={`/stacks/${encodeURIComponent(s.name)}/pivot`}
-              onClick={e => e.stopPropagation()}
-              className="text-accent underline decoration-transparent hover:decoration-inherit">pivot</Link>
-            <button onClick={e => { e.stopPropagation(); deleteStack(s.name) }}
-              className="opacity-0 group-hover:opacity-100 text-danger hover:text-danger leading-none ml-2 pl-2 border-l border-line">✕</button>
-          </div>
-        ))}
-        {creating ? (
-          <div className="flex items-center gap-1">
-            <input autoFocus className="border border-accent rounded px-2 py-1 text-xs focus:outline-none w-32"
-              placeholder="stack name" value={newName} onChange={e => setNewName(e.target.value)}
-              onKeyDown={e => { if (e.key === 'Enter') createStack(); if (e.key === 'Escape') setCreating(false) }} />
-            <button onClick={createStack} className="text-xs bg-blue-600 text-white px-2 py-1 rounded">Create</button>
-            <button onClick={() => setCreating(false)} className="text-xs text-muted px-1">✕</button>
-            {error && <p className="text-xs text-danger">{error}</p>}
-          </div>
-        ) : (
-          <button onClick={() => setCreating(true)} className="text-xs text-accent hover:text-accent px-2 py-1.5">+ New</button>
-        )}
-      </div>
-
-      {stackDetail ? (
-        <div className="flex gap-6 items-start">
-          {/* Left: config panel */}
-          <div className="flex-1 min-w-0">
-            <h2 className="text-base font-semibold text-ink mb-4">
-              {stackDetail.label || stackDetail.name}
-              {stackDetail.label && <span className="text-sm text-muted font-normal ml-2">{stackDetail.name}</span>}
-            </h2>
-            <StackPanel
-              key={stackDetail.name}
-              stack={stackDetail}
-              sources={sources}
-              onUpdated={() => { load(); loadDetail(stackDetail.name) }}
-              onStale={onStackStale}
-              onViewGenerated={onStackViewGenerated}
-              onSqlGenerated={sql => { setSqlDraft(prettySql(sql)); setSqlResult(null) }}
-            />
-          </div>
-
-          {/* Right: SQL panel */}
-          <div className="flex-1 min-w-0">
-            <div className="bg-surface border border-line rounded p-4 sticky top-4">
-              <div className="flex items-center justify-between mb-3">
-                <h3 className="text-sm font-semibold text-ink-soft">Generated SQL</h3>
-                <button
-                  onClick={runSql}
-                  disabled={!sqlDraft.trim() || sqlRunning}
-                  className="text-sm bg-blue-600 text-white px-3 py-1 rounded hover:bg-blue-700 disabled:opacity-40">
-                  {sqlRunning ? 'Running…' : 'Run'}
-                </button>
-              </div>
-              {sqlDraft ? (
-                <textarea
-                  className="w-full font-mono text-xs text-ink-soft bg-raised border border-line rounded p-2 focus:outline-none focus:border-accent resize-none leading-relaxed"
-                  style={{ minHeight: '60vh' }}
-                  value={sqlDraft}
-                  onChange={e => { setSqlDraft(e.target.value); setSqlResult(null) }}
-                  spellCheck={false}
-                />
-              ) : (
-                <p className="text-xs text-muted">Generate a view to see the SQL here.</p>
-              )}
-              {sqlResult && (
-                <p className={`text-xs mt-2 ${sqlResult.success ? 'text-ok' : 'text-danger'}`}>
-                  {sqlResult.success ? 'View updated successfully.' : sqlResult.error}
-                </p>
-              )}
-            </div>
-          </div>
-        </div>
-      ) : (
-        <p className="text-sm text-muted">Select a stack or create one.</p>
-      )}
-    </div>
-  )
-}

ui/src/theme.jsx

@@ -1,25 +0,0 @@
-import { createContext, useContext, useState, useEffect } from 'react'
-
-const ThemeContext = createContext()
-
-export function ThemeProvider({ children }) {
-  const [dark, setDark] = useState(() => {
-    const saved = localStorage.getItem('df_dark')
-    if (saved !== null) return saved === 'true'
-    return window.matchMedia('(prefers-color-scheme: dark)').matches
-  })
-
-  useEffect(() => {
-    localStorage.setItem('df_dark', dark)
-    document.documentElement.classList.toggle('dark', dark)
-  }, [dark])
-
-  return (
-    <ThemeContext.Provider value={{ dark, setDark }}>
-      {children}
-    </ThemeContext.Provider>
-  )
-}
-
-const useTheme = () => useContext(ThemeContext)
-export default useTheme

uninstall.sh

@@ -0,0 +1,85 @@
+#!/bin/bash
+#
+# Dataflow Uninstall Script
+# Removes database user, database, and optionally .env
+#
+
+echo "⚠️  Dataflow Uninstall"
+echo "====================="
+echo ""
+
+# Load .env if it exists
+if [ -f .env ]; then
+    export $(cat .env | grep -v '^#' | xargs)
+fi
+
+DB_NAME=${DB_NAME:-dataflow}
+DB_USER=${DB_USER:-dataflow}
+
+echo "⚠️  This will permanently delete:"
+echo "  - Database: $DB_NAME"
+echo "  - User: $DB_USER"
+echo ""
+read -p "Type 'delete' to confirm: " CONFIRM
+if [ "$CONFIRM" != "delete" ]; then
+    echo "Cancelled."
+    exit 0
+fi
+
+# Prompt for admin credentials
+echo ""
+echo "📋 PostgreSQL Admin Credentials"
+echo ""
+read -p "Admin username [postgres]: " ADMIN_USER
+ADMIN_USER=${ADMIN_USER:-postgres}
+read -s -p "Admin password: " ADMIN_PASS
+echo ""
+
+DB_HOST=${DB_HOST:-localhost}
+DB_PORT=${DB_PORT:-5432}
+DB_NAME=${DB_NAME:-dataflow}
+DB_USER=${DB_USER:-dataflow}
+
+# Test admin connection
+echo ""
+echo "🔍 Testing PostgreSQL admin connection..."
+export PGPASSWORD="$ADMIN_PASS"
+if ! psql -U "$ADMIN_USER" -h "$DB_HOST" -p "$DB_PORT" -d postgres -c '\q' 2>/dev/null; then
+    echo "✗ Cannot connect to PostgreSQL"
+    exit 1
+fi
+echo "✓ Connected"
+
+# Drop database
+echo ""
+echo "🗄️  Dropping database..."
+psql -U "$ADMIN_USER" -h "$DB_HOST" -p "$DB_PORT" -d postgres -c "DROP DATABASE IF EXISTS $DB_NAME;" 2>/dev/null || true
+echo "✓ Database dropped"
+
+# Drop user
+echo ""
+echo "👤 Dropping user..."
+psql -U "$ADMIN_USER" -h "$DB_HOST" -p "$DB_PORT" -d postgres -c "DROP USER IF EXISTS $DB_USER;" 2>/dev/null || true
+echo "✓ User dropped"
+
+unset PGPASSWORD
+
+# Optionally remove .env
+echo ""
+read -p "Remove .env file? [y/N]: " REMOVE_ENV
+if [ "$REMOVE_ENV" == "y" ] || [ "$REMOVE_ENV" == "Y" ]; then
+    rm -f .env
+    echo "✓ .env removed"
+fi
+
+# Optionally remove node_modules
+echo ""
+read -p "Remove node_modules? [y/N]: " REMOVE_MODULES
+if [ "$REMOVE_MODULES" == "y" ] || [ "$REMOVE_MODULES" == "Y" ]; then
+    rm -rf node_modules
+    echo "✓ node_modules removed"
+fi
+
+echo ""
+echo "✅ Uninstall complete!"
+echo ""