Update all docs to reflect current state

- perspective-pivot.md: npm install pattern, v4.5.1/v4.4.1 versions
- README.md: Node 18+, port 3020, add stacks routes, fix project structure
- SPEC.md: add stacks/status routes, pages, SQL functions; update Perspective version
- ui/README.md: replace Vite boilerplate with project-specific content
- Remove docs/refactor-transformed-split.md (completed work)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

README.md

@@ -46,7 +46,7 @@ Map extracted values to clean, standardized output.
 
 ### Prerequisites
 - PostgreSQL 12+
-- Node.js 16+
+- Node.js 18+
 - Python 3 (for `manage.py`)
 
 ### Installation
@@ -66,7 +66,7 @@ For development with auto-reload:
 npm run dev
 ```
 
-The UI is available at `http://localhost:3000`. The API is at `http://localhost:3000/api`.
+The UI is available at `http://localhost:3020`. The API is at `http://localhost:3020/api` (port set by `API_PORT` in `.env`).
 
 ## Management Script (`manage.py`)
 
@@ -154,6 +154,20 @@ All `/api` routes require HTTP Basic authentication.
 | DELETE | `/api/records/:id` | Delete a record |
 | DELETE | `/api/records/source/:source_name/all` | Delete all records for a source |
 
+### Stacks — `/api/stacks`
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/stacks` | List all stacks |
+| POST | `/api/stacks` | Create a stack |
+| GET | `/api/stacks/:name` | Get a stack |
+| PUT | `/api/stacks/:name` | Update a stack |
+| DELETE | `/api/stacks/:name` | Delete a stack |
+| GET | `/api/stacks/:name/view-data` | Query stacked data (paginated) |
+| GET | `/api/stacks/:name/layouts` | List saved pivot layouts |
+| POST | `/api/stacks/:name/layouts` | Save a pivot layout |
+| DELETE | `/api/stacks/:name/layouts/:id` | Delete a pivot layout |
+
 ## Typical Workflow
 
 ```
@@ -175,7 +189,13 @@ See `examples/GETTING_STARTED.md` for a complete walkthrough with curl examples.
 dataflow/
 ├── database/
 │   ├── schema.sql           # Table definitions
-│   └── functions.sql      # Import/transform/query functions
+│   └── queries/             # SQL functions, one file per route
+│       ├── sources.sql
+│       ├── rules.sql
+│       ├── mappings.sql
+│       ├── records.sql
+│       ├── stacks.sql
+│       └── status.sql
 ├── api/
 │   ├── server.js            # Express server
 │   ├── middleware/
@@ -186,7 +206,9 @@ dataflow/
 │       ├── sources.js
 │       ├── rules.js
 │       ├── mappings.js
-│       └── records.js
+│       ├── records.js
+│       ├── stacks.js
+│       └── status.js
 ├── public/                  # Built React UI (served as static files)
 ├── examples/
 │   ├── GETTING_STARTED.md

SPEC.md

@@ -50,6 +50,8 @@ api/
     rules.js           — HTTP handlers for rule management
     mappings.js        — HTTP handlers for mapping management
     records.js         — HTTP handlers for record queries
+    stacks.js          — HTTP handlers for stack management
+    status.js          — HTTP handler for deployment status
 ui/
   src/
     api.js             — fetch wrapper, credential management
@@ -135,6 +137,12 @@ Each file in `database/queries/` maps 1-to-1 with a route file.
 **records.sql**
 `list_records`, `get_record`, `search_records` (JSONB containment on data and transformed), `delete_record`, `delete_source_records`
 
+**stacks.sql**
+`list_stacks`, `get_stack`, `create_stack`, `update_stack`, `delete_stack`, `get_stack_view_data` (union of source views with field mapping and running balance), `list_pivot_layouts`, `save_pivot_layout`, `delete_pivot_layout`
+
+**status.sql**
+`get_status` — returns deployment state (schema version, function presence, service status)
+
 ---
 
 ## API
@@ -182,6 +190,16 @@ All routes are under `/api`. Every route requires HTTP Basic Auth. The `GET /hea
 | POST | /api/records/search | Search by JSONB containment |
 | DELETE | /api/records/:id | Delete record |
 | DELETE | /api/records/source/:name/all | Delete all records for a source |
+| GET | /api/stacks | List all stacks |
+| POST | /api/stacks | Create stack |
+| GET | /api/stacks/:name | Get stack |
+| PUT | /api/stacks/:name | Update stack |
+| DELETE | /api/stacks/:name | Delete stack |
+| GET | /api/stacks/:name/view-data | Paginated stacked data with running balance |
+| GET | /api/stacks/:name/layouts | List saved pivot layouts |
+| POST | /api/stacks/:name/layouts | Save pivot layout |
+| DELETE | /api/stacks/:name/layouts/:id | Delete pivot layout |
+| GET | /api/status | Deployment status |
 
 ---
 
@@ -218,11 +236,11 @@ Built with React + Vite + Tailwind CSS. Compiled output goes to `public/`. The s
 
 - **Records** — Paginated table showing the `dfv.{source}` view. Server-side sorting (column validated against `information_schema.columns`, interpolated with `quote_ident`). Dates are formatted `YYYY-MM-DD` for correct lexicographic sort. Regex filters can be added per column. If the view cast fails (e.g. a field typed as `date` contains text), the error is shown inline rather than a blank page.
 
-- **Pivot** — Interactive pivot/crosstab powered by [Perspective](https://perspective.finos.org/) (`@perspective-dev` v4.4.0, loaded from CDN at runtime). Loads all rows from the source view into an in-browser Perspective worker and renders a `<perspective-viewer>` web component. Supports grouping, splitting, filtering, sorting, and charting interactively.
+- **Pivot** — Interactive pivot/crosstab powered by [Perspective](https://perspective.finos.org/) (`@perspective-dev` client/viewer/datagrid v4.5.1, viewer-d3fc v4.4.1 — installed via npm). Loads all rows from the source view into an in-browser Perspective worker and renders a `<perspective-viewer>` web component. Supports grouping, splitting, filtering, sorting, and charting interactively.
 
   **Toolbar (above the viewer):**
   - Named layouts — saved per source in the `pivot_layouts` DB table. Each chip recalls the full viewer state including group_by, split_by, filters, expressions, selection mode, and expand depth. A blue **Save** button overwrites the active layout in place; **+ Save as…** saves to a new name. The × on each chip deletes it.
-  - **depth: 0 1 2 3** — collapses or expands all grouped rows to the specified hierarchy level. Implemented via `view.set_depth(d)` + `plugin.draw(view)` (the only working mechanism found in v4.4.0 — `plugin_config.expand_depth` and `viewer.flush()` alone have no effect).
+  - **depth: 0 1 2 3** — collapses or expands all grouped rows to the specified hierarchy level. Implemented via `view.set_depth(d)` + `plugin.draw(view)` (the only working mechanism found — `plugin_config.expand_depth` and `viewer.flush()` alone have no effect).
   - The Perspective built-in **selection mode button** (Read-Only / Select Row / Select Column / Select Region) defaults to **Select Region** on fresh load, set directly via `plugin.restore({ edit_mode: 'SELECT_REGION' })` after the viewer loads.
 
   **Cell inspector (right panel):**
@@ -237,6 +255,8 @@ Built with React + Vite + Tailwind CSS. Compiled output goes to `public/`. The s
 
   See `docs/perspective-pivot.md` for the full technical reference on controlling Perspective programmatically.
 
+- **Stacks** — Named unions of multiple sources. Each stack defines a field mapping (how source fields map to common output columns), an amount field, a date field, and an optional balance offset. The view-data endpoint unions the underlying source views and computes a running balance sorted by date. The Pivot page supports stacks as well as individual sources, with layouts stored in the same `pivot_layouts` table.
+
 - **Log** — Global import log across all sources. Same expandable key detail and delete capability as the Import page, plus a source name column.
 
 ---

docs/perspective-pivot.md

@@ -1,27 +1,22 @@
 # Perspective Pivot — Technical Reference
 
-Version tested: `@perspective-dev` v4.4.0 (client, viewer, viewer-datagrid, viewer-d3fc), loaded from CDN.
+Packages: `@perspective-dev` client/viewer/viewer-datagrid at **v4.5.1**, viewer-d3fc at **v4.4.1** — installed via npm. API notes that reference v4.4.0 behaviour have not been re-verified at 4.5.1 but are believed to still apply.
 
 This document captures everything learned about controlling Perspective programmatically. The official docs are incomplete for some of these APIs — treat this as a ground-truth supplement.
 
 ---
 
-## Loading from CDN
+## Loading via npm
 
 ```js
-const [{ default: perspective }] = await Promise.all([
+import perspective from '@perspective-dev/client/inline'
-  import('https://cdn.jsdelivr.net/npm/@perspective-dev/client@4.4.0/dist/cdn/perspective.js'),
+import '@perspective-dev/viewer/inline'
-  import('https://cdn.jsdelivr.net/npm/@perspective-dev/viewer@4.4.0/dist/cdn/perspective-viewer.js'),
+import '@perspective-dev/viewer-datagrid'
-  import('https://cdn.jsdelivr.net/npm/@perspective-dev/viewer-datagrid@4.4.0/dist/cdn/perspective-viewer-datagrid.js'),
+import '@perspective-dev/viewer-d3fc'
-  import('https://cdn.jsdelivr.net/npm/@perspective-dev/viewer-d3fc@4.4.0/dist/cdn/perspective-viewer-d3fc.js'),
+import '@perspective-dev/viewer/themes'
-])
 ```
 
-Stylesheet:
+The `inline` builds embed WebAssembly directly into the JS bundle — no separate `.wasm` file to serve. viewer-datagrid and viewer-d3fc have no inline variant; they import normally. viewer-d3fc is currently at v4.4.1 (no v4.5.x release yet); its chart plugins register but may not appear in the viewer due to an API change in v4.5.x's `registerPlugin`.
-```html
-<link rel="stylesheet" crossorigin="anonymous"
-  href="https://cdn.jsdelivr.net/npm/@perspective-dev/viewer/dist/css/themes.css" />
-```
 
 ---
 

docs/refactor-transformed-split.md

@@ -1,62 +0,0 @@
-# Refactor: Split `transformed` into three columns
-
-## Goal
-
-Separate `records` into three clean JSONB layers with clear semantics:
-
-| Column | Meaning | Wins over |
-|---|---|---|
-| `data` | Raw import values, never mutated | — |
-| `transformed` | Rule/mapping-derived fields only | `data` |
-| `overrides` | Manual user overrides | `data`, `transformed` |
-
-Consumers merge them at read time:
-
-```sql
-data || COALESCE(transformed, '{}'::jsonb) || COALESCE(overrides, '{}'::jsonb)
-```
-
-## Why
-
-Currently `transformed` duplicates `data` keys because `apply_transformations` was originally
-written as `data || rule_additions`. This makes it impossible to tell what the rules actually
-changed vs. what was carried from the original import.
-
-## Current State (branch: `transformed-refactor`)
-
-### Already done in functions.sql
-
-- `apply_transformations` — already stores only rule additions (`COALESCE(ra.additions, '{}')`)
-- `generate_source_view` — already uses the 3-way coalesce for `dfv.*` views
-- `set_record_overrides`, `clear_record_overrides`, `bulk_set_record_overrides` — exist
-- API routes — `PUT /api/records/:id/overrides`, `DELETE /:id/overrides`, `POST /bulk-overrides` exist
-
-### Still needed
-
-1. **`database/schema.sql`** — add `overrides JSONB` column to `records` table and a GIN index.
-   Also fix the syntax error: trailing comma before `)` on line 48.
-
-2. **`ui/src/pages/Records.jsx`** — right panel currently iterates `selectedRecord.transformed`
-   for all fields. Split into three sections:
-   - **Original** (`data`) — read-only, muted style
-   - **Transformed** (`transformed`) — rule-derived delta only, highlighted
-   - **Overrides** (`overrides`) — editable, amber style (existing draft UI already works here)
-
-3. **Deploy + reprocess** (user-triggered, not automated):
-   - `psql -d dataflow -f database/schema.sql` (drop/recreate schema)
-   - `psql -d dataflow -f database/functions.sql` (redeploy functions)
-   - Regenerate all `dfv.*` views via the API for each source
-   - Run `reprocess_records` on every source to strip stale `data` keys from existing `transformed` rows
-
-## Rollback
-
-Branch `stacks` is the stable point. A pg_dump taken before deployment is the DB rollback.
-
-## File Checklist
-
-- [ ] `database/schema.sql` — add `overrides` column + index, fix syntax error
-- [ ] `database/functions.sql` — no changes needed (already correct)
-- [ ] `ui/src/pages/Records.jsx` — split inspector panel into 3 sections
-- [ ] Build UI: `cd ui && npm run build`
-- [ ] Deploy DB (user-triggered)
-- [ ] Reprocess all sources (user-triggered)

ui/README.md

@@ -1,16 +1,25 @@
-# React + Vite
+# Dataflow UI
 
-This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
+React + Vite + Tailwind CSS frontend for Dataflow.
 
-Currently, two official plugins are available:
+## Development
 
-- [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Oxc](https://oxc.rs)
+```bash
-- [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/)
+npm install
+npm run dev      # dev server on :5173, proxies /api to :3020
+```
 
-## React Compiler
+## Build
 
-The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation).
+```bash
+npm run build    # outputs to ../public/
+```
 
-## Expanding the ESLint configuration
+The Express server serves `../public/` as static files — no separate web server needed in production.
 
-If you are developing a production application, we recommend using TypeScript with type-aware lint rules enabled. Check out the [TS template](https://github.com/vitejs/vite/tree/main/packages/create-vite/template-react-ts) for information on how to integrate TypeScript and [`typescript-eslint`](https://typescript-eslint.io) in your project.
+## Key packages
+
+- `react` / `react-router-dom` — SPA routing
+- `@perspective-dev/client`, `viewer`, `viewer-datagrid`, `viewer-d3fc` — pivot table (npm, inline WASM builds)
+- `tailwindcss` — utility CSS
+- `sql-formatter` — SQL display formatting