Compare commits

..

No commits in common. "efa65d84091de069baf6995517e2f84cd0cc70bf" and "bef3d6d89c5d510340c002f3d81cce49b418a064" have entirely different histories.

24 changed files with 329 additions and 1756 deletions

View File

@ -140,17 +140,6 @@ records.data → apply_transformations() →
- Server.js has global error handler - Server.js has global error handler
- Database functions return JSON with `success` boolean - Database functions return JSON with `success` boolean
## Light / dark mode
Theme state lives in `ui/src/theme.jsx` — a React context (`ThemeContext`) with a `ThemeProvider` that wraps the app in `main.jsx`.
- **Storage key:** `df_dark` in `localStorage`; falls back to `window.matchMedia('(prefers-color-scheme: dark)')` on first visit
- **Toggle:** button in the sidebar header in `App.jsx`; effect writes `localStorage` and toggles the `.dark` class on `<html>`
- **CSS:** `ui/src/index.css` defines CSS custom properties under `:root` (light) and `.dark`. All Tailwind color overrides are written as `.dark .bg-white { ... }` etc.
- **Palette:** dark mode uses Perspective's "Pro Dark" colours (`--bg-primary: #242526`, panels `#2a2c2f`, gridlines `#3b3f46`, text `#c5c9d0`)
- **Perspective viewer:** `Pivot.jsx` calls `viewer.setAttribute('theme', dark ? 'Pro Dark' : 'Pro Light')` on initial load and in a `useEffect([dark])` so the viewer stays in sync when the toggle fires
- **Consuming the theme:** `import useTheme from '../theme.jsx'` then `const { dark, setDark } = useTheme()`
## UI (React + Vite) ## UI (React + Vite)
The frontend lives in `ui/src/` and is built to `public/` via `npm run build` from the `ui/` directory. **Always run `npm run build` from `ui/` after any changes to `ui/src/` files.** The frontend lives in `ui/src/` and is built to `public/` via `npm run build` from the `ui/` directory. **Always run `npm run build` from `ui/` after any changes to `ui/src/` files.**
@ -158,7 +147,7 @@ The frontend lives in `ui/src/` and is built to `public/` via `npm run build` fr
### Pages ### Pages
- **Sources / Rules / Mappings / Records** — standard CRUD pages - **Sources / Rules / Mappings / Records** — standard CRUD pages
- **Pivot** (`ui/src/pages/Pivot.jsx`) — interactive pivot/crosstab powered by Perspective (`@perspective-dev` v4.5.1, installed via npm). See `docs/perspective-pivot.md` for the full Perspective API reference. - **Pivot** (`ui/src/pages/Pivot.jsx`) — interactive pivot/crosstab powered by Perspective (`@perspective-dev` v4.4.0, loaded from CDN). See `docs/perspective-pivot.md` for the full Perspective API reference.
- **Stacks** — multi-source union views with running balance - **Stacks** — multi-source union views with running balance
- **Log** — import audit trail - **Log** — import audit trail
@ -173,10 +162,6 @@ Clicking a data cell opens a right-hand inspector panel showing the underlying t
- The panel is resizable via a drag handle on its left edge (`paneWidth` state, min 240px). - The panel is resizable via a drag handle on its left edge (`paneWidth` state, min 240px).
- The transaction table is sortable (click header) and shows column totals for all-numeric columns. - The transaction table is sortable (click header) and shows column totals for all-numeric columns.
### Pivot layout persistence
Named layouts are stored in `dataflow.pivot_layouts` for both sources and stacks. The `source_name` column holds either a source name or a stack name — the FK to `sources(name)` was dropped to allow this. Source layouts use `/api/sources/:name/layouts`; stack layouts use `/api/stacks/:name/layouts`. Both call the same DB functions (`list_pivot_layouts`, `save_pivot_layout`, `delete_pivot_layout`). `localStorage` is still used to remember the *last active layout* for a view (the `psp_layout_<name>` key), but named layout definitions live in the DB so they persist across machines.
## File Structure ## File Structure
``` ```

View File

@ -46,7 +46,7 @@ Map extracted values to clean, standardized output.
### Prerequisites ### Prerequisites
- PostgreSQL 12+ - PostgreSQL 12+
- Node.js 18+ - Node.js 16+
- Python 3 (for `manage.py`) - Python 3 (for `manage.py`)
### Installation ### Installation
@ -66,7 +66,7 @@ For development with auto-reload:
npm run dev npm run dev
``` ```
The UI is available at `http://localhost:3020`. The API is at `http://localhost:3020/api` (port set by `API_PORT` in `.env`). The UI is available at `http://localhost:3000`. The API is at `http://localhost:3000/api`.
## Management Script (`manage.py`) ## Management Script (`manage.py`)
@ -154,20 +154,6 @@ All `/api` routes require HTTP Basic authentication.
| DELETE | `/api/records/:id` | Delete a record | | DELETE | `/api/records/:id` | Delete a record |
| DELETE | `/api/records/source/:source_name/all` | Delete all records for a source | | DELETE | `/api/records/source/:source_name/all` | Delete all records for a source |
### Stacks — `/api/stacks`
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/stacks` | List all stacks |
| POST | `/api/stacks` | Create a stack |
| GET | `/api/stacks/:name` | Get a stack |
| PUT | `/api/stacks/:name` | Update a stack |
| DELETE | `/api/stacks/:name` | Delete a stack |
| GET | `/api/stacks/:name/view-data` | Query stacked data (paginated) |
| GET | `/api/stacks/:name/layouts` | List saved pivot layouts |
| POST | `/api/stacks/:name/layouts` | Save a pivot layout |
| DELETE | `/api/stacks/:name/layouts/:id` | Delete a pivot layout |
## Typical Workflow ## Typical Workflow
``` ```
@ -189,13 +175,7 @@ See `examples/GETTING_STARTED.md` for a complete walkthrough with curl examples.
dataflow/ dataflow/
├── database/ ├── database/
│ ├── schema.sql # Table definitions │ ├── schema.sql # Table definitions
│ └── queries/ # SQL functions, one file per route │ └── functions.sql # Import/transform/query functions
│ ├── sources.sql
│ ├── rules.sql
│ ├── mappings.sql
│ ├── records.sql
│ ├── stacks.sql
│ └── status.sql
├── api/ ├── api/
│ ├── server.js # Express server │ ├── server.js # Express server
│ ├── middleware/ │ ├── middleware/
@ -206,9 +186,7 @@ dataflow/
│ ├── sources.js │ ├── sources.js
│ ├── rules.js │ ├── rules.js
│ ├── mappings.js │ ├── mappings.js
│ ├── records.js │ └── records.js
│ ├── stacks.js
│ └── status.js
├── public/ # Built React UI (served as static files) ├── public/ # Built React UI (served as static files)
├── examples/ ├── examples/
│ ├── GETTING_STARTED.md │ ├── GETTING_STARTED.md

24
SPEC.md
View File

@ -50,8 +50,6 @@ api/
rules.js — HTTP handlers for rule management rules.js — HTTP handlers for rule management
mappings.js — HTTP handlers for mapping management mappings.js — HTTP handlers for mapping management
records.js — HTTP handlers for record queries records.js — HTTP handlers for record queries
stacks.js — HTTP handlers for stack management
status.js — HTTP handler for deployment status
ui/ ui/
src/ src/
api.js — fetch wrapper, credential management api.js — fetch wrapper, credential management
@ -137,12 +135,6 @@ Each file in `database/queries/` maps 1-to-1 with a route file.
**records.sql** **records.sql**
`list_records`, `get_record`, `search_records` (JSONB containment on data and transformed), `delete_record`, `delete_source_records` `list_records`, `get_record`, `search_records` (JSONB containment on data and transformed), `delete_record`, `delete_source_records`
**stacks.sql**
`list_stacks`, `get_stack`, `create_stack`, `update_stack`, `delete_stack`, `get_stack_view_data` (union of source views with field mapping and running balance), `list_pivot_layouts`, `save_pivot_layout`, `delete_pivot_layout`
**status.sql**
`get_status` — returns deployment state (schema version, function presence, service status)
--- ---
## API ## API
@ -190,16 +182,6 @@ All routes are under `/api`. Every route requires HTTP Basic Auth. The `GET /hea
| POST | /api/records/search | Search by JSONB containment | | POST | /api/records/search | Search by JSONB containment |
| DELETE | /api/records/:id | Delete record | | DELETE | /api/records/:id | Delete record |
| DELETE | /api/records/source/:name/all | Delete all records for a source | | DELETE | /api/records/source/:name/all | Delete all records for a source |
| GET | /api/stacks | List all stacks |
| POST | /api/stacks | Create stack |
| GET | /api/stacks/:name | Get stack |
| PUT | /api/stacks/:name | Update stack |
| DELETE | /api/stacks/:name | Delete stack |
| GET | /api/stacks/:name/view-data | Paginated stacked data with running balance |
| GET | /api/stacks/:name/layouts | List saved pivot layouts |
| POST | /api/stacks/:name/layouts | Save pivot layout |
| DELETE | /api/stacks/:name/layouts/:id | Delete pivot layout |
| GET | /api/status | Deployment status |
--- ---
@ -236,11 +218,11 @@ Built with React + Vite + Tailwind CSS. Compiled output goes to `public/`. The s
- **Records** — Paginated table showing the `dfv.{source}` view. Server-side sorting (column validated against `information_schema.columns`, interpolated with `quote_ident`). Dates are formatted `YYYY-MM-DD` for correct lexicographic sort. Regex filters can be added per column. If the view cast fails (e.g. a field typed as `date` contains text), the error is shown inline rather than a blank page. - **Records** — Paginated table showing the `dfv.{source}` view. Server-side sorting (column validated against `information_schema.columns`, interpolated with `quote_ident`). Dates are formatted `YYYY-MM-DD` for correct lexicographic sort. Regex filters can be added per column. If the view cast fails (e.g. a field typed as `date` contains text), the error is shown inline rather than a blank page.
- **Pivot** — Interactive pivot/crosstab powered by [Perspective](https://perspective.finos.org/) (`@perspective-dev` client/viewer/datagrid v4.5.1, viewer-d3fc v4.4.1 — installed via npm). Loads all rows from the source view into an in-browser Perspective worker and renders a `<perspective-viewer>` web component. Supports grouping, splitting, filtering, sorting, and charting interactively. - **Pivot** — Interactive pivot/crosstab powered by [Perspective](https://perspective.finos.org/) (`@perspective-dev` v4.4.0, loaded from CDN at runtime). Loads all rows from the source view into an in-browser Perspective worker and renders a `<perspective-viewer>` web component. Supports grouping, splitting, filtering, sorting, and charting interactively.
**Toolbar (above the viewer):** **Toolbar (above the viewer):**
- Named layouts — saved per source in the `pivot_layouts` DB table. Each chip recalls the full viewer state including group_by, split_by, filters, expressions, selection mode, and expand depth. A blue **Save** button overwrites the active layout in place; **+ Save as…** saves to a new name. The × on each chip deletes it. - Named layouts — saved per source in the `pivot_layouts` DB table. Each chip recalls the full viewer state including group_by, split_by, filters, expressions, selection mode, and expand depth. A blue **Save** button overwrites the active layout in place; **+ Save as…** saves to a new name. The × on each chip deletes it.
- **depth: 0 1 2 3** — collapses or expands all grouped rows to the specified hierarchy level. Implemented via `view.set_depth(d)` + `plugin.draw(view)` (the only working mechanism found — `plugin_config.expand_depth` and `viewer.flush()` alone have no effect). - **depth: 0 1 2 3** — collapses or expands all grouped rows to the specified hierarchy level. Implemented via `view.set_depth(d)` + `plugin.draw(view)` (the only working mechanism found in v4.4.0 `plugin_config.expand_depth` and `viewer.flush()` alone have no effect).
- The Perspective built-in **selection mode button** (Read-Only / Select Row / Select Column / Select Region) defaults to **Select Region** on fresh load, set directly via `plugin.restore({ edit_mode: 'SELECT_REGION' })` after the viewer loads. - The Perspective built-in **selection mode button** (Read-Only / Select Row / Select Column / Select Region) defaults to **Select Region** on fresh load, set directly via `plugin.restore({ edit_mode: 'SELECT_REGION' })` after the viewer loads.
**Cell inspector (right panel):** **Cell inspector (right panel):**
@ -255,8 +237,6 @@ Built with React + Vite + Tailwind CSS. Compiled output goes to `public/`. The s
See `docs/perspective-pivot.md` for the full technical reference on controlling Perspective programmatically. See `docs/perspective-pivot.md` for the full technical reference on controlling Perspective programmatically.
- **Stacks** — Named unions of multiple sources. Each stack defines a field mapping (how source fields map to common output columns), an amount field, a date field, and an optional balance offset. The view-data endpoint unions the underlying source views and computes a running balance sorted by date. The Pivot page supports stacks as well as individual sources, with layouts stored in the same `pivot_layouts` table.
- **Log** — Global import log across all sources. Same expandable key detail and delete capability as the Import page, plus a source name column. - **Log** — Global import log across all sources. Same expandable key detail and delete capability as the Import page, plus a source name column.
--- ---

View File

@ -49,7 +49,7 @@ module.exports = (pool) => {
} }
}); });
// Set overrides for all selected records // Set overrides for all selected records and immediately merge into transformed
router.post('/bulk-overrides', async (req, res, next) => { router.post('/bulk-overrides', async (req, res, next) => {
try { try {
const { source_name, record_ids, overrides } = req.body; const { source_name, record_ids, overrides } = req.body;
@ -57,25 +57,25 @@ module.exports = (pool) => {
return res.status(400).json({ error: 'source_name, record_ids array, and overrides object required' }); return res.status(400).json({ error: 'source_name, record_ids array, and overrides object required' });
const idList = record_ids.map(id => parseInt(id)).join(','); const idList = record_ids.map(id => parseInt(id)).join(',');
const result = await pool.query( const result = await pool.query(
`SELECT bulk_set_record_overrides(${lit(source_name)}, ARRAY[${idList}]::int[], ${lit(overrides)}) as result` `SELECT bulk_set_record_overrides(${lit(source_name)}, ARRAY[${idList}]::int[], ${lit(overrides)}) as updated`
); );
res.json(result.rows[0].result); res.json({ updated: Number(result.rows[0].updated) });
} catch (err) { } catch (err) {
next(err); next(err);
} }
}); });
// Set overrides for a record // Set overrides for a record and immediately merge into transformed
router.put('/:id/overrides', async (req, res, next) => { router.put('/:id/overrides', async (req, res, next) => {
try { try {
const { overrides } = req.body; const { overrides } = req.body;
if (!overrides || typeof overrides !== 'object') if (!overrides || typeof overrides !== 'object')
return res.status(400).json({ error: 'overrides object required' }); return res.status(400).json({ error: 'overrides object required' });
const result = await pool.query( const result = await pool.query(
`SELECT set_record_overrides(${lit(parseInt(req.params.id))}, ${lit(overrides)}) as rec` `SELECT * FROM set_record_overrides(${lit(parseInt(req.params.id))}, ${lit(overrides)})`
); );
if (!result.rows[0].rec) return res.status(404).json({ error: 'Record not found' }); if (result.rows.length === 0) return res.status(404).json({ error: 'Record not found' });
res.json(result.rows[0].rec); res.json(result.rows[0]);
} catch (err) { } catch (err) {
next(err); next(err);
} }
@ -84,13 +84,13 @@ module.exports = (pool) => {
// Clear overrides and reprocess that record to restore computed values // Clear overrides and reprocess that record to restore computed values
router.delete('/:id/overrides', async (req, res, next) => { router.delete('/:id/overrides', async (req, res, next) => {
try { try {
const result = await pool.query( const rec = await pool.query(
`SELECT clear_record_overrides(${lit(parseInt(req.params.id))}) as rec` `SELECT * FROM clear_record_overrides(${lit(parseInt(req.params.id))})`
); );
if (!result.rows[0].rec) return res.status(404).json({ error: 'Record not found' }); if (rec.rows.length === 0) return res.status(404).json({ error: 'Record not found' });
const { source_name } = result.rows[0].rec; // Reprocess this record so transformed reflects rules/mappings without overrides
await pool.query( await pool.query(
`SELECT apply_transformations(${lit(source_name)}, ARRAY[${lit(parseInt(req.params.id))}::int], true)` `SELECT apply_transformations(${lit(rec.rows[0].source_name)}, ARRAY[${lit(parseInt(req.params.id))}::int], true)`
); );
const updated = await pool.query(`SELECT * FROM get_record(${lit(parseInt(req.params.id))})`); const updated = await pool.query(`SELECT * FROM get_record(${lit(parseInt(req.params.id))})`);
res.json(updated.rows[0]); res.json(updated.rows[0]);

View File

@ -170,32 +170,5 @@ module.exports = (pool) => {
} catch (err) { next(err); } } catch (err) { next(err); }
}); });
// Pivot layouts (same DB table as sources; FK was dropped to allow stack names)
router.get('/:name/layouts', async (req, res, next) => {
try {
const result = await pool.query(`SELECT * FROM list_pivot_layouts(${lit(req.params.name)})`);
res.json(result.rows);
} catch (err) { next(err); }
});
router.post('/:name/layouts', async (req, res, next) => {
try {
const { layout_name, config } = req.body;
if (!layout_name || !config) return res.status(400).json({ error: 'layout_name and config required' });
const result = await pool.query(
`SELECT * FROM save_pivot_layout(${lit(req.params.name)}, ${lit(layout_name)}, ${lit(config)})`
);
res.json(result.rows[0]);
} catch (err) { next(err); }
});
router.delete('/:name/layouts/:id', async (req, res, next) => {
try {
const result = await pool.query(`SELECT * FROM delete_pivot_layout(${lit(parseInt(req.params.id))})`);
if (result.rows.length === 0) return res.status(404).json({ error: 'Layout not found' });
res.json({ success: true });
} catch (err) { next(err); }
});
return router; return router;
}; };

View File

@ -3,7 +3,7 @@
* Simple REST API for data transformation * Simple REST API for data transformation
*/ */
require('dotenv').config({ quiet: true }); require('dotenv').config();
const express = require('express'); const express = require('express');
const { Pool } = require('pg'); const { Pool } = require('pg');
@ -16,8 +16,7 @@ const pool = new Pool({
port: process.env.DB_PORT, port: process.env.DB_PORT,
database: process.env.DB_NAME, database: process.env.DB_NAME,
user: process.env.DB_USER, user: process.env.DB_USER,
password: process.env.DB_PASSWORD, password: process.env.DB_PASSWORD
options: '-c search_path=dataflow,public'
}); });
// Middleware // Middleware
@ -32,6 +31,11 @@ app.use('/api', auth);
const path = require('path'); const path = require('path');
app.use(express.static(path.join(__dirname, '../public'))); app.use(express.static(path.join(__dirname, '../public')));
// Set search path for all queries
pool.on('connect', (client) => {
client.query('SET search_path TO dataflow, public');
});
// Test database connection // Test database connection
pool.query('SELECT NOW()', (err, res) => { pool.query('SELECT NOW()', (err, res) => {
if (err) { if (err) {

File diff suppressed because it is too large Load Diff

View File

@ -1,40 +0,0 @@
--
-- Migration: add overrides column to records
--
-- Separates the three data layers:
-- data — original import values, never mutated
-- transformed — rule/mapping output fields only (delta)
-- overrides — manual user overrides (highest precedence)
--
-- Consumers merge as: data || COALESCE(transformed,'{}') || COALESCE(overrides,'{}')
--
-- Safe to run multiple times (IF NOT EXISTS guards).
--
SET search_path TO dataflow, public;
-- 1. Add overrides column
ALTER TABLE dataflow.records
ADD COLUMN IF NOT EXISTS overrides JSONB;
-- 2. Add partial GIN index (only indexes rows that have overrides)
CREATE INDEX IF NOT EXISTS idx_records_overrides
ON dataflow.records USING gin(overrides)
WHERE overrides IS NOT NULL;
-- 3. Redeploy functions (CREATE OR REPLACE — non-destructive)
\i functions.sql
-- 4. Reprocess all sources to strip stale data keys from transformed
-- (apply_transformations now writes only rule additions, not data || additions)
DO $$
DECLARE
src TEXT;
result JSON;
BEGIN
FOR src IN SELECT name FROM dataflow.sources ORDER BY name LOOP
SELECT dataflow.reprocess_records(src) INTO result;
RAISE NOTICE 'Reprocessed %: %', src, result;
END LOOP;
END;
$$;

View File

@ -1,4 +0,0 @@
-- Drop the foreign key from pivot_layouts.source_name so stack view names can also
-- be used as layout keys (stacks are not rows in the sources table).
ALTER TABLE dataflow.pivot_layouts
DROP CONSTRAINT pivot_layouts_source_name_fkey;

View File

@ -37,27 +37,26 @@ CREATE TABLE records (
-- Data -- Data
data JSONB NOT NULL, -- Original imported data data JSONB NOT NULL, -- Original imported data
constraint_key JSONB, -- Fields that uniquely identify this record (set on import) constraint_key JSONB, -- Fields that uniquely identify this record (set on import)
transformed JSONB, -- Rule/mapping output fields only (delta, not raw data) transformed JSONB, -- Data after transformations applied
overrides JSONB, -- Manual user overrides (highest precedence)
-- Metadata -- Metadata
import_id INTEGER REFERENCES import_log(id) ON DELETE CASCADE, import_id INTEGER REFERENCES import_log(id) ON DELETE CASCADE, -- Which import batch this came from
imported_at TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP, imported_at TIMESTAMPTZ DEFAULT CURRENT_TIMESTAMP,
transformed_at TIMESTAMPTZ transformed_at TIMESTAMPTZ,
); );
COMMENT ON TABLE records IS 'Imported records with raw and transformed data'; COMMENT ON TABLE records IS 'Imported records with raw and transformed data';
COMMENT ON COLUMN records.data IS 'Original data as imported — never mutated after import'; COMMENT ON COLUMN records.data IS 'Original data as imported';
COMMENT ON COLUMN records.constraint_key IS 'JSONB object of constraint field values — uniquely identifies this record within its source'; COMMENT ON COLUMN records.constraint_key IS 'JSONB object of constraint field values — uniquely identifies this record within its source';
COMMENT ON COLUMN records.transformed IS 'Rule/mapping output fields only (delta); merge as data || transformed || overrides for final values'; COMMENT ON COLUMN records.transformed IS 'Data after applying transformation rules';
COMMENT ON COLUMN records.overrides IS 'Manual user overrides; highest precedence in data || transformed || overrides merge';
-- Indexes -- Indexes
CREATE INDEX idx_records_source ON records(source_name); CREATE INDEX idx_records_source ON records(source_name);
CREATE INDEX idx_records_constraint ON records USING gin(constraint_key); CREATE INDEX idx_records_constraint ON records USING gin(constraint_key);
CREATE INDEX idx_records_data ON records USING gin(data); CREATE INDEX idx_records_data ON records USING gin(data);
CREATE INDEX idx_records_transformed ON records USING gin(transformed); CREATE INDEX idx_records_transformed ON records USING gin(transformed);
CREATE INDEX idx_records_overrides ON records USING gin(overrides) WHERE overrides IS NOT NULL;
------------------------------------------------------ ------------------------------------------------------
-- Table: rules -- Table: rules

View File

@ -1,22 +1,27 @@
# Perspective Pivot — Technical Reference # Perspective Pivot — Technical Reference
Packages: `@perspective-dev` client/viewer/viewer-datagrid at **v4.5.1**, viewer-d3fc at **v4.4.1** — installed via npm. API notes that reference v4.4.0 behaviour have not been re-verified at 4.5.1 but are believed to still apply. Version tested: `@perspective-dev` v4.4.0 (client, viewer, viewer-datagrid, viewer-d3fc), loaded from CDN.
This document captures everything learned about controlling Perspective programmatically. The official docs are incomplete for some of these APIs — treat this as a ground-truth supplement. This document captures everything learned about controlling Perspective programmatically. The official docs are incomplete for some of these APIs — treat this as a ground-truth supplement.
--- ---
## Loading via npm ## Loading from CDN
```js ```js
import perspective from '@perspective-dev/client/inline' const [{ default: perspective }] = await Promise.all([
import '@perspective-dev/viewer/inline' import('https://cdn.jsdelivr.net/npm/@perspective-dev/client@4.4.0/dist/cdn/perspective.js'),
import '@perspective-dev/viewer-datagrid' import('https://cdn.jsdelivr.net/npm/@perspective-dev/viewer@4.4.0/dist/cdn/perspective-viewer.js'),
import '@perspective-dev/viewer-d3fc' import('https://cdn.jsdelivr.net/npm/@perspective-dev/viewer-datagrid@4.4.0/dist/cdn/perspective-viewer-datagrid.js'),
import '@perspective-dev/viewer/themes' import('https://cdn.jsdelivr.net/npm/@perspective-dev/viewer-d3fc@4.4.0/dist/cdn/perspective-viewer-d3fc.js'),
])
``` ```
The `inline` builds embed WebAssembly directly into the JS bundle — no separate `.wasm` file to serve. viewer-datagrid and viewer-d3fc have no inline variant; they import normally. viewer-d3fc is currently at v4.4.1 (no v4.5.x release yet); its chart plugins register but may not appear in the viewer due to an API change in v4.5.x's `registerPlugin`. Stylesheet:
```html
<link rel="stylesheet" crossorigin="anonymous"
href="https://cdn.jsdelivr.net/npm/@perspective-dev/viewer/dist/css/themes.css" />
```
--- ---

View File

@ -18,11 +18,11 @@
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"bcrypt": "^6.0.0", "bcrypt": "^6.0.0",
"csv-parse": "^6.2.1", "csv-parse": "^5.5.2",
"dotenv": "^17.4.2", "dotenv": "^16.3.1",
"express": "^5.2.1", "express": "^4.18.2",
"multer": "^2.1.1", "multer": "^1.4.5-lts.1",
"pg": "^8.21.0" "pg": "^8.11.3"
}, },
"devDependencies": { "devDependencies": {
"nodemon": "^3.0.1" "nodemon": "^3.0.1"

View File

@ -1,25 +1,16 @@
# Dataflow UI # React + Vite
React + Vite + Tailwind CSS frontend for Dataflow. This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
## Development Currently, two official plugins are available:
```bash - [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Oxc](https://oxc.rs)
npm install - [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/)
npm run dev # dev server on :5173, proxies /api to :3020
```
## Build ## React Compiler
```bash The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation).
npm run build # outputs to ../public/
```
The Express server serves `../public/` as static files — no separate web server needed in production. ## Expanding the ESLint configuration
## Key packages If you are developing a production application, we recommend using TypeScript with type-aware lint rules enabled. Check out the [TS template](https://github.com/vitejs/vite/tree/main/packages/create-vite/template-react-ts) for information on how to integrate TypeScript and [`typescript-eslint`](https://typescript-eslint.io) in your project.
- `react` / `react-router-dom` — SPA routing
- `@perspective-dev/client`, `viewer`, `viewer-datagrid`, `viewer-d3fc` — pivot table (npm, inline WASM builds)
- `tailwindcss` — utility CSS
- `sql-formatter` — SQL display formatting

View File

@ -10,26 +10,22 @@
"preview": "vite preview" "preview": "vite preview"
}, },
"dependencies": { "dependencies": {
"@perspective-dev/client": "^4.5.1", "react": "^19.2.4",
"@perspective-dev/viewer": "^4.5.1", "react-dom": "^19.2.4",
"@perspective-dev/viewer-d3fc": "^4.4.1", "react-router-dom": "^7.13.2",
"@perspective-dev/viewer-datagrid": "^4.5.1", "sql-formatter": "^15.7.3"
"react": "^19.2.7",
"react-dom": "^19.2.7",
"react-router-dom": "^7.17.0",
"sql-formatter": "^15.8.1"
}, },
"devDependencies": { "devDependencies": {
"@eslint/js": "^9.39.4", "@eslint/js": "^9.39.4",
"@tailwindcss/vite": "^4.3.1", "@tailwindcss/vite": "^4.2.2",
"@types/react": "^19.2.17", "@types/react": "^19.2.14",
"@types/react-dom": "^19.2.3", "@types/react-dom": "^19.2.3",
"@vitejs/plugin-react": "^6.0.2", "@vitejs/plugin-react": "^6.0.1",
"eslint": "^9.39.4", "eslint": "^9.39.4",
"eslint-plugin-react-hooks": "^7.1.1", "eslint-plugin-react-hooks": "^7.0.1",
"eslint-plugin-react-refresh": "^0.5.2", "eslint-plugin-react-refresh": "^0.5.2",
"globals": "^17.6.0", "globals": "^17.4.0",
"tailwindcss": "^4.3.1", "tailwindcss": "^4.2.2",
"vite": "^8.0.16" "vite": "^8.0.1"
} }
} }

View File

@ -1,8 +1,6 @@
import { useState, useEffect } from 'react' import { useState, useEffect } from 'react'
import { BrowserRouter, Routes, Route, Navigate } from 'react-router-dom' import { BrowserRouter, Routes, Route, NavLink, Navigate } from 'react-router-dom'
import { api, setCredentials, clearCredentials } from './api' import { api, setCredentials, clearCredentials } from './api'
import StatusBar from './components/StatusBar.jsx'
import Sidebar from './components/Sidebar.jsx'
import Login from './pages/Login' import Login from './pages/Login'
import Sources from './pages/Sources' import Sources from './pages/Sources'
import Import from './pages/Import' import Import from './pages/Import'
@ -14,14 +12,24 @@ import Pivot from './pages/Pivot'
import Remap from './pages/Remap' import Remap from './pages/Remap'
import Stacks from './pages/Stacks' import Stacks from './pages/Stacks'
const NAV = [
{ to: '/sources', label: 'Sources' },
{ to: '/import', label: 'Import' },
{ to: '/rules', label: 'Rules' },
{ to: '/mappings', label: 'Mappings' },
{ to: '/remap', label: 'Remap' },
{ to: '/records', label: 'Records' },
{ to: '/pivot', label: 'Pivot' },
{ to: '/stacks', label: 'Stacks' },
{ to: '/log', label: 'Log' },
]
export default function App() { export default function App() {
const [authed, setAuthed] = useState(false) const [authed, setAuthed] = useState(false)
const [loginUser, setLoginUser] = useState('') const [loginUser, setLoginUser] = useState('')
const [sources, setSources] = useState([]) const [sources, setSources] = useState([])
const [stacks, setStacks] = useState([])
const [source, setSource] = useState(() => localStorage.getItem('selectedSource') || '') const [source, setSource] = useState(() => localStorage.getItem('selectedSource') || '')
const [selectedStack, setSelectedStack] = useState(null) const [sidebarOpen, setSidebarOpen] = useState(false)
const [sidebarExpanded, setSidebarExpanded] = useState(() => localStorage.getItem('df_sidebar') !== 'collapsed')
// Sets of names whose dfv view is out of sync with current definitions // Sets of names whose dfv view is out of sync with current definitions
const [staleSources, setStaleSources] = useState(new Set()) const [staleSources, setStaleSources] = useState(new Set())
const [staleStacks, setStaleStacks] = useState(new Set()) const [staleStacks, setStaleStacks] = useState(new Set())
@ -30,14 +38,14 @@ export default function App() {
async function handleLogin(user, pass) { async function handleLogin(user, pass) {
setCredentials(user, pass) setCredentials(user, pass)
const s = await api.getSources() await api.getSources().then(s => {
sessionStorage.setItem('df_user', user) sessionStorage.setItem('df_user', user)
sessionStorage.setItem('df_pass', pass) sessionStorage.setItem('df_pass', pass)
setSources(s) setSources(s)
if (!source && s.length > 0) setSource(s[0].name) if (!source && s.length > 0) setSource(s[0].name)
setAuthed(true) setAuthed(true)
setLoginUser(user) setLoginUser(user)
api.getStacks().then(setStacks).catch(() => {}) })
} }
function handleLogout() { function handleLogout() {
@ -47,17 +55,11 @@ export default function App() {
setAuthed(false) setAuthed(false)
setLoginUser('') setLoginUser('')
setSources([]) setSources([])
setStacks([])
setSelectedStack(null)
setStaleSources(new Set()) setStaleSources(new Set())
setStaleStacks(new Set()) setStaleStacks(new Set())
setReprocessSources(new Set()) setReprocessSources(new Set())
} }
function refreshStacks() {
api.getStacks().then(setStacks).catch(() => {})
}
// Load initial stale state from DB once on login // Load initial stale state from DB once on login
useEffect(() => { useEffect(() => {
if (!authed) return if (!authed) return
@ -117,29 +119,83 @@ export default function App() {
if (source) localStorage.setItem('selectedSource', source) if (source) localStorage.setItem('selectedSource', source)
}, [source]) }, [source])
useEffect(() => {
localStorage.setItem('df_sidebar', sidebarExpanded ? 'expanded' : 'collapsed')
}, [sidebarExpanded])
if (!authed) return <Login onLogin={handleLogin} /> if (!authed) return <Login onLogin={handleLogin} />
const sidebar = (
<div className="flex flex-col h-full">
{/* Header */}
<div className="px-4 py-3 border-b border-gray-200">
<div className="flex items-center justify-between">
<span className="text-sm font-semibold text-gray-800 tracking-wide uppercase">Dataflow</span>
<button onClick={() => setSidebarOpen(false)} className="md:hidden text-gray-400 hover:text-gray-600 leading-none" title="Close"></button>
</div>
<div className="flex items-center justify-between mt-1">
<span className="text-xs text-gray-400">{loginUser}</span>
<button onClick={handleLogout} className="text-xs text-gray-400 hover:text-red-500" title="Sign out">Sign out</button>
</div>
</div>
{/* Source selector */}
<div className="px-3 py-3 border-b border-gray-200">
<div className="flex items-center justify-between mb-1">
<label className="text-xs text-gray-500">Source</label>
<NavLink to="/sources?new=1" className="text-xs text-blue-400 hover:text-blue-600 leading-none" title="New source" onClick={() => setSidebarOpen(false)}>+</NavLink>
</div>
<select
className="w-full text-sm border border-gray-200 rounded px-2 py-1 bg-white focus:outline-none focus:border-blue-400"
value={source}
onChange={e => setSource(e.target.value)}
>
{sources.length === 0 && <option value=""></option>}
{sources.map(s => <option key={s.name} value={s.name}>{s.name}</option>)}
</select>
</div>
{/* Nav */}
<nav className="flex-1 py-2">
{NAV.map(({ to, label }) => (
<NavLink
key={to}
to={to}
onClick={() => setSidebarOpen(false)}
className={({ isActive }) =>
`block px-4 py-2 text-sm ${isActive
? 'bg-blue-50 text-blue-700 font-medium'
: 'text-gray-600 hover:bg-gray-50'}`
}
>
{label}
</NavLink>
))}
</nav>
</div>
)
return ( return (
<BrowserRouter> <BrowserRouter>
<div className="flex h-screen"> <div className="flex h-screen bg-gray-50">
<Sidebar {/* Mobile overlay */}
expanded={sidebarExpanded} {sidebarOpen && (
setExpanded={setSidebarExpanded} <div className="fixed inset-0 z-20 bg-black/30 md:hidden" onClick={() => setSidebarOpen(false)} />
loginUser={loginUser} )}
onLogout={handleLogout}
/> {/* Sidebar — fixed on mobile, static on desktop */}
<div className={`
fixed inset-y-0 left-0 z-30 w-44 bg-white border-r border-gray-200 transform transition-transform duration-200
md:static md:translate-x-0 md:z-auto md:transition-none
${sidebarOpen ? 'translate-x-0' : '-translate-x-full'}
`}>
{sidebar}
</div>
{/* Main */} {/* Main */}
<div className="flex-1 overflow-hidden flex flex-col min-w-0"> <div className="flex-1 overflow-auto flex flex-col min-w-0">
<StatusBar {/* Mobile top bar */}
sources={sources} source={source} setSource={setSource} <div className="md:hidden flex items-center px-3 py-2 bg-white border-b border-gray-200">
stacks={stacks} selectedStack={selectedStack} setSelectedStack={setSelectedStack} <button onClick={() => setSidebarOpen(true)} className="text-gray-500 hover:text-gray-700 mr-3 text-lg leading-none"></button>
/> <span className="text-sm font-semibold text-gray-800 tracking-wide uppercase">Dataflow</span>
</div>
{(staleSources.size > 0 || staleStacks.size > 0) && ( {(staleSources.size > 0 || staleStacks.size > 0) && (
<div className="bg-amber-50 border-b border-amber-200 px-4 py-1.5 text-xs text-amber-800 flex flex-wrap items-center gap-x-3 gap-y-1"> <div className="bg-amber-50 border-b border-amber-200 px-4 py-1.5 text-xs text-amber-800 flex flex-wrap items-center gap-x-3 gap-y-1">
@ -198,8 +254,8 @@ export default function App() {
<Route path="/mappings" element={<Mappings source={source} onNeedsReprocess={markNeedsReprocess} />} /> <Route path="/mappings" element={<Mappings source={source} onNeedsReprocess={markNeedsReprocess} />} />
<Route path="/remap" element={<Remap />} /> <Route path="/remap" element={<Remap />} />
<Route path="/records" element={<Records source={source} />} /> <Route path="/records" element={<Records source={source} />} />
<Route path="/pivot" element={<Pivot source={source} selectedStack={selectedStack} setSelectedStack={setSelectedStack} />} /> <Route path="/pivot" element={<Pivot source={source} />} />
<Route path="/stacks" element={<Stacks sources={sources} onStackStale={markStackStale} onStackViewGenerated={clearStackStale} onStacksChange={refreshStacks} />} /> <Route path="/stacks" element={<Stacks sources={sources} onStackStale={markStackStale} onStackViewGenerated={clearStackStale} />} />
<Route path="/log" element={<Log />} /> <Route path="/log" element={<Log />} />
</Routes> </Routes>
</div> </div>

View File

@ -108,16 +108,11 @@ export const api = {
getMappingsByOutputField: (col, val) => request('GET', `/mappings/outputs/${encodeURIComponent(col)}/${encodeURIComponent(val)}`), getMappingsByOutputField: (col, val) => request('GET', `/mappings/outputs/${encodeURIComponent(col)}/${encodeURIComponent(val)}`),
remapOutputField: (col, from_val, to_val) => request('POST', '/mappings/remap-field', { col, from_val, to_val }), remapOutputField: (col, from_val, to_val) => request('POST', '/mappings/remap-field', { col, from_val, to_val }),
// Pivot layouts (sources) // Pivot layouts
getPivotLayouts: (source) => request('GET', `/sources/${source}/layouts`), getPivotLayouts: (source) => request('GET', `/sources/${source}/layouts`),
savePivotLayout: (source, layout_name, config) => request('POST', `/sources/${source}/layouts`, { layout_name, config }), savePivotLayout: (source, layout_name, config) => request('POST', `/sources/${source}/layouts`, { layout_name, config }),
deletePivotLayout: (source, id) => request('DELETE', `/sources/${source}/layouts/${id}`), deletePivotLayout: (source, id) => request('DELETE', `/sources/${source}/layouts/${id}`),
// Pivot layouts (stacks)
getStackPivotLayouts: (name) => request('GET', `/stacks/${name}/layouts`),
saveStackPivotLayout: (name, layout_name, config) => request('POST', `/stacks/${name}/layouts`, { layout_name, config }),
deleteStackPivotLayout: (name, id) => request('DELETE', `/stacks/${name}/layouts/${id}`),
// Stacks // Stacks
getStacks: () => request('GET', '/stacks'), getStacks: () => request('GET', '/stacks'),
getStack: (name) => request('GET', `/stacks/${name}`), getStack: (name) => request('GET', `/stacks/${name}`),

View File

@ -1,183 +0,0 @@
import { NavLink } from 'react-router-dom'
const NAV = [
{
to: '/sources',
label: 'Sources',
icon: (
<svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
<ellipse cx="10" cy="5.5" rx="7" ry="2.5"/>
<path d="M3 5.5v9c0 1.4 3.1 2.5 7 2.5s7-1.1 7-2.5v-9"/>
<path d="M3 10.5c0 1.4 3.1 2.5 7 2.5s7-1.1 7-2.5"/>
</svg>
),
},
{
to: '/import',
label: 'Import',
icon: (
<svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
<line x1="10" y1="3" x2="10" y2="14"/>
<polyline points="6,10 10,14 14,10"/>
<line x1="3" y1="18" x2="17" y2="18"/>
</svg>
),
},
{
to: '/rules',
label: 'Rules',
icon: (
<svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
<polyline points="6,7 2,10 6,13"/>
<polyline points="14,7 18,10 14,13"/>
<line x1="12" y1="4" x2="8" y2="16"/>
</svg>
),
},
{
to: '/mappings',
label: 'Mappings',
icon: (
<svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
<line x1="2" y1="7" x2="12" y2="7"/>
<polyline points="9,4 12,7 9,10"/>
<line x1="8" y1="13" x2="18" y2="13"/>
<polyline points="11,10 14,13 11,16"/>
</svg>
),
},
{
to: '/remap',
label: 'Remap',
icon: (
<svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
<polyline points="2,8 2,4 6,4"/>
<path d="M2 4a8 8 0 0 1 14 2"/>
<polyline points="18,12 18,16 14,16"/>
<path d="M18 16a8 8 0 0 1-14-2"/>
</svg>
),
},
{
to: '/records',
label: 'Records',
icon: (
<svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
<rect x="2" y="3" width="16" height="14" rx="1.5"/>
<line x1="2" y1="8" x2="18" y2="8"/>
<line x1="7" y1="8" x2="7" y2="17"/>
</svg>
),
},
{
to: '/pivot',
label: 'Pivot',
icon: (
<svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
<rect x="2" y="2" width="7" height="7" rx="1"/>
<rect x="11" y="2" width="7" height="7" rx="1"/>
<rect x="2" y="11" width="7" height="7" rx="1"/>
<rect x="11" y="11" width="7" height="7" rx="1"/>
</svg>
),
},
{
to: '/stacks',
label: 'Stacks',
icon: (
<svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
<polygon points="10,2 18,6 10,10 2,6"/>
<polyline points="2,10 10,14 18,10"/>
<polyline points="2,14 10,18 18,14"/>
</svg>
),
},
{
to: '/log',
label: 'Log',
icon: (
<svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
<circle cx="10" cy="10" r="8"/>
<polyline points="10,5 10,10 14,12"/>
</svg>
),
},
]
export default function Sidebar({ expanded, setExpanded, loginUser, onLogout }) {
return (
<div
className="bg-white border-r border-gray-200 flex flex-col shrink-0 overflow-hidden transition-all duration-150"
style={{ width: expanded ? 200 : 48 }}
>
{/* Header */}
<div className="h-12 flex items-center px-3 border-b border-gray-100 gap-2 shrink-0">
<button
onClick={() => setExpanded(e => !e)}
className="w-8 h-8 flex items-center justify-center rounded hover:bg-gray-100 text-gray-400 shrink-0"
title="Toggle sidebar"
>
<svg width="16" height="16" viewBox="0 0 16 16" fill="none">
<rect x="1" y="3" width="14" height="1.5" rx="0.75" fill="currentColor"/>
<rect x="1" y="7.25" width="14" height="1.5" rx="0.75" fill="currentColor"/>
<rect x="1" y="11.5" width="14" height="1.5" rx="0.75" fill="currentColor"/>
</svg>
</button>
<span
className="text-xs font-semibold text-gray-600 tracking-wide uppercase whitespace-nowrap transition-opacity duration-100"
style={{ opacity: expanded ? 1 : 0, pointerEvents: expanded ? 'auto' : 'none' }}
>
Dataflow
</span>
</div>
{/* Nav */}
<nav className="flex flex-col gap-0.5 p-2 flex-1">
{NAV.map(({ to, label, icon }) => (
<NavLink
key={to}
to={to}
title={!expanded ? label : undefined}
className={({ isActive }) =>
`flex items-center gap-3 px-2 py-2 rounded w-full transition-colors ${
isActive
? 'bg-blue-50 text-blue-700'
: 'text-gray-500 hover:bg-gray-100 hover:text-gray-800'
}`
}
>
<span className="shrink-0">{icon}</span>
<span
className="text-sm whitespace-nowrap transition-opacity duration-100"
style={{ opacity: expanded ? 1 : 0, pointerEvents: expanded ? 'auto' : 'none', width: expanded ? 'auto' : 0, overflow: 'hidden' }}
>
{label}
</span>
</NavLink>
))}
</nav>
{/* User / logout */}
<div className="border-t border-gray-100 px-3 py-2.5 flex items-center gap-2 shrink-0 overflow-hidden">
<div
className="w-6 h-6 rounded-full bg-gray-200 text-gray-500 flex items-center justify-center shrink-0 text-xs font-medium"
title={!expanded ? loginUser : undefined}
>
{loginUser ? loginUser[0].toUpperCase() : '?'}
</div>
<div
className="flex-1 flex items-center justify-between min-w-0 transition-opacity duration-100"
style={{ opacity: expanded ? 1 : 0, pointerEvents: expanded ? 'auto' : 'none', width: expanded ? 'auto' : 0, overflow: 'hidden' }}
>
<span className="text-xs text-gray-400 truncate">{loginUser}</span>
<button
onClick={onLogout}
className="text-xs text-gray-400 hover:text-red-500 ml-2 shrink-0"
>
Sign out
</button>
</div>
</div>
</div>
)
}

View File

@ -1,73 +0,0 @@
import { NavLink } from 'react-router-dom'
import useTheme from '../theme.jsx'
export default function StatusBar({ sources = [], source, setSource, stacks = [], selectedStack, setSelectedStack }) {
const { dark, setDark } = useTheme()
return (
<div className="bg-white border-b border-gray-200 px-3 h-9 flex items-center gap-3 shrink-0 text-xs">
<span className="text-gray-400">Source</span>
<select
value={source || ''}
onChange={e => setSource(e.target.value)}
disabled={sources.length === 0}
className="border border-gray-200 rounded px-2 py-0.5 bg-white focus:outline-none focus:border-blue-400"
>
{sources.length === 0
? <option value=""> no sources </option>
: sources.map(s => <option key={s.name} value={s.name}>{s.name}</option>)}
</select>
<NavLink
to="/sources?new=1"
className="text-blue-400 hover:text-blue-600 leading-none"
title="New source"
>+</NavLink>
{stacks.length > 0 && (
<>
<span className="text-gray-200">|</span>
<span className="text-gray-400">Stacks</span>
{stacks.map(s => (
<button
key={s.name}
onClick={() => setSelectedStack(n => n === s.name ? null : s.name)}
className={`rounded px-2 py-0.5 border transition-colors ${
selectedStack === s.name
? 'bg-purple-50 border-purple-300 text-purple-700'
: 'bg-white border-gray-200 text-gray-500 hover:border-gray-400'
}`}
>
{s.name}
</button>
))}
</>
)}
<div className="ml-auto">
<button
onClick={() => setDark(d => !d)}
className="w-6 h-6 flex items-center justify-center rounded hover:bg-gray-100 text-gray-500"
title={dark ? 'Switch to light mode' : 'Switch to dark mode'}
>
{dark ? (
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
<circle cx="12" cy="12" r="4"/>
<line x1="12" y1="2" x2="12" y2="5"/>
<line x1="12" y1="19" x2="12" y2="22"/>
<line x1="4.93" y1="4.93" x2="7.05" y2="7.05"/>
<line x1="16.95" y1="16.95" x2="19.07" y2="19.07"/>
<line x1="2" y1="12" x2="5" y2="12"/>
<line x1="19" y1="12" x2="22" y2="12"/>
<line x1="4.93" y1="19.07" x2="7.05" y2="16.95"/>
<line x1="16.95" y1="7.05" x2="19.07" y2="4.93"/>
</svg>
) : (
<svg width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" strokeWidth="2" strokeLinecap="round" strokeLinejoin="round">
<path d="M21 12.79A9 9 0 1 1 11.21 3 7 7 0 0 0 21 12.79z"/>
</svg>
)}
</button>
</div>
</div>
)
}

View File

@ -1,97 +1,6 @@
@import "tailwindcss"; @import "tailwindcss";
:root, .light {
--bg-primary: #f3f4f6;
--bg-secondary: #ffffff;
--bg-tertiary: #f9fafb;
--text-primary: #1f2937;
--text-secondary: #374151;
--text-muted: #9ca3af;
--border-color: #e5e7eb;
--border-light: #f3f4f6;
--accent-bg: #eff6ff;
--accent-text: #1d4ed8;
}
/* Dark palette tuned to Perspective's "Pro Dark" theme:
bg #242526, tooltip #2a2c2f, gridline #3b3f46, inactive #61656e,
inactive border #4c505b, active #2770a9, legend #c5c9d0. */
.dark {
--bg-primary: #242526;
--bg-secondary: #2a2c2f;
--bg-tertiary: #3b3f46;
--text-primary: #ffffff;
--text-secondary: #c5c9d0;
--text-muted: #61656e;
--border-color: #4c505b;
--border-light: #3b3f46;
--accent-bg: rgba(39, 113, 170, 0.32);
--accent-text: #4778c2;
}
body { body {
margin: 0; margin: 0;
font-family: system-ui, -apple-system, sans-serif; font-family: system-ui, -apple-system, sans-serif;
background-color: var(--bg-primary);
color: var(--text-primary);
} }
.dark .bg-white { background-color: var(--bg-secondary); }
.dark .bg-gray-50 { background-color: var(--bg-tertiary); }
.dark .bg-gray-100 { background-color: var(--bg-tertiary); }
.dark .bg-gray-200 { background-color: var(--bg-tertiary); }
.dark .bg-gray-300 { background-color: var(--bg-tertiary); }
.dark .text-gray-300 { color: var(--text-muted); }
.dark .text-gray-400 { color: var(--text-muted); }
.dark .text-gray-500 { color: var(--text-muted); }
.dark .text-gray-600 { color: var(--text-secondary); }
.dark .text-gray-700 { color: var(--text-secondary); }
.dark .text-gray-800 { color: var(--text-primary); }
.dark .text-gray-900 { color: var(--text-primary); }
.dark .bg-blue-50 { background-color: var(--accent-bg); }
.dark .bg-blue-100 { background-color: var(--accent-bg); }
.dark .text-blue-400 { color: var(--accent-text); }
.dark .text-blue-600 { color: var(--accent-text); }
.dark .text-blue-700 { color: var(--accent-text); }
.dark .text-blue-800 { color: var(--accent-text); }
.dark .border-blue-200 { border-color: var(--accent-text); }
.dark .border-blue-300 { border-color: var(--accent-text); }
.dark .hover\:bg-blue-50:hover { background-color: var(--accent-bg); }
/* Status accents — desaturated to sit on Pro Dark's neutral background */
.dark .bg-green-50 { background-color: #1a3d2c; }
.dark .text-green-600 { color: #6ee7b7; }
.dark .text-green-700 { color: #6ee7b7; }
.dark .text-green-400 { color: #6ee7b7; }
.dark .bg-amber-50 { background-color: #3a2e14; }
.dark .text-amber-800 { color: #f5c66f; }
.dark .border-amber-200 { border-color: #5a4a26; }
.dark .bg-amber-200 { background-color: #5a4a26; }
.dark .hover\:bg-amber-300:hover { background-color: #6b5830; }
.dark .bg-red-50 { background-color: #3d1f1f; }
.dark .text-red-500 { color: #ff9485; }
.dark .text-red-700 { color: #ff9485; }
.dark .border-gray-100 { border-color: var(--border-light); }
.dark .border-gray-200 { border-color: var(--border-color); }
.dark .border-gray-300 { border-color: var(--border-color); }
.dark .border-blue-100 { border-color: var(--border-color); }
.dark .border-b { border-color: var(--border-color); }
.dark .border-t { border-color: var(--border-color); }
.dark .border-r { border-color: var(--border-color); }
.dark .border-l { border-color: var(--border-color); }
.dark .hover\:bg-gray-50:hover { background-color: var(--bg-tertiary); }
.dark .hover\:bg-gray-100:hover { background-color: var(--bg-tertiary); }
.dark .hover\:bg-gray-200:hover { background-color: var(--bg-tertiary); }
.dark .hover\:text-gray-500:hover { color: var(--text-secondary); }
.dark .hover\:text-gray-600:hover { color: var(--text-secondary); }
.dark .hover\:text-gray-700:hover { color: var(--text-primary); }
.dark .hover\:text-gray-800:hover { color: var(--text-primary); }
.dark .hover\:border-gray-300:hover { border-color: var(--border-color); }
.dark .hover\:border-gray-400:hover { border-color: var(--border-color); }
.dark .focus\:border-gray-300:focus { border-color: var(--border-color); }
.dark .focus\:border-blue-400:focus { border-color: var(--accent-text); }
.dark ::selection { background-color: var(--accent-bg); color: var(--text-primary); }
.dark input { background-color: var(--bg-secondary); color: var(--text-primary); border-color: var(--border-color); }
.dark select { background-color: var(--bg-secondary); color: var(--text-primary); border-color: var(--border-color); }
.dark textarea { background-color: var(--bg-secondary); color: var(--text-primary); border-color: var(--border-color); }
.dark .bg-transparent { background-color: transparent; }

View File

@ -1,13 +1,10 @@
import { StrictMode } from 'react' import { StrictMode } from 'react'
import { createRoot } from 'react-dom/client' import { createRoot } from 'react-dom/client'
import { ThemeProvider } from './theme.jsx'
import './index.css' import './index.css'
import App from './App.jsx' import App from './App.jsx'
createRoot(document.getElementById('root')).render( createRoot(document.getElementById('root')).render(
<StrictMode> <StrictMode>
<ThemeProvider>
<App /> <App />
</ThemeProvider>
</StrictMode>, </StrictMode>,
) )

View File

@ -1,19 +1,33 @@
import { useEffect, useRef, useState, useCallback } from 'react' import { useEffect, useRef, useState, useCallback } from 'react'
import { api } from '../api' import { api } from '../api'
import useTheme from '../theme.jsx'
import perspective from '@perspective-dev/client/inline'
import '@perspective-dev/viewer/inline'
import '@perspective-dev/viewer-datagrid'
import '@perspective-dev/viewer-d3fc'
import '@perspective-dev/viewer/themes'
async function fetchAllRows(source) { async function fetchAllRows(source) {
const res = await api.getViewData(source, 100000, 0) const res = await api.getViewData(source, 100000, 0)
return res.rows || [] return res.rows || []
} }
let perspectivePromise = null
function loadPerspective() { function loadPerspective() {
return Promise.resolve(perspective) if (perspectivePromise) return perspectivePromise
perspectivePromise = (async () => {
if (!document.getElementById('psp-theme')) {
const link = document.createElement('link')
link.id = 'psp-theme'
link.rel = 'stylesheet'
link.crossOrigin = 'anonymous'
link.href = 'https://cdn.jsdelivr.net/npm/@perspective-dev/viewer/dist/css/themes.css'
document.head.appendChild(link)
}
const [{ default: perspective }] = await Promise.all([
import(/* @vite-ignore */ 'https://cdn.jsdelivr.net/npm/@perspective-dev/client@4.4.0/dist/cdn/perspective.js'),
import(/* @vite-ignore */ 'https://cdn.jsdelivr.net/npm/@perspective-dev/viewer@4.4.0/dist/cdn/perspective-viewer.js'),
import(/* @vite-ignore */ 'https://cdn.jsdelivr.net/npm/@perspective-dev/viewer-datagrid@4.4.0/dist/cdn/perspective-viewer-datagrid.js'),
import(/* @vite-ignore */ 'https://cdn.jsdelivr.net/npm/@perspective-dev/viewer-d3fc@4.4.0/dist/cdn/perspective-viewer-d3fc.js'),
])
return perspective
})()
return perspectivePromise
} }
function formatVal(v, decimals = 2) { function formatVal(v, decimals = 2) {
@ -66,8 +80,7 @@ const LAYOUT_KEY = (source) => `psp_layout_${source}`
const DEFAULT_PLUGIN_CONFIG = { edit_mode: 'SELECT_REGION' } const DEFAULT_PLUGIN_CONFIG = { edit_mode: 'SELECT_REGION' }
export default function Pivot({ source, selectedStack, setSelectedStack }) { export default function Pivot({ source }) {
const { dark } = useTheme()
const viewerRef = useRef() const viewerRef = useRef()
const workerRef = useRef() const workerRef = useRef()
const tableRef = useRef() const tableRef = useRef()
@ -84,12 +97,20 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
const [sortCol, setSortCol] = useState(null) const [sortCol, setSortCol] = useState(null)
const [sortDir, setSortDir] = useState('asc') const [sortDir, setSortDir] = useState('asc')
const selectedView = selectedStack ?? source // View selector: source or a stack
const viewType = selectedStack ? 'stack' : 'source' const [stacks, setStacks] = useState([])
const [selectedView, setSelectedView] = useState(source) // name of active dfv view
const [viewType, setViewType] = useState('source') // 'source' | 'stack'
useEffect(() => { api.getStacks().then(setStacks).catch(() => {}) }, [])
// When sidebar source changes, reset to that source
useEffect(() => { useEffect(() => {
if (viewerRef.current) viewerRef.current.setAttribute('theme', dark ? 'Pro Dark' : 'Pro Light') if (viewType === 'source') setSelectedView(source)
}, [dark]) }, [source])
function selectSource() { setViewType('source'); setSelectedView(source) }
function selectStack(name) { setViewType('stack'); setSelectedView(name) }
// Named layouts stacks use localStorage only (no server FK to sources) // Named layouts stacks use localStorage only (no server FK to sources)
const [layouts, setLayouts] = useState([]) const [layouts, setLayouts] = useState([])
@ -106,12 +127,16 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
const loadLayouts = useCallback(async () => { const loadLayouts = useCallback(async () => {
if (!selectedView) return if (!selectedView) return
try { try {
const rows = viewType === 'source' if (viewType === 'source') {
? await api.getPivotLayouts(selectedView) const rows = await api.getPivotLayouts(selectedView)
: await api.getStackPivotLayouts(selectedView)
setLayouts(rows) setLayouts(rows)
} else {
// Stacks: localStorage only
const stored = localStorage.getItem(`psp_layouts_stack_${selectedView}`)
setLayouts(stored ? JSON.parse(stored) : [])
}
} catch {} } catch {}
}, [selectedView]) }, [selectedView, viewType])
useEffect(() => { useEffect(() => {
if (!selectedView) return if (!selectedView) return
@ -239,7 +264,6 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
await plugin.restore(DEFAULT_PLUGIN_CONFIG) await plugin.restore(DEFAULT_PLUGIN_CONFIG)
} }
await viewer.flush() await viewer.flush()
viewer.setAttribute('theme', dark ? 'Pro Dark' : 'Pro Light')
setStatus('ready') setStatus('ready')
} catch (err) { } catch (err) {
@ -294,6 +318,11 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
localStorage.setItem(LAYOUT_KEY(selectedView), JSON.stringify(cleaned)) localStorage.setItem(LAYOUT_KEY(selectedView), JSON.stringify(cleaned))
} catch { } catch {
// Layout references columns that no longer exist remove it // Layout references columns that no longer exist remove it
if (viewType === 'stack') {
const updated = layouts.filter(l => l.id !== layout.id)
setLayouts(updated)
localStorage.setItem(`psp_layouts_stack_${selectedView}`, JSON.stringify(updated))
}
localStorage.removeItem(LAYOUT_KEY(selectedView)) localStorage.removeItem(LAYOUT_KEY(selectedView))
setActiveLayoutId(null) setActiveLayoutId(null)
await viewer.restore({ table: selectedView, settings: false }) await viewer.restore({ table: selectedView, settings: false })
@ -308,22 +337,19 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
return { ...viewerConfig, plugin_config: pluginConfig, expand_depth: expandDepthRef.current } return { ...viewerConfig, plugin_config: pluginConfig, expand_depth: expandDepthRef.current }
} }
const saveLayout = (name, config) => viewType === 'source'
? api.savePivotLayout(selectedView, name, config)
: api.saveStackPivotLayout(selectedView, name, config)
const deleteLayout = (id) => viewType === 'source'
? api.deletePivotLayout(selectedView, id)
: api.deleteStackPivotLayout(selectedView, id)
async function handleSaveOver() { async function handleSaveOver() {
const layout = layouts.find(l => l.id === activeLayoutId) const layout = layouts.find(l => l.id === activeLayoutId)
if (!layout) return if (!layout) return
const config = await captureConfig() const config = await captureConfig()
if (!config) return if (!config) return
try { try {
const saved = await saveLayout(layout.layout_name, config) if (viewType === 'source') {
const saved = await api.savePivotLayout(selectedView, layout.layout_name, config)
setActiveLayoutId(saved.id) setActiveLayoutId(saved.id)
} else {
const updated = layouts.map(l => l.id === activeLayoutId ? { ...l, config } : l)
localStorage.setItem(`psp_layouts_stack_${selectedView}`, JSON.stringify(updated))
}
localStorage.setItem(LAYOUT_KEY(selectedView), JSON.stringify(config)) localStorage.setItem(LAYOUT_KEY(selectedView), JSON.stringify(config))
await loadLayouts() await loadLayouts()
flashMsg('Saved!') flashMsg('Saved!')
@ -338,10 +364,18 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
const config = await captureConfig() const config = await captureConfig()
if (!config) return if (!config) return
try { try {
const saved = await saveLayout(name, config) let newId
if (viewType === 'source') {
const saved = await api.savePivotLayout(selectedView, name, config)
newId = saved.id
} else {
newId = Date.now()
const updated = [...layouts, { id: newId, layout_name: name, config }]
localStorage.setItem(`psp_layouts_stack_${selectedView}`, JSON.stringify(updated))
}
localStorage.setItem(LAYOUT_KEY(selectedView), JSON.stringify(config)) localStorage.setItem(LAYOUT_KEY(selectedView), JSON.stringify(config))
await loadLayouts() await loadLayouts()
setActiveLayoutId(saved.id) setActiveLayoutId(newId)
setShowSaveAs(false) setShowSaveAs(false)
setSaveAsName('') setSaveAsName('')
flashMsg('Saved!') flashMsg('Saved!')
@ -353,7 +387,12 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
async function handleDelete(layout, e) { async function handleDelete(layout, e) {
e.stopPropagation() e.stopPropagation()
try { try {
await deleteLayout(layout.id) if (viewType === 'source') {
await api.deletePivotLayout(selectedView, layout.id)
} else {
const updated = layouts.filter(l => l.id !== layout.id)
localStorage.setItem(`psp_layouts_stack_${selectedView}`, JSON.stringify(updated))
}
if (activeLayoutId === layout.id) setActiveLayoutId(null) if (activeLayoutId === layout.id) setActiveLayoutId(null)
await loadLayouts() await loadLayouts()
flashMsg('Deleted') flashMsg('Deleted')
@ -415,12 +454,29 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
return ( return (
<div className="w-full h-full flex flex-col"> <div className="w-full h-full flex flex-col">
{/* Layouts sub-bar */} {/* Layout toolbar */}
<div className="flex items-center gap-2 px-3 h-9 bg-white border-b border-gray-200 shrink-0 text-xs"> <div className="flex items-center gap-2 px-3 py-1.5 bg-white border-b border-gray-200 flex-shrink-0">
{/* View selector */}
<div className="flex items-center gap-1 mr-2 border-r border-gray-200 pr-3">
<button
onClick={selectSource}
className={`text-xs rounded px-2 py-0.5 border transition-colors ${viewType === 'source' ? 'bg-blue-50 border-blue-300 text-blue-700' : 'bg-white border-gray-200 text-gray-500 hover:border-gray-400'}`}
>{source}</button>
{stacks.map(s => (
<button key={s.name}
onClick={() => selectStack(s.name)}
className={`text-xs rounded px-2 py-0.5 border transition-colors ${viewType === 'stack' && selectedView === s.name ? 'bg-purple-50 border-purple-300 text-purple-700' : 'bg-white border-gray-200 text-gray-500 hover:border-gray-400'}`}
>{s.name}</button>
))}
</div>
<span className="text-xs text-gray-400 uppercase tracking-wide mr-1">Layouts</span>
{layouts.map(l => ( {layouts.map(l => (
<div key={l.id} <div key={l.id}
onClick={() => applyLayout(l)} onClick={() => applyLayout(l)}
className={`flex items-center gap-1 rounded px-2 py-0.5 cursor-pointer border transition-colors className={`flex items-center gap-1 text-xs rounded px-2 py-0.5 cursor-pointer border transition-colors
${activeLayoutId === l.id ${activeLayoutId === l.id
? 'bg-blue-50 border-blue-300 text-blue-700' ? 'bg-blue-50 border-blue-300 text-blue-700'
: 'bg-white border-gray-200 text-gray-600 hover:border-gray-400'}`}> : 'bg-white border-gray-200 text-gray-600 hover:border-gray-400'}`}>
@ -433,7 +489,7 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
{activeLayoutId !== null && !showSaveAs && ( {activeLayoutId !== null && !showSaveAs && (
<button onClick={handleSaveOver} <button onClick={handleSaveOver}
className="text-blue-500 hover:text-blue-700 border border-blue-200 rounded px-2 py-0.5"> className="text-xs text-blue-500 hover:text-blue-700 border border-blue-200 rounded px-2 py-0.5">
Save Save
</button> </button>
)} )}
@ -446,27 +502,30 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
onChange={e => setSaveAsName(e.target.value)} onChange={e => setSaveAsName(e.target.value)}
onKeyDown={e => { if (e.key === 'Enter') handleSaveAs(); if (e.key === 'Escape') { setShowSaveAs(false); setSaveAsName('') } }} onKeyDown={e => { if (e.key === 'Enter') handleSaveAs(); if (e.key === 'Escape') { setShowSaveAs(false); setSaveAsName('') } }}
placeholder="Layout name…" placeholder="Layout name…"
className="border border-gray-300 rounded px-2 py-0.5 w-36 focus:outline-none focus:border-blue-400" className="text-xs border border-gray-300 rounded px-2 py-0.5 w-36 focus:outline-none focus:border-blue-400"
/> />
<button onClick={handleSaveAs} className="text-blue-600 hover:text-blue-800 px-1">Save</button> <button onClick={handleSaveAs} className="text-xs text-blue-600 hover:text-blue-800 px-1">Save</button>
<button onClick={() => { setShowSaveAs(false); setSaveAsName('') }} className="text-gray-400 hover:text-gray-600 px-1">Cancel</button> <button onClick={() => { setShowSaveAs(false); setSaveAsName('') }} className="text-xs text-gray-400 hover:text-gray-600 px-1">Cancel</button>
</div> </div>
) : ( ) : (
<button <button
onClick={() => setShowSaveAs(true)} onClick={() => setShowSaveAs(true)}
className="text-gray-400 hover:text-gray-600 border border-dashed border-gray-200 rounded px-2 py-0.5"> className="text-xs text-gray-400 hover:text-gray-600 border border-dashed border-gray-200 rounded px-2 py-0.5">
+ Save as + Save as
</button> </button>
)} )}
{activeLayoutId !== null && ( {activeLayoutId !== null && (
<button onClick={handleResetToDefault} className="text-gray-300 hover:text-gray-500 ml-1">reset</button> <button onClick={handleResetToDefault}
className="text-xs text-gray-300 hover:text-gray-500 ml-1">
reset
</button>
)} )}
{layoutMsg && <span className="text-green-600 ml-1">{layoutMsg}</span>} {layoutMsg && <span className="text-xs text-green-600 ml-1">{layoutMsg}</span>}
<div className="ml-auto flex items-center gap-1"> <div className="ml-auto flex items-center gap-1">
<span className="text-gray-400">depth:</span> <span className="text-xs text-gray-400">depth:</span>
{[0, 1, 2, 3].map(d => ( {[0, 1, 2, 3].map(d => (
<button key={d} onClick={async () => { <button key={d} onClick={async () => {
const v = viewerRef.current; if (!v) return const v = viewerRef.current; if (!v) return
@ -475,7 +534,7 @@ export default function Pivot({ source, selectedStack, setSelectedStack }) {
const p = await v.getPlugin() const p = await v.getPlugin()
await p.draw(view) await p.draw(view)
expandDepthRef.current = d expandDepthRef.current = d
}} className="border border-gray-200 rounded px-1.5 py-0.5 text-gray-500 hover:border-gray-400"> }} className="text-xs border border-gray-200 rounded px-1.5 py-0.5 text-gray-500 hover:border-gray-400">
{d} {d}
</button> </button>
))} ))}

View File

@ -271,7 +271,7 @@ export default function Records({ source }) {
const displayCols = (rows.length > 0 ? Object.keys(rows[0]) : cols).filter(c => !HIDDEN_COLS.has(c)) const displayCols = (rows.length > 0 ? Object.keys(rows[0]) : cols).filter(c => !HIDDEN_COLS.has(c))
const visCols = cols.filter(c => !HIDDEN_COLS.has(c)) const visCols = cols.filter(c => !HIDDEN_COLS.has(c))
// For bulk bar: only established override keys // All override cols: known from DB + new ones added this session
const allOverrideCols = [...new Set([...overrideCols, ...extraCols])] const allOverrideCols = [...new Set([...overrideCols, ...extraCols])]
const savedOverrides = selectedRecord?.overrides || {} const savedOverrides = selectedRecord?.overrides || {}
@ -493,94 +493,43 @@ export default function Records({ source }) {
</div> </div>
)} )}
{/* Raw fields — read only */} {/* Read-only transformed fields */}
<div className="border-b border-gray-100"> <div className="border-b border-gray-100">
<div className="px-3 py-1.5 bg-gray-50 border-b border-gray-100"> {Object.entries(selectedRecord.transformed || {}).map(([field, val]) => (
<span className="text-xs font-medium text-gray-400 uppercase tracking-wide">Raw</span>
</div>
{Object.entries(selectedRecord.data || {}).map(([field, val]) => (
<div key={field} className="flex items-baseline gap-2 px-3 py-1 border-t border-gray-50 first:border-t-0"> <div key={field} className="flex items-baseline gap-2 px-3 py-1 border-t border-gray-50 first:border-t-0">
<span className="text-xs font-mono text-gray-400 w-28 shrink-0 truncate">{field}</span> <span className="text-xs font-mono text-gray-400 w-28 shrink-0 truncate">{field}</span>
<span className="text-xs font-mono text-gray-500 truncate">{formatVal(val) ?? <span className="text-gray-300"></span>}</span> <span className="text-xs font-mono text-gray-600 truncate">{formatVal(val) ?? <span className="text-gray-300"></span>}</span>
</div> </div>
))} ))}
</div> </div>
{/* Transformed fields — read only delta */} {/* Override cols — Mappings-style */}
<div className="border-b border-gray-100"> <div className="flex-1">
<div className="px-3 py-1.5 bg-gray-50 border-b border-gray-100">
<span className="text-xs font-medium text-gray-400 uppercase tracking-wide">Transformed</span>
</div>
{Object.entries(selectedRecord.transformed || {}).filter(([k]) => !HIDDEN_COLS.has(k)).length === 0
? <div className="px-3 py-2 text-xs text-gray-300">No rule output yet.</div>
: Object.entries(selectedRecord.transformed || {}).filter(([k]) => !HIDDEN_COLS.has(k)).map(([field, val]) => (
<div key={field} className="flex items-baseline gap-2 px-3 py-1 border-t border-gray-50 first:border-t-0">
<span className="text-xs font-mono text-gray-400 w-28 shrink-0 truncate">{field}</span>
<span className="text-xs font-mono text-blue-600 truncate">{formatVal(val) ?? <span className="text-gray-300"></span>}</span>
</div>
))
}
</div>
{/* Overrides — editable */}
<div className="flex-1 border-b border-gray-100">
<div className="flex items-center justify-between px-3 py-1.5 bg-gray-50 border-b border-gray-100"> <div className="flex items-center justify-between px-3 py-1.5 bg-gray-50 border-b border-gray-100">
<span className="text-xs font-medium text-gray-400 uppercase tracking-wide">Overrides</span> <span className="text-xs font-medium text-gray-500 uppercase tracking-wide">Override</span>
<button <button
onClick={() => setExtraCols(ec => [...ec, ''])} onClick={() => setExtraCols(ec => [...ec, ''])}
className="text-gray-400 hover:text-gray-700 font-medium text-sm leading-none" className="text-gray-400 hover:text-gray-700 font-medium text-sm leading-none"
title="Add field">+</button> title="Add column">+</button>
</div> </div>
<table className="w-full text-xs"> <table className="w-full text-xs">
<tbody> <tbody>
{[...new Set([ {allOverrideCols.map((col, idx) => {
...Object.keys(selectedRecord.transformed || {}), const isExtra = idx >= overrideCols.length
...Object.keys(selectedRecord.overrides || {}),
...overrideCols
])].filter(k => !HIDDEN_COLS.has(k)).map(col => {
const override = overrideDraft[col] ?? ''
const placeholder = formatVal(selectedRecord.transformed?.[col]) ?? ''
const suggestions = [...(globalValues[col] || [])].sort() const suggestions = [...(globalValues[col] || [])].sort()
return (
<tr key={col} className="border-t border-gray-50">
<td className="px-3 py-1.5 w-28 shrink-0">
<span className="font-mono text-gray-500 truncate block">{col}</span>
</td>
<td className="px-1 py-1.5">
<AutocompleteInput
className={`w-full text-xs font-mono px-2 py-0.5 rounded border focus:outline-none ${
override ? 'border-amber-300 bg-amber-50 text-amber-800' : 'border-gray-200 text-gray-600'
}`}
value={override}
placeholder={placeholder}
onChange={v => setOverrideDraft(d => ({ ...d, [col]: v }))}
onEnter={handleSaveOverrides}
suggestions={suggestions}
/>
</td>
<td className="pr-2 text-center w-6">
{override && (
<button
onClick={() => setOverrideDraft(d => { const n = { ...d }; delete n[col]; return n })}
className="text-gray-300 hover:text-red-400 leading-none text-base">×</button>
)}
</td>
</tr>
)
})}
{extraCols.map((col, i) => {
const val = overrideDraft[col] ?? '' const val = overrideDraft[col] ?? ''
const suggestions = [...(globalValues[col] || [])].sort()
return ( return (
<tr key={`extra-${i}`} className="border-t border-gray-50"> <tr key={col || `extra-${idx}`} className="border-t border-gray-50">
<td className="px-3 py-1.5 w-28 shrink-0"> <td className="px-3 py-1 w-28 shrink-0">
{isExtra ? (
<input <input
className="w-full text-xs font-mono border border-gray-200 rounded px-1 py-0.5 focus:outline-none focus:border-blue-400" className="w-full text-xs font-mono border border-gray-200 rounded px-1 py-0.5 focus:outline-none focus:border-blue-400"
value={col} value={col}
placeholder="field name" placeholder="field name"
onChange={e => { onChange={e => {
const newName = e.target.value const newName = e.target.value
setExtraCols(ec => { const c = [...ec]; c[i] = newName; return c }) setExtraCols(ec => { const c = [...ec]; c[idx - overrideCols.length] = newName; return c })
if (val) setOverrideDraft(d => { if (val) setOverrideDraft(d => {
const n = { ...d } const n = { ...d }
delete n[col] delete n[col]
@ -589,11 +538,14 @@ export default function Records({ source }) {
}) })
}} }}
/> />
) : (
<span className="font-mono text-gray-500 truncate block">{col}</span>
)}
</td> </td>
<td className="px-1 py-1.5"> <td className="px-1 py-1">
<AutocompleteInput <AutocompleteInput
className={`w-full text-xs font-mono px-2 py-0.5 rounded border focus:outline-none ${ className={`w-full text-xs font-mono px-2 py-0.5 rounded border focus:outline-none ${
val ? 'border-amber-300 bg-amber-50 text-amber-800' : 'border-gray-200 text-gray-600' val ? 'border-amber-300 bg-amber-50 text-amber-800' : 'border-gray-200 text-gray-700'
}`} }`}
value={val} value={val}
onChange={v => setOverrideDraft(d => ({ ...d, [col]: v }))} onChange={v => setOverrideDraft(d => ({ ...d, [col]: v }))}
@ -601,7 +553,7 @@ export default function Records({ source }) {
suggestions={suggestions} suggestions={suggestions}
/> />
</td> </td>
<td className="pr-2 text-center w-6"> <td className="pr-2 text-center">
{val && ( {val && (
<button <button
onClick={() => setOverrideDraft(d => { const n = { ...d }; delete n[col]; return n })} onClick={() => setOverrideDraft(d => { const n = { ...d }; delete n[col]; return n })}
@ -615,7 +567,7 @@ export default function Records({ source }) {
</table> </table>
</div> </div>
<div className="flex gap-2 px-3 py-2 border-t border-gray-100 shrink-0"> <div className="flex gap-2 px-3 py-2 border-t border-gray-100">
<button <button
onClick={handleSaveOverrides} onClick={handleSaveOverrides}
disabled={panelSaving || !isDirty} disabled={panelSaving || !isDirty}

View File

@ -675,7 +675,7 @@ function StackPanel({ stack, sources, onUpdated, onStale, onViewGenerated, onSql
// Main page // Main page
export default function Stacks({ sources, onStackStale, onStackViewGenerated, onStacksChange }) { export default function Stacks({ sources, onStackStale, onStackViewGenerated }) {
const [stacks, setStacks] = useState([]) const [stacks, setStacks] = useState([])
const [selected, setSelected] = useState(null) const [selected, setSelected] = useState(null)
const [stackDetail, setStackDetail] = useState(null) const [stackDetail, setStackDetail] = useState(null)
@ -716,7 +716,6 @@ export default function Stacks({ sources, onStackStale, onStackViewGenerated, on
await api.createStack({ name: newName, fields: [] }) await api.createStack({ name: newName, fields: [] })
setNewName(''); setCreating(false) setNewName(''); setCreating(false)
await load() await load()
onStacksChange?.()
loadDetail(newName) loadDetail(newName)
} catch (e) { setError(e.message) } } catch (e) { setError(e.message) }
} }
@ -726,7 +725,6 @@ export default function Stacks({ sources, onStackStale, onStackViewGenerated, on
await api.deleteStack(name) await api.deleteStack(name)
if (selected === name) { setSelected(null); setStackDetail(null); setSqlDraft(''); setSqlResult(null) } if (selected === name) { setSelected(null); setStackDetail(null); setSqlDraft(''); setSqlResult(null) }
load() load()
onStacksChange?.()
} }
async function runSql() { async function runSql() {

View File

@ -1,25 +0,0 @@
import { createContext, useContext, useState, useEffect } from 'react'
const ThemeContext = createContext()
export function ThemeProvider({ children }) {
const [dark, setDark] = useState(() => {
const saved = localStorage.getItem('df_dark')
if (saved !== null) return saved === 'true'
return window.matchMedia('(prefers-color-scheme: dark)').matches
})
useEffect(() => {
localStorage.setItem('df_dark', dark)
document.documentElement.classList.toggle('dark', dark)
}, [dark])
return (
<ThemeContext.Provider value={{ dark, setDark }}>
{children}
</ThemeContext.Provider>
)
}
const useTheme = () => useContext(ThemeContext)
export default useTheme