- Two-column layout: config left, SQL panel right (equal halves)
- SQL panel shows formatted SQL (sql-formatter, 4-space indent)
- Live preview: SQL updates 400ms after any field/source/mapping change
- Run button executes edited SQL directly via new exec-sql endpoint
- generate_stack_view gains p_dry_run mode for preview without executing
- CASCADE drop detects dependent stacks, marks them stale in DB and status bar
- net_balance moved to last column in generated view
- Backfill 458 missing dcard rows and 123 missing chase rows from TPS migration bug
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add get_status() SQL and /api/status route; load stale state on login
- Replace polling with immediate client-side stale tracking via callbacks
- Amber status bar with per-item Generate buttons for sources and stacks
- Pivot: add stack selector to view any dfv.stack view via Perspective
- Stack views: DROP CASCADE, add id to source views, per-source balance columns
- net_balance = sum(all amounts) + total_offset guarantees chase+dcard=net per row
- CLAUDE.md: document correct dedup spec (within-batch duplicates always allowed)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Open panel immediately on row click (panelOpen state), then load full
record async. Previously the panel condition depended on selectedRecord
or panelLoading both of which are set after async work, so if id was
missing or the API call failed the panel never appeared.
Also shows a message if id is missing (view needs regeneration).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Schema:
- records.overrides JSONB column (ALTER TABLE, already applied)
- apply_transformations merges overrides on top: data || rules || overrides
- generate_source_view always includes id and _overridden columns
- set_record_overrides(id, overrides): stores and immediately merges into transformed
- clear_record_overrides(id): clears overrides then reprocesses record
API:
- PUT /records/:id/overrides — set overrides
- DELETE /records/:id/overrides — clear and reprocess
UI (Records page):
- Rows are clickable; overridden rows highlighted amber
- Side panel shows all transformed fields as editable inputs
- Overridden fields highlighted amber with pencil indicator
- Save stores overrides; Clear removes them and restores computed values
- id and _overridden hidden from table display
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- SQL: search_mapping_outputs(search) — distinct (col, val, count) groups
get_mappings_by_output_field(col, val) — individual mappings
remap_output_field(col, from, to) — bulk UPDATE via jsonb_set
- API: GET /mappings/outputs?search=, GET /mappings/outputs/:col/:val,
POST /mappings/remap-field
- UI: Remap page — search output values, click to select, edit the
replacement value, see all affected mappings, apply globally
- Nav: Remap added between Mappings and Records
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SPEC.md: rewrite Pivot page description to cover named layouts, depth
control, selection mode, inspector filtering, and layout persistence.
docs/perspective-pivot.md: new file documenting all discovered Perspective
v4.4.0 APIs — viewer/plugin/view methods, selection modes, set_depth
mechanism, perspective-click event shape, full state save/restore pattern,
and common pitfalls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
formatVal now rounds numeric values using toLocaleString with
configurable decimal places (default 2, range 0-8). Adds -/+ controls
in the inspector header to adjust on the fly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Without group_by there are no coordinate filters, so the view query
would return the full dataset and hang. Early-return on click if
config.group_by is empty.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Default selection mode is now SELECT_REGION
- plugin.save()/restore() used to capture and apply edit mode
- expand_depth tracked in ref and included in layout config
- applyExpandDepth helper restores depth on layout recall and page load
- Save button overwrites active layout in place (no re-typing name)
- captureConfig() helper shared by save-over and save-as flows
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace debug test buttons with a minimal 'depth: 0 1 2 3' control
in the pivot toolbar right side.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After testing plugin_config.expand_depth (no effect) and view.set_depth
+ flush() (no effect), confirmed that view.set_depth(d) followed by
plugin.draw(view) correctly collapses/expands all rows to depth d.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Temporary UI for testing programmatic row expansion control via
plugin_config.expand_depth in Perspective viewer.restore().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- pivot_layouts table (source_name, layout_name, config JSONB)
- list/save/delete SQL functions and API routes
- Pivot toolbar above viewer: layout chips, save-as inline input,
delete per layout, reset to default
- Applying a named layout also updates localStorage working state
- Layouts reload on source change
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Always display all non-null metric columns from the clicked row.
When a specific cell can be identified (split_by in use, cell mode),
highlight that row in blue/bold. Fixes row mode showing only one value.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace __ROW_PATH__ zip approach with direct application of
perspective-click event filters against raw rows. Fields not
present in the raw data (Perspective computed columns like Month,
YearDate) are skipped. Also removes debug console.log calls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Show row path prominently, filter to non-null metric values,
use group_by › split_by as section header.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix stale import_records in sources.sql that referenced deleted generate_constraint_key
- Auto-transform after import, auto-generate view after create
- New source form matches existing source layout (In view, Seq, type dropdown)
- Sample data table (50 rows) shown below field config in both new and existing source views
- Import sample CSV on create (checked by default)
- Sortable column headers on field table
- Choose CSV styled as a button showing filename
- + button in sidebar opens new source form
- Records tab shows error message when view cast fails instead of blank
- Pivot page with Perspective viewer, per-source saved layouts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Rename dedup_key/dedup_fields → constraint_key/constraint_fields everywhere
(schema, functions, routes, UI, migration script, docs)
- Change constraint_key from MD5 TEXT hash to readable JSONB object
- Drop unique constraint on (source_name, constraint_key); dedup is now
enforced at import time via CTE, allowing intra-file duplicate rows
- Add import_id FK (ON DELETE CASCADE) so deleting a log entry removes its records
- Add info JSONB to import_log with inserted_keys and excluded_keys arrays
- Add get_import_log, get_all_import_logs, delete_import SQL functions
- Auto-apply transformations immediately after import
- Import UI: expandable key detail, checkbox selection, delete with confirm,
import ID column, transform result display
- New Log page: global import log across all sources
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add import_id column to records (links each record to its import batch)
- import_records() now stores readable dedup field values (not hashes) in
info.inserted_keys / info.excluded_keys, and stamps import_id on insert
- delete_import() simplified to delete log row; ON DELETE CASCADE removes records
- Add get_import_log() and get_all_import_logs() DB functions
- Add DELETE /api/sources/:name/import-log/:id endpoint
- Add GET /api/sources/import-log global log endpoint
- Import route now auto-applies transformations to new records after import
- Import page: show ID column, expandable key detail, checkbox delete
- New Log page: global view of all imports across sources
- Update README API reference and workflow
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Documents manage.py menu, adds full API reference tables, fixes
incorrect route in quick example, and removes stale sections
(docs/ dir, initial development status).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add database/queries/{sources,rules,mappings,records}.sql — one file per
route, all business logic in PostgreSQL functions
- Replace parameterized queries in all four route files with lit()/jsonLit()
literal interpolation for debuggability
- Add api/lib/sql.js with lit(), jsonLit(), arr() helpers
- Fix get_view_data to use json_agg (preserves column order) with subquery
(guarantees sort order is respected before aggregation)
- Fix jsonLit() for JSONB params so plain strings become valid JSON
- Update manage.py option 3 to deploy database/queries/ instead of functions.sql
- Add SPEC.md covering architecture, philosophy, and manage.py
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After createMapping, the new mapping's id was not stored in allValues
state, so editing the row again fell into the create path instead of
update. Now stores created.id so subsequent saves correctly use updateMapping.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Credentials are saved to sessionStorage on login and restored on mount,
so a page refresh re-authenticates silently. Closing the tab clears them.
Logout explicitly removes them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Clicking a column header reloads from page 1 with ORDER BY col ASC/DESC
NULLS LAST passed to the view query. Sort column is validated against
information_schema.columns to prevent injection. Pagination preserves
the active sort across prev/next.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Click any column header to sort asc/desc (⇅ / ▲ / ▼ indicators)
- Sort is client-side within the current page, numeric-aware
- Dates matching ISO format are displayed as e.g. "Apr 5, 26"
- Sort resets on source change
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Username and 'Sign out' link now sit on their own row under the
Dataflow title, so they always have room to display regardless of
title width.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
On mobile: hamburger button in top bar opens sidebar as a slide-over
with a backdrop overlay. Nav links and source selector close it on tap.
On desktop: sidebar is static as before.
Logout button now sits alongside the close button in the sidebar header
with a gap so both are always visible.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Express auth middleware checks Authorization: Basic header on all /api
routes using bcrypt against LOGIN_USER/LOGIN_PASSWORD_HASH in .env
- React login screen shown before app loads, stores credentials in memory,
sends them with every API request, clears and returns to login on 401
- Logout button in sidebar header
- manage.py option 9: set login credentials (bcrypt via node, writes to .env)
- manage.py status shows whether login credentials are configured
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Show exact commands that will be run before each confirm prompt
- Step 1 dialog now offers schema and function deployment after writing .env
- Steps 2/3 relabeled as 'Redeploy only' for standalone use
- Option 5 (nginx) detects existing config and warns before overwriting
- Option 1 menu label clarified as 'Database configuration and deployment dialog'
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
sudo cp creates the file as root:root 0600, making it unreadable by the
app user. Add sudo chmod 644 after writing so status detection can read
it without sudo, matching how other nginx configs are set up.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
nginx config files written by sudo are root-only (rw-------), so
nginx_domain() was silently failing to read them. Now uses 'sudo -n cat'
with fallback to direct read for world-readable files.
Also fix PermissionError on cert_path.exists() — /etc/letsencrypt/live/
requires root, so use 'sudo test -f' instead of Path.exists().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
capture_output=True on sudo_run suppresses the sudo password prompt,
causing silent auth failure and permission denied on subsequent calls.
Removed capture_output from nginx -t and systemctl enable so the
password prompt and any error output appear on screen.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Every status line, action header, confirm prompt, and ok/err message now
names exactly what it refers to — schema name, database, host, file paths,
and systemd commands. Menu items include source/target context. No ambiguous
shorthand anywhere.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Menu item: 'Configure database' → 'Configure database connection'
Intro text: disambiguates that the database is what gets created, not a connection.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
No external dependencies — uses psql CLI via subprocess. Interactive menu
detects current state (DB connection, schema, UI build, service status) and
guides through configure, deploy, rebuild, nginx setup, and service management.
Handles both new installs and existing databases (grants access vs. creating).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
