Commit Graph

5 Commits

Author SHA1 Message Date
ebd88a2df8 Source setup UX, Pivot page, and import/view fixes
- Fix stale import_records in sources.sql that referenced deleted generate_constraint_key
- Auto-transform after import, auto-generate view after create
- New source form matches existing source layout (In view, Seq, type dropdown)
- Sample data table (50 rows) shown below field config in both new and existing source views
- Import sample CSV on create (checked by default)
- Sortable column headers on field table
- Choose CSV styled as a button showing filename
- + button in sidebar opens new source form
- Records tab shows error message when view cast fails instead of blank
- Pivot page with Perspective viewer, per-source saved layouts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 21:31:44 -04:00
d495ef2fc5 Records filters, global picklist, autocomplete, and rule reprocess
- Records tab: regex filter bar (postgres ~*), add/remove filters, debounced,
  ANDed together; get_view_data gains p_filters JSONB param
- Global picklist: sources.global_picklist flag (default true) controls whether
  a source's mapped output values feed the cross-source autocomplete suggestion pool;
  toggle on Sources page; get_global_output_values() SQL function
- Mappings: replace native datalist with custom AutocompleteInput component —
  Alt+Down opens, Tab cycles, Enter selects, arrow keys navigate, Escape closes
- Rules: auto-reprocess source records when a rule is created or updated
- preview_rule: fix BIGINT/INT return type mismatch
- Stale get_import_log removed from sources.sql
- TSV export: fetch with auth headers instead of plain <a href> (fixes 401)
- + column button: more visible styling

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-14 16:28:26 -04:00
d63d70cd52 Import log, constraint key overhaul, and dedup improvements
- Rename dedup_key/dedup_fields → constraint_key/constraint_fields everywhere
  (schema, functions, routes, UI, migration script, docs)
- Change constraint_key from MD5 TEXT hash to readable JSONB object
- Drop unique constraint on (source_name, constraint_key); dedup is now
  enforced at import time via CTE, allowing intra-file duplicate rows
- Add import_id FK (ON DELETE CASCADE) so deleting a log entry removes its records
- Add info JSONB to import_log with inserted_keys and excluded_keys arrays
- Add get_import_log, get_all_import_logs, delete_import SQL functions
- Auto-apply transformations immediately after import
- Import UI: expandable key detail, checkbox selection, delete with confirm,
  import ID column, transform result display
- New Log page: global import log across all sources

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-13 23:44:30 -04:00
928a54932d Add multi-capture regex, computed view fields, collapsible rules, and live preview
- Support multi-capture-group regex: mappings.input_value changed to JSONB,
  regexp_match() result stored as scalar or array JSONB in transformed column
- Computed expression fields in generated views: {fieldname} refs substituted
  with (transformed->>'fieldname')::numeric for arithmetic in view columns
- Fix generate_source_view to DROP VIEW before CREATE (avoids column drop error)
- Collapsible rule cards that open directly to inline edit form
- Debounced live regex preview (extract + replace) with popout modal for 50 rows
- Records page now shows dfv.<source> view output instead of raw records
- Unified field table in Sources: single table with In view, Seq, expression columns
- Fix "Rule already exists" error when editing by passing rule.id directly to submit
- Fix Sources page clearing on F5 by watching sourceObj?.name in useEffect dep

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 16:37:15 -04:00
eb50704ca0 Add React UI and backend enhancements for dataflow
- Add full React + Vite UI (src/pages: Sources, Rules, Mappings, Records, Import)
- Sidebar layout with source selector persisted to localStorage
- Sources: unified field table with Dedup/In-view checkboxes, CSV suggest, generate dfv view
- Rules: extract/replace function types, regex flags, input field picklist, test results
- Mappings: unmapped values with sample records, inline key/value editor, edit existing mappings
- Records: expanded row shows per-rule extraction and mapping output breakdown
- Import: drag-drop CSV, transform/reprocess buttons, import history
- Backend: add flags/function_type to rules, get_unmapped_values with samples, generate_source_view, fields endpoint, reprocess endpoint
- database/functions.sql: apply_transformations supports replace mode and flags; generate_source_view builds typed dfv views
- Server bound to 0.0.0.0, SPA fallback for client-side routing

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-29 00:35:33 -04:00