get_source_fields only unioned schema fields, raw data keys, rule
output_field and mapping output keys. Keys that live solely in
records.transformed or records.overrides — a manual override such as
dcard's "Note", or a transformed key whose rule was since deleted — never
appeared on the source page, so there was no way to add them to the view.
Read both columns off the records directly.
Records showed no id column, leaving no handle to identify a row. Split
the hidden-column set: HIDDEN_COLS still keeps id out of the override
editor, GRID_HIDDEN_COLS hides only _overridden, and gridCols() pins id
first in the grid and the filter dropdown.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BDFU3ueEbCYGE37oDgeu2j
The page loaded unsorted, so the most recent rows were rarely on screen. On
source change it now looks up the source's first date-typed field and sorts
descending on it, falling back to unsorted when a source has no date field.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- transformed now stores only rule additions (not merged data+overrides)
- View dynamically computes data || transformed || overrides at query time
- New DB functions: set/clear/bulk_set_record_overrides
- Records panel now includes source-wide override keys so party/reason etc.
appear even on records that don't have them set yet
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Stack selection lifted to App.jsx: stacks fetched on login, selectedStack
state shared via StatusBar (pills) and Pivot (view switching); Stacks page
calls onStacksChange to keep list fresh
- Pivot: derive selectedView/viewType from props, remove local stack state;
toolbar replaced with dedicated layouts sub-bar (h-9, layouts only)
- Records panel: merge read-only and override sections into single field list;
known cols seeded from record's transformed fields; rule-derived fields
(transformed minus data) will be editable in follow-up refactor
- Pivot theme: setAttribute moved to after flush() so restore() can't reset it
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- bulk_set_record_overrides() DB function merges overrides into multiple
records at once using a CTE with RETURNING for accurate count
- POST /records/bulk-overrides calls the function (consistent with rest
of API — no raw SQL in routes)
- UI: regex input on loaded rows selects rows for bulk override; labeled
"Bulk select:" / "DB query:" to distinguish from server-side filters
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Open panel immediately on row click (panelOpen state), then load full
record async. Previously the panel condition depended on selectedRecord
or panelLoading both of which are set after async work, so if id was
missing or the API call failed the panel never appeared.
Also shows a message if id is missing (view needs regeneration).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Schema:
- records.overrides JSONB column (ALTER TABLE, already applied)
- apply_transformations merges overrides on top: data || rules || overrides
- generate_source_view always includes id and _overridden columns
- set_record_overrides(id, overrides): stores and immediately merges into transformed
- clear_record_overrides(id): clears overrides then reprocesses record
API:
- PUT /records/:id/overrides — set overrides
- DELETE /records/:id/overrides — clear and reprocess
UI (Records page):
- Rows are clickable; overridden rows highlighted amber
- Side panel shows all transformed fields as editable inputs
- Overridden fields highlighted amber with pencil indicator
- Save stores overrides; Clear removes them and restores computed values
- id and _overridden hidden from table display
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix stale import_records in sources.sql that referenced deleted generate_constraint_key
- Auto-transform after import, auto-generate view after create
- New source form matches existing source layout (In view, Seq, type dropdown)
- Sample data table (50 rows) shown below field config in both new and existing source views
- Import sample CSV on create (checked by default)
- Sortable column headers on field table
- Choose CSV styled as a button showing filename
- + button in sidebar opens new source form
- Records tab shows error message when view cast fails instead of blank
- Pivot page with Perspective viewer, per-source saved layouts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Clicking a column header reloads from page 1 with ORDER BY col ASC/DESC
NULLS LAST passed to the view query. Sort column is validated against
information_schema.columns to prevent injection. Pagination preserves
the active sort across prev/next.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Click any column header to sort asc/desc (⇅ / ▲ / ▼ indicators)
- Sort is client-side within the current page, numeric-aware
- Dates matching ISO format are displayed as e.g. "Apr 5, 26"
- Sort resets on source change
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Support multi-capture-group regex: mappings.input_value changed to JSONB,
regexp_match() result stored as scalar or array JSONB in transformed column
- Computed expression fields in generated views: {fieldname} refs substituted
with (transformed->>'fieldname')::numeric for arithmetic in view columns
- Fix generate_source_view to DROP VIEW before CREATE (avoids column drop error)
- Collapsible rule cards that open directly to inline edit form
- Debounced live regex preview (extract + replace) with popout modal for 50 rows
- Records page now shows dfv.<source> view output instead of raw records
- Unified field table in Sources: single table with In view, Seq, expression columns
- Fix "Rule already exists" error when editing by passing rule.id directly to submit
- Fix Sources page clearing on F5 by watching sourceObj?.name in useEffect dep
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
