Replace idempotent schema script with tracked forward-only migrations

setup_sql/01_schema.sql was an idempotent bootstrap script — CREATE TABLE
IF NOT EXISTS plus a tail of ALTER ... ADD COLUMN IF NOT EXISTS. It had no
record of what any given database had applied, which is how a branch could
declare col_meta.in_grain while the running database lacked it, with
nothing able to detect the mismatch. The symptom would have been a
confusing 'column "in_grain" does not exist' inside an unrelated request.

Migrations-only, no hand-maintained current-state file to drift:

- setup_sql/migrations/*.sql applied in filename order, recorded in
  pf.schema_version with a checksum. Split along the schema's actual
  evolution, so each column is declared exactly once — 01_schema.sql had
  grown to declare dim_group, dim_period_col and in_grain twice each.
- lib/migrations.js holds the bookkeeping, shared by the CLI and the boot
  check. scripts/migrate.js provides up | status | baseline.
- server.js refuses to start when the database is behind, listing what is
  pending. This converts silent drift into a clear boot message, which was
  the whole point. PF_SKIP_MIGRATION_CHECK=1 bypasses.
- Four integrity guards, each verified to fire: a migration modified after
  being applied, one recorded as applied but missing from disk, one that
  would apply out of order, and a re-run when already current.
- No IF NOT EXISTS on new migrations. The bookkeeping already guarantees
  one run each, and the guards hide ordering mistakes — that is exactly why
  01_schema.sql had ALTERs sitting above the CREATE TABLE they depended on,
  broken for anyone installing from scratch. 0004 keeps the guard only
  because it was applied by hand before migrations existed.

Verified: replaying all four migrations into a throwaway schema reproduces
the live pf schema exactly, 41 columns, column for column.

schema.generated.sql is a pg_dump snapshot for reading, refreshed by
npm run schema:dump. It excludes the runtime fc_* tables and strips
pg_dump's random \restrict token and version banner, so regenerating an
unchanged schema yields an identical file rather than a spurious diff.

pf.dim_period stays out of migrations — it is a parameterised data load
(fiscal year start month), not a schema change.

The dev database (ubm) has been baselined at all four migrations.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

.env.example

@@ -4,18 +4,3 @@ DB_NAME=your_database
 DB_USER=your_user
 DB_PASSWORD=your_password
 PORT=3010
-
-# Signs the session cookie. Generate with:
-#   node -e 'console.log(require("crypto").randomBytes(32).toString("hex"))'
-# or let ./pf.sh config do it. Changing it signs everyone out.
-SESSION_SECRET=
-
-# Send the session cookie over HTTPS only. Keep true behind a TLS proxy;
-# set false only to reach the app over plain HTTP on a trusted network.
-COOKIE_SECURE=true
-
-# Reverse proxy hops express should trust for req.ip / protocol. Default 1.
-#TRUST_PROXY=1
-
-# Only needed if the UI is served from a different origin than the API.
-#CORS_ORIGIN=https://forecast.example.com

.gitignore

@@ -1,4 +1,3 @@
 node_modules/
 .env
-.env.*
 public/app/

CLAUDE.md

@@ -4,8 +4,8 @@
 
 A web app for building named forecast scenarios against any PostgreSQL table. The workflow: load historical actuals as a baseline (optionally date-shifted into the forecast period), then apply incremental adjustments (scale, recode, clone) to build a plan. All changes are append-only, fully audited, and reversible by log entry.
 
-Full spec, including the rationale and history this file leaves out: `docs/pf_spec.md`
-Data transport architecture options: `docs/pf_perspective_options.md`
+Full spec: `pf_spec.md`
+Data transport architecture options: `pf_perspective_options.md`
 
 ---
 
@@ -14,58 +14,40 @@ Data transport architecture options: `docs/pf_perspective_options.md`
 - **Backend:** Node.js / Express (`server.js`)
 - **Database:** PostgreSQL — isolated `pf` schema
 - **Frontend:** React + Vite + Tailwind CSS in `ui/`; built output lands in `public/app/`
-- **Pivot:** [Perspective](https://github.com/perspective-dev/perspective) (`@perspective-dev/*` distribution, **not** FINOS `@finos/perspective`) 5.4.0 from a **patched build vendored in `ui/vendor`** — see its README for what is patched and which host-tool versions the rebuild needs; **bundled inline via the `/inline` entrypoints — never from a CDN** (the CDN bundle resolves its server WASM to an unversioned path and silently pulls whatever is newest). See `docs/PERSPECTIVE.md`.
+- **Pivot:** [Perspective](https://github.com/perspective-dev/perspective) (`@perspective-dev/*` distribution, **not** FINOS `@finos/perspective`) 4.4.0 loaded from CDN at runtime — see `PERSPECTIVE.md` for config/deploy guidance
 - **Dev:** `npm run dev` (nodemon) in root; `npm run build` in `ui/`
+- **Schema:** forward-only SQL migrations in `setup_sql/migrations/`, applied by `npm run migrate` and tracked in `pf.schema_version`. `server.js` refuses to start when the database is behind. See `setup_sql/README.md`
 
 ---
 
 ## Project layout
 
 ```
-server.js               Express entry point; pg pool; session; type parsers for bigint/numeric
-pf.sh                   Install, deploy and admin: config, db-setup, deploy, install-service,
-                        add-user, passwd, set-territory, … (fresh install: config → db-setup
-                        → add-user → deploy → install-service)
+server.js               Express entry point; pg pool; type parsers for bigint/numeric
 routes/
-  auth.js               POST /api/login, /api/logout, GET /api/me; login throttle
   tables.js             GET /api/tables, /api/tables/:schema/:tname/preview
-  sources.js            Source registration, col_meta, SQL generation;
-                        GET /sources/:id/values/:col, /sources/:id/lookup,
-                        /sources/:id/dim/:group (+ POST …/refresh), /dim-period/cols
-  versions.js           Version CRUD and clone; GET /versions/:id/tags (+ PATCH),
-                        /versions/:id/bridge, /versions/:id/values/:col, /versions/:id/table-info
-  operations.js         Data streams (GET /versions/:id/data, /versions/:id/agg); loads
-                        (baseline, reference); scale, recode, clone
-  log.js                GET /api/versions/:id/log, /log/:logid/debug; DELETE and PATCH /log/:logid
-  layouts.js            Named pivot layouts — list per version, create, patch, delete
-  plans.js              Planned load segments — list, create, patch, delete
+  sources.js            Source registration, col_meta, SQL generation
+  versions.js           Version CRUD, baseline/reference load, data stream
+  operations.js         scale, recode, clone, undo — the core forecast ops
+  log.js                GET /api/versions/:id/log, DELETE /api/log/:logid
 lib/
-  sql_generator.js      grainOf, buildWhere/buildWhereAny, SCOPE_OPS, token substitution helpers
-  auth.js               scrypt hash/verify, requireAuth, requireAdmin, sessionUser; `node lib/auth.js hash` CLI
-  utils.js              RELATION_COLUMNS_SQL, mapType
+  sql_generator.js      buildFilterClause, token substitution helpers
+  utils.js
 setup_sql/
-  01_schema.sql         pf schema DDL — idempotent, not a migration runner
-  02_auth.sql           pf.app_user + pf.session
-  gen_dim_period.sql    Populates pf.dim_period; fiscal year start month is configurable
-ui/vendor/              Patched Perspective build and its patches (see its README)
+  README.md             migration workflow — read before changing the schema
+  migrations/           ordered .sql, applied once, tracked in pf.schema_version
+  gen_dim_period.sql    parameterised calendar load (not a migration)
+  schema.generated.sql  pg_dump reference snapshot; generated, never edited
+scripts/
+  migrate.js            migration CLI (up | status | baseline)
+  schema-dump.js        regenerates schema.generated.sql
 ui/src/
-  main.jsx              ThemeProvider, AuthProvider, Gate (mounts the app only once signed in)
-  App.jsx               Page routing; source/version selection and its localStorage restore
-  theme.jsx             ThemeContext / useTheme
-  auth.jsx              AuthProvider/useAuth; wraps fetch so any 401 returns to login
   views/
-    Login.jsx           Sign-in form
     Setup.jsx           DB browser, source registration, col_meta editor
-    Baseline.jsx        Version management, baseline workbench, reference load, tags panel
-    Forecast.jsx        Perspective pivot, selection handling, operation dispatch
-  components/
-    DepthButtons.jsx    One axis's collapse stepper; both axes are drawn from it
-    LayoutMenu.jsx      The Layout ▾ control — Published / Mine, with the write actions
-    OperationPanel.jsx  The adjustment workbench — ledger + scale/recode/clone forms; effectiveScaleInputs
-    BridgeView.jsx      Baseline → current waterfall by tag (exports buildSteps/layoutSteps)
-    Sidebar.jsx         Forecast list + the 3 steps; collapsible
-    StatusBar.jsx       Forecast name · row count · calculating · theme; source on Setup only
-    ReadOnlyNotice.jsx  The banner over a page a non-admin may read but not change
+    Baseline.jsx        Version management, baseline workbench, reference load
+    Forecast.jsx        Perspective pivot + operation panel (Scale/Recode/Clone)
+    Sidebar.jsx         3-step collapsible nav
+    StatusBar.jsx       Source · version · row count · status
     Timeline.jsx        Date-range preview bar for baseline segments
 ```
 
@@ -74,581 +56,83 @@ ui/src/
 ## Database schema (`pf`)
 
 - **`pf.source`** — registered source tables
-- **`pf.col_meta`** — column roles: `dimension` | `value` | `units` | `date` | `filter` | `ignore`; `dim_group` groups functionally dependent columns (e.g. a part and its attributes, or a date and its derived year/month dimensions); `dim_period_col` maps a dimension to a `pf.dim_period` column so date-adjacent values are derived at load time rather than copied raw; `in_grain` flags dimension/date columns that define the **display grain**; `is_territory` marks the column territories are expressed in (one per source); `is_key` is described under §`is_key` below
-- **`pf.dim_member`** — members of each `dim_group`, keyed on the group's key column; refreshed by `POST /sources/:id/dim/:group/refresh`
-- **`pf.version`** — named forecast scenarios; `exclude_iters` (default `["reference"]`) blocks those iter values from all operations; `adjustment_segment` / `adjustment_bucket` name new adjustments; `is_default` (one per source)
-- **`pf.fc_{tname}_{version_id}`** — one forecast table per version; contains both operational rows (`pf_iter = baseline|scale|recode|clone`) and reference rows (`pf_iter = reference`). Indexed on `pf_logid`, which is how undo, the change-log aggregate and the grain key all find their rows. Tables created before that index was added do not have it.
-- **`pf.log`** — audit log; every write gets one entry; `slice` + `params` stored as jsonb; `label` / `bucket` required (`log_named` CHECK). Stamped at write time: `row_count`, `value_total`, `units_total`, `measure_cols` (which columns the totals are denominated in), `env` (the state the intent executed against), `sql_text` (the statement as executed)
-- **`pf.layout`** — named Perspective view configs; see §Pivot layouts
-- **`pf.load_plan`** — load segment definitions, before and apart from having been run; see §Cloning a version's definition
+- **`pf.col_meta`** — column roles: `dimension` | `value` | `units` | `date` | `filter` | `ignore`; `is_key` marks dimensions used in slice WHERE clauses; `dim_group` groups functionally dependent columns (e.g. date + its derived year/month dimensions); `dim_period_col` maps a dimension to a `pf.dim_period` column so date-adjacent values are derived at load time rather than copied raw; `in_grain` flags dimension/date columns that define the **display grain** (see below)
+- **`pf.version`** — named forecast scenarios; `exclude_iters` (default `["reference"]`) blocks those iter values from all operations
+- **`pf.fc_{tname}_{version_id}`** — one forecast table per version; contains both operational rows (`pf_iter = baseline|scale|recode|clone`) and reference rows (`pf_iter = reference`)
+- **`pf.log`** — audit log; every write gets one entry; `slice` + `params` stored as jsonb
 - **`pf.sql`** — generated SQL templates per source/operation; tokens substituted at request time
-- **`pf.app_user`** — login accounts; scrypt `pass_hash`, `is_active`, `last_login_at`, `is_admin`, `territory` (jsonb array)
-- **`pf.session`** — express-session store (connect-pg-simple layout)
-- **`pf.dim_period`** — calendar lookup table (2018–2035); one row per month keyed on `sdat` (month start date); provides cal/fiscal year, quarter, and month columns; populated by `setup_sql/gen_dim_period.sql`
-
-Dead columns, kept but read by nothing: `pf.version.unlabeled_load`, `pf.version.bucket_order`, `pf.log.seq`, `pf.source.default_layout`.
-
-### `is_key`
-
-Read in four places, and **not** the one it sounds like — slices are validated
-against `filterCols`, which is every `dimension` plus every `date` column
-regardless of `is_key`.
-
-1. **The key of a `dim_group`** — `resolveGroup()` in `routes/sources.js` takes
-   `members.find(c => c.is_key)`, the column every other member is keyed on for
-   `pf.dim_member`
-2. **Value completion** — only `is_key` columns get a dropdown, and
-   `/sources/:id/values/:col` refuses anything else
-3. **Sibling autofill** — fires on blur only when `is_key && dim_group`
-4. **The `dim_period` anchor** — `role === 'date' && is_key && dim_group` picks the
-   date whose siblings are derived from the calendar
-
-Uses 2 and 3 want several columns flagged; use 1 needs exactly one per group.
-**When a group has more than one, `.find()` silently takes the lowest `opos`** —
-a refresh keyed on the wrong column matches nothing and reports success with
-zero members. Until the two meanings are separated, a refresh that finds two
-keys should refuse rather than guess. Rationale: docs/pf_spec.md → §Data model.
+- **`pf.dim_period`** — calendar lookup table (2018–2035); one row per month keyed on `sdat` (month start date); provides cal/fiscal year, quarter, and month columns; populated by `setup_sql/gen_dim_period.sql` with a configurable fiscal year start month
+- **`pf.schema_version`** — applied migrations (filename + checksum); owned by the migration runner, never edited by hand
 
 ### Key token substitution tokens
-`{{fc_table}}`, `{{where_clause}}`, `{{exclude_clause}}`, `{{territory_clause}}`, `{{logid}}`, `{{pf_user}}`, `{{label}}`, `{{bucket}}`, `{{value_incr}}`, `{{units_incr}}`, `{{pct}}`, `{{set_clause}}`, `{{scale_factor}}`, `{{date_offset}}`, `{{filter_clause}}`
+`{{fc_table}}`, `{{where_clause}}`, `{{exclude_clause}}`, `{{logid}}`, `{{pf_user}}`, `{{value_incr}}`, `{{units_incr}}`, `{{pct}}`, `{{set_clause}}`, `{{scale_factor}}`, `{{date_offset}}`, `{{filter_clause}}`
 
 ---
 
 ## Core data flow
 
 ### Initial load (Forecast view)
-`Forecast.jsx` fetches col_meta first, then picks the endpoint (both in `routes/operations.js`):
+`Forecast.jsx` fetches col_meta first, then picks the endpoint:
 
 - **grain mode** (any `in_grain` column) — `GET /api/versions/:id/agg`, rows pre-aggregated to the grain, table indexed on `pf_gkey`
 - **raw mode** (no grain) — `GET /api/versions/:id/data`, raw forecast rows, table indexed on `pf_id`
 
 Either way: Arrow IPC binary stream → `worker.table(buffer)` in Perspective WASM. `fetchArrow()` handles both.
 
-**One batch, not streaming.** Per-batch Arrow encoding creates independent dictionaries that crash Perspective WASM on dictionary replacement messages, so the server accumulates all rows and emits one record batch. pg returns `bigint`/`numeric` as strings by default — the type parsers in `server.js` coerce them to numbers; do not remove them.
+**Why one batch (not streaming):** pg returns `bigint`/`numeric` as strings by default — type parsers in `server.js` coerce them to numbers. Per-batch Arrow encoding creates independent dictionaries that cause Perspective WASM to crash on dictionary replacement messages. Server accumulates all rows, emits one record batch.
 
 ### Display grain
-Aggregating to the grain the pivot displays is the load-time fix, and it keeps the **native** Perspective engine, so expand/collapse/depth/sort/filter all still work. Set the grain in Setup (`in_grain` per column); it is baked into `pf.sql` at Generate SQL time so load and operations agree. **`grainOf()` in `lib/sql_generator.js` is the single definition of the grain** — `routes/log.js` calls it directly and `Setup.jsx` mirrors it. Design: docs/pf_spec.md → §Display-grain pre-aggregation. Why not a DuckDB virtual server: `docs/pf_perspective_options.md` → §Spike findings.
-
-### Log-derived columns
-`/data` and `/agg` both LEFT JOIN `pf.log` and emit four columns the forecast
-table does not itself carry:
-
-- **`pf_segment`** — `pf.log.label` (`SEGMENT_EXPR`)
-- **`pf_bucket`** — `pf.log.bucket` (`BUCKET_EXPR`)
-- **`pf_note`** — tag, else note, on a scale/recode/clone; null on loads (`NOTE_EXPR`)
-- **`pf_op`** — `pf.log.operation`
-
-Only segment and bucket are stored names (see §Every log entry is named); the
-note is derived. `SEGMENT_EXPR` / `BUCKET_EXPR` / `NOTE_EXPR` in
-`lib/sql_generator.js` are the single definition, shared with the `/data`
-cursor in `routes/operations.js` — `/agg` is generated, `/data` is not, and the
-two have to agree. `LABEL_GROUP_COLS` lists the `pf.log` columns they read, for
-`/agg`'s GROUP BY. In grain mode `pf_logid` is part of the grain, so the join
-adds no rows. The operation routes stamp the same fields onto the rows they push
-back incrementally, since those come from `RETURNING *` and would otherwise
-arrive without them.
-
-### Column order is stored text
-
-Perspective orders column groups by the value string, so a `"01 - "` prefix in
-**`pf.log.label`** / **`pf.log.bucket`**, typed by whoever names the segment, is
-the only lever; nothing derives it. `ADJUSTMENT_SEGMENT` = `'99 - Adjustments'`
-lands last only when the loads carry `01`–`0n` prefixes (digits precede
-letters). Do not reintroduce ordering as Perspective expressions — `restore()`
-replaces `expressions` wholesale and ExprTK cannot handle non-ASCII labels;
-`DEAD_ORDER_EXPRS` in `Forecast.jsx` strips the old ones from saved layouts.
-Rationale: docs/pf_spec.md → §Naming: segments, buckets, notes.
-
-### Every log entry is named
-
-`pf.log.label` is a row's segment and `pf.log.bucket` what it counts toward, and
-**every entry carries both** — the `log_named` CHECK refuses one without. A
-report gets the pivot's segment and bucket by joining the forecast table to
-`pf.log` on `pf_logid`; there is no rule to copy. Never derive them at read time.
-
-They are stamped when the entry is written:
-
-- **A load** — `label` is required (the routes answer 400 without one); `bucket`
-  defaults to the label. `loadNames()` in `routes/operations.js`.
-- **An adjustment** — scale, recode and clone have no name inputs, so they are
-  stamped with the version's `adjustment_segment` / `adjustment_bucket`, else the
-  built-ins `ADJUSTMENT_SEGMENT` = `99 - Adjustments` and `ADJUSTMENT_BUCKET` =
-  `04 - Forecast` (`ADJUSTMENT DEFAULTS` in `lib/sql_generator.js`).
-  `adjustmentNames()` resolves them once per request; they go into the template's
-  INSERT as `{{label}}` / `{{bucket}}` and onto the rows pushed back to the pivot.
-
-The two version defaults are set on the Baseline page under **Defaults for new
-adjustments**. Changing one names the *next* adjustment; existing entries keep
-what they were written with. Renaming past entries is `PATCH /api/log/:logid`,
-one at a time, which renames but refuses to clear. A relabel needs a page
-reload to show, since the label is part of the aggregated row. A source
-registered before names were stored needs **Generate SQL** run once so its load
-templates write `label` and `bucket` at all. Rationale: docs/pf_spec.md → §Naming: segments, buckets, notes.
+Aggregating to the grain the pivot actually displays is the load-time fix — measured 534,902 → 6,154 rows on `osm_stack`. It keeps the **native** Perspective engine, so expand/collapse/depth/sort/filter all still work. Set the grain in Setup (`in_grain` per column); it is baked into `pf.sql` at Generate SQL time so load and operations agree. `grainOf()` in `lib/sql_generator.js` is the single definition of what the grain is — `Setup.jsx` and `routes/log.js` mirror it. Full design: `pf_spec.md` → §Display-grain pre-aggregation. Why not a DuckDB virtual server: `pf_perspective_options.md` → §Spike findings.
 
 ### Forecast operations
 POST to `/api/versions/:id/{scale|recode|clone}` → SQL executed with `RETURNING *` → new rows returned as JSON → `pspTable.update(rows)` — no full reload. In grain mode the operation's final CTE aggregates its own new rows to grain first; since `pf_logid` is part of `pf_gkey` those keys are always new, so `update()` **appends** and the view re-sums.
 
-### Price or volume (`plug`)
-
-**Price does not move unless it is named.** A price move is invisible in the
-figure being edited — the dollars land on target either way — so every default
-holds price. All resolution is in `resolveIncrs()` in `routes/operations.js`;
-the semantics come from the predecessor Excel model
-(`/opt/forecast_api/VBA/fpvt.frm` → `calc_val` / `calc_price`).
-
-- **Dollars only** — `plug` decides: `'volume'` (the default: price holds, units
-  scale in proportion) or `'price'` (volume holds, price moves).
-- **Qty only** — dollars follow at the current price. A slice with no units has
-  no price to hold, so the dollars stay put.
-- **Dollars and qty** — both land as named; price falls out.
-- **A named price (`target_price`) holds**, and the other named measure decides
-  the rest: price alone → qty holds; price + qty → qty lands as named, dollars
-  follow; price + dollars → dollars land, qty follows. **All three is refused**
-  server-side (400).
-- **The held price is the adjustable rows' own**, never blended with the fixed
-  (excluded-iter) rows, even under `target_basis: 'selected'` — the scale SQL
-  prorates over the adjustable rows only, so a blended price drives qty and
-  dollars apart (and qty negative when a slice is taken to its fixed total).
-
-**The three boxes foot.** At most two are inputs and the third is derived.
-`effectiveScaleInputs()` in `OperationPanel.jsx` is the single resolver: the
-panel renders from it and `buildPayload` in `Forecast.jsx` builds the request
-from it, so what is posted is what is shown. Typing a third drops one — the plug
-decides when dollars came last, else the older of the other two. The ledger's
-**Result** line previews value, units and price by the same rules, and the
-column that cannot move is labelled `· holds`.
-
-**The plug control is labelled "Moves"**: it lights whatever the edit moves
-(both if both do) and takes a click only while it is the one deciding; otherwise
-it stays on screen, not hidden.
-
-**Logged.** `resolved.method` (`price` | `both` | `volume` | the plug) records
-which side moved, since the request alone cannot say. `params` also keeps
-`plug` and `expected` (what the Result line promised), beside `resolved`, so a
-panel/server disagreement can be found by reading the log — nothing compares
-them yet. The client's stored preference is keyed `pf_scale_plug_v2`; do not
-reuse the old key, which could carry a stale `'price'`. Rationale: docs/pf_spec.md → §Operations.
-
 ### Undo
 `DELETE /api/log/:logid` → removes rows by logid → `table.remove()` of the affected index values (`pf_gkeys` in grain mode, `pf_ids` in raw mode); the view re-sums. No full reload.
 
 ---
 
-## Axis depth (collapse / expand)
-
-Both axes collapse the same way: a **depth in `ViewConfig`**, set through
-`restore()` — `group_by_depth` via `applyDepth()`, `split_by_depth` via
-`applySplitDepth()`. Both steppers are drawn by `components/DepthButtons.jsx` from the axis's own field list
-(`groupFull` / `splitFull`), as a stepper `◂ level ▸`, the middle button opening
-the full level list. `depth` is null until the first layout has been read back;
-the stepper then shows an em dash.
-
-**Depths are 1-based counts of levels to show**, on both axes and in
-`DepthButtons` (`levels[d - 1]`); the value crosses unchanged in each direction.
-The engine agrees (`set_depth(n - 1)`, and the traversal adds the 1 back). **Never
-send 0**: it reaches the engine as `set_depth(-1)` on an unsigned `t_depth`,
-underflows, and renders fully expanded. `applyDepth` floors at 1, so the
-stepper's "Total" slot shows the first level rather than a bare total — kept
-deliberately. A real total would need `group_rollup_mode: 'total'`, unverified
-against the local engine.
-
-A depth lives in the config, so it survives view rebuilds and rides into saved
-layouts through `viewer.save()`. **This required patching the engine** —
-`ViewConfig::apply_update` dropped both depth fields; see
-`ui/vendor/0001-apply-depth-fields-on-config-update.patch`. Do not go back to
-imperative `view.set_depth()` alone: the viewer discards that view on every
-re-render. `split_full` is still *read* on load, for layouts saved under the
-old truncated-`split_by` scheme.
-
-**Auto-pause is off.** `initViewer()` calls `viewer.setAutoPause(false)` right
-after `viewer.load()`. Pausing deletes the view, so returning to the tab would
-rebuild the whole grain and lose per-node expansion.
-
-**Per-node expansion is not solved.** `expand(row_index)` / `collapse(row_index)`
-are set-only with no getter, so they cannot be captured and replayed and are
-lost on every rebuild. The route is a `ViewConfig` field of expanded row paths
-applied in `server.cpp`. Depth is also whole-axis — one level collapses every
-group at that level. Rationale: docs/pf_spec.md → §The pivot.
-
-## Tag completions are per version
-
-`GET /versions/:id/tags` — the tags used on **this** version, with `uses`,
-`last_used`, `owners` and `can_edit`; `Forecast.jsx` refetches on `versionId`
-and after every tagged write. A tag is not a row anywhere — only a value
-repeated across `pf.log` entries.
-
-**Managed on the Baseline page's Tags panel**, which lists each with its use
-count and last-used date. `PATCH /versions/:id/tags` takes `{ from, to }` and
-rewrites every entry carrying `from`; a `to` that already exists **merges**, an
-empty `to` clears; the panel confirms both. Permission is the `pf.log` rule
-applied to *every* entry the tag is on (a half-applied rename leaves the
-initiative split), so a row with `can_edit` false is greyed out. One entry's
-tag is edited in the Forecast page's change log.
-
-**Completion matches on words, not substrings.** `TagInput` in
-`OperationPanel.jsx` normalises both sides (case, punctuation, stopwords, light
-stemming) and says so when a typed tag reduces to an existing one; nothing is
-merged automatically. Rationale: docs/pf_spec.md → §Bridge and change log.
-
----
-
-## Choosing a forecast
-
-The **sidebar** lists every source's versions (`allVersions`, a client-side
-fan-out over `/sources/:id/versions`) and sets `sourceId` from the one picked.
-`versions` stays scoped to the selected source, because Baseline and Forecast
-are — do not widen it. The source picker is only on Setup. Known collision:
-"forecast" is both the scenario and the third step; undecided.
-
-**Where a browser lands.** `App.jsx` restores `pf_sourceId` / `pf_versionId`
-from `localStorage`, re-checked against the live list. With nothing stored it
-falls back to **`pf.version.is_default`** (one per source, partial unique index
-`version_one_default`), not `versions[0]`, which is the newest. A stored
-selection wins. Setting the default is **admin only**, in `PUT /api/versions/:id`,
-as two statements in one transaction — the index refuses the instant between.
-
-**Renaming a version** is safe at any time; nothing derives from the name.
-`PUT /api/versions/:id` answers 409 on a `(source_id, name)` collision, and
-COALESCEs a falsy name back to the existing one, so a blank box is refused
-client-side.
-
----
-
-## Cloning a version's definition
-
-`POST /sources/:id/versions` takes `clone_from`. It copies the template
-version's settings (`description`, `exclude_iters`, the fallback names,
-`bucket_order`), its **version-scoped** layouts, and its load segments — and
-**runs nothing**, so the dates can be changed before anything loads.
-
-Not copied: adjustments, in any form (a scale's target was sized against the
-rows that were there), and source-wide layouts (`version_id IS NULL`), which
-already apply to the new version.
-
-### `pf.load_plan`
-
-A plan is a load definition on its own — `kind`, `label`, `bucket`, `tag`,
-`note`, `filters`, `raw_where`, `date_offset`, `seq` — with `run_logid` naming
-the `pf.log` entry it produced, or null while unrun. **Deliberately not
-`pf.log`**: every row there means a write happened.
-
-`run_logid` is `ON DELETE SET NULL`, so undoing a load returns its plan to unrun.
-The plans list reads the row count back through that FK.
-
-`cloneDefinition()` in `routes/versions.js` reads `pf.load_plan` when the
-template has plans and otherwise derives them from its `baseline`/`reference`
-log entries (`pf.log.params` stores `filters`, `raw_where`, `date_offset`).
-
-**Run all stops at the first failure** — segments are not independent.
-Rationale: docs/pf_spec.md → §Loads and segments.
-
----
-
-## Pivot layouts
-
-A layout is a named `ViewConfig` in **`pf.layout`**, owned by an account:
-**published** (everyone on the forecast lists and can apply it; only its owner
-or an admin may change it) or **private**. Scope is the **version** by default;
-`version_id IS NULL` means every version of the source. `is_default` (at most
-one per scope, by partial unique index) is what `initViewer()` restores on a
-first load — a version-scoped default beating a source-wide one.
-
-**Permissions are the `pf.log` rule verbatim** — your own, or an admin's
-override; `can_edit` rides on every row and the UI greys out the rest. Applying
-is never restricted; Save is withheld on a layout that isn't yours, Save as…
-forks it. No territory clause — a layout is display config.
-
-**Local state.** `LAYOUT_KEY` (`pf_layout_v{vid}`), the unnamed last-used
-config, stays in `localStorage`; `loadLayouts()` lifts the old named list
-(`LAYOUTS_KEY`) into `pf.layout` as private rows once, then clears it.
-
-**The dirty dot is a comparison, not a flag.** `restore()` itself fires
-`perspective-config-update`, so `activeConfigRef` holds what the pivot last
-matched and `sameConfig()` compares against it, ignoring `table` and `theme`.
-
-**Grand totals ride in the layout under `pf_totals`** — `{ row, col }`. The
-engine has no "subtotals but no grand total" mode, so both are hidden in the
-grid's DOM; `captureConfig()` adds the key and `cleanLayout()` strips it before
-`restore()`. Without it: row shown, column hidden.
-
-**`cleanLayout()`** drops anything the live schema lacks — axes, `aggregates`
-(including the weight column of the multi-arg form), `theme`, `pf_totals`, dead
-expressions — dropping the offending *entry* rather than the layout, since
-`restore()` is all-or-nothing. Rationale: docs/pf_spec.md → §The pivot.
-
----
-
-## The Forecast toolbar is two zones
-
-Left, **view** — `Layout ▾` and the `Rows` / `Columns` steppers with their
-`Total` toggles; all `ViewConfig`, nothing reaches the server. Right,
-**forecast** — `Refresh data`, `Change log`, `Bridge`, `Operations`. **Blue is
-for actions, pressed grey (`.pf-pressed`) for state.** `Fit widths` was removed
-deliberately; do not restore it without fixing auto-size (see git history).
-
-**`Calculating…`** in the status bar: `Forecast.jsx` raises a counter (rebuilds
-overlap) in `applyDepth`, `applySplitDepth` and `applyLayout`, lowers it after
-`viewer.flush()`, and broadcasts `pf-pivot-busy` on the window for `StatusBar`.
-**Not** driven from `perspective-config-update`, which arrives after the
-rebuild. Rationale: docs/pf_spec.md → §The pivot.
-
----
-
 ## Slice mechanics
 
-A click (`perspective-click`) or a dragged region becomes a **slice** — a
-`{col: value}` object. A plain click replaces the selection; ctrl/⌘/shift-click
-toggles a slice in or out, so the panel holds a **list** sent as `slices` in
-operation POST bodies (a single `slice` is still accepted server-side).
+When the user clicks a pivot cell, `perspective-click` fires. The handler in `Forecast.jsx` extracts `[col, '==', value]` filters from `detail.config.filter` — only `role = dimension` columns are kept as the slice. This slice populates the operation panel and is sent as the `slice` object in all operation POST bodies.
 
-**The slice is re-derived by `cellFilters()` in `Forecast.jsx`**, from
-`detail.row.__ROW_PATH__` (row axis, raw values, so dates stay epoch millis)
-and the clicked column name (column axis, split by `|`). The click and a dragged
-region go through the same derivation. `detail.config.filter` is used only as
-the fallback when there is no cell to index the column axis with — it cannot be
-trusted otherwise, because the engine builds it with a falsy test that drops
-blank and `0` pivot values, and a dropped key **widens** the write.
-
-**Regions.** The datagrid runs in `edit_mode: SELECT_REGION` (forced on
-restore). `perspective-select` carries a **ViewWindow** —
-`{ start_row, end_row, start_col, end_col }` — and fires on every mouseover, so
-the handler records the latest window and a window-level `mouseup` commits it.
-A single-cell region is ignored there: `perspective-click` owns it, and handling
-it twice would undo a ctrl-click toggle.
-
-**A total cell is its displayed amount.** The grand-total row resolves to no
-row dimension, so its slice is whatever the column axis and the pivot's `==`
-filters leave — possibly `{}`. An empty slice is accepted only alongside a
-non-empty `scope`; with no filter at all it would be the whole version, and is
-refused on both sides. `buildWhereAny()` reads a union containing `{}` as
-`TRUE`, the scope then ANDed on — dropping it would narrow the write.
-
-**The pivot's `pf_` columns stay in the slice.** `pf_segment`, `pf_bucket`,
-`pf_note` and `pf_op` are resolved through the log (`COMPUTED_SLICE_COLS`);
-`pf_iter`, `pf_user` and `pf_logid` are compared directly (`TABLE_SLICE_COLS`).
-Stripping any of them made the write wider than the ledger's total. They narrow
-a slice but do not make it selective on their own. Duplicates are collapsed
-before the request — otherwise `apply_mode: each` applies the change twice.
-
-**Selection highlight.** The datagrid wipes `model._selection_state.selected_areas`
-on every mousedown, so `Forecast.jsx` keeps `areasRef` (`sliceKey -> rectangles`,
-parallel to `slices`) and an effect pushes the full set back and redraws after
-every change. Every deselect path prunes it by live slice key.
-
-**A blank dimension value is a value.** `null` and `''` are different rows and
-stay distinct through to the WHERE clause: `IS NULL` vs `= ''`. Only `undefined`
-means absent (a subtotal row's shorter path, a collapsed column level). On the
-column axis, `t_tscalar::to_string` renders a null as the literal text `"null"`
-and an empty string as `''`, so `cellFilters()` maps `"null"` → null. The ledger
-filters a null with `is null`, never `== null` (the engine matches on type). The
-panel prints `(null)` and `(blank)`, and `—` only when the column is absent from
-the slice.
-
-**The pivot's own filter travels alongside, as `scope`** — `[col, op, value]`
-triples read off `viewer.save()`, which is also what the ledger totals against.
-`SCOPE_OPS` in `lib/sql_generator.js` is the whole allowed list, under
-Perspective's operator names; anything outside it is **refused, not dropped**.
-
-- Substring operators (`contains`, `begins with`, `ends with` and negations) map
-  to **`ILIKE`**, matching the engine's case-insensitive `t_tscalar::contains`;
-  `%`, `_` and `\` are escaped out of the term.
-- `matches` / `not matches` stay refused: the engine is RE2, Postgres is not.
-- Every operator that needs a term refuses an empty one (an unfinished filter
-  row arrives as `[col, '==', null]`).
-
-**Limitation:** computed columns created by Perspective expressions don't map
-back to raw rows — only native columns work for slice extraction. Rationale:
-docs/pf_spec.md → §Selection.
+**Limitation:** computed columns created by Perspective's split_by (e.g. Month, YearDate) don't map back to raw rows — only native dimension columns work for slice extraction.
 
 ---
 
 ## Operation SQL patterns
 
-All three operations follow the same structure: insert a `pf.log` row in a CTE, then insert forecast rows referencing its id. `{{where_clause}}` is built from the slices; `{{exclude_clause}}` blocks `exclude_iters` rows; the territory clause is ANDed on last.
+All three operations follow the same structure: insert a `pf.log` row in a CTE, then insert forecast rows referencing its id. `{{where_clause}}` is built from the slice; `{{exclude_clause}}` blocks `exclude_iters` rows.
 
-**`buildWhere()` in `lib/sql_generator.js`** turns a slice into SQL: it allows
-`dimension` + `date` columns plus `COMPUTED_SLICE_COLS` and `TABLE_SLICE_COLS`,
-**skips** unknown non-`pf_` keys, and **refuses** an unknown `pf_` key (skipping
-one would widen the write). A null renders as `IS NULL`; in a list the null arm
-is ORed on, since `IN (NULL)` is never true. Values are escaped but not
-parameterized — consistent with existing patterns, debuggable in pg logs. The
-"must name a filterable column" refusal is **`assertSelective()`** in
-`routes/operations.js`.
-
-**Every date column travels.** `dataCols` is `[...dims, ...dateCols, value, units]`
-— never just the first date column, or scale and recode write rows with null
-secondary dates under correctly labelled period dimensions.
-
-- **Scale** — distributes `value_incr`/`units_incr` proportionally across rows in the slice using window functions; refuses a pool whose net is ~zero relative to its gross (`assertProratable`)
+- **Scale** — distributes `value_incr`/`units_incr` proportionally across rows in the slice using window functions
 - **Recode** — inserts negative rows (zero out original) + positive rows with `{{set_clause}}` dimension overrides; both share the same logid
 - **Clone** — copies the slice with `{{set_clause}}` overrides and `{{scale_factor}}` multiplier; original untouched
 
-**Recode takes a date shift too.** `date_offset` moves the *put-back* leg only,
-so a shifted recode is a timing move, and with a shift `set` may be empty. The
-moved rows' period dimensions are re-derived from `pf.dim_period` against the
-shifted date; a row whose date is null keeps the label it had. Without a shift
-nothing is derived. Clone still blanks those labels on a null date.
-
-**A date offset must be whole days.** `assertInterval()` refuses anything finer,
-because every shifted date is cast back to a date — and Postgres reads a bare
-number as *seconds*. `30 days` is not a month; use `1 month`.
-
-Scale refuses with "Nothing to scale…" and recode with "Nothing to recode…"
-when there is nothing to write. Rationale: docs/pf_spec.md → §Operations.
+`build_where()` validates every slice key against col_meta (only `role = dimension` allowed). Values are escaped but not parameterized — consistent with existing patterns, debuggable in pg logs.
 
 ---
 
-## Authentication
-
-Everything under `/api` except the auth routes sits behind a session; the React
-app is only mounted once there is one (`Gate` in `main.jsx`).
-
-- **Accounts:** `pf.app_user` — scrypt hashes from `lib/auth.js`, never plaintext.
-  Managed with `./pf.sh add-user | passwd | list-users | disable-user | enable-user`;
-  the password is read on stdin and hashed before it reaches psql.
-- **Sessions:** `pf.session` (connect-pg-simple); revoke by deleting the row.
-  Cookie `pf.sid`: httpOnly, SameSite=Lax, Secure unless `COOKIE_SECURE=false`, 12h rolling.
-- **Config:** `SESSION_SECRET` is required — the server exits at boot without one.
-  `TRUST_PROXY` (default 1) for `req.ip` and secure cookies behind the TLS proxy.
-  `CORS_ORIGIN` is the only way CORS is enabled; never a wildcard with a session
-  cookie.
-- **Login hardening:** `routes/auth.js` throttles to 10 failures per IP per 15
-  minutes (in-memory), returns one message for unknown/wrong/disabled alike, and
-  regenerates the session id on success.
-
-**Identity is server-side.** `pf_user`, `created_by` and `closed_by` come from
-`sessionUser(req)`, **never from the request body**.
-
-## Setup and Baseline are admin-only to write
-
-`requireAdmin` (`lib/auth.js`) is on:
-
-- **sources.js** — `POST /sources`, `PUT /sources/:id/cols`, `POST
-  /sources/:id/generate-sql`, `POST /sources/:id/dim/:group/refresh`,
-  `DELETE /sources/:id`
-- **versions.js** — `POST /sources/:id/versions`, `PUT /versions/:id`,
-  `POST /versions/:id/{close,reopen}`, `DELETE /versions/:id`
-- **operations.js** — the loads: `POST /versions/:id/{baseline,reference}`,
-  `PUT /versions/:id/baseline/:logid`, `DELETE /versions/:id/baseline`
-- **plans.js** — every write
-
-**Not gated, deliberately:** reads (gating what a page needs to render turns it
-into a 403); scale/recode/clone/undo, which carry their own owner and territory
-rules; `PATCH /versions/:id/tags`, whose permission is ownership.
-
-**The pages stay visible and go read-only** — one `<fieldset disabled>` per page
-plus a `ReadOnlyNotice` banner. Setup wraps a real box; Baseline uses
-`display: contents`. Baseline's Tags panel sits outside the fieldset. **None of
-this is the permission** — the server check is. Setup's list fetches coerce
-responses to an array: an error body rendered with `.map` throws during render
-and unmounts the whole app. Rationale: docs/pf_spec.md → §Access.
-
-## Source columns come from pg_catalog
-
-`information_schema.columns` omits **materialized views**, and `gs.osm_skinny` is
-one. `RELATION_COLUMNS_SQL` in `lib/utils.js` is the replacement, used by version
-creation, source registration and the table preview; it returns the
-information_schema shape (`data_type` from `format_type` with the modifier
-stripped, precision/scale from `atttypmod`). The table browser lists from
-`pg_class` by `relkind`. Do not query `information_schema.columns` for source
-columns.
-
-## Territory scoping
-
-An account sees and changes only its own territory: `pf.app_user.territory`
-(jsonb array), `is_admin` for accounts that see everything, and
-`col_meta.is_territory` marks the column (one per source).
-
-**Fail closed.** `buildTerritoryClause()` returns `FALSE`, not `TRUE`, for an
-empty list or an unflagged source.
-
-**Built from the session, never the request.** Unlike `scope`, the territory
-predicate is ANDed on last, in `sliceUnits()` for writes and per route for
-reads, where nothing in the payload can remove it.
-
-Enforced at:
-
-- `/data` — clause on the cursor *and* on the count behind `X-Row-Count`
-- `/agg` — `{{territory_clause}}`, applied **before** the GROUP BY, since the
-  territory column need not be in the grain
-- every operation, through `sliceUnits()`
-- `/sources/:id/values/:col` — completion reads the source table
-- `DELETE /log/:logid` and `PATCH /log/:logid` — by **owner**, not territory:
-  your own entries, or an admin's override. Undo removes an entry wholesale, and
-  `label`/`bucket` name the pivot's columns for everyone in the version
-- recode's `set` — a scoped account cannot set the territory column at all
-
-**The change log** lists everyone's entries, with an **Everyone / Mine** control
-(resets to Mine on every open, not persisted) and the reader's own entries
-marked. `isMine()` is strict authorship, deliberately not `canUndo()`. The fetch
-stays unfiltered, since the control's counts need the whole set. It shows an
-entry's **full** impact — the totals on `pf.log` are company-wide, deliberately.
-
-Managed with `./pf.sh set-territory | set-admin | orphan-territory`.
-`orphan-territory` lists values present in the data that no account owns.
-Rationale: docs/pf_spec.md → §Access.
-
 ## Light / dark mode
 
-`ui/src/theme.jsx` — `ThemeProvider` (in `main.jsx`) and `useTheme()` →
-`{ dark, setDark }`. Stored as `pf_dark` in `localStorage`, else
-`prefers-color-scheme`; the effect toggles `.dark` on `<html>`. `ui/src/index.css`
-defines custom properties under `:root` and `.dark`, with Tailwind overrides
-written as `.dark .bg-white { ... }` — no Tailwind dark-mode config. Palette is
-Perspective's "Pro Dark".
+Theme state lives in `ui/src/theme.jsx` — a React context (`ThemeContext`) with a `ThemeProvider` that wraps the app in `main.jsx`.
 
-The viewer's theme is `viewer.restore({ theme: dark ? 'Pro Dark' : 'Pro Light' })`,
-on load and in `useEffect([dark, versionId])` — a **partial** restore, no
-rebuild. **Not** `setAttribute('theme', …)`, which does nothing in 5.4.0.
-`cleanLayout` drops `theme` and `sameConfig()` ignores it: the theme is the
-reader's choice, never a layout's.
-
-## After a change: restart, and Generate SQL
-
-- **Restart the server** for anything in `routes/`, `lib/` or `server.js`.
-- **Generate SQL** (Setup, per source) whenever `lib/sql_generator.js` changes.
-  The templates are *stored* in `pf.sql`, so editing the generator changes
-  nothing until they are rebuilt — and a template carrying a token the running
-  code does not substitute fails at the database, which reads as an unrelated
-  client-side error.
-
-**Order matters: restart first, then Generate SQL**, or the old code writes the
-templates.
-
-Schema changes are applied to the live database directly and mirrored into
-`setup_sql/` for a fresh install; `01_schema.sql` is idempotent but is not a
-migration runner.
+- **Storage key:** `pf_dark` in `localStorage`; falls back to `window.matchMedia('(prefers-color-scheme: dark)')` on first visit
+- **Toggle:** `setDark(d => !d)` in `StatusBar.jsx`; effect writes `localStorage` and toggles the `.dark` class on `<html>`
+- **CSS:** `ui/src/index.css` defines CSS custom properties under `:root` (light) and `.dark`. All Tailwind color overrides are written as `.dark .bg-white { ... }` etc. — no Tailwind dark-mode config needed
+- **Palette:** dark mode uses Perspective's "Pro Dark" colours (`--bg-primary: #242526`, panels `#2a2c2f`, gridlines `#3b3f46`, text `#c5c9d0`)
+- **Perspective viewer:** `Forecast.jsx` calls `viewer.setAttribute('theme', dark ? 'Pro Dark' : 'Pro Light')` both on initial load and in a `useEffect([dark, versionId])` so the viewer stays in sync when the toggle fires
+- **Consuming the theme:** `import useTheme from '../theme.jsx'` then `const { dark, setDark } = useTheme()`
 
 ## Known issues / active work
 
-- **Rows written before the every-date-column fix have null secondary dates**
-  (`rdate`/`sdate_e`) on scale and recode rows. Deleting and replaying those log
-  entries is the honest repair; a backfill would be inference
-- **Zero-row clones report success** — clone commits an empty log entry and
-  returns `rows_affected: 0`, where scale and recode refuse
-- **The change log does not show an entry's id**, so there is no way to name
-  one when asking about it
-- **Territory is read onto the session at login**, so a change does not reach a
-  signed-in account until it signs in again
-- **The bridge dedupes overlapping slices by `pf_id`**, which grain-mode rows do
-  not carry, so overlapping slices double-count there; and it keeps only
-  dimension/date slice keys, dropping the `pf_` keys operations now honour
-  (`BridgeView.jsx`, the per-slice view loop)
-- **Depth buttons rebuild the view** — `restore()` tears down and rebuilds, a
-  full traversal, where a manual collapse mutates in place. Fix: detect a
-  depth-only change and call `view.set_depth()` too, at the cost of drift
-- **Load time is dominated by row count, not payload size** — serialising rows
-  out of Postgres into JS dominates. The lever is **dynamic grain**; see
-  docs/pf_spec.md → §Display-grain pre-aggregation
-- **Per-node expand/collapse** is lost whenever the view rebuilds; see §Axis depth
-- **Col_meta / version schema drift:** if col_meta roles change after a version's
-  forecast table is created, SQL and DDL go out of sync — delete and recreate
-  the version
-- **Grain drift:** changing `in_grain` after a load requires Generate SQL + a page
-  reload. `routes/log.js` derives the grain from live col_meta, so a grain
-  changed mid-session yields `pf_gkeys` that don't match the loaded table and
-  undo silently removes nothing
-- **Grain is static per source** — an unflagged dimension cannot be pivoted on
+- Operation panel (Scale/Recode/Clone) SQL generation and dim_period JOIN are complete; UI wiring to API still needs completion
+- Load progress bar is jittery — needs throttle (~10 updates/sec)
+- Default pivot layout should be configurable per source (currently hardcodes first 2 dimensions)
+- Source/version selection doesn't persist across page reload
+- Col_meta / version schema drift: if col_meta roles change after a version's forecast table is created, SQL and DDL go out of sync — workaround is to delete and recreate the version
+- Grain drift: changing `in_grain` after a load requires Generate SQL + a page reload, since the loaded table's index and columns are fixed at load time. `routes/log.js` derives the grain from live col_meta, so a grain changed mid-session yields `pf_gkeys` that don't match the loaded table and undo silently removes nothing
+- Migrations are forward-only — there are no down migrations. Rolling back a schema change means writing a new migration that reverses it
+- Grain is static per source — a dimension left unflagged cannot be pivoted on. Dynamic per-cut grain (intersect the viewer's field set with the eligible set) is the additive next step; see `pf_spec.md` → §Display-grain pre-aggregation
 
 ## Deferred (not in v1)
-Baseline replay (not built), approval workflow, export, version comparison, multi-DB connections. Live server-side aggregation (Path A / DuckDB virtual server) is parked on branch `spike/duckdb-virtual-server`. See docs/pf_spec.md → §Open questions.
+Baseline replay (`replay: true` returns 501), approval workflow, territory filtering, export, version comparison, multi-DB connections. Live server-side aggregation (Path A / DuckDB virtual server) is parked on branch `spike/duckdb-virtual-server`.

PERSPECTIVE.md

@@ -0,0 +1,199 @@
+# Perspective — configuration & deployment reference
+
+Canonical guide for how Perspective should be configured, fed, and shipped across our
+apps (**pf_app**, **dataflow**). Both embed the same `<perspective-viewer>` web component
+but made different early choices; this doc defines the target state and the rationale, so
+the two converge instead of drifting.
+
+> **Distribution note:** we use the **`@perspective-dev/*`** packages
+> (repo: <https://github.com/perspective-dev/perspective>, home:
+> <https://perspective-dev.github.io>) — **not** the FINOS/OpenJS `@finos/perspective`
+> packages from <https://perspective.finos.org/>. Same engine lineage, but a separate
+> npm scope, release cadence, and "Pro" theme set (`Pro Dark`/`Pro Light`). This is why
+> `viewer-d3fc` versions on a different schedule than `viewer`/`client` (§2), and why the
+> client's bundled `apache-arrow` (17.x) can lag the server's (§3). Don't mix the two
+> scopes.
+
+---
+
+## Core principle: Perspective is one locked unit
+
+These four things must move together and be pinned together. Bumping one without the
+others is the source of nearly every Perspective bug we've hit:
+
+1. **Loader** — how the JS/WASM gets into the page (npm-inline vs CDN)
+2. **Package version** — `client` / `viewer` / `viewer-datagrid` / `viewer-d3fc`
+3. **Data format** — Arrow IPC vs JSON rows
+4. **`apache-arrow` version** (server-side, only if using Arrow) — must speak an IPC
+   format the client WASM understands
+
+Treat a Perspective upgrade as a coordinated change to all four, gated by the smoke test
+at the bottom of this doc. Never let viewer/client drift ahead of `viewer-d3fc`.
+
+---
+
+## 1. Loading — npm `/inline`, pinned exact (not CDN)
+
+```js
+import perspective from '@perspective-dev/client/inline'
+import '@perspective-dev/viewer/inline'
+import '@perspective-dev/viewer-datagrid'
+import '@perspective-dev/viewer-d3fc'
+import '@perspective-dev/viewer/themes'
+```
+
+- The `/inline` entrypoints bundle the WASM into the Vite build — **no runtime network
+  fetch, works offline / behind a firewall, reproducible from the lockfile.**
+- **Do not load from a CDN at runtime.** It's convenient for a prototype (smaller build,
+  one-line version bumps) but in production it means: app breaks if the CDN is
+  unreachable, version isn't captured in `package-lock.json`, slower cold start, and you
+  pull executable WASM from a third party on every load. (pf_app currently does this in
+  `ui/src/views/Forecast.jsx` — migrating off it is the main open item.)
+- The themes CSS is imported in JS (`@perspective-dev/viewer/themes`), **not** via a
+  `<link>` in `index.html` — so it's bundled and versioned too.
+
+---
+
+## 2. Version policy — a real trilemma (read carefully)
+
+The version choice is constrained by two hard facts about the `@perspective-dev` packages
+**(verified against installed metadata, 2026-06)**:
+
+- **`viewer-d3fc` caps at 4.4.1** — npm publishes no 4.5.x. The d3fc charts (Bar / Line /
+  Treemap / Heatmap / etc.) live only in this package.
+- **The `/inline` and `/themes` entrypoints are 4.5.x-only** — `@perspective-dev/client/inline`,
+  `@perspective-dev/viewer/inline`, and `@perspective-dev/viewer/themes` do **not** exist
+  in 4.4.1's `exports` map. Bundling inline WASM requires 4.5.x.
+
+So you can have at most **two** of these three:
+
+| Want | Requires |
+|---|---|
+| Inline WASM bundling (`/inline`, `/themes`) | **4.5.x** viewer/client |
+| One coherent single-version suite | **4.4.1** everything (d3fc ceiling) |
+| d3fc chart plugins | **4.4.1** viewer-d3fc |
+
+There is **no** version where all three hold. Pick by what the app needs:
+
+- **Inline-bundled + charts** (dataflow's case) → `^4.5.1` viewer/client/datagrid **+
+  `^4.4.1` viewer-d3fc`. This is a deliberate, necessary mixed-version pair, *not* an
+  accident — it's the only combo that keeps both. Accept it; pin the lockfile and gate
+  bumps on the smoke test (§7). Do **not** "fix" it by pinning everything to 4.4.1 — the
+  build breaks (`"./inline" is not exported`).
+- **Coherent single suite, no inline** (e.g. CDN or `.`-entry loading) → pin all four to
+  **4.4.1 exact**. Charts work; you give up `/inline` bundling.
+
+Whatever you pick, **commit the lockfile** so the resolved set can't drift on
+`npm install`. Re-evaluate the whole policy only when `viewer-d3fc` ships a 4.5.x (then a
+fully-coherent inline-capable 4.5.x suite becomes possible).
+
+---
+
+## 3. Data delivery — match the format to the workload
+
+| Workload | Format | Why |
+|---|---|---|
+| Large (100k+ rows), numeric-heavy, writes/incremental updates | **Arrow IPC** | Compact columnar binary, near-zero-copy ingest, carries types (no string coercion). Powers pf_app's 500k-row path. |
+| Small (≤100k), read-only, click-to-inspect | **JSON rows** | Simpler, no encoding step, no dictionary pitfalls. dataflow's model. |
+
+### Arrow constraints (read before choosing it)
+
+If you deliver Arrow, three pieces are coupled and must stay aligned:
+
+- **Numeric type parsers, server-side.** pg returns `bigint`/`numeric` as *strings*; you
+  must coerce them to JS numbers before encoding, or `apache-arrow` infers
+  `Dictionary<Utf8>` instead of `Int`/`Float64`. See `server.js` type parsers (oid 20,
+  1700).
+- **Single record batch.** Per-batch Arrow builds independent dictionaries; the
+  Perspective WASM crashes on dictionary-replacement messages. The server must
+  accumulate all rows and emit **one** batch (`tableToIPC(tableFromJSON(allRows),
+  'stream')`). Consequence: the client "stream" is just a chunked *download* of one
+  batch — nothing renders progressively, and the server holds the full result set in
+  memory per request.
+- **`apache-arrow` pinned to match the client WASM.** Pin it exact in the server
+  `package.json` and treat it as part of the locked unit (principle above). Note the
+  `@perspective-dev/client` build is tested against **`apache-arrow@17.0.0`**, while
+  pf_app's server currently pins **`^21.1.0`** — a real IPC version gap. Prefer aligning
+  the server toward the arrow major the client was built against, or at minimum make
+  Arrow ingestion (smoke test §7) the gate on any arrow bump.
+
+JSON avoids all three but pays in payload size and parse cost, and pushes type handling
+to client-side heuristics — acceptable only at small scale.
+
+---
+
+## 4. Theming
+
+- One toggle drives both app CSS and the viewer:
+  `viewer.setAttribute('theme', dark ? 'Pro Dark' : 'Pro Light')`.
+- Apply it on initial load **and** in an effect keyed on the dark flag, so the viewer
+  re-themes when the toggle fires (not just on mount).
+- App-level dark mode is plain CSS custom properties + a `.dark` class on `<html>`; the
+  viewer theme name is the only Perspective-specific piece.
+
+---
+
+## 5. Layout persistence
+
+- Persist the full viewer config (`await viewer.save()`, incl. `plugin_config`) to
+  `localStorage`, keyed per source/version. Restore with `viewer.restore(cfg)`.
+- **Guard restores against schema drift.** Before restoring, filter the saved config's
+  `columns`/`group_by`/`split_by`/`sort`/`filter` against the columns that actually
+  exist in the current dataset (plus any `expressions`). dataflow's `cleanLayout()` is
+  the reference implementation; a stale layout referencing a dropped column otherwise
+  throws on restore.
+
+---
+
+## 6. Build & deploy (target)
+
+- **Build:** `vite build` emits a static bundle (pf_app → `public/app`, via
+  `outDir: '../public/app'`). The API server serves it statically (`express.static`).
+- **Process:** run the Node API under **systemd** with `Restart=always` and an
+  `EnvironmentFile=.env`; front it with **nginx** (reverse proxy + TLS via certbot).
+  dataflow's `dataflow.service` + `deploy.sh` are the reference; **pf_app has no deploy
+  automation yet** and should adopt the same pattern.
+- **`deploy.sh`** should be idempotent: first run installs (db/schema, UI build, nginx,
+  systemd unit); later runs rebuild the UI and restart the service.
+- Keep secrets in `.env` (db creds), loaded by both the app (`dotenv`) and the systemd
+  unit — never commit it.
+
+---
+
+## 7. Upgrade checklist / smoke test
+
+Run this whenever bumping **any** Perspective package or `apache-arrow`:
+
+1. Confirm `viewer-d3fc` publishes the target version
+   (`npm view @perspective-dev/viewer-d3fc versions`). If not, **don't bump** the others.
+2. Pin all four packages + `apache-arrow` to exact, matching versions; `npm install`;
+   commit the lockfile.
+3. `vite build` — no unresolved imports.
+4. **Arrow apps:** load a real dataset and confirm `worker.table(buffer)` ingests
+   without a WASM dictionary error; verify a numeric column is `Float64`/`Int`, not a
+   string/dictionary.
+5. Open a d3fc **chart** plugin (not just datagrid) and confirm it renders.
+6. Toggle dark/light; confirm the viewer re-themes.
+7. Save a layout, reload, confirm it restores; then drop a column and confirm the
+   cleaned restore doesn't throw.
+
+---
+
+## Per-project state (2026-06)
+
+| | pf_app | dataflow | Target |
+|---|---|---|---|
+| Loader | CDN (runtime) | npm `/inline` | **npm `/inline`** |
+| Version | 4.4.0 (CDN URLs) | 4.5.1 viewer/client + 4.4.1 d3fc | depends on loader (§2) |
+| Data | Arrow IPC (single batch) | JSON (≤100k) | per workload (§3) |
+| `apache-arrow` | `^21.1.0` (client built vs 17) | n/a | pin exact, match WASM |
+| Deploy | none | systemd + nginx + `deploy.sh` | **systemd + nginx + `deploy.sh`** |
+
+**dataflow's 4.5.1/4.4.1 pair is correct** — it's the only combo giving both inline
+bundling and d3fc charts (§2). Leave it; just keep the lockfile committed.
+
+**Open items:**
+- pf_app → move off CDN. Note this forces the §2 choice: going npm-`/inline` means
+  4.5.x viewer/client + 4.4.1 d3fc (same pair as dataflow); or stay coherent at 4.4.x and
+  load via the `.` entry instead of `/inline`. Either way, pin + commit the lockfile, and
+  add deploy automation (systemd + nginx + `deploy.sh`).

docs/PERSPECTIVE.md

@@ -1,187 +0,0 @@
-# Perspective — lessons and rules
-
-What we have learned the hard way about configuring, feeding and upgrading
-Perspective, written for any app that embeds `<perspective-viewer>` (**pf_app**,
-**dataflow**). The pf_app specifics — the vendored 5.4.0 fork, its patch, and how
-to rebuild it — live in `ui/vendor/README.md`; how pf_app uses the viewer day to
-day is in `CLAUDE.md` and `pf_spec.md` → §The pivot.
-
-> **Distribution:** we use the **`@perspective-dev/*`** packages
-> (<https://github.com/perspective-dev/perspective>) — **not** FINOS/OpenJS
-> `@finos/perspective`. Same engine lineage, but a separate npm scope, release
-> cadence and "Pro" theme set (`Pro Dark` / `Pro Light`). Don't mix the scopes.
-
----
-
-## Perspective is one locked unit
-
-These move together and are pinned together. Bumping one without the others is
-the source of nearly every Perspective bug we have hit:
-
-1. **Loader** — how the JS/WASM gets into the page (npm `/inline`, never a CDN)
-2. **Package versions** — `client`, `server`, `viewer`, `viewer-datagrid` (and
-   `viewer-d3fc` where charts are used)
-3. **Data format** — Arrow IPC vs JSON rows
-4. **`apache-arrow` on the server**, when delivering Arrow — it must write an IPC
-   format the client WASM reads
-
-**Pin `@perspective-dev/server` explicitly.** `@perspective-dev/client` declares
-it as `"@perspective-dev/server": ""` — an empty range, which npm resolves to
-`latest`. Installing an older client pulls whatever server is newest and the WASM
-fails to link (`Import #8 "env" "psp_opfs_load": function import requires a
-callable`). Five packages, not four.
-
----
-
-## Load inline, never from a CDN
-
-```js
-import perspective from '@perspective-dev/client/inline'
-import '@perspective-dev/viewer/inline'
-import '@perspective-dev/viewer-datagrid'
-import '@perspective-dev/viewer/themes'
-```
-
-The `/inline` entrypoints bundle the WASM into the Vite build: no runtime fetch,
-works behind a firewall, reproducible from the lockfile. The themes CSS is
-imported in JS for the same reason, not `<link>`ed from `index.html`.
-
-**This is not hypothetical — a CDN load took pf_app down on 2026-08-10.** The 4.x
-CDN bundle resolves its server WASM with
-`new URL("../../../server/dist/wasm/perspective-server.wasm", import.meta.url)`,
-which from `.../client@4.4.0/dist/cdn/` lands on
-`.../npm/@perspective-dev/server/dist/wasm/perspective-server.wasm` — **no
-version**, so jsdelivr serves `@latest`. The moment a 5.x server published, every
-page load linked a 5.x WASM against a 4.4.0 client and threw the `LinkError`
-above. Nobody changed anything. The security framing matters as much as the
-outage: an unversioned URL means users execute whatever that package publishes
-next, unreviewed. `/inline` has no runtime fetch to hijack.
-
----
-
-## Charts versus inline bundling (apps that need d3fc)
-
-pf_app uses the datagrid only and does not carry `viewer-d3fc`. For an app that
-does, the constraint as of 2026-08: `viewer-d3fc` published nothing past 4.4.1,
-while `/inline` and `/themes` exist only from 4.5.x. **A 4.5.x viewer with a 4.4.1
-d3fc does not work** — every chart plugin throws `get_static_config is not a
-function` (reproduced in isolation, and seen in dataflow's install, where every
-chart type but the datagrid fails). So it is inline bundling **or** d3fc charts,
-not both: go coherent 4.4.1 for charts (still unverified that a bundled 4.4.1
-suite renders them — check before relying on it), or 5.x without them. Re-check
-when d3fc publishes a newer version.
-
----
-
-## Arrow delivery: three coupled constraints
-
-Arrow IPC is the right format for large, numeric, incrementally-updated tables;
-JSON rows are simpler and fine for small read-only ones. Delivering Arrow couples
-three things:
-
-- **Numeric type parsers, server-side.** pg returns `bigint` / `numeric` as
-  strings; coerce them to numbers before encoding, or `apache-arrow` infers
-  `Dictionary<Utf8>` instead of `Int` / `Float64`. pf_app: the type parsers in
-  `server.js` (oids 20 and 1700).
-- **One record batch.** Per-batch encoding builds independent dictionaries, and
-  the WASM crashes on dictionary-replacement messages. Accumulate all rows and emit
-  a single batch — pf_app does `tableToIPC(tableFromArrays(columns), 'stream')`.
-  The client's "stream" is therefore a chunked download of one batch; nothing
-  renders progressively.
-- **`apache-arrow` versus the client WASM.** The client is built against an older
-  arrow major than pf_app's server pins; the pair was verified working, but any
-  arrow bump has to be gated on the ingest smoke test below.
-
----
-
-## Expression columns are row-level: ratios need a weighted mean
-
-Perspective evaluates every expression against each **raw row**, then feeds the
-result into the column's aggregate; there is no post-aggregate expression stage.
-So `price: '"revenue" / "qty"'` under the default `sum` **sums the per-row
-ratios** — wrong at every grouped level, with or without `split_by`. `avg` gives
-the average of ratios, which is also wrong.
-
-Weight the ratio by its own denominator — `sum(price_i × qty_i) / sum(qty_i)` is
-exactly `sum(revenue) / sum(qty)`, at every level of both axes:
-
-```js
-expressions: { price: '"revenue" / "qty"' },
-aggregates:  { price: ['weighted mean', ['qty']] }   // note the NESTED array
-```
-
-**The nested array matters.** `Aggregate = string | [string, Array<string>]`, so
-the weight column goes in its own array. The flat `['weighted mean', 'qty']` is
-rejected with `data did not match any variant of untagged enum Aggregate`, which
-reads like the feature is missing. Being engine-side, it stays correct across
-incremental `table.update()`.
-
-- A **sum of a measure** (including `if(...)` subtotal expressions) → default
-  `sum`.
-- A **ratio, rate, price or per-unit figure** → must carry
-  `['weighted mean', ['<denominator>']]`.
-
-**Type inference can silently break expressions.** `if("Year" == '2026',
-"Amount", 0)` returns 0 on every row if `Year` was *inferred* as integer from
-numeric-looking strings, with no error, and neither literal form fixes it. Give
-period and year columns an explicit `string` type at table creation.
-
----
-
-## Theming
-
-The viewer theme is a `ViewerConfig` field: `viewer.restore({ theme: 'Pro Dark' })`.
-**`setAttribute('theme', …)` does nothing in 5.x** — the element observes no such
-attribute. Apply it on load and whenever the app toggle changes. `viewer.save()`
-includes `theme`, so strip it from stored layouts and ignore it when comparing
-configs — otherwise restoring a layout reasserts its author's theme.
-
----
-
-## Layouts: guard every restore against schema drift
-
-Before `viewer.restore(cfg)`, filter the saved `columns` / `group_by` /
-`split_by` / `sort` / `filter` against the columns the current table actually has
-(plus its `expressions`); a stale layout naming a dropped column otherwise throws.
-**`aggregates` needs the same guard** — once a layout carries a weighted mean, a
-dropped column aborts the whole restore (`Could not get dtype for column 'X'`).
-An aggregate names a target column and, in the multi-arg form, a weight column;
-validate both and drop the *entry*, never the layout, since `restore()` is
-all-or-nothing. pf_app's `cleanLayout()` does all of this; dataflow's does not yet
-guard `aggregates`.
-
----
-
-## Auto-pause deletes the view — turn it off for a static pivot
-
-`<perspective-viewer>` auto-pauses by default (an `IntersectionObserver` on itself
-plus the document's `visibilitychange`). Pausing is not a paint optimisation:
-`session.set_pause(true)` runs `view_sub.take().delete()`, so the view is
-destroyed, and becoming visible again rebuilds it with a full traversal — a
-multi-second stall per tab switch on a large table, and any per-node
-expand/collapse is lost with it.
-
-```js
-await viewer.load(table)
-try { if (viewer.setAutoPause) await viewer.setAutoPause(false) } catch {}
-```
-
-Leave it on only where a live stream feeds a table nobody needs kept current
-while away. Long-standing viewer behaviour, not a regression.
-
----
-
-## Upgrade smoke test
-
-Run on any change to a Perspective package or `apache-arrow` (for pf_app, after
-rebuilding the vendored tarballs per `ui/vendor/README.md`):
-
-1. All five packages (and `apache-arrow`) pinned exact and matching; lockfile
-   committed.
-2. `vite build` — no unresolved imports.
-3. Arrow apps: load a real dataset; `worker.table(buffer)` ingests without a
-   dictionary error, and a numeric column comes through as `Float64` / `Int`.
-4. Charts, where used: open a d3fc plugin and confirm it renders.
-5. Toggle dark/light; the viewer re-themes.
-6. Save a layout, reload, confirm it restores; drop a column and confirm the
-   cleaned restore does not throw.

install.sh

@@ -0,0 +1,65 @@
+#!/usr/bin/env bash
+set -e
+
+echo ""
+echo "========================================"
+echo "  Pivot Forecast — Install"
+echo "========================================"
+echo ""
+
+# ── DB connection ─────────────────────────────────────────────
+read -p "DB host     [192.168.1.110]: " DB_HOST
+DB_HOST=${DB_HOST:-192.168.1.110}
+
+read -p "DB port     [5432]: " DB_PORT
+DB_PORT=${DB_PORT:-5432}
+
+read -p "DB name     [ubm]: " DB_NAME
+DB_NAME=${DB_NAME:-ubm}
+
+read -p "DB user     [ptrowbridge]: " DB_USER
+DB_USER=${DB_USER:-ptrowbridge}
+
+read -s -p "DB password: " DB_PASSWORD
+echo ""
+
+read -p "App port    [3030]: " PORT
+PORT=${PORT:-3030}
+
+# ── Write .env ────────────────────────────────────────────────
+cat > .env <<EOF
+DB_HOST=${DB_HOST}
+DB_PORT=${DB_PORT}
+DB_NAME=${DB_NAME}
+DB_USER=${DB_USER}
+DB_PASSWORD=${DB_PASSWORD}
+PORT=${PORT}
+EOF
+echo "✓ .env written"
+
+# ── npm install ───────────────────────────────────────────────
+echo ""
+echo "Installing node dependencies..."
+npm install --silent
+echo "✓ dependencies installed"
+
+# ── schema install ────────────────────────────────────────────
+echo ""
+echo "Installing pf schema into ${DB_NAME}..."
+PGPASSWORD=${DB_PASSWORD} psql \
+    -h "${DB_HOST}" \
+    -p "${DB_PORT}" \
+    -U "${DB_USER}" \
+    -d "${DB_NAME}" \
+    -f setup_sql/01_schema.sql
+
+echo "✓ schema installed"
+
+# ── done ─────────────────────────────────────────────────────
+echo ""
+echo "========================================"
+echo "  Install complete"
+echo "  Start with:  npm run dev"
+echo "  Open:        http://$(hostname -I | awk '{print $1}'):${PORT}"
+echo "========================================"
+echo ""

lib/auth.js

@@ -1,92 +0,0 @@
-// Password hashing and the route guard.
-//
-// Hashes are scrypt, from node's own crypto — no native build step, and the
-// stored form carries its own parameters so they can be raised later without
-// invalidating existing rows:
-//
-//     scrypt$<N>$<r>$<p>$<salt base64>$<derived key base64>
-
-const crypto = require('crypto');
-
-const SCRYPT = { N: 16384, r: 8, p: 1, keylen: 64 };
-
-function hashPassword(password, params = SCRYPT) {
-    const { N, r, p, keylen } = params;
-    const salt = crypto.randomBytes(16);
-    const dk   = crypto.scryptSync(password, salt, keylen, { N, r, p, maxmem: 256 * 1024 * 1024 });
-    return `scrypt$${N}$${r}$${p}$${salt.toString('base64')}$${dk.toString('base64')}`;
-}
-
-// Constant-time compare. Returns false rather than throwing on a malformed or
-// legacy hash, so one bad row can't 500 the login route.
-function verifyPassword(password, stored) {
-    if (typeof stored !== 'string') return false;
-    const parts = stored.split('$');
-    if (parts.length !== 6 || parts[0] !== 'scrypt') return false;
-
-    const [, N, r, p, saltB64, dkB64] = parts;
-    try {
-        const salt     = Buffer.from(saltB64, 'base64');
-        const expected = Buffer.from(dkB64, 'base64');
-        const actual   = crypto.scryptSync(password, salt, expected.length, {
-            N: Number(N), r: Number(r), p: Number(p), maxmem: 256 * 1024 * 1024,
-        });
-        return crypto.timingSafeEqual(actual, expected);
-    } catch {
-        return false;
-    }
-}
-
-// Every /api route except the auth ones sits behind this.
-function requireAuth(req, res, next) {
-    if (req.session?.user?.username) return next();
-    res.status(401).json({ error: 'Not authenticated' });
-}
-
-// The identity used for pf.log.pf_user and the created_by/closed_by columns.
-// Read from the session only — never from the request body, which the browser
-// controls and which used to carry a hardcoded 'admin'.
-function sessionUser(req) {
-    return req.session?.user?.username || null;
-}
-
-// What this account may see and change, read from the session for the same
-// reason the username is: the browser must not be able to widen it.
-//
-// Returns { admin: true } for an account that sees everything, or
-// { admin: false, values: [...] } for a scoped one. An empty list is a real
-// answer meaning "nothing", not a missing one meaning "everything" -- an
-// account created without a territory sees no rows until it is granted some.
-function sessionTerritory(req) {
-    const u = req.session?.user;
-    if (!u) return { admin: false, values: [] };
-    if (u.is_admin) return { admin: true, values: null };
-    return { admin: false, values: Array.isArray(u.territory) ? u.territory : [] };
-}
-
-function requireAdmin(req, res, next) {
-    if (req.session?.user?.is_admin) return next();
-    res.status(403).json({ error: 'Administrator access required' });
-}
-
-module.exports = {
-    hashPassword, verifyPassword, requireAuth, requireAdmin,
-    sessionUser, sessionTerritory, SCRYPT,
-};
-
-// CLI: `node lib/auth.js hash` reads a password on stdin and prints its hash,
-// so ./pf.sh can create users without the plaintext touching argv or psql.
-if (require.main === module) {
-    if (process.argv[2] !== 'hash') {
-        console.error('usage: node lib/auth.js hash   (password on stdin)');
-        process.exit(2);
-    }
-    let input = '';
-    process.stdin.setEncoding('utf8');
-    process.stdin.on('data', chunk => { input += chunk; });
-    process.stdin.on('end', () => {
-        const password = input.replace(/\r?\n$/, '');
-        if (!password) { console.error('empty password'); process.exit(2); }
-        process.stdout.write(hashPassword(password) + '\n');
-    });
-}

lib/migrations.js

@@ -0,0 +1,158 @@
+// Migration bookkeeping shared by the runner (scripts/migrate.js) and the
+// startup check in server.js.
+//
+// Migrations are plain .sql files in setup_sql/migrations, applied in filename
+// order and recorded in pf.schema_version. They are immutable once applied: the
+// runner stores a checksum and refuses to proceed if a file has changed, since
+// editing an applied migration means databases silently disagree about what the
+// schema is.
+//
+// pf.dim_period is deliberately not a migration — it is a parameterised data
+// load (fiscal year start month), so it stays in setup_sql/gen_dim_period.sql.
+
+const fs   = require('fs');
+const path = require('path');
+const crypto = require('crypto');
+
+const MIGRATIONS_DIR = path.join(__dirname, '..', 'setup_sql', 'migrations');
+
+// Bookkeeping table has to exist before the first migration can be recorded, so
+// the runner creates it rather than a migration doing so.
+const BOOKKEEPING_DDL = `
+    CREATE SCHEMA IF NOT EXISTS pf;
+    CREATE TABLE IF NOT EXISTS pf.schema_version (
+        filename   text PRIMARY KEY,
+        checksum   text NOT NULL,
+        applied_at timestamptz NOT NULL DEFAULT now(),
+        applied_by text
+    );
+`;
+
+function checksum(sql) {
+    return crypto.createHash('sha256').update(sql).digest('hex').slice(0, 16);
+}
+
+// every .sql file on disk, in apply order
+function readMigrations() {
+    if (!fs.existsSync(MIGRATIONS_DIR)) return [];
+    return fs.readdirSync(MIGRATIONS_DIR)
+        .filter(f => f.endsWith('.sql'))
+        .sort()
+        .map(filename => {
+            const sql = fs.readFileSync(path.join(MIGRATIONS_DIR, filename), 'utf8');
+            return { filename, sql, checksum: checksum(sql) };
+        });
+}
+
+async function ensureBookkeeping(client) {
+    await client.query(BOOKKEEPING_DDL);
+}
+
+async function readApplied(client) {
+    const { rows } = await client.query(
+        `SELECT filename, checksum, applied_at FROM pf.schema_version ORDER BY filename`
+    );
+    return rows;
+}
+
+// Compare disk against the database. Returns pending migrations plus any
+// integrity problems worth refusing to run on.
+async function status(client) {
+    await ensureBookkeeping(client);
+    const onDisk  = readMigrations();
+    const applied = await readApplied(client);
+    const appliedBy = new Map(applied.map(r => [r.filename, r]));
+    const diskBy    = new Map(onDisk.map(m => [m.filename, m]));
+
+    const pending  = onDisk.filter(m => !appliedBy.has(m.filename));
+    // an applied file whose contents changed — the schema is now undefined
+    const modified = onDisk
+        .filter(m => appliedBy.has(m.filename) && appliedBy.get(m.filename).checksum !== m.checksum)
+        .map(m => m.filename);
+    // recorded as applied but no longer on disk — someone deleted or renamed it
+    const missing  = applied.filter(r => !diskBy.has(r.filename)).map(r => r.filename);
+    // a pending migration ordered before an applied one would apply out of sequence
+    const lastApplied = applied.length ? applied[applied.length - 1].filename : null;
+    const outOfOrder  = lastApplied ? pending.filter(m => m.filename < lastApplied).map(m => m.filename) : [];
+
+    return { onDisk, applied, pending, modified, missing, outOfOrder };
+}
+
+// Apply pending migrations, each in its own transaction so a failure leaves
+// earlier ones committed and the failing one fully rolled back.
+async function migrate(client, { user, log = console.log } = {}) {
+    const st = await status(client);
+    if (st.modified.length) {
+        throw new Error(
+            `Applied migrations have been modified: ${st.modified.join(', ')}. ` +
+            `Migrations are immutable — revert the edit and add a new migration instead.`
+        );
+    }
+    if (st.missing.length) {
+        throw new Error(
+            `Migrations recorded as applied are missing from disk: ${st.missing.join(', ')}.`
+        );
+    }
+    if (st.outOfOrder.length) {
+        throw new Error(
+            `Migrations would apply out of order: ${st.outOfOrder.join(', ')} sort before ` +
+            `already-applied migrations. Renumber them after the latest applied migration.`
+        );
+    }
+    if (!st.pending.length) {
+        log('Schema is up to date — no migrations to apply.');
+        return [];
+    }
+
+    const done = [];
+    for (const m of st.pending) {
+        log(`applying ${m.filename} …`);
+        try {
+            await client.query('BEGIN');
+            await client.query(m.sql);
+            await client.query(
+                `INSERT INTO pf.schema_version (filename, checksum, applied_by) VALUES ($1, $2, $3)`,
+                [m.filename, m.checksum, user || null]
+            );
+            await client.query('COMMIT');
+            done.push(m.filename);
+        } catch (err) {
+            await client.query('ROLLBACK').catch(() => {});
+            throw new Error(`${m.filename} failed: ${err.message}`);
+        }
+    }
+    log(`Applied ${done.length} migration(s).`);
+    return done;
+}
+
+// Record migrations as applied without running them — for a database whose schema
+// already matches, from before migrations existed.
+async function baseline(client, { user, upTo, log = console.log } = {}) {
+    const st = await status(client);
+    const target = upTo
+        ? st.pending.filter(m => m.filename <= upTo)
+        : st.pending;
+    if (!target.length) {
+        log('Nothing to baseline — no pending migrations.');
+        return [];
+    }
+    await client.query('BEGIN');
+    try {
+        for (const m of target) {
+            await client.query(
+                `INSERT INTO pf.schema_version (filename, checksum, applied_by)
+                 VALUES ($1, $2, $3)
+                 ON CONFLICT (filename) DO NOTHING`,
+                [m.filename, m.checksum, user ? `${user} (baseline)` : 'baseline']
+            );
+        }
+        await client.query('COMMIT');
+    } catch (err) {
+        await client.query('ROLLBACK').catch(() => {});
+        throw err;
+    }
+    log(`Marked ${target.length} migration(s) as applied without running them.`);
+    return target.map(m => m.filename);
+}
+
+module.exports = { MIGRATIONS_DIR, readMigrations, status, migrate, baseline, checksum };

lib/sql_generator.js

@@ -8,65 +8,9 @@
 // Tokens baked in at generation time:  column names, source schema.table
 // Tokens substituted at request time:  {{fc_table}}, {{where_clause}}, {{exclude_clause}},
 //                                      {{version_id}}, {{logid}}, {{pf_user}}, {{note}},
-//                                      {{label}}, {{bucket}}, {{tag}}, {{territory_clause}},
 //                                      {{params}}, {{slice}}, {{date_from}}, {{date_to}},
 //                                      {{value_incr}}, {{units_incr}}, {{set_clause}}, {{scale_factor}}
 
-// What the pivot shows for a row's segment and its bucket.
-//
-// Both are STORED on the row's pf.log entry -- l.label and l.bucket -- and read
-// back as plain columns. Every entry carries both: pf.log's log_named CHECK
-// refuses one without, so a report can join pf.log and select them with no rule
-// of its own.
-//
-// They used to be derived at read time, through a chain of fallbacks (a load's
-// label, else its tag, else its note, else the version's unlabeled_load; an
-// adjustment's label, else the version's adjustment_segment, else a constant).
-// That made the pivot the only reader that knew what bucket a row was in: the
-// same CASE was repeated in /agg, the /data cursor and the decorate proc in
-// /opt/forecast, and anything querying pf directly had to copy it again. It also
-// meant editing a version's fallback name silently renamed every past adjustment.
-//
-// The ordering prefix is part of the stored text, not computed here. Perspective
-// orders column groups by the value string, so "01 - Actual" is the only way an
-// arbitrary order can be expressed -- and l.label is where a person types it.
-//
-// ---------------------------------------------------------------------------
-// ADJUSTMENT DEFAULTS -- what a scale/recode/clone is stamped with.
-//
-// An adjustment has no label or bucket input, so the route stamps these onto its
-// log entry when it is written: the version's own adjustment_segment /
-// adjustment_bucket when set, else the constants below. Resolved once, at write
-// time -- changing a version's default afterwards names new adjustments, not old
-// ones. Exported so the operation routes stamp the rows they push back to the
-// pivot with the same values they wrote.
-//
-// A load has no default: its label is required, and its bucket is its label
-// unless one is given (the routes enforce both).
-// ---------------------------------------------------------------------------
-
-// An adjustment with no label of its own. The 99 keeps it after every numbered
-// segment -- ordering is string ordering, so this only works while the loads
-// carry 01-0n.
-const ADJUSTMENT_SEGMENT = '99 - Adjustments';
-
-// What an adjustment counts toward. Prefixed to match the segments it adjusts,
-// so the adjustments sit in the same column as the rows they adjust.
-const ADJUSTMENT_BUCKET = '04 - Forecast';
-
-const SEGMENT_EXPR = `l.label`;
-const BUCKET_EXPR  = `l.bucket`;
-
-const NOTE_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
-          THEN NULL
-          ELSE COALESCE(NULLIF(l.tag, ''), NULLIF(l.note, ''))
-     END`;
-
-// Every pf.log column the expressions above read, for /agg's GROUP BY: they
-// are functionally dependent on pf_logid, which is in the grain, but Postgres
-// will not infer that.
-const LABEL_GROUP_COLS = ['l.operation', 'l.label', 'l.tag', 'l.note', 'l.bucket'];
-
 // wrap a column name in double quotes for safe use in SQL
 function q(name) { return `"${name}"`; }
 
@@ -90,149 +34,50 @@ function grainOf(colMeta) {
     // parts and chr(30) stands in for NULL, so ('a', NULL) cannot collide with
     // (NULL, 'a') and a NULL stays distinct from an empty string — a collision
     // would silently merge two groups into one indexed row.
-    //
-    // md5 of that, rather than the concatenation itself, because the key is an
-    // opaque handle -- nothing reads it but table.update() and table.remove().
-    // The raw form averaged 233 chars on a 24-column grain and, being unique per
-    // row, defeated Arrow's dictionary encoding: 65.6 MB of a 109 MB payload,
-    // more than every other column combined. 128 bits keeps collisions unreachable.
-    const key = (pfx = '') => `md5(concat_ws(chr(31), ${[
+    const key = (pfx = '') => `concat_ws(chr(31), ${[
         ...cols.map(c => `COALESCE(${pfx}${q(c)}::text, chr(30))`),
         `${pfx}pf_iter`,
         `${pfx}pf_logid::text`
-    ].join(', ')}))`;
+    ].join(', ')})`;
 
     const groupCols = (pfx = '') => [...cols.map(c => `${pfx}${q(c)}`), `${pfx}pf_iter`, `${pfx}pf_logid`];
 
     return { cols, key, groupCols };
 }
 
-// Date columns that anchor a dim_group, each with the dimensions derived from
-// pf.dim_period for it. Shared so the generator and the routes agree on both the
-// membership and the join aliases -- the same reason grainOf is a single function.
-function dateGroupsOf(colMeta) {
-    return colMeta
-        .filter(c => c.role === 'date' && c.is_key && c.dim_group)
-        .map((keyCol, i) => ({
-            alias:   `dp${i + 1}`,
-            dateCol: keyCol.cname,
-            group:   keyCol.dim_group,
-            derived: colMeta
-                .filter(c => c.role === 'dimension'
-                          && c.dim_group === keyCol.dim_group
-                          && c.dim_period_col)
-                .map(c => ({ cname: c.cname, periodCol: c.dim_period_col })),
-        }))
-        .filter(g => g.derived.length > 0);
-}
-
-// cname -> which join it comes from and which of its columns
-function dimPeriodMapOf(dateGroups) {
-    return new Map(
-        dateGroups.flatMap(g => g.derived.map(d => [d.cname, { alias: g.alias, periodCol: d.periodCol }]))
-    );
-}
-
-// The joins themselves, against a date that has already had {{date_offset}}
-// applied. LEFT because a null date -- an order not yet shipped has no ship date
-// -- must leave the period columns empty rather than drop the row.
-function dimPeriodJoins(dateGroups, alias = 's') {
-    return dateGroups.map(g =>
-        `\n    LEFT JOIN pf.dim_period ${g.alias}`
-        + ` ON ${g.alias}.drange @> (${alias}."${g.dateCol}" + '{{date_offset}}'::interval)::date`
-    ).join('');
-}
-
 function generateSQL(source, colMeta) {
     const dims = colMeta
         .filter(c => c.role === 'dimension')
         .sort((a, b) => (a.opos || 0) - (b.opos || 0))
         .map(c => c.cname);
 
-    // Every column of each measure/date role, in col_meta order. Loads carry all of
-    // them; the adjustment operations are single-measure (scale distributes one
-    // {{value_incr}}) and use only the first of each, below.
-    const byRole = role => colMeta
-        .filter(c => c.role === role)
-        .sort((a, b) => (a.opos || 0) - (b.opos || 0))
-        .map(c => c.cname);
-
-    const valueCols = byRole('value');
-    const unitsCols = byRole('units');
-    const dateCols  = byRole('date');
-
-    const valueCol = valueCols[0];
-    const unitsCol = unitsCols[0];
+    const valueCol = colMeta.find(c => c.role === 'value')?.cname;
+    const unitsCol = colMeta.find(c => c.role === 'units')?.cname;
+    const dateCol  = colMeta.find(c => c.role === 'date')?.cname;
 
     if (!valueCol) throw new Error('No value column defined in col_meta');
-    if (!dateCols.length) throw new Error('No date column defined in col_meta');
+    if (!dateCol)  throw new Error('No date column defined in col_meta');
     if (dims.length === 0) throw new Error('No dimension columns defined in col_meta');
 
     const srcTable        = `"${source.schema}"."${source.tname}"`;
-    // Every date column, not just the primary one. Scale and recode used to
-    // carry dateCols[0] alone, so a source with order, requested and ship dates
-    // wrote adjustment rows whose rdate and sdate_e were *null* -- not stale,
-    // absent. Those columns are role 'date' and therefore sliceable, so a later
-    // adjustment scoped to a ship date matched no adjustment row ever written,
-    // and any reader that dated the forecast by ship date lost every
-    // adjustment silently. The derived period dimensions (rseas, smon_e) were
-    // fine throughout, being dimensions and copied with the rest -- which is
-    // what made it invisible: the labels were right and the dates underneath
-    // them were gone.
-    const dataCols        = [...dims, ...dateCols, valueCol, unitsCol].filter(Boolean);
+    const dataCols        = [...dims, dateCol, valueCol, unitsCol].filter(Boolean);
     const effectiveValue  = dataCols.includes(valueCol) ? valueCol : null;
     const effectiveUnits  = dataCols.includes(unitsCol) ? unitsCol : null;
     const insertCols      = [...dataCols.map(q), 'pf_iter', 'pf_logid', 'pf_user', 'pf_created_at'].join(', ');
     const selectData      = dataCols.map(q).join(', ');
     const dimsJoined      = dims.map(q).join(', ');
-    // The date columns as a select list, in the same order dataCols lists them,
-    // so it lines up with insertCols positionally.
-    const datesJoined     = dateCols.map(q).join(', ');
 
-    // Baseline and reference copy the source row wholesale, so they carry every
-    // measure and every date — not just the primary one the operations act on.
-    // Dropping the others would leave those columns null for the life of the version.
-    // Clone carries every date column, not just the primary one, because it is the
-    // operation that moves rows through time: {{date_offset}} shifts them all
-    // together, and the period dimensions are re-derived from pf.dim_period against
-    // the shifted dates rather than copied from the row being cloned. Cloning last
-    // year's mix forward a year otherwise produces rows dated 2027 still labelled
-    // with 2026's periods.
-    const cloneCols = [...dims, ...dateCols, effectiveValue, effectiveUnits].filter(Boolean);
-    const cloneInsertCols = [...cloneCols.map(q), 'pf_iter', 'pf_logid', 'pf_user', 'pf_created_at'].join(', ');
-
-    // An adjustment writes one row per *coordinate* it touches, not one per row
-    // it reads.
-    //
-    // Reading rows one-for-one meant every operation inherited the row count of
-    // everything before it: the baseline's rows plus every prior adjustment's
-    // rows at the same coordinate, so the table grew super-linearly with how
-    // much work had been done on it. Eight Pull Forward entries and the next
-    // scale over the same slice writes nine times what it needs to.
-    //
-    // Collapsing is over pf_logid and pf_iter only -- every stored dimension and
-    // date stays in the GROUP BY -- so no column goes null and nothing becomes
-    // unsliceable later. The distribution maths is untouched either way, since
-    // the window sums see the same totals whether or not the rows underneath
-    // them have been added up first.
-    const groupCols       = (cols) => cols.map(q).join(',\n        ');
-
-    const loadCols        = [...dims, ...dateCols, ...valueCols, ...unitsCols];
-    const loadInsertCols  = [...loadCols.map(q), 'pf_iter', 'pf_logid', 'pf_user', 'pf_created_at'].join(', ');
-    const dateColSet      = new Set(dateCols);
-
-    // dim_period JOIN support: a date column that is the is_key of a dim_group
-    // anchors that group, and dimension siblings with dim_period_col set are
-    // derived from pf.dim_period instead of copied raw. Derivation is against the
-    // date *after* {{date_offset}}, which is the whole point -- shift a baseline
-    // forward a year and its period columns follow, rather than still naming the
-    // year it came from.
-    //
-    // Every such group, not just the first. This used to be a find(), so a source
-    // with order, requested and ship date groups derived the order one and copied
-    // the other two raw -- shifted dates against unshifted period labels.
-    const dateGroups   = dateGroupsOf(colMeta);
-    const dimPeriodMap = dimPeriodMapOf(dateGroups);
+    // dim_period JOIN support: if the date column is the is_key of a dim_group,
+    // dimension siblings with dim_period_col set are derived from pf.dim_period
+    // instead of being copied raw from the source on baseline/reference load.
+    const dateKeyGroup = colMeta.find(c => c.role === 'date' && c.is_key && c.dim_group)?.dim_group;
+    const dimPeriodMap = new Map(
+        dateKeyGroup
+            ? colMeta
+                .filter(c => c.role === 'dimension' && c.dim_group === dateKeyGroup && c.dim_period_col)
+                .map(c => [c.cname, c.dim_period_col])
+            : []
+    );
     const hasDimPeriod = dimPeriodMap.size > 0;
 
     // display grain — when set, initial load and operations both return rows
@@ -272,23 +117,12 @@ function generateSQL(source, colMeta) {
     // table indexed on pf_gkey and its view sums across these rows, exactly as an
     // Excel pivot cache sums its data tab.
     function buildGetAgg() {
-        // pf_logid is part of the grain, so joining pf.log adds no rows — each group
-        // already belongs to exactly one log entry. Without this the segment labels
-        // that /data surfaces would vanish the moment a source declares a grain.
         return `
 SELECT
-    ${grainSelect('t.')}
-    ,${SEGMENT_EXPR} AS pf_segment
-    ,${BUCKET_EXPR} AS pf_bucket
-    ,${NOTE_EXPR} AS pf_note
-    ,l.operation AS pf_op
-FROM {{fc_table}} t
-LEFT JOIN pf.log l
-    ON l.id = t.pf_logid
-WHERE {{territory_clause}}
+    ${grainSelect()}
+FROM {{fc_table}}
 GROUP BY
-    ${grain.groupCols('t.').join('\n    ,')}
-    ,${LABEL_GROUP_COLS.join('\n    ,')}`.trim();
+    ${grain.groupCols().join('\n    ,')}`.trim();
     }
 
     // grain columns + pf_gkey + summed measures, in the leading-comma style the
@@ -317,33 +151,29 @@ GROUP BY
 
     function buildLoadSelect(pfx) {
         // pfx: table alias prefix ('s.' when joining dim_period, '' otherwise)
-        // The offset shifts every date column, so order date and ship date stay in step.
-        return loadCols.map(c => {
-            if (dateColSet.has(c))      return `(${pfx}${q(c)} + '{{date_offset}}'::interval)::date`;
-            if (dimPeriodMap.has(c)) {
-                const { alias, periodCol } = dimPeriodMap.get(c);
-                return `${alias}.${q(periodCol)} AS ${q(c)}`;
-            }
+        return dataCols.map(c => {
+            if (c === dateCol)          return `(${pfx}${q(c)} + '{{date_offset}}'::interval)::date`;
+            if (dimPeriodMap.has(c))    return `dp.${q(dimPeriodMap.get(c))} AS ${q(c)}`;
             return `${pfx}${q(c)}`;
         }).join(',\n        ');
     }
 
     function buildFromClause() {
         if (!hasDimPeriod) return srcTable;
-        return srcTable + ' s' + dimPeriodJoins(dateGroups);
+        return `${srcTable} s\n    JOIN pf.dim_period dp`
+            + ` ON dp.drange @> (s.${q(dateCol)} + '{{date_offset}}'::interval)::date`;
     }
 
     function buildBaseline() {
         return `
 WITH
 ilog AS (
-    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note, label, bucket, tag)
-    VALUES ({{version_id}}, '{{pf_user}}', 'baseline', NULL, '{{params}}'::jsonb, '{{note}}',
-            NULLIF('{{label}}', ''), NULLIF('{{bucket}}', ''), NULLIF('{{tag}}', ''))
+    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note)
+    VALUES ({{version_id}}, '{{pf_user}}', 'baseline', NULL, '{{params}}'::jsonb, '{{note}}')
     RETURNING id
 )
 ,ins AS (
-    INSERT INTO {{fc_table}} (${loadInsertCols})
+    INSERT INTO {{fc_table}} (${insertCols})
     SELECT
         ${buildLoadSelect(hasDimPeriod ? 's.' : '')},
         'baseline', (SELECT id FROM ilog), '{{pf_user}}', now()
@@ -351,20 +181,19 @@ ilog AS (
     WHERE {{filter_clause}}
     RETURNING *
 )
-SELECT count(*) AS rows_affected, (SELECT id FROM ilog) AS log_id FROM ins`.trim();
+SELECT count(*) AS rows_affected FROM ins`.trim();
     }
 
     function buildReference() {
         return `
 WITH
 ilog AS (
-    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note, label, bucket, tag)
-    VALUES ({{version_id}}, '{{pf_user}}', 'reference', NULL, '{{params}}'::jsonb, '{{note}}',
-            NULLIF('{{label}}', ''), NULLIF('{{bucket}}', ''), NULLIF('{{tag}}', ''))
+    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note)
+    VALUES ({{version_id}}, '{{pf_user}}', 'reference', NULL, '{{params}}'::jsonb, '{{note}}')
     RETURNING id
 )
 ,ins AS (
-    INSERT INTO {{fc_table}} (${loadInsertCols})
+    INSERT INTO {{fc_table}} (${insertCols})
     SELECT
         ${buildLoadSelect(hasDimPeriod ? 's.' : '')},
         'reference', (SELECT id FROM ilog), '{{pf_user}}', now()
@@ -372,7 +201,7 @@ ilog AS (
     WHERE {{filter_clause}}
     RETURNING *
 )
-SELECT count(*) AS rows_affected, (SELECT id FROM ilog) AS log_id FROM ins`.trim();
+SELECT count(*) AS rows_affected FROM ins`.trim();
     }
 
     function buildScale() {
@@ -382,23 +211,19 @@ SELECT count(*) AS rows_affected, (SELECT id FROM ilog) AS log_id FROM ins`.trim
         const uSel = effectiveUnits
             ? `round((${q(effectiveUnits)} / NULLIF(total_units, 0)) * {{units_incr}}, 5)`
             : `0`;
-        // sum(sum(x)) OVER () is the aggregate of the aggregates: the window runs
-        // after the GROUP BY, so the total is over collapsed coordinates and
-        // comes to the same figure the ungrouped window produced.
         const baseSelectParts = [
             ...dimsJoined ? [dimsJoined] : [],
-            datesJoined,
-            effectiveValue  ? `sum(${q(effectiveValue)}) AS ${q(effectiveValue)}` : null,
-            effectiveUnits  ? `sum(${q(effectiveUnits)}) AS ${q(effectiveUnits)}` : null,
-            effectiveValue  ? `sum(sum(${q(effectiveValue)})) OVER () AS total_value` : null,
-            effectiveUnits  ? `sum(sum(${q(effectiveUnits)})) OVER () AS total_units` : null
+            q(dateCol),
+            effectiveValue  ? q(effectiveValue)  : null,
+            effectiveUnits  ? q(effectiveUnits)  : null,
+            effectiveValue  ? `sum(${q(effectiveValue)}) OVER () AS total_value`  : null,
+            effectiveUnits  ? `sum(${q(effectiveUnits)}) OVER () AS total_units` : null
         ].filter(Boolean).join(',\n        ');
         return `
 WITH
 ilog AS (
-    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note, label, bucket)
-    VALUES ({{version_id}}, '{{pf_user}}', 'scale', '{{slice}}'::jsonb, '{{params}}'::jsonb, '{{note}}',
-            NULLIF('{{label}}', ''), NULLIF('{{bucket}}', ''))
+    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note)
+    VALUES ({{version_id}}, '{{pf_user}}', 'scale', '{{slice}}'::jsonb, '{{params}}'::jsonb, '{{note}}')
     RETURNING id
 )
 ,base AS (
@@ -407,13 +232,11 @@ ilog AS (
     FROM {{fc_table}}
     WHERE {{where_clause}}
     {{exclude_clause}}
-    GROUP BY
-        ${groupCols([...dims, ...dateCols])}
 )
 ,ins AS (
     INSERT INTO {{fc_table}} (${insertCols})
     SELECT
-        ${[dimsJoined, datesJoined, ...(effectiveValue ? [vSel] : []), ...(effectiveUnits ? [uSel] : [])].join(',\n        ')},
+        ${[dimsJoined, q(dateCol), ...(effectiveValue ? [vSel] : []), ...(effectiveUnits ? [uSel] : [])].join(',\n        ')},
         'scale', (SELECT id FROM ilog), '{{pf_user}}', now()
     FROM base
     RETURNING *
@@ -425,41 +248,28 @@ ${opTail('ins')}`.trim();
         return `
 WITH
 ilog AS (
-    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note, label, bucket)
-    VALUES ({{version_id}}, '{{pf_user}}', 'recode', '{{slice}}'::jsonb, '{{params}}'::jsonb, '{{note}}',
-            NULLIF('{{label}}', ''), NULLIF('{{bucket}}', ''))
+    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note)
+    VALUES ({{version_id}}, '{{pf_user}}', 'recode', '{{slice}}'::jsonb, '{{params}}'::jsonb, '{{note}}')
     RETURNING id
 )
 ,src AS (
-    SELECT
-        ${dimsJoined},
-        ${datesJoined}${effectiveValue ? `,\n        sum(${q(effectiveValue)}) AS ${q(effectiveValue)}` : ''}${effectiveUnits ? `,\n        sum(${q(effectiveUnits)}) AS ${q(effectiveUnits)}` : ''}
+    SELECT ${selectData}
     FROM {{fc_table}}
     WHERE {{where_clause}}
     {{exclude_clause}}
-    GROUP BY
-        ${groupCols([...dims, ...dateCols])}
-    -- a coordinate whose earlier adjustments already net it to zero has nothing to
-    -- move; writing it anyway is two empty rows per coordinate. Rare on a first
-    -- recode, but a second recode over cells a first one emptied would rewrite
-    -- every one of them
-    HAVING ${[effectiveValue, effectiveUnits].filter(Boolean).map(c => `sum(${q(c)}) <> 0`).join(' OR ')}
 )
 ,neg AS (
     INSERT INTO {{fc_table}} (${insertCols})
-    SELECT ${dimsJoined}, ${datesJoined}, ${effectiveValue ? `-${q(effectiveValue)}` : '0'}${effectiveUnits ? `, -${q(effectiveUnits)}` : ''},
+    SELECT ${dimsJoined}, ${q(dateCol)}, ${effectiveValue ? `-${q(effectiveValue)}` : '0'}${effectiveUnits ? `, -${q(effectiveUnits)}` : ''},
            'recode', (SELECT id FROM ilog), '{{pf_user}}', now()
     FROM src
     RETURNING *
 )
--- the put-back leg, moved by {{date_offset}} ('0 days' for a plain recode). The
--- route resolves {{set_clause}} against alias s, deriving the period dimensions
--- from the dim_period joins below only when the offset is non-zero.
 ,ins AS (
     INSERT INTO {{fc_table}} (${insertCols})
-    SELECT {{set_clause}}, ${dateCols.map(c => `(s.${q(c)} + '{{date_offset}}'::interval)::date`).join(', ')}, ${effectiveValue ? `s.${q(effectiveValue)}` : '0'}${effectiveUnits ? `, s.${q(effectiveUnits)}` : ''},
+    SELECT {{set_clause}}, ${q(dateCol)}, ${effectiveValue ? q(effectiveValue) : '0'}${effectiveUnits ? `, ${q(effectiveUnits)}` : ''},
            'recode', (SELECT id FROM ilog), '{{pf_user}}', now()
-    FROM src s${hasDimPeriod ? dimPeriodJoins(dateGroups) : ''}
+    FROM src
     RETURNING *
 )
 ${grain ? `,allrows AS (
@@ -471,38 +281,23 @@ ${opTail('allrows')}` : 'SELECT * FROM neg UNION ALL SELECT * FROM ins'}`.trim()
     }
 
     function buildClone() {
-        const select = [
-            // dims: whatever {{set_clause}} resolves them to. The route builds it,
-            // and substitutes the dim_period expression for any derived dimension
-            // the caller has not overridden outright.
-            '{{set_clause}}',
-            ...dateCols.map(c => `(s.${q(c)} + '{{date_offset}}'::interval)::date`),
-            effectiveValue ? `round(s.${q(effectiveValue)} * {{scale_factor}}, 2)` : null,
-            effectiveUnits ? `round(s.${q(effectiveUnits)} * {{scale_factor}}, 5)` : null,
-        ].filter(Boolean).join(',\n        ');
-
         return `
 WITH
 ilog AS (
-    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note, label, bucket)
-    VALUES ({{version_id}}, '{{pf_user}}', 'clone', '{{slice}}'::jsonb, '{{params}}'::jsonb, '{{note}}',
-            NULLIF('{{label}}', ''), NULLIF('{{bucket}}', ''))
+    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note)
+    VALUES ({{version_id}}, '{{pf_user}}', 'clone', '{{slice}}'::jsonb, '{{params}}'::jsonb, '{{note}}')
     RETURNING id
 )
 ,ins AS (
-    INSERT INTO {{fc_table}} (${cloneInsertCols})
+    INSERT INTO {{fc_table}} (${insertCols})
     SELECT
-        ${select},
+        {{set_clause}},
+        ${q(dateCol)},
+        ${effectiveValue ? `round(${q(effectiveValue)} * {{scale_factor}}, 2)` : '0'}${effectiveUnits ? `,\n        round(${q(effectiveUnits)} * {{scale_factor}}, 5)` : ''},
         'clone', (SELECT id FROM ilog), '{{pf_user}}', now()
-    FROM (
-        SELECT
-            ${groupCols([...dims, ...dateCols])}${effectiveValue ? `,\n            sum(${q(effectiveValue)}) AS ${q(effectiveValue)}` : ''}${effectiveUnits ? `,\n            sum(${q(effectiveUnits)}) AS ${q(effectiveUnits)}` : ''}
-        FROM {{fc_table}}
-        WHERE {{where_clause}}
-        {{exclude_clause}}
-        GROUP BY
-            ${groupCols([...dims, ...dateCols])}
-    ) s${hasDimPeriod ? dimPeriodJoins(dateGroups) : ''}
+    FROM {{fc_table}}
+    WHERE {{where_clause}}
+    {{exclude_clause}}
     RETURNING *
 )
 ${opTail('ins')}`.trim();
@@ -532,98 +327,17 @@ function applyTokens(sql, tokens) {
 
 // build a SQL WHERE clause string from a slice object
 // only dimension columns are included; unrecognised keys are silently skipped
-// pf_segment and pf_bucket are not columns on the forecast table -- they are
-// stored on the row's pf.log entry -- so a slice naming one cannot be compared
-// directly. It resolves to a set of log ids instead, which is
-// exact: the name lives on the log row, and every forecast row carries the
-// pf_logid that points at it.
-//
-// Without this they were dropped from the slice, and clicking a single bucket's
-// cell scaled every bucket at that dimension intersection while the panel showed
-// only the one clicked.
-//
-// pf_note and pf_op are the same kind of column -- the pivot shows them, from
-// the log entry -- and used to be refused, which the client pre-empted by
-// stripping them. The ledger did not strip them, so a pivot grouped by one
-// totalled a narrower set of rows than the operation then wrote.
-const COMPUTED_SLICE_COLS = {
-    pf_segment: SEGMENT_EXPR,
-    pf_bucket:  BUCKET_EXPR,
-    pf_note:    NOTE_EXPR,
-    pf_op:      'l.operation',
-};
-
-// pf_ columns that are real columns of the forecast table, compared directly.
-// pf_iter was stripped by the client and skipped here, on the reasoning that
-// two cells differing only by iter band are the same slice. They are not to the
-// ledger, which filtered on it: with the default layout grouped by pf_iter, a
-// click on a customer's `scale` row showed the scale rows' total and wrote to
-// every row the customer has.
-const TABLE_SLICE_COLS = new Set(['pf_iter', 'pf_user', 'pf_logid']);
-
-function computedSlicePredicate(col, val, versionId) {
-    if (versionId == null) {
-        const err = new Error(`Cannot filter on ${col} without a version`);
-        err.status = 500;
-        throw err;
-    }
-    // a blank is a value here too -- a load has no note -- and `IN (NULL)` is
-    // never true, so the null arm is ORed on as buildWhere does
-    const list   = Array.isArray(val) ? val : [val];
-    const nulls  = list.some(v => v === null);
-    const vals   = list.filter(v => v !== null).map(v => `'${esc(v)}'`);
-    const expr   = COMPUTED_SLICE_COLS[col];
-    const inList = vals.length ? `(${expr}) IN (${vals.join(', ')})` : null;
-    const test   = nulls && inList ? `((${expr}) IS NULL OR ${inList})`
-                 : nulls           ? `(${expr}) IS NULL`
-                 :                   inList;
-    return `pf_logid IN (
-    SELECT l.id
-    FROM pf.log l
-    WHERE l.version_id = ${parseInt(versionId)}
-      AND ${test}
-)`;
-}
-
-function buildWhere(slice, dimCols, versionId) {
+function buildWhere(slice, dimCols) {
     if (!slice || Object.keys(slice).length === 0) return 'TRUE';
 
     const allowed = new Set(dimCols);
     const parts = [];
 
     for (const [col, val] of Object.entries(slice)) {
-        if (COMPUTED_SLICE_COLS[col]) {
-            parts.push(computedSlicePredicate(col, val, versionId));
-            continue;
-        }
-        // A pf_ key this does not understand is refused rather than skipped.
-        // Skipping is how a selection silently widened: the operation ran against
-        // everything the dropped key would have excluded.
-        if (col.startsWith('pf_') && !TABLE_SLICE_COLS.has(col)) {
-            const err = new Error(`Slice names ${col}, which cannot be filtered on`);
-            err.status = 400;
-            throw err;
-        }
-        if (!allowed.has(col) && !TABLE_SLICE_COLS.has(col)) continue;
-        // A blank dimension value is a value. Both blanks exist in this data and
-        // they are different rows -- on fc_osm_skinny_34, partgroup is null on
-        // 51,964 and '' on 111,696 -- so a null has to become IS NULL. It used
-        // to fall through esc() to `= ''`, which selected the *other* blank:
-        // the slice named one node and the operation wrote the other.
-        if (val === null) {
-            parts.push(`"${col}" IS NULL`);
-        } else if (Array.isArray(val)) {
-            const nulls   = val.some(v => v === null);
-            const escaped = val.filter(v => v !== null).map(v => esc(v));
-            const inList  = escaped.length ? `"${col}" IN ('${escaped.join("', '")}')` : null;
-            // an IN list cannot carry a null -- `IN (NULL)` is never true -- so
-            // the null arm is ORed on rather than folded in
-            parts.push(
-                nulls && inList ? `("${col}" IS NULL OR ${inList})`
-              : nulls           ? `"${col}" IS NULL`
-              : inList          ? inList
-              : 'FALSE'
-            );
+        if (!allowed.has(col)) continue;
+        if (Array.isArray(val)) {
+            const escaped = val.map(v => esc(v));
+            parts.push(`"${col}" IN ('${escaped.join("', '")}')`);
         } else {
             parts.push(`"${col}" = '${esc(val)}'`);
         }
@@ -632,153 +346,6 @@ function buildWhere(slice, dimCols, versionId) {
     return parts.length ? parts.join('\nAND ') : 'TRUE';
 }
 
-// The pivot's own filter, carried alongside the slices so an operation writes
-// exactly the rows the ledger counted.
-//
-// A slice is {col: value} and can only ever mean equality, so a view filtered to
-// sseas_e <= 2027 could not be expressed as one. Refusing it was safe but
-// useless -- a bounded season is an ordinary way to scope a forecast -- so the
-// operators travel as [col, op, value] triples instead.
-//
-// Perspective's operator names, not SQL's, since that is where these come from.
-// Anything outside this list is refused rather than ignored: a scope silently
-// dropped is a write that is wider than the panel that authorised it.
-//
-// The substring operators are the ones a text dimension is actually filtered
-// with -- `partd contains 'Keysto'` picks a product family out of a description
-// nobody would enumerate by hand -- and they are **case-insensitive** in the
-// engine: `t_tscalar::contains` lowercases both sides before
-// `std::string::find` (scalar.cpp), as do `begins_with` and `ends_with`. So the
-// SQL is ILIKE, not LIKE. LIKE would be *narrower* than the grid that
-// authorised the write, which is the failure this whole mechanism exists to
-// prevent, and narrower is no safer than wider: it writes a set the ledger did
-// not count either way.
-//
-// Their term is matched literally there, so LIKE's own metacharacters are
-// escaped out of it by `likeTerm` -- a filter on `50%` means those three
-// characters, not "50 followed by anything".
-//
-// Not identical in one corner: the engine folds case byte by byte, so ASCII
-// only, where ILIKE follows the database collation. They differ only on
-// non-ASCII letters.
-//
-// `matches` / `not matches` are deliberately still refused. They are RE2 in the
-// engine and Postgres regexes are not RE2 -- `\b` is a word boundary in one and
-// a backspace in the other -- so a pattern valid in both could quietly match a
-// different set of rows, which is worse than answering 400.
-const likeTerm = (val, pre, post) =>
-    `'${esc(pre + String(val).replace(/[\\%_]/g, m => `\\${m}`) + post)}'`;
-
-const SCOPE_OPS = {
-    '==':                (c, v) => `${c} = ${v[0]}`,
-    '!=':                (c, v) => `${c} != ${v[0]}`,
-    '>':                 (c, v) => `${c} > ${v[0]}`,
-    '>=':                (c, v) => `${c} >= ${v[0]}`,
-    '<':                 (c, v) => `${c} < ${v[0]}`,
-    '<=':                (c, v) => `${c} <= ${v[0]}`,
-    'in':                (c, v) => `${c} IN (${v.join(', ')})`,
-    'not in':            (c, v) => `${c} NOT IN (${v.join(', ')})`,
-    'is null':           (c)    => `${c} IS NULL`,
-    'is not null':       (c)    => `${c} IS NOT NULL`,
-    'contains':          (c, v, raw) => `${c} ILIKE ${likeTerm(raw[0], '%', '%')}`,
-    'not contains':      (c, v, raw) => `${c} NOT ILIKE ${likeTerm(raw[0], '%', '%')}`,
-    'begins with':       (c, v, raw) => `${c} ILIKE ${likeTerm(raw[0], '', '%')}`,
-    'not begins with':   (c, v, raw) => `${c} NOT ILIKE ${likeTerm(raw[0], '', '%')}`,
-    'ends with':         (c, v, raw) => `${c} ILIKE ${likeTerm(raw[0], '%', '')}`,
-    'not ends with':     (c, v, raw) => `${c} NOT ILIKE ${likeTerm(raw[0], '%', '')}`,
-};
-
-// the operators that need a term, so an empty one is refused rather than
-// becoming ILIKE '%%' -- which is every non-null row, i.e. no scope at all
-const SCOPE_OPS_NEED_TERM = new Set(Object.keys(SCOPE_OPS).filter(op => !op.startsWith('is ')));
-
-function buildScopeClause(scope, dimCols, versionId) {
-    if (!Array.isArray(scope) || scope.length === 0) return '';
-    const allowed = new Set(dimCols);
-    const parts = scope.map((entry) => {
-        if (!Array.isArray(entry) || entry.length < 2) {
-            const err = new Error(`Malformed scope entry ${JSON.stringify(entry)}`);
-            err.status = 400; throw err;
-        }
-        const [col, op, ...rest] = entry;
-        const vals = (Array.isArray(rest[0]) ? rest[0] : rest).filter(v => v !== undefined);
-
-        if (COMPUTED_SLICE_COLS[col]) {
-            if (op !== '==' && op !== 'in') {
-                const err = new Error(`${col} can only be scoped with == or in, not ${op}`);
-                err.status = 400; throw err;
-            }
-            return computedSlicePredicate(col, vals, versionId);
-        }
-        if (!allowed.has(col)) {
-            const err = new Error(`Column "${col}" is not available for filtering`);
-            err.status = 400; throw err;
-        }
-        const fn = SCOPE_OPS[op];
-        if (!fn) {
-            const err = new Error(`Unsupported filter operator "${op}" on ${col}`);
-            err.status = 400; throw err;
-        }
-        if (SCOPE_OPS_NEED_TERM.has(op) && (vals.length === 0 || vals[0] === '' || vals[0] === null)) {
-            const err = new Error(`Filter "${col} ${op}" has no value`);
-            err.status = 400; throw err;
-        }
-        return fn(`"${col}"`, vals.map(v => `'${esc(String(v))}'`), vals);
-    });
-    return parts.join('\nAND ');
-}
-
-// The territory predicate: what this account may see and change.
-//
-// Server-side by construction. Today's `scope` is sent by the browser, which is
-// right for a filter the user chose and would be fatal for a permission -- so
-// this one is built from the session and ANDed on last, where nothing in the
-// request can remove it.
-//
-// FALSE, not TRUE, for an account with no territory. The whole point is that a
-// missing grant means no rows: an empty list that fell through to TRUE would
-// hand the entire book to the first account somebody forgot to configure.
-function buildTerritoryClause(territory, territoryCol, alias = '') {
-    if (!territory || territory.admin) return '';
-    if (!territoryCol) return 'FALSE';
-    const vals = (territory.values || []).filter(v => v != null && v !== '');
-    if (vals.length === 0) return 'FALSE';
-    const pfx = alias ? `${alias}.` : '';
-    return `${pfx}"${territoryCol}" IN (${vals.map(v => `'${esc(String(v))}'`).join(', ')})`;
-}
-
-// build a WHERE clause spanning several slices — an OR of AND-groups.
-// A union of slices cannot be flattened into one IN list per column: slices
-// {Region:East, State:NY} and {Region:West, State:CA} would become
-// Region IN (East,West) AND State IN (NY,CA), which also matches East/CA.
-function buildWhereAny(slices, dimCols, versionId) {
-    const list = (slices || []).filter(Boolean);
-    // An empty slice is a total cell, and the union with a total is the total.
-    // Dropping it instead would narrow the write to the other cells. The caller
-    // only lets one through under a scope, which is ANDed on afterwards.
-    if (list.length === 0 || list.some(s => Object.keys(s).length === 0)) return 'TRUE';
-    if (list.length === 1) return buildWhere(list[0], dimCols, versionId);
-
-    const groups = list
-        .map(s => buildWhere(s, dimCols, versionId))
-        .filter(w => w !== 'TRUE');
-
-    // any slice that reduced to TRUE selects everything, so the union does too
-    if (groups.length !== list.length) return 'TRUE';
-
-    // outer parens matter: the caller appends `AND pf_iter NOT IN (...)`,
-    // and AND binds tighter than OR
-    return `(${groups.map(g => `(${g.replace(/\n/g, ' ')})`).join('\n     OR ')})`;
-}
-
-// the bare predicate for "this row participates in operations", for use in a
-// FILTER clause where the excluded rows still need to be counted separately
-function buildExcludePredicate(excludeIters) {
-    if (!excludeIters || excludeIters.length === 0) return 'TRUE';
-    const list = excludeIters.map(i => `'${esc(i)}'`).join(', ');
-    return `pf_iter NOT IN (${list})`;
-}
-
 // build AND iter NOT IN (...) from a version's exclude_iters array
 function buildExcludeClause(excludeIters) {
     if (!excludeIters || excludeIters.length === 0) return '';
@@ -788,21 +355,12 @@ function buildExcludeClause(excludeIters) {
 
 // build the dimension columns portion of a SELECT for recode/clone
 // replaces named dimensions with literal values, passes others through unchanged
-// derivedExprs: cname -> a SQL expression to use when the caller has not set the
-// column outright. Clone passes the dim_period expressions through here, so a
-// cloned row's period dimensions come from the calendar against its shifted date
-// rather than from the row it was copied from.
-function buildSetClause(dimCols, setObj, opts = {}) {
-    const { derivedExprs, alias } = opts;
-    const pfx = alias ? `${alias}.` : '';
+function buildSetClause(dimCols, setObj) {
     return dimCols.map(col => {
         if (setObj && setObj[col] !== undefined) {
             return `'${esc(setObj[col])}' AS "${col}"`;
         }
-        if (derivedExprs && derivedExprs[col]) {
-            return `${derivedExprs[col]} AS "${col}"`;
-        }
-        return `${pfx}"${col}"`;
+        return `"${col}"`;
     }).join(', ');
 }
 
@@ -846,6 +404,4 @@ function esc(val) {
     return String(val).replace(/'/g, "''");
 }
 
-module.exports = { generateSQL, grainOf, COMPUTED_SLICE_COLS, TABLE_SLICE_COLS, buildScopeClause, buildTerritoryClause,
-    SEGMENT_EXPR, BUCKET_EXPR, NOTE_EXPR, LABEL_GROUP_COLS,
-    ADJUSTMENT_SEGMENT, ADJUSTMENT_BUCKET, dateGroupsOf, dimPeriodMapOf, dimPeriodJoins, applyTokens, buildWhere, buildWhereAny, buildExcludeClause, buildExcludePredicate, buildSetClause, buildFilterClause, esc };
+module.exports = { generateSQL, grainOf, applyTokens, buildWhere, buildExcludeClause, buildSetClause, buildFilterClause, esc };

lib/utils.js

@@ -3,41 +3,7 @@ function fcTable(tname, versionId) {
     return `pf.fc_${tname}_${versionId}`;
 }
 
-// The columns of a relation, from pg_catalog rather than information_schema.
-//
-// information_schema.columns omits materialized views -- they are not in the
-// SQL standard -- so a source built on one looked like it had no columns at
-// all: registering it seeded nothing, and creating a version failed with "No
-// usable columns in col_meta" while col_meta plainly had thirty-six.
-//
-// The shape matches what information_schema returned, so mapType and the
-// callers did not have to change. data_type is format_type with the modifier
-// stripped, which gives the same spelling information_schema uses ('character
-// varying', 'numeric'), and the numeric precision and scale are unpacked from
-// atttypmod the way information_schema does internally.
-//
-// Takes $1 = schema, $2 = relation name.
-const RELATION_COLUMNS_SQL = `
-    SELECT a.attname                                              AS column_name
-          ,regexp_replace(format_type(a.atttypid, a.atttypmod), '\\(.*\\)$', '') AS data_type
-          ,a.attnum                                               AS ordinal_position
-          ,CASE WHEN a.attnotnull THEN 'NO' ELSE 'YES' END        AS is_nullable
-          ,CASE WHEN a.atttypid = 'numeric'::regtype AND a.atttypmod > 4
-                THEN ((a.atttypmod - 4) >> 16) & 65535 END        AS numeric_precision
-          ,CASE WHEN a.atttypid = 'numeric'::regtype AND a.atttypmod > 4
-                THEN (a.atttypmod - 4) & 65535 END                AS numeric_scale
-    FROM pg_attribute a
-    JOIN pg_class     c ON c.oid = a.attrelid
-    JOIN pg_namespace n ON n.oid = c.relnamespace
-    WHERE TRUE
-      AND n.nspname = $1
-      AND c.relname = $2
-      AND c.relkind IN ('r', 'v', 'm', 'f', 'p')
-      AND a.attnum > 0
-      AND NOT a.attisdropped
-`;
-
-// map a data_type name to a clean postgres column type
+// map information_schema data_type to a clean postgres column type
 function mapType(dataType, numericPrecision, numericScale) {
     switch (dataType) {
         case 'character varying':
@@ -70,4 +36,4 @@ function mapType(dataType, numericPrecision, numericScale) {
     }
 }
 
-module.exports = { fcTable, mapType, RELATION_COLUMNS_SQL };
+module.exports = { fcTable, mapType };

package-lock.json

@@ -7,14 +7,11 @@
     "": {
       "name": "pf_app",
       "version": "1.0.0",
-      "license": "MIT",
       "dependencies": {
         "apache-arrow": "^21.1.0",
-        "connect-pg-simple": "^10.0.0",
         "cors": "^2.8.5",
         "dotenv": "^16.0.0",
         "express": "^4.18.2",
-        "express-session": "^1.19.0",
         "pg": "^8.11.3"
       },
       "devDependencies": {
@@ -372,18 +369,6 @@
         "node": ">=12.20.0"
       }
     },
-    "node_modules/connect-pg-simple": {
-      "version": "10.0.0",
-      "resolved": "https://registry.npmjs.org/connect-pg-simple/-/connect-pg-simple-10.0.0.tgz",
-      "integrity": "sha512-pBGVazlqiMrackzCr0eKhn4LO5trJXsOX0nQoey9wCOayh80MYtThCbq8eoLsjpiWgiok/h+1/uti9/2/Una8A==",
-      "license": "MIT",
-      "dependencies": {
-        "pg": "^8.12.0"
-      },
-      "engines": {
-        "node": "^18.18.0 || ^20.9.0 || >=22.0.0"
-      }
-    },
     "node_modules/content-disposition": {
       "version": "0.5.4",
       "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-0.5.4.tgz",
@@ -597,29 +582,6 @@
         "url": "https://opencollective.com/express"
       }
     },
-    "node_modules/express-session": {
-      "version": "1.19.0",
-      "resolved": "https://registry.npmjs.org/express-session/-/express-session-1.19.0.tgz",
-      "integrity": "sha512-0csaMkGq+vaiZTmSMMGkfdCOabYv192VbytFypcvI0MANrp+4i/7yEkJ0sbAEhycQjntaKGzYfjfXQyVb7BHMA==",
-      "license": "MIT",
-      "dependencies": {
-        "cookie": "~0.7.2",
-        "cookie-signature": "~1.0.7",
-        "debug": "~2.6.9",
-        "depd": "~2.0.0",
-        "on-headers": "~1.1.0",
-        "parseurl": "~1.3.3",
-        "safe-buffer": "~5.2.1",
-        "uid-safe": "~2.1.5"
-      },
-      "engines": {
-        "node": ">= 0.8.0"
-      },
-      "funding": {
-        "type": "opencollective",
-        "url": "https://opencollective.com/express"
-      }
-    },
     "node_modules/fill-range": {
       "version": "7.1.1",
       "resolved": "https://registry.npmjs.org/fill-range/-/fill-range-7.1.1.tgz",
@@ -1123,15 +1085,6 @@
         "node": ">= 0.8"
       }
     },
-    "node_modules/on-headers": {
-      "version": "1.1.0",
-      "resolved": "https://registry.npmjs.org/on-headers/-/on-headers-1.1.0.tgz",
-      "integrity": "sha512-737ZY3yNnXy37FHkQxPzt4UZ2UWPWiCZWLvFZ4fu5cueciegX0zGPnrlY6bwRg4FdQOe9YU8MkmJwGhoMybl8A==",
-      "license": "MIT",
-      "engines": {
-        "node": ">= 0.8"
-      }
-    },
     "node_modules/parseurl": {
       "version": "1.3.3",
       "resolved": "https://registry.npmjs.org/parseurl/-/parseurl-1.3.3.tgz",
@@ -1323,15 +1276,6 @@
         "url": "https://github.com/sponsors/ljharb"
       }
     },
-    "node_modules/random-bytes": {
-      "version": "1.0.0",
-      "resolved": "https://registry.npmjs.org/random-bytes/-/random-bytes-1.0.0.tgz",
-      "integrity": "sha512-iv7LhNVO047HzYR3InF6pUcUsPQiHTM1Qal51DcGSuZFBil1aBBWG5eHPNek7bvILMaYJ/8RU1e8w1AMdHmLQQ==",
-      "license": "MIT",
-      "engines": {
-        "node": ">= 0.8"
-      }
-    },
     "node_modules/range-parser": {
       "version": "1.2.1",
       "resolved": "https://registry.npmjs.org/range-parser/-/range-parser-1.2.1.tgz",
@@ -1648,18 +1592,6 @@
         "node": ">=12.17"
       }
     },
-    "node_modules/uid-safe": {
-      "version": "2.1.5",
-      "resolved": "https://registry.npmjs.org/uid-safe/-/uid-safe-2.1.5.tgz",
-      "integrity": "sha512-KPHm4VL5dDXKz01UuEd88Df+KzynaohSL9fBh096KWAxSKZQDI2uBrVqtvRM4rwrIrRRKsdLNML/lnaaVSRioA==",
-      "license": "MIT",
-      "dependencies": {
-        "random-bytes": "~1.0.0"
-      },
-      "engines": {
-        "node": ">= 0.8"
-      }
-    },
     "node_modules/undefsafe": {
       "version": "2.0.5",
       "resolved": "https://registry.npmjs.org/undefsafe/-/undefsafe-2.0.5.tgz",

package.json

@@ -7,15 +7,17 @@
   "scripts": {
     "start": "node server.js",
     "dev": "nodemon server.js",
-    "build": "cd ui && npm run build"
+    "build": "cd ui && npm run build",
+    "migrate": "node scripts/migrate.js up",
+    "migrate:status": "node scripts/migrate.js status",
+    "migrate:baseline": "node scripts/migrate.js baseline",
+    "schema:dump": "node scripts/schema-dump.js"
   },
   "dependencies": {
     "apache-arrow": "^21.1.0",
-    "connect-pg-simple": "^10.0.0",
     "cors": "^2.8.5",
     "dotenv": "^16.0.0",
     "express": "^4.18.2",
-    "express-session": "^1.19.0",
     "pg": "^8.11.3"
   },
   "devDependencies": {

pf.sh

@@ -4,8 +4,6 @@ set -euo pipefail
 # ---------------------------------------------------------------------------
 # pf.sh — Pivot Forecast management script
 # Usage: ./pf.sh [deploy|start|stop|restart|status|logs|db-setup|config]
-#        ./pf.sh [add-user|passwd|list-users|disable-user|enable-user]
-#        ./pf.sh [set-territory|set-admin|orphan-territory]
 #        ./pf.sh          (interactive menu)
 # ---------------------------------------------------------------------------
 
@@ -69,33 +67,13 @@ require_service() {
     service_installed || die "systemd service not installed. Run: ./pf.sh install-service"
 }
 
-# The keys cmd_config manages; anything else in .env is left alone.
-ENV_KEYS=(DB_HOST DB_PORT DB_NAME DB_USER DB_PASSWORD PORT SESSION_SECRET COOKIE_SECURE)
-
-env_get() {
-    [[ -f "$ENV_FILE" ]] || return 0
-    grep -E "^$1=" "$ENV_FILE" | tail -1 | cut -d= -f2- | tr -d '"' || true
-}
-
-# psql against the DB_* connection in .env; extra args are passed through.
-run_psql() {
-    PGPASSWORD="${DB_PASSWORD:-}" psql \
-        -h "${DB_HOST:-localhost}" \
-        -p "${DB_PORT:-5432}" \
-        -U "${DB_USER}" \
-        -d "${DB_NAME}" \
-        "$@"
-}
-
 db_ping() {
     load_env
-    if [[ -z "${DB_NAME:-}" || -z "${DB_USER:-}" ]]; then
-        warn "DB_NAME / DB_USER not set in .env"
-        return 1
-    fi
+    local url="${DATABASE_URL:-}"
+    [[ -z "$url" ]] && { warn "DATABASE_URL not set in .env"; return 1; }
     # Use psql if available for a real connectivity check
     if command -v psql &>/dev/null; then
-        run_psql -tAc "SELECT 1" &>/dev/null && return 0 || return 1
+        psql "$url" -c "SELECT 1" &>/dev/null && return 0 || return 1
     else
         warn "psql not in PATH — skipping live DB check"
         return 0
@@ -190,27 +168,18 @@ cmd_logs() {
 
 cmd_db_setup() {
     require_env; load_env
-    [[ -n "${DB_NAME:-}" && -n "${DB_USER:-}" ]] || die "DB_NAME / DB_USER not set in .env — run: ./pf.sh config"
+    local url="${DATABASE_URL:-}"
+    [[ -z "$url" ]] && die "DATABASE_URL not set in .env"
     command -v psql &>/dev/null || die "psql not found — install postgresql-client"
 
     echo
-    bold "DB Setup — will run: setup_sql/01_schema.sql, setup_sql/gen_dim_period.sql, setup_sql/02_auth.sql"
-    warn "This creates the pf schema and tables, the pf.dim_period calendar, and the account/session tables."
-    warn "The calendar's fiscal year starts in the month set in gen_dim_period.sql (fiscal_start_month) — check it before a first install."
-    warn "Safe to re-run (CREATE IF NOT EXISTS; existing calendar rows are never overwritten)."
+    bold "DB Setup — will run: setup_sql/01_schema.sql"
+    warn "This creates the pf schema and tables. Safe to re-run (CREATE IF NOT EXISTS)."
     read -rp "  Continue? [y/N] " confirm
     [[ "$confirm" =~ ^[Yy]$ ]] || { echo "Aborted."; return; }
 
-    run_psql -v ON_ERROR_STOP=1 -f "${APP_DIR}/setup_sql/01_schema.sql"
-    run_psql -v ON_ERROR_STOP=1 -f "${APP_DIR}/setup_sql/gen_dim_period.sql"
-    run_psql -v ON_ERROR_STOP=1 -f "${APP_DIR}/setup_sql/02_auth.sql"
+    psql "$url" -f "${APP_DIR}/setup_sql/01_schema.sql"
     success "Schema applied."
-
-    local n
-    n=$(run_psql -tAc "SELECT count(*) FROM pf.app_user WHERE is_active" 2>/dev/null || echo 0)
-    if [[ "${n:-0}" == "0" ]]; then
-        warn "No active accounts yet — create one with: ./pf.sh add-user"
-    fi
 }
 
 cmd_config() {
@@ -219,284 +188,41 @@ cmd_config() {
     echo "  File: $ENV_FILE"
     echo
 
-    local cur_host cur_port cur_name cur_user cur_pass cur_app_port
-    cur_host=$(env_get DB_HOST)
-    cur_port=$(env_get DB_PORT)
-    cur_name=$(env_get DB_NAME)
-    cur_user=$(env_get DB_USER)
-    cur_pass=$(env_get DB_PASSWORD)
-    cur_app_port=$(env_get PORT)
+    local current_url=""
+    local current_port=""
+    local current_user=""
 
-    local input
-    read -rp "  DB_HOST [${cur_host:-localhost}]: " input
-    local host="${input:-${cur_host:-localhost}}"
-
-    read -rp "  DB_PORT [${cur_port:-5432}]: " input
-    local port="${input:-${cur_port:-5432}}"
-
-    read -rp "  DB_NAME [${cur_name:-not set}]: " input
-    local name="${input:-$cur_name}"
-    [[ -z "$name" ]] && die "DB_NAME is required."
-
-    read -rp "  DB_USER [${cur_user:-$USER}]: " input
-    local user="${input:-${cur_user:-$USER}}"
-
-    if [[ -n "$cur_pass" ]]; then
-        read -rsp "  DB_PASSWORD [keep existing]: " input; echo
-    else
-        read -rsp "  DB_PASSWORD: " input; echo
-    fi
-    local pass="${input:-$cur_pass}"
-
-    read -rp "  PORT (app) [${cur_app_port:-3010}]: " input
-    local app_port="${input:-${cur_app_port:-3010}}"
-
-    # Session cookies are signed with this; regenerating it signs everyone out,
-    # so an existing secret is kept rather than re-rolled on every config run.
-    local secret
-    secret=$(env_get SESSION_SECRET)
-    if [[ -z "$secret" ]]; then
-        secret=$(node -e 'console.log(require("crypto").randomBytes(32).toString("hex"))')
-        success "SESSION_SECRET generated."
-    else
-        success "SESSION_SECRET kept (delete the line in .env to re-roll)."
-    fi
-
-    local cur_secure
-    cur_secure=$(env_get COOKIE_SECURE)
-    read -rp "  COOKIE_SECURE — HTTPS-only cookie [${cur_secure:-true}]: " input
-    local cookie_secure="${input:-${cur_secure:-true}}"
-
-    # Rewrite the managed keys, carrying over any other lines already in .env.
-    local tmp
-    tmp=$(mktemp)
-    cat > "$tmp" <<EOF
-DB_HOST=${host}
-DB_PORT=${port}
-DB_NAME=${name}
-DB_USER=${user}
-DB_PASSWORD=${pass}
-PORT=${app_port}
-SESSION_SECRET=${secret}
-COOKIE_SECURE=${cookie_secure}
-EOF
     if [[ -f "$ENV_FILE" ]]; then
-        local managed
-        managed=$(IFS='|'; echo "${ENV_KEYS[*]}")
-        grep -vE "^(${managed})=" "$ENV_FILE" | grep -vE '^[[:space:]]*$' >> "$tmp" || true
+        current_url=$(grep -E '^DATABASE_URL=' "$ENV_FILE" | cut -d= -f2- | tr -d '"' || true)
+        current_port=$(grep -E '^PORT=' "$ENV_FILE" | cut -d= -f2- | tr -d '"' || true)
+        current_user=$(grep -E '^PF_USER=' "$ENV_FILE" | cut -d= -f2- | tr -d '"' || true)
     fi
-    mv "$tmp" "$ENV_FILE"
+
+    read -rp "  DATABASE_URL [${current_url:-not set}]: " input_url
+    local url="${input_url:-$current_url}"
+    [[ -z "$url" ]] && die "DATABASE_URL is required."
+
+    read -rp "  PORT [${current_port:-3010}]: " input_port
+    local port="${input_port:-${current_port:-3010}}"
+
+    read -rp "  PF_USER [${current_user:-$USER}]: " input_user
+    local pf_user="${input_user:-${current_user:-$USER}}"
+
+    cat > "$ENV_FILE" <<EOF
+DATABASE_URL=${url}
+PORT=${port}
+PF_USER=${pf_user}
+EOF
     chmod 600 "$ENV_FILE"
     success ".env written."
 
     if db_ping; then
         success "Database connection verified."
     else
-        warn "Could not reach the database — double-check the DB_* settings."
+        warn "Could not reach the database — double-check DATABASE_URL."
     fi
 }
 
-# -- Accounts ----------------------------------------------------------------
-
-# Reads a password twice without echo and hashes it with lib/auth.js, so the
-# plaintext never reaches argv, psql, or the shell history.
-read_new_password() {
-    local p1 p2
-    # Prompts and their newlines go to stderr: stdout is the hash, and a stray
-    # newline there ends up prefixed to it by the caller's $( ).
-    read -rsp "  Password: " p1; echo >&2
-    [[ -z "$p1" ]] && { error "Password cannot be empty."; return 1; }
-    read -rsp "  Confirm : " p2; echo >&2
-    [[ "$p1" != "$p2" ]] && { error "Passwords do not match."; return 1; }
-    printf '%s' "$p1" | node "${APP_DIR}/lib/auth.js" hash
-}
-
-# psql single-quoted literal: double any embedded quote.
-sql_lit() { printf "%s" "${1//\'/\'\'}"; }
-
-cmd_add_user() {
-    require_env; load_env
-    check_node >/dev/null
-
-    echo; bold "Add account"
-    local username display hash
-    read -rp "  Username: " username
-    [[ -z "$username" ]] && die "Username is required."
-    read -rp "  Display name [${username}]: " display
-    display="${display:-$username}"
-
-    hash=$(read_new_password) || return 1
-
-    run_psql -v ON_ERROR_STOP=1 -tAc "
-        WITH ins AS (
-            INSERT INTO pf.app_user (username, pass_hash, display_name)
-            VALUES ('$(sql_lit "$username")', '$(sql_lit "$hash")', '$(sql_lit "$display")')
-            ON CONFLICT (username) DO NOTHING
-            RETURNING id
-        ) SELECT id FROM ins" | grep -q . \
-        && success "Account '${username}' created." \
-        || die "Account '${username}' already exists — change its password with: ./pf.sh passwd"
-}
-
-cmd_passwd() {
-    require_env; load_env
-    check_node >/dev/null
-
-    echo; bold "Change password"
-    local username hash
-    read -rp "  Username: " username
-    [[ -z "$username" ]] && die "Username is required."
-
-    hash=$(read_new_password) || return 1
-
-    run_psql -v ON_ERROR_STOP=1 -tAc "
-        WITH upd AS (
-            UPDATE pf.app_user SET pass_hash = '$(sql_lit "$hash")'
-            WHERE lower(username) = lower('$(sql_lit "$username")')
-            RETURNING id
-        ) SELECT id FROM upd" | grep -q . \
-        && success "Password updated for '${username}'." \
-        || die "No such account: ${username}"
-}
-
-cmd_list_users() {
-    require_env; load_env
-    echo; bold "Accounts"
-    run_psql -c "
-        SELECT username, display_name, is_active, is_admin,
-               coalesce(jsonb_array_length(territory), 0) AS territory_values,
-               to_char(last_login_at, 'YYYY-MM-DD HH24:MI') AS last_login
-        FROM pf.app_user ORDER BY username"
-}
-
-# Territory is what an account may see and change, as a list of values in the
-# source's is_territory column. No territory and not an admin means no rows --
-# so a new account is blind until this is run, which is the intended direction
-# to fail in.
-#
-# Values are given comma-separated and have to match the column exactly, since
-# that is what the SQL compares. set-territory with no values clears it.
-cmd_set_territory() {
-    require_env; load_env
-    local username="${1:-}"; shift || true
-    [[ -z "$username" ]] && { read -rp "  Username: " username; }
-    [[ -z "$username" ]] && die "Username is required."
-    local values="${*:-}"
-    [[ -z "$values" ]] && { read -rp "  Territory values (comma separated, blank to clear): " values; }
-
-    local json="null"
-    if [[ -n "$values" ]]; then
-        json=$(python3 - "$values" <<'PYEOF'
-import json, sys
-vals = [v.strip() for v in sys.argv[1].split(',') if v.strip()]
-print(json.dumps(vals))
-PYEOF
-)
-    fi
-
-    run_psql -v ON_ERROR_STOP=1 -tAc "
-        WITH upd AS (
-            UPDATE pf.app_user SET territory = $(if [[ "$json" == "null" ]]; then echo NULL; else echo "'$(sql_lit "$json")'::jsonb"; fi)
-            WHERE lower(username) = lower('$(sql_lit "$username")')
-            RETURNING username
-        )
-        SELECT count(*) FROM upd" | grep -q '^1$' \
-        || die "No such account: $username"
-    success "Territory updated for $username"
-    run_psql -c "SELECT username, is_admin, territory FROM pf.app_user WHERE lower(username) = lower('$(sql_lit "$username")')"
-}
-
-# An admin sees and changes everything, and is the only account that can recode
-# the territory column or undo someone else's entry.
-cmd_set_admin() {
-    require_env; load_env
-    local username="${1:-}" flag="${2:-true}"
-    [[ -z "$username" ]] && { read -rp "  Username: " username; }
-    [[ -z "$username" ]] && die "Username is required."
-    [[ "$flag" != "true" && "$flag" != "false" ]] && die "Second argument must be true or false."
-
-    run_psql -v ON_ERROR_STOP=1 -tAc "
-        WITH upd AS (
-            UPDATE pf.app_user SET is_admin = $flag
-            WHERE lower(username) = lower('$(sql_lit "$username")')
-            RETURNING username
-        )
-        SELECT count(*) FROM upd" | grep -q '^1$' \
-        || die "No such account: $username"
-    success "$username is_admin = $flag"
-}
-
-# Territory values present in the data that belong to no account. Work under one
-# is invisible to everybody but an admin, which is easy to cause by a typo and
-# impossible to notice from inside the app.
-cmd_orphan_territory() {
-    require_env; load_env
-    local source_id="${1:-}"
-    [[ -z "$source_id" ]] && { read -rp "  Source id: " source_id; }
-    [[ -z "$source_id" ]] && die "Source id is required."
-
-    # The column and table are data, so the query is built in two steps rather
-    # than one clever one: read the names, then run the listing.
-    local meta col schema tname
-    meta=$(run_psql -tAF'|' -c "
-        SELECT m.cname, x.schema, x.tname
-        FROM pf.col_meta m JOIN pf.source x ON x.id = m.source_id
-        WHERE m.source_id = $source_id AND m.is_territory")
-    [[ -z "$meta" ]] && die "Source $source_id has no column marked is_territory."
-    IFS='|' read -r col schema tname <<< "$meta"
-
-    echo; bold "Territory values in $schema.$tname with no account"
-    run_psql -c "
-        SELECT DISTINCT s.\"$col\" AS unassigned
-        FROM \"$schema\".\"$tname\" s
-        WHERE TRUE
-          AND s.\"$col\" IS NOT NULL
-          AND s.\"$col\"::text NOT IN (
-              SELECT jsonb_array_elements_text(territory)
-              FROM pf.app_user
-              WHERE territory IS NOT NULL
-          )
-        ORDER BY 1"
-}
-
-# Deactivating leaves the row (and its history) in place, and drops any live
-# session so the account loses access immediately rather than at cookie expiry.
-cmd_disable_user() {
-    require_env; load_env
-    local username="${1:-}"
-    [[ -z "$username" ]] && { read -rp "  Username to disable: " username; }
-    [[ -z "$username" ]] && die "Username is required."
-
-    run_psql -v ON_ERROR_STOP=1 -tAc "
-        WITH upd AS (
-            UPDATE pf.app_user SET is_active = false
-            WHERE lower(username) = lower('$(sql_lit "$username")')
-            RETURNING id
-        ) SELECT id FROM upd" | grep -q . \
-        || die "No such account: ${username}"
-
-    run_psql -v ON_ERROR_STOP=1 -c "
-        DELETE FROM pf.session
-        WHERE sess::jsonb -> 'user' ->> 'username' ILIKE '$(sql_lit "$username")'" >/dev/null
-    success "Account '${username}' disabled and signed out."
-}
-
-cmd_enable_user() {
-    require_env; load_env
-    local username="${1:-}"
-    [[ -z "$username" ]] && { read -rp "  Username to enable: " username; }
-    [[ -z "$username" ]] && die "Username is required."
-
-    run_psql -v ON_ERROR_STOP=1 -tAc "
-        WITH upd AS (
-            UPDATE pf.app_user SET is_active = true
-            WHERE lower(username) = lower('$(sql_lit "$username")')
-            RETURNING id
-        ) SELECT id FROM upd" | grep -q . \
-        && success "Account '${username}' enabled." \
-        || die "No such account: ${username}"
-}
-
 cmd_install_service() {
     require_systemd
     require_env
@@ -576,18 +302,10 @@ interactive_menu() {
         echo "  4) restart         restart server"
         echo "  5) status          service + DB + git info"
         echo "  6) logs            tail journald logs"
-        echo "  7) db-setup        apply the pf schema, calendar and auth tables"
-        echo "  8) config          set DB connection + app PORT"
+        echo "  7) db-setup        apply setup_sql/01_schema.sql"
+        echo "  8) config          set DATABASE_URL / PORT / PF_USER"
         echo "  9) install-service create systemd unit file"
         echo " 10) uninstall-service remove systemd unit file"
-        echo " 11) add-user       create a login account"
-        echo " 12) passwd         change an account password"
-        echo " 13) list-users     show accounts"
-        echo " 14) disable-user   deactivate an account and sign it out"
-        echo " 15) enable-user    reactivate an account"
-        echo " 16) set-territory  grant an account the territory values it may see"
-        echo " 17) set-admin      make an account an administrator"
-        echo " 18) orphan-territory  territory values no account owns"
         echo "  q) quit"
         echo
         read -rp "  Choice: " choice
@@ -602,14 +320,6 @@ interactive_menu() {
             8|config)            cmd_config ;;
             9|install-service)   cmd_install_service ;;
             10|uninstall-service) cmd_uninstall_service ;;
-            11|add-user)         cmd_add_user ;;
-            12|passwd)           cmd_passwd ;;
-            13|list-users)       cmd_list_users ;;
-            14|disable-user)     cmd_disable_user ;;
-            15|enable-user)      cmd_enable_user ;;
-            16|set-territory)    cmd_set_territory ;;
-            17|set-admin)        cmd_set_admin ;;
-            18|orphan-territory) cmd_orphan_territory ;;
             q|Q|quit|exit)       echo "Bye."; exit 0 ;;
             *) warn "Unknown option: $choice" ;;
         esac
@@ -629,14 +339,6 @@ case "${1:-}" in
     config)             cmd_config ;;
     install-service)    cmd_install_service ;;
     uninstall-service)  cmd_uninstall_service ;;
-    add-user)           cmd_add_user ;;
-    passwd)             cmd_passwd ;;
-    list-users)         cmd_list_users ;;
-    set-territory)      shift; cmd_set_territory "$@" ;;
-    set-admin)          shift; cmd_set_admin "$@" ;;
-    orphan-territory)   shift; cmd_orphan_territory "$@" ;;
-    disable-user)       cmd_disable_user "${2:-}" ;;
-    enable-user)        cmd_enable_user "${2:-}" ;;
     "")                 interactive_menu ;;
-    *) die "Unknown command: $1. Valid: deploy start stop restart status logs db-setup config install-service uninstall-service add-user passwd list-users disable-user enable-user set-territory set-admin orphan-territory" ;;
+    *) die "Unknown command: $1. Valid: deploy start stop restart status logs db-setup config install-service uninstall-service" ;;
 esac

pf_perspective_options.md

@@ -1,17 +1,5 @@
 # Perspective Architecture Options
 
-> **Decision record, written 2026-08.** The outcome was the display-grain
-> pre-aggregation (§Spike findings onward), and it shipped. **"Current
-> architecture" below describes the app as it was then**, not now: it has
-> since moved to a vendored Perspective 5.4.0 bundled inline, `tableFromArrays`
-> on the server, `table.remove()` on undo, and `/data` / `/agg` emitting
-> `pf_segment`, `pf_bucket`, `pf_note` and `pf_op`. For today's data flow see
-> `CLAUDE.md` → §Core data flow and `pf_spec.md` → §Display-grain
-> pre-aggregation. What remains current here is the spike findings, the
-> display-grain reasoning, and the Path A / Path B analysis behind the
-> "dynamic grain" next step. The DuckDB virtual-server spike is parked on
-> branch `spike/duckdb-virtual-server`.
-
 This document weighs how the Forecast view should source data for the Perspective
 pivot. The current implementation hits practical limits on initial load
 (~30s for 350k rows × ~55 cols), and growth is expected. Choosing an architecture

pf_spec.md

@@ -0,0 +1,950 @@
+# Pivot Forecast — Application Spec
+
+## Overview
+
+A web application for building named forecast scenarios against any PostgreSQL table. The core workflow is: load known historical actuals as a baseline, shift those dates forward by a specified interval into the forecast period to establish a no-change starting point, then apply incremental adjustments (scale, recode, clone) to build the plan. An admin configures a source table, generates a baseline, and opens it for users to make adjustments. Users interact with a pivot table to select slices of data and apply forecast operations. All changes are incremental (append-only), fully audited, and reversible.
+
+---
+
+## Tech Stack
+
+- **Backend:** Node.js / Express
+- **Database:** PostgreSQL — isolated `pf` schema, installs into any existing DB
+- **Frontend:** React + Vite + Tailwind CSS; Perspective (forecast pivot)
+- **Pattern:** Follows fc_webapp (shell) + pivot_forecast (operations)
+
+---
+
+## Database Schema: `pf`
+
+Everything lives in the `pf` schema. Install via sequential SQL scripts.
+
+### `pf.source`
+Registered source tables available for forecasting.
+
+```sql
+CREATE TABLE pf.source (
+    id              serial PRIMARY KEY,
+    schema          text NOT NULL,
+    tname           text NOT NULL,
+    label           text,                   -- friendly display name
+    status          text DEFAULT 'active',  -- active | archived
+    default_layout  jsonb,                  -- Perspective view config used as per-source default
+    created_at      timestamptz DEFAULT now(),
+    created_by      text,
+    UNIQUE (schema, tname)
+);
+```
+
+### `pf.col_meta`
+Column configuration for each registered source table. Determines how the app treats each column.
+
+```sql
+CREATE TABLE pf.col_meta (
+    id              serial PRIMARY KEY,
+    source_id       integer REFERENCES pf.source(id),
+    cname           text NOT NULL,          -- column name in source table
+    label           text,                   -- friendly display name
+    role            text NOT NULL,          -- 'dimension' | 'value' | 'units' | 'date' | 'filter' | 'ignore'
+    is_key          boolean DEFAULT false,  -- true = part of natural key (used in WHERE slice)
+    opos            integer,                -- ordinal position (for ordering)
+    dim_group       text,                   -- groups functionally dependent columns (see below)
+    dim_period_col  text,                   -- maps this dimension to a pf.dim_period column
+    UNIQUE (source_id, cname)
+);
+```
+
+**Roles:**
+- `dimension` — categorical field (customer, part, channel, rep, geography, etc.) — appears as pivot rows/cols, used in WHERE filters for operations
+- `value` — the money/revenue field to scale (**required** — SQL generation fails without it)
+- `units` — the quantity field to scale (**optional** — if absent, units columns are omitted from the forecast table and all SQL patterns)
+- `date` — the primary date field; used for baseline/reference date range and stored in the forecast table (**required**)
+- `filter` — columns available as filter conditions in the Baseline Workbench (e.g. order status, ship date, open flag); used in baseline WHERE clauses but **not stored** in the forecast table
+- `ignore` — exclude from forecast table entirely
+
+**`dim_group`** — a free-text group name linking a `date` column to its derived dimension siblings. When the `date` column has `is_key = true` and a `dim_group` value, the SQL generator looks for `dimension` columns in the same group that also have a `dim_period_col` value. Those columns are sourced from `pf.dim_period` on baseline/reference load (via a JOIN on `drange @> date`) rather than copied raw from the source table. This allows fiscal year, quarter, and month columns to be stored in the forecast table with calendar-correct values even if those columns don't exist in the source.
+
+**`dim_period_col`** — names the column in `pf.dim_period` to use as the value for this dimension on load. Only meaningful when the column is in a `dim_group` whose `date` key has `is_key = true`. Example: `cal_year`, `fisc_quarter`, `fisc_label`.
+
+### `pf.dim_period`
+Calendar lookup table. One row per month from 2018-01-01 through 2035-12-01. Keyed on `sdat` (month start date). Used to derive fiscal/calendar period columns at baseline load time when `dim_group` / `dim_period_col` are configured on col_meta.
+
+Populated by `setup_sql/gen_dim_period.sql` (safe to re-run; `ON CONFLICT DO NOTHING`). Fiscal year start month is configurable at the top of that script (default: June, i.e. fiscal month 1 = June).
+
+Key columns: `sdat`, `edat`, `drange` (GiST-indexed daterange), `cal_year`, `cal_quarter`, `cal_month`, `cal_month_abbr`, `cal_month_name`, `cal_label`, `fisc_year`, `fisc_quarter`, `fisc_quarter_label`, `fisc_month`, `fisc_month_abbr`, `fisc_month_name`, `fisc_label`, `period_key`.
+
+The baseline/reference SQL JOINs this table when `hasDimPeriod` is true: `JOIN pf.dim_period dp ON dp.drange @> (s.{date_col} + '{{date_offset}}'::interval)::date`.
+
+### `pf.version`
+Named forecast scenarios. One forecast table (`pf.fc_{tname}_{version_id}`) is created per version.
+
+```sql
+CREATE TABLE pf.version (
+    id              serial PRIMARY KEY,
+    source_id       integer REFERENCES pf.source(id),
+    name            text NOT NULL,
+    description     text,
+    status          text DEFAULT 'open',        -- open | closed
+    exclude_iters   jsonb DEFAULT '["reference"]', -- iter values excluded from all operations
+    created_at      timestamptz DEFAULT now(),
+    created_by      text,
+    closed_at       timestamptz,
+    closed_by       text,
+    UNIQUE (source_id, name)
+);
+```
+
+**`exclude_iters`:** jsonb array of `iter` values that are excluded from operation WHERE clauses. Defaults to `["reference"]`. Reference rows are still returned by `get_data` (visible in pivot) but are never touched by scale/recode/clone. Additional iters can be added to lock them from further adjustment.
+
+**Forecast table naming:** `pf.fc_{tname}_{version_id}` — e.g., `pf.fc_sales_3`. One table per version, physically isolated. Contains both operational rows and reference rows.
+
+Creating a version → `CREATE TABLE pf.fc_{tname}_{version_id} (...)`
+Deleting a version → `DROP TABLE pf.fc_{tname}_{version_id}` + delete from `pf.version` + delete from `pf.log`
+
+### `pf.log`
+Audit log. Every write operation gets one entry here.
+
+```sql
+CREATE TABLE pf.log (
+    id          bigserial PRIMARY KEY,
+    version_id  integer REFERENCES pf.version(id),
+    pf_user     text NOT NULL,
+    stamp       timestamptz DEFAULT now(),
+    operation   text NOT NULL,  -- 'baseline' | 'reference' | 'scale' | 'recode' | 'clone'
+    slice       jsonb,          -- the WHERE conditions that defined the selection
+    params      jsonb,          -- operation parameters (increments, new values, scale factor, etc.)
+    note        text            -- user-provided comment
+);
+```
+
+### `pf.fc_{tname}_{version_id}` (dynamic, one per version)
+Created when a version is created. Mirrors source table dimension/value/date columns (and units if configured) plus any `dim_period_col`-derived dimension columns, plus forecast metadata. Contains both operational rows (`pf_iter = 'baseline' | 'scale' | 'recode' | 'clone'`) and reference rows (`pf_iter = 'reference'`).
+
+```sql
+-- Example: source table "sales", version id 3 → pf.fc_sales_3
+CREATE TABLE pf.fc_sales_3 (
+    id              bigserial PRIMARY KEY,
+
+    -- mirrored from source (role = dimension | value | units | date only):
+    customer        text,
+    channel         text,
+    part            text,
+    geography       text,
+    order_date      date,
+    value           numeric,
+    units           numeric,    -- omitted if no 'units' role in col_meta
+
+    -- forecast metadata:
+    pf_iter         text,       -- 'baseline' | 'reference' | 'scale' | 'recode' | 'clone'
+    pf_logid        bigint REFERENCES pf.log(id),
+    pf_user         text,
+    pf_created_at   timestamptz DEFAULT now()
+);
+```
+
+Note: no `version_id` column on the forecast table — it's implied by the table itself. The `units` column is only present when a column with `role = 'units'` exists in col_meta.
+
+### `pf.sql`
+Generated SQL stored per source and operation. Built once when col_meta is finalized, fetched at request time.
+
+```sql
+CREATE TABLE pf.sql (
+    id           serial PRIMARY KEY,
+    source_id    integer REFERENCES pf.source(id),
+    operation    text NOT NULL,  -- 'baseline' | 'reference' | 'scale' | 'recode' | 'clone' | 'get_data' | 'undo'
+    sql          text NOT NULL,
+    generated_at timestamptz DEFAULT now(),
+    UNIQUE (source_id, operation)
+);
+```
+
+**Column names are baked in at generation time.** Runtime substitution tokens:
+
+| Token | Resolved from |
+|-------|--------------|
+| `{{fc_table}}` | `pf.fc_{tname}_{version_id}` — derived at request time |
+| `{{where_clause}}` | built from `slice` JSON by `build_where()` in JS |
+| `{{exclude_clause}}` | built from `version.exclude_iters` — e.g. `AND pf_iter NOT IN ('reference')` |
+| `{{logid}}` | newly inserted `pf.log` id |
+| `{{pf_user}}` | from request body |
+| `{{date_from}}` / `{{date_to}}` | baseline/reference date range (source period) |
+| `{{date_offset}}` | PostgreSQL interval string to shift dates into the forecast period — e.g. `1 year`, `6 months`, `2 years 3 months` (baseline only; empty string = no shift) |
+| `{{value_incr}}` / `{{units_incr}}` | scale operation increments |
+| `{{pct}}` | scale mode: absolute or percentage |
+| `{{set_clause}}` | recode/clone dimension overrides |
+| `{{scale_factor}}` | clone multiplier |
+
+**Request-time flow:**
+1. Fetch SQL from `pf.sql` for `source_id` + `operation`
+2. Fetch `version.exclude_iters`, build `{{exclude_clause}}`
+3. Build `{{where_clause}}` from `slice` JSON via `build_where()`
+4. Substitute all tokens
+5. Execute — single round trip
+
+**WHERE clause safety:** `build_where()` validates every key in the slice against col_meta (only `role = 'dimension'` columns are permitted). Values are sanitized (escaped single quotes). No parameterization — consistent with existing projects, debuggable in Postgres logs.
+
+---
+
+## Setup / Install Scripts
+
+```
+setup_sql/
+  migrations/            -- forward-only .sql, applied once each, tracked in pf.schema_version
+    0001_initial_schema.sql          CREATE SCHEMA pf + metadata tables
+    0002_source_default_layout.sql
+    0003_col_meta_dim_group_period.sql
+    0004_col_meta_in_grain.sql
+  gen_dim_period.sql     -- parameterised calendar load (fiscal year start month); not a migration
+  schema.generated.sql   -- pg_dump reference snapshot; generated, never edited or applied
+  README.md              -- migration workflow
+```
+
+Install with `npm run migrate`, then run `gen_dim_period.sql`. For a database that
+already matches the schema, `npm run migrate:baseline` records the migrations as
+applied without running them. `server.js` refuses to start when the database is
+behind, so drift surfaces at boot rather than as a column-not-found error inside
+an unrelated request.
+
+Source registration, col_meta configuration, SQL generation, version creation, and forecast table DDL all happen via API.
+
+---
+
+## API Routes
+
+### DB Browser
+
+| Method | Route | Description |
+|--------|-------|-------------|
+| GET | `/api/tables` | List all tables in the DB with row counts |
+| GET | `/api/tables/:schema/:tname/preview` | Preview columns + sample rows |
+
+### Source Management
+
+| Method | Route | Description |
+|--------|-------|-------------|
+| GET | `/api/sources` | List registered sources |
+| POST | `/api/sources` | Register a source table |
+| GET | `/api/sources/:id/cols` | Get col_meta for a source |
+| PUT | `/api/sources/:id/cols` | Save col_meta configuration |
+| POST | `/api/sources/:id/generate-sql` | Generate/regenerate all operation SQL into `pf.sql` |
+| GET | `/api/sources/:id/sql` | View generated SQL for a source (inspection/debug) |
+| DELETE | `/api/sources/:id` | Deregister a source (does not affect existing forecast tables) |
+
+### Forecast Versions
+
+| Method | Route | Description |
+|--------|-------|-------------|
+| GET | `/api/sources/:id/versions` | List versions for a source |
+| POST | `/api/sources/:id/versions` | Create a new version (CREATE TABLE for forecast table) |
+| PUT | `/api/versions/:id` | Update version (name, description, exclude_iters) |
+| POST | `/api/versions/:id/close` | Close a version (blocks further edits) |
+| POST | `/api/versions/:id/reopen` | Reopen a closed version |
+| DELETE | `/api/versions/:id` | Delete a version (DROP TABLE + delete log entries) |
+
+### Baseline & Reference Data
+
+| Method | Route | Description |
+|--------|-------|-------------|
+| POST | `/api/versions/:id/baseline` | Load one baseline segment (additive — does not clear existing baseline rows) |
+| DELETE | `/api/versions/:id/baseline` | Clear all baseline rows and baseline log entries for this version |
+| POST | `/api/versions/:id/reference` | Load reference rows from source table for a date range (additive) |
+
+**Baseline load request body:**
+```json
+{
+  "date_offset":  "1 year",
+  "filters": [
+    [
+      { "col": "order_date",   "op": "BETWEEN", "values": ["2024-01-01", "2024-12-31"] },
+      { "col": "order_status", "op": "IN",      "values": ["OPEN", "PENDING"] }
+    ],
+    [
+      { "col": "order_status", "op": "IS NULL" }
+    ]
+  ],
+  "pf_user":  "admin",
+  "note":     "FY2024 actuals + open orders projected to FY2025",
+  "replay":   false
+}
+```
+
+The example above generates: `(order_date BETWEEN '2024-01-01' AND '2024-12-31' AND order_status IN ('OPEN','PENDING')) OR (order_status IS NULL)`
+
+- `date_offset` — PostgreSQL interval string applied to the primary `role = 'date'` column at insert time. Examples: `"1 year"`, `"6 months"`, `"2 years 3 months"`. Defaults to `"0 days"`. Applied to the stored date value only — filter columns are never shifted.
+- `filters` — an array of **groups**. Conditions within a group are AND-ed; groups are OR-ed together. Each group is an array of one or more condition objects:
+  - `col` — must be `role = 'date'` or `role = 'filter'` in col_meta
+  - `op` — one of `=`, `!=`, `IN`, `NOT IN`, `BETWEEN`, `IS NULL`, `IS NOT NULL`
+  - `values` — array of strings; two elements for `BETWEEN`; multiple for `IN`/`NOT IN`; omitted for `IS NULL`/`IS NOT NULL`
+  - Backward compatibility: a flat array of condition objects (non-nested) is treated as a single group (all AND).
+- At least one group with at least one condition is required.
+- `raw_where` — optional string. When present, bypasses `filters` entirely and injects the value verbatim as the WHERE clause body. **Admin-only** — rejected with `403` if the requesting `pf_user` is not in the admin list. Not validated against col_meta. Caller is responsible for correctness and SQL safety. Stored as-is in `pf.log.params` for audit. Cannot be combined with `filters` — if both are present the request is rejected with `400`.
+- Baseline loads are **additive** — existing `iter = 'baseline'` rows are not touched. Each load is its own log entry and is independently undoable.
+
+`replay` controls behavior when incremental rows exist (applies to Clear + reload, not individual segments):
+
+- `replay: false` (default) — after clearing, re-load baseline segments, leave incremental rows untouched
+- `replay: true` — after clearing, re-load baseline, then re-execute each incremental log entry in chronological order
+
+**v1 note:** `replay: true` returns `501 Not Implemented` until the replay engine is built.
+
+**Clear baseline (`DELETE /api/versions/:id/baseline`)** — deletes all rows where `iter = 'baseline'` and all `operation = 'baseline'` log entries. Irreversible (no undo). Returns `{ rows_deleted, log_entries_deleted }`.
+
+**Reference request body:** same shape as baseline load without `replay`. Reference dates land verbatim (no offset). Additive — multiple reference loads stack independently, each undoable by logid.
+
+### Forecast Data
+
+| Method | Route | Description |
+|--------|-------|-------------|
+| GET | `/api/versions/:id/data` | Stream all rows for this version as an Arrow IPC binary |
+
+**Transport format — Apache Arrow IPC stream**
+
+The endpoint returns `Content-Type: application/vnd.apache.arrow.stream` (binary). JSON is not used for this route. The client fetches the response as `arrayBuffer()` and passes it directly to `worker.table(buffer)` — Perspective's native ingestion path with no JS deserialization overhead.
+
+Arrow's columnar layout with dictionary encoding on string dimension columns keeps payload size manageable at scale (typically 50–150 MB for 1M rows depending on string cardinality), compared to several times that for equivalent JSON.
+
+**Server-side streaming (cursor-based)**
+
+For datasets that may reach 1M+ rows, the server must not buffer the full query result in memory before writing the response. Instead:
+
+1. Open a PostgreSQL cursor over the `SELECT * FROM {{fc_table}}` query
+2. Fetch rows in batches (target: 10 000 rows per batch)
+3. For each batch, append a serialized Arrow record batch to the HTTP response using chunked transfer encoding
+4. Close the cursor and end the response when all batches are written
+
+This means the first bytes of the Arrow stream reach the client while the server is still reading from the database, and Node.js heap stays bounded regardless of dataset size.
+
+**Client-side loading**
+
+- **Moderate datasets (< ~500k rows):** accumulate the full `arrayBuffer()` then call `worker.table(buffer)` once. Perspective becomes interactive after the stream completes.
+- **Large datasets (≥ ~500k rows):** process Arrow record batches incrementally — call `worker.table(firstBatch)` to create the table, then `pspTable.update(batch)` for each subsequent batch. Perspective is interactive and browseable while remaining batches are still arriving.
+
+The client detects which path to use by checking the `X-Row-Count` response header (see below).
+
+**Row-count pre-check**
+
+Before opening the cursor, the server runs `SELECT COUNT(*) FROM {{fc_table}}`. The result is attached as the `X-Row-Count` response header so the client can choose its loading strategy. If the count exceeds 500 000, the UI displays a non-blocking notice ("Loading large dataset — pivot will become interactive as data arrives") rather than a blank screen.
+
+### Forecast Operations
+
+All operations share a common request envelope:
+
+```json
+{
+  "pf_user": "paul.trowbridge",
+  "note":    "optional comment",
+  "slice": {
+    "channel":   "WHS",
+    "geography": "WEST"
+  }
+}
+```
+
+`slice` keys must be `role = 'dimension'` columns per col_meta. Stored in `pf.log` as the implicit link to affected rows.
+
+#### Scale
+`POST /api/versions/:id/scale`
+
+```json
+{
+  "pf_user":    "paul.trowbridge",
+  "note":       "10% volume lift Q3 West",
+  "slice":      { "channel": "WHS", "geography": "WEST" },
+  "value_incr": null,
+  "units_incr": 5000,
+  "pct":        false
+}
+```
+
+- `value_incr` / `units_incr` — absolute amounts to add (positive or negative). Either can be null.
+- `pct: true` — treat as percentage of current slice total instead of absolute
+- Excludes `exclude_iters` rows from the source selection
+- Distributes increment proportionally across rows in the slice
+- Inserts rows tagged `iter = 'scale'`
+
+#### Recode
+`POST /api/versions/:id/recode`
+
+```json
+{
+  "pf_user": "paul.trowbridge",
+  "note":    "Part discontinued, replaced by new SKU",
+  "slice":   { "part": "OLD-SKU-001" },
+  "set":     { "part": "NEW-SKU-002" }
+}
+```
+
+- `set` — one or more dimension fields to replace (can swap multiple at once)
+- Inserts negative rows to zero out the original slice
+- Inserts positive rows with replaced dimension values
+- Both sets of rows share the same `logid` — undone together
+- Inserts rows tagged `iter = 'recode'`
+
+#### Clone
+`POST /api/versions/:id/clone`
+
+```json
+{
+  "pf_user": "paul.trowbridge",
+  "note":    "New customer win, similar profile to existing",
+  "slice":   { "customer": "EXISTING CO", "channel": "DIR" },
+  "set":     { "customer": "NEW CO" },
+  "scale":   0.75
+}
+```
+
+- `set` — dimension values to override on cloned rows
+- `scale` — optional multiplier on value/units (default 1.0)
+- Does not offset original slice
+- Inserts rows tagged `iter = 'clone'`
+
+### Audit & Undo
+
+| Method | Route | Description |
+|--------|-------|-------------|
+| GET | `/api/versions/:id/log` | List all log entries for a version, newest first |
+| DELETE | `/api/log/:logid` | Undo: delete all forecast rows with this logid, then delete log entry |
+
+---
+
+## Frontend (Web UI)
+
+### Navigation (sidebar)
+
+Three-step collapsible sidebar (200 px expanded / 48 px collapsed, state persisted to `localStorage`):
+
+1. **① Setup** — browse DB tables, register sources, configure col_meta, generate SQL. One-time admin task.
+2. **② Baseline** — create/manage versions, load baseline segments, timeline preview. One-time per version.
+3. **③ Forecast** — main working view: Perspective pivot + operation panel. Primary ongoing use.
+
+### Setup View (① Setup)
+
+- Left panel: DB table browser — all tables with row counts; click a table to open a preview modal (column list + sample rows)
+- Right panel: Registered sources list; click a source to open col_meta editor below
+- Col_meta editor: inline table — role dropdown per column, is_key checkbox, label text input, ordinal position
+- "Save" button — upserts col_meta; "Generate SQL" button — triggers generate-sql route, shows confirmation
+- "Register source" button available in the table preview modal
+- New columns default to role `dimension` on registration
+- Must generate SQL before a version can be created against this source
+
+### Baseline View (② Baseline)
+
+Source and version selectors at top. Version management inline: create new version (explains that a forecast table will be created), Close / Reopen / Delete buttons. Delete drops the forecast table and removes all version records.
+
+### Baseline Workbench
+
+A dedicated view for constructing the baseline for the selected version. The baseline is built from one or more **segments** — each segment is an independent query against the source table that appends rows to `iter = 'baseline'`. Segments are additive; clearing is explicit.
+
+**Layout:**
+```
+┌─────────────────────────────────────────────────────────────┐
+│  Baseline — [Version name]              [Clear Baseline]     │
+├─────────────────────────────────────────────────────────────┤
+│  Segments loaded (from log):                                 │
+│  ┌──────┬────────────────┬──────────┬───────┬──────────┐    │
+│  │  ID  │  Description   │  Rows    │  By   │  [Undo]  │    │
+│  └──────┴────────────────┴──────────┴───────┴──────────┘    │
+├─────────────────────────────────────────────────────────────┤
+│  Add Segment                                                 │
+│                                                              │
+│  Description  [_______________________________________]      │
+│                                                              │
+│  Date range   [date_from] to [date_to]  on [date col ▾]     │
+│  Date offset  [0] years  [0] months                         │
+│                                                              │
+│  Additional filters:                                         │
+│  [ + Add filter ]                                            │
+│  ┌──────────────────┬──────────┬──────────────┬───────┐     │
+│  │  Column          │  Op      │  Value(s)    │  [ x ]│     │
+│  └──────────────────┴──────────┴──────────────┴───────┘     │
+│                                                              │
+│  Preview: [projected month chips]                            │
+│                                                              │
+│  Note  [___________]          [Load Segment]                 │
+└─────────────────────────────────────────────────────────────┘
+```
+
+**Segments list** — shows all `operation = 'baseline'` log entries for this version, newest first. Each has an Undo button. Undo removes only that segment's rows (by logid), leaving other segments intact.
+
+**Clear Baseline** — deletes ALL `iter = 'baseline'` rows and all `operation = 'baseline'` log entries for this version. Prompts for confirmation. Used when starting over from scratch.
+
+**Add Segment form:**
+
+- **Description** — free text label stored as the log `note`, shown in the segments list
+- **Date offset** — years + months spinners; shifts the primary `role = 'date'` column forward on insert
+- **Filters** — one or more filter groups that define what rows to pull. Conditions within a group are AND-ed; groups are OR-ed. There is no separate "date range" section — period selection is just a filter like any other:
+  - Each group has a header row ("Group 1", "Group 2 — OR", …) and a `+ Add condition` link
+  - Within a group: Column (any `role = 'date'` or `role = 'filter'`), Operator (`=`, `!=`, `IN`, `NOT IN`, `BETWEEN`, `IS NULL`, `IS NOT NULL`), Value(s)
+  - Value inputs: `BETWEEN` → two date/text inputs; `IN`/`NOT IN` → comma-separated list; `=`/`!=` → single input; omitted for `IS NULL`/`IS NOT NULL`
+  - `+ Add OR group` button appends a new empty group below, joined by an "OR" separator label
+  - Groups with more than one condition render an "AND" badge between rows to make the logic explicit
+  - A group can be removed with `×` on its header (not available when only one group remains)
+  - At least one group with at least one condition is required to load a segment
+- **Manual WHERE clause** (admin only) — a toggle link ("Switch to manual SQL") that replaces the filter builder with a plain textarea. The admin types a raw PostgreSQL WHERE clause body (no `WHERE` keyword). Switching back to the builder clears the textarea. When active, the filter builder is hidden and the structured `filters` field is not sent; `raw_where` is sent instead. A prominent warning banner reads: "Raw SQL is not validated. You are responsible for correctness and security."
+- **Timeline preview** — rendered when any condition in any group is a `BETWEEN` or `=` on a `role = 'date'` column. Shows a horizontal bar (number-line style) for the source period and, if offset > 0, a second bar below for the projected period. Each bar shows start date on the left, end date on the right, duration in the centre. The two bars share the same visual width so the shift is immediately apparent. Not shown in manual WHERE mode or when no date condition is present.
+- **Note** — optional free text
+- **Load Segment** — submits; appends rows, does not clear existing baseline rows
+
+**Example — three-segment baseline:**
+
+| # | Description | Filter logic | Offset |
+|---|-------------|--------------|--------|
+| 1 | All orders taken 6/1/25–3/31/26 | `order_date BETWEEN 2025-06-01 AND 2026-03-31` | 0 |
+| 2 | Open or unshipped orders (status missing or explicit) | `(status IN ('OPEN','PENDING')) OR (status IS NULL)` | 0 |
+| 3 | Prior year book-and-ship 4/1/25–5/31/25 | `order_date BETWEEN 2025-04-01 AND 2025-05-31 AND ship_date BETWEEN 2025-04-01 AND 2025-05-31` | 0 |
+
+Segment 2 uses two OR groups; segment 3 has two AND conditions in one group. Any combination is valid as long as at least one group with at least one condition is present.
+
+### Forecast View
+
+**Layout:**
+```
+┌─────────────────────────────────────────────────────────────────┐
+│  [Version label]  [Refresh]  [Save layout]  [Reset layout]       │
+├──────────────────────────────────────┬──────────────────────────┤
+│                                      │                           │
+│  Perspective Viewer                  │  Operation Panel          │
+│  (interactive pivot web component)   │  (active when slice set)  │
+│                                      │                           │
+│                                      │  Slice:                   │
+│                                      │    channel = WHS          │
+│                                      │    geography = WEST       │
+│                                      │                           │
+│                                      │  [ Scale ] [ Recode ]     │
+│                                      │  [ Clone ]                │
+│                                      │                           │
+│                                      │  ... operation form ...   │
+│                                      │                           │
+│                                      │  [ Submit ]               │
+│                                      │                           │
+└──────────────────────────────────────┴──────────────────────────┘
+```
+
+**Pivot control:** [Perspective](https://perspective.finos.org/) 4.4.0, loaded from CDN at runtime. Data is fetched from `GET /api/versions/:id/data` as an Arrow IPC binary stream and loaded into an in-browser Perspective worker — Perspective's native ingestion path. Supports grouping, splitting, filtering, sorting, and charting interactively. Layout (group_by, split_by, filters, plugin) is saved per version to `localStorage` via Save layout / Reset layout buttons.
+
+**Large-dataset loading sequence:**
+1. Client issues `GET /api/versions/:id/data`
+2. Server responds with `X-Row-Count` header and begins streaming Arrow record batches
+3. If `X-Row-Count` ≥ 500 000, UI shows a non-blocking loading banner; otherwise no indicator
+4. Client calls `worker.table(firstBatch)` on the first batch to make the pivot interactive immediately
+5. Each subsequent batch is applied with `pspTable.update(batch)` as it arrives
+6. Banner clears when the stream closes
+
+**Interaction flow:**
+1. Click a cell or row in the pivot — the `perspective-click` event fires
+2. `detail.config.filter` from the event is parsed: only `==` filters on `role = dimension` columns are extracted as the slice
+3. Slice populates the Operation Panel — pick operation tab, fill in parameters
+4. Submit → POST to API → new rows returned via `RETURNING *` are streamed directly into the Perspective table (`pspTable.update(rows)`) — no full reload needed
+5. For recode, both the negative offset rows and positive replacement rows are returned and streamed
+
+**Pivot default layout:** built from col_meta — first two `dimension` columns as `group_by`, `date` column as `split_by`. User can rearrange in Perspective settings panel and save.
+
+**Reference rows** (`pf_iter = 'reference'`) are visible in the pivot for comparison context. Operations never affect them (enforced by `exclude_iters` in the version).
+
+### Log View
+
+AG Grid list of log entries — user, timestamp, operation, slice, note, rows affected.
+"Undo" button per row → `DELETE /api/log/:logid` → grid and pivot refresh (full reload of Perspective table).
+
+---
+
+## Forecast SQL Patterns
+
+Column names baked in at generation time. Tokens substituted at request time. Metadata columns are `pf_iter`, `pf_logid`, `pf_user`, `pf_created_at`.
+
+**Units conditionality:** `{units_col}` appears in INSERT column lists and SELECT expressions only when a `units` role is configured in col_meta. The SQL generator omits it entirely otherwise — no placeholder column, no zero-fill.
+
+**dim_period JOIN:** when any `dimension` column has `dim_period_col` set (and its group's `date` key has `is_key = true`), the FROM clause becomes `{schema}.{tname} s JOIN pf.dim_period dp ON dp.drange @> (s.{date_col} + '{{date_offset}}'::interval)::date`. Those dimension columns are selected as `dp.{dim_period_col} AS {col}` instead of `s.{col}`.
+
+### Baseline Load (one segment)
+
+```sql
+WITH ilog AS (
+    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note)
+    VALUES ({{version_id}}, '{{pf_user}}', 'baseline', NULL, '{{params}}'::jsonb, '{{note}}')
+    RETURNING id
+)
+,ins AS (
+    INSERT INTO {{fc_table}} (
+        {dimension_cols}, {date_col}, {value_col} [, {units_col}],
+        pf_iter, pf_logid, pf_user, pf_created_at
+    )
+    SELECT
+        {dimension_cols},
+        ({date_col} + '{{date_offset}}'::interval)::date,
+        {value_col} [, {units_col}],
+        'baseline', (SELECT id FROM ilog), '{{pf_user}}', now()
+    FROM
+        {schema}.{tname}  -- or with dim_period JOIN (see above)
+    WHERE
+        {{filter_clause}}
+    RETURNING *
+)
+SELECT count(*) AS rows_affected FROM ins
+```
+
+Baseline loads are **additive** — no DELETE before INSERT. Each segment appends independently.
+
+Token details:
+- `{{date_offset}}` — PostgreSQL interval string (e.g. `1 year`); defaults to `0 days`; applied only to the primary `role = 'date'` column on insert
+- `{{filter_clause}}` — built from `filters` or `raw_where` at request time (not baked into stored SQL since conditions vary per segment).
+  - Structured path (`filters`): each group becomes a parenthesized AND block; groups are joined with `OR`. Every column is validated against col_meta (`role = 'date'` or `role = 'filter'`). Values are escaped (single quotes doubled). Supported operators: `=`, `!=`, `IN`, `NOT IN`, `BETWEEN`, `IS NULL`, `IS NOT NULL`.
+  - Raw path (`raw_where`): the string is injected verbatim. No col_meta validation. Admin-only.
+
+### Clear Baseline
+
+Two queries, run in a transaction:
+```sql
+DELETE FROM {{fc_table}} WHERE pf_iter = 'baseline';
+DELETE FROM pf.log WHERE version_id = {{version_id}} AND operation = 'baseline';
+```
+
+### Reference Load
+
+```sql
+WITH ilog AS (
+    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note)
+    VALUES ({{version_id}}, '{{pf_user}}', 'reference', NULL, '{{params}}'::jsonb, '{{note}}')
+    RETURNING id
+)
+,ins AS (
+    INSERT INTO {{fc_table}} (
+        {dimension_cols}, {date_col}, {value_col} [, {units_col}],
+        pf_iter, pf_logid, pf_user, pf_created_at
+    )
+    SELECT
+        {dimension_cols}, {date_col}, {value_col} [, {units_col}],
+        'reference', (SELECT id FROM ilog), '{{pf_user}}', now()
+    FROM
+        {schema}.{tname}  -- or with dim_period JOIN (see above)
+    WHERE
+        {{filter_clause}}
+    RETURNING *
+)
+SELECT count(*) AS rows_affected FROM ins
+```
+
+No date offset applied — reference rows land at their original dates for prior-period comparison. Same dim_period JOIN logic applies as baseline.
+
+### Scale
+
+```sql
+WITH ilog AS (
+    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note)
+    VALUES ({{version_id}}, '{{pf_user}}', 'scale', '{{slice}}'::jsonb, '{{params}}'::jsonb, '{{note}}')
+    RETURNING id
+)
+,base AS (
+    SELECT
+        {dimension_cols}, {date_col},
+        {value_col} [, {units_col}],
+        sum({value_col}) OVER () AS total_value
+        [, sum({units_col}) OVER () AS total_units]
+    FROM {{fc_table}}
+    WHERE {{where_clause}}
+    {{exclude_clause}}
+)
+,ins AS (
+    INSERT INTO {{fc_table}} (
+        {dimension_cols}, {date_col}, {value_col} [, {units_col}],
+        pf_iter, pf_logid, pf_user, pf_created_at
+    )
+    SELECT
+        {dimension_cols}, {date_col},
+        round(({value_col} / NULLIF(total_value, 0)) * {{value_incr}}, 2)
+        [, round(({units_col} / NULLIF(total_units, 0)) * {{units_incr}}, 5)],
+        'scale', (SELECT id FROM ilog), '{{pf_user}}', now()
+    FROM base
+    RETURNING *
+)
+SELECT * FROM ins
+```
+
+`{{value_incr}}` / `{{units_incr}}` are pre-computed in JS when `pct: true` (multiply slice total by pct). Units expressions are omitted when no units column is configured.
+
+### Recode
+
+```sql
+WITH ilog AS (
+    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note)
+    VALUES ({{version_id}}, '{{pf_user}}', 'recode', '{{slice}}'::jsonb, '{{params}}'::jsonb, '{{note}}')
+    RETURNING id
+)
+,src AS (
+    SELECT {dimension_cols}, {date_col}, {value_col} [, {units_col}]
+    FROM {{fc_table}}
+    WHERE {{where_clause}}
+    {{exclude_clause}}
+)
+,neg AS (
+    INSERT INTO {{fc_table}} ({dimension_cols}, {date_col}, {value_col} [, {units_col}], pf_iter, pf_logid, pf_user, pf_created_at)
+    SELECT {dimension_cols}, {date_col}, -{value_col} [, -{units_col}], 'recode', (SELECT id FROM ilog), '{{pf_user}}', now()
+    FROM src
+    RETURNING *
+)
+,ins AS (
+    INSERT INTO {{fc_table}} ({dimension_cols}, {date_col}, {value_col} [, {units_col}], pf_iter, pf_logid, pf_user, pf_created_at)
+    SELECT {{set_clause}}, {date_col}, {value_col} [, {units_col}], 'recode', (SELECT id FROM ilog), '{{pf_user}}', now()
+    FROM src
+    RETURNING *
+)
+SELECT * FROM neg UNION ALL SELECT * FROM ins
+```
+
+`{{set_clause}}` replaces the listed dimension columns with new values, passes others through unchanged. Both the negative (zero-out) and positive (replacement) rows share the same `pf_logid` and are undone together.
+
+### Clone
+
+```sql
+WITH ilog AS (
+    INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note)
+    VALUES ({{version_id}}, '{{pf_user}}', 'clone', '{{slice}}'::jsonb, '{{params}}'::jsonb, '{{note}}')
+    RETURNING id
+)
+,ins AS (
+    INSERT INTO {{fc_table}} ({dimension_cols}, {date_col}, {value_col} [, {units_col}], pf_iter, pf_logid, pf_user, pf_created_at)
+    SELECT
+        {{set_clause}}, {date_col},
+        round({value_col} * {{scale_factor}}, 2)
+        [, round({units_col} * {{scale_factor}}, 5)],
+        'clone', (SELECT id FROM ilog), '{{pf_user}}', now()
+    FROM {{fc_table}}
+    WHERE {{where_clause}}
+    {{exclude_clause}}
+    RETURNING *
+)
+SELECT * FROM ins
+```
+
+### Undo
+
+Two queries run sequentially (not in a CTE — FK ordering):
+
+```sql
+DELETE FROM {{fc_table}} WHERE pf_logid = {{logid}};
+DELETE FROM pf.log WHERE id = {{logid}};
+```
+
+---
+
+## Display-grain pre-aggregation
+
+**Status:** built (static grain). This is **Path B** (pre-aggregated extract →
+native Perspective table) of two candidate designs; rationale, the Path A
+alternative (live virtual-server aggregation), the spike evidence, and the
+A-vs-B trade-off live in `pf_perspective_options.md` (§Two candidate designs,
+§Spike findings).
+
+The grain is **static** — set once per source in Setup and baked into the stored
+`pf.sql` templates, so load and operations agree by construction. `in_grain`
+means *eligible for the grain*, and in this version the grain is exactly the set
+of eligible columns. Deriving a narrower grain per pivot at request time (the
+dynamic variant) is then additive: intersect the viewer's field set with the
+eligible set. Leaving high-cardinality columns (`part`, raw day dates,
+currency-level detail) unflagged is what keeps the grain from exploding back
+toward raw, regardless of what a user drags into the pivot.
+
+**Measured on `pf.fc_osm_stack_20`** at `pending_rep × customer × smon`:
+534,902 → **6,154** rows (≈87×), `pf_gkey` unique across all 6,154, and both
+measures reconcile exactly to the raw totals (283,296,087.67 / 962,142,261.46).
+
+**Problem it solves.** The current transport ships every raw forecast row to the
+browser (≈535k rows / ~250 MB / ~2 min on `osm_stack`). Perspective then pivots
+them in WASM. Pre-aggregating server-side to the grain the pivot actually
+displays collapses the payload dramatically — measured 534,902 → **4,642 rows**
+at `rep × customer × month` (≈115×) — while keeping Perspective's **native**
+engine, so expand/collapse/depth/sort/filter all keep working (unlike the DuckDB
+virtual-server path, which loses them).
+
+### Grain definition — `pf.col_meta.in_grain`
+
+Add a boolean `in_grain` to `pf.col_meta`. At **Generate SQL** time the flagged
+columns (plus `pf_iter`, always) define the display grain; `value`/`units`
+columns are the additive measures summed to that grain. The grain is baked into
+the stored `pf.sql` templates, so initial load and operations agree on it.
+
+- A date column enters the grain at **period** resolution (via `dim_period_col`,
+  e.g. month `sdat`), not raw date.
+- **Constraint — additive measures only.** Sums (sales, qty, units) are exact.
+  Averages must ship as `sum` + `count` and be derived; `count(DISTINCT …)`
+  cannot be pre-aggregated. Forecast measures are sums, so this holds.
+- Dimensions **not** in the grain are unavailable for pivot/filter on that view.
+  Either include every dimension users pivot on (cardinality permitting — a
+  high-cardinality dim like `part` explodes the grain back toward raw), or
+  re-fetch at a new grain when the user changes `group_by`/`split_by`/`filter`
+  (each change = one ~25 ms aggregate returning a small native dataset).
+
+### Initial load — `GET /api/versions/:id/agg`
+
+Replaces the raw `/data` stream for grain-based versions. Aggregates the forecast
+table to the stored grain and returns Arrow IPC. The template is stored in
+`pf.sql` as operation `get_agg`, generated only when a grain is defined; clearing
+the grain and regenerating removes it, and the client falls back to `/data`.
+Both endpoints speak the same protocol (one record batch plus an `X-Row-Count`
+header), so the client only chooses the URL:
+
+```sql
+SELECT
+     {{grain_cols}}
+    ,pf_iter
+    ,pf_logid
+    ,{{grain_key}}                       AS pf_gkey   -- synthetic index, see below
+    ,SUM({{value_col}})                  AS {{value_col}}
+    ,SUM({{units_col}})                  AS {{units_col}}
+FROM {{fc_table}}
+GROUP BY {{grain_cols}}, pf_iter, pf_logid
+```
+
+Client loads the result into a native `worker.table(buffer, { index: 'pf_gkey' })`.
+The Perspective **view** sums measures across these rows for whatever the user
+pivots on — exactly as an Excel pivot cache sums the data tab.
+
+**Model — append deltas, let the view sum (the Excel pivot-cache pattern).** This
+is a direct port of the prior Excel workflow: pre-aggregated rows go to the table,
+the pivot cuts them locally, and each adjustment is *appended* — never a
+recomputed total. So rows are aggregated to grain **per `pf_logid`** (one row per
+grain × `pf_iter` × originating log entry), and the view sums them. This is also
+the smallest change from today's code, which already appends operation results via
+`table.update()` and lets the view sum — we just feed pre-aggregated rows.
+
+**Synthetic key.** Perspective's index is a single column, so emit
+`pf_gkey = {{grain_cols}} || pf_iter || pf_logid` (concatenated). Including
+`pf_logid` keeps each operation's contribution a **distinct** row, so appends
+accumulate (rather than replacing a bucket) and a delete can remove exactly that
+operation's rows.
+
+As built, the concatenation is
+`concat_ws(chr(31), COALESCE(col::text, chr(30)), …, pf_iter, pf_logid::text)`.
+The separator and NULL sentinel matter: plain `concat_ws` skips NULLs, so
+`('a', NULL)` and `(NULL, 'a')` would produce the same key and silently merge two
+groups into one indexed row. `chr(30)` also keeps NULL distinct from `''`.
+
+### Write path (scale / recode / clone) — append the new log entry's rows
+
+Operations INSERT raw rows into `{{fc_table}}` under a new `pf_logid` as today; the
+final CTE returns just **that log entry's rows aggregated to grain**:
+
+```sql
+WITH
+ins AS (
+    INSERT INTO {{fc_table}} ( … ) SELECT … RETURNING {{grain_cols}}, pf_iter, pf_logid
+)
+SELECT
+     {{grain_cols}}
+    ,pf_iter
+    ,pf_logid
+    ,{{grain_key}}      AS pf_gkey
+    ,SUM({{value_col}}) AS {{value_col}}
+    ,SUM({{units_col}}) AS {{units_col}}
+FROM ins
+GROUP BY {{grain_cols}}, pf_iter, pf_logid
+```
+
+Client applies `table.update(rows)` — these are new `pf_gkey`s, so they append and
+the view re-sums. No bucket-total recomputation. (Aggregating `FROM ins` is safe
+because each row carries the new, unique `pf_logid`.)
+
+### Delete path (undo) — remove that log entry's rows
+
+A logid's rows are uniquely keyed, so undo just removes them and lets the view
+re-sum — no re-aggregation, no emptied-bucket handling, no snapshot caveat:
+
+`RETURNING` does not accept `DISTINCT`, so the delete feeds a CTE that reduces its
+output to the distinct grain keys. `rows_deleted` still counts raw rows removed:
+
+```sql
+WITH
+del AS (
+    DELETE FROM {{fc_table}}
+    WHERE pf_logid = {{logid}}
+    RETURNING {{grain_cols}}, pf_iter, pf_logid
+)
+SELECT
+     count(*)::int                    AS rows_deleted
+    ,array_agg(DISTINCT {{grain_key}}) AS pf_gkeys
+FROM del;
+DELETE FROM pf.log WHERE id = {{logid}};
+```
+
+Client applies `table.remove(pf_gkeys)`. (A wholesale re-pull of `/agg` is an even
+simpler fallback and is now cheap — ~25 ms.)
+
+### What this preserves / changes
+
+- **Preserves:** native Perspective interactivity, layout persistence, and the
+  slice-click → operation flow (a clicked grain cell still maps to a `WHERE` on
+  raw rows). pg raw rows remain the single source of truth.
+- **Changes:** operation responses return grain-aggregated rows for the new
+  `pf_logid` (not raw `RETURNING *`); `/data` is superseded by `/agg` for grain
+  versions; the table is indexed by `pf_gkey` instead of `pf_id`; undo becomes a
+  targeted `table.remove()` and the "full reload" wart disappears.
+- **Lineage:** this is a port of the prior Excel model — pre-agg rows → pivot
+  cache → double-click slice → VBA adjustment UI → append incremental rows →
+  refresh cache. The append-and-sum write/undo model mirrors that directly.
+
+---
+
+## Admin Setup Flow (end-to-end)
+
+1. Open **Sources** view → browse DB tables → register source table
+2. Open col_meta editor → assign roles to columns (`dimension`, `value`, `units`, `date`, `filter`, `ignore`), mark is_key dimensions, set labels
+3. Click **Generate SQL** → app writes operation SQL to `pf.sql`
+4. Open **Versions** view → create a named version (sets `exclude_iters`, creates forecast table)
+5. Open **Baseline Workbench** → build the baseline from one or more segments:
+   - Each segment specifies a date range (on any date/filter column), date offset, and optional additional filter conditions
+   - Add segments until the baseline is complete; each is independently undoable
+   - Use "Clear Baseline" to start over if needed
+6. Optionally load **Reference** → pick prior period date range → inserts `iter = 'reference'` rows at their original dates (for comparison in the pivot)
+7. Open **Forecast** view → share with users
+
+## User Forecast Flow (end-to-end)
+
+1. Open **Forecast** view → select version
+2. Pivot loads — explore data, identify slice to adjust
+3. Select cells → Operation Panel populates with slice
+4. Choose operation → fill in parameters → Submit
+5. Grid refreshes — adjustment visible immediately
+6. Repeat as needed
+7. Admin closes version when forecasting is complete
+
+---
+
+## Open Questions / Future Scope
+
+- **Display-grain pre-aggregation** — ship the pivot at its display grain instead of raw rows; see §Display-grain pre-aggregation above and `pf_perspective_options.md`. This is the chosen direction for the load-time + interactivity problem.
+- **Baseline replay** — re-execute change log against a restated baseline (`replay: true`); v1 returns 501
+- **Approval workflow** — user submits, admin approves before changes are visible to others (deferred)
+- **Territory filtering** — restrict what a user can see/edit by dimension value (deferred)
+- **Export** — download forecast as CSV or push results to a reporting table
+- **Version comparison** — side-by-side view of two versions (facilitated by isolated tables via UNION)
+- **Col meta / version schema drift** — if col_meta roles are changed after a version's forecast table is already created, the generated SQL and the table DDL go out of sync. UI should detect this: compare col_meta against the forecast table's actual columns via `information_schema`, warn the user, and offer to rebuild the version (drop + recreate table, preserving the version record and log). Workaround: delete and recreate the version manually.
+- **Multi-connection support** — currently one DB via `.env`. Full vision: `pf.connection` table (host, port, dbname, user, password as env-var ref), `connection_id` on `pf.source`, per-connection pg pools at runtime. `pf` schema stays on a "home" connection; source data can live anywhere. Connections UI in Setup. Safe to defer while in dev — requires clean reinstall when added since it changes the source schema.
+
+---
+
+## Project Status — 2026-06-12
+
+### What's working
+- Full backend: source registration, col_meta, SQL generation, versions, baseline segments, reference load, scale, recode, clone, undo
+- `units` column is optional — sources without a units column register and generate SQL correctly
+- `dim_group` / `dim_period_col` on col_meta: baseline/reference load JOINs `pf.dim_period` to derive fiscal/calendar period columns rather than copying them raw from the source
+- `pf.dim_period` calendar table (2018–2035): populated by `setup_sql/gen_dim_period.sql`, configurable fiscal year start
+- React + Vite + Tailwind CSS frontend in `ui/`, built output to `public/app/`, served by Express
+- Data transport: Arrow IPC binary stream (`GET /api/versions/:id/data`); server accumulates all rows into one record batch; client hands buffer directly to Perspective WASM
+- 3-step collapsible sidebar (Setup / Baseline / Forecast)
+- Setup view: DB table browser with preview modal, source registration, col_meta editor (`dim_group`/`dim_period_col` fields included), SQL generation
+- Baseline view: version management (create/close/reopen/delete), multi-segment baseline workbench, canvas timeline, filter builder
+- Perspective pivot in Forecast view: loads all version rows, interactive group/split/filter/chart, layout saved per version to localStorage
+- Slice extraction from `perspective-click` event feeds operation panel directly
+- Incremental row streaming: operation results (`RETURNING *`) applied to Perspective table via `pspTable.update()` — no full reload
+- Status bar: shows current source · version · baseline row count · status
+
+### Known issues / next focus
+
+- **Forecast view** — operation panel SQL generation complete; UI wiring to API still needed
+- **Load progress bar** — jittery at high throughput; throttle to ~10 updates/sec
+- **Default pivot layout** — per-source configurable layout not yet implemented; currently hardcodes first 2 dimensions
+- **No "current version" persistence** — source/version selection resets on page reload
+- **Perspective slice limitation** — computed date columns (Month, YearDate) from split_by don't map back to raw rows; only native dimension columns work for slice extraction
+- **Col_meta / version schema drift** — if col_meta changes after a version's forecast table is created, SQL and DDL go out of sync. Workaround: delete and recreate the version.
+

routes/auth.js

@@ -1,110 +0,0 @@
-const express = require('express');
-const { verifyPassword } = require('../lib/auth');
-
-// Per-IP login throttle. In-memory on purpose: it only has to blunt online
-// guessing, and a counter that resets on restart is the acceptable cost of not
-// writing a failed-attempt row for every knock on an internet-facing port.
-const WINDOW_MS   = 15 * 60 * 1000;
-const MAX_ATTEMPTS = 10;
-const attempts = new Map();   // ip -> { count, resetAt }
-
-function tooManyAttempts(ip) {
-    const rec = attempts.get(ip);
-    if (!rec || Date.now() > rec.resetAt) return false;
-    return rec.count >= MAX_ATTEMPTS;
-}
-
-function recordFailure(ip) {
-    const rec = attempts.get(ip);
-    if (!rec || Date.now() > rec.resetAt) {
-        attempts.set(ip, { count: 1, resetAt: Date.now() + WINDOW_MS });
-    } else {
-        rec.count += 1;
-    }
-}
-
-// Keep the map from growing without bound on a long-lived process.
-setInterval(() => {
-    const now = Date.now();
-    for (const [ip, rec] of attempts) if (now > rec.resetAt) attempts.delete(ip);
-}, WINDOW_MS).unref();
-
-module.exports = function(pool) {
-    const router = express.Router();
-
-    router.post('/login', async (req, res) => {
-        const ip = req.ip;
-        if (tooManyAttempts(ip)) {
-            return res.status(429).json({ error: 'Too many failed attempts. Try again later.' });
-        }
-
-        const username = String(req.body?.username || '').trim();
-        const password = String(req.body?.password || '');
-        if (!username || !password) {
-            return res.status(400).json({ error: 'Username and password are required' });
-        }
-
-        try {
-            const result = await pool.query(
-                `SELECT id, username, display_name, pass_hash, is_active,
-                        is_admin, territory
-                 FROM pf.app_user WHERE lower(username) = lower($1)`,
-                [username]
-            );
-            const user = result.rows[0];
-
-            // Same message and roughly the same work either way: no unknown
-            // user / wrong password / disabled distinction to enumerate.
-            const ok = user && user.is_active && verifyPassword(password, user.pass_hash);
-            if (!ok) {
-                recordFailure(ip);
-                return res.status(401).json({ error: 'Invalid username or password' });
-            }
-
-            // New session id on login — an existing cookie can't be fixated.
-            req.session.regenerate(err => {
-                if (err) {
-                    console.error(err);
-                    return res.status(500).json({ error: 'Could not start session' });
-                }
-                req.session.user = {
-                    id:           user.id,
-                    username:     user.username,
-                    display_name: user.display_name,
-                    is_admin:     !!user.is_admin,
-                    territory:    Array.isArray(user.territory) ? user.territory : [],
-                };
-                pool.query(`UPDATE pf.app_user SET last_login_at = now() WHERE id = $1`, [user.id])
-                    .catch(e => console.error('last_login_at update failed', e));
-                req.session.save(err2 => {
-                    if (err2) {
-                        console.error(err2);
-                        return res.status(500).json({ error: 'Could not start session' });
-                    }
-                    attempts.delete(ip);
-                    res.json({ user: req.session.user });
-                });
-            });
-        } catch (err) {
-            console.error(err);
-            res.status(500).json({ error: err.message });
-        }
-    });
-
-    router.post('/logout', (req, res) => {
-        const name = req.session?.cookie && req.app.get('session cookie name');
-        req.session.destroy(err => {
-            if (err) console.error(err);
-            res.clearCookie(name || 'pf.sid');
-            res.json({ ok: true });
-        });
-    });
-
-    // The UI calls this on load to decide between the login screen and the app.
-    router.get('/me', (req, res) => {
-        if (!req.session?.user) return res.status(401).json({ error: 'Not authenticated' });
-        res.json({ user: req.session.user });
-    });
-
-    return router;
-};

routes/layouts.js

@@ -1,193 +0,0 @@
-const express = require('express');
-const { sessionUser } = require('../lib/auth');
-
-// Named Perspective view configs. Two kinds, one table:
-//
-//   private    — yours, nobody else lists it
-//   published  — everyone on the version sees it; only the owner or an admin
-//                may change it, the same rule pf.log uses for its entries
-//
-// Scope is the version by default, because that is where people enter the app.
-// A layout with version_id NULL applies to every version of the source, which
-// is where the old pf.source.default_layout went and what a brand-new version
-// picks up before anyone has published anything for it.
-module.exports = function(pool) {
-    const router = express.Router();
-
-    const SELECT_COLS = `id, source_id, version_id, name, config, owner,
-                         visibility, is_default, created_at, updated_at`;
-
-    // What the client needs to know about a row it cannot write, so the UI can
-    // grey the control out rather than offer a click that answers 403.
-    const decorate = (row, req) => ({
-        ...row,
-        scope:    row.version_id == null ? 'source' : 'version',
-        can_edit: !!req.session?.user?.is_admin || row.owner === sessionUser(req)
-    });
-
-    // The owner/admin gate, shared by every write. Returns the row, or sends the
-    // response itself and returns null.
-    async function ownedLayout(req, res) {
-        const id = parseInt(req.params.lid);
-        const { rows } = await pool.query(
-            `SELECT ${SELECT_COLS} FROM pf.layout WHERE id = $1`, [id]
-        );
-        if (!rows.length) { res.status(404).json({ error: 'Layout not found' }); return null; }
-        const row = rows[0];
-        if (!req.session?.user?.is_admin && row.owner !== sessionUser(req)) {
-            res.status(403).json({
-                error: `“${row.name}” belongs to ${row.owner} — only they or an administrator can change it`
-            });
-            return null;
-        }
-        return row;
-    }
-
-    // Everything applicable to a version: its own published layouts, the
-    // source-wide published ones, and the caller's own private layouts at either
-    // scope. Deliberately not territory-filtered — a layout is display config,
-    // and territory restricts rows, not columns.
-    router.get('/versions/:id/layouts', async (req, res) => {
-        const versionId = parseInt(req.params.id);
-        try {
-            const ver = await pool.query(
-                `SELECT source_id FROM pf.version WHERE id = $1`, [versionId]
-            );
-            if (!ver.rows.length) return res.status(404).json({ error: 'Version not found' });
-            const sourceId = ver.rows[0].source_id;
-
-            const { rows } = await pool.query(
-                `SELECT ${SELECT_COLS}
-                 FROM pf.layout
-                 WHERE TRUE
-                   AND source_id = $1
-                   AND (version_id = $2 OR version_id IS NULL)
-                   AND (visibility = 'published' OR owner = $3)
-                 ORDER BY visibility DESC, is_default DESC, lower(name)`,
-                [sourceId, versionId, sessionUser(req)]
-            );
-            res.json(rows.map(r => decorate(r, req)));
-        } catch (err) {
-            res.status(500).json({ error: err.message });
-        }
-    });
-
-    // Create. scope 'source' saves it against every version of the source;
-    // anything else is this version.
-    router.post('/versions/:id/layouts', async (req, res) => {
-        const versionId = parseInt(req.params.id);
-        const { name, config, visibility = 'private', scope = 'version' } = req.body || {};
-        if (!name || !String(name).trim()) return res.status(400).json({ error: 'Name is required' });
-        if (!config) return res.status(400).json({ error: 'Config is required' });
-        if (!['private', 'published'].includes(visibility)) {
-            return res.status(400).json({ error: 'visibility must be private or published' });
-        }
-        try {
-            const ver = await pool.query(
-                `SELECT source_id FROM pf.version WHERE id = $1`, [versionId]
-            );
-            if (!ver.rows.length) return res.status(404).json({ error: 'Version not found' });
-
-            const { rows } = await pool.query(
-                `INSERT INTO pf.layout (source_id, version_id, name, config, owner, visibility)
-                 VALUES ($1, $2, $3, $4, $5, $6)
-                 RETURNING ${SELECT_COLS}`,
-                [ver.rows[0].source_id, scope === 'source' ? null : versionId,
-                 String(name).trim(), config, sessionUser(req), visibility]
-            );
-            res.json(decorate(rows[0], req));
-        } catch (err) {
-            // the partial unique indexes, reported in the terms the user typed
-            if (err.code === '23505') {
-                return res.status(409).json({ error: `A layout named “${name}” already exists here` });
-            }
-            res.status(500).json({ error: err.message });
-        }
-    });
-
-    // Update any of name / config / visibility / is_default. Each is optional;
-    // a missing key leaves the column alone, which is why the flag-and-value
-    // pair is used rather than COALESCE on the value.
-    router.patch('/layouts/:lid', async (req, res) => {
-        const { name, config, visibility, is_default, scope } = req.body || {};
-        if (name === undefined && config === undefined && visibility === undefined
-            && is_default === undefined && scope === undefined) {
-            return res.status(400).json({ error: 'Nothing to update' });
-        }
-        if (visibility !== undefined && !['private', 'published'].includes(visibility)) {
-            return res.status(400).json({ error: 'visibility must be private or published' });
-        }
-        const client = await pool.connect();
-        try {
-            const row = await ownedLayout(req, res);
-            if (!row) return;
-
-            // Unpublishing a default would leave a default nobody can see, which
-            // the table's CHECK refuses; clear the flag with it rather than
-            // failing on a constraint the user never mentioned.
-            const clearsDefault = visibility === 'private';
-
-            // One default per scope, enforced by a partial unique index. Stand
-            // the others down first rather than letting the update collide --
-            // and in one transaction, or a name clash on the second statement
-            // leaves the version with no default at all.
-            const target_version = scope === undefined ? row.version_id
-                                 : (scope === 'source' ? null : row.version_id);
-            await client.query('BEGIN');
-            if (is_default === true) {
-                await client.query(
-                    `UPDATE pf.layout SET is_default = false
-                     WHERE TRUE
-                       AND source_id = $1
-                       AND COALESCE(version_id, 0) = COALESCE($2::int, 0)
-                       AND id <> $3
-                       AND is_default`,
-                    [row.source_id, target_version, row.id]
-                );
-            }
-
-            const { rows } = await client.query(
-                `UPDATE pf.layout SET
-                     name       = CASE WHEN $2::bool THEN $3::text  ELSE name       END,
-                     config     = CASE WHEN $4::bool THEN $5::jsonb ELSE config     END,
-                     visibility = CASE WHEN $6::bool THEN $7::text  ELSE visibility END,
-                     is_default = CASE WHEN $10::bool THEN false
-                                       WHEN $8::bool  THEN $9::bool ELSE is_default END,
-                     version_id = CASE WHEN $11::bool THEN $12::int ELSE version_id END,
-                     updated_at = now()
-                 WHERE id = $1
-                 RETURNING ${SELECT_COLS}`,
-                [row.id,
-                 name !== undefined,       name !== undefined ? String(name).trim() : null,
-                 config !== undefined,     config !== undefined ? config : null,
-                 visibility !== undefined, visibility !== undefined ? visibility : null,
-                 is_default !== undefined, is_default === true,
-                 clearsDefault,
-                 scope !== undefined,      target_version]
-            );
-            await client.query('COMMIT');
-            res.json(decorate(rows[0], req));
-        } catch (err) {
-            await client.query('ROLLBACK').catch(() => {});
-            if (err.code === '23505') {
-                return res.status(409).json({ error: 'A layout with that name already exists here' });
-            }
-            res.status(500).json({ error: err.message });
-        } finally {
-            client.release();
-        }
-    });
-
-    router.delete('/layouts/:lid', async (req, res) => {
-        try {
-            const row = await ownedLayout(req, res);
-            if (!row) return;
-            await pool.query(`DELETE FROM pf.layout WHERE id = $1`, [row.id]);
-            res.json({ deleted: row.id });
-        } catch (err) {
-            res.status(500).json({ error: err.message });
-        }
-    });
-
-    return router;
-};

routes/log.js

@@ -1,6 +1,5 @@
 const express = require('express');
 const { grainOf } = require('../lib/sql_generator');
-const { sessionUser } = require('../lib/auth');
 const { fcTable } = require('../lib/utils');
 
 module.exports = function(pool) {
@@ -31,96 +30,17 @@ module.exports = function(pool) {
                 unitsCol ? `sum(f."${unitsCol}")::float8 AS units_total` : `NULL::float8 AS units_total`
             ].join(', ');
 
-            // The totals are stamped onto the entry when it is written, so the
-            // normal read is a scan of a few dozen log rows rather than a join
-            // against millions of forecast rows.
-            //
-            // ?recount=1 does it the old way. Stored totals are fixed at write
-            // time and cannot drift on their own, but nothing stops someone
-            // deleting forecast rows by hand, and a stored figure has no way to
-            // notice. This is the way back -- and the backfill for entries
-            // written before the columns existed.
-            const recount = req.query.recount === '1' || req.query.recount === 'true';
-            const stamped = !recount && (await pool.query(
-                `SELECT count(*)::int AS n FROM pf.log
-                 WHERE version_id = $1 AND row_count IS NULL`, [versionId]
-            )).rows[0].n === 0;
-
-            // ?kind=adjustments drops the baseline and reference entries. That is not
-            // only about what gets listed: the aggregate below joins the whole
-            // forecast table, and on a real version the load entries own almost
-            // every row of it -- 2.5M against a few thousand for the adjustments.
-            // Filtering in the WHERE keeps them out of the join rather than
-            // totalling them and discarding the answer.
-            const adjustmentsOnly = req.query.kind === 'adjustments';
-            const opFilter = adjustmentsOnly
-                ? `AND l.operation NOT IN ('baseline', 'reference')`
-                : '';
-
-            const result = stamped
-                ? await pool.query(`
-                    SELECT l.*,
-                           $2::text AS value_col,
-                           $3::text AS units_col
-                    FROM pf.log l
-                    WHERE l.version_id = $1
-                    ${opFilter}
-                    ORDER BY l.id DESC
-                `, [versionId, valueCol || null, unitsCol || null])
-                : await pool.query(`
-                    SELECT l.*, ${aggCols},
-                           $2::text AS value_col,
-                           $3::text AS units_col
-                    FROM pf.log l
-                    LEFT JOIN ${table} f ON f.pf_logid = l.id
-                    WHERE l.version_id = $1
-                    ${opFilter}
-                    GROUP BY l.id
-                    ORDER BY l.id DESC
-                `, [versionId, valueCol || null, unitsCol || null]);
-
-            // A recount is also a repair: write back what it found, so the next
-            // read is cheap again and the stored figure matches the rows.
-            if (recount) {
-                for (const r of result.rows) {
-                    await pool.query(
-                        `UPDATE pf.log SET row_count = $2, value_total = $3, units_total = $4,
-                                           measure_cols = $5::jsonb
-                         WHERE id = $1`,
-                        [r.id, r.row_count, r.value_total, r.units_total,
-                         JSON.stringify({ value: valueCol || null, units: unitsCol || null })]
-                    );
-                }
-            }
-            // The statement is kilobytes per entry and the list is opened to scan,
-            // not to read SQL. It stays in the row for the debug endpoint below.
-            res.json(result.rows.map(({ sql_text, ...r }) => ({
-                ...r, has_sql: !!sql_text,
-            })));
-        } catch (err) {
-            console.error(err);
-            res.status(err.status || 500).json({ error: err.message });
-        }
-    });
-
-    // Everything about one entry, for when the rows look wrong: what was asked
-    // for (params), what it ran against (env), and the statement that actually
-    // executed with territory and scope resolved into it (sql_text).
-    //
-    // Both the intent and the SQL, because the translation between them is
-    // exactly what is in doubt when a result is surprising.
-    router.get('/log/:logid/debug', async (req, res) => {
-        const logId = parseInt(req.params.logid);
-        try {
-            const { rows } = await pool.query(
-                `SELECT l.*, v.name AS version_name, s.schema, s.tname
-                 FROM pf.log l
-                 JOIN pf.version v ON v.id = l.version_id
-                 JOIN pf.source s  ON s.id = v.source_id
-                 WHERE l.id = $1`, [logId]
-            );
-            if (!rows.length) return res.status(404).json({ error: 'Log entry not found' });
-            res.json(rows[0]);
+            const result = await pool.query(`
+                SELECT l.*, ${aggCols},
+                       $2::text AS value_col,
+                       $3::text AS units_col
+                FROM pf.log l
+                LEFT JOIN ${table} f ON f.pf_logid = l.id
+                WHERE l.version_id = $1
+                GROUP BY l.id
+                ORDER BY l.id DESC
+            `, [versionId, valueCol || null, unitsCol || null]);
+            res.json(result.rows);
         } catch (err) {
             console.error(err);
             res.status(err.status || 500).json({ error: err.message });
@@ -141,17 +61,6 @@ module.exports = function(pool) {
             if (!logResult.rows.length) return res.status(404).json({ error: 'Log entry not found' });
             const log = logResult.rows[0];
             if (log.status === 'closed') return res.status(403).json({ error: 'Version is closed' });
-
-            // Undo deletes rows wholesale by logid, so it cannot be territory
-            // filtered the way a read or a write can -- half-undoing an entry
-            // would leave the version in a state nothing describes. Ownership
-            // instead: your own entries, or an admin's override. Territory alone
-            // would not do it anyway, since two accounts can share one.
-            if (!req.session?.user?.is_admin && log.pf_user !== sessionUser(req)) {
-                return res.status(403).json({
-                    error: `That entry was made by ${log.pf_user || 'someone else'} — only they or an administrator can undo it`
-                });
-            }
             const table = fcTable(log.tname, log.version_id);
 
             // In grain mode the client's table is indexed on pf_gkey, so undo has to
@@ -205,58 +114,13 @@ module.exports = function(pool) {
         }
     });
 
-    // update the note and/or tag on a log entry. Both are annotations — they never
-    // affect the forecast rows — so they stay editable after the fact, including on
-    // a closed version, where relabelling history is still legitimate.
+    // update the note on a log entry
     router.patch('/log/:logid', async (req, res) => {
         const logId = parseInt(req.params.logid);
-        const { note, tag, bucket, label } = req.body;
-        if (note === undefined && tag === undefined
-            && bucket === undefined && label === undefined) {
-            return res.status(400).json({
-                error: 'Nothing to update — send note, tag, bucket and/or label'
-            });
-        }
-        // Every entry carries a label and a bucket (pf.log's log_named CHECK), so
-        // they can be renamed but not cleared. Refused here for the message; the
-        // constraint would refuse it anyway.
-        for (const [field, value] of [['label', label], ['bucket', bucket]]) {
-            if (value !== undefined && !String(value ?? '').trim()) {
-                return res.status(400).json({ error: `The ${field} can be renamed but not cleared` });
-            }
-        }
+        const { note } = req.body;
         try {
-            // Same rule as undo: your own entries, or an admin's. These are
-            // annotations, but label and bucket name the pivot's columns for
-            // everyone who opens the version, so an unguarded PATCH let any
-            // account rename the company's segments -- including on loads whose
-            // rows it cannot see.
-            const owner = await pool.query(
-                `SELECT pf_user FROM pf.log WHERE id = $1`, [logId]
-            );
-            if (!owner.rows.length) return res.status(404).json({ error: 'Log entry not found' });
-            if (!req.session?.user?.is_admin && owner.rows[0].pf_user !== sessionUser(req)) {
-                return res.status(403).json({
-                    error: `That entry was made by ${owner.rows[0].pf_user || 'someone else'} — only they or an administrator can change it`
-                });
-            }
-
-            // COALESCE on the flag, not the value: an explicit null or '' must be
-            // able to clear a field, which COALESCE on the value alone would ignore
             const result = await pool.query(
-                `UPDATE pf.log SET
-                     note   = CASE WHEN $2::bool THEN $3::text ELSE note   END,
-                     tag    = CASE WHEN $4::bool THEN $5::text ELSE tag    END,
-                     bucket = CASE WHEN $6::bool THEN $7::text ELSE bucket END,
-                     label  = CASE WHEN $8::bool THEN $9::text ELSE label  END
-                 WHERE id = $1 RETURNING *`,
-                [
-                    logId,
-                    note   !== undefined, note   === undefined ? null : (String(note).trim()   || null),
-                    tag    !== undefined, tag    === undefined ? null : (String(tag).trim()    || null),
-                    bucket !== undefined, bucket === undefined ? null : (String(bucket).trim() || null),
-                    label  !== undefined, label  === undefined ? null : (String(label).trim()  || null),
-                ]
+                `UPDATE pf.log SET note = $1 WHERE id = $2 RETURNING *`, [note ?? null, logId]
             );
             if (!result.rows.length) return res.status(404).json({ error: 'Log entry not found' });
             res.json(result.rows[0]);

routes/plans.js

@@ -1,125 +0,0 @@
-const express = require('express');
-const { sessionUser, requireAdmin } = require('../lib/auth');
-
-// Planned load segments — a version's definition, before it has been run.
-//
-// A plan carries exactly what POST /versions/:id/{baseline|reference} takes, so
-// running one is that same call with the plan's fields; the client then stamps
-// run_logid. It is two round trips rather than one because the alternative is a
-// second copy of the load route's token substitution, and a plan that reads
-// unrun while its rows exist is visible and fixable, where a divergent copy of
-// the load SQL is neither.
-module.exports = function(pool) {
-    const router = express.Router();
-
-    const FIELDS = ['seq', 'kind', 'label', 'bucket', 'tag', 'note',
-                    'filters', 'raw_where', 'date_offset'];
-
-    // A plan whose log entry has been undone is a plan again: the FK is
-    // ON DELETE SET NULL, so run_logid clears itself. The join is for the
-    // row count, which is what makes "already run" worth seeing.
-    const SELECT_PLANS = `
-        SELECT
-             p.*
-            ,l.row_count   AS run_rows
-            ,l.stamp       AS run_at
-        FROM pf.load_plan p
-        LEFT JOIN pf.log l
-            ON l.id = p.run_logid
-        WHERE TRUE
-          AND p.version_id = $1
-        ORDER BY
-             p.seq
-            ,p.id`;
-
-    router.get('/versions/:id/plans', async (req, res) => {
-        try {
-            const { rows } = await pool.query(SELECT_PLANS, [parseInt(req.params.id)]);
-            res.json(rows);
-        } catch (err) {
-            res.status(500).json({ error: err.message });
-        }
-    });
-
-    router.post('/versions/:id/plans', requireAdmin, async (req, res) => {
-        const versionId = parseInt(req.params.id);
-        const b = req.body || {};
-        if (!['baseline', 'reference'].includes(b.kind)) {
-            return res.status(400).json({ error: 'kind must be baseline or reference' });
-        }
-        try {
-            // Append by default: a new plan runs after the ones already there.
-            const next = await pool.query(
-                `SELECT COALESCE(max(seq), -1) + 1 AS seq FROM pf.load_plan WHERE version_id = $1`,
-                [versionId]
-            );
-            const { rows } = await pool.query(
-                `INSERT INTO pf.load_plan
-                     (version_id, seq, kind, label, bucket, tag, note, filters, raw_where,
-                      date_offset, created_by)
-                 VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, COALESCE($10, '0 days'), $11)
-                 RETURNING *`,
-                [versionId, b.seq ?? next.rows[0].seq, b.kind,
-                 b.label || null, b.bucket || null, b.tag || null, b.note || null,
-                 b.filters ? JSON.stringify(b.filters) : null, b.raw_where || null,
-                 b.date_offset || null, sessionUser(req)]
-            );
-            res.status(201).json(rows[0]);
-        } catch (err) {
-            res.status(500).json({ error: err.message });
-        }
-    });
-
-    // Any subset of the fields, plus run_logid when a run has just happened.
-    // The flag-and-value pair rather than COALESCE on the value: clearing a
-    // label to blank has to be distinguishable from not mentioning it.
-    router.patch('/plans/:pid', requireAdmin, async (req, res) => {
-        const pid = parseInt(req.params.pid);
-        const b   = req.body || {};
-        const keys = [...FIELDS, 'run_logid'].filter(k => b[k] !== undefined);
-        if (!keys.length) return res.status(400).json({ error: 'Nothing to update' });
-        if (b.kind !== undefined && !['baseline', 'reference'].includes(b.kind)) {
-            return res.status(400).json({ error: 'kind must be baseline or reference' });
-        }
-        try {
-            const sets   = [];
-            const params = [pid];
-            for (const k of keys) {
-                let v = k === 'filters'
-                    ? (b[k] === null ? null : JSON.stringify(b[k]))
-                    : (typeof b[k] === 'string' ? (b[k].trim() || null) : b[k]);
-                // date_offset is NOT NULL, so a blank one means the default.
-                // Coerced here rather than in the statement: a second assignment
-                // to the same column is an error, not a fallback.
-                if (k === 'date_offset' && !v) v = '0 days';
-                params.push(v);
-                sets.push(`${k} = $${params.length}${k === 'filters' ? '::jsonb' : ''}`);
-            }
-            const { rows } = await pool.query(
-                `UPDATE pf.load_plan
-                 SET ${sets.join(', ')}, updated_at = now()
-                 WHERE id = $1
-                 RETURNING *`,
-                params
-            );
-            if (!rows.length) return res.status(404).json({ error: 'Plan not found' });
-            res.json(rows[0]);
-        } catch (err) {
-            res.status(500).json({ error: err.message });
-        }
-    });
-
-    router.delete('/plans/:pid', requireAdmin, async (req, res) => {
-        try {
-            const { rows } = await pool.query(
-                `DELETE FROM pf.load_plan WHERE id = $1 RETURNING id`, [parseInt(req.params.pid)]
-            );
-            if (!rows.length) return res.status(404).json({ error: 'Plan not found' });
-            res.json({ deleted: rows[0].id });
-        } catch (err) {
-            res.status(500).json({ error: err.message });
-        }
-    });
-
-    return router;
-};

routes/sources.js

@@ -1,8 +1,5 @@
 const express = require('express');
-const { generateSQL, buildTerritoryClause } = require('../lib/sql_generator');
-const { RELATION_COLUMNS_SQL } = require('../lib/utils');
-const { sessionTerritory, requireAdmin } = require('../lib/auth');
-const { sessionUser } = require('../lib/auth');
+const { generateSQL } = require('../lib/sql_generator');
 
 module.exports = function(pool) {
     const router = express.Router();
@@ -22,9 +19,8 @@ module.exports = function(pool) {
 
     // register a source table
     // auto-populates col_meta from information_schema with role='ignore'
-    router.post('/sources', requireAdmin, async (req, res) => {
-        const { schema, tname, label } = req.body;
-        const created_by = sessionUser(req);
+    router.post('/sources', async (req, res) => {
+        const { schema, tname, label, created_by } = req.body;
         if (!schema || !tname) {
             return res.status(400).json({ error: 'schema and tname are required' });
         }
@@ -43,14 +39,15 @@ module.exports = function(pool) {
             );
             const source = src.rows[0];
 
-            // seed col_meta from the source's real columns
+            // seed col_meta from information_schema
             await client.query(`
                 INSERT INTO pf.col_meta (source_id, cname, role, opos)
-                SELECT $3, column_name, 'dimension', ordinal_position
-                FROM (${RELATION_COLUMNS_SQL}) c
+                SELECT $1, column_name, 'dimension', ordinal_position
+                FROM information_schema.columns
+                WHERE table_schema = $2 AND table_name = $3
                 ORDER BY ordinal_position
                 ON CONFLICT (source_id, cname) DO NOTHING
-            `, [schema, tname, source.id]);
+            `, [source.id, schema, tname]);
 
             await client.query('COMMIT');
             res.status(201).json(source);
@@ -81,29 +78,19 @@ module.exports = function(pool) {
     });
 
     // save col_meta — accepts full array, upserts each row
-    router.put('/sources/:id/cols', requireAdmin, async (req, res) => {
+    router.put('/sources/:id/cols', async (req, res) => {
         const sourceId = parseInt(req.params.id);
         const cols = req.body;
         if (!Array.isArray(cols)) {
             return res.status(400).json({ error: 'body must be an array' });
         }
-        // Exactly one per source: the scope is a single IN list against a single
-        // column, and two flagged would silently mean whichever one a .find()
-        // reached first -- the trap is_key already fell into (see CLAUDE.md).
-        const territoryCols = cols.filter(c => c.is_territory).map(c => c.cname);
-        if (territoryCols.length > 1) {
-            return res.status(400).json({
-                error: `Only one column can be the territory. Flagged: ${territoryCols.join(', ')}`
-            });
-        }
-
         const client = await pool.connect();
         try {
             await client.query('BEGIN');
             for (const col of cols) {
                 await client.query(`
-                    INSERT INTO pf.col_meta (source_id, cname, label, role, is_key, dim_group, dim_period_col, in_grain, is_territory, opos)
-                    VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
+                    INSERT INTO pf.col_meta (source_id, cname, label, role, is_key, dim_group, dim_period_col, in_grain, opos)
+                    VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
                     ON CONFLICT (source_id, cname) DO UPDATE SET
                         label          = EXCLUDED.label,
                         role           = EXCLUDED.role,
@@ -111,7 +98,6 @@ module.exports = function(pool) {
                         dim_group      = EXCLUDED.dim_group,
                         dim_period_col = EXCLUDED.dim_period_col,
                         in_grain       = EXCLUDED.in_grain,
-                        is_territory   = EXCLUDED.is_territory,
                         opos           = EXCLUDED.opos
                 `, [
                     sourceId,
@@ -122,7 +108,6 @@ module.exports = function(pool) {
                     col.dim_group      || null,
                     col.dim_period_col || null,
                     col.in_grain       || false,
-                    col.is_territory   || false,
                     col.opos           || null
                 ]);
             }
@@ -142,7 +127,7 @@ module.exports = function(pool) {
     });
 
     // generate SQL for all operations from current col_meta and store in pf.sql
-    router.post('/sources/:id/generate-sql', requireAdmin, async (req, res) => {
+    router.post('/sources/:id/generate-sql', async (req, res) => {
         const sourceId = parseInt(req.params.id);
         try {
             const srcResult = await pool.query(
@@ -238,38 +223,10 @@ module.exports = function(pool) {
                 return res.status(400).json({ error: `"${col}" is not a key column` });
             }
 
-            // ?q= narrows, ?limit= caps. A key column can be very wide -- part on
-            // osm_skinny has 11,290 distinct values -- so returning the lot to fill a
-            // completion list is both a slow query and a large response for a control
-            // that can only usefully show a handful.
             const { schema, tname } = srcResult.rows[0];
-            const q     = (req.query.q || '').trim();
-            const limit = Math.min(parseInt(req.query.limit) || 5000, 5000);
-
-            const params = [];
-            let filter = `WHERE "${col}" IS NOT NULL`;
-
-            // Completion reads the *source* table, which no territory scope has
-            // touched -- so without this a scoped account could enumerate every
-            // customer, part and rep in the business from a dropdown, having
-            // been shown none of their rows.
-            const terrRow = (await pool.query(
-                `SELECT cname FROM pf.col_meta WHERE source_id = $1 AND is_territory LIMIT 1`,
-                [req.params.id]
-            )).rows[0];
-            const terrClause = buildTerritoryClause(sessionTerritory(req), terrRow?.cname || null);
-            if (terrClause) filter += ` AND ${terrClause}`;
-
-            if (q) {
-                params.push(`%${q}%`);
-                filter += ` AND "${col}"::text ILIKE $${params.length}`;
-            }
-            params.push(limit);
-
             const result = await pool.query(
-                `SELECT DISTINCT "${col}"::text AS val FROM ${schema}.${tname}
-                 ${filter} ORDER BY 1 LIMIT $${params.length}`,
-                params
+                `SELECT DISTINCT "${col}" AS val FROM ${schema}.${tname}
+                 WHERE "${col}" IS NOT NULL ORDER BY "${col}"`
             );
             res.json(result.rows.map(r => r.val));
         } catch (err) {
@@ -278,142 +235,6 @@ module.exports = function(pool) {
         }
     });
 
-    // Resolve a dim_group to its key column and siblings, or explain why it cannot be.
-    async function resolveGroup(sourceId, group) {
-        const { rows: meta } = await pool.query(
-            `SELECT * FROM pf.col_meta WHERE source_id = $1 ORDER BY opos`, [sourceId]);
-        const members = meta.filter(c => c.dim_group === group);
-        if (!members.length) {
-            const err = new Error(`No columns are grouped as "${group}" on this source`);
-            err.status = 404; throw err;
-        }
-        const keyCol = members.find(c => c.is_key);
-        if (!keyCol) {
-            const err = new Error(
-                `Group "${group}" has no is_key column, so its members have nothing to be keyed on`);
-            err.status = 400; throw err;
-        }
-        return {
-            keyCol,
-            siblings:  members.filter(c => c.cname !== keyCol.cname),
-            // recency column: the source's primary date, the same one the generator
-            // treats as the date for loads
-            dateCol: meta.find(c => c.role === 'date')?.cname || null,
-        };
-    }
-
-    // The member list for a group, as one array. Small enough to send whole --
-    // 11,290 parts on osm_skinny -- so the client holds it and filters locally
-    // instead of querying per keystroke.
-    router.get('/sources/:id/dim/:group', async (req, res) => {
-        try {
-            const sourceId = parseInt(req.params.id);
-            const { keyCol, siblings } = await resolveGroup(sourceId, req.params.group);
-            const includeInactive = req.query.all === '1';
-
-            const { rows } = await pool.query(`
-                SELECT key_value, attrs, is_active, source_seen
-                FROM pf.dim_member
-                WHERE source_id = $1 AND dim_group = $2
-                ${includeInactive ? '' : 'AND is_active'}
-                ORDER BY key_value
-            `, [sourceId, req.params.group]);
-
-            res.json({
-                group:    req.params.group,
-                key_col:  keyCol.cname,
-                siblings: siblings.map(c => c.cname),
-                members:  rows,
-            });
-        } catch (err) {
-            console.error(err);
-            res.status(err.status || 500).json({ error: err.message });
-        }
-    });
-
-    // Rebuild a group's members from the source. A merge, not a replace: curation
-    // (a member deactivated by hand, or added before it ever sold) has to survive a
-    // refresh, so absent members are marked source_seen = false rather than deleted.
-    //
-    // Slow by nature -- it reads the whole source, which for a view over a
-    // transaction table is millions of rows -- so it is a deliberate action rather
-    // than something that happens on a page load.
-    router.post('/sources/:id/dim/:group/refresh', requireAdmin, async (req, res) => {
-        const sourceId = parseInt(req.params.id);
-        const group    = req.params.group;
-        try {
-            const srcResult = await pool.query(
-                `SELECT schema, tname FROM pf.source WHERE id = $1`, [sourceId]);
-            if (!srcResult.rows.length) return res.status(404).json({ error: 'Source not found' });
-            const { schema, tname } = srcResult.rows[0];
-
-            const { keyCol, siblings, dateCol } = await resolveGroup(sourceId, group);
-            if (!siblings.length) {
-                return res.status(400).json({ error: `Group "${group}" has no sibling columns to store` });
-            }
-
-            const q      = (n) => `"${n}"`;
-            const attrs  = siblings.map(c => `'${c.cname}', s.${q(c.cname)}::text`).join(', ');
-            // A key can carry more than one attribute set across history -- 11,290
-            // parts against 13,662 combinations on osm_skinny. Take the most recent
-            // by the source's date column, which is the live definition.
-            const recency = dateCol ? `s.${q(dateCol)} DESC NULLS LAST` : `1`;
-
-            const started = Date.now();
-            // An explicit stamp rather than now(): inside a transaction now() is the
-            // transaction's start time, so "refreshed in this run" and "refreshed in
-            // a run that began at the same instant" would be indistinguishable.
-            const runAt   = new Date();
-            const client  = await pool.connect();
-            try {
-                await client.query('BEGIN');
-                const { rows: [{ n }] } = await client.query(`
-                    WITH ranked AS (
-                        SELECT s.${q(keyCol.cname)}::text AS key_value,
-                               jsonb_build_object(${attrs}) AS attrs,
-                               row_number() OVER (
-                                   PARTITION BY s.${q(keyCol.cname)} ORDER BY ${recency}
-                               ) AS rn
-                        FROM ${q(schema)}.${q(tname)} s
-                        WHERE s.${q(keyCol.cname)} IS NOT NULL
-                    )
-                    ,upserted AS (
-                        INSERT INTO pf.dim_member
-                            (source_id, dim_group, key_value, attrs, refreshed_at, source_seen)
-                        SELECT $1, $2, key_value, attrs, $3, true
-                        FROM ranked WHERE rn = 1
-                        ON CONFLICT (source_id, dim_group, key_value) DO UPDATE SET
-                            attrs        = EXCLUDED.attrs,
-                            source_seen  = true,
-                            refreshed_at = $3,
-                            updated_at   = now()
-                        RETURNING 1
-                    )
-                    SELECT count(*)::int AS n FROM upserted
-                `, [sourceId, group, runAt]);
-
-                // anything this run did not touch is no longer in the source
-                const { rowCount: dropped } = await client.query(`
-                    UPDATE pf.dim_member
-                    SET source_seen = false, updated_at = now()
-                    WHERE source_id = $1 AND dim_group = $2 AND source_seen
-                      AND refreshed_at IS DISTINCT FROM $3
-                `, [sourceId, group, runAt]);
-
-                await client.query('COMMIT');
-                res.json({ group, members: n, no_longer_in_source: dropped, ms: Date.now() - started });
-            } catch (err) {
-                await client.query('ROLLBACK');
-                throw err;
-            } finally {
-                client.release();
-            }
-        } catch (err) {
-            console.error(err);
-            res.status(err.status || 500).json({ error: err.message });
-        }
-    });
-
     // given a key column value, look up sibling dim_group column values from source
     // returns { sibling_col: value, ... } if exactly one match, null if none or ambiguous
     router.get('/sources/:id/lookup', async (req, res) => {
@@ -449,12 +270,23 @@ module.exports = function(pool) {
         }
     });
 
-    // PUT /sources/:id/default-layout is gone. It wrote pf.source.default_layout,
-    // one anonymous blob per source that any account could overwrite for every
-    // other account -- a published layout with no owner. Its successor is
-    // pf.layout: named, owned, and writable only by its owner or an admin. The
-    // old column is left in place, already migrated into pf.layout by
-    // setup_sql/01_schema.sql, and read by nothing.
+    // set or clear the default Perspective layout for a source.
+    // Body: a Perspective view config (group_by, split_by, columns, plugin_config, …).
+    // Pass null or {} to clear.
+    router.put('/sources/:id/default-layout', async (req, res) => {
+        try {
+            const layout = req.body && Object.keys(req.body).length > 0 ? req.body : null;
+            const result = await pool.query(
+                `UPDATE pf.source SET default_layout = $1 WHERE id = $2 RETURNING *`,
+                [layout, req.params.id]
+            );
+            if (result.rows.length === 0) return res.status(404).json({ error: 'Source not found' });
+            res.json(result.rows[0]);
+        } catch (err) {
+            console.error(err);
+            res.status(500).json({ error: err.message });
+        }
+    });
 
     // deregister a source — does not drop existing forecast tables
     router.get('/dim-period/cols', async (req, res) => {
@@ -473,7 +305,7 @@ module.exports = function(pool) {
         }
     });
 
-    router.delete('/sources/:id', requireAdmin, async (req, res) => {
+    router.delete('/sources/:id', async (req, res) => {
         try {
             const result = await pool.query(
                 `DELETE FROM pf.source WHERE id = $1 RETURNING *`,

routes/tables.js

@@ -1,9 +1,5 @@
 const express = require('express');
-const { RELATION_COLUMNS_SQL } = require('../lib/utils');
 
-// Reads, deliberately open to everyone: Setup is visible read-only to
-// non-admins, so gating what the page needs in order to render turned it into
-// a 403 the client rendered as a crash. The writes it leads to are gated.
 module.exports = function(pool) {
     const router = express.Router();
 
@@ -11,18 +7,15 @@ module.exports = function(pool) {
     router.get('/tables', async (req, res) => {
         try {
             const result = await pool.query(`
-                -- pg_class, not information_schema.tables, which omits
-                -- materialized views: gs.osm_skinny is one, and the browser
-                -- could not offer what the app is already built on.
                 SELECT
-                    n.nspname           AS schema,
-                    c.relname           AS tname,
+                    t.table_schema  AS schema,
+                    t.table_name    AS tname,
                     c.reltuples::bigint AS row_estimate
-                FROM pg_class c
-                JOIN pg_namespace n ON n.oid = c.relnamespace
-                WHERE c.relkind IN ('r', 'v', 'm', 'f', 'p')
-                  AND n.nspname NOT IN ('pg_catalog', 'information_schema', 'pf')
-                ORDER BY n.nspname, c.relname
+                FROM information_schema.tables t
+                LEFT JOIN pg_namespace n ON n.nspname = t.table_schema
+                LEFT JOIN pg_class c ON c.relname = t.table_name AND c.relnamespace = n.oid
+                WHERE t.table_schema NOT IN ('pg_catalog', 'information_schema', 'pf')
+                ORDER BY t.table_schema, t.table_name
             `);
             res.json(result.rows);
         } catch (err) {
@@ -38,8 +31,12 @@ module.exports = function(pool) {
             return res.status(400).json({ error: 'Invalid schema or table name' });
         }
         try {
-            const cols = await pool.query(
-                `${RELATION_COLUMNS_SQL} ORDER BY ordinal_position`, [schema, tname]);
+            const cols = await pool.query(`
+                SELECT column_name, data_type, is_nullable, ordinal_position
+                FROM information_schema.columns
+                WHERE table_schema = $1 AND table_name = $2
+                ORDER BY ordinal_position
+            `, [schema, tname]);
 
             const rows = await pool.query(
                 `SELECT * FROM ${schema}.${tname} LIMIT 5`

routes/versions.js

@@ -1,75 +1,5 @@
 const express = require('express');
-const { fcTable, mapType, RELATION_COLUMNS_SQL } = require('../lib/utils');
-const { sessionUser, sessionTerritory, requireAdmin } = require('../lib/auth');
-const { buildTerritoryClause } = require('../lib/sql_generator');
-
-// Copy a version's definition onto a freshly created one: its load segments as
-// plans, and its saved layouts.
-//
-// The plans come from pf.load_plan when the template has them, and are derived
-// from its baseline/reference log entries when it does not — which is every
-// version created before plans existed, so a definition can be lifted out of
-// one that was built by hand. `params` on a load entry already stores the
-// filters, the raw WHERE when one was used, and the date offset, so the
-// derivation is a read rather than a reconstruction.
-async function cloneDefinition(client, template, version, created_by) {
-    const existing = await client.query(
-        `SELECT * FROM pf.load_plan WHERE version_id = $1 ORDER BY seq, id`,
-        [template.id]
-    );
-
-    let source = existing.rows.map(p => ({
-        kind: p.kind, label: p.label, bucket: p.bucket, tag: p.tag, note: p.note,
-        filters: p.filters, raw_where: p.raw_where, date_offset: p.date_offset
-    }));
-
-    if (!source.length) {
-        const log = await client.query(
-            `SELECT operation, label, bucket, tag, note, params
-             FROM pf.log
-             WHERE TRUE
-               AND version_id = $1
-               AND operation IN ('baseline', 'reference')
-             ORDER BY id`,
-            [template.id]
-        );
-        source = log.rows.map(e => ({
-            kind: e.operation,
-            label: e.label, bucket: e.bucket, tag: e.tag, note: e.note,
-            filters:     e.params?.filters   || null,
-            raw_where:   e.params?.raw_where || null,
-            date_offset: e.params?.date_offset || '0 days'
-        }));
-    }
-
-    for (const [i, p] of source.entries()) {
-        await client.query(
-            `INSERT INTO pf.load_plan
-                 (version_id, seq, kind, label, bucket, tag, note, filters, raw_where,
-                  date_offset, created_by)
-             VALUES ($1, $2, $3, $4, $5, $6, $7, $8::jsonb, $9, COALESCE($10, '0 days'), $11)`,
-            [version.id, i, p.kind, p.label, p.bucket, p.tag, p.note,
-             p.filters ? JSON.stringify(p.filters) : null, p.raw_where,
-             p.date_offset, created_by]
-        );
-    }
-
-    // Version-scoped layouts only. The source-wide ones (version_id IS NULL)
-    // already apply to the new version, so copying them would duplicate them
-    // under a name that now exists twice.
-    const layouts = await client.query(
-        `INSERT INTO pf.layout (source_id, version_id, name, config, owner, visibility, is_default)
-         SELECT source_id, $2, name, config, owner, visibility, is_default
-         FROM pf.layout
-         WHERE TRUE
-           AND version_id = $1
-         RETURNING id`,
-        [template.id, version.id]
-    );
-
-    return { from: template.id, from_name: template.name,
-             plans: source.length, layouts: layouts.rowCount };
-}
+const { fcTable, mapType } = require('../lib/utils');
 
 module.exports = function(pool) {
     const router = express.Router();
@@ -90,10 +20,9 @@ module.exports = function(pool) {
 
     // create a new version
     // inserts version row, then CREATE TABLE pf.fc_{tname}_{version_id} in one transaction
-    router.post('/sources/:id/versions', requireAdmin, async (req, res) => {
+    router.post('/sources/:id/versions', async (req, res) => {
         const sourceId = parseInt(req.params.id);
-        const { name, description, exclude_iters, clone_from } = req.body;
-        const created_by = sessionUser(req);
+        const { name, description, created_by, exclude_iters } = req.body;
         if (!name) return res.status(400).json({ error: 'name is required' });
 
         const client = await pool.connect();
@@ -107,9 +36,8 @@ module.exports = function(pool) {
             }
             const source = srcResult.rows[0];
 
-            // col_meta joined to the source's real columns, for the data types
+            // fetch col_meta joined to information_schema for data types
             const colResult = await client.query(`
-                WITH cols AS (${RELATION_COLUMNS_SQL})
                 SELECT
                     m.cname,
                     m.role,
@@ -118,11 +46,14 @@ module.exports = function(pool) {
                     i.numeric_precision,
                     i.numeric_scale
                 FROM pf.col_meta m
-                JOIN cols i ON i.column_name = m.cname
-                WHERE m.source_id = $3
+                JOIN information_schema.columns i
+                    ON  i.table_schema = $2
+                    AND i.table_name   = $3
+                    AND i.column_name  = m.cname
+                WHERE m.source_id = $1
                   AND m.role NOT IN ('ignore')
                 ORDER BY m.opos
-            `, [source.schema, source.tname, sourceId]);
+            `, [sourceId, source.schema, source.tname]);
 
             if (colResult.rows.length === 0) {
                 return res.status(400).json({
@@ -130,52 +61,19 @@ module.exports = function(pool) {
                 });
             }
 
-            // clone_from copies a version's *definition* -- its settings, its
-            // saved layouts, and its load segments as plans -- and stops there.
-            // The loads are deliberately not run: the reason to clone a
-            // definition is to change the dates before it loads anything, and a
-            // clone that fetched two million rows first would have to be undone
-            // before it could be edited.
-            //
-            // Adjustments are not copied in any form. A scale's target is a
-            // figure sized against the rows that were there, so replaying one
-            // against a different load is arithmetic nobody asked for.
-            let template = null;
-            if (clone_from) {
-                const t = await client.query(
-                    `SELECT * FROM pf.version WHERE id = $1 AND source_id = $2`,
-                    [parseInt(clone_from), sourceId]
-                );
-                if (!t.rows.length) {
-                    return res.status(404).json({
-                        error: 'Version to clone not found, or it belongs to a different source'
-                    });
-                }
-                template = t.rows[0];
-            }
-
             await client.query('BEGIN');
 
             // insert version to get id
             const verResult = await client.query(`
-                INSERT INTO pf.version (source_id, name, description, created_by, exclude_iters,
-                                        adjustment_segment, adjustment_bucket, unlabeled_load,
-                                        bucket_order)
-                VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
+                INSERT INTO pf.version (source_id, name, description, created_by, exclude_iters)
+                VALUES ($1, $2, $3, $4, $5)
                 RETURNING *
             `, [
                 sourceId,
                 name,
-                // An explicit description wins; otherwise the template's is
-                // carried, since it describes the definition being copied.
-                description || template?.description || null,
+                description   || null,
                 created_by    || null,
-                exclude_iters ? JSON.stringify(exclude_iters)
-                              : (template ? JSON.stringify(template.exclude_iters) : '["reference"]'),
-                template?.adjustment_segment || null,
-                template?.adjustment_bucket  || null,
-                template?.unlabeled_load     || null,
-                template?.bucket_order ? JSON.stringify(template.bucket_order) : null
+                exclude_iters ? JSON.stringify(exclude_iters) : '["reference"]'
             ]);
             const version = verResult.rows[0];
 
@@ -202,19 +100,8 @@ ${colDefs},
             `;
             await client.query(ddl);
 
-            // pf_logid is how every entry-level operation finds its rows: undo
-            // deletes by it, the change log aggregates by it, and it is part of the
-            // grain key. Without an index each of those is a sequential scan of the
-            // whole forecast table -- 2.5M rows to total two adjustments.
-            await client.query(
-                `CREATE INDEX ${table.split('.').pop()}_logid_idx ON ${table} (pf_logid)`
-            );
-
-            let cloned = null;
-            if (template) cloned = await cloneDefinition(client, template, version, created_by);
-
             await client.query('COMMIT');
-            res.status(201).json({ ...version, fc_table: table, cloned });
+            res.status(201).json({ ...version, fc_table: table });
         } catch (err) {
             await client.query('ROLLBACK');
             console.error(err);
@@ -227,376 +114,36 @@ ${colDefs},
         }
     });
 
-    // where this version's writes actually land: the physical forecast table,
-    // its current row count, and the source table rows are read from.
-    // Surfaced in the status bar so the write target is never a mystery.
-    // Distinct values of a dimension as they appear in one version's forecast
-    // table, for completing recode and clone.
-    //
-    // Deliberately not the source: the source view reaches back over all of
-    // history, so completing from it offers parts discontinued years ago. The
-    // version holds what was actually loaded, which is what a forecast is being
-    // written against.
-    //
-    // Held in memory because the scan is not cheap -- 2.0s for 11,290 parts across
-    // 2.5M rows on fc_osm_skinny_29 -- and completion is typed into. Keyed on the
-    // version's latest log id, so any load, adjustment or undo rebuilds it on the
-    // next request without anything having to remember to invalidate.
-    const valueCache = new Map();
-
-    router.get('/versions/:id/values/:col', async (req, res) => {
-        const versionId = parseInt(req.params.id);
-        const col = req.params.col;
-        try {
-            const verResult = await pool.query(`
-                SELECT v.id, s.tname, s.id AS source_id
-                FROM pf.version v JOIN pf.source s ON s.id = v.source_id
-                WHERE v.id = $1
-            `, [versionId]);
-            if (!verResult.rows.length) return res.status(404).json({ error: 'Version not found' });
-            const { tname, source_id } = verResult.rows[0];
-
-            // the column name is interpolated, so it has to be one col_meta names
-            const okCol = await pool.query(`
-                SELECT 1 FROM pf.col_meta
-                WHERE source_id = $1 AND cname = $2 AND role IN ('dimension', 'date')
-            `, [source_id, col]);
-            if (!okCol.rows.length) {
-                return res.status(400).json({ error: `"${col}" is not a dimension on this source` });
-            }
-
-            const table = fcTable(tname, versionId);
-            const { rows: [{ rev }] } = await pool.query(
-                `SELECT coalesce(max(id), 0)::text AS rev FROM pf.log WHERE version_id = $1`,
-                [versionId]
-            );
-
-            const key = `${versionId}:${col}`;
-            let entry = valueCache.get(key);
-            if (!entry || entry.rev !== rev) {
-                const { rows } = await pool.query(
-                    `SELECT DISTINCT "${col}"::text AS val FROM ${table}
-                     WHERE "${col}" IS NOT NULL ORDER BY 1`
-                );
-                entry = { rev, values: rows.map(r => r.val) };
-                valueCache.set(key, entry);
-            }
-
-            const q      = (req.query.q || '').trim().toLowerCase();
-            const limit  = Math.min(parseInt(req.query.limit) || 50, 500);
-            const picked = q
-                ? entry.values.filter(v => v.toLowerCase().includes(q))
-                : entry.values;
-            res.json(picked.slice(0, limit));
-        } catch (err) {
-            console.error(err);
-            res.status(err.status || 500).json({ error: err.message });
-        }
-    });
-
-    router.get('/versions/:id/table-info', async (req, res) => {
-        try {
-            const verResult = await pool.query(`
-                SELECT v.id, v.name, v.status, s.schema, s.tname, s.id AS source_id
-                FROM pf.version v
-                JOIN pf.source s ON s.id = v.source_id
-                WHERE v.id = $1
-            `, [req.params.id]);
-            if (verResult.rows.length === 0) return res.status(404).json({ error: 'Version not found' });
-
-            const v       = verResult.rows[0];
-            const fc      = fcTable(v.tname, v.id);
-            const [schema, table] = fc.split('.');
-
-            const existsResult = await pool.query(
-                `SELECT to_regclass($1) IS NOT NULL AS exists`, [fc]
-            );
-            const exists = existsResult.rows[0].exists;
-
-            // Scoped like every other read. Unscoped it reported the whole
-            // table to an account that can see a fraction of it -- the status
-            // bar's row count, and the figure the load progress promises, both
-            // came from here: a rep with 330k rows was told the load was
-            // fetching 2.8M.
-            const terrCol = (await pool.query(
-                `SELECT cname FROM pf.col_meta WHERE source_id = $1 AND is_territory LIMIT 1`,
-                [v.source_id]
-            )).rows[0]?.cname || null;
-            const terr = buildTerritoryClause(sessionTerritory(req), terrCol);
-
-            let rows = null, byIter = [];
-            if (exists) {
-                const countResult = await pool.query(
-                    `SELECT pf_iter, count(*)::int AS n FROM ${fc}
-                     ${terr ? `WHERE ${terr}` : ''}
-                     GROUP BY pf_iter ORDER BY pf_iter`
-                );
-                byIter = countResult.rows;
-                rows   = byIter.reduce((a, r) => a + r.n, 0);
-            }
-
-            res.json({
-                version_id:   v.id,
-                version_name: v.name,
-                status:       v.status,
-                source:       `${v.schema}.${v.tname}`,
-                fc_table:     fc,
-                fc_schema:    schema,
-                fc_tname:     table,
-                exists,
-                rows,
-                by_iter:      byIter
-            });
-        } catch (err) {
-            console.error(err);
-            res.status(500).json({ error: err.message });
-        }
-    });
-
-    // Tags already used on this source, newest first — feeds the tag autocomplete.
-    // Scoped to the source rather than the version so an initiative name carries
-    // across versions, which is the point of naming it.
-    // Tags already used on one version, for completing the next adjustment.
-    //
-    // Scoped to the version, not the source. A tag names an initiative inside a
-    // scenario -- what this plan is doing -- so offering every tag anyone has
-    // ever used on the source put last year's work in the way of this year's,
-    // and the `uses` count read as activity on a version it did not belong to.
-    // The cost is that a fresh version completes from nothing until it has a
-    // tag of its own, which is the honest state of it: cloning a definition
-    // copies no adjustments, so it has no initiatives yet.
-    router.get('/versions/:id/tags', async (req, res) => {
+    // update version name, description, or exclude_iters
+    router.put('/versions/:id', async (req, res) => {
+        const { name, description, exclude_iters } = req.body;
         try {
             const result = await pool.query(`
-                SELECT l.tag,
-                       count(*)::int                    AS uses,
-                       max(l.stamp)                     AS last_used,
-                       array_agg(DISTINCT l.pf_user)    AS owners
-                FROM pf.log l
-                WHERE TRUE
-                  AND l.version_id = $1
-                  AND l.tag IS NOT NULL
-                  AND l.tag <> ''
-                GROUP BY l.tag
-                ORDER BY max(l.stamp) DESC
-            `, [parseInt(req.params.id)]);
-
-            // Renaming a tag rewrites every entry that carries it, so the right
-            // to do it is the right to edit all of them -- not most of them.
-            // The tag manager greys the row out rather than offering a rename
-            // that would half-apply.
-            const me = sessionUser(req);
-            const admin = !!req.session?.user?.is_admin;
-            res.json(result.rows.map(r => ({
-                ...r,
-                can_edit: admin || (r.owners || []).every(o => o === me)
-            })));
-        } catch (err) {
-            console.error(err);
-            res.status(500).json({ error: err.message });
-        }
-    });
-
-    // Rename a tag across the whole version, or clear it. `to` empty means
-    // clear; `to` naming a tag that already exists merges the two, which is the
-    // point -- "Pull Forward" and "Pulled Forward" are one initiative reported
-    // as two, and the bridge groups on this column.
-    //
-    // A PATCH on the version rather than on a tag: a tag is not a row anywhere,
-    // only a value repeated across pf.log, so there is no /tags/:tag to address.
-    router.patch('/versions/:id/tags', async (req, res) => {
-        const versionId = parseInt(req.params.id);
-        const from = req.body.from === undefined ? null : String(req.body.from).trim();
-        const to   = req.body.to   === undefined ? null : String(req.body.to).trim();
-        if (!from) return res.status(400).json({ error: 'Which tag? Send `from`' });
-        if (to === from) return res.status(400).json({ error: 'That is the same tag' });
-
-        const client = await pool.connect();
-        try {
-            await client.query('BEGIN');
-            // Lock the rows before counting them, so the owner check and the
-            // rewrite see the same set.
-            const rows = await client.query(
-                `SELECT id, pf_user FROM pf.log
-                 WHERE version_id = $1 AND tag = $2
-                 FOR UPDATE`,
-                [versionId, from]
-            );
-            if (!rows.rows.length) {
-                await client.query('ROLLBACK');
-                return res.status(404).json({ error: `No entry on this version is tagged "${from}"` });
-            }
-
-            // Same rule as undo and PATCH /log: your own entries, or an admin's
-            // override. Applied to every entry carrying the tag, because a
-            // partial rename leaves the initiative split in two.
-            const me = sessionUser(req);
-            if (!req.session?.user?.is_admin) {
-                const others = [...new Set(rows.rows.map(r => r.pf_user).filter(u => u !== me))];
-                if (others.length) {
-                    await client.query('ROLLBACK');
-                    return res.status(403).json({
-                        error: `"${from}" is also on entries made by ${others.join(', ')} — only they or an administrator can change those`
-                    });
-                }
-            }
-
-            const result = await client.query(
-                `UPDATE pf.log SET tag = $3 WHERE version_id = $1 AND tag = $2`,
-                [versionId, from, to || null]
-            );
-            await client.query('COMMIT');
-            res.json({ from, to: to || null, rows_affected: result.rowCount });
-        } catch (err) {
-            await client.query('ROLLBACK').catch(() => {});
-            console.error(err);
-            res.status(500).json({ error: err.message });
-        } finally {
-            client.release();
-        }
-    });
-
-    // Bridge: how this version got from its baseline to where it stands, grouped by
-    // initiative. Amounts come from the version's own forecast table, so the figures
-    // reconcile with the pivot rather than being recomputed from the log's params.
-    router.get('/versions/:id/bridge', async (req, res) => {
-        try {
-            const verResult = await pool.query(`
-                SELECT v.id, v.exclude_iters, s.schema, s.tname, s.id AS source_id
-                FROM pf.version v JOIN pf.source s ON s.id = v.source_id
-                WHERE v.id = $1
-            `, [req.params.id]);
-            if (verResult.rows.length === 0) return res.status(404).json({ error: 'Version not found' });
-            const v  = verResult.rows[0];
-            const fc = fcTable(v.tname, v.id);
-
-            const exists = await pool.query(`SELECT to_regclass($1) IS NOT NULL AS ok`, [fc]);
-            if (!exists.rows[0].ok) return res.json({ fc_table: fc, exists: false, rows: [] });
-
-            const colResult = await pool.query(
-                `SELECT cname, role FROM pf.col_meta WHERE source_id = $1`, [v.source_id]);
-            const valueCol = colResult.rows.find(c => c.role === 'value')?.cname;
-            const unitsCol = colResult.rows.find(c => c.role === 'units')?.cname;
-            if (!valueCol) return res.status(400).json({ error: 'No value column configured' });
-
-            const excl = (v.exclude_iters || []).length
-                ? `t.pf_iter NOT IN (${v.exclude_iters.map(i => `'${String(i).replace(/'/g, "''")}'`).join(', ')})`
-                : 'TRUE';
-
-            const result = await pool.query(`
-                SELECT CASE WHEN t.pf_iter = 'baseline' THEN '(baseline)'
-                            ELSE coalesce(nullif(l.tag, ''), '(untagged)') END AS tag,
-                       bool_or(t.pf_iter = 'baseline')            AS is_baseline,
-                       count(DISTINCT l.id)::int                  AS entries,
-                       count(*)::int                              AS row_count,
-                       round(sum(t."${valueCol}")::numeric, 2)    AS value
-                       ${unitsCol ? `, round(sum(t."${unitsCol}")::numeric, 2) AS units` : ''}
-                FROM ${fc} t
-                LEFT JOIN pf.log l ON l.id = t.pf_logid
-                WHERE ${excl}
-                GROUP BY 1
-                ORDER BY bool_or(t.pf_iter = 'baseline') DESC, min(l.id)
-            `);
-            res.json({ fc_table: fc, exists: true, value_col: valueCol, units_col: unitsCol, rows: result.rows });
-        } catch (err) {
-            console.error(err);
-            res.status(500).json({ error: err.message });
-        }
-    });
-
-    // update version name, description, exclude_iters, or the fallback display
-    // names.
-    //
-    // bucket_order is deliberately not settable: the bucket column order is the
-    // text in pf.log.bucket now, so a stored order would be a second answer to
-    // the same question, and a silent one -- nothing reads it.
-    //
-    // The three name columns are flag-and-value pairs rather than COALESCE:
-    // clearing one back to the built-in means writing null, which COALESCE on
-    // the value alone cannot tell from "not mentioned".
-    router.put('/versions/:id', requireAdmin, async (req, res) => {
-        const { name, description, exclude_iters,
-                adjustment_segment, adjustment_bucket, unlabeled_load,
-                is_default } = req.body;
-        const set = (v) => (v === undefined ? null : (String(v).trim() || null));
-
-        // is_default decides where every account without a saved selection
-        // lands, so it is not the version's own setting the way its name is:
-        // it follows the pf.log rule, an admin override and nothing else. The
-        // UI greys the box out rather than offering a click that answers 403.
-        if (is_default !== undefined && !req.session?.user?.is_admin) {
-            return res.status(403).json({
-                error: 'Only an admin can set the default forecast'
-            });
-        }
-
-        // The write is two statements when the default moves -- clear the old
-        // one, set the new -- and the partial unique index refuses the moment
-        // in between, so they share a transaction.
-        const client = await pool.connect();
-        try {
-            await client.query('BEGIN');
-            if (is_default === true) {
-                await client.query(`
-                    UPDATE pf.version SET is_default = false
-                    WHERE source_id = (SELECT source_id FROM pf.version WHERE id = $1)
-                      AND id <> $1
-                      AND is_default
-                `, [req.params.id]);
-            }
-            const result = await client.query(`
                 UPDATE pf.version SET
-                    name               = COALESCE($2, name),
-                    description        = COALESCE($3, description),
-                    exclude_iters      = COALESCE($4, exclude_iters),
-                    adjustment_segment = CASE WHEN $5::bool  THEN $6::text  ELSE adjustment_segment END,
-                    adjustment_bucket  = CASE WHEN $7::bool  THEN $8::text  ELSE adjustment_bucket  END,
-                    unlabeled_load     = CASE WHEN $9::bool  THEN $10::text ELSE unlabeled_load     END,
-                    is_default         = CASE WHEN $11::bool THEN $12::bool ELSE is_default         END
+                    name          = COALESCE($2, name),
+                    description   = COALESCE($3, description),
+                    exclude_iters = COALESCE($4, exclude_iters)
                 WHERE id = $1
                 RETURNING *
             `, [
                 req.params.id,
                 name          || null,
                 description   || null,
-                exclude_iters ? JSON.stringify(exclude_iters) : null,
-                adjustment_segment !== undefined, set(adjustment_segment),
-                adjustment_bucket  !== undefined, set(adjustment_bucket),
-                unlabeled_load     !== undefined, set(unlabeled_load),
-                is_default !== undefined, is_default === true
+                exclude_iters ? JSON.stringify(exclude_iters) : null
             ]);
             if (result.rows.length === 0) {
-                await client.query('ROLLBACK');
                 return res.status(404).json({ error: 'Version not found' });
             }
-            await client.query('COMMIT');
             res.json(result.rows[0]);
         } catch (err) {
-            await client.query('ROLLBACK').catch(() => {});
             console.error(err);
-            // (source_id, name) is unique, so a rename can collide. The POST has
-            // said so for as long as it has existed; this said it with a raw
-            // Postgres message, which reads as a server fault rather than a name
-            // that is already taken.
-            if (err.code === '23505') {
-                return res.status(409).json({
-                    // Two unique constraints raise 23505 here, and the name is
-                    // only one of them -- version_one_default is the other,
-                    // reachable only if two requests set the default at once.
-                    error: err.constraint === 'version_one_default'
-                        ? 'Another forecast was just made the default — reload and try again'
-                        : 'A version with that name already exists for this source'
-                });
-            }
             res.status(500).json({ error: err.message });
-        } finally {
-            client.release();
         }
     });
 
     // close a version — blocks further edits
-    router.post('/versions/:id/close', requireAdmin, async (req, res) => {
-        const pf_user = sessionUser(req);
+    router.post('/versions/:id/close', async (req, res) => {
+        const { pf_user } = req.body;
         try {
             const result = await pool.query(`
                 UPDATE pf.version
@@ -615,7 +162,7 @@ ${colDefs},
     });
 
     // reopen a closed version
-    router.post('/versions/:id/reopen', requireAdmin, async (req, res) => {
+    router.post('/versions/:id/reopen', async (req, res) => {
         try {
             const result = await pool.query(`
                 UPDATE pf.version
@@ -635,7 +182,7 @@ ${colDefs},
 
     // delete a version — drops forecast table then deletes version record
     // log entries are removed by ON DELETE CASCADE on pf.log.version_id
-    router.delete('/versions/:id', requireAdmin, async (req, res) => {
+    router.delete('/versions/:id', async (req, res) => {
         const versionId = parseInt(req.params.id);
         const client = await pool.connect();
         try {

scripts/migrate.js

@@ -0,0 +1,72 @@
+#!/usr/bin/env node
+// Migration CLI.
+//
+//   npm run migrate            apply pending migrations
+//   npm run migrate:status     show what is applied and what is pending
+//   npm run migrate:baseline   record pending migrations as applied WITHOUT running
+//                              them — for a database that already matches
+//
+// Pass --up-to=<filename> to baseline only through a given migration.
+
+require('dotenv').config();
+const os = require('os');
+const { Client } = require('pg');
+const { status, migrate, baseline } = require('../lib/migrations');
+
+const args    = process.argv.slice(2);
+const command = args.find(a => !a.startsWith('--')) || 'up';
+const upTo    = (args.find(a => a.startsWith('--up-to=')) || '').split('=')[1] || null;
+
+function connect() {
+    return new Client({
+        host:     process.env.DB_HOST,
+        port:     parseInt(process.env.DB_PORT) || 5432,
+        database: process.env.DB_NAME,
+        user:     process.env.DB_USER,
+        password: process.env.DB_PASSWORD,
+        ssl:      false
+    });
+}
+
+async function showStatus(client) {
+    const st = await status(client);
+    console.log(`\n${process.env.DB_NAME}  (${st.applied.length} applied, ${st.pending.length} pending)\n`);
+    for (const m of st.onDisk) {
+        const hit = st.applied.find(a => a.filename === m.filename);
+        const mark = !hit ? 'PENDING'
+            : st.modified.includes(m.filename) ? 'MODIFIED'
+            : 'applied';
+        const when = hit ? hit.applied_at.toISOString().slice(0, 19).replace('T', ' ') : '';
+        console.log(`  ${mark.padEnd(9)} ${m.filename.padEnd(40)} ${when}`);
+    }
+    for (const f of st.missing) console.log(`  MISSING   ${f.padEnd(40)} recorded as applied but not on disk`);
+    if (st.modified.length)   console.log(`\n  ! modified after being applied: ${st.modified.join(', ')}`);
+    if (st.outOfOrder.length) console.log(`  ! would apply out of order: ${st.outOfOrder.join(', ')}`);
+    console.log('');
+    return st;
+}
+
+(async () => {
+    const client = connect();
+    await client.connect();
+    try {
+        const user = `${os.userInfo().username}@${os.hostname()}`;
+        if (command === 'status') {
+            const st = await showStatus(client);
+            process.exitCode = (st.modified.length || st.missing.length) ? 1 : 0;
+        } else if (command === 'baseline') {
+            await baseline(client, { user, upTo });
+            await showStatus(client);
+        } else if (command === 'up') {
+            await migrate(client, { user });
+        } else {
+            console.error(`Unknown command "${command}" — expected up, status, or baseline.`);
+            process.exitCode = 2;
+        }
+    } catch (err) {
+        console.error(`\nmigration error: ${err.message}\n`);
+        process.exitCode = 1;
+    } finally {
+        await client.end();
+    }
+})();

scripts/schema-dump.js

@@ -0,0 +1,50 @@
+#!/usr/bin/env node
+// Regenerate setup_sql/schema.generated.sql — a readable snapshot of the current
+// pf schema, for reference only. Migrations are the source of truth; this file is
+// generated so it cannot drift from the database the way a hand-maintained
+// schema file does. Never edit it, and never apply it to create a database.
+
+require('dotenv').config();
+const { spawnSync } = require('child_process');
+const fs   = require('fs');
+const path = require('path');
+
+const out = path.join(__dirname, '..', 'setup_sql', 'schema.generated.sql');
+
+const result = spawnSync('pg_dump', [
+    '--schema-only', '--no-owner', '--no-privileges',
+    '--schema=pf',
+    // per-version forecast tables are created at runtime — including them would
+    // make this file churn every time a version is created or dropped
+    '--exclude-table=pf.fc_*',
+    '-f', out
+], {
+    stdio: 'inherit',
+    env: {
+        ...process.env,
+        PGHOST:     process.env.DB_HOST,
+        PGPORT:     process.env.DB_PORT || '5432',
+        PGDATABASE: process.env.DB_NAME,
+        PGUSER:     process.env.DB_USER,
+        PGPASSWORD: process.env.DB_PASSWORD
+    }
+});
+
+if (result.error) {
+    console.error(`pg_dump failed: ${result.error.message}`);
+    process.exit(1);
+}
+if (result.status !== 0) process.exit(result.status);
+
+// Strip the lines pg_dump varies between runs — a random \restrict token and the
+// server/client version banner — so regenerating an unchanged schema produces an
+// identical file instead of a spurious diff.
+const cleaned = fs.readFileSync(out, 'utf8')
+    .split('\n')
+    .filter(l => !/^\\(un)?restrict /.test(l))
+    .filter(l => !/^-- Dumped (from|by) /.test(l))
+    .join('\n')
+    .replace(/\n{3,}/g, '\n\n');
+fs.writeFileSync(out, cleaned);
+
+console.log(`wrote ${path.relative(process.cwd(), out)} (${cleaned.split('\n').length} lines)`);

server.js

@@ -1,26 +1,15 @@
 require('dotenv').config();
 const express = require('express');
 const cors    = require('cors');
-const session = require('express-session');
-const PgSession = require('connect-pg-simple')(session);
 const { Pool, types } = require('pg');
-const { requireAuth } = require('./lib/auth');
 
 // Return bigint (oid 20) and numeric (oid 1700) as JS numbers instead of strings,
-// so apache-arrow encodes them as Int/Float64 rather than Dictionary<Utf8>.
+// so apache-arrow's tableFromJSON infers Int/Float64 rather than Dictionary<Utf8>.
 types.setTypeParser(20, v => v === null ? null : Number(v));
 types.setTypeParser(1700, v => v === null ? null : Number(v));
 
 const app = express();
-
-// Sessions ride on a cookie, so a wildcard CORS origin would let any site make
-// credentialed calls on behalf of a logged-in user. The UI is served from this
-// same origin and needs no CORS at all; set CORS_ORIGIN only for a separate
-// front-end host, and it is then allowed by name, never by wildcard.
-if (process.env.CORS_ORIGIN) {
-    app.use(cors({ origin: process.env.CORS_ORIGIN.split(',').map(o => o.trim()), credentials: true }));
-}
-
+app.use(cors());
 app.use(express.json());
 app.use(express.static('public/app'));
 
@@ -37,52 +26,45 @@ pool.on('error', (err) => {
     console.error('pg pool error', err);
 });
 
-// ── Authentication ────────────────────────────────────────────
-// Refuse to boot without a secret rather than fall back to a default one:
-// a predictable secret means forgeable session cookies.
-const sessionSecret = process.env.SESSION_SECRET;
-if (!sessionSecret) {
-    console.error('SESSION_SECRET is not set. Run: ./pf.sh config');
-    process.exit(1);
-}
-
-// TLS terminates at the reverse proxy, so express has to trust its headers for
-// req.ip (the login throttle) and for secure-cookie detection to be right.
-app.set('trust proxy', process.env.TRUST_PROXY || 1);
-
-const cookieSecure = process.env.COOKIE_SECURE !== 'false';
-if (!cookieSecure) {
-    console.warn('COOKIE_SECURE=false — session cookie will be sent over plain HTTP.');
-}
-
-app.use(session({
-    name:   'pf.sid',
-    store:  new PgSession({ pool, schemaName: 'pf', tableName: 'session', createTableIfMissing: false }),
-    secret: sessionSecret,
-    resave: false,
-    saveUninitialized: false,
-    rolling: true,
-    cookie: {
-        httpOnly: true,
-        sameSite: 'lax',
-        secure:   cookieSecure,
-        maxAge:   1000 * 60 * 60 * 12,
-    },
-}));
-
-app.use('/api', require('./routes/auth')(pool));
-
-// Everything below this line requires a session.
-app.use('/api', requireAuth);
-
 app.use('/api', require('./routes/tables')(pool));
 app.use('/api', require('./routes/sources')(pool));
 app.use('/api', require('./routes/versions')(pool));
 app.use('/api', require('./routes/operations')(pool));
 app.use('/api', require('./routes/log')(pool));
-app.use('/api', require('./routes/layouts')(pool));
-app.use('/api', require('./routes/plans')(pool));
 
 
+// Refuse to start against a database whose schema is behind the migrations on
+// disk. Without this the mismatch surfaces later as a confusing query error
+// ('column "x" does not exist') deep inside an unrelated request.
+// Set PF_SKIP_MIGRATION_CHECK=1 to bypass.
+async function checkSchema() {
+    if (process.env.PF_SKIP_MIGRATION_CHECK === '1') return;
+    const { status } = require('./lib/migrations');
+    const client = await pool.connect();
+    try {
+        const st = await status(client);
+        const problems = [
+            st.pending.length  && `${st.pending.length} pending: ${st.pending.map(m => m.filename).join(', ')}`,
+            st.modified.length && `modified after being applied: ${st.modified.join(', ')}`,
+            st.missing.length  && `applied but missing from disk: ${st.missing.join(', ')}`
+        ].filter(Boolean);
+        if (problems.length) {
+            console.error('\n  Database schema is out of date:');
+            for (const p of problems) console.error(`    - ${p}`);
+            console.error('\n  Run "npm run migrate" (or "npm run migrate:status" for detail).');
+            console.error('  For a database that already matches, "npm run migrate:baseline".\n');
+            throw new Error('schema out of date');
+        }
+        console.log(`schema up to date (${st.applied.length} migrations applied)`);
+    } finally {
+        client.release();
+    }
+}
+
 const port = process.env.PORT || 3010;
-app.listen(port, '0.0.0.0', () => console.log(`pf_app started on port ${port}`));
+checkSchema()
+    .then(() => app.listen(port, '0.0.0.0', () => console.log(`pf_app started on port ${port}`)))
+    .catch((err) => {
+        if (err.message !== 'schema out of date') console.error('startup failed:', err.message);
+        process.exit(1);
+    });

setup_sql/01_schema.sql

@@ -1,361 +0,0 @@
--- Pivot Forecast schema install
--- Run once against target database: psql -d <db> -f setup_sql/01_schema.sql
-
-CREATE SCHEMA IF NOT EXISTS pf;
-
-CREATE TABLE IF NOT EXISTS pf.source (
-    id             serial PRIMARY KEY,
-    schema         text NOT NULL,
-    tname          text NOT NULL,
-    label          text,
-    status         text NOT NULL DEFAULT 'active',  -- active | archived
-    default_layout jsonb,                            -- Perspective view config used as the per-source default
-    created_at     timestamptz NOT NULL DEFAULT now(),
-    created_by     text,
-    UNIQUE (schema, tname)
-);
-
--- backfill columns for existing installs
-ALTER TABLE pf.source ADD COLUMN IF NOT EXISTS default_layout jsonb;
-
--- pf.dim_period: run setup_sql/gen_dim_period.sql to create and populate
-
-CREATE TABLE IF NOT EXISTS pf.col_meta (
-    id          serial PRIMARY KEY,
-    source_id   integer NOT NULL REFERENCES pf.source(id) ON DELETE CASCADE,
-    cname       text NOT NULL,
-    label       text,
-    role        text NOT NULL DEFAULT 'ignore',   -- dimension | value | units | date | ignore
-    is_key      boolean NOT NULL DEFAULT false,   -- true = usable in WHERE slice
-    dim_group   text,                             -- groups functionally dependent columns
-    dim_period_col text,                          -- pf.dim_period column this dimension derives from
-    in_grain    boolean NOT NULL DEFAULT false,   -- true = column defines the display grain
-    opos        integer,
-    UNIQUE (source_id, cname)
-);
-
--- backfill columns for existing installs (must follow the CREATE above)
-ALTER TABLE pf.col_meta ADD COLUMN IF NOT EXISTS dim_group text;
-ALTER TABLE pf.col_meta ADD COLUMN IF NOT EXISTS dim_period_col text;
-ALTER TABLE pf.col_meta ADD COLUMN IF NOT EXISTS in_grain boolean NOT NULL DEFAULT false;
-
-CREATE TABLE IF NOT EXISTS pf.version (
-    id              serial PRIMARY KEY,
-    source_id       integer NOT NULL REFERENCES pf.source(id) ON DELETE RESTRICT,
-    name            text NOT NULL,
-    description     text,
-    status          text NOT NULL DEFAULT 'open',               -- open | closed
-    exclude_iters   jsonb NOT NULL DEFAULT '["reference"]'::jsonb,
-    created_at      timestamptz NOT NULL DEFAULT now(),
-    created_by      text,
-    closed_at       timestamptz,
-    closed_by       text,
-    UNIQUE (source_id, name)
-);
-
-CREATE TABLE IF NOT EXISTS pf.log (
-    id          bigserial PRIMARY KEY,
-    version_id  integer NOT NULL REFERENCES pf.version(id) ON DELETE CASCADE,
-    pf_user     text NOT NULL,
-    stamp       timestamptz NOT NULL DEFAULT now(),
-    operation   text NOT NULL,   -- baseline | reference | scale | recode | clone
-    slice       jsonb,
-    params      jsonb,
-    note        text,
-    tag         text            -- initiative label, e.g. 'reduce_spend'; groups
-                                -- adjustments into a bridge from baseline to current
-);
-
--- adding tags to an install that predates them
-ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS tag text;
-CREATE INDEX IF NOT EXISTS log_tag_idx ON pf.log (tag) WHERE tag IS NOT NULL;
-
--- seed tags for loads that predate the column: a baseline/reference note is the
--- segment's name ('Open Orders', 'Prior Year'), which is exactly what tag holds.
--- Adjustment notes are free text, not labels, so they are left alone.
-UPDATE pf.log
-SET tag = note
-WHERE TRUE
-    AND tag IS NULL
-    AND note IS NOT NULL
-    AND note <> ''
-    AND operation IN ('baseline', 'reference');
-
--- Display order for the pivot's segment and bucket columns.
---
--- The segment's display name in the pivot, falling back to tag then note.
---
--- Separate from both because those have jobs already -- tag groups adjustments
--- into initiatives for the bridge, note is free commentary -- and because the
--- label carries the sort order. Perspective orders column groups by the value
--- string, so a leading "01 - " is how ordering is expressed; putting that in the
--- note would put it in every note.
-ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS label text;
-
--- Vestigial, both of them. They held the ordinal when the "01 - " prefix was
--- computed for the pivot rather than typed into label and bucket: bucket_order
--- sequenced the bucket columns, log.seq the segments within them. Nothing reads
--- either now, and nothing writes them -- kept only because dropping a column is
--- not worth a migration to reclaim two that cost nothing.
-ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS bucket_order jsonb;
-ALTER TABLE pf.log     ADD COLUMN IF NOT EXISTS seq integer;
-
--- What a segment contributes to, independent of pf_iter.
---
--- pf_iter answers "can operations write to these rows"; bucket answers "does this
--- belong in the forecast number". Those are not the same question -- Open Orders is
--- loaded as reference so nothing adjusts it, yet it is part of the forecast -- so
--- neither can be derived from the other.
---
--- Free text with suggested values (Forecast / Prior Year / Prior Prior Year / Plan)
--- rather than an enum, so a new banner does not need a migration. Blank by default:
--- until a segment is labelled, the pivot falls back to showing its own name.
-ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS bucket text;
-CREATE INDEX IF NOT EXISTS log_bucket_idx ON pf.log (bucket) WHERE bucket IS NOT NULL;
-
--- The names a row falls back to when nobody has named it, per scenario. Null
--- means "use the built-in", which is the DISPLAY DEFAULTS block in
--- lib/sql_generator.js. Read through a join at query time, not baked into
--- pf.sql: those templates are keyed on (source_id, operation) and shared by
--- every version of a source.
-ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS adjustment_segment text;
-ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS adjustment_bucket  text;
-ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS unlabeled_load     text;
-
--- Every log entry is named. label is the row's segment and bucket what it counts
--- toward, both stored rather than derived, so a report reads them off pf.log with
--- no rule of its own. The routes stamp them on write: a load's label is required
--- and its bucket defaults to the label; an adjustment gets the version's
--- adjustment_segment / adjustment_bucket, else the built-ins in sql_generator.
---
--- The backfill names entries written before that, with the rule the pivot used to
--- apply at read time, so nothing already on screen changes name. It is the last
--- reader of pf.version.unlabeled_load; the column remains.
-WITH
--- the read-time rule, verbatim
-named AS (
-    SELECT
-         l.id
-        ,CASE
-            WHEN l.operation IN ('baseline', 'reference')
-                THEN COALESCE(NULLIF(l.label, ''), NULLIF(l.tag, ''), NULLIF(l.note, ''), NULLIF(v.unlabeled_load, ''), 'Unlabeled')
-            ELSE COALESCE(NULLIF(l.label, ''), NULLIF(v.adjustment_segment, ''), '99 - Adjustments')
-         END AS label
-        ,l.operation
-        ,l.bucket
-        ,v.adjustment_bucket
-    FROM
-        pf.log l
-        LEFT JOIN pf.version v
-            ON v.id = l.version_id
-    WHERE TRUE
-        AND (NULLIF(l.label, '') IS NULL OR NULLIF(l.bucket, '') IS NULL)
-)
-UPDATE pf.log l
-SET
-     label  = n.label
-    ,bucket = COALESCE(
-         NULLIF(n.bucket, '')
-        ,CASE
-            WHEN n.operation IN ('baseline', 'reference') THEN n.label
-            ELSE COALESCE(NULLIF(n.adjustment_bucket, ''), '04 - Forecast')
-         END
-     )
-FROM
-    named n
-WHERE TRUE
-    AND l.id = n.id;
-
-ALTER TABLE pf.log DROP CONSTRAINT IF EXISTS log_named;
-ALTER TABLE pf.log ADD CONSTRAINT log_named
-    CHECK (NULLIF(label, '') IS NOT NULL AND NULLIF(bucket, '') IS NOT NULL);
-
--- Where an account lands when it has no saved selection of its own.
---
--- Without it the app fell back to the newest version, which is whatever
--- somebody created last -- a scratch scenario as readily as the plan. Sales
--- accounts, which never pick a version, therefore followed whatever the
--- forecasting work happened to be doing that morning.
---
--- At most one per source, by partial unique index, the way pf.layout does it:
--- two defaults is not a state anything could resolve, so the database refuses
--- it rather than the route picking one.
-ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS is_default boolean NOT NULL DEFAULT false;
-
-CREATE UNIQUE INDEX IF NOT EXISTS version_one_default
-    ON pf.version (source_id)
-    WHERE is_default;
-
--- What the entry did, stamped when it did it.
---
--- Not a cache: a log entry's forecast rows never change after it is written.
--- Only the operation that owns the logid inserts them, and the only thing that
--- removes them is undo, which deletes this row too -- so these totals are fixed
--- at write time rather than derived from something that can move underneath
--- them. The change log was joining the whole forecast table to recompute them
--- on every open, 2.5M rows to total a few thousand.
---
--- measure_cols records which columns they are denominated in, since the value
--- and units roles can be reassigned in col_meta and the numbers would otherwise
--- quietly come to mean something else.
-ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS row_count    integer;
-ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS value_total  double precision;
-ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS units_total  double precision;
-ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS measure_cols jsonb;
-
--- The column an account's territory is expressed in -- the rep, the region,
--- whatever this source divides ownership by. Exactly one per source.
---
--- Flagged here rather than named in code because it is source-specific, the
--- same way the grain is: a second source should not need a code change to be
--- scoped by something other than a sales rep.
---
--- It does two jobs. The server scopes every read and write to the values on the
--- account, and recode refuses to *set* this column unless the account is an
--- admin -- moving a row between territories is reassignment, not forecasting.
-ALTER TABLE pf.col_meta ADD COLUMN IF NOT EXISTS is_territory boolean NOT NULL DEFAULT false;
-
--- The debug path: what the entry was trying to do, and what that became.
---
--- Both, deliberately. params records the intent -- the slice, the scope, the
--- resolved increments -- and sql_text records the statement as executed, with
--- territory and scope already resolved into it. Keeping only one of them
--- assumes the translation between them is correct, which is exactly the
--- assumption in doubt when the rows look wrong.
---
--- env captures the state the intent was executed against that cannot be
--- reconstructed afterwards: the territory in force, the version's
--- exclude_iters, and when the template was generated. All three are mutable
--- rows elsewhere, and nothing remembers what they were.
---
--- The template generation is a fingerprint, not a version: it cannot reproduce
--- the old template, but it can tell you the entry ran under a different one,
--- which is what would otherwise make a replay quietly wrong.
-ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS env      jsonb;
-ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS sql_text text;
-
--- Master data for a dim_group: one row per key value, with its sibling columns.
---
--- The source is transactional and often a view over all history, so deriving a
--- member list from it is both slow and wrong -- slow because it means scanning
--- millions of rows, wrong because it can only describe what was transacted and
--- has no way to say a part is discontinued or that a new one exists before it
--- has sold. This table is the app's own list, refreshed from the source but
--- curatable independently of it.
-CREATE TABLE IF NOT EXISTS pf.dim_member (
-    source_id    integer NOT NULL REFERENCES pf.source(id) ON DELETE CASCADE,
-    dim_group    text    NOT NULL,   -- matches pf.col_meta.dim_group
-    key_value    text    NOT NULL,   -- the is_key column's value
-    attrs        jsonb   NOT NULL DEFAULT '{}'::jsonb,  -- the sibling columns
-    is_active    boolean NOT NULL DEFAULT true,
-    source_seen  boolean NOT NULL DEFAULT true,  -- present in the source at last refresh
-    added_at     timestamptz NOT NULL DEFAULT now(),
-    updated_at   timestamptz NOT NULL DEFAULT now(),
-    refreshed_at timestamptz,
-    PRIMARY KEY (source_id, dim_group, key_value)
-);
-
-CREATE INDEX IF NOT EXISTS dim_member_active_idx
-    ON pf.dim_member (source_id, dim_group) WHERE is_active;
-
--- generated operation SQL per source, stored after col_meta is configured
-CREATE TABLE IF NOT EXISTS pf.sql (
-    id           serial PRIMARY KEY,
-    source_id    integer NOT NULL REFERENCES pf.source(id) ON DELETE CASCADE,
-    operation    text NOT NULL,  -- get_data | baseline | reference | scale | recode | clone | undo
-    sql          text NOT NULL,
-    generated_at timestamptz NOT NULL DEFAULT now(),
-    UNIQUE (source_id, operation)
-);
-
--- pf.layout: named Perspective view configs.
---
--- Replaces the per-browser localStorage lists (pf_layouts_v*) and the single
--- anonymous pf.source.default_layout. A layout is either `private` -- visible
--- only to its owner -- or `published`, visible to everyone on the version and
--- writable only by its owner or an admin, the same rule pf.log already uses.
-CREATE TABLE IF NOT EXISTS pf.layout (
-    id          serial PRIMARY KEY,
-    source_id   integer NOT NULL REFERENCES pf.source(id)  ON DELETE CASCADE,
-    -- null = applies to every version of the source; that is where the old
-    -- source default lives, and what a new version picks up before anyone has
-    -- published a layout of its own.
-    version_id  integer          REFERENCES pf.version(id) ON DELETE CASCADE,
-    name        text    NOT NULL,
-    config      jsonb   NOT NULL,
-    owner       text    NOT NULL,
-    visibility  text    NOT NULL DEFAULT 'private',   -- private | published
-    is_default  boolean NOT NULL DEFAULT false,       -- applied on first load
-    created_at  timestamptz NOT NULL DEFAULT now(),
-    updated_at  timestamptz NOT NULL DEFAULT now(),
-    CHECK (visibility IN ('private', 'published')),
-    -- only a published layout can be the default: a private one would be a
-    -- default nobody else could see.
-    CHECK (NOT is_default OR visibility = 'published')
-);
-
--- COALESCE rather than the bare column: version_id is nullable, and in a unique
--- index NULLs are distinct, so source-wide rows would not be constrained at all.
-CREATE UNIQUE INDEX IF NOT EXISTS layout_private_name
-    ON pf.layout (source_id, COALESCE(version_id, 0), owner, lower(name))
-    WHERE visibility = 'private';
-
-CREATE UNIQUE INDEX IF NOT EXISTS layout_published_name
-    ON pf.layout (source_id, COALESCE(version_id, 0), lower(name))
-    WHERE visibility = 'published';
-
--- one default per scope
-CREATE UNIQUE INDEX IF NOT EXISTS layout_one_default
-    ON pf.layout (source_id, COALESCE(version_id, 0))
-    WHERE is_default;
-
-CREATE INDEX IF NOT EXISTS layout_lookup ON pf.layout (source_id, version_id);
-
--- Carry the old single source default across as a published, source-wide row.
--- Idempotent: skipped once a default exists for that source.
-INSERT INTO pf.layout (source_id, version_id, name, config, owner, visibility, is_default)
-SELECT s.id, NULL, 'Source default', s.default_layout, COALESCE(s.created_by, 'admin'), 'published', true
-FROM pf.source s
-WHERE TRUE
-  AND s.default_layout IS NOT NULL
-  AND NOT EXISTS (
-      SELECT 1 FROM pf.layout l
-      WHERE TRUE
-        AND l.source_id = s.id
-        AND l.version_id IS NULL
-        AND l.is_default
-  );
-
--- pf.load_plan: a segment's definition, before and apart from having been run.
---
--- Until now a segment existed only once it had been loaded -- as a pf.log entry
--- with forecast rows behind it -- so cloning a version's definition meant
--- re-running every load, and there was nowhere to hold "this is the cut I want,
--- but the dates need changing first".
---
--- Deliberately not pf.log. That table is the audit log: every row in it means a
--- write happened, and putting unexecuted intent there would cost it that
--- meaning. A plan is intent; run_logid is the pf.log entry it produced.
-CREATE TABLE IF NOT EXISTS pf.load_plan (
-    id          serial PRIMARY KEY,
-    version_id  integer NOT NULL REFERENCES pf.version(id) ON DELETE CASCADE,
-    seq         integer NOT NULL DEFAULT 0,      -- run order
-    kind        text    NOT NULL,                -- baseline | reference
-    label       text,
-    bucket      text,
-    tag         text,
-    note        text,
-    filters     jsonb,                           -- the structured filter builder's own shape
-    raw_where   text,                            -- the raw-SQL escape hatch, when used instead
-    date_offset text    NOT NULL DEFAULT '0 days',
-    -- the pf.log entry this plan produced, or null while it is still a plan.
-    -- ON DELETE SET NULL so undoing a load returns its plan to unrun rather than
-    -- deleting the definition with it.
-    run_logid   bigint           REFERENCES pf.log(id) ON DELETE SET NULL,
-    created_at  timestamptz NOT NULL DEFAULT now(),
-    updated_at  timestamptz NOT NULL DEFAULT now(),
-    created_by  text,
-    CHECK (kind IN ('baseline', 'reference'))
-);
-
-CREATE INDEX IF NOT EXISTS load_plan_version ON pf.load_plan (version_id, seq, id);

setup_sql/02_auth.sql

@@ -1,38 +0,0 @@
--- Pivot Forecast — authentication
--- Run after 01_schema.sql: psql -d <db> -f setup_sql/02_auth.sql
--- Safe to re-run.
-
--- Application accounts. Passwords are scrypt hashes written by lib/auth.js;
--- the plaintext never reaches the database. Manage with ./pf.sh add-user.
-CREATE TABLE IF NOT EXISTS pf.app_user (
-    id            serial PRIMARY KEY,
-    username      text NOT NULL UNIQUE,
-    pass_hash     text NOT NULL,
-    display_name  text,
-    is_active     boolean NOT NULL DEFAULT true,
-    created_at    timestamptz NOT NULL DEFAULT now(),
-    last_login_at timestamptz
-);
-
--- Session store for express-session (connect-pg-simple layout). Sessions live
--- here rather than in memory so a restart doesn't sign everyone out, and so a
--- session can be revoked by deleting its row.
-CREATE TABLE IF NOT EXISTS pf.session (
-    sid    varchar PRIMARY KEY NOT NULL COLLATE "default",
-    sess   json NOT NULL,
-    expire timestamp(6) NOT NULL
-);
-
-CREATE INDEX IF NOT EXISTS session_expire_idx ON pf.session (expire);
-
--- What an account may see and change.
---
--- territory is the list of values allowed in the source's territory column
--- (col_meta.is_territory). It is a filter the server applies to every read and
--- every write; it is never sent by the client, which could remove it.
---
--- Fail closed: no territory and not an admin means no rows. An account created
--- without one sees nothing until someone grants it, rather than seeing the whole
--- book because a column was left null.
-ALTER TABLE pf.app_user ADD COLUMN IF NOT EXISTS territory jsonb;
-ALTER TABLE pf.app_user ADD COLUMN IF NOT EXISTS is_admin  boolean NOT NULL DEFAULT false;

setup_sql/README.md

@@ -0,0 +1,64 @@
+# Database setup
+
+## Migrations
+
+`migrations/*.sql` are applied in filename order and recorded in
+`pf.schema_version`. They are the **source of truth** for the `pf` schema — there
+is no hand-maintained current-state file to drift out of sync.
+
+```bash
+npm run migrate            # apply pending migrations
+npm run migrate:status     # what is applied, what is pending
+npm run migrate:baseline   # record pending as applied WITHOUT running them
+```
+
+`server.js` refuses to start when the database is behind, so drift surfaces at
+boot rather than as `column "x" does not exist` inside an unrelated request. Set
+`PF_SKIP_MIGRATION_CHECK=1` to bypass.
+
+### Fresh database
+
+```bash
+npm run migrate
+psql -d <db> -f setup_sql/gen_dim_period.sql
+```
+
+### Existing database that already matches
+
+Use `baseline` so the runner does not try to re-create tables that exist:
+
+```bash
+npm run migrate:baseline
+```
+
+To baseline only part of the way — the schema matches through `0003` but not
+`0004` — pass `--up-to` and then migrate the rest:
+
+```bash
+node scripts/migrate.js baseline --up-to=0003_col_meta_dim_group_period.sql
+npm run migrate
+```
+
+## Writing a migration
+
+- Name it `NNNN_short_description.sql`, numbered after the highest existing file.
+- One concern per file. Keep it forward-only; there are no down migrations.
+- **Applied migrations are immutable.** The runner stores a checksum and refuses
+  to proceed if a file changes after being applied, because editing one means
+  databases silently disagree about what the schema is. To fix a mistake, add a
+  new migration.
+- No `IF NOT EXISTS` guards on new migrations. The bookkeeping already guarantees
+  each runs once, and the guards hide ordering mistakes — the reason the old
+  `01_schema.sql` had `ALTER`s sitting above the `CREATE TABLE` they depended on,
+  broken for anyone installing from scratch. `0004` is the one exception, since it
+  was applied by hand before migrations existed.
+
+## Not migrations
+
+- **`gen_dim_period.sql`** — creates and populates `pf.dim_period`. It is a
+  parameterised data load (configurable fiscal year start month), not a schema
+  change, so it stays a script you run deliberately.
+- **`pf.fc_{tname}_{version_id}`** — per-version forecast tables, created and
+  dropped at runtime by `routes/versions.js` from `col_meta`. Never migrated.
+- **`schema.generated.sql`** — a `pg_dump` snapshot for reading, refreshed with
+  `npm run schema:dump`. Generated, never edited, never applied.

setup_sql/migrations/0001_initial_schema.sql

@@ -0,0 +1,63 @@
+-- Initial pf schema: sources, column metadata, versions, audit log, generated SQL.
+--
+-- This is the schema as it stood before the additive columns in later migrations.
+-- Columns added afterwards are declared once, in their own migration — not here.
+
+CREATE SCHEMA IF NOT EXISTS pf;
+
+CREATE TABLE pf.source (
+    id         serial PRIMARY KEY,
+    schema     text NOT NULL,
+    tname      text NOT NULL,
+    label      text,
+    status     text NOT NULL DEFAULT 'active',  -- active | archived
+    created_at timestamptz NOT NULL DEFAULT now(),
+    created_by text,
+    UNIQUE (schema, tname)
+);
+
+CREATE TABLE pf.col_meta (
+    id        serial PRIMARY KEY,
+    source_id integer NOT NULL REFERENCES pf.source(id) ON DELETE CASCADE,
+    cname     text NOT NULL,
+    label     text,
+    role      text NOT NULL DEFAULT 'ignore',   -- dimension | value | units | date | filter | ignore
+    is_key    boolean NOT NULL DEFAULT false,   -- true = usable in WHERE slice
+    opos      integer,
+    UNIQUE (source_id, cname)
+);
+
+CREATE TABLE pf.version (
+    id            serial PRIMARY KEY,
+    source_id     integer NOT NULL REFERENCES pf.source(id) ON DELETE RESTRICT,
+    name          text NOT NULL,
+    description   text,
+    status        text NOT NULL DEFAULT 'open',  -- open | closed
+    exclude_iters jsonb NOT NULL DEFAULT '["reference"]'::jsonb,
+    created_at    timestamptz NOT NULL DEFAULT now(),
+    created_by    text,
+    closed_at     timestamptz,
+    closed_by     text,
+    UNIQUE (source_id, name)
+);
+
+CREATE TABLE pf.log (
+    id         bigserial PRIMARY KEY,
+    version_id integer NOT NULL REFERENCES pf.version(id) ON DELETE CASCADE,
+    pf_user    text NOT NULL,
+    stamp      timestamptz NOT NULL DEFAULT now(),
+    operation  text NOT NULL,   -- baseline | reference | scale | recode | clone
+    slice      jsonb,
+    params     jsonb,
+    note       text
+);
+
+-- generated operation SQL per source, stored after col_meta is configured
+CREATE TABLE pf.sql (
+    id           serial PRIMARY KEY,
+    source_id    integer NOT NULL REFERENCES pf.source(id) ON DELETE CASCADE,
+    operation    text NOT NULL,  -- get_data | get_agg | baseline | reference | scale | recode | clone | undo
+    sql          text NOT NULL,
+    generated_at timestamptz NOT NULL DEFAULT now(),
+    UNIQUE (source_id, operation)
+);

setup_sql/migrations/0002_source_default_layout.sql

@@ -0,0 +1,4 @@
+-- Per-source default Perspective view config, applied when a version has no
+-- saved layout of its own.
+
+ALTER TABLE pf.source ADD COLUMN default_layout jsonb;

setup_sql/migrations/0003_col_meta_dim_group_period.sql

@@ -0,0 +1,7 @@
+-- dim_group groups functionally dependent columns (e.g. a date and the year/month
+-- dimensions derived from it). dim_period_col maps such a dimension to a
+-- pf.dim_period column, so date-adjacent values are derived by JOIN at load time
+-- rather than copied raw from the source.
+
+ALTER TABLE pf.col_meta ADD COLUMN dim_group      text;
+ALTER TABLE pf.col_meta ADD COLUMN dim_period_col text;

setup_sql/migrations/0004_col_meta_in_grain.sql

@@ -0,0 +1,8 @@
+-- in_grain flags dimension/date columns that define the display grain: the
+-- forecast load is pre-aggregated to the flagged columns instead of shipping raw
+-- rows. See pf_spec.md -> Display-grain pre-aggregation.
+--
+-- Already applied by hand on the original dev database before migrations existed;
+-- IF NOT EXISTS keeps replay safe there. New migrations should not need the guard.
+
+ALTER TABLE pf.col_meta ADD COLUMN IF NOT EXISTS in_grain boolean NOT NULL DEFAULT false;

setup_sql/schema.generated.sql

@@ -0,0 +1,384 @@
+--
+-- PostgreSQL database dump
+--
+
+SET statement_timeout = 0;
+SET lock_timeout = 0;
+SET idle_in_transaction_session_timeout = 0;
+SET transaction_timeout = 0;
+SET client_encoding = 'UTF8';
+SET standard_conforming_strings = on;
+SELECT pg_catalog.set_config('search_path', '', false);
+SET check_function_bodies = false;
+SET xmloption = content;
+SET client_min_messages = warning;
+SET row_security = off;
+
+--
+-- Name: pf; Type: SCHEMA; Schema: -; Owner: -
+--
+
+CREATE SCHEMA pf;
+
+SET default_tablespace = '';
+
+SET default_table_access_method = heap;
+
+--
+-- Name: col_meta; Type: TABLE; Schema: pf; Owner: -
+--
+
+CREATE TABLE pf.col_meta (
+    id integer NOT NULL,
+    source_id integer NOT NULL,
+    cname text NOT NULL,
+    label text,
+    role text DEFAULT 'ignore'::text NOT NULL,
+    is_key boolean DEFAULT false NOT NULL,
+    opos integer,
+    dim_group text,
+    dim_period_col text,
+    in_grain boolean DEFAULT false NOT NULL
+);
+
+--
+-- Name: col_meta_id_seq; Type: SEQUENCE; Schema: pf; Owner: -
+--
+
+CREATE SEQUENCE pf.col_meta_id_seq
+    AS integer
+    START WITH 1
+    INCREMENT BY 1
+    NO MINVALUE
+    NO MAXVALUE
+    CACHE 1;
+
+--
+-- Name: col_meta_id_seq; Type: SEQUENCE OWNED BY; Schema: pf; Owner: -
+--
+
+ALTER SEQUENCE pf.col_meta_id_seq OWNED BY pf.col_meta.id;
+
+--
+-- Name: dim_period; Type: TABLE; Schema: pf; Owner: -
+--
+
+CREATE TABLE pf.dim_period (
+    sdat date NOT NULL,
+    edat date NOT NULL,
+    drange daterange NOT NULL,
+    ndays integer NOT NULL,
+    cal_year integer NOT NULL,
+    cal_quarter integer NOT NULL,
+    cal_month integer NOT NULL,
+    cal_month_abbr text NOT NULL,
+    cal_month_name text NOT NULL,
+    cal_label text NOT NULL,
+    fisc_year integer NOT NULL,
+    fisc_quarter integer NOT NULL,
+    fisc_quarter_label text NOT NULL,
+    fisc_month integer NOT NULL,
+    fisc_month_abbr text NOT NULL,
+    fisc_month_name text NOT NULL,
+    fisc_label text NOT NULL,
+    period_key text NOT NULL
+);
+
+--
+-- Name: log; Type: TABLE; Schema: pf; Owner: -
+--
+
+CREATE TABLE pf.log (
+    id bigint NOT NULL,
+    version_id integer NOT NULL,
+    pf_user text NOT NULL,
+    stamp timestamp with time zone DEFAULT now() NOT NULL,
+    operation text NOT NULL,
+    slice jsonb,
+    params jsonb,
+    note text
+);
+
+--
+-- Name: log_id_seq; Type: SEQUENCE; Schema: pf; Owner: -
+--
+
+CREATE SEQUENCE pf.log_id_seq
+    START WITH 1
+    INCREMENT BY 1
+    NO MINVALUE
+    NO MAXVALUE
+    CACHE 1;
+
+--
+-- Name: log_id_seq; Type: SEQUENCE OWNED BY; Schema: pf; Owner: -
+--
+
+ALTER SEQUENCE pf.log_id_seq OWNED BY pf.log.id;
+
+--
+-- Name: schema_version; Type: TABLE; Schema: pf; Owner: -
+--
+
+CREATE TABLE pf.schema_version (
+    filename text NOT NULL,
+    checksum text NOT NULL,
+    applied_at timestamp with time zone DEFAULT now() NOT NULL,
+    applied_by text
+);
+
+--
+-- Name: source; Type: TABLE; Schema: pf; Owner: -
+--
+
+CREATE TABLE pf.source (
+    id integer NOT NULL,
+    schema text NOT NULL,
+    tname text NOT NULL,
+    label text,
+    status text DEFAULT 'active'::text NOT NULL,
+    created_at timestamp with time zone DEFAULT now() NOT NULL,
+    created_by text,
+    default_layout jsonb
+);
+
+--
+-- Name: source_id_seq; Type: SEQUENCE; Schema: pf; Owner: -
+--
+
+CREATE SEQUENCE pf.source_id_seq
+    AS integer
+    START WITH 1
+    INCREMENT BY 1
+    NO MINVALUE
+    NO MAXVALUE
+    CACHE 1;
+
+--
+-- Name: source_id_seq; Type: SEQUENCE OWNED BY; Schema: pf; Owner: -
+--
+
+ALTER SEQUENCE pf.source_id_seq OWNED BY pf.source.id;
+
+--
+-- Name: sql; Type: TABLE; Schema: pf; Owner: -
+--
+
+CREATE TABLE pf.sql (
+    id integer NOT NULL,
+    source_id integer NOT NULL,
+    operation text NOT NULL,
+    sql text NOT NULL,
+    generated_at timestamp with time zone DEFAULT now() NOT NULL
+);
+
+--
+-- Name: sql_id_seq; Type: SEQUENCE; Schema: pf; Owner: -
+--
+
+CREATE SEQUENCE pf.sql_id_seq
+    AS integer
+    START WITH 1
+    INCREMENT BY 1
+    NO MINVALUE
+    NO MAXVALUE
+    CACHE 1;
+
+--
+-- Name: sql_id_seq; Type: SEQUENCE OWNED BY; Schema: pf; Owner: -
+--
+
+ALTER SEQUENCE pf.sql_id_seq OWNED BY pf.sql.id;
+
+--
+-- Name: version; Type: TABLE; Schema: pf; Owner: -
+--
+
+CREATE TABLE pf.version (
+    id integer NOT NULL,
+    source_id integer NOT NULL,
+    name text NOT NULL,
+    description text,
+    status text DEFAULT 'open'::text NOT NULL,
+    exclude_iters jsonb DEFAULT '["reference"]'::jsonb NOT NULL,
+    created_at timestamp with time zone DEFAULT now() NOT NULL,
+    created_by text,
+    closed_at timestamp with time zone,
+    closed_by text
+);
+
+--
+-- Name: version_id_seq; Type: SEQUENCE; Schema: pf; Owner: -
+--
+
+CREATE SEQUENCE pf.version_id_seq
+    AS integer
+    START WITH 1
+    INCREMENT BY 1
+    NO MINVALUE
+    NO MAXVALUE
+    CACHE 1;
+
+--
+-- Name: version_id_seq; Type: SEQUENCE OWNED BY; Schema: pf; Owner: -
+--
+
+ALTER SEQUENCE pf.version_id_seq OWNED BY pf.version.id;
+
+--
+-- Name: col_meta id; Type: DEFAULT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.col_meta ALTER COLUMN id SET DEFAULT nextval('pf.col_meta_id_seq'::regclass);
+
+--
+-- Name: log id; Type: DEFAULT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.log ALTER COLUMN id SET DEFAULT nextval('pf.log_id_seq'::regclass);
+
+--
+-- Name: source id; Type: DEFAULT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.source ALTER COLUMN id SET DEFAULT nextval('pf.source_id_seq'::regclass);
+
+--
+-- Name: sql id; Type: DEFAULT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.sql ALTER COLUMN id SET DEFAULT nextval('pf.sql_id_seq'::regclass);
+
+--
+-- Name: version id; Type: DEFAULT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.version ALTER COLUMN id SET DEFAULT nextval('pf.version_id_seq'::regclass);
+
+--
+-- Name: col_meta col_meta_pkey; Type: CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.col_meta
+    ADD CONSTRAINT col_meta_pkey PRIMARY KEY (id);
+
+--
+-- Name: col_meta col_meta_source_id_cname_key; Type: CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.col_meta
+    ADD CONSTRAINT col_meta_source_id_cname_key UNIQUE (source_id, cname);
+
+--
+-- Name: dim_period dim_period_pkey; Type: CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.dim_period
+    ADD CONSTRAINT dim_period_pkey PRIMARY KEY (sdat);
+
+--
+-- Name: log log_pkey; Type: CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.log
+    ADD CONSTRAINT log_pkey PRIMARY KEY (id);
+
+--
+-- Name: schema_version schema_version_pkey; Type: CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.schema_version
+    ADD CONSTRAINT schema_version_pkey PRIMARY KEY (filename);
+
+--
+-- Name: source source_pkey; Type: CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.source
+    ADD CONSTRAINT source_pkey PRIMARY KEY (id);
+
+--
+-- Name: source source_schema_tname_key; Type: CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.source
+    ADD CONSTRAINT source_schema_tname_key UNIQUE (schema, tname);
+
+--
+-- Name: sql sql_pkey; Type: CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.sql
+    ADD CONSTRAINT sql_pkey PRIMARY KEY (id);
+
+--
+-- Name: sql sql_source_id_operation_key; Type: CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.sql
+    ADD CONSTRAINT sql_source_id_operation_key UNIQUE (source_id, operation);
+
+--
+-- Name: version version_pkey; Type: CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.version
+    ADD CONSTRAINT version_pkey PRIMARY KEY (id);
+
+--
+-- Name: version version_source_id_name_key; Type: CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.version
+    ADD CONSTRAINT version_source_id_name_key UNIQUE (source_id, name);
+
+--
+-- Name: dim_period_cal_idx; Type: INDEX; Schema: pf; Owner: -
+--
+
+CREATE INDEX dim_period_cal_idx ON pf.dim_period USING btree (cal_year, cal_month);
+
+--
+-- Name: dim_period_drange_idx; Type: INDEX; Schema: pf; Owner: -
+--
+
+CREATE INDEX dim_period_drange_idx ON pf.dim_period USING gist (drange);
+
+--
+-- Name: dim_period_fisc_idx; Type: INDEX; Schema: pf; Owner: -
+--
+
+CREATE INDEX dim_period_fisc_idx ON pf.dim_period USING btree (fisc_year, fisc_month);
+
+--
+-- Name: col_meta col_meta_source_id_fkey; Type: FK CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.col_meta
+    ADD CONSTRAINT col_meta_source_id_fkey FOREIGN KEY (source_id) REFERENCES pf.source(id) ON DELETE CASCADE;
+
+--
+-- Name: log log_version_id_fkey; Type: FK CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.log
+    ADD CONSTRAINT log_version_id_fkey FOREIGN KEY (version_id) REFERENCES pf.version(id) ON DELETE CASCADE;
+
+--
+-- Name: sql sql_source_id_fkey; Type: FK CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.sql
+    ADD CONSTRAINT sql_source_id_fkey FOREIGN KEY (source_id) REFERENCES pf.source(id) ON DELETE CASCADE;
+
+--
+-- Name: version version_source_id_fkey; Type: FK CONSTRAINT; Schema: pf; Owner: -
+--
+
+ALTER TABLE ONLY pf.version
+    ADD CONSTRAINT version_source_id_fkey FOREIGN KEY (source_id) REFERENCES pf.source(id) ON DELETE RESTRICT;
+
+--
+-- PostgreSQL database dump complete
+--
+

ui/README.md

@@ -0,0 +1,16 @@
+# React + Vite
+
+This template provides a minimal setup to get React working in Vite with HMR and some ESLint rules.
+
+Currently, two official plugins are available:
+
+- [@vitejs/plugin-react](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react) uses [Oxc](https://oxc.rs)
+- [@vitejs/plugin-react-swc](https://github.com/vitejs/vite-plugin-react/blob/main/packages/plugin-react-swc) uses [SWC](https://swc.rs/)
+
+## React Compiler
+
+The React Compiler is not enabled on this template because of its impact on dev & build performances. To add it, see [this documentation](https://react.dev/learn/react-compiler/installation).
+
+## Expanding the ESLint configuration
+
+If you are developing a production application, we recommend using TypeScript with type-aware lint rules enabled. Check out the [TS template](https://github.com/vitejs/vite/tree/main/packages/create-vite/template-react-ts) for information on how to integrate TypeScript and [`typescript-eslint`](https://typescript-eslint.io) in your project.

ui/index.html

@@ -5,6 +5,7 @@
     <link rel="icon" type="image/svg+xml" href="/favicon.svg" />
     <meta name="viewport" content="width=device-width, initial-scale=1.0" />
     <title>Pivot Forecast</title>
+    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@perspective-dev/viewer@4.4.0/dist/css/themes.css" crossorigin="anonymous">
   </head>
   <body>
     <div id="root"></div>

ui/package.json

@@ -10,10 +10,6 @@
     "preview": "vite preview"
   },
   "dependencies": {
-    "@perspective-dev/client": "file:vendor/perspective-dev-client-5.4.0.tgz",
-    "@perspective-dev/server": "file:vendor/perspective-dev-server-5.4.0.tgz",
-    "@perspective-dev/viewer": "file:vendor/perspective-dev-viewer-5.4.0.tgz",
-    "@perspective-dev/viewer-datagrid": "file:vendor/perspective-dev-viewer-datagrid-5.4.0.tgz",
     "react": "^19.2.5",
     "react-dom": "^19.2.5"
   },

ui/src/App.jsx

@@ -10,10 +10,8 @@ export default function App() {
   const [sidebarExpanded, setSidebarExpanded] = useState(() => localStorage.getItem('pf_sidebar') !== 'collapsed')
 
   const [sources,   setSources]   = useState([])
-  const [sourcesLoaded, setSourcesLoaded] = useState(false)
   const [sourceId,  setSourceId]  = useState(() => localStorage.getItem('pf_sourceId')  || '')
   const [versions,  setVersions]  = useState([])
-  const [versionsLoaded, setVersionsLoaded] = useState(false)
   const [versionId, setVersionId] = useState(() => localStorage.getItem('pf_versionId') || '')
 
   useEffect(() => { localStorage.setItem('pf_view', view) }, [view])
@@ -23,92 +21,37 @@ export default function App() {
 
   const refreshSources = useCallback(async () => {
     const data = await fetch('/api/sources').then(r => r.json())
-    const list = Array.isArray(data) ? data : []
-    setSources(list)
-    setSourcesLoaded(true)
-    return list
+    setSources(data)
+    return data
   }, [])
 
   const refreshVersions = useCallback(async (sid) => {
     const id = sid ?? sourceId
-    if (!id) { setVersions([]); setVersionsLoaded(true); return [] }
+    if (!id) { setVersions([]); return [] }
     const data = await fetch(`/api/sources/${id}/versions`).then(r => r.json())
-    const list = Array.isArray(data) ? data : []
-    setVersions(list)
-    setVersionsLoaded(true)
-    return list
+    setVersions(data)
+    return data
   }, [sourceId])
 
-  // Every source's forecasts, for the sidebar only.
-  //
-  // `versions` stays scoped to the selected source, because Baseline and
-  // Forecast are, and widening it would change what it means everywhere. The
-  // sidebar is the one place that has to be cross-source: it is how a forecast
-  // gets chosen, and a forecast carries its own source_id, so choosing one
-  // settles the source. Without this, a second registered source would be
-  // unreachable from anywhere but Setup -- and invisible from everywhere else,
-  // which is worse.
-  //
-  // A fan-out rather than an endpoint: there is one source today, so this is one
-  // request, and a `GET /versions` is not worth adding until there are enough to
-  // make the difference measurable.
-  const [allVersions, setAllVersions] = useState([])
-  const refreshAllVersions = useCallback(async (list) => {
-    const srcs = list ?? sources
-    if (!srcs.length) { setAllVersions([]); return }
-    const lists = await Promise.all(srcs.map(s =>
-      fetch(`/api/sources/${s.id}/versions`).then(r => r.json()).catch(() => [])))
-    setAllVersions(lists.flat().filter(v => v && v.id != null))
-  }, [sources])
-
-  useEffect(() => { refreshSources() }, [])
-
-  // Recomputed when the selected source's list changes, which is what creating
-  // or deleting a version does -- so the sidebar follows without every caller
-  // having to know it exists.
-  useEffect(() => { if (sourcesLoaded) refreshAllVersions() }, [sourcesLoaded, sources, versions])
-
-  // The selection is restored from localStorage and survives a deregister, so it
-  // has to be re-checked against the list itself rather than only at mount:
-  // deleting the selected source otherwise leaves a dead id behind and every
-  // call 404s "Source not found" until the page is reloaded.
   useEffect(() => {
-    if (!sourcesLoaded) return
-    if (sources.length === 0) { setSourceId(''); return }
-    if (!sourceId || !sources.some(s => String(s.id) === String(sourceId))) {
-      setSourceId(String(sources[0].id))
-    }
-  }, [sources, sourcesLoaded, sourceId])
+    refreshSources().then(data => {
+      if (data.length === 0) { setSourceId(''); return }
+      if (!sourceId || !data.some(s => String(s.id) === String(sourceId))) {
+        setSourceId(String(data[0].id))
+      }
+    })
+  }, [])
 
   useEffect(() => {
-    if (!sourceId) { setVersions([]); setVersionId(''); setVersionsLoaded(true); return }
-    // The list belongs to the previous source until the fetch lands.
-    setVersionsLoaded(false)
-    refreshVersions(sourceId)
+    if (!sourceId) { setVersions([]); setVersionId(''); return }
+    refreshVersions(sourceId).then(data => {
+      if (data.length === 0) { setVersionId(''); return }
+      if (!versionId || !data.some(v => String(v.id) === String(versionId))) {
+        setVersionId(String(data[0].id))
+      }
+    })
   }, [sourceId])
 
-  // Same reasoning as sources: a deleted version must not stay selected.
-  //
-  // versionsLoaded matters more than it looks: without it, the empty initial
-  // state reads as "this source has no versions", so a versionId restored from
-  // localStorage is cleared and then set straight back when the fetch lands.
-  // Forecast's load effect is keyed on that id, so the round trip made every
-  // page load fetch and aggregate the whole version twice.
-  //
-  // What it falls back TO is the source's default version, set on the Baseline
-  // page, and only the newest version when nobody has named one. The list is
-  // ordered created_at DESC, so the old fallback landed an account on whatever
-  // scenario was created last -- a scratch copy as readily as the plan. Sales
-  // accounts never pick a version themselves, so that is where they stayed.
-  useEffect(() => {
-    if (!sourceId || !versionsLoaded) return
-    if (versions.length === 0) { setVersionId(''); return }
-    if (!versionId || !versions.some(v => String(v.id) === String(versionId))) {
-      const landing = versions.find(v => v.is_default) || versions[0]
-      setVersionId(String(landing.id))
-    }
-  }, [versions, versionsLoaded, sourceId, versionId])
-
   const ctx = {
     sources, sourceId, setSourceId,
     versions, versionId, setVersionId,
@@ -117,12 +60,7 @@ export default function App() {
 
   return (
     <div className="flex h-screen w-full text-sm overflow-hidden">
-      <Sidebar
-        view={view} setView={setView}
-        expanded={sidebarExpanded} setExpanded={setSidebarExpanded}
-        versions={allVersions} versionId={versionId} setVersionId={setVersionId}
-        sourceId={sourceId} setSourceId={setSourceId}
-      />
+      <Sidebar view={view} setView={setView} expanded={sidebarExpanded} setExpanded={setSidebarExpanded} />
       <div className="flex flex-col flex-1 overflow-hidden min-w-0">
         <StatusBar view={view} {...ctx} />
         <div className="flex-1 overflow-hidden">

ui/src/auth.jsx

@@ -1,62 +0,0 @@
-import { createContext, useContext, useState, useEffect, useCallback } from 'react'
-
-const AuthContext = createContext()
-
-// A session can expire while the app is open. Rather than teach every one of
-// the app's fetch calls to check for it, wrap fetch once: any 401 from /api
-// drops the whole UI back to the login screen. Cookies ride along on their own
-// — fetch defaults to same-origin credentials, and the UI is served from the
-// same origin as the API.
-function installUnauthorizedHandler(onUnauthorized) {
-  const original = window.fetch
-  window.fetch = async (...args) => {
-    const res = await original(...args)
-    const url = typeof args[0] === 'string' ? args[0] : args[0]?.url || ''
-    if (res.status === 401 && url.includes('/api/') && !url.includes('/api/login')) {
-      onUnauthorized()
-    }
-    return res
-  }
-  return () => { window.fetch = original }
-}
-
-export function AuthProvider({ children }) {
-  const [user, setUser]       = useState(null)
-  const [checking, setCheck]  = useState(true)
-
-  useEffect(() => {
-    fetch('/api/me')
-      .then(r => r.ok ? r.json() : null)
-      .then(d => setUser(d?.user || null))
-      .catch(() => setUser(null))
-      .finally(() => setCheck(false))
-  }, [])
-
-  useEffect(() => installUnauthorizedHandler(() => setUser(null)), [])
-
-  const login = useCallback(async (username, password) => {
-    const r = await fetch('/api/login', {
-      method:  'POST',
-      headers: { 'Content-Type': 'application/json' },
-      body:    JSON.stringify({ username, password }),
-    })
-    const data = await r.json().catch(() => ({}))
-    if (!r.ok) throw new Error(data.error || 'Login failed')
-    setUser(data.user)
-    return data.user
-  }, [])
-
-  const logout = useCallback(async () => {
-    try { await fetch('/api/logout', { method: 'POST' }) } catch {}
-    setUser(null)
-  }, [])
-
-  return (
-    <AuthContext.Provider value={{ user, checking, login, logout }}>
-      {children}
-    </AuthContext.Provider>
-  )
-}
-
-const useAuth = () => useContext(AuthContext)
-export default useAuth

ui/src/components/BridgeView.jsx

@@ -1,677 +0,0 @@
-// Bridge (waterfall): how a version got from its baseline to where it stands,
-// one step per initiative tag.
-//
-// Data is computed from the Perspective table already in the browser rather than
-// from the /bridge endpoint, so the figures always reconcile with what the pivot
-// is showing — including when the view is scoped to the pivot's current filters.
-//
-// Colour is a POLARITY job, not a categorical one: increases and decreases are two
-// poles of one scale, with baseline and current as neutral anchors. Blue/red is the
-// validated diverging pair (CVD ΔE 21.6, normal-vision 32.3 against white);
-// green/red is avoided precisely because it is the classic CVD failure.
-
-import { useState, useEffect, useRef, useCallback } from 'react'
-
-const UP      = '#2a78d6'   // increase
-const DOWN    = '#e34948'   // decrease
-const ANCHOR  = '#6b7280'   // baseline / current — neutral, 4.83:1 on white
-// Everything that is not a bar follows the theme. These were hex literals, so in
-// dark mode the value labels stayed #374151 on a #2a2c2f panel and could not be
-// read. Applied through `style` rather than the presentation attribute, where
-// var() is not reliable across browsers.
-const GRID    = 'var(--border-color)'
-const AXIS    = 'var(--chart-axis)'
-const INK     = 'var(--text-secondary)'
-const INK_DIM = 'var(--chart-ink-dim)'
-const PANEL   = 'var(--bg-secondary)'   // the bars' separating stroke
-
-const fmt = (n, dp = 2) =>
-  n == null || !isFinite(n) ? '—'
-    : n.toLocaleString(undefined, { minimumFractionDigits: dp, maximumFractionDigits: dp })
-
-const fmtSigned = (n, dp = 2) =>
-  n == null || !isFinite(n) ? '—' : `${n > 0 ? '+' : n < 0 ? '−' : ''}${fmt(Math.abs(n), dp)}`
-
-// compact axis ticks — full precision belongs on the marks and in the table
-function fmtAxis(n) {
-  const a = Math.abs(n)
-  if (a >= 1e9) return `${(n / 1e9).toFixed(1)}B`
-  if (a >= 1e6) return `${(n / 1e6).toFixed(1)}M`
-  if (a >= 1e3) return `${(n / 1e3).toFixed(1)}k`
-  return String(Math.round(n))
-}
-
-function niceTicks(min, max, count = 5) {
-  if (!isFinite(min) || !isFinite(max) || min === max) return [min || 0]
-  const span = max - min
-  const raw  = span / count
-  const mag  = Math.pow(10, Math.floor(Math.log10(raw)))
-  const step = [1, 2, 2.5, 5, 10].map(m => m * mag).find(s => s >= raw) || mag * 10
-  const out = []
-  for (let t = Math.ceil(min / step) * step; t <= max + 1e-9; t += step) out.push(t)
-  return out
-}
-
-// Turn raw forecast rows into the walk: baseline anchor, one floating step per
-// initiative tag, current anchor. Pure and exported so the arithmetic can be
-// checked against real data without a browser.
-// The walk from a basis to the current forecast.
-//
-// Without a basis this is the forecast's own composition: its loads as the
-// opening anchor, then one step per adjustment tag.
-//
-// With one -- Plan when building a forecast, Prior Year when building the AOP --
-// it starts there instead, and the identity that makes it exact is
-//
-//     Forecast - Basis = (Forecast loads - Basis) + adjustments
-//
-// so the opening step is the difference between the forecast's own loads and the
-// basis, and every tagged adjustment explains the rest. No residual, no plug: the
-// bars sum to the endpoint by construction rather than by hoping the tags cover
-// everything.
-//
-// Membership comes from pf_bucket, not pf_iter. Those answer different questions
-// -- Open Orders is loaded as reference so nothing adjusts it, and is still part
-// of the forecast -- so a bridge keyed on iter silently dropped it.
-export function buildSteps(rows, {
-  valueCol, unitsCol, logMeta = {},
-  excludeIters = ['reference'],   // kept for callers with no bucket data
-  basis = null,                   // a pf_bucket name, or null for composition
-  forecastBucket,
-}) {
-  const hasBuckets = rows.some(r => r.pf_bucket != null)
-  const excl = new Set(excludeIters)
-
-  // Which bucket is the forecast. It used to be the literal 'Forecast', and the
-  // moment the buckets were renamed to carry their sort prefix -- '04 - Forecast'
-  // -- no row matched it: every row counted as a comparison, the walk had no
-  // steps, and the bridge showed the basis cancelling itself to zero.
-  //
-  // The caller passes the version's adjustment_bucket, which is the same value
-  // an unbucketed adjustment is labelled with, so the two agree by construction.
-  // Falling back to whichever bucket actually holds the adjustments keeps a
-  // renamed or unconfigured version working rather than silently empty.
-  const fcBucket = (() => {
-    if (forecastBucket && rows.some(r => r.pf_bucket === forecastBucket)) return forecastBucket
-    const adjusted = rows.find(r => r.pf_bucket && !['baseline', 'reference'].includes(r.pf_iter))
-    if (adjusted) return adjusted.pf_bucket
-    const baseline = rows.find(r => r.pf_bucket && r.pf_iter === 'baseline')
-    return baseline ? baseline.pf_bucket : (forecastBucket || 'Forecast')
-  })()
-
-  const num = (r, col) => (col ? (parseFloat(r[col]) || 0) : 0)
-  const blank = () => ({ value: 0, units: 0, rows: 0 })
-
-  const loads   = blank()          // the forecast's own segments
-  const basisT  = blank()          // the comparison bucket
-  const byTag   = new Map()
-  const buckets = new Map()        // every bucket, for the picker and the markers
-
-  for (const r of rows) {
-    const v = num(r, valueCol)
-    const u = num(r, unitsCol)
-    const bucket = hasBuckets ? (r.pf_bucket || '') : null
-
-    if (bucket != null) {
-      const b = buckets.get(bucket) || blank()
-      b.value += v; b.units += u; b.rows += 1
-      buckets.set(bucket, b)
-    }
-
-    // Anything outside the forecast is a comparison, never a step.
-    if (hasBuckets && bucket !== fcBucket) {
-      if (basis && bucket === basis) { basisT.value += v; basisT.units += u; basisT.rows += 1 }
-      continue
-    }
-    if (!hasBuckets && excl.has(r.pf_iter)) continue
-
-    const isLoad = r.pf_iter === 'baseline' || r.pf_iter === 'reference'
-    if (isLoad) { loads.value += v; loads.units += u; loads.rows += 1; continue }
-
-    const meta = logMeta[r.pf_logid] || {}
-    // The tag alone. This used to be label first, to match pf_segment, but every
-    // adjustment now carries a label -- the version's adjustment segment, the
-    // same one for all of them -- so the walk collapsed into a single bar and
-    // no tag was ever shown. The label is the segment; the tag is the initiative.
-    const tag = (meta.tag || '').trim()
-    // Untagged entries collapse into one step, the same as the ledger's
-    // "N untagged adjustments" line in OperationPanel.jsx -- one bar per
-    // untagged scale/recode would otherwise turn a routine cleanup pass into
-    // a wall of identical-looking bars. Tagged entries group case-
-    // insensitively for the same reason the ledger does: "Pull Forward" and
-    // "pull forward" are one initiative, not two, and the most-used spelling
-    // is what gets shown.
-    const key = tag ? `tag:${tag.toLowerCase()}` : 'untagged'
-    const g = byTag.get(key) || {
-      key, tagged: !!tag, value: 0, units: 0, rows: 0, logIds: new Set(), first: r.pf_logid,
-      spellings: tag ? new Map() : null,
-    }
-    g.value += v; g.units += u; g.rows += 1
-    if (r.pf_logid != null) { g.logIds.add(r.pf_logid); g.first = Math.min(g.first ?? r.pf_logid, r.pf_logid) }
-    if (tag) g.spellings.set(tag, (g.spellings.get(tag) || 0) + 1)
-    byTag.set(key, g)
-  }
-
-  const mid = [...byTag.values()]
-    .sort((a, b) => (a.first ?? 0) - (b.first ?? 0))
-    .map(g => ({
-      ...g,
-      label: g.tagged
-        ? [...g.spellings.entries()].sort((a, b) => b[1] - a[1])[0]?.[0] || ''
-        : `${g.logIds.size} untagged adjustment${g.logIds.size === 1 ? '' : 's'}`,
-    }))
-  const useBasis = !!basis && basisT.rows > 0
-
-  const out = []
-  if (useBasis) {
-    out.push({
-      key: 'basis', label: basis, kind: 'anchor',
-      delta: basisT.value, start: 0, end: basisT.value,
-      units: basisT.units, rows: basisT.rows, entries: 1,
-    })
-    const gap = loads.value - basisT.value
-    out.push({
-      // "Loads" is our word for a segment import and means nothing to anyone
-      // reading a waterfall. This step is simply where the forecast began
-      // relative to the comparison.
-      //
-      // tagged, though it has no tag: the flag drives an "· untagged" suffix
-      // meant for adjustments nobody grouped into an initiative, and this is
-      // not an adjustment at all.
-      key: 'loads-vs-basis', label: `Starting point vs ${basis}`, kind: 'step',
-      tagged: true,
-      delta: gap, start: basisT.value, end: loads.value,
-      units: loads.units - basisT.units, rows: loads.rows, entries: 1,
-    })
-  } else {
-    out.push({
-      key: 'baseline', label: hasBuckets ? fcBucket + ' loads' : 'Baseline', kind: 'anchor',
-      delta: loads.value, start: 0, end: loads.value,
-      units: loads.units, rows: loads.rows, entries: 1,
-    })
-  }
-
-  let running = loads.value
-  for (const g of mid) {
-    const start = running
-    running += g.value
-    out.push({ ...g, kind: 'step', delta: g.value, start, end: running, entries: g.logIds.size })
-  }
-
-  out.push({
-    key: 'current', label: hasBuckets ? fcBucket : 'Current', kind: 'anchor',
-    delta: running, start: 0, end: running,
-    units: loads.units + mid.reduce((a, g) => a + (g.units || 0), 0),
-    rows: loads.rows + mid.reduce((a, g) => a + g.rows, 0),
-    entries: mid.reduce((a, g) => a + g.logIds.size, 0) + 1,
-  })
-
-  // Every bucket present, so the view can offer them as bases and show the ones
-  // that are not the basis as comparison markers.
-  out.buckets = [...buckets.entries()]
-    .map(([name, t]) => ({ name, ...t }))
-    .sort((a, b) => b.value - a.value)
-  return out
-}
-
-// Plot geometry, also pure: given the steps and a canvas size, where does each
-// bar and label land? Exported so collisions and overflow can be checked.
-// Break a label to the bar's width, on word boundaries.
-//
-// SVG text does not wrap, so the labels were cut at twelve characters and
-// "04 - Forecast" and "04 - Forecasting" read the same. Character width is
-// estimated rather than measured -- measuring means a DOM round trip per label
-// on every render, and at this font a digit is about 0.55em, which is close
-// enough for a centred label with a bar's width to play with.
-//
-// Three lines maximum, and a word longer than the line is left to overflow
-// rather than broken: a truncated word helps nobody, and the bars are wide
-// enough that it only happens to things like a pasted part number.
-function wrapLabel(label, barW, fontSize = 10, maxLines = 3) {
-  const perLine = Math.max(6, Math.floor(barW / (fontSize * 0.55)))
-  const words = String(label || '').split(/\s+/).filter(Boolean)
-  const lines = []
-  let line = ''
-  for (const w of words) {
-    const next = line ? `${line} ${w}` : w
-    if (next.length <= perLine) { line = next; continue }
-    if (line) lines.push(line)
-    line = w
-    if (lines.length === maxLines) break
-  }
-  if (line && lines.length < maxLines) lines.push(line)
-  if (!lines.length) return ['']
-  // anything that did not fit is marked on the last line rather than dropped
-  const used = lines.join(' ').length
-  if (used < String(label).replace(/\s+/g, ' ').length) {
-    lines[lines.length - 1] = `${lines[lines.length - 1]}…`
-  }
-  return lines
-}
-
-export function layoutSteps(steps, width, H = 340, PAD = { t: 24, r: 16, b: 84, l: 68 }) {
-  const plotW = Math.max(120, width - PAD.l - PAD.r)
-  const plotH = H - PAD.t - PAD.b
-  const values = steps.flatMap(s => [s.start, s.end])
-  const rawMin = Math.min(0, ...values)
-  const rawMax = Math.max(0, ...values)
-  const span   = (rawMax - rawMin) || 1
-  const yMin   = rawMin - span * 0.08
-  const yMax   = rawMax + span * 0.12
-  const y      = (v) => PAD.t + plotH - ((v - yMin) / (yMax - yMin)) * plotH
-
-  const n    = steps.length || 1
-  const band = plotW / n
-  const barW = Math.max(10, Math.min(64, band - 14))
-  const bars = steps.map((s, i) => {
-    const x   = PAD.l + band * i + (band - barW) / 2
-    const top = y(Math.max(s.start, s.end))
-    const bot = y(Math.min(s.start, s.end))
-    return { key: s.key, x, w: barW, top, h: Math.max(2, bot - top), labelY: top - 6 }
-  })
-  return { PAD, plotW, plotH, yMin, yMax, y, band, barW, bars, H, width }
-}
-
-export default function BridgeView({
-  open, onClose, tableRef, viewerRef, logMeta = {},
-  valueCol, unitsCol, colMeta = [], slices = [],
-  excludeIters = ['reference'], versionName, forecastBucket,
-}) {
-  const hasSelection = slices.length > 0
-  // 'selection' | 'filtered'. There is no whole-version scope: the pivot's
-  // filter is part of what the figures mean, and dropping it moves one end of
-  // the walk and not the other. On the version this was built against the grid
-  // is scoped `sseas_e <= 2027`, and the rows outside it -- orders with a null
-  // ship date, plus a tail beyond 2027 -- sit entirely in the forecast loads,
-  // with no counterpart in Plan. Ignoring the filter therefore added $9.4M to
-  // the endpoint and nothing to the basis, and the bridge said nothing about it.
-  const [scope, setScope] = useState(hasSelection ? 'selection' : 'filtered')
-  // Whether the user has picked a scope button themselves. Bridge mounts once,
-  // up front, before anything has ever been selected on the pivot — so the
-  // useState initializer above always seeds 'filtered' and nothing follows a
-  // selection made afterward. Until the user overrides it, scope tracks the
-  // live selection instead.
-  const [scopeTouched, setScopeTouched] = useState(false)
-  // Which bucket the walk starts from. Plan when building a forecast, Prior Year
-  // when building the AOP; empty means show the forecast's own composition.
-  const [basis, setBasis] = useState(() => localStorage.getItem('pf_bridge_basis') || '')
-  const [asTable, setAsTable] = useState(false)
-  const [steps,   setSteps]   = useState(null)
-  const [loading, setLoading] = useState(false)
-  const [error,   setError]   = useState(null)
-  const [hover,   setHover]   = useState(null)
-  const [width,   setWidth]   = useState(880)
-  const boxRef = useRef(null)
-  // Where the panel has been dragged to, as an offset from centre. It opens
-  // centred each time rather than wherever it was last left.
-  const [pos, setPos] = useState({ x: 0, y: 0 })
-  const dragRef = useRef(null)
-
-  // Build the steps: baseline anchor, one floating step per tag, current anchor.
-  const compute = useCallback(async () => {
-    if (!tableRef?.current || !valueCol) return
-    setLoading(true); setError(null)
-    try {
-      let rows
-      if (scope === 'selection') {
-        // The union of the selected slices — the same reach an operation would
-        // have. Perspective view filters are AND-only, so each slice needs its own
-        // view; rows matching more than one slice are counted once.
-        //
-        // pf_bucket/pf_segment are deliberately left out here, even though a
-        // click can carry one (e.g. the column axis split by bucket): the walk
-        // needs rows from every bucket, not just the one the slice landed on,
-        // or there is nothing left for "Compare to" to compare against and
-        // every basis collapses to the one bucket fetched. Which bucket the
-        // walk starts and ends on is decided inside buildSteps, from
-        // forecastBucket, not by filtering it out of the fetch.
-        const dimNames  = new Set(colMeta.filter(c => c.role === 'dimension').map(c => c.cname))
-        const dateNames = new Set(colMeta.filter(c => c.role === 'date').map(c => c.cname))
-        const schema = await tableRef.current.schema()
-        const typed = (col, val) => {
-          switch (schema[col]) {
-            case 'integer': case 'float': return Number(val)
-            case 'boolean': return val === true || val === 'true'
-            case 'date': case 'datetime': return Number(val)
-            default: return String(val)
-          }
-        }
-        // The pivot's own filter (e.g. sseas_e <= 2027) is not part of a
-        // clicked slice — perspective-click reports only the cell's own
-        // dimensions — so it has to be read off the viewer and applied
-        // alongside every per-slice view, the same reason Forecast.jsx's
-        // ledger totals do this. Without it, a grid scoped narrower than the
-        // whole version bridges rows the grid is hiding.
-        let viewFilter = []
-        let expressions = {}
-        if (viewerRef?.current) {
-          const cfg = await viewerRef.current.save()
-          viewFilter = (cfg.filter || []).filter(f => Array.isArray(f) && f.length >= 2)
-          expressions = cfg.expressions || {}
-        }
-        const seen = new Set()
-        rows = []
-        for (const sl of slices) {
-          const sliceFilters = Object.entries(sl)
-            .filter(([c]) => (dimNames.has(c) || dateNames.has(c)) && !viewFilter.some(vf => vf[0] === c))
-            // A null is filtered with `is null`, not `== null`: the engine
-            // matches on type, and a null threshold against a string column is
-            // the same dropped predicate that widens a slice back toward its
-            // parent group — see totalsFor in Forecast.jsx for the same fix.
-            .map(([c, v]) => (v === null ? [c, 'is null'] : [c, '==', typed(c, v)]))
-          if (!sliceFilters.length) continue
-          const viewCfg = { filter: [...viewFilter, ...sliceFilters] }
-          if (Object.keys(expressions).length) viewCfg.expressions = expressions
-          const view = await tableRef.current.view(viewCfg)
-          const part = await view.to_json()
-          await view.delete()
-          for (const r of part) {
-            if (r.pf_id != null && seen.has(r.pf_id)) continue
-            if (r.pf_id != null) seen.add(r.pf_id)
-            rows.push(r)
-          }
-        }
-      } else {
-        let filter = []
-        // Expression columns have to come with the filter that uses them: a
-        // filter can name a column that exists only as an expression, and a view
-        // built without it cannot resolve the column and fails outright.
-        let expressions = {}
-        if (viewerRef?.current) {
-          const cfg = await viewerRef.current.save()
-          filter = (cfg.filter || []).filter(f => Array.isArray(f) && f.length >= 2)
-          expressions = cfg.expressions || {}
-        }
-        const viewCfg = {}
-        if (filter.length) viewCfg.filter = filter
-        if (Object.keys(expressions).length) viewCfg.expressions = expressions
-        const view = await tableRef.current.view(viewCfg)
-        rows = await view.to_json()
-        await view.delete()
-      }
-
-      setSteps(buildSteps(rows, {
-        valueCol, unitsCol, logMeta, excludeIters, basis: basis || null, forecastBucket,
-      }))
-          } catch (err) {
-      setError(err.message || String(err))
-      setSteps(null)
-    } finally {
-      setLoading(false)
-    }
-  }, [tableRef, viewerRef, scope, logMeta, valueCol, unitsCol, excludeIters, slices, colMeta, basis, forecastBucket])
-
-  // Follow the pivot's selection until the user chooses a scope explicitly.
-  useEffect(() => {
-    if (scopeTouched) return
-    setScope(hasSelection ? 'selection' : 'filtered')
-  }, [hasSelection, scopeTouched])
-
-  // Once the user has picked "Selection" -- themselves or via the effect above
-  // -- and the selection then clears, there is nothing left to show under it;
-  // fall back regardless of whether the choice was manual.
-  useEffect(() => {
-    if (!hasSelection && scope === 'selection') { setScope('filtered'); setScopeTouched(false) }
-  }, [hasSelection, scope])
-
-  useEffect(() => { try { localStorage.setItem('pf_bridge_basis', basis) } catch {} }, [basis])
-  useEffect(() => { if (open) compute() }, [open, compute])
-
-  useEffect(() => { if (open) setPos({ x: 0, y: 0 }) }, [open])
-
-  // Escape closes it. There is no backdrop to click any more -- see below.
-  useEffect(() => {
-    if (!open) return
-    const onKey = (e) => { if (e.key === 'Escape') onClose() }
-    window.addEventListener('keydown', onKey)
-    return () => window.removeEventListener('keydown', onKey)
-  }, [open, onClose])
-
-  // Drag by the header. Pointer capture keeps the drag alive when the cursor
-  // outruns the panel; the close button is excluded so clicking it still closes.
-  const startDrag = (e) => {
-    if (e.button !== 0 || e.target.closest('button')) return
-    e.currentTarget.setPointerCapture(e.pointerId)
-    dragRef.current = { px: e.clientX, py: e.clientY, x: pos.x, y: pos.y }
-  }
-  const moveDrag = (e) => {
-    const d = dragRef.current
-    if (!d) return
-    // keep the header on screen, or there is nothing left to drag it back by
-    const maxX = window.innerWidth / 2 - 40
-    const maxY = window.innerHeight / 2 - 20
-    setPos({
-      x: Math.max(-maxX, Math.min(maxX, d.x + e.clientX - d.px)),
-      y: Math.max(-maxY, Math.min(maxY, d.y + e.clientY - d.py)),
-    })
-  }
-  const endDrag = () => { dragRef.current = null }
-
-  useEffect(() => {
-    if (!open || !boxRef.current) return
-    const ro = new ResizeObserver(([e]) => setWidth(Math.max(420, e.contentRect.width)))
-    ro.observe(boxRef.current)
-    return () => ro.disconnect()
-  }, [open])
-
-  if (!open) return null
-
-  const geom = layoutSteps(steps || [{ start: 0, end: 0 }], width)
-  const { PAD, plotW, plotH, yMin, yMax, y, barW, H, bars } = geom
-  const xOf = (i) => bars[i]?.x ?? PAD.l
-
-  const ticks = niceTicks(yMin, yMax, 5)
-
-  return (
-    // Not modal. The overlay passes clicks through, so the pivot stays usable
-    // with the bridge open -- moving it aside to read the grid under it, or
-    // changing the selection it is scoped to, which recomputes it. It used to
-    // be a dimmed backdrop that closed on click, which made the panel immovable
-    // and the pivot unreachable until it was dismissed.
-    <div className="fixed inset-0 z-50 flex items-center justify-center pointer-events-none">
-      <div className="pointer-events-auto bg-white rounded-lg shadow-2xl border border-gray-200 w-full max-w-5xl mx-4 flex flex-col max-h-[88vh]"
-           style={{ transform: `translate(${pos.x}px, ${pos.y}px)` }}>
-
-        <div className="flex items-center justify-between px-5 py-3 border-b border-gray-200 shrink-0 cursor-move select-none touch-none"
-             onPointerDown={startDrag} onPointerMove={moveDrag}
-             onPointerUp={endDrag} onPointerCancel={endDrag}>
-          <div className="flex items-baseline gap-2">
-            <span className="font-medium text-gray-700 text-sm">Bridge</span>
-            {versionName && <span className="text-gray-600 text-xs">{versionName}</span>}
-            <span className="text-gray-600 text-xs">
-              · {scope === 'selection'
-                  ? `${slices.length} selected slice${slices.length === 1 ? '' : 's'}`
-                  : "pivot's filters"}
-            </span>
-          </div>
-          <button onClick={onClose} className="text-gray-600 hover:text-gray-800 text-lg leading-none">×</button>
-        </div>
-
-        {/* controls — one row above the chart */}
-        <div className="flex items-center gap-3 px-5 py-2 border-b border-gray-100 shrink-0 text-xs flex-wrap">
-          <span className="text-gray-600">Scope</span>
-          <div className="inline-flex rounded border border-gray-200 overflow-hidden">
-            {[
-              ['selection', hasSelection ? `Selection (${slices.length})` : 'Selection',
-               hasSelection ? 'The slices selected in the operation panel'
-                            : 'Select one or more pivot rows first'],
-              ['filtered',  "Pivot's filters", 'Everything the pivot currently shows'],
-            ].map(([v, l, title]) => (
-              <button key={v} onClick={() => { setScope(v); setScopeTouched(true) }} title={title}
-                disabled={v === 'selection' && !hasSelection}
-                className={`px-3 py-1 disabled:opacity-40 disabled:cursor-not-allowed ${
-                  scope === v ? 'bg-blue-600 text-white' : 'bg-white text-gray-700 hover:bg-gray-50'}`}>
-                {l}
-              </button>
-            ))}
-          </div>
-          {/* Offered only once something carries a bucket, and only listing buckets
-              other than the forecast itself -- a walk from Forecast to Forecast is
-              the composition view, which is what the empty option gives. */}
-          {(steps?.buckets || []).some(b => b.name && b.name !== 'Forecast') && (
-            <>
-              <div className="w-px h-4 bg-gray-200" />
-              <span className="text-gray-600">Compare to</span>
-              <select value={basis} onChange={e => setBasis(e.target.value)}
-                className="border border-gray-200 rounded px-2 py-1 text-gray-700 bg-white">
-                <option value="">nothing — show composition</option>
-                {(steps?.buckets || [])
-                  .filter(b => b.name && b.name !== 'Forecast')
-                  .map(b => <option key={b.name} value={b.name}>{b.name}</option>)}
-              </select>
-            </>
-          )}
-
-          <div className="w-px h-4 bg-gray-200" />
-          <button onClick={() => setAsTable(t => !t)}
-            className="border border-gray-200 rounded px-2 py-1 text-gray-700 hover:bg-gray-50">
-            {asTable ? 'Show chart' : 'Show table'}
-          </button>
-          <button onClick={compute} disabled={loading}
-            className="border border-gray-200 rounded px-2 py-1 text-gray-700 hover:bg-gray-50 disabled:opacity-40">
-            {loading ? 'Computing…' : 'Refresh'}
-          </button>
-
-          {/* legend — identity is never colour alone, but say it anyway */}
-          <div className="ml-auto flex items-center gap-3 text-gray-700">
-            {[['Increase', UP], ['Decrease', DOWN], ['Total', ANCHOR]].map(([l, c]) => (
-              <span key={l} className="inline-flex items-center gap-1.5">
-                <span className="inline-block w-2.5 h-2.5 rounded-sm" style={{ background: c }} />
-                {l}
-              </span>
-            ))}
-          </div>
-        </div>
-
-        <div className="overflow-auto p-5" ref={boxRef}>
-          {error && <p className="text-red-600">{error}</p>}
-          {!error && !steps && <p className="text-gray-600">Computing…</p>}
-          {!error && steps && steps.length <= 2 && (
-            <p className="text-gray-600">
-              No adjustments in scope — the bridge shows the walk from baseline to current,
-              and this selection has only a baseline.
-            </p>
-          )}
-
-          {!error && steps && steps.length > 2 && !asTable && (
-            <div className="relative">
-              <svg width={width} height={H} role="img"
-                   aria-label={`Bridge from baseline ${fmt(steps[0].end)} to current ${fmt(steps[steps.length - 1].end)}`}>
-                {/* recessive grid */}
-                {ticks.map(t => (
-                  <g key={t}>
-                    <line x1={PAD.l} x2={PAD.l + plotW} y1={y(t)} y2={y(t)}
-                          style={{ stroke: t === 0 ? AXIS : GRID }} strokeWidth={t === 0 ? 1.5 : 1} />
-                    <text x={PAD.l - 8} y={y(t) + 3} textAnchor="end" fontSize="10" style={{ fill: INK_DIM }}>
-                      {fmtAxis(t)}
-                    </text>
-                  </g>
-                ))}
-
-                {steps.map((s, i) => {
-                  const isAnchor = s.kind === 'anchor'
-                  const up   = s.delta >= 0
-                  const fill = isAnchor ? ANCHOR : (up ? UP : DOWN)
-                  const top  = y(Math.max(s.start, s.end))
-                  const bot  = y(Math.min(s.start, s.end))
-                  const h    = Math.max(2, bot - top)
-                  const x    = xOf(i)
-                  const on   = hover?.key === s.key
-                  const labelLines = wrapLabel(s.label, barW)
-                  const subY = PAD.t + plotH + 16 + labelLines.length * 11 + 2
-                  return (
-                    <g key={s.key}
-                       onMouseEnter={() => setHover({ ...s, x: x + barW / 2, y: top })}
-                       onMouseLeave={() => setHover(null)}>
-                      {/* connector to the next bar, drawn behind */}
-                      {i < steps.length - 1 && (
-                        <line x1={x + barW} x2={xOf(i + 1)} y1={y(s.end)} y2={y(s.end)}
-                              style={{ stroke: AXIS }} strokeWidth="1" strokeDasharray="2 2" />
-                      )}
-                      {/* hit target larger than the mark */}
-                      <rect x={x - 6} y={PAD.t} width={barW + 12} height={plotH} fill="transparent" />
-                      <rect x={x} y={top} width={barW} height={h} rx="4" fill={fill}
-                            opacity={on ? 1 : 0.92}
-                            style={{ stroke: PANEL }} strokeWidth="2" />
-                      {/* direct label: few bars, so every one is labelled */}
-                      <text x={x + barW / 2} y={top - 6} textAnchor="middle" fontSize="10"
-                            style={{ fill: INK }} fontWeight="500">
-                        {isAnchor ? fmt(s.end, 0) : fmtSigned(s.delta, 0)}
-                      </text>
-                      <text x={x + barW / 2} y={PAD.t + plotH + 16} textAnchor="middle" fontSize="10" style={{ fill: INK }}>
-                        {labelLines.map((ln, li) => (
-                          <tspan key={li} x={x + barW / 2} dy={li === 0 ? 0 : 11}>{ln}</tspan>
-                        ))}
-                      </text>
-                      {/* below the label, however many lines it took. Skipped for the
-                          untagged bucket -- its label already reads "N untagged
-                          adjustments", so a ×N here would just repeat the count. */}
-                      {!isAnchor && s.tagged && s.entries > 1 && (
-                        <text x={x + barW / 2} y={subY} textAnchor="middle" fontSize="9" style={{ fill: INK_DIM }}>
-                          ×{s.entries}
-                        </text>
-                      )}
-                    </g>
-                  )
-                })}
-              </svg>
-
-              {hover && (
-                <div className="absolute pointer-events-none bg-white border border-gray-300 rounded shadow-lg px-2.5 py-1.5 text-xs"
-                     style={{ left: Math.min(hover.x + 10, width - 190), top: Math.max(0, hover.y - 10) }}>
-                  <div className="font-medium text-gray-800">{hover.label}</div>
-                  <div className="text-gray-700 font-mono tabular-nums">
-                    {hover.kind === 'anchor' ? fmt(hover.end) : fmtSigned(hover.delta)}
-                  </div>
-                  {hover.kind === 'step' && (
-                    <div className="text-gray-600">
-                      running → <span className="font-mono tabular-nums">{fmt(hover.end)}</span>
-                    </div>
-                  )}
-                  <div className="text-gray-600">
-                    {hover.rows} row{hover.rows === 1 ? '' : 's'}
-                    {hover.entries > 1 ? ` · ${hover.entries} adjustments` : ''}
-                  </div>
-                </div>
-              )}
-            </div>
-          )}
-
-          {/* table view — the same numbers, at full precision */}
-          {!error && steps && steps.length > 2 && asTable && (
-            <table className="w-full text-xs">
-              <thead>
-                <tr className="text-gray-600 border-b border-gray-200">
-                  <th className="text-left py-1.5 pr-3 font-medium">Step</th>
-                  <th className="text-right py-1.5 px-2 font-medium">{valueCol}</th>
-                  {unitsCol && <th className="text-right py-1.5 px-2 font-medium">{unitsCol}</th>}
-                  <th className="text-right py-1.5 px-2 font-medium">Running</th>
-                  <th className="text-right py-1.5 px-2 font-medium">Adjustments</th>
-                  <th className="text-right py-1.5 pl-2 font-medium">Rows</th>
-                </tr>
-              </thead>
-              <tbody>
-                {steps.map(s => (
-                  <tr key={s.key} className="border-b border-gray-100">
-                    <td className="py-1.5 pr-3 text-gray-800">{s.label}</td>
-                    <td className="py-1.5 px-2 text-right font-mono tabular-nums text-gray-800">
-                      {s.kind === 'anchor' ? fmt(s.end) : fmtSigned(s.delta)}
-                    </td>
-                    {unitsCol && (
-                      <td className="py-1.5 px-2 text-right font-mono tabular-nums text-gray-700">
-                        {s.kind === 'anchor' ? fmt(s.units) : fmtSigned(s.units)}
-                      </td>
-                    )}
-                    <td className="py-1.5 px-2 text-right font-mono tabular-nums text-gray-700">{fmt(s.end)}</td>
-                    <td className="py-1.5 px-2 text-right text-gray-700">{s.kind === 'step' ? s.entries : '—'}</td>
-                    <td className="py-1.5 pl-2 text-right text-gray-700">{s.rows}</td>
-                  </tr>
-                ))}
-              </tbody>
-            </table>
-          )}
-        </div>
-      </div>
-    </div>
-  )
-}

ui/src/components/DepthButtons.jsx

@@ -1,108 +0,0 @@
-import { useEffect, useRef, useState } from 'react'
-
-// One axis's collapse control, as a stepper.
-//
-// Rows and columns are the same thing — a depth in ViewConfig, counting the
-// levels to show — so one component draws both.
-//
-// It used to be one button per level, which is N+1 controls that grow with the
-// pivot: a 5-by-3 layout rendered ten of them, and the active level was a blue
-// button somewhere among the greys, so reading the current state meant scanning.
-// A stepper is three controls whatever the depth, and says where you are rather
-// than making you find it. The full list is still one click away, for jumping
-// rather than stepping.
-export default function DepthButtons({ label, levels, depth, onPick, totalLabel = 'Total', children }) {
-  const [open, setOpen] = useState(false)
-  const boxRef = useRef(null)
-
-  useEffect(() => {
-    if (!open) return
-    const onDown = (e) => { if (!boxRef.current?.contains(e.target)) setOpen(false) }
-    const onKey  = (e) => { if (e.key === 'Escape') setOpen(false) }
-    // Capture phase: the pivot lives in a shadow root and retargets its events,
-    // so a bubbled listener sees the host rather than the real target.
-    document.addEventListener('mousedown', onDown, true)
-    window.addEventListener('keydown', onKey)
-    return () => {
-      document.removeEventListener('mousedown', onDown, true)
-      window.removeEventListener('keydown', onKey)
-    }
-  }, [open])
-
-  const max = levels.length
-  // depth is null until the first layout has been read back. Showing the last
-  // level would be a guess; an em dash is honest and the arrows still work.
-  const known   = depth != null
-  const current = !known ? '—' : (depth === 0 ? totalLabel : levels[depth - 1])
-  const step    = (d) => onPick(Math.min(max, Math.max(0, d)))
-  // Capped so one long field name cannot push the rest of the toolbar off;
-  // past the cap the label truncates, with the full name in the title.
-  const widest  = Math.min(18, Math.max(...[totalLabel, ...levels].map(l => String(l).length)))
-
-  // One joined control, in the style of the operation panel's
-  // Scale | Recode | Clone. The arrows used to be separate outlined boxes a
-  // few pixels wide, which in dark mode were barely visible and looked the
-  // same enabled or disabled; joined, the axis reads as one object and the
-  // arrows are halves of it rather than specks beside it.
-  const seg = 'px-1.5 py-0.5 leading-none bg-white transition-colors ' +
-              'text-gray-600 hover:bg-gray-100 disabled:text-gray-300 disabled:hover:bg-white disabled:cursor-default'
-
-  return (
-    <div ref={boxRef} className="relative flex items-center gap-1.5">
-      <span className="pf-section-label">{label}</span>
-
-      <div className="inline-flex rounded border border-gray-300 divide-x divide-gray-300 overflow-hidden">
-        <button className={seg} disabled={!known || depth <= 0}
-          title={`Collapse ${label.toLowerCase()} by one level`}
-          onClick={() => step((depth ?? 0) - 1)}>◂</button>
-
-        {/* Sized to the axis's longest level, not the current one. Sized to
-            its content it resized on every step, so the ▸ moved out from
-            under the cursor between clicks. */}
-        <button onClick={() => setOpen(o => !o)}
-          title={`${label}: showing ${current}`}
-          style={{ width: `calc(${widest}ch + 1rem)` }}
-          className={`px-2 py-0.5 truncate text-left transition-colors
-            ${open ? 'bg-blue-50 text-blue-700' : 'bg-white text-gray-700 hover:bg-gray-100'}`}>
-          {current}
-        </button>
-
-        <button className={seg} disabled={!known || depth >= max}
-          title={`Expand ${label.toLowerCase()} by one level`}
-          onClick={() => step((depth ?? 0) + 1)}>▸</button>
-      </div>
-
-      {children}
-
-      {open && (
-        <div className="absolute top-full left-0 mt-1 z-50 min-w-[10rem] bg-white border
-                        border-gray-200 rounded shadow-lg py-1 text-xs">
-          {Array.from({ length: max + 1 }, (_, n) => (
-            <button key={n} onClick={() => { onPick(n); setOpen(false) }}
-              title={n === 0
-                ? `Collapse ${label.toLowerCase()} to a single total`
-                : `Show ${label.toLowerCase()} down to ${levels.slice(0, n).join(' › ')}`}
-              className={`block w-full text-left px-3 py-1 truncate
-                ${depth === n ? 'bg-blue-50 text-blue-700' : 'text-gray-700 hover:bg-gray-50'}`}>
-              {n === 0 ? totalLabel : levels[n - 1]}
-            </button>
-          ))}
-        </div>
-      )}
-    </div>
-  )
-}
-
-// An on/off control that looks pressed when on — the same state the toolbar's
-// Operations button uses for "open". Replaces the native checkboxes, which
-// drew in the light style on the dark theme and were the brightest thing on
-// the toolbar.
-export function ToggleChip({ on, onChange, title, children }) {
-  return (
-    <button type="button" aria-pressed={on} title={title} onClick={() => onChange(!on)}
-      className={`border rounded px-2 py-0.5 transition-colors ${
-        on ? 'pf-pressed' : 'border-gray-300 bg-white text-gray-500 hover:bg-gray-100'}`}>
-      {children}
-    </button>
-  )
-}

ui/src/components/LayoutMenu.jsx

@@ -1,197 +0,0 @@
-import { useEffect, useRef, useState } from 'react'
-
-// The pivot layout, as one control rather than a row of equal chips.
-//
-// Two lists, because a layout is one of two different things: Published, which
-// everyone on this forecast sees and only its owner or an admin may change, and
-// Mine, which nobody else lists. Everything you cannot write is still there to
-// apply -- it is only the write controls that are withheld, so a click never
-// answers 403.
-export default function LayoutMenu({
-  layouts = [], activeLayoutId, dirty,
-  onApply, onSave, onSaveAs, onSetVisibility, onSetDefault, onRename, onDelete, onReset
-}) {
-  const [open,       setOpen]       = useState(false)
-  const [saveName,   setSaveName]   = useState('')
-  const [savePublic, setSavePublic] = useState(false)
-  const [renaming,   setRenaming]   = useState(null)
-  const [renameTo,   setRenameTo]   = useState('')
-  const boxRef = useRef(null)
-
-  // Close on an outside click or Escape. Capture phase: the pivot lives in a
-  // shadow root and retargets its events, so a bubbled listener on document
-  // sees the host rather than the real target and cannot tell inside from out.
-  useEffect(() => {
-    if (!open) return
-    const onDown = (e) => { if (!boxRef.current?.contains(e.target)) setOpen(false) }
-    const onKey  = (e) => { if (e.key === 'Escape') setOpen(false) }
-    document.addEventListener('mousedown', onDown, true)
-    window.addEventListener('keydown', onKey)
-    return () => {
-      document.removeEventListener('mousedown', onDown, true)
-      window.removeEventListener('keydown', onKey)
-    }
-  }, [open])
-
-  const active    = layouts.find(l => l.id === activeLayoutId)
-  const published = layouts.filter(l => l.visibility === 'published')
-  const mine      = layouts.filter(l => l.visibility === 'private')
-
-  const submitSaveAs = () => {
-    const name = saveName.trim()
-    if (!name) return
-    onSaveAs(name, savePublic ? 'published' : 'private')
-    setSaveName('')
-    setOpen(false)
-  }
-
-  const submitRename = (l) => {
-    const name = renameTo.trim()
-    if (name && name !== l.name) onRename(l, name)
-    setRenaming(null)
-  }
-
-  const iconBtn = 'px-1 rounded text-gray-400 hover:text-gray-700 hover:bg-gray-100 leading-none'
-
-  const row = (l) => (
-    <div key={l.id}
-      className={`group flex items-center gap-1 px-2 py-1 rounded cursor-pointer
-        ${l.id === activeLayoutId ? 'bg-blue-50 text-blue-700' : 'hover:bg-gray-50 text-gray-700'}`}
-      onClick={() => { if (renaming !== l.id) { onApply(l); setOpen(false) } }}>
-
-      {renaming === l.id ? (
-        <input autoFocus value={renameTo} onClick={e => e.stopPropagation()}
-          onChange={e => setRenameTo(e.target.value)}
-          onBlur={() => submitRename(l)}
-          onKeyDown={e => {
-            if (e.key === 'Enter')  submitRename(l)
-            if (e.key === 'Escape') setRenaming(null)
-          }}
-          className="flex-1 min-w-0 border border-blue-300 rounded px-1 py-0 outline-none bg-white" />
-      ) : (
-        <>
-          <span className="flex-1 min-w-0 truncate">{l.name}</span>
-          {l.is_default && (
-            <span title="Applied when this forecast is opened for the first time"
-              className="text-amber-500 shrink-0">★</span>
-          )}
-          {l.scope === 'source' && (
-            <span title="Applies to every forecast of this source"
-              className="text-gray-300 shrink-0" style={{fontSize:'9px'}}>ALL</span>
-          )}
-        </>
-      )}
-
-      {/* Withheld rather than offered-and-refused: the server allows a write only
-          to the owner or an admin, and can_edit says which this is. */}
-      {l.can_edit && renaming !== l.id && (
-        <span className="hidden group-hover:flex items-center gap-0.5 shrink-0"
-          onClick={e => e.stopPropagation()}>
-          <button className={iconBtn} title="Rename"
-            onClick={() => { setRenaming(l.id); setRenameTo(l.name) }}>✎</button>
-          {l.visibility === 'private' ? (
-            <button className={iconBtn} title="Publish — everyone on this forecast sees it"
-              onClick={() => onSetVisibility(l, 'published')}>↑</button>
-          ) : (
-            <>
-              {!l.is_default && (
-                <button className={iconBtn} title="Apply this when the forecast is first opened"
-                  onClick={() => onSetDefault(l)}>★</button>
-              )}
-              <button className={iconBtn} title="Unpublish — keep it, but only for you"
-                onClick={() => onSetVisibility(l, 'private')}>↓</button>
-            </>
-          )}
-          <button className={`${iconBtn} hover:text-red-500`} title="Delete"
-            onClick={() => onDelete(l)}>×</button>
-        </span>
-      )}
-
-      {!l.can_edit && (
-        <span className="hidden group-hover:inline text-gray-300 shrink-0 truncate"
-          style={{fontSize:'9px'}} title={`Published by ${l.owner}`}>{l.owner}</span>
-      )}
-    </div>
-  )
-
-  return (
-    <div ref={boxRef} className="relative flex items-center gap-1.5">
-      <span className="pf-section-label">Layout</span>
-
-      <button onClick={() => setOpen(o => !o)}
-        className={`flex items-center gap-1 border rounded px-2 py-0.5 max-w-[22rem] transition-colors
-          ${open ? 'border-blue-300 text-blue-700 bg-blue-50'
-                 : 'border-gray-200 text-gray-600 hover:border-gray-400'}`}>
-        <span className="truncate">{active ? active.name : 'Unsaved'}</span>
-        {/* Always rendered, hidden when clean, so the button does not widen
-            by a dot the moment the pivot changes. */}
-        <span title={dirty ? 'Changed since it was saved' : undefined}
-          className={`text-amber-500 leading-none ${dirty ? '' : 'invisible'}`}>•</span>
-        <span className="text-gray-400 leading-none">▾</span>
-      </button>
-
-      {/* Save is the common action and stays outside the menu, but only when
-          there is something to save it to and it is yours to overwrite. Its
-          slot is held while the layout is clean: appearing and vanishing, it
-          shoved the Rows and Columns controls sideways under the cursor. */}
-      {active?.can_edit && (
-        <button onClick={() => onSave(active)} disabled={!dirty}
-          aria-hidden={!dirty} tabIndex={dirty ? 0 : -1}
-          className={`border border-blue-200 text-blue-600 hover:text-blue-800 rounded px-2 py-0.5
-            ${dirty ? '' : 'invisible'}`}>
-          Save
-        </button>
-      )}
-
-      {open && (
-        <div className="absolute top-full left-0 mt-1 z-50 w-72 bg-white border border-gray-200
-                        rounded shadow-lg py-1 text-xs">
-
-          {published.length > 0 && (
-            <>
-              <div className="px-2 pt-1 pb-0.5 text-gray-400 uppercase tracking-wide"
-                style={{fontSize:'9px'}}>Published</div>
-              <div className="px-1">{published.map(row)}</div>
-            </>
-          )}
-
-          {mine.length > 0 && (
-            <>
-              <div className="px-2 pt-2 pb-0.5 text-gray-400 uppercase tracking-wide"
-                style={{fontSize:'9px'}}>Mine</div>
-              <div className="px-1">{mine.map(row)}</div>
-            </>
-          )}
-
-          {!layouts.length && (
-            <div className="px-3 py-2 text-gray-400">No saved layouts yet</div>
-          )}
-
-          <div className="border-t border-gray-100 mt-1 pt-1 px-2">
-            <div className="flex items-center gap-1 py-1">
-              <input value={saveName} onChange={e => setSaveName(e.target.value)}
-                onKeyDown={e => { if (e.key === 'Enter') submitSaveAs() }}
-                placeholder="Save current view as…"
-                className="flex-1 min-w-0 border border-gray-300 rounded px-1.5 py-0.5
-                           outline-none focus:border-blue-400" />
-              <button onClick={submitSaveAs} disabled={!saveName.trim()}
-                className="text-blue-600 hover:text-blue-800 px-1 disabled:opacity-30">Save</button>
-            </div>
-            <label className="flex items-center gap-1.5 py-0.5 text-gray-500 cursor-pointer">
-              <input type="checkbox" checked={savePublic}
-                onChange={e => setSavePublic(e.target.checked)} />
-              Publish to everyone on this forecast
-            </label>
-          </div>
-
-          <div className="border-t border-gray-100 mt-1 pt-1 px-2 pb-0.5">
-            <button onClick={() => { onReset(); setOpen(false) }}
-              className="text-gray-400 hover:text-gray-700 py-0.5">
-              Reset to a blank pivot
-            </button>
-          </div>
-        </div>
-      )}
-    </div>
-  )
-}

ui/src/components/ReadOnlyNotice.jsx

@@ -1,22 +0,0 @@
-// Says why everything on the page is greyed out.
-//
-// The disabling itself is one `<fieldset disabled>` around the page rather than
-// a flag threaded through every control -- Baseline alone has forty-five of
-// them, and the one that got missed would be the one that mattered. A disabled
-// fieldset takes its descendant form controls out with it natively, so nothing
-// has to remember to opt in.
-//
-// None of this is the permission. The routes carry requireAdmin; this only
-// stops an account being offered a button that would answer 403.
-export default function ReadOnlyNotice({ what }) {
-  return (
-    <div className="flex items-center gap-2 px-3 py-1.5 bg-amber-50 border-b border-amber-200 text-amber-800 text-xs shrink-0">
-      <svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor"
-           strokeWidth="2" strokeLinecap="round" className="shrink-0">
-        <rect x="4" y="11" width="16" height="10" rx="2" />
-        <path d="M8 11V7a4 4 0 0 1 8 0v4" />
-      </svg>
-      Read-only — {what} is managed by an administrator.
-    </div>
-  )
-}

ui/src/components/Sidebar.jsx

@@ -32,41 +32,7 @@ const NAV = [
   },
 ]
 
-// The forecasts are listed here, above the steps, rather than offered as a
-// dropdown in the status bar. Two reasons: a scenario is a thing you open, not
-// a setting you change, and the status bar is otherwise read-only, so a control
-// sitting in it did not read as one.
-//
-// Not nested under the Forecast button, which is where this started: Baseline
-// works on the same version, so hanging it off one step would either
-// misrepresent the scope or duplicate the list under two. It reads
-// "what am I working on" and then "what step am I on".
-//
-// The source is deliberately absent. `gs.osm_skinny` means nothing to someone
-// editing a forecast, and a version already carries its source, so choosing a
-// forecast settles it. Picking the source is a Setup concern and the status bar
-// only offers it there.
-export default function Sidebar({
-  view, setView, expanded, setExpanded,
-  versions = [], versionId, setVersionId,
-  sourceId, setSourceId,
-}) {
-  const selected = versions.find(v => String(v.id) === String(versionId))
-
-  function pick(v) {
-    // The source comes with the forecast. This is the whole reason the source
-    // needs no control of its own outside Setup: every version carries its
-    // source_id, so choosing one settles both. Set first, so the version is not
-    // briefly validated against the outgoing source's list.
-    if (v.source_id != null && String(v.source_id) !== String(sourceId)) {
-      setSourceId(String(v.source_id))
-    }
-    setVersionId(String(v.id))
-    // Setup is the one step a version means nothing to, so choosing one there
-    // is a statement of intent to go and work on it.
-    if (view === 'setup') setView('forecast')
-  }
-
+export default function Sidebar({ view, setView, expanded, setExpanded }) {
   return (
     <div
       className="bg-white border-r border-gray-200 flex flex-col shrink-0 overflow-hidden transition-all duration-150"
@@ -92,61 +58,6 @@ export default function Sidebar({
         </span>
       </div>
 
-      {expanded ? (
-        <div className="px-2 pt-2 pb-1.5 border-b border-gray-100 shrink-0">
-          <div className="px-2 pb-1 text-gray-400 uppercase tracking-wide" style={{ fontSize: '10px' }}>
-            Forecasts
-          </div>
-          {/* Capped and scrollable: the list grows with the work, and the steps
-              below it must never be pushed off the bottom. */}
-          <div className="max-h-52 overflow-y-auto">
-            {versions.length === 0 && (
-              <div className="px-2 py-1 text-xs text-gray-400">none yet</div>
-            )}
-            {versions.map(v => {
-              const active = String(v.id) === String(versionId)
-              return (
-                <button
-                  key={v.id}
-                  onClick={() => pick(v)}
-                  title={`${v.name}${v.is_default ? ' · default' : ''} · ${v.status}`}
-                  className={`w-full flex items-center gap-1.5 rounded px-2 py-1 text-left transition-colors ${
-                    active ? 'bg-blue-50 text-blue-700' : 'text-gray-600 hover:bg-gray-100'
-                  }`}
-                >
-                  <span className={`w-1.5 h-1.5 rounded-full shrink-0 ${active ? 'bg-blue-600' : 'bg-transparent'}`} />
-                  <span className="text-xs truncate flex-1">{v.name}</span>
-                  {v.is_default && (
-                    <span className="text-amber-500 shrink-0" style={{ fontSize: '10px' }} title="Default">★</span>
-                  )}
-                  <span
-                    className={`shrink-0 ${v.status === 'open' ? 'text-green-600' : 'text-gray-400'}`}
-                    style={{ fontSize: '10px' }}
-                  >
-                    {v.status}
-                  </span>
-                </button>
-              )
-            })}
-          </div>
-        </div>
-      ) : (
-        // Collapsed, the list has nowhere to go, so all that survives is which
-        // one is open — as a tooltip, on a button that opens the rail again.
-        <div className="p-2 border-b border-gray-100 shrink-0">
-          <button
-            onClick={() => setExpanded(true)}
-            title={selected ? `Forecast: ${selected.name}` : 'Forecasts'}
-            className="w-8 h-8 flex items-center justify-center rounded hover:bg-gray-100 text-gray-400"
-          >
-            <svg width="18" height="18" viewBox="0 0 20 20" fill="none" stroke="currentColor" strokeWidth="1.6" strokeLinecap="round" strokeLinejoin="round">
-              <path d="M4 3h8l4 4v10a1 1 0 0 1-1 1H4a1 1 0 0 1-1-1V4a1 1 0 0 1 1-1z"/>
-              <polyline points="12,3 12,7 16,7"/>
-            </svg>
-          </button>
-        </div>
-      )}
-
       <nav className="flex flex-col gap-0.5 p-2 flex-1">
         {NAV.map(item => {
           const active = view === item.id

ui/src/components/StatusBar.jsx

@@ -1,171 +1,47 @@
-import { useState, useEffect, useCallback } from 'react'
 import useTheme from '../theme.jsx'
-import useAuth from '../auth.jsx'
 
-export default function StatusBar({ view, sources = [], sourceId, setSourceId, versions = [], versionId }) {
+export default function StatusBar({ view, sources = [], sourceId, setSourceId, versions = [], versionId, setVersionId }) {
   const { dark, setDark } = useTheme()
-  const { user, logout } = useAuth()
   const showVersion = view === 'baseline' || view === 'forecast'
   const selectedVersion = versions.find(v => String(v.id) === String(versionId))
 
-  const [info, setInfo]     = useState(null)
-  const [showInfo, setShow] = useState(false)
-  const [copied, setCopied] = useState(false)
-  // Forecast.jsx mirrors the pivot's own updating state onto this event. The
-  // viewer shows the same thing as a beaker in its settings panel, which is
-  // exactly where you cannot see it once the panel is closed.
-  const [busy, setBusy]     = useState(false)
-
-  const refreshInfo = useCallback(async () => {
-    if (!versionId || !showVersion) { setInfo(null); return }
-    try {
-      const r = await fetch(`/api/versions/${versionId}/table-info`)
-      setInfo(r.ok ? await r.json() : null)
-    } catch { setInfo(null) }
-  }, [versionId, showVersion])
-
-  useEffect(() => { refreshInfo() }, [refreshInfo])
-
-  // operations broadcast this after a write so the row count stays honest
-  useEffect(() => {
-    const onChange = () => refreshInfo()
-    window.addEventListener('pf-data-changed', onChange)
-    return () => window.removeEventListener('pf-data-changed', onChange)
-  }, [refreshInfo])
-
-  useEffect(() => {
-    const onBusy = (e) => setBusy(!!e.detail?.busy)
-    window.addEventListener('pf-pivot-busy', onBusy)
-    return () => window.removeEventListener('pf-pivot-busy', onBusy)
-  }, [])
-
-  async function copyTable() {
-    if (!info?.fc_table) return
-    try {
-      await navigator.clipboard.writeText(info.fc_table)
-      setCopied(true)
-      setTimeout(() => setCopied(false), 1200)
-    } catch {}
-  }
-
-  const fmt = (n) => n == null ? '—' : n.toLocaleString()
-
   return (
-    <div className="bg-white border-b border-gray-200 px-3 h-9 flex items-center gap-3 shrink-0 text-xs relative">
-      {/* Source is a Setup concern: the raw table name means nothing to someone
-          editing a forecast, and a version carries its own source, so picking a
-          forecast in the sidebar settles it everywhere else. */}
-      {view === 'setup' && (
-        <>
-          <span className="text-gray-400">Source</span>
-          <select
-            value={sourceId || ''}
-            onChange={e => setSourceId(e.target.value)}
-            disabled={sources.length === 0}
-            className="border border-gray-200 rounded px-2 py-0.5 bg-white"
-          >
-            {sources.length === 0
-              ? <option value="">— no sources —</option>
-              : sources.map(s => <option key={s.id} value={s.id}>{s.tname}</option>)}
-          </select>
-        </>
-      )}
+    <div className="bg-white border-b border-gray-200 px-3 h-9 flex items-center gap-3 shrink-0 text-xs">
+      <span className="text-gray-400">Source</span>
+      <select
+        value={sourceId || ''}
+        onChange={e => setSourceId(e.target.value)}
+        disabled={sources.length === 0}
+        className="border border-gray-200 rounded px-2 py-0.5 bg-white"
+      >
+        {sources.length === 0
+          ? <option value="">— no sources —</option>
+          : sources.map(s => <option key={s.id} value={s.id}>{s.tname}</option>)}
+      </select>
 
       {showVersion && (
         <>
-          {/* What is open, and how big it is -- the two things a forecaster
-              wants from a status bar. The physical table it writes to is an
-              implementation detail and means nothing to a sales account, so it
-              has moved into the hover panel and onto the Baseline page, which
-              is where a version's plumbing is managed. */}
-          {info && (
-            <>
-              <span className="text-gray-200">|</span>
-              <span
-                onMouseEnter={() => setShow(true)}
-                onMouseLeave={() => setShow(false)}
-                className="font-medium text-gray-700 cursor-default"
-                title="Hover for the write target and row breakdown"
-              >
-                {selectedVersion?.name || '—'}
-              </span>
-              <span className={`font-mono ${info.exists ? 'text-gray-400' : 'text-amber-700'}`}>
-                {info.exists ? `${fmt(info.rows)} rows` : 'not created'}
-              </span>
-
-              {showInfo && (
-                <div className="absolute top-9 left-0 z-30 bg-white border border-gray-200 rounded shadow-lg p-3 text-xs min-w-[260px]">
-                  <div className="text-gray-400 uppercase tracking-wide mb-2" style={{ fontSize: '10px' }}>Write target</div>
-                  <div className="grid grid-cols-[auto_1fr] gap-x-3 gap-y-1">
-                    <span className="text-gray-400">table</span>
-                    {/* Still copyable, just no longer the headline: pasting it
-                        into psql is the reason it was ever on screen. */}
-                    <button
-                      onClick={copyTable}
-                      title="Click to copy table name"
-                      className="font-mono text-gray-700 text-left hover:text-blue-700"
-                    >
-                      {copied ? 'copied!' : info.fc_table}
-                    </button>
-                    <span className="text-gray-400">reads from</span>
-                    <span className="font-mono text-gray-700">{info.source}</span>
-                    <span className="text-gray-400">total rows</span>
-                    <span className="font-mono text-gray-700">{fmt(info.rows)}</span>
-                  </div>
-                  {info.by_iter?.length > 0 && (
-                    <>
-                      <div className="text-gray-400 uppercase tracking-wide mt-3 mb-1" style={{ fontSize: '10px' }}>Rows by iter</div>
-                      <table className="w-full">
-                        <tbody>
-                          {info.by_iter.map(r => (
-                            <tr key={r.pf_iter}>
-                              <td className="text-gray-500 capitalize pr-3">{r.pf_iter}</td>
-                              <td className="text-right font-mono text-gray-700">{fmt(r.n)}</td>
-                            </tr>
-                          ))}
-                        </tbody>
-                      </table>
-                    </>
-                  )}
-                </div>
-              )}
-            </>
+          <span className="text-gray-200">|</span>
+          <span className="text-gray-400">Version</span>
+          <select
+            value={versionId || ''}
+            onChange={e => setVersionId(e.target.value)}
+            disabled={versions.length === 0}
+            className="border border-gray-200 rounded px-2 py-0.5 bg-white"
+          >
+            {versions.length === 0
+              ? <option value="">— no versions —</option>
+              : versions.map(v => <option key={v.id} value={v.id}>{v.name}</option>)}
+          </select>
+          {selectedVersion && (
+            <span className={`font-medium ${selectedVersion.status === 'open' ? 'text-green-600' : 'text-gray-400'}`}>
+              {selectedVersion.status}
+            </span>
           )}
         </>
       )}
 
-      {/* Left of the ml-auto block, so it sits beside the pivot's own figures
-          rather than out among the account controls. Occupies no space at all
-          when idle: a permanently reserved slot reads as a thing that is broken
-          when empty. */}
-      {busy && view === 'forecast' && (
-        <span
-          className="flex items-center gap-1.5 text-blue-600"
-          title="The pivot is rebuilding its view — sorting, filtering, or changing depth on a large grain takes a moment"
-        >
-          <svg width="12" height="12" viewBox="0 0 24 24" fill="none" stroke="currentColor"
-               strokeWidth="3" strokeLinecap="round" className="animate-spin">
-            <path d="M12 3a9 9 0 1 0 9 9" />
-          </svg>
-          Calculating…
-        </span>
-      )}
-
-      <div className="ml-auto flex items-center gap-2">
-        {user && (
-          <>
-            <span className="text-gray-500" title={`Signed in as ${user.username}`}>
-              {user.display_name || user.username}
-            </span>
-            <button
-              onClick={logout}
-              className="text-xs text-gray-500 hover:text-gray-700 border border-gray-200 px-2 py-0.5 rounded"
-              title="Sign out"
-            >
-              Sign out
-            </button>
-          </>
-        )}
+      <div className="ml-auto">
         <button
           onClick={() => setDark(d => !d)}
           className="w-6 h-6 flex items-center justify-center rounded hover:bg-gray-100"

ui/src/components/Timeline.jsx

@@ -32,16 +32,11 @@ function roundRect(ctx, x, y, w, h, r, fill, stroke) {
   if (stroke) ctx.stroke()
 }
 
-export default function Timeline({ dateFrom, dateTo, offsetMonths = 0, offsetDays = 0, type = 'baseline' }) {
+export default function Timeline({ dateFrom, dateTo, offsetYr, offsetMo, type = 'baseline' }) {
   const canvasRef = useRef(null)
 
-  // Months and days are kept apart because they are not interchangeable: a month
-  // shift lands on the same day of a different month, a day shift can cross a
-  // month boundary. The preview adds months first, then days, as Postgres does.
-  const offsetMoTotal = offsetMonths || 0
-  const offsetDayTotal = offsetDays || 0
-  const shifted = offsetMoTotal !== 0 || offsetDayTotal !== 0
-  const twoBands = type === 'baseline' && shifted
+  const offsetMoTotal = (offsetYr || 0) * 12 + (offsetMo || 0)
+  const twoBands = type === 'baseline' && offsetMoTotal > 0
   const canvasH  = twoBands ? 90 : 52
 
   useEffect(() => {
@@ -66,9 +61,8 @@ export default function Timeline({ dateFrom, dateTo, offsetMonths = 0, offsetDay
       const srcEnd   = parseDate(dateTo)
       if (!srcStart || !srcEnd || isNaN(srcStart) || isNaN(srcEnd)) return
 
-      const addDays = (d, n) => { const x = new Date(d); x.setDate(x.getDate() + n); return x }
-      const projStart = addDays(addMonths(srcStart, offsetMoTotal), offsetDayTotal)
-      const projEnd   = addDays(addMonths(srcEnd,   offsetMoTotal), offsetDayTotal)
+      const projStart = addMonths(srcStart, offsetMoTotal)
+      const projEnd   = addMonths(srcEnd,   offsetMoTotal)
 
       const winStart = addMonths(srcStart, -1)
       const winEnd   = addMonths(twoBands ? projEnd : srcEnd, 1)
@@ -153,14 +147,7 @@ export default function Timeline({ dateFrom, dateTo, offsetMonths = 0, offsetDay
         ctx.lineTo(px1 - 4, arrowY + 4)
         ctx.closePath()
         ctx.fill()
-        const yrs  = Math.trunc(offsetMoTotal / 12)
-        const mos  = offsetMoTotal % 12
-        const sign = (offsetMoTotal + offsetDayTotal) < 0 ? '' : '+'
-        const offsetLabel = sign + [
-          yrs ? `${yrs}yr` : '',
-          mos ? `${mos}mo` : '',
-          offsetDayTotal ? `${offsetDayTotal}d` : '',
-        ].filter(Boolean).join(' ')
+        const offsetLabel = '+' + (offsetYr ? offsetYr + 'yr ' : '') + (offsetMo ? offsetMo + 'mo' : '')
         ctx.fillStyle = '#64748b'
         ctx.font = '9px system-ui'
         ctx.textAlign = 'center'
@@ -169,7 +156,7 @@ export default function Timeline({ dateFrom, dateTo, offsetMonths = 0, offsetDay
     }
     raf = requestAnimationFrame(draw)
     return () => cancelAnimationFrame(raf)
-  }, [dateFrom, dateTo, offsetMoTotal, offsetDayTotal, type, twoBands, canvasH])
+  }, [dateFrom, dateTo, offsetYr, offsetMo, type, twoBands, canvasH])
 
   return <canvas ref={canvasRef} height={canvasH} style={{ width: '100%', display: 'block' }} />
 }

ui/src/index.css

@@ -11,8 +11,6 @@
   --border-light: #f3f4f6;
   --accent-bg: #eff6ff;
   --accent-text: #1d4ed8;
-  --chart-ink-dim: #6b7280;
-  --chart-axis: #d1d5db;
 }
 
 /* Dark palette tuned to Perspective's "Pro Dark" theme:
@@ -29,11 +27,6 @@
   --border-light: #3b3f46;
   --accent-bg: rgba(39, 113, 170, 0.32);
   --accent-text: #4778c2;
-  --chart-ink-dim: #9ca3af;
-  --chart-axis: #61656e;
-  /* Native controls — checkboxes, scrollbars, date pickers — draw in the dark
-     style. Without it every checkbox in the app was a white square. */
-  color-scheme: dark;
 }
 
 body { margin: 0; background-color: var(--bg-primary); color: var(--text-primary); }
@@ -41,68 +34,9 @@ body { margin: 0; background-color: var(--bg-primary); color: var(--text-primary
 
 .dark .bg-white { background-color: var(--bg-secondary); }
 .dark .bg-gray-50 { background-color: var(--bg-tertiary); }
-/* The opacity variant is its own class and the rule above does not reach it, so
-   without this the operation panel's header band stayed light grey in dark mode.
-   --bg-tertiary at the same 60%. */
-.dark .bg-gray-50\/60 { background-color: rgb(59 63 70 / 0.6); }
 .dark .bg-gray-100 { background-color: var(--bg-tertiary); }
 .dark .bg-gray-200 { background-color: var(--bg-tertiary); }
 .dark .bg-gray-300 { background-color: var(--bg-tertiary); }
-/* Toolbar section labels (LAYOUT / ROWS / COLUMNS). Their own class rather than
-   a grey utility: in dark mode text-gray-400 and text-gray-500 both collapse to
-   --text-muted, so changing the utility would have fixed light mode only. At
-   10px/gray-400 these sat near 2.5:1 on white and 2:1 on the dark panel, under
-   the 4.5:1 WCAG wants for text this size. */
-/* Names the group of controls beside it — Layout, Rows, Columns. Weighted and
-   tracked so it reads as a label rather than as another small grey control:
-   the toolbar is dense, and at the old weight the labels and the buttons they
-   name were the same visual object. */
-.pf-section-label {
-  font-size: 10px;
-  font-weight: 600;
-  line-height: 1;
-  letter-spacing: 0.08em;
-  text-transform: uppercase;
-  color: #4b5563;
-  white-space: nowrap;
-}
-.dark .pf-section-label { color: #c5c9d0; }
-
-/* The rule between groups inside a zone. Taller and lighter than the old
-   `w-px h-4 bg-gray-200`, which was the same weight as the zone borders, so
-   the groups within a zone and the zones themselves separated identically. */
-.pf-group-rule {
-  width: 1px;
-  height: 20px;
-  flex-shrink: 0;
-  background: #b6bcc5;
-}
-.dark .pf-group-rule { background: #5a616b; }
-
-/* A toggle that is on — the Total chips, and Operations while its panel is
-   open. Pressed grey rather than blue: blue is Save and Post, actions, and
-   these are states. Its own class so the dark values are defined once rather
-   than hoping each utility has a dark override. */
-.pf-pressed {
-  background-color: #e5e7eb;
-  border-color: #9ca3af;
-  color: #1f2937;
-  box-shadow: inset 0 1px 2px rgb(0 0 0 / 0.08);
-}
-.dark .pf-pressed {
-  background-color: var(--bg-tertiary);
-  border-color: #61656e;
-  color: var(--text-primary);
-  box-shadow: inset 0 1px 2px rgb(0 0 0 / 0.3);
-}
-
-/* The joined controls — the depth steppers and the operation panel's
-   Scale | Recode | Clone — draw their inner rules with divide-*, which had no
-   dark override and stayed light grey. */
-.dark .divide-gray-300 > :not(:last-child) { border-color: var(--border-color); }
-.dark .disabled\:text-gray-300:disabled { color: var(--text-muted); }
-.dark .disabled\:hover\:bg-white:disabled:hover { background-color: var(--bg-secondary); }
-
 .dark .text-gray-300 { color: var(--text-muted); }
 .dark .text-gray-400 { color: var(--text-muted); }
 .dark .text-gray-500 { color: var(--text-muted); }

ui/src/main.jsx

@@ -1,27 +1,13 @@
 import { StrictMode } from 'react'
 import { createRoot } from 'react-dom/client'
 import { ThemeProvider } from './theme.jsx'
-import { AuthProvider } from './auth.jsx'
-import useAuth from './auth.jsx'
-import Login from './views/Login.jsx'
 import './index.css'
 import App from './App.jsx'
 
-// App is mounted only once there is a session — its load effects call /api
-// straight away, and mounting it logged-out would just fire a burst of 401s.
-function Gate() {
-  const { user, checking } = useAuth()
-  if (checking) return <div className="flex items-center justify-center h-screen text-sm text-gray-400">Loading…</div>
-  if (!user)    return <Login />
-  return <App />
-}
-
 createRoot(document.getElementById('root')).render(
   <StrictMode>
     <ThemeProvider>
-      <AuthProvider>
-        <Gate />
-      </AuthProvider>
+      <App />
     </ThemeProvider>
   </StrictMode>,
 )

ui/src/views/Login.jsx

@@ -1,68 +0,0 @@
-import { useState } from 'react'
-import useAuth from '../auth.jsx'
-
-export default function Login() {
-  const { login } = useAuth()
-  const [username, setUsername] = useState('')
-  const [password, setPassword] = useState('')
-  const [error, setError]       = useState('')
-  const [busy, setBusy]         = useState(false)
-
-  async function submit(e) {
-    e.preventDefault()
-    setError(''); setBusy(true)
-    try {
-      await login(username.trim(), password)
-    } catch (err) {
-      setError(err.message)
-      setPassword('')
-    } finally {
-      setBusy(false)
-    }
-  }
-
-  return (
-    <div className="flex items-center justify-center h-screen w-full text-sm">
-      <form onSubmit={submit} className="bg-white border border-gray-200 rounded p-6 w-80 flex flex-col gap-4">
-        <div>
-          <div className="text-base font-medium">Pivot Forecast</div>
-          <div className="text-xs text-gray-500 mt-0.5">Sign in to continue</div>
-        </div>
-
-        <label className="flex flex-col gap-1">
-          <span className="text-xs text-gray-500">Username</span>
-          <input
-            value={username}
-            onChange={e => setUsername(e.target.value)}
-            autoFocus
-            autoComplete="username"
-            className="border border-gray-200 rounded px-2 py-1.5 text-sm"
-          />
-        </label>
-
-        <label className="flex flex-col gap-1">
-          <span className="text-xs text-gray-500">Password</span>
-          <input
-            type="password"
-            value={password}
-            onChange={e => setPassword(e.target.value)}
-            autoComplete="current-password"
-            className="border border-gray-200 rounded px-2 py-1.5 text-sm"
-          />
-        </label>
-
-        {error && (
-          <div className="px-3 py-2 text-xs rounded font-medium bg-red-50 text-red-700">{error}</div>
-        )}
-
-        <button
-          type="submit"
-          disabled={busy || !username.trim() || !password}
-          className="bg-blue-600 text-white text-xs px-3 py-2 rounded hover:bg-blue-700 disabled:opacity-50"
-        >
-          {busy ? 'Signing in…' : 'Sign in'}
-        </button>
-      </form>
-    </div>
-  )
-}

ui/src/views/Setup.jsx

@@ -1,7 +1,5 @@
 import { useState, useEffect, useRef } from 'react'
 import useTheme from '../theme.jsx'
-import useAuth from '../auth.jsx'
-import ReadOnlyNotice from '../components/ReadOnlyNotice.jsx'
 
 const ROLES = ['ignore', 'dimension', 'value', 'units', 'date', 'filter']
 
@@ -15,8 +13,6 @@ const ROLE_STYLE = {
 }
 
 export default function Setup({ refreshSources }) {
-  const { user: me } = useAuth()
-  const isAdmin = !!me?.is_admin
   const { dark } = useTheme()
   const [tables,         setTables]         = useState([])
   const [sources,        setSources]        = useState([])
@@ -29,22 +25,14 @@ export default function Setup({ refreshSources }) {
   const [sqlStatus,      setSqlStatus]      = useState({})     // sourceId -> bool
   const [saving,         setSaving]         = useState(false)
   const [generating,     setGenerating]     = useState(false)
-  const [refreshingDims, setRefreshingDims] = useState(false)
   const [msg,            setMsg]            = useState(null)
   const [dimPeriodCols,  setDimPeriodCols]  = useState([])
   const [openPeriodIdx,  setOpenPeriodIdx]  = useState(null)
   const periodDropRef = useRef(null)
 
-  // Every list here is rendered with .map, so a response that is not an array
-  // takes the whole app down rather than this one panel -- a 403 body is
-  // {error}, and `{error}.map` threw during render, which unmounts React and
-  // leaves a blank page. An endpoint answering something unexpected should cost
-  // an empty list, not the session.
-  const asList = (set) => (d) => set(Array.isArray(d) ? d : [])
-
   useEffect(() => {
-    fetch('/api/tables').then(r => r.json()).then(asList(setTables)).catch(console.error)
-    fetch('/api/dim-period/cols').then(r => r.json()).then(asList(setDimPeriodCols)).catch(console.error)
+    fetch('/api/tables').then(r => r.json()).then(setTables).catch(console.error)
+    fetch('/api/dim-period/cols').then(r => r.json()).then(setDimPeriodCols).catch(console.error)
     loadSources()
   }, [])
 
@@ -59,8 +47,7 @@ export default function Setup({ refreshSources }) {
   }, [openPeriodIdx])
 
   function loadSources() {
-    fetch('/api/sources').then(r => r.json()).then(raw => {
-      const data = Array.isArray(raw) ? raw : []
+    fetch('/api/sources').then(r => r.json()).then(data => {
       setSources(data)
       // check sql status for each source
       data.forEach(s => {
@@ -75,8 +62,7 @@ export default function Setup({ refreshSources }) {
   function selectSource(source) {
     setSelectedSource(source)
     setColsDirty(false)
-    fetch(`/api/sources/${source.id}/cols`).then(r => r.json()).then(raw => {
-      const data = Array.isArray(raw) ? raw : []
+    fetch(`/api/sources/${source.id}/cols`).then(r => r.json()).then(data => {
       setCols(data)
       setEditedCols(data.map(c => ({ ...c })))
     })
@@ -167,35 +153,6 @@ export default function Setup({ refreshSources }) {
     }
   }
 
-  // Rebuild every keyed dim_group's member list from the source. Deliberate rather
-  // than automatic: it reads the whole source, which for a view over a transaction
-  // table is millions of rows, and the answer only changes when the catalogue does.
-  async function refreshDimMembers() {
-    const groups = [...new Set(cols.filter(c => c.dim_group && c.is_key).map(c => c.dim_group))]
-    if (groups.length === 0) {
-      flash('No dim_group has an is_key column, so there is nothing to build a list from', 'error')
-      return
-    }
-    setRefreshingDims(true)
-    try {
-      const done = []
-      for (const g of groups) {
-        const res  = await fetch(`/api/sources/${selectedSource.id}/dim/${encodeURIComponent(g)}/refresh`,
-                                 { method: 'POST' })
-        const data = await res.json()
-        if (!res.ok) { flash(`${g}: ${data.error}`, 'error'); return }
-        done.push(`${g}: ${data.members.toLocaleString()} members`
-                  + (data.no_longer_in_source ? `, ${data.no_longer_in_source} no longer in source` : '')
-                  + ` (${(data.ms / 1000).toFixed(1)}s)`)
-      }
-      flash(done.join(' · '))
-    } catch (err) {
-      flash(err.message, 'error')
-    } finally {
-      setRefreshingDims(false)
-    }
-  }
-
   async function deleteSource(id, e) {
     e.stopPropagation()
     if (!confirm('Deregister this source? Existing forecast tables are not affected.')) return
@@ -222,13 +179,7 @@ export default function Setup({ refreshSources }) {
     .map(c => c.cname)
 
   return (
-    <div className="h-full flex flex-col overflow-hidden text-sm">
-    {!isAdmin && <ReadOnlyNotice what="source registration and column roles" />}
-    {/* A real box here rather than `display: contents`, because this one has to
-        be the flex row the two panes sit in. Baseline uses `contents` instead,
-        where the parent's layout must be left alone. `disabled` is a DOM
-        property and applies either way. */}
-    <fieldset disabled={!isAdmin} className="flex-1 min-h-0 flex overflow-hidden border-0 p-0 m-0">
+    <div className="h-full flex overflow-hidden text-sm">
 
       {/* All Tables */}
       <div className="w-64 bg-white border-r border-gray-200 flex flex-col shrink-0">
@@ -344,17 +295,6 @@ export default function Setup({ refreshSources }) {
                       {saving ? 'Saving…' : 'Save'}
                     </button>
                   )}
-                  {cols.some(c => c.dim_group && c.is_key) && (
-                    <button
-                      onClick={refreshDimMembers}
-                      disabled={refreshingDims || colsDirty}
-                      className="text-xs border border-gray-200 px-3 py-1 rounded hover:bg-gray-50 disabled:opacity-50"
-                      title={colsDirty ? 'Save col meta first'
-                                       : 'Rebuild the member list for each keyed dim_group from the source. Reads the whole source, so it takes a while.'}
-                    >
-                      {refreshingDims ? 'Refreshing…' : 'Refresh master data'}
-                    </button>
-                  )}
                   <button
                     onClick={generateSQL}
                     disabled={generating || colsDirty}
@@ -373,7 +313,6 @@ export default function Setup({ refreshSources }) {
                       <th className="px-3 py-1.5 font-medium">role</th>
                       <th className="px-3 py-1.5 font-medium text-center">key</th>
                       <th className="px-3 py-1.5 font-medium text-center" title="Include this column in the display grain — the load is pre-aggregated to the flagged columns">grain</th>
-                      <th className="px-3 py-1.5 font-medium text-center" title="The column an account's territory is expressed in. Accounts see and change only rows whose value here is on their list. One per source.">territory</th>
                       <th className="px-3 py-1.5 font-medium">group</th>
                       <th className="px-3 py-1.5 font-medium">period col</th>
                       <th className="px-3 py-1.5 font-medium">label</th>
@@ -410,25 +349,6 @@ export default function Setup({ refreshSources }) {
                             className="cursor-pointer disabled:opacity-20"
                           />
                         </td>
-                        {/* Radio, not a checkbox: exactly one column per source,
-                            and the shape of the control should say so rather than
-                            leaving it to a save-time error. */}
-                        <td className="px-3 py-1.5 text-center">
-                          <input
-                            type="radio"
-                            name="pf-territory-col"
-                            checked={!!col.is_territory}
-                            onChange={() => {}}
-                            onClick={() => setEditedCols(prev => {
-                              // clicking the chosen one again clears it, since a
-                              // radio otherwise has no way back to "no territory"
-                              const already = !!prev[i].is_territory
-                              return prev.map((c, x) => ({ ...c, is_territory: !already && x === i }))
-                            })}
-                            disabled={col.role !== 'dimension'}
-                            className="cursor-pointer disabled:opacity-20"
-                          />
-                        </td>
                         <td className="px-3 py-1.5">
                           <input
                             type="text"
@@ -572,7 +492,6 @@ export default function Setup({ refreshSources }) {
         </div>
       )}
 
-    </fieldset>
     </div>
   )
 }

ui/vendor/0001-apply-depth-fields-on-config-update.patch

@@ -1,45 +0,0 @@
-diff --git a/rust/perspective-client/src/rust/config/view_config.rs b/rust/perspective-client/src/rust/config/view_config.rs
-index fa4f36e..360b816 100644
---- a/rust/perspective-client/src/rust/config/view_config.rs
-+++ b/rust/perspective-client/src/rust/config/view_config.rs
-@@ -526,6 +526,23 @@ impl ViewConfig {
-         }
-     }
- 
-+    /// `_apply` for a field which is itself `Option`, where `None` in the update
-+    /// means "not mentioned" rather than "clear it". `Option<Option<T>>` would be
-+    /// needed to express both, and the wire format cannot carry the difference:
-+    /// these fields are `skip_serializing_if = "Option::is_none"`, so an absent
-+    /// field and an explicit null arrive identically. To lift a depth, send the
-+    /// number of levels on that axis rather than clearing it.
-+    fn _apply_optional<T: PartialEq>(field: &mut Option<T>, update: Option<T>) -> bool {
-+        match update {
-+            None => false,
-+            Some(_) if *field == update => false,
-+            Some(_) => {
-+                *field = update;
-+                true
-+            },
-+        }
-+    }
-+
-     pub fn reset(&mut self, reset_expressions: bool) {
-         let mut config = Self::default();
-         if !reset_expressions {
-@@ -568,6 +585,16 @@ impl ViewConfig {
-         changed = Self::_apply(&mut self.windows, update.windows) || changed;
-         changed = Self::_apply(&mut self.group_rollup_mode, update.group_rollup_mode) || changed;
-         changed = Self::_apply(&mut self.split_rollup_mode, update.split_rollup_mode) || changed;
-+
-+        // Without these two, a depth can be set when a view is created --
-+        // `table.view({ group_by_depth })` -- but never through `restore()`, which
-+        // merges a `ViewConfigUpdate` onto the live config. The field arrives,
-+        // deserializes, and is then dropped here, so the viewer's config and the
-+        // engine never see it and nothing happens. That makes expand/collapse
-+        // unreachable for anything driven by `restore()`, which is how a viewer
-+        // changes its own configuration.
-+        changed = Self::_apply_optional(&mut self.group_by_depth, update.group_by_depth) || changed;
-+        changed = Self::_apply_optional(&mut self.split_by_depth, update.split_by_depth) || changed;
-         if self.group_rollup_mode == GroupRollupMode::Total && !self.group_by.is_empty() {
-             tracing::info!("`total` incompatible with `group_by`");
-             changed = true;

ui/vendor/PROVENANCE.txt

@@ -1,8 +0,0 @@
-Built from https://github.com/fleetside72/perspective
-  branch  expand-blank-tree-nodes
-  commit  751dcfe136c6455e3cd02a7b75a6df79cbe8e064
-  based on unknown
-  built   2026-09-22T04:18:51Z on usmidsap02
-  dirty   0 uncommitted file(s) in the source tree at build time
-
-Regenerate with ui/vendor/rebuild-perspective.sh

ui/vendor/README.md

@@ -1,187 +0,0 @@
-# Vendored Perspective
-
-pf_app runs a **patched build of Perspective**. Upstream's C++ engine has always
-implemented column-axis expand/collapse — `t_ctx2::set_depth(HEADER_COLUMN, …)`
-and `open`/`close(HEADER_COLUMN, idx)` are fully written — but nothing above C++
-could reach it: `set_column_pivot_depth()` was never called, and
-`View<t_ctx2>::expand/collapse` hardcoded `HEADER_ROW`. The patch is wiring, not
-new engine logic.
-
-It buys two things the released packages cannot do at all:
-
-- `split_by_depth` in `ViewConfig`, the `split_by` counterpart to `group_by_depth`
-- `expand_column()` / `collapse_column()`, so one column branch can fold to its
-  subtotal while its siblings stay expanded — the Excel behaviour
-
-**Source:** https://github.com/fleetside72/perspective, branch
-`expand-blank-tree-nodes` — the tip of the stack, carrying
-`column-axis-expand-collapse` and `apply-depth-on-config-update` beneath it.
-See `PROVENANCE.txt` for the exact commit these tarballs were built from.
-
-## Why tarballs and not npm
-
-The feature is not released upstream. Until it is, the four packages are built
-from the fork and committed here as npm tarballs. `npm install` expands them
-exactly as it expands anything from the registry — no special tooling, and
-`pf.sh deploy` works unchanged. A deploy machine needs node and nothing else:
-no emscripten, no cmake, no protoc, no Rust.
-
-All four move together, never a subset. Perspective couples loader, package
-versions, data format and `apache-arrow`; vendoring a partial set reintroduces
-exactly the drift that causes trouble.
-
-## Changing the engine
-
-    ./rebuild-perspective.sh          # builds the fork, repacks, rewrites PROVENANCE.txt
-    cd .. && npm install ./vendor/perspective-dev-*-5.4.0.tgz   # by path — see below
-    git add vendor package-lock.json && git commit
-
-Push the fork first — the script warns if the source tree is dirty, because a
-tarball built from uncommitted code has no recoverable source.
-
-The build itself needs cmake >= 3.29.5, protoc >= 22 (its version silently
-selects which protobuf source tree gets cloned), pnpm, and the Rust nightly the
-repo pins. Roughly 40 minutes cold. Only ever on a machine changing the engine.
-
-### `npm install` alone does not pick up a rebuilt tarball
-
-The version in the filename never changes, and `package-lock.json` pins an
-`integrity` hash per package — so npm satisfies the dependency from its own
-cache and the freshly built tarball on disk is never read. Nothing fails; the
-old code is simply still there. Re-resolve by path instead, which recomputes
-the hashes and rewrites the lock:
-
-```bash
-cd ui
-rm -rf node_modules/@perspective-dev
-npm install ./vendor/perspective-dev-{client,server,viewer,viewer-datagrid}-5.4.0.tgz
-```
-
-Check the install rather than trusting it — the packages ship their TypeScript
-sources, so the change is readable without unminifying anything:
-
-```bash
-grep -n is_label node_modules/@perspective-dev/viewer-datagrid/src/ts/style_handlers/table_cell/row_header.ts
-```
-
-`cargo check` on this patch was clean — the 125 errors it reports without
-`protoc` installed are unresolved generated protobuf modules, none of them in
-`view_config.rs`.
-
-`applyDepth()` in `Forecast.jsx` reads the config back after restoring it and
-falls back to the imperative `view.set_depth()` when the value did not stick.
-With the patched engine installed that fallback should no longer run; it is kept
-for now so an unpatched engine still works, and can be removed once the
-declarative path is confirmed.
-
-## Getting rid of this
-
-This is a fork, with the maintenance that implies. The exit is upstream taking
-the change — the patch is small and additive, and the engine work is already
-theirs. When a release ships it, delete this directory and put normal version
-ranges back in `ui/package.json`.
-
-## Applied: expand a tree node whose group key is blank
-
-In the vendored tarballs as of the 2026-09-22 build, from branch
-`expand-blank-tree-nodes` (which is `apply-depth-on-config-update` plus this).
-Upstream bug, in `packages/viewer-datagrid/src/ts/style_handlers/table_cell/row_header.ts`.
-
-A tree row's header is the label cell padded on its left by one blank cell per
-level above it, and `cell_style_row_header` told padding from label by testing
-the cell's own **text**. A node whose group key is null or `''` therefore read
-as padding: no `psp-tree-label` class, so no `+` caret — and because
-`createClickListener` in `event_handlers/header_click.ts` gates expand/collapse
-on that same class, no way to open the node at all. The node was in the
-traversal and its children existed; only the affordance was missing. On
-`gs.osm_skinny` that is `partgroup`, which has 93k null and 111k empty rows.
-
-The fix tells them apart structurally — the label is the last defined entry in
-`row_header` — which is how `is_collapse` two lines below already reads the
-following row. Flat rollup (`group_rollup_mode: 'flat'`) writes each level into
-its own cell with no padding, so there is nothing structural to test there and
-it keeps the text test.
-
-The class was only half of it. `tbody th:empty { pointer-events: none }` in
-`regular_table.css` makes the padding cells inert, and a blank label cell is
-`:empty` as well — a `::before` caret is not content — so the fixed node drew
-its `+`, then ignored the click and would not even take the hover. The rule now
-excludes the tree classes, which is the distinction it was reaching for: a
-padding cell never carries one.
-
-`row-hover.css` uses `:empty` the same way in six border selectors. Those are a
-hover tint and are left alone; a row whose first level is blank loses a border
-highlight, and nothing else.
-
-Separate from the data question: null and `''` group as two distinct rows that
-look identical, so a `COALESCE(NULLIF(col,''), '(none)')` at load time is still
-worth doing on its own merits.
-
-## Applied: depth fields on a config update
-
-`0001-apply-depth-fields-on-config-update.patch` is **in** the vendored
-tarballs as of the 2026-09-17 build, from branch
-`apply-depth-on-config-update`. It fixes an upstream bug in
-`rust/perspective-client/src/rust/config/view_config.rs`:
-`ViewConfig::apply_update` applies ten fields and **neither `group_by_depth`
-nor `split_by_depth` is among them**. So a depth can be set when a view is
-created (`table.view({ group_by_depth: 1 })` — which is what the fork's own
-`depth_test.mjs` exercises) but never through `restore()`, which is how a
-viewer changes its own configuration. The field arrives, deserializes, and is
-discarded before the engine sees it.
-
-`group_by_depth` and the omission are both upstream; the fork mirrored
-`split_by_depth` alongside it faithfully, including the omission.
-
-Note the semantics, from `server.cpp`:
-
-```cpp
-ctx1->set_depth(row_pivot_depth - 1);                        // one-sided
-ctx2->set_depth(t_header::HEADER_ROW, row_pivot_depth - 1);  // two-sided
-```
-
-The config field counts **levels to show**; `view.set_depth()` counts the
-boundary below them. So `group_by_depth: n` equals `set_depth(n - 1)`, and
-`t_traversal::set_depth` adds the 1 back, so `n` renders `n` levels.
-`Forecast.jsx` therefore sends the stepper's level count unchanged, floored at
-1 — `0` would reach the engine as `set_depth(-1)` on an unsigned depth,
-underflow, and clamp to fully expanded. (It used to send `d + 1` to dodge that,
-which was off by one against the grid everywhere.)
-
-The patch is already committed on the fork's build branch (see
-`PROVENANCE.txt`), so rebuilding needs only the script. The `.patch` file here
-is the change on its own, for review or for carrying to a clean upstream
-checkout:
-
-```bash
-./ui/vendor/rebuild-perspective.sh             # from the fork branch
-# then re-resolve by path, as above -- a plain `npm install` keeps the old code
-
-# only on a fresh upstream checkout:
-cd $PSP_DIR && git apply /path/to/pf_app/ui/vendor/0001-apply-depth-fields-on-config-update.patch
-```
-
-### Host tools this tree needs
-
-The repo pins its Rust toolchain (`nightly-2026-06-01`), Emscripten (4.0.9) and
-Binaryen (132) exactly, and pins **nothing** for the host tools — no
-`packageManager`, no `.nvmrc`. So `npm i -g pnpm` and `pip install cmake` both
-give versions this tree was not written against. The script's preflight now
-rejects them, but for the record:
-
-- **pnpm 10.x.** 11+ rejects a repeated `--if-present`, which
-  `sh_perspective.mjs:189` emits once per package in scope.
-- **cmake 3.x**, not 4. Both satisfy the `>= 3.29.5` floor; CMake 4 removed
-  support for `cmake_minimum_required < 3.5`, which the C++ dependencies still
-  declare, and fails partway through the Arrow build.
-- **protoc >= 22** (33.2 known good).
-- **`pnpm pack`, not `npm pack`.** These packages depend on each other as
-  `workspace:^`, and only pnpm rewrites that to a concrete version. An
-  `npm pack` tarball is rejected at install: `Unsupported URL Type
-  "workspace:"`.
-
-Two commits on the build branch are local build concessions rather than
-fixes, and should be dropped before the real patch goes anywhere upstream:
-removing the `postinstall:playwright` step (it runs `playwright install
---with-deps`, which apt-installs system libraries and so needs root), and
-filling in the `allowBuilds` block pnpm 12 demanded.

ui/vendor/rebuild-perspective.sh

@@ -1,132 +0,0 @@
-#!/usr/bin/env bash
-set -euo pipefail
-
-# ---------------------------------------------------------------------------
-# rebuild-perspective.sh — rebuild the patched Perspective and re-vendor it
-#
-# pf_app runs a patched build of Perspective that exposes the column axis
-# expand/collapse the engine already implements (split_by_depth, and
-# expand_column/collapse_column). Upstream does not ship this yet, so the
-# built packages are vendored into this directory as npm tarballs.
-#
-# Source of truth:  https://github.com/fleetside72/perspective
-#                   branch expand-blank-tree-nodes (tip of the patch stack)
-#
-# This script exists because vendored binaries are opaque: once the .tgz files
-# are committed, nothing in the repo records how to regenerate them. Run this
-# after changing the fork, then commit the resulting tarballs.
-#
-# Only needed on a machine that is changing the engine. Deploys just run
-# `npm install`, which expands the committed tarballs - see ../README in this
-# directory.
-# ---------------------------------------------------------------------------
-
-PSP="${PSP_DIR:-$HOME/perspective}"
-BRANCH="${PSP_BRANCH:-expand-blank-tree-nodes}"
-VENDOR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-
-# The four packages that must move together. Perspective's own docs are
-# emphatic that loader, packages, data format and apache-arrow are one unit;
-# vendoring a subset would reintroduce exactly the drift that causes trouble.
-PACKAGES=(
-    "rust/perspective-js"
-    "rust/perspective-server"
-    "rust/perspective-viewer"
-    "packages/viewer-datagrid"
-)
-
-info() { echo -e "\033[0;34m==>\033[0m $*"; }
-ok()   { echo -e "\033[0;32m  ✓\033[0m $*"; }
-die()  { echo -e "\033[0;31m  ✗\033[0m $*" >&2; exit 1; }
-
-# -- preflight --------------------------------------------------------------
-[[ -d "$PSP" ]] || die "No Perspective checkout at $PSP.
-    git clone https://github.com/fleetside72/perspective.git $PSP
-    cd $PSP && git checkout $BRANCH
-    Set PSP_DIR to use a different path."
-
-command -v pnpm >/dev/null || die "pnpm not found. Perspective builds with pnpm, not npm."
-
-# The repo pins its Rust toolchain, emsdk and Binaryen exactly, but pins nothing
-# for the host tools -- no packageManager, no .nvmrc -- so `npm i -g pnpm` gets
-# whatever is newest and that is not what this tree was written against. pnpm 11+
-# rejects a repeated `--if-present`, which sh_perspective.mjs emits once per
-# package in scope, and the build dies with:
-#     error: the argument '--if-present' cannot be used multiple times
-pnpm_major=$(pnpm --version | cut -d. -f1)
-if (( pnpm_major > 10 )); then
-    die "pnpm $(pnpm --version) is too new; this tree needs pnpm 10.x.
-    npm i -g pnpm@10"
-fi
-command -v protoc >/dev/null || die "protoc not found.
-    Its VERSION selects which protobuf source tree the build clones, and a
-    version below 22 pulls a layout the build cannot consume. Needs >= 22
-    (33.2 known good). Distro packages are usually far too old."
-
-cmake_ver=$(cmake --version 2>/dev/null | head -1 | grep -oE '[0-9]+\.[0-9]+(\.[0-9]+)?') || die "cmake not found"
-cmake_major=${cmake_ver%%.*}; cmake_minor=$(echo "$cmake_ver" | cut -d. -f2)
-if (( cmake_major < 3 || (cmake_major == 3 && cmake_minor < 29) )); then
-    die "cmake $cmake_ver is too old; Perspective needs >= 3.29.5.
-    A user-level install works: pip3 install --user 'cmake==3.31.*'"
-fi
-# And not too new: CMake 4 dropped compatibility with cmake_minimum_required
-# below 3.5, which Perspective's C++ dependencies still declare. `pip install
-# cmake` gives 4.x by default, which satisfies the floor above and then fails
-# in the middle of the Arrow build.
-if (( cmake_major >= 4 )); then
-    die "cmake $cmake_ver is too new; CMake 4 removed support for
-    cmake_minimum_required < 3.5, which the C++ dependencies still use.
-    pip3 install --user 'cmake==3.31.*'"
-fi
-
-info "Perspective checkout: $PSP"
-git -C "$PSP" rev-parse --abbrev-ref HEAD | grep -qx "$BRANCH" \
-    || echo "  ! on branch $(git -C "$PSP" rev-parse --abbrev-ref HEAD), expected $BRANCH"
-commit=$(git -C "$PSP" rev-parse --short HEAD)
-dirty=$(git -C "$PSP" status --porcelain | wc -l)
-echo "  commit $commit$([[ $dirty -gt 0 ]] && echo " (+$dirty uncommitted files)")"
-
-# -- build ------------------------------------------------------------------
-# `metadata` first: it generates docs/expression_gen.md, which perspective-client
-# includes at compile time. Building a scope without it fails on the missing file.
-info "Building (this takes ~40 minutes cold, a few minutes warm)…"
-( cd "$PSP" && PSP_ONCE=1 PACKAGE="metadata,server,client,viewer,viewer-datagrid" pnpm run build )
-ok "build complete"
-
-# -- pack -------------------------------------------------------------------
-# pnpm pack, not npm pack. These packages depend on each other as
-# `workspace:^`, and only pnpm rewrites that to the concrete version on pack.
-# npm leaves it, and npm install then refuses the tarball outright:
-#     npm error Unsupported URL Type "workspace:": workspace:^
-info "Packing tarballs into $VENDOR"
-rm -f "$VENDOR"/*.tgz
-for p in "${PACKAGES[@]}"; do
-    ( cd "$PSP/$p" && pnpm pack --pack-destination "$VENDOR" >/dev/null )
-    ok "$(basename "$p")"
-done
-
-# -- record provenance ------------------------------------------------------
-# A committed .tgz is an opaque binary; without this the tie back to source is
-# only in someone's memory.
-cat > "$VENDOR/PROVENANCE.txt" <<EOF
-Built from https://github.com/fleetside72/perspective
-  branch  $BRANCH
-  commit  $(git -C "$PSP" rev-parse HEAD)
-  based on $(git -C "$PSP" describe --tags --abbrev=0 2>/dev/null || echo 'unknown')
-  built   $(date -u +%Y-%m-%dT%H:%M:%SZ) on $(hostname)
-  dirty   $dirty uncommitted file(s) in the source tree at build time
-
-Regenerate with ui/vendor/rebuild-perspective.sh
-EOF
-
-echo
-ls -la "$VENDOR"/*.tgz | awk '{printf "  %-52s %5.1f MB\n", $NF, $5/1048576}'
-echo
-ok "Done. Now, from ui/ — note the paths: npm resolves these from its cache
-    otherwise, since the version in the filename never changes.
-
-    rm -rf node_modules/@perspective-dev
-    npm install ./vendor/perspective-dev-{client,server,viewer,viewer-datagrid}-5.4.0.tgz
-    git add vendor package-lock.json && git commit"
-[[ $dirty -gt 0 ]] && echo -e "\033[1;33m  !\033[0m source tree had uncommitted changes — push them to the fork first"
-exit 0