pf_app/docs/pf_spec.md

# Pivot Forecast — Design

What the app does, how the pieces fit, and why — including what each design
replaced, where that history still explains the current shape. `CLAUDE.md`
holds the rules an agent needs while editing; this holds the reasoning. Where
either disagrees with the code (`routes/`, `lib/`, `ui/src/`, `setup_sql/`),
the code is the truth.

Related: `pf_perspective_options.md` (transport options, the DuckDB spike, why
the pre-aggregated path won), `PERSPECTIVE.md` (bundling), `ui/vendor/README.md`
(the patched Perspective build and how to rebuild it).

---

## Overview

A web app for building named forecast scenarios against a PostgreSQL table:

1. **Load actuals as a baseline** — segments pulled from the source,
   optionally date-shifted into the forecast period, so the starting point is
   "no change from last year".
2. **Load reference rows** — prior year, plan, open orders — for comparison.
   Operations do not adjust them.
3. **Adjust** — select pivot cells and scale, recode or clone them.

Every write is **append-only** and produces exactly one `pf.log` entry. An
adjustment inserts new rows carrying its entry's id; undo deletes those rows
and the entry. So a version is always the sum of its log, every figure is
attributable to the entry that wrote it, and any entry can be reversed alone.

Postgres holds the raw rows and is the single source of truth. The browser
holds a copy in [Perspective](https://github.com/perspective-dev/perspective)
(WASM), pre-aggregated to the display grain, and updates it incrementally from
each operation's response rather than reloading.

### Lineage

A port of an Excel model: a data tab of pre-aggregated rows, a pivot cache over
it, a double-click on a cell opening a VBA form (`/opt/forecast_api/VBA/fpvt.frm`)
that appended incremental rows, and a cache refresh. The append-and-let-the-pivot-sum
model follows that directly, and the price/volume rules in §Operations come from
the form's `calc_val` / `calc_price`.

### Stack

Node/Express (`server.js`, one `pg` pool, type parsers coercing `bigint` and
`numeric` to numbers so Arrow does not infer them as dictionary text);
PostgreSQL, everything in schema `pf`; React + Vite + Tailwind in `ui/`, built to
`public/app/`; Perspective 5.4.0 from a vendored patched fork (§The pivot).

### Pages and navigation

Three steps in a collapsible sidebar: **Setup** (register sources, assign column
roles, Generate SQL), **Baseline** (versions, load segments, version defaults,
tags), **Forecast** (the pivot and the operation panel). Setup and Baseline are
admin-only to write (§Access).

**The forecasts are listed in the sidebar**, above the steps, each with its
status and a star for the default. They used to be two dropdowns in the status
bar, which is otherwise read-only, so the one pair of controls in it did not read
as controls. The list is *not* nested under the Forecast step, where the idea
started: Baseline works on the same version, so hanging it off one step would
misrepresent its scope or duplicate it. The sidebar reads *what am I working on*,
then *what step am I on*.

**The source is not in the nav.** A raw table name means nothing to someone
editing a forecast. Picking a forecast sets the source, since
`pf.version.source_id` comes with it; the source picker appears only on Setup.
`versions` (the selected source's, which Baseline and Forecast need) and
`allVersions` (the sidebar's, a client-side fan-out over `/sources/:id/versions`)
are kept apart so widening one does not change the other.

The status bar keeps what follows from the selection: the forecast's name and row
count, with the physical table in a click-to-copy hover panel — pasting it into
psql is why it was ever on screen.

**Known collision:** "forecast" names both the scenario and the third step. The
scenario should own the word, leaving the step wanting another (`Adjust`).
Undecided.

### Where a browser lands

`App.jsx` restores the last source and version from `localStorage`, re-validated
against the live list so a deleted version re-points rather than 404ing every
call. With nothing stored it falls back to **`pf.version.is_default`** (one per
source, partial unique index). It used to fall back to `versions[0]` — newest
first, so whatever somebody created last, a scratch copy as readily as the plan —
and accounts that never pick a version, which is every sales account, followed
the forecasting work around.

Setting the default is **admin only**, since it decides where everyone else
lands; the Baseline checkbox is greyed for others. It is two statements (clear
old, set new) in one transaction, because the index refuses the instant between.
**A stored selection still wins** — the default is a fallback, so an account that
landed in the wrong scenario keeps landing there until it switches once.

### Renaming a version

Safe at any time, including closed. Nothing derives from the name: the table is
`fc_{tname}_{version_id}`, `pf.log` and `pf.layout` key on the id, and client
storage keys are id-based. The Baseline page's **This forecast** panel saves name
and description on blur via `PUT /versions/:id`; a name collision answers 409,
and a blank name is refused client-side because the route COALESCEs it away.

### API surface

All under `/api`; all but the auth routes need a session.

| Area | Routes |
|---|---|
| Auth | `POST /login`, `POST /logout`, `GET /me` |
| Browser | `GET /tables`, `GET /tables/:schema/:tname/preview` |
| Sources | `GET/POST /sources`, `GET/PUT /sources/:id/cols`, `POST /sources/:id/generate-sql`, `GET /sources/:id/sql`, `GET /sources/:id/values/:col`, `GET /sources/:id/dim/:group`, `POST /sources/:id/dim/:group/refresh`, `GET /sources/:id/lookup`, `GET /dim-period/cols`, `DELETE /sources/:id` |
| Versions | `GET/POST /sources/:id/versions`, `PUT/DELETE /versions/:id`, `POST /versions/:id/{close,reopen}`, `GET /versions/:id/table-info`, `GET /versions/:id/values/:col`, `GET/PATCH /versions/:id/tags`, `GET /versions/:id/bridge` |
| Data | `GET /versions/:id/data`, `GET /versions/:id/agg` |
| Loads | `POST /versions/:id/{baseline,reference}`, `PUT /versions/:id/baseline/:logid`, `DELETE /versions/:id/baseline` |
| Plans | `GET/POST /versions/:id/plans`, `PATCH/DELETE /plans/:pid` |
| Operations | `POST /versions/:id/{scale,recode,clone}` |
| Log | `GET /versions/:id/log`, `GET /log/:logid/debug`, `PATCH/DELETE /log/:logid` |
| Layouts | `GET/POST /versions/:id/layouts`, `PATCH/DELETE /layouts/:lid` |

`DELETE /sources/:id` answers 409 while the source has versions
(`pf.version.source_id` is `ON DELETE RESTRICT`) and never drops forecast tables.

---

## Data model

Install scripts in `setup_sql/`: `01_schema.sql` (metadata tables, with
`ADD COLUMN IF NOT EXISTS` backfills and one-off data migrations inline),
`02_auth.sql` (`pf.app_user`, `pf.session`), `gen_dim_period.sql` (creates and
fills `pf.dim_period`; fiscal start month set at the top; re-runnable).

`01_schema.sql` is idempotent but **not a migration runner**. Schema changes are
applied to the live database directly and mirrored into `setup_sql/` for fresh
installs.

### `pf.source`

`schema`, `tname`, `label`, `status`, `created_by`. `default_layout` is dead —
migrated into `pf.layout`, read by nothing. Registration seeds one `col_meta` row
per column as `dimension`.

**Columns come from `pg_catalog`, not `information_schema`**, which omits
materialized views — they are not in the SQL standard — and the app's primary
source is one. It looked columnless: registration seeded nothing and version
creation failed "No usable columns in col_meta" while col_meta plainly held them.
`RELATION_COLUMNS_SQL` in `lib/utils.js` returns the shape `information_schema`
did (`format_type` minus modifier; precision and scale unpacked from
`atttypmod`), so `mapType` and callers did not change. The table browser lists
from `pg_class` by `relkind` for the same reason.

### `pf.col_meta`

| Column | Meaning |
|---|---|
| `role` | `dimension` \| `value` \| `units` \| `date` \| `filter` \| `ignore` |
| `label`, `opos` | display name; ordinal, which orders everything generated |
| `is_key` | four separate uses, below |
| `dim_group` | groups functionally dependent columns — a part and its attributes, a date and its periods |
| `dim_period_col` | the `pf.dim_period` column this dimension is derived from at load |
| `in_grain` | part of the display grain (§Display-grain pre-aggregation) |
| `is_territory` | the column territory is expressed in; at most one per source, enforced on save |

Roles: `dimension` is categorical and sliceable. `value` (required) and `units`
(optional) are the measures — several may be marked, loads carry all, but the
operations use only the first of each by `opos` (§Proposed: additional
measures). `date` (at least one) — every date column is stored, shifted and
carried by every operation, and sliceable. `filter` feeds the baseline filter
builder and is not stored. `ignore` is not stored. Slices are validated against
every `dimension` and `date` column regardless of `is_key`.

**`dim_group` + `dim_period_col`.** A `date` column that is `is_key` with a
`dim_group` anchors the group: its sibling dimensions with a `dim_period_col` are
looked up in `pf.dim_period` against the *shifted* date at load time instead of
copied raw, so fiscal periods follow a baseline moved forward a year. Every such
group is honoured; this used to be a `.find()`, so a source with order, requested
and ship dates derived one set and copied the other two raw — shifted dates under
unshifted labels.

**`is_key`** is read in four places, none of them what the name suggests:

1. **The key of a `dim_group`** — `resolveGroup()` in `routes/sources.js` takes
   `members.find(c => c.is_key)` as the column `pf.dim_member` is keyed on
2. **Value completion** — `/sources/:id/values/:col` serves only `is_key`
   columns (the operation panel now completes from `/versions/:id/values/:col`
   instead, which takes any dimension or date)
3. **Sibling autofill** — `/sources/:id/lookup`, for `is_key && dim_group`
4. **The `dim_period` anchor** — `role = 'date' && is_key && dim_group`

Uses 2 and 3 want several flagged; use 1 needs exactly one per group, and with
more `.find()` silently takes the lowest `opos`. That has happened: a column null
throughout outranked its group's real key, so a member refresh matched nothing
and reported success with zero members. The meanings want separating — a
per-group key choice, or the rule that the key is the column the group is named
after — and until then a refresh finding two keys should refuse rather than guess.

### `pf.dim_period`

One row per month keyed on `sdat`, a GiST-indexed `drange`, and calendar and
fiscal year/quarter/month columns in several spellings. Joined as
`LEFT JOIN pf.dim_period dpN ON dpN.drange @> (date + offset)::date` — left,
because an unshipped order has no ship date and must keep its row.

### `pf.dim_member`

Master data per `dim_group`: one row per key value, siblings in `attrs`, with
`is_active` and `source_seen`. The source is transactional, often a view over all
history, so deriving members from it is slow (a full scan) and wrong (it describes
only what has been transacted — it cannot say a part is discontinued, or that a
new one exists before it sells). This is the app's own list, refreshed from the
source by an admin action but curatable apart from it. A refresh **merges**:
members gone from the source become `source_seen = false`, not deleted, so
curation survives; a key with several attribute sets across history takes the
most recent. It stamps an explicit run time rather than `now()`, which inside a
transaction is the transaction start.

### `pf.version`

| Column | Meaning |
|---|---|
| `name`, `description` | `(source_id, name)` unique |
| `status`, `closed_at`, `closed_by` | a closed version refuses every write but annotation |
| `exclude_iters` | jsonb, default `["reference"]` — `pf_iter` values scale and recode will not write |
| `is_default` | where a browser with no saved selection lands |
| `adjustment_segment`, `adjustment_bucket` | names a new adjustment is stamped with (§Naming) |
| `unlabeled_load`, `bucket_order` | dead — read by nothing |

Creating a version inserts the row and creates its forecast table and `pf_logid`
index in one transaction. Deleting drops the table; `pf.log`, `pf.layout` and
`pf.load_plan` cascade.

### `pf.fc_{tname}_{version_id}`

One table per version holding operational rows (`pf_iter` = `baseline`, `scale`,
`recode`, `clone`) and reference rows (`reference`):

```sql
CREATE TABLE pf.fc_{tname}_{id} (
    pf_id          bigserial   PRIMARY KEY,
    -- every non-ignored col_meta column, typed from the source by mapType
    pf_iter        text        NOT NULL,
    pf_logid       bigint      NOT NULL,
    pf_user        text,
    pf_created_at  timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX fc_{tname}_{id}_logid_idx ON pf.fc_{tname}_{id} (pf_logid);
```

No `version_id` column: the table is the version. **`pf_logid` is not a foreign
key** — undo deletes rows then log row in one transaction, and those are the only
two paths that remove either. **The index** is how undo, the change-log recount
and the grain key find an entry's rows; without it each is a sequential scan.
Tables created before it was added lack it.

### `pf.log`

| Column | Meaning |
|---|---|
| `version_id` | `ON DELETE CASCADE` |
| `pf_user`, `stamp` | the session account; when |
| `operation` | `baseline` \| `reference` \| `scale` \| `recode` \| `clone` |
| `slice`, `params` | the slice(s) written to (null on loads); intent plus what the server resolved |
| `label`, `bucket` | segment name and what it counts toward; required by the `log_named` CHECK |
| `tag`, `note` | initiative name, which the bridge groups on; free commentary |
| `row_count`, `value_total`, `units_total`, `measure_cols` | what the entry wrote, stamped at write time |
| `env`, `sql_text` | what it ran against; the statement as executed |
| `seq` | dead |

**Who writes what.** Load templates insert `label`, `bucket` and `tag` in their
`ilog` CTE. Adjustment templates insert `label` and `bucket`; their `tag` goes on
in a follow-up `UPDATE` in the same transaction (`tagLog()`), a hold-over from
when a new `{{tag}}` token would have stranded sources that had not re-run
Generate SQL. Totals, `env` and `sql_text` are written after commit by
`stampLogTotals()`.

**Stamped totals are not a cache.** An entry's rows never change after it is
written — only its own operation inserts them and only undo removes them, taking
this row too — so the totals are fixed. The change log used to join the whole
forecast table on every open to recompute them. `measure_cols` records which
columns they are in, since roles can be reassigned. Stamping is best-effort: a
failure does not roll back a write that succeeded, since totals can be recomputed
and the adjustment cannot. `GET /versions/:id/log?recount=1` recomputes by join
and writes back — the repair for rows deleted by hand, and the backfill for
entries older than the columns.

**`params` and `sql_text` are both kept on purpose.** One is the intent (slices,
scope, resolved increments), the other what it became. Keeping only one assumes
the translation is correct, which is exactly what is in doubt when rows look
wrong. `env` holds what cannot be reconstructed afterwards — the territory, the
version's `exclude_iters`, and the template's `generated_at`, a fingerprint (not
a version) showing whether an entry ran under a different template.
`GET /log/:logid/debug` returns the whole row; the list omits `sql_text` and
reports `has_sql`.

### `pf.sql`

One generated template per `(source_id, operation)`: `get_data`, `get_agg` (only
with a grain), `baseline`, `reference`, `scale`, `recode`, `clone`, `undo`.
`generateSQL()` in `lib/sql_generator.js` bakes in column names and leaves
`{{token}}`s — `fc_table`, `version_id`, `pf_user`, `note`, `tag`, `label`,
`bucket`, `params`, `slice`, `filter_clause`, `date_offset`, `where_clause`,
`exclude_clause`, `value_incr`, `units_incr`, `set_clause`, `scale_factor`,
`territory_clause` — for `applyTokens()` at request time. Generation deletes any
operation it no longer produces, since a stale `get_agg` after clearing the grain
would still be picked up. The stored `undo` is documentation; the route runs its
own statements.

Values are escaped (`esc()` doubles quotes), not parameterised — consistent with
the codebase, and the executed SQL reads as-is in the server log and
`pf.log.sql_text`.

**Editing the generator changes nothing until Generate SQL runs**, since the
templates are stored; a template carrying a token the running code does not
substitute fails at the database, reading as an unrelated client error. Restart
first, then Generate SQL, or the old code writes the templates.

### Other tables

`pf.layout` (§The pivot), `pf.load_plan` (§Loads and segments), `pf.app_user` and
`pf.session` (§Access).

---

## Loads and segments

A version's rows start as **segments**: independent queries against the source,
each appended as one log entry, each undoable alone. Two kinds, differing only in
`pf_iter`:

- **baseline** — the rows operations adjust
- **reference** — kept out of scale and recode by `exclude_iters`

`pf_iter` answers *can operations write these rows*. Whether rows belong in the
forecast number is the segment's **bucket** (§Naming): open orders load as
reference so nothing adjusts them, and are still part of the forecast. Neither
can be derived from the other. All load routes are admin-only.

### How a load's WHERE is built

The Baseline page's filter builder takes **groups** of conditions — AND within,
OR between — over `date` and `filter` columns (`=`, `!=`, `IN`, `NOT IN`,
`BETWEEN`, `IS NULL`, `IS NOT NULL`). A period is a filter like any other. A
toggle swaps the builder for a raw WHERE body.

**The client builds the clause; the route injects it.** `buildFilterClause()` in
`Baseline.jsx` renders `(a AND b) OR (c)` and sends it as `where_clause` beside
the structured `filters` (or `raw_where`). The route substitutes
`raw_where || where_clause` as `{{filter_clause}}` verbatim and keeps `filters` /
`raw_where` in `params` only so a segment can be re-opened and a plan derived.
Nothing server-side validates columns or escapes values, and the client does not
escape them either; `lib/sql_generator.js`'s own `buildFilterClause()`, which
does, is called by no route. The protection is `requireAdmin`: whoever can reach
the route can already send raw SQL, so validating the structured path would not
narrow what they can do. An empty clause becomes `TRUE` — the whole source — and
the client refuses to send one.

### Date offsets

`date_offset` is a Postgres interval (`1 year`, `-90 days`) applied to **every
date column**, so order and ship dates stay in step, with period dimensions
looked up against the shifted dates. Filter columns are never shifted: the offset
moves where rows land, not which are chosen.

It must be **whole days** (`assertInterval()`): shifted dates are cast back to
dates, so anything finer is rounded away, and Postgres reads a bare number as
seconds — `"30"` used to be accepted and move nothing. `30 days` is not a month
either; use `1 month`. Checked on baseline loads, segment edits, recode and clone
— **not** on reference loads, where a bad offset surfaces as a Postgres error.

The offset is typed as an interval string. It replaced year/month spinners,
which could not express days and clamped at zero, so a segment could only move
forward in whole months. The timeline preview parses it only far enough to draw
the source and shifted periods; Postgres is the authority.

### Naming, editing, clearing

A load's **label is required** (400 without) — the segment name the pivot shows
and reports group on — and its bucket defaults to it (`loadNames()`).

`PUT /versions/:id/baseline/:logid` edits a segment by deleting its rows and log
row and re-running it in one transaction, under a **new** logid. Refused (409)
once any adjustment exists, since those were sized against the old totals.
Annotations carry over with `??`, not `||`: the form has no tag input, so an
absent tag must survive, while an explicit `''` clears.

`DELETE /versions/:id/baseline` removes every `baseline` row and entry (not
reference). Not undoable.

### Load plans — `pf.load_plan`

A segment used to exist only once run, so cloning a definition meant re-running
every load, and there was nowhere to hold "this cut, but with the dates changed
first". A plan is the definition alone — `kind`, `label`, `bucket`, `tag`, `note`,
`filters`, `raw_where`, `date_offset`, `seq` — with `run_logid` naming the entry
it produced, or null.

- **Deliberately not `pf.log`**: every row there means a write happened, and
  unexecuted intent would cost it that meaning.
- **`run_logid` is `ON DELETE SET NULL`**, so undoing a load returns its plan to
  unrun instead of deleting the definition. The list reads the entry's
  `row_count` through the FK to tell run from unrun.
- **Running is two round trips** — the ordinary load POST with the plan's fields,
  then `PATCH /plans/:pid` with the `log_id`. The alternative was a second copy of
  the load route's substitution; a plan that reads unrun while its rows exist is
  visible and fixable, a divergent copy of the load SQL is neither.
- **Run all stops at the first failure.** A definition's segments are not
  independent — references date-shift off a baseline — so carrying on past a
  broken one builds something nobody described.

Known issue: `runPlan()` does not send the plan's `tag`, so a tagged plan loads
untagged.

### Cloning a version's definition

`POST /sources/:id/versions` with `clone_from` copies the template's settings
(description unless given, `exclude_iters`, adjustment names), its
**version-scoped** layouts, and its segments as plans — and **runs nothing**. The
point of cloning a definition is to change the dates before anything loads; a
clone that fetched the baseline first would have to be undone to be edited.

Not copied: **adjustments**, in any form — a scale's target was sized against the
rows that were there, so replaying it against another load is arithmetic nobody
asked for; and **source-wide layouts**, which already apply and would duplicate
by name.

`cloneDefinition()` reads the template's plans, or derives them from its
`baseline`/`reference` entries when it has none — every version built before
plans existed. `params` already stores `filters`, `raw_where` and `date_offset`,
so this is a read, not a reconstruction.

### Replay

Re-running a version's adjustments against a restated baseline is **not built**.
Nothing accepts a `replay` flag and nothing answers 501, whatever earlier specs
said. See §Open questions.

---

## Naming: segments, buckets, notes

Four pivot columns come from `pf.log`, joined on `pf_logid`, not from the
forecast table:

| Column | From |
|---|---|
| `pf_segment` | `l.label` — which load, or which kind of adjustment |
| `pf_bucket` | `l.bucket` — what it counts toward: Actual, Plan, Forecast, Prior Year |
| `pf_note` | adjustments: the tag, else the note; loads: null |
| `pf_op` | `l.operation` — `/agg` and pushed-back rows only |

Separate on purpose: commingling a segment name with an adjustment's note makes
neither pivotable. In grain mode `pf_logid` is in the grain, so the join adds no
rows. `SEGMENT_EXPR` / `BUCKET_EXPR` / `NOTE_EXPR` in `lib/sql_generator.js` are
the single definition, shared by the generated `/agg` and the hand-written
`/data` cursor, which must agree. The operation routes stamp the same fields on
the rows they return, since `RETURNING *` lacks them. The `/data` cursor does not
emit `pf_op`, so a raw-mode pivot cannot group on it.

### Every log entry is named

`label` and `bucket` are **stored, not derived**, and every entry has both —
`log_named` refuses one without. A report gets the pivot's segment and bucket by
joining `pf.log` and selecting two columns, with no rule of its own. Stamped at
write time:

- **A load** — `label` required, `bucket` defaulting to it.
- **An adjustment** — no name inputs, so the version's `adjustment_segment` /
  `adjustment_bucket`, else `ADJUSTMENT_SEGMENT = '99 - Adjustments'` and
  `ADJUSTMENT_BUCKET = '04 - Forecast'`. `adjustmentNames()` resolves them once
  per request, for the INSERT and the returned rows alike.

The version defaults are set on the Baseline page (**Defaults for new
adjustments**) and name the *next* adjustment; existing entries keep theirs.
Renaming an entry is `PATCH /log/:logid` — from the Baseline page's segment list —
which renames but refuses to clear, owner or admin only because the names are the
pivot's columns for everyone. Relabelling needs a page reload to show, since the
label is part of the aggregated row. A source registered before names were
stored needs Generate SQL once so its templates write them.

**What this replaced.** The names were derived at read time: a load's `label`,
else `tag`, else `note`, else the version's `unlabeled_load`, else `Unlabeled`;
an adjustment's `label`, else the version default, else the constant — through a
join on `pf.version`. That made the pivot the only reader that knew a row's
bucket: the CASE was copied into `/agg`, the `/data` cursor and a reporting
procedure in `/opt/forecast`, and every direct query had to copy it again. And
editing a version's fallback silently renamed every past adjustment. The backfill
in `01_schema.sql` stamped existing entries with that rule, so nothing on screen
changed name.

### Column order is stored text

Perspective orders column groups by the value string, and `asc`/`desc` only
reverses that, so Prior Year → Plan → Actual → Forecast is expressible in neither
direction. A `"01 - "` prefix is the only lever, and it lives in the stored label
and bucket, typed by whoever names the segment. Nothing derives it.

The one hardcoded ordinal is `99 - Adjustments`, keeping unnamed adjustments
after every *numbered* segment. That proviso is the scheme: ordering is string
ordering, so `99` lands last only while loads carry `01`–`0n`, and an unnumbered
segment sorts after it (digits precede letters). The old `'(adjustment)'` sorted
*first* for the same reason read the other way. Relabelling an adjustment's log
row is how one kind of adjustment is split out from the rest.

**What this replaced.** The prefix was computed client-side as Perspective
expression columns (`pf_bucket_ord`, `pf_segment_ord`) from `pf.log.seq` and
`pf.version.bucket_order`. It ordered the pivot and nothing else, so every other
reader disagreed; `restore()` replaces `expressions` wholesale, so it had to be
re-applied after every layout load; and ExprTK's scanner tests each *byte* with
`isprint()`, so a label with anything outside printable ASCII could not be
ordered at all. `DEAD_ORDER_EXPRS` in `Forecast.jsx` strips those names from old
layouts.

### Tags

A tag names an **initiative** — a line in the ledger and the bridge. It never
affects rows, so it stays editable, even on a closed version. It is not a row
anywhere, only a value repeated across a version's entries; it exists because
somebody typed it.

**Completions are per version.** `GET /versions/:id/tags` returns this version's
tags with `uses`, `last_used`, `owners`, `can_edit`, refetched on version change
and after every tagged write. It used to be `GET /sources/:id/tags`, across every
version of the source — which put last year's initiatives in front of this
year's and reported use counts earned elsewhere. The cost is that a fresh version
completes from nothing until it has a tag, which is honest: cloning a definition
copies no adjustments.

**Managed on the Baseline page.** `PATCH /versions/:id/tags` `{ from, to }`
rewrites every entry carrying `from`. A `to` that already exists **merges** —
usually the point, since "Pull Forward" and "Pulled Forward" are one initiative
reported as two. An empty `to` clears. Both are confirmed first, being invisible
as a diff afterwards. Permission is ownership of *every* entry the tag is on — a
half-applied rename leaves the initiative split, the state being fixed — so rows
the reader cannot rename are greyed; the route locks the rows before checking, so
check and rewrite see the same set. The Tags panel sits outside the page's
read-only fieldset: a rep may rename what they named.

**Completion matches on words.** `TagInput` in `OperationPanel.jsx` lowercases,
strips punctuation and stopwords and lightly stems both sides, so "Pulled
Forward" finds "Pull Forward" and one inserted word does not hide an existing
tag. Something that reduces to an existing tag without being it says so
(*"Already used as … — use that"*). Nothing merges automatically. The change
log's inline tag cell still uses a plain datalist.

The panel used to carry six quick-pick pills under the tag field; they wrapped to
two rows once tags were longer than a word and pushed the docked form past its
border. The list shows the tag alone — an option's text renders as a second
column, so a use count widened every row for a number nobody picks on.

---

## Selection

A selection is a list of **slices** (`{column: value}`) plus the pivot's own
filter as **scope**. The server turns them into a WHERE clause; the ledger totals
the loaded table through the same filters, so the panel and the write agree.

### From a click to a slice

`perspective-click`'s own filter list cannot be trusted: the datagrid's
`getCellConfig` builds it with `pivot_value ? [pivot, '==', pivot_value] :
undefined`, so **the engine drops a blank group key** — and, being a falsy test,
a `0`. A dropped key widens the write: clicking the blank node under a customer
selected the whole customer, and the ledger agreed, totalling through the same
list. So `cellFilters()` in `Forecast.jsx` re-derives both axes:

- **Rows** from `detail.row.__ROW_PATH__` — raw values, so dates stay epoch
  millis. A subtotal's shorter path leaves deeper levels absent.
- **Columns** from the clicked column's name (taken from the mousedown's cell;
  the event names only the measure). The name is the visible split values joined
  with `|`, measure last, so the measure is popped before mapping `split_by`
  positionally. Mapping every segment used to read the measure as a value of the
  first collapsed dimension — a slice matching no row.

A plain click replaces the selection; ctrl/⌘/shift toggles a slice. The event
carries no modifier flags, so they are read off the preceding mousedown.
`buildEffectiveSlice()` keeps dimension, date and `pf_` keys, turns dates into
ISO strings, and drops the rest; duplicates are collapsed, since under `each` a
repeat would apply twice. A Perspective computed column does not map back to
stored rows, so only native columns slice.

### Dragged regions

The datagrid runs in `edit_mode: SELECT_REGION`, forced on every restore so a
layout cannot switch it off. A drag fires `perspective-select` with a
**ViewWindow** — `{ start_row, end_row, start_col, end_col }` — on every
mouseover, so the handler keeps the latest and a window `mouseup` commits it,
each cell going through `cellFilters()`. A single-cell region is ignored:
`perspective-click` owns clicks, and handling both would undo a ctrl-click.

### Blanks: null and `''` are different values

Both occur in real data as different rows, so they stay distinct from click to
WHERE; only `undefined` means absent.

- `buildWhere()` renders null as `IS NULL`. It used to fall through `esc()` to
  `= ''`, selecting *the other blank* — the slice named one node and the
  operation wrote the other. In a list the null arm is ORed on, since `IN (NULL)`
  is never true.
- The ledger filters null with `is null`, not `== null`: the engine matches on
  type, and a null threshold on a string column is another dropped predicate.
- On the column axis, `t_tscalar::to_string` renders `''` as `''` and a null as
  the text `"null"`, so `"null"` is read back as null. This was believed
  impossible — a null was thought to join as `''` too — so `''` cells were
  refused and `"null"` was taken literally, a slice matching nothing. The one
  ambiguity left is a value that really is the text `null`.
- The panel prints `(null)` / `(blank)`, and `—` only for a column not in the
  slice: isolating the blank node and ignoring the column are opposite
  statements and used to look identical.

### The pivot's `pf_` columns are part of the slice

- **Table columns** — `pf_iter`, `pf_user`, `pf_logid` (`TABLE_SLICE_COLS`) —
  compared directly.
- **Log columns** — `pf_segment`, `pf_bucket`, `pf_note`, `pf_op`
  (`COMPUTED_SLICE_COLS`) — resolved to `pf_logid IN (SELECT id FROM pf.log WHERE
  version_id = … AND <expr> …)`, exact because the name lives on the log row.
- **Any other `pf_` key is refused** — skipping is how selections widened.

They **narrow a slice but do not make it selective** (`pf_iter = scale` alone is
every adjustment): `assertSelective()` needs a dimension, date, `pf_segment` or
`pf_bucket` key.

**What this replaced.** `pf_iter` was stripped on the theory that cells differing
only by iter band are one slice; `pf_note` and `pf_op` were refused, and the
client pre-empted that by stripping; `pf_segment` and `pf_bucket` were dropped
before that. The ledger stripped none of them. So under the default layout,
grouped by `pf_iter`, a click on a customer's `scale` row totalled the scale rows
and wrote to all the customer's rows; a click on one bucket's cell scaled every
bucket there.

### Several slices, and total cells

Several slices become an OR of AND-groups, not per-column `IN` lists —
`{A:1,B:1}` + `{A:2,B:2}` as `A IN (1,2) AND B IN (1,2)` would also match
`A:1,B:2` — parenthesised, since callers append `AND pf_iter NOT IN (…)`.

**A total cell is its displayed amount.** The grand-total row names no row
dimension, so its slice is whatever the column axis and the pivot filter leave —
possibly `{}`, *everything the filter shows*. `{}` is accepted only with a
non-empty scope (`normalizeSlices()`, `assertSelective()`); alone it would be the
whole version and is refused on both sides. `buildWhereAny()` reads a union
containing `{}` as `TRUE`, the scope then ANDed on — dropping it would narrow the
write to the other cells. The panel names it "Total".

### Scope: the pivot's filter travels with the slices

A slice can only mean equality, so a bounded season, an in-list or a substring
has no slice form. The filter is read off `viewer.save()` — in the table's own
types, with the user's operators — and sent as `scope`, `[col, op, value]`
triples; the ledger totals against the same list. Without it the ledger counted
rows the grid was hiding and the operation wrote them. It is ANDed onto each
statement rather than folded into the slices, which under `each` would repeat it
per statement.

`SCOPE_OPS` in `lib/sql_generator.js` is the whole allowed list, under
**Perspective's** names: `==`, `!=`, `>`, `>=`, `<`, `<=`, `in`, `not in`,
`is null`, `is not null`, and the substring family. Anything else is refused —
a scope silently dropped is a write wider than the panel that authorised it.
`pf_` log columns take `==` or `in` only.

- **Substring operators map to `ILIKE`**, because the engine's
  `t_tscalar::contains` / `begins_with` / `ends_with` lowercase both sides. `LIKE`
  would write a fraction of what the grid showed, and narrower is no safer than
  wider — neither is the set the ledger counted. The engine matches the term
  literally, so `%`, `_` and `\` are escaped. Residual difference: the engine
  folds ASCII only, `ILIKE` follows the collation.
- **`matches` / `not matches` stay refused.** RE2 in the engine, not in Postgres —
  `\b` is a word boundary in one and a backspace in the other — so a pattern
  valid in both could match different rows, which is worse than a 400.
- **An empty term is refused.** An unfinished filter row arrives as
  `[col, '==', null]`, and `ILIKE '%%'` or `= ''` would read as every row or none
  rather than as the mistake it is.

The ledger used to drop a slice's column whenever the scope named it, assuming
they could not disagree; under a substring filter, clicking one part then totalled
every matching part. It now drops only an exact `==` repeat. Territory (§Access)
is ANDed on after the scope, from the session.

### Selection highlight

The datagrid highlights `model._selection_state.selected_areas` and wipes it on
every mousedown, so a ctrl-click selection would show only its last cell.
`areasRef`, a `sliceKey → rectangles` map beside `slices`, is pushed back after
every change and pruned by live slice key on any deselect, so grid and panel
cannot disagree.

Rectangles are **re-derived when the grid moves**. A sort reorders rows and a new
segment column shifts every column; tracked coordinates would then highlight
whoever took the position while `slices`, being value-based, still named the
right row — the grid would be the only thing lying. `relocateAreas()` sweeps the
displayed rows as a drag would and keeps cells whose slice is still selected. A
slice can therefore light more cells than were clicked (a slice not naming the
segment matches the customer under old and new segments alike), which is what it
covers. Past `MAX_RELOCATE_ROWS` the highlight is cleared rather than left wrong.

---

## Operations

`POST /api/versions/:id/{scale|recode|clone}`, common body:

```json
{ "slices": [ { "channel": "WHS" } ], "scope": [ ["season", "<=", 2027] ],
  "apply_mode": "prorate", "tag": "volume_push", "note": "optional" }
```

`slices` — one or more (a legacy single `slice` is accepted). `apply_mode` —
`prorate` (default) is one statement over the union and one entry; `each` is one
per slice, so each reaches its target alone and undoes separately (for recode and
clone it only decides how many entries). `pf_user` is the session's. A closed
version answers 403. Operations are not admin-gated; territory and ownership
govern them. The response is the new rows — whole in raw mode, aggregated in
grain mode — for `table.update()`; no reload.

### SQL shape

An `ilog` CTE inserts the log row and the insert CTEs reference its id;
`{{where_clause}}` is the selection and `{{exclude_clause}}` keeps
`exclude_iters` out.

**Every date column travels** — the written columns are every dimension, every
date and the first value and units column. Scale and recode used to carry only
the first date, so on a source with order, requested and ship dates they wrote
the others **null**. The period dimensions were fine, being dimensions — which
hid it: labels right, dates gone. Those dates are sliceable, so an adjustment
scoped to a ship date matched no adjustment ever written. Clone and loads were
always right. Rows written before the fix keep the nulls; deleting and replaying
those entries is the honest repair, since re-deriving them would be inference.

**One row per coordinate, not per row read.** Reads GROUP BY every dimension and
date. One-for-one, every operation inherited the row count of everything before
it at that coordinate, and the table grew super-linearly with the work done.
Only `pf_logid` and `pf_iter` collapse, so nothing goes null or unsliceable; the
maths is unchanged, `sum(sum(x)) OVER ()` being the aggregate of aggregates.

### Scale

Distributes an increment in proportion to each coordinate's current amount:

```sql
round(("value" / NULLIF(total_value, 0)) * {{value_incr}}, 2)
round(("units" / NULLIF(total_units, 0)) * {{units_incr}}, 5)
```

`resolveIncrs()` in `routes/operations.js` computes the two increments per
statement, which is what makes `each` land per slice. Value and units each take
at most one of a target (`target_value`/`target_units`), a percentage
(`value_pct`/`units_pct`) or a change (`value_incr`/`units_incr`), resolved
independently; `target_price` is a third input; legacy `pct: true` is honoured.

**`target_basis`** — what a target or percentage measures against:

- `adjustable` — only the rows the operation can write. The **server default**.
- `selected` — everything the pivot shows, `exclude_iters` rows included. Those
  cannot move, so the adjustable rows absorb the difference and the pivot lands
  on the number asked for; otherwise a target set against a visible total
  overshoots by the fixed rows.

The panel defaults to `selected` and sends it only when the selection holds
excluded rows (otherwise they are identical). The basis row is tinted and tagged
"target", and clicking either row switches.

#### Price does not move unless it is named

A price move is invisible in the figure being edited — the dollars land either
way, and only the implied price says whether volume followed. So:

**`plug`** answers the one ambiguous case, dollars alone: `volume` (default,
price holds, units scale in proportion) or `price` (volume holds). From the Excel
form:

```
plug volume:  pchange = fVal/(pVal+bVal); fVol = (pVol+bVol)*pchange
plug price:   fVol = pVol + bVol
```

**A qty input follows the same rule**: with dollars unnamed they follow at the
current price, `value = (curValue / curUnits) * unitsIncr`. Holding them made
every qty edit a price edit — more cases at the same revenue is a price cut
nobody asked for. A slice with no units has no price, so its dollars stay put.

**The price held is the adjustable rows' own**, never blended with the fixed
rows. The SQL prorates over the adjustable rows only, so an increment sized at a
blended price moved qty by a different factor than dollars wherever reference
rows priced differently — and taking a selection to its fixed total drove qty past
zero, grains with positive sales and negative qty. The ledger preview uses the
same price.

**Naming the price** (`target_price`, the Excel form's Edit Price) holds it, and
the other named measure decides the rest, on the chosen basis:

| Named | Result |
|---|---|
| price alone | qty holds; dollars = price × qty |
| price + qty | qty lands as named; dollars = price × that qty |
| price + dollars | dollars land as named; qty = dollars / price |
| all three | refused — two degrees of freedom, three numbers |

This branch used to read only a qty *target* (a qty change sized dollars on the
old qty), took the fixed qty off a target but not the fixed dollars, and let a
dollar figure outrank the price. Worse, the panel sends no `plug` with a price,
so the server read `volume` and scaled qty by the dollar change too, putting
price back where it started. A named price now bypasses the plug.

So the deliberate ways to move price are **dollars and qty together** and
**naming the price**; both outrank the plug.

**What this replaced.** `plug` defaulted to `price`, being what the API did
before the flag existed — so an unqualified dollar edit and every qty edit moved
price, and only a hand-picked `volume` did not. The client's preference is keyed
`pf_scale_plug_v2` so a browser carrying the old `price` choice does not go on
silently moving price.

#### The ledger and its three boxes

The scale form is one continuous statement: baseline, adjustments grouped by tag
(untagged ones collapsed), fixed segments, total, then the edit, with a running
total closing on the selected total. Value, qty and price each have `New` /
`Change` / `% change` boxes; typing in one derives the others, and the box typed
in is what is sent — the row you type in *is* the mode. A **Result** line
previews all three under the server's rules and marks the column that cannot
move `· holds`.

**The boxes foot** — price = value / qty — so at most two are inputs.
`effectiveScaleInputs()` in `OperationPanel.jsx` decides which when all three are
typed: value typed last lets the plug decide (`volume` drops qty, `price` drops
price), since a dollar change says nothing about price or volume; otherwise the
more recently touched of the other two stays. The dropped box keeps its text, so
flipping the plug brings it back. `buildPayload()` in `Forecast.jsx` uses the same
function, so what is posted is what is shown. Clicking into a derived figure pins
it — "units are this" rather than "units may move" — and **unpin** releases it.
Rough edge: all three typed, dollars last, the choice is really which earlier
number to keep, and "Volume" there moves price too.

**The Moves control** is the plug. It lights whatever the edit moves — both if
both do — and takes a click only while it is the one deciding. It used to render
only for a dollars-only edit, so it was absent before anything was typed and
vanished when a measure was pinned, even by an accidental click — two states that
both looked like a missing control. It now stays, greyed when the inputs have
answered it, showing their answer.

A price change or percentage is resolved client-side to an absolute
`target_price`; the server has no price increment form.

#### What the log records

`params` carries the intent (`pickIntent()`: inputs, `target_basis`, `scope`,
`plug`) plus:

- **`resolved`** — `value_incr`, `units_incr` and **`method`**: `price` (named),
  `both` (dollars and qty named), `volume` (qty named), else the plug. The
  request alone cannot say which moved — the plug is a per-browser preference —
  so a price edit used to be findable only by noticing qty had not moved.
- **`expected`** — the panel's Result line as a change on its basis (under
  `each`, combined and flagged so), beside `resolved` so a panel/server
  disagreement shows in the log. Nothing compares them yet.

#### Refusals

- **A pool netting to ~zero** (below 1% of gross): shares are `row / total`, so
  as the net nears zero rows fly to extreme opposite values. Offsetting slices
  are the usual cause; `each` handles them.
- **Nothing to scale** — every unit already on target or empty; individually
  skipped slices come back in `slices_skipped`.
- **Holding price on a selection with no value** — no price to hold.

Rounding each row to 2dp can leave a target a cent off; landing exactly would
need a correction row.

### Recode

Moves the selection to other dimension values in one entry: **negative rows**
zero it out where it was, **positive rows** put it back with `{{set_clause}}`
replacing the named dimensions.

**A date shift moves the put-back leg only**, so a shifted recode is a timing move
— pull forward, push out — and `set` may be empty. Moved rows' period dimensions
are re-derived against the shifted date, as clone's are; an explicit `set` wins;
a row with a null date keeps its label, since a lookup on nothing would blank it.
Without a shift nothing is derived.

**Coordinates already netting to zero are skipped** (`HAVING sum(x) <> 0`),
rather than writing two empty rows each — which a second recode over cells a
first one emptied would do for all of them. If nothing is left the request is
refused ("Nothing to recode") rather than committing an empty entry. A scoped
account cannot set the territory column (§Access).

### Clone

Copies the selection as new business: `set` overrides dimensions, `scale`
multiplies (default 1), `date_offset` shifts every date; the original is
untouched. Period dimensions come from the calendar against the shifted date —
else a mix moved forward a year keeps last year's labels — and `set` wins. Unlike
recode, a null date blanks them.

**`exclude_iters` does not apply to clone.** It stops operations *modifying*
reference rows — scale would attribute movement to prior-year rows, recode would
write negatives against them. Clone does neither; copying a plan or prior year
out of reference is the operation working as intended, and excluding them made a
deliberate selection silently produce nothing. **`from_logid`** narrows instead,
for a selection spanning segments where only one is wanted.

Known issue: a clone matching nothing commits an empty entry with
`rows_affected: 0`, where scale and recode refuse.

### Undo

`DELETE /log/:logid` deletes the entry's rows then its log row in one transaction
and returns `pf_gkeys` (grain mode) or `pf_ids` (raw) for `table.remove()`; the
view re-sums. Undo is by **ownership**, not territory — your own entries or an
admin's override — because it removes rows wholesale, and half-undoing one would
leave a state nothing describes (and two accounts can share a territory). Closed
versions refuse. Undoing a load returns its plan to unrun.

Known issues: the `pf_gkeys` come from **live** col_meta, so after a grain change
they match nothing in the loaded table and the pivot keeps the rows (the database
is still right); recode and clone log `apply_mode: unit.mode`, never set, so
their `params` omit it.

---

## Display-grain pre-aggregation

**Built, with a static grain.** Path B of `pf_perspective_options.md` — a
pre-aggregated extract in a **native** Perspective table — rather than Path A,
live aggregation through a DuckDB virtual server (parked on
`spike/duckdb-virtual-server`), which loses expand, collapse, depth, sort and
filter.

**The problem.** Shipping raw rows makes the browser pivot all of them and load
time grows with raw rows. Aggregating server-side to the grain the pivot
displays cuts the row count by orders of magnitude while Perspective still does
all the rollup locally.

### Grain definition — `pf.col_meta.in_grain`

The flagged `dimension`/`date` columns, plus always `pf_iter` and `pf_logid`.
`grainOf()` in `lib/sql_generator.js` is the single definition (`Setup.jsx` and
`routes/log.js` mirror it), baked into the templates at Generate SQL so load and
operations agree by construction.

- **Additive measures only.** Sums are exact; an average would ship as
  sum + count; `count(DISTINCT …)` cannot be pre-aggregated. Forecast measures
  are sums.
- **An unflagged dimension cannot be pivoted or filtered on.** Leaving
  high-cardinality columns (part numbers, raw dates) out is what keeps the grain
  from exploding back toward raw; the price is that they are unavailable. Dates
  usually enter through their derived period dimensions.
- Generation refuses a grain column the table never populates, or a grain with
  no measure.

`in_grain` means *eligible*; today the grain is exactly the eligible set.

### The model: append deltas, let the view sum

The Excel pivot cache again: pre-aggregated rows go to the table, the pivot cuts
them locally, and each adjustment is **appended**, never a recomputed total. Rows
are aggregated **per `pf_logid`** — one per grain tuple × `pf_iter` × entry — and
the view sums them. It was also the smallest change from raw mode, which already
appended results with `table.update()`.

**Synthetic key.** Perspective indexes on one column, so each row carries

```sql
md5(concat_ws(chr(31),
    COALESCE(col1::text, chr(30)), …, pf_iter, pf_logid::text)) AS pf_gkey
```

`pf_logid` keeps each entry's contribution distinct, so an append accumulates
rather than replacing a bucket and undo removes exactly that entry. Plain
`concat_ws` skips NULLs, so `('a', NULL)` and `(NULL, 'a')` would collide and
silently merge two groups; `chr(30)` also keeps NULL apart from `''`. The md5
because the key is an opaque handle read only by `update()`/`remove()`: raw, it
was long and, unique per row, defeated Arrow's dictionary encoding — the largest
column in the payload. 128 bits keeps collisions unreachable.

### Initial load — `GET /api/versions/:id/agg`

```sql
SELECT
     t.<grain cols>, t.pf_iter, t.pf_logid, <pf_gkey>
    ,SUM(t.<value>) AS <value>, SUM(t.<units>) AS <units>
    ,l.label AS pf_segment, l.bucket AS pf_bucket
    ,<NOTE_EXPR> AS pf_note, l.operation AS pf_op
FROM {{fc_table}} t
LEFT JOIN pf.log l
    ON l.id = t.pf_logid
WHERE {{territory_clause}}
GROUP BY <grain cols>, pf_iter, pf_logid, l.operation, l.label, l.tag, l.note, l.bucket
```

The `pf.log` columns depend on `pf_logid`, but Postgres will not infer it, so
they are grouped (`LABEL_GROUP_COLS`). **Territory applies before the GROUP BY**,
since its column need not be in the grain. The route sets `LOCAL enable_sort` and
`enable_incremental_sort` off: seeing `pf_logid` pre-sorted off its index, the
planner picks a sort-based GroupAggregate for the "free" sort, but comparing
dozens of keys per row makes it several times slower than a HashAggregate,
whatever `work_mem` or statistics say. The client loads the result with
`worker.table(buffer, { index: 'pf_gkey' })`.

### Write and delete paths

Operations insert raw rows under a new logid as always; in grain mode the final
select aggregates **that entry's rows** to grain (`FROM ins`, or recode's
`neg UNION ALL ins`) — safe because they all carry the new logid. The keys are
therefore always new, `update()` **appends**, and nothing is recomputed.

Undo removes an entry's keys and lets the view re-sum — no re-aggregation, no
emptied-bucket handling. `RETURNING` does not take `DISTINCT`, so:

```sql
WITH del AS (
    DELETE FROM {{fc_table}} WHERE pf_logid = $1
    RETURNING <grain cols>, pf_iter, pf_logid
)
SELECT count(*)::int AS rows_deleted, array_agg(DISTINCT <pf_gkey>) AS pf_gkeys
FROM del;
```

### Raw mode

With nothing flagged, no `get_agg` exists and the client uses `/data`: every row
plus `pf_segment`, `pf_bucket` and `pf_note`, indexed on `pf_id`; operations
return whole rows and undo returns `pf_ids`.

### Transport

Both endpoints return Arrow IPC in **one record batch** with `X-Row-Count`.
`/data` reads a cursor in batches into column arrays (row objects multiplied peak
heap) and builds one table with `tableFromArrays`; `/agg` does the same from its
result. One batch because per-batch encoding creates independent dictionaries,
and Perspective's WASM reader crashes on the replacement messages. An earlier
plan to stream batches into `update()` behind a large-dataset banner was never
built, for that reason.

**Load time is dominated by row count, not payload size.** Even a good grain on a
wide source leaves enough rows that getting them out of Postgres and into JS
dwarfs the aggregation and the Arrow build; halving the payload (the md5 key)
barely moved it. The remaining lever is the dynamic grain.

### The dynamic variant

The static grain means an unflagged dimension is never pivotable and every
flagged one is paid for even when the pivot uses none of them. The additive next
step: group by the fields the current pivot uses — the viewer's `group_by`,
`split_by` and `filter` intersected with the eligible set — and re-fetch when they
change. That needs the grain passed per request rather than baked into `pf.sql`,
with operation responses and undo keys computed at the loaded table's grain.

**Grain drift.** Changing `in_grain` after a load needs Generate SQL and a page
reload, since the loaded table's index and columns are fixed; until then undo's
keys, from live col_meta, match nothing.

---

## The pivot

### Perspective, vendored

Perspective 5.4.0, `@perspective-dev/*` — **not** FINOS `@finos/perspective` —
built from a patched fork and committed as tarballs in `ui/vendor`, whose README
covers the patches (column-axis expand/collapse, `split_by_depth`, depth applied
on config update), provenance and rebuild. Bundled through the `/inline`
entrypoints, **never a CDN**: the 4.x CDN bundle resolved its server WASM to an
unversioned path and silently pulled whatever was newest.

### Layouts — `pf.layout`

A named `ViewConfig` owned by an account. **Published** — everyone on the
forecast lists and can apply it, only owner or admin may change it — or
**private**. Scope is the **version** by default, where people enter;
`version_id IS NULL` applies to every version of the source. `is_default` (one
per scope, published only) is restored on a first load, version-scoped beating
source-wide — the narrower answer is the one somebody chose for this forecast.
With no default: `group_by: ['pf_iter']` over the first value column.

Load order: this version's last-used unnamed config in `localStorage`
(`pf_layout_v{vid}`) if any — per-browser continuity, not something anyone names
or shares — else the default. The old named list (`pf_layouts_v{vid}`) is lifted
into `pf.layout` as private rows once and cleared.

**Permissions are the `pf.log` rule** and `can_edit` rides on every row so the
**Layout ▾** menu greys the rest. Applying is never restricted: the guarantee is
that a published layout cannot be *changed* under people, not that it cannot be
adapted — Save is withheld on others' layouts, Save as… forks. **No territory
clause**: layouts are display config and territory restricts rows.

`cleanLayout()` gates every stored config before `restore()`, which is
all-or-nothing: it drops unknown columns from the axes and `aggregates`
(including a multi-arg weight column) entry by entry rather than dropping the
layout, and strips `theme`, the dead ordering expressions and `pf_totals`.

**The dirty dot is a comparison, not a flag.** `restore()` itself fires
`perspective-config-update`, so a flag set there would light the moment a layout
was applied. `sameConfig()` compares against `activeConfigRef`, ignoring `table`
(per-load) and `theme`.

**Grand totals ride in the layout as `pf_totals: { row, col }`** — the Total
toggles. The engine has no "subtotals but no grand total" mode, so both are
hidden in the grid's DOM; `captureConfig()` adds the key and `cleanLayout()`
strips it. Without it: row shown, column hidden.

**What this replaced.** Named layouts lived only in `localStorage`, invisible to
others and gone on another machine. The one server-side layout was
`pf.source.default_layout`, an anonymous blob `PUT /sources/:id/default-layout`
let *any* account overwrite for *every* account — a published layout with no
owner. The route is gone; the column is migrated and unread.

### Axis depth

Both axes collapse by a **depth in `ViewConfig`** set through `restore()` —
`group_by_depth` (`applyDepth()`) and `split_by_depth` (`applySplitDepth()`). In
the config it survives every rebuild, rides into layouts via `viewer.save()`, and
needs nothing re-applied.

Each axis is a **stepper** from `DepthButtons.jsx` — `◂ level ▸`, the middle
opening the level list for a jump — three controls whatever the depth. It was one
button per level, growing with the pivot, the current one a blue among greys so
reading it meant scanning; before that a fixed `0 1 2 3` that named nothing and
offered levels a two-field `group_by` lacks. It shows an em dash until a layout
has been read back, and its middle is sized to the longest level so the toolbar
holds still.

**Depths are 1-based counts of levels to show**, on the stepper and in the
config, so the value crosses unchanged; `server.cpp` sets `set_depth(n - 1)` and
`t_traversal::set_depth` adds 1 back. The toolbar used to send `d + 1` and
subtract it on read-back — self-consistent, and one level off the grid
everywhere. The `+1` dodged depth 0, which reaches the engine as
`set_depth(-1)` on an unsigned `t_depth`, underflows and clamps to fully
expanded. `applyDepth` now floors at 1, so the **Total slot still shows the first
level, not a bare total** — a wart kept rather than traded for the underflow. A
real total needs `group_rollup_mode: 'total'`, unverified on this engine.

**This required patching the engine.** `ViewConfig::apply_update` applied ten
fields and neither depth, so a depth through `restore()` was accepted and
dropped — and `Session::update_view_config` returns early when nothing changed,
so no view was rebuilt. `ui/vendor/0001-apply-depth-fields-on-config-update.patch`.

**What this replaced:**

- Row depth was imperative — `view.set_depth()` on an object the viewer discards
  whenever it re-renders — so restoring it meant guessing when that happened: an
  `observerShim.js` patching `IntersectionObserver` and `ResizeObserver`, focus
  and visibility listeners, a retry loop for `getView()` throwing `No table set`,
  a flag for whether the viewer had "gone away". That produced three visible
  faults: the tree fully expanding on refocus, snapping on any reflow, and
  snapping when the settings sidebar opened.
- Column collapse restored a **truncated `split_by`**, so discarded levels were
  remembered separately (`splitFull`, persisted as `split_full`), our collapse
  had to be told apart from the user rearranging, and the selection was cleared
  on every collapse. `split_full` and `expand_depth` are still read from old
  layouts.

**Depth changes rebuild the view** — `restore()` makes the session tear down and
rebuild it, a full traversal, where a manual collapse mutates in place.
Noticeable on a large grain. Detecting a depth-only change and calling
`view.set_depth()` while still writing the config would fix it, at the risk of the
two drifting.

**Per-node expansion is not solved.** It is view state with no config form, and
the API is set-only (`expand(row_index)` / `collapse(row_index)`, no getter), so
it cannot be captured and replayed and is lost on every rebuild. The honest route
is a `ViewConfig` field of expanded row *paths* (indices shift as the tree
opens), applied in `server.cpp` beside the depths — needing node enumeration in
the C++ traversal, a proto field and the apply step. `expand_column` /
`collapse_column` share the limitation. And depth is whole-axis: Excel can
collapse one year while the next stays open.

### Auto-pause is off

`<perspective-viewer>` auto-pauses by default — an `IntersectionObserver` plus
`visibilitychange` — and pausing **deletes the view**, so returning to the tab
means `restore_and_render()`: a fresh traversal of the whole grain, a
multi-second chug per tab switch on a large one, losing per-node expansion.
`initViewer()` calls `viewer.setAutoPause(false)` after `load()`. Nothing updates
the table while hidden — every operation is driven from this page — so the pause
bought nothing.

### Toolbar

Two zones. Left, **view**: `Layout ▾` and the Rows and Columns steppers with
their Total chips — all `ViewConfig`, nothing reaching the server. Right,
**forecast**: Refresh data, Change log, Bridge, Operations — re-running the load,
opening something over the pivot, or writing.

The distance between them is the separation. Each was once an outlined box too,
the view one tinted too faintly to see and, in dark mode, reading as a sibling;
before that everything was one row of identical hairlines, making a
column-width control and the version's history look like peers. Three weights do
the separating: `.pf-section-label` names a group and is heavier than its
controls, `.pf-group-rule` divides groups within a zone, the gap divides zones.
Each axis is one joined control, `◂ | level | ▸`, then its Total chip.

**Blue is for actions, pressed grey for state.** Save and Post are blue; the
Total chips and an open Operations use `.pf-pressed`. Operations used to relabel
itself "Hide panel" in a blue outline, looking like a second Save.

**Fit widths is gone**, for three independent reasons: auto-size measures over
the rows *in the viewport*, so anything below the fold never counted;
`draw({invalid_columns})` passes an option `DrawOptions` lacks, and `draw()` is
throttled, so a click during a pending draw cleared widths without re-measuring;
and the `persistLayout` after it saved nothing, since `saveColumnSizes()` returns
only overrides, which `resetAutoSize()` had emptied — and this fork discards
`column_size_override` whenever `split_by` is set, which here is always.
Recoverable from git if ever worth fixing.

The operation panel docks **bottom**, **right** or **floats**, persisted locally;
the toolbar toggle shows the selection count while it is shut.

### Calculating indicator

**Calculating…** shows in the status bar while the pivot rebuilds. Perspective's
beaker renders only while its settings panel is open, vanishing exactly when
nothing else says to wait. A counter — rebuilds overlap — rises in `applyDepth`,
`applySplitDepth` and `applyLayout`, falls after `viewer.flush()`, and travels as
`pf-pivot-busy` on the window, like `pf-data-changed`.

**Not** driven by `perspective-config-update`, the obvious hook: on a depth
change over a large grain it arrives *after* the rebuild, seconds late, and the
flush after it resolves at once. The cost is that sorting or filtering from the
viewer's own chrome shows nothing, since the app is never told they began.

### Light and dark

`ui/src/theme.jsx` holds a context stored as `pf_dark`, defaulting to
`prefers-color-scheme`; the status-bar toggle sets `.dark` on `<html>`.
`index.css` defines properties under `:root` and `.dark`, with Tailwind overrides
written as `.dark .bg-white { … }` — no Tailwind dark config. Dark uses
Perspective's Pro Dark palette.

The pivot takes a **partial** `viewer.restore({ theme: 'Pro Dark' | 'Pro Light' })`
on load and on toggle. It used to be `setAttribute('theme', …)`, which does
nothing in 5.4.0 — the theme is a `ViewerConfig` field — so the pivot stayed light
whatever the toggle said. The other half is `cleanLayout()` dropping `theme`:
`viewer.save()` includes it, so every layout reasserted its author's theme. It is
the reader's choice, so a published layout cannot impose one.

---

## Access

### Authentication and sessions

Everything under `/api` but `/login`, `/logout`, `/me` sits behind
`requireAuth`; the React app mounts only with a session (`Gate` in `main.jsx`),
since its effects call the API at once, and `auth.jsx` wraps `fetch` so a 401
returns to login.

- **Accounts** — `pf.app_user`, scrypt hashes whose stored form carries its
  parameters, so they can be raised without invalidating rows. `./pf.sh add-user
  | passwd | list-users | disable-user | enable-user`; passwords go in on stdin
  through `node lib/auth.js hash`, never argv or psql.
- **Sessions** — `connect-pg-simple` in `pf.session`, so restarts keep people
  signed in and a session is revoked by deleting its row (`disable-user` does).
  Cookie `pf.sid`: httpOnly, SameSite=Lax, Secure unless `COOKIE_SECURE=false`,
  12h rolling.
- **Config** — the server exits without `SESSION_SECRET` rather than using a
  guessable default. `TRUST_PROXY` (default 1) keeps `req.ip` and secure cookies
  right behind the TLS proxy. CORS is enabled only by `CORS_ORIGIN`, by name —
  a wildcard origin plus a session cookie is CSRF by construction.
- **Login hardening** — ten failures per IP per fifteen minutes (in memory), one
  message for unknown/wrong/disabled, session id regenerated on success.

**Identity is server-side.** `pf_user`, `created_by`, `closed_by` come from
`sessionUser(req)`, never the body. The UI used to send a hardcoded
`pf_user: 'admin'`, which any client could set to anything.

### Setup and Baseline are admin-only to write

`requireAdmin` sat exported in `lib/auth.js` and applied to nothing, so every
non-admin could call every Setup and Baseline endpoint. It now guards source
writes (register, cols, Generate SQL, member refresh, deregister), version writes
(create, update, close, reopen, delete), loads (baseline, reference, segment edit,
clear) and every plan write. `is_default` is further admin-checked inside
`PUT /versions/:id`.

**Not gated, each for a reason.** No read, including the table browser — the
pages stay visible, and gating what a page needs to *render* turns it into a 403.
Not the forecast operations, which carry territory and ownership rules. Not
`PATCH /versions/:id/tags`, whose permission is ownership.

**The pages go read-only rather than hidden**: one `<fieldset disabled>` per page
instead of a flag threaded through every control — Baseline alone has dozens, and
the one missed would be the one that mattered — plus a `ReadOnlyNotice` banner.
Setup wraps a real box because the fieldset *is* its flex row; Baseline uses
`display: contents` so its layout is untouched. **None of this is the
permission**; it only stops offering buttons that would answer 403.

**A 403 used to blank the whole app.** Setup put responses straight into state
and `.map`ped them; an error body is `{error}`, and throwing in render unmounts
React. Its list fetches now coerce to arrays, so a bad response costs an empty
panel, not the session. The fragility predated the gating, which only made it
reachable.

### Territory scoping

An account sees and changes only its territory: `pf.app_user.territory` (jsonb
list), `is_admin` for the unrestricted, and `col_meta.is_territory` naming which
column of a source the values belong to — flagged, not coded, so another source
can be divided by something other than a sales rep.

**Fail closed.** No territory and not admin means no rows:
`buildTerritoryClause()` returns `FALSE`, not `TRUE`, for an empty list or an
unflagged source, so a forgotten account sees nothing rather than the book.

**Built from the session, never the request** — the difference from `scope`,
which the browser rightly sends as a filter the user chose. A permission cannot
come from the thing it restrains, so the predicate is ANDed on last, in
`sliceUnits()` for writes and per route for reads. Enforced at:

- `/data` — cursor *and* the `X-Row-Count` count
- `/agg` — `{{territory_clause}}`, before the GROUP BY
- `/versions/:id/table-info` — unscoped, it told an account its load was
  fetching the whole table
- every operation (`pf.log.env` records the territory in force)
- `/sources/:id/values/:col` — it reads the *source*, untouched by any scope
- recode's `set` — a scoped account cannot set the territory column. That is
  reassignment, not forecasting, and the row would vanish from the view that
  would have shown what happened.

**Ownership, not territory**, governs `DELETE /log/:logid`, `PATCH /log/:logid`
and `PATCH /versions/:id/tags`. The log PATCH looks like a private annotation and
is not: `label` and `bucket` name the pivot's columns for everyone, so unguarded
it let any account rename the company's segments, including on loads whose rows
it cannot see.

Known gaps, verified in code:

- **Completion now uses unscoped routes.** Recode and clone complete from
  `GET /versions/:id/values/:col` (the version's own table, cached in memory
  keyed on the latest logid), the member list from `/sources/:id/dim/:group`,
  sibling autofill from `/sources/:id/lookup`. None applies the territory clause.
- **Territory is read onto the session at login**, so a change reaches a
  signed-in account only when it signs in again. Re-reading it in `requireAuth`
  would also make disabling immediate.

`./pf.sh set-territory | set-admin | orphan-territory`; the last lists values in
the data that no account owns — work under one is invisible to all but admins,
which a typo causes easily and nothing in the app reveals.

---

## Bridge and change log

### Bridge

A waterfall of *how this forecast got from where it started to where it stands*,
one step per initiative tag. `BridgeView.jsx`; the arithmetic is in exported pure
`buildSteps()` / `layoutSteps()`, checkable against real data without a browser.

**Computed from the table already in the browser**, not `GET /versions/:id/bridge`,
so it reconciles with the pivot. The endpoint remains for API consumers, grouping
by tag over rows outside `exclude_iters`, with no notion of buckets.

**Scope** is the selection (the union of its slices, an operation's reach) or the
pivot's filter, following the live selection until a scope button is picked.
There is **no whole-version scope**: the filter is part of what the figures
mean, and dropping it moves one end of the walk and not the other — rows outside
a season filter sat wholly in the forecast loads with no counterpart in the plan,
inflating the endpoint without explanation. View filters are AND-only, so
selection scope builds a view per slice, each with the pivot's filter.

**Membership comes from `pf_bucket`, not `pf_iter`**: open orders load as
reference and are still forecast, and an iter-keyed bridge dropped them. The
forecast bucket is the version's `adjustment_bucket` — the value unbucketed
adjustments carry, so the two agree by construction. It used to be the literal
`'Forecast'`; once buckets carried their sort prefix nothing matched, every row
became comparison, and the basis cancelled itself to zero. With no match it
falls back to whichever bucket holds the adjustments. Selection scope does not
filter on `pf_bucket`/`pf_segment` even when a slice names one, since Compare to
needs rows from every bucket.

**Compare to** sets a bucket as basis — Plan when building a forecast, Prior Year
when building the plan — resting on

    Forecast − Basis = (Forecast loads − Basis) + adjustments

The opening step is the forecast's loads against the basis and the tagged
adjustments explain the rest: no residual, no plug, the bars summing to the
endpoint by construction. With no basis it shows the forecast's composition.

**Steps group by tag alone**, case-insensitively, showing the commonest spelling.
They grouped by label first once, to match `pf_segment`, but every adjustment
carries the same default label, so no tag ever showed — the label is the segment,
the tag the initiative. Untagged adjustments collapse to one "N untagged
adjustments" step, as in the ledger, rather than a wall of identical bars.

**Colour encodes polarity**: increase and decrease as two poles of a validated
blue/red diverging pair (green/red is the classic colour-vision failure), neutral
grey anchors. Everything not a bar follows the theme via CSS variables set
through `style`, where `var()` is reliable. Every bar is labelled, a table view
gives full precision, and the panel can be dragged.

Known issues:

- **Overlapping slices are deduplicated by `pf_id`**, which grain rows lack, so
  in grain mode a row matching two selected slices counts twice; it should key on
  `pf_gkey`.
- **Selection scope keeps only dimension and date keys**, dropping the `pf_iter` /
  `pf_note` / `pf_op` / `pf_user` / `pf_logid` keys operations honour, so it
  bridges a wider set than the operation would write; a total-cell `{}` slice is
  skipped outright.
- **No drill-down** from a step to its entries.

### Change log

A modal of the version's entries, newest first — time, operation, slice, tag,
note, rows and totals, author — with undo per row and tag and note edited inline
(Enter saves, Esc cancels). `PATCH /log/:logid` branches on whether a field was
sent, so `""` clears rather than meaning "leave alone" — except `label` and
`bucket`, renamed from the Baseline page and never cleared. A saved tag regroups
the ledger and bridge at once, so history can be reclassified. Multi-slice entries
list a line per slice, pulling out shared fields when only one dimension varies
(a dragged region); `{}` reads "Total within the pivot filter". An entry with
stored SQL opens its debug record — intent, environment, executed statement.

**It shows an entry's full impact**, not the reader's share: stamped totals are
company-wide, so an admin's version-wide scale reads the same everywhere —
deliberately, and labelled, rather than re-aggregated per territory.

**Everyone / Mine.** Everyone's entries are listed, with the reader's own marked
in place — a blue rule down the row and a blue `By` cell. With several active
accounts, finding your own meant reading `By` down the list. Marked *and*
filterable: the mark tells without hiding anybody, the filter is for wanting only
yours. It opens on Mine every time, unpersisted — the log is almost always opened
to check your own recent work, and a stale "Everyone" is the worse default.
`isMine()` is strict authorship, not `canUndo()`: an admin can undo everything,
and marking every row as theirs would say nothing. The fetch stays unfiltered,
since the control's counts need the whole set. Rows the reader cannot change are
greyed rather than offering a 403.

`?kind=adjustments` drops load entries, which own nearly every row, from the
recount join.

Known issue: the log does not show an entry's id, so there is no way to name one
when asking about it.

---

## Proposed: additional measures

Status: **proposed — not built.** Nothing below exists in code yet.

### The problem

The app forecasts exactly one dollar measure (`role = value`) and one quantity
(`role = units`). `col_meta` lets a second column be marked `value`, but only
the loads carry it (`generateSQL` in `lib/sql_generator.js`: "the adjustment
operations are single-measure … and use only the first of each"). So a second
measure is right on the baseline and wrong everywhere an adjustment has been
made:

- scale, recode and clone write adjustment rows with that column **null**
- a recode moves the sales and leaves the cost behind
- in grain mode `/agg` sums only the first value and the units column, so it
  is not in the pivot at all

Extended cost is the first case wanted — margin is sales minus cost — and the
same shape recurs for rebates, surcharges, freight and the like.

### The model: every extra measure is a rate on a driver

An extra measure is almost never forecast on its own. It is a rate applied to
something already being forecast:

| Measure            | Driver | Rate held by an ordinary edit     |
|--------------------|--------|-----------------------------------|
| Extended cost      | qty    | unit cost                         |
| Surcharge per unit | qty    | surcharge per unit                |
| Rebate             | sales  | rebate as a % of sales            |
| Freight (fixed)    | none   | — carried, not moved by scale     |

Declared once, in Setup: `col_meta.role = 'measure'`, plus a new
`col_meta.driver` of `units` | `value` | `none`. The existing `value` and
`units` roles are unchanged and remain the two the ledger edits.

### How each operation treats a measure

The rule is that **an edit never changes a measure's rate unless the rate is
what is being edited** — the same rule the scale already applies to price.

| Operation          | Measure driven by qty           | by sales                       | none                 |
|--------------------|---------------------------------|--------------------------------|----------------------|
| Load               | copied                          | copied                         | copied               |
| Scale              | moves by the qty factor         | moves by the dollar factor     | untouched            |
| Recode             | both legs, like the other cols  | both legs                      | both legs            |
| Clone              | × scale factor                  | × scale factor                 | × scale factor       |
| Undo               | removed with its rows           | removed with its rows          | removed              |

So a price-only edit moves no qty-driven cost (qty did not move) but does
move a sales-driven rebate (dollars did). The scale SQL already prorates
value and units by one factor per coordinate; each measure is prorated by its
driver's factor the same way, per row, so the rate holds per row and not only
in total.

Because scale moves a measure at the slice's **current** rate, a rate set
earlier is carried forward by every later edit, which is what makes Set rate
(below) stick.

### Setting a rate: its own operation, not the ledger

The ledger stays three boxes — value, qty, price — and the measures stay out of
it. The ledger is already the hardest part of the panel to read, and a rate is
a different question from a price-or-volume edit.

The case that needs it: a part used as a proxy for a new part not yet set up,
where the new part's cost is roughly known. The workflow is clone or recode the
proxy's volume onto the new part, then set the new part's cost.

    Set rate · measure ext_cost · per qty · new rate 0.31

- Writes adjustment rows that change **only that measure**: the slice's cost
  goes to rate × its qty; sales and qty do not move. The same shape as a price
  edit, on a different measure.
- A sales-driven measure takes its rate as a % of sales.
- A `none` measure takes a plain target or change amount.
- One `pf.log` entry, `operation = 'rate'`, `params` recording the measure,
  the rate and the resolved change; undoable, listed in the change log, and a
  line in the bridge like any other adjustment.
- Measured against the same basis choice as scale (adjustable rows or the
  selected total), with the same territory and scope rules.

### In the pivot

Each measure is summed like sales and qty — `/agg` and the `/data` cursor carry
every `measure` column. Rates and margins are not stored: they are Perspective
expression columns computed from the sums (unit cost = cost / qty, rebate % =
rebate / sales, margin = sales − cost, margin % = margin / sales), so they are
right at every level of the tree.

### What has to change

- `col_meta`: `role = 'measure'`, new `driver` column; Setup's editor offers both.
- `lib/sql_generator.js`: loads and `/agg` carry every measure; scale prorates
  each by its driver's factor; recode and clone carry them; a new `rate`
  template.
- `routes/operations.js`: a `rate` route; scale passes the factors through.
- `OperationPanel.jsx`: a Set rate form beside Scale / Recode / Clone.
- Existing versions: the forecast table needs the measure columns, so a version
  created before a measure was added needs them added (or recreating) — the
  same col_meta drift noted under §Open questions.

### Open questions

- Should a measure be allowed more than one driver (a surcharge that is part
  per-unit, part percentage)? Proposed: no — split it into two measures.
- Does Set rate on a slice with no qty (a qty-driven measure) refuse, as the
  price hold does on a slice with no value? Proposed: yes.

---

## Open questions

- **Replay** — re-running a version's adjustments against a restated baseline.
  Not built: no route takes `replay` and nothing returns 501. The hard part is
  the one cloning a definition declines: a scale's target was sized against the
  rows that were there, so a replay must choose between the *figure* and the
  *intent* (`params` holds both inputs and `resolved`). `env`'s template
  fingerprint exists partly so a replay can tell an entry ran under another
  template.
- **Approval workflow** — a submission an admin approves before others see it.
  Deferred.
- **Export** — CSV, or a push to a reporting table. The bridge's table view is a
  partial stand-in; there is no download. Stored names (§Naming) mean a reporting
  query needs only a join to `pf.log`.
- **Cross-version comparison** — side by side, or a bridge between versions; the
  isolated tables make it a UNION. Compare to answers the within-version form
  (against a plan bucket loaded into the same version); across versions is open.
- **Bridge drill-down** — a step listing its entries, or selecting them back in
  the pivot.
- **Multi-connection support** — one database via `.env` today. The vision: a
  `pf.connection` table (host, port, dbname, user, password as an env-var
  reference), `connection_id` on `pf.source`, per-connection pools; `pf` stays on
  a home connection and sources live anywhere, with a Connections panel in Setup.
  It changes the source schema, so it wants a clean install.
- **Col_meta / version schema drift** — roles changed after a forecast table
  exists put the generated SQL and the table's columns out of sync, unnoticed.
  Setup should compare col_meta with each table's columns, warn, and offer to
  rebuild (recreate the table, keep the version and log). Today: delete and
  recreate the version. Additional measures would hit this on every version.
- **Separating `is_key`'s meanings** — §Data model → `pf.col_meta`.
- **A per-source default layout** beyond the `pf_iter` fallback, built from
  col_meta when nothing is published.
- **Dynamic grain** — §Display-grain pre-aggregation.
- **Per-node expansion that survives a rebuild** — §The pivot → Axis depth.
- **Tests** — SQL generation is token substitution against append-only tables,
  exercised only by hand.