Put the fallback display names on the version

adjustment_segment, adjustment_bucket and unlabeled_load are columns on
pf.version now, edited under "Fallback names" on the Baseline page, with the
constants in sql_generator left as the built-in for a version that sets none.

Read through a join, not substituted at generation: pf.sql is keyed on
(source_id, operation) and shared by every version of a source, so a baked-in
value could not vary by version and regenerating for one would change the
others. The join costs three more GROUP BY columns on /agg, all functionally
dependent on a version id that is already fixed for the whole query.

The built-ins stay a convention guess -- ADJUSTMENT_BUCKET's "04 - " suits one
numbering -- which is now a default to override rather than the only answer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Paul Trowbridge 2026-09-17 23:47:17 -04:00
parent aa03e74b5e
commit 0cabe9bcd2
6 changed files with 113 additions and 38 deletions

View File

@ -158,28 +158,26 @@ the `DISPLAY DEFAULTS` block at the top of `lib/sql_generator.js`, exported so
the `/data` cursor and the operation routes' incremental row stamps use the same the `/data` cursor and the operation routes' incremental row stamps use the same
values the generated `/agg` does. values the generated `/agg` does.
| constant | value | applies to | | constant | `pf.version` column | built-in | applies to |
|---|---|---| |---|---|---|---|
| `ADJUSTMENT_SEGMENT` | `99 - Adjustments` | `pf_segment` for a scale/recode/clone with no `label` | | `ADJUSTMENT_SEGMENT` | `adjustment_segment` | `99 - Adjustments` | `pf_segment` for a scale/recode/clone with no `label` |
| `ADJUSTMENT_BUCKET` | `04 - Forecast` | `pf_bucket` for a scale/recode/clone with no `bucket` | | `ADJUSTMENT_BUCKET` | `adjustment_bucket` | `04 - Forecast` | `pf_bucket` for a scale/recode/clone with no `bucket` |
| `UNLABELED_LOAD` | `Unlabeled` | `pf_segment` and `pf_bucket` for a load with no `label`, `tag` or `note` | | `UNLABELED_LOAD` | `unlabeled_load` | `Unlabeled` | `pf_segment` and `pf_bucket` for a load with no `label`, `tag` or `note` |
Anything typed on the log row overrides its fallback, so none of these appears Each is set per scenario on the Baseline page, under **Fallback names**; blank
once a segment is named. falls back to the built-in. Anything typed on the log row overrides both, so
none of these appears once a segment is named.
**They should be per-version and are not.** `ADJUSTMENT_BUCKET` in particular **Why the join rather than a token.** `pf.sql` is keyed on
carries a `04 - ` that only suits one naming convention: unprefixed it read `(source_id, operation)` — one template shared by every version of a source — so
`Forecast` while the loads read `04 - Forecast`, which split the column in two a value baked in at Generate SQL time could not vary by version, and
and sat the adjustments apart from the rows they adjust. Changing it changes regenerating for one version would silently change the others. The names are
every version on every source. therefore read through `VERSION_JOIN` at query time, which also means changing
one takes effect on the next load with nothing regenerated.
The fix is two columns on `pf.version` — it already holds per-scenario config in The built-ins are still a convention guess: `ADJUSTMENT_BUCKET`'s `04 - ` only
`exclude_iters` — read through a join in `/data` and `/agg` rather than suits one numbering. A version that numbers its buckets differently sets its
substituted at generation time. That last part is the constraint worth own rather than inheriting that.
remembering: `pf.sql` is keyed on `(source_id, operation)`, one template shared
by every version of a source, so a value baked in at Generate SQL time cannot
vary by version, and regenerating for one version would silently change the
others.
**What this replaced.** The prefix used to be computed client-side, as **What this replaced.** The prefix used to be computed client-side, as
Perspective expression columns (`pf_bucket_ord`, `pf_segment_ord`) built from Perspective expression columns (`pf_bucket_ord`, `pf_segment_ord`) built from

View File

@ -36,11 +36,12 @@
// that is not in pf.log, it came from here. CLAUDE.md has the same list under // that is not in pf.log, it came from here. CLAUDE.md has the same list under
// "Hardcoded display names". // "Hardcoded display names".
// //
// They belong on pf.version, so a scenario can name its own. Until then they // They live on pf.version -- adjustment_segment, adjustment_bucket,
// are global, and changing one changes it for every version on every source -- // unlabeled_load -- and the constants below are only the fallback for a version
// which is why ADJUSTMENT_BUCKET carries a prefix that only suits the current // that has not set one. Read through a join at query time rather than
// naming convention. Read at query time, not baked in: pf.sql templates are // substituted at generation: pf.sql templates are keyed on (source_id,
// per source, so a per-version value cannot be substituted at generation. // operation) and shared by every version of a source, so a value baked in could
// not vary by version and regenerating for one would change the others.
// //
// Exported because /data builds its own statement in routes/operations.js while // Exported because /data builds its own statement in routes/operations.js while
// /agg is generated here, and the two have to agree. // /agg is generated here, and the two have to agree.
@ -64,11 +65,12 @@ const ADJUSTMENT_BUCKET = '04 - Forecast';
// since '(' is 0x28 and digits begin at 0x30. // since '(' is 0x28 and digits begin at 0x30.
const UNLABELED_LOAD = 'Unlabeled'; const UNLABELED_LOAD = 'Unlabeled';
const LOAD_SEGMENT = `COALESCE(NULLIF(l.label, ''), NULLIF(l.tag, ''), NULLIF(l.note, ''), '${UNLABELED_LOAD}')`; const LOAD_SEGMENT = `COALESCE(NULLIF(l.label, ''), NULLIF(l.tag, ''), NULLIF(l.note, ''),
NULLIF(v.unlabeled_load, ''), '${UNLABELED_LOAD}')`;
const SEGMENT_EXPR = `CASE WHEN l.operation IN ('baseline','reference') const SEGMENT_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
THEN ${LOAD_SEGMENT} THEN ${LOAD_SEGMENT}
ELSE COALESCE(NULLIF(l.label, ''), '${ADJUSTMENT_SEGMENT}') ELSE COALESCE(NULLIF(l.label, ''), NULLIF(v.adjustment_segment, ''), '${ADJUSTMENT_SEGMENT}')
END`; END`;
// What the row counts towards. A load falls back to its own name until it is // What the row counts towards. A load falls back to its own name until it is
@ -78,7 +80,7 @@ const SEGMENT_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
const BUCKET_EXPR = `COALESCE(NULLIF(l.bucket, ''), const BUCKET_EXPR = `COALESCE(NULLIF(l.bucket, ''),
CASE WHEN l.operation IN ('baseline','reference') CASE WHEN l.operation IN ('baseline','reference')
THEN ${LOAD_SEGMENT} THEN ${LOAD_SEGMENT}
ELSE '${ADJUSTMENT_BUCKET}' ELSE COALESCE(NULLIF(v.adjustment_bucket, ''), '${ADJUSTMENT_BUCKET}')
END)`; END)`;
const NOTE_EXPR = `CASE WHEN l.operation IN ('baseline','reference') const NOTE_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
@ -89,7 +91,16 @@ const NOTE_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
// Every pf.log column the two expressions above read, for /agg's GROUP BY: they // Every pf.log column the two expressions above read, for /agg's GROUP BY: they
// are functionally dependent on pf_logid, which is in the grain, but Postgres // are functionally dependent on pf_logid, which is in the grain, but Postgres
// will not infer that. // will not infer that.
const LABEL_GROUP_COLS = ['l.operation', 'l.label', 'l.tag', 'l.note', 'l.bucket']; const LABEL_GROUP_COLS = ['l.operation', 'l.label', 'l.tag', 'l.note', 'l.bucket',
'v.adjustment_segment', 'v.adjustment_bucket', 'v.unlabeled_load'];
// The version carries the fallback names, so every statement that reads the
// expressions above needs it in scope as `v`. LEFT, not inner: a forecast row
// whose log entry somehow has no version should still come back, named by the
// constants.
const VERSION_JOIN = `
LEFT JOIN pf.version v
ON v.id = l.version_id`;
// wrap a column name in double quotes for safe use in SQL // wrap a column name in double quotes for safe use in SQL
function q(name) { return `"${name}"`; } function q(name) { return `"${name}"`; }
@ -280,7 +291,7 @@ SELECT
,l.operation AS pf_op ,l.operation AS pf_op
FROM {{fc_table}} t FROM {{fc_table}} t
LEFT JOIN pf.log l LEFT JOIN pf.log l
ON l.id = t.pf_logid ON l.id = t.pf_logid${VERSION_JOIN}
GROUP BY GROUP BY
${grain.groupCols('t.').join('\n ,')} ${grain.groupCols('t.').join('\n ,')}
,${LABEL_GROUP_COLS.join('\n ,')}`.trim(); ,${LABEL_GROUP_COLS.join('\n ,')}`.trim();
@ -617,5 +628,5 @@ function esc(val) {
} }
module.exports = { generateSQL, grainOf, module.exports = { generateSQL, grainOf,
SEGMENT_EXPR, BUCKET_EXPR, NOTE_EXPR, LABEL_GROUP_COLS, SEGMENT_EXPR, BUCKET_EXPR, NOTE_EXPR, LABEL_GROUP_COLS, VERSION_JOIN,
ADJUSTMENT_SEGMENT, ADJUSTMENT_BUCKET, UNLABELED_LOAD, dateGroupsOf, dimPeriodMapOf, dimPeriodJoins, applyTokens, buildWhere, buildWhereAny, buildExcludeClause, buildExcludePredicate, buildSetClause, buildFilterClause, esc }; ADJUSTMENT_SEGMENT, ADJUSTMENT_BUCKET, UNLABELED_LOAD, dateGroupsOf, dimPeriodMapOf, dimPeriodJoins, applyTokens, buildWhere, buildWhereAny, buildExcludeClause, buildExcludePredicate, buildSetClause, buildFilterClause, esc };

View File

@ -1,7 +1,8 @@
const express = require('express'); const express = require('express');
const { tableFromArrays, tableToIPC } = require('apache-arrow'); const { tableFromArrays, tableToIPC } = require('apache-arrow');
const { applyTokens, buildWhere, buildWhereAny, buildExcludeClause, buildExcludePredicate, buildSetClause, dateGroupsOf, dimPeriodMapOf, esc, const { applyTokens, buildWhere, buildWhereAny, buildExcludeClause, buildExcludePredicate, buildSetClause, dateGroupsOf, dimPeriodMapOf, esc,
SEGMENT_EXPR, BUCKET_EXPR, NOTE_EXPR, ADJUSTMENT_SEGMENT, ADJUSTMENT_BUCKET } = require('../lib/sql_generator'); SEGMENT_EXPR, BUCKET_EXPR, NOTE_EXPR, VERSION_JOIN,
ADJUSTMENT_SEGMENT, ADJUSTMENT_BUCKET } = require('../lib/sql_generator');
const { sessionUser } = require('../lib/auth'); const { sessionUser } = require('../lib/auth');
const { fcTable } = require('../lib/utils'); const { fcTable } = require('../lib/utils');
@ -340,7 +341,7 @@ module.exports = function(pool) {
,${NOTE_EXPR} AS pf_note ,${NOTE_EXPR} AS pf_note
FROM ${tbl} t FROM ${tbl} t
LEFT JOIN pf.log l LEFT JOIN pf.log l
ON l.id = t.pf_logid ON l.id = t.pf_logid${VERSION_JOIN}
`); `);
// Accumulate into column arrays (not row objects) to avoid allocating one JS // Accumulate into column arrays (not row objects) to avoid allocating one JS

View File

@ -307,26 +307,39 @@ ${colDefs},
} }
}); });
// update version name, description, or exclude_iters // update version name, description, exclude_iters, or the fallback display
// names.
// //
// bucket_order is deliberately not settable: the bucket column order is the // bucket_order is deliberately not settable: the bucket column order is the
// text in pf.log.bucket now, so a stored order would be a second answer to // text in pf.log.bucket now, so a stored order would be a second answer to
// the same question, and a silent one -- nothing reads it. // the same question, and a silent one -- nothing reads it.
//
// The three name columns are flag-and-value pairs rather than COALESCE:
// clearing one back to the built-in means writing null, which COALESCE on
// the value alone cannot tell from "not mentioned".
router.put('/versions/:id', async (req, res) => { router.put('/versions/:id', async (req, res) => {
const { name, description, exclude_iters } = req.body; const { name, description, exclude_iters,
adjustment_segment, adjustment_bucket, unlabeled_load } = req.body;
const set = (v) => (v === undefined ? null : (String(v).trim() || null));
try { try {
const result = await pool.query(` const result = await pool.query(`
UPDATE pf.version SET UPDATE pf.version SET
name = COALESCE($2, name), name = COALESCE($2, name),
description = COALESCE($3, description), description = COALESCE($3, description),
exclude_iters = COALESCE($4, exclude_iters) exclude_iters = COALESCE($4, exclude_iters),
adjustment_segment = CASE WHEN $5::bool THEN $6::text ELSE adjustment_segment END,
adjustment_bucket = CASE WHEN $7::bool THEN $8::text ELSE adjustment_bucket END,
unlabeled_load = CASE WHEN $9::bool THEN $10::text ELSE unlabeled_load END
WHERE id = $1 WHERE id = $1
RETURNING * RETURNING *
`, [ `, [
req.params.id, req.params.id,
name || null, name || null,
description || null, description || null,
exclude_iters ? JSON.stringify(exclude_iters) : null exclude_iters ? JSON.stringify(exclude_iters) : null,
adjustment_segment !== undefined, set(adjustment_segment),
adjustment_bucket !== undefined, set(adjustment_bucket),
unlabeled_load !== undefined, set(unlabeled_load)
]); ]);
if (result.rows.length === 0) { if (result.rows.length === 0) {
return res.status(404).json({ error: 'Version not found' }); return res.status(404).json({ error: 'Version not found' });

View File

@ -113,6 +113,15 @@ ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS seq integer;
ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS bucket text; ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS bucket text;
CREATE INDEX IF NOT EXISTS log_bucket_idx ON pf.log (bucket) WHERE bucket IS NOT NULL; CREATE INDEX IF NOT EXISTS log_bucket_idx ON pf.log (bucket) WHERE bucket IS NOT NULL;
-- The names a row falls back to when nobody has named it, per scenario. Null
-- means "use the built-in", which is the DISPLAY DEFAULTS block in
-- lib/sql_generator.js. Read through a join at query time, not baked into
-- pf.sql: those templates are keyed on (source_id, operation) and shared by
-- every version of a source.
ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS adjustment_segment text;
ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS adjustment_bucket text;
ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS unlabeled_load text;
-- Master data for a dim_group: one row per key value, with its sibling columns. -- Master data for a dim_group: one row per key value, with its sibling columns.
-- --
-- The source is transactional and often a view over all history, so deriving a -- The source is transactional and often a view over all history, so deriving a

View File

@ -347,6 +347,24 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
setTimeout(() => setMsg(null), 3000) setTimeout(() => setMsg(null), 3000)
} }
// The names a row falls back to when nobody has named it. Blank means "use the
// built-in", so these save on blur like the segment fields and an empty box is
// a meaningful value rather than a missing one.
async function saveVersionName(field, value) {
const next = value.trim()
if (next === (selectedVersion?.[field] || '')) return
try {
const res = await fetch(`/api/versions/${versionId}`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ [field]: next }),
})
if (!res.ok) { const d = await res.json(); flash(d.error, 'error'); return }
await refreshVersions(sourceId)
flash('Saved — reload the Forecast view to see it')
} catch (err) { flash(err.message, 'error') }
}
const selectedVersion = versions.find(v => String(v.id) === versionId) const selectedVersion = versions.find(v => String(v.id) === versionId)
return ( return (
@ -379,6 +397,31 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
)} )}
</div> </div>
{/* Fallback names. Not part of any segment -- they are what the pivot shows
for rows nobody has named, so they belong to the version rather than to
a log entry. Blank falls back to the built-in in sql_generator's
DISPLAY DEFAULTS block. */}
{versionId && (
<div className="bg-white border border-gray-200 rounded p-3 flex items-end gap-3 flex-wrap">
<span className="text-xs text-gray-500 uppercase tracking-wide w-full">Fallback names</span>
{[
['adjustment_segment', 'Adjustment segment', '99 - Adjustments'],
['adjustment_bucket', 'Adjustment bucket', '04 - Forecast'],
['unlabeled_load', 'Unlabeled load', 'Unlabeled'],
].map(([field, label, builtin]) => (
<div key={field} className="flex flex-col gap-1">
<label className="text-xs text-gray-500">{label}</label>
<input
key={`${field}-${versionId}-${selectedVersion?.[field] || ''}`}
defaultValue={selectedVersion?.[field] || ''}
onBlur={e => saveVersionName(field, e.target.value)}
placeholder={builtin}
className="border border-gray-200 rounded px-2 py-1 text-sm w-48" />
</div>
))}
</div>
)}
{showNewVersion && ( {showNewVersion && (
<div className="bg-white border border-gray-200 rounded p-3 flex flex-col gap-3"> <div className="bg-white border border-gray-200 rounded p-3 flex flex-col gap-3">
<div className="flex items-end gap-3"> <div className="flex items-end gap-3">