Put the fallback display names on the version
adjustment_segment, adjustment_bucket and unlabeled_load are columns on pf.version now, edited under "Fallback names" on the Baseline page, with the constants in sql_generator left as the built-in for a version that sets none. Read through a join, not substituted at generation: pf.sql is keyed on (source_id, operation) and shared by every version of a source, so a baked-in value could not vary by version and regenerating for one would change the others. The join costs three more GROUP BY columns on /agg, all functionally dependent on a version id that is already fixed for the whole query. The built-ins stay a convention guess -- ADJUSTMENT_BUCKET's "04 - " suits one numbering -- which is now a default to override rather than the only answer. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
aa03e74b5e
commit
0cabe9bcd2
36
CLAUDE.md
36
CLAUDE.md
@ -158,28 +158,26 @@ the `DISPLAY DEFAULTS` block at the top of `lib/sql_generator.js`, exported so
|
|||||||
the `/data` cursor and the operation routes' incremental row stamps use the same
|
the `/data` cursor and the operation routes' incremental row stamps use the same
|
||||||
values the generated `/agg` does.
|
values the generated `/agg` does.
|
||||||
|
|
||||||
| constant | value | applies to |
|
| constant | `pf.version` column | built-in | applies to |
|
||||||
|---|---|---|
|
|---|---|---|---|
|
||||||
| `ADJUSTMENT_SEGMENT` | `99 - Adjustments` | `pf_segment` for a scale/recode/clone with no `label` |
|
| `ADJUSTMENT_SEGMENT` | `adjustment_segment` | `99 - Adjustments` | `pf_segment` for a scale/recode/clone with no `label` |
|
||||||
| `ADJUSTMENT_BUCKET` | `04 - Forecast` | `pf_bucket` for a scale/recode/clone with no `bucket` |
|
| `ADJUSTMENT_BUCKET` | `adjustment_bucket` | `04 - Forecast` | `pf_bucket` for a scale/recode/clone with no `bucket` |
|
||||||
| `UNLABELED_LOAD` | `Unlabeled` | `pf_segment` and `pf_bucket` for a load with no `label`, `tag` or `note` |
|
| `UNLABELED_LOAD` | `unlabeled_load` | `Unlabeled` | `pf_segment` and `pf_bucket` for a load with no `label`, `tag` or `note` |
|
||||||
|
|
||||||
Anything typed on the log row overrides its fallback, so none of these appears
|
Each is set per scenario on the Baseline page, under **Fallback names**; blank
|
||||||
once a segment is named.
|
falls back to the built-in. Anything typed on the log row overrides both, so
|
||||||
|
none of these appears once a segment is named.
|
||||||
|
|
||||||
**They should be per-version and are not.** `ADJUSTMENT_BUCKET` in particular
|
**Why the join rather than a token.** `pf.sql` is keyed on
|
||||||
carries a `04 - ` that only suits one naming convention: unprefixed it read
|
`(source_id, operation)` — one template shared by every version of a source — so
|
||||||
`Forecast` while the loads read `04 - Forecast`, which split the column in two
|
a value baked in at Generate SQL time could not vary by version, and
|
||||||
and sat the adjustments apart from the rows they adjust. Changing it changes
|
regenerating for one version would silently change the others. The names are
|
||||||
every version on every source.
|
therefore read through `VERSION_JOIN` at query time, which also means changing
|
||||||
|
one takes effect on the next load with nothing regenerated.
|
||||||
|
|
||||||
The fix is two columns on `pf.version` — it already holds per-scenario config in
|
The built-ins are still a convention guess: `ADJUSTMENT_BUCKET`'s `04 - ` only
|
||||||
`exclude_iters` — read through a join in `/data` and `/agg` rather than
|
suits one numbering. A version that numbers its buckets differently sets its
|
||||||
substituted at generation time. That last part is the constraint worth
|
own rather than inheriting that.
|
||||||
remembering: `pf.sql` is keyed on `(source_id, operation)`, one template shared
|
|
||||||
by every version of a source, so a value baked in at Generate SQL time cannot
|
|
||||||
vary by version, and regenerating for one version would silently change the
|
|
||||||
others.
|
|
||||||
|
|
||||||
**What this replaced.** The prefix used to be computed client-side, as
|
**What this replaced.** The prefix used to be computed client-side, as
|
||||||
Perspective expression columns (`pf_bucket_ord`, `pf_segment_ord`) built from
|
Perspective expression columns (`pf_bucket_ord`, `pf_segment_ord`) built from
|
||||||
|
|||||||
@ -36,11 +36,12 @@
|
|||||||
// that is not in pf.log, it came from here. CLAUDE.md has the same list under
|
// that is not in pf.log, it came from here. CLAUDE.md has the same list under
|
||||||
// "Hardcoded display names".
|
// "Hardcoded display names".
|
||||||
//
|
//
|
||||||
// They belong on pf.version, so a scenario can name its own. Until then they
|
// They live on pf.version -- adjustment_segment, adjustment_bucket,
|
||||||
// are global, and changing one changes it for every version on every source --
|
// unlabeled_load -- and the constants below are only the fallback for a version
|
||||||
// which is why ADJUSTMENT_BUCKET carries a prefix that only suits the current
|
// that has not set one. Read through a join at query time rather than
|
||||||
// naming convention. Read at query time, not baked in: pf.sql templates are
|
// substituted at generation: pf.sql templates are keyed on (source_id,
|
||||||
// per source, so a per-version value cannot be substituted at generation.
|
// operation) and shared by every version of a source, so a value baked in could
|
||||||
|
// not vary by version and regenerating for one would change the others.
|
||||||
//
|
//
|
||||||
// Exported because /data builds its own statement in routes/operations.js while
|
// Exported because /data builds its own statement in routes/operations.js while
|
||||||
// /agg is generated here, and the two have to agree.
|
// /agg is generated here, and the two have to agree.
|
||||||
@ -64,11 +65,12 @@ const ADJUSTMENT_BUCKET = '04 - Forecast';
|
|||||||
// since '(' is 0x28 and digits begin at 0x30.
|
// since '(' is 0x28 and digits begin at 0x30.
|
||||||
const UNLABELED_LOAD = 'Unlabeled';
|
const UNLABELED_LOAD = 'Unlabeled';
|
||||||
|
|
||||||
const LOAD_SEGMENT = `COALESCE(NULLIF(l.label, ''), NULLIF(l.tag, ''), NULLIF(l.note, ''), '${UNLABELED_LOAD}')`;
|
const LOAD_SEGMENT = `COALESCE(NULLIF(l.label, ''), NULLIF(l.tag, ''), NULLIF(l.note, ''),
|
||||||
|
NULLIF(v.unlabeled_load, ''), '${UNLABELED_LOAD}')`;
|
||||||
|
|
||||||
const SEGMENT_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
|
const SEGMENT_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
|
||||||
THEN ${LOAD_SEGMENT}
|
THEN ${LOAD_SEGMENT}
|
||||||
ELSE COALESCE(NULLIF(l.label, ''), '${ADJUSTMENT_SEGMENT}')
|
ELSE COALESCE(NULLIF(l.label, ''), NULLIF(v.adjustment_segment, ''), '${ADJUSTMENT_SEGMENT}')
|
||||||
END`;
|
END`;
|
||||||
|
|
||||||
// What the row counts towards. A load falls back to its own name until it is
|
// What the row counts towards. A load falls back to its own name until it is
|
||||||
@ -78,7 +80,7 @@ const SEGMENT_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
|
|||||||
const BUCKET_EXPR = `COALESCE(NULLIF(l.bucket, ''),
|
const BUCKET_EXPR = `COALESCE(NULLIF(l.bucket, ''),
|
||||||
CASE WHEN l.operation IN ('baseline','reference')
|
CASE WHEN l.operation IN ('baseline','reference')
|
||||||
THEN ${LOAD_SEGMENT}
|
THEN ${LOAD_SEGMENT}
|
||||||
ELSE '${ADJUSTMENT_BUCKET}'
|
ELSE COALESCE(NULLIF(v.adjustment_bucket, ''), '${ADJUSTMENT_BUCKET}')
|
||||||
END)`;
|
END)`;
|
||||||
|
|
||||||
const NOTE_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
|
const NOTE_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
|
||||||
@ -89,7 +91,16 @@ const NOTE_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
|
|||||||
// Every pf.log column the two expressions above read, for /agg's GROUP BY: they
|
// Every pf.log column the two expressions above read, for /agg's GROUP BY: they
|
||||||
// are functionally dependent on pf_logid, which is in the grain, but Postgres
|
// are functionally dependent on pf_logid, which is in the grain, but Postgres
|
||||||
// will not infer that.
|
// will not infer that.
|
||||||
const LABEL_GROUP_COLS = ['l.operation', 'l.label', 'l.tag', 'l.note', 'l.bucket'];
|
const LABEL_GROUP_COLS = ['l.operation', 'l.label', 'l.tag', 'l.note', 'l.bucket',
|
||||||
|
'v.adjustment_segment', 'v.adjustment_bucket', 'v.unlabeled_load'];
|
||||||
|
|
||||||
|
// The version carries the fallback names, so every statement that reads the
|
||||||
|
// expressions above needs it in scope as `v`. LEFT, not inner: a forecast row
|
||||||
|
// whose log entry somehow has no version should still come back, named by the
|
||||||
|
// constants.
|
||||||
|
const VERSION_JOIN = `
|
||||||
|
LEFT JOIN pf.version v
|
||||||
|
ON v.id = l.version_id`;
|
||||||
|
|
||||||
// wrap a column name in double quotes for safe use in SQL
|
// wrap a column name in double quotes for safe use in SQL
|
||||||
function q(name) { return `"${name}"`; }
|
function q(name) { return `"${name}"`; }
|
||||||
@ -280,7 +291,7 @@ SELECT
|
|||||||
,l.operation AS pf_op
|
,l.operation AS pf_op
|
||||||
FROM {{fc_table}} t
|
FROM {{fc_table}} t
|
||||||
LEFT JOIN pf.log l
|
LEFT JOIN pf.log l
|
||||||
ON l.id = t.pf_logid
|
ON l.id = t.pf_logid${VERSION_JOIN}
|
||||||
GROUP BY
|
GROUP BY
|
||||||
${grain.groupCols('t.').join('\n ,')}
|
${grain.groupCols('t.').join('\n ,')}
|
||||||
,${LABEL_GROUP_COLS.join('\n ,')}`.trim();
|
,${LABEL_GROUP_COLS.join('\n ,')}`.trim();
|
||||||
@ -617,5 +628,5 @@ function esc(val) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
module.exports = { generateSQL, grainOf,
|
module.exports = { generateSQL, grainOf,
|
||||||
SEGMENT_EXPR, BUCKET_EXPR, NOTE_EXPR, LABEL_GROUP_COLS,
|
SEGMENT_EXPR, BUCKET_EXPR, NOTE_EXPR, LABEL_GROUP_COLS, VERSION_JOIN,
|
||||||
ADJUSTMENT_SEGMENT, ADJUSTMENT_BUCKET, UNLABELED_LOAD, dateGroupsOf, dimPeriodMapOf, dimPeriodJoins, applyTokens, buildWhere, buildWhereAny, buildExcludeClause, buildExcludePredicate, buildSetClause, buildFilterClause, esc };
|
ADJUSTMENT_SEGMENT, ADJUSTMENT_BUCKET, UNLABELED_LOAD, dateGroupsOf, dimPeriodMapOf, dimPeriodJoins, applyTokens, buildWhere, buildWhereAny, buildExcludeClause, buildExcludePredicate, buildSetClause, buildFilterClause, esc };
|
||||||
|
|||||||
@ -1,7 +1,8 @@
|
|||||||
const express = require('express');
|
const express = require('express');
|
||||||
const { tableFromArrays, tableToIPC } = require('apache-arrow');
|
const { tableFromArrays, tableToIPC } = require('apache-arrow');
|
||||||
const { applyTokens, buildWhere, buildWhereAny, buildExcludeClause, buildExcludePredicate, buildSetClause, dateGroupsOf, dimPeriodMapOf, esc,
|
const { applyTokens, buildWhere, buildWhereAny, buildExcludeClause, buildExcludePredicate, buildSetClause, dateGroupsOf, dimPeriodMapOf, esc,
|
||||||
SEGMENT_EXPR, BUCKET_EXPR, NOTE_EXPR, ADJUSTMENT_SEGMENT, ADJUSTMENT_BUCKET } = require('../lib/sql_generator');
|
SEGMENT_EXPR, BUCKET_EXPR, NOTE_EXPR, VERSION_JOIN,
|
||||||
|
ADJUSTMENT_SEGMENT, ADJUSTMENT_BUCKET } = require('../lib/sql_generator');
|
||||||
const { sessionUser } = require('../lib/auth');
|
const { sessionUser } = require('../lib/auth');
|
||||||
const { fcTable } = require('../lib/utils');
|
const { fcTable } = require('../lib/utils');
|
||||||
|
|
||||||
@ -340,7 +341,7 @@ module.exports = function(pool) {
|
|||||||
,${NOTE_EXPR} AS pf_note
|
,${NOTE_EXPR} AS pf_note
|
||||||
FROM ${tbl} t
|
FROM ${tbl} t
|
||||||
LEFT JOIN pf.log l
|
LEFT JOIN pf.log l
|
||||||
ON l.id = t.pf_logid
|
ON l.id = t.pf_logid${VERSION_JOIN}
|
||||||
`);
|
`);
|
||||||
|
|
||||||
// Accumulate into column arrays (not row objects) to avoid allocating one JS
|
// Accumulate into column arrays (not row objects) to avoid allocating one JS
|
||||||
|
|||||||
@ -307,26 +307,39 @@ ${colDefs},
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// update version name, description, or exclude_iters
|
// update version name, description, exclude_iters, or the fallback display
|
||||||
|
// names.
|
||||||
//
|
//
|
||||||
// bucket_order is deliberately not settable: the bucket column order is the
|
// bucket_order is deliberately not settable: the bucket column order is the
|
||||||
// text in pf.log.bucket now, so a stored order would be a second answer to
|
// text in pf.log.bucket now, so a stored order would be a second answer to
|
||||||
// the same question, and a silent one -- nothing reads it.
|
// the same question, and a silent one -- nothing reads it.
|
||||||
|
//
|
||||||
|
// The three name columns are flag-and-value pairs rather than COALESCE:
|
||||||
|
// clearing one back to the built-in means writing null, which COALESCE on
|
||||||
|
// the value alone cannot tell from "not mentioned".
|
||||||
router.put('/versions/:id', async (req, res) => {
|
router.put('/versions/:id', async (req, res) => {
|
||||||
const { name, description, exclude_iters } = req.body;
|
const { name, description, exclude_iters,
|
||||||
|
adjustment_segment, adjustment_bucket, unlabeled_load } = req.body;
|
||||||
|
const set = (v) => (v === undefined ? null : (String(v).trim() || null));
|
||||||
try {
|
try {
|
||||||
const result = await pool.query(`
|
const result = await pool.query(`
|
||||||
UPDATE pf.version SET
|
UPDATE pf.version SET
|
||||||
name = COALESCE($2, name),
|
name = COALESCE($2, name),
|
||||||
description = COALESCE($3, description),
|
description = COALESCE($3, description),
|
||||||
exclude_iters = COALESCE($4, exclude_iters)
|
exclude_iters = COALESCE($4, exclude_iters),
|
||||||
|
adjustment_segment = CASE WHEN $5::bool THEN $6::text ELSE adjustment_segment END,
|
||||||
|
adjustment_bucket = CASE WHEN $7::bool THEN $8::text ELSE adjustment_bucket END,
|
||||||
|
unlabeled_load = CASE WHEN $9::bool THEN $10::text ELSE unlabeled_load END
|
||||||
WHERE id = $1
|
WHERE id = $1
|
||||||
RETURNING *
|
RETURNING *
|
||||||
`, [
|
`, [
|
||||||
req.params.id,
|
req.params.id,
|
||||||
name || null,
|
name || null,
|
||||||
description || null,
|
description || null,
|
||||||
exclude_iters ? JSON.stringify(exclude_iters) : null
|
exclude_iters ? JSON.stringify(exclude_iters) : null,
|
||||||
|
adjustment_segment !== undefined, set(adjustment_segment),
|
||||||
|
adjustment_bucket !== undefined, set(adjustment_bucket),
|
||||||
|
unlabeled_load !== undefined, set(unlabeled_load)
|
||||||
]);
|
]);
|
||||||
if (result.rows.length === 0) {
|
if (result.rows.length === 0) {
|
||||||
return res.status(404).json({ error: 'Version not found' });
|
return res.status(404).json({ error: 'Version not found' });
|
||||||
|
|||||||
@ -113,6 +113,15 @@ ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS seq integer;
|
|||||||
ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS bucket text;
|
ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS bucket text;
|
||||||
CREATE INDEX IF NOT EXISTS log_bucket_idx ON pf.log (bucket) WHERE bucket IS NOT NULL;
|
CREATE INDEX IF NOT EXISTS log_bucket_idx ON pf.log (bucket) WHERE bucket IS NOT NULL;
|
||||||
|
|
||||||
|
-- The names a row falls back to when nobody has named it, per scenario. Null
|
||||||
|
-- means "use the built-in", which is the DISPLAY DEFAULTS block in
|
||||||
|
-- lib/sql_generator.js. Read through a join at query time, not baked into
|
||||||
|
-- pf.sql: those templates are keyed on (source_id, operation) and shared by
|
||||||
|
-- every version of a source.
|
||||||
|
ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS adjustment_segment text;
|
||||||
|
ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS adjustment_bucket text;
|
||||||
|
ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS unlabeled_load text;
|
||||||
|
|
||||||
-- Master data for a dim_group: one row per key value, with its sibling columns.
|
-- Master data for a dim_group: one row per key value, with its sibling columns.
|
||||||
--
|
--
|
||||||
-- The source is transactional and often a view over all history, so deriving a
|
-- The source is transactional and often a view over all history, so deriving a
|
||||||
|
|||||||
@ -347,6 +347,24 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
|
|||||||
setTimeout(() => setMsg(null), 3000)
|
setTimeout(() => setMsg(null), 3000)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// The names a row falls back to when nobody has named it. Blank means "use the
|
||||||
|
// built-in", so these save on blur like the segment fields and an empty box is
|
||||||
|
// a meaningful value rather than a missing one.
|
||||||
|
async function saveVersionName(field, value) {
|
||||||
|
const next = value.trim()
|
||||||
|
if (next === (selectedVersion?.[field] || '')) return
|
||||||
|
try {
|
||||||
|
const res = await fetch(`/api/versions/${versionId}`, {
|
||||||
|
method: 'PUT',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify({ [field]: next }),
|
||||||
|
})
|
||||||
|
if (!res.ok) { const d = await res.json(); flash(d.error, 'error'); return }
|
||||||
|
await refreshVersions(sourceId)
|
||||||
|
flash('Saved — reload the Forecast view to see it')
|
||||||
|
} catch (err) { flash(err.message, 'error') }
|
||||||
|
}
|
||||||
|
|
||||||
const selectedVersion = versions.find(v => String(v.id) === versionId)
|
const selectedVersion = versions.find(v => String(v.id) === versionId)
|
||||||
|
|
||||||
return (
|
return (
|
||||||
@ -379,6 +397,31 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
|
|||||||
)}
|
)}
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
|
{/* Fallback names. Not part of any segment -- they are what the pivot shows
|
||||||
|
for rows nobody has named, so they belong to the version rather than to
|
||||||
|
a log entry. Blank falls back to the built-in in sql_generator's
|
||||||
|
DISPLAY DEFAULTS block. */}
|
||||||
|
{versionId && (
|
||||||
|
<div className="bg-white border border-gray-200 rounded p-3 flex items-end gap-3 flex-wrap">
|
||||||
|
<span className="text-xs text-gray-500 uppercase tracking-wide w-full">Fallback names</span>
|
||||||
|
{[
|
||||||
|
['adjustment_segment', 'Adjustment segment', '99 - Adjustments'],
|
||||||
|
['adjustment_bucket', 'Adjustment bucket', '04 - Forecast'],
|
||||||
|
['unlabeled_load', 'Unlabeled load', 'Unlabeled'],
|
||||||
|
].map(([field, label, builtin]) => (
|
||||||
|
<div key={field} className="flex flex-col gap-1">
|
||||||
|
<label className="text-xs text-gray-500">{label}</label>
|
||||||
|
<input
|
||||||
|
key={`${field}-${versionId}-${selectedVersion?.[field] || ''}`}
|
||||||
|
defaultValue={selectedVersion?.[field] || ''}
|
||||||
|
onBlur={e => saveVersionName(field, e.target.value)}
|
||||||
|
placeholder={builtin}
|
||||||
|
className="border border-gray-200 rounded px-2 py-1 text-sm w-48" />
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
{showNewVersion && (
|
{showNewVersion && (
|
||||||
<div className="bg-white border border-gray-200 rounded p-3 flex flex-col gap-3">
|
<div className="bg-white border border-gray-200 rounded p-3 flex flex-col gap-3">
|
||||||
<div className="flex items-end gap-3">
|
<div className="flex items-end gap-3">
|
||||||
|
|||||||
Loading…
Reference in New Issue
Block a user