Compare commits

...

109 Commits

Author SHA1 Message Date
9e9782419a Filter aggregates with the rest of the layout
cleanLayout's job is to let a layout outlive the columns it names, and it
walked every config field that carries a column name except aggregates --
which carries one as its key and, in the multi-arg form, a second as the
weight. Empty in practice today, so nothing was breaking; the first
explicit aggregate would have made a later column change abort the whole
restore rather than lose one entry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 14:43:00 -04:00
d2ba944d41 Make a pivot layout a thing people own and publish
Named layouts lived in localStorage: invisible to anyone else, gone on the
next machine, and nothing to publish from. The one server-side layout was
pf.source.default_layout -- a single anonymous blob any account could
overwrite for every account, which is a published layout with no owner.

pf.layout replaces both. A layout is named, owned, and either private or
published; published ones are listed by everyone on the forecast and
writable only by their owner or an admin, the same rule pf.log already uses
for its entries. Scope is the version, since that is the entry point, with
version_id NULL for the source-wide default a new version inherits.
Applying is never restricted -- Save is withheld on a layout that is not
yours, Save as forks it -- because the guarantee wanted is that a published
layout cannot be changed out from under people, not that it cannot be
adapted.

The toolbar's flat chip row becomes one Layout menu: Published and Mine,
rename/publish/default/delete shown only where they would be allowed, and
a dirty dot computed by comparing the live config against the one the pivot
was applied from, since restore() fires the change event itself.

Existing localStorage lists are lifted into pf.layout on first load.
PUT /sources/:id/default-layout is removed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 14:20:04 -04:00
7217518cfd Write down what auto-pause actually does
It reads like a paint optimisation and is not: pausing deletes the view, so
becoming visible again is a full rebuild. dataflow embeds the same viewer and
would hit the same stall, so it belongs in the shared reference next to the
other things that cost us a day to find.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 12:25:22 -04:00
d14036d1b5 Keep the view across a tab switch
Perspective's auto-pause is on by default, and pausing deletes the view:
AutoPauseState::apply() fires on the viewer's own IntersectionObserver and
on the document's visibilitychange, and set_pause(true) does
view_sub.take().delete(). Coming back is therefore not a redraw but
restore_and_render() -- a fresh view and a fresh traversal of the whole
grain -- which is the multi-second chug on every tab switch, and takes any
per-node expansion with it.

Nothing updates the table while the tab is hidden; every operation is
driven from this page. So the pause bought nothing and cost a rebuild.

Also drops a comment that still described the old set_depth re-apply on
refocus, whose machinery is long gone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-19 09:08:01 -04:00
a028cdb79a Write down the deploy ritual and today's loose ends
The restart / Generate SQL pair caught us out four times today, twice
appearing as an unrelated client-side error: a stored template carrying a
token the running code does not substitute fails at the database, and what
surfaces is Perspective aborting on an empty dataset.

Also records the four things left open -- recode and clone reporting success
on zero rows, the change log not showing an entry id, territory being read
only at login, and the depth buttons rebuilding the view.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 16:58:07 -04:00
23032a0658 Read source columns from pg_catalog, not information_schema
information_schema.columns omits materialized views -- they are not in the
SQL standard -- and gs.osm_skinny is one. So the source this app runs on
looked like it had no columns at all: creating a version failed with "No
usable columns in col_meta" while col_meta plainly held thirty-six, and
registering such a source would have seeded nothing.

RELATION_COLUMNS_SQL returns the same shape information_schema did, so
mapType and every caller are unchanged: data_type is format_type with the
modifier stripped, which spells things the same way ('character varying',
'numeric'), and precision and scale are unpacked from atttypmod as
information_schema does internally. Verified against the live matview -- 36
usable columns, and the types map to exactly what fc_osm_skinny_29 already
has.

The table browser had the same blind spot from information_schema.tables and
now lists from pg_class by relkind, so a materialized view can be registered
rather than merely used by a source registered when it was still a table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 16:12:05 -04:00
469e4cc957 Name each loaded table uniquely
Refresh Data aborted with Table "fc_29" already exists. The name was fixed
per version and the cleanup before it is best-effort: the viewer is still
holding the previous table when the new one is built, so deleting it does
not free the registry entry, and creating a second under the same name
fails.

Nothing reads the name -- the viewer is loaded by reference, and cleanLayout
strips it out of saved configs -- so it only has to be unique. The previous
name is remembered and freed on the next load, when the viewer has let go of
it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 13:49:44 -04:00
0d05dc3baa Show that an operation is running
Apply Scale sat there live and silent for the whole round trip, so on a large
slice the only sign anything was happening was the absence of a result -- and
a second click applied the change twice.

The button now spins, reads "Applying…", and refuses further clicks until the
write returns, with a line beside it saying the pivot updates when it
finishes. Cleared in a finally, so a failure releases it too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 13:32:59 -04:00
f2734ab5b2 Scope the table-info counts to the territory
table-info feeds two things and was scoped for neither: the status bar's
"total rows", and the figure the load progress promises while it waits. So
an account that can see a fraction of the table was told the whole size of
it -- jbukowski's 542k rows reported as 2.8M.

It only showed on the larger territories. The seeded figure is replaced by
X-Row-Count when the headers arrive, and on a small territory the aggregate
returns fast enough that the wrong number barely appears.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 13:29:52 -04:00
5cc59b5792 Wrap the bridge's bar labels instead of cutting them
SVG text does not wrap, so labels were cut at twelve characters -- which with
the sort prefixes now on every bucket meant "04 - Forecas…" and told you
nothing the position of the bar had not already.

Word wrapping to the bar's width, three lines at most, with the ×n and
untagged markers moved below however many lines the label took and the plot's
bottom padding raised to make room.

Character width is estimated rather than measured: measuring means a DOM
round trip per label on every render, and at this size a digit is about
0.55em, which is close enough for a centred label with a bar's width to play
with. A word longer than the line overflows rather than breaking, since half
a word helps nobody.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 13:16:35 -04:00
96342f3527 Say "Starting point vs Plan" rather than "Loads vs Plan"
"Loads" is our word for a segment import. It means nothing to a salesperson
reading a waterfall, and the step is simply where the forecast began relative
to the comparison.

It also carried a stray "· untagged". That suffix marks an adjustment nobody
grouped into an initiative, and this is not an adjustment at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 13:14:31 -04:00
344168abdd Run the ledger's cumulative across the whole selection
It started at the baseline, so it accumulated only the part that can still
move and stopped short of the number on the screen. Now it accumulates in
display order from the top -- billed, then booked, then the baseline, then
each adjustment -- and closes on Selected total.

Adjustable keeps its own figure but no running cell: it is a subtotal of the
walk, not a point on the line, and printing the running there would put two
different totals side by side in one row.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 12:59:45 -04:00
62c815da63 Show the walk's running total in the ledger
The lines sum to Adjustable, and checking that they do meant adding
eight-digit numbers in your head. The column closes on the Adjustable row, so
the walk visibly lands where it says it does.

Value only. A cumulative price is meaningless -- prices do not add -- and a
second running column for units doubles the width to say what the value
column already implies. It appears only when there is more than one line to
accumulate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 12:57:50 -04:00
2814b4073c Tell the bridge which bucket is the forecast
It was the literal 'Forecast'. The moment the buckets were renamed to carry
their sort prefix -- '04 - Forecast' -- nothing matched: every row counted as
a comparison rather than a step, so the walk had no middle, the loads came to
nothing, and the bridge showed the basis cancelling itself exactly to zero
with a Forecast anchor of 0.00 over 0 rows.

The version's adjustment_bucket is the right source, being the same value an
unbucketed adjustment is labelled with, so the bridge and the pivot agree by
construction rather than by both hardcoding the same string.

If that value names no bucket in the data -- renamed since, or never
configured -- it falls back to whichever bucket actually holds the
adjustments. A bridge that is merely mislabelled beats one that is silently
empty.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 12:42:27 -04:00
50f4c50b3a Group the digits as you type in the ledger
-2000000 and -20000000 are the same shape at a glance, and the ledger deals
in both.

Display only: what leaves the input is always the raw string, so nothing
upstream ever sees a comma. Partly-typed numbers survive intact -- "1." and
"-" and "1.50" are all states on the way to a value, and reformatting them
into something else mid-keystroke makes the field unusable.

The caret is restored by counting digits rather than remembering an offset,
since inserting a comma shifts every character after it and a remembered
position lands one place off for the rest of the number.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 12:28:31 -04:00
b4f579c4b1 Merge territory scoping and the write audit trail
An account sees and changes only its own territory: the values live on
pf.app_user, the column they name is flagged per source in col_meta, and the
predicate is built from the session and ANDed on last where no request can
remove it. Fail closed -- an account nobody configured sees nothing.

Enforced on both reads, on every operation, and on the value completion
endpoint, which reads the source table and would otherwise enumerate the
whole business to someone shown none of their rows. Undo and annotation are
gated by author, since label and bucket name the pivot's columns for
everyone; recode refuses to move a row between territories unless you are an
admin.

Each entry now records what it ran against and the statement it ran, beside
the intent it already recorded -- the three things that cannot be
reconstructed afterwards, and the SQL that the intent actually became.
2026-09-18 12:26:08 -04:00
c6005bd17c Record what a write ran against, and the statement it ran
params says what was asked for. That is not enough to explain a surprising
result, because the same intent produces different rows depending on state
the entry does not carry, and because the translation from intent to SQL is
itself a place bugs live.

So both, not one. env records the state that cannot be reconstructed later:
the territory in force, the version's exclude_iters, and when the template
was generated -- all mutable rows elsewhere with nothing remembering what
they were. sql_text records the statement as executed, territory and scope
already resolved into it.

The template generation is a fingerprint rather than a version. It cannot
bring the old template back; it can tell you the entry did not run under the
current one, which is what would otherwise make a comparison quietly wrong.
Generate SQL has overwritten those templates four times today.

The statement is fetched on demand through GET /log/:logid/debug and left out
of the list, which is opened to scan rather than to read SQL. Under an
entry's payload in the change log there is now an "executed SQL" toggle.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 12:19:40 -04:00
39b2a7e4a2 Only the author or an admin can edit a log entry
PATCH /log/:logid had no check, so any account could edit the tag, note,
label and bucket of any entry -- including loads whose rows it cannot see.
That reads as harmless annotation and is not: label and bucket name the
pivot's columns for everyone in the version, so a rep could rename the
company's segments.

Same rule as undo now, author or admin, with the fields shown read-only
rather than editable-then-403 -- in the change log's tag and note cells and
on the Baseline page's label and bucket.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 12:06:13 -04:00
61268f2a7a Say who made each change, and which ones you can undo
The change log showed what happened and never who did it, which stops being
a detail the moment more than one person is in the version.

The Undo button greys out on entries belonging to someone else, with the
reason on hover. The server already refused them; the button offered the
click anyway and answered with a 403, which reads as a fault rather than a
rule.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 12:03:06 -04:00
9d08638a92 Read territory at login, and survive having none
The login query names its columns, and territory and is_admin were not among
them -- so every session carried an empty list, every account scoped to
FALSE, and the forecast page came back with nothing however the grant was
set. The CLI had written it correctly; nothing read it.

The empty case then aborted twice over. First on the index, fixed already.
Then on the layout: an empty table has no schema, so restoring a saved
config asks for the dtype of a column that is not there and the worker dies
-- "Could not get dtype for column `sseas_e`". With no rows there is nothing
to lay out, so nothing is restored, and the saved layout waits in
localStorage for rows to come back.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:52:00 -04:00
68e2d81c8c Let an empty result be an answer, not an abort
worker.table([], { index: 'pf_gkey' }) aborts: an empty array carries no
columns, so the index names one that does not exist and the page dies with
"Specified index `pf_gkey` does not exist in dataset" instead of saying it
found nothing.

Nothing is a legitimate answer -- an empty version, and now a territory with
no rows in it, which is what surfaced this. The empty table is built without
an index and the page says which of the two it is.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:48:57 -04:00
e94c364406 Fix the territory commands' success message and menu order
They called ok(), which this script does not have -- the helper is
success() -- so set-territory ended on "ok: command not found" after having
worked. The three new entries also sat between 13 and 14 in the menu, having
been appended where the list-users case was rather than at the end.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:47:02 -04:00
e60dd3ad96 Put the territory column in the Setup editor
The flag was enforced everywhere and settable nowhere but psql, so the one
piece of configuration a second account depends on was invisible.

A radio rather than a checkbox, because exactly one column per source can be
the territory -- the control should say so rather than leaving it to an
error on save. Clicking the chosen one again clears it, which a radio has no
other way to express. The save still refuses two, since the UI is not the
only caller, and two would mean whichever a .find() reached first -- the trap
is_key already fell into.

Restricted to dimension columns: a territory is something rows are divided
by, and scoping on a date or a measure is not a thing to offer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:38:55 -04:00
712d114dc5 Scope what an account can see and change to its territory
Territory filtering was deferred from v1, so every account saw and could
write every row. With the sales team about to adjust their own territories
that is the thing standing in the way, and it is also why a rep would wait
fifteen seconds to load 2.7M rows to work on a few thousand.

The list lives on pf.app_user.territory with is_admin beside it, and
col_meta.is_territory marks which column of a source the values belong to --
flagged rather than named in code, so a second source can be divided by
something other than a sales rep.

Fail closed: buildTerritoryClause returns FALSE for an empty list or an
unflagged source. An account nobody configured sees nothing, rather than
everything because a column was left null.

Built from the session, never the request. That is what separates it from
`scope`, which the browser sends and should: a filter the user chose belongs
in the payload, a permission cannot come from the thing it restrains. It is
ANDed on last, where nothing in the request can undo it.

Enforced on /data (the cursor and the count behind X-Row-Count), on /agg
before the GROUP BY since the territory column need not be in the grain, on
every operation through sliceUnits, and on the value completion endpoint --
which reads the source table, so without it a dropdown enumerates every
customer and rep in the business to someone shown none of their rows.

Undo is gated by owner rather than territory: it removes an entry's rows
wholesale, so half-undoing one would leave a state nothing describes. Recode
refuses to set the territory column unless you are an admin, since moving a
row between territories is reassignment, not forecasting.

./pf.sh gains set-territory, set-admin and orphan-territory. The last lists
territory values no account owns -- work under one is invisible to everybody
but an admin, which a typo causes easily and nothing in the app reveals.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 11:14:03 -04:00
b173a3ddaf Merge adjustment row collapse and stamped log totals
Adjustments write one row per coordinate rather than one per row read, so an
operation no longer inherits the row count of every layer before it.
Verified live: a scale reading 2,948 rows wrote 908.

Each log entry records what it did -- row count, value and units, and the
columns those are denominated in -- stamped after the write rather than
recomputed by joining the whole forecast table every time the change log is
opened.
2026-09-18 10:57:06 -04:00
496642c545 Stamp each log entry with what it did
The change log joined the whole forecast table on every open to total rows
it had just written -- 2.5M rows to report a few thousand, and the Baseline
page's row and value columns paid the same cost again.

The totals go onto pf.log at write time instead. This is not a cache that
can drift: a log entry's forecast rows never change once written, because
only the operation owning the logid inserts them and the only thing that
removes them is undo, which deletes the log row too. measure_cols records
which columns the figures are denominated in, since the value and units
roles can be reassigned in col_meta and the numbers would otherwise quietly
come to mean something else.

Stamping is best-effort and runs after the commit: a failure to record what
happened must not roll back the thing that happened. The loads return their
new log id to make it possible, the adjustments take theirs from the rows
they return, and apply_mode 'each' writes one entry per slice, so it is a
set rather than a single id.

?recount=1 does it the old way and writes back what it finds. Stored totals
cannot drift on their own, but nothing stops someone deleting forecast rows
by hand, and a stored figure has no way to notice -- so there is a way back,
which doubles as the backfill for entries written before the columns
existed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 10:40:39 -04:00
f70de94e61 Write one adjustment row per coordinate, not per row read
Every operation inherited the row count of everything before it. Scale read
the baseline's rows plus every prior adjustment's rows sitting at the same
dimensional coordinate, and wrote a delta for each -- so eight Pull Forward
entries meant the next scale over that slice wrote nine rows where one would
do, and the table grew super-linearly with how much work had been done on it.

The base sets are grouped now: scale's `base`, recode's `src`, and clone's
source, each by every stored dimension and date, summing the measures. The
collapse is over pf_logid and pf_iter alone, so no column goes null and
nothing becomes unsliceable by a later operation -- which is the trap in
collapsing to the display grain instead, where the non-grain dimensions would
have to be null and the next slice naming one would silently miss these rows.

The maths is unchanged. Scale's proportional split needs the total over the
pool, and sum(sum(x)) OVER () gives the same figure over collapsed
coordinates that sum(x) OVER () gave over raw ones -- the window runs after
the GROUP BY. Verified on a real slice: 3,989 rows collapse to 3,187 at
1,303,545.75 either way. Across the version's existing adjustments it is
149,458 rows against 124,474, and that understates it, since the point is
that the next layer no longer multiplies the last.

All three templates planned against the live table before committing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 09:59:14 -04:00
74fd030638 Merge segment labels, stored ordering, and slice fidelity
The pivot's column order is stored text now, typed into pf.log.label, rather
than a prefix computed in the browser from a seq column -- which ordered the
pivot and nothing else, had to be re-applied after every layout load, and
could not express a label outside printable ASCII. 250 lines of expression
machinery go with it.

The fallback names an unnamed row shows live on pf.version, editable per
scenario. A segment edit no longer wipes the annotations it was not handed.

Slices mean what they say: pf_segment and pf_bucket resolve to log ids
instead of being dropped, the measure name no longer leaks in from a
collapsed column, and the pivot's filter scopes both the ledger and the
write. Each of those was a case of the panel showing one number and the
operation changing a wider set.

The ledger names its lines the way the rest of the app does, leads with what
cannot move, and breaks that out per segment.
2026-09-18 09:56:23 -04:00
afab81d770 Merge ledger and slice fidelity work
Slices now mean what they say -- pf_segment and pf_bucket resolve to log ids
rather than being dropped, the measure name no longer leaks in from a
collapsed column, and the pivot's own filter scopes both the ledger and the
write. The ledger names its lines from label like everything else does, shows
what cannot move first and per segment, and the column groups are ruled off
in the grid.
2026-09-18 09:53:37 -04:00
28aa7012f1 Put what cannot move at the top of the ledger
The immovable rows sat between Adjustable and Selected total, which made
them read as an afterthought to a figure they in fact constrain. They come
first now: this much is already booked and billed, this is what is left to
work with, and here is how that got to where it is.

The walk stays immediately above Adjustable, because it sums to it -- the
two are one statement and separating them would leave a column of numbers
adding up to nothing on the page.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 09:43:12 -04:00
2f862a6c8b Name the ledger's lines the way everything else names them
The walk read from tag and note, and hardcoded the word "Baseline" for the
baseline load -- so a segment called 03 - New Orders in the pivot, in the
bridge and on the Baseline page read as "Baseline" in the one place you go
to check a number before changing it. label comes first now, the same
precedence pf_segment uses, in the ledger and the bridge alike. logMeta did
not carry label at all, which is why neither could reach it.

The immovable rows split one line per segment. Combined, "01 - YTD Sales ·
02 - Open Orders" said 1.6m was untouchable without saying how much of it was
billed and how much was booked -- different things a forecaster treats
differently. The FINAL badge also gains the space it was missing, having
rendered as "02 - Open Ordersfinal".

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 09:38:37 -04:00
f9424d9c42 Recognise a subtotal by its zero-width padding
Perspective pads a subtotal's column path to full length rather than
shortening it -- ['04 - Forecast', '​', 'sales_usd'] -- so testing for
an empty string found no subtotals and nothing was tinted. The blank test now
strips zero-width spaces and the other invisibles alongside whitespace.

The grand total falls out of the same rule, its path being blank at every
level above the measure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 09:26:21 -04:00
54d49ebc75 Rule off the column groups and mark their subtotals
Prior, plan and forecast each carry twelve months and a total, so scanning
across is thirty-odd columns of identical-looking numbers with nothing to say
where one domain ends and the next begins -- annual figures read as just
another month.

Each group's first column now takes a left rule and each group's subtotal a
tint and a heavier weight. Both are derived from the cell's column path: the
deepest path is a leaf, so anything shorter is an aggregate of the levels
below it, which is what makes a subtotal a subtotal.

Through regular_table's style listener rather than CSS, because the DOM cells
are recycled across columns as you scroll -- a stylesheet would paint the
wrong ones the moment the grid virtualised. The styles go into the grid's own
shadow root, since a page stylesheet cannot reach it, and use currentColor so
the rule follows the theme instead of disappearing against Pro Dark.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 09:20:27 -04:00
cdb40e7368 Make the request preview show the request
The panel's preview calls buildPayload itself, and the scope arrived as a
parameter that the preview had no way to supply -- so it defaulted to empty
and printed a payload with no scope for a write that had one. A preview that
disagrees with what is sent is worse than no preview: it is the one place
someone looks to check before committing a change.

buildPayload reads the scope from the ref instead, so there is one payload
and both callers get it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 09:10:09 -04:00
d983e2b1df Carry the pivot filter as a scope, operators and all
Refusing anything but == was safe and useless: a view bounded to
sseas_e <= 2027 is an ordinary way to scope a forecast, and it has no slice
form at all, a slice being {col: value}. The filter now travels beside the
slices as [col, op, value] triples and is ANDed onto every unit -- not folded
into the slices, since it applies to all of them equally and under
apply_mode 'each' would just repeat itself in every statement.

Operators are Perspective's, since that is where they come from, and the list
is a whitelist: anything outside it is refused rather than ignored, because a
scope silently dropped is a write wider than the panel that authorised it.
The scope goes into the log's params too, so the audit trail records what
bounded the write and not only what was clicked.

The panel prints it above the selection as "within sseas_e <= 2027". It
scopes every figure below it and every row the operation writes while
appearing in none of the slices, so without it the panel showed a selection
wider than the one it was acting on -- which is exactly what made the
ledger's 956,485.13 look plausible against a cell of 921,225.71.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 08:46:23 -04:00
1a0a9db8d0 Show the row count while the load is still waiting on it
X-Row-Count is exact but travels with the response headers, and in grain mode
the server aggregates the whole table before sending any -- so the number
appeared just as the fifteen-second wait ended, which is no use to anyone
watching it.

The forecast table's own count goes up first instead, from the same
table-info the status bar already reads, and the exact figure replaces it
when the headers arrive. The count query is deliberately not awaited: it
scans the whole table, and the load must not wait on a progress message.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 08:41:24 -04:00
9753846d34 Make the pivot's filter scope the ledger and the write
perspective-click reports only the clicked cell's own dimensions -- the
view-level filter is not in it -- so a slice never carried the season the
grid was scoped to. The ledger therefore counted rows the grid was hiding
(921,225.71 on screen against 956,485.13 in the panel) and an operation
would have written them.

Both now read the filter off the viewer. The ledger applies it to its own
view, where the values are already in the table's types and any operator
works. The operation merges the equalities into each slice, cell values
winning on a shared column since a cell cannot contradict the filter it was
drawn inside, and refuses outright on any other operator: a range or an
in-list cannot travel in a slice, and dropping it silently is the widening
this is meant to stop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 08:39:44 -04:00
6b63e9a5f3 Say how many rows the load is waiting on
X-Row-Count arrives with the headers, long before the body has been read, so
the overlay can name the wait instead of saying "Loading…" over a grey
screen for fifteen seconds. On this data the row count *is* the wait -- the
bytes are quick and the rows are not -- so it is the number worth showing
beside the transfer bar, which only ever measured the fast part.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 08:38:04 -04:00
c6ef350283 Type a slice's values before filtering the ledger with them
A slice carries every value as a string -- built from filters the grid
reports, and shaped to survive JSON on the way to the API. Perspective
matches on type, and a string '2027' against an integer column is not a
filter that matches nothing, it is a filter that is dropped.

So the pivot's own season filter never reached the ledger: with the grid
scoped to sseas_e = 2027 the ledger totalled 956,485.13 against a cell
reading 921,225.71, the difference being eleven rows of a baseline segment
whose shipments fall in the next season. Only dates were being coerced, and
only because someone had hit this before with them.

Values are now typed against the loaded table's schema rather than against
col_meta's role, which is the thing that actually decides the match. The
server side was already right -- Postgres casts the literal -- and returns
921,225.71 for the same slice.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 08:33:34 -04:00
f94aa4ec99 Drop the measure name from a collapsed column's slice
Clicking a bucket subtotal with the month level collapsed produced
{"customer": "...", "pf_bucket": "04 - Forecast", "smon_e": "sales_usd"} --
a month equal to a measure, matching nothing, so the ledger came back empty
and an operation would have had no rows to act on.

Perspective maps split_by positionally over the column name, and a collapsed
axis has fewer segments than there are split_by levels, so the measure lands
on the first hidden dimension. Both slice paths now drop any == filter whose
value is one of the view's measures; the region path additionally takes the
measure off the end of the column name before mapping, which is the same
error made in our own code.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 08:16:09 -04:00
50a0bb42aa Make a slice mean what it says, and say what cannot move
The phantom: pf_segment and pf_bucket are computed from pf.log when the rows
are served, so buildWhere had no column to compare and dropped them. Clicking
one bucket's cell and scaling therefore wrote every bucket at that dimension
intersection, while the panel showed only the bucket clicked. On the example
slice that is 350,524.74 displayed against 503,446.08 written.

They resolve exactly, without a new column: the name lives on the log row and
every forecast row carries the pf_logid that points at it, so the predicate is
pf_logid IN (SELECT id FROM pf.log WHERE <the same expression> = ...). Verified
against version 29 -- the clause returns 350,524.74 over 12 rows.

Any other pf_ key is now refused rather than skipped, since skipping is the
mechanism by which a selection silently widens. pf_iter stays exempt: the
client drops it deliberately, two cells differing only by iter band being the
same slice.

Client side they are ordinary columns in the loaded table, so both the
dispatch path and the panel's own totals filter on them directly -- the latter
matters as much, or the ledger reconciles against a wider selection than the
operation writes.

The ledger: excluded rows read "02 - Prior Year · FINAL" in amber rather than
"reference · fixed" -- named by the segment a forecaster recognises instead of
the iter band that happens to exclude it, and coloured because immovable is a
property worth seeing before reading a number. When the whole selection is
immovable it now says so in a sentence, where before it printed a row of zeros
and left the reason to be inferred from the edit rows failing below.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-18 00:17:25 -04:00
0cabe9bcd2 Put the fallback display names on the version
adjustment_segment, adjustment_bucket and unlabeled_load are columns on
pf.version now, edited under "Fallback names" on the Baseline page, with the
constants in sql_generator left as the built-in for a version that sets none.

Read through a join, not substituted at generation: pf.sql is keyed on
(source_id, operation) and shared by every version of a source, so a baked-in
value could not vary by version and regenerating for one would change the
others. The join costs three more GROUP BY columns on /agg, all functionally
dependent on a version id that is already fixed for the whole query.

The built-ins stay a convention guess -- ADJUSTMENT_BUCKET's "04 - " suits one
numbering -- which is now a default to override rather than the only answer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 23:47:17 -04:00
aa03e74b5e Prefix the adjustment bucket, and list the hardcoded names
Adjustments fell back to a bare 'Forecast' while the loads they adjust read
'04 - Forecast', so the bucket column split in two and the adjustments sat
apart from the rows they came from. The fallback now matches.

The three fallback names are gathered into one DISPLAY DEFAULTS block at the
top of sql_generator, exported, and tabulated in CLAUDE.md, so the answer to
"where did that name come from" is one place rather than a grep. The
incremental row stamps in the operation routes use the constant now instead
of restating the literal, which is how they drifted apart in the first place.

None of this belongs in the source. ADJUSTMENT_BUCKET carries a number that
only suits one convention and changing it changes every version on every
source; the note says what per-version would take.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 23:42:51 -04:00
1286351be3 One name for one column: Note
Description was the original field and Note replaced it, but nothing removed
it -- so the edit form carried both, writing the same pf.log.note, with
`note: description || segNote` letting Description win silently over whatever
was typed in Note right below it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 23:18:41 -04:00
2306315a17 Stop a segment edit from wiping its annotations
PUT /versions/:id/baseline/:logid deletes the log row and inserts a fresh one
from the stored template, so every field the form does not send comes back
null. That took the label and the bucket with it, and the tag besides -- the
segment form has no tag input at all, so a tag could not survive an edit made
for any other reason.

The route now hands back what it was not given, reading the row it is about
to replace. `??` rather than `||`: an empty string is the form clearing a
field deliberately, undefined is the form not carrying it.

The load templates gained a tag token to receive it. Stored templates are
per-source and were generated before any of this existed, which is the other
half of why labels vanished -- source 14's had no label column to write to.
Regenerated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 23:06:17 -04:00
3162759f93 Size the table to its content, not to the window
Measuring the columns was not enough while the table itself was w-full inside
an uncapped page: it stretched to the window and handed the slack back out,
so the measurements only decided who got squeezed. The table now sizes to its
content and the page uses items-start, so each block is as wide as it needs.

Ceilings pulled in a little too -- with nothing competing for slack they are
the actual width, not a limit on a fight.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 23:02:38 -04:00
602effde73 Give the segment table room, and size its text columns by content
The page was capped at max-w-4xl. Eleven columns in 896px meant something was
always smashed, and the previous fix just moved which one -- w-full on the note
cell let it claim the slack, and w-48 on label is only a hint in an auto-layout
table, so the browser shrank the label input to min-content. The cap is gone;
the blocks that read better narrow keep their own.

label, note and counts-toward are now measured off the longest value in the
log, in ch, with floors so an empty table keeps its headers and ceilings so one
long note cannot push the numbers off the side.

The note's one-line clip moved onto its inner div: a max-width on a table cell
is only a hint too, so pinning it to the cell could collapse the column to
min-content or let it grow past the measurement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 23:00:27 -04:00
98f7bbef34 Stop the note column wrapping the whole row
It was the only one of eleven columns with no width, so it lived on whatever
slack was left once kind, label and counts-toward took theirs -- and it holds
the longest text of any of them. Before label and bucket existed it also held
the operation badge and the segment name, and had the room for them.

Now one line with an ellipsis, the full text on hover, and unclipped in the
expand panel underneath, which already renders it. w-full with max-w-0 is
what lets a cell in an auto-layout table absorb the slack and still clip;
with only w-full the column grows to fit and nothing truncates.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 22:57:20 -04:00
98322a6080 Close the write-only surface left behind by the ordinals
bucket_order and log.seq were still settable -- PUT /versions/:id took the
first, PATCH /log/:logid the second -- with nothing left to read either. A
field that only ever gets written is worse than a missing one: the call
succeeds, so the caller has no way to find out it did nothing.

The columns themselves stay, marked vestigial where they are declared.
Dropping a column is not worth a migration to reclaim two that cost nothing.

pf.log.bucket is untouched and stays exactly as it was -- what a row counts
toward, read first by BUCKET_EXPR. It is the bucket *order* that no longer
needs storing, the text having become the order.

Also removes a comment head in Forecast.jsx that survived its function and
had glued itself onto fitColumns.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 22:51:34 -04:00
3299bfe10b Be exact about what the 99 fallback does
It puts adjustments after every *numbered* segment, not last outright: on the
existing version, whose segments are still named AOP and YTD Sales with no
prefix, "99 - Adjustments" sorts first, because digits precede letters. That
is the scheme working as designed rather than an edge case, so the note says
so.

Also corrects the regeneration claim: editing a label is a PATCH and needs
nothing regenerated. Generate SQL is a one-time thing per source, so its
stored load templates write label and bucket at all.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 22:45:56 -04:00
03278eb091 Drop the bucket reorder buttons, and document the new mechanism
bucket_order fed the ordering expressions and nothing else, so its up/down
list is gone. The datalist it shared state with stays, now offering the
buckets actually in use plus the four conventional names carried with their
prefixes -- a near-miss spelling silently splits a column in two, so the
options are worth more than they were.

saveBucket was a duplicate of saveLogField left behind by af9e6de's refactor;
the bucket cell goes through saveLogField like the label does. Both
confirmations now say to reload the Forecast view, which is true of a label
for the same reason it was true of a bucket: it is part of the aggregated row.

pf.log.seq and pf.version.bucket_order are no longer read anywhere. The
columns stay -- dropping them is not worth the migration, and nothing costs
anything by their being there.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 22:45:11 -04:00
7738b904bf Delete the client-side ordering expressions
With the prefix stored in pf.log.label there is nothing left to compute, so
all of it goes: ORDER_EXPR_NAMES, buildOrderExpression and its ExprTK
printable-ASCII-per-byte guard, SYNTHETIC_SEGMENTS and its 98/99 ordinals,
syncOrderExpressions with its two self-issued fetches, the dbgOrder tracing,
and the three places it had to be re-applied because restore() replaces
`expressions` wholesale.

What remains is a list of the names it used to manage, stripped by
cleanLayout so a layout saved under the old scheme does not keep ordering by
a rule nothing updates. The strip goes before the axis filter: dropping them
from `expressions` is what makes the existing ok() reject them on every axis,
which restore() requires -- an expression the pivot is using cannot vanish
from underneath it.

Net 250 lines out. The Baseline page's reorder buttons no longer feed
anything and go next.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 22:44:07 -04:00
885c9abe83 Put the ordering prefix in the stored label, not in an expression
Perspective orders column groups by the value string, so "01 - Actual" is
the only way an arbitrary order can be expressed. That prefix now lives in
pf.log.label, typed by whoever names the segment, rather than being built
from a seq column by client-side expressions.

pf_segment and pf_bucket read label first, and the expressions are shared
between /agg and /data instead of being spelled out in each -- they have to
agree, and they had drifted apart in whitespace already.

The synthetic values lose their parentheses and their ordinals, except the
adjustment fallback: '(adjustment)' sorted *before* '01 - ...', since '(' is
0x28 and digits begin at 0x30, so it becomes '99 - Adjustments' to sit last.
Labelling an adjustment's own log row overrides that, which is how one kind
of adjustment splits out from the rest. '(unlabeled load)' becomes plain
'Unlabeled', which needs no ordinal -- letters already follow digits.

The load routes carry label and bucket onto the log row, so the fields the
segment form has been offering since af9e6de are no longer a silent no-op.
startEdit now reads them back, which it never did: editing a segment for any
other reason blanked both.

Existing sources need Generate SQL re-run -- the load templates are stored.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 22:42:57 -04:00
893a395529 Give the operation badge its own column, and drop seq
"referenceYTD Sales" ran together because the badge shared the note column,
and that column had lost width to label and counts-toward. The badge is a
fixed-width token, so it gets a column of its own and stops competing with
free text.

seq goes with it. The label carries the sort order now -- it is typed with
its own "01 - " prefix -- so a separate ordinal column is one more thing to
keep in agreement with it for no gain. saveSeq and its state go too.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 22:22:46 -04:00
af9e6de88e Give a segment an editable label and bucket, at creation and after
Two gaps you hit. There was nowhere to set "counts toward" while defining a
segment -- only in the list afterwards -- and the Edit buttons disappear
entirely once any adjustment exists.

That guard is right in principle and too broad in practice. Editing a
segment's filters or date offset after a scale would silently recalibrate a
distribution that was sized against the old rows, so it stays gated. But the
label and the bucket are presentation: they change what the pivot shows and
what the segment counts toward, never which rows were loaded. Those are now
editable in the list at any time, and settable on the create form.

pf.log.label is new: the segment's display name, falling back to tag then
note. Separate from both because those have jobs already -- tag groups
adjustments into initiatives for the bridge, note is commentary -- and
because the label is where sort order lives. Perspective orders column
groups by the value string, so a leading "01 - " is how ordering gets
expressed, and putting that in the note would put it in every note.

The load routes do not yet carry bucket and label through: they return only
rows_affected, with no log id to attach them to. That comes with the switch
away from computed prefixes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 22:15:25 -04:00
0fdb08291d Carry the pivot's expressions into the bridge's own view
The bridge builds its own view from the pivot's filters, and those filters
can name pf_bucket_ord or pf_segment_ord — columns that exist only as
expressions. A view created without them cannot resolve the column, so the
bridge failed outright as soon as anyone filtered on an ordering column.

The per-slice path is left alone: its filters are built from col_meta names,
so they only ever reference real columns. Said so in place, since the
asymmetry otherwise reads as an oversight.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 15:38:03 -04:00
a83138b3ce Make the row-label floor three characters, not 130px
130px was far too wide — it was a guess, and it turned the row labels into a
quarter of the sheet. The floor only needs to stop a column coming back
unusable after a layout restore, not to fit anything.

Three characters, measured in the grid's own font plus its cell padding
rather than fixed in pixels, so it survives a theme or zoom change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 15:25:38 -04:00
4045d336c4 Put a floor under the row-label columns
Loading a saved layout brought them back a few pixels wide, needing to be
dragged open by hand. Restoring resets the widths, and the row-header
columns are then sized from their header — which for row headers is a blank
corner cell — so they measure as empty.

A minimum rather than a fit. Fitting to content is the other extreme: each
group_by level is its own column, so the first widens to its longest label
and shoves the second rightwards, which is the spacing that read worse than
the default. 130px leaves a reasonable default alone and only intervenes
where a column came back unusable; anything already wider, whether dragged
or recorded in a layout, is untouched.

Applied after the initial load and after a layout restore — the two places
that reset widths.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 15:21:44 -04:00
a9062c398c Leave row labels at their default width
Each group_by level is its own row-header column, so fitting the first to
its longest label pushes the second to start after it — and the default
spacing reads better than the fitted result, cap or no cap. Fit now sizes
the data columns only, which is what it was doing when it was useful.

fitRowLabels is kept, unused, one call away: the measurement was the hard
part and the judgement about whether to apply it may change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 15:13:48 -04:00
df8b6f56c9 Re-apply the ordering columns after loading a saved layout
Switching to a saved layout dropped pf_bucket_ord and pf_segment_ord.
restore() replaces `expressions` wholesale rather than merging, and a layout
saved before the feature existed carries none to restore — the same fault
initViewer had, fixed there and not generalised.

The layout is persisted from the merged config afterwards, so the saved copy
picks the expressions up rather than re-dropping them on the next load.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 15:10:40 -04:00
ec5d9e1dc3 Cap the row-label width, and never narrow a column
Each group_by level is its own row-header column, so fitting the first to
its longest label pushes the second to start after it — which reads as the
deeper level being indented past the end of the shallower one, and sends the
rest of the pivot off to the right when a label is long.

Capped at 260px, which trades a rare truncation for a sheet that stays
legible. And taking the max with the current width means Fit only ever
widens, so a width set by dragging is not undone by pressing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 15:05:36 -04:00
9ccaabd4c1 Pin the row-label columns by index, not through __ROW_PATH__
Row labels still did not fit, because the override was landing on the wrong
column. restore_column_size_overrides maps the key "__ROW_PATH__" to index
tree_header_offset - 1, which with two group_by levels is index 2 — the
first data column. That column is the grand total, which the stylesheet
hides, so pinning it changed nothing visible.

The row labels are separate columns (rt-col-0 and rt-col-1 here, one per
group_by level), so each is measured on its own and set by index through
regular-table's saveColumnSizes / restoreColumnSizes, which are index-based
and public.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 14:59:34 -04:00
e7c266a871 Quiet the ordering trace behind pf_debug
It works now — inputs carrying a bucket_order and three seq values, both
expressions applied — so the console does not need narrating, least of all
during a demo.

Kept rather than deleted, gated like the depth tracing: the failure it
diagnosed (running before its inputs existed, then reporting nothing) is the
kind that recurs, and rebuilding this each time is wasted work. The
initViewer scaffolding goes, having served its purpose.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 14:47:38 -04:00
424199c1aa Name the ordering columns pf_bucket_ord and pf_segment_ord
"Bucket" and "Segment" sit too close to the source data -- segment_new is an
actual column here -- and a name collision would be worse than a confusing
label: an expression named after an existing column shadows or rejects it
rather than just reading ambiguously.

pf_ prefixed like every other synthesised column, so they sort beside
pf_bucket and pf_segment in the column list and read as belonging to the app.

The old names are kept in the managed list so they are cleared from configs
that still carry them. Without that they would sit in saved layouts forever,
ordering by a rule nothing updates -- which is exactly what a stale Segment
expression was already doing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 14:46:13 -04:00
f7f4fbb4c6 Have the ordering sync fetch its own inputs
The instrumentation finally said it plainly:

    inputs {versionId: '29', versionFound: false, bucket_order: null,
            logMetaCount: 0, seqs: []}

It read the `versions` prop and the `logMeta` state, both populated
asynchronously, while running from initViewer — which finishes well before
them on a large load. So it was called with nothing every time, could never
build an expression, and the effect meant to re-run it once the data landed
never fired. Two small queries beat depending on that timing, the same
correction the master-data effect needed for the same reason.

Also explains why the ordering half-worked: `existing: Array(1)` with
Segment already applied. An earlier session had built it and it has been
riding in the saved layout since, so segment ordering appeared to work while
Bucket never existed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 14:43:30 -04:00
6c9d0eef11 Actually instrument the ordering sync
The earlier attempt at this never landed: its patch failed an assertion, and
I read a `grep -c "pf-order"` of 1 as confirmation when that count is just
the logger's own template literal — every call site was missing. So the
function was reached, returned silently, and reported nothing, twice over.

Logs on entry, the inputs it resolved, the expressions already in the saved
config, and each exit. Verified against the source and against the
individual strings in the built bundle rather than a count.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 14:40:22 -04:00
a5bb814a68 Trace how far initViewer gets before the ordering sync
No [pf-order] line appeared at all, and the sync call is unconditional
inside initViewer, so initViewer is not reaching it. Logs on entry, on the
superseded-by-a-newer-load guard, and immediately before the call, so the
next reload says which.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 14:35:25 -04:00
e8e95bf4b9 Log every entry and exit of the ordering sync
Neither the success nor the failure line appeared, and save() reports no
expressions at all -- so the function is either not being called or leaving
by a path that says nothing. Two of its exits were silent: the no-viewer
guard, and the case where the computed expressions already match what is
applied.

Logs on entry, logs the inputs it actually saw (the version it resolved, its
bucket_order, how many log entries carry a seq, and what it decided to
build), and logs every exit. One reload should end the guessing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 14:29:43 -04:00
33bfec838a Say why the ordering columns are missing instead of failing silently
syncOrderExpressions caught its errors and logged them, and returned quietly
when there was nothing to build. Those two outcomes are indistinguishable
from the outside -- the Bucket and Segment columns are simply absent -- which
is most of why this has taken several rounds to pin down.

A restore() rejection now raises a message in the status bar, and the
no-ordering case logs the inputs it saw: the version's bucket_order and
every log entry carrying a seq. One reload should say which of the two is
happening.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 14:18:24 -04:00
b75ce939b1 Fit the row-label column to the labels it actually shows
Values fitted and row labels did not. The measurement pairs each column's
header with its body cell, and for the row-header columns the header is a
blank corner cell — so the column was being sized from an empty string,
never from the labels beneath it.

Measured with a canvas instead of the DOM, for the same reason the headers
cannot size themselves: the cell is clipped, so reading its box back returns
the width it was allotted rather than the width of its text. Tree
indentation is added, since it occupies real width. The result is pinned as
a column_size_override on __ROW_PATH__ — the key
restore_column_size_overrides special-cases for this column, and an override
is what survives subsequent draws, which is precisely why overrides were
defeating resetAutoSize earlier.

Group headers are deliberately left alone. pro.css is explicit:

    /* Header groups should overflow and not contribute to auto-sizing. */
    thead tr:not(.rt-autosize) th { overflow: hidden; max-width: 0px; }

Only the leaf header row participates, because a group label spans many
columns and letting it set width would widen all of them. With the ordinal
prefix in front, "01 - Prior…" still reads when truncated. Fixing it
properly means distributing a group's label width across its span, which is
a different job.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 14:17:22 -04:00
194134ea5f Apply the ordering expressions after the layout, not before
The Bucket and Segment columns never appeared in the column list, and
viewer.save() reported expressions: {}. syncOrderExpressions ran immediately
after viewer.load() — and the saved layout was restored on the next line.
restore() replaces `expressions` wholesale rather than merging, so the
expressions were created and then wiped every single time, before anything
could see them.

Moved to after both restore branches. The effect on [logMeta, versions,
versionId] then keeps them in step, as intended.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 14:10:55 -04:00
9a748b8178 Let headers and row labels size their own column
Fit worked on the values and not on the headings. Auto-fit measures cells
with getBoundingClientRect() and sets each column's min-width from the
result, but the datagrid's CSS wraps and clips header text — so a clipped th
measures at the width it was *allotted*, not the width of its content, and a
column can never grow to fit its own heading. Row labels are th elements in
tbody and clip for the same reason.

white-space: nowrap on those cells, and nothing else: no width, no overflow.
The measurement then sees the full text and the existing sizing logic does
the rest.

Folded into the stylesheet already being injected for the grand-total
column, and renamed from pf-hide-split-total to pf-grid-css now that it does
more than the one thing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 14:02:59 -04:00
81c4672147 Fit columns by driving regular-table, not the plugin's draw
The Fit button did nothing. Setting _reset_column_size and redrawing cannot
work, because draw() undoes the reset on the very next line:

    const old_sizes = save_column_size_overrides.call(this);
    ... if (this._reset_column_size) { resetAutoSize() }
    restore_column_size_overrides.call(this, old_sizes);

and old_sizes comes from regular_table.saveColumnSizes() -- the *live*
widths -- not from plugin_config. So each draw preserves whatever the
columns currently are, whatever the config says, and clearing
column_size_override released nothing.

Calls regular_table.resetAutoSize() and draw() directly instead: the same
draw the plugin performs, without the save/restore wrapped around it.
_cached_column_sizes is cleared too, since it short-circuits the next save
and would be restored over the measurement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 13:57:33 -04:00
7975acc0fc Add a Fit action to size columns to their contents
The datagrid already measures content -- draw() calls
regular_table.resetAutoSize() -- but only when _reset_column_size is set,
and it puts any pinned widths back immediately afterwards:

    const old_sizes = save_column_size_overrides.call(this);
    ... if (this._reset_column_size) { resetAutoSize() }
    restore_column_size_overrides.call(this, old_sizes);

So a width pinned by dragging a column edge, or carried in a saved layout,
outlives every measurement -- including the automatic ones the datagrid does
when split_by or columns change. Clearing the overrides is what releases
them; the flag then makes the next draw measure instead of reuse.

The flag is set after restore(), not before: building the model recomputes
it from what changed in the config, and would clear it again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 13:52:18 -04:00
7aaf017532 Sort the synthetic segment labels last, and drop their parentheses
Unordered, '(adjustment)' sorted *first* rather than last: '(' is 0x28 and
digits begin at 0x30, so it precedes '01 - Prior Year Sales'. The
parenthesised labels exist to mark a value as not a real segment, which is
exactly what an ordinal now does, so they get high ordinals and plain names
-- 98 - Unlabeled, 99 - Adjustments.

99 rather than max(seq) + 1 so the position does not shift every time a
segment is added or renumbered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 13:47:49 -04:00
5455d8089a Move the bucket order row and its datalist out of the table
Both were direct children of <table>, which is invalid: a browser hoists
stray non-table content out of the element, and in doing so it disturbed the
column widths of the segment listing below.

The datalist was already there before this change and had been getting away
with it; adding a visible div beside it is what made the consequence
obvious.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 13:43:09 -04:00
8c1f5b8f60 Use an ASCII separator: ExprTK string literals are byte-wise isprint
"01 · Prior Year" would not parse -- Invalid `expressions` "Bucket": Invalid
string token: 01. ExprTK's string scanner validates each *byte*:

    is_valid_string_char(c) = isprint((unsigned char) c) || is_whitespace(c)

"·" is U+00B7, two bytes 0xC2 0xB7 in UTF-8, and isprint(0xC2) is false in
the C locale. The literal fails at that byte and the parser reports from the
start of it, which is why the message names "01" rather than the character
it actually objected to.

" - " instead, which matches the "07 - Dec" already in the source data rather
than introducing a second convention.

The same rule applies to the label being compared, so a bucket or segment
named with anything outside printable ASCII is left unordered rather than
emitted into an expression that will not parse -- an unparseable expression
loses the whole column, not just that one case.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 13:36:55 -04:00
916229bdab Sequence the bucket and segment columns with expression columns
Perspective orders column groups by the value string, and SortDir's col asc /
col desc only reverses that -- so Prior Year -> Plan -> Actual -> Forecast is
expressible as neither, being alphabetical in neither direction. The order
has to be part of the value, as a "01 · " prefix.

Done as Perspective expression columns, named Bucket and Segment, generated
from pf.version.bucket_order and pf.log.seq. The first attempt computed the
prefix in the served SQL (archived on feature/column-sequencing-sql), which
was the wrong layer: the prefix is a pivot-ordering concern, and putting it
in the query put it in every other reader too -- the change log and the
bridge's basis list would both have read "04 · Forecast". It also meant a
Generate SQL to introduce the placeholder, and a reload to see any change.

As expressions it stays in the pivot, travels with saved layouts because it
lives in ViewConfig, and reordering on the Baseline page takes effect
immediately -- the expressions are rebuilt and the pivot re-renders, no
reload.

Kept from the SQL attempt: pf.version.bucket_order and pf.log.seq, which are
needed either way, and the Baseline controls -- a seq column per segment and
a reorderable row of bucket chips. bucket_order is on the version because
the Baseline page is version-scoped; Setup is the only source-level context
and not where anyone would look for this.

Anything unordered falls through to the raw column, so it sorts after the
ordered entries (digits before letters) rather than silently landing first.
The expressions are merged into the live config rather than replacing it, so
a user's own expressions survive, and an expression that stops existing is
dropped from the axes first -- restore() rejects a config that pivots on an
expression it no longer defines.

Needs 01_schema.sql for the two columns. No Generate SQL.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 13:33:12 -04:00
ebe0288202 Find the datagrid by asking the DOM, not by guessing the nesting
The rule was injected into perspective-viewer-datagrid's shadow root, which
is not where the cells live, so it applied to nothing and the grand-total
column stayed visible.

Locates regular-table by walking through shadow roots and injects into
whatever root contains it, via getRootNode(). That makes no assumption about
how the plugin nests -- which is the assumption that was wrong, and the kind
that breaks on a Perspective upgrade anyway.

The grid is built asynchronously after load, so hideSplitTotal now reports
whether it found anything and is retried once on a short delay, as well as
from the config-update handler.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 13:06:28 -04:00
484798cac5 Hide the grand-total column group in rollup mode
Rollup mode emits a grand-total column group alongside the subtotals. The
subtotals are the point -- they are what per-branch collapse needs -- but
the grand total sums across the split, and when the split is Prior Year /
Plan / Forecast that sum is meaningless while looking exactly like a real
figure to anyone scanning the sheet.

The engine cannot separate the two: t_totals is { BEFORE, HIDDEN, AFTER },
and in a rollup the grand total *is* the root of the hierarchy that produces
the subtotals. Turning it off turns the subtotals off with it. So it is
hidden rather than suppressed -- the view still computes the column, it is
simply not painted, and collapsed to zero width so there is no gap where it
was.

Targeted by psp-split-total, excluding psp-split-subtotal, which is how the
datagrid already distinguishes them. That survives adding a measure or
rearranging the pivot; hiding by column position would not -- the group
spans one column per measure in `columns`.

Injected into the plugin's shadow root, since a document stylesheet cannot
reach inside it, and re-asserted on config updates because the plugin
element is replaced when the plugin changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 13:03:46 -04:00
4d2af589b9 Document both axes collapsing through ViewConfig
§Row depth and the observer shim and §Column hierarchy described two
different mechanisms and an asymmetry between the axes that no longer
exists: rows were imperative view state chased by an observer shim, and
columns collapsed by truncating split_by. Both are a depth in ViewConfig
now, so the two sections are one.

Records what matters for reading the code: that the depths are 1-based
against server.cpp's `set_depth(depth - 1)`, that this needed the engine
patch in ui/vendor because apply_update dropped the fields, and that
Session::update_view_config returning early on "no change" is why nothing
happened at all rather than happening wrongly.

Keeps a short account of what was deleted -- the shim, the focus listeners,
the retry loop, splitFull, collapsingRef, the prefix test -- because someone
will otherwise wonder why the surrounding code looks so plain, and because
split_full is still read for layouts saved under the old scheme.

Per-node expansion moves to its own subsection with the reason it is hard
(set-only API, no getter) and what fixing it would take. The stale claim that
the selection is cleared on every column change is gone: with a depth the
dimensions a slice was cut from are all still present.

Also: observerShim.js dropped from the project layout, and the tech stack
now says 5.4.0 from the patched vendored build rather than 5.2.0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 12:38:18 -04:00
607dac221d Collapse the column axis with split_by_depth, like the row axis
Column collapse did not survive flipping away from the browser and back,
because it was the one piece of state still held outside the config:
applySplitDepth restored a *truncated* split_by, and a rebuilt view took its
split_by from the config while the levels it had dropped lived only in
splitFull.

split_by_depth is the config-level equivalent, and it works now that
apply_update applies it -- server.cpp does
ctx2->set_depth(HEADER_COLUMN, column_pivot_depth - 1), so it is 1-based
exactly like group_by_depth.

Everything that existed to support truncation goes with it:

- splitFull no longer has to outlive split_by, because split_by keeps every
  level. It is just the live axis, for rendering the buttons.
- split_full is no longer persisted beside the config. Still read on load,
  for layouts saved by the old scheme.
- collapsingRef and the prefix test are gone. They existed to tell our own
  collapse from the user rearranging the pivot, which only mattered because
  a collapse looked like a shorter split_by.
- The selection survives a collapse. It was cleared because slices name the
  split_by dimensions they were cut from and the axis was changing shape;
  with a depth those dimensions are all still there.

CLAUDE.md's §Column hierarchy describes the old mechanism throughout, and is
now wrong; updating it separately.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 12:13:51 -04:00
251692a3f9 Vendor a Perspective build that applies depth on a config update
Rebuilt from fleetside72/perspective at apply-depth-on-config-update, which
carries the patch ui/vendor has been describing: ViewConfig::apply_update
applied ten fields and neither group_by_depth nor split_by_depth, so a depth
set through restore() was dropped before the engine saw it. The new
perspective-js wasm is 81 bytes larger than the old, which is about right
for two calls and a generic.

Four things went wrong getting here, all of them host tooling rather than
the patch, and the script's preflight now catches three:

- pnpm 12 rejects a repeated `--if-present`, which sh_perspective.mjs emits
  once per package in scope. The tree needs pnpm 10; it pins no
  packageManager, so `npm i -g pnpm` gets something too new.
- `pip install cmake` gives 4.x, which clears the existing >= 3.29.5 floor
  and then fails in the Arrow build, because CMake 4 dropped support for
  cmake_minimum_required < 3.5.
- The pack step used `npm pack`, which leaves these packages' mutual
  `workspace:^` dependencies as-is; npm install then refuses the tarball
  with `Unsupported URL Type "workspace:"`. Now `pnpm pack`, which rewrites
  them -- and which is evidently how the previous tarballs were made, since
  theirs read `^5.4.0`.
- postinstall:playwright runs `playwright install --with-deps`, which
  apt-installs system libraries and needs root. Removed on the build branch.

npm normalised the vendor paths in package.json from ./vendor to vendor;
same meaning, left alone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 10:47:09 -04:00
f5d0f6b2f6 Carry a patch for the engine dropping depth on a config update
ViewConfig::apply_update in perspective-client applies ten fields and
neither group_by_depth nor split_by_depth is among them. A depth therefore
works when a view is created -- table.view({ group_by_depth: 1 }), which is
what the fork's own depth_test.mjs exercises -- and is silently discarded by
restore(), which is how a viewer changes its own configuration. That is why
the EXPAND buttons did nothing however the value was sent.

group_by_depth and the omission are both upstream; the fork mirrored
split_by_depth alongside it faithfully, including the omission. So the column
axis expand/collapse the fork added has the same hole, and pf_app only avoids
it by collapsing columns through a truncated split_by instead.

The patch adds an Option-aware sibling to _apply and applies both fields. It
cannot be built here -- protoc is absent, so the generated protobuf modules
are missing and the crate does not compile for unrelated reasons -- but
cargo check reports nothing against view_config.rs. It sits in ui/vendor with
the rebuild instructions, beside the tarballs it is not yet in.

Meanwhile applyDepth reads the config back after restoring and falls back to
view.set_depth() when the value did not stick, so the buttons work on the
current build. The fallback loses depth on a view rebuild, as it always did;
what is not back is the shim and listeners that used to chase that. The check
clears itself once the engine honours the field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 03:29:28 -04:00
744342d519 Count row depth the way group_by_depth does — levels, not boundaries
The EXPAND buttons did nothing because their 0..3 were written against
view.set_depth(), which is 0-based, while the config field is 1-based.
server.cpp does

    ctx1->set_depth(row_pivot_depth - 1)     // one-sided
    ctx2->set_depth(HEADER_ROW, row_pivot_depth - 1)

for both contexts, so group_by_depth counts the levels to show. Passing 0
asked the engine for set_depth(-1), and every other button was off by one
level. The fork's own depth_test.mjs uses group_by_depth: 1 as its working
control, which is the same arithmetic seen from the outside.

So the toolbar sends d + 1, and subtracts one again when reading a saved
layout back for the button state.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 03:22:42 -04:00
02a0db386e Send the whole config when changing row depth
A partial restore({ group_by_depth: d }) had no effect -- the buttons did
nothing, and the depth appearing to survive every rebuild was simply a tree
that had never been collapsed. So the full config goes back with the one
field changed.

table is dropped from it: the table is loaded by reference, and a stale name
in a restored config fails the lookup -- the same reason every other restore
in this file strips it.

If this still does nothing then group_by_depth is not applied when the view
is constructed, and the fix belongs in the fork beside split_by_depth.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 03:12:38 -04:00
033176eb43 Set row depth through ViewConfig instead of restoring it after every rebuild
ViewConfig carries group_by_depth -- alongside the split_by_depth this fork
added -- so row depth can be set declaratively:

    await viewer.restore({ group_by_depth: d })

The config is what the viewer rebuilds its view from, so the depth survives
every rebuild by construction, and viewer.save() carries it into the
persisted and named layouts for free.

It had been imperative: getView() then view.set_depth(), which puts the
depth on an object the viewer discards whenever it re-renders. Everything
that grew around that existed only to guess when a rebuild had happened and
put the depth back -- a shim patching window.IntersectionObserver and
window.ResizeObserver, focus/visibilitychange/pageshow listeners, a retry
loop for getView() throwing "No table set" while getTable() resolved, a flag
tracking whether the viewer had "gone away", and tracing to debug all of it.

That guesswork caused three separate visible faults in a single session: the
tree fully expanding on refocus, snapping on any reflow, and snapping when
Perspective's own settings sidebar was opened. Each fix was a finer
heuristic about which browser event meant what, which is the shape of
fighting a framework rather than using it.

312 lines out, 35 in. applySplitDepth no longer re-applies the row depth
after truncating split_by, because the rebuilt view brings it along.
expand_depth stays readable on load: it is the legacy key, from when the
depth had to be stored beside the config rather than in it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 03:05:36 -04:00
8aa0ae2ece Stop the pivot snapping when its settings sidebar is opened
Two causes, both mine, both triggered by adjusting the layout.

A zero size was being treated as the viewer going away, and opening
Perspective's settings sidebar collapses the datagrid for a frame. The
return then re-applied the stored row depth over whatever had been expanded
by hand. Only lost intersection counts as a departure now; a zero size is a
transient of layout, not an absence.

And perspective-config-update fires for anything in the config, the settings
flag included -- not only for split_by. While collapsed the live split_by is
a truncation of the full hierarchy, so adopting it recorded the truncation
as the hierarchy and put the deeper levels out of reach. A genuine
rearrangement is not a prefix of what we already hold, which is the
difference the handler now tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 02:58:10 -04:00
7cfd0068e7 Restore row depth only after the viewer has actually gone away
Driving the re-apply from the observer callbacks caught the refocus case but
fired on far more than that: ResizeObserver reports every reflow, so
dragging the panel, a scrollbar appearing, or anything that changed the
layout re-applied the stored depth and discarded whatever had just been
expanded by hand. From the outside that is the pivot snapping to a different
layout while clicking around.

Going away is what discards the view; a reflow is not. So the callbacks now
record the departure -- lost intersection, collapsed to zero size, or a
hidden document -- and only a return after one of those restores anything.
The flag clears once the depth is back, so one departure causes one restore.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 02:55:39 -04:00
507cae4e51 Lay the adjustment panel out on a grid, and count what a clone will copy
The clone form was four controls whose labels ran to different lengths --
"copy rows from", "scale cloned rows by", "tag", "note" -- each row starting
wherever its label happened to end, with a preview sentence floating after
them and a row of unexplained chips between the tag and the note. Nothing
separated where the rows come from, what happens to them, and what the
change is called.

Every row now shares a label column, so the controls line up, and the three
groups carry headers in the same 10px uppercase the ledger headers already
use. The tag suggestions sit inside the tag's own column, where they read as
values for it. Hints appear under the control they qualify and only when
relevant.

Also: the preview said "Copying 0.00 sales_usd" while five AOP rows worth
68,904 were selected. It reported the adjustable total, which is zero when
the selection is entirely prior year or plan -- the case clone exists for,
now that it can read reference rows. It counts them for clone, and still
does not for recode, which cannot touch them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 02:46:05 -04:00
cc3d83268f Show the clone date offset without requiring a segment
The offset field was rendered only when a segment had been named, from when
that was the only way clone could reach reference rows. Once clone could read
them directly the field became unreachable in the ordinary case -- so
"shift dates by" was invisible and the only typeable-looking control was
"copy rows from", which is a dropdown.

It is shown whenever clone is open now, and the payload carries the offset
independently of from_logid: shifting a selection through time is the common
case, and narrowing to one segment is the occasional one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 02:40:58 -04:00
ee4a60475e Type the segment date offset as an interval, not year and month spinners
The Date offset on a baseline or reference segment was a pair of
type="number" inputs, years and months, both clamped at min 0. So no
characters could be typed, days could not be expressed at all, and a segment
could only ever be shifted forward in whole months -- while the stored value
is a Postgres interval that happily accepts "4 months", "1 year" or
"-90 days". parseOffset only read year and month, so anything else would not
have survived a round trip through the edit form either.

One text field now, with suggestions, matching what clone already offers.
parseInterval parses it far enough to draw the timeline preview; Postgres
stays the authority, and assertInterval -- now shared by the baseline,
reference and clone routes -- rejects what it will not accept, with a
message naming the field rather than a parse error from inside a CTE.

Timeline takes months and days separately rather than years and months,
because the two are not interchangeable: a month shift lands on the same day
of another month, a day shift can cross a month boundary. It adds months
then days, as Postgres does, and its label handles negatives.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 02:36:50 -04:00
8f96fa3f7f Let clone read reference rows — copying them out is the point
A selection of five AOP rows cloned nothing, with no explanation. The cause
was exclude_iters, which clone applied along with scale and recode.

It should not. That exclusion exists to stop operations *modifying*
reference rows: scale distributes an increment across its pool, so including
reference would attribute forecast movement to prior-year rows, and recode
writes negative rows that zero the original out. Clone does neither -- it
reads rows and inserts new pf_iter = 'clone' rows, leaving the source
untouched. Copying a plan or a prior year out of reference and into
adjustments is the operation doing exactly what it is for.

So from_logid stops being the only way to reach those rows and becomes what
it should be: a narrowing, for a selection spanning AOP and Prior Year where
only one is wanted.

The "would just duplicate the rows" guard no longer fires when the selection
is entirely non-adjustable, since moving rows from reference into adjustments
changes what they are even at factor 1 with no shift.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 02:33:28 -04:00
13e49c14b6 Offer negative clone offsets, and validate the interval before using it
Shifting backwards already worked -- the offset is added as a Postgres
interval, and '-90 days' is a perfectly good one -- but the suggestions only
listed forward shifts, so nothing said so. Pulling a plan back a quarter is
a normal thing to want.

A day-level shift is also the case that most needs the dim_period
derivation: 15 Mar 2027 less 90 days is 15 Dec 2026, which crosses from
2027/10 - Mar into 2027/07 - Dec. Copying the period columns across would
have labelled it March.

The offset is interpolated into the statement, so a typo surfaced as a
Postgres parse error from the middle of a CTE. It is parsed on its own
first, where the failure is cheap and can name the field.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 02:11:04 -04:00
0a50122add Let a clone proceed on a date shift with no dimension override
The panel required at least one override for both recode and clone. For
recode that is right -- rewriting rows as themselves does nothing. For clone
it is not: copying prior year forward changes the dates and, through
dim_period, the season and month dimensions, which is the entire point of
cloning from a reference segment. The server-side requirement was dropped
when from_logid went in; this one was missed, so the operation was blocked
in the UI with "Enter at least one override value".

Clone now only has to be doing something -- an override, a shift, or a
factor other than 1 -- and says so when it is doing none of the three.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 02:09:28 -04:00
5acac2738a Build the bridge on buckets, and let it start from a comparison basis
The bridge filtered on exclude_iters, so it dropped every reference row --
including Open Orders, which is loaded as reference precisely so nothing
adjusts it and is still part of the forecast. Membership is pf_bucket now,
which is the axis that answers this question; pf_iter answers a different
one.

What you compare against depends on what you are building: an AOP is built
off a prior period, a forecast off an update to the AOP. So the basis is
chosen rather than assumed, and the walk stays exact either way because

    Forecast - Basis = (Forecast loads - Basis) + adjustments

The opening step is that first term, every tagged adjustment explains the
rest, and the bars sum to the endpoint by construction. No residual to
explain away -- which is what a bridge from prior year to forecast would
otherwise be, since the adjustments describe movement from the forecast's
own loads and not from last year.

With no basis it is the composition instead: the forecast's loads as the
opening anchor, then the adjustments. The picker only appears once something
carries a bucket, so a version that has not been labelled behaves as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 01:46:40 -04:00
1904428fbb Say that a clone's selection is the source, not the destination
The slice filters the rows being copied, so cloning prior year forward means
selecting last December and shifting it, not selecting the December you want
to fill. Selecting the destination matches nothing -- prior year's rows are
labelled with prior year's periods, so a 2027 slice and a 2026 segment share
no rows -- and it fails silently, as zero rows cloned.

Nothing in the form said which way round it was, and "copy from <segment>"
reads like the selection is the target. It now says the selection is what
gets copied, and the segment narrows it rather than replacing it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 01:37:33 -04:00
518a0ca5ba Clone from a named segment, shifted, with period dimensions re-derived
Three changes, one feature: a period with no baseline borrows a shape from
prior year or plan.

The generator handled one date group. sql_generator.js used find(), so a
source with order, requested and ship date groups derived the order one and
copied the other two raw -- shifted dates against unshifted period labels.
Every group now gets its own join, and they are LEFT rather than inner: an
order that has not shipped has no ship date, and an inner join would have
dropped the row from the load entirely rather than leaving its period
columns empty. dateGroupsOf() is now shared, like grainOf, so the routes and
the generator agree on membership and on join aliases.

Clone carries every date column rather than only the primary one, since it
is the operation that moves rows through time, and re-derives the period
dimensions from pf.dim_period against the shifted date instead of copying
them from the row being cloned.

from_logid names the segment to copy from, replacing the exclude clause for
that one entry rather than widening it. Rows written stay pf_iter = 'clone',
so scale applies to them afterwards as a second step.

set is no longer required: copying a segment forward unchanged is a
reasonable thing to ask for.

Needs dim_period_col set in Setup (fisc_year on oseas/rseas/sseas_e,
fisc_month_abbr on omon/rmon/smon_e) and Generate SQL.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 01:21:29 -04:00
2689e95b2c Count adjustments as Forecast rather than their own banner
pf_bucket mirrored pf_segment for scale/recode/clone rows, so adjustments
showed as '(adjustment)' -- a banner of their own, sitting outside the
forecast they are adjustments to. Grouping by pf_bucket therefore split the
forecast in two.

An adjustment is always part of the forecast: exclude_iters keeps operations
off the reference segments, so there is no adjustment that is not. It falls
back to 'Forecast' now, and still yields to an explicit bucket on its own
log entry.

pf_segment keeps '(adjustment)', which is right there -- it answers which
segment a row came from, and an adjustment came from none.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 01:03:34 -04:00
0049a391c0 Let a segment say what it counts toward, and pivot on it
There was no way to express "these segments together are the forecast".
pf_iter cannot say it: it answers whether operations may write to a row, and
Open Orders is loaded as reference precisely so nothing adjusts it while
still being part of the forecast number. The two questions are independent,
so one cannot be derived from the other.

pf.log.bucket is the second axis. Free text with suggestions -- Forecast,
Prior Year, Prior Prior Year, Plan -- rather than an enum, so another banner
needs no migration. Blank by default, falling back in the pivot to the
segment's own name, so nothing changes until something is labelled.

/data and /agg emit it as pf_bucket beside pf_segment, through the pf.log
join that is already there. Set it per segment in the Baseline list, which is
where loads live now that the change log only shows adjustments.

Needs 01_schema.sql for the column and Generate SQL for /agg to select it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 00:59:00 -04:00
15d9ecf319 Fetch col_meta in the member-list effect instead of reading a ref
The effect decided which dim_groups to load from colMetaRef.current, but
that ref is filled inside initViewer, which is async. The effect ran first,
saw an empty array, found no groups, and fetched nothing -- and a ref
changing does not re-run an effect, so it never recovered. dimMembers stayed
{} for the whole session.

Everything therefore fell back to the source lookup, which is precisely the
query pf.dim_member exists to replace: XCP06500G18B112 has two attribute
combinations across history, so DISTINCT ... LIMIT 2 returned two rows and
the route answered null. The member row had the answer all along, picked as
the most recent by order date.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 00:53:12 -04:00
708e5662ab Correct what is_key does, and record the trap in it
CLAUDE.md said is_key "marks dimensions used in slice WHERE clauses", which
is not true and never was: assertSelective and buildWhere validate against
filterCols, which is every dimension plus every date column regardless.

What it actually drives is the key of a dim_group, value completion, the
sibling autofill trigger, and the dim_period anchor -- and the first of
those wants exactly one column per group while the others want several. When
a group has more than one, find() silently takes the lowest opos, which is
how segment_new outranked part and a master-data refresh keyed on a column
that is null throughout: zero members, reported as success.

Written down because the failure gives no signal at all -- the refresh
succeeds, the list is simply empty, and every symptom appears somewhere
else entirely.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 00:47:20 -04:00
89026e3440 Replace sibling attributes on a key change, and say when none were found
Autofill filled only empty boxes, so it appeared to work exactly once. Type a
part, get its nine attributes; type a different part, and every box is
already full, so nothing updates and the form still describes the previous
part. That is worse than blank -- the recode would have been submitted with
one part's code and another's attributes.

These columns describe the key that was just entered, so they are replaced
outright now.

A lookup that finds nothing also said nothing, which is indistinguishable
from a lookup that did not run. It warns, and names the value it could not
find, so an unknown part reads differently from a broken autofill.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 00:40:27 -04:00
a3550eabc3 Fall back to the source when the member list misses
Autofill stopped filling anything as soon as the member endpoint answered,
because a hit was treated as the only possible answer: if the key was not in
the list, it returned rather than trying the source. An empty list counted as
an answer too, which is the state while a refresh is still running -- so
wiring up master data broke the behaviour it was meant to improve, for
everyone who had not finished building the list yet.

A miss is not an answer. Empty list, or a key the list does not know, both
fall through to the source lookup as before. It costs one request, on a path
that only runs when someone types a value.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 00:31:54 -04:00
07d92ccd74 Keep master data for a dim_group instead of re-deriving it from the source
Every question about a part -- what values exist, what attributes go with
one -- was answered by querying the source, and the source is the wrong
place to ask. It is a view over a transaction table, so the query is slow
(76s for one ILIKE against 6.9M rows), it describes only what was
transacted, and it cannot express intent: there is no way to say a part is
discontinued, or to name one that has not sold yet.

pf.dim_member holds the app's own list: one row per key value per group,
siblings in jsonb, keyed on (source_id, dim_group, key_value). Refresh is a
merge rather than a replace, so curation survives it -- members absent from
the source are marked source_seen = false, not deleted. Triggered from
Setup, next to Generate SQL, because it reads the whole source and the
answer only changes when the catalogue does.

A key can carry several attribute sets across history -- 11,290 parts
against 13,662 combinations on osm_skinny -- so the refresh takes the most
recent by the source's date column. That also fixes the sibling autofill,
which used to run a DISTINCT ... LIMIT 2 against the source and silently
fill nothing whenever a part came back ambiguous. A member row is one
definition by construction.

The client fetches each group's list once per source and does both
completion and autofill against it in memory, so neither costs a request.
Columns outside a group, or a group never refreshed, still fall back to the
version's values endpoint.

Run 01_schema.sql to create the table.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 00:19:39 -04:00
401b6b9b42 Define sliceUnits, which recode and clone have always called
55814ee introduced calls to sliceUnits() in the recode and clone routes
without ever defining it, so both have thrown "sliceUnits is not defined"
since that commit. Scale worked only because it inlined the same three lines
rather than calling the helper.

Defined from scale's copy, and scale now calls it too. Recode and clone
passed req.body.apply_mode straight through, where scale normalised it
first, so an absent or unrecognised value would have fallen to the 'each'
branch by accident; they normalise the same way now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 00:11:35 -04:00
812678bb7f Complete recode values from the version, not the source view
Pointing completion at the source made every keystroke a 76-second query:
gs.osm_skinny is a plain view over rlarp.osm_stack, so ILIKE '%1601%'
scanned 6.9M rows and read 1.3M buffers off disk, 64s of it I/O. Nothing had
called that endpoint before, so the cost only appeared once a debounced
input was wired to it.

The source is the wrong list anyway. It reaches back over all of history and
would offer parts discontinued years ago; the version holds what was
actually loaded, which is what the forecast is being written against.

So GET /versions/:id/values/:col reads the version's own forecast table --
2.0s for 11,290 parts on fc_osm_skinny_29 -- and holds the result in memory,
keyed on that version's latest pf.log id. Any load, adjustment or undo moves
the id and the next request rebuilds, so nothing has to remember to
invalidate. Filtering happens over the cached array, so typing costs one
small max(id) query.

The column name is interpolated into the DISTINCT, so it is checked against
col_meta first.

The source-side endpoint keeps its new q/limit but no longer has a caller;
its ILIKE path against a view is the expensive one and should stay unused.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-17 00:03:05 -04:00
b65da53360 Complete key dimension values as you type them in recode and clone
Recoding to a part meant typing a code from memory into a plain text box,
with the only feedback being that the sibling autofill either fired or
silently did nothing.

The values endpoint already existed but returned every distinct value with
no filter and no limit, which for part on osm_skinny is 11,290 rows -- too
slow to open and no easier to read than a short list. It now takes ?q= and
?limit=, so the field fetches matches for what has been typed so far,
debounced 200ms, and offers them through a native datalist.

Only key columns get it, which is the same condition the endpoint already
enforced, and the same one that decides whether the dim_group sibling
lookup runs on blur. So on osm_skinny it is part and customer: type 1601
and the eight parts containing it are offered; pick one and the nine
part-group columns fill themselves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 23:55:37 -04:00
088b6a30c5 Document the plug, the segment columns, and the observer shim
Three things a reader would otherwise have to rediscover, and one of them is
load-bearing: observerShim.js has to be main.jsx's first import or it
silently intercepts nothing, which no amount of reading the shim itself
tells you. Also records the two false trails found while getting there --
getTable() resolving while getView() throws, and getView() returning a fresh
wrapper every call so identity cannot detect a rebuild.

Known issues rewritten against what is actually true now: the operation
panel wiring and the progress-bar throttle are done, and the load-time entry
now says what was measured -- that the cost is row count, not payload, and
dynamic grain is the remaining lever.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-16 23:48:25 -04:00
35 changed files with 4298 additions and 839 deletions

426
CLAUDE.md
View File

@ -14,7 +14,7 @@ Data transport architecture options: `pf_perspective_options.md`
- **Backend:** Node.js / Express (`server.js`)
- **Database:** PostgreSQL — isolated `pf` schema
- **Frontend:** React + Vite + Tailwind CSS in `ui/`; built output lands in `public/app/`
- **Pivot:** [Perspective](https://github.com/perspective-dev/perspective) (`@perspective-dev/*` distribution, **not** FINOS `@finos/perspective`) 5.2.0, **bundled inline via the `/inline` entrypoints — never from a CDN** (the 4.x CDN bundle resolves its server WASM to an unversioned path and silently pulls whatever is newest). See `PERSPECTIVE.md`.
- **Pivot:** [Perspective](https://github.com/perspective-dev/perspective) (`@perspective-dev/*` distribution, **not** FINOS `@finos/perspective`) 5.4.0 from a **patched build vendored in `ui/vendor`** — see its README for what is patched and which host-tool versions the rebuild needs; **bundled inline via the `/inline` entrypoints — never from a CDN** (the 4.x CDN bundle resolves its server WASM to an unversioned path and silently pulls whatever is newest). See `PERSPECTIVE.md`.
- **Dev:** `npm run dev` (nodemon) in root; `npm run build` in `ui/`
---
@ -30,6 +30,7 @@ routes/
versions.js Version CRUD, baseline/reference load, data stream
operations.js scale, recode, clone, undo — the core forecast ops
log.js GET /api/versions/:id/log, DELETE /api/log/:logid
layouts.js Named pivot layouts — list per version, create, patch, delete
lib/
sql_generator.js buildFilterClause, token substitution helpers
auth.js scrypt hash/verify, requireAuth, sessionUser; `node lib/auth.js hash` CLI
@ -45,6 +46,7 @@ ui/src/
Baseline.jsx Version management, baseline workbench, reference load
Forecast.jsx Perspective pivot, selection handling, operation dispatch
components/
LayoutMenu.jsx The Layout ▾ control — Published / Mine, with the write actions
OperationPanel.jsx The adjustment workbench — ledger + scale/recode/clone forms
BridgeView.jsx Baseline → current waterfall by tag (exports buildSteps/layoutSteps)
Sidebar.jsx 3-step collapsible nav
@ -57,15 +59,43 @@ ui/src/
## Database schema (`pf`)
- **`pf.source`** — registered source tables
- **`pf.col_meta`** — column roles: `dimension` | `value` | `units` | `date` | `filter` | `ignore`; `is_key` marks dimensions used in slice WHERE clauses; `dim_group` groups functionally dependent columns (e.g. date + its derived year/month dimensions); `dim_period_col` maps a dimension to a `pf.dim_period` column so date-adjacent values are derived at load time rather than copied raw; `in_grain` flags dimension/date columns that define the **display grain** (see below)
- **`pf.col_meta`** — column roles: `dimension` | `value` | `units` | `date` | `filter` | `ignore`; `dim_group` groups functionally dependent columns (e.g. a part and its attributes, or a date and its derived year/month dimensions); `dim_period_col` maps a dimension to a `pf.dim_period` column so date-adjacent values are derived at load time rather than copied raw; `in_grain` flags dimension/date columns that define the **display grain** (see below); `is_key` is described under §`is_key` below
- **`pf.version`** — named forecast scenarios; `exclude_iters` (default `["reference"]`) blocks those iter values from all operations
- **`pf.fc_{tname}_{version_id}`** — one forecast table per version; contains both operational rows (`pf_iter = baseline|scale|recode|clone`) and reference rows (`pf_iter = reference`)
- **`pf.fc_{tname}_{version_id}`** — one forecast table per version; contains both operational rows (`pf_iter = baseline|scale|recode|clone`) and reference rows (`pf_iter = reference`). Indexed on `pf_logid`, which is how undo, the change-log aggregate and the grain key all find their rows — without it each is a sequential scan of the whole table. Tables created before that index was added do not have it.
- **`pf.log`** — audit log; every write gets one entry; `slice` + `params` stored as jsonb
- **`pf.layout`** — named Perspective view configs; see §Pivot layouts
- **`pf.sql`** — generated SQL templates per source/operation; tokens substituted at request time
- **`pf.app_user`** — login accounts; scrypt `pass_hash`, `is_active`, `last_login_at`
- **`pf.session`** — express-session store (connect-pg-simple layout)
- **`pf.dim_period`** — calendar lookup table (20182035); one row per month keyed on `sdat` (month start date); provides cal/fiscal year, quarter, and month columns; populated by `setup_sql/gen_dim_period.sql` with a configurable fiscal year start month
### `is_key`
Read in four places, and **not** the one it sounds like — slices are validated
against `filterCols`, which is every `dimension` plus every `date` column
regardless of `is_key`.
1. **The key of a `dim_group`**`resolveGroup()` in `routes/sources.js` takes
`members.find(c => c.is_key)`, the column every other member is keyed on for
`pf.dim_member`
2. **Value completion** — only `is_key` columns get a dropdown, and
`/sources/:id/values/:col` refuses anything else
3. **Sibling autofill** — fires on blur only when `is_key && dim_group`
4. **The `dim_period` anchor**`role === 'date' && is_key && dim_group` picks the
date whose siblings are derived from the calendar
Uses 2 and 3 want several columns flagged; use 1 needs exactly one per group.
**When a group has more than one, `.find()` silently takes the lowest `opos`.**
That is not hypothetical: `segment_new` (opos 12) outranked `part` (opos 15) in
the `part` group, so a refresh keyed on a column that is null throughout, matched
nothing, and reported success with zero members. `customer`'s group has four keys
and picks the right one only by `opos` luck.
The two meanings want separating — a per-group key choice, or a rule that the
group key is the column named by the group (which these groups nearly follow
already, except `sdate``sdate_e`). Until then, a refresh that finds two keys
should refuse rather than guess.
### Key token substitution tokens
`{{fc_table}}`, `{{where_clause}}`, `{{exclude_clause}}`, `{{logid}}`, `{{pf_user}}`, `{{value_incr}}`, `{{units_incr}}`, `{{pct}}`, `{{set_clause}}`, `{{scale_factor}}`, `{{date_offset}}`, `{{filter_clause}}`
@ -86,14 +116,257 @@ Either way: Arrow IPC binary stream → `worker.table(buffer)` in Perspective WA
### Display grain
Aggregating to the grain the pivot actually displays is the load-time fix — measured 534,902 → 6,154 rows on `osm_stack`. It keeps the **native** Perspective engine, so expand/collapse/depth/sort/filter all still work. Set the grain in Setup (`in_grain` per column); it is baked into `pf.sql` at Generate SQL time so load and operations agree. `grainOf()` in `lib/sql_generator.js` is the single definition of what the grain is — `Setup.jsx` and `routes/log.js` mirror it. Full design: `pf_spec.md` → §Display-grain pre-aggregation. Why not a DuckDB virtual server: `pf_perspective_options.md` → §Spike findings.
### Segment and note columns
`/data` and `/agg` both LEFT JOIN `pf.log` and emit two columns the forecast table
does not itself carry:
- **`pf_segment`** — `pf.log.label`, else `tag`, else `note`, else `Unlabeled`;
`'99 - Adjustments'` for an adjustment that has no label of its own
- **`pf_note`** — the free text on a scale/recode/clone; null on loads
They are deliberately separate: commingling a segment name with an adjustment note
makes neither pivotable. In grain mode `pf_logid` is part of the grain, so the join
adds no rows. The operation routes stamp the same two fields onto the rows they push
back incrementally, since those come from `RETURNING *` and would otherwise arrive
without them.
### Column order is stored text
Perspective orders column groups by the value string, and `SortDir`'s `col asc` /
`col desc` only reverses that — so Prior Year → Plan → Actual → Forecast is
alphabetical in neither direction and expressible as neither. A `"01 - "` prefix
is the only lever, and it lives in **`pf.log.label`** (and `pf.log.bucket`),
typed by whoever names the segment. Nothing derives it.
`SEGMENT_EXPR` / `BUCKET_EXPR` / `NOTE_EXPR` in `lib/sql_generator.js` are the
single definition, shared with the `/data` cursor in `routes/operations.js`
`/agg` is generated, `/data` is not, and the two have to agree.
The one hardcoded ordinal is `ADJUSTMENT_SEGMENT` = `'99 - Adjustments'`, which
keeps unlabelled adjustments after every *numbered* segment. That proviso is the
whole scheme, not a caveat on it: ordering is string ordering, so `99` only lands
last once the loads carry `01``0n`, and an unnumbered segment sorts after it
(digits precede letters — `9` is `0x39`, `A` is `0x41`). The old `'(adjustment)'`
sorted *first* for the same reason read the other way, `(` being `0x28`.
Unlabelled loads read plain `Unlabeled` and so land at the very end, which is
where a segment nobody has named belongs. Labelling an adjustment's own log row
overrides the fallback, which is how one kind of adjustment is split out from the
rest.
### Hardcoded display names
Every name the pivot can show that does not come from `pf.log`. If a segment or
bucket appears under a name nobody typed, it is one of these. All three are in
the `DISPLAY DEFAULTS` block at the top of `lib/sql_generator.js`, exported so
the `/data` cursor and the operation routes' incremental row stamps use the same
values the generated `/agg` does.
| constant | `pf.version` column | built-in | applies to |
|---|---|---|---|
| `ADJUSTMENT_SEGMENT` | `adjustment_segment` | `99 - Adjustments` | `pf_segment` for a scale/recode/clone with no `label` |
| `ADJUSTMENT_BUCKET` | `adjustment_bucket` | `04 - Forecast` | `pf_bucket` for a scale/recode/clone with no `bucket` |
| `UNLABELED_LOAD` | `unlabeled_load` | `Unlabeled` | `pf_segment` and `pf_bucket` for a load with no `label`, `tag` or `note` |
Each is set per scenario on the Baseline page, under **Fallback names**; blank
falls back to the built-in. Anything typed on the log row overrides both, so
none of these appears once a segment is named.
**Why the join rather than a token.** `pf.sql` is keyed on
`(source_id, operation)` — one template shared by every version of a source — so
a value baked in at Generate SQL time could not vary by version, and
regenerating for one version would silently change the others. The names are
therefore read through `VERSION_JOIN` at query time, which also means changing
one takes effect on the next load with nothing regenerated.
The built-ins are still a convention guess: `ADJUSTMENT_BUCKET`'s `04 - ` only
suits one numbering. A version that numbers its buckets differently sets its
own rather than inheriting that.
**What this replaced.** The prefix used to be computed client-side, as
Perspective expression columns (`pf_bucket_ord`, `pf_segment_ord`) built from
`pf.log.seq` and `pf.version.bucket_order`. It ordered the pivot and nothing
else, so every other reader disagreed with it; `restore()` replaces
`expressions` wholesale, so it had to be re-applied after every layout load; and
ExprTK's string scanner tests each *byte* with `isprint()`, so a label
containing anything outside printable ASCII could not be ordered at all (`·` is
two bytes, of which `isprint(0xC2)` is false). `DEAD_ORDER_EXPRS` in
`Forecast.jsx` strips the expression names out of layouts saved under that
scheme. `pf.log.seq` and `pf.version.bucket_order` are no longer read; the
columns remain.
Relabelling now needs a page reload to show, because the label is part of the
aggregated row rather than something the pivot can re-derive. Editing a label
afterwards is a `PATCH /api/log/:logid` and needs nothing regenerated, but a
source registered before this change needs **Generate SQL** run once, so its
stored load templates write `label` and `bucket` onto the log row at all.
### Forecast operations
POST to `/api/versions/:id/{scale|recode|clone}` → SQL executed with `RETURNING *` → new rows returned as JSON → `pspTable.update(rows)` — no full reload. In grain mode the operation's final CTE aggregates its own new rows to grain first; since `pf_logid` is part of `pf_gkey` those keys are always new, so `update()` **appends** and the view re-sums.
### Price or volume (`plug`)
A sales figure alone does not say which of price or volume moved, so scale takes
`plug``'price'` (default, volume holds) or `'volume'` (price holds, units scale
in proportion). Resolved in `resolveIncrs()` in `routes/operations.js`; the panel
only offers it when the edit is dollars-only, because naming units or price has
already answered it. The semantics come from the predecessor Excel model,
`/opt/forecast_api/VBA/fpvt.frm``calc_val` / `calc_price`:
```
plug volume: pchange = fVal/(pVal+bVal); fVol = (pVol+bVol)*pchange
plug price: fVol = pVol + bVol
```
A `target_price` with a `target_units` alongside is that form's Edit Price mode,
where both are inputs and dollars fall out. The ledger's **Result** line previews
value, units and price together using the same rules, so what you see is what
gets written.
### Undo
`DELETE /api/log/:logid` → removes rows by logid → `table.remove()` of the affected index values (`pf_gkeys` in grain mode, `pf_ids` in raw mode); the view re-sums. No full reload.
---
## Axis depth (collapse / expand)
Both axes collapse the same way: a **depth in `ViewConfig`**, set through
`restore()`.
- **Rows**`group_by_depth`, driven by the `EXPAND 0 1 2 3` buttons via
`applyDepth()`
- **Columns**`split_by_depth`, driven by the `COLUMNS` buttons via
`applySplitDepth()`
Both are **1-based**: they count the levels to show, where the imperative
`view.set_depth()` counts the boundary below them. `server.cpp` does
`ctx1->set_depth(row_pivot_depth - 1)` and
`ctx2->set_depth(HEADER_COLUMN, column_pivot_depth - 1)`, so the toolbar sends
`d + 1` and subtracts one again when reading a layout back.
Because a depth lives in the config, it survives every view rebuild, rides into
the persisted and named layouts through `viewer.save()`, and needs nothing
re-applied afterwards.
**This required patching the engine.** `ViewConfig::apply_update` applied ten
fields and neither depth, so a depth sent through `restore()` was accepted,
deserialized and dropped — and since `Session::update_view_config` returns early
when `apply_update` reports no change, no view was rebuilt at all. See
`ui/vendor/0001-apply-depth-fields-on-config-update.patch`; the vendored build
carries it.
### What this replaced
Worth knowing, because a lot of machinery existed to work around it and is now
gone:
- Row depth used to be imperative — `getView()` then `view.set_depth()` — which
put it on an object the viewer discards whenever it re-renders. Restoring it
meant guessing when that had happened: an `observerShim.js` patching
`window.IntersectionObserver` and `window.ResizeObserver`, focus and
visibility listeners, a retry loop for `getView()` throwing `No table set`
while `getTable()` resolved, and a flag tracking whether the viewer had "gone
away". That guesswork produced three distinct visible faults — the tree fully
expanding on refocus, snapping on any reflow, and snapping when Perspective's
settings sidebar opened.
- Column collapse used to restore a **truncated `split_by`**. The discarded
levels therefore had to be remembered separately (`splitFull`, persisted as
`split_full`), our own collapse had to be told apart from the user rearranging
the pivot (`collapsingRef` plus a prefix test), and the selection was cleared
on every collapse because the axis was changing shape.
`split_full` is still *read* on load, for layouts saved under the old scheme.
### Auto-pause is off
`<perspective-viewer>` auto-pauses by default: an `IntersectionObserver` on
itself plus the document's `visibilitychange` drive `AutoPauseState::apply()`,
and pausing **deletes the view** (`session.set_pause(true)` →
`view_sub.take().delete()`). Returning to the tab is therefore not a redraw but
`restore_and_render()` — a fresh view and a fresh traversal of the whole grain,
which on a large one is a multi-second chug on every tab switch, and takes any
per-node expansion with it.
`initViewer()` calls `viewer.setAutoPause(false)` right after `viewer.load()`.
Nothing updates the table while the tab is hidden — every operation is driven
from this page — so the pause bought nothing and cost a rebuild. The view is now
held while the tab is backgrounded.
### Still not solved: per-node expansion
Expanding one specific branch is view state with no config representation, and
the API has **no getter**:
```
expand(row_index: number): Promise<number>
collapse(row_index: number): Promise<number>
```
It can be set but not read, so it cannot be captured and replayed — this is why
it is lost on every rebuild, and why no client-side fix has worked. The honest
route is a `ViewConfig` field carrying expanded row *paths* (indices shift as
the tree opens), applied in `server.cpp` where the depths are. Bigger than the
depth patch: it needs a way to enumerate expanded nodes in the C++ traversal, a
proto field, and the apply step.
The fork's own `header_click.ts` / `expand_column` / `collapse_column` is the
column-axis equivalent and has the same limitation.
**Limitation that remains either way:** depth is whole-axis. Excel can collapse
2025 while 2026 stays expanded; a depth collapses every group at that level
together. `columns` selects which *measures* appear, not individual split
combinations.
## Pivot layouts
A layout is a named `ViewConfig`, stored in **`pf.layout`** and owned by an
account. Two kinds, one table:
- **published** — everyone on the forecast lists it and can apply it; only its
owner or an admin may change it
- **private** — yours, nobody else lists it
Scope is the **version** by default, because that is where people enter the app.
`version_id IS NULL` means the layout applies to every version of the source;
that is where the old source default went, and what a brand-new version picks up
before anyone has published anything for it. `is_default` (at most one per scope,
by partial unique index) is what `initViewer()` restores on a first load — a
version-scoped default beating a source-wide one, the narrower answer winning.
**Permissions are the `pf.log` rule verbatim** — your own, or an admin's
override, and the UI greys out the rest rather than offering a click that answers
403. `can_edit` rides on every row so the menu knows which. Applying is never
restricted: the guarantee is that a published layout cannot be *changed* out from
under people, not that it cannot be adapted — Save is withheld on a layout that
isn't yours, Save as… forks it into your own.
**No territory clause.** A layout is display config, and territory restricts rows,
not columns; `cleanLayout()` already drops anything the live schema lacks.
**What is still local.** `LAYOUT_KEY` (`pf_layout_v{vid}`) — the unnamed
last-used config — stays in `localStorage`, because it is per-browser session
continuity rather than a thing anyone names or shares. `LAYOUTS_KEY`
(`pf_layouts_v{vid}`) is the old named list; `loadLayouts()` lifts it into
`pf.layout` as private rows once per version and then clears the key.
**The dirty dot is a comparison, not a flag.** `restore()` itself fires
`perspective-config-update`, so anything set unconditionally in that handler
would light up the moment a layout was applied. `activeConfigRef` holds what the
pivot last matched and `sameConfig()` compares against it, ignoring `table`
(the per-load table name, different on every refresh).
**What this replaced.** Named layouts lived only in `localStorage` — invisible to
anyone else, gone on another machine. The one server-side layout was
`pf.source.default_layout`, a single anonymous blob that `PUT
/sources/:id/default-layout` let *any* account overwrite for *every* account: a
published layout with no owner. That route is gone; the column remains, migrated
and read by nothing.
`cleanLayout()` guards `aggregates` along with the axes — including the weight
column of the multi-arg form — and drops the offending *entry* rather than the
layout, since `restore()` is all-or-nothing.
---
## Slice mechanics
When the user clicks a pivot cell, `perspective-click` fires. The handler in `Forecast.jsx` extracts `[col, '==', value]` filters from `detail.config.filter` — only `role = dimension` and `role = date` columns are kept as the slice. A plain click replaces the selection; ctrl/⌘/shift-click toggles a slice in or out of it, so the panel holds a **list** of slices sent as `slices` in operation POST bodies (the single `slice` object is still accepted server-side).
@ -110,43 +383,6 @@ Turning a region back into slices re-derives, per cell, the same filters Perspec
---
## Column hierarchy (collapse / expand)
The two pivot axes collapse by completely different mechanisms, and the asymmetry is a
Perspective constraint, not a choice:
- **Rows.** The `GROUP BY ROLLUP` view holds every level at once; `view.set_depth()` — which
lives on the view, not the config — hides the deeper ones. That is what the `EXPAND 0 1 2 3`
buttons drive, via `applyDepth()`.
- **Columns.** There is no equivalent. `expand()` / `collapse()` take a **row index**,
`ViewConfig` has `group_by_depth` but no `split_by_depth`, and `split_rollup_mode`
(`'flat' | 'rollup'`) only chooses whether subtotal column groups are *emitted* — it is a
view shape, not an interaction. So `applySplitDepth(n)` collapses by restoring a
**truncated `split_by`**, which rebuilds the view.
Three things follow from the rebuild, and each is handled:
1. The full hierarchy has to be remembered separately — once collapsed, `viewer.save()`
only reports the short `split_by`. `splitFullRef` / `splitFull` hold it, and it is
persisted into the saved layout as `split_full` so a reload while collapsed can still
expand back. `adoptSplit()` is the single place it is set.
2. `perspective-config-update` fires for our own restore as well as the user rearranging
the pivot. `collapsingRef` distinguishes them — without it, a collapse would overwrite
the full hierarchy with the truncated one and the deeper levels would be unreachable.
3. Row depth lives on the discarded view, so `applyDepth(expandDepthRef.current)` is
re-applied afterwards — the same wart as the refocus re-apply.
The selection is cleared on every change: slices name the split_by dimensions they were
cut from, and the highlight is keyed on grid coordinates. Neither survives a column axis
that just changed shape.
**Limitation:** this is whole-axis, not per-branch. Excel can collapse 2025 while 2026
stays expanded; truncating `split_by` collapses every column group at that level together.
Per-branch is not reachable — `columns` selects which *measures* appear, not individual
split combinations.
---
## Operation SQL patterns
All three operations follow the same structure: insert a `pf.log` row in a CTE, then insert forecast rows referencing its id. `{{where_clause}}` is built from the slice; `{{exclude_clause}}` blocks `exclude_iters` rows.
@ -186,6 +422,70 @@ effects call the API immediately.
`pf_user: 'admin'`, which any client could have set to anything. The audit log
now names the account that made the change.
## Source columns come from pg_catalog
`information_schema.columns` omits **materialized views** — they are not in the
SQL standard — and `gs.osm_skinny` is one. So the source the whole app is built
on looked like it had no columns: registering it seeded nothing, and creating a
version failed with "No usable columns in col_meta" while col_meta plainly held
thirty-six.
`RELATION_COLUMNS_SQL` in `lib/utils.js` is the replacement, used by version
creation, source registration and the table preview. It returns the same shape
information_schema did, so `mapType` and the callers were unchanged:
`data_type` is `format_type` with the modifier stripped, which gives the same
spelling (`character varying`, `numeric`), and precision and scale are unpacked
from `atttypmod`. The table browser lists from `pg_class` by `relkind` for the
same reason.
## Territory scoping
An account sees and changes only its own territory. The list lives on
`pf.app_user.territory` (jsonb array) with `is_admin` for the accounts that see
everything, and `col_meta.is_territory` marks which column of a given source
the values belong to — one per source, flagged rather than named in code so a
second source can be divided by something other than a sales rep.
**Fail closed.** No territory and not an admin means no rows.
`buildTerritoryClause()` returns `FALSE`, not `TRUE`, for an empty list or an
unflagged source: an account somebody forgot to configure sees nothing instead
of the whole book.
**Built from the session, never the request.** This is the difference between
it and `scope`, which the browser sends and which is right to send, being a
filter the user chose. A permission cannot come from the thing it restrains, so
the territory predicate is ANDed on last, in `sliceUnits()` for writes and per
route for reads, where nothing in the payload can remove it.
Enforced at:
- `/data` — clause on the cursor *and* on the count behind `X-Row-Count`
- `/agg` — a `{{territory_clause}}` token applied **before** the GROUP BY, since
the territory column need not be part of the grain and may not survive it
- every operation, through `sliceUnits()`
- `/sources/:id/values/:col` — completion reads the *source* table, which no
scope has touched, so without it a dropdown enumerates the whole business
- `DELETE /log/:logid` and `PATCH /log/:logid` — by owner, not territory. Undo
removes an entry's rows wholesale, and half-undoing one would leave a state
nothing describes. The PATCH looks like a private annotation and is not:
`label` and `bucket` name the pivot's columns for everyone in the version, so
unguarded it let any account rename the company's segments. Your own entries,
or an admin's override, and the UI greys out the rest rather than offering a
click that answers 403.
- recode's `set` — a scoped account cannot set the territory column at all.
Moving a row between territories is reassignment, not forecasting, and it
would vanish from the view that would have shown what happened.
**The change log shows an entry's full impact**, not the reader's share. The
totals stamped on `pf.log` are company-wide, so an admin's version-wide scale
reads the same in every account — deliberate, and labelled, rather than
re-aggregating per territory.
Managed with `./pf.sh set-territory | set-admin | orphan-territory`.
`orphan-territory` lists values present in the data that no account owns; work
under one is invisible to everyone but an admin, which a typo causes easily and
nothing inside the app reveals.
## Light / dark mode
Theme state lives in `ui/src/theme.jsx` — a React context (`ThemeContext`) with a `ThemeProvider` that wraps the app in `main.jsx`.
@ -197,10 +497,52 @@ Theme state lives in `ui/src/theme.jsx` — a React context (`ThemeContext`) wit
- **Perspective viewer:** `Forecast.jsx` calls `viewer.setAttribute('theme', dark ? 'Pro Dark' : 'Pro Light')` both on initial load and in a `useEffect([dark, versionId])` so the viewer stays in sync when the toggle fires
- **Consuming the theme:** `import useTheme from '../theme.jsx'` then `const { dark, setDark } = useTheme()`
## After a change: restart, and Generate SQL
Two steps, and which one you need depends on what changed.
- **Restart the server** for anything in `routes/`, `lib/` or `server.js`.
- **Generate SQL** (Setup, per source) whenever `lib/sql_generator.js` changes.
The templates are *stored* in `pf.sql`, so editing the generator changes
nothing until they are rebuilt — and a template carrying a token the running
code does not substitute fails at the database rather than in JS, which reads
as an unrelated client-side error.
The order matters: restart first, then Generate SQL, or the old code writes the
templates.
Schema changes are applied to the live database directly and mirrored into
`setup_sql/` for a fresh install; `01_schema.sql` is idempotent but is not a
migration runner, so running it is not how an existing database gets a new
column.
## Known issues / active work
- Operation panel (Scale/Recode/Clone) SQL generation and dim_period JOIN are complete; UI wiring to API still needs completion
- Load progress bar is jittery — needs throttle (~10 updates/sec)
- **Zero-row operations report success.** Scale refuses with "Nothing to
scale…" when its slice matches nothing; recode and clone commit an empty log
entry and return `rows_affected: 0`. A recode of a rep whose rows are all
`reference` looked like it worked and did nothing
- **The change log does not show an entry's id**, so there is no way to name
one when asking about it
- **Territory is read onto the session at login**, so granting or changing one
does not reach a signed-in account until it signs in again. Re-reading it per
request in `requireAuth` would also make disabling someone immediate
- **Depth buttons rebuild the view.** A depth lives in `ViewConfig`, so changing
it goes through `restore()` and `Session::update_view_config` tears down and
rebuilds the view — a full traversal — where a manual collapse mutates the
existing one in place. Noticeable on a large grain. The fix is to detect a
depth-only change and call `view.set_depth()` imperatively while still writing
it to the config, at the cost of the two being able to drift
- **Load time is dominated by row count, not payload size.** On `fc_osm_skinny_29`
(2.56M raw rows) a 24-column grain still yields 285,685 rows: ~2s to aggregate in
pg, but ~15s to serialise those rows out of Postgres and parse them into JS, then
~3s to build Arrow. Halving the payload (the `pf_gkey` md5) barely moved it. The
remaining lever is **dynamic grain** — group by the fields the current pivot
actually uses rather than every `in_grain` column; see `pf_spec.md`
§Display-grain pre-aggregation, "the dynamic variant"
- Per-node expand/collapse is lost whenever the view rebuilds — set-only API, no
getter; see §Axis depth
- Default pivot layout should be configurable per source (currently hardcodes first 2 dimensions)
- Source/version selection persists in `localStorage` (`pf_sourceId` / `pf_versionId`,
`App.jsx`). It is re-validated against the live list whenever that list changes, so a

View File

@ -257,17 +257,43 @@ it would retire the `if(...)`-expression workaround. It costs the d3fc charts, t
exist in the current dataset (plus any `expressions`). dataflow's `cleanLayout()` is
the reference implementation; a stale layout referencing a dropped column otherwise
throws on restore.
- **`aggregates` needs the same guard, and doesn't currently have it** (verified 2026-08,
pf_app). Both existing `cleanLayout()` implementations filter
`columns`/`group_by`/`split_by`/`sort`/`filter` but leave `aggregates` untouched. That
is harmless *today* only because `viewer.save()` emits `aggregates: {}` until someone
sets one explicitly. The moment a layout adopts the weighted-mean pattern (§3a), a
- **`aggregates` needs the same guard.** pf_app's `cleanLayout()` has it as of 2026-09;
**dataflow's does not** and should adopt it. Filtering
`columns`/`group_by`/`split_by`/`sort`/`filter` and stopping there is harmless only
while `viewer.save()` emits `aggregates: {}`, which it does until someone sets an
aggregate explicitly. The moment a layout adopts the weighted-mean pattern (§3a), a
dropped column aborts the entire restore — both `table.view()` and `viewer.restore()`
throw `Could not get dtype for column 'X' as it does not exist in the schema`. An
aggregate entry references a *target* column and, in the multi-arg form, a *weight*
column; both need validating, and the entry should be dropped rather than the layout.
column; both need validating, and the entry is what gets dropped, never the layout.
Add this guard as part of adopting §3a, not after.
### Auto-pause deletes the view — turn it off for a static pivot
`<perspective-viewer>` auto-pauses by default: an `IntersectionObserver` on itself
(scrolled out of the viewport, `display: none`) combined with the document's
`visibilitychange` (backgrounded tab, minimized window). "Pause" is not a paint
optimisation — `session.set_pause(true)` runs `view_sub.take().delete()`, so the
**view object is destroyed**. Becoming visible again calls
`restore_and_render(…, ViewerConfigUpdate::default())`: a new view and a full
traversal, every time.
For a viewer streaming live updates nobody is watching, that is the right trade.
For a pivot over a large static table it is the wrong one — on pf_app's
`fc_osm_skinny_29` grain it is a multi-second stall on every tab switch, and it
silently discards per-node expand/collapse (§"Not fixed by any of this"), which
has no config representation and so cannot be restored.
```js
await viewer.load(table)
try { if (viewer.setAutoPause) await viewer.setAutoPause(false) } catch {}
```
Guard the call: it is a method on the custom element and absent on older builds.
Leave auto-pause **on** where the table is fed by a live stream the user does not
need to have kept up with while away. This is long-standing viewer behaviour, not
a 5.x regression — worth knowing before blaming a rebuild on your own code.
---
## 6. Build & deploy (target)

View File

@ -50,7 +50,29 @@ function sessionUser(req) {
return req.session?.user?.username || null;
}
module.exports = { hashPassword, verifyPassword, requireAuth, sessionUser, SCRYPT };
// What this account may see and change, read from the session for the same
// reason the username is: the browser must not be able to widen it.
//
// Returns { admin: true } for an account that sees everything, or
// { admin: false, values: [...] } for a scoped one. An empty list is a real
// answer meaning "nothing", not a missing one meaning "everything" -- an
// account created without a territory sees no rows until it is granted some.
function sessionTerritory(req) {
const u = req.session?.user;
if (!u) return { admin: false, values: [] };
if (u.is_admin) return { admin: true, values: null };
return { admin: false, values: Array.isArray(u.territory) ? u.territory : [] };
}
function requireAdmin(req, res, next) {
if (req.session?.user?.is_admin) return next();
res.status(403).json({ error: 'Administrator access required' });
}
module.exports = {
hashPassword, verifyPassword, requireAuth, requireAdmin,
sessionUser, sessionTerritory, SCRYPT,
};
// CLI: `node lib/auth.js hash` reads a password on stdin and prints its hash,
// so ./pf.sh can create users without the plaintext touching argv or psql.

View File

@ -8,9 +8,100 @@
// Tokens baked in at generation time: column names, source schema.table
// Tokens substituted at request time: {{fc_table}}, {{where_clause}}, {{exclude_clause}},
// {{version_id}}, {{logid}}, {{pf_user}}, {{note}},
// {{label}}, {{bucket}}, {{tag}}, {{territory_clause}},
// {{params}}, {{slice}}, {{date_from}}, {{date_to}},
// {{value_incr}}, {{units_incr}}, {{set_clause}}, {{scale_factor}}
// What the pivot shows for a row's segment and its bucket.
//
// The ordering prefix is part of the stored text, not computed here. Perspective
// orders column groups by the value string, so "01 - Actual" is the only way an
// arbitrary order can be expressed -- and l.label is where a person types it.
// Nothing derives it, which is deliberate: an earlier design built the prefix from
// a separate seq column, as Perspective expressions on the client, and the prefix
// then existed only inside the pivot -- so every other reader disagreed with it,
// and a label that could not be expressed in ExprTK's printable-ASCII-per-byte
// string scanner could not be ordered at all. Stored text has neither problem.
//
// The single exception is the adjustment fallback, whose 99 keeps unlabelled
// adjustments last. Labelling an adjustment's log row overrides it, which is how
// one kind of adjustment is split out from the rest -- l.label rather than tag or
// note, so a segment name stays separable from adjustment commentary (pf_note).
//
// ---------------------------------------------------------------------------
// DISPLAY DEFAULTS -- every hardcoded name the pivot can show.
//
// These are the values a row falls back to when nobody has named it. They are
// the complete list: if a segment or bucket appears in the pivot under a name
// that is not in pf.log, it came from here. CLAUDE.md has the same list under
// "Hardcoded display names".
//
// They live on pf.version -- adjustment_segment, adjustment_bucket,
// unlabeled_load -- and the constants below are only the fallback for a version
// that has not set one. Read through a join at query time rather than
// substituted at generation: pf.sql templates are keyed on (source_id,
// operation) and shared by every version of a source, so a value baked in could
// not vary by version and regenerating for one would change the others.
//
// Exported because /data builds its own statement in routes/operations.js while
// /agg is generated here, and the two have to agree.
// ---------------------------------------------------------------------------
// An adjustment with no label of its own. The 99 keeps it after every numbered
// segment -- ordering is string ordering, so this only works while the loads
// carry 01-0n. Labelling an adjustment's log row overrides it, which is how one
// kind of adjustment is split out from the rest.
const ADJUSTMENT_SEGMENT = '99 - Adjustments';
// What an adjustment counts toward. Prefixed to match the segments it adjusts:
// unprefixed it read 'Forecast' while the loads read '04 - Forecast', and the
// column split in two -- the adjustments sitting apart from the rows they
// adjust. The number is a guess at the convention in use, which is the clearest
// argument for making this per-version.
const ADJUSTMENT_BUCKET = '04 - Forecast';
// A load nobody named. No prefix, so it sorts after everything numbered --
// letters follow digits in ASCII. The old '(unlabeled load)' sorted *first*,
// since '(' is 0x28 and digits begin at 0x30.
const UNLABELED_LOAD = 'Unlabeled';
const LOAD_SEGMENT = `COALESCE(NULLIF(l.label, ''), NULLIF(l.tag, ''), NULLIF(l.note, ''),
NULLIF(v.unlabeled_load, ''), '${UNLABELED_LOAD}')`;
const SEGMENT_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
THEN ${LOAD_SEGMENT}
ELSE COALESCE(NULLIF(l.label, ''), NULLIF(v.adjustment_segment, ''), '${ADJUSTMENT_SEGMENT}')
END`;
// What the row counts towards. A load falls back to its own name until it is
// bucketed; an adjustment falls back to the forecast bucket, because that is
// what an adjustment is -- exclude_iters keeps operations off the reference
// segments, so there is no adjustment that is not part of the forecast.
const BUCKET_EXPR = `COALESCE(NULLIF(l.bucket, ''),
CASE WHEN l.operation IN ('baseline','reference')
THEN ${LOAD_SEGMENT}
ELSE COALESCE(NULLIF(v.adjustment_bucket, ''), '${ADJUSTMENT_BUCKET}')
END)`;
const NOTE_EXPR = `CASE WHEN l.operation IN ('baseline','reference')
THEN NULL
ELSE COALESCE(NULLIF(l.tag, ''), NULLIF(l.note, ''))
END`;
// Every pf.log column the two expressions above read, for /agg's GROUP BY: they
// are functionally dependent on pf_logid, which is in the grain, but Postgres
// will not infer that.
const LABEL_GROUP_COLS = ['l.operation', 'l.label', 'l.tag', 'l.note', 'l.bucket',
'v.adjustment_segment', 'v.adjustment_bucket', 'v.unlabeled_load'];
// The version carries the fallback names, so every statement that reads the
// expressions above needs it in scope as `v`. LEFT, not inner: a forecast row
// whose log entry somehow has no version should still come back, named by the
// constants.
const VERSION_JOIN = `
LEFT JOIN pf.version v
ON v.id = l.version_id`;
// wrap a column name in double quotes for safe use in SQL
function q(name) { return `"${name}"`; }
@ -51,6 +142,42 @@ function grainOf(colMeta) {
return { cols, key, groupCols };
}
// Date columns that anchor a dim_group, each with the dimensions derived from
// pf.dim_period for it. Shared so the generator and the routes agree on both the
// membership and the join aliases -- the same reason grainOf is a single function.
function dateGroupsOf(colMeta) {
return colMeta
.filter(c => c.role === 'date' && c.is_key && c.dim_group)
.map((keyCol, i) => ({
alias: `dp${i + 1}`,
dateCol: keyCol.cname,
group: keyCol.dim_group,
derived: colMeta
.filter(c => c.role === 'dimension'
&& c.dim_group === keyCol.dim_group
&& c.dim_period_col)
.map(c => ({ cname: c.cname, periodCol: c.dim_period_col })),
}))
.filter(g => g.derived.length > 0);
}
// cname -> which join it comes from and which of its columns
function dimPeriodMapOf(dateGroups) {
return new Map(
dateGroups.flatMap(g => g.derived.map(d => [d.cname, { alias: g.alias, periodCol: d.periodCol }]))
);
}
// The joins themselves, against a date that has already had {{date_offset}}
// applied. LEFT because a null date -- an order not yet shipped has no ship date
// -- must leave the period columns empty rather than drop the row.
function dimPeriodJoins(dateGroups, alias = 's') {
return dateGroups.map(g =>
`\n LEFT JOIN pf.dim_period ${g.alias}`
+ ` ON ${g.alias}.drange @> (${alias}."${g.dateCol}" + '{{date_offset}}'::interval)::date`
).join('');
}
function generateSQL(source, colMeta) {
const dims = colMeta
.filter(c => c.role === 'dimension')
@ -88,21 +215,47 @@ function generateSQL(source, colMeta) {
// Baseline and reference copy the source row wholesale, so they carry every
// measure and every date — not just the primary one the operations act on.
// Dropping the others would leave those columns null for the life of the version.
// Clone carries every date column, not just the primary one, because it is the
// operation that moves rows through time: {{date_offset}} shifts them all
// together, and the period dimensions are re-derived from pf.dim_period against
// the shifted dates rather than copied from the row being cloned. Cloning last
// year's mix forward a year otherwise produces rows dated 2027 still labelled
// with 2026's periods.
const cloneCols = [...dims, ...dateCols, effectiveValue, effectiveUnits].filter(Boolean);
const cloneInsertCols = [...cloneCols.map(q), 'pf_iter', 'pf_logid', 'pf_user', 'pf_created_at'].join(', ');
// An adjustment writes one row per *coordinate* it touches, not one per row
// it reads.
//
// Reading rows one-for-one meant every operation inherited the row count of
// everything before it: the baseline's rows plus every prior adjustment's
// rows at the same coordinate, so the table grew super-linearly with how
// much work had been done on it. Eight Pull Forward entries and the next
// scale over the same slice writes nine times what it needs to.
//
// Collapsing is over pf_logid and pf_iter only -- every stored dimension and
// date stays in the GROUP BY -- so no column goes null and nothing becomes
// unsliceable later. The distribution maths is untouched either way, since
// the window sums see the same totals whether or not the rows underneath
// them have been added up first.
const groupCols = (cols) => cols.map(q).join(',\n ');
const loadCols = [...dims, ...dateCols, ...valueCols, ...unitsCols];
const loadInsertCols = [...loadCols.map(q), 'pf_iter', 'pf_logid', 'pf_user', 'pf_created_at'].join(', ');
const dateColSet = new Set(dateCols);
// dim_period JOIN support: if the date column is the is_key of a dim_group,
// dimension siblings with dim_period_col set are derived from pf.dim_period
// instead of being copied raw from the source on baseline/reference load.
const dateKeyGroup = colMeta.find(c => c.role === 'date' && c.is_key && c.dim_group)?.dim_group;
const dimPeriodMap = new Map(
dateKeyGroup
? colMeta
.filter(c => c.role === 'dimension' && c.dim_group === dateKeyGroup && c.dim_period_col)
.map(c => [c.cname, c.dim_period_col])
: []
);
// dim_period JOIN support: a date column that is the is_key of a dim_group
// anchors that group, and dimension siblings with dim_period_col set are
// derived from pf.dim_period instead of copied raw. Derivation is against the
// date *after* {{date_offset}}, which is the whole point -- shift a baseline
// forward a year and its period columns follow, rather than still naming the
// year it came from.
//
// Every such group, not just the first. This used to be a find(), so a source
// with order, requested and ship date groups derived the order one and copied
// the other two raw -- shifted dates against unshifted period labels.
const dateGroups = dateGroupsOf(colMeta);
const dimPeriodMap = dimPeriodMapOf(dateGroups);
const hasDimPeriod = dimPeriodMap.size > 0;
// display grain — when set, initial load and operations both return rows
@ -148,21 +301,17 @@ function generateSQL(source, colMeta) {
return `
SELECT
${grainSelect('t.')}
,CASE WHEN l.operation IN ('baseline','reference')
THEN COALESCE(NULLIF(l.tag, ''), NULLIF(l.note, ''), '(unlabeled load)')
ELSE '(adjustment)' END AS pf_segment
,CASE WHEN l.operation IN ('baseline','reference')
THEN NULL
ELSE COALESCE(NULLIF(l.tag, ''), NULLIF(l.note, '')) END AS pf_note
,l.operation AS pf_op
,${SEGMENT_EXPR} AS pf_segment
,${BUCKET_EXPR} AS pf_bucket
,${NOTE_EXPR} AS pf_note
,l.operation AS pf_op
FROM {{fc_table}} t
LEFT JOIN pf.log l
ON l.id = t.pf_logid
ON l.id = t.pf_logid${VERSION_JOIN}
WHERE {{territory_clause}}
GROUP BY
${grain.groupCols('t.').join('\n ,')}
,l.operation
,l.tag
,l.note`.trim();
,${LABEL_GROUP_COLS.join('\n ,')}`.trim();
}
// grain columns + pf_gkey + summed measures, in the leading-comma style the
@ -194,23 +343,26 @@ GROUP BY
// The offset shifts every date column, so order date and ship date stay in step.
return loadCols.map(c => {
if (dateColSet.has(c)) return `(${pfx}${q(c)} + '{{date_offset}}'::interval)::date`;
if (dimPeriodMap.has(c)) return `dp.${q(dimPeriodMap.get(c))} AS ${q(c)}`;
if (dimPeriodMap.has(c)) {
const { alias, periodCol } = dimPeriodMap.get(c);
return `${alias}.${q(periodCol)} AS ${q(c)}`;
}
return `${pfx}${q(c)}`;
}).join(',\n ');
}
function buildFromClause() {
if (!hasDimPeriod) return srcTable;
return `${srcTable} s\n JOIN pf.dim_period dp`
+ ` ON dp.drange @> (s.${q(dateCol)} + '{{date_offset}}'::interval)::date`;
return srcTable + ' s' + dimPeriodJoins(dateGroups);
}
function buildBaseline() {
return `
WITH
ilog AS (
INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note)
VALUES ({{version_id}}, '{{pf_user}}', 'baseline', NULL, '{{params}}'::jsonb, '{{note}}')
INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note, label, bucket, tag)
VALUES ({{version_id}}, '{{pf_user}}', 'baseline', NULL, '{{params}}'::jsonb, '{{note}}',
NULLIF('{{label}}', ''), NULLIF('{{bucket}}', ''), NULLIF('{{tag}}', ''))
RETURNING id
)
,ins AS (
@ -222,15 +374,16 @@ ilog AS (
WHERE {{filter_clause}}
RETURNING *
)
SELECT count(*) AS rows_affected FROM ins`.trim();
SELECT count(*) AS rows_affected, (SELECT id FROM ilog) AS log_id FROM ins`.trim();
}
function buildReference() {
return `
WITH
ilog AS (
INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note)
VALUES ({{version_id}}, '{{pf_user}}', 'reference', NULL, '{{params}}'::jsonb, '{{note}}')
INSERT INTO pf.log (version_id, pf_user, operation, slice, params, note, label, bucket, tag)
VALUES ({{version_id}}, '{{pf_user}}', 'reference', NULL, '{{params}}'::jsonb, '{{note}}',
NULLIF('{{label}}', ''), NULLIF('{{bucket}}', ''), NULLIF('{{tag}}', ''))
RETURNING id
)
,ins AS (
@ -242,7 +395,7 @@ ilog AS (
WHERE {{filter_clause}}
RETURNING *
)
SELECT count(*) AS rows_affected FROM ins`.trim();
SELECT count(*) AS rows_affected, (SELECT id FROM ilog) AS log_id FROM ins`.trim();
}
function buildScale() {
@ -252,13 +405,16 @@ SELECT count(*) AS rows_affected FROM ins`.trim();
const uSel = effectiveUnits
? `round((${q(effectiveUnits)} / NULLIF(total_units, 0)) * {{units_incr}}, 5)`
: `0`;
// sum(sum(x)) OVER () is the aggregate of the aggregates: the window runs
// after the GROUP BY, so the total is over collapsed coordinates and
// comes to the same figure the ungrouped window produced.
const baseSelectParts = [
...dimsJoined ? [dimsJoined] : [],
q(dateCol),
effectiveValue ? q(effectiveValue) : null,
effectiveUnits ? q(effectiveUnits) : null,
effectiveValue ? `sum(${q(effectiveValue)}) OVER () AS total_value` : null,
effectiveUnits ? `sum(${q(effectiveUnits)}) OVER () AS total_units` : null
effectiveValue ? `sum(${q(effectiveValue)}) AS ${q(effectiveValue)}` : null,
effectiveUnits ? `sum(${q(effectiveUnits)}) AS ${q(effectiveUnits)}` : null,
effectiveValue ? `sum(sum(${q(effectiveValue)})) OVER () AS total_value` : null,
effectiveUnits ? `sum(sum(${q(effectiveUnits)})) OVER () AS total_units` : null
].filter(Boolean).join(',\n ');
return `
WITH
@ -273,6 +429,8 @@ ilog AS (
FROM {{fc_table}}
WHERE {{where_clause}}
{{exclude_clause}}
GROUP BY
${groupCols([...dims, dateCol])}
)
,ins AS (
INSERT INTO {{fc_table}} (${insertCols})
@ -294,10 +452,14 @@ ilog AS (
RETURNING id
)
,src AS (
SELECT ${selectData}
SELECT
${dimsJoined},
${q(dateCol)}${effectiveValue ? `,\n sum(${q(effectiveValue)}) AS ${q(effectiveValue)}` : ''}${effectiveUnits ? `,\n sum(${q(effectiveUnits)}) AS ${q(effectiveUnits)}` : ''}
FROM {{fc_table}}
WHERE {{where_clause}}
{{exclude_clause}}
GROUP BY
${groupCols([...dims, dateCol])}
)
,neg AS (
INSERT INTO {{fc_table}} (${insertCols})
@ -322,6 +484,16 @@ ${opTail('allrows')}` : 'SELECT * FROM neg UNION ALL SELECT * FROM ins'}`.trim()
}
function buildClone() {
const select = [
// dims: whatever {{set_clause}} resolves them to. The route builds it,
// and substitutes the dim_period expression for any derived dimension
// the caller has not overridden outright.
'{{set_clause}}',
...dateCols.map(c => `(s.${q(c)} + '{{date_offset}}'::interval)::date`),
effectiveValue ? `round(s.${q(effectiveValue)} * {{scale_factor}}, 2)` : null,
effectiveUnits ? `round(s.${q(effectiveUnits)} * {{scale_factor}}, 5)` : null,
].filter(Boolean).join(',\n ');
return `
WITH
ilog AS (
@ -330,15 +502,19 @@ ilog AS (
RETURNING id
)
,ins AS (
INSERT INTO {{fc_table}} (${insertCols})
INSERT INTO {{fc_table}} (${cloneInsertCols})
SELECT
{{set_clause}},
${q(dateCol)},
${effectiveValue ? `round(${q(effectiveValue)} * {{scale_factor}}, 2)` : '0'}${effectiveUnits ? `,\n round(${q(effectiveUnits)} * {{scale_factor}}, 5)` : ''},
${select},
'clone', (SELECT id FROM ilog), '{{pf_user}}', now()
FROM {{fc_table}}
WHERE {{where_clause}}
{{exclude_clause}}
FROM (
SELECT
${groupCols([...dims, ...dateCols])}${effectiveValue ? `,\n sum(${q(effectiveValue)}) AS ${q(effectiveValue)}` : ''}${effectiveUnits ? `,\n sum(${q(effectiveUnits)}) AS ${q(effectiveUnits)}` : ''}
FROM {{fc_table}}
WHERE {{where_clause}}
{{exclude_clause}}
GROUP BY
${groupCols([...dims, ...dateCols])}
) s${hasDimPeriod ? dimPeriodJoins(dateGroups) : ''}
RETURNING *
)
${opTail('ins')}`.trim();
@ -368,13 +544,53 @@ function applyTokens(sql, tokens) {
// build a SQL WHERE clause string from a slice object
// only dimension columns are included; unrecognised keys are silently skipped
function buildWhere(slice, dimCols) {
// pf_segment and pf_bucket are not columns on the forecast table -- they are
// computed at read time from the row's pf.log entry -- so a slice naming one
// cannot be compared directly. It resolves to a set of log ids instead, which is
// exact: the name lives on the log row, and every forecast row carries the
// pf_logid that points at it.
//
// Without this they were dropped from the slice, and clicking a single bucket's
// cell scaled every bucket at that dimension intersection while the panel showed
// only the one clicked.
const COMPUTED_SLICE_COLS = { pf_segment: SEGMENT_EXPR, pf_bucket: BUCKET_EXPR };
function computedSlicePredicate(col, val, versionId) {
if (versionId == null) {
const err = new Error(`Cannot filter on ${col} without a version`);
err.status = 500;
throw err;
}
const vals = (Array.isArray(val) ? val : [val]).map(v => `'${esc(v)}'`).join(', ');
return `pf_logid IN (
SELECT l.id
FROM pf.log l${VERSION_JOIN}
WHERE l.version_id = ${parseInt(versionId)}
AND ${COMPUTED_SLICE_COLS[col]} IN (${vals})
)`;
}
function buildWhere(slice, dimCols, versionId) {
if (!slice || Object.keys(slice).length === 0) return 'TRUE';
const allowed = new Set(dimCols);
const parts = [];
for (const [col, val] of Object.entries(slice)) {
if (COMPUTED_SLICE_COLS[col]) {
parts.push(computedSlicePredicate(col, val, versionId));
continue;
}
// A pf_ key this does not understand is refused rather than skipped.
// Skipping is how a selection silently widened: the operation ran against
// everything the dropped key would have excluded. pf_iter is the one
// exception -- the client strips it deliberately, since two cells that
// differ only by iter band are the same slice.
if (col.startsWith('pf_') && col !== 'pf_iter') {
const err = new Error(`Slice names ${col}, which cannot be filtered on`);
err.status = 400;
throw err;
}
if (!allowed.has(col)) continue;
if (Array.isArray(val)) {
const escaped = val.map(v => esc(v));
@ -387,17 +603,92 @@ function buildWhere(slice, dimCols) {
return parts.length ? parts.join('\nAND ') : 'TRUE';
}
// The pivot's own filter, carried alongside the slices so an operation writes
// exactly the rows the ledger counted.
//
// A slice is {col: value} and can only ever mean equality, so a view filtered to
// sseas_e <= 2027 could not be expressed as one. Refusing it was safe but
// useless -- a bounded season is an ordinary way to scope a forecast -- so the
// operators travel as [col, op, value] triples instead.
//
// Perspective's operator names, not SQL's, since that is where these come from.
// Anything outside this list is refused rather than ignored: a scope silently
// dropped is a write that is wider than the panel that authorised it.
const SCOPE_OPS = {
'==': (c, v) => `${c} = ${v[0]}`,
'!=': (c, v) => `${c} != ${v[0]}`,
'>': (c, v) => `${c} > ${v[0]}`,
'>=': (c, v) => `${c} >= ${v[0]}`,
'<': (c, v) => `${c} < ${v[0]}`,
'<=': (c, v) => `${c} <= ${v[0]}`,
'in': (c, v) => `${c} IN (${v.join(', ')})`,
'not in': (c, v) => `${c} NOT IN (${v.join(', ')})`,
'is null': (c) => `${c} IS NULL`,
'is not null': (c) => `${c} IS NOT NULL`,
};
function buildScopeClause(scope, dimCols, versionId) {
if (!Array.isArray(scope) || scope.length === 0) return '';
const allowed = new Set(dimCols);
const parts = scope.map((entry) => {
if (!Array.isArray(entry) || entry.length < 2) {
const err = new Error(`Malformed scope entry ${JSON.stringify(entry)}`);
err.status = 400; throw err;
}
const [col, op, ...rest] = entry;
const vals = (Array.isArray(rest[0]) ? rest[0] : rest).filter(v => v !== undefined);
if (COMPUTED_SLICE_COLS[col]) {
if (op !== '==' && op !== 'in') {
const err = new Error(`${col} can only be scoped with == or in, not ${op}`);
err.status = 400; throw err;
}
return computedSlicePredicate(col, vals, versionId);
}
if (!allowed.has(col)) {
const err = new Error(`Column "${col}" is not available for filtering`);
err.status = 400; throw err;
}
const fn = SCOPE_OPS[op];
if (!fn) {
const err = new Error(`Unsupported filter operator "${op}" on ${col}`);
err.status = 400; throw err;
}
return fn(`"${col}"`, vals.map(v => `'${esc(String(v))}'`));
});
return parts.join('\nAND ');
}
// The territory predicate: what this account may see and change.
//
// Server-side by construction. Today's `scope` is sent by the browser, which is
// right for a filter the user chose and would be fatal for a permission -- so
// this one is built from the session and ANDed on last, where nothing in the
// request can remove it.
//
// FALSE, not TRUE, for an account with no territory. The whole point is that a
// missing grant means no rows: an empty list that fell through to TRUE would
// hand the entire book to the first account somebody forgot to configure.
function buildTerritoryClause(territory, territoryCol, alias = '') {
if (!territory || territory.admin) return '';
if (!territoryCol) return 'FALSE';
const vals = (territory.values || []).filter(v => v != null && v !== '');
if (vals.length === 0) return 'FALSE';
const pfx = alias ? `${alias}.` : '';
return `${pfx}"${territoryCol}" IN (${vals.map(v => `'${esc(String(v))}'`).join(', ')})`;
}
// build a WHERE clause spanning several slices — an OR of AND-groups.
// A union of slices cannot be flattened into one IN list per column: slices
// {Region:East, State:NY} and {Region:West, State:CA} would become
// Region IN (East,West) AND State IN (NY,CA), which also matches East/CA.
function buildWhereAny(slices, dimCols) {
function buildWhereAny(slices, dimCols, versionId) {
const list = (slices || []).filter(s => s && Object.keys(s).length > 0);
if (list.length === 0) return 'TRUE';
if (list.length === 1) return buildWhere(list[0], dimCols);
if (list.length === 1) return buildWhere(list[0], dimCols, versionId);
const groups = list
.map(s => buildWhere(s, dimCols))
.map(s => buildWhere(s, dimCols, versionId))
.filter(w => w !== 'TRUE');
// any slice that reduced to TRUE selects everything, so the union does too
@ -425,12 +716,21 @@ function buildExcludeClause(excludeIters) {
// build the dimension columns portion of a SELECT for recode/clone
// replaces named dimensions with literal values, passes others through unchanged
function buildSetClause(dimCols, setObj) {
// derivedExprs: cname -> a SQL expression to use when the caller has not set the
// column outright. Clone passes the dim_period expressions through here, so a
// cloned row's period dimensions come from the calendar against its shifted date
// rather than from the row it was copied from.
function buildSetClause(dimCols, setObj, opts = {}) {
const { derivedExprs, alias } = opts;
const pfx = alias ? `${alias}.` : '';
return dimCols.map(col => {
if (setObj && setObj[col] !== undefined) {
return `'${esc(setObj[col])}' AS "${col}"`;
}
return `"${col}"`;
if (derivedExprs && derivedExprs[col]) {
return `${derivedExprs[col]} AS "${col}"`;
}
return `${pfx}"${col}"`;
}).join(', ');
}
@ -474,4 +774,6 @@ function esc(val) {
return String(val).replace(/'/g, "''");
}
module.exports = { generateSQL, grainOf, applyTokens, buildWhere, buildWhereAny, buildExcludeClause, buildExcludePredicate, buildSetClause, buildFilterClause, esc };
module.exports = { generateSQL, grainOf, COMPUTED_SLICE_COLS, buildScopeClause, buildTerritoryClause,
SEGMENT_EXPR, BUCKET_EXPR, NOTE_EXPR, LABEL_GROUP_COLS, VERSION_JOIN,
ADJUSTMENT_SEGMENT, ADJUSTMENT_BUCKET, UNLABELED_LOAD, dateGroupsOf, dimPeriodMapOf, dimPeriodJoins, applyTokens, buildWhere, buildWhereAny, buildExcludeClause, buildExcludePredicate, buildSetClause, buildFilterClause, esc };

View File

@ -3,7 +3,41 @@ function fcTable(tname, versionId) {
return `pf.fc_${tname}_${versionId}`;
}
// map information_schema data_type to a clean postgres column type
// The columns of a relation, from pg_catalog rather than information_schema.
//
// information_schema.columns omits materialized views -- they are not in the
// SQL standard -- so a source built on one looked like it had no columns at
// all: registering it seeded nothing, and creating a version failed with "No
// usable columns in col_meta" while col_meta plainly had thirty-six.
//
// The shape matches what information_schema returned, so mapType and the
// callers did not have to change. data_type is format_type with the modifier
// stripped, which gives the same spelling information_schema uses ('character
// varying', 'numeric'), and the numeric precision and scale are unpacked from
// atttypmod the way information_schema does internally.
//
// Takes $1 = schema, $2 = relation name.
const RELATION_COLUMNS_SQL = `
SELECT a.attname AS column_name
,regexp_replace(format_type(a.atttypid, a.atttypmod), '\\(.*\\)$', '') AS data_type
,a.attnum AS ordinal_position
,CASE WHEN a.attnotnull THEN 'NO' ELSE 'YES' END AS is_nullable
,CASE WHEN a.atttypid = 'numeric'::regtype AND a.atttypmod > 4
THEN ((a.atttypmod - 4) >> 16) & 65535 END AS numeric_precision
,CASE WHEN a.atttypid = 'numeric'::regtype AND a.atttypmod > 4
THEN (a.atttypmod - 4) & 65535 END AS numeric_scale
FROM pg_attribute a
JOIN pg_class c ON c.oid = a.attrelid
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE TRUE
AND n.nspname = $1
AND c.relname = $2
AND c.relkind IN ('r', 'v', 'm', 'f', 'p')
AND a.attnum > 0
AND NOT a.attisdropped
`;
// map a data_type name to a clean postgres column type
function mapType(dataType, numericPrecision, numericScale) {
switch (dataType) {
case 'character varying':
@ -36,4 +70,4 @@ function mapType(dataType, numericPrecision, numericScale) {
}
}
module.exports = { fcTable, mapType };
module.exports = { fcTable, mapType, RELATION_COLUMNS_SQL };

105
pf.sh
View File

@ -5,6 +5,7 @@ set -euo pipefail
# pf.sh — Pivot Forecast management script
# Usage: ./pf.sh [deploy|start|stop|restart|status|logs|db-setup|config]
# ./pf.sh [add-user|passwd|list-users|disable-user|enable-user]
# ./pf.sh [set-territory|set-admin|orphan-territory]
# ./pf.sh (interactive menu)
# ---------------------------------------------------------------------------
@ -360,11 +361,102 @@ cmd_list_users() {
require_env; load_env
echo; bold "Accounts"
run_psql -c "
SELECT username, display_name, is_active,
SELECT username, display_name, is_active, is_admin,
coalesce(jsonb_array_length(territory), 0) AS territory_values,
to_char(last_login_at, 'YYYY-MM-DD HH24:MI') AS last_login
FROM pf.app_user ORDER BY username"
}
# Territory is what an account may see and change, as a list of values in the
# source's is_territory column. No territory and not an admin means no rows --
# so a new account is blind until this is run, which is the intended direction
# to fail in.
#
# Values are given comma-separated and have to match the column exactly, since
# that is what the SQL compares. set-territory with no values clears it.
cmd_set_territory() {
require_env; load_env
local username="${1:-}"; shift || true
[[ -z "$username" ]] && { read -rp " Username: " username; }
[[ -z "$username" ]] && die "Username is required."
local values="${*:-}"
[[ -z "$values" ]] && { read -rp " Territory values (comma separated, blank to clear): " values; }
local json="null"
if [[ -n "$values" ]]; then
json=$(python3 - "$values" <<'PYEOF'
import json, sys
vals = [v.strip() for v in sys.argv[1].split(',') if v.strip()]
print(json.dumps(vals))
PYEOF
)
fi
run_psql -v ON_ERROR_STOP=1 -tAc "
WITH upd AS (
UPDATE pf.app_user SET territory = $(if [[ "$json" == "null" ]]; then echo NULL; else echo "'$(sql_lit "$json")'::jsonb"; fi)
WHERE lower(username) = lower('$(sql_lit "$username")')
RETURNING username
)
SELECT count(*) FROM upd" | grep -q '^1$' \
|| die "No such account: $username"
success "Territory updated for $username"
run_psql -c "SELECT username, is_admin, territory FROM pf.app_user WHERE lower(username) = lower('$(sql_lit "$username")')"
}
# An admin sees and changes everything, and is the only account that can recode
# the territory column or undo someone else's entry.
cmd_set_admin() {
require_env; load_env
local username="${1:-}" flag="${2:-true}"
[[ -z "$username" ]] && { read -rp " Username: " username; }
[[ -z "$username" ]] && die "Username is required."
[[ "$flag" != "true" && "$flag" != "false" ]] && die "Second argument must be true or false."
run_psql -v ON_ERROR_STOP=1 -tAc "
WITH upd AS (
UPDATE pf.app_user SET is_admin = $flag
WHERE lower(username) = lower('$(sql_lit "$username")')
RETURNING username
)
SELECT count(*) FROM upd" | grep -q '^1$' \
|| die "No such account: $username"
success "$username is_admin = $flag"
}
# Territory values present in the data that belong to no account. Work under one
# is invisible to everybody but an admin, which is easy to cause by a typo and
# impossible to notice from inside the app.
cmd_orphan_territory() {
require_env; load_env
local source_id="${1:-}"
[[ -z "$source_id" ]] && { read -rp " Source id: " source_id; }
[[ -z "$source_id" ]] && die "Source id is required."
# The column and table are data, so the query is built in two steps rather
# than one clever one: read the names, then run the listing.
local meta col schema tname
meta=$(run_psql -tAF'|' -c "
SELECT m.cname, x.schema, x.tname
FROM pf.col_meta m JOIN pf.source x ON x.id = m.source_id
WHERE m.source_id = $source_id AND m.is_territory")
[[ -z "$meta" ]] && die "Source $source_id has no column marked is_territory."
IFS='|' read -r col schema tname <<< "$meta"
echo; bold "Territory values in $schema.$tname with no account"
run_psql -c "
SELECT DISTINCT s.\"$col\" AS unassigned
FROM \"$schema\".\"$tname\" s
WHERE TRUE
AND s.\"$col\" IS NOT NULL
AND s.\"$col\"::text NOT IN (
SELECT jsonb_array_elements_text(territory)
FROM pf.app_user
WHERE territory IS NOT NULL
)
ORDER BY 1"
}
# Deactivating leaves the row (and its history) in place, and drops any live
# session so the account loses access immediately rather than at cookie expiry.
cmd_disable_user() {
@ -491,6 +583,9 @@ interactive_menu() {
echo " 13) list-users show accounts"
echo " 14) disable-user deactivate an account and sign it out"
echo " 15) enable-user reactivate an account"
echo " 16) set-territory grant an account the territory values it may see"
echo " 17) set-admin make an account an administrator"
echo " 18) orphan-territory territory values no account owns"
echo " q) quit"
echo
read -rp " Choice: " choice
@ -510,6 +605,9 @@ interactive_menu() {
13|list-users) cmd_list_users ;;
14|disable-user) cmd_disable_user ;;
15|enable-user) cmd_enable_user ;;
16|set-territory) cmd_set_territory ;;
17|set-admin) cmd_set_admin ;;
18|orphan-territory) cmd_orphan_territory ;;
q|Q|quit|exit) echo "Bye."; exit 0 ;;
*) warn "Unknown option: $choice" ;;
esac
@ -532,8 +630,11 @@ case "${1:-}" in
add-user) cmd_add_user ;;
passwd) cmd_passwd ;;
list-users) cmd_list_users ;;
set-territory) shift; cmd_set_territory "$@" ;;
set-admin) shift; cmd_set_admin "$@" ;;
orphan-territory) shift; cmd_orphan_territory "$@" ;;
disable-user) cmd_disable_user "${2:-}" ;;
enable-user) cmd_enable_user "${2:-}" ;;
"") interactive_menu ;;
*) die "Unknown command: $1. Valid: deploy start stop restart status logs db-setup config install-service uninstall-service add-user passwd list-users disable-user enable-user" ;;
*) die "Unknown command: $1. Valid: deploy start stop restart status logs db-setup config install-service uninstall-service add-user passwd list-users disable-user enable-user set-territory set-admin orphan-territory" ;;
esac

View File

@ -46,7 +46,8 @@ module.exports = function(pool) {
try {
const result = await pool.query(
`SELECT id, username, display_name, pass_hash, is_active
`SELECT id, username, display_name, pass_hash, is_active,
is_admin, territory
FROM pf.app_user WHERE lower(username) = lower($1)`,
[username]
);
@ -70,6 +71,8 @@ module.exports = function(pool) {
id: user.id,
username: user.username,
display_name: user.display_name,
is_admin: !!user.is_admin,
territory: Array.isArray(user.territory) ? user.territory : [],
};
pool.query(`UPDATE pf.app_user SET last_login_at = now() WHERE id = $1`, [user.id])
.catch(e => console.error('last_login_at update failed', e));

193
routes/layouts.js Normal file
View File

@ -0,0 +1,193 @@
const express = require('express');
const { sessionUser } = require('../lib/auth');
// Named Perspective view configs. Two kinds, one table:
//
// private — yours, nobody else lists it
// published — everyone on the version sees it; only the owner or an admin
// may change it, the same rule pf.log uses for its entries
//
// Scope is the version by default, because that is where people enter the app.
// A layout with version_id NULL applies to every version of the source, which
// is where the old pf.source.default_layout went and what a brand-new version
// picks up before anyone has published anything for it.
module.exports = function(pool) {
const router = express.Router();
const SELECT_COLS = `id, source_id, version_id, name, config, owner,
visibility, is_default, created_at, updated_at`;
// What the client needs to know about a row it cannot write, so the UI can
// grey the control out rather than offer a click that answers 403.
const decorate = (row, req) => ({
...row,
scope: row.version_id == null ? 'source' : 'version',
can_edit: !!req.session?.user?.is_admin || row.owner === sessionUser(req)
});
// The owner/admin gate, shared by every write. Returns the row, or sends the
// response itself and returns null.
async function ownedLayout(req, res) {
const id = parseInt(req.params.lid);
const { rows } = await pool.query(
`SELECT ${SELECT_COLS} FROM pf.layout WHERE id = $1`, [id]
);
if (!rows.length) { res.status(404).json({ error: 'Layout not found' }); return null; }
const row = rows[0];
if (!req.session?.user?.is_admin && row.owner !== sessionUser(req)) {
res.status(403).json({
error: `${row.name}” belongs to ${row.owner} — only they or an administrator can change it`
});
return null;
}
return row;
}
// Everything applicable to a version: its own published layouts, the
// source-wide published ones, and the caller's own private layouts at either
// scope. Deliberately not territory-filtered — a layout is display config,
// and territory restricts rows, not columns.
router.get('/versions/:id/layouts', async (req, res) => {
const versionId = parseInt(req.params.id);
try {
const ver = await pool.query(
`SELECT source_id FROM pf.version WHERE id = $1`, [versionId]
);
if (!ver.rows.length) return res.status(404).json({ error: 'Version not found' });
const sourceId = ver.rows[0].source_id;
const { rows } = await pool.query(
`SELECT ${SELECT_COLS}
FROM pf.layout
WHERE TRUE
AND source_id = $1
AND (version_id = $2 OR version_id IS NULL)
AND (visibility = 'published' OR owner = $3)
ORDER BY visibility DESC, is_default DESC, lower(name)`,
[sourceId, versionId, sessionUser(req)]
);
res.json(rows.map(r => decorate(r, req)));
} catch (err) {
res.status(500).json({ error: err.message });
}
});
// Create. scope 'source' saves it against every version of the source;
// anything else is this version.
router.post('/versions/:id/layouts', async (req, res) => {
const versionId = parseInt(req.params.id);
const { name, config, visibility = 'private', scope = 'version' } = req.body || {};
if (!name || !String(name).trim()) return res.status(400).json({ error: 'Name is required' });
if (!config) return res.status(400).json({ error: 'Config is required' });
if (!['private', 'published'].includes(visibility)) {
return res.status(400).json({ error: 'visibility must be private or published' });
}
try {
const ver = await pool.query(
`SELECT source_id FROM pf.version WHERE id = $1`, [versionId]
);
if (!ver.rows.length) return res.status(404).json({ error: 'Version not found' });
const { rows } = await pool.query(
`INSERT INTO pf.layout (source_id, version_id, name, config, owner, visibility)
VALUES ($1, $2, $3, $4, $5, $6)
RETURNING ${SELECT_COLS}`,
[ver.rows[0].source_id, scope === 'source' ? null : versionId,
String(name).trim(), config, sessionUser(req), visibility]
);
res.json(decorate(rows[0], req));
} catch (err) {
// the partial unique indexes, reported in the terms the user typed
if (err.code === '23505') {
return res.status(409).json({ error: `A layout named “${name}” already exists here` });
}
res.status(500).json({ error: err.message });
}
});
// Update any of name / config / visibility / is_default. Each is optional;
// a missing key leaves the column alone, which is why the flag-and-value
// pair is used rather than COALESCE on the value.
router.patch('/layouts/:lid', async (req, res) => {
const { name, config, visibility, is_default, scope } = req.body || {};
if (name === undefined && config === undefined && visibility === undefined
&& is_default === undefined && scope === undefined) {
return res.status(400).json({ error: 'Nothing to update' });
}
if (visibility !== undefined && !['private', 'published'].includes(visibility)) {
return res.status(400).json({ error: 'visibility must be private or published' });
}
const client = await pool.connect();
try {
const row = await ownedLayout(req, res);
if (!row) return;
// Unpublishing a default would leave a default nobody can see, which
// the table's CHECK refuses; clear the flag with it rather than
// failing on a constraint the user never mentioned.
const clearsDefault = visibility === 'private';
// One default per scope, enforced by a partial unique index. Stand
// the others down first rather than letting the update collide --
// and in one transaction, or a name clash on the second statement
// leaves the version with no default at all.
const target_version = scope === undefined ? row.version_id
: (scope === 'source' ? null : row.version_id);
await client.query('BEGIN');
if (is_default === true) {
await client.query(
`UPDATE pf.layout SET is_default = false
WHERE TRUE
AND source_id = $1
AND COALESCE(version_id, 0) = COALESCE($2::int, 0)
AND id <> $3
AND is_default`,
[row.source_id, target_version, row.id]
);
}
const { rows } = await client.query(
`UPDATE pf.layout SET
name = CASE WHEN $2::bool THEN $3::text ELSE name END,
config = CASE WHEN $4::bool THEN $5::jsonb ELSE config END,
visibility = CASE WHEN $6::bool THEN $7::text ELSE visibility END,
is_default = CASE WHEN $10::bool THEN false
WHEN $8::bool THEN $9::bool ELSE is_default END,
version_id = CASE WHEN $11::bool THEN $12::int ELSE version_id END,
updated_at = now()
WHERE id = $1
RETURNING ${SELECT_COLS}`,
[row.id,
name !== undefined, name !== undefined ? String(name).trim() : null,
config !== undefined, config !== undefined ? config : null,
visibility !== undefined, visibility !== undefined ? visibility : null,
is_default !== undefined, is_default === true,
clearsDefault,
scope !== undefined, target_version]
);
await client.query('COMMIT');
res.json(decorate(rows[0], req));
} catch (err) {
await client.query('ROLLBACK').catch(() => {});
if (err.code === '23505') {
return res.status(409).json({ error: 'A layout with that name already exists here' });
}
res.status(500).json({ error: err.message });
} finally {
client.release();
}
});
router.delete('/layouts/:lid', async (req, res) => {
try {
const row = await ownedLayout(req, res);
if (!row) return;
await pool.query(`DELETE FROM pf.layout WHERE id = $1`, [row.id]);
res.json({ deleted: row.id });
} catch (err) {
res.status(500).json({ error: err.message });
}
});
return router;
};

View File

@ -1,5 +1,6 @@
const express = require('express');
const { grainOf } = require('../lib/sql_generator');
const { sessionUser } = require('../lib/auth');
const { fcTable } = require('../lib/utils');
module.exports = function(pool) {
@ -30,6 +31,21 @@ module.exports = function(pool) {
unitsCol ? `sum(f."${unitsCol}")::float8 AS units_total` : `NULL::float8 AS units_total`
].join(', ');
// The totals are stamped onto the entry when it is written, so the
// normal read is a scan of a few dozen log rows rather than a join
// against millions of forecast rows.
//
// ?recount=1 does it the old way. Stored totals are fixed at write
// time and cannot drift on their own, but nothing stops someone
// deleting forecast rows by hand, and a stored figure has no way to
// notice. This is the way back -- and the backfill for entries
// written before the columns existed.
const recount = req.query.recount === '1' || req.query.recount === 'true';
const stamped = !recount && (await pool.query(
`SELECT count(*)::int AS n FROM pf.log
WHERE version_id = $1 AND row_count IS NULL`, [versionId]
)).rows[0].n === 0;
// ?kind=adjustments drops the baseline and reference entries. That is not
// only about what gets listed: the aggregate below joins the whole
// forecast table, and on a real version the load entries own almost
@ -41,18 +57,70 @@ module.exports = function(pool) {
? `AND l.operation NOT IN ('baseline', 'reference')`
: '';
const result = await pool.query(`
SELECT l.*, ${aggCols},
$2::text AS value_col,
$3::text AS units_col
FROM pf.log l
LEFT JOIN ${table} f ON f.pf_logid = l.id
WHERE l.version_id = $1
${opFilter}
GROUP BY l.id
ORDER BY l.id DESC
`, [versionId, valueCol || null, unitsCol || null]);
res.json(result.rows);
const result = stamped
? await pool.query(`
SELECT l.*,
$2::text AS value_col,
$3::text AS units_col
FROM pf.log l
WHERE l.version_id = $1
${opFilter}
ORDER BY l.id DESC
`, [versionId, valueCol || null, unitsCol || null])
: await pool.query(`
SELECT l.*, ${aggCols},
$2::text AS value_col,
$3::text AS units_col
FROM pf.log l
LEFT JOIN ${table} f ON f.pf_logid = l.id
WHERE l.version_id = $1
${opFilter}
GROUP BY l.id
ORDER BY l.id DESC
`, [versionId, valueCol || null, unitsCol || null]);
// A recount is also a repair: write back what it found, so the next
// read is cheap again and the stored figure matches the rows.
if (recount) {
for (const r of result.rows) {
await pool.query(
`UPDATE pf.log SET row_count = $2, value_total = $3, units_total = $4,
measure_cols = $5::jsonb
WHERE id = $1`,
[r.id, r.row_count, r.value_total, r.units_total,
JSON.stringify({ value: valueCol || null, units: unitsCol || null })]
);
}
}
// The statement is kilobytes per entry and the list is opened to scan,
// not to read SQL. It stays in the row for the debug endpoint below.
res.json(result.rows.map(({ sql_text, ...r }) => ({
...r, has_sql: !!sql_text,
})));
} catch (err) {
console.error(err);
res.status(err.status || 500).json({ error: err.message });
}
});
// Everything about one entry, for when the rows look wrong: what was asked
// for (params), what it ran against (env), and the statement that actually
// executed with territory and scope resolved into it (sql_text).
//
// Both the intent and the SQL, because the translation between them is
// exactly what is in doubt when a result is surprising.
router.get('/log/:logid/debug', async (req, res) => {
const logId = parseInt(req.params.logid);
try {
const { rows } = await pool.query(
`SELECT l.*, v.name AS version_name, s.schema, s.tname
FROM pf.log l
JOIN pf.version v ON v.id = l.version_id
JOIN pf.source s ON s.id = v.source_id
WHERE l.id = $1`, [logId]
);
if (!rows.length) return res.status(404).json({ error: 'Log entry not found' });
res.json(rows[0]);
} catch (err) {
console.error(err);
res.status(err.status || 500).json({ error: err.message });
@ -73,6 +141,17 @@ module.exports = function(pool) {
if (!logResult.rows.length) return res.status(404).json({ error: 'Log entry not found' });
const log = logResult.rows[0];
if (log.status === 'closed') return res.status(403).json({ error: 'Version is closed' });
// Undo deletes rows wholesale by logid, so it cannot be territory
// filtered the way a read or a write can -- half-undoing an entry
// would leave the version in a state nothing describes. Ownership
// instead: your own entries, or an admin's override. Territory alone
// would not do it anyway, since two accounts can share one.
if (!req.session?.user?.is_admin && log.pf_user !== sessionUser(req)) {
return res.status(403).json({
error: `That entry was made by ${log.pf_user || 'someone else'} — only they or an administrator can undo it`
});
}
const table = fcTable(log.tname, log.version_id);
// In grain mode the client's table is indexed on pf_gkey, so undo has to
@ -131,22 +210,44 @@ module.exports = function(pool) {
// a closed version, where relabelling history is still legitimate.
router.patch('/log/:logid', async (req, res) => {
const logId = parseInt(req.params.logid);
const { note, tag } = req.body;
if (note === undefined && tag === undefined) {
return res.status(400).json({ error: 'Nothing to update — send note and/or tag' });
const { note, tag, bucket, label } = req.body;
if (note === undefined && tag === undefined
&& bucket === undefined && label === undefined) {
return res.status(400).json({
error: 'Nothing to update — send note, tag, bucket and/or label'
});
}
try {
// Same rule as undo: your own entries, or an admin's. These are
// annotations, but label and bucket name the pivot's columns for
// everyone who opens the version, so an unguarded PATCH let any
// account rename the company's segments -- including on loads whose
// rows it cannot see.
const owner = await pool.query(
`SELECT pf_user FROM pf.log WHERE id = $1`, [logId]
);
if (!owner.rows.length) return res.status(404).json({ error: 'Log entry not found' });
if (!req.session?.user?.is_admin && owner.rows[0].pf_user !== sessionUser(req)) {
return res.status(403).json({
error: `That entry was made by ${owner.rows[0].pf_user || 'someone else'} — only they or an administrator can change it`
});
}
// COALESCE on the flag, not the value: an explicit null or '' must be
// able to clear a field, which COALESCE on the value alone would ignore
const result = await pool.query(
`UPDATE pf.log SET
note = CASE WHEN $2::bool THEN $3::text ELSE note END,
tag = CASE WHEN $4::bool THEN $5::text ELSE tag END
note = CASE WHEN $2::bool THEN $3::text ELSE note END,
tag = CASE WHEN $4::bool THEN $5::text ELSE tag END,
bucket = CASE WHEN $6::bool THEN $7::text ELSE bucket END,
label = CASE WHEN $8::bool THEN $9::text ELSE label END
WHERE id = $1 RETURNING *`,
[
logId,
note !== undefined, note === undefined ? null : (String(note).trim() || null),
tag !== undefined, tag === undefined ? null : (String(tag).trim() || null),
note !== undefined, note === undefined ? null : (String(note).trim() || null),
tag !== undefined, tag === undefined ? null : (String(tag).trim() || null),
bucket !== undefined, bucket === undefined ? null : (String(bucket).trim() || null),
label !== undefined, label === undefined ? null : (String(label).trim() || null),
]
);
if (!result.rows.length) return res.status(404).json({ error: 'Log entry not found' });

View File

@ -1,7 +1,9 @@
const express = require('express');
const { tableFromArrays, tableToIPC } = require('apache-arrow');
const { applyTokens, buildWhere, buildWhereAny, buildExcludeClause, buildExcludePredicate, buildSetClause, esc } = require('../lib/sql_generator');
const { sessionUser } = require('../lib/auth');
const { applyTokens, buildWhere, buildWhereAny, COMPUTED_SLICE_COLS, buildScopeClause, buildTerritoryClause, buildExcludeClause, buildExcludePredicate, buildSetClause, dateGroupsOf, dimPeriodMapOf, esc,
SEGMENT_EXPR, BUCKET_EXPR, NOTE_EXPR, VERSION_JOIN,
ADJUSTMENT_SEGMENT, ADJUSTMENT_BUCKET } = require('../lib/sql_generator');
const { sessionUser, sessionTerritory } = require('../lib/auth');
const { fcTable } = require('../lib/utils');
module.exports = function(pool) {
@ -32,12 +34,60 @@ module.exports = function(pool) {
return clean;
}
// How a multi-slice request is split into statements.
//
// 'each' — one statement per slice, so every slice reaches its target on
// its own and gets its own log entry
// 'prorate' — a single statement over all of them, letting the SQL's
// sum() OVER () distribute across the whole pool
//
// Only scale has a target to prorate towards; recode and clone rewrite rows
// rather than distribute an amount, so for them this only decides whether the
// work lands as one log entry or several.
// The scope is ANDed onto every unit rather than folded into the slices: it
// applies to all of them equally, and under apply_mode 'each' folding it in
// would repeat the same predicate in every statement for no gain.
function sliceUnits(slices, ctx, applyMode, scope, req) {
const vid = ctx.version.id;
const scl = buildScopeClause(scope, ctx.filterCols, vid);
const terr = req ? territoryOf(req, ctx) : '';
const and = (w) => {
let out = w;
for (const extra of [scl, terr]) {
if (!extra) continue;
out = (out === 'TRUE' || !out) ? extra : `${out}\nAND ${extra}`;
}
return out;
};
return applyMode === 'each'
? slices.map(sl => ({ slices: [sl], where: and(buildWhere(sl, ctx.filterCols, vid)) }))
: [{ slices, where: and(buildWhereAny(slices, ctx.filterCols, vid)) }];
}
// The offset is interpolated into the statement as an interval literal, so a
// typo would surface as a Postgres parse error from the middle of a CTE. Ask
// Postgres to parse it alone first, where the failure is cheap and can name the
// field it came from. Negative intervals are valid and useful -- '-90 days'
// pulls a plan back a quarter -- so this checks validity, not sign.
async function assertInterval(value, res) {
try {
await pool.query(`SELECT $1::interval`, [value]);
return true;
} catch {
res.status(400).json({
error: `"${value}" is not a valid interval. Try something like `
+ `"4 months", "1 year", "-90 days" or "0 days".`
});
return false;
}
}
// A slice is only meaningful if at least one of its keys is a filterable
// column. buildWhere silently drops unknown keys, so {"typo": "x"} would
// otherwise reduce to TRUE and apply the operation to the whole version.
// Refuse rather than let a malformed selection rewrite every row.
function assertSelective(slices, ctx) {
const allowed = new Set(ctx.filterCols);
const allowed = new Set([...ctx.filterCols, ...Object.keys(COMPUTED_SLICE_COLS)]);
slices.forEach((sl, i) => {
const hits = Object.keys(sl).filter(k => allowed.has(k));
if (hits.length === 0) {
@ -51,10 +101,76 @@ module.exports = function(pool) {
});
}
// Stamp what the entry did onto the entry itself.
//
// An indexed lookup on pf_logid, run once at write time, in place of the
// change log joining the whole forecast table on every open. Safe to store
// rather than derive because these rows never change: only this operation
// inserts them, and the only thing that removes them is undo, which deletes
// the log row too.
//
// Best-effort by design. A failure here must not roll back a write that
// succeeded -- the totals can always be recomputed, the adjustment cannot.
async function stampLogTotals(client, ctx, logId, extra = {}) {
if (!logId) return;
const v = ctx.valueCol, u = ctx.unitsCol;
// The state the write ran against, none of which can be reconstructed
// later: territory and exclude_iters are mutable rows elsewhere, and the
// template is overwritten in place every time Generate SQL runs. The
// generation timestamp is a fingerprint, not a version -- it cannot bring
// the old template back, only tell you the entry did not run under this
// one.
const env = {
territory: extra.territory ?? null,
territory_col: ctx.territoryCol || null,
exclude_iters: ctx.version.exclude_iters ?? null,
sql_generated_at: ctx.sqlGeneratedAt || null,
};
try {
await client.query(`
UPDATE pf.log SET
row_count = t.n,
value_total = t.v,
units_total = t.u,
measure_cols = $2::jsonb,
env = $3::jsonb,
sql_text = $4::text
FROM (
SELECT count(*)::int AS n
,${v ? `sum(f."${v}")::float8` : 'NULL::float8'} AS v
,${u ? `sum(f."${u}")::float8` : 'NULL::float8'} AS u
FROM ${ctx.table} f
WHERE f.pf_logid = $1
) t
WHERE pf.log.id = $1
`, [logId, JSON.stringify({ value: v || null, units: u || null }),
JSON.stringify(env), extra.sql || null]);
} catch (err) {
console.error('[stampLogTotals]', err);
}
}
// Moving a row between territories is reassignment, not forecasting, so a
// scoped account cannot set the territory column -- otherwise a rep could
// recode work into their own book, or quietly out of it, and the row would
// be gone from the view that would have shown what happened.
//
// Reads and writes are already scoped, so the *source* rows are safely the
// account's own; this is only about the destination.
function assertMayRecodeTerritory(req, ctx, set, res) {
if (!ctx.territoryCol) return true;
if (req.session?.user?.is_admin) return true;
if (!set || set[ctx.territoryCol] === undefined) return true;
res.status(403).json({
error: `Only an administrator can recode ${ctx.territoryCol} — that moves rows between territories`
});
return false;
}
// echo back what the caller asked for, for the audit log
function pickIntent(body) {
const keys = ['mode', 'target_basis', 'value_incr', 'units_incr', 'value_pct', 'units_pct', 'pct',
'target_value', 'target_units', 'target_price'];
'target_value', 'target_units', 'target_price', 'scope'];
const out = {};
for (const k of keys) if (body[k] !== undefined && body[k] !== null && body[k] !== '') out[k] = body[k];
return out;
@ -244,7 +360,7 @@ module.exports = function(pool) {
const unitsCol = colMeta.find(c => c.role === 'units')?.cname;
const sqlResult = await pool.query(
`SELECT sql FROM pf.sql WHERE source_id = $1 AND operation = $2`,
`SELECT sql, generated_at FROM pf.sql WHERE source_id = $1 AND operation = $2`,
[version.source_id, operation]
);
if (sqlResult.rows.length === 0) {
@ -261,10 +377,26 @@ module.exports = function(pool) {
filterCols: [...dimCols, ...dateCols],
valueCol,
unitsCol,
sql: sqlResult.rows[0].sql
territoryCol: colMeta.find(c => c.is_territory)?.cname || null,
sql: sqlResult.rows[0].sql,
sqlGeneratedAt: sqlResult.rows[0].generated_at
};
}
// Every read and every write goes through this, so a scoped account cannot
// reach a row outside its territory by any route. ANDed on last, after the
// slice and the client's own scope, where nothing in the request can undo
// it.
function territoryOf(req, ctx) {
return buildTerritoryClause(sessionTerritory(req), ctx.territoryCol);
}
function andTerritory(where, req, ctx) {
const t = territoryOf(req, ctx);
if (!t) return where;
return (!where || where === 'TRUE') ? t : `${where}\nAND ${t}`;
}
function guardOpen(version, res) {
if (version.status === 'closed') {
res.status(403).json({ error: 'Version is closed' });
@ -287,7 +419,19 @@ module.exports = function(pool) {
}
const tbl = fcTable(verResult.rows[0].tname, versionId);
const { rows: [{ count }] } = await pool.query(`SELECT COUNT(*) FROM ${tbl}`);
// /data does not go through getContext, so it resolves the territory
// column itself. The count is scoped too, or the progress bar
// promises rows this account will never be sent.
const terrCol = (await pool.query(
`SELECT cname FROM pf.col_meta WHERE source_id = $1 AND is_territory LIMIT 1`,
[verResult.rows[0].source_id]
)).rows[0]?.cname || null;
const territory = sessionTerritory(req);
const terrBare = buildTerritoryClause(territory, terrCol);
const terrAlias = buildTerritoryClause(territory, terrCol, 't');
const terrWhere = terrBare ? `WHERE ${terrBare}` : '';
const { rows: [{ count }] } = await pool.query(`SELECT COUNT(*) FROM ${tbl} ${terrWhere}`);
const rowCount = parseInt(count);
res.setHeader('Content-Type', 'application/vnd.apache.arrow.stream');
@ -300,15 +444,13 @@ module.exports = function(pool) {
await client.query(`
DECLARE pf_cur CURSOR FOR
SELECT t.*
,CASE WHEN l.operation IN ('baseline','reference')
THEN COALESCE(NULLIF(l.tag, ''), NULLIF(l.note, ''), '(unlabeled load)')
ELSE '(adjustment)' END AS pf_segment
,CASE WHEN l.operation IN ('baseline','reference')
THEN NULL
ELSE COALESCE(NULLIF(l.tag, ''), NULLIF(l.note, '')) END AS pf_note
,${SEGMENT_EXPR} AS pf_segment
,${BUCKET_EXPR} AS pf_bucket
,${NOTE_EXPR} AS pf_note
FROM ${tbl} t
LEFT JOIN pf.log l
ON l.id = t.pf_logid
ON l.id = t.pf_logid${VERSION_JOIN}
${terrAlias ? `WHERE ${terrAlias}` : ''}
`);
// Accumulate into column arrays (not row objects) to avoid allocating one JS
@ -352,7 +494,13 @@ module.exports = function(pool) {
router.get('/versions/:id/agg', async (req, res) => {
try {
const ctx = await getContext(parseInt(req.params.id), 'get_agg');
const sql = applyTokens(ctx.sql, { fc_table: ctx.table });
// Before the GROUP BY, not after: the territory column need not be
// part of the grain, so an aggregated row may not carry it at all.
const sql = applyTokens(ctx.sql, {
fc_table: ctx.table,
territory_clause:
buildTerritoryClause(sessionTerritory(req), ctx.territoryCol, 't') || 'TRUE',
});
const { rows } = await runSQL(sql);
res.setHeader('Content-Type', 'application/vnd.apache.arrow.stream');
@ -377,9 +525,10 @@ module.exports = function(pool) {
// load baseline rows from source table — additive, no delete
router.post('/versions/:id/baseline', async (req, res) => {
const { where_clause, date_offset, note, filters, raw_where } = req.body;
const { where_clause, date_offset, note, filters, raw_where, label, bucket, tag } = req.body;
const pf_user = sessionUser(req);
const dateOffset = date_offset || '0 days';
if (!await assertInterval(dateOffset, res)) return;
const filterClause = (raw_where || where_clause || '').trim() || 'TRUE';
try {
const ctx = await getContext(parseInt(req.params.id), 'baseline');
@ -394,12 +543,16 @@ module.exports = function(pool) {
version_id: ctx.version.id,
pf_user: esc(pf_user || ''),
note: esc(note || ''),
label: esc(label || ''),
bucket: esc(bucket || ''),
tag: esc(tag || ''),
params: esc(paramsJson),
filter_clause: filterClause,
date_offset: esc(dateOffset)
});
const result = await runSQL(sql);
await stampLogTotals(pool, ctx, result.rows[0]?.log_id, { sql, territory: sessionTerritory(req) });
res.json(result.rows[0]);
} catch (err) {
console.error(err);
@ -413,9 +566,10 @@ module.exports = function(pool) {
router.put('/versions/:id/baseline/:logid', async (req, res) => {
const versionId = parseInt(req.params.id);
const logid = parseInt(req.params.logid);
const { where_clause, date_offset, note, filters, raw_where } = req.body;
const { where_clause, date_offset, note, filters, raw_where, label, bucket, tag } = req.body;
const pf_user = sessionUser(req);
const dateOffset = date_offset || '0 days';
if (!await assertInterval(dateOffset, res)) return;
const filterClause = (raw_where || where_clause || '').trim() || 'TRUE';
const client = await pool.connect();
@ -451,11 +605,21 @@ module.exports = function(pool) {
date_offset: dateOffset,
...(raw_where ? { raw_where } : (filters ? { filters } : {}))
});
// This route deletes the log row and inserts a fresh one, so every
// annotation on it has to be handed back or it is lost. `??`, not `||`:
// an empty string is the form clearing a field on purpose, undefined is
// the form not carrying it at all -- the segment form has no tag input,
// so tag is always the latter and must survive an edit made for any
// other reason.
const keep = (sent, prior) => esc(sent ?? prior ?? '');
const sql = applyTokens(ctx.sql, {
fc_table: ctx.table,
version_id: ctx.version.id,
pf_user: esc(pf_user || ''),
note: esc(note || ''),
label: keep(label, oldLog.label),
bucket: keep(bucket, oldLog.bucket),
tag: keep(tag, oldLog.tag),
params: esc(paramsJson),
filter_clause: filterClause,
date_offset: esc(dateOffset)
@ -469,6 +633,7 @@ module.exports = function(pool) {
await client.query(`DELETE FROM pf.log WHERE id = $1`, [logid]);
const insResult = await client.query(sql);
await client.query('COMMIT');
await stampLogTotals(pool, ctx, insResult.rows[0]?.log_id, { sql, territory: sessionTerritory(req) });
res.json({
rows_deleted: delRows.rowCount,
@ -520,7 +685,7 @@ module.exports = function(pool) {
// load reference rows from source table (additive — does not clear prior reference rows)
router.post('/versions/:id/reference', async (req, res) => {
const { where_clause, date_offset, note, filters, raw_where } = req.body;
const { where_clause, date_offset, note, filters, raw_where, label, bucket, tag } = req.body;
const pf_user = sessionUser(req);
const dateOffset = date_offset || '0 days';
const filterClause = (raw_where || where_clause || '').trim() || 'TRUE';
@ -537,12 +702,16 @@ module.exports = function(pool) {
version_id: ctx.version.id,
pf_user: esc(pf_user || ''),
note: esc(note || ''),
label: esc(label || ''),
bucket: esc(bucket || ''),
tag: esc(tag || ''),
params: esc(paramsJson),
filter_clause: filterClause,
date_offset: esc(dateOffset)
});
const result = await runSQL(sql);
await stampLogTotals(pool, ctx, result.rows[0]?.log_id, { sql, territory: sessionTerritory(req) });
res.json(result.rows[0]);
} catch (err) {
console.error(err);
@ -572,15 +741,15 @@ module.exports = function(pool) {
// sum() OVER () distribute the increment across the whole pool.
// 'each' runs the same statement once per slice, so every slice
// reaches the target on its own and gets its own log entry.
const units = applyMode === 'each'
? slices.map(sl => ({ slices: [sl], where: buildWhere(sl, ctx.filterCols) }))
: [{ slices, where: buildWhereAny(slices, ctx.filterCols) }];
const units = sliceUnits(slices, ctx, applyMode, req.body.scope, req);
const client = await pool.connect();
let committed = false;
try {
await client.query('BEGIN');
const allRows = [];
// the statement that produced each entry, for pf.log.sql_text
const sqlByLogId = new Map();
let applied = 0;
const skipped = [];
@ -610,6 +779,9 @@ module.exports = function(pool) {
});
const result = await runSQL(sql, client);
await tagLog(client, result.rows, req.body.tag);
for (const r of result.rows) {
if (r.pf_logid != null) sqlByLogId.set(r.pf_logid, sql);
}
allRows.push(...result.rows);
}
@ -622,8 +794,15 @@ module.exports = function(pool) {
await client.query('COMMIT');
committed = true;
// one log id per unit: apply_mode 'each' writes an entry per slice
for (const id of new Set(allRows.map(r => r.pf_logid).filter(Boolean))) {
await stampLogTotals(pool, ctx, id, {
sql: sqlByLogId.get(id) || null,
territory: sessionTerritory(req),
});
}
const opLabel = (req.body.tag || '').trim() || note || null;
const rows = allRows.map(r => ({ ...r, pf_segment: '(adjustment)', pf_note: opLabel, pf_op: 'scale' }));
const rows = allRows.map(r => ({ ...r, pf_segment: ADJUSTMENT_SEGMENT, pf_bucket: ADJUSTMENT_BUCKET, pf_note: opLabel, pf_op: 'scale' }));
res.json({
rows,
rows_affected: rows.length,
@ -653,16 +832,19 @@ module.exports = function(pool) {
const ctx = await getContext(parseInt(req.params.id), 'recode');
if (!guardOpen(ctx.version, res)) return;
assertSelective(slices, ctx);
if (!assertMayRecodeTerritory(req, ctx, set, res)) return;
const excludeClause = buildExcludeClause(ctx.version.exclude_iters);
const setClause = buildSetClause(ctx.dimCols, set);
const units = sliceUnits(slices, ctx, apply_mode);
const units = sliceUnits(slices, ctx, apply_mode === 'each' ? 'each' : 'prorate', req.body.scope, req);
const client = await pool.connect();
let committed = false;
try {
await client.query('BEGIN');
const allRows = [];
// the statement that produced each entry, for pf.log.sql_text
const sqlByLogId = new Map();
for (const unit of units) {
const loggedSlice = unit.slices.length === 1 ? unit.slices[0] : unit.slices;
const sql = applyTokens(ctx.sql, {
@ -678,12 +860,22 @@ module.exports = function(pool) {
});
const result = await runSQL(sql, client);
await tagLog(client, result.rows, req.body.tag);
for (const r of result.rows) {
if (r.pf_logid != null) sqlByLogId.set(r.pf_logid, sql);
}
allRows.push(...result.rows);
}
await client.query('COMMIT');
committed = true;
// one log id per unit: apply_mode 'each' writes an entry per slice
for (const id of new Set(allRows.map(r => r.pf_logid).filter(Boolean))) {
await stampLogTotals(pool, ctx, id, {
sql: sqlByLogId.get(id) || null,
territory: sessionTerritory(req),
});
}
const opLabel = (req.body.tag || '').trim() || note || null;
const rows = allRows.map(r => ({ ...r, pf_segment: '(adjustment)', pf_note: opLabel, pf_op: 'recode' }));
const rows = allRows.map(r => ({ ...r, pf_segment: ADJUSTMENT_SEGMENT, pf_bucket: ADJUSTMENT_BUCKET, pf_note: opLabel, pf_op: 'recode' }));
res.json({ rows, rows_affected: rows.length, slices_applied: units.length });
} finally {
if (!committed) try { await client.query('ROLLBACK'); } catch {}
@ -698,27 +890,62 @@ module.exports = function(pool) {
// clone one or more slices as new business under new dimension values
// does not offset the original slice
router.post('/versions/:id/clone', async (req, res) => {
const { note, set, scale, apply_mode } = req.body;
const { note, set, scale, apply_mode, from_logid, date_offset } = req.body;
const pf_user = sessionUser(req);
const slices = normalizeSlices(req.body);
if (slices.length === 0) return res.status(400).json({ error: 'slice is required' });
if (!set || Object.keys(set).length === 0) return res.status(400).json({ error: 'set is required' });
try {
const ctx = await getContext(parseInt(req.params.id), 'clone');
if (!guardOpen(ctx.version, res)) return;
assertSelective(slices, ctx);
const scaleFactor = (scale != null) ? parseFloat(scale) : 1.0;
const excludeClause = buildExcludeClause(ctx.version.exclude_iters);
const setClause = buildSetClause(ctx.dimCols, set);
const units = sliceUnits(slices, ctx, apply_mode);
const scaleFactor = (scale != null) ? parseFloat(scale) : 1.0;
const dateOffset = (date_offset || '0 days').trim() || '0 days';
if (!await assertInterval(dateOffset, res)) return;
// exclude_iters deliberately does not apply here. It exists to stop
// operations *modifying* reference rows: scale would attribute forecast
// movement to prior-year rows by distributing across them, and recode
// writes negative rows that zero the original out. Clone does neither --
// it reads rows and writes new pf_iter = 'clone' rows, leaving the source
// untouched. Copying a plan or a prior year out of reference and into
// adjustments is the operation working as intended, and excluding them
// meant a visible, deliberate selection silently produced nothing.
//
// from_logid narrows instead: a selection spanning AOP and Prior Year
// where only one is wanted.
let excludeClause = '';
if (from_logid != null) {
const srcLog = await pool.query(
`SELECT id FROM pf.log WHERE id = $1 AND version_id = $2`,
[parseInt(from_logid), ctx.version.id]
);
if (!srcLog.rows.length) {
return res.status(400).json({ error: `No log entry ${from_logid} on this version` });
}
excludeClause = `AND pf_logid = ${parseInt(from_logid)}`;
}
// Period dimensions come from the calendar against the shifted date, not
// from the row being copied -- otherwise a mix moved forward a year
// keeps last year's period labels. An explicit set wins over both.
const dateGroups = dateGroupsOf(ctx.colMeta);
const derivedExprs = Object.fromEntries(
[...dimPeriodMapOf(dateGroups)].map(([cname, { alias, periodCol }]) =>
[cname, `${alias}."${periodCol}"`])
);
const setClause = buildSetClause(ctx.dimCols, set, { derivedExprs, alias: 's' });
const units = sliceUnits(slices, ctx, apply_mode === 'each' ? 'each' : 'prorate', req.body.scope, req);
const client = await pool.connect();
let committed = false;
try {
await client.query('BEGIN');
const allRows = [];
// the statement that produced each entry, for pf.log.sql_text
const sqlByLogId = new Map();
for (const unit of units) {
const loggedSlice = unit.slices.length === 1 ? unit.slices[0] : unit.slices;
const sql = applyTokens(ctx.sql, {
@ -726,21 +953,36 @@ module.exports = function(pool) {
version_id: ctx.version.id,
pf_user: esc(pf_user || ''),
note: esc(note || ''),
params: esc(JSON.stringify({ slices: unit.slices, set, scale: scaleFactor, apply_mode: unit.mode })),
params: esc(JSON.stringify({
slices: unit.slices, set, scale: scaleFactor, apply_mode: unit.mode,
date_offset: dateOffset,
...(from_logid != null ? { from_logid: parseInt(from_logid) } : {}),
})),
slice: esc(JSON.stringify(loggedSlice)),
where_clause: unit.where,
exclude_clause: excludeClause,
set_clause: setClause,
scale_factor: scaleFactor
scale_factor: scaleFactor,
date_offset: esc(dateOffset)
});
const result = await runSQL(sql, client);
await tagLog(client, result.rows, req.body.tag);
for (const r of result.rows) {
if (r.pf_logid != null) sqlByLogId.set(r.pf_logid, sql);
}
allRows.push(...result.rows);
}
await client.query('COMMIT');
committed = true;
// one log id per unit: apply_mode 'each' writes an entry per slice
for (const id of new Set(allRows.map(r => r.pf_logid).filter(Boolean))) {
await stampLogTotals(pool, ctx, id, {
sql: sqlByLogId.get(id) || null,
territory: sessionTerritory(req),
});
}
const opLabel = (req.body.tag || '').trim() || note || null;
const rows = allRows.map(r => ({ ...r, pf_segment: '(adjustment)', pf_note: opLabel, pf_op: 'clone' }));
const rows = allRows.map(r => ({ ...r, pf_segment: ADJUSTMENT_SEGMENT, pf_bucket: ADJUSTMENT_BUCKET, pf_note: opLabel, pf_op: 'clone' }));
res.json({ rows, rows_affected: rows.length, slices_applied: units.length });
} finally {
if (!committed) try { await client.query('ROLLBACK'); } catch {}

View File

@ -1,5 +1,7 @@
const express = require('express');
const { generateSQL } = require('../lib/sql_generator');
const { generateSQL, buildTerritoryClause } = require('../lib/sql_generator');
const { RELATION_COLUMNS_SQL } = require('../lib/utils');
const { sessionTerritory } = require('../lib/auth');
const { sessionUser } = require('../lib/auth');
module.exports = function(pool) {
@ -41,15 +43,14 @@ module.exports = function(pool) {
);
const source = src.rows[0];
// seed col_meta from information_schema
// seed col_meta from the source's real columns
await client.query(`
INSERT INTO pf.col_meta (source_id, cname, role, opos)
SELECT $1, column_name, 'dimension', ordinal_position
FROM information_schema.columns
WHERE table_schema = $2 AND table_name = $3
SELECT $3, column_name, 'dimension', ordinal_position
FROM (${RELATION_COLUMNS_SQL}) c
ORDER BY ordinal_position
ON CONFLICT (source_id, cname) DO NOTHING
`, [source.id, schema, tname]);
`, [schema, tname, source.id]);
await client.query('COMMIT');
res.status(201).json(source);
@ -86,13 +87,23 @@ module.exports = function(pool) {
if (!Array.isArray(cols)) {
return res.status(400).json({ error: 'body must be an array' });
}
// Exactly one per source: the scope is a single IN list against a single
// column, and two flagged would silently mean whichever one a .find()
// reached first -- the trap is_key already fell into (see CLAUDE.md).
const territoryCols = cols.filter(c => c.is_territory).map(c => c.cname);
if (territoryCols.length > 1) {
return res.status(400).json({
error: `Only one column can be the territory. Flagged: ${territoryCols.join(', ')}`
});
}
const client = await pool.connect();
try {
await client.query('BEGIN');
for (const col of cols) {
await client.query(`
INSERT INTO pf.col_meta (source_id, cname, label, role, is_key, dim_group, dim_period_col, in_grain, opos)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9)
INSERT INTO pf.col_meta (source_id, cname, label, role, is_key, dim_group, dim_period_col, in_grain, is_territory, opos)
VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10)
ON CONFLICT (source_id, cname) DO UPDATE SET
label = EXCLUDED.label,
role = EXCLUDED.role,
@ -100,6 +111,7 @@ module.exports = function(pool) {
dim_group = EXCLUDED.dim_group,
dim_period_col = EXCLUDED.dim_period_col,
in_grain = EXCLUDED.in_grain,
is_territory = EXCLUDED.is_territory,
opos = EXCLUDED.opos
`, [
sourceId,
@ -110,6 +122,7 @@ module.exports = function(pool) {
col.dim_group || null,
col.dim_period_col || null,
col.in_grain || false,
col.is_territory || false,
col.opos || null
]);
}
@ -225,10 +238,38 @@ module.exports = function(pool) {
return res.status(400).json({ error: `"${col}" is not a key column` });
}
// ?q= narrows, ?limit= caps. A key column can be very wide -- part on
// osm_skinny has 11,290 distinct values -- so returning the lot to fill a
// completion list is both a slow query and a large response for a control
// that can only usefully show a handful.
const { schema, tname } = srcResult.rows[0];
const q = (req.query.q || '').trim();
const limit = Math.min(parseInt(req.query.limit) || 5000, 5000);
const params = [];
let filter = `WHERE "${col}" IS NOT NULL`;
// Completion reads the *source* table, which no territory scope has
// touched -- so without this a scoped account could enumerate every
// customer, part and rep in the business from a dropdown, having
// been shown none of their rows.
const terrRow = (await pool.query(
`SELECT cname FROM pf.col_meta WHERE source_id = $1 AND is_territory LIMIT 1`,
[req.params.id]
)).rows[0];
const terrClause = buildTerritoryClause(sessionTerritory(req), terrRow?.cname || null);
if (terrClause) filter += ` AND ${terrClause}`;
if (q) {
params.push(`%${q}%`);
filter += ` AND "${col}"::text ILIKE $${params.length}`;
}
params.push(limit);
const result = await pool.query(
`SELECT DISTINCT "${col}" AS val FROM ${schema}.${tname}
WHERE "${col}" IS NOT NULL ORDER BY "${col}"`
`SELECT DISTINCT "${col}"::text AS val FROM ${schema}.${tname}
${filter} ORDER BY 1 LIMIT $${params.length}`,
params
);
res.json(result.rows.map(r => r.val));
} catch (err) {
@ -237,6 +278,142 @@ module.exports = function(pool) {
}
});
// Resolve a dim_group to its key column and siblings, or explain why it cannot be.
async function resolveGroup(sourceId, group) {
const { rows: meta } = await pool.query(
`SELECT * FROM pf.col_meta WHERE source_id = $1 ORDER BY opos`, [sourceId]);
const members = meta.filter(c => c.dim_group === group);
if (!members.length) {
const err = new Error(`No columns are grouped as "${group}" on this source`);
err.status = 404; throw err;
}
const keyCol = members.find(c => c.is_key);
if (!keyCol) {
const err = new Error(
`Group "${group}" has no is_key column, so its members have nothing to be keyed on`);
err.status = 400; throw err;
}
return {
keyCol,
siblings: members.filter(c => c.cname !== keyCol.cname),
// recency column: the source's primary date, the same one the generator
// treats as the date for loads
dateCol: meta.find(c => c.role === 'date')?.cname || null,
};
}
// The member list for a group, as one array. Small enough to send whole --
// 11,290 parts on osm_skinny -- so the client holds it and filters locally
// instead of querying per keystroke.
router.get('/sources/:id/dim/:group', async (req, res) => {
try {
const sourceId = parseInt(req.params.id);
const { keyCol, siblings } = await resolveGroup(sourceId, req.params.group);
const includeInactive = req.query.all === '1';
const { rows } = await pool.query(`
SELECT key_value, attrs, is_active, source_seen
FROM pf.dim_member
WHERE source_id = $1 AND dim_group = $2
${includeInactive ? '' : 'AND is_active'}
ORDER BY key_value
`, [sourceId, req.params.group]);
res.json({
group: req.params.group,
key_col: keyCol.cname,
siblings: siblings.map(c => c.cname),
members: rows,
});
} catch (err) {
console.error(err);
res.status(err.status || 500).json({ error: err.message });
}
});
// Rebuild a group's members from the source. A merge, not a replace: curation
// (a member deactivated by hand, or added before it ever sold) has to survive a
// refresh, so absent members are marked source_seen = false rather than deleted.
//
// Slow by nature -- it reads the whole source, which for a view over a
// transaction table is millions of rows -- so it is a deliberate action rather
// than something that happens on a page load.
router.post('/sources/:id/dim/:group/refresh', async (req, res) => {
const sourceId = parseInt(req.params.id);
const group = req.params.group;
try {
const srcResult = await pool.query(
`SELECT schema, tname FROM pf.source WHERE id = $1`, [sourceId]);
if (!srcResult.rows.length) return res.status(404).json({ error: 'Source not found' });
const { schema, tname } = srcResult.rows[0];
const { keyCol, siblings, dateCol } = await resolveGroup(sourceId, group);
if (!siblings.length) {
return res.status(400).json({ error: `Group "${group}" has no sibling columns to store` });
}
const q = (n) => `"${n}"`;
const attrs = siblings.map(c => `'${c.cname}', s.${q(c.cname)}::text`).join(', ');
// A key can carry more than one attribute set across history -- 11,290
// parts against 13,662 combinations on osm_skinny. Take the most recent
// by the source's date column, which is the live definition.
const recency = dateCol ? `s.${q(dateCol)} DESC NULLS LAST` : `1`;
const started = Date.now();
// An explicit stamp rather than now(): inside a transaction now() is the
// transaction's start time, so "refreshed in this run" and "refreshed in
// a run that began at the same instant" would be indistinguishable.
const runAt = new Date();
const client = await pool.connect();
try {
await client.query('BEGIN');
const { rows: [{ n }] } = await client.query(`
WITH ranked AS (
SELECT s.${q(keyCol.cname)}::text AS key_value,
jsonb_build_object(${attrs}) AS attrs,
row_number() OVER (
PARTITION BY s.${q(keyCol.cname)} ORDER BY ${recency}
) AS rn
FROM ${q(schema)}.${q(tname)} s
WHERE s.${q(keyCol.cname)} IS NOT NULL
)
,upserted AS (
INSERT INTO pf.dim_member
(source_id, dim_group, key_value, attrs, refreshed_at, source_seen)
SELECT $1, $2, key_value, attrs, $3, true
FROM ranked WHERE rn = 1
ON CONFLICT (source_id, dim_group, key_value) DO UPDATE SET
attrs = EXCLUDED.attrs,
source_seen = true,
refreshed_at = $3,
updated_at = now()
RETURNING 1
)
SELECT count(*)::int AS n FROM upserted
`, [sourceId, group, runAt]);
// anything this run did not touch is no longer in the source
const { rowCount: dropped } = await client.query(`
UPDATE pf.dim_member
SET source_seen = false, updated_at = now()
WHERE source_id = $1 AND dim_group = $2 AND source_seen
AND refreshed_at IS DISTINCT FROM $3
`, [sourceId, group, runAt]);
await client.query('COMMIT');
res.json({ group, members: n, no_longer_in_source: dropped, ms: Date.now() - started });
} catch (err) {
await client.query('ROLLBACK');
throw err;
} finally {
client.release();
}
} catch (err) {
console.error(err);
res.status(err.status || 500).json({ error: err.message });
}
});
// given a key column value, look up sibling dim_group column values from source
// returns { sibling_col: value, ... } if exactly one match, null if none or ambiguous
router.get('/sources/:id/lookup', async (req, res) => {
@ -272,23 +449,12 @@ module.exports = function(pool) {
}
});
// set or clear the default Perspective layout for a source.
// Body: a Perspective view config (group_by, split_by, columns, plugin_config, …).
// Pass null or {} to clear.
router.put('/sources/:id/default-layout', async (req, res) => {
try {
const layout = req.body && Object.keys(req.body).length > 0 ? req.body : null;
const result = await pool.query(
`UPDATE pf.source SET default_layout = $1 WHERE id = $2 RETURNING *`,
[layout, req.params.id]
);
if (result.rows.length === 0) return res.status(404).json({ error: 'Source not found' });
res.json(result.rows[0]);
} catch (err) {
console.error(err);
res.status(500).json({ error: err.message });
}
});
// PUT /sources/:id/default-layout is gone. It wrote pf.source.default_layout,
// one anonymous blob per source that any account could overwrite for every
// other account -- a published layout with no owner. Its successor is
// pf.layout: named, owned, and writable only by its owner or an admin. The
// old column is left in place, already migrated into pf.layout by
// setup_sql/01_schema.sql, and read by nothing.
// deregister a source — does not drop existing forecast tables
router.get('/dim-period/cols', async (req, res) => {

View File

@ -1,4 +1,5 @@
const express = require('express');
const { RELATION_COLUMNS_SQL } = require('../lib/utils');
module.exports = function(pool) {
const router = express.Router();
@ -7,15 +8,18 @@ module.exports = function(pool) {
router.get('/tables', async (req, res) => {
try {
const result = await pool.query(`
-- pg_class, not information_schema.tables, which omits
-- materialized views: gs.osm_skinny is one, and the browser
-- could not offer what the app is already built on.
SELECT
t.table_schema AS schema,
t.table_name AS tname,
n.nspname AS schema,
c.relname AS tname,
c.reltuples::bigint AS row_estimate
FROM information_schema.tables t
LEFT JOIN pg_namespace n ON n.nspname = t.table_schema
LEFT JOIN pg_class c ON c.relname = t.table_name AND c.relnamespace = n.oid
WHERE t.table_schema NOT IN ('pg_catalog', 'information_schema', 'pf')
ORDER BY t.table_schema, t.table_name
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE c.relkind IN ('r', 'v', 'm', 'f', 'p')
AND n.nspname NOT IN ('pg_catalog', 'information_schema', 'pf')
ORDER BY n.nspname, c.relname
`);
res.json(result.rows);
} catch (err) {
@ -31,12 +35,8 @@ module.exports = function(pool) {
return res.status(400).json({ error: 'Invalid schema or table name' });
}
try {
const cols = await pool.query(`
SELECT column_name, data_type, is_nullable, ordinal_position
FROM information_schema.columns
WHERE table_schema = $1 AND table_name = $2
ORDER BY ordinal_position
`, [schema, tname]);
const cols = await pool.query(
`${RELATION_COLUMNS_SQL} ORDER BY ordinal_position`, [schema, tname]);
const rows = await pool.query(
`SELECT * FROM ${schema}.${tname} LIMIT 5`

View File

@ -1,6 +1,7 @@
const express = require('express');
const { fcTable, mapType } = require('../lib/utils');
const { sessionUser } = require('../lib/auth');
const { fcTable, mapType, RELATION_COLUMNS_SQL } = require('../lib/utils');
const { sessionUser, sessionTerritory } = require('../lib/auth');
const { buildTerritoryClause } = require('../lib/sql_generator');
module.exports = function(pool) {
const router = express.Router();
@ -38,8 +39,9 @@ module.exports = function(pool) {
}
const source = srcResult.rows[0];
// fetch col_meta joined to information_schema for data types
// col_meta joined to the source's real columns, for the data types
const colResult = await client.query(`
WITH cols AS (${RELATION_COLUMNS_SQL})
SELECT
m.cname,
m.role,
@ -48,14 +50,11 @@ module.exports = function(pool) {
i.numeric_precision,
i.numeric_scale
FROM pf.col_meta m
JOIN information_schema.columns i
ON i.table_schema = $2
AND i.table_name = $3
AND i.column_name = m.cname
WHERE m.source_id = $1
JOIN cols i ON i.column_name = m.cname
WHERE m.source_id = $3
AND m.role NOT IN ('ignore')
ORDER BY m.opos
`, [sourceId, source.schema, source.tname]);
`, [source.schema, source.tname, sourceId]);
if (colResult.rows.length === 0) {
return res.status(400).json({
@ -127,10 +126,74 @@ ${colDefs},
// where this version's writes actually land: the physical forecast table,
// its current row count, and the source table rows are read from.
// Surfaced in the status bar so the write target is never a mystery.
// Distinct values of a dimension as they appear in one version's forecast
// table, for completing recode and clone.
//
// Deliberately not the source: the source view reaches back over all of
// history, so completing from it offers parts discontinued years ago. The
// version holds what was actually loaded, which is what a forecast is being
// written against.
//
// Held in memory because the scan is not cheap -- 2.0s for 11,290 parts across
// 2.5M rows on fc_osm_skinny_29 -- and completion is typed into. Keyed on the
// version's latest log id, so any load, adjustment or undo rebuilds it on the
// next request without anything having to remember to invalidate.
const valueCache = new Map();
router.get('/versions/:id/values/:col', async (req, res) => {
const versionId = parseInt(req.params.id);
const col = req.params.col;
try {
const verResult = await pool.query(`
SELECT v.id, s.tname, s.id AS source_id
FROM pf.version v JOIN pf.source s ON s.id = v.source_id
WHERE v.id = $1
`, [versionId]);
if (!verResult.rows.length) return res.status(404).json({ error: 'Version not found' });
const { tname, source_id } = verResult.rows[0];
// the column name is interpolated, so it has to be one col_meta names
const okCol = await pool.query(`
SELECT 1 FROM pf.col_meta
WHERE source_id = $1 AND cname = $2 AND role IN ('dimension', 'date')
`, [source_id, col]);
if (!okCol.rows.length) {
return res.status(400).json({ error: `"${col}" is not a dimension on this source` });
}
const table = fcTable(tname, versionId);
const { rows: [{ rev }] } = await pool.query(
`SELECT coalesce(max(id), 0)::text AS rev FROM pf.log WHERE version_id = $1`,
[versionId]
);
const key = `${versionId}:${col}`;
let entry = valueCache.get(key);
if (!entry || entry.rev !== rev) {
const { rows } = await pool.query(
`SELECT DISTINCT "${col}"::text AS val FROM ${table}
WHERE "${col}" IS NOT NULL ORDER BY 1`
);
entry = { rev, values: rows.map(r => r.val) };
valueCache.set(key, entry);
}
const q = (req.query.q || '').trim().toLowerCase();
const limit = Math.min(parseInt(req.query.limit) || 50, 500);
const picked = q
? entry.values.filter(v => v.toLowerCase().includes(q))
: entry.values;
res.json(picked.slice(0, limit));
} catch (err) {
console.error(err);
res.status(err.status || 500).json({ error: err.message });
}
});
router.get('/versions/:id/table-info', async (req, res) => {
try {
const verResult = await pool.query(`
SELECT v.id, v.name, v.status, s.schema, s.tname
SELECT v.id, v.name, v.status, s.schema, s.tname, s.id AS source_id
FROM pf.version v
JOIN pf.source s ON s.id = v.source_id
WHERE v.id = $1
@ -146,10 +209,23 @@ ${colDefs},
);
const exists = existsResult.rows[0].exists;
// Scoped like every other read. Unscoped it reported the whole
// table to an account that can see a fraction of it -- the status
// bar's row count, and the figure the load progress promises, both
// came from here: a rep with 330k rows was told the load was
// fetching 2.8M.
const terrCol = (await pool.query(
`SELECT cname FROM pf.col_meta WHERE source_id = $1 AND is_territory LIMIT 1`,
[v.source_id]
)).rows[0]?.cname || null;
const terr = buildTerritoryClause(sessionTerritory(req), terrCol);
let rows = null, byIter = [];
if (exists) {
const countResult = await pool.query(
`SELECT pf_iter, count(*)::int AS n FROM ${fc} GROUP BY pf_iter ORDER BY pf_iter`
`SELECT pf_iter, count(*)::int AS n FROM ${fc}
${terr ? `WHERE ${terr}` : ''}
GROUP BY pf_iter ORDER BY pf_iter`
);
byIter = countResult.rows;
rows = byIter.reduce((a, r) => a + r.n, 0);
@ -243,22 +319,39 @@ ${colDefs},
}
});
// update version name, description, or exclude_iters
// update version name, description, exclude_iters, or the fallback display
// names.
//
// bucket_order is deliberately not settable: the bucket column order is the
// text in pf.log.bucket now, so a stored order would be a second answer to
// the same question, and a silent one -- nothing reads it.
//
// The three name columns are flag-and-value pairs rather than COALESCE:
// clearing one back to the built-in means writing null, which COALESCE on
// the value alone cannot tell from "not mentioned".
router.put('/versions/:id', async (req, res) => {
const { name, description, exclude_iters } = req.body;
const { name, description, exclude_iters,
adjustment_segment, adjustment_bucket, unlabeled_load } = req.body;
const set = (v) => (v === undefined ? null : (String(v).trim() || null));
try {
const result = await pool.query(`
UPDATE pf.version SET
name = COALESCE($2, name),
description = COALESCE($3, description),
exclude_iters = COALESCE($4, exclude_iters)
name = COALESCE($2, name),
description = COALESCE($3, description),
exclude_iters = COALESCE($4, exclude_iters),
adjustment_segment = CASE WHEN $5::bool THEN $6::text ELSE adjustment_segment END,
adjustment_bucket = CASE WHEN $7::bool THEN $8::text ELSE adjustment_bucket END,
unlabeled_load = CASE WHEN $9::bool THEN $10::text ELSE unlabeled_load END
WHERE id = $1
RETURNING *
`, [
req.params.id,
name || null,
description || null,
exclude_iters ? JSON.stringify(exclude_iters) : null
exclude_iters ? JSON.stringify(exclude_iters) : null,
adjustment_segment !== undefined, set(adjustment_segment),
adjustment_bucket !== undefined, set(adjustment_bucket),
unlabeled_load !== undefined, set(unlabeled_load)
]);
if (result.rows.length === 0) {
return res.status(404).json({ error: 'Version not found' });

View File

@ -80,6 +80,7 @@ app.use('/api', require('./routes/sources')(pool));
app.use('/api', require('./routes/versions')(pool));
app.use('/api', require('./routes/operations')(pool));
app.use('/api', require('./routes/log')(pool));
app.use('/api', require('./routes/layouts')(pool));
const port = process.env.PORT || 3010;

View File

@ -81,6 +81,119 @@ WHERE TRUE
AND note <> ''
AND operation IN ('baseline', 'reference');
-- Display order for the pivot's segment and bucket columns.
--
-- The segment's display name in the pivot, falling back to tag then note.
--
-- Separate from both because those have jobs already -- tag groups adjustments
-- into initiatives for the bridge, note is free commentary -- and because the
-- label carries the sort order. Perspective orders column groups by the value
-- string, so a leading "01 - " is how ordering is expressed; putting that in the
-- note would put it in every note.
ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS label text;
-- Vestigial, both of them. They held the ordinal when the "01 - " prefix was
-- computed for the pivot rather than typed into label and bucket: bucket_order
-- sequenced the bucket columns, log.seq the segments within them. Nothing reads
-- either now, and nothing writes them -- kept only because dropping a column is
-- not worth a migration to reclaim two that cost nothing.
ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS bucket_order jsonb;
ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS seq integer;
-- What a segment contributes to, independent of pf_iter.
--
-- pf_iter answers "can operations write to these rows"; bucket answers "does this
-- belong in the forecast number". Those are not the same question -- Open Orders is
-- loaded as reference so nothing adjusts it, yet it is part of the forecast -- so
-- neither can be derived from the other.
--
-- Free text with suggested values (Forecast / Prior Year / Prior Prior Year / Plan)
-- rather than an enum, so a new banner does not need a migration. Blank by default:
-- until a segment is labelled, the pivot falls back to showing its own name.
ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS bucket text;
CREATE INDEX IF NOT EXISTS log_bucket_idx ON pf.log (bucket) WHERE bucket IS NOT NULL;
-- The names a row falls back to when nobody has named it, per scenario. Null
-- means "use the built-in", which is the DISPLAY DEFAULTS block in
-- lib/sql_generator.js. Read through a join at query time, not baked into
-- pf.sql: those templates are keyed on (source_id, operation) and shared by
-- every version of a source.
ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS adjustment_segment text;
ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS adjustment_bucket text;
ALTER TABLE pf.version ADD COLUMN IF NOT EXISTS unlabeled_load text;
-- What the entry did, stamped when it did it.
--
-- Not a cache: a log entry's forecast rows never change after it is written.
-- Only the operation that owns the logid inserts them, and the only thing that
-- removes them is undo, which deletes this row too -- so these totals are fixed
-- at write time rather than derived from something that can move underneath
-- them. The change log was joining the whole forecast table to recompute them
-- on every open, 2.5M rows to total a few thousand.
--
-- measure_cols records which columns they are denominated in, since the value
-- and units roles can be reassigned in col_meta and the numbers would otherwise
-- quietly come to mean something else.
ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS row_count integer;
ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS value_total double precision;
ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS units_total double precision;
ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS measure_cols jsonb;
-- The column an account's territory is expressed in -- the rep, the region,
-- whatever this source divides ownership by. Exactly one per source.
--
-- Flagged here rather than named in code because it is source-specific, the
-- same way the grain is: a second source should not need a code change to be
-- scoped by something other than a sales rep.
--
-- It does two jobs. The server scopes every read and write to the values on the
-- account, and recode refuses to *set* this column unless the account is an
-- admin -- moving a row between territories is reassignment, not forecasting.
ALTER TABLE pf.col_meta ADD COLUMN IF NOT EXISTS is_territory boolean NOT NULL DEFAULT false;
-- The debug path: what the entry was trying to do, and what that became.
--
-- Both, deliberately. params records the intent -- the slice, the scope, the
-- resolved increments -- and sql_text records the statement as executed, with
-- territory and scope already resolved into it. Keeping only one of them
-- assumes the translation between them is correct, which is exactly the
-- assumption in doubt when the rows look wrong.
--
-- env captures the state the intent was executed against that cannot be
-- reconstructed afterwards: the territory in force, the version's
-- exclude_iters, and when the template was generated. All three are mutable
-- rows elsewhere, and nothing remembers what they were.
--
-- The template generation is a fingerprint, not a version: it cannot reproduce
-- the old template, but it can tell you the entry ran under a different one,
-- which is what would otherwise make a replay quietly wrong.
ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS env jsonb;
ALTER TABLE pf.log ADD COLUMN IF NOT EXISTS sql_text text;
-- Master data for a dim_group: one row per key value, with its sibling columns.
--
-- The source is transactional and often a view over all history, so deriving a
-- member list from it is both slow and wrong -- slow because it means scanning
-- millions of rows, wrong because it can only describe what was transacted and
-- has no way to say a part is discontinued or that a new one exists before it
-- has sold. This table is the app's own list, refreshed from the source but
-- curatable independently of it.
CREATE TABLE IF NOT EXISTS pf.dim_member (
source_id integer NOT NULL REFERENCES pf.source(id) ON DELETE CASCADE,
dim_group text NOT NULL, -- matches pf.col_meta.dim_group
key_value text NOT NULL, -- the is_key column's value
attrs jsonb NOT NULL DEFAULT '{}'::jsonb, -- the sibling columns
is_active boolean NOT NULL DEFAULT true,
source_seen boolean NOT NULL DEFAULT true, -- present in the source at last refresh
added_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
refreshed_at timestamptz,
PRIMARY KEY (source_id, dim_group, key_value)
);
CREATE INDEX IF NOT EXISTS dim_member_active_idx
ON pf.dim_member (source_id, dim_group) WHERE is_active;
-- generated operation SQL per source, stored after col_meta is configured
CREATE TABLE IF NOT EXISTS pf.sql (
id serial PRIMARY KEY,
@ -90,3 +203,61 @@ CREATE TABLE IF NOT EXISTS pf.sql (
generated_at timestamptz NOT NULL DEFAULT now(),
UNIQUE (source_id, operation)
);
-- pf.layout: named Perspective view configs.
--
-- Replaces the per-browser localStorage lists (pf_layouts_v*) and the single
-- anonymous pf.source.default_layout. A layout is either `private` -- visible
-- only to its owner -- or `published`, visible to everyone on the version and
-- writable only by its owner or an admin, the same rule pf.log already uses.
CREATE TABLE IF NOT EXISTS pf.layout (
id serial PRIMARY KEY,
source_id integer NOT NULL REFERENCES pf.source(id) ON DELETE CASCADE,
-- null = applies to every version of the source; that is where the old
-- source default lives, and what a new version picks up before anyone has
-- published a layout of its own.
version_id integer REFERENCES pf.version(id) ON DELETE CASCADE,
name text NOT NULL,
config jsonb NOT NULL,
owner text NOT NULL,
visibility text NOT NULL DEFAULT 'private', -- private | published
is_default boolean NOT NULL DEFAULT false, -- applied on first load
created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(),
CHECK (visibility IN ('private', 'published')),
-- only a published layout can be the default: a private one would be a
-- default nobody else could see.
CHECK (NOT is_default OR visibility = 'published')
);
-- COALESCE rather than the bare column: version_id is nullable, and in a unique
-- index NULLs are distinct, so source-wide rows would not be constrained at all.
CREATE UNIQUE INDEX IF NOT EXISTS layout_private_name
ON pf.layout (source_id, COALESCE(version_id, 0), owner, lower(name))
WHERE visibility = 'private';
CREATE UNIQUE INDEX IF NOT EXISTS layout_published_name
ON pf.layout (source_id, COALESCE(version_id, 0), lower(name))
WHERE visibility = 'published';
-- one default per scope
CREATE UNIQUE INDEX IF NOT EXISTS layout_one_default
ON pf.layout (source_id, COALESCE(version_id, 0))
WHERE is_default;
CREATE INDEX IF NOT EXISTS layout_lookup ON pf.layout (source_id, version_id);
-- Carry the old single source default across as a published, source-wide row.
-- Idempotent: skipped once a default exists for that source.
INSERT INTO pf.layout (source_id, version_id, name, config, owner, visibility, is_default)
SELECT s.id, NULL, 'Source default', s.default_layout, COALESCE(s.created_by, 'admin'), 'published', true
FROM pf.source s
WHERE TRUE
AND s.default_layout IS NOT NULL
AND NOT EXISTS (
SELECT 1 FROM pf.layout l
WHERE TRUE
AND l.source_id = s.id
AND l.version_id IS NULL
AND l.is_default
);

View File

@ -24,3 +24,15 @@ CREATE TABLE IF NOT EXISTS pf.session (
);
CREATE INDEX IF NOT EXISTS session_expire_idx ON pf.session (expire);
-- What an account may see and change.
--
-- territory is the list of values allowed in the source's territory column
-- (col_meta.is_territory). It is a filter the server applies to every read and
-- every write; it is never sent by the client, which could remove it.
--
-- Fail closed: no territory and not an admin means no rows. An account created
-- without one sees nothing until someone grants it, rather than seeing the whole
-- book because a column was left null.
ALTER TABLE pf.app_user ADD COLUMN IF NOT EXISTS territory jsonb;
ALTER TABLE pf.app_user ADD COLUMN IF NOT EXISTS is_admin boolean NOT NULL DEFAULT false;

16
ui/package-lock.json generated
View File

@ -8,10 +8,10 @@
"name": "ui",
"version": "0.0.0",
"dependencies": {
"@perspective-dev/client": "file:./vendor/perspective-dev-client-5.4.0.tgz",
"@perspective-dev/server": "file:./vendor/perspective-dev-server-5.4.0.tgz",
"@perspective-dev/viewer": "file:./vendor/perspective-dev-viewer-5.4.0.tgz",
"@perspective-dev/viewer-datagrid": "file:./vendor/perspective-dev-viewer-datagrid-5.4.0.tgz",
"@perspective-dev/client": "file:vendor/perspective-dev-client-5.4.0.tgz",
"@perspective-dev/server": "file:vendor/perspective-dev-server-5.4.0.tgz",
"@perspective-dev/viewer": "file:vendor/perspective-dev-viewer-5.4.0.tgz",
"@perspective-dev/viewer-datagrid": "file:vendor/perspective-dev-viewer-datagrid-5.4.0.tgz",
"react": "^19.2.5",
"react-dom": "^19.2.5"
},
@ -578,7 +578,7 @@
"node_modules/@perspective-dev/client": {
"version": "5.4.0",
"resolved": "file:vendor/perspective-dev-client-5.4.0.tgz",
"integrity": "sha512-l9xCJ0W42wm9fLlwhoU838KyTE131qQTS686uQb/VcsLs30kCVjkoermCCUs0YHCDwDTJhly1y9DfAZDbkg9Ng==",
"integrity": "sha512-DOk5TM4+SpDW29w6wJoHtH6oZVSon9YZjkMfKap4Lmf8SLp2PYwec/LouetkMfg4CvveGJemG4domsl8XeNafA==",
"license": "Apache-2.0",
"dependencies": {
"@perspective-dev/server": "^5.4.0",
@ -590,13 +590,13 @@
"node_modules/@perspective-dev/server": {
"version": "5.4.0",
"resolved": "file:vendor/perspective-dev-server-5.4.0.tgz",
"integrity": "sha512-iTseRJB6TL6D9xjaMKMhh2NEKMIi9JR881J+GyQflHIQXK43fDlsIWtByUAoyZzZ7uA9KNZJZicirvONxIpLuw==",
"integrity": "sha512-Z5oGEOYqTHMGoMElYbrfJL3YCtoGrAX6+22+3KwrxWDDLB5OlOOdsctfeEAdGqVeJ1/WFhqZp0fWBPEn4vcXrQ==",
"license": "Apache-2.0"
},
"node_modules/@perspective-dev/viewer": {
"version": "5.4.0",
"resolved": "file:vendor/perspective-dev-viewer-5.4.0.tgz",
"integrity": "sha512-7D6jNn7tDZ3W84MsydplWAJbqqpXIz5OlsHx5YG4sHvJ5q8ngZQvP+oZdxLQXL4hIbaxpskMld1gJbGltpcvUw==",
"integrity": "sha512-2y2xRb5k9BedpnRXcaweRYC2LiP/mblNqbFaRtpqNhnVmp+c3pwO6kx57oRuLhyzkcmNZUAzlqZvs1MJNDn50g==",
"license": "Apache-2.0",
"dependencies": {
"@perspective-dev/client": "^5.4.0",
@ -607,7 +607,7 @@
"node_modules/@perspective-dev/viewer-datagrid": {
"version": "5.4.0",
"resolved": "file:vendor/perspective-dev-viewer-datagrid-5.4.0.tgz",
"integrity": "sha512-7ITOmrIh1ZpiQbUR/KmHYck1sLA4cpNgpVMI/qJjQc9k/uypkT1vJmKYxv4MKR24TmEkOouBLNYiO+ItFKs8vg==",
"integrity": "sha512-ccJOioKVPAGp/lv7RGSP2rtYj0PfwkJrNZWFMRJkmdz+eAPiHViDZhIHudOuA/2DOmcz6LbD617XaPasTU0Uog==",
"license": "Apache-2.0",
"dependencies": {
"@perspective-dev/client": "^5.4.0",

View File

@ -10,10 +10,10 @@
"preview": "vite preview"
},
"dependencies": {
"@perspective-dev/client": "file:./vendor/perspective-dev-client-5.4.0.tgz",
"@perspective-dev/server": "file:./vendor/perspective-dev-server-5.4.0.tgz",
"@perspective-dev/viewer": "file:./vendor/perspective-dev-viewer-5.4.0.tgz",
"@perspective-dev/viewer-datagrid": "file:./vendor/perspective-dev-viewer-datagrid-5.4.0.tgz",
"@perspective-dev/client": "file:vendor/perspective-dev-client-5.4.0.tgz",
"@perspective-dev/server": "file:vendor/perspective-dev-server-5.4.0.tgz",
"@perspective-dev/viewer": "file:vendor/perspective-dev-viewer-5.4.0.tgz",
"@perspective-dev/viewer-datagrid": "file:vendor/perspective-dev-viewer-datagrid-5.4.0.tgz",
"react": "^19.2.5",
"react-dom": "^19.2.5"
},

View File

@ -49,25 +49,85 @@ function niceTicks(min, max, count = 5) {
// Turn raw forecast rows into the walk: baseline anchor, one floating step per
// initiative tag, current anchor. Pure and exported so the arithmetic can be
// checked against real data without a browser.
export function buildSteps(rows, { valueCol, unitsCol, logMeta = {}, excludeIters = ['reference'] }) {
// The walk from a basis to the current forecast.
//
// Without a basis this is the forecast's own composition: its loads as the
// opening anchor, then one step per adjustment tag.
//
// With one -- Plan when building a forecast, Prior Year when building the AOP --
// it starts there instead, and the identity that makes it exact is
//
// Forecast - Basis = (Forecast loads - Basis) + adjustments
//
// so the opening step is the difference between the forecast's own loads and the
// basis, and every tagged adjustment explains the rest. No residual, no plug: the
// bars sum to the endpoint by construction rather than by hoping the tags cover
// everything.
//
// Membership comes from pf_bucket, not pf_iter. Those answer different questions
// -- Open Orders is loaded as reference so nothing adjusts it, and is still part
// of the forecast -- so a bridge keyed on iter silently dropped it.
export function buildSteps(rows, {
valueCol, unitsCol, logMeta = {},
excludeIters = ['reference'], // kept for callers with no bucket data
basis = null, // a pf_bucket name, or null for composition
forecastBucket,
}) {
const hasBuckets = rows.some(r => r.pf_bucket != null)
const excl = new Set(excludeIters)
const baseline = { value: 0, units: 0, rows: 0 }
const byTag = new Map()
// Which bucket is the forecast. It used to be the literal 'Forecast', and the
// moment the buckets were renamed to carry their sort prefix -- '04 - Forecast'
// -- no row matched it: every row counted as a comparison, the walk had no
// steps, and the bridge showed the basis cancelling itself to zero.
//
// The caller passes the version's adjustment_bucket, which is the same value
// an unbucketed adjustment is labelled with, so the two agree by construction.
// Falling back to whichever bucket actually holds the adjustments keeps a
// renamed or unconfigured version working rather than silently empty.
const fcBucket = (() => {
if (forecastBucket && rows.some(r => r.pf_bucket === forecastBucket)) return forecastBucket
const adjusted = rows.find(r => r.pf_bucket && !['baseline', 'reference'].includes(r.pf_iter))
if (adjusted) return adjusted.pf_bucket
const baseline = rows.find(r => r.pf_bucket && r.pf_iter === 'baseline')
return baseline ? baseline.pf_bucket : (forecastBucket || 'Forecast')
})()
const num = (r, col) => (col ? (parseFloat(r[col]) || 0) : 0)
const blank = () => ({ value: 0, units: 0, rows: 0 })
const loads = blank() // the forecast's own segments
const basisT = blank() // the comparison bucket
const byTag = new Map()
const buckets = new Map() // every bucket, for the picker and the markers
for (const r of rows) {
const iter = r.pf_iter
if (excl.has(iter)) continue
const v = parseFloat(r[valueCol]) || 0
const u = unitsCol ? (parseFloat(r[unitsCol]) || 0) : 0
const v = num(r, valueCol)
const u = num(r, unitsCol)
const bucket = hasBuckets ? (r.pf_bucket || '') : null
if (iter === 'baseline') {
baseline.value += v; baseline.units += u; baseline.rows += 1
if (bucket != null) {
const b = buckets.get(bucket) || blank()
b.value += v; b.units += u; b.rows += 1
buckets.set(bucket, b)
}
// Anything outside the forecast is a comparison, never a step.
if (hasBuckets && bucket !== fcBucket) {
if (basis && bucket === basis) { basisT.value += v; basisT.units += u; basisT.rows += 1 }
continue
}
if (!hasBuckets && excl.has(r.pf_iter)) continue
const isLoad = r.pf_iter === 'baseline' || r.pf_iter === 'reference'
if (isLoad) { loads.value += v; loads.units += u; loads.rows += 1; continue }
const meta = logMeta[r.pf_logid] || {}
const tag = (meta.tag || '').trim()
// label first, the same precedence pf_segment uses, so the bridge and the
// pivot call a step by the same name
const tag = (meta.label || meta.tag || '').trim()
const label = tag || (meta.note || '').trim() ||
`${(meta.operation || iter || 'adj')}${r.pf_logid != null ? ` #${r.pf_logid}` : ''}`
`${(meta.operation || r.pf_iter || 'adj')}${r.pf_logid != null ? ` #${r.pf_logid}` : ''}`
const key = tag ? `tag:${tag}` : `log:${r.pf_logid}`
const g = byTag.get(key) ||
{ key, label, tagged: !!tag, value: 0, units: 0, rows: 0, logIds: new Set(), first: r.pf_logid }
@ -77,31 +137,96 @@ export function buildSteps(rows, { valueCol, unitsCol, logMeta = {}, excludeIter
}
const mid = [...byTag.values()].sort((a, b) => (a.first ?? 0) - (b.first ?? 0))
const useBasis = !!basis && basisT.rows > 0
let running = baseline.value
const out = [{
key: 'baseline', label: 'Baseline', kind: 'anchor',
delta: baseline.value, start: 0, end: baseline.value,
units: baseline.units, rows: baseline.rows, entries: 1,
}]
const out = []
if (useBasis) {
out.push({
key: 'basis', label: basis, kind: 'anchor',
delta: basisT.value, start: 0, end: basisT.value,
units: basisT.units, rows: basisT.rows, entries: 1,
})
const gap = loads.value - basisT.value
out.push({
// "Loads" is our word for a segment import and means nothing to anyone
// reading a waterfall. This step is simply where the forecast began
// relative to the comparison.
//
// tagged, though it has no tag: the flag drives an "· untagged" suffix
// meant for adjustments nobody grouped into an initiative, and this is
// not an adjustment at all.
key: 'loads-vs-basis', label: `Starting point vs ${basis}`, kind: 'step',
tagged: true,
delta: gap, start: basisT.value, end: loads.value,
units: loads.units - basisT.units, rows: loads.rows, entries: 1,
})
} else {
out.push({
key: 'baseline', label: hasBuckets ? fcBucket + ' loads' : 'Baseline', kind: 'anchor',
delta: loads.value, start: 0, end: loads.value,
units: loads.units, rows: loads.rows, entries: 1,
})
}
let running = loads.value
for (const g of mid) {
const start = running
running += g.value
out.push({ ...g, kind: 'step', delta: g.value, start, end: running, entries: g.logIds.size })
}
out.push({
key: 'current', label: 'Current', kind: 'anchor',
key: 'current', label: hasBuckets ? fcBucket : 'Current', kind: 'anchor',
delta: running, start: 0, end: running,
units: out.reduce((a, s) => a + (s.kind === 'anchor' ? 0 : s.units || 0), baseline.units),
rows: rows.filter(r => !excl.has(r.pf_iter)).length,
units: loads.units + mid.reduce((a, g) => a + (g.units || 0), 0),
rows: loads.rows + mid.reduce((a, g) => a + g.rows, 0),
entries: mid.reduce((a, g) => a + g.logIds.size, 0) + 1,
})
// Every bucket present, so the view can offer them as bases and show the ones
// that are not the basis as comparison markers.
out.buckets = [...buckets.entries()]
.map(([name, t]) => ({ name, ...t }))
.sort((a, b) => b.value - a.value)
return out
}
// Plot geometry, also pure: given the steps and a canvas size, where does each
// bar and label land? Exported so collisions and overflow can be checked.
export function layoutSteps(steps, width, H = 340, PAD = { t: 24, r: 16, b: 64, l: 68 }) {
// Break a label to the bar's width, on word boundaries.
//
// SVG text does not wrap, so the labels were cut at twelve characters and
// "04 - Forecast" and "04 - Forecasting" read the same. Character width is
// estimated rather than measured -- measuring means a DOM round trip per label
// on every render, and at this font a digit is about 0.55em, which is close
// enough for a centred label with a bar's width to play with.
//
// Three lines maximum, and a word longer than the line is left to overflow
// rather than broken: a truncated word helps nobody, and the bars are wide
// enough that it only happens to things like a pasted part number.
function wrapLabel(label, barW, fontSize = 10, maxLines = 3) {
const perLine = Math.max(6, Math.floor(barW / (fontSize * 0.55)))
const words = String(label || '').split(/\s+/).filter(Boolean)
const lines = []
let line = ''
for (const w of words) {
const next = line ? `${line} ${w}` : w
if (next.length <= perLine) { line = next; continue }
if (line) lines.push(line)
line = w
if (lines.length === maxLines) break
}
if (line && lines.length < maxLines) lines.push(line)
if (!lines.length) return ['']
// anything that did not fit is marked on the last line rather than dropped
const used = lines.join(' ').length
if (used < String(label).replace(/\s+/g, ' ').length) {
lines[lines.length - 1] = `${lines[lines.length - 1]}`
}
return lines
}
export function layoutSteps(steps, width, H = 340, PAD = { t: 24, r: 16, b: 84, l: 68 }) {
const plotW = Math.max(120, width - PAD.l - PAD.r)
const plotH = H - PAD.t - PAD.b
const values = steps.flatMap(s => [s.start, s.end])
@ -127,11 +252,14 @@ export function layoutSteps(steps, width, H = 340, PAD = { t: 24, r: 16, b: 64,
export default function BridgeView({
open, onClose, tableRef, viewerRef, logMeta = {},
valueCol, unitsCol, colMeta = [], slices = [],
excludeIters = ['reference'], versionName,
excludeIters = ['reference'], versionName, forecastBucket,
}) {
const hasSelection = slices.length > 0
// 'selection' | 'filtered' | 'all'
const [scope, setScope] = useState(hasSelection ? 'selection' : 'filtered')
// Which bucket the walk starts from. Plan when building a forecast, Prior Year
// when building the AOP; empty means show the forecast's own composition.
const [basis, setBasis] = useState(() => localStorage.getItem('pf_bridge_basis') || '')
const [asTable, setAsTable] = useState(false)
const [steps, setSteps] = useState(null)
const [loading, setLoading] = useState(false)
@ -160,6 +288,8 @@ export default function BridgeView({
...Object.entries(sl).filter(([c]) => dateNames.has(c)).map(([c, v]) => [c, '==', Number(v)]),
]
if (!f.length) continue
// No expressions needed here: these filters are built from col_meta
// names, so they only ever reference real columns.
const view = await tableRef.current.view({ filter: f })
const part = await view.to_json()
await view.delete()
@ -171,28 +301,39 @@ export default function BridgeView({
}
} else {
let filter = []
// Expression columns have to come with the filter that uses them: a
// filter can name a column that exists only as an expression, and a view
// built without it cannot resolve the column and fails outright.
let expressions = {}
if (scope === 'filtered' && viewerRef?.current) {
const cfg = await viewerRef.current.save()
filter = (cfg.filter || []).filter(f => Array.isArray(f) && f.length >= 2)
expressions = cfg.expressions || {}
}
const view = await tableRef.current.view(filter.length ? { filter } : {})
const viewCfg = {}
if (filter.length) viewCfg.filter = filter
if (Object.keys(expressions).length) viewCfg.expressions = expressions
const view = await tableRef.current.view(viewCfg)
rows = await view.to_json()
await view.delete()
}
setSteps(buildSteps(rows, { valueCol, unitsCol, logMeta, excludeIters }))
setSteps(buildSteps(rows, {
valueCol, unitsCol, logMeta, excludeIters, basis: basis || null, forecastBucket,
}))
} catch (err) {
setError(err.message || String(err))
setSteps(null)
} finally {
setLoading(false)
}
}, [tableRef, viewerRef, scope, logMeta, valueCol, unitsCol, excludeIters, slices, colMeta])
}, [tableRef, viewerRef, scope, logMeta, valueCol, unitsCol, excludeIters, slices, colMeta, basis, forecastBucket])
useEffect(() => {
if (!hasSelection && scope === 'selection') setScope('filtered')
}, [hasSelection, scope])
useEffect(() => { try { localStorage.setItem('pf_bridge_basis', basis) } catch {} }, [basis])
useEffect(() => { if (open) compute() }, [open, compute])
useEffect(() => {
@ -247,6 +388,23 @@ export default function BridgeView({
</button>
))}
</div>
{/* Offered only once something carries a bucket, and only listing buckets
other than the forecast itself -- a walk from Forecast to Forecast is
the composition view, which is what the empty option gives. */}
{(steps?.buckets || []).some(b => b.name && b.name !== 'Forecast') && (
<>
<div className="w-px h-4 bg-gray-200" />
<span className="text-gray-600">Compare to</span>
<select value={basis} onChange={e => setBasis(e.target.value)}
className="border border-gray-200 rounded px-2 py-1 text-gray-700 bg-white">
<option value="">nothing show composition</option>
{(steps?.buckets || [])
.filter(b => b.name && b.name !== 'Forecast')
.map(b => <option key={b.name} value={b.name}>{b.name}</option>)}
</select>
</>
)}
<div className="w-px h-4 bg-gray-200" />
<button onClick={() => setAsTable(t => !t)}
className="border border-gray-200 rounded px-2 py-1 text-gray-700 hover:bg-gray-50">
@ -302,6 +460,8 @@ export default function BridgeView({
const h = Math.max(2, bot - top)
const x = xOf(i)
const on = hover?.key === s.key
const labelLines = wrapLabel(s.label, barW)
const subY = PAD.t + plotH + 16 + labelLines.length * 11 + 2
return (
<g key={s.key}
onMouseEnter={() => setHover({ ...s, x: x + barW / 2, y: top })}
@ -322,15 +482,18 @@ export default function BridgeView({
{isAnchor ? fmt(s.end, 0) : fmtSigned(s.delta, 0)}
</text>
<text x={x + barW / 2} y={PAD.t + plotH + 16} textAnchor="middle" fontSize="10" fill={INK}>
{s.label.length > 12 ? `${s.label.slice(0, 11)}` : s.label}
{labelLines.map((ln, li) => (
<tspan key={li} x={x + barW / 2} dy={li === 0 ? 0 : 11}>{ln}</tspan>
))}
</text>
{/* below the label, however many lines it took */}
{!isAnchor && s.entries > 1 && (
<text x={x + barW / 2} y={PAD.t + plotH + 29} textAnchor="middle" fontSize="9" fill={INK_DIM}>
<text x={x + barW / 2} y={subY} textAnchor="middle" fontSize="9" fill={INK_DIM}>
×{s.entries}
</text>
)}
{!s.tagged && !isAnchor && (
<text x={x + barW / 2} y={PAD.t + plotH + 29} textAnchor="middle" fontSize="9" fill={INK_DIM}>
<text x={x + barW / 2} y={subY} textAnchor="middle" fontSize="9" fill={INK_DIM}>
untagged
</text>
)}

View File

@ -0,0 +1,190 @@
import { useEffect, useRef, useState } from 'react'
// The pivot layout, as one control rather than a row of equal chips.
//
// Two lists, because a layout is one of two different things: Published, which
// everyone on this forecast sees and only its owner or an admin may change, and
// Mine, which nobody else lists. Everything you cannot write is still there to
// apply -- it is only the write controls that are withheld, so a click never
// answers 403.
export default function LayoutMenu({
layouts = [], activeLayoutId, dirty,
onApply, onSave, onSaveAs, onSetVisibility, onSetDefault, onRename, onDelete, onReset
}) {
const [open, setOpen] = useState(false)
const [saveName, setSaveName] = useState('')
const [savePublic, setSavePublic] = useState(false)
const [renaming, setRenaming] = useState(null)
const [renameTo, setRenameTo] = useState('')
const boxRef = useRef(null)
// Close on an outside click or Escape. Capture phase: the pivot lives in a
// shadow root and retargets its events, so a bubbled listener on document
// sees the host rather than the real target and cannot tell inside from out.
useEffect(() => {
if (!open) return
const onDown = (e) => { if (!boxRef.current?.contains(e.target)) setOpen(false) }
const onKey = (e) => { if (e.key === 'Escape') setOpen(false) }
document.addEventListener('mousedown', onDown, true)
window.addEventListener('keydown', onKey)
return () => {
document.removeEventListener('mousedown', onDown, true)
window.removeEventListener('keydown', onKey)
}
}, [open])
const active = layouts.find(l => l.id === activeLayoutId)
const published = layouts.filter(l => l.visibility === 'published')
const mine = layouts.filter(l => l.visibility === 'private')
const submitSaveAs = () => {
const name = saveName.trim()
if (!name) return
onSaveAs(name, savePublic ? 'published' : 'private')
setSaveName('')
setOpen(false)
}
const submitRename = (l) => {
const name = renameTo.trim()
if (name && name !== l.name) onRename(l, name)
setRenaming(null)
}
const iconBtn = 'px-1 rounded text-gray-400 hover:text-gray-700 hover:bg-gray-100 leading-none'
const row = (l) => (
<div key={l.id}
className={`group flex items-center gap-1 px-2 py-1 rounded cursor-pointer
${l.id === activeLayoutId ? 'bg-blue-50 text-blue-700' : 'hover:bg-gray-50 text-gray-700'}`}
onClick={() => { if (renaming !== l.id) { onApply(l); setOpen(false) } }}>
{renaming === l.id ? (
<input autoFocus value={renameTo} onClick={e => e.stopPropagation()}
onChange={e => setRenameTo(e.target.value)}
onBlur={() => submitRename(l)}
onKeyDown={e => {
if (e.key === 'Enter') submitRename(l)
if (e.key === 'Escape') setRenaming(null)
}}
className="flex-1 min-w-0 border border-blue-300 rounded px-1 py-0 outline-none bg-white" />
) : (
<>
<span className="flex-1 min-w-0 truncate">{l.name}</span>
{l.is_default && (
<span title="Applied when this forecast is opened for the first time"
className="text-amber-500 shrink-0"></span>
)}
{l.scope === 'source' && (
<span title="Applies to every forecast of this source"
className="text-gray-300 shrink-0" style={{fontSize:'9px'}}>ALL</span>
)}
</>
)}
{/* Withheld rather than offered-and-refused: the server allows a write only
to the owner or an admin, and can_edit says which this is. */}
{l.can_edit && renaming !== l.id && (
<span className="hidden group-hover:flex items-center gap-0.5 shrink-0"
onClick={e => e.stopPropagation()}>
<button className={iconBtn} title="Rename"
onClick={() => { setRenaming(l.id); setRenameTo(l.name) }}></button>
{l.visibility === 'private' ? (
<button className={iconBtn} title="Publish — everyone on this forecast sees it"
onClick={() => onSetVisibility(l, 'published')}></button>
) : (
<>
{!l.is_default && (
<button className={iconBtn} title="Apply this when the forecast is first opened"
onClick={() => onSetDefault(l)}></button>
)}
<button className={iconBtn} title="Unpublish — keep it, but only for you"
onClick={() => onSetVisibility(l, 'private')}></button>
</>
)}
<button className={`${iconBtn} hover:text-red-500`} title="Delete"
onClick={() => onDelete(l)}>×</button>
</span>
)}
{!l.can_edit && (
<span className="hidden group-hover:inline text-gray-300 shrink-0 truncate"
style={{fontSize:'9px'}} title={`Published by ${l.owner}`}>{l.owner}</span>
)}
</div>
)
return (
<div ref={boxRef} className="relative flex items-center gap-1.5">
<span className="text-gray-400 uppercase tracking-wide" style={{fontSize:'10px'}}>Layout</span>
<button onClick={() => setOpen(o => !o)}
className={`flex items-center gap-1 border rounded px-2 py-0.5 max-w-[14rem] transition-colors
${open ? 'border-blue-300 text-blue-700 bg-blue-50'
: 'border-gray-200 text-gray-600 hover:border-gray-400'}`}>
<span className="truncate">{active ? active.name : 'Unsaved'}</span>
{dirty && <span title="Changed since it was saved" className="text-amber-500 leading-none"></span>}
<span className="text-gray-400 leading-none"></span>
</button>
{/* Save is the common action and stays outside the menu, but only when
there is something to save it to and it is yours to overwrite. */}
{active?.can_edit && dirty && (
<button onClick={() => onSave(active)}
className="border border-blue-200 text-blue-600 hover:text-blue-800 rounded px-2 py-0.5">
Save
</button>
)}
{open && (
<div className="absolute top-full left-0 mt-1 z-50 w-72 bg-white border border-gray-200
rounded shadow-lg py-1 text-xs">
{published.length > 0 && (
<>
<div className="px-2 pt-1 pb-0.5 text-gray-400 uppercase tracking-wide"
style={{fontSize:'9px'}}>Published</div>
<div className="px-1">{published.map(row)}</div>
</>
)}
{mine.length > 0 && (
<>
<div className="px-2 pt-2 pb-0.5 text-gray-400 uppercase tracking-wide"
style={{fontSize:'9px'}}>Mine</div>
<div className="px-1">{mine.map(row)}</div>
</>
)}
{!layouts.length && (
<div className="px-3 py-2 text-gray-400">No saved layouts yet</div>
)}
<div className="border-t border-gray-100 mt-1 pt-1 px-2">
<div className="flex items-center gap-1 py-1">
<input value={saveName} onChange={e => setSaveName(e.target.value)}
onKeyDown={e => { if (e.key === 'Enter') submitSaveAs() }}
placeholder="Save current view as…"
className="flex-1 min-w-0 border border-gray-300 rounded px-1.5 py-0.5
outline-none focus:border-blue-400" />
<button onClick={submitSaveAs} disabled={!saveName.trim()}
className="text-blue-600 hover:text-blue-800 px-1 disabled:opacity-30">Save</button>
</div>
<label className="flex items-center gap-1.5 py-0.5 text-gray-500 cursor-pointer">
<input type="checkbox" checked={savePublic}
onChange={e => setSavePublic(e.target.checked)} />
Publish to everyone on this forecast
</label>
</div>
<div className="border-t border-gray-100 mt-1 pt-1 px-2 pb-0.5">
<button onClick={() => { onReset(); setOpen(false) }}
className="text-gray-400 hover:text-gray-700 py-0.5">
Reset to a blank pivot
</button>
</div>
</div>
)}
</div>
)
}

View File

@ -11,7 +11,7 @@
// flex-1 buttons grew to absurd sizes; everything here is fixed-width and
// left-aligned instead.
import { useState } from 'react'
import { useState, useEffect, useRef, useLayoutEffect } from 'react'
const INPUT = 'border border-gray-200 rounded px-2 py-1 text-xs bg-white w-28 text-right font-mono tabular-nums'
const TEXT = 'border border-gray-200 rounded px-2 py-1 text-xs bg-white w-40 font-mono'
@ -61,6 +61,34 @@ function Button({ onClick, active, children, title }) {
)
}
// A label and its control, on a grid. Every row in the panel uses the same label
// width, so the controls line up down the column instead of each row starting
// wherever its label happens to end -- which was the whole problem with
// "copy rows from" sitting above "scale cloned rows by" above "tag".
const LABEL_W = 'w-24'
function Field({ label, children, hint }) {
return (
<div className="flex items-start gap-2">
<span className={`text-gray-600 whitespace-nowrap shrink-0 pt-1 ${LABEL_W}`}>{label}</span>
<div className="flex flex-col gap-1 min-w-0">
<div className="flex items-center gap-2 flex-wrap">{children}</div>
{hint && <p className="text-gray-500 text-[11px] leading-snug max-w-xs">{hint}</p>}
</div>
</div>
)
}
// Same 10px uppercase as the ledger table headers, so the groupings read as part
// of the same family rather than as a second style.
function SectionLabel({ children }) {
return (
<div className="text-gray-400 uppercase tracking-wide pt-1" style={{ fontSize: '10px' }}>
{children}
</div>
)
}
function Segmented({ options, value, onChange }) {
return (
<div className="inline-flex rounded border border-gray-200 overflow-hidden w-auto self-start">
@ -71,22 +99,37 @@ function Segmented({ options, value, onChange }) {
)
}
function Submit({ onClick, children, disabled }) {
function Submit({ onClick, children, disabled, busy }) {
return (
<button onClick={onClick} disabled={disabled}
<button onClick={onClick} disabled={disabled || busy}
className="self-start px-4 py-1.5 rounded text-xs font-medium bg-blue-600 text-white hover:bg-blue-700
disabled:bg-gray-200 disabled:text-gray-600 disabled:cursor-not-allowed whitespace-nowrap">
disabled:bg-gray-200 disabled:text-gray-600 disabled:cursor-not-allowed whitespace-nowrap
inline-flex items-center gap-2">
{busy && (
<span className="inline-block w-3 h-3 rounded-full border-2 border-white/40 border-t-white animate-spin" />
)}
{children}
</button>
)
}
// 1. Selection
function SelectionList({ slices, currentTotals, onRemove, onClear }) {
function SelectionList({ slices, viewScope = [], currentTotals, onRemove, onClear }) {
const multi = slices.length > 1
const perSlice = currentTotals?.perSlice || []
const valueCol = currentTotals?.valueCol
// The pivot's own filter. Shown because it scopes every figure below and
// every row the operation writes, while appearing in none of the slices --
// perspective-click reports only the cell's own dimensions, so without this
// the panel prints a selection wider than the one it is acting on.
const scopeLine = viewScope
.map(([col, op, ...rest]) => {
const vals = (Array.isArray(rest[0]) ? rest[0] : rest).filter(v => v !== undefined)
return `${col} ${op}${vals.length ? ' ' + vals.join(', ') : ''}`
})
.join(' · ')
if (!slices.length) {
return (
<p className="text-gray-600 italic leading-relaxed">
@ -98,6 +141,12 @@ function SelectionList({ slices, currentTotals, onRemove, onClear }) {
return (
<div className="min-w-0">
{scopeLine && (
<div className="mb-1 flex items-baseline gap-1.5 text-[11px]">
<span className="text-gray-400 uppercase tracking-wide shrink-0">within</span>
<span className="font-mono text-gray-600 truncate" title={scopeLine}>{scopeLine}</span>
</div>
)}
<div className="overflow-auto max-h-36 -mx-1 px-1">
<table className="w-full">
<tbody>
@ -214,11 +263,59 @@ function derive(current, edit, dp = 2) {
return out
}
// Grouped while you type, because -2000000 and -20000000 are the same shape at
// a glance and the ledger deals in both.
//
// Formats for display only: what leaves here is always the raw string, so the
// arithmetic upstream never sees a comma. A partly-typed number has to survive
// intact -- "1." and "-" and "1.50" are all states on the way to a value, and
// reformatting them into something else as you type makes the field unusable.
function groupDigits(raw) {
const str = String(raw ?? '')
if (str === '' || str === '-') return str
const neg = str.startsWith('-')
const body = neg ? str.slice(1) : str
const dot = body.indexOf('.')
const whole = (dot === -1 ? body : body.slice(0, dot)).replace(/\D/g, '')
const frac = dot === -1 ? null : body.slice(dot + 1).replace(/\D/g, '')
if (whole === '' && frac === null) return neg ? '-' : ''
const grouped = whole.replace(/\B(?=(\d{3})+(?!\d))/g, ',')
return `${neg ? '-' : ''}${grouped}${dot === -1 ? '' : `.${frac}`}`
}
function LedgerInput({ value, active, onChange, onFocus, suffix }) {
const ref = useRef(null)
const caret = useRef(null)
const display = groupDigits(value)
// The commas shift every character after them, so a remembered offset lands
// in the wrong place. Count digits instead -- those are what the caret is
// actually sitting between -- and find that many digits into the new text.
useLayoutEffect(() => {
const el = ref.current
if (!el || caret.current == null) return
const wanted = caret.current
caret.current = null
let seen = 0, pos = display.length
for (let i = 0; i < display.length; i++) {
if (/[\d.-]/.test(display[i])) seen++
if (seen === wanted) { pos = i + 1; break }
}
if (wanted === 0) pos = 0
try { el.setSelectionRange(pos, pos) } catch {}
}, [display])
return (
<span className="inline-flex items-center gap-1">
<input
type="text" inputMode="decimal" value={value} onChange={e => onChange(e.target.value)}
ref={ref}
type="text" inputMode="decimal" value={display}
onChange={e => {
const el = e.target
const upto = el.value.slice(0, el.selectionStart ?? el.value.length)
caret.current = (upto.match(/[\d.-]/g) || []).length
onChange(el.value.replace(/,/g, ''))
}}
onFocus={onFocus} placeholder="—"
className={`border rounded px-2 py-0.5 text-xs w-24 text-right font-mono tabular-nums
${active ? 'border-blue-400 bg-blue-50/40 text-gray-800' : 'border-gray-200 bg-white text-gray-700'}`} />
@ -242,9 +339,16 @@ function ScaleLedger({ currentTotals, scaleInputs, setScaleInputs, scalePlug, se
const loose = []
const byTag = new Map()
for (const e of entries) {
if (e.key === 'baseline') { baseline.push({ ...e, label: 'Baseline', kind: 'baseline' }); continue }
const meta = logMeta[e.logid] || {}
const tag = (meta.tag || '').trim()
// Named like every other line: the label the pivot shows, then the older
// fallbacks. "Baseline" was hardcoded, so a segment called 03 - New Orders
// everywhere else read as "Baseline" here alone.
if (e.key === 'baseline') {
const name = (meta.label || meta.tag || meta.note || '').trim()
baseline.push({ ...e, label: name || 'Baseline', kind: 'baseline' })
continue
}
const tag = (meta.label || meta.tag || '').trim()
if (tag) {
const g = byTag.get(tag) ||
{ key: `tag:${tag}`, label: tag, kind: 'tag', value: 0, units: 0, count: 0, first: e.logid }
@ -257,7 +361,8 @@ function ScaleLedger({ currentTotals, scaleInputs, setScaleInputs, scalePlug, se
const op = meta.operation || e.iter || 'adjustment'
loose.push({
...e, kind: 'entry', count: 1,
label: (meta.note || '').trim() || `${op.charAt(0).toUpperCase()}${op.slice(1)} #${e.logid}`,
label: (meta.label || meta.note || '').trim()
|| `${op.charAt(0).toUpperCase()}${op.slice(1)} #${e.logid}`,
})
}
}
@ -271,7 +376,22 @@ function ScaleLedger({ currentTotals, scaleInputs, setScaleInputs, scalePlug, se
const hasExcl = excl.rows > 0 && (excl.value !== 0 || excl.units !== 0)
const onTotal = hasExcl && targetBasis !== 'adjustable' // 'selected total' is the default
const grand = { value: total.value + excl.value, units: total.units + excl.units }
const exclName = (currentTotals?.excludedIters || []).join(' / ') || 'excluded'
// Named by the segments themselves where we have them -- "02 - Prior Year"
// reads as a thing a forecaster recognises, where "reference" names only the
// iter band that happens to exclude it.
const exclName = (currentTotals?.excluded?.names || []).join(' · ')
|| (currentTotals?.excludedIters || []).join(' / ')
|| 'excluded'
// Everything in the selection is immovable. Worth saying outright: the panel
// otherwise prints a row of zeros and leaves the reason to be worked out.
const nothingToAdjust = hasExcl && !total.value && !total.units
// One line per immovable segment. Falls back to the combined figure for a
// selection whose rows carry no segment name.
const exclLines = (currentTotals?.excluded?.bySegment?.length
? currentTotals.excluded.bySegment
: (hasExcl ? [{ name: exclName, ...excl }] : []))
// the basis decides which line the editable rows are measured from
const basisOf = (key) => {
@ -367,6 +487,27 @@ function ScaleLedger({ currentTotals, scaleInputs, setScaleInputs, scalePlug, se
return { ...prev, [key]: { field, raw: cur ? raw : '' } }
})
// A running total across the walk, in the primary measure. The lines sum to
// Adjustable, and without this you are adding eight-digit numbers in your
// head to check that they do.
//
// Value only: a cumulative price is meaningless -- prices do not add -- and a
// second running column for units doubles the width to say something the
// value column already implies.
// Accumulates in display order across everything: the immovable segments
// first, then the walk. So it reads as the whole number being built up --
// billed, then booked, then the baseline, then each adjustment -- and closes
// on Selected total rather than on Adjustable, which is only the part of it
// that can still move.
const runningByKey = (() => {
const out = new Map()
let acc = 0
for (const seg of exclLines) { acc += seg.value || 0; out.set(`final:${seg.name}`, acc) }
for (const e of lines) { acc += e.value || 0; out.set(e.key, acc) }
return out
})()
const showRunning = !!valueCol && (lines.length + exclLines.length) > 1
const numCell = 'text-right font-mono tabular-nums whitespace-nowrap px-2'
const rule = <td className="p-0"><div className="border-t border-gray-300 my-1" /></td>
@ -382,9 +523,42 @@ function ScaleLedger({ currentTotals, scaleInputs, setScaleInputs, scalePlug, se
{m.hint && <span className="text-gray-500"> · {m.hint}</span>}
</th>
))}
{showRunning && (
<th className="text-right font-normal pb-1 px-2 whitespace-nowrap text-gray-500">
running
</th>
)}
</tr>
</thead>
<tbody>
{/* What cannot move comes first: it is the constraint the rest is
worked out against. Then the walk, which sums to Adjustable, and
the two together make the selected total. */}
{exclLines.map(seg => (
<tr key={seg.name} className="text-amber-700">
<td className="pr-3 whitespace-nowrap max-w-[16rem] truncate" title={seg.name}>
{seg.name}
<span className="ml-1.5 px-1 py-0.5 rounded bg-amber-50 text-amber-700 text-[10px] uppercase tracking-wide">
final
</span>
</td>
{measures.map(m => (
<td key={m.key} className={`${numCell} text-amber-700`}>
{m.key === 'price' ? fmtNum(priceOf(seg), m.dp) : fmtNum(seg[m.key], m.dp)}
</td>
))}
{showRunning && (
<td className={`${numCell} text-amber-700`}>
{fmtNum(runningByKey.get(`final:${seg.name}`), 0)}
</td>
)}
</tr>
))}
{exclLines.length > 0 && (
<tr>{rule}{measures.map(m => <td key={m.key} className="p-0 px-2"><div className="border-t border-gray-300 my-1" /></td>)}{showRunning && <td className="p-0 px-2"><div className="border-t border-gray-300 my-1" /></td>}</tr>
)}
{/* the walk from baseline to current, by initiative */}
{lines.map(e => (
<tr key={e.key} className="text-gray-600">
@ -398,10 +572,13 @@ function ScaleLedger({ currentTotals, scaleInputs, setScaleInputs, scalePlug, se
{m.key === 'price' ? fmtNum(priceOf(e), m.dp) : fmtNum(e[m.key], m.dp)}
</td>
))}
{showRunning && (
<td className={`${numCell} text-gray-500`}>{fmtNum(runningByKey.get(e.key), 0)}</td>
)}
</tr>
))}
<tr>{rule}{measures.map(m => <td key={m.key} className="p-0 px-2"><div className="border-t border-gray-300 my-1" /></td>)}</tr>
<tr>{rule}{measures.map(m => <td key={m.key} className="p-0 px-2"><div className="border-t border-gray-300 my-1" /></td>)}{showRunning && <td className="p-0 px-2"><div className="border-t border-gray-300 my-1" /></td>}</tr>
<tr className={onTotal ? 'text-gray-600' : 'font-semibold text-gray-700'}>
<td className="pr-3 whitespace-nowrap">
@ -410,22 +587,9 @@ function ScaleLedger({ currentTotals, scaleInputs, setScaleInputs, scalePlug, se
{measures.map(m => (
<td key={m.key} className={numCell}>{fmtNum(m.current, m.dp)}</td>
))}
{showRunning && <td className={numCell} />}
</tr>
{/* Rows the pivot shows but operations cannot write. Listed so the
panel's figures reconcile with what the grid displays. */}
{hasExcl && (
<tr className="text-gray-600">
<td className="pr-3 whitespace-nowrap">
{exclName} <span className="text-gray-500">· fixed</span>
</td>
{measures.map(m => (
<td key={m.key} className={numCell}>
{m.key === 'price' ? fmtNum(priceOf(excl), m.dp) : fmtNum(excl[m.key], m.dp)}
</td>
))}
</tr>
)}
{hasExcl && (
<tr className={onTotal ? 'font-semibold text-gray-700' : 'text-gray-600'}>
@ -437,10 +601,26 @@ function ScaleLedger({ currentTotals, scaleInputs, setScaleInputs, scalePlug, se
: fmtNum(grand[m.key], m.dp)}
</td>
))}
{showRunning && (
<td className={`${numCell} font-semibold text-gray-700`}>{fmtNum(grand.value, 0)}</td>
)}
</tr>
)}
<tr>{rule}{measures.map(m => <td key={m.key} className="p-0 px-2"><div className="border-t border-gray-300 my-1" /></td>)}</tr>
{/* Zeros in the Adjustable row are a true answer to the wrong question:
they say how much can move, not why none of it can. Spell it out
where the eye already is, rather than leaving the edit rows to fail
silently below. */}
{nothingToAdjust && (
<tr>
<td colSpan={measures.length + 1 + (showRunning ? 1 : 0)} className="pt-2 text-amber-700 leading-snug">
Nothing in this selection can be adjusted all of it is {exclName},
loaded as {(currentTotals?.excludedIters || []).join(' / ') || 'reference'}.
</td>
</tr>
)}
<tr>{rule}{measures.map(m => <td key={m.key} className="p-0 px-2"><div className="border-t border-gray-300 my-1" /></td>)}{showRunning && <td className="p-0 px-2"><div className="border-t border-gray-300 my-1" /></td>}</tr>
{/* the edit — three equivalent ways to say the same thing */}
{FIELDS.map(([field, label]) => (
@ -467,7 +647,7 @@ function ScaleLedger({ currentTotals, scaleInputs, setScaleInputs, scalePlug, se
{outcome && (
<>
<tr>{rule}{measures.map(m => <td key={m.key} className="p-0 px-2"><div className="border-t border-gray-300 my-1" /></td>)}</tr>
<tr>{rule}{measures.map(m => <td key={m.key} className="p-0 px-2"><div className="border-t border-gray-300 my-1" /></td>)}{showRunning && <td className="p-0 px-2"><div className="border-t border-gray-300 my-1" /></td>}</tr>
<tr className="font-semibold text-gray-700">
<td className="pr-3 whitespace-nowrap">Result</td>
{measures.map(m => (
@ -538,9 +718,62 @@ function ScaleLedger({ currentTotals, scaleInputs, setScaleInputs, scalePlug, se
)
}
// Completion for a key dimension. The list is fetched as you type rather than up
// front: part alone has 11,290 distinct values, and a native datalist given all of
// them is slow to open and no easier to read than a short filtered one.
function DimValueInput({ col, members, versionId, value, onChange, onBlur, className }) {
const [fallback, setFallback] = useState([])
const listId = `pf-vals-${col.cname}`
const hasList = !!members?.length
// Without a member list for this group -- never refreshed, or the column is not
// in one -- fall back to the version's own values. That is a 2s scan held in
// memory server-side, so it stays debounced rather than firing per keystroke.
useEffect(() => {
if (hasList || !versionId || !col.is_key) return
let cancelled = false
const t = setTimeout(async () => {
try {
const url = `/api/versions/${versionId}/values/${encodeURIComponent(col.cname)}`
+ `?limit=50${value ? `&q=${encodeURIComponent(value)}` : ''}`
const rows = await fetch(url).then(r => r.ok ? r.json() : [])
if (!cancelled) setFallback(Array.isArray(rows) ? rows : [])
} catch { if (!cancelled) setFallback([]) }
}, 200)
return () => { cancelled = true; clearTimeout(t) }
}, [hasList, versionId, col.cname, col.is_key, value])
// The member list is already in memory, so filtering it costs nothing and needs
// no debounce -- the options move with the keystroke.
const options = hasList
? (() => {
const q = (value || '').trim().toLowerCase()
const all = members.map(m => m.key_value)
return (q ? all.filter(v => v.toLowerCase().includes(q)) : all).slice(0, 50)
})()
: fallback
return (
<>
<input
value={value}
list={col.is_key ? listId : undefined}
onChange={onChange}
onBlur={onBlur}
placeholder="keep"
className={className} />
{col.is_key && (
<datalist id={listId}>
{options.map(o => <option key={o} value={o} />)}
</datalist>
)}
</>
)
}
// 2b. Recode / clone form
// Same pairing: the dimension's current value sits beside the box that replaces it.
function DimForm({ dimCols, setObj, setSet, slices, lookupDerivedCols, extra }) {
function DimForm({ dimCols, setObj, setSet, slices, lookupDerivedCols, dimMembers, versionId, extra }) {
const multi = slices.length > 1
const first = slices[0] || {}
return (
@ -563,13 +796,15 @@ function DimForm({ dimCols, setObj, setSet, slices, lookupDerivedCols, extra })
<td className="pr-3 py-0.5 text-gray-500 whitespace-nowrap" title={c.cname}>{c.label || c.cname}</td>
<td className="px-2 py-0.5 font-mono text-gray-600 max-w-[10rem] truncate" title={String(cur)}>{cur}</td>
<td className="pl-2 py-0.5">
<input
<DimValueInput
col={c}
members={c.dim_group ? dimMembers?.[c.dim_group]?.members : null}
versionId={versionId}
value={setObj[c.cname] || ''}
onChange={e => setSet(s => ({ ...s, [c.cname]: e.target.value }))}
onBlur={c.is_key && c.dim_group
? e => lookupDerivedCols(c.cname, e.target.value, setSet)
: undefined}
placeholder="keep"
className={TEXT} />
</td>
</tr>
@ -584,9 +819,16 @@ function DimForm({ dimCols, setObj, setSet, slices, lookupDerivedCols, extra })
// Recode and clone change dimensions rather than amounts, but you still want to
// see how much is on the move and for clone, what it becomes after scaling.
function MovingTotal({ currentTotals, verb, factor }) {
const t = currentTotals?.total
if (!t) return null
// includeExcluded: clone reads reference rows too, so its preview has to count
// them. Reporting the adjustable total alone said "Copying 0.00" for a selection
// made entirely of prior year or plan -- the exact case clone exists for.
function MovingTotal({ currentTotals, verb, factor, includeExcluded }) {
const adj = currentTotals?.total
if (!adj) return null
const ex = currentTotals?.excluded
const t = includeExcluded && ex
? { value: (adj.value || 0) + (ex.value || 0), units: (adj.units || 0) + (ex.units || 0) }
: adj
const { valueCol, unitsCol } = currentTotals
const scaled = factor != null && factor !== 1
return (
@ -648,6 +890,8 @@ function RequestPreview({ payload }) {
export default function OperationPanel({
dock,
slices, setSlices, distinctSlices,
viewScope = [],
opBusy = null,
applyMode, setApplyMode,
currentTotals,
activeOp, setActiveOp,
@ -660,8 +904,9 @@ export default function OperationPanel({
recodeNote, setRecodeNote,
cloneSet, setCloneSet,
cloneScale, setCloneScale,
cloneFrom, setCloneFrom, cloneOffset, setCloneOffset, cloneSources,
cloneNote, setCloneNote,
dimCols, lookupDerivedCols,
dimCols, lookupDerivedCols, dimMembers, versionId,
buildPayload, submitOp,
}) {
const hasSlice = slices.length > 0
@ -669,8 +914,10 @@ export default function OperationPanel({
const horizontal = dock === 'bottom'
const note = activeOp === 'scale' ? scaleNote : activeOp === 'recode' ? recodeNote : cloneNote
const shifting = !!cloneOffset && cloneOffset.trim() !== '' && cloneOffset.trim() !== '0 days'
const setNote = activeOp === 'scale' ? setScaleNote : activeOp === 'recode' ? setRecodeNote : setCloneNote
const OP_LABEL = { scale: 'Apply Scale', recode: 'Apply Recode', clone: 'Apply Clone' }
const OP_BUSY_LABEL = { scale: 'Applying…', recode: 'Recoding…', clone: 'Cloning…' }
return (
<div className={horizontal ? 'flex flex-row items-start p-3 gap-5 min-w-0' : 'flex flex-col p-3 gap-3 min-w-0'}>
@ -688,6 +935,7 @@ export default function OperationPanel({
)}
<SelectionList
slices={slices}
viewScope={viewScope}
currentTotals={currentTotals}
onRemove={(i) => setSlices(prev => prev.filter((_, x) => x !== i))}
onClear={() => setSlices([])}
@ -722,21 +970,76 @@ export default function OperationPanel({
)}
{activeOp === 'recode' && (
<DimForm dimCols={dimCols} setObj={recodeSet} setSet={setRecodeSet}
slices={slices} lookupDerivedCols={lookupDerivedCols}
extra={<MovingTotal currentTotals={currentTotals} verb="Moving" />} />
slices={slices} lookupDerivedCols={lookupDerivedCols} dimMembers={dimMembers} versionId={versionId}
extra={
<div className="pt-1 border-t border-gray-100 mt-1">
<MovingTotal currentTotals={currentTotals} verb="Moving" />
</div>
} />
)}
{activeOp === 'clone' && (
<DimForm dimCols={dimCols} setObj={cloneSet} setSet={setCloneSet}
slices={slices} lookupDerivedCols={lookupDerivedCols}
slices={slices} lookupDerivedCols={lookupDerivedCols} dimMembers={dimMembers} versionId={versionId}
extra={
<div className="flex flex-col gap-2">
<div className="flex items-center gap-2">
<span className="text-gray-500">scale cloned rows by</span>
<div className="flex flex-col gap-2 pt-1 border-t border-gray-100 mt-1">
<SectionLabel>source</SectionLabel>
{/* The selection is the SOURCE, not the destination: pick the
cells you want to copy -- last December, say -- and the
shift is what lands them in the target period. Naming a
segment narrows that selection to one entry, including the
reference ones operations are normally kept away from,
which is the point: a period with no baseline borrows its
shape from prior year or plan. */}
<Field label="rows from"
hint={cloneFrom
? 'Only rows from that segment, out of everything selected.'
: undefined}>
<select value={cloneFrom} onChange={e => setCloneFrom(e.target.value)}
className={`${TEXT} w-48`}>
<option value="">the whole selection</option>
{(cloneSources || []).map(s => (
<option key={s.id} value={s.id}>
{s.label || `${s.operation} #${s.id}`}
</option>
))}
</select>
</Field>
<SectionLabel>changes</SectionLabel>
<Field label="shift dates"
hint={shifting
? 'Every date column moves; season and month are re-derived from the calendar. So select the period you are copying from, not the one you are filling.'
: undefined}>
<input value={cloneOffset} list="pf-clone-offsets"
onChange={e => setCloneOffset(e.target.value)}
placeholder="0 days" className={`${TEXT} w-28`} />
<datalist id="pf-clone-offsets">
<option value="12 months" />
<option value="24 months" />
<option value="-90 days" />
<option value="-12 months" />
<option value="0 days" />
</datalist>
</Field>
<Field label="scale by">
<input type="number" step="any" value={cloneScale}
onChange={e => setCloneScale(e.target.value)} className={INPUT} />
onChange={e => setCloneScale(e.target.value)}
className={`${INPUT} w-28`} />
<span className="text-gray-400 text-[11px]">×</span>
</Field>
{/* Sits under the controls it reflects rather than floating
after them, and counts the reference rows clone can now
read -- it used to report the adjustable total, which is
zero when the selection is entirely prior year or plan. */}
<div className="pt-1">
<MovingTotal currentTotals={currentTotals} verb="Copying"
factor={parseFloat(cloneScale) || 1}
includeExcluded />
</div>
<MovingTotal currentTotals={currentTotals} verb="Copying"
factor={parseFloat(cloneScale) || 1} />
</div>
} />
)}
@ -747,11 +1050,12 @@ export default function OperationPanel({
{hasSlice && (
<Block horizontal={horizontal}>
<div className="flex flex-col gap-2.5 min-w-0">
<SectionLabel>label this change</SectionLabel>
{/* Tag first: it is the field that gives an adjustment meaning later,
in the ledger and in the bridge. Completes from initiatives already
used on this source; free text is still accepted. */}
<div className="flex items-center gap-2">
<span className="text-gray-600 whitespace-nowrap w-9">tag</span>
<Field label="tag">
<input
value={opTag} onChange={e => setOpTag(e.target.value)}
list="pf-tag-options" placeholder="initiative, e.g. reduce_spend"
@ -760,27 +1064,42 @@ export default function OperationPanel({
<button onClick={() => setOpTag('')} title="Clear tag"
className="text-gray-500 hover:text-red-500 leading-none px-1">×</button>
)}
</div>
</Field>
{/* Under the tag field and inside the same column, so they read as
values for it rather than as a row of unexplained buttons. */}
{knownTags.length > 0 && (
<div className="flex items-center gap-1 flex-wrap">
{knownTags.slice(0, 6).map(t => (
<button key={t.tag} onClick={() => setOpTag(t.tag)}
title={`${t.uses} previous use${t.uses === 1 ? '' : 's'}`}
className={`px-2 py-0.5 rounded-full border text-xs whitespace-nowrap ${
opTag.trim() === t.tag
? 'bg-blue-600 border-blue-600 text-white'
: 'bg-white border-gray-300 text-gray-700 hover:border-blue-400 hover:text-blue-700'}`}>
{t.tag}
</button>
))}
</div>
<Field label="">
<div className="flex items-center gap-1 flex-wrap">
{knownTags.slice(0, 6).map(t => (
<button key={t.tag} onClick={() => setOpTag(t.tag)}
title={`${t.uses} previous use${t.uses === 1 ? '' : 's'}`}
className={`px-2 py-0.5 rounded-full border text-xs whitespace-nowrap ${
opTag.trim() === t.tag
? 'bg-blue-600 border-blue-600 text-white'
: 'bg-white border-gray-300 text-gray-700 hover:border-blue-400 hover:text-blue-700'}`}>
{t.tag}
</button>
))}
</div>
</Field>
)}
<div className="flex items-center gap-2">
<span className="text-gray-600 whitespace-nowrap w-9">note</span>
<input value={note} onChange={e => setNote(e.target.value)} placeholder="optional" className={TEXT} />
<Field label="note">
<input value={note} onChange={e => setNote(e.target.value)}
placeholder="optional" className={`${TEXT} w-48`} />
</Field>
<div className="pt-1 flex items-center gap-3">
<Submit onClick={() => submitOp(activeOp)} busy={!!opBusy}>
{opBusy ? OP_BUSY_LABEL[opBusy] : OP_LABEL[activeOp]}
</Submit>
{opBusy && (
<span className="text-gray-500 leading-snug">
Writing rows the pivot updates when it finishes.
</span>
)}
</div>
<Submit onClick={() => submitOp(activeOp)}>{OP_LABEL[activeOp]}</Submit>
<RequestPreview payload={buildPayload(activeOp)} />
</div>
</Block>

View File

@ -32,11 +32,16 @@ function roundRect(ctx, x, y, w, h, r, fill, stroke) {
if (stroke) ctx.stroke()
}
export default function Timeline({ dateFrom, dateTo, offsetYr, offsetMo, type = 'baseline' }) {
export default function Timeline({ dateFrom, dateTo, offsetMonths = 0, offsetDays = 0, type = 'baseline' }) {
const canvasRef = useRef(null)
const offsetMoTotal = (offsetYr || 0) * 12 + (offsetMo || 0)
const twoBands = type === 'baseline' && offsetMoTotal > 0
// Months and days are kept apart because they are not interchangeable: a month
// shift lands on the same day of a different month, a day shift can cross a
// month boundary. The preview adds months first, then days, as Postgres does.
const offsetMoTotal = offsetMonths || 0
const offsetDayTotal = offsetDays || 0
const shifted = offsetMoTotal !== 0 || offsetDayTotal !== 0
const twoBands = type === 'baseline' && shifted
const canvasH = twoBands ? 90 : 52
useEffect(() => {
@ -61,8 +66,9 @@ export default function Timeline({ dateFrom, dateTo, offsetYr, offsetMo, type =
const srcEnd = parseDate(dateTo)
if (!srcStart || !srcEnd || isNaN(srcStart) || isNaN(srcEnd)) return
const projStart = addMonths(srcStart, offsetMoTotal)
const projEnd = addMonths(srcEnd, offsetMoTotal)
const addDays = (d, n) => { const x = new Date(d); x.setDate(x.getDate() + n); return x }
const projStart = addDays(addMonths(srcStart, offsetMoTotal), offsetDayTotal)
const projEnd = addDays(addMonths(srcEnd, offsetMoTotal), offsetDayTotal)
const winStart = addMonths(srcStart, -1)
const winEnd = addMonths(twoBands ? projEnd : srcEnd, 1)
@ -147,7 +153,14 @@ export default function Timeline({ dateFrom, dateTo, offsetYr, offsetMo, type =
ctx.lineTo(px1 - 4, arrowY + 4)
ctx.closePath()
ctx.fill()
const offsetLabel = '+' + (offsetYr ? offsetYr + 'yr ' : '') + (offsetMo ? offsetMo + 'mo' : '')
const yrs = Math.trunc(offsetMoTotal / 12)
const mos = offsetMoTotal % 12
const sign = (offsetMoTotal + offsetDayTotal) < 0 ? '' : '+'
const offsetLabel = sign + [
yrs ? `${yrs}yr` : '',
mos ? `${mos}mo` : '',
offsetDayTotal ? `${offsetDayTotal}d` : '',
].filter(Boolean).join(' ')
ctx.fillStyle = '#64748b'
ctx.font = '9px system-ui'
ctx.textAlign = 'center'
@ -156,7 +169,7 @@ export default function Timeline({ dateFrom, dateTo, offsetYr, offsetMo, type =
}
raf = requestAnimationFrame(draw)
return () => cancelAnimationFrame(raf)
}, [dateFrom, dateTo, offsetYr, offsetMo, type, twoBands, canvasH])
}, [dateFrom, dateTo, offsetMoTotal, offsetDayTotal, type, twoBands, canvasH])
return <canvas ref={canvasRef} height={canvasH} style={{ width: '100%', display: 'block' }} />
}

View File

@ -1,9 +1,3 @@
// MUST be first: it swaps window.IntersectionObserver / window.ResizeObserver
// for wrappers, and Perspective's viewer captures those constructors when its
// module is evaluated. Any import that reaches perspective-viewer before this
// one leaves the shim with nothing to intercept.
import './observerShim.js'
import { StrictMode } from 'react'
import { createRoot } from 'react-dom/client'
import { ThemeProvider } from './theme.jsx'

View File

@ -1,113 +0,0 @@
// Perspective's viewer captures the observer constructors at module-evaluation
// time:
//
// var it = window.ResizeObserver; var st = window.IntersectionObserver;
//
// so replacing them on `window` before that module is imported puts us in front
// of every callback it receives. This file must therefore be the FIRST import in
// main.jsx -- ES modules evaluate in declaration order, and an import that lands
// after the chain reaching perspective-viewer is too late to matter.
//
// Why bother: the WASM engine has no notion of focus or tab visibility. All it
// ever sees is an observer callback saying "you are visible again, at this
// size", and it re-renders in response -- which discards the view, and with it
// set_depth() and per-node expansion, neither of which lives in ViewConfig.
// Listening for window 'focus' is only a guess at when that happens. This is the
// actual event.
//
// Everything is passed through to the native observer untouched. We add one
// CustomEvent, and only for targets that are (or contain) a perspective-viewer,
// so nothing else on the page pays for this.
export const PF_OBSERVER_EVENT = 'pf-viewer-observed'
function dbg(msg, extra) {
let on = false
try { on = !!localStorage.getItem('pf_debug') } catch { /* private mode */ }
if (!on) return
const t = new Date().toISOString().slice(11, 23)
if (extra !== undefined) console.log(`[pf-obs ${t}] ${msg}`, extra)
else console.log(`[pf-obs ${t}] ${msg}`)
}
// Perspective observes elements inside its own shadow root, and closest() stops
// at a shadow boundary -- it will not climb from a shadow child out to the host.
// So walk the tree explicitly, hopping host to host, or the match never fires.
function touchesViewer(target) {
if (!(target instanceof Element)) return false
let node = target
for (let hops = 0; node && hops < 20; hops++) {
if (node.tagName === 'PERSPECTIVE-VIEWER') return true
if (node.closest?.('perspective-viewer')) return true
const root = node.getRootNode?.()
node = root && root.host ? root.host : node.parentElement
}
return !!target.querySelector?.('perspective-viewer')
}
function describe(target) {
if (!(target instanceof Element)) return String(target)
const root = target.getRootNode?.()
return `${target.tagName.toLowerCase()}${target.className ? '.' + String(target.className).split(' ')[0] : ''}`
+ (root && root.host ? ` (in shadow of ${root.host.tagName.toLowerCase()})` : '')
}
function wrap(Native, kind) {
if (typeof Native !== 'function') return Native
return class PfObserver extends Native {
constructor(callback, options) {
super((entries, observer) => {
// Perspective's own handler runs first and unchanged. If it throws,
// that is its business -- we still report, so a failure upstream is
// visible rather than silently swallowing our notification too.
try {
callback(entries, observer)
} finally {
const hit = entries.some(e => touchesViewer(e.target))
dbg(`${kind} fired on ${entries.length} entr${entries.length === 1 ? 'y' : 'ies'}`
+ ` -> ${hit ? 'MATCHED viewer' : 'no viewer match'}`,
entries.map(e => describe(e.target)))
if (hit) {
window.dispatchEvent(new CustomEvent(PF_OBSERVER_EVENT, {
detail: {
kind,
at: Date.now(),
// IntersectionObserver entries carry visibility; Resize
// ones carry geometry. Report whichever exists so the
// listener can tell a re-show from a relayout.
entries: entries.map(e => ({
isIntersecting: e.isIntersecting,
intersectionRatio: e.intersectionRatio,
width: e.contentRect?.width,
height: e.contentRect?.height,
})),
},
}))
}
}
}, options)
}
}
}
let installed = false
export function installObserverShim() {
if (installed) return
installed = true
try {
if (window.IntersectionObserver) {
window.IntersectionObserver = wrap(window.IntersectionObserver, 'intersection')
}
if (window.ResizeObserver) {
window.ResizeObserver = wrap(window.ResizeObserver, 'resize')
}
} catch {
// A browser that refuses the assignment just means we fall back to the
// focus/visibilitychange listeners, which still work -- they are only
// less precise about when the rebuild happened.
}
}
installObserverShim()

View File

@ -1,5 +1,6 @@
import { useState, useEffect } from 'react'
import Timeline from '../components/Timeline.jsx'
import useAuth from '../auth.jsx'
const OPERATORS = ['BETWEEN', '=', '!=', 'IN', 'NOT IN', 'IS NULL', 'IS NOT NULL']
@ -48,11 +49,28 @@ function getDateRange(groups) {
return null
}
function parseOffset(offsetStr) {
if (!offsetStr || offsetStr === '0 days') return { yr: 0, mo: 0 }
const yr = parseInt(offsetStr.match(/(\d+)\s+year/)?.[1] || 0)
const mo = parseInt(offsetStr.match(/(\d+)\s+month/)?.[1] || 0)
return { yr, mo }
// The offset is stored and sent as a Postgres interval, so it is typed as one --
// "4 months", "1 year", "-90 days". This only parses it far enough to draw the
// timeline preview; Postgres remains the authority on what is valid, and the
// server rejects anything it will not accept.
//
// It replaced a pair of year/month number spinners, which could not express days
// at all and were clamped at zero, so a segment could only ever be shifted
// forward in whole months.
export function parseInterval(str) {
let months = 0, days = 0
if (!str) return { months, days }
const re = /([+-]?\d+(?:\.\d+)?)\s*(years?|yrs?|y|months?|mons?|mo|weeks?|wks?|w|days?|d)\b/gi
for (const [, n, unit] of str.matchAll(re)) {
const v = parseFloat(n)
const u = unit.toLowerCase()
if (u.startsWith('y')) months += v * 12
else if (u.startsWith('mo') || u === 'mons' || u === 'mon') months += v
else if (u.startsWith('w')) days += v * 7
else if (u.startsWith('d')) days += v
else months += v // 'm' alone reads as months here
}
return { months, days }
}
function emptyCondition(cols) {
@ -69,6 +87,7 @@ function normalizeFilters(stored) {
}
export default function Baseline({ sources = [], sourceId, versions = [], versionId, setVersionId, refreshVersions }) {
const { user: me } = useAuth()
const [filterCols, setFilterCols] = useState([])
const [log, setLog] = useState([])
@ -80,13 +99,15 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
// segment form
const [segType, setSegType] = useState('baseline')
const [description, setDescription] = useState('')
const [filters, setFilters] = useState([]) // [[cond,...], [cond,...]]
const [useRaw, setUseRaw] = useState(false)
const [rawSql, setRawSql] = useState('')
const [offsetYr, setOffsetYr] = useState(0)
const [offsetMo, setOffsetMo] = useState(0)
const [offset, setOffset] = useState('0 days')
const [segNote, setSegNote] = useState('')
// Presentation, not definition: what the segment counts toward and how it is
// labelled in the pivot. Safe to set at any time, unlike its filters.
const [segBucket, setSegBucket] = useState('')
const [segLabel, setSegLabel] = useState('')
const [submitting, setSubmitting] = useState(false)
const [editingLogId, setEditingLogId] = useState(null)
const [showAddForm, setShowAddForm] = useState(false)
@ -113,10 +134,65 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
loadLog()
}, [versionId])
// A segment's banner: what it counts toward, independent of pf_iter. Held
// locally while typing so the field does not fight the fetched value, and
// written on blur.
const [buckets, setBuckets] = useState({})
// Buckets already in use on this version, for the datalist. There is no stored
// bucket order any more: the order is whatever the typed text sorts as, so a
// bucket is named "02 - Forecast" and that is the whole mechanism.
const [bucketsInUse, setBucketsInUse] = useState([])
// Label and bucket are presentation, not definition: they change what the pivot
// shows and what the segment counts toward, never which rows were loaded. So
// they stay editable after adjustments exist, unlike the filters and offset,
// where an edit would silently recalibrate scales sized against the old rows.
//
// Both are only read at load time, hence the reload in the confirmation: the
// label is part of the aggregated row the pivot holds, not something it can
// re-derive in place.
// Same rule the server enforces: your own entries, or an admin's. A segment's
// label and bucket name the pivot's columns for everyone in the version, so
// they are not the private annotation they look like.
const canEdit = (entry) => !!me && (me.is_admin || entry.pf_user === me.username)
async function saveLogField(entry, field, value) {
const next = value.trim()
if (next === (entry[field] || '')) return
try {
const res = await fetch(`/api/log/${entry.id}`, {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ [field]: next }),
})
if (!res.ok) { const d = await res.json(); flash(d.error, 'error'); return }
loadLog()
flash(next ? `Saved — reload the Forecast view to see it` : 'Cleared')
} catch (err) { flash(err.message, 'error') }
}
// Column widths read off the content rather than guessed. ch is the width of a
// '0', so for proportional text it runs slightly generous -- which is what is
// wanted for an input you are about to type a longer name into. The floors keep
// an empty table from collapsing its headers; the ceilings keep one long note
// from pushing the numbers off the side.
function widthCh(values, min, max) {
const longest = values.reduce((n, v) => Math.max(n, String(v || '').length), 0)
return `${Math.min(max, Math.max(min, longest + 2))}ch`
}
const labelW = widthCh(log.map(e => e.label || e.tag || e.note), 18, 34)
const bucketW = widthCh(log.map(e => e.bucket), 16, 24)
const noteW = widthCh(log.map(e => e.note), 22, 44)
function loadLog() {
fetch(`/api/versions/${versionId}/log`).then(r => r.json()).then(data => {
setLog(data.filter(e => e.operation === 'baseline' || e.operation === 'reference'))
setHasForecastOps(data.some(e => ['scale', 'recode', 'clone'].includes(e.operation)))
// Every bucket in use, adjustments included: typing one on a new segment
// should offer the ones already there rather than inviting a near-miss
// spelling, which would silently split the column in two.
setBucketsInUse([...new Set(data.map(e => (e.bucket || '').trim()).filter(Boolean))].sort())
})
}
@ -148,12 +224,14 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
const clause = useRaw ? rawSql.trim() : buildFilterClause(filters)
if (!clause) { flash(useRaw ? 'Enter a WHERE clause' : 'Add at least one filter', 'error'); return }
const isRef = segType === 'reference'
const offsetStr = [offsetYr > 0 ? `${offsetYr} year` : '', offsetMo > 0 ? `${offsetMo} month` : ''].filter(Boolean).join(' ') || '0 days'
const offsetStr = offset.trim() || '0 days'
const endpoint = isRef ? 'reference' : 'baseline'
const body = {
where_clause: clause,
note: description || segNote,
note: segNote,
date_offset: offsetStr,
label: segLabel.trim(),
bucket: segBucket.trim(),
...(useRaw ? { raw_where: clause } : { filters }),
}
setSubmitting(true)
@ -189,10 +267,9 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
const params = entry.params || {}
setSegType(entry.operation)
setSegNote(entry.note || '')
setDescription('')
const off = parseOffset(params.date_offset)
setOffsetYr(off.yr)
setOffsetMo(off.mo)
setSegLabel(entry.label || '')
setSegBucket(entry.bucket || '')
setOffset(params.date_offset || '0 days')
const groups = normalizeFilters(params.filters)
if (groups) {
setUseRaw(false)
@ -218,8 +295,9 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
function cancelEdit() {
setEditingLogId(null)
setShowAddForm(false)
setDescription('')
setSegNote('')
setSegLabel('')
setSegBucket('')
setOffsetYr(0)
setOffsetMo(0)
setUseRaw(false)
@ -276,11 +354,33 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
setTimeout(() => setMsg(null), 3000)
}
// The names a row falls back to when nobody has named it. Blank means "use the
// built-in", so these save on blur like the segment fields and an empty box is
// a meaningful value rather than a missing one.
async function saveVersionName(field, value) {
const next = value.trim()
if (next === (selectedVersion?.[field] || '')) return
try {
const res = await fetch(`/api/versions/${versionId}`, {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ [field]: next }),
})
if (!res.ok) { const d = await res.json(); flash(d.error, 'error'); return }
await refreshVersions(sourceId)
flash('Saved — reload the Forecast view to see it')
} catch (err) { flash(err.message, 'error') }
}
const selectedVersion = versions.find(v => String(v.id) === versionId)
return (
<div className="h-full overflow-y-auto bg-gray-50">
<div className="p-4 flex flex-col gap-4 max-w-4xl">
{/* No page-wide cap and no stretching: at max-w-4xl (896px) the eleven-column
segment table always had something smashed, and uncapped it ran to the
window. items-start makes each block as wide as its own content needs,
which for the table is the measured column widths below. */}
<div className="p-4 flex flex-col gap-4 items-start max-w-full">
{msg && (
<div className={`px-3 py-2 text-xs rounded font-medium ${msg.type === 'error' ? 'bg-red-50 text-red-700' : 'bg-green-50 text-green-700'}`}>
@ -304,6 +404,31 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
)}
</div>
{/* Fallback names. Not part of any segment -- they are what the pivot shows
for rows nobody has named, so they belong to the version rather than to
a log entry. Blank falls back to the built-in in sql_generator's
DISPLAY DEFAULTS block. */}
{versionId && (
<div className="bg-white border border-gray-200 rounded p-3 flex items-end gap-3 flex-wrap">
<span className="text-xs text-gray-500 uppercase tracking-wide w-full">Fallback names</span>
{[
['adjustment_segment', 'Adjustment segment', '99 - Adjustments'],
['adjustment_bucket', 'Adjustment bucket', '04 - Forecast'],
['unlabeled_load', 'Unlabeled load', 'Unlabeled'],
].map(([field, label, builtin]) => (
<div key={field} className="flex flex-col gap-1">
<label className="text-xs text-gray-500">{label}</label>
<input
key={`${field}-${versionId}-${selectedVersion?.[field] || ''}`}
defaultValue={selectedVersion?.[field] || ''}
onBlur={e => saveVersionName(field, e.target.value)}
placeholder={builtin}
className="border border-gray-200 rounded px-2 py-1 text-sm w-48" />
</div>
))}
</div>
)}
{showNewVersion && (
<div className="bg-white border border-gray-200 rounded p-3 flex flex-col gap-3">
<div className="flex items-end gap-3">
@ -329,17 +454,27 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
{versionId && <>
{/* Segments loaded */}
<div className="bg-white border border-gray-200 rounded">
<div className="bg-white border border-gray-200 rounded max-w-full">
<div className="px-3 py-2 border-b border-gray-100 text-xs font-medium text-gray-500 uppercase tracking-wide flex items-center justify-between">
<span>Segments loaded</span>
<button onClick={clearBaseline} className="text-red-400 hover:text-red-600 text-xs normal-case font-normal">Clear all baseline</button>
</div>
<table className="w-full text-xs">
<datalist id="pf-bucket-options">
{[...new Set([...bucketsInUse,
'01 - Prior Prior Year', '02 - Prior Year',
'03 - Plan', '04 - Forecast'])].map(b => (
<option key={b} value={b} />
))}
</datalist>
<table className="text-xs">
<thead className="bg-gray-50">
<tr className="text-left text-gray-400 border-b border-gray-100">
<th className="px-3 py-1.5 font-medium w-6"></th>
<th className="px-3 py-1.5 font-medium">#</th>
<th className="px-3 py-1.5 font-medium">note</th>
<th className="px-3 py-1.5 font-medium w-20">kind</th>
<th className="px-3 py-1.5 font-medium" style={{ width: labelW }}>label</th>
<th className="px-3 py-1.5 font-medium" style={{ width: noteW }}>note</th>
<th className="px-3 py-1.5 font-medium" style={{ width: bucketW }}>counts toward</th>
<th className="px-3 py-1.5 font-medium text-right">rows</th>
<th className="px-3 py-1.5 font-medium text-right">{log[0]?.value_col || 'value'}</th>
<th className="px-3 py-1.5 font-medium">by</th>
@ -349,11 +484,11 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
</thead>
<tbody>
{log.length === 0 && (
<tr><td colSpan={8} className="px-3 py-3 text-gray-300 italic">No segments loaded yet</td></tr>
<tr><td colSpan={11} className="px-3 py-3 text-gray-300 italic">No segments loaded yet</td></tr>
)}
{!showAddForm && !editingLogId && (
<tr className="border-t border-gray-100">
<td colSpan={8} className="p-0">
<td colSpan={11} className="p-0">
<button
onClick={() => setShowAddForm(true)}
className="w-full px-3 py-2 text-xs text-blue-600 hover:bg-blue-50 text-left font-medium"
@ -375,11 +510,49 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
>
<td className="px-3 py-2 text-gray-400 w-6"><span className="text-gray-300 text-xs">{isOpen ? '▾' : '▸'}</span></td>
<td className="px-3 py-2 text-gray-400">{log.length - i}</td>
{/* The operation badge gets its own column. Sharing one with
the note put "reference" hard against "YTD Sales" as soon
as the note column lost width to label and bucket. */}
<td className="px-3 py-2">
<span className={`inline-block mr-2 px-1.5 py-0.5 rounded text-xs font-medium ${entry.operation === 'reference' ? 'bg-purple-50 text-purple-600' : 'bg-blue-50 text-blue-600'}`}>
<span className={`inline-block px-1.5 py-0.5 rounded text-xs font-medium ${entry.operation === 'reference' ? 'bg-purple-50 text-purple-600' : 'bg-blue-50 text-blue-600'}`}>
{entry.operation}
</span>
{entry.note || <span className="text-gray-300"></span>}
</td>
<td className="px-3 py-2" onClick={e => e.stopPropagation()}>
<input
defaultValue={entry.label || ''}
key={`label-${entry.id}-${entry.label || ''}`}
onBlur={e => saveLogField(entry, 'label', e.target.value)}
readOnly={!canEdit(entry)}
title={canEdit(entry) ? '' : `${entry.pf_user || 'Another account'} made this segment`}
placeholder={entry.tag || entry.note || '—'}
className="w-full border border-transparent hover:border-gray-200
focus:border-blue-400 rounded px-1 py-0.5 text-xs
focus:outline-none bg-transparent" />
</td>
{/* One line, clipped against the measured width above. The
note is provenance and can run long, so left to itself it
wrapped and pushed every row to two or three lines.
Expanding the row shows it in full. The cap is on the div,
not the cell: a max-width on a cell in an auto-layout table
is only a hint, and the column can still collapse to
min-content or grow past it. */}
<td className="px-3 py-2">
{entry.note
? <div className="truncate" style={{ maxWidth: noteW }} title={entry.note}>{entry.note}</div>
: <span className="text-gray-300"></span>}
</td>
<td className="px-3 py-2" onClick={e => e.stopPropagation()}>
<input
value={buckets[entry.id] ?? entry.bucket ?? ''}
list="pf-bucket-options"
onChange={e => setBuckets(b => ({ ...b, [entry.id]: e.target.value }))}
onBlur={e => saveLogField(entry, 'bucket', e.target.value)}
readOnly={!canEdit(entry)}
title={canEdit(entry) ? '' : `${entry.pf_user || 'Another account'} made this segment`}
placeholder="—"
className="w-full border border-transparent hover:border-gray-200 focus:border-blue-400
rounded px-1 py-0.5 text-xs focus:outline-none bg-transparent" />
</td>
<td className="px-3 py-2 text-right text-gray-700 font-mono">
{entry.row_count != null ? entry.row_count.toLocaleString() : '—'}
@ -398,7 +571,7 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
</tr>
{isOpen && (
<tr key={`${entry.id}-detail`} className="bg-blue-50 border-t border-blue-100">
<td colSpan={6} className="px-2 py-2">
<td colSpan={9} className="px-2 py-2">
<div className="bg-white border border-gray-200 rounded">
<SegmentForm mode="view" {...view} filterCols={filterCols} />
</div>
@ -437,10 +610,10 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
filters={filters} setFilters={setFilters}
useRaw={useRaw} setUseRaw={setUseRaw}
rawSql={rawSql} setRawSql={setRawSql}
description={description} setDescription={setDescription}
segNote={segNote} setSegNote={setSegNote}
offsetYr={offsetYr} setOffsetYr={setOffsetYr}
offsetMo={offsetMo} setOffsetMo={setOffsetMo}
segBucket={segBucket} setSegBucket={setSegBucket}
segLabel={segLabel} setSegLabel={setSegLabel}
offset={offset} setOffset={setOffset}
filterCols={filterCols}
onSubmit={loadSegment}
submitting={submitting}
@ -459,17 +632,16 @@ export default function Baseline({ sources = [], sourceId, versions = [], versio
// derive view-mode props for a saved segment
function segmentValuesFor(entry, filterCols) {
const params = entry.params || {}
const off = parseOffset(params.date_offset)
const groups = normalizeFilters(params.filters)
return {
segType: entry.operation === 'reference' ? 'reference' : 'baseline',
filters: groups || (filterCols.length > 0 ? [emptyGroup(filterCols)] : []),
useRaw: !groups && !!params.where_clause,
rawSql: params.where_clause || '',
description: '',
segNote: entry.note || '',
offsetYr: off.yr,
offsetMo: off.mo,
segBucket: entry.bucket || '',
segLabel: entry.label || '',
offset: params.date_offset || '0 days',
}
}
@ -479,10 +651,10 @@ function SegmentForm({
filters, setFilters,
useRaw, setUseRaw,
rawSql, setRawSql,
description, setDescription,
segNote, setSegNote,
offsetYr, setOffsetYr,
offsetMo, setOffsetMo,
segBucket, setSegBucket,
segLabel, setSegLabel,
offset, setOffset,
filterCols,
onSubmit,
submitting,
@ -552,14 +724,6 @@ function SegmentForm({
</div>
</div>
{/* Description (edit only) */}
{mode === 'edit' && (
<div className="flex items-center gap-3">
<label className="text-xs text-gray-500 w-28 shrink-0">Description</label>
<input value={description} onChange={e => setDescription(e.target.value)} placeholder="e.g. FY25 actuals +1yr" className="border border-gray-200 rounded px-2 py-1.5 text-sm flex-1 max-w-sm" />
</div>
)}
{/* Filters */}
<div>
<div className="flex items-center justify-between mb-2">
@ -659,10 +823,19 @@ function SegmentForm({
<div className="flex items-center gap-3">
<label className="text-xs text-gray-500 w-28 shrink-0">Date offset</label>
<div className="flex items-center gap-2">
<input disabled={disabled} type="number" value={offsetYr} min={0} onChange={e => setOffsetYr(parseInt(e.target.value) || 0)} className={`${baseInp} text-sm w-16 text-center`} />
<span className="text-xs text-gray-500">yr</span>
<input disabled={disabled} type="number" value={offsetMo} min={0} max={11} onChange={e => setOffsetMo(parseInt(e.target.value) || 0)} className={`${baseInp} text-sm w-16 text-center`} />
<span className="text-xs text-gray-500">mo</span>
<input disabled={disabled} value={offset} list="pf-offset-options"
onChange={e => setOffset(e.target.value)}
placeholder="0 days"
className={`${baseInp} text-sm w-32`} />
<datalist id="pf-offset-options">
<option value="12 months" />
<option value="1 year" />
<option value="4 months" />
<option value="-90 days" />
<option value="-12 months" />
<option value="0 days" />
</datalist>
<span className="text-xs text-gray-400">any Postgres interval</span>
</div>
</div>
@ -673,16 +846,32 @@ function SegmentForm({
<Timeline
dateFrom={dateRange.from}
dateTo={dateRange.to}
offsetYr={offsetYr}
offsetMo={offsetMo}
offsetMonths={parseInterval(offset).months}
offsetDays={parseInterval(offset).days}
type={segType}
/>
</div>
</div>
)}
{/* Note + submit */}
<div className="flex items-end gap-3">
{/* Label, bucket, note + submit.
Label and bucket are presentation: the label is what the pivot shows for
this segment, the bucket is what it counts toward. Both are free text and
both sort by what is typed, so a leading "01 - " is how ordering is set
which is why they belong here, at the point the segment is defined, as
well as being editable in the list afterwards. */}
<div className="flex items-end gap-3 flex-wrap">
<div className="flex flex-col gap-1 max-w-xs">
<label className="text-xs text-gray-500">Label</label>
<input disabled={disabled} value={segLabel} onChange={e => setSegLabel(e.target.value)}
placeholder="defaults to the note" className={`${baseInp} text-sm py-1.5`} />
</div>
<div className="flex flex-col gap-1 max-w-xs">
<label className="text-xs text-gray-500">Counts toward</label>
<input disabled={disabled} value={segBucket} onChange={e => setSegBucket(e.target.value)}
list="pf-bucket-options" placeholder="e.g. 02 - Forecast"
className={`${baseInp} text-sm py-1.5`} />
</div>
<div className="flex flex-col gap-1 flex-1 max-w-xs">
<label className="text-xs text-gray-500">Note</label>
<input disabled={disabled} value={segNote} onChange={e => setSegNote(e.target.value)} placeholder="optional" className={`${baseInp} text-sm py-1.5`} />

File diff suppressed because it is too large Load Diff

View File

@ -25,6 +25,7 @@ export default function Setup({ refreshSources }) {
const [sqlStatus, setSqlStatus] = useState({}) // sourceId -> bool
const [saving, setSaving] = useState(false)
const [generating, setGenerating] = useState(false)
const [refreshingDims, setRefreshingDims] = useState(false)
const [msg, setMsg] = useState(null)
const [dimPeriodCols, setDimPeriodCols] = useState([])
const [openPeriodIdx, setOpenPeriodIdx] = useState(null)
@ -153,6 +154,35 @@ export default function Setup({ refreshSources }) {
}
}
// Rebuild every keyed dim_group's member list from the source. Deliberate rather
// than automatic: it reads the whole source, which for a view over a transaction
// table is millions of rows, and the answer only changes when the catalogue does.
async function refreshDimMembers() {
const groups = [...new Set(cols.filter(c => c.dim_group && c.is_key).map(c => c.dim_group))]
if (groups.length === 0) {
flash('No dim_group has an is_key column, so there is nothing to build a list from', 'error')
return
}
setRefreshingDims(true)
try {
const done = []
for (const g of groups) {
const res = await fetch(`/api/sources/${selectedSource.id}/dim/${encodeURIComponent(g)}/refresh`,
{ method: 'POST' })
const data = await res.json()
if (!res.ok) { flash(`${g}: ${data.error}`, 'error'); return }
done.push(`${g}: ${data.members.toLocaleString()} members`
+ (data.no_longer_in_source ? `, ${data.no_longer_in_source} no longer in source` : '')
+ ` (${(data.ms / 1000).toFixed(1)}s)`)
}
flash(done.join(' · '))
} catch (err) {
flash(err.message, 'error')
} finally {
setRefreshingDims(false)
}
}
async function deleteSource(id, e) {
e.stopPropagation()
if (!confirm('Deregister this source? Existing forecast tables are not affected.')) return
@ -295,6 +325,17 @@ export default function Setup({ refreshSources }) {
{saving ? 'Saving…' : 'Save'}
</button>
)}
{cols.some(c => c.dim_group && c.is_key) && (
<button
onClick={refreshDimMembers}
disabled={refreshingDims || colsDirty}
className="text-xs border border-gray-200 px-3 py-1 rounded hover:bg-gray-50 disabled:opacity-50"
title={colsDirty ? 'Save col meta first'
: 'Rebuild the member list for each keyed dim_group from the source. Reads the whole source, so it takes a while.'}
>
{refreshingDims ? 'Refreshing…' : 'Refresh master data'}
</button>
)}
<button
onClick={generateSQL}
disabled={generating || colsDirty}
@ -313,6 +354,7 @@ export default function Setup({ refreshSources }) {
<th className="px-3 py-1.5 font-medium">role</th>
<th className="px-3 py-1.5 font-medium text-center">key</th>
<th className="px-3 py-1.5 font-medium text-center" title="Include this column in the display grain — the load is pre-aggregated to the flagged columns">grain</th>
<th className="px-3 py-1.5 font-medium text-center" title="The column an account's territory is expressed in. Accounts see and change only rows whose value here is on their list. One per source.">territory</th>
<th className="px-3 py-1.5 font-medium">group</th>
<th className="px-3 py-1.5 font-medium">period col</th>
<th className="px-3 py-1.5 font-medium">label</th>
@ -349,6 +391,25 @@ export default function Setup({ refreshSources }) {
className="cursor-pointer disabled:opacity-20"
/>
</td>
{/* Radio, not a checkbox: exactly one column per source,
and the shape of the control should say so rather than
leaving it to a save-time error. */}
<td className="px-3 py-1.5 text-center">
<input
type="radio"
name="pf-territory-col"
checked={!!col.is_territory}
onChange={() => {}}
onClick={() => setEditedCols(prev => {
// clicking the chosen one again clears it, since a
// radio otherwise has no way back to "no territory"
const already = !!prev[i].is_territory
return prev.map((c, x) => ({ ...c, is_territory: !already && x === i }))
})}
disabled={col.role !== 'dimension'}
className="cursor-pointer disabled:opacity-20"
/>
</td>
<td className="px-3 py-1.5">
<input
type="text"

View File

@ -0,0 +1,45 @@
diff --git a/rust/perspective-client/src/rust/config/view_config.rs b/rust/perspective-client/src/rust/config/view_config.rs
index fa4f36e..360b816 100644
--- a/rust/perspective-client/src/rust/config/view_config.rs
+++ b/rust/perspective-client/src/rust/config/view_config.rs
@@ -526,6 +526,23 @@ impl ViewConfig {
}
}
+ /// `_apply` for a field which is itself `Option`, where `None` in the update
+ /// means "not mentioned" rather than "clear it". `Option<Option<T>>` would be
+ /// needed to express both, and the wire format cannot carry the difference:
+ /// these fields are `skip_serializing_if = "Option::is_none"`, so an absent
+ /// field and an explicit null arrive identically. To lift a depth, send the
+ /// number of levels on that axis rather than clearing it.
+ fn _apply_optional<T: PartialEq>(field: &mut Option<T>, update: Option<T>) -> bool {
+ match update {
+ None => false,
+ Some(_) if *field == update => false,
+ Some(_) => {
+ *field = update;
+ true
+ },
+ }
+ }
+
pub fn reset(&mut self, reset_expressions: bool) {
let mut config = Self::default();
if !reset_expressions {
@@ -568,6 +585,16 @@ impl ViewConfig {
changed = Self::_apply(&mut self.windows, update.windows) || changed;
changed = Self::_apply(&mut self.group_rollup_mode, update.group_rollup_mode) || changed;
changed = Self::_apply(&mut self.split_rollup_mode, update.split_rollup_mode) || changed;
+
+ // Without these two, a depth can be set when a view is created --
+ // `table.view({ group_by_depth })` -- but never through `restore()`, which
+ // merges a `ViewConfigUpdate` onto the live config. The field arrives,
+ // deserializes, and is then dropped here, so the viewer's config and the
+ // engine never see it and nothing happens. That makes expand/collapse
+ // unreachable for anything driven by `restore()`, which is how a viewer
+ // changes its own configuration.
+ changed = Self::_apply_optional(&mut self.group_by_depth, update.group_by_depth) || changed;
+ changed = Self::_apply_optional(&mut self.split_by_depth, update.split_by_depth) || changed;
if self.group_rollup_mode == GroupRollupMode::Total && !self.group_by.is_empty() {
tracing::info!("`total` incompatible with `group_by`");
changed = true;

View File

@ -1,7 +1,8 @@
Built from https://github.com/fleetside72/perspective
branch column-axis-expand-collapse
commit 2e3901d652650a33eaf19c2ddf049f7e525ea95b
based on v5.4.0
built 2026-09-14T02:40:11Z on r710.hptrow.me
branch apply-depth-on-config-update
commit 71d1a7508aa86d7e7224ad91e247afe4c201313d
based on unknown
built 2026-09-17T14:34:21Z on usmidsap02
dirty 0 uncommitted file(s) in the source tree at build time
Regenerate with ui/vendor/rebuild-perspective.sh

71
ui/vendor/README.md vendored
View File

@ -48,3 +48,74 @@ This is a fork, with the maintenance that implies. The exit is upstream taking
the change — the patch is small and additive, and the engine work is already
theirs. When a release ships it, delete this directory and put normal version
ranges back in `ui/package.json`.
## Applied: depth fields on a config update
`0001-apply-depth-fields-on-config-update.patch` is **in** the vendored
tarballs as of the 2026-09-17 build, from branch
`apply-depth-on-config-update`. It fixes an upstream bug in
`rust/perspective-client/src/rust/config/view_config.rs`:
`ViewConfig::apply_update` applies ten fields and **neither `group_by_depth`
nor `split_by_depth` is among them**. So a depth can be set when a view is
created (`table.view({ group_by_depth: 1 })` — which is what the fork's own
`depth_test.mjs` exercises) but never through `restore()`, which is how a
viewer changes its own configuration. The field arrives, deserializes, and is
discarded before the engine sees it.
`group_by_depth` and the omission are both upstream; the fork mirrored
`split_by_depth` alongside it faithfully, including the omission.
Note the semantics, from `server.cpp`:
```cpp
ctx1->set_depth(row_pivot_depth - 1); // one-sided
ctx2->set_depth(t_header::HEADER_ROW, row_pivot_depth - 1); // two-sided
```
The config field counts **levels to show**; `view.set_depth()` counts the
boundary below them. So `group_by_depth: n` equals `set_depth(n - 1)`, and
`Forecast.jsx` sends `d + 1`.
To apply:
```bash
cd $PSP_DIR # default ~/perspective
git apply /path/to/pf_app/ui/vendor/0001-apply-depth-fields-on-config-update.patch
./ui/vendor/rebuild-perspective.sh
cd ui && npm install
```
`cargo check` on this patch was clean — the 125 errors it reports without
`protoc` installed are unresolved generated protobuf modules, none of them in
`view_config.rs`.
`applyDepth()` in `Forecast.jsx` reads the config back after restoring it and
falls back to the imperative `view.set_depth()` when the value did not stick.
With the patched engine installed that fallback should no longer run; it is kept
for now so an unpatched engine still works, and can be removed once the
declarative path is confirmed.
### Host tools this tree needs
The repo pins its Rust toolchain (`nightly-2026-06-01`), Emscripten (4.0.9) and
Binaryen (132) exactly, and pins **nothing** for the host tools — no
`packageManager`, no `.nvmrc`. So `npm i -g pnpm` and `pip install cmake` both
give versions this tree was not written against. The script's preflight now
rejects them, but for the record:
- **pnpm 10.x.** 11+ rejects a repeated `--if-present`, which
`sh_perspective.mjs:189` emits once per package in scope.
- **cmake 3.x**, not 4. Both satisfy the `>= 3.29.5` floor; CMake 4 removed
support for `cmake_minimum_required < 3.5`, which the C++ dependencies still
declare, and fails partway through the Arrow build.
- **protoc >= 22** (33.2 known good).
- **`pnpm pack`, not `npm pack`.** These packages depend on each other as
`workspace:^`, and only pnpm rewrites that to a concrete version. An
`npm pack` tarball is rejected at install: `Unsupported URL Type
"workspace:"`.
Two commits on the build branch are local build concessions rather than
fixes, and should be dropped before the real patch goes anywhere upstream:
removing the `postinstall:playwright` step (it runs `playwright install
--with-deps`, which apt-installs system libraries and so needs root), and
filling in the `allowBuilds` block pnpm 12 demanded.

Binary file not shown.

Binary file not shown.

Binary file not shown.

View File

@ -46,6 +46,18 @@ die() { echo -e "\033[0;31m ✗\033[0m $*" >&2; exit 1; }
Set PSP_DIR to use a different path."
command -v pnpm >/dev/null || die "pnpm not found. Perspective builds with pnpm, not npm."
# The repo pins its Rust toolchain, emsdk and Binaryen exactly, but pins nothing
# for the host tools -- no packageManager, no .nvmrc -- so `npm i -g pnpm` gets
# whatever is newest and that is not what this tree was written against. pnpm 11+
# rejects a repeated `--if-present`, which sh_perspective.mjs emits once per
# package in scope, and the build dies with:
# error: the argument '--if-present' cannot be used multiple times
pnpm_major=$(pnpm --version | cut -d. -f1)
if (( pnpm_major > 10 )); then
die "pnpm $(pnpm --version) is too new; this tree needs pnpm 10.x.
npm i -g pnpm@10"
fi
command -v protoc >/dev/null || die "protoc not found.
Its VERSION selects which protobuf source tree the build clones, and a
version below 22 pulls a layout the build cannot consume. Needs >= 22
@ -55,7 +67,16 @@ cmake_ver=$(cmake --version 2>/dev/null | head -1 | grep -oE '[0-9]+\.[0-9]+(\.[
cmake_major=${cmake_ver%%.*}; cmake_minor=$(echo "$cmake_ver" | cut -d. -f2)
if (( cmake_major < 3 || (cmake_major == 3 && cmake_minor < 29) )); then
die "cmake $cmake_ver is too old; Perspective needs >= 3.29.5.
A user-level install works: pip3 install --user 'cmake>=3.29.5'"
A user-level install works: pip3 install --user 'cmake==3.31.*'"
fi
# And not too new: CMake 4 dropped compatibility with cmake_minimum_required
# below 3.5, which Perspective's C++ dependencies still declare. `pip install
# cmake` gives 4.x by default, which satisfies the floor above and then fails
# in the middle of the Arrow build.
if (( cmake_major >= 4 )); then
die "cmake $cmake_ver is too new; CMake 4 removed support for
cmake_minimum_required < 3.5, which the C++ dependencies still use.
pip3 install --user 'cmake==3.31.*'"
fi
info "Perspective checkout: $PSP"
@ -73,10 +94,14 @@ info "Building (this takes ~40 minutes cold, a few minutes warm)…"
ok "build complete"
# -- pack -------------------------------------------------------------------
# pnpm pack, not npm pack. These packages depend on each other as
# `workspace:^`, and only pnpm rewrites that to the concrete version on pack.
# npm leaves it, and npm install then refuses the tarball outright:
# npm error Unsupported URL Type "workspace:": workspace:^
info "Packing tarballs into $VENDOR"
rm -f "$VENDOR"/*.tgz
for p in "${PACKAGES[@]}"; do
( cd "$PSP/$p" && npm pack --pack-destination "$VENDOR" >/dev/null )
( cd "$PSP/$p" && pnpm pack --pack-destination "$VENDOR" >/dev/null )
ok "$(basename "$p")"
done