Values fitted and row labels did not. The measurement pairs each column's
header with its body cell, and for the row-header columns the header is a
blank corner cell — so the column was being sized from an empty string,
never from the labels beneath it.
Measured with a canvas instead of the DOM, for the same reason the headers
cannot size themselves: the cell is clipped, so reading its box back returns
the width it was allotted rather than the width of its text. Tree
indentation is added, since it occupies real width. The result is pinned as
a column_size_override on __ROW_PATH__ — the key
restore_column_size_overrides special-cases for this column, and an override
is what survives subsequent draws, which is precisely why overrides were
defeating resetAutoSize earlier.
Group headers are deliberately left alone. pro.css is explicit:
/* Header groups should overflow and not contribute to auto-sizing. */
thead tr:not(.rt-autosize) th { overflow: hidden; max-width: 0px; }
Only the leaf header row participates, because a group label spans many
columns and letting it set width would widen all of them. With the ordinal
prefix in front, "01 - Prior…" still reads when truncated. Fixing it
properly means distributing a group's label width across its span, which is
a different job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Bucket and Segment columns never appeared in the column list, and
viewer.save() reported expressions: {}. syncOrderExpressions ran immediately
after viewer.load() — and the saved layout was restored on the next line.
restore() replaces `expressions` wholesale rather than merging, so the
expressions were created and then wiped every single time, before anything
could see them.
Moved to after both restore branches. The effect on [logMeta, versions,
versionId] then keeps them in step, as intended.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Fit worked on the values and not on the headings. Auto-fit measures cells
with getBoundingClientRect() and sets each column's min-width from the
result, but the datagrid's CSS wraps and clips header text — so a clipped th
measures at the width it was *allotted*, not the width of its content, and a
column can never grow to fit its own heading. Row labels are th elements in
tbody and clip for the same reason.
white-space: nowrap on those cells, and nothing else: no width, no overflow.
The measurement then sees the full text and the existing sizing logic does
the rest.
Folded into the stylesheet already being injected for the grand-total
column, and renamed from pf-hide-split-total to pf-grid-css now that it does
more than the one thing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Fit button did nothing. Setting _reset_column_size and redrawing cannot
work, because draw() undoes the reset on the very next line:
const old_sizes = save_column_size_overrides.call(this);
... if (this._reset_column_size) { resetAutoSize() }
restore_column_size_overrides.call(this, old_sizes);
and old_sizes comes from regular_table.saveColumnSizes() -- the *live*
widths -- not from plugin_config. So each draw preserves whatever the
columns currently are, whatever the config says, and clearing
column_size_override released nothing.
Calls regular_table.resetAutoSize() and draw() directly instead: the same
draw the plugin performs, without the save/restore wrapped around it.
_cached_column_sizes is cleared too, since it short-circuits the next save
and would be restored over the measurement.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The datagrid already measures content -- draw() calls
regular_table.resetAutoSize() -- but only when _reset_column_size is set,
and it puts any pinned widths back immediately afterwards:
const old_sizes = save_column_size_overrides.call(this);
... if (this._reset_column_size) { resetAutoSize() }
restore_column_size_overrides.call(this, old_sizes);
So a width pinned by dragging a column edge, or carried in a saved layout,
outlives every measurement -- including the automatic ones the datagrid does
when split_by or columns change. Clearing the overrides is what releases
them; the flag then makes the next draw measure instead of reuse.
The flag is set after restore(), not before: building the model recomputes
it from what changed in the config, and would clear it again.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Unordered, '(adjustment)' sorted *first* rather than last: '(' is 0x28 and
digits begin at 0x30, so it precedes '01 - Prior Year Sales'. The
parenthesised labels exist to mark a value as not a real segment, which is
exactly what an ordinal now does, so they get high ordinals and plain names
-- 98 - Unlabeled, 99 - Adjustments.
99 rather than max(seq) + 1 so the position does not shift every time a
segment is added or renumbered.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Both were direct children of <table>, which is invalid: a browser hoists
stray non-table content out of the element, and in doing so it disturbed the
column widths of the segment listing below.
The datalist was already there before this change and had been getting away
with it; adding a visible div beside it is what made the consequence
obvious.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
"01 · Prior Year" would not parse -- Invalid `expressions` "Bucket": Invalid
string token: 01. ExprTK's string scanner validates each *byte*:
is_valid_string_char(c) = isprint((unsigned char) c) || is_whitespace(c)
"·" is U+00B7, two bytes 0xC2 0xB7 in UTF-8, and isprint(0xC2) is false in
the C locale. The literal fails at that byte and the parser reports from the
start of it, which is why the message names "01" rather than the character
it actually objected to.
" - " instead, which matches the "07 - Dec" already in the source data rather
than introducing a second convention.
The same rule applies to the label being compared, so a bucket or segment
named with anything outside printable ASCII is left unordered rather than
emitted into an expression that will not parse -- an unparseable expression
loses the whole column, not just that one case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Perspective orders column groups by the value string, and SortDir's col asc /
col desc only reverses that -- so Prior Year -> Plan -> Actual -> Forecast is
expressible as neither, being alphabetical in neither direction. The order
has to be part of the value, as a "01 · " prefix.
Done as Perspective expression columns, named Bucket and Segment, generated
from pf.version.bucket_order and pf.log.seq. The first attempt computed the
prefix in the served SQL (archived on feature/column-sequencing-sql), which
was the wrong layer: the prefix is a pivot-ordering concern, and putting it
in the query put it in every other reader too -- the change log and the
bridge's basis list would both have read "04 · Forecast". It also meant a
Generate SQL to introduce the placeholder, and a reload to see any change.
As expressions it stays in the pivot, travels with saved layouts because it
lives in ViewConfig, and reordering on the Baseline page takes effect
immediately -- the expressions are rebuilt and the pivot re-renders, no
reload.
Kept from the SQL attempt: pf.version.bucket_order and pf.log.seq, which are
needed either way, and the Baseline controls -- a seq column per segment and
a reorderable row of bucket chips. bucket_order is on the version because
the Baseline page is version-scoped; Setup is the only source-level context
and not where anyone would look for this.
Anything unordered falls through to the raw column, so it sorts after the
ordered entries (digits before letters) rather than silently landing first.
The expressions are merged into the live config rather than replacing it, so
a user's own expressions survive, and an expression that stops existing is
dropped from the axes first -- restore() rejects a config that pivots on an
expression it no longer defines.
Needs 01_schema.sql for the two columns. No Generate SQL.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The rule was injected into perspective-viewer-datagrid's shadow root, which
is not where the cells live, so it applied to nothing and the grand-total
column stayed visible.
Locates regular-table by walking through shadow roots and injects into
whatever root contains it, via getRootNode(). That makes no assumption about
how the plugin nests -- which is the assumption that was wrong, and the kind
that breaks on a Perspective upgrade anyway.
The grid is built asynchronously after load, so hideSplitTotal now reports
whether it found anything and is retried once on a short delay, as well as
from the config-update handler.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rollup mode emits a grand-total column group alongside the subtotals. The
subtotals are the point -- they are what per-branch collapse needs -- but
the grand total sums across the split, and when the split is Prior Year /
Plan / Forecast that sum is meaningless while looking exactly like a real
figure to anyone scanning the sheet.
The engine cannot separate the two: t_totals is { BEFORE, HIDDEN, AFTER },
and in a rollup the grand total *is* the root of the hierarchy that produces
the subtotals. Turning it off turns the subtotals off with it. So it is
hidden rather than suppressed -- the view still computes the column, it is
simply not painted, and collapsed to zero width so there is no gap where it
was.
Targeted by psp-split-total, excluding psp-split-subtotal, which is how the
datagrid already distinguishes them. That survives adding a measure or
rearranging the pivot; hiding by column position would not -- the group
spans one column per measure in `columns`.
Injected into the plugin's shadow root, since a document stylesheet cannot
reach inside it, and re-asserted on config updates because the plugin
element is replaced when the plugin changes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Column collapse did not survive flipping away from the browser and back,
because it was the one piece of state still held outside the config:
applySplitDepth restored a *truncated* split_by, and a rebuilt view took its
split_by from the config while the levels it had dropped lived only in
splitFull.
split_by_depth is the config-level equivalent, and it works now that
apply_update applies it -- server.cpp does
ctx2->set_depth(HEADER_COLUMN, column_pivot_depth - 1), so it is 1-based
exactly like group_by_depth.
Everything that existed to support truncation goes with it:
- splitFull no longer has to outlive split_by, because split_by keeps every
level. It is just the live axis, for rendering the buttons.
- split_full is no longer persisted beside the config. Still read on load,
for layouts saved by the old scheme.
- collapsingRef and the prefix test are gone. They existed to tell our own
collapse from the user rearranging the pivot, which only mattered because
a collapse looked like a shorter split_by.
- The selection survives a collapse. It was cleared because slices name the
split_by dimensions they were cut from and the axis was changing shape;
with a depth those dimensions are all still there.
CLAUDE.md's §Column hierarchy describes the old mechanism throughout, and is
now wrong; updating it separately.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rebuilt from fleetside72/perspective at apply-depth-on-config-update, which
carries the patch ui/vendor has been describing: ViewConfig::apply_update
applied ten fields and neither group_by_depth nor split_by_depth, so a depth
set through restore() was dropped before the engine saw it. The new
perspective-js wasm is 81 bytes larger than the old, which is about right
for two calls and a generic.
Four things went wrong getting here, all of them host tooling rather than
the patch, and the script's preflight now catches three:
- pnpm 12 rejects a repeated `--if-present`, which sh_perspective.mjs emits
once per package in scope. The tree needs pnpm 10; it pins no
packageManager, so `npm i -g pnpm` gets something too new.
- `pip install cmake` gives 4.x, which clears the existing >= 3.29.5 floor
and then fails in the Arrow build, because CMake 4 dropped support for
cmake_minimum_required < 3.5.
- The pack step used `npm pack`, which leaves these packages' mutual
`workspace:^` dependencies as-is; npm install then refuses the tarball
with `Unsupported URL Type "workspace:"`. Now `pnpm pack`, which rewrites
them -- and which is evidently how the previous tarballs were made, since
theirs read `^5.4.0`.
- postinstall:playwright runs `playwright install --with-deps`, which
apt-installs system libraries and needs root. Removed on the build branch.
npm normalised the vendor paths in package.json from ./vendor to vendor;
same meaning, left alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ViewConfig::apply_update in perspective-client applies ten fields and
neither group_by_depth nor split_by_depth is among them. A depth therefore
works when a view is created -- table.view({ group_by_depth: 1 }), which is
what the fork's own depth_test.mjs exercises -- and is silently discarded by
restore(), which is how a viewer changes its own configuration. That is why
the EXPAND buttons did nothing however the value was sent.
group_by_depth and the omission are both upstream; the fork mirrored
split_by_depth alongside it faithfully, including the omission. So the column
axis expand/collapse the fork added has the same hole, and pf_app only avoids
it by collapsing columns through a truncated split_by instead.
The patch adds an Option-aware sibling to _apply and applies both fields. It
cannot be built here -- protoc is absent, so the generated protobuf modules
are missing and the crate does not compile for unrelated reasons -- but
cargo check reports nothing against view_config.rs. It sits in ui/vendor with
the rebuild instructions, beside the tarballs it is not yet in.
Meanwhile applyDepth reads the config back after restoring and falls back to
view.set_depth() when the value did not stick, so the buttons work on the
current build. The fallback loses depth on a view rebuild, as it always did;
what is not back is the shim and listeners that used to chase that. The check
clears itself once the engine honours the field.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The EXPAND buttons did nothing because their 0..3 were written against
view.set_depth(), which is 0-based, while the config field is 1-based.
server.cpp does
ctx1->set_depth(row_pivot_depth - 1) // one-sided
ctx2->set_depth(HEADER_ROW, row_pivot_depth - 1)
for both contexts, so group_by_depth counts the levels to show. Passing 0
asked the engine for set_depth(-1), and every other button was off by one
level. The fork's own depth_test.mjs uses group_by_depth: 1 as its working
control, which is the same arithmetic seen from the outside.
So the toolbar sends d + 1, and subtracts one again when reading a saved
layout back for the button state.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A partial restore({ group_by_depth: d }) had no effect -- the buttons did
nothing, and the depth appearing to survive every rebuild was simply a tree
that had never been collapsed. So the full config goes back with the one
field changed.
table is dropped from it: the table is loaded by reference, and a stale name
in a restored config fails the lookup -- the same reason every other restore
in this file strips it.
If this still does nothing then group_by_depth is not applied when the view
is constructed, and the fix belongs in the fork beside split_by_depth.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ViewConfig carries group_by_depth -- alongside the split_by_depth this fork
added -- so row depth can be set declaratively:
await viewer.restore({ group_by_depth: d })
The config is what the viewer rebuilds its view from, so the depth survives
every rebuild by construction, and viewer.save() carries it into the
persisted and named layouts for free.
It had been imperative: getView() then view.set_depth(), which puts the
depth on an object the viewer discards whenever it re-renders. Everything
that grew around that existed only to guess when a rebuild had happened and
put the depth back -- a shim patching window.IntersectionObserver and
window.ResizeObserver, focus/visibilitychange/pageshow listeners, a retry
loop for getView() throwing "No table set" while getTable() resolved, a flag
tracking whether the viewer had "gone away", and tracing to debug all of it.
That guesswork caused three separate visible faults in a single session: the
tree fully expanding on refocus, snapping on any reflow, and snapping when
Perspective's own settings sidebar was opened. Each fix was a finer
heuristic about which browser event meant what, which is the shape of
fighting a framework rather than using it.
312 lines out, 35 in. applySplitDepth no longer re-applies the row depth
after truncating split_by, because the rebuilt view brings it along.
expand_depth stays readable on load: it is the legacy key, from when the
depth had to be stored beside the config rather than in it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two causes, both mine, both triggered by adjusting the layout.
A zero size was being treated as the viewer going away, and opening
Perspective's settings sidebar collapses the datagrid for a frame. The
return then re-applied the stored row depth over whatever had been expanded
by hand. Only lost intersection counts as a departure now; a zero size is a
transient of layout, not an absence.
And perspective-config-update fires for anything in the config, the settings
flag included -- not only for split_by. While collapsed the live split_by is
a truncation of the full hierarchy, so adopting it recorded the truncation
as the hierarchy and put the deeper levels out of reach. A genuine
rearrangement is not a prefix of what we already hold, which is the
difference the handler now tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Driving the re-apply from the observer callbacks caught the refocus case but
fired on far more than that: ResizeObserver reports every reflow, so
dragging the panel, a scrollbar appearing, or anything that changed the
layout re-applied the stored depth and discarded whatever had just been
expanded by hand. From the outside that is the pivot snapping to a different
layout while clicking around.
Going away is what discards the view; a reflow is not. So the callbacks now
record the departure -- lost intersection, collapsed to zero size, or a
hidden document -- and only a return after one of those restores anything.
The flag clears once the depth is back, so one departure causes one restore.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The clone form was four controls whose labels ran to different lengths --
"copy rows from", "scale cloned rows by", "tag", "note" -- each row starting
wherever its label happened to end, with a preview sentence floating after
them and a row of unexplained chips between the tag and the note. Nothing
separated where the rows come from, what happens to them, and what the
change is called.
Every row now shares a label column, so the controls line up, and the three
groups carry headers in the same 10px uppercase the ledger headers already
use. The tag suggestions sit inside the tag's own column, where they read as
values for it. Hints appear under the control they qualify and only when
relevant.
Also: the preview said "Copying 0.00 sales_usd" while five AOP rows worth
68,904 were selected. It reported the adjustable total, which is zero when
the selection is entirely prior year or plan -- the case clone exists for,
now that it can read reference rows. It counts them for clone, and still
does not for recode, which cannot touch them.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The offset field was rendered only when a segment had been named, from when
that was the only way clone could reach reference rows. Once clone could read
them directly the field became unreachable in the ordinary case -- so
"shift dates by" was invisible and the only typeable-looking control was
"copy rows from", which is a dropdown.
It is shown whenever clone is open now, and the payload carries the offset
independently of from_logid: shifting a selection through time is the common
case, and narrowing to one segment is the occasional one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Date offset on a baseline or reference segment was a pair of
type="number" inputs, years and months, both clamped at min 0. So no
characters could be typed, days could not be expressed at all, and a segment
could only ever be shifted forward in whole months -- while the stored value
is a Postgres interval that happily accepts "4 months", "1 year" or
"-90 days". parseOffset only read year and month, so anything else would not
have survived a round trip through the edit form either.
One text field now, with suggestions, matching what clone already offers.
parseInterval parses it far enough to draw the timeline preview; Postgres
stays the authority, and assertInterval -- now shared by the baseline,
reference and clone routes -- rejects what it will not accept, with a
message naming the field rather than a parse error from inside a CTE.
Timeline takes months and days separately rather than years and months,
because the two are not interchangeable: a month shift lands on the same day
of another month, a day shift can cross a month boundary. It adds months
then days, as Postgres does, and its label handles negatives.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A selection of five AOP rows cloned nothing, with no explanation. The cause
was exclude_iters, which clone applied along with scale and recode.
It should not. That exclusion exists to stop operations *modifying*
reference rows: scale distributes an increment across its pool, so including
reference would attribute forecast movement to prior-year rows, and recode
writes negative rows that zero the original out. Clone does neither -- it
reads rows and inserts new pf_iter = 'clone' rows, leaving the source
untouched. Copying a plan or a prior year out of reference and into
adjustments is the operation doing exactly what it is for.
So from_logid stops being the only way to reach those rows and becomes what
it should be: a narrowing, for a selection spanning AOP and Prior Year where
only one is wanted.
The "would just duplicate the rows" guard no longer fires when the selection
is entirely non-adjustable, since moving rows from reference into adjustments
changes what they are even at factor 1 with no shift.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Shifting backwards already worked -- the offset is added as a Postgres
interval, and '-90 days' is a perfectly good one -- but the suggestions only
listed forward shifts, so nothing said so. Pulling a plan back a quarter is
a normal thing to want.
A day-level shift is also the case that most needs the dim_period
derivation: 15 Mar 2027 less 90 days is 15 Dec 2026, which crosses from
2027/10 - Mar into 2027/07 - Dec. Copying the period columns across would
have labelled it March.
The offset is interpolated into the statement, so a typo surfaced as a
Postgres parse error from the middle of a CTE. It is parsed on its own
first, where the failure is cheap and can name the field.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The panel required at least one override for both recode and clone. For
recode that is right -- rewriting rows as themselves does nothing. For clone
it is not: copying prior year forward changes the dates and, through
dim_period, the season and month dimensions, which is the entire point of
cloning from a reference segment. The server-side requirement was dropped
when from_logid went in; this one was missed, so the operation was blocked
in the UI with "Enter at least one override value".
Clone now only has to be doing something -- an override, a shift, or a
factor other than 1 -- and says so when it is doing none of the three.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The bridge filtered on exclude_iters, so it dropped every reference row --
including Open Orders, which is loaded as reference precisely so nothing
adjusts it and is still part of the forecast. Membership is pf_bucket now,
which is the axis that answers this question; pf_iter answers a different
one.
What you compare against depends on what you are building: an AOP is built
off a prior period, a forecast off an update to the AOP. So the basis is
chosen rather than assumed, and the walk stays exact either way because
Forecast - Basis = (Forecast loads - Basis) + adjustments
The opening step is that first term, every tagged adjustment explains the
rest, and the bars sum to the endpoint by construction. No residual to
explain away -- which is what a bridge from prior year to forecast would
otherwise be, since the adjustments describe movement from the forecast's
own loads and not from last year.
With no basis it is the composition instead: the forecast's loads as the
opening anchor, then the adjustments. The picker only appears once something
carries a bucket, so a version that has not been labelled behaves as before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The slice filters the rows being copied, so cloning prior year forward means
selecting last December and shifting it, not selecting the December you want
to fill. Selecting the destination matches nothing -- prior year's rows are
labelled with prior year's periods, so a 2027 slice and a 2026 segment share
no rows -- and it fails silently, as zero rows cloned.
Nothing in the form said which way round it was, and "copy from <segment>"
reads like the selection is the target. It now says the selection is what
gets copied, and the segment narrows it rather than replacing it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three changes, one feature: a period with no baseline borrows a shape from
prior year or plan.
The generator handled one date group. sql_generator.js used find(), so a
source with order, requested and ship date groups derived the order one and
copied the other two raw -- shifted dates against unshifted period labels.
Every group now gets its own join, and they are LEFT rather than inner: an
order that has not shipped has no ship date, and an inner join would have
dropped the row from the load entirely rather than leaving its period
columns empty. dateGroupsOf() is now shared, like grainOf, so the routes and
the generator agree on membership and on join aliases.
Clone carries every date column rather than only the primary one, since it
is the operation that moves rows through time, and re-derives the period
dimensions from pf.dim_period against the shifted date instead of copying
them from the row being cloned.
from_logid names the segment to copy from, replacing the exclude clause for
that one entry rather than widening it. Rows written stay pf_iter = 'clone',
so scale applies to them afterwards as a second step.
set is no longer required: copying a segment forward unchanged is a
reasonable thing to ask for.
Needs dim_period_col set in Setup (fisc_year on oseas/rseas/sseas_e,
fisc_month_abbr on omon/rmon/smon_e) and Generate SQL.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was no way to express "these segments together are the forecast".
pf_iter cannot say it: it answers whether operations may write to a row, and
Open Orders is loaded as reference precisely so nothing adjusts it while
still being part of the forecast number. The two questions are independent,
so one cannot be derived from the other.
pf.log.bucket is the second axis. Free text with suggestions -- Forecast,
Prior Year, Prior Prior Year, Plan -- rather than an enum, so another banner
needs no migration. Blank by default, falling back in the pivot to the
segment's own name, so nothing changes until something is labelled.
/data and /agg emit it as pf_bucket beside pf_segment, through the pf.log
join that is already there. Set it per segment in the Baseline list, which is
where loads live now that the change log only shows adjustments.
Needs 01_schema.sql for the column and Generate SQL for /agg to select it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The effect decided which dim_groups to load from colMetaRef.current, but
that ref is filled inside initViewer, which is async. The effect ran first,
saw an empty array, found no groups, and fetched nothing -- and a ref
changing does not re-run an effect, so it never recovered. dimMembers stayed
{} for the whole session.
Everything therefore fell back to the source lookup, which is precisely the
query pf.dim_member exists to replace: XCP06500G18B112 has two attribute
combinations across history, so DISTINCT ... LIMIT 2 returned two rows and
the route answered null. The member row had the answer all along, picked as
the most recent by order date.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Autofill filled only empty boxes, so it appeared to work exactly once. Type a
part, get its nine attributes; type a different part, and every box is
already full, so nothing updates and the form still describes the previous
part. That is worse than blank -- the recode would have been submitted with
one part's code and another's attributes.
These columns describe the key that was just entered, so they are replaced
outright now.
A lookup that finds nothing also said nothing, which is indistinguishable
from a lookup that did not run. It warns, and names the value it could not
find, so an unknown part reads differently from a broken autofill.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Autofill stopped filling anything as soon as the member endpoint answered,
because a hit was treated as the only possible answer: if the key was not in
the list, it returned rather than trying the source. An empty list counted as
an answer too, which is the state while a refresh is still running -- so
wiring up master data broke the behaviour it was meant to improve, for
everyone who had not finished building the list yet.
A miss is not an answer. Empty list, or a key the list does not know, both
fall through to the source lookup as before. It costs one request, on a path
that only runs when someone types a value.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every question about a part -- what values exist, what attributes go with
one -- was answered by querying the source, and the source is the wrong
place to ask. It is a view over a transaction table, so the query is slow
(76s for one ILIKE against 6.9M rows), it describes only what was
transacted, and it cannot express intent: there is no way to say a part is
discontinued, or to name one that has not sold yet.
pf.dim_member holds the app's own list: one row per key value per group,
siblings in jsonb, keyed on (source_id, dim_group, key_value). Refresh is a
merge rather than a replace, so curation survives it -- members absent from
the source are marked source_seen = false, not deleted. Triggered from
Setup, next to Generate SQL, because it reads the whole source and the
answer only changes when the catalogue does.
A key can carry several attribute sets across history -- 11,290 parts
against 13,662 combinations on osm_skinny -- so the refresh takes the most
recent by the source's date column. That also fixes the sibling autofill,
which used to run a DISTINCT ... LIMIT 2 against the source and silently
fill nothing whenever a part came back ambiguous. A member row is one
definition by construction.
The client fetches each group's list once per source and does both
completion and autofill against it in memory, so neither costs a request.
Columns outside a group, or a group never refreshed, still fall back to the
version's values endpoint.
Run 01_schema.sql to create the table.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pointing completion at the source made every keystroke a 76-second query:
gs.osm_skinny is a plain view over rlarp.osm_stack, so ILIKE '%1601%'
scanned 6.9M rows and read 1.3M buffers off disk, 64s of it I/O. Nothing had
called that endpoint before, so the cost only appeared once a debounced
input was wired to it.
The source is the wrong list anyway. It reaches back over all of history and
would offer parts discontinued years ago; the version holds what was
actually loaded, which is what the forecast is being written against.
So GET /versions/:id/values/:col reads the version's own forecast table --
2.0s for 11,290 parts on fc_osm_skinny_29 -- and holds the result in memory,
keyed on that version's latest pf.log id. Any load, adjustment or undo moves
the id and the next request rebuilds, so nothing has to remember to
invalidate. Filtering happens over the cached array, so typing costs one
small max(id) query.
The column name is interpolated into the DISTINCT, so it is checked against
col_meta first.
The source-side endpoint keeps its new q/limit but no longer has a caller;
its ILIKE path against a view is the expensive one and should stay unused.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Recoding to a part meant typing a code from memory into a plain text box,
with the only feedback being that the sibling autofill either fired or
silently did nothing.
The values endpoint already existed but returned every distinct value with
no filter and no limit, which for part on osm_skinny is 11,290 rows -- too
slow to open and no easier to read than a short list. It now takes ?q= and
?limit=, so the field fetches matches for what has been typed so far,
debounced 200ms, and offers them through a native datalist.
Only key columns get it, which is the same condition the endpoint already
enforced, and the same one that decides whether the dim_group sibling
lookup runs on blur. So on osm_skinny it is part and customer: type 1601
and the eight parts containing it are offered; pick one and the nine
part-group columns fill themselves.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Adding the Value column pushed the table past max-w-4xl: roughly 624px of
declared column widths inside an 896px dialog, leaving Slice and Note to
fight over the rest. With auto table layout the cells won and the table grew
wider than its container, so the dialog got a horizontal scrollbar along the
bottom.
table-fixed makes the declared widths hold and the free-text columns
truncate instead of widening the table -- which is what the truncate classes
on them already assumed. Dialog widened to max-w-6xl, gutters down from
px-4 to px-3, and Note given an explicit share rather than whatever was
left.
max-w-xs on the slice and note cells was doing nothing useful under fixed
layout and is now overflow-hidden, which is what makes truncate work in a
table cell.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three things about the change log, from using it.
The slice column grew to a block per entry once it listed lines, which is
too much for a table you scan. It is one truncated line now: a single slice
reads as its fields, and a dragged region as "5 slices · omon = 07 - Dec …
11 - Apr" -- enough to recognise the entry.
There was no indication of what an entry did to the numbers, which is the
first thing you want from a change log. The route already returned
value_total and it simply was not rendered. Added, signed and coloured, so
a list of adjustments reads as a list of impacts.
And clicking a row now opens slice and params as formatted JSON -- the exact
payload that went over the route, including the increments and the plug,
which no amount of summarising in the cell was going to convey.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A multi-slice operation logs an array of slices, and Object.entries over an
array yields its indices -- so the column read "0 = [object Object], 1 =
[object Object], ...", which says nothing about what was adjusted.
Each slice now gets a line. And since a dragged region is one dimension
walked across a fixed set of others, the shared part is pulled out: log 118
is five slices differing only in omon, which now reads as one line of the
four fixed fields and one listing Dec through Apr, rather than the same four
fields five times over.
Only collapsed when exactly one key varies. Two independent axes would lose
their pairing when flattened, so those stay listed slice by slice.
Dates arrive as epoch millis and are shown as dates; the raw payload is on
the cell's title for anything the rendering has flattened.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Filtering the loads out in the dialog left the server still joining the
whole forecast table for them and discarding the answer. ?kind=adjustments
moves the filter into the WHERE, so they never enter the join: on version 29
the join goes from 2,556,821 rows to 25, and the query from 1666ms to 719ms.
Still 719ms, because nothing indexes pf_logid and it stays a sequential scan
of 2.5M rows. New forecast tables now get an index on it. That column is how
every entry-level operation finds its rows -- undo deletes by it, this
aggregate groups by it, and it is part of the grain key -- so the scan was
being paid on all of them.
Existing tables predate the index and still scan; fc_osm_skinny_29 would
want it added by hand.
The route keeps returning everything by default: Baseline.jsx lists the
segments from the same endpoint, and the ledger's tag lookup needs every
entry to label its lines.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Baseline and reference loads are segment construction, not forecasting.
Listing them beside the adjustments buries what the log is actually for --
on version 29 that is four load entries against however many adjustments --
and offers an undo next to them that would silently gut the version.
They are managed in the Baseline view instead, which is where they can be
edited in place and their date ranges seen. Filtered in the dialog rather
than in the route, since Baseline.jsx and the ledger's tag lookup both read
the same endpoint and do want every entry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
sales_usd and qty run to eight digits here, where two decimals are noise.
Price is the opposite case: it sits around 0.27, so two places barely show a
move and four still round away part of one.
derive() hardcoded two decimals, so the editable rows would have disagreed
with the lines above them. It now takes the measure's own precision. The
percentage row keeps one place, being its own scale.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Price was blanked on the bridge lines and on the excluded row, so a ledger
that had both measures on every line showed a price on two of them. Baseline
sat there as 2,242,180.67 over 8,111,717.10 with the column empty.
A bridge line's price is the implied price of that initiative's own
contribution -- Scale #116 above works out to 0.2397 against a current
0.2735 -- which is the number that says whether the initiative was a price
move or a volume move. That is the same question the plug toggle asks about
the edit, answered for the adjustments already made.
A line with no units is left blank, since its price is undefined rather
than zero -- the "price" bridge line in that example moved dollars alone.
The price column's hint said "holds units", which stopped being true when
plug arrived: it holds units under plug = price and moves them under plug =
volume. It now just says what the column is.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Each ledger row derived only from its own input, so typing a dollar figure
left the units and price rows blank -- which are precisely the two numbers
that say whether you just asked for a volume change or a price change. The
plug toggle decided it and then showed you nothing.
Adds a Result line under the edit rows carrying value, units and price
together, resolved the same way the server resolves the increments: a price
input multiplies through volume, a dollar input with plug = volume scales
units in proportion, anything else holds units. Measures that actually move
are picked out in blue against the ones that hold, so plug = price reads as
price moving alone and plug = volume as price standing still.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A sales figure on its own does not say which of price or volume moved, and
scale was answering silently: it wrote the value increment and left units
alone, so volume stayed flat and price absorbed everything. That is one of
the three behaviours the Excel form offered, picked by default rather than
by the user.
/opt/forecast_api/VBA/fpvt.frm has the semantics. Edit Sales, then plug one
side:
plug volume: pchange = fVal/(pVal+bVal); fVol = (pVol+bVol)*pchange
plug price: fVol = pVol + bVol
So 'volume' moves units in the same proportion as the dollars, holding
price; 'price' leaves units alone, as now. Checked against those formulas:
+100 on 1000/500 gives units 500 -> 550 with price held at 2.0000, and
-400 on 2000/800 gives 800 -> 640 at 2.5000.
Price still defaults, so nothing changes for an existing caller that does
not send `plug`. The control only appears once the edit is dollars-only --
naming units or price has already settled the question. A price target now
also honours a units target alongside it, which is the form's Edit Price
mode where both are inputs and dollars fall out.
Holding price when the selection has no value is refused rather than
divided by zero, the same case the form guarded with "Zero times any number
is zero".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two things the trace showed, one of them a wrong call on my part.
getTable() was the wrong thing to wait on. The re-apply threw "No table
set" two milliseconds after the event, meaning getTable() had already
resolved while getView() still had not -- it is the view that is missing
around a rebuild, not the table. There is no predicate for "the view is
ready", so attempt applyDepth and retry until it stops throwing, up to 5s.
And the shim matched nothing: not one callback reported a viewer. closest()
stops at a shadow boundary, so it cannot climb from an element inside
Perspective's shadow root out to the host, which is where the observed
elements live. Walk host to host via getRootNode().host instead.
The shim now also logs every callback it sees under [pf-obs], matched or
not, with a description of each target. If it still reports nothing at all,
the shim is not installed and the problem is import order rather than
matching.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The version re-validation effect could not tell "the versions fetch has not
landed yet" from "this source has no versions", because both look like an
empty array. So a versionId restored from localStorage was cleared on the
first render and set straight back when the fetch returned: 29 -> '' -> 29.
Forecast's load effect is keyed on [versionId, sourceId], so that round
trip ran initViewer twice. Two /agg requests, two full aggregations in
Postgres, two Arrow payloads -- on version 29 that is the 285k-row
aggregate computed twice at ~17s each, which is most of the "it hangs
before it displays" and the Postgres process sitting at the top of htop.
sourcesLoaded already guarded the identical effect for sources; versions
just never got the same treatment. Cleared when the source changes, since
the list belongs to the previous source until the new fetch lands.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The WASM engine has no notion of focus or tab visibility. It re-renders
because IntersectionObserver or ResizeObserver told it the element is
visible again at some size, and that re-render discards the view -- taking
set_depth() and per-node expansion with it, since neither lives in
ViewConfig. Listening for window 'focus' was only a guess at when that
happened, which is why the re-apply kept landing while the viewer was still
detached from its table.
Perspective's viewer captures the constructors at module-evaluation time
(`var it=window.ResizeObserver; var st=window.IntersectionObserver`), so
replacing them on window before that import puts us in front of every
callback it gets. observerShim.js does exactly that and nothing else: the
original callback runs first and unchanged, and we re-emit as a CustomEvent
only when an entry's target is or contains a perspective-viewer. That has
to be the first import in main.jsx or there is nothing left to intercept.
focus/visibilitychange/pageshow stay as a backstop for the case where the
shim could not be installed. ResizeObserver fires on every frame of a panel
drag, so observer events trail by 150ms and act once the burst settles.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The refocus timing is the sort of thing that will shift again -- a
Perspective upgrade, a different machine, a slower load -- and rebuilding
this instrumentation from scratch each time is wasted work. So it stays,
behind a flag, instead of being reverted.
probeView() returns before touching the viewer when the flag is off, since
it calls getView() purely to report identity.
Note for whoever reads the log next: the view ids it prints cannot be used
to detect a rebuild. getView() hands back a fresh wrapper object on every
call, so consecutive calls always look "rebuilt". The reliable signal is
whether the re-apply throws "No table set".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The instrumentation showed the re-apply mostly throwing "No table set": the
viewer is detached from its table for a while after a refocus, the fixed
60ms delay fired inside that window, set_depth threw, and the tree rendered
fully expanded. The handful of times the delay happened to be long enough,
the log reads "re-apply done depth=0" and the collapse survived.
So poll for the table instead of guessing, up to 5s. queued now clears in a
finally after the work rather than at the top of the callback, so the focus
and visibilitychange that both fire for one window switch no longer each
run a re-apply -- that was the doubled applyDepth in the log.
Does not address per-node +/- collapse, which is still not recorded at all:
expandDepthRef stays null because only the toolbar buttons set it, and the
view exposes expand()/collapse() with no getter to read the state back.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Temporary. Row grouping re-expands whenever the browser regains focus, for
both the toolbar EXPAND buttons and the per-row +/-. set_depth() and
per-node expansion both live on the view rather than in ViewConfig, so a
rebuilt view loses them -- but nothing so far distinguishes "the view was
rebuilt" from "the re-apply lost its race with the redraw".
Tags each view object with an id via a WeakMap and logs it at every point
that could rebuild one: focus/visibilitychange/pageshow, config-update,
applyDepth, the theme effect, and initViewer. A changed id across a refocus
means the view was discarded; an unchanged id means the depth re-apply
itself is at fault. Each bail-out in the refocus handler now says which
branch it took, which also shows whether expandDepthRef was ever set.
Prefix [pf-depth]. Revert this commit once the cause is known.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Brings in the /agg endpoint, col_meta.in_grain, the pf_gkey index and the
append/remove write model that replaces undo's full reload. On the live
2,574,287-row forecast this collapses to 32,411 rows at a
rep/customer/channel/season/month grain, with totals tying exactly
(857,792,111.91 either way) and pf_gkey unique across every group.
Three conflicts, all from work done on this branch after the grain branch
was cut:
- sql_generator exports: union of both sides, adding grainOf.
- Forecast.jsx fetchArrow: the grain branch factored the inline progress
reader into a helper; kept the helper, and the tag/note ledger functions
beside it, since the two were only textually adjacent.
- Forecast.jsx initViewer: took the grain branch's endpoint selection, but
dropped its loadPerspective() -- 99375bb replaced that lazy CDN loader
with a static inline import, so awaiting fetchArrow directly is correct
here.
Carried the segment labels into grain mode as well: /agg now joins pf.log
the way /data does. pf_logid is part of the grain, so the join adds no
rows. Without it the labels would have disappeared exactly when a source
declared a grain -- which is the mode that will actually be used.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The load filter dropdown listed role 'date' and role 'filter' only, so a
segment could not be cut on sseas without recoding it -- and role is a
single value, so that trade would have taken it off the pivot to put it in
a dropdown.
Nothing downstream required the restriction: the server drops filter_clause
into the load's WHERE against the source table without consulting col_meta,
and this form takes values as free text rather than through the key-only
values endpoint. So the fix is to stop filtering them out.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ordinary npm churn: dropped peer markers where those packages are now
direct dependents, plus the optional @emnapi platform packages.
.env is ignored but a .env.bak sitting next to it is not, and it holds the
same credentials. Widen the pattern to .env.* so a backup can't be staged
by a careless git add -A.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The selection is restored from localStorage, but it was only validated once
at mount. Deregistering the selected source left App holding an id that no
longer exists, so every subsequent call 404'd "Source not found" with no way
out but a reload — Setup.deleteSource clears its own selectedSource and never
tells App. Deleting the selected version had the same shape.
Move both checks into effects keyed on the lists themselves. A sourcesLoaded
flag keeps the source effect from firing against the initial empty array and
wiping the restored id before the fetch resolves.
Also coerce both refresh callbacks to an array. A 401 returns an error object,
and data.some() then threw inside an unhandled promise, stranding the
selection instead of clearing it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
(cherry picked from commit e6e2faeb37)
The server had no authentication: every /api route was open, CORS
allowed any origin, and the identity written to the audit log came from
the request body — the UI sent a hardcoded pf_user: 'admin', which any
client could have set to anything it liked.
Accounts live in pf.app_user with scrypt hashes from node's own crypto,
so there is no native build step and the parameters travel with each
hash. Sessions are express-session over connect-pg-simple in pf.session:
a restart no longer signs everyone out, and a session can be revoked by
deleting its row, which is how disable-user cuts off access immediately
rather than at cookie expiry.
Everything under /api except login/logout/me now requires a session, and
the React app is mounted only once there is one — its load effects call
the API on mount, so a logged-out mount would just fire a burst of 401s.
A session that expires while the app is open lands back on the login
screen: auth.jsx wraps fetch once rather than teaching every call site
to check.
Identity is now read from the session for pf_user, created_by and
closed_by, and the body values are ignored.
Hardened for an internet-facing deployment: trust proxy so req.ip and
secure-cookie detection are right behind TLS termination, httpOnly +
SameSite=Lax + Secure cookies, ten login failures per IP per fifteen
minutes, one error message for unknown, wrong and disabled alike, and a
fresh session id on success. CORS is off entirely unless CORS_ORIGIN
names an origin — a wildcard alongside a session cookie would be CSRF by
construction. The server refuses to boot without SESSION_SECRET rather
than falling back to a guessable default.
pf.sh grows add-user, passwd, list-users, disable-user and enable-user;
passwords are read on stdin and hashed before they reach psql, so no
plaintext in argv or shell history. install.sh generates the secret,
applies 02_auth.sql, and creates the first account.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Perspective's column axis cannot be collapsed. The row axis has had it
forever — GROUP BY ROLLUP holds every level and view.set_depth() hides the
deeper ones — but nothing equivalent is exposed for split_by, so a Year over
Month pivot can only ever be shown fully expanded.
The engine already implements it. t_ctx2 is symmetric: set_depth(t_header,
depth), open(t_header, idx) and close(t_header, idx) each have a real
HEADER_COLUMN branch on m_ctraversal mirroring m_rtraversal, and
t_view_config carries m_column_pivot_depth which server.cpp already applies.
None of it is reachable: set_column_pivot_depth() is never called, so the
depth stays -1, and View<t_ctx2>::expand/collapse hardcode HEADER_ROW. The
patch is 193 lines of wiring across the protobuf, the Rust client and the
datagrid — no new engine logic.
Two capabilities result, mirroring the row axis:
- split_by_depth in ViewConfig, the split_by counterpart to group_by_depth
- expand_column()/collapse_column(), addressed by column traversal index
exactly as the row methods are addressed by row index
which together give the Excel behaviour — one year folded to its subtotal
while its siblings stay expanded — that no combination of existing config
could produce. Verified in this app against fc_cash_9: clicking a Year
header goes from 27 columns to 15, totals reconciling at every level.
Vendored rather than aliased
- The previous approach pointed vite at a local checkout, which built only on
one laptop and left package.json claiming npm 5.2.0 while the build used
something else. The four packages are now committed as npm tarballs and
package.json names them, so the declaration is true and `pf.sh deploy`
works unchanged — npm install expands them like any registry package.
- Packed with `pnpm pack`, not `npm pack`: Perspective is a pnpm workspace and
cross-package deps are `workspace:^`, which npm rejects outright. pnpm
rewrites those to real version ranges at pack time.
- All four move together. Perspective couples loader, package versions, data
format and apache-arrow; a partial vendor reintroduces exactly that drift.
Cost, stated plainly: 12MB of opaque binaries in git that do not delta, a
fork to maintain, and a second engine build for anyone changing it.
rebuild-perspective.sh makes that repeatable and PROVENANCE.txt records the
commit each tarball came from, because a committed .tgz otherwise has no
recoverable source. README.md says how to delete all of it once upstream
ships the feature.
This also moves the app from 5.2.0 to 5.4.0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoxNi8cFsQLPUSw3obb5NH
The row axis has had Expand 0/1/2/3 since the pivot landed; the column
axis had nothing. With a year over month split there was no way to step
back to whole years short of dragging split_by apart in the settings
panel and putting it back afterwards.
Perspective gives the two axes nothing in common here. Rows collapse
because the GROUP BY ROLLUP view holds every level at once and
view.set_depth() hides the deeper ones. For columns there is no
equivalent: expand()/collapse() take a row index, ViewConfig has
group_by_depth but no split_by_depth, and split_rollup_mode only chooses
whether subtotal column groups are emitted — a view shape, not an
interaction. So applySplitDepth() collapses by restoring a truncated
split_by, which rebuilds the view.
Three consequences of that rebuild, each handled:
- Once collapsed, viewer.save() only reports the short split_by, so the
full hierarchy is held separately (splitFullRef) and persisted into the
layout as split_full. Without it, collapsing would be a one-way door:
reload while collapsed and the deeper levels are gone. adoptSplit() is
the single place it is set.
- perspective-config-update fires for our own restore as well as for the
user rearranging the pivot, and the two mean opposite things — one must
adopt the new hierarchy, the other must not. collapsingRef separates
them.
- Row depth lives on the discarded view, so it is re-applied afterwards.
The selection is cleared on each change: slices name the split_by
dimensions they were cut from, and the highlight is keyed on grid
coordinates. Neither survives a column axis that just changed shape.
Buttons are named for the level they show — Total, then one per split_by
column — rather than numbered like Expand, since the levels are named and
a number would say nothing about what you are collapsing to.
Whole-axis, not per-branch: Excel can collapse 2025 while 2026 stays
expanded, and this cannot. `columns` selects which measures appear, not
individual split combinations, so there is no way to hide one branch's
leaves while keeping another's.
Verified in the browser against cash/test with split_by Year x Reason:
Reason -> Total -> Year -> Reason all render the expected column sets and
the right button highlights; and a reload while collapsed to Year comes
back collapsed with Reason still offered.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoxNi8cFsQLPUSw3obb5NH
Two gaps in how the pivot reports a selection. Dragging across cells did
nothing, and a selection built from several ctrl-clicks was invisible on
the grid — the panel listed the slices but nothing on screen said which
cells they came from.
Drag-select
- The perspective-select handler was reading detail.selected and
detail.insertConfigs. In 5.2.0 that event carries a ViewWindow —
{ start_row, end_row, start_col, end_col }; insertConfigs only appears
on perspective-global-filter, and only in SELECT_ROW_TREE mode. So the
handler always returned early and the whole path was dead.
- It fires on every mouseover as the region grows, so the handler now
records the latest window and a window-level mouseup commits it. A
single-cell region is skipped there: perspective-click already owns
plain and modifier clicks, and handling it in both places would undo a
ctrl-click toggle. Modifier+drag adds to the selection.
- Turning a region back into slices re-derives, per cell, the same
filters Perspective attaches to a click — row dimensions from the
view's __ROW_PATH__, column dimensions from the split_by segments of
the column name. Reading the path rather than the rendered cell is
what keeps dates as epoch millis instead of whatever the grid
formatted them as. The grand-total row resolves to no dimension at
all — that would mean the whole version — and is skipped.
Highlight
- The datagrid already highlights whatever sits in its own
model._selection_state.selected_areas, and wipes that list on every
mousedown, so a multi-click selection only ever showed the last cell.
Keep a sliceKey -> rectangles map parallel to `slices` and push the
full set back after each change, which gets the native highlight for
every selected cell without any styling of our own.
- Deselecting anywhere — ctrl-click, the panel's x, Clear selection —
prunes the map by live slice key, so the grid and the panel cannot
disagree.
Also: edit_mode is forced to SELECT_REGION on restore rather than
defaulted, since a saved layout's plugin_config could previously
override it and turn selection off entirely.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LoxNi8cFsQLPUSw3obb5NH
Forecast operations could only act on one clicked row at a time, and the
panel that drove them separated the numbers you were reading from the
inputs that changed them. This reworks both, and adds initiative tags so
a version's history can be read as a bridge.
Operations
- Accept `slices` (array) alongside the legacy single `slice`, with
apply_mode 'prorate' (one pool) or 'each' (independent per slice).
- buildWhereAny() ORs the slices into one predicate. A union of slices
cannot be flattened into per-column IN lists without over-selecting,
and the result is parenthesised so the appended exclude clause does not
bind wrong.
- resolveIncrs() now resolves each measure independently: target, percent
or change amount per measure, so a target on value and a percent on
units can be submitted together. Replaces the single global `mode`.
- target_basis chooses what a target measures against: only the rows an
operation can write, or everything the pivot shows for the slice.
Excluded iters are visible in the grid but immovable, so a target set
against the visible total previously overshot by their contribution.
Two latent bugs surfaced by the above, both pre-existing:
- A slice naming no filterable column reduced to TRUE and applied the
operation to the entire version. Now rejected on all three operations.
- Prorating across a pool that nets to ~zero multiplies each row's share
by an exploding factor, sending rows to extreme opposite values to hit
the target. Refused when the net falls below 1% of gross.
Tags and the bridge
- pf.log gains a nullable `tag`, written by a follow-up UPDATE rather
than through the generated SQL: those templates are stored per source
in pf.sql, so a {{tag}} token would strand any source that had not
re-run "Generate SQL".
- Tag is editable after the fact in the change log, with completion from
tags already used on the source. PATCH branches on whether a field was
sent, so a tag can be cleared as well as set.
- BridgeView renders the walk from baseline to current as a waterfall,
one step per tag, scoped to the selection, the pivot's filters, or the
whole version. Computed from the loaded Perspective table so the
figures always reconcile with what is on screen; overlapping slices are
deduped by pf_id to match the OR semantics operations use.
- Colour is a polarity job, so it uses the validated diverging pair
(blue/red, CVD dE 21.6) with neutral anchors, not categorical hues.
Every bar is directly labelled and a table view is available.
Panel
- Extracted to OperationPanel; the scale form is one continuous ledger:
baseline, each adjustment, current, then New value / Change / % change
as three interchangeable editable rows. Typing in any one derives the
others, which removes the target/delta/percent mode toggle entirely.
- Dockable bottom, right, or floating (drag to move, grip to resize), and
closable via header, Esc, or the toolbar. Placement persists.
- Controls no longer stretch to the dock width, and text contrast now
clears WCAG AA against white throughout.
Also: the status bar names the physical table writes land in, with live
row counts; and the pivot's expand depth is re-applied when the tab
regains focus, since Perspective rebuilds its view on redraw and a
ROLLUP view with no depth set renders fully expanded.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W1TQiBYZbbWWkMNoCtUd8M
Ship rows pre-aggregated to the grain the pivot displays instead of raw
forecast rows. This is Path B from pf_perspective_options.md: it keeps
Perspective's native WASM engine — so expand/collapse/depth/sort/filter
all still work — and fixes load time by cutting rows, not transport.
Measured on pf.fc_osm_stack_20 at pending_rep x customer x smon:
534,902 -> 6,154 rows (~87x), pf_gkey unique across all 6,154, and both
measures reconcile exactly to the raw totals.
The grain is static: flagged once per source in Setup and baked into the
stored pf.sql templates, so load and operations agree by construction.
Sources with no flagged column keep the previous raw-row behaviour, so
this is backward compatible.
- pf.col_meta gains in_grain; grainOf() in lib/sql_generator.js is the
single definition of the grain and is reused by routes/log.js.
- New get_agg template + GET /api/versions/:id/agg, generated only when a
grain is defined. Regenerating drops templates no longer produced, so
clearing the grain falls back to /data.
- scale/recode/clone now aggregate their own new rows to grain before
returning. Because pf_logid is part of pf_gkey those keys are always
new, so table.update() appends and the view re-sums — the Excel
pivot-cache pattern, no bucket recomputation.
- Undo reports pf_gkeys (RETURNING cannot take DISTINCT, so the delete
feeds a CTE that reduces to distinct keys); the client removes those
index values and the view re-sums.
- pf_gkey is concat_ws(chr(31), COALESCE(col::text, chr(30)), ...).
The separator and NULL sentinel are load-bearing: plain concat_ws skips
NULLs, so ('a',NULL) and (NULL,'a') would collide and silently merge two
groups into one indexed row.
- Forecast.jsx reads col_meta first to pick /agg vs /data; the Arrow
streaming logic is extracted to fetchArrow() since both share it.
- Setup.jsx gains a grain checkbox and shows the resulting grain.
- 01_schema.sql: move the col_meta ALTERs after its CREATE TABLE — they
referenced the table before it existed on a fresh install.
All six generated statements verified to plan against the real forecast
table; the in_grain column has been added to the dev database.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The pivot stopped rendering with:
LinkError: WebAssembly.instantiate(): Import #8 "env" "psp_opfs_load":
function import requires a callable
Nobody changed anything. The 4.x CDN bundle resolves its server WASM with
new URL("../../../server/dist/wasm/perspective-server.wasm", import.meta.url)
which from .../client@4.4.0/dist/cdn/ resolves to
.../npm/@perspective-dev/server/dist/wasm/perspective-server.wasm -- with no
version. jsdelivr serves @latest, so when @perspective-dev/server@5.2.0 was
published on 2026-08-10 every page load began linking a 5.2.0 WASM against a
4.4.0 client. 4.4.1 and 4.5.2 carry the identical unversioned pattern, so no
4.x pin is safe over CDN. Beyond the outage, an unversioned URL means users
execute whatever that package publishes next, unreviewed.
Switch to the /inline entrypoints, which embed the WASM in the Vite build:
no runtime fetch, and the version is fixed by package-lock.json (verified:
zero `new URL(...perspective-server...)` in perspective.inline.js).
- pin client/viewer/viewer-datagrid/server exact at 5.2.0. The explicit
`server` pin matters: client declares it as "" (an empty range), which npm
also resolves to latest -- the same break, at install time instead.
- drop the viewer-d3fc import. pf_app never selects a chart plugin, and d3fc
has no 5.x; loading 4.4.1 against a 5.x viewer only emits
`get_static_config is not a function` per plugin.
- themes move from a CDN <link> to @perspective-dev/viewer/themes.
Verified end-to-end with every non-localhost request aborted: no external
requests are attempted, both custom elements register, the Arrow stream
ingests, and the pivot renders (TOTAL 17,235.97 = -7,573.97 + 30,907.47
- 6,097.53). apache-arrow 21.1.0 ingests cleanly against the 5.2.0 WASM.
Bundle grows 263 KB -> 11.6 MB (5.4 MB gzipped); that is the embedded WASM.
PERSPECTIVE.md also records findings from the same investigation:
- §3a: expression columns are row-level, evaluated before aggregation, so a
ratio like "revenue"/"qty" summed per row is wrong under any pivot (not
just split_by). Fix is a weighted-mean aggregate, whose weight column must
be a NESTED array: ['weighted mean', ['qty']]. Works in 4.4.0 and survives
incremental table.update().
- §2: withdraws the recommendation of the 4.5.1-core + 4.4.1-d3fc pair. It
does not deliver charts, so the trilemma is really a dilemma: inline
bundling XOR charts. dataflow is on that pair and needs the same migration.
- §5: cleanLayout() does not sanitize `aggregates`; guard it before adopting
the weighted-mean pattern or a dropped column aborts the whole restore.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SBt3EtKaP9D2mmWJ6Q4bov
Replaced native <select> (macOS ignores CSS on option elements) with a
custom button+ul dropdown. Background/text/border colors are applied via
useTheme so they respond correctly to dark mode toggle.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GET /api/dim-period/cols queries information_schema for pf.dim_period columns
(excluding sdat/edat/drange/ndays) so the UI always reflects actual columns.
Setup col_meta editor now shows a dropdown populated from that endpoint instead
of a free-text field, preventing invalid column names like the cash source had.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- SQL generator no longer requires a units col; recode/clone/scale omit units
expressions when none is configured in col_meta
- Source registration validation drops units from required roles (value + date
are the only hard requirements)
- DELETE /api/sources/:id returns 409 when existing versions reference the source
- Setup.jsx surfaces the 409 error via flash instead of silently failing
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- col_meta gets dim_period_col field: maps a dimension column to its pf.dim_period counterpart (e.g. year -> cal_year, month -> cal_month)
- When the date column is is_key of a dim_group and any sibling dimension has dim_period_col set, baseline and reference SQL JOIN pf.dim_period on the shifted date instead of copying raw source values
- No dim_period config = identical SQL to before (fully backwards compatible)
- Setup UI: period col input in col_meta editor, enabled for dimension columns with a dim_group set
- Schema migration applied: dim_period_col text null on pf.col_meta
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GET /api/sources/:id/lookup?col=X&value=Y — given a key column value, queries the source table for sibling column values in the same dim_group; returns null if no match or ambiguous
- Recode and Clone panels: key columns (is_key + dim_group) trigger lookup on blur and auto-fill sibling inputs that the user hasn't already typed into
- Row labels now use col_meta label field when set, falling back to cname
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- col_meta: add dim_group field to group related columns (dimension hierarchies, date-adjacent columns); is_key now enabled for date role to mark group parent
- sources.js: upsert includes dim_group
- Setup.jsx: group column in col_meta editor, key checkbox enabled for date role
- gen_dim_period.sql: create and populate pf.dim_period with calendar and fiscal period cuts (monthly grain, 2018-2035)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Switch server Arrow encoding from tableFromJSON (row objects) to
tableFromArrays (column arrays) — cuts peak Node heap 3-5x for large
datasets by avoiding one JS object per row
- Remove unused pf.log JOIN from data endpoint; forecast rows only
- Load Perspective viewer with direct table reference instead of worker
Server object — fixes "No Table attached" error on large datasets where
named-table registry lookup raced against WASM initialization
- Pre-emptively clean up stale named table in worker registry before
creating, eliminating the "already exists" retry path that silently
swallowed errors (finally ran but flash never fired)
- Strip cfg.table from restore configs since table is loaded by reference
- Throttle progress bar updates to 100ms intervals (was every chunk)
- Persist load errors until dismissed; add console.error for devtools
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
- Add PayloadPreview component showing the exact JSON that will be POSTed,
live-updating as form fields change (value_incr shown as computed delta)
- buildEffectiveSlice strips expression/system columns and converts
Perspective ms-timestamps to ISO date strings for date-role columns
- fetchCurrentTotals now includes date columns in Perspective view filter
(passing ms number as Perspective expects) so subtotals respect the
clicked date
- Server buildWhere now receives filterCols (dimensions + date cols) so
date values reach the SQL WHERE clause correctly
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace Bootstrap fill icons with Feather-style stroke SVGs (sun with
rays + crescent moon) in StatusBar toggle.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Forecast falls back to a saved per-source layout when no version-local
layout is cached, so new versions of a source open with a sensible pivot
without each user reconfiguring it.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reference segments can now apply a date offset just like baselines.
SQL template gains the {{date_offset}} token; both POST /reference and
PUT baseline/:logid pass it through. Existing sources need to
regenerate SQL to pick up the new template — old stored reference SQL
ignores the token (preserving prior verbatim behavior). The Baseline
form drops the "dates land verbatim" hint and shows the offset
control for both segment types.
Editing a segment now color-codes the source row amber with a ring
and tints the form border + header amber so the active connection is
visually obvious. Header label reads "Edit segment #3 — baseline —
note" instead of just "#43" (the internal log id).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New forecasts opened the pivot with all dimensions stacked as
group_by and the date column as split_by — wide and slow to read.
Open with just the value column showing and pf_iter as rows so the
first thing you see is iteration totals.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The segment form is now one component rendered in either 'view' or
'edit' mode — the expanded segment row in the list and the
add/edit form below share the same layout, view mode just disables
the inputs. Edit and View are visually identical so toggling between
them feels like enabling fields, not switching tools.
Filters become groups (conditions AND-ed inside, groups OR-ed
between) with + AND condition and + Add OR group affordances. The
compiled WHERE renders live below the groups so you can see what's
being built. A "Switch to manual SQL" toggle flips to a textarea
seeded with the compiled clause; backend baseline POST/PUT and
reference POST accept raw_where alongside filters and store whichever
arrived in pf.log.params for round-tripping.
The Add form is hidden until you click "+ Add segment" at the
bottom of the segments table; Edit also opens it. Cancel/Close
returns the table to its compact state.
/versions/:id/log now also returns value_total, units_total, and the
column names so the segments table can show row count and value sum
inline (header uses the source's actual value column name).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds PUT /versions/:id/baseline/:logid that, in one transaction, drops
the segment's rows and log entry and replays the baseline or reference
SQL with new params. The endpoint refuses (409) if any scale, recode,
or clone has been applied — those operations were calibrated against
the old totals and would silently misreconcile.
Baseline view gets an Edit button on each segment (hidden once
forecast operations exist), populating the form with the original
filters, offset, and note. Submit issues PUT in edit mode, POST
otherwise. POST baseline and POST reference now also persist the
structured filters in pf.log.params so edit can reload them.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Save/restore went through both viewer and plugin, where the explicit
plugin.restore could stomp the column formatting the viewer had
already applied. Capture via viewer.save() alone (it includes
plugin_config) and restore via a single viewer.restore call with
edit_mode merged in. Added a perspective-config-update listener so
formatting, sort, and other in-place changes persist to the last-used
cache without an explicit Save.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The slice panel was a single muted line; now it shows a breakdown
table — value, units, and derived price for baseline / scale / recode /
clone, with a bold total row when more than one iteration applies.
Numbers use full text contrast so the current state is legible at a
glance during adjustments. Scale gains a price input that holds units
constant and translates to a value-target call (target value =
new_price × current_units).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Reuse a single Perspective worker across version switches and delete
the previous table instead of terminating the worker — terminate was
returning a rejecting promise the sync try/catch missed, and each new
worker leaked WASM memory. applyLayout no longer leaks a view per call;
it reads schema directly from the table. An init id guards against
concurrent runs (StrictMode, rapid version switches) clobbering each
other, and a catch on "already exists" recovers via open_table+delete
when a stale table from a previous run is still hosted.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
App.jsx owns sources/sourceId/versions/versionId and persists them
across reloads. StatusBar renders the dropdowns plus a status badge —
Source-only on Setup, Source · Version · status on Baseline/Forecast.
The duplicate in-view selector bars in Forecast and Baseline are gone;
Baseline keeps its version actions (New/Close/Reopen/Delete) inline.
Setup reports source-list mutations up via refreshSources so the bar
stays in sync after register/deregister.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
App chrome now uses Pro Dark's neutral grays (#242526 background,
#2a2c2f panels, #4c505b borders) so the surrounding UI sits cleanly
against the viewer instead of clashing with its warmer tone. Status
accents are desaturated to match. Forecast view sets theme="Pro Dark"
or "Pro Light" on the perspective-viewer in sync with the toggle.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Perspective table is now created with index: 'pf_id'. Delete endpoints
return the pf_ids they removed; the client calls table.remove(pf_ids)
in undoEntry. Avoids the full /data refetch that dominated undo time.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
pg now returns bigint/numeric as JS numbers so Arrow infers Int/Float64
instead of Dictionary<Utf8>. /data accumulates rows and emits a single
record batch to avoid dictionary REPLACEMENT messages that crash
Perspective's WASM reader. Forecast view streams the response body and
shows received/total bytes while loading. Drops stale public/ static
middleware that was shadowing the React build at /.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Server streams rows from a pg cursor in 10k-row batches, building Arrow
record batches incrementally and piping them as chunked HTTP response —
Node.js heap stays bounded regardless of dataset size.
Client fetches as arrayBuffer() and loads directly into Perspective worker
(native Arrow path, no JSON deserialization). X-Row-Count header drives
a non-blocking banner for datasets >= 500k rows. validCols now derived
from col_meta rather than from row keys.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Re-applies fetchCurrentTotals dimension-only filter (prevents Perspective errors
from split_by/expression columns in the filter), toolbar three-group reorganization
(Layout | Expand | Data with dividers), always-visible Save as…, msg in toolbar,
resizable panel, change log modal with undo and inline note editing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Expression columns (bucket, computed) are defined in cfg.expressions and
are valid pivot axes, but weren't in validCols (raw table columns), so
they were filtered out of group_by/split_by on every layout restore.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GET /api/versions/:id/log — log entries with row counts via JOIN
- DELETE /api/log/:logid — undo in a transaction (delete fc rows + log entry)
- PATCH /api/log/:logid — update note text
- History button opens a modal: op badge, slice, editable note, row count, Undo per entry
- Undo triggers full Perspective table reload via initViewer
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix perspective-click handler to use event filter triples instead of
__ROW_PATH__ — Perspective encodes row position as [col,'==',val] in
detail.config.filter
- buildWhere now skips unrecognised slice keys (e.g. pf_iter) instead of
throwing, so only dimension columns reach the WHERE clause
- Add draggable resize handle on the operation panel (160–480px)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Baseline.jsx: merge Reference section into Add Segment form with baseline/reference toggle; segment rows now clickable to expand stored WHERE clause + timeline; date filter inputs use type="date" for date-role columns
- Timeline.jsx: add type prop ('baseline'|'reference'); reference band uses purple; single-band height shrinks to 52px; canvas uses requestAnimationFrame to fix offsetWidth=0 on mount
- operations.js: reference route now accepts where_clause like baseline (drops date_from/date_to)
- sql_generator.js: reference SQL template uses {{filter_clause}} instead of hardcoded BETWEEN
Note: existing sources need Generate SQL re-run to pick up the new reference template.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- ui/: React + Vite + Tailwind app (Setup, Baseline, Forecast views, collapsible sidebar, status bar, canvas timeline)
- server.js: serve built UI from public/app/
- package.json: add build script (cd ui && npm run build)
- routes/sources.js: default new col_meta role to 'dimension' instead of 'ignore'
- .gitignore: exclude public/app/ build output
- pf_spec.md: update tech stack, nav, frontend section, and project status to reflect current implementation
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
