Commit Graph

3 Commits

Author SHA1 Message Date
584df9dc03 module adds 2026-09-04 09:26:09 -04:00
b855ab91d7 Make the three AP history modules incremental
The Accounts Payable group ran ~9.5 min, 83% of it in four full
reloads. The cost is the GPSERVER linked-server hop, not row volume
(pm30200 managed only ~590 rows/sec), so cutting rows crossing the link
is the whole win. Predicates go inside OPENQUERY to push down to the
remote — a 4-part name would drag the table across before filtering.

  pm30200  own DEX_ROW_TS
  pm30600  changed set: parent PM30200
  pm30300  changed set: PM30200 union PM20000

All three key on (vchrnmbr, doctype), verified exactly unique in
PM30200 at 108,803/108,803. Watermark resolves off gp.pm30200 with a
7-day lookback, so run order within the group must stay pm30200 first
and the lookback must exceed the sync interval.

The two-parent union on pm30300 is load-bearing: 38 rows have a parent
only in the open table, and a PM30200-only join would strand them
permanently. pm30600 needs no union (0 orphans, verified).

pm30700 stays full — 213 of its rows have no parent in either table, so
no changed-set join can reach them, and it only costs 13s. pm20100 and
pm00400 stay full because open and index tables delete rows, which a
delete-by-key incremental structurally cannot propagate (cf. icstt).

391s -> 15s. reconcile.py --super-quick reports IN SYNC for all three.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 00:21:59 -04:00
e8a3acaebd Add GP Payables (PM) modules and an Accounts Payable group
Only pm00200 (vendor master) covered the AP side; every transactional
PM table was missing even though the parallel receivables (rm*) set was
fully built out. Add ten modules mirroring that AR structure:

  pm00100  vendor class setup        pm20100  open apply detail
  pm00300  vendor address master     pm30200  paid transaction history
  pm00400  key master (doc index)    pm30300  apply history
  pm10100  open GL distributions     pm30600  GL distribution history
  pm20000  open transactions         pm30700  tax history

All full reloads. The whole set is <=230k rows and loads in seconds over
the Postgres COPY path, so a watermark buys nothing — and a date-keyed
incremental carries the row-level purge drift documented for icstt.
pm20000 and pm30200 do have DEX_ROW_TS if that changes.

Column maps come from pipekit's own /api/introspect/columns over the
GPSERVER linked server. Descriptions are null throughout: GP populates
no MS_Description extended properties (CHG has zero), and the metadata
lives in the Dexterity dictionary rather than SQL Server.

The group is scheduled 0 1 * * * but left disabled, matching the
Accounts Receivable group.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 00:06:09 -04:00