Commit Graph

2 Commits

Author SHA1 Message Date
855dd1a402 Make pm20100 incremental on MODIFDT
My earlier reasoning for leaving this full — "open tables delete rows,
so delete-by-key strands them" — does not hold for PM20100. The table
accumulates rather than sheds: 46,841 of its 46,886 rows belong to a
document already in PM30200 (history), and 46,864 also appear in
PM30300. MODIFDT spans 2017-2026 and is populated on every row.

The changed set is self-referential rather than parent-driven: only 42
of 46,886 rows have a parent in PM20000, so a PM20000 join would sync
almost nothing. Instead, stage every apply row belonging to a voucher
that has any row modified since the watermark, and key the merge on
(vchrnmbr, doctype) so the delete replaces the voucher's whole apply
set. The natural grain (vchrnmbr, doctype, aptvchnm, aptodcty) has 42
duplicates and would drop rows if used as the key directly.

7-day lookback stages 60 rows (738 at 30 days) against 46,886 for a
full reload. 84s -> 5s. reconcile.py --super-quick reports IN SYNC.

The watermark guards MAX(modifdt) with LEAST(..., CURRENT_DATE) so a
future-dated row cannot advance it past real data.

Residual risk, same class as icstt: an upstream delete cannot
propagate. Low here given the table demonstrably retains rows whose
documents went historical years ago.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 00:34:43 -04:00
e8a3acaebd Add GP Payables (PM) modules and an Accounts Payable group
Only pm00200 (vendor master) covered the AP side; every transactional
PM table was missing even though the parallel receivables (rm*) set was
fully built out. Add ten modules mirroring that AR structure:

  pm00100  vendor class setup        pm20100  open apply detail
  pm00300  vendor address master     pm30200  paid transaction history
  pm00400  key master (doc index)    pm30300  apply history
  pm10100  open GL distributions     pm30600  GL distribution history
  pm20000  open transactions         pm30700  tax history

All full reloads. The whole set is <=230k rows and loads in seconds over
the Postgres COPY path, so a watermark buys nothing — and a date-keyed
incremental carries the row-level purge drift documented for icstt.
pm20000 and pm30200 do have DEX_ROW_TS if that changes.

Column maps come from pipekit's own /api/introspect/columns over the
GPSERVER linked server. Descriptions are null throughout: GP populates
no MS_Description extended properties (CHG has zero), and the metadata
lives in the Dexterity dictionary rather than SQL Server.

The group is scheduled 0 1 * * * but left disabled, matching the
Accounts Receivable group.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 00:06:09 -04:00