Commit Graph

7 Commits

Author SHA1 Message Date
584df9dc03 module adds 2026-09-04 09:26:09 -04:00
d0e72e270f Add the CMS AP subledger modules (avtx, vchr, open)
Completes the AP source layer: GP was covered by the pm* modules, but the
CMS side had only cms.vend. These three are what
SQL/fiscal/analysis/open_ap_aging.db2.sql actually joins.

  avtx  966,553  A/P voucher transaction events   incremental on AVTSTM
  vchr  792,348  vouchers entered, period to date full
  open  538,544  open payables; FHCBAL is live    full

AVTX is the artrn analog -- the per-event apply detail. It matters
because the AR aging in /opt/clean reconstructs balances from
application events rather than a single settle_date, so AP needs an
applications fact too, not just documents.

Grains verified exactly unique before drafting: AVTX
(avtco#,avtvh#,avttx#) 966,553/966,553; VCHR
(idcom#,idvch#,idbnk#,idfisy,idfisp) 792,348/792,348; OPEN
(fhcom#,fhvch#,fhpyy,fhpper) 538,544/538,544. All three reconcile
IN SYNC.

vchr and open stay full: no change signal, only business dates, and
FHCBAL is a live mutable balance.

Two things worth knowing for the next DB2 module:

- pipekit maps DB2 TIMESTMP to Postgres text, so a watermark resolver
  doing COALESCE(MAX(ts), TIMESTAMP '...') fails on mismatched types.
  Cast explicitly. cms.gtran was already the precedent.
- Watermarks resolve at engine step 2 but --create-dest builds the
  table at step 3b, so a watermark reading its own dest table fails on
  the first run. Pre-create the table or seed with a constant.

Note LGDAT.AP* is a dead end -- APVH has 14 rows, APVD has 22. CMS AP
does not live in AP-prefixed files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 01:05:34 -04:00
e8a3acaebd Add GP Payables (PM) modules and an Accounts Payable group
Only pm00200 (vendor master) covered the AP side; every transactional
PM table was missing even though the parallel receivables (rm*) set was
fully built out. Add ten modules mirroring that AR structure:

  pm00100  vendor class setup        pm20100  open apply detail
  pm00300  vendor address master     pm30200  paid transaction history
  pm00400  key master (doc index)    pm30300  apply history
  pm10100  open GL distributions     pm30600  GL distribution history
  pm20000  open transactions         pm30700  tax history

All full reloads. The whole set is <=230k rows and loads in seconds over
the Postgres COPY path, so a watermark buys nothing — and a date-keyed
incremental carries the row-level purge drift documented for icstt.
pm20000 and pm30200 do have DEX_ROW_TS if that changes.

Column maps come from pipekit's own /api/introspect/columns over the
GPSERVER linked server. Descriptions are null throughout: GP populates
no MS_Description extended properties (CHG has zero), and the metadata
lives in the Dexterity dictionary rather than SQL Server.

The group is scheduled 0 1 * * * but left disabled, matching the
Accounts Receivable group.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-09-04 00:06:09 -04:00
4108342dee updates 2026-08-31 16:59:04 -04:00
0f30c4b921 Export config to match the database
config/ had drifted from pipekit.db: group memberships and three
schedules existed only in the DB, cjobdr and cjobh had no sidecars at
all, and ab.{json,sql} was left over from a module no longer in the DB
(an apply would have recreated it). pipekit apply --dry-run now reports
that the database matches config.

Two real inconsistencies fixed in ffpdglr1.json: it was missing the
ffpdglr1_hot watermark added with the rollover hot set, so an apply from
files would have deleted the watermark while the committed SQL still
referenced {ffpdglr1_hot}; and its period resolver had been edited in the
UI without an export, so the file carried a stale format('%L', ...)
variant.

Purchasing members were all run_order 0, which happened to sort
poh, poh_deleted, poi alphabetically and satisfy the real constraints by
accident. Now explicit: poh 1, poi 2, poh_deleted 3 -- poi resolves its
watermark off cms.poh, and poh_deleted's first hook reads cms.poh.

Also describes eight GP modules that had no description (dest_description
is the half the sidecar format carries; the module-level description
lives only in the DB).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-19 00:43:10 -04:00
e5c1ed37f4 updates 2026-08-06 15:39:22 -04:00
dd9f89f4b5 feat: version-control pipeline definitions via export/apply
Add `pipekit export` / `pipekit apply` (pipekit/config_io.py) to serialise the
DB's config — drivers, connections, modules (+ columns, watermarks, hooks),
groups, schedules — to a git-trackable text tree under config/, and rehydrate
it. SQLite stays runtime state; definitions become diffable/reviewable/
revertible.

- config only: run_log/group_run/settings and per-run state columns excluded
- name-keyed refs (portable across databases); source_query in .sql sidecars;
  columns as real JSON arrays for line-by-line diffs
- newline-normalised so CRLF-vs-LF is never a spurious change
- apply is create/update by name; child collections fully synced; top-level
  deletes gated behind --prune; --dry-run prints the plan
- round-trip is identity (export -> apply --dry-run == nothing to do)

Commits the current config/ as the first baseline, capturing the freshly
populated columns_json for rm00101/rm00301/iv00101. Documented in SPEC.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 10:56:55 -04:00