Commit Graph

5 Commits

Author SHA1 Message Date
560056c39c Add Bucket 2 CMS incremental modules (6 tables)
Watermarked incremental syncs for qcrh, qcri, qtnote, iprcct, icstt, methh,
ported from the legacy /opt/sync merge logic. Rows initially backfilled from
lgdat; columns match the cms.* dest (lgdat subset), pulled from DB2.

Merge keys: qcrh dcord#, qcri ddord#, qtnote ggkey, iprcct tadate,
icstt jhdate (legacy INSERT..ON CONFLICT -> keyed delete+insert),
methh composite anpart,anplnt (legacy DELETE WHERE EXISTS).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 22:17:39 -04:00
6a3ac0e436 Add Bucket 1 CMS→pipekit modules (41 tables)
Full-reload modules for the CMS/DB2 tables migrated off /opt/sync, landing
in cms.*. Columns introspected fresh from DB2 (QSYS2.SYSCOLUMNS). Three large
tables use incremental merge ported from the legacy sync instead of full
reload: iprcctn/iprccto (date watermark on t8date/t7date), ffpdglr1 (fiscal-
period watermark via rprh join, keyed on btid).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 12:33:48 -04:00
c073e70114 Fix _custom_sales_line_connector map: dest_names to match table
Five dest_names in the column map were junk from a crude source-name
sanitizer (ext._price, margin__, _hd_grower, ext._cost,
ext._margin_dollar) and didn't match the physical gp table's clean names
(ext_price, margin_pct, hd_grower, ext_cost, ext_margin_dollar). The load
is positional so this was harmless for years — until the new dest-table
preflight's name check started (correctly) flagging the drift and failing
the daily run. Align the map to the table.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-27 10:09:52 -04:00
720ea52877 activate the ffsbglr1 2026-07-22 15:16:07 -04:00
dd9f89f4b5 feat: version-control pipeline definitions via export/apply
Add `pipekit export` / `pipekit apply` (pipekit/config_io.py) to serialise the
DB's config — drivers, connections, modules (+ columns, watermarks, hooks),
groups, schedules — to a git-trackable text tree under config/, and rehydrate
it. SQLite stays runtime state; definitions become diffable/reviewable/
revertible.

- config only: run_log/group_run/settings and per-run state columns excluded
- name-keyed refs (portable across databases); source_query in .sql sidecars;
  columns as real JSON arrays for line-by-line diffs
- newline-normalised so CRLF-vs-LF is never a spurious change
- apply is create/update by name; child collections fully synced; top-level
  deletes gated behind --prune; --dry-run prints the plan
- round-trip is identity (export -> apply --dry-run == nothing to do)

Commits the current config/ as the first baseline, capturing the freshly
populated columns_json for rm00101/rm00301/iv00101. Documented in SPEC.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-22 10:56:55 -04:00