From 890f10cbabc4bd7876b967eb391a9f391101e281 Mon Sep 17 00:00:00 2001 From: Paul Trowbridge Date: Tue, 11 Aug 2026 11:02:57 -0400 Subject: [PATCH] Catch order status changes via an open-order hot set Hold/release rewrites OCRH.DCSTAT and DCHREA without touching any date column, so no watermark could see it. A full-table row diff of LGDAT.OCRH vs cms.ocrh (468,823 rows both sides) found 168 drifted rows, 0 missing, 0 extra -- and 159 of them changed with no date moving at all (checked dcudat/dcodat/dccdat/dcmdat/dchdat/dcxdat/dcsdat/dcmxdt). 155 of the drifts were DCSTAT, 4 DCHREA. Every drifted row was still open (DCSTAT in A/N/H/B); none had reached 'C'. Only ~2,044 of 468,823 orders are open, so re-pulling the whole open set each run is cheap and catches 100% of the observed drift. Applied to ocrh and to the two modules that read its changed-set, ocri and ocrs. Result: drift 168 -> 11 rows, DCSTAT 155 -> 3. No invoice-driven branch: tested against LGDAT.OCRHT (CMS's header change log, keyed DC3TMSP-leading via logical OCRHTX1) over the live watermark window -- 1,107 headers changed, 65 missed by the date branches, 0 missed by dates+open, and a 5- or 30-day invoiced-orders branch added exactly 0. Closing an order writes dcudat/dccdat, so the date window covers the flip. Also evaluated LGDAT.OCRIT (the order-line change log) as a replacement for the open-order branch and rejected it: at the live 8-day watermark it reports 0 changed orders not already caught, at ~12x the predicate cost, and its lower row count means it declines to pull 18k lines the open set would -- safe only if CMS's logging is complete, which is unverified. Co-Authored-By: Claude Opus 5 --- config/modules/ocrh.sql | 9 +++++++++ config/modules/ocri.sql | 8 ++++++++ config/modules/ocrs.sql | 5 +++++ 3 files changed, 22 insertions(+) diff --git a/config/modules/ocrh.sql b/config/modules/ocrh.sql index 3f91101..f115770 100644 --- a/config/modules/ocrh.sql +++ b/config/modules/ocrh.sql @@ -178,4 +178,13 @@ WHERE (dcudat BETWEEN '{ocrh_wm}' AND CURRENT_DATE) OR (dcodat BETWEEN '{ocrh_wm}' AND CURRENT_DATE) OR (dccdat BETWEEN '{ocrh_wm}' AND CURRENT_DATE) + -- Hold / release and other status flips rewrite DCSTAT and DCHREA and + -- touch no date column at all, so no watermark can see them. Open orders + -- are ~2k rows out of 469k -- cheap to re-pull in full on every run. + OR RTRIM(DCSTAT) <> 'C' + -- No invoice-driven branch here on purpose: measured 2026-08-11 against + -- LGDAT.OCRHT (CMS's header change log) over the live watermark window, + -- the two predicates above already catch 100% of changed headers, and a + -- 5- or 30-day invoiced-orders branch added exactly 0. Closing an order + -- writes dcudat/dccdat, so the date window covers the flip to 'C'. diff --git a/config/modules/ocri.sql b/config/modules/ocri.sql index 2beb13b..114adb7 100644 --- a/config/modules/ocri.sql +++ b/config/modules/ocri.sql @@ -4,6 +4,14 @@ WITH changed AS ( SELECT dcord# FROM lgdat.ocrh WHERE dcodat BETWEEN '{ocri_wm}' AND CURRENT_DATE UNION SELECT dcord# FROM lgdat.ocrh WHERE dccdat BETWEEN '{ocri_wm}' AND CURRENT_DATE + UNION + -- Hold / release and other status flips rewrite DCSTAT and DCHREA and + -- move no date column, so no watermark can see them. Only ~2k orders are + -- open, so re-pulling the whole open set every run is cheap. + -- Checked 2026-08-11 against LGDAT.OCRIT (CMS's own order-line change + -- log): OCRIT reports 0 changed orders that are not already caught here, + -- at ~12x the predicate cost -- this branch is a proven superset. + SELECT dcord# FROM lgdat.ocrh WHERE RTRIM(DCSTAT) <> 'C' ) SELECT "DDORD#" AS "ddord#", diff --git a/config/modules/ocrs.sql b/config/modules/ocrs.sql index 14077d5..9e9c016 100644 --- a/config/modules/ocrs.sql +++ b/config/modules/ocrs.sql @@ -2,6 +2,11 @@ WITH changed AS ( SELECT dcord# FROM lgdat.ocrh WHERE dcudat BETWEEN '{ocrs_wm}' AND CURRENT_DATE UNION SELECT dcord# FROM lgdat.ocrh WHERE dcodat BETWEEN '{ocrs_wm}' AND CURRENT_DATE UNION SELECT dcord# FROM lgdat.ocrh WHERE dccdat BETWEEN '{ocrs_wm}' AND CURRENT_DATE + UNION + -- status flips (hold/release) move no date column; ~2k open orders + SELECT dcord# FROM lgdat.ocrh WHERE RTRIM(DCSTAT) <> 'C' + -- No invoice-driven branch on purpose -- see ocrh. Measured against + -- LGDAT.OCRHT: the two predicates above catch 100% of changed headers. ) SELECT "FFORD#" AS "fford#",