Commit Graph

216 Commits

Author SHA1 Message Date
Paul Trowbridge
caf3ffe3b4 build template for applying gl on the server side per item with offset 2017-11-01 01:12:33 -04:00
Paul Trowbridge
0b4a638430 thought on evt.log 2017-10-30 17:38:50 -04:00
Paul Trowbridge
242ebb8029 add a conversion to pay_month 2017-10-30 14:43:04 -04:00
Paul Trowbridge
e9b185fc05 seperate out maps for ach credits vs debits 2017-10-30 10:37:03 -04:00
Paul Trowbridge
9a8144a2e2 Merge branch 'pt' of https://github.com/fleetside72/tps_etl into pt 2017-10-29 13:22:36 -04:00
Paul Trowbridge
f056c9a3f9 update spreadsheets 2017-10-29 13:22:25 -04:00
pt motog5
0ee5e0ce5f add thoughts on log gl modules, a module equates to a json schema 2017-10-28 02:44:55 -04:00
Paul Trowbridge
afa16ce925 add items to log and create summary pivot 2017-10-28 02:35:24 -04:00
Paul Trowbridge
2d6212e3cd add index to srce and couple import jobs 2017-10-27 16:26:00 -04:00
Paul Trowbridge
273b7c9673 limit mapper to allj records are null 2017-10-27 15:50:42 -04:00
Paul Trowbridge
fac52588ae update map to exclude 6 digits at end of company name 2017-10-27 14:18:57 -04:00
Paul Trowbridge
f88584f835 convert map flag to be at the regex item level 2017-10-27 13:51:08 -04:00
Paul Trowbridge
417bdca27d move all current map_rm into update scripts for editing 2017-10-27 13:23:08 -04:00
Paul Trowbridge
b94948b1b9 add new mappings and clean up old ones 2017-10-27 13:12:45 -04:00
Paul Trowbridge
2829887a57 get rid of bad maps 2017-10-27 13:08:03 -04:00
Paul Trowbridge
30da16dd0a add map definition query 2017-10-27 13:07:52 -04:00
Paul Trowbridge
4a62ca84b1 implement sequence of target, and insure that the link to the map does not return duplicates 2017-10-27 13:06:44 -04:00
Paul Trowbridge
69d5698ff7 add transactions 2017-10-27 01:45:38 -04:00
Paul Trowbridge
9293fbfd1f add transactions 2017-10-27 01:44:55 -04:00
Paul Trowbridge
e0c5cf47a5 build interpretation of evt.log for running totals 2017-10-27 01:44:41 -04:00
Paul Trowbridge
a5b48325a2 add jounral entries 2017-10-27 01:04:04 -04:00
Paul Trowbridge
7ad5b8dd6f add column to event_log. demonstact GL extraction and load initial row 2017-10-26 23:38:36 -04:00
Paul Trowbridge
2838ff4cc5 update maps 2017-10-26 22:51:46 -04:00
Paul Trowbridge
a3b2928619 Merge branch 'pt' 2017-10-26 20:01:45 -04:00
Paul Trowbridge
6a97c5ca69 update schema 2017-10-26 20:01:26 -04:00
Paul Trowbridge
3486281c7c revert back to pt branch rows of data 2017-10-26 19:03:48 -04:00
Paul Trowbridge
11aa70684d merge wk 2017-10-26 18:46:42 -04:00
Paul Trowbridge
6b08ea8fb8 add new maps, need to convert OBI & RFB maps to a single target 2017-10-26 17:36:03 -04:00
Paul Trowbridge
3924776161 update readme 2017-10-26 17:34:45 -04:00
Paul Trowbridge
9623853387 add map to parse all ACH transactions 2017-10-26 17:34:37 -04:00
Paul Trowbridge
7eba5981ef trim retained values 2017-10-26 17:34:21 -04:00
Paul Trowbridge
6a90cc4b32 add maps 2017-10-26 14:31:37 -04:00
Paul Trowbridge
2448ccf641 add new map, add column for all json, and apply index 2017-10-26 13:39:50 -04:00
Paul Trowbridge
bf8e901ffd add intermediate selects, add coalesce to target aggregation since some items are SQL null, then apply a strip_nulls, but exclude from subsequent step. add update line at end 2017-10-26 13:32:14 -04:00
Paul Trowbridge
fb93f73b6c create another mapping approach to better handle option g on regexp_ functions. 2017-10-26 12:15:58 -04:00
Paul Trowbridge
21c3f74de0 couple changes 2017-10-26 00:26:20 -04:00
Paul Trowbridge
6b8d14ddb7 add running discover card balance 2017-10-26 00:16:46 -04:00
Paul Trowbridge
1af3f05a1e overhaul of do_map, add seperate column for parse versus mapped, accomodate rexexp_replace 2017-10-25 16:28:42 -04:00
Paul Trowbridge
a01c5c1a04 add mapping template 2017-10-25 15:21:13 -04:00
Paul Trowbridge
4a85732890 edit function to recognize slice of array, ad bank schema to hold types there, add function to json regex definition to accomodate differntiating extracts from replaces 2017-10-25 12:07:04 -04:00
Paul Trowbridge
8929b4eea9 try using correlated subquery for unique list of keys, is pretty slow 2017-10-25 12:05:28 -04:00
Paul Trowbridge
ba4a404980 added partion by major item, and then sort on import id 2017-10-25 10:45:24 -04:00
Paul Trowbridge
44373bc616 copy source template from pt branch 2017-10-25 10:44:57 -04:00
Paul Trowbridge
65ee6d703b modify to print log info 2017-10-25 10:44:32 -04:00
Paul Trowbridge
6056491ace merge srce changes from pt branch 2017-10-25 09:07:51 -04:00
Paul Trowbridge
a630f46833 add sqitch 2017-10-25 01:35:21 -04:00
Paul Trowbridge
8c9cc81be4 add source, some trans, and trans_log records 2017-10-25 01:07:07 -04:00
Paul Trowbridge
bd39c3fd12 add source template 2017-10-25 01:05:45 -04:00
Paul Trowbridge
bfe602762a rename fle to srce_unq 2017-10-25 00:57:41 -04:00
Paul Trowbridge
ed1a903718 finish out logging the inserts and getting them into the main do block 2017-10-25 00:47:26 -04:00