Commit Graph

377 Commits

Author SHA1 Message Date
Paul Trowbridge
4a85732890 edit function to recognize slice of array, ad bank schema to hold types there, add function to json regex definition to accomodate differntiating extracts from replaces 2017-10-25 12:07:04 -04:00
Paul Trowbridge
8929b4eea9 try using correlated subquery for unique list of keys, is pretty slow 2017-10-25 12:05:28 -04:00
Paul Trowbridge
ba4a404980 added partion by major item, and then sort on import id 2017-10-25 10:45:24 -04:00
Paul Trowbridge
44373bc616 copy source template from pt branch 2017-10-25 10:44:57 -04:00
Paul Trowbridge
65ee6d703b modify to print log info 2017-10-25 10:44:32 -04:00
Paul Trowbridge
6056491ace merge srce changes from pt branch 2017-10-25 09:07:51 -04:00
Paul Trowbridge
a630f46833 add sqitch 2017-10-25 01:35:21 -04:00
Paul Trowbridge
8c9cc81be4 add source, some trans, and trans_log records 2017-10-25 01:07:07 -04:00
Paul Trowbridge
bd39c3fd12 add source template 2017-10-25 01:05:45 -04:00
Paul Trowbridge
bfe602762a rename fle to srce_unq 2017-10-25 00:57:41 -04:00
Paul Trowbridge
ed1a903718 finish out logging the inserts and getting them into the main do block 2017-10-25 00:47:26 -04:00
Paul Trowbridge
e2d42467af catch up with wk branch 2017-10-25 00:42:22 -04:00
Paul Trowbridge
937cf38af1 updates in progress 2017-10-24 17:37:20 -04:00
Paul Trowbridge
57fa123adf add transaction log 2017-10-23 10:13:32 -04:00
Paul Trowbridge
51fbec1244 push inserts into do block so they can consume local variables, need to setup log inserts 2017-10-23 10:12:37 -04:00
Paul Trowbridge
b3775210b5 merge pt branch updates 2017-10-20 10:09:55 -04:00
Paul Trowbridge
c7f88271c5 add map_rv and apply some mapping to trans 2017-10-20 01:16:53 -04:00
Paul Trowbridge
1aa4cb938f build out map_rv 2017-10-20 01:05:15 -04:00
Paul Trowbridge
7abb8ce65d only assign a map if an item actually joins to map_rv 2017-10-20 01:04:56 -04:00
Paul Trowbridge
ea0084ed75 update maps for first 20, get rid of pretty and tag on summary of mappings 2017-10-19 23:57:03 -04:00
Paul Trowbridge
085f794693 merge with wk 2017-10-19 22:22:11 -04:00
Paul Trowbridge
41cf7af55b merge wk for srce 2017-10-19 22:13:28 -04:00
Paul Trowbridge
ccd685ca22 implement retain flag to drop keys used only for linking to the appropriate map value, but not acutally included in the map 2017-10-19 18:02:52 -04:00
Paul Trowbridge
9001c44b46 regex mapping script 2017-10-19 17:39:52 -04:00
Paul Trowbridge
75d511f6a4 regex modification template 2017-10-19 17:39:29 -04:00
Paul Trowbridge
2979fd3437 add jsonb object aggregate functions, change regex to jsonb, remove seq from primary key of map_rm, add foreign key to map_rv pointing map_rm 2017-10-19 17:38:58 -04:00
Paul Trowbridge
351b83a206 alter regex defs to have a retention flag indicating if the return value shoudl be mapped or thrown away 2017-10-19 17:37:46 -04:00
Paul Trowbridge
5369a39f34 add mapping tables & data 2017-10-19 14:13:57 -04:00
Paul Trowbridge
f203c7010a add calc columns to collateral 2017-10-19 13:10:36 -04:00
Paul Trowbridge
e68b3e0349 revise import to work with a set of unique keys in determining which rows to insert. prior used many of the same key, creating large duplication on join to tps 2017-10-19 13:10:25 -04:00
Paul Trowbridge
2a2652e397 qualify the order by id 2017-10-19 11:31:54 -04:00
Paul Trowbridge
f0838b28c0 retain the id number in the json rec, and order by id asc before inserting to trans. if mult records per key, this helps sort 2017-10-19 11:22:03 -04:00
Paul Trowbridge
2190f55acb add running collateral & loan balances 2017-10-19 11:20:52 -04:00
Paul Trowbridge
774540a83e remove assn & coll 2017-10-19 10:30:40 -04:00
Paul Trowbridge
e03a5cdbac backup 2017-10-19 10:29:33 -04:00
Paul Trowbridge
e0f8bad70a add ADPRP 2017-10-19 09:53:52 -04:00
Paul Trowbridge
1ae5298557 add index on tps.rec 2017-10-19 09:25:42 -04:00
Paul Trowbridge
dcf2fcbb6c update srce definitions 2017-10-19 09:25:31 -04:00
Paul Trowbridge
bc3cecec09 add order by clause 2017-10-19 09:25:07 -04:00
Paul Trowbridge
995eb181c1 dont export tps.trans table 2017-10-19 09:24:55 -04:00
Paul Trowbridge
e0163382f9 added sources 2017-10-19 08:49:33 -04:00
Paul Trowbridge
c5a9dfe0db add order by clause on transactions range, add hungtington transactions, add data type to match sources 2017-10-19 00:42:30 -04:00
Paul Trowbridge
5fabbd7fb3 add odbc query 2017-10-19 00:41:24 -04:00
Paul Trowbridge
dabd38b98b add huntington transactions 2017-10-19 00:13:28 -04:00
Paul Trowbridge
419017f98c switch to PNCC 2017-10-18 14:11:57 -04:00
Paul Trowbridge
748d6ddb3c merge summary of records not inserted from pt branch 2017-10-18 14:10:07 -04:00
Paul Trowbridge
a18774473b move exception union into CTE and summarize at end for records not inserted 2017-10-18 02:07:33 -04:00
Paul Trowbridge
4f958485d5 merge pt 2017-10-18 01:57:24 -04:00
Paul Trowbridge
9dbe11619f Merge branch 'pt' 2017-10-18 01:51:10 -04:00
Paul Trowbridge
144fd62bd8 fix conflicts 2017-10-18 01:49:09 -04:00