Commit Graph

97 Commits

Author SHA1 Message Date
Paul Trowbridge
9001c44b46 regex mapping script 2017-10-19 17:39:52 -04:00
Paul Trowbridge
75d511f6a4 regex modification template 2017-10-19 17:39:29 -04:00
Paul Trowbridge
2979fd3437 add jsonb object aggregate functions, change regex to jsonb, remove seq from primary key of map_rm, add foreign key to map_rv pointing map_rm 2017-10-19 17:38:58 -04:00
Paul Trowbridge
351b83a206 alter regex defs to have a retention flag indicating if the return value shoudl be mapped or thrown away 2017-10-19 17:37:46 -04:00
Paul Trowbridge
5369a39f34 add mapping tables & data 2017-10-19 14:13:57 -04:00
Paul Trowbridge
f203c7010a add calc columns to collateral 2017-10-19 13:10:36 -04:00
Paul Trowbridge
e68b3e0349 revise import to work with a set of unique keys in determining which rows to insert. prior used many of the same key, creating large duplication on join to tps 2017-10-19 13:10:25 -04:00
Paul Trowbridge
2a2652e397 qualify the order by id 2017-10-19 11:31:54 -04:00
Paul Trowbridge
f0838b28c0 retain the id number in the json rec, and order by id asc before inserting to trans. if mult records per key, this helps sort 2017-10-19 11:22:03 -04:00
Paul Trowbridge
2190f55acb add running collateral & loan balances 2017-10-19 11:20:52 -04:00
Paul Trowbridge
774540a83e remove assn & coll 2017-10-19 10:30:40 -04:00
Paul Trowbridge
e03a5cdbac backup 2017-10-19 10:29:33 -04:00
Paul Trowbridge
e0f8bad70a add ADPRP 2017-10-19 09:53:52 -04:00
Paul Trowbridge
1ae5298557 add index on tps.rec 2017-10-19 09:25:42 -04:00
Paul Trowbridge
dcf2fcbb6c update srce definitions 2017-10-19 09:25:31 -04:00
Paul Trowbridge
bc3cecec09 add order by clause 2017-10-19 09:25:07 -04:00
Paul Trowbridge
995eb181c1 dont export tps.trans table 2017-10-19 09:24:55 -04:00
Paul Trowbridge
e0163382f9 added sources 2017-10-19 08:49:33 -04:00
Paul Trowbridge
419017f98c switch to PNCC 2017-10-18 14:11:57 -04:00
Paul Trowbridge
748d6ddb3c merge summary of records not inserted from pt branch 2017-10-18 14:10:07 -04:00
Paul Trowbridge
4f958485d5 merge pt 2017-10-18 01:57:24 -04:00
Paul Trowbridge
144fd62bd8 fix conflicts 2017-10-18 01:49:09 -04:00
Paul Trowbridge
160002404c fix conflicts 2017-10-18 01:48:22 -04:00
Paul Trowbridge
c582d5f176 merge from c9 2017-10-18 01:45:44 -04:00
Paul Trowbridge
e219827d93 more discover transactions 2017-10-18 01:41:19 -04:00
Paul Trowbridge
4766f157fc get rid of id key in json, remove constraint 2017-10-18 01:40:09 -04:00
Paul Trowbridge
fe761c6975 include pt branch srce.pgsql changes & transaction range query 2017-10-18 01:06:00 -04:00
Paul Trowbridge
3afe033fa7 add transaction query and new data 2017-10-18 01:01:54 -04:00
Paul Trowbridge
63a3bd3b97 snap to pt branch on srce.pgsql 2017-10-18 01:00:10 -04:00
Paul Trowbridge
a970e87b95 add discover transactions 2017-10-18 00:57:40 -04:00
Paul Trowbridge
006a1b0103 and query to show range of key values in transation table 2017-10-18 00:55:15 -04:00
Paul Trowbridge
c123cbfa84 add insert as part of with a return rows not inserted, probably needs modified to show the keys not inserted and sorted 2017-10-18 00:55:00 -04:00
whats_that_flashing
fe3a9c4bed switch rebuild to ubuntu syntax 2017-10-16 17:12:41 +00:00
whats_that_flashing
92e67ad3c5 Merge branch 'pt' into c9
forward c9 to reflect pt branch
2017-10-16 16:58:03 +00:00
Paul Trowbridge
ac4d9f68d5 add link to tps trans to find any potential import conflicts 2017-10-16 00:14:05 -04:00
Paul Trowbridge
668ca9f44a update data from dcard to reflect to 2 field unique constrain, aggregate several unique constraonts back to 1 row per id 2017-10-15 23:31:51 -04:00
Paul Trowbridge
8bbf575f93 get rid of insert 2017-10-15 13:05:20 -04:00
Paul Trowbridge
f6ab0e42f5 change COPY command to dynamic due to column list 2017-10-14 01:48:13 -04:00
Paul Trowbridge
a0b3173741 add another variable to collect column list, add a serial column to temp, copy using column list, and then build json object based on json defn 2017-10-14 01:40:26 -04:00
Paul Trowbridge
c81a7c7dc4 extract key at import 2017-10-13 17:17:56 -04:00
Paul Trowbridge
dc9e160c3e set sequence to 1 2017-10-13 12:00:34 -04:00
Paul Trowbridge
5e8768e4a6 get rid of test insert data 2017-10-13 11:57:25 -04:00
Paul Trowbridge
811fbb90f2 add rebuild script 2017-10-13 11:41:43 -04:00
Paul Trowbridge
9040c3249e Merge branch 'wk' of https://github.com/fleetside72/tps_etl into wk 2017-10-13 09:19:20 -04:00
Paul Trowbridge
b3c0b3355c add credit cards 2017-10-13 09:18:48 -04:00
Paul Trowbridge
94d9a501de add batch file to re-create database 2017-10-13 03:23:41 -04:00
Paul Trowbridge
cc567e86b9 merge wk 2017-10-13 03:06:51 -04:00
Paul Trowbridge
a41ea3cfa8 Merge branch 'pt' 2017-10-13 02:59:45 -04:00
Paul Trowbridge
e560e49686 get rid of commented out sample json 2017-10-13 02:55:28 -04:00
Paul Trowbridge
78ebfd5e8f change out to sql extension 2017-10-13 02:53:21 -04:00