pt/tps
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
236 Commits 6 Branches 0 Tags 2 MiB
PLpgSQL 67.8%
JavaScript 12.9%
CSV 10.4%
HTML 8.7%
Shell 0.2%
ff18ce5bd3
Go to file
Paul Trowbridge ff18ce5bd3 simplify readme
2018-03-05 23:34:53 -05:00
deploy update import function to trap all error at the end 2018-03-05 22:12:31 -05:00
functions update import function to trap all error at the end 2018-03-05 22:12:31 -05:00
reports list out columns 2018-03-03 11:52:34 -05:00
sample_discovercard create function to insert many updates from one json 2018-03-03 11:47:31 -05:00
sample_google_api add scenario to test storing json, exmaple google api 2018-03-02 01:01:46 -05:00
templates put stuff in folders, delete a couple more 2018-02-05 23:18:27 -05:00
triggers initial trigger work 2018-03-04 23:57:59 -05:00
.gitignore add to ignore 2018-02-27 22:35:27 -05:00
LICENSE add event record example 2017-07-24 23:13:34 -04:00
readme.md simplify readme 2018-03-05 23:34:53 -05:00

readme.md

Generic Data Transformation Tool

The goal is to:

  1. house external data and prevent duplication on insert
  2. facilitate regular exression operations to extract meaningful data
  3. be able to reference it from outside sources (no action required) and maintain reference to original data

It is well suited for data from outside systems that

  • requires complex transformation (parsing and mapping)
  • original data is retained for reference

use cases:

  • on-going bank feeds
  • jumbled product lists
  • storing api results

The data is converted to json by the importing program and inserted to the database. Regex expressions are applied to specified json components and the results can be mapped to other values.

Major Interactions

  • Source Definitions (Maint/Inquire)
  • Regex Instructions (Maint/Inquire)
  • Cross Reference List (Maint/Inquire)
  • Run Import (Run Job)