2018-03-05 23:34:53 -05:00
|
|
|
Generic Data Transformation Tool
|
2018-02-02 17:34:10 -05:00
|
|
|
----------------------------------------------
|
|
|
|
|
|
|
|
|
|
The goal is to:
|
|
|
|
|
1. house external data and prevent duplication on insert
|
2018-03-05 23:34:53 -05:00
|
|
|
2. facilitate regular exression operations to extract meaningful data
|
|
|
|
|
3. be able to reference it from outside sources (no action required) and maintain reference to original data
|
2017-10-26 17:34:45 -04:00
|
|
|
|
2017-07-24 23:13:34 -04:00
|
|
|
|
2018-03-05 23:34:53 -05:00
|
|
|
It is well suited for data from outside systems that
|
|
|
|
|
* requires complex transformation (parsing and mapping)
|
|
|
|
|
* original data is retained for reference
|
2017-07-24 23:13:34 -04:00
|
|
|
|
2018-03-05 23:34:53 -05:00
|
|
|
use cases:
|
|
|
|
|
* on-going bank feeds
|
|
|
|
|
* jumbled product lists
|
|
|
|
|
* storing api results
|
2017-07-24 23:13:34 -04:00
|
|
|
|
2017-10-27 13:06:44 -04:00
|
|
|
|
2018-03-05 23:34:53 -05:00
|
|
|
The data is converted to json by the importing program and inserted to the database.
|
|
|
|
|
Regex expressions are applied to specified json components and the results can be mapped to other values.
|
2017-10-26 17:34:45 -04:00
|
|
|
|
2017-07-24 23:13:34 -04:00
|
|
|
|
2018-03-05 23:34:53 -05:00
|
|
|
Major Interactions
|
|
|
|
|
------------------------
|
2017-07-24 23:13:34 -04:00
|
|
|
|
2018-03-05 23:34:53 -05:00
|
|
|
* Source Definitions (Maint/Inquire)
|
|
|
|
|
* Regex Instructions (Maint/Inquire)
|
|
|
|
|
* Cross Reference List (Maint/Inquire)
|
|
|
|
|
* Run Import (Run Job)
|
