tps/readme.md

24 lines
665 B
Markdown
Raw Normal View History

2017-07-07 14:11:25 -04:00
Concepts
======================================
2017-07-07 13:54:13 -04:00
2017-07-07 14:11:25 -04:00
pull various static files into postgres and do basic transformation without losing the original document
or getting into custom code for each scenario
2017-07-07 14:12:05 -04:00
## Storage
2017-07-07 14:11:25 -04:00
all records are jsonb
applied mappings are in associated jsonb documents
2017-07-07 14:12:05 -04:00
## Import
2017-07-07 14:11:25 -04:00
`COPY` function utilized
2017-07-07 14:12:05 -04:00
## Mappings
2017-07-07 14:11:25 -04:00
1. regular expressions are used to extract pieces of the json objects
2. the results of the regular expressions are bumped up against a list of basic mappings and written to an associated jsonb document
2017-07-07 14:12:05 -04:00
## Transformation tools
2017-07-07 14:11:25 -04:00
* `COPY`
* `regexp_matches()`
2017-07-07 14:12:05 -04:00
## Difficulties
2017-07-07 14:11:25 -04:00
Non standard file formats will require additional logic