From 01f87ce7a6dd7d8b9fbda283ad80fe683db5fcbb Mon Sep 17 00:00:00 2001 From: Paul Trowbridge Date: Fri, 7 Jul 2017 14:11:25 -0400 Subject: [PATCH] initialize readme --- readme.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/readme.md b/readme.md index 8b13789..3a6d2e5 100644 --- a/readme.md +++ b/readme.md @@ -1 +1,24 @@ +Concepts +====================================== +pull various static files into postgres and do basic transformation without losing the original document +or getting into custom code for each scenario + +##Storage +all records are jsonb +applied mappings are in associated jsonb documents + +##Import +`COPY` function utilized + +##Mappings +1. regular expressions are used to extract pieces of the json objects +2. the results of the regular expressions are bumped up against a list of basic mappings and written to an associated jsonb document + +##Transformation tools +* `COPY` +* `regexp_matches()` + +##Difficulties +Non standard file formats will require additional logic +This