pt/tps
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

add throughts on new source definition

Browse Source
This commit is contained in:
Paul Trowbridge 2018-05-20 02:51:25 -04:00
parent ba6a06e30a
commit d3114d5759
1 changed files with 32 additions and 10 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

42
readme.md
View File

@ -66,16 +66,38 @@ source definition
---------------------------------------------------------------------- ----------------------------------------------------------------------
* **load data** * **load data**
* parsing function reference * the brwosers role is to extract the contents of a file and send them as a post body to the backend for processing under target function `based on srce defintion`
* csv_from_client --> load file dialog, send content as post body to backend, backend sends array of json as argument to database? * the backend builds a json array of all the rows to be added and sends as an argument to a database insert function
* custom_pnc_parse --> load file dialog, send content as post body to backend, parse out data to array of json and send to db as large json object? * build constraint key `based on srce definition`
* _note_ : the database will have to store in the json a reference to these functions which the browser will need to read in order to act on an import request * handle violations
* increment global key list (this may not be possible depending on if a json with variable length arrays can be traversed)
* the brwosers role is to extract the contents of a file and send them as a post body to the backend for processing under target function (/parse_csv&q=source) * build an import log
* run maps (as opposed to relying on trigger)
* constraints
* **read data** * **read data**
* top level key to table as type? * top level key to table as type?
* function that returns a string that is a list of columns? * view that `uses the source definiton` to extrapolate a table?
* custom function to read and return the table? * a materialized table is built `based on the source definition` and any addtional regex?
* add regex = alter table add column with historic updates?
* no primary key?
* every document must work out to one row
{
"name":"sample_name",
"source":"client_file",
"loading_function":"csv"
"constrain":[
"{date}",
"{level1,item3,field}"
],
"table_schema":[
{
"top_level_key":"column"name",
"type":"numeric"
},
{
"top_level_key":"column"name",
"type":"numeric"
}
]
}