Commit Graph

315 Commits

Author SHA1 Message Date
Maxime Beauchemin e53f3032bb [dashboard] adding an option to duplicate slices when "Saving AS" (#3391)
* [dashboard] adding an option to duplicate slices when "Saving AS"

* Fix tests
2017-08-30 14:09:29 -07:00
timifasubaa 8d877e8a35 [explore] Fix and test slice id logging issue (#3339)
* [explore] fixed padding bug on filter section

* fix slice_id logging issue

* [superset-sqllab] fix slice_id population in appropriate column

* [explore-logging] test the slice_id logging fix

* fix travis errors

* fix nits pointed out in PR comments

* cleanup tests

* made python more beautiful

* made python even more beautiful

* made python even more more beautiful

* made python even more more more beautiful

* fix lint error

* make exception handling more specific

* fixed silly error

* fixed argument indentation
2017-08-24 09:11:41 -07:00
Rogan a5320a0f37 [add] Save filters to dashboard (#3183)
* [add] Save filters to dashboard

* format code

* fix CI error

* add semicolon  semi

* fix none object

* add test data
optimize the js code
fix the compatibility issue

* fix urllib to urllib.parse

* add space

* update test case

* remove  'return'

* fix error

* update test case
2017-08-10 20:49:49 -07:00
Brian Wolfe 1e325d9645 [druid] Allow custom druid postaggregators (#3146)
* [druid] Allow custom druid postaggregators

Also, fix the postaggregation for approxHistogram quantiles so it adds
the dependent field and that can show up in the graphs/tables.

In general, postAggregators add significant power, we should probably
support including custom postAggregators. Plywood has standard
postAggregators here, and a customAggregator escape hatch that allows
you to define custom postAggregators.

This commit adds a similar capability for Superset and a additional
field/fields/fieldName breakdown of the typical naming for dependent
aggregations, which should make it significantly easier to develop
approxHistogram and custom postAggregation-required dashboards.

* [druid] Minor style cleanup in tests file.

* [druid] Apply code review suggestions

* break out CustomPostAggregator into separate class. This just cleans
  up the creation of the postaggregator a little bit.
* minor style issues.
* move the function around so the git diff is more readable
2017-07-28 11:45:59 -07:00
Maxime Beauchemin b888802e05 [sqllab] improve Hive support (#3187)
* [sqllab] improve Hive support

* Fix "Transport not open" bug
* Getting progress bar to show
* Bump pyhive to 0.4.0
* Getting [Track Job] button to show

* Fix testzz
2017-07-27 14:00:19 -07:00
Ke Zhu e834154030 Fixes #3134 by correct response content-type of /testconn (#3135) 2017-07-15 09:52:37 -07:00
Maxime Beauchemin 06fcaa3095 Stabilizing Celery / SQL Lab (#2981)
* upgrade celery to 4.0.2
* using Redis for unit tests (sqla broker not supported in Celery 4)
* Setting Celery's soft_time_limit based on `SQLLAB_ASYNC_TIME_LIMIT_SEC` config
* Better error handling in async tasks
* Better statsd logging in async tasks
* show [pending/running] query status in Results tab
* systematically using sqla NullPool on worker (async) to limit number
  of database connections
2017-06-20 13:55:03 -07:00
Grace Guo 8329ea2b9b Edit Dashboard title and Slice title in place (#2940)
* Edit Dashboard title and Slice title in place

Add EditableTitle component into Dashboard and Explore view to support edit title inline.
2017-06-14 12:52:12 -07:00
Alanna Scott e1751c065c add new slice test (#2939)
* sort explicitly on label

* add simple test for /slicemodelview/add endpoint

* make comments and method names more clear

* fix test name

* be more explicit, test status_code
2017-06-13 09:44:26 -07:00
timfeirg 17bd7512ff remove reference for CSRF_ENABLED, and use WTF_CSRF_ENABLED instead (#2946) 2017-06-12 13:17:59 -07:00
ShengyaoQian b0e2904c24 Updating permission when refreshing druid datasource (#2655)
* Updating permission when refreshing druid datasource

* Adding test

* Fix style

* Deletion view_menu after db, table, cluster, ds deletion

* Update table model

* Linting

* Override _delete instead of post_delete

* fix

* lint

* fix multi delete

* fix

* Refactoring

* Amending
2017-05-22 16:41:32 -07:00
YOU 46d7a925bb chore: remove unused methods with invalid models.Query usage (#2721)
* fix: models.Query is not defined

* chore: remove unused Query and methods
2017-05-06 21:23:17 -07:00
Maxime Beauchemin cb14640a82 Removing uneeded results_backends.py (#2717) 2017-05-04 23:41:10 -07:00
robert-digit 1df37e6e4d add option for pulling favourited dashboards by username (#2661)
* add option for pulling favourited dashboards by username

add tests

* fix too many lines pep8 error
2017-04-24 20:48:25 -07:00
Maxime Beauchemin 91fe02cdc8 Setting adjust_database_uri for HiveEngineSpec (#2636)
* Setting adjust_database_uri for HiveEngineSpec

* fix tests
2017-04-18 12:29:13 -07:00
Maxime Beauchemin 31283f1424 Fix metric formating in Dashboard view + some refactoring (#2598)
* Fix metric formating in Dashboard view + some refactoring

* Fixing build
2017-04-12 09:37:49 -07:00
Maxime Beauchemin ac84fc2b65 Fixing confusion when selecting schema across engines (#2572) 2017-04-10 15:36:58 -07:00
rumbin c581ea8661 Alternative PR for: Some bytes/str issues in py3 w/ zlib and json (#2558)
* sql_lab.py: compress via utils

* utils.py: added zlib_compress and zlib_compress_to_string

* core.py: converted to use zlib_decompress_to_string; renamed uncompress to decompress in utils.py

* utils_tests.py: added test for compress/decompress

* fixed broken utils test; removed redundant code and empty lines from utils.py

* utils.py: corrected docstrings, removed unnecessary 'else'

* removed yet another superfluous else
2017-04-06 09:42:43 -07:00
Maxime Beauchemin 122891c29b [sql lab] allow users to save their queries (#2528)
* Allow users to save their queries

Fixing tests .

* Adding placeholder for Query Description

* initJQueryCSRF -> initJQueryAjaxCSRF
2017-04-04 20:15:19 -07:00
Maxime Beauchemin 1f8e48b374 [sqllab] assign types for visualize flow (#2458)
* [sqllab] assign types for visualize flow

Somehow when using the visualize flow, the types were not
assigned at all, creating some bugs downstream. This PR attempts to get
the information required based on what pandas is knows and the types in
the data itself.

* Fixing tests

* Fixing tests

* Fixing more tests

* Fixing the last py3 tests
2017-03-24 09:23:51 -07:00
Maxime Beauchemin 1590b8c7e5 Speeding up polling by not checking access (#2466)
* Speeding up polling by not checking access

* Fix tests
2017-03-23 22:46:25 -07:00
Maxime Beauchemin 5e43d074c3 [explore ] templating can now reference query elements (#2388)
Second take on
https://github.com/mistercrunch/superset/pull/3
2017-03-14 13:23:20 -07:00
Bogdan 0779da6d24 Keep column order in .csv (#2377) 2017-03-10 14:49:11 -08:00
Maxime Beauchemin 2969cc9993 Refactoring Druid & SQLa into a proper "Connector" interface (#2362)
* Formalizing the Connector interface

* Checkpoint

* Fixing views

* Fixing tests

* Adding migrtion

* Tests

* Final

* Addressing comments
2017-03-10 09:11:51 -08:00
Bogdan 9114d86ecd Add hive to superset + monkey patch the pyhive (#2134)
* Initial hive implementation

* Fix select star query for hive.

* Exclude generated code.

* Address code coverage and linting.

* Exclude generated code from coveralls.

* Fix lint errors

* Move TCLIService to it's own repo.

* Address comments

* Implement special postgres case,
2017-03-06 16:20:55 -08:00
Sungju Jin d4b59b36a8 Fixed a bug when querying with schema path to Redshift/Postgresql (#1789) 2017-02-26 22:54:48 -08:00
Maxime Beauchemin 0cc8eff1c3 [WiP] Deprecate Explore v1 (#2064)
* Simplifying the viz interface (#2005)

* Working on dashes

* Making this a collaborative branch

* Fixing some bugs

* Fixing bugs

* More improvements

* Add datasource back in bootstrap data

* Decent state

* Linting

* Moving forward

* Some more linting

* Fix the timer

* Triggering events through state

* Lingint

* Put filters in an array instead of flt strings (#2090)

* Put filters in an array instead of flt strings

* Remove query_filter(), put opChoices into Filter

* Update version_info.json

* Fix migrations

* More renderTrigger=true

* Fixing bugs

* Working on standalone

* getting standalone to work

* Fixed forcedHeight for standalone =view

* Linting

* Get save slice working in v2 (#2106)

* Filter bugfix

* Fixing empty series limit bug

* Fixed dashboard view

* Fixing short urls

* Only allow owners to overwrite slice (#2142)

* Raise exception when date range is wrong

* Only allow owner to overwrite a slice

* Fix tests for deprecate v1 (#2140)

* Fixed tests for control panels container and filters

* Fixed python tests for explorev2

* Fix linting errors

* Add in stop button during slice querying/rendering (#2121)

* Add in stop button during slice querying/rendering

* Abort ajax request on stop

* Adding missing legacy module

* Removing select2.sortable.js because of license

* Allow query to display while slice is loading (#2100)

* Allow query to display while slice is loading

* Put latestQueryFormData in store

* Reorganized query function, got rid of tu[le return values

* Merging migrations

* Wrapping up shortner migration

* Fixing tests

* Add folder creation to syncBackend

* Fixing edit URL in explore view

* Fix look of Stop button

* Adding syntax highlighting to query modal

* Fix cast_form_data and flase checkbox on dash

* Bugfix

* Going deeper

* Fix filtering

* Deleing invalid filters when changing datasource

* Minor adjustments

* Fixing calendar heatmap examples

* Moving edit datasource button to header's right side

* Fixing mapbox example

* Show stack trace when clicking alert

* Adding npm sync-backend command to build instruction

* Bumping up JS dependencies

* rm dep on select2

* Fix py3 urlparse

* rm superset-select2.js

* Improving migration scripts

* Bugfixes on staging

* Fixing Markup viz
2017-02-16 17:28:35 -08:00
Bogdan c564881867 Implement caching and dynamic data fetching. (#1466)
* Rename rv => o in the decorator.

* Address comments.

* Permissions cleanup: remove none and duplicates. (#1967)

* Updates

* Rename var and dropdown text

* Cleanup

* Resolve comments.

* Add user to the perm check.
2017-02-13 16:14:55 -08:00
Saleh Hindi daa1420c8e adding tests for #1131 (#1902)
* Add login test

* Add add-slice test

* Fix linting errors

* Use get_resp()helper method instead of self.client.get()

* Fix failing test.

* Add checks for URLs in the tablemodelview

* Fix linting errors

* Address pull request comments

* Address pull request comments

* Remove accidentally added files

* Fix failing test
2017-02-10 01:17:49 -08:00
Benedict Jin 1f58e18b6f Some code refactoring (#2139) 2017-02-08 11:52:58 -08:00
Benjamin Yolken 716406198e Clean up imports of cPickle and StringIO 2017-02-03 21:50:04 -08:00
Benjamin Yolken ce50e6e4fe Fix python3 cPickle import errors 2017-02-03 21:27:21 -08:00
Benjamin Yolken 167ed33bba Fix name of test in results_backends_tests module 2017-02-03 16:29:28 -08:00
Benjamin Yolken 00b6b0ac68 Misc. style tweaks to S3Cache changes and tests 2017-02-03 15:43:43 -08:00
Benjamin Yolken 1546b1ae71 Add tests for S3Cache 2017-02-03 15:40:18 -08:00
vera-liu 85806624db Use a key-value store model for sharing long queries (#1951)
* Add KeyValue model for storing id-value pairs
use it for storing shared queries

* Change string to text and added test

* Put getQueryLink in one place

* Changed migration down version

* Changes based on comments

* Update bcf3126872fc_add_keyvalue.py
2017-01-27 10:20:24 -08:00
Bogdan b1bba96d04 Fix csv download. (#2036) 2017-01-25 18:06:29 -08:00
vera-liu 27ed0b37bf Cleanup fulfilled requests after approve (#1953)
* Cleanup fulfilled requests after approve

* Modified tests

* Moved to separate test, add user to access functions

* Moved to separate test and added test cases

* Fixed issue with dryrun

* More changes based on comments
2017-01-24 18:11:51 -08:00
Bogdan 495f6460a4 Add email functionality (#1914)
* Add email functionality

* Add email templates.

* Test notifications

* Move email to utils
2017-01-13 19:30:17 -08:00
Maxime Beauchemin 2d866e3ffa [hotfix] fix the logging fix that broke the build (#1940)
* [hotfix] fix the logging fix that broke the build

* Fixing tests
2017-01-11 07:53:24 -08:00
Bogdan 2ab6a411f4 Druid dashboard import/export. (#1930)
* Druid dashboard import

* Fix tests.

* Parse params with trailing commas

* Resolved issue with datasource is not in DB yet when slice perms are set

* Finish rebase

* Fix heads alembic problem.
2017-01-09 10:02:03 -08:00
Maxime Beauchemin 49e6fd5bfb Revert "Druid dashboard import/export. " (#1923) 2017-01-07 14:02:13 -08:00
Bogdan af872fa4d4 Druid dashboard import/export. (#1811)
* Druid dashboard import

* Fix tests.

* Parse params with trailing commas

* Resolved issue with datasource is not in DB yet when slice perms are set

* Finish rebase
2017-01-06 16:48:21 -08:00
Maxime Beauchemin 222671675c [exploreV2] mapStateToProps for fields (#1882)
* Controls support for mapStateToProps

* Binding methods in the constructor

* Adressing comments

* Fixing tests
2017-01-06 12:38:44 -08:00
Maxime Beauchemin 9a62d94630 [sqllab] bugfix visualizing a query with a semi-colon (#1869)
* [sqllab] bugfix visualizing a query with a semi-colon

* Fixing tests
2017-01-06 12:24:07 -08:00
Maxime Beauchemin c14c7edc5e [explore] show the broken query when failing (#1871)
* Return query when failing

* Linting

* sjson -> simplejson
2017-01-05 10:00:39 -08:00
Riccardo Magliocchetti e3b296c558 utils: teach our json serializer to handle more types (#1907)
Namely datetime.time and numpy.bool_

Refs: #1900
Refs: #1903
2017-01-05 09:56:07 -08:00
willgroves ea8e6634d6 read anon user role from config, remove reference to public role (#1878)
* read anon user role from configuration, add anon user role as 'Public' to testing configuration

* apply suggestions
2016-12-27 14:30:01 -08:00
Dongkyu Hwangbo 3e6f90cf72 Upgrading pydruid version and adopt 'merge' flag during refresh_druid operation (#1879)
* Initial

* rewrite some line to make it short and setting merge variable temporarily

* rewrite commit author

* add emitted attribute

* Fix typo

* fix test error

* fix typo

* test added
2016-12-27 14:27:55 -08:00
vera-liu 6732f01cb7 Enable freeform-select with fetched column values for filter values (#1697)
* Enable freeform-select with fetched column values for filter values
 - db migration to add filter_select_enabled
 - add freeform-multi option for Selectfield
 - modify formatFilter() function on query to accomodate filter-select

* Fix js tests

* Fix codeclimate issue

* Changes based on comments

* Add test for filter endpoint

* Extract out renderFilterFormField function from render

* Fix landscape issues
2016-12-16 14:23:48 -08:00
Bogdan 92aa1a6124 Permissions refactoring, optimizations and unit testing. (#1798)
* Refactor and speed up superset init

* Add unit tests.

* Test fixes.

* More test updates.

* Fix read only perms

* Address comments.
2016-12-15 08:38:34 -05:00
vera-liu e1e20b8757 Sort searched queries by recency (#1735)
* Sort searched queries by recency

* Fixed sqllab tests

* Add one more tr to QueryTable spec for pagination

* Change desc to asc as we reverse queries in component
2016-12-14 10:14:54 -08:00
Maxime Beauchemin e099088012 [hotfix] fixing the build 2016-12-13 16:05:05 -08:00
Alanna Scott 34e107e7d3 [explore-v2] add config option for explore v2 beta users, and send through v2 path (#1671)
* add config option for explore v2 beta users, and send through v2 path

* fix long lines

* use role rather than user ids in config

* add test

* simplify role check

* remove extra  line

* use different test user for v2 tests
2016-12-12 16:42:38 -08:00
vera-liu 9f7486f402 remove extra call to get_viz in explorev2 (#1812)
* Not working errors

* Remove update_explore endpoint, only update explore endpoints in reducer on query

* Change scroll to auto and make container reponse to height:

* Remove update_explore endpoint

* Remove can_update_explore perm
2016-12-12 10:58:07 -08:00
Bogdan 3597fdb7f8 Filter table list based on the user permissions. (#1769)
* Filter table list based on the user permissions.

* Fix tests
2016-12-06 02:18:16 -05:00
Bogdan 69702e3a19 Create users if not found. (#1753)
* Create users if not found.

* Do not fail is user is a duplicate.

* Make endpoint faster.
2016-12-05 20:03:40 -05:00
Bogdan 9d4c3d83d0 Update role based on usernames not emails. (#1749) 2016-12-02 16:30:21 -05:00
Maxime Beauchemin 324205f77a [sqllab] bugfix where a query has the same alias twice as output (#1734) 2016-12-01 19:53:23 -08:00
Maxime Beauchemin 7eef46e941 Adding links pointing to the new user profile page (#1704)
* Adding links pointing to the new user profile page

* Raising coverage

* Fixing tests
2016-12-01 15:21:18 -08:00
Benjamin Yolken 50da4f8c07 Support running superset via pex (#1713)
* Support running superset via pex

* [superset] Update default port in superset/bin/superset

* Fix codeclimate line length issues

* Fix another line length issue, in config.py

* Add trivial utils test to increase test coverage

* Clean up runserver handling
2016-12-01 15:18:55 -08:00
Bogdan e822d5a1b7 Make edit / add / delete perms available to all users. (#1722)
* Make edit / add / delete perms available to all users.

* Add tests and restrict from editing the datasources.
2016-11-30 17:05:09 -05:00
Bogdan dc98c6739f Implement table name extraction. (#1598)
* Implement table name extraction tests.

* Address comments.

* Fix tests and reimplement the token processing.

* Exclude aliases.

* Clean up print statements and code.

* Reverse select test.

* Fix failing test.

* Test JOINs

* refactore as a class

* Check for permissions in SQL Lab.

* Implement permissions check for the datasources in sql_lab

* Address comments.
2016-11-29 15:43:36 -05:00
the-dcruz 84e8f741ae Add 'Save As' feature for dashboards (#1669)
* Add 'Save As' feature for dashboards

* Address code review comments
2016-11-28 08:34:14 -08:00
vera-liu 39ce4aa049 Added filter in ControlPanelsContainer for explore V2 (#1647)
* Added filter in ControlPanelsContainer

* Move function for getting url params object to utils

* Fixed python test

* Move Filter to separate component

* Added specs and made changes based on comments

* Moved specs to right folder
2016-11-23 09:51:19 -08:00
Bogdan c90dd4902f Programatically sync the role with user list. (#1619) 2016-11-21 13:06:43 -05:00
Maxime Beauchemin 7e1852ee88 User profile pages (favorites, created content, recent activity, security & access) (#1615)
* Super

* User profile page

* Fixing python style

* Python unit tests

* Touchups and js tests

* Addressing comments
2016-11-19 21:23:44 -08:00
Bogdan 1624e7de7d Add all_tables endpoint to allow airpal / superset perm sync. (#1614) 2016-11-18 20:14:26 -05:00
vera-liu d5ef937b31 Fixed bugs with viz in exploreV2 (#1609)
* Fixed bugs with viz in exploreV2

Done:
 - fix typo in pie viz
 - add metrics/groupby to dist_bar viz
 - fix typo in heatmap viz
 - add big_number_total to viz
 - fixed problem with fetching columns for datasource
 - add sqlaTimeSeries to viz
 - change row_limit and limit from number to strings so that we
 don't need to parse integers in bootstrap data

* Fix python tests

* Added order_bars checkbox for dist_bar viz
2016-11-17 12:31:36 -08:00
Maxime Beauchemin bce02e3f51 [security] improving the security scheme (#1587)
* [security] improving the security scheme

* Addressing comments

* improving docs

* Creating security module to organize things

* Moving CLI to its own module

* perms

* Materializung perms

* progrss

* Addressing comments, linting
2016-11-17 11:58:33 -08:00
vera-liu 83d08b8b8f Get query button working in explorev2 (#1581)
* Get query buttonw working in explorev2

 - Create new endpoint for updating explore viz
 - Send over new form_data when query button is pressed

* Added endpoint test

* Changes based on comments

* Added docstring for endpoint, and query spec

* Remove white space around docstring
2016-11-16 13:21:53 -08:00
Maxime Beauchemin 15b67b2c6c [WiP] rename project from Caravel to Superset (#1576)
* Change in files

* Renamin files and folders

* cleaning up a single piece of lint

* Removing boat picture from docs

* add superset word mark

* Update rename note in docs

* Fixing images

* Pinning datatables

* Fixing issues with mapbox-gl

* Forgot to rename one file

* Linting

* v0.13.0

* adding pyyaml to dev-reqs
2016-11-09 23:08:22 -08:00
Alanna Scott 51c0470f0b [explore v2] populate dynamic select field options (#1543)
* pass field options in viz json

* move field options to fetch_datasource_metadata

* on control panels container mount, fetch datasource meta data and set dynamic field choices

* render options for select fields

* use component class rather than sic

* fix linting

* fix whitespace

* delete unused var

* only render fields once datasource meta has returned

* fix typo

* add datasources and fix column formatting

* fix tests

* never used function

* fix tests

* add test for fetch_datasource_metadata

* remove unneeded props
2016-11-08 15:55:49 -08:00
Maxime Beauchemin e4bd1884d3 [druid] adding support for dimensionspecs (#1545)
More about it here:
http://druid.io/docs/latest/querying/dimensionspecs.html

fixes https://github.com/airbnb/caravel/issues/1086
2016-11-07 17:05:41 -08:00
Maxime Beauchemin 1d7d5469a9 [hotfix] remove failing Druid test 2016-11-04 09:47:02 -07:00
Bogdan ae46561648 Support week_ending_saturday for Druid. (#1491)
* Support week_ending_saturday for Druid.

* Use period granularity

* Use ISO 8601 for period definitions.

* Fix tests

* More flexibility for the freeform choices.
2016-11-02 18:45:10 -07:00
Maxime Beauchemin 1700a807e9 [sqllab] templating refactor (#1504)
* Add support for jinja templates in WHERE/HAVING clauses

* Generalizing

* bugfix
2016-11-02 13:22:07 -07:00
Maxime Beauchemin 1b124bfb87 [druid] optimize Druid queries where possible (#1517)
* [druid] optimize Druid queries where possible

Trying to use timeseries, topn where possible, falling back on 2-phases
groupby only where needed

* Fixing py3 bug
2016-11-02 11:25:33 -07:00
vera-liu a13bed2db6 Moved sqllab tests from core_tests to sqllab_tests (#1502)
* Moved sqllab tests from core_tests to sqllab_tests

* Minor changes based on comments
2016-11-01 20:48:31 -07:00
Maxime Beauchemin 61509bbd44 [sqllab] surfacing more table metadata (indices, pk, fks) (#1485)
* Expose more table/column metadata

* [sqllab] expose more table metadata

* more tests
2016-10-31 23:52:37 -07:00
Ron Baker 518fbf562c Minor Fixes (#1484)
* Fix typo accomodate

* Fix typo calender

* Fix typo dimentions
2016-10-30 16:55:56 -07:00
vera-liu 45efcb381c Added time filter to query search page (#1329)
* Added time filter to query search page

* Added start date

* Updated python endpoint test

* changed spec

* Added specs and tests

* Modified python/js tests and some function/file names
based on code review comments

* Resolved conflicts in DashboardSelect_spec and QuerySearch_spec

* Break python tests for separate functions, Move sql queries to setUp()

* Get around eslint error for spec

* Small changes based on comments
2016-10-28 14:12:53 -07:00
Maxime Beauchemin 5944643da6 [sqllab] add support for Jinja templating (#1426)
* [sqllab] add support for Jinja templating

* Adressing comments

* Presto macros

* Progress

* Addressing coments
2016-10-26 11:09:27 -07:00
Bogdan 7c5933732b Filter immune slices array stores strings. (#1402) 2016-10-25 11:08:08 -07:00
Maxime Beauchemin 63161b11c3 [sqllab] proper, quoted, select * on the server side (#1404)
* [sqllab] proper, quoted, select * on the server side

* fixing tests
2016-10-21 16:55:37 -07:00
Bogdan 4f886d65ec Fix None view_menues in permissions. (#1409) 2016-10-21 16:09:51 -07:00
Bogdan 5c3966a32d Override the role with perms for given datasources. (#1399)
* Override the role with perms for give datasources.

* Address comments.
2016-10-20 15:30:09 -07:00
Bogdan c198535292 Change slice ids in the position json during dashboard import. (#1380)
* Change slice ids in the position json during dashboard import.

* Update slice ids in the dashboard json metadata.
2016-10-20 09:17:51 -07:00
Maxime Beauchemin c255e89219 [sqllab] show partition metadata for Presto (#1342) 2016-10-19 09:17:08 -07:00
Bogdan f0f8478922 Revert "Override the role with perms for give datasources." (#1345) 2016-10-13 19:21:21 -07:00
Bogdan 40e7057bce Override the role with perms for give datasources. (#1335)
* Override the role with perms for give datasources.

* Address comments.
2016-10-13 18:18:03 -07:00
Bogdan 11a8e3591d Some dashboard import/export fixes. (#1340) 2016-10-13 14:36:39 -07:00
Maxime Beauchemin ef2670ca32 Using inheritance scheme to organize db specific code (#1294)
* Using inheritance scheme to organize db specific code

* Addressing comments
2016-10-12 13:50:47 -07:00
Bogdan 73cd2ea3b1 Import / export of the dashboards. (#1197)
* Implement import / export dashboard functionality.

* Address comments from discussion.

* Add function descriptions.

* Minor fixes

* Fix tests for python 3.

* Export datasources.

* Implement tables import.

* Json.loads does not support trailing commas.

* Improve alter_dict func

* Resolve comments.

* Refactor tests

* Move params_dict and alter_params to the ImportMixin

* Fix flask menues.
2016-10-11 17:54:40 -07:00
Dennis O'Brien cd2ab42abc do not overwrite the stored password with the masked password (#1209)
* do not over-write the stored password with a masked password.
his addresses issue #1199

* added a test to validate that sending a password-masked sqlalchemy_uri does not over-write the stored sqlalchemy_uri

* requery the database object after the update.
use self.assertEqual for more informative error messages.
2016-10-11 16:49:40 -07:00
Juho Lamminmäki 1e6e144d24 Fixed viewing dashboards as anonymous (#1320) 2016-10-11 08:04:00 -07:00
Maxime Beauchemin f70d301f0d Refactor the explore view (#1252)
* Refactor the explore view

* Fixing the tests

* Addressing comments
2016-10-07 16:24:39 -07:00
Maxime Beauchemin 472679bb38 [security] allow for requesting access when denied on a dashboard view (#1192)
* Request access on dashboard view

* Fixing the unit tests

* Refactored much in the tests
2016-10-02 18:03:19 -07:00
vera-liu aa5bbe6149 Fixed error of inserializable json for druid test (#1213)
* Fixed error of inserializable json for druid test

* Fixed indentation
2016-09-29 09:12:43 -07:00
Bogdan 9c83b900ae Bring DB in sync with the models.py (#1172)
* Bring DB in sync with the models.py

* Make sure that migrations run on multiple dbs
2016-09-28 13:55:06 -07:00
vera-liu 5f6ef84c4e Vliu query search (#1187)
* Query search page under SQL Lab tab

* Modifications based on comments

* Hash

* Added spec and endpoint test with modifications
based on second round comments

* Changed permission menu in https://github.com/airbnb/caravel/pull/1095/files
2016-09-23 17:41:24 -07:00
vera-liu 551c97112c Revert "Query Search Page" (#1186) 2016-09-23 16:28:21 -07:00
vera-liu d5c5c0d6ac Query Search Page (#1122)
* Query search page under SQL Lab tab

* Modifications based on comments

* Hash

* Added spec and endpoint test with modifications
based on second round comments

* Changed permission menu in https://github.com/airbnb/caravel/pull/1095/files
2016-09-23 16:13:18 -07:00
Bogdan df89bec712 Infer types. Smart defaults for the visualize window. Basic implementation. (#1134)
* Implement smart suggestions for the visualize flow.

* Address JS comments.

* Implement caravel dataframe wrapper.
2016-09-23 11:14:38 -07:00
Maxime Beauchemin fc921d63a1 Simplifying source_registry (#1180) 2016-09-22 21:19:03 -07:00
Bogdan cbc70d3738 Implement permission request/approve flow. (#1095)
* Implement permission request/approve flow

* Address the comments.

* Refactor the code to support multiple datasources.

* Reformat the queries.
2016-09-22 09:53:14 -07:00
ShengyaoQian 5a0e06e7a2 Generalize switch between different datasources (#1078)
* Generalize switch between different datasources.

* Fix previous migration since slice model changed

* Fix warm up cache and other small stuff

* Adding modules and datasources through config

* Replace tabs w/ spaces

* Fix other style issues

* Change add method for SliceModelView to pick the first non-empty ds

* Remove tests on slice add redirect

* Change way of db migration

* Fix styling

* Fix create slice

* Small fixes

* Fix code climate check

* Adding notes on how to create new datasource in CONTRIBUTING.md

* Fix last merge

* A commit just to trigger travis build again

* Add migration to merge two heads

* Fix codeclimate

* Simplify source_registry

* Fix codeclimate

* Remove all getter methods
2016-09-21 09:52:05 -07:00
Maxime Beauchemin e8088d5c9a More improvements to SQL Lab (#1104)
* Handling timeouts

* Fixing timer on non-utc server

* Allowing async with results

* [bugfix] database is not selected

* Making sure the session is up and running

* Cleaning up query results and query objects

* Picking a groupby and metric field on visualize flow

* Showing local time in query history

* Using pull-left pull-right instead of grid layout for table metdata

Long column name were looking weird and icons were wrapping oddly

* Linting

* Eliminating east buttons under the sql editor

* Sort database dropdown by name

* Linting

* Allowing non-SELECT statements to run

* Adding a db config

* Making sqla checkout check cross-db
2016-09-19 15:28:10 -07:00
Dennis O'Brien ca66ba4893 Fix initialization of Database sqlalchemy_uri and password (#1137)
* move initialization of Database sqlalchemy_uri and password from DatabaseView.pre_add to utils.get_or_create_main_db.
Unit tests for mysql and postgres include username and password in the SQLALCHEMY_DATABASE_URI.

* modified test_testconn to work with sqlalchemy uri with a username and password.
2016-09-19 15:14:00 -07:00
Riccardo Magliocchetti afa1f0916b Make sql test connection work with saved Database instance (#694)
Fix #596
2016-09-17 12:32:41 -07:00
Bogdan a871ee7858 Clean up the druid sync api. (#1101) 2016-09-13 17:30:36 -07:00
vera-liu 1f761c61dd Single quote filter values with comma (#1084)
* Single quote filter values with comma

* refactor for codeclimate limite

* Added unit tests and tooltip
2016-09-12 13:41:05 -07:00
Maxime Beauchemin 1971bf653c Numerous improvements to SQL Lab (#1088)
* Improving the Visualize flow

* Fixed the timer

* CTAS

* Expiclit engine handling

* make tab full height, stretch for longer content (#1081)

* Better error handling for queries

* Hooked and fixed CSV export

* Linting

* Tying in the dttm in the viz flow

* Indicator showing when going offline

* Addressing comments, fixing the build

* Fixing unit tests
2016-09-11 07:39:07 -07:00
Bogdan c20ee0c129 Filter Druid Datasources by user permissions. (#1090) 2016-09-09 17:12:09 -07:00
Bogdan d454fb402b Add refresh druid datasources endpoint. (#1065) 2016-09-07 17:27:35 -07:00
Bogdan e783219a76 Add cache warmup endpoint (#1063)
* Refactor permissions in the views.py.

* Add get_viz to the slice.

* Add warm up cache endpoint.

* Resolve comments.
2016-09-06 13:58:09 -07:00
Chris Williams bae21194c1 Chris/remove redirect from slice id endpoint (#1044)
* call explore() directly from '/caravel/slice/id/' endpoint instead of redirecting to it.

* pass slice_id instead of slice_params in 'caravel/slice/id/' endpoint to avoid logging errors. add fix for MultiDict bug

* address max's comments

* remove the /slice/id/ endpoint in preference of /datasource_type/datasource_id/slice_id/

* pep8
2016-08-31 16:45:03 -07:00
Maxime Beauchemin fc1e63761c Adding celery_tests.py 2016-08-29 22:11:57 -07:00
Maxime Beauchemin 38b8db8051 SQL Lab - A multi-tab SQL editor (#514)
* Carapal react mockup

This is really just a mock up written in React to try different
components. It could become scaffolding to build a prototype, or not.

* Merging in Alanna's theme tweaks for SQL lab

* Tweak the display of the alert message in navbar

* Sketching the middleware refresh for Queries

* Adjustments

* Implement timer sync.

* CTAS

* Refactor the queries to be stored as a dict. (#994)

* Download csv endpoint. (#992)

* CSV download engdpoint.

* Use lower case booleans.

* Replcate loop with the object lookup by key.

* First changes for the sync

* Address comments

* Fix query deletions. Update only the queries from the store.

* Sync queries using tmp_id.

* simplify

* Fix the tests in the carapal. (#1023)

* Sync queries using tmp_id.

* Fix the unit tests

* Bux fixes. Pass 2.

* Tweakin' & linting

* Adding alpha label to the SQL LAb navbar entry

* Fixing the python unit tests
2016-08-29 21:55:31 -07:00
Adam Jones 88f4260777 Change default location for db and logs to ~/.caravel Fix #915 (#947) 2016-08-16 21:35:31 -07:00
Gustavo Brian 198226a39f Fix date serialization (#873)
* [panoramix] -> [dashed]

* merge from caravel/master

* Updated from airbnb

* Cleaning

* Rebase with upstream/master

* merge from caravel/master

* Updated from airbnb

* Cleaning

* Manual rebase

* Last pending change to rebase

* Convert date to datetime before serialization.
Approach choosen: transform data before serialize and keep just one way to serialize

* Unit test created

* stupid error :(

* remove uneeded code and rename test

* Avoid double type checking
Test updated
note: isinstance(<datetime>, <date>) == True, check order changed

* Increase coverage

* Fix assertRaises
2016-08-10 23:13:59 -07:00
Chris Williams 2bfb9cc7dd pass the `standalone` request arg in the /caravel/slices/<slice_id>/ endpoint redirect (#876)
* pass the  request arg in the /caravel/slices/<slice_id>/ endpoint.

remove unused import.

* test that a single slice redirects rather than testing them all. update standalone redirect logic for Javascript 'false' instead of Python False
2016-08-10 23:11:53 -07:00
Bogdan d6bb8c6935 Add per database permissions for the SQL Lab. (#885) 2016-08-09 17:53:23 -07:00
Riccardo Magliocchetti e7ce38b486 Smarter redirect on slice creation (#691)
After ea8a7ec1ba creating a slice
started redirecting to druid datasource from sqlalchemy tables.
That's quite painful for sqlalchemy tables users.
Instead of hardcoding a choice just query the db, if we don't
have any druid datasource fallback to sqlalchemy tables.
Bonus points we remove hacky javascript and make the message
translatable.

While at it fix druid client test to not hardcode datasource id.
2016-08-01 23:06:19 -07:00
Maxime Beauchemin ee9141a31a New endpoint that receives SQL and returns data as JSON (#842)
* New endpoint that receives SQL and returns data as JSON

* Addressing comments
2016-07-29 22:39:33 -07:00
George Ke 04f3e3bc8f "Add Slices" modal on dashboard page (#678)
* Add slice modal

* use datatables, filter by slice creator

* tests & landscaping

* code review + react-bootstrap-table + modularity
2016-07-07 21:40:33 -07:00
Maxime Beauchemin 917bc984eb Make DruidDatasource.version_higher support funky version strings (#706) 2016-07-01 14:44:25 -07:00
kkalyan 30da408ace Druid Intervals Issues prevents metadata pull (#526)
* Druid Intervals Issues

* fetching druid version

* fixes

* space

* version fix

* landscape issues
2016-06-21 10:03:56 -07:00
x4base 13095eb550 Show right messages as soon as possible (#632)
* Flashed messaged should be flushed in every page

* Show error messages in AJAX style

* Introduce the decorator "api"

* Move toggleCheckbox() to the right place and trigger it in jQuery style
2016-06-21 09:42:54 -07:00
Chris Williams 668ede1133 expose /slice/<slice_id>/ endpoint to redirect to a slice's url (#633)
* expose /slice/<slice_id>/ endpoint to redirect to a slice's url

* remove residual print statement

* add unit test for caravel/slices/id endpoint
2016-06-20 09:18:03 -07:00
Maxime Beauchemin fa13b77cfa Adding postgres to the build matrix (#604) 2016-06-12 11:01:16 -07:00
Maxime Beauchemin c490138afe Fixing json issues (#602) 2016-06-11 20:39:25 -07:00
Maxime Beauchemin 55baab413a [hotfix] fixing the build (#594) 2016-06-09 22:58:20 -07:00
Maxime Beauchemin 2644dd1984 Adding a test for welcome page 2016-06-09 18:11:24 -07:00
Maxime Beauchemin 60ed3e4050 TOX / Travis build matrix (#593)
* Building on many dbs

* Fixing some translation related error msg
2016-06-09 18:05:58 -07:00
Maxime Beauchemin 3ee9a68c09 Only owners can update their objects (#507) 2016-06-02 19:17:34 -07:00
Maxime Beauchemin 57990bfd83 Implementing druid's regex filters (#501)
* Implementing druid's regex filters

* Debugging

* Debuggin'
2016-05-23 13:06:35 -07:00
Andrii Sydorchuk 8a406b18f5 Add PUBLIC_ROLE_LIKE_GAMMA config flag (#473) 2016-05-16 21:24:43 -07:00
andrewhn 6941f1de64 add unicode data to tests (#432)
* add unicode data to tests

* make tests pass on 2.7

* clean up data loading

- remove duplicate keys in slice_data
- reduce line length

* change manager option flag to -t, --load-test-data

* test --> load_test_data
2016-05-05 08:46:16 -07:00
Andrii Sydorchuk 0bedaed367 Make sure anonymous user with proper permissions can access data (#415)
* Make sure anonymous user with proper permissions can access data

* Review fixes: naming changes

* Review fixes: add more granular tests for public user dashboard access

* Review fixes: test that public user has access only to permitted data sets
2016-05-03 22:31:37 -07:00
Maxime Beauchemin 26d273643b Allowing for druid post aggregations (#418)
* Druid post aggregations

* Fixing tests
2016-05-02 10:00:39 -07:00
Maxime Beauchemin 0ca3f5ec80 Improving SQLA query generation (#421)
* Improving SQLA query generation

* Fixing debug
2016-05-02 10:00:28 -07:00
Maxime Beauchemin b634d03ac3 Show only Slices and Dashboards users have access to (#404)
* Introducing more security features

* Many to many owners for slices and dashboards
* Slices are filtered to only slices that the user has access to

* Adding unit tests
2016-04-26 16:44:51 -07:00
Maxime Beauchemin 7d90f26554 Fixing #388 (#391) 2016-04-21 21:10:09 -07:00
Maxime Beauchemin f1e10d8d25 One more Druid test (#387) 2016-04-21 08:17:15 -07:00
Maxime Beauchemin a5f33fecd8 Adding login call for Druid test 2016-04-20 20:13:41 -07:00
Maxime Beauchemin 17e711fda2 Druid unit tests using Mock (#384)
* Initial Druid mock unit tests

* More unit tests

* Test for Druid query

* Adding a groupby test
2016-04-20 15:08:10 -07:00
antbell a9af6b1b96 Fix for undefined inner_time_filter on missing granularity in models.SqlaTable (#309) 2016-04-10 16:22:58 -07:00
Siddharth Gupta eff0beb195 Optimizing import statements for better readability and to avoid unused, excessive and duplicating imports in the project. (#294) 2016-04-07 23:01:40 -07:00
Maxime Beauchemin 8e27099866 Fixing the tests for py3 2016-03-29 10:02:15 -07:00
Maxime Beauchemin 60bce9ed59 Fixing the order and coverage report for the unit tests 2016-03-28 23:50:11 -07:00
Maxime Beauchemin 619d35878f [dashed->caravel] Replace in files 2016-03-28 22:01:21 -07:00
Maxime Beauchemin d48796f00e Caravel - renaming files 2016-03-28 21:53:24 -07:00
Maxime Beauchemin 301dce2dd1 Using setup.py nosetests to run tests 2016-03-27 19:15:21 -07:00
Maxime Beauchemin 2102e04fec Adding tests for ping and health enpoint 2016-03-27 14:23:49 -07:00
Maxime Beauchemin be6b2fe556 [panoramix] -> [dashed] 2016-03-17 23:44:58 -07:00
Maxime Beauchemin 27fb810dd7 Dynamic time granularity on any datetime column 2016-03-16 15:12:16 -07:00
Maxime Beauchemin e039547762 More tests using doctests 2016-02-10 11:20:13 -08:00
Maxime Beauchemin b18d117852 Renaming Classes related to Druid 2016-02-10 06:56:35 -08:00
Maxime Beauchemin ef5511cccf Getting stuff to load 2016-01-13 17:19:12 -08:00
Maxime Beauchemin 1cd63c3e4a Adding a unit test 2015-12-17 18:14:40 -08:00
Maxime Beauchemin e3e3d0b1f5 py3 compat changes 2015-10-20 12:50:11 -07:00
Maxime Beauchemin c2049f837c Automate Panoramix role creation 2015-10-04 16:23:19 -07:00
Maxime Beauchemin c3f0597c07 Small improvments to tests 2015-09-28 21:31:04 -07:00
Maxime Beauchemin 703b6f612b Forgot to add some files for unit tests 2015-09-26 15:56:08 -07:00