Commit Graph

501 Commits

Author SHA1 Message Date
Daniel Vaz Gaspar 123246fca6
[dashboards] New, export api (#8941)
* [dashboards] Multiple exports

* [dashboards] Fix, mulexport permission missing

* [dashboards] Test for security filtered export

* [dashboards] Address PR comments
2020-01-15 18:09:55 +00:00
serenajiang 1f6f4ed879 [db migration] change datasources-clusters foreign key to cluster_id (#8576)
* [db migration] change datasources foreign key to cluster_id

* address pr comments

* address pr comment, fix ci
2020-01-13 11:02:36 -08:00
Rob DiCiuccio 7ff18f060c Ensure proper JSON serialization of numpy.ndarray (#8946) 2020-01-10 13:24:03 -05:00
John Bodley 2d456e88eb
[fix] Enforce the query result to contain a data-frame (#8935) 2020-01-08 11:50:26 -08:00
Daniel Vaz Gaspar 2726f21cbc
[dashboard] Fix, prevent delete and update on dashes not owned (#8911) 2020-01-06 15:29:39 +00:00
Rob DiCiuccio 6537d5ed8c Replace pandas.DataFrame with PyArrow.Table for nullable int typing (#8733)
* Use PyArrow Table for query result serialization

* Cleanup dev comments

* Additional cleanup

* WIP: tests

* Remove explicit dtype logic from db_engine_specs

* Remove obsolete  column property

* SupersetTable column types

* Port SupersetDataFrame methods to SupersetTable

* Add test for nullable boolean columns

* Support datetime values with timezone offsets

* Black formatting

* Pylint

* More linting/formatting

* Resolve issue with timezones not appearing in results

* Types

* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip

* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks

* Dataframe typing and nits

* Renames to reduce ambiguity
2020-01-03 11:55:39 -05:00
Craig Rueda 4ce5add5df Making test_config and main config the same (#8838) 2020-01-02 13:57:54 -05:00
Rob DiCiuccio 8fc814fc62 Enable running of tests in tests/db_engine_specs (#8902)
* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip
2019-12-31 14:45:33 +02:00
Ville Brofeldt 5b690f9411
chore: refactor, add typing and fix uncovered errors (#8900)
* Add type annotations and fix inconsistencies

* Address review comments

* Remove incorrect typing of jsonable obj
2019-12-31 09:26:23 +02:00
Daniel Vaz Gaspar 3d9181d270
[dashboard] New, get releated owners and slices (#8872) 2019-12-20 10:16:57 +00:00
David Aaron Suddjian 016f202423 Refactor Dashboard and Slice models (#8820)
* refactor dashboard and slice models

* appease various linters

* remove shortcuts & import indirection

* appease mypy

* fix bad imports

* lint

* address various issues

* ignore type issue

* remove unused imports

* lint
2019-12-18 11:40:45 -08:00
David Aaron Suddjian 7a68cb7ca0 factor out datasource_access_request model (#8809) 2019-12-17 16:17:49 -08:00
Daniel Vaz Gaspar 2f2ac00a09
[dashboard] feat: REST API (#8694) 2019-12-16 21:10:33 +00:00
Will Barrett b87f21fe1f Move function in utils folder to fix circular import issues (#8814)
* Move function in utils folder to fix circular import issues

* CI fixes - black, isort
2019-12-13 16:44:56 +00:00
Will Barrett ed54f6ec8d Require the SQLAlchemy URI on the database model (#8720)
* Require the SQLAlchemy URI when creating a database

* Add migration to make dbs.sqlalchemy_uri not-nullable

* Fixes for black, isort, tests

* Alter migration to use current revision from master as downgrade target

* Update tests to support new db constraint

* black
2019-12-11 14:05:23 -08:00
Bogdan e6be51953c Avoid circular dependency between superset config and superset package (#8254)
Resolve comments

Avoid circular dependency between superset config and superset package

Resolve comments
2019-12-10 09:17:34 +00:00
Will Barrett 7f8db7ab8d Re-enable pylint for common, connector, and test files (#8774)
* re-enable pylint for superset/common/query_context.py

* Re-enable pylint for superset/connectors/base/views.py

* re-enable pylint for superset/connectors/connector_registry.py

* Re-enable pylint for tests/fixtures/pyodbcRow.py

* re-enable pylint for superset/cli.py

* Fix for mypy

* Improve variable naming

* A few more naming fixes
2019-12-09 22:32:54 -08:00
Beto Dealmeida 5477f42740
Limit payload returned when using SQLLAB_BACKEND_PERSISTENCE (#8769)
* Limit queries

* WIP

* Adding unit test

* WIP

* Add unit test

* Fix test

* Try to fix remote unit tests

* Fix unit test

* Fix  for postgres
2019-12-09 16:12:40 -08:00
Dinesh Sawant cae0583e0f Support timeFormat extraction function column for Druid in Filter (#8728) 2019-12-07 17:54:37 -08:00
Hossein Torabi b053d66ea7 fix urlib opener bug (#8788) 2019-12-07 16:07:09 -08:00
ʈᵃᵢ a94464b9c9 guard against duplicate columns in datasource save (#8712)
* catch IntegrityError in datasource save

* catch duplicate columns and wrap in exception handling decorators

* use 409

* isort
2019-12-04 11:05:06 -08:00
John Bodley c74bb6cb9b
[fix] Updating bound filters for PyDruid 0.5.7 (#8742) 2019-12-03 17:44:04 -08:00
David Aaron Suddjian b0b93768bb handle nulls in bar chart series/breakdown (#8724)
* handle nulls in bar chart series/breakdown

* snake_case

* add tests for null groups in bar charts

* black

* comments will save us
2019-12-03 16:11:15 -08:00
Bogdan 003e98c231 Support schema permissions (#8219)
* Build support for schema access in Superset and SQLab

* Lint
* Test for old and new flask
* Black formatting
* Cleanup modified files
* Support schema permissions
* Add migration
* Make it work for slices as well
* TODO and remove unused field
* Apply schema access check to dashboards as well

Cleaner schema permissions check

Fetch schema permissions in 1 query

Bugfix

Better datasource check

Use security manager for permissions checks and remove dead code

Properly handle anon user

Add database permissions on the database updates

Fix schema filtering

Lint / flake changes

Increase cypress request timeout

Run black

Resolve multiple alembic heads

* Use set comprehensions

* Fixes for the pylint
2019-12-03 14:57:24 -08:00
Beto Dealmeida f8e15e1c55
Add protocol to cache warmup (#8721)
* Add protocol to cache warmup

* Update unit tests

* Fix wrong commit

* Update UPDATING.md

* Fix pylint
2019-12-03 10:51:06 -08:00
John Bodley 4e998be695
[fix] Druid IS NULL/IS NOT NULL filters (#8714) 2019-12-02 20:38:42 -08:00
Will Barrett 16c0599470 Disable deprecated druid connector by default (#8512)
* Disable deprecated druid connector by default

* Add a line in UPDATING.md for the configuration change

* Remove security tests related default-disabled feature

* More test updates

* black
2019-11-27 11:28:23 -08:00
Craig Rueda df2ee5cbcb Adding app context wrapper to Celery tasks (#8653)
* Adding app context wrapper to Celery tasks
2019-11-27 15:06:06 +00:00
Beto Dealmeida 5f4e3adfd2
Pass full response to `query_cost_formatter` (#8652)
* Return full info when doing query cost estimation

* Add unit test

* Fix isort
2019-11-26 11:49:19 -08:00
Ville Brofeldt 7104b04817
Pass url parameters from dashboard to charts (#8536)
* Pass url_params from dashboard to charts

* Update params to form_data instead of overwriting

* Add cypress tests

* Add python test

* Add docs

* Move reserved url params to utils

* Bump cypress
2019-11-21 07:07:08 +02:00
Craig Rueda e490414484 Flask App factory PR #1 (#8418)
* First cut at app factory

* Setting things back to master

* Working with new FLASK_APP

* Still need to refactor Celery

* CLI mostly working

* Working on unit tests

* Moving cli stuff around a bit

* Removing get in config

* Defaulting test config

* Adding flask-testing

* flask-testing casing

* resultsbackend property bug

* Fixing up cli

* Quick fix for KV api

* Working on save slice

* Fixed core_tests

* Fixed utils_tests

* Most tests working - still need to dig into remaining app_context issue in tests

* All tests passing locally - need to update code comments

* Fixing dashboard tests again

* Blacking

* Sorting imports

* linting

* removing envvar mangling

* blacking

* Fixing unit tests

* isorting

* licensing

* fixing mysql tests

* fixing cypress?

* fixing .flaskenv

* fixing test app_ctx

* fixing cypress

* moving manifest processor around

* moving results backend manager around

* Cleaning up __init__ a bit more

* Addressing PR comments

* Addressing PR comments

* Blacking

* Fixes for running celery worker

* Tuning isort

* Blacking
2019-11-20 15:47:06 +00:00
ʈᵃᵢ 9d36fa348f Move from MVC FAB menu to Superset React menu (#8289)
* adds react based menu for welcome view

* bump fab

* better logic in common_bootstrap_payload

* split up files

* fix spacing and module scope styles

* lint

* render correct template for welcome
2019-11-19 22:56:27 -08:00
Beto Dealmeida 3b97ae3b9d
Set owner when creating SQL Lab view (#8597)
* Set owner when creating SQL Lab view

* Add unit test

* Small fix
2019-11-19 10:48:42 -08:00
Beto Dealmeida 78f41b38e1
Fix more places where feature flags are serialized (#8567) 2019-11-14 10:01:57 -08:00
Beto Dealmeida d66bc5ad90
SIP-23: Persist SQL Lab state in the backend (#8060)
* Squash all commits from VIZ-689

* Fix javascript

* Fix black

* WIP fixing javascript

* Add feature flag SQLLAB_BACKEND_PERSISTENCE

* Use feature flag

* Small fix

* Fix lint

* Fix setQueryEditorSql

* Improve unit tests

* Add unit tests for backend sync

* Rename results to description in table_schema

* Add integration tests

* Fix black

* Migrate query history

* Handle no results backend

* Small improvement

* Address comments

* Store SQL directly instead of reference to query

* Small fixes

* Fix clone tab

* Fix remove query

* Cascade delete

* Cascade deletes

* Fix tab closing

* Small fixes

* Small fix

* Fix error when deleting tab

* Catch 404 when tab is deleted

* Remove tables from state on tab close

* Add index, autoincrement and cascade

* Prevent duplicate table schemas

* Fix mapStateToProps

* Fix lint

* Fix head

* Fix javascript

* Fix mypy

* Fix isort

* Fix javascript

* Fix merge

* Fix heads

* Fix heads

* Fix displayLimit

* Recreate migration script trying to fix heads

* Fix heads
2019-11-14 09:44:57 -08:00
Ville Brofeldt 90275fe991
Add convert_dttm method to SnowflakeEngineSpec (#8551) 2019-11-13 11:13:49 +02:00
Beto Dealmeida a58b3920db
Encode feature flags to JSON pessimistically (#8529)
* Encode feature flags to JSON pessimistically

* Add unit test

* Remove old imports
2019-11-12 11:10:36 -08:00
Daniel Vaz Gaspar 650c848243
[cli] Fix, import datasources exported by UI (#8477)
* [cli] Fix, import datasources exported by UI
2019-11-10 16:29:51 +00:00
Ville Brofeldt 49ea232c3a
[fix] Improve csv upload functionality (#8457)
* [fix] csv upload when table metadata present

* Remove table from hive spec

* Move upload before table metadata creation

* Refine upload logic, dd unit tests and fix translations

* Use ALLOWED_EXTENSIONS from config

* Address review comments

* Fix error message grammar

* Add return type to hive csv upload and replace first with one_or_none
2019-11-07 20:03:42 +02:00
John Bodley 1e3d833b02
[sip-15] Fixing time range endpoints from dashboards (#8513) 2019-11-06 09:15:58 -08:00
John Bodley 5dba2f4b56
[SIP-15] Adding grace period (#8490) 2019-11-04 12:00:41 -08:00
John Bodley eb150eba1f
[sip-15] Fix time range endpoints encoding (#8481) 2019-10-31 07:11:48 -07:00
Will Barrett e4ca44e95f Use `config[]` not `config.get()` (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
Kim Truong 3cba1bd990
feat: add expand_data parameter (#8472)
* feat: add expand_data parameter

* fix: reformat files
2019-10-30 11:45:35 -07:00
John Bodley 0a3b121244
[sql] Fixing datetime SQL literal (#8464) 2019-10-29 23:24:48 -07:00
Beto Dealmeida e704e29174
Allow fetching all rows from results endpoint (#8389)
* Allow bypassing DISPLAY_MAX_ROW

* Add unit tests and docs

* Fix tests

* Fix mock

* Fix unit test

* Revert config change after test

* Change behavior

* Address comments
2019-10-25 10:22:16 -07:00
Will Barrett 82b174701f Split up tests/db_engine_test.py (#8449)
* Split up db_engine_specs_test.py into a number of targeted files

* Remove db_engine_specs_test.py

* isort
2019-10-24 20:46:45 -07:00
John Bodley 786d7706e7
[fix] Updating parse_human_timedelta typing (#8436) 2019-10-23 16:04:46 -07:00
Marcus 4cfd6b3ce1 [tests] Fix, removing deprecated function for future Python release compatibility. (#8353)
* removing deprecated function for future Python release compatibility.

* also removed assertNotEquals per @dpgaspar
2019-10-21 15:49:12 +01:00
John Bodley 9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00