Commit Graph

810 Commits

Author SHA1 Message Date
Ville Brofeldt b532c4609f
chore: bump python dependencies (#9786)
* chore: bump pyhton dependencies

* bump flask-testing

* Add note to UPDATING and bump flask-testing in requirements-dev.txt

* fix old sqlparse test case that is fixed with sqlparse==0.3.1

* Fix migration and sqlparse set test

* downgrade alembic

* Downgrade sqlalchemy-utils

* pin sqlalchemy-utils due to regression in 0.36.5

* Refine comment in setup.py

* make cachelib an explicit requirement
2020-05-13 18:49:34 +03:00
Erik Ritter 83ec7365a7
feat: convert backend chart errors to the new error type (#9753) 2020-05-12 18:15:53 -07:00
John Bodley 04e3a4a72d
[caching] Using request context rather than globals (#9715)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-11 15:55:05 -07:00
Ville Brofeldt a8d5a8aaa2
fix: make tests work individually (#9782) 2020-05-11 11:48:22 -07:00
Ville Brofeldt 42b10aecae
Add QueryContext deserialization test (#9778)
* Add QueryContext deserialization test

* deserialize using marshmallow and assert error dict
2020-05-11 14:10:14 +03:00
Bogdan 903217f64d
Fix SQL Lab schema permission checks (#9756)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-05-08 14:59:49 -07:00
Daniel Vaz Gaspar 453806fc4c
perf(dataset): improve performance on get list (#9739) 2020-05-06 11:48:32 +01:00
Daniel Vaz Gaspar 0d85d25314
docs(api): improve openapi documentation for dash, charts and queries (#9724) 2020-05-05 14:42:18 +01:00
Bogdan 11972ed60d
Fix bad test (#9735)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-05-04 23:52:18 -07:00
ʈᵃᵢ 4c522ac65f
[sqllab] fix, strip comments before parsing statements (#9692) 2020-05-04 19:47:29 +03:00
Daniel Vaz Gaspar 2b59075d57
tests(engine_specs): full bigquery engine coverage (#9702) 2020-04-30 18:31:15 +01:00
John Bodley 3b0f8e9c8a
[sql] Adding lighweight Table class (#9649)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-30 08:38:02 -07:00
Bogdan 283504479e
Fix ENABLE_ROW_LEVEL_SECURITY flag usage (#9689)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-04-29 23:57:06 +03:00
John Bodley 7f89f12c4a
[debug] Debugging caching issue (#9665)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-29 12:16:47 -07:00
Daniel Vaz Gaspar 527bee5051
tests(engine_specs): full postgres engine coverage (#9682) 2020-04-29 18:20:25 +01:00
Daniel Vaz Gaspar f13ba2561c
fix(tests): custom filter flaky tests on dash and charts (#9679) 2020-04-29 08:36:11 +01:00
Lily Kuang b27a81e0da
[logging] deprecation notices for SQLite (#9662)
* [logging] deprecation notices for SQLite

* [logging] use warning for SQLite notice

* [logging] update grammar
2020-04-28 14:28:09 -07:00
Ville Brofeldt a52cfcd234
feat: Add geospatial post processing operations (#9661)
* feat: Add geospatial post processing operations

* Linting

* Refactor

* Add tests

* Improve docs

* Address comments

* fix latitude/longitude mixup

* fix: bad refactor by pycharm
2020-04-28 20:15:16 +03:00
Daniel Vaz Gaspar f07ca7d836
fix(database): test connection error message for module not found (#9634) 2020-04-27 09:24:41 +01:00
Daniel Vaz Gaspar 516bdf6db1
fix(mssql): apply limit and set alias for functions (#9644) 2020-04-27 09:23:08 +01:00
Lily Kuang b2720074bd
[datesets] feat: add statsd to datasets api (#9577)
* add statsd to tables api

* [datasets] update export endpoint

* [datasets] update charts api test
2020-04-24 16:38:29 +01:00
John Bodley 955a4fe792
[fix] Automatically add relevant Jinja methods to cache key if present (#9572)
* [fix] Adding URL params to cache key if present

* [cache] Wrapping Jinja methods

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-24 08:18:22 -07:00
John Bodley 8ae92b50ad
[fix] Fixing regression from #9161 (#9641)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-23 22:51:33 -07:00
Ville Brofeldt a6cedaaa87
chore: Improve chart data API + schemas + tests (#9599)
* Make all fields optional in QueryObject and fix having_druid schema

* fix: datasource type sql to table

* lint

* Add missing fields

* Refactor tests

* Linting

* Refactor query context fixtures

* Add typing to test func
2020-04-23 14:30:48 +03:00
Ville Brofeldt 85778f5bec
fix: broken IS NULL and IS NOT NULL operator (#9613)
* fix: broken is null and is not null operator

* add unit tests

* Rename filter operator enum
2020-04-22 19:11:45 +03:00
ʈᵃᵢ 0b999e3b91
[charts] adds new filters ui (#9530)
* [charts] adds new filters ui

* move null check to be more visible

* better filter lists and async filter functionality
2020-04-21 12:04:11 -07:00
Lily Kuang 7cefc89c64
[charts] feat: add statsd to charts api (#9571)
* add statsd to charts api

* update test for charts api

* [charts] add statsd asserts wrapper

* [charts] update api test

* removed white space
2020-04-21 19:57:42 +01:00
John Bodley 237ac59474
[fix] Fixing issue with Jinja filter_value (#9582)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-20 09:44:45 -07:00
Ville Brofeldt 0472c11157
fix: Add deprecated fields to QueryObject schema (#9579)
* fix: Add deprecated fields to QueryObject schema

* linting
2020-04-18 20:28:57 +03:00
Ville Brofeldt 8e439b1115
chore: Add OpenAPI docs to /api/v1/chart/data EP (#9556)
* Add OpenAPI docs to /api/v1/chart/data EP

* Add missing fields to QueryObject, fix linting and unit test errors

* Fix unit test errors

* abc

* Fix errors uncovered by schema validation and add unit test for invalid payload

* Add schema for response

* Remove unnecessary pylint disable
2020-04-17 16:44:16 +03:00
John Bodley 074e3653e2
[fix] warm up cache error handling (#9560)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-16 15:19:32 -07:00
Daniel Vaz Gaspar 7b11b44abe
[dashboard] New, add statsd metrics to the API (#9519) 2020-04-16 10:54:45 +01:00
Bogdan 3574a3a4fb
Make email parsing more robust (#9523)
* Make email parsing more robust

* Address comment

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-04-15 16:58:03 +03:00
Aliaksei Kushniarevich ef5e11f45b
[copy] fix: Row Level Security get_rls_filters func SELECT statement (#9541)
* fix: Row Level Security get_rls_filters func SELECT statement

* More general RowLevelSecurityTests case to avoid improper ids matching
2020-04-15 16:41:54 +03:00
Daniel Vaz Gaspar d81f720502
[thumbnails] API and celery task for dashboards and charts (#8947) 2020-04-15 09:40:14 +01:00
Jianchao Yang 7b4b0d1c61
[Build] Add Github workflows (#9517)
As an replacement to Travis CI.
2020-04-14 13:38:09 -07:00
Daniel Vaz Gaspar 925d8b5315
[dashboard] Fix, improve test for custom filter (#9521) 2020-04-14 10:10:14 +01:00
Ville Brofeldt 9447381549
deprecate groupby controls in query_obj (#9366)
* Deprecate groupby from query_obj

* Fix query_object bug

* Fix histogram

* Remove groupby from legacy druid connector and fix first batch of unit tests

* Deprecate some unnecessary tests + fix a few others

* Address comments

* hide SIP-38 changes behind feature flag

* Break out further SIP-38 related tests

* Reslove test errors

* Add feature flag to QueryContext

* Resolve tests and bad rebase

* Backport recent changes from viz.py and fix broken DeckGL charts

* Fix bad rebase

* backport #9522 and address comments
2020-04-14 12:06:10 +03:00
Daniel Vaz Gaspar 622561f756
[charts] New, custom filter for name OR description (#9492)
* [charts] New, custom filter for name OR description

* Improve test
2020-04-11 07:25:54 +01:00
Bogdan 02b3fb1d63
Add visualization flow to the CTA queries (#9370)
* Visualize after running Ctas query (#4)

* Visualize after running Ctas query.

Stub for table viz

Work on the viz

Add JS functionality for Ctas viz

Add test for the table viz flow

Fix test

Add js test for the ctas viz

* Fix tests

* Resolve comments

* Leverate tmpSchema in the query object

* Fix i18n string

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-04-11 00:46:46 +03:00
Ville Brofeldt a8ce3bccdf
feat: Add post processing to QueryObject (#9427)
* Add post processing to QueryObject

* Simplify sort signature and require explicit sort order

* Add new operations and unit tests

* linting

* Address comments

* Simplify test method names

* Address comments

* Linting

* remove unnecessary logic

* Apply strict whitelisting to all getattr calls

* Add checking of rolling_type_options and add/improve docs
2020-04-10 20:50:11 +03:00
Daniel Vaz Gaspar b39e78fca5
[dashboards] New, tittle and slug OR filter (#9435)
* [dashboards] New, tittle and slug OR filter

* Update requirements, because of prison bump

* Tests

* Fix tests

* Avoid like filter on empty string value

* merge master brings strict typing to the table
2020-04-09 11:14:28 +01:00
Daniel Vaz Gaspar bb80ceaccc
[dashboards] Fix, update dashboard owners not propagating to charts o… (#9484)
* [dashboards] Fix, update dashboard owners not propagating to charts owners

* Add tests

* Fix tests

* better naming
2020-04-09 09:37:37 +01:00
Daniel Vaz Gaspar 0610c489bb
[query] Migrate api v1 query to new location (#9479)
* [query] Migrate api v1 query to new location

* Improved errors and tests

* [query] nit and revert undesired change

* [query] lint

* [query] revert nan change

* Update superset/queries/api.py

Co-Authored-By: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Update tests/queries/api_tests.py

Co-Authored-By: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* change endpoint location to charts

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-04-09 09:05:49 +01:00
Daniel Vaz Gaspar 4485800e21
[tests] refactor, change datasets and charts to it's own folder (#9487) 2020-04-08 08:44:35 +01:00
dandanhub 72f051f3ce
[sqllab] Add CUSTOM_TEMPLATE_PROCESSOR config (#9376)
Co-authored-by: Dandan Shi <dshi@dropbox.com>
2020-04-07 21:00:42 +01:00
David Aaron Suddjian 5e535062da
Filter owners select by text input (#9337)
* filter owners select by text input

* use rison

* fix backend owners filter logic

* use fullname, not username on owners inputs

* fix some tests

* fixing tests

* deterministic tests

* appease linter

* add back search by username

* more comprehensive filter test

* add clarifying text

* formatting...
2020-04-07 09:09:02 -07:00
Daniel Vaz Gaspar c0807c1af7
[query] New, readonly API (#9403)
* [query] New, readonly API

* Update tests/queries/api_tests.py

Co-Authored-By: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-04-06 11:18:32 +01:00
Jianchao Yang 801e2f1777
chore: bump black to 19.10b0 and mypy to 0.770 (#9378)
* Bump black to 19.10b0

* Upgrade mypy to 0.770

* Update how inline type is defined
2020-04-04 13:23:18 -07:00
Daniel Vaz Gaspar 752de8fe9d
[dashboards] Fix, API update slug uniqueness refusing empty string (#9417)
* [dashboards] Fix, API update slug uniqueness refusing empty string

* [dashboards] tests
2020-03-30 16:10:08 +01:00
Ville Brofeldt 499f9c8fca
feat: add SSL certificate validation for Druid (#9396)
* feat: add SSL certificate feature

* Address comments

* don't mutate extras

* Address comments and add polish

* Add further polish
2020-03-27 19:07:07 +02:00
Daniel Vaz Gaspar 81971967c3
[charts] New, bulk delete API endpoint (#9387)
* [charts] New, bulk delete API endpoint

* [charts] Fix, typos

* [charts] Fix wrong model name
2020-03-27 09:31:01 +00:00
Daniel Vaz Gaspar 40bc1b33d2
[dataset] New, export API endpoint (#9367)
* [dataset] New, export API endpoint

* Fix, lint
2020-03-27 09:30:23 +00:00
John Bodley 19da9042ff
[cache] Cleaning up viz/cache logic (#9375)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-03-26 09:22:06 -07:00
Grace Guo 6b0f62a36e
[explore view] fix long query issue from Run in SQL LAB Button (#9345)
* [explore view] fix long query issue from Run in SQL LAB Button

* SQL Lab page needs to take the post form data, too

* fix variable names

* updated payload dict, rename hidden form

Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
2020-03-25 13:15:52 -07:00
Daniel Vaz Gaspar ccb22dc976
[dataset] columns and metrics API (nested) (#9268)
* [dataset] columns and metrics API (nested)

* [dataset] tests and validation

* [datasets] Fix, revert list field name to database_name
2020-03-24 17:24:08 +00:00
Daniel Vaz Gaspar f51ab59748
[charts] Refactor API using SIP-35 (#9329)
* [charts] Refactor charts API using SIP-35

* [charts] Fix, copy pasta

* [charts] simplify
2020-03-24 10:05:11 +00:00
Maxime Beauchemin 98a71be80b
feat: [explore] don't save filters inherited from a dashboard (#9340)
* feat: [explore] don't save filters inherited from a dashboard

When navigating to explore from a dashboard context, the current
dashboard filter(s) are passed along to explore so that the context is
kept. So say you're filtering on "country=Romania", in your dashboard
and pivot to explore, that filter is still there and keep on exploring.

Now a common issue is that you'll want to make some tweak to your chart
that are unrelated to the filter, say toggling the legend off for
instance, and then save it. Now you back to your dashboard and even
though you started with an "all countries" dashboard, with a global
filter on country, now that one chart is stuck on "Romania". Typically
you notice this when filtering on something else, say "Italy" and then
that one chart now has two mutually exclusive filters, and show "No data".

Now, the fix is to flag the filter as "extra" (that's the not-so-good internal
name we use for these inherited filters) and make it clear that that
specific filter is special and won't be saved when saving the chart.

* fix build
2020-03-23 23:05:00 -07:00
Daniel Vaz Gaspar c34df6b7b3
[dashboard] Refactor API using SIP-35 (#9315)
* [dashboard] Refactor API using SIP-35

* [dashboard] Fix, import

* [dashboard] more tests

* [dashboards] a misc of improvements

* [charts] Fix, DAO and tests

* [dashboards] small exceptions refactor

* [dashboards] lint

* [dashboards] Improves comments on base classes

* [dashboards] lint
2020-03-20 16:32:03 +00:00
Ville Brofeldt f1edcebc79
fix: handle list of lists from fetch_data (#9322)
* fix: handle list of lists from fetch_data

* Address comments
2020-03-18 22:04:26 +02:00
Grace Guo adebd40d30
[cache warm_up] warm_up slice with dashboard default_filters (#9311)
* [cache warm_up] warm_up slice with dashboard default_filters

* update Celery warmup tasks

* fix code review comments

* add try catch and type checking for parsed dash metadata

* extra code review fix
2020-03-18 08:21:10 -07:00
Ville Brofeldt 982c234a50
fix: remove character set and collate column info by default (#9316)
* fix: remove character set and collate column info by default

* lint

* remove collation and charset info before compile
2020-03-17 21:34:39 +02:00
Maxime Beauchemin 6cf36c91ea
fix: big number to handle NULL as it did in the past (#9314) 2020-03-17 10:46:59 -07:00
ʈᵃᵢ 5767fb15cd
[datasets] new, listview (react) (#9197)
* [datasets] new, react listview

* add hidden columns to support filtering by columns not rendered

* throw exception if config is incorrect

* fix database filter

* update endpoints to point to datasets; fix translation strings

* move Link into src/components

* add add new record button to datasets listview
2020-03-13 21:35:00 +02:00
Erik Ritter f80fadff0e
Reduce dashboard bootstrap payload (#9284) 2020-03-13 12:18:22 -07:00
Daniel Vaz Gaspar 78ba7d52f6
[dashboard][api] Fix, PUT publish/draft to not clean slug and owners (#9279) 2020-03-12 15:35:27 +00:00
Ville Brofeldt 3682702e91
fix: add connection testing params for snowflake (#9272)
* fix: add connection testingt params for snowflake

* Linting
2020-03-11 06:51:57 +02:00
Grace Guo 724b8a3c31
[fix] copy filter_scopes with duplicate charts (#9271) 2020-03-10 10:23:15 -07:00
Maxime Beauchemin c04d6163e7
feat: add rolling window support to 'Big Number with Trendline' viz (#9107)
* Rolling big number

* addressing comment
2020-03-10 10:19:12 -07:00
Maxime Beauchemin 116200cf73
fix: change database save in DatasourceEditor (#9255)
* fix: change database save in DatasourceEditor

This addresses the issue where pointing a datasource to another database
in the datasource editor is not reflected.

Also addresses:
- a minorcosmetic issue in the datasource editor.
- user/owners list not getting populated

* tests
2020-03-10 09:20:37 -07:00
Daniel Vaz Gaspar 52c59d6890
[datasets] new, API using command pattern (#9129)
* [datasets] new, API using command pattern

* [datasets] tests and improvements

* [datasets] lint

* [database] address comments

* [datasets] lint

* [datasets] Address PR comments

* [dataset] Fix, dataset expects a Dict now

* [dataset] lint and optional commits

* [dataset] mypy

* [dataset] Fix, license and parent class

* [dataset] Make CRUD DAO raise exceptions
2020-03-08 09:13:08 +00:00
Erik Ritter f34e9e393a
Replace No data error with help message (#9249) 2020-03-06 16:11:01 -08:00
Grace Guo 000a038af1
[fix] use filter_scopes in dashboard warmup strategy (#9235)
* [fix] use filter_scopes in dashboard warmup strategy

* remove unnecessary comment
2020-03-05 17:50:32 -08:00
Ville Brofeldt ef2ebbd570
Add option to specify type specific date truncation functions (#9238) 2020-03-05 07:25:50 +02:00
Ville Brofeldt 969bc87431
fix: Oracle fetch_query and datetime conversion (#9240) 2020-03-04 23:43:28 +02:00
Ville Brofeldt 7a91498cf1
fix: share column type matching between model and result set (#9161)
* Share column type matching between model and result set

* Address comments
2020-03-04 16:33:22 +02:00
Bogdan 4e1fa95035
Make schema name for the CTA queries and limit configurable (#8867)
* Make schema name configurable

Fixing unit tests

Fix table quoting

Mypy

Split tests out for sqlite

Grant more permissions for mysql user

Postgres doesn't support if not exists

More logging

Commit for table creation

Priviliges for postgres

Update tests

Resolve comments

Lint

No limits for the CTA queries if configures

* CTA -> CTAS and dict -> {}

* Move database creation to the .travis file

* Black

* Move tweaks to travis db setup

* Remove left over version

* Address comments

* Quote table names in the CTAS queries

* Pass tmp_schema_name for the query execution

* Rebase alembic migration

* Switch to python3 mypy

* SQLLAB_CTA_SCHEMA_NAME_FUNC -> SQLLAB_CTAS_SCHEMA_NAME_FUNC

* Black
2020-03-03 19:52:20 +02:00
David Aaron Suddjian e01f24f833
Prevent database connections to sqlite (#9218)
* prevent database connections to sqlite

* tweaks and tests

* add entry to UPDATING.md
2020-03-02 13:13:11 -08:00
Grace Guo ccd6e44edf
refactor copy filter_scopes and add tests (#9224)
* refactor copy filter_scopes and add tests

* fix review comments
2020-03-02 12:55:30 -08:00
John Bodley 7b2c2e8a33
[fix] Adding SIP-15 support for the query context (#9219) 2020-03-02 10:10:54 -08:00
michellethomas 4f73f8a1f9
[Bug Fix] Returning timeseries_limit_metric in table viz get_data (#9196)
* Returning timeseries_limit_metric in table viz get_data

* Fixing issue with include_time field

* Reformatting and adding a test

* Changing if/else structure

* Reformatting
2020-02-25 21:34:36 -08:00
Daniel Vaz Gaspar 27b371a485
[core] Fix, sanitize errors returned from testconn (#9178) 2020-02-24 10:40:51 +00:00
altef dee16de03e
[SIP-29] Add support for row-level security (#8699)
* Support and apply filters.

* Added the UI for row level security, and moved it all under SQLA in order to access the Table model more easily.

* Added a row level security filter documentation entry.

* Accidentally added two new lines to this file.

* Blacked and iSorted, hopefully.  Also, sometimes g.user may not be set.

* Another isort, and handling g not having a user attribute another way.

* Let's try this again #CI tests.

* Adjusted import order for isort; I was sure I'd already done this..

* Row level filters should be wrapped in parentheses in case one contains an OR.

* Oops, did not think that would change Black's formatting.

* Changes as per @mistercrunch.

* RLS filters are now many-to-many with Roles.

* Updated documentation to reflect RLS filters supporting multiple rows.

* Let's see what happens when I set it to the previous revision ID

* Updated from upstream.

* There was a pylint error.

* Added RLS ids to the cache keys; modified documentation; added template processing to RLS filters.

* A new migration was merged in.

* Removed RLS cache key from query_object.

* RLS added to the cache_key from query_context.

* Changes as per @etr2460.

* Updating entry for RLS pull request.

* Another migration to skip.

* Changes as per @serenajiang.

* Blacked.

* Blacked and added some attributes to check for.

* Changed to a manual query as per @mistercrunch.

* Blacked.

* Another migration in the meantime.

* Black wanted some whitespace changes.

* AttributeError: 'AnonymousUserMixin' object has no attribute 'id'.

* Oops, did hasattr backwards.

* Changes as per @mistercrunch.

* Doesn't look like text us required here anymore.

* Changes as per @dpgaspar

* Two RLS tests.

* Row level security is now disabled by default via the feature flag ENABLE_ROW_LEVEL_SECURITY.

* New head to revise.

* Changed the comment.
2020-02-22 11:21:31 +02:00
Daniel Vaz Gaspar e55fe43ca6
[database] Fix, tables API endpoint (#9144) 2020-02-20 10:15:22 +00:00
Will Barrett 38f3fd0c9f
Add feature flags to control query sharing, KV exposure (#9120)
* Add feature flags to control query sharing, KV exposure

* Add tests, fix bug

* Skip test for kv endpoints when they are disabled

* ESLint fixes

* Remove unnecessary binds

* Fix eslint errors

* Add note to UPDATING.md RE: new feature flag options

* Use expanded version of RBAC

* Enable KV_STORE and SHARE_QUERIES_VIA_KV_STORE feature flags in the test environment

* Fix black
2020-02-19 09:51:50 -08:00
Grace Guo 84b42d28b9
[dashboard] use filter_scopes metadata when import old dashboard (#9145)
* [dashboard] update filter_scopes metadata when import old dashboard

* fix review comments
2020-02-19 09:06:52 -08:00
Rob DiCiuccio 0992445ae9
Catch TypeError on PyArrow array instantiation (#9139) 2020-02-17 08:34:00 -08:00
John Bodley 9f7466ef90
[fix] Fix table viz column order (#9122) 2020-02-16 22:51:35 -08:00
John Bodley a7e433a512
[mypy] Disallowing implicit optional (#9150) 2020-02-16 22:34:15 -08:00
Daniel Vaz Gaspar f993bdc7ef
[database] new, select star API migration (#9054) 2020-02-14 14:30:49 +00:00
Craig Rueda 607cfd1f29
Moving away from using the root logger everywhere (#9099)
* Moving away from using the root logger everywhere

* self.logger -> logger
2020-02-07 23:38:48 -08:00
Rob DiCiuccio 161d211c07
SQL Lab: Use numpy structured arrays, fallback to JSON serialization (#9096)
* Use numpy structured arrays, fallback to JSON serialization

* Explicitly cast data as list when creating numpy array
2020-02-06 17:25:22 -08:00
John Bodley 8a138fbd03
[tox] Allowing running of specific tests (#9097) 2020-02-06 16:45:37 -08:00
ʈᵃᵢ e5f5eed425
[chart] new, list view (react) (#8999)
* [charts] new, list view (react)

* DRY up template rendering

* fix i18n

* lint package.json
2020-02-05 12:42:42 -08:00
Will Barrett 47b5780c5c
Wrap tagging endpoints in a feature flag (disabled by default) (#9078)
* Wrap tagging endpoints in a feature flag (disabled by default)

* Add tests for new feature flag

* Reuse existing feature flag

* Add TAGGING_SYSTEM to feature flags defaulted to False
2020-02-05 11:10:13 -08:00
Daniel Vaz Gaspar 916d184076
[query] deprecate can_only_access_owned_queries (#9046) 2020-02-05 08:58:11 +00:00
Will Barrett fc1c9428e3
Do not show stacktraces on some intentionally-thrown errors (#9056)
* Do not show stacktraces on some intentionally-thrown errors

* Fix pylint error

* Fix JavaScript looking for the wrong key in error response from connection check

* format -> f-string

* Run black because PyCharm missed it on the last save
2020-02-05 07:01:43 +02:00
John Bodley 848c565c8c
[sip-15] Enabling SIP-15 by default (#9017) 2020-02-03 10:37:29 -08:00
Daniel Vaz Gaspar d5dbfc886d
[database] new, API table metadata (#9002)
Migrates and completely removes: '/superset/table/<database_id>/<table_name>/<schema>/'
- Guarantees database filtering based on the user permissions
- Full OpenAPI spec
- Better error handling
2020-01-29 09:51:06 +00:00
Daniel Vaz Gaspar fb17acd122
[api] fix, set default columns to just id when not defined (#8982) 2020-01-29 09:41:40 +00:00
Rob DiCiuccio 2fc3d84b5a
Serialize nested columns as JSON strings (#9007) 2020-01-28 15:50:36 -08:00
John Bodley dc60db2a43
[fix] Reverting metic logic from #8901 (#9030) 2020-01-27 22:28:42 -08:00
Daniel Vaz Gaspar 1f21bf877f
[log] fix, log model view permissions (#8993)
Limit MVC access to admin role only and limit REST API permissions
2020-01-26 12:15:57 +00:00
Erik Ritter 68e85ab1b6 [Caching] Ensure cache is always created (#9020)
* [Caching] Ensure cache is always created

* Update cache_manager.py

* Refactoring cache typing
2020-01-25 22:49:05 +00:00
Maxime Beauchemin 315a11dfe2
fix: shut off unneeded endpoints (#8960)
* fix: shut off all uneeded endpoints

We recently added a new feature to FAB allowing to whitelist the needed
endpoints in ModelView and ModelRestApi.

First, we set our base wrapper class to an empty set, forcing each
class inheriting from it to explicitely turn on the endpoints that
Superset intends to use.

Second, we go ModelView by ModelView to whitelist the actual endpoints
used in the app.

Notes:
* as a result a large set of [unneeded] permissions should be cleaned up
* outside of the "private" use of endpoints in the app, people that have
  been using endpoints in their environment for other purposes may
  experience loss of functionality

* Tweaking

* Reduce the amount of endpoints using white lists

* Fix, included needed endpoints for dashboard and druid

* Drying things up

* fixes

* limiting more endpoints

* Read only on some FAB model views

* fixing some tests

* fixes

* Fixing more tests

* Addressing comments

* Drying up route_methods

* further drying

Co-authored-by: Daniel Vaz Gaspar <danielvazgaspar@gmail.com>
2020-01-23 11:25:15 -05:00
Daniel Vaz Gaspar d02cf2f509 [dashboards] New, API for Bulk delete (#8972)
* [dashboards] New, API for Bulk delete

* [dashboards] Tests

* [dashboards] Fix not found on multiple delete

* [dashboards] Bulk delete partial deletes on not owned dashes

* [dashboards] Improve OpenAPI spec and tests

* [dashboards] Test for bad request

* [dashboards] i18n

* [dashboards] black

* [dashboard] make bulk all or nothing

* [dashboard] Log on sqlalchemy error

* [dashboard] Log on sqlalchemy error
2020-01-21 17:14:53 -05:00
Daniel Vaz Gaspar 74158694c5 [charts] New, REST API (#8917)
* [charts] New REST API

* [charts] Small improvements

* [charts] Fix, lint

* [charts] Tests and datasource validation

* [charts] Fix, lint

* [charts] DRY post schemas

* [charts] lint and improve type declarations

* [charts] DRY owned REST APIs

* [charts] Small fixes

* [charts] More tests

* [charts] Tests and DRY

* [charts] Tests for update

* [charts] More tests

* [charts] Fix, isort

* [charts] DRY and improve quality

* [charts] DRY and more tests

* [charts] Refactor base for api and schemas

* [charts] Fix bug on partial updates for dashboards

* [charts] Fix missing apache license

* black app.py after merge

* [charts] Fix, missing imports and black

* [api] Log on sqlalchemy error

* [api] isort
2020-01-21 13:04:52 -05:00
Ville Brofeldt c087a48d52 fix: add datasource.changed_on to cache_key (#8901)
* Add datasource.changed_on to cache_key and add+fix related unit tests

* Add note to UPDATING.md

* Remove redundant comment about metric names
2020-01-16 00:51:13 -05:00
ʈᵃᵢ 7b97764dbc [dashboard] New, list view (react) (#8845)
* adds dashboard listview component

* use new api

* use json over rison

* lint

* adds seperate dashboard list view

* edit and delete actions

* fix lint ignore

* fix common_bootstrap_payload is now a function

* fix license

* fix pylint

* isort

* fix tests

* lint

* lint ts

* fix js tests

* fix double import from bad rebase

* fix indent error

* lookup permissions

* generic permission lookup

* get tslint to pass

* adds js specs

* lint

* fix rebase

* lint

* lint again

* fix type errors preventing build

* adds more specs

* fix tslint error

* fix null check

* remove unecessary code

* use translations provided by api

* more translations

* linting

* fix spec

* i18n

* fix register order
2020-01-16 00:48:55 -05:00
Daniel Vaz Gaspar 123246fca6
[dashboards] New, export api (#8941)
* [dashboards] Multiple exports

* [dashboards] Fix, mulexport permission missing

* [dashboards] Test for security filtered export

* [dashboards] Address PR comments
2020-01-15 18:09:55 +00:00
serenajiang 1f6f4ed879 [db migration] change datasources-clusters foreign key to cluster_id (#8576)
* [db migration] change datasources foreign key to cluster_id

* address pr comments

* address pr comment, fix ci
2020-01-13 11:02:36 -08:00
Rob DiCiuccio 7ff18f060c Ensure proper JSON serialization of numpy.ndarray (#8946) 2020-01-10 13:24:03 -05:00
John Bodley 2d456e88eb
[fix] Enforce the query result to contain a data-frame (#8935) 2020-01-08 11:50:26 -08:00
Daniel Vaz Gaspar 2726f21cbc
[dashboard] Fix, prevent delete and update on dashes not owned (#8911) 2020-01-06 15:29:39 +00:00
Rob DiCiuccio 6537d5ed8c Replace pandas.DataFrame with PyArrow.Table for nullable int typing (#8733)
* Use PyArrow Table for query result serialization

* Cleanup dev comments

* Additional cleanup

* WIP: tests

* Remove explicit dtype logic from db_engine_specs

* Remove obsolete  column property

* SupersetTable column types

* Port SupersetDataFrame methods to SupersetTable

* Add test for nullable boolean columns

* Support datetime values with timezone offsets

* Black formatting

* Pylint

* More linting/formatting

* Resolve issue with timezones not appearing in results

* Types

* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip

* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks

* Dataframe typing and nits

* Renames to reduce ambiguity
2020-01-03 11:55:39 -05:00
Craig Rueda 4ce5add5df Making test_config and main config the same (#8838) 2020-01-02 13:57:54 -05:00
Rob DiCiuccio 8fc814fc62 Enable running of tests in tests/db_engine_specs (#8902)
* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip
2019-12-31 14:45:33 +02:00
Ville Brofeldt 5b690f9411
chore: refactor, add typing and fix uncovered errors (#8900)
* Add type annotations and fix inconsistencies

* Address review comments

* Remove incorrect typing of jsonable obj
2019-12-31 09:26:23 +02:00
Daniel Vaz Gaspar 3d9181d270
[dashboard] New, get releated owners and slices (#8872) 2019-12-20 10:16:57 +00:00
David Aaron Suddjian 016f202423 Refactor Dashboard and Slice models (#8820)
* refactor dashboard and slice models

* appease various linters

* remove shortcuts & import indirection

* appease mypy

* fix bad imports

* lint

* address various issues

* ignore type issue

* remove unused imports

* lint
2019-12-18 11:40:45 -08:00
David Aaron Suddjian 7a68cb7ca0 factor out datasource_access_request model (#8809) 2019-12-17 16:17:49 -08:00
Daniel Vaz Gaspar 2f2ac00a09
[dashboard] feat: REST API (#8694) 2019-12-16 21:10:33 +00:00
Will Barrett b87f21fe1f Move function in utils folder to fix circular import issues (#8814)
* Move function in utils folder to fix circular import issues

* CI fixes - black, isort
2019-12-13 16:44:56 +00:00
Will Barrett ed54f6ec8d Require the SQLAlchemy URI on the database model (#8720)
* Require the SQLAlchemy URI when creating a database

* Add migration to make dbs.sqlalchemy_uri not-nullable

* Fixes for black, isort, tests

* Alter migration to use current revision from master as downgrade target

* Update tests to support new db constraint

* black
2019-12-11 14:05:23 -08:00
Bogdan e6be51953c Avoid circular dependency between superset config and superset package (#8254)
Resolve comments

Avoid circular dependency between superset config and superset package

Resolve comments
2019-12-10 09:17:34 +00:00
Will Barrett 7f8db7ab8d Re-enable pylint for common, connector, and test files (#8774)
* re-enable pylint for superset/common/query_context.py

* Re-enable pylint for superset/connectors/base/views.py

* re-enable pylint for superset/connectors/connector_registry.py

* Re-enable pylint for tests/fixtures/pyodbcRow.py

* re-enable pylint for superset/cli.py

* Fix for mypy

* Improve variable naming

* A few more naming fixes
2019-12-09 22:32:54 -08:00
Beto Dealmeida 5477f42740
Limit payload returned when using SQLLAB_BACKEND_PERSISTENCE (#8769)
* Limit queries

* WIP

* Adding unit test

* WIP

* Add unit test

* Fix test

* Try to fix remote unit tests

* Fix unit test

* Fix  for postgres
2019-12-09 16:12:40 -08:00
Dinesh Sawant cae0583e0f Support timeFormat extraction function column for Druid in Filter (#8728) 2019-12-07 17:54:37 -08:00
Hossein Torabi b053d66ea7 fix urlib opener bug (#8788) 2019-12-07 16:07:09 -08:00
ʈᵃᵢ a94464b9c9 guard against duplicate columns in datasource save (#8712)
* catch IntegrityError in datasource save

* catch duplicate columns and wrap in exception handling decorators

* use 409

* isort
2019-12-04 11:05:06 -08:00
John Bodley c74bb6cb9b
[fix] Updating bound filters for PyDruid 0.5.7 (#8742) 2019-12-03 17:44:04 -08:00
David Aaron Suddjian b0b93768bb handle nulls in bar chart series/breakdown (#8724)
* handle nulls in bar chart series/breakdown

* snake_case

* add tests for null groups in bar charts

* black

* comments will save us
2019-12-03 16:11:15 -08:00
Bogdan 003e98c231 Support schema permissions (#8219)
* Build support for schema access in Superset and SQLab

* Lint
* Test for old and new flask
* Black formatting
* Cleanup modified files
* Support schema permissions
* Add migration
* Make it work for slices as well
* TODO and remove unused field
* Apply schema access check to dashboards as well

Cleaner schema permissions check

Fetch schema permissions in 1 query

Bugfix

Better datasource check

Use security manager for permissions checks and remove dead code

Properly handle anon user

Add database permissions on the database updates

Fix schema filtering

Lint / flake changes

Increase cypress request timeout

Run black

Resolve multiple alembic heads

* Use set comprehensions

* Fixes for the pylint
2019-12-03 14:57:24 -08:00
Beto Dealmeida f8e15e1c55
Add protocol to cache warmup (#8721)
* Add protocol to cache warmup

* Update unit tests

* Fix wrong commit

* Update UPDATING.md

* Fix pylint
2019-12-03 10:51:06 -08:00
John Bodley 4e998be695
[fix] Druid IS NULL/IS NOT NULL filters (#8714) 2019-12-02 20:38:42 -08:00
Will Barrett 16c0599470 Disable deprecated druid connector by default (#8512)
* Disable deprecated druid connector by default

* Add a line in UPDATING.md for the configuration change

* Remove security tests related default-disabled feature

* More test updates

* black
2019-11-27 11:28:23 -08:00
Craig Rueda df2ee5cbcb Adding app context wrapper to Celery tasks (#8653)
* Adding app context wrapper to Celery tasks
2019-11-27 15:06:06 +00:00
Beto Dealmeida 5f4e3adfd2
Pass full response to `query_cost_formatter` (#8652)
* Return full info when doing query cost estimation

* Add unit test

* Fix isort
2019-11-26 11:49:19 -08:00
Ville Brofeldt 7104b04817
Pass url parameters from dashboard to charts (#8536)
* Pass url_params from dashboard to charts

* Update params to form_data instead of overwriting

* Add cypress tests

* Add python test

* Add docs

* Move reserved url params to utils

* Bump cypress
2019-11-21 07:07:08 +02:00
Craig Rueda e490414484 Flask App factory PR #1 (#8418)
* First cut at app factory

* Setting things back to master

* Working with new FLASK_APP

* Still need to refactor Celery

* CLI mostly working

* Working on unit tests

* Moving cli stuff around a bit

* Removing get in config

* Defaulting test config

* Adding flask-testing

* flask-testing casing

* resultsbackend property bug

* Fixing up cli

* Quick fix for KV api

* Working on save slice

* Fixed core_tests

* Fixed utils_tests

* Most tests working - still need to dig into remaining app_context issue in tests

* All tests passing locally - need to update code comments

* Fixing dashboard tests again

* Blacking

* Sorting imports

* linting

* removing envvar mangling

* blacking

* Fixing unit tests

* isorting

* licensing

* fixing mysql tests

* fixing cypress?

* fixing .flaskenv

* fixing test app_ctx

* fixing cypress

* moving manifest processor around

* moving results backend manager around

* Cleaning up __init__ a bit more

* Addressing PR comments

* Addressing PR comments

* Blacking

* Fixes for running celery worker

* Tuning isort

* Blacking
2019-11-20 15:47:06 +00:00
ʈᵃᵢ 9d36fa348f Move from MVC FAB menu to Superset React menu (#8289)
* adds react based menu for welcome view

* bump fab

* better logic in common_bootstrap_payload

* split up files

* fix spacing and module scope styles

* lint

* render correct template for welcome
2019-11-19 22:56:27 -08:00
Beto Dealmeida 3b97ae3b9d
Set owner when creating SQL Lab view (#8597)
* Set owner when creating SQL Lab view

* Add unit test

* Small fix
2019-11-19 10:48:42 -08:00
Beto Dealmeida 78f41b38e1
Fix more places where feature flags are serialized (#8567) 2019-11-14 10:01:57 -08:00
Beto Dealmeida d66bc5ad90
SIP-23: Persist SQL Lab state in the backend (#8060)
* Squash all commits from VIZ-689

* Fix javascript

* Fix black

* WIP fixing javascript

* Add feature flag SQLLAB_BACKEND_PERSISTENCE

* Use feature flag

* Small fix

* Fix lint

* Fix setQueryEditorSql

* Improve unit tests

* Add unit tests for backend sync

* Rename results to description in table_schema

* Add integration tests

* Fix black

* Migrate query history

* Handle no results backend

* Small improvement

* Address comments

* Store SQL directly instead of reference to query

* Small fixes

* Fix clone tab

* Fix remove query

* Cascade delete

* Cascade deletes

* Fix tab closing

* Small fixes

* Small fix

* Fix error when deleting tab

* Catch 404 when tab is deleted

* Remove tables from state on tab close

* Add index, autoincrement and cascade

* Prevent duplicate table schemas

* Fix mapStateToProps

* Fix lint

* Fix head

* Fix javascript

* Fix mypy

* Fix isort

* Fix javascript

* Fix merge

* Fix heads

* Fix heads

* Fix displayLimit

* Recreate migration script trying to fix heads

* Fix heads
2019-11-14 09:44:57 -08:00
Ville Brofeldt 90275fe991
Add convert_dttm method to SnowflakeEngineSpec (#8551) 2019-11-13 11:13:49 +02:00
Beto Dealmeida a58b3920db
Encode feature flags to JSON pessimistically (#8529)
* Encode feature flags to JSON pessimistically

* Add unit test

* Remove old imports
2019-11-12 11:10:36 -08:00
Daniel Vaz Gaspar 650c848243
[cli] Fix, import datasources exported by UI (#8477)
* [cli] Fix, import datasources exported by UI
2019-11-10 16:29:51 +00:00
Ville Brofeldt 49ea232c3a
[fix] Improve csv upload functionality (#8457)
* [fix] csv upload when table metadata present

* Remove table from hive spec

* Move upload before table metadata creation

* Refine upload logic, dd unit tests and fix translations

* Use ALLOWED_EXTENSIONS from config

* Address review comments

* Fix error message grammar

* Add return type to hive csv upload and replace first with one_or_none
2019-11-07 20:03:42 +02:00
John Bodley 1e3d833b02
[sip-15] Fixing time range endpoints from dashboards (#8513) 2019-11-06 09:15:58 -08:00
John Bodley 5dba2f4b56
[SIP-15] Adding grace period (#8490) 2019-11-04 12:00:41 -08:00
John Bodley eb150eba1f
[sip-15] Fix time range endpoints encoding (#8481) 2019-10-31 07:11:48 -07:00
Will Barrett e4ca44e95f Use `config[]` not `config.get()` (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
Kim Truong 3cba1bd990
feat: add expand_data parameter (#8472)
* feat: add expand_data parameter

* fix: reformat files
2019-10-30 11:45:35 -07:00
John Bodley 0a3b121244
[sql] Fixing datetime SQL literal (#8464) 2019-10-29 23:24:48 -07:00
Beto Dealmeida e704e29174
Allow fetching all rows from results endpoint (#8389)
* Allow bypassing DISPLAY_MAX_ROW

* Add unit tests and docs

* Fix tests

* Fix mock

* Fix unit test

* Revert config change after test

* Change behavior

* Address comments
2019-10-25 10:22:16 -07:00
Will Barrett 82b174701f Split up tests/db_engine_test.py (#8449)
* Split up db_engine_specs_test.py into a number of targeted files

* Remove db_engine_specs_test.py

* isort
2019-10-24 20:46:45 -07:00
John Bodley 786d7706e7
[fix] Updating parse_human_timedelta typing (#8436) 2019-10-23 16:04:46 -07:00
Marcus 4cfd6b3ce1 [tests] Fix, removing deprecated function for future Python release compatibility. (#8353)
* removing deprecated function for future Python release compatibility.

* also removed assertNotEquals per @dpgaspar
2019-10-21 15:49:12 +01:00
John Bodley 9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
serenajiang 876d329474 [fix] make names non-nullable (#8371) 2019-10-15 16:51:04 -07:00
serenajiang 7e7ea3d9a0 Revert "[fix] make datasource names non-nullable (#8332)" (#8363)
This reverts commit 65a05ca47e.
2019-10-09 11:38:57 -07:00
Beto Dealmeida 04c3d1f58d
Remove unused methods from Presto (#8350)
* Remove unused methods from Presto

* Remove tests

* Fix bug in PRESTO_SPLIT_VIEWS_FROM_TABLES

* Fix lint
2019-10-04 11:22:02 -07:00
Will Barrett ec86d9de17 Fix lint in `superset/db_engine_spec` (#8338)
* Enable lint checking for files in db_engine_spec that have few to no
lint issues

* Enable lint and fix issue in db_engine_spec/mysql.py

* Enable pylint and fix lint for db_engine_spec/pinot.py

* Enable lint and fix issues for db_engine_specs/hive.py

* Enable lint and fix for db_engine_spec/presto.py

* Re-enable lint on base.py, fix/disable specific failures, including one
bad method signature

* Make flake8 happy after a number of pylint fixes

* Update db_engine_spec_test test cases related to Presto to support
different method naming

* automated reformatting

* One more pylint disable for druid.py

* Find the magic invocation that makes all the lint tools happy
2019-10-04 09:19:21 -07:00
serenajiang 65a05ca47e [fix] make datasource names non-nullable (#8332) 2019-10-04 09:13:25 -07:00
John Bodley 05b67673c3
[metric] Adding security for restricted metrics (#8175) 2019-10-01 16:52:15 -07:00
John Bodley 5d1bf4245d
[sql_json] Ensuring the request body is JSON encoded (#8256) 2019-09-23 09:09:12 -07:00
Ben bc83b5fbe6 More intuitive display of negative time deltas (#8274) (#8276)
* WIP - Add easily interpretable negative timedelta formatting

* Add unit tests for utils.core.timedelta_f

* Reformat tests.utils_tests.test_timedelta_f test code. Fixes #8274.

* Add type annotations to utils.core.timedelta_f (#8274)

* Replace recursion with absolute value in utils.core.timedelta_f (#8274)

* Rename utils.core.timedelta_f to format_timedelta (#8274)

* Fix small typo (#8274)
2019-09-22 19:37:43 +03:00
Beto Dealmeida 7090725de9
Fix no data in Presto (#8268)
* Fix no data in Presto

* Fix test
2019-09-20 14:31:13 +02:00
Beto Dealmeida 6df74c6b8e
Remove __row_id (#8257) 2019-09-18 20:32:33 -07:00
Beto Dealmeida 12fb8e70cc
Show Presto views as views, not tables (#8243)
* WIP

* Implement views in Presto

* Clean up

* Fix CSS

* Fix unit tests

* Add types to database

* Fix circular import
2019-09-18 12:47:10 -07:00
Beto Dealmeida 4088a84eb4
Small fix for Presto dtype map (#8251)
* Small fix for Presto dtype map

* Add unit test
2019-09-18 12:46:50 -07:00
1AB9502 0cd4ef54f8 Add RegisteredLookupExtraction support to extraction function (#8185)
* Add RegisteredLookupExtraction support to extraction function

* Fix formatting issues

* Reformat druid_func_tests through black
2019-09-17 23:45:47 -07:00
Beto Dealmeida 88777943fa
Better distinction between tables and views, and show CREATE VIEW (#8213)
* WIP

* Add missing file

* WIP

* Clean up

* Use label instead

* Address comments

* Add docstring

* Fix lint

* Fix typo

* Fix unit test
2019-09-17 14:24:38 -07:00
Beto Dealmeida 4132d8fb0f
Simplify and improve `expand_data` in Presto (#8233)
* WIP

* Working version, needs cleanup

* Add unit tests to split

* Small fixes

* Dedupe array columns

* Fix lint
2019-09-17 14:04:14 -07:00
Beto Dealmeida b9be01fcd8
Handle int64 columns with missing data in SQL Lab (#8226)
* Handle int64 columns with missing data in SQL Lab

* Fix docstring

* Add unit test

* Small fix

* Small fixes

* Fix cursor description update

* Better fix

* Fix unit test, black

* Fix nan comparison in unit test
2019-09-17 08:16:09 -07:00
serenajiang 4e2d1c1a62 more detailed async error messages (#8164) 2019-09-09 09:09:15 -07:00
Maxime Beauchemin 68c4c3a0b9
Prevent 'main' database connection creation (#8038)
* prevent 'main' database connection creation

* fix tests

* removing get_main_database

* Kill get_main_database

* Point to examples tables
2019-09-08 10:18:09 -07:00
Ville Brofeldt 3250c5ac94
[bugfix] fix timegrain addon regression (#8165)
* Fix regression in time grain addons

* Revert privatization of time_grain_functions

* Fix test

* Rename variable

* Fix test

* Fix typing error

* Refactor and add tests

* Add TODO
2019-09-08 08:34:40 +02:00
Ville Brofeldt 4e1e54b538
[bugfix] Correctly quote table and schema in select_star (#8181)
* Fix select_star table quoting bug

* Add unit test for fully qualified names in select_star

* Rename main_db to db

* Rename test function
2019-09-05 22:44:34 +03:00
Rob DiCiuccio 7595d9e5fd [SQL Lab] Async query results serialization with MessagePack and PyArrow (#8069)
* Add support for msgpack results_backend serialization

* Serialize DataFrame with PyArrow rather than JSON

* Adjust dependencies, de-lint

* Add tests for (de)serialization methods

* Add MessagePack config info to Installation docs

* Enable msgpack/arrow serialization by default

* [Fix] Prevent msgpack serialization on synchronous queries

* Add type annotations
2019-08-27 14:23:40 -07:00
Ville Brofeldt 1982b74af2 Add check for calls to cache_key_wrapper (#8128)
* Add check for calls to cache_key_wrapper to avoid unavoidable compilation of query

* Add fetch_values_predicate to check

* Only check relevant attributes

* Address nit
2019-08-27 13:36:05 -07:00
Dave Smith 6dc760a054 Add LogConfigurator, and add logging for CSV export (#8085)
* Add LogConfigurator, and add logging for CSV export

* tweak for PR feedback

* fix linter issue
2019-08-27 13:22:18 -07:00
Ville Brofeldt fb51632e18
Add docstrings and typing to db_engine_specs and sql_parse (#8058)
* Add typing to db_engine_specs

* Add more type annotations and docstrings

* Add docstrings and typing to sql_parse and db_engine_specs

* Refine select_star

* Fix execute and add more docstrings

* Revert kwargs change from execute

* Remove redundant or

* Align view and table getter schema types

* Fix return type of latest_partition

* Remove some typing from presto

* Improve docstring for __extract_from_token
2019-08-22 06:29:32 +03:00
Beto Dealmeida aae9b56232 Move more Presto nested types behind feature flag (#8075)
* Move more stuff behind ff

* Fix unit tests

* Fix more tests

* Fix lint
2019-08-20 16:10:55 -07:00
Maxime Beauchemin 4e7ea3f3e2
Improve birth example dashboard (#8042)
* Improve birth example dashboard

* Fix tests

* pylint
2019-08-18 22:36:27 -07:00
Beto Dealmeida 478d0969a8
Add feature flag for Presto expand data (#8056)
* Add feature flag for Presto expand data

* Fix unit tests

* Fix black

* Revert temporary file change
2019-08-15 20:10:05 -07:00
serenajiang 17f0740692 Fix bug where lists in queries cannot contain multiple types (#8032)
* use set instead of sorted to check equality of lists

* run black
2019-08-13 23:06:44 +03:00
Dave Smith 9233a63a16 Event logger config takes instance instead of class (#7997)
* allow preconfigured event logger instance; deprecate specifying class

* add func docs and simplify conditions

* modify docs to reflect EVENT_LOGGER cfg change

* commit black formatting fixes and license header

* add type checking, fix other pre-commit failues

* remove superfluous/wordy condition

* fix flake8 failure

* fix new black failure

* dedent warning msg; use f-strings
2019-08-08 13:47:18 -07:00
Daniel Vaz Gaspar 73cdad2375 [SQLLab] Fix, database api unlimited page size v2 (#8002)
* [database] Fix, Removes the limit for the page size, Bump FAB to 2.1.8

Old FAB API had no limits by default, this will keep this behaviour
but only for this endpoint

* [sqllab] Add test for database API

* [sqllab] Add test for database API

* [sqllab] Include page zero on request

* [sqllab] Fix, Black and requirements

* [sqllab] Make database API return unlimited results

* [sqllab] just a test

* [sqllab] Bump FAB to 2.1.9

* [sqllab] Remove unused import
2019-08-08 11:37:00 -07:00
John Bodley fdb62e9865
[flake8] Ignoring I202 (#7988) 2019-08-05 15:01:33 -07:00
Erik Ritter d58dbad076 [DB Engine] Support old and new Presto syntax (#7977) 2019-08-05 11:56:56 -07:00
John Bodley f7af50c399
[security] Adding docstrings and type hints (#7952) 2019-08-05 11:24:13 -07:00
Gianluca Ciccarelli e23920b8ba 7620: Start removing dependencies on requests (#7643)
* 7620: Start removing dependencies on requests

* Patch urllib.request.urlopen instead of requests.get

* Try to fix flake8

* More work on flake8 import errors

* First attempt at using urllib with cookies

* Fix pylint/flake8

* Fix test_deliver_slice_csv_attachment

* Fix test_deliver_slice_csv_inline

* Import requests and pydruid conditionally, remove dependency on prison

* Fix flake errors

* Fix load_examples

* Please flake

* Skip tests depending on optional deps

* Try to please flake

* Address review comments

* Remove Druid-related UI

* Revert "Remove Druid-related UI"

This reverts commit d7e0f166cc3f3dd2496b4a666e177f0c191aeb0f.

* Skip a few tests more

* Put imports in right order

* Apply black patch

* Please flake

* Please black, silence flake

* Use flake8 silencing the right way

* Add deps for CI
2019-08-02 10:01:28 -07:00
John Bodley af462fe79f
[revert] Partial revert of #7888 (#7933) 2019-07-29 09:19:21 -07:00
Daniel Vaz Gaspar 994ac04c1f [cli] New, command line option to create or set a db URI (#7918)
* [cli] New, command line option to create or set a db URI

* [tests] New, get or create db on utils

* [tests] flake8 and black

* [tests] Fix, remove dummy comments

* [cli] dummy commit to re-trigger build

* [cli] dummy commit to re-trigger build
2019-07-27 17:43:39 -07:00
Ville Brofeldt 72d1011023
[sqllab] Fix limit parsing bug when using limit-offset comma notation (#7912)
* Fix limit parsing bug when using limit-offset comma notation

* Use native sqlparse semantics to find limit

* black
2019-07-24 08:18:39 +03:00
Ville Brofeldt 07a76f83b1
[Bugfix] Remove prequery properties from query_obj (#7896)
* Create query_obj for every filter

* Deprecate is_prequery and prequeries from query_obj

* Fix tests

* Fix typos and remove redundant ; from sql

* Add typing to namedtuples and move all query str logic to one place

* Fix unit test
2019-07-23 22:13:58 +03:00
Maxim Sukharev 648f9fa54b Set owner to dashboards and charts on import (#7894)
* Allow to pass user for dashboard import cli

Dashboard import assign current user from flask context during import.
But in case of cli import there is no flask user and imported charts
don't have an owner which prevents ability to edit them.

* Reset ownership on dashboard import

For overriding existing charts it requires `owners` property to be set.

* Add tests for reset ownership

* Use ORM to decode dashboards json

Creating instances using ORM allows to normally work with relations

* Fix test_import_dashboard_1_slice test

Previously tests used side-effect of slices import which kept id from
json on insert into db.
2019-07-22 22:30:28 -07:00
Daniel Vaz Gaspar b65ab51b63 [api] [database] New, migrate to new FAB API side by side (#7665)
* [api] [database] New, migrate to new FAB API side by side

* [database] view and api module reorganization

* [style] Fix, flake8

* [test] [database] Fix, tests for new database view structure

* [style] [database] black

* [database] Fix, missing apache license

* Migrate databaasync api to the new api

* flake8

* More efficient api call

* Revert querySearch to old API, test

* dummy commit

* revert config.py commit

* Remove unused import

* Remove old API view

* Remove new API view

* Add database filter

* Add database filter

* Add database filter

* Remove comments on js

* Fix JS test

* Fix pylint

* QuerySearch new databaseasync API endpoint

* Fix, query search new api endpoint

* Reintroduce old API for smooth side by side migration

* Better naming and use common hooks for view and API

* black it

* Fix, database mixin must come first

* Fix, lint
2019-07-22 21:25:16 -07:00
Arpit ee3430cce5 [Bugfix] Response header and response mismatch on explore result from sqllab (#7907)
* fix(sql-lab explore):   fixed response data and header type to be aligned

* fix(sql-lab explore):   fixed test error caused by incorrect test at master

* fix(sql-lab explore):   reformatted

* fix(sql-lab explore):   removed unused import
2019-07-22 08:47:34 +03:00
Ville Brofeldt 4568b2a532 Add cache_key_wrapper to Jinja template processor (#7816) 2019-07-20 09:12:35 -07:00
John Bodley ccedbea506
[fix] Adding space after -- for SQL comments (#7897) 2019-07-18 13:17:26 -07:00
John Bodley 2b3e7fe4de
[sqla] Adding check for invalid filter columns (#7888) 2019-07-18 08:51:41 -07:00
Maxim Sukharev b512502d72 Remove unnecessary fields from dashboard exported json (#7879)
* Remove unnecessary fields from dashboard exported json

This commit makes export respect export_fields and doesn't export
unnecessary relations (like users with passwords hashes) which are
ignored during the import.

* Allow to import dashboard without position_json

In case charts were added from chart-edit page and wasn't re-ordered on
the dashboard position_json field is empty and import was failing with
error.

* Update export/import tests
2019-07-17 22:22:10 -07:00
Maxime Beauchemin d65b039219
Improve examples & related tests (#7773)
* [WiP] improve load_examples

related to #7472, longer term we will generate the examples by exporting
them into tarball as in #7472. In the meantime, we need this subset of
the features:

* allowing specifying an alternate database connection for examples
* allowing a --only-metadata flag to `load_examples` to load only
  dashboard and chart definitions, no actual data is loaded

* Improve logging

* Rename data->examples

* Load only if not exist

* By default do not load, add a force flag

* fix build

* set published to true
2019-07-16 21:36:56 -07:00
Rob DiCiuccio df051813d5 Adds the ability to replace/extend caching backend (#7856)
* Add ability to override cache backend with custom module

* Tests for setup_cache

* Add custom CACHE_CONFIG documentation

* Fix linter errors and documentation link

* Fix black formatting errors
2019-07-12 14:06:56 -07:00
Maxime Beauchemin 87f50ba0d8
Make SHOW_STACKTRACE config apply across the board (#7845) 2019-07-12 08:58:44 -07:00
Tresdon Jones 97ffb762d0 Add "Published" feature to dashboards (#4725)
* Allow users to publish dashboards

* Rework publish dashboards feature

- The eye next to the title has been replaced with a [draft] badge
- Published status is now toggled in the Header Action Dropdown
- CRUD list shows published status

* Fix linter errors

* Update javascript tests

* Add tests and change DashboardFilter

Add some tests to make sure the published status is rendered and
Make it so that users cannot see dashboards that are published
if they don't have access to any of the slices within

* Fix some linter errors

* Remove commas from core.py

* Fix some failing tests

* More linter errors I introduced

* Fix more linter errors I introduced

* update alembic migration

* Update design of publish dash feature

* Upgrade migration version

* Secure publish endpoint

* Remove bad quotes

* Give publish span its own style

* fix publish rendering

* Add new test for publish feature

* Update migration

* update slug in test

* Update migration

* Address reviwer comments

* Fix linter errors

* Add licenses

* Remove fetchPublished(), use bootstrap data

* Update migration

* Update croniter to existing version

* Fix linter errors

* Upgrade DB Revisions

* Fix flake8 linter error

* Set all dashboards to published on migration

* Migration proper line spacing

* Fix migration to work with postgres

* UPDATE statement works with postgresql and sqlite hopefully

* Update wording to kick off travis
2019-07-10 23:14:13 -07:00
Maxime Beauchemin ab8af536b0
Fixing the examples (#7376)
Fixing a few failing examples, examples were referencing metrics that
were getting auto-generated in the past (like `sum__num`),
that we now prevent from getting created.
2019-07-04 13:23:02 -07:00
John Bodley 9dac805e7e
[schema] Deprecating the table_columns.database_expression column (#7653) 2019-07-03 14:05:59 -07:00
John Bodley 6d7701e684
[fix] Updating Pandas resample logic (#7799) 2019-07-02 11:10:50 -07:00
John Bodley ffa7c617ad
[travis] Adding Black tox environment (#7809) 2019-07-01 18:05:15 -07:00
John Bodley e0d040c377
[fix] Handling of non-existent datasource (#7755) 2019-07-01 11:55:25 -07:00
Kim Truong 5994e432d4 feat: query based on all partitions (#7782)
* feat: query based on all partitions

* fix: address Beto's comments
2019-06-27 11:19:55 -07:00
John Bodley 5c58fd1802
[format] Using Black (#7769) 2019-06-25 13:34:48 -07:00
Maxime Beauchemin 859d6e7c6a
chore: taking db dependencies out of requirements-dev.txt (#7605)
* chore: taking db dependencies out of requirements-dev.txt

The deps on mysqlclient and psycopg2

* Fix unit tests

* fix tox.ini

* fix tests
2019-06-23 22:37:41 -07:00
Ville Brofeldt 1c4092c61c Add csv upload support for BigQuery (#7756)
* Add extra_require for bigquery to setup.py

* Refactor df_to_db and add df upload capability for BigQuery

* Fix unit tests and clarify kwarg logic

* Fix flake8 errors

* Add minimum versions for bigquery dependencies

* wrap to_gbq in try-catch block and raise error if pandas-gbq is missing

* Fix linting error and make error more generic
2019-06-23 14:20:09 -07:00
michellethomas 7a575ce20a
Fixing time comparison to look for past deltas (#7616)
* Fixing time comparison to look for past deltas

* Adding note to updating about time_compare
2019-06-19 10:10:18 -07:00
Ville Brofeldt 95291facff
Break up db_engine_specs (#7676)
* Refactor db_engine_specs into package

* Rename bigquery class and add epoch funcs

* Fix flake8 errors

* Dynamically load all engine specs

* Fix linting errors and unit tests

* Implement Snowflake epoch time funcs

* Implement Teradata epoch time func

* Fix presto datasource query and remove unused import

* Fix broken datasource query

* Add mypy ignore for false positive

* Add missing license files

* Make create_time_grains_tuple public

* Fix flake8 quote

* Fix incorrect licence header
2019-06-08 11:27:13 -07:00
Erik Ritter f7812a3f27 [SQL Lab] Clarify SQL Lab query and display limits (#7641) 2019-06-07 17:16:38 -07:00
John Bodley 45b41aadcc
[fix] SQL parsing of table names (#7490) 2019-06-03 11:07:57 -07:00
Jim Horng 78c1674dc7 make filter_values() macro to properly accept value from filter box with single value setting (#7494) 2019-06-03 09:54:07 -07:00
Maxime Beauchemin 45b9880516
fix: SqlaColumn.type overflow on mysql (#7606)
* fix: SqlaColumn.type overflow on mysql

Hitting a new error when loading examples around a datatype that exceeds
the 32 chars limit on SqlaColumn.type

Type includes the 'COLLATE utf8mb4_general_ci' suffix which is too
verbose and not needed in that context.

* fix tests
2019-06-03 09:52:25 -07:00
Maxime Beauchemin 4b5931f637
Alternative fix for #7559 (#7575)
* Alternative fix for #7559

Just an idea...

* logging
2019-06-01 09:21:35 -07:00
Erik Ritter 722043c672 Allow trailing spaces in simple filter values (#7617) 2019-05-31 17:06:41 -07:00
Kim Truong 9acafd5b75 fix: handle presto columns with whitespace (#7630) 2019-05-31 13:38:05 -07:00
Kim Truong d2967340d9 View Presto row and array objects clearly in the data grid (#7625)
* feat: rough check in for Presto rows and arrays

* fix: presto arrays

* fix: return selected and expanded columns

* fix: add helper methods and unit tests

* fix: only allow exploration of selected columns

* fix: address Beto's comments and add more unit tests
2019-05-31 11:25:07 -07:00
Ville Brofeldt c1712e5d10
Add relative start param for time filters (#7525)
* Add relative start param for time filters

* Fix typo and add types to parse_human_datetime

* Add relative start/end to query_object

* Fix linting error
2019-05-31 16:55:26 +03:00
Ville Brofeldt 34407e8962
Make timestamp expression native SQLAlchemy element (#7131)
* Add native sqla component for time expressions

* Add unit tests and remove old tests

* Remove redundant _grains_dict method

* Clarify time_grain logic

* Add docstrings and typing

* Fix flake8 errors

* Add missing typings

* Rename to TimestampExpression

* Remove redundant tests

* Fix broken reference to db.database_name due to refactor
2019-05-30 08:28:37 +03:00
Maxime Beauchemin fd5befee38
Bump python libs, address insecure releases (#7550)
* Bump python libs, address insecure releases

Using
https://requires.io/github/mistercrunch/superset/requirements/?branch=apache_release_improv
to identify insecure, old releases we're using and bumping.

* redis <3.0

* fix new flakes
2019-05-28 21:03:49 -07:00
Ville Brofeldt f7d3413a50
Add support for period character in table names (#7453)
* Move schema name handling in table names from frontend to backend

* Rename all_schema_names to get_all_schema_names

* Fix js errors

* Fix additional js linting errors

* Refactor datasource getters and fix linting errors

* Update js unit tests

* Add python unit test for get_table_names method

* Add python unit test for get_table_names method

* Fix js linting error
2019-05-26 06:13:16 +03:00
John Bodley 6b9790c8b1
[ad-hoc] Fixing type for count distinct (#7573) 2019-05-22 09:41:25 -07:00
Daniel Vaz Gaspar 74704f68c7 [security] New, deprecate merge_perm, FAB method is fixed (#7355)
* [security] New, deprecate merge_perm, FAB method is fixed

* [style] Fix, flakes

* [tests] Fix, change merge_perm to add_permission_view_menu

* [security] Fix, maintain merge_perm for compatibility

* [security] New, deprecation warning on merge_perm method

* [style] Fix, flake8 C812
2019-05-20 17:30:00 -07:00
Maxime Beauchemin d8be0a7dd5
Break line before LIMIT statement to prevent trailing comment issue (#7485)
* Break line before LIMIT statement to prevent trailing comment issue

This may not be a perfect solution but it addresses the issue in 7483

closes https://github.com/apache/incubator-superset/issues/7483

* fix tests
2019-05-13 00:34:34 -05:00
Ville Brofeldt 959c35d506 bugfix: Improve support for special characters in schema and table names (#7297)
* Bugfix to SQL Lab to support tables and schemas with characters that require quoting

* Remove debugging prints

* Add uri encoding to secondary tables call

* Quote schema names for presto

* Quote selected_schema on Snowflake, MySQL and Hive

* Remove redundant parens

* Add python unit tests

* Add js unit test

* Fix flake8 linting error
2019-05-08 00:37:44 -05:00
Alex Berghage 24970485cf feat: Add `validate_sql_json` endpoint for checking that a given sql query is valid for the chosen database (#7422) (#7462)
merge from lyft-release-sp8 to master
2019-05-06 10:21:02 -07:00
Kim Truong 88e6ec992c feat: view presto row objects in data grid (#7445)
* Merge lastest from master into lyft-release-sp8 (#7405)

* filter out all nan series (#7313)

* improve not rich tooltip (#7345)

* Create issue_label_bot.yaml (#7341)

* fix: do not save colors without a color scheme (#7347)

* [wtforms] Strip leading/trailing whitespace (#7084)

* [schema] Updating the datasources schema (#5451)

* limit tables/views returned if schema is not provided (#7358)

* limit tables/views returned if schema is not provided

* fix typo

* improve code performance

* handle the case when table name or view name does not present a schema

* Add type anno (#7342)

* Updated local dev instructions to include missing step

* First pass at type annotations

* [schema] Updating the base column schema (#5452)

* Update 937d04c16b64_update_datasources.py (#7361)

* Feature flag for client cache (#7348)

* Feature flag for client cache

* Fix integration test

* Revert "Fix integration test"

This reverts commit 58434ab98a.

* Feature flag for client cache

* Fix integration tests

* Add feature flag to config.py

* Add another feature check

* Fix more integration tests

* Fix raw HTML in SliceAdder (#7338)

* remove backendSync.json (#7331)

* [bubbles] issue when using duplicated metrics (#7087)

* SUPERSET-7: Docker compose config version breaks on Ubuntu 16.04 (#7359)

* SUPERSET-8: Update text in docs copyright footer (#7360)

* SUPERSET-7: Docker compose config version breaks on Ubuntu 16.04

* SUPERSET-8: Extra text in docs copyright footer

* [schema] Adding commits and removing unnecessary foreign-key definitions (#7371)

*  Store last selected dashboard in sessionStorage (#7181)

* Store last selected dashboard in sessionStorage

* Fix tests

* [schema] Updating the base metric schema (#5453)

* Fix NoneType bug & fill the test recipients with original recipients if empty (#7365)

* feat: see Presto row and array data types (#7391)

* feat: see Presto row and array data types

* fix: address PR comments

* fix: lint and build issues

* fix: add types

* Incorporate feedback from initial PR (prematurely merged to lyft-release-sp8) (#7415)

* add stronger type hints where possible

* fix: lint issues and add select_star func in Hive

* add missing pkg init

* fix: build issues

* fix: pylint issues

* fix: use logging instead of print

* feat: view presto row objects in data grid

* fix: address feedback

* fix: spacing
2019-05-03 16:29:57 -07:00
John Bodley fb627ba376
[fix] Fixing SQL parsing issue (#7374) 2019-05-01 22:07:01 -07:00
Dave Smith a6aabf8268 feature: see Presto row and array data types (#7413)
* Merge lastest from master into lyft-release-sp8 (#7405)

* filter out all nan series (#7313)

* improve not rich tooltip (#7345)

* Create issue_label_bot.yaml (#7341)

* fix: do not save colors without a color scheme (#7347)

* [wtforms] Strip leading/trailing whitespace (#7084)

* [schema] Updating the datasources schema (#5451)

* limit tables/views returned if schema is not provided (#7358)

* limit tables/views returned if schema is not provided

* fix typo

* improve code performance

* handle the case when table name or view name does not present a schema

* Add type anno (#7342)

* Updated local dev instructions to include missing step

* First pass at type annotations

* [schema] Updating the base column schema (#5452)

* Update 937d04c16b64_update_datasources.py (#7361)

* Feature flag for client cache (#7348)

* Feature flag for client cache

* Fix integration test

* Revert "Fix integration test"

This reverts commit 58434ab98a.

* Feature flag for client cache

* Fix integration tests

* Add feature flag to config.py

* Add another feature check

* Fix more integration tests

* Fix raw HTML in SliceAdder (#7338)

* remove backendSync.json (#7331)

* [bubbles] issue when using duplicated metrics (#7087)

* SUPERSET-7: Docker compose config version breaks on Ubuntu 16.04 (#7359)

* SUPERSET-8: Update text in docs copyright footer (#7360)

* SUPERSET-7: Docker compose config version breaks on Ubuntu 16.04

* SUPERSET-8: Extra text in docs copyright footer

* [schema] Adding commits and removing unnecessary foreign-key definitions (#7371)

*  Store last selected dashboard in sessionStorage (#7181)

* Store last selected dashboard in sessionStorage

* Fix tests

* [schema] Updating the base metric schema (#5453)

* Fix NoneType bug & fill the test recipients with original recipients if empty (#7365)

* feat: see Presto row and array data types (#7391)

* feat: see Presto row and array data types

* fix: address PR comments

* fix: lint and build issues

* fix: add types

* add stronger type hints where possible

* fix: lint issues and add select_star func in Hive

* add missing pkg init

* fix: build issues

* fix: pylint issues

* fix: use logging instead of print
2019-05-01 09:17:34 -07:00
Daniel Vaz Gaspar 06c4610e8e Bump FAB to 2.0.0 (#7323)
* Bump FAB to 2.0.0

* [tests] whitelist SecurityApi login and refresh endpoints

* [style] Fix, C812 missing trailing commas

* [security] Remove SUPERSET_UPDATE_PERMS flag

Registering sources needs to be performed after the views are
initialized on UPDATE_PERMS=False configuration

* [docs] New, FAB_UPDATE_PERMS and flask fab cli

* [docs] Fix, db upgrade needs to come first, create-admin needs a db

* [cli] New, superset init bootstraps all permissions for FAB and Superset

* [style] Fix, flakes
2019-04-30 09:01:18 -07:00
John Bodley f488a3281d
[schema] Updating the base metric schema (#5453) 2019-04-24 21:50:14 -07:00
John Bodley 1388452169
[schema] Updating the base column schema (#5452) 2019-04-23 18:06:21 -07:00
Maxime Beauchemin 36a219da7f
Allow Gamma and Alpha to access '/users/userinfo/' (#6936)
* Allow Gamma and Alpha to access '/users/userinfo/'

closes https://github.com/apache/incubator-superset/issues/4919

* Fix unit test

* Fix test
2019-04-22 11:23:54 -07:00
Maxime Beauchemin 5f28027ce7 Merging lyftga into master 2019-04-18 22:47:47 -07:00
michellethomas 51068f007e
Adding permission for can_only_access_owned_queries (#7234)
* Adding permission for can_only_access_owned_queries

* Fixing lint adding typing to variable

* Adding test for queryview and enabling /queryview/api/read

* Fixing issues with python typing
2019-04-17 16:11:11 -07:00
Maxime Beauchemin 9dfa80d5c8
Add all database access permission to 'Alpha' role (#7271)
* Add all database access permission to 'Alpha' role

* Grant all database access to Alpha
2019-04-15 09:22:54 -07:00
Beto Dealmeida 4ab89dbcf7 Celery task for warming up cache (#7148)
* Sparkline dates aren't formatting in Time Series Table (#6976)

* Exclude venv for python linter to ignore

* Fix NaN error

* Fix the white background shown in SQL editor on drag (#7021)

This PR sets the background-color css property on `.ace_scroller` instead of `.ace_content` to prevent the white background shown during resizing of the SQL editor before drag ends.

* Show tooltip with time frame (#6979)

* Fix time filter control (#6978)

* Enhancement of query context and object. (#6962)

* added more functionalities for query context and object.

* fixed cache logic

* added default value for groupby

* updated comments and removed print

(cherry picked from commit d5b9795f87)

* [fix] /superset/slice/id url is too long (#6989)


(cherry picked from commit 6a4d507ab6)

* [WIP] fix user specified JSON metadata not updating dashboard on refresh (#7027)


(cherry picked from commit cc58f0e661)

* feat: add ability to change font size in big number (#7003)

* Add ability to change font sizes in Big Number

* rename big number to header

* Add comment to clarify font size values

* Allow LIMIT to be specified in parameters (#7052)

* [fix] Cursor jumping when editing chart and dashboard titles (#7038)


(cherry picked from commit fc1770f7b7)

* Changing time table viz to pass formatTime a date (#7020)

(cherry picked from commit 7f3c145b1f)

* [db-engine-spec] Aligning Hive/Presto partition logic (#7007)


(cherry picked from commit 05be866117)

* [fix] explore chart from dashboard missed slice title (#7046)


(cherry picked from commit a6d48d4052)

* fix inaccurate data calculation with adata rolling and contribution (#7035)


(cherry picked from commit 0782e831cd)

* Adding warning message for sqllab save query (#7028)


(cherry picked from commit ead3d48133)

* [datasource] Ensuring consistent behavior of datasource editing/saving. (#7037)

* Update datasource.py

* Update datasource.py

(cherry picked from commit c771625f10)

* [csv-upload] Fixing message encoding (#6971)


(cherry picked from commit 48431ab5b9)

* [sql-parse] Fixing LIMIT exceptions (#6963)


(cherry picked from commit 3e076cb60b)

* Adding custom control overrides (#6956)

* Adding extraOverrides to line chart

* Updating extraOverrides to fit with more cases

* Moving extraOverrides to index.js

* Removing webpack-merge in package.json

* Fixing metrics control clearing metric

(cherry picked from commit e6194051f4)

* [sqlparse] Fixing table name extraction for ill-defined query (#7029)


(cherry picked from commit 07c340cf82)

* [missing values] Removing replacing missing values (#4905)


(cherry picked from commit 61add606ca)

* [SQL Lab] Improved query and results tabs rendering reliability (#7082)

closes #7080

(cherry picked from commit 9b58e9f492)

* Fix filter_box migration PR #6523 (#7066)

* Fix filter_box migration PR #6523

* Fix druid-related bug

(cherry picked from commit b210742ad2)

* SQL editor layout makeover (#7102)

This PR includes the following layout and css tweaks:
- Using flex to layout the north and south sub panes of query pane so resizing works properly in both Chrome and Firefox
- Removal of necessary wrapper divs and tweaking of css in sql lab so we can scroll to the bottom of both the table list and the results pane
- Make sql lab's content not overflow vertically and layout the query result area to eliminate double scroll bars
- css tweaks on the basic.html page so the loading animation appears in the center of the page across the board

(cherry picked from commit 71f1bbd2ec)

* [forms] Fix handling of NULLs

(cherry picked from commit e83a07d3df)

* handle null column_name in sqla and druid models

(cherry picked from commit 2ff721ae07)

* Use metric name instead of metric in filter box (#7106)


(cherry picked from commit 003364e74e)

* Bump python lib croniter to an existing version (#7132)

Package maintainers should really never delete packages, but it appears
this happened with croniter and resulted in breaking our builds.

This PR bumps to a more recent existing version of the library

(cherry picked from commit 215ed392a1)

* Revert PR #6933 (#7162)

* Celery worker for warming up cache

* Remove testing changes

* Add documentation

* Fix lint

* WIP dashboard filters

* Use new cache so it works with dashboards

* Add more unit tests, fix old ones

* Fix flake8 and docs

* Sparkline dates aren't formatting in Time Series Table (#6976)

* Exclude venv for python linter to ignore

* Fix NaN error

* Changing time table viz to pass formatTime a date (#7020)

(cherry picked from commit 7f3c145b1f)

* SQL editor layout makeover (#7102)

This PR includes the following layout and css tweaks:
- Using flex to layout the north and south sub panes of query pane so resizing works properly in both Chrome and Firefox
- Removal of necessary wrapper divs and tweaking of css in sql lab so we can scroll to the bottom of both the table list and the results pane
- Make sql lab's content not overflow vertically and layout the query result area to eliminate double scroll bars
- css tweaks on the basic.html page so the loading animation appears in the center of the page across the board

(cherry picked from commit 71f1bbd2ec)

* Celery worker for warming up cache

* Remove testing changes

* Add documentation

* Fix lint

* WIP dashboard filters

* Use new cache so it works with dashboards

* Add more unit tests, fix old ones

* Fix flake8 and docs

* Fix bad merge and pylint
2019-04-03 16:57:59 -07:00
Kim Truong e974a23f90 [Lyft-GA] Enable color consistency in a dashboard (#7135)
* Enable color consistency in a dashboard

Moved actions, minor UI, allowed dashboard copy

Fix linting errors

Undo unintentional change

Updated and added unit tests

Fail quietly if package has not been updated

Fail quietly on dashboard copy if package is old

* Update packages

* Remove unnecessary code

* Addressed Grace's comments

* Small fix for item key

* Reset chart's color during exploration

* Do not reset chart form data when exploring chart
2019-04-01 22:04:31 -07:00
Dave Smith ba19a62412 Injectable statsd client (#7138)
* Add ability to inject statsd client; some py test/reqs updates

- Updated the metrics logger to allow construction with an existing
statsd client, so that it can be configured by external systems or libs.
- added requirements to requirements-dev.txt which are needed to run
  tests-eg coverage, nose
- removed dependency on mock lib, it is in python stdlib now
- updated tox.ini to remove the now-superfluous deps

* add license to test file, and remove blank line at EOF
2019-03-27 14:08:36 -07:00
Ville Brofeldt 23e823f0c1 Bump SQLAlchemy to 1.3 (#7099)
* Bump sqla to >=1.3.1

* Refine mssql column types to only use N-prefixing when necessary

* make join explicit

* replace set with list

* Add additional test case for N-prefix

* Replace engine with dialect and fix linting error

* Remove unneeded import
2019-03-24 20:44:05 -07:00