Commit Graph

979 Commits

Author SHA1 Message Date
ʈᵃᵢ fbe4a6622e
feat: SQL preview modal for Query History (#11634) 2020-11-20 16:01:06 -08:00
Beto Dealmeida a3a2a68f01
feat: API endpoint to import charts (#11744)
* ImportChartsCommand

* feat: API endpoint to import charts

* Add dispatcher

* Fix docstring
2020-11-20 14:40:27 -08:00
Beto Dealmeida 2f4f87795d
feat: add a command to import charts (#11743)
* ImportChartsCommand

* Fix type
2020-11-20 14:20:13 -08:00
Ville Brofeldt 7ae8cd07cc
fix: do not drop calculated column on metadata sync (#11731) 2020-11-18 08:33:14 -08:00
Beto Dealmeida cce716a821
feat: API endpoints to upload dataset/db (#11728)
* feat: API endpoints to upload dataset/db

* Fix method call
2020-11-17 14:49:33 -08:00
Rob DiCiuccio 01d15f5368
feat(templating): Safer Jinja template processing (#11704)
* Enable safer Jinja template processing

* Allow JINJA_CONTEXT_ADDONS with SAFE_JINJA_PROCESSING

* Make template processor initialization less magical, refactor classes

* Consolidat Jinja logic, remove config flag in favor of sane defaults

* Restore previous ENABLE_TEMPLATE_PROCESSING default

* Add recursive type checking, update tests

* remove erroneous config file

* Remove TableColumn models from template context

* pylint refactoring

* Add entry to UPDATING.md

* Resolve botched merge conflict

* Update docs on running single python test

* Refactor template context checking to support engine-specific methods
2020-11-17 11:55:47 -08:00
Beto Dealmeida 7bc353f8a8
feat: new import commands for dataset and databases (#11670)
* feat: commands for importing databases and datasets

* Refactor code
2020-11-16 17:11:20 -08:00
Bogdan a16a2b87c7
chore: statds metrics for cache invalidation (#11701)
* Add more logging to the cache invalidation

* Address linter & tests

* Fix linter

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-11-16 06:48:26 -08:00
Jesse Yang 4cfcaebb61
feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG (#11509)
* feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG

The corresponding cache will now also cache the query results.

* Slice use DATA_CACHE_CONFIG CACHE_DEFAULT_TIMEOUT

* Add test for default cache timeout

* rename FAR_FUTURE to ONE_YEAR_IN_SECS
2020-11-13 22:35:10 -08:00
Jesse Yang ec8ccd4cf1
feat: keep modal open when saving database failed (#11618) 2020-11-12 16:10:14 -08:00
ʈᵃᵢ 432e5ab460
feat: Query History CRUD list view (#11574) 2020-11-12 11:55:13 -10:00
Daniel Vaz Gaspar 12cb27f5cb
feat: new reports models api (#11606)
* feat: new report schedule models

* lint and unique constraint

* support sqlite

* fix sqlite

* add audit mixin and minor fixes

* feat(api): alerts and reports REST API

* feat: new report schedule models

* lint and unique constraint

* support sqlite

* fix sqlite

* add audit mixin and minor fixes

* feat(api): alerts and reports REST API

* draft working version

* add tests

* test

* black

* remove copy pasta

* solve dashboard object representation being used on cache

* tests and custom filter

* fix PUT has PATCH on active field

* create feature flag

* fix lint

* address comments
2020-11-12 13:21:01 -08:00
Grace Guo 0338b48fdd
fix (dashboard): remove default filters parameters in dashboard url (#11661) 2020-11-12 13:00:27 -08:00
Jesse Yang 542d2e3b06
chore: clean up a debug line from #11653 (#11663) 2020-11-12 08:37:49 -08:00
Ville Brofeldt 2718909314
feat: add ECharts BoxPlot chart (#11199)
* feat: add ECharts BoxPlot chart

* lint

* fix cypress tests

* lint

* remove viz.py shim

* bump plugin package

* skip non-legacy plugin cypress dashboard tests

* fix cypress tests

* disable cypress tests for non-leagcy charts

* fix bad rebase

* use midpoint interpolation for quartile calculation

* bump packages and add support for no groupby

* whitespace

* whitespace

* linting

* fix tests

* xit mathjs load test

* bump mathjs to 8.0.1

* disable cypress filter test for v1 charts
2020-11-12 10:01:04 +02:00
Jesse Yang 302c9603c4
fix: database connection validation when creation (#11653) 2020-11-12 08:00:38 +00:00
Beto Dealmeida 45738ffc1d
chore: consolidate datasource import logic (#11533)
* Consolidate dash import logic

* WIP

* Add license

* Fix lint

* Retrigger tests

* Fix lint
2020-11-11 22:04:16 -08:00
Beto Dealmeida 4a070cfceb
chore: consolidate dashboard import logic (#11529)
* Consolidate dash import logic

* Fix lint

* Remove Slice.import_obj

* Remove unused import

* Fix log
2020-11-11 14:50:18 -08:00
Ville Brofeldt 77dff0e4e0
fix(chart-data): ignore orderby on sample result type (#11656) 2020-11-11 16:27:56 +02:00
Beto Dealmeida 51712bf7ce
fix: CommandInvalidError had invalid init (#11635) 2020-11-10 07:41:03 -08:00
Ville Brofeldt 600a6fa92a
chore(rls): move to feature flag and disable related view (#11575)
* chore(rls): move to feature flag and disable related view

* rename feature flag
2020-11-05 12:19:48 +02:00
Lily Kuang eef4809978
feat: annotation edit modal with antd datepicker (#11500) 2020-11-03 12:59:13 -08:00
Daniel Vaz Gaspar 6f2e36dd1b
fix(tests): flaky test with dttm on Query API (#11526)
* fix(tests): flaky test wirh dttm on Query API
2020-11-03 09:01:17 +00:00
ʈᵃᵢ edb9619731
refactor: reduce number of api calls needed to fetch favorite status for charts and dashboards (#11502) 2020-11-02 21:26:14 -08:00
Beto Dealmeida fd10c47bc6
chore: remove sanitize (#11532)
* chore: remove sanitize

* Fix lint
2020-11-02 15:13:44 -08:00
Daniel Vaz Gaspar 34ae29e59b
feat: query REST API more fields and tests (#11482)
* feat: query REST API more fields and tests

* fix missing field
2020-10-31 10:18:34 +00:00
Moriah Kreeger 01ddbd0697
feat: annotation layers modal + filters (#11494) 2020-10-30 15:51:46 -07:00
Beto Dealmeida fbcfaacda3
feat: create base class for export commands (#11463)
* Add UUID to saved_query

* Reuse function from previous migration

* Point to new head

* feat: add backend to export saved queries using new format

* Rename ImportMixin to ImportExportMixin

* Create base class for exports

* Add saved queries as well

* Add constant, small fixes

* Fix wrong import

* Fix lint
2020-10-30 11:52:11 -07:00
Erik Ritter ca40877640
feat: add certifications to tables (#11450) 2020-10-30 11:28:01 -07:00
Beto Dealmeida 21f722c9bc
feat: add endpoint to export saved queries using new format (#11447)
* Add UUID to saved_query

* Reuse function from previous migration

* Point to new head

* feat: add backend to export saved queries using new format
2020-10-30 08:32:16 -07:00
John Bodley a8eb3fe8e7
chore: Cleaning up ENABLE_REACT_CRUD_VIEWS config (#11496)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-10-29 16:47:11 -07:00
ʈᵃᵢ f918ca14aa
fix(datasets): add custom filter for virtual datasets based on sql attribute (#11452) 2020-10-29 13:11:33 -07:00
David Aaron Suddjian 18658f45be
feat(dashboards): Filter status indicators (#10936)
* Initial commit of new filters badge.

* refactor applied/rejected filters code

* finished filter indicators

* filter badge tested

* unnecessary imports

* formatting and types

* fixes

* license

* code quality tweaks

* state management for showing focused filter scope

* clean up filter key extraction code

* remove unnecessary styles

* temp css to demonstrate highlighting

* fix focused filter logic

* no more color badges

* new toys for highlighting dash components (#11144)

* tweak style for the filter chart when filter is focused

* style: Filters p0 css2 (#11151)

* nixing background tweak

* src paths

* another quick theme color

* src paths, adjusting pill icon color, changing icons, showing applied/busted counts

* linting stuff

* fixing and tweaking tests

* show filter indicator when filters are not active

* chart title bar cleanup

* open the right panel when popover opens

* unused import

* fix EditableTitle tests

* margin on dashboard header

* show the chart dropdown menu

* fix blur filter breaking dropdowns

* style tweak - no pointer events when irrelevant charts are blurred

* fix box shadow on filter highlight

* it's an array

* attempt fixing e2e

* style: filters p0 icon churn (#11215)

* new filters icon

* icon styling

* bigger icons in list views

* better sizing of table actions and favStars

* more icon sizing...

* fixing more button size jankiness

* linting

* Filters performance (#11255)

* fixing time filter "ok" button

* making unset filter menu collapsible

* sort alphabetically

* fix highlighting when removing items

* try a flex layout (for browser render perf)

* more specific transitioning

* temp: comment out some code as a test

* temp: comment out more code

* temp: remove possibly expensive computations from ChartHolder

* Revert "temp: comment out some code as a test"

This reverts commit 309b880e90.

* Revert "temp: comment out more code"

This reverts commit 64c88b2cba.

* Revert "temp: remove possibly expensive computations from ChartHolder"

This reverts commit 37ce0214f0.

* experiment: upgrade react-select to v3

* Revert "experiment: upgrade react-select to v3"

This reverts commit c3972ba486.

* fix the damn problem

* remove code used for testing purposes

* awful hack to avoid adding a class to a container

* approaching infinity... and not beyond!

* fix ref forwarding

* add theme to tests as necessary

* fix(extra-filters): add logic for identifying applied extra filters (#11325)

* fix: use dashboard id for stable cache key (#11293)

* fix: button translations missing (#11187)

* button translations missing

* blank space before text

* feat: update time_compare description and choices (#11294)

* feat: update time_compare description and choices

* Update sections.jsx

* fix(extra-filters): add logic for identifying applied extra filters

* lint

Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
Co-authored-by: rubenSastre <ruben.sastre@decathlon.com>
Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>

* address design feedback

* slight tweak to panel logic, keep panels open that user has opened

* rearrange code to be more graceful

* fix: bump superset-ui/core (#11385)

* use is_dttm instead of is_temporal

* types, names

* only show unset filter panel if there are unset filters

* fix highlighting the filter control

* fix filterbox layout

* translations

* fix cypress

* actually add the test attribute

* Update superset-frontend/src/dashboard/components/DashboardBuilder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/DashboardBuilder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* formatting

* add link comment to hack

* Update superset-frontend/src/dashboard/components/gridComponents/ChartHolder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* stop importing lodash

* Update superset-frontend/src/dashboard/components/gridComponents/ChartHolder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* skip broken test

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* adjust colors of titles

* linting

* no indicators when chart is loading

* support all time fields

* fix lock file

Co-authored-by: Natalie Ruhe <natalie@preset.io>
Co-authored-by: Evan Rusackas <evan@preset.io>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
Co-authored-by: rubenSastre <ruben.sastre@decathlon.com>
Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2020-10-28 15:46:24 -07:00
Moriah Kreeger e9dba18466
feat: annotation layers CRUD list view (#11432) 2020-10-28 15:45:07 -07:00
Ville Brofeldt ecdff724ea
feat: enable metadata sync for virtual tables (#10645)
* feat: enable metadata sync for virtual tables

* add migration and check for empty schema name

* simplify request

* truncate trailing column attributes for MySQL

* add unit test

* use db_engine_spec func to truncate collation and charset

* Remove redundant migration

* add more tests

* address review comments and apply templating to query

* add todo for refactoring

* remove schema from tests

* check column datatype
2020-10-27 07:58:38 +02:00
Ville Brofeldt 8575439f48
fix: is_temporal should be overridden by is_dttm value (#11429) 2020-10-27 07:22:27 +02:00
Daniel Vaz Gaspar 144b279aa2
feat: saved queries with execution info (#11391)
* feat: add rows and last_run info to saved queries

* feat: add rows to saved query

* refactor and tests

* lint

* fix tests
2020-10-26 11:20:07 -07:00
Daniel Vaz Gaspar 6f69212f28
feat: annotations REST API (#11344)
* feat(api): annotations and layers CRUD REST API

* lint

* annotations API and tests

* fix openapi spec

* fix openapi spec

* fix openapi spec

* annotations bulk delete

* fix openapi spec

* fix older tests

* fix older tests

* small lint fixes

* layer_id to pk to keep broad coherence on openapi spec

* fix openapi spec

* one more test and validation

* fix test name

* fix test

* fix bulk delete

* add name validation

* annotation uniqueness validation

* lint

* add sorting reqs and tests

* add statsd metrics

* Update superset/annotation_layers/annotations/dao.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Update tests/core_tests.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* address comments

* address my comment :)

* fix, address comments

* lint

Co-authored-by: riahk <moar.riah@gmail.com>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-10-23 09:41:17 +01:00
Beto Dealmeida c81204aeef
feat: export dashboards as ZIP files (#11351)
* Export datasets as ZIP files

* Add logging when failing to parse extra

* Export datasets as ZIP files

* Export charts as Zip file

* Export dashboards as a Zip file

* Add logging
2020-10-22 20:02:40 -07:00
Beto Dealmeida 633355ab00
feat: export charts as ZIP files (#11349)
* Export datasets as ZIP files

* Add logging when failing to parse extra

* Export charts as Zip file

* Fix lint
2020-10-22 12:06:58 -07:00
Beto Dealmeida 00e394451f
feat: export datasets as ZIP files (#11332)
* Export datasets as ZIP files

* Add logging when failing to parse extra

* Fix logging
2020-10-22 10:32:08 -07:00
Daniel Vaz Gaspar b86f779dc1
feat: add statsd metrics to FAB rest API post, put and delete (#11362) 2020-10-22 08:56:26 +01:00
Moriah Kreeger a2a614d760
feat: CSS Templates List Actions (#11271) 2020-10-21 20:32:59 -07:00
Daniel Vaz Gaspar f14cf9d6dc
fix: saved_query search on schema and database not working (#11286)
* fix: saved_query search on schema and database not working

* add tests
2020-10-21 21:17:25 +01:00
Hugh A. Miles II dfbcbcce67
fix: Allow "EXPLAIN" queries when "Allow DML" setting is False (#11348)
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2020-10-21 12:58:57 -07:00
Maxime Beauchemin 796a2a6924
fix: error around latest partition in BigQuery (#11274)
* fix: error around latest partition in BigQuery

* lint

* Going with a backend-first approach

* fix test

* add an extra test
2020-10-20 21:34:46 -07:00
Lily Kuang 9f8d0e7a06
fix: update table_name and schema on dataset editor (#11218)
* fix dataset editor for updating table_name and schema

* update test

* remove table_name from restrict list

* fix pylint

* fix cypress test

* fix test
2020-10-20 09:38:57 -07:00
hnle 7369039713
tests: Parameterize some tests in alerts_tests.py (#11313)
* SO-1099 Parameterize some tests in alerts_tests.py

* Update alerts_tests.py

Remove commented out line

* Update alerts_tests.py

Reformatted using black
2020-10-19 23:05:04 -07:00
Kasia Kucharczyk 0e97c4f66c
tests: added fixtures to created dashboards and updated test with proper value (#11290)
* Added fixtures for hidden and published dashboards. Added fixture to restore copied dashboard in dashboard tests. Changed number of dashboards in datasets/api_tests.py because copied dashboard is removed.

* Changed number of dashboards in database api tests after cleanup of dashboards in dashboards_tests
2020-10-19 15:07:26 -07:00
Beto Dealmeida 94e23bfc82
feat: export databases as a ZIP bundle (#11229)
* Export databases as Zip file

* Fix tests

* Address comments

* Implement mulexport for database

* Fix lint

* Fix lint
2020-10-16 11:10:39 -07:00
Kasia Kucharczyk 9caf875263
Refactored api_tests in charts. Split filter test into several unit tests cases. Added table id based on database id (#11249) 2020-10-14 12:41:37 -07:00
Jesse Yang 634676d467
refactor: use contextmanager for event_logger decorators (#11222) 2020-10-14 10:44:06 -07:00
Yongjie Zhao 22d8171dd6
chore(engine): adapt postgres backend connection URI (#11233)
* chore(engine): adapt postgres backend connection URI

* fix tests

* Update superset/db_engine_specs/__init__.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-10-14 18:57:41 +03:00
Jesse Yang 2c649ac20f
perf: cache dashboard bootstrap data (#11234) 2020-10-13 18:43:06 -07:00
Grace Guo 4f4367edf3
feat: prevent co-edit dashboard collision (#11220)
* feat: prevent co-edit dashboard collision

* fix comments
2020-10-12 17:58:32 -07:00
Daniel Vaz Gaspar 9e9dac68f7
feat(datasets): REST API bulk delete (#11237)
* feat(datasets): REST API bulk delete

* doc HTTP 400
2020-10-12 13:40:05 +01:00
Ville Brofeldt 9f3d089655
chore(sqla): assert query is single read-only statement (#11236) 2020-10-12 15:11:43 +03:00
Bogdan cb3f649a7f
chore: simplify alerting data model to leverage a single class (#11179)
* Collapse alerting models into a single one

Fixing upgrade migration & tests

* Address feedback

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-10-09 19:20:12 -07:00
Kasia Kucharczyk 56d5e8a1cb
Changed cache uid to be depending on table in test_csv_response_format (#11214) 2020-10-09 09:01:26 -07:00
Kasia Kucharczyk d93b1afe79
test: test_cache_logging is not depending on other dashboard (#11213)
* Changed test_cache_logging to be idempotent and not depending on other data than tested one

* Changed cache uid to be depending on table, not datashource
2020-10-09 09:00:41 -07:00
Grace Guo a10e86ab31
fix: revert eTag cache feature for dashboard (#11203)
* revert #11137

* revert #10963
2020-10-08 12:15:08 -07:00
Beto Dealmeida b6728d87a0
fix: skip unit test that is failing in master for test-postgres-hive (#11196)
* Get full diff

* Test

* Test

* Test

* Test

* Test

* Explicitly detect backend to skip test
2020-10-08 09:17:09 -07:00
Daniel Vaz Gaspar 7c60939429
chore: turn SQL templating off by default (#11172)
* feat: possible to turn off SQL templating

* turn SQL templating off by default

* Update UPDATING.md

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* fix missing PR number

* fix missing PR number

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-10-08 11:55:39 +01:00
Beto Dealmeida 9785667a0d
feat: add UUID column to ImportMixin (#11098)
* Add UUID column to ImportMixin

* Fix default value

* Fix lint

* Fix order of downgrade

* Add logging when downgrade fails

* Migrate position_json to contain UUIDs, and add schedule tables

* Save UUID when adding charts to dashboard

* Fix heads

* Rename migration file

* Fix dashboard serialization

* Fix migration script with Postgres

* Fix unique contraint name

* Handle UUID when exporting dashboard

* Fix Dataset PUT

* Add UUID JSON serialization

* Fix tests

* Simplify logic

* Try binary=True
2020-10-07 09:00:55 -07:00
Kasia Kucharczyk 32e174d4b8
test: removed unicode_test example from unit tests (#11131)
* Removed depemdency to unicode example in tests config.

* Added common methods for creating dashboards for tests.

* Added fixtures to all tests which were using unicode example.

* Added cleanup for unicode_test table

* Removed unnecessary fixture parts of unicode dashboard tests

* Parametrized creating slice for tests

* Moved fixtures for unicode test to separate file and refactored to several methods. Added param types and return types.

* Cleandup after fix

* Changed variable names to more readable

* Added cleanup for dashboards and slices

* Applied unicode fixture to charts api tests

* Update schema variable to dtype in dashboard utils

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Changed variable schema to dtype in dashboards. Replaced accessing first element with one_or_none

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-10-07 18:29:37 +03:00
Daniel Vaz Gaspar 4c85d33109
feat: custom favorite filter for dashboards, charts and saved queries (#11083)
* feat: custom favorite filter for dashboards

* lint and sort

* add favored for charts

* fix tests and lint

* more tests and saved query filter

* fix tests

* fix tests

* lint

* lint and fix conflict

* remove unnecessary prop

* separate tests
2020-10-01 15:08:40 -07:00
Daniel Vaz Gaspar fdb26f6131
feat: CRUD REST API for CSS Templates (#11114)
* feat: CSS Template CRUD API

* fix API docs

* fix copy pasta

* lint
2020-10-01 11:46:25 +01:00
Moriah Kreeger e7a4265c30
feat: saved query list actions (#11109) 2020-09-30 18:18:34 -07:00
Lily Kuang 5b284e67eb
feat: add created by chart CRUD view (#11067) 2020-09-30 13:53:04 -07:00
Ville Brofeldt ada66e30dd
fix(chart-data-api): ignore missing filters (#11112) 2020-09-30 09:34:23 +01:00
Bogdan 03eebd3019
fix: fix table existence validation function (#11066)
* Fix table existance validation function

* Drop left over table name index in mysql db

* Do not modify model

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-29 15:01:01 -07:00
Moriah Kreeger b8d616a9d2
feat: saved query list view + sort/filters (#11005) 2020-09-29 13:39:10 -07:00
Grace Guo 6633409a15
feat: enable ETag header for dashboard GET requests (#10963)
* feat: add etag for dashboard load requests

* fix review comments
2020-09-29 10:57:16 -07:00
Daniel Vaz Gaspar 5d08a426d3
fix(api): unable to delete virtual dataset, wrong permission name (#11019)
* fix(api): unable to delete virtual dataset because of wrong permission name

* Still delete the dataset even when no permission was found

* migration script to fix possible existing faulty permissions on the db

* black

* fix db migration and one more test

* add more comments to the migration script

* freeze a partial schema of the model on the migration step

* fix mig script

* Update superset/migrations/versions/3fbbc6e8d654_fix_data_access_permissions_for_virtual_.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-09-29 12:33:07 +01:00
Ville Brofeldt 80e395559d
fix: echarts timeseries groupby (#11103)
* fix: echarts timeseries groupby

* address review comment
2020-09-29 14:25:38 +03:00
Lily Kuang e337355162
refactor: table selector on dataset editor (#10914)
Co-authored-by: Maxime Beauchemin <maximebeauchemin@gmail.com>
2020-09-28 11:16:03 -07:00
ʈᵃᵢ 1dc05d4c12
feat(charts): modify custom api filter to include more fields (#11054) 2020-09-28 10:18:34 -07:00
Lily Kuang ea105bc23b
feat: add "created by" to dashboard CRUD view (#11030) 2020-09-24 14:33:55 -07:00
ʈᵃᵢ eded51b2f8
feat(saved_queries): add custom api filter for all string & text fields (#11031) 2020-09-24 11:05:25 -07:00
Bogdan b2fdf12f62
Support jinja templates (#11008)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-24 10:29:44 -07:00
Ville Brofeldt a9eab33d64
feat: add ECharts Pie chart (#10966)
* feat: introduce echarts pie chart

* lint

* remove viz.py shim

* remove tests

* fix cypress test

* fix test
2020-09-22 19:27:40 +03:00
Ville Brofeldt bd140e018a
fix(presto): default unknown types to string type (#10753)
* fix(presto): default unknown types to string type

* lint
2020-09-22 13:16:54 +03:00
Ville Brofeldt 448a41a4e7
feat(row-level-security): add base filter type and filter grouping (#10946)
* feat(row-level-security): add filter type and group key

* simplify tests and add custom list widget

* address comments

* use enum value to ensure case sensitive value is used
2020-09-22 08:54:10 +03:00
Bogdan 56d001835e
fix: use nullpool even for user lookup in the celery (#10938)
* Use nullpool even for user lookup in the celery

* Address feedback

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-21 10:34:03 -07:00
Bogdan 801fb4063c
Allow empty observations in alerting (#10939)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-21 08:48:02 -07:00
Daniel Vaz Gaspar e375ed4678
fix(jinja): make context attrs private on SQL templates (#10934)
* fix(jinja): make SQLAlchemy models private on SQL templates

* add missing privates

* fix test
2020-09-18 12:56:07 +01:00
Bogdan c3f1720456
fix: superset alerting misc fixes (#10891)
* Superset alerting misc fixes

* Test 0 threshold

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-16 11:09:11 -07:00
John Bodley b48dd4b7d9
chore: Using cache factory method (#10887)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-09-15 12:48:19 -07:00
Bogdan 9c420d6efe
feat: implement cache invalidation api (#10761)
* Add cache endpoints

* Implement cache endpoint

* Tests and address feedback

* Set cache config

* Address feedback

* Expose only invalidate endpoint

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-15 11:17:21 -07:00
Yongjie Zhao 38edb69d95
Feat: Adding table comment and columns comment for SQLLab (#10844)
* Adding table comment and columns comment for backend

* fix mypy

* Fix CI

* adding wider catch

* use logger

* fix lint
2020-09-14 21:11:48 -07:00
Daniel Vaz Gaspar 7b2200437e
fix(test): missing auth on tests (#10842)
* fix(test): missing auth on tests

* fix mock

* make test login more inline with other tests
2020-09-11 14:28:41 +01:00
Daniel Vaz Gaspar 136f90f87c
feat: SavedQuery REST API for bulk delete and new API fields (#10793)
* feat: SavedQuery REST API for bulk delete

* fix, singular msg and test

* remove 403 from OpenAPI spec

* filter by current user using created_by add sql_tables field

* fixes for new filter, add user field on pre_update, pre_add

* add lru cache to property

* Revert "add lru cache to property"

This reverts commit ad0d9428
2020-09-11 09:36:47 +01:00
Lily Kuang f0545bfe50
fix(databases): test connection api endpoint (#10824)
* fix test connection with extra

* fix lint and allow_none server_cert

* update test connection tests
2020-09-10 21:49:14 +01:00
Bogdan ac2937a6c5
fix: use nullpool in the celery workers (#10819)
* Use nullpool in the celery workers

* Address comments

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-10 13:29:57 -07:00
Daniel Vaz Gaspar e6a4808cb7
fix(tests): export dataset tests fails with presto (#10818)
* fix(tests): export dataset tests fails with presto

* debug

* debug

* skip dataset export test on presto

* add TODO on failing test
2020-09-09 21:46:28 -07:00
Lily Kuang 8a3ac70c06
feat(databases): test connection api (#10723)
* test connection api on databases

* update test connection tests

* update database api test and open api description

* moved test connection to commands

* update error message

* fix isort

* fix mypy

* fix black

* fix mypy pre commit
2020-09-09 13:37:48 -07:00
ʈᵃᵢ 50672bb11b
feat: database delete warning (#10800) 2020-09-08 18:54:02 -07:00
Ville Brofeldt 3ae80d3b98
test: add rls jinja extra cache key test (#10810) 2020-09-08 07:42:48 +03:00
Daniel Vaz Gaspar 5199423ad1
feat: CRUD REST API for saved queries (#10777)
* feat: CRUD REST API for saved queries

* debug test

* fix test

* use pytest fixtures
2020-09-03 15:46:27 -07:00
Ville Brofeldt 70c6764780
fix: pivot table timestamp grouping (#10774)
* fix: pivot table timestamp grouping

* address comments
2020-09-03 19:49:54 +03:00
Jason Davis 54ae3b044f
fix: add validator information to email/slack alerts (#10762)
* added validator info to alerts

* adjusted format of messages

* added nits

Co-authored-by: Jason Davis <@dropbox.com>
2020-09-03 08:15:05 -07:00
John Bodley 1ee87cc4d1
fix: dashboard extra filters (#10692)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-09-02 16:03:25 -07:00
ʈᵃᵢ 7bccb38a60
feat: SIP-34 table list view for databases (#10705) 2020-09-02 11:48:21 -07:00
ʈᵃᵢ 5a4370012b
refactor: convert DatasetList schema filter to use new distinct api (#10746) 2020-09-02 11:07:37 -07:00
Daniel Vaz Gaspar 77a3167412
feat(database): POST, PUT, DELETE API endpoints (#10741)
* feat(database): POST, PUT, DELETE API endpoints

* post tests

* more tests

* lint

* lint

* debug ci

* fix test

* fix test

* fix test

* fix test

* fix test

* fix test

* cleanup

* handle db connection failures

* lint

* skip hive and presto for connection fail test

* fix typo
2020-09-02 09:58:08 -07:00
Jason Davis b59f6b1833
feat: refractored SQL-based alerting framework (#10605)
* added new tables for alerting refractor

* reformatted inheritance structure

* added workflow for updated framework

* added suggested changes

* cleaned up changes

* added obervations to alert table to enable view

* added comments

* added requested changes

* fix tests

* added styling changes

* mypy

* added requested changes

* updated operator logic

* requested changes, 1 validator, styling changes

* refactored tests

* fix test alert workflow

* fixed create_alert in test

Co-authored-by: Jason Davis <@dropbox.com>
2020-09-01 13:36:02 -07:00
Bogdan 4572ebb600
chore: log cache keys to the logs (#10678)
* Log cache keys to the logs

* Add tests

* Use separate table for the cache keys

* Add migration for the cache lookup table

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-01 09:41:25 -07:00
Maxime Beauchemin 3d8857ecc8
style: rename 'Source' top nav item to 'Data' (#10715)
* style: rename 'Source' top nav item to 'Data'

* Fix tests

* tables -> datasets

* rename datasource -> dataset

* lint

* more renaming

* use npm ci
2020-08-29 18:09:47 -07:00
Bogdan 33fa9ebff1
Covert celery tests to the pytest (#10704)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-08-28 17:04:30 -07:00
Ville Brofeldt 58b075bc17
fix(jinja): extract form_data from json body (#10684)
* fix(jinja): extract form_data from json body

* add test

* disable test for presto
2020-08-28 21:26:07 +03:00
Daniel Vaz Gaspar 3e374dab07
fix: change public role like gamma procedure (#10674)
* fix: change public role like gamma procedure

* lint and updating UPDATING with breaking change

* fix updating text

* add test and support PUBLIC_ROLE_LIKE_GAMMA

* fix, cleanup tests

* fix, new test

* fix, public default

* Update superset/config.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* add simple public welcome page

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-08-28 10:49:10 +01:00
Bogdan 6ed36552e6
Use test name for the custom macros testing: (#10695)
:

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-08-27 13:12:24 -07:00
Bogdan 19a9bcc9c5
chore: ci Initial hive support (#10593)
* Initial hive support

* Clone hive setup

* Make hive tests work locally

* Debugging presto failure

* sleep in dataset test

* Address comments

* Address comments

* Pin ipython, exclude new pylint rules

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-08-27 09:49:18 -07:00
Lily Kuang bb92c1f84d
feat: chart and dashboard card list view e2e cypress test (#10669)
* card view cypress tests

* update test config

* chart filter cypress test

* move setting to superset test config

* update test description card view
2020-08-27 09:38:12 -07:00
John Bodley 502f4db6f9
feature: Adding extra_filters to warm_up_cache (#10675)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-08-25 15:19:34 -07:00
Ville Brofeldt 9461f9c1e0
fix(db_engine_specs): improve Presto column type matching (#10658)
* fix: improve Presto column type matching

* add optional callback to type map and add tests

* lint

* change private to public
2020-08-24 22:42:07 +03:00
Ville Brofeldt 0177c2f591
fix(db_engine_specs): mysql longtext type should not be numeric (#10661) 2020-08-24 21:24:43 +03:00
Daniel Vaz Gaspar 692266f4f5
feat: dataset REST API for distinct values (#10595)
* feat: dataset REST API for distinct values

* add tests and fix lint

* fix mypy, and tests

* fix docs

* fix test

* lint

* fix test
2020-08-17 15:46:59 +01:00
Ville Brofeldt acb00f509c
fix(chart-data-api): assert referenced columns are present in datasource (#10451)
* fix(chart-data-api): assert requested columns are present in datasource

* add filter tests

* add column_names to AnnotationDatasource

* add assertion for simple metrics

* lint
2020-08-14 20:58:24 +03:00
Ville Brofeldt c0ebd7f434
fix: show error if rolling window returns empty df (#10572)
* fix: show error if rolling window returns empty df

* add test
2020-08-13 20:51:03 +03:00
Daniel Vaz Gaspar 11da6ee812
fix: dataset delete and perm delete (#10578) 2020-08-13 10:18:13 +01:00
Craig Rueda 2aaa4d92d9
chore: Migrating reports to AuthWebdriverProxy (#10567)
* Migrating reports to AuthWebdriverProxy

* Extracting out webdriver proxy / Adding thumbnail tests to CI

* Adding license

* Adding license again

* Empty commit

* Adding thumbnail tests to CI

* Switching thumbnail test to Postgres

* Linting

* Adding mypy:ignore / removing thumbnail tests from CI

* Putting ignore statement back

* Updating docs

* First cut at authprovider

* First cut at authprovider mostly working - still needs more tests

* Auth provider tests added

* Linting

* Linting again...

* Linting again...

* Busting CI cache

* Reverting workflow change

* Fixing dataclasses

* Reverting back to master

* linting?

* Reverting installation.rst

* Reverting package-lock.json

* Addressing feedback

* Blacking

* Lazy logging strings

* UPDATING.md note
2020-08-12 13:28:41 -07:00
Ville Brofeldt 85c0cad816
fix: add None checking to cast_to_num (#10584)
* fix: add None checking to cast_to_num

* fix doctest
2020-08-12 19:58:14 +03:00
Jason Davis c0bb86d0d6
feat: slack integration for SQL-based alerts (#10566)
* add slack functionality

* deleted unused variable

* updated test

* black

* fix rebase

* added nits

* added slack no screenshot integration

* isort

* added namedtuple for screenshot

* added test

* fix precommit

Co-authored-by: Jason Davis <@dropbox.com>
2020-08-11 11:15:31 -07:00
Bogdan 0071d374da
chore: move load examples to the fixture (#10545)
* Move load examples to the fixture

* Update tests/celery_tests.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Address feedback

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-08-10 13:20:19 -07:00
Bogdan 101e5b670b
chore: clean up csv tests (#10556)
* Clean up csv tests

* Update tests/base_tests.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Update tests/base_tests.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* import optional

* Fix mypy error

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-08-10 11:22:38 -07:00
Jason Davis 8b9292ed05
fix: add retry to SQL-based alerting celery task (#10542)
* added retry and minimized sqlalchemy object lives

* pylint

* added try catch

* adjusted naming

* added scoped session

* update tests for dbsession

* added requested changes

* nit todo

Co-authored-by: Jason Davis <@dropbox.com>
2020-08-10 10:20:43 -07:00
Ville Brofeldt 08358d623b
fix: handle query exceptions gracefully (#10548)
* fix: handle query exceptions gracefully

* add more recasts

* add test

* disable test for presto

* switch to SQLA error
2020-08-07 17:37:40 +03:00
Ville Brofeldt 363abfa1a5
fix: remove unnecessary exception when exploring non-legacy viz plugins (#10538)
* fix: remove unnecessary exception when exploring non-legacy viz plugins

* lint
2020-08-07 08:41:39 +03:00
Erik Ritter fd2d1c58c5
Revert "chore: Cleanup database sessions (#10427)" (#10537)
This reverts commit 7645fc85c3.
2020-08-06 15:33:48 -07:00
Bogdan 62b873e3da
feat: welcome presto to the suite of tested databases (#10498)
* Add presto to the CI

Sample test data

Datetime conversion

Sample test data

Fix tests

* TODO to switch to timestamps

* Address feedback

* Update requirements

* Add TODOs

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-08-06 12:07:22 -07:00
Jason Davis 9c5b0e1c86
fix: refactored SQL-based alerts to not pass sqlalchemy objects as args (#10506)
* refractored alerting to not pass sqlalchemy obj as args

* updated to pass only alert id as arg

* used object id instead of argument

* updated alerts_tests.py to reflect change

Co-authored-by: Jason Davis <@dropbox.com>
2020-08-04 09:52:32 -07:00
Bogdan ab404ea2cf
Sample test data (#10487)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-08-03 09:08:49 -07:00
Jesse Yang ce10e13ec3
test: schedules test refactor (#10492) 2020-07-31 12:05:39 -07:00
Ville Brofeldt 9d9c34835a
fix: support non-string groupbys for pie chart (#10493)
* chore: add unit tests to pie chart

* refine logic for floats and nans and add more tests
2020-07-31 11:19:21 +03:00
John Bodley 7645fc85c3
chore: Cleanup database sessions (#10427)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-07-30 23:07:56 -07:00
Jesse Yang 7ff1757448
build: optimize GitHub Actions for building speed and stability (#10485) 2020-07-30 22:28:56 -07:00
Jason Davis 7a329c25e9
feat: add test email functionality to SQL-based email alerts (#10476)
* added test email functionality

* formatting changes

* more formatting

* applied requested changes

* mypy

Co-authored-by: Jason Davis <@dropbox.com>
2020-07-30 13:27:22 -07:00
Hossein Torabi 259a344fd1
test: upload excel (#10467) 2020-07-29 12:50:39 -07:00
ʈᵃᵢ 78cad9a4a8
refactor(listviews): use correct filter endpoints for charts and datasets (#10442) 2020-07-29 11:00:19 -07:00
Daniel Vaz Gaspar 671461d0d0
feat(api): database schemas migration to new API (#10436)
* fix(log): log crashes if expired or not authenticated

* fix lint and rison

* add tests

* more tests

* perm fix

* fix test not found

* JS lint

* fix Jest test
2020-07-29 09:33:15 +01:00
Jason Davis d065633c81
fix: Implement updates to SQL-based email alerts (#10454)
* implemented updates to alerting

* fixed imports and teardown

* changed unittest to pytest conventions

* add app_context to tests

Co-authored-by: Jason Davis <@dropbox.com>
2020-07-28 16:48:42 -07:00
Beto Dealmeida 4f678272d6
fix(presto): Handle ROW data stored as string (#10456)
* Handle ROW data stored as string

* Use destringify

* Fix mypy

* Fix mypy with cast

* Bypass pylint
2020-07-28 16:05:58 -07:00
Ville Brofeldt fc28c92f57
feat: support non-numeric columns in pivot table (#10389)
* fix: support non-numeric columns in pivot table

* bump package and add unit tests

* mypy
2020-07-28 10:40:53 +03:00
ʈᵃᵢ 6f56cd5e9d
feat(listviews): SIP-34 filters for charts, dashboards, datasets (#10335) 2020-07-27 10:14:11 -07:00
Daniel Vaz Gaspar 562012c586
fix(permissions): alpha role has all full features (#10241)
* fix(permissions): alpha role is inconsistent

* reverse and allow Alpha to access manager menu

* Bump FAB to 3.0.1rc1 to include del permission fix

* add docs, tests and UPDATING

* EOL

* Fix query view for Alpha
2020-07-27 10:32:30 +01:00
Ville Brofeldt 5b6ff4944c
fix: incorrect filter operator emitted by Filter Box (#10421)
* fix: equals operator in filter box

* fix time range endpoint schema

* fix test

* bump packages

* fix tests

* lint
2020-07-26 12:40:03 +03:00
Daniel Vaz Gaspar 2fd37b18e3
fix(api): fixes perf on charts and introduces sorting by database on datasets (#10392) 2020-07-23 11:19:05 -07:00
Daniel Vaz Gaspar 98a11e7cf2
fix(api): fixes openapi spec errors and adds a test to validate all spec (#10393) 2020-07-23 10:43:41 +01:00
Daniel Vaz Gaspar 961108625e
fix(datasets): sort and humanized modified by (#10380)
* fix(datasets): sort and humanized modified by

* size xl on changed_on_delta_humanized

* Fix, tests

* Fix, tests

* Fix, frontend tests

* remove debug flag
2020-07-22 11:34:47 +01:00
Maxime Beauchemin 85169f2f1c
chore: remove QueryView (ModelView) (#10162)
* chore: deprecate QueryView (ModelView)

* lint

* fix unit tests
2020-07-21 09:14:15 -07:00
Erik Ritter 5fa4680447
feat: update timeout error UX (#10274) 2020-07-20 15:32:17 -07:00
Bogdan 7075c2ff69
Pin slack client, breaking change (#10345)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-07-20 09:20:05 -07:00
Ville Brofeldt 7af8b2b3f8
feat: add optional prophet forecasting functionality to chart data api (#10324)
* feat: add prophet post processing operation

* add tests

* lint

* whitespace

* remove whitespace

* address comments

* add note to UPDATING.md
2020-07-20 18:46:51 +03:00
bryanck 73797b8b64
check for empty dataframes before processing results (#10361)
Co-authored-by: Bryan Keller <bkeller@netflix.com>
2020-07-19 17:19:30 +03:00
Daniel Vaz Gaspar 74cb82e1ad
fix: humanised changed on UTC on dashboards and charts (#10321)
* fix: API marshmallow3 drop utc for naive datetime fields

* fix: API marshmallow3 drop utc for naive datetime fields

* fix, tests

* isort and test

* black

* add and fix test

* fix comment
2020-07-15 19:09:32 +01:00
Ville Brofeldt ac85aebe4a
chore: prefer allow/deny terminology (#10320)
* chore: prefer allow/deny terminology

* fix tests

* add PR reference
2020-07-15 19:22:46 +03:00
Ville Brofeldt 42b6347fca
fix: fix csv and query result type and QueryObject schema (#10312) 2020-07-14 16:37:19 +03:00
Ville Brofeldt c44ee06b5d
fix(chart-data-api): improve chart data endpoint errors (#10300)
* fix: improve chart data error response

* Populate error_message in QueryResult

* add tests

* Lint + fix incorrect raise

* add more tests
2020-07-14 12:40:00 +03:00
Bogdan 7abe3e5182
Add more fields to the log API (#10271)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-07-13 18:06:33 -07:00
Ville Brofeldt b316f723a1
fix(chart-data-api): case insensitive evaluation of filter op (#10299)
* fix(chart-data-api): case insensitive evaluation of filter op

* fix(chart-data-api): case insensitive evaluation of filter op

* mypy

* remove print statement

* add test
2020-07-13 17:21:02 +03:00
Ville Brofeldt 14260f9843
feat: add contribution operation and fix cache_key bug (#10286)
* feat: add contribution operation and fix cache_key_bug

* Add contribution schema
2020-07-10 17:06:05 +03:00
serenajiang c67b1abfd8
fix(csv upload): hive params typo (#10273)
* fix(csv upload): fix :params typo

* fix tests
2020-07-09 11:25:35 -07:00
Daniel Vaz Gaspar 6224edd42d
fix: dashboard endpoint sig changed (#10220)
* fix(thumbnails): dashboard endpoint sig changed

* fix, flask get url for Superset.dashboard

* add simple test
2020-07-09 11:42:38 +01:00
Ville Brofeldt baeacc3c56
feat(chart-data-api): make pivoted columns flattenable (#10255)
* feat(chart-data-api): make pivoted columns flattenable

* Linting + improve tests
2020-07-08 13:35:53 +03:00
Daniel Vaz Gaspar 7ecff0e513
fix: datasets related objects, apply filter and openapi spec (#10252)
* fix, datasets related objects, apply filter and openapi spec

* fix, add 404 test

* fix, add 404 test

* fix test
2020-07-08 10:04:05 +01:00
Daniel Vaz Gaspar 878dbcda3f
feat(api): bump marshmallow and FAB to version 3 (#9964)
* feat(api): bump marshmallow and FAB to version 3

* revert query context tests changes

* obey mypy

* fix tests

* ignore types that collide with marshmallow

* preparing for RC2

* fix tests for marshmallow 3

* typing fixes for marshmallow

* fix tests and black

* fix tests

* bump to RC3 and lint

* Test RC4

* Final 3.0.0

* Address comments, fix tests, better naming, docs

* fix test

* couple of fixes, addressing comments

* bumping marshmallow
2020-07-07 13:26:54 +01:00
Lily Kuang b9e0678752
feat: dataset api endpoint for charts and dashboards count (#10235)
* create GET endpoint for charts and dashboards count associated to a dataset

* add test for chart and dashboard count dataset
2020-07-06 16:25:57 -07:00
Bogdan 84f8a51458
feat: support nulls in the csv uploads (#10208)
* Support more table properties for the hive upload

Refactor

Add tests, and refactor them to be pytest friendly

Use lowercase table names

Ignore isort

* Use sql params

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-07-06 13:26:43 -07:00
Maxime Beauchemin 318e5347bc
feat: Alerts! allowing users to set SQL-based email alerts with screenshots (#9944)
* feat: add support for alerts

* Add ModelViews

* First pass at adding scheduled Alert support

* Fix syntax errors

* Add log_retention and database to model

* Improving screenshots

* Still refactoring screenshots

* Pass down thumb_size properly

* Progress on screenshot endpoint

* Add alerts.last_eval_dttm and alert query logic

* Getting ready to split compute_screenshot and screenshot/{SHA}

* split the endpoints

* Improving alerting loop

* empty methods

* Add CLI command 'superset alert' that runs a sched loop

* Getting things to work

* email delivery

* A working email\!

* Add feature flag

* Add some description_columns to AlertModelView

* Little tweaks

* Use database.get_df, eval cells for truthiness

* Migrate thumbnail/screenshot functions to use new arg pattern

* Addressing PR feedback

* Update alerts DB migration down_revision

* Resolve _deliver_email arg conflict

* Make mypy happy

* Make isort happy

* Make pylint happy

Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
2020-07-06 13:12:27 -07:00
Bogdan b7c45fed80
fix: downgrade sqlparse and add unit test (#10165)
* Downgrade sqlparse and add unit test

* Explain why sqlparse is pinned

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-07-06 11:10:46 -07:00
Maxime Beauchemin 2314aad450
feat: add database dropdown to dashboard import (#10118)
* feat: add database dropdown to dashboard import

Currently, when importing a database from a JSON file, the process
looks at the database name from the source (the info is in the file)
and matches the datasources to that name. If no database by that name
exists, it simply fails.

With this PR, we add a database dropdown that allows the user to specify
which databases the datasources should target as the get upserted.

I want to stress that the code in this area is not in a great shape,
and that the challenge of serializing/deser the nested objects is
challenging, but that there should be a much better way to do this.
One of the improvement (out of scope for this PR) that would allow to
simplify those import/export would be to use UUIDs for
importable/exportable objects.

Another identified issue is the indirections between
`utils/import_expor_{model}.py` on top of `{Model}.import_object`. Not
addressing that here.

Next topic is the MVC stuff. Decided to stick with it for now as this is
more of a [obious missing feat:] than a rewrite.

* isort \!? 0%^$%Y$&?%$^?%0^?

* fix tests

* pre-committing to py3.6

* address dpgaspar's comments

* revert isort
2020-07-05 15:08:37 -07:00
John Bodley b181e48f5c
[log] Updating form-data logic (#10197)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-07-04 12:51:43 -07:00
Bogdan 77b36783e7
s/assertEqual/assertEquals (#10195)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-29 16:38:06 -07:00
Bogdan 4e340c8368
feat: Switch from nosetest to pytest (#10177)
* Switch from nosetest to pytest

Fix schedule tests

Collect pytest coverage

Move pytest config into pytest.ini

Move cov to the pytest.ini

* Append coverage for the 2nd run

* Add coverage to all commands

* Coverage only for tests

* Get coverage from 1 place

* Rename classes to be  pytest compatible

* Test coverage for examples and tests

* Max diff to -1

* Explain how to run pytest for the whole project

* Do not append code coverage for the main run

* Do not run coverage on examples

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-29 15:36:06 -07:00
Jason Davis 72fc581b15
added explore database for ctas/cvas (#10174)
Co-authored-by: Jason Davis <@dropbox.com>
2020-06-29 11:13:54 -07:00
Ville Brofeldt 26d34727d3
fix: make time grain nullable in chart data endpoint (#10187)
* fix: make time grain nullable

* add test

* lint
2020-06-29 09:54:01 +03:00
Jesse Yang 9bdfa055ac
feat(viz): add query mode switch to table chart (#10113)
1, Replace table chart rendering from jquery.DataTables to react-table: apache-superset/superset-ui#623
2. Rearrange the control panel, replace GROUP BY and NOT GROUP BY with a "Query Mode" switch: apache-superset/superset-ui#609
2020-06-28 21:37:04 -07:00
Will Barrett 4965d87505
Break some static methods out of superset.views.core.Superset (#10175) 2020-06-26 14:34:45 -07:00
Jason Davis 9de9e1c19d
feat: Added configuration to SQL Lab results "Explore" button (#10164)
* added configuration to virtual table explore button

* added description to mixin

* fixed unit tests

Co-authored-by: Jason Davis <@dropbox.com>
2020-06-26 09:54:55 -07:00
Will Barrett df71fac1e0
refactor: Fix lint on superset/utils/core.py (#10120)
* Fix lint on superset/utils/core.py

* black

* mypy

* Fix some missing renames
2020-06-26 08:49:12 -07:00
Ville Brofeldt 410c5be2f8
fix: return slice owners for /user_slices ep (#10171)
* fix: return slice wowners for /user_slices ep

* put back created_by and changed_by

* improve comment
2020-06-26 18:10:56 +03:00
Maxime Beauchemin 1781ebbaa4
fix: [search query view] edit link is broken (#10128)
* fix: [search query view] edit link is broken

* eslint + mypy

* rever app.py changes

* addressing comments

* use api/v1/query

* fix test
2020-06-25 12:20:10 -07:00
Ville Brofeldt b205ce32b0
feat: Add new timegrains and convert_dttm to Druid engine spec (#10160)
* feat: Add new timegrains and convert_dttm to Druid engine spec

* Add TemporalType enum and fix test case

* Remove DATETIME for athena (original spec)
2020-06-25 12:18:37 +03:00
Jason Davis c544ebd876
fix: Fixed Attribute Error when running core_tests.py (#10154)
* fix: changed import order to fix core-test error

* reformatted using black

Co-authored-by: Jason Davis <@dropbox.com>
2020-06-24 12:46:38 -07:00
John Bodley 37777f382f
fix(security): dbs/clusters perm (#10130)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-24 10:06:25 -07:00
Bogdan 3db76c6fdc
Implement create view as functionality (#9794)
Implement create view as button in sqllab

Make CVAS configurable

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-24 09:50:41 -07:00
John Bodley aefef9ca55
chore(security): Updating assert logic (#10034)
* chore(security): Updating assert logic

* Deprecating rejected_tables

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-23 20:49:39 -07:00
Aliaksei Kushniarevich 550e78ff7c
feat: Add multiple table filters for Row Level Security (#9751)
* Add multiple table filters for Row Level Security

* Set ENABLE_ROW_LEVEL_SECURITY back to False (default)

* Merge DB migrations

* Drop table_id column and foreign key on PostgreSQL, MySQL, SQLite

* Support db records migration also

* Support downgrading from the new-fashioned formatted records

* Straighten up migrations

* Update migration's down_revision to comply master branch
2020-06-22 12:51:08 +03:00
Daniel Vaz Gaspar dbc43d7c7b
perf(datasets): improve datasets API performance for attr kind (#10111)
* perf(datasets): improve datasets API performance for attr kind

* fix test
2020-06-22 08:21:06 +01:00
Bogdan be6b9b8fec
feat: implement dttm column configuration through db extra config (#9444)
* Implement table mutator and examples how to set date defaults

Fix tests

* Fix flaky test

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 13:46:45 -07:00
Will Barrett 01aede0652
refactor: Break up superset/views/core.py (#10078)
* Remove unreferenced function from views/core.py

* Remove excess constants from views/core.py

* Extract CssTemplate-related views to their own file from core.py

* Remove duplicate constant declaration and make the constant less racist

* Move free-floating functions in views/core.py to views/utils.py

* Move AccessRequestsModelView out of views/core.py into its own module

* Move health checks and KV ModelView out of core.py and into their own modules

* Move R model view to its own module

* Move after-request header setting to views/base.py from views/core.py

* black

* mypy

* isort

* Fix reference to imported app

* pylint

* Fix some imports

* Add some missing view imports

* Fix a missing import
2020-06-17 13:42:13 -07:00
Bogdan 29e9f2c70b
feat: superset report slack integration (#9810)
* First draft for the slack integration

Fix slack

another typo

another typo

Fix slack

Add channels to the form

another typo

Another set of changes

Make code more transparent

Fix tests

Add logging

logging

use logger

import logging

import logging

import logging

add assert

more logging

Fix channels

Fix channels

* Address comments

* Move slack into a separate module

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 11:01:25 -07:00
John Bodley eba1fe2ebb
fix: Ensuring queries route accepts float or int (#10079)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-17 08:02:49 -07:00
ʈᵃᵢ 26120467af
fix: use custom int parsing over flask int parsing in sqllab queries endpoint (#10070) 2020-06-16 12:58:32 -07:00
Erik Ritter 7504b1cf20
fix: caching on viz with relative time ranges and time compare (#10061) 2020-06-16 08:47:35 -07:00
Ville Brofeldt a3393c1bc4
fix: native annotations (#10037)
* fix: native annotations

* Add test

* Add comment to test
2020-06-12 21:53:26 +03:00
John Bodley 9532bff48f
chore(security): Renaming access methods (#10031)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-11 13:12:23 -07:00
John Bodley 54c6ddbdb7
chore(security): Renaming schemas_accessible_by_user (#10030)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-11 11:29:43 -07:00
Will Barrett 4ef26993e2
feat: Add owners and physical/virtual status to Datasets API list endpoint (#10025)
* Add owners and physical/virtual status to Datasets API list endpoint

* black

* Update type definition for DatasourceKind

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Fixin' tests and mypy

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-06-10 22:04:40 +03:00