Commit Graph

864 Commits

Author SHA1 Message Date
Daniel Vaz Gaspar
790ac5ef87
feat(databases): security perm simplification (#12036)
* feat(databases): security perm simplification

* fix tests

* fix JS tests
2020-12-17 12:37:05 +00:00
Daniel Vaz Gaspar
dd5cdb1b5f
feat(dashboards): security permissions simplification (#12012)
* feat(dashboards): security perm simplification

* fix tests

* fix tests

* remove unnecessary mapping

* fix JS tests

* fix cypress test and explore endpoint

* update alembic down revision

* Merge branch 'master' into feat/security-converge-dashboards

# Conflicts:
#	superset/constants.py
#	tests/security_tests.py
2020-12-17 09:53:13 +00:00
Kasia Kucharczyk
700dee6db6
feat(logs): security permissions simplification (#12061)
* Added migration for logs security converge

* Changed class permission name and method permission in LogModelView and LogRestApi

* Updated recent revision and filename

* Changed name of Log perm in manager. Updated TestRolePermission to have correct menu and permission.

* Updated latest migration revision

* Updated latest migration revision
2020-12-17 00:50:31 -08:00
Daniel Vaz Gaspar
2302adb61a
feat(datasets): security perm simplification (#12000)
* feat(datasets): security perm simplification

* feat(datasets): security perm simplification

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* include SqlMetricInlineView converge and fix JS tests

* update to current alembic revision
2020-12-16 11:49:03 +00:00
Kasia Kucharczyk
9c8b65d03f
feat(annotations): security permissions simplification (#12014)
* Changed security permissions for annotations and annotation layers

* Updated permissions in annotation layers list

* Created test for retrieving premissions info. Updated uris from f-strings to strings

* Updated annotations in security_tests and added annotations to NEW_SECURITY_CONVERGE_VIEWS

* Added migration for annotations security converge

* Updated current revision after rebase master

* Updated migration name to annotations security converge

* Updated annotations permissions names in AnnotationLayersList and updated test since 'can_write' has wider permissions

* Updated annotations migration to current
2020-12-16 09:08:06 +00:00
Beto Dealmeida
8bda6b0bd9
feat: show missing parameters in query (#12049)
* feat: show missing parameters in query

* Fix lint

* Address comments

* Simplify error message

* Use f-string in helper function
2020-12-15 18:47:40 -08:00
Grace Guo
8da1900d8a
feat: add hook for dataset health check (#11970)
* feat: add hook for dataset health check

* add event log

* optimize datasource json data like certified data

* add unit test

* fix review comments

* extra code review comments
2020-12-15 18:12:06 -08:00
Jesse Yang
76f9f185fb
refactor: optimize backend log payload (#11927) 2020-12-15 17:22:23 -08:00
Daniel Vaz Gaspar
f79e52f48e
feat(charts): security perm simplification (#11981)
* feat(charts): security perm simplification

* fix superset explore

* fix JS test

* fix cypress test

* fix split heads

* fix favorite permission

* fix permission

* update with new async permission

* fix new permission coming from master

* fix core permission assert

* black

* update alembic down revision
2020-12-15 11:27:06 +00:00
Daniel Vaz Gaspar
0f979dea06
feat(reports): security perm simplification (#11853)
* feat: security converge report

* black

* fix: comment

* add frontend changes and rebase

* fix multiple heads
2020-12-15 08:48:00 +00:00
Daniel Vaz Gaspar
20b1aa7d6c
fix(reports): apply owners security validation (#12035)
* fix(reports): apply owners security validation

* fix pylint
2020-12-15 08:43:31 +00:00
ʈᵃᵢ
6fcda5dac1
feat: add cron picker to AlertReportModal (#12032)
* humanize cron display in list view
2020-12-14 22:29:31 -08:00
Beto Dealmeida
45703a1dea
feat: update IDs when importing dashboards (#11991)
* feat: update IDs when importing dashboards

* Fix typos
2020-12-11 18:19:36 -08:00
Rob DiCiuccio
4d329071a1
feat(SIP-39): Async query support for charts (#11499)
* Generate JWT in Flask app

* Refactor chart data API query logic, add JWT validation and async worker

* Add redis stream implementation, refactoring

* Add chart data cache endpoint, refactor QueryContext caching

* Typing, linting, refactoring

* pytest fixes and openapi schema update

* Enforce caching be configured for async query init

* Async query processing for explore_json endpoint

* Add /api/v1/async_event endpoint

* Async frontend for dashboards [WIP]

* Chart async error message support, refactoring

* Abstract asyncEvent middleware

* Async chart loading for Explore

* Pylint fixes

* asyncEvent middleware -> TypeScript, JS linting

* Chart data API: enforce forced_cache, add tests

* Add tests for explore_json endpoints

* Add test for chart data cache enpoint (no login)

* Consolidate set_and_log_cache and add STORE_CACHE_KEYS_IN_METADATA_DB flag

* Add tests for tasks/async_queries and address PR comments

* Bypass non-JSON result formats for async queries

* Add tests for redux middleware

* Remove debug statement

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Skip force_cached if no queryObj

* SunburstViz: don't modify self.form_data

* Fix failing annotation test

* Resolve merge/lint issues

* Reduce polling delay

* Fix new getClientErrorObject reference

* Fix flakey unit tests

* /api/v1/async_event: increment redis stream ID, add tests

* PR feedback: refactoring, configuration

* Fixup: remove debugging

* Fix typescript errors due to redux upgrade

* Update UPDATING.md

* Fix failing py tests

* asyncEvent_spec.js -> asyncEvent_spec.ts

* Refactor flakey Python 3.7 mock assertions

* Fix another shared state issue in Py tests

* Use 'sub' claim in JWT for user_id

* Refactor async middleware config

* Fixup: restore FeatureFlag boolean type

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-12-10 20:21:56 -08:00
Beto Dealmeida
475f59cb1c
feat: confirm overwrite when importing (#11982)
* feat: confirm overwrite when importing

* Skip flaky test
2020-12-10 14:50:10 -08:00
Kasia Kucharczyk
4da47f1635
tests: added fixture with energy usage instead of example (#11308)
* Added energy usage fixture. Updated dashboard utils and unicode fixture with new method parameters.

* Add energy fixture to tests/access_tests.py

* Add energy fixture to tests/core_tests.py

* Add energy fixture to tests/dashboard_tests.py

* Add energy fixture to tests/datasets/api_tests.py

* Add energy fixture to tests/db_engine_specs/base_engine_spec_tests.py

* Add energy fixture to tests/import_export_tests.py

* Add energy fixture to tests/model_tests.py

* Add energy fixture to tests/query_context_tests.py

* Add energy fixture to tests/security_tests.py

* Add energy fixture to tests/charts/api_tests.py

* Changed formatting of slices' parameters in energy usage fixture

* Removed loading energy udage data from test conf file

* Add energy fixture to tests/databases/api_tests.py

* Fixes after review: removed isort:skip, load_charts->load_energy_charts, removed unused import.

* Added energy fixture to tests/charts/commands_tests.py and retrieving proper Slice by name

* Fixed charts/api_tests.py to use energy_usage from fixtures

* Fixed datasets/commands_tests.py to retrieve dataset by name and use energy_usage fixture

* Changed energy usage data to generated data and fixed chart tests which was checking energy usage data
2020-12-09 12:02:29 -08:00
Daniel Vaz Gaspar
1e3aaab590
fix(reports): validator_config, report state machine, working_timeout (#11890)
* fix(reports): expect more exceptions and fix validator config

* use a state pattern on command reports

* use a state pattern on command reports continue

* fix multiple heads

* fix unittests

* add more tests

* fix api tests after enum rename

* fix alembic multiple heads

* fix tests

* fix fixture cleanup

* fix mysql tests

* fix initial and not found state

* fix schema, and private public methods, addressing comments

* add new col to the API
2020-12-09 18:19:07 +00:00
Beto Dealmeida
33325f9fa6
feat: add modal to import charts (#11956)
* WIP

* Splat props
2020-12-07 18:33:59 -08:00
Beto Dealmeida
2b9695c520
feat: add modal to import databases (#11884)
* feat: add modal to import databases

* Fix test

* Improve hook

* Remove log and needless store.

* Change JS functions
2020-12-07 11:22:45 -08:00
Beto Dealmeida
66cd565bff
feat: add Postgres SQL validator (#11538)
* Add Postgres SQL validator

* Strip line number from message

* Add unit tests

* Run tests only with postgres backend

* Add dep

* Add dep to bashlib
2020-12-04 19:17:23 -08:00
Erik Ritter
77d362d306
fix: Remove expensive logs table migration (#11920) 2020-12-04 07:59:28 -08:00
Ville Brofeldt
327a2817d3
feat: add event and interval annotation support to chart data ep (#11665)
* feat: add event and interval annotation support to chart data ep

* add tests + refactor fixtures

* use chart dao
2020-12-04 14:40:31 +02:00
Beto Dealmeida
e0288bf76b
chore: add unit tests for the current import functionality (#11786)
* chore: add unit tests for the current import functionality

* Improve comment

* Fix unit test
2020-12-03 14:48:43 -08:00
Bogdan
0396c705d4
chore: Support SET & SHOW commands as read only SQL commands (#11868)
* Support SET & SHOW commands as read only SQL commands

* Move is_readonly definition into the engine spec

* Rename & use super()

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-12-03 10:44:11 -08:00
Daniel Vaz Gaspar
71c567bf3a
feat(css templates): security perm simplification (#11856)
* feat: security converge css templates

* fix security tests

* fix JS test

* add migration

* black and fix migration
2020-12-03 08:45:32 +00:00
Lily Kuang
a76eadd838
feat: report/alert list CRUD view (#11802) 2020-12-01 17:57:15 -08:00
Kasia Kucharczyk
462b39d762
fix: update chart doesn't remove all connections to dashboards (#11830)
* Added possibility to not change dashboards connection when chart is changed

* Added two chart unit tests which checks if dashboards properly change on update
2020-11-30 09:48:14 -08:00
Hugh A. Miles II
0689738e7c
feat: Sqllab to Explore UX improvements api changes (#11836) 2020-11-30 09:23:11 -08:00
Daniel Vaz Gaspar
9dd33d5566
feat(saved queries): security perm simplification (#11764)
* feat(saved queries): security perm simplification

* migration script and frontend

* add downgrade procedure

* downgrade procedure, inferred from the upgrade data

* fix JS test

* improve code

* add tests for role migration

* more tests and a short description for each one

* Improve readability

* simplify dataclass creation

* fix dataclass
2020-11-30 13:07:39 +00:00
Daniel Vaz Gaspar
bac84a3aac
fix: delete chart, dashboards, dbs with assoc reports (#11801)
* fix: delete chart or dashboards with assoc reports

* database constraint to reports and tests

* add tests for dashboards and database

* fix exceptions default text
2020-11-26 08:45:49 +00:00
Beto Dealmeida
c354e7e0ab
fix: remove root dir from ZIP bundle (#11805)
* fix: remove root dir from ZIP bundle

* Fix lint

* Fix tests

* Improve tests

* Fix dashboard as well
2020-11-25 11:47:48 -08:00
Daniel Vaz Gaspar
f27ebc4be5
feat: new reports scheduler (#11711)
* feat(reports): scheduler and delivery system

* working version

* improvements and fix grace_period

* add tests and fix bugs

* fix report API test

* test MySQL test fail

* delete-orphans

* fix MySQL tests

* address comments

* lint
2020-11-25 08:50:30 +00:00
Beto Dealmeida
501b9d47c5
feat: API endpoint to import dashboards (#11751)
* ImportChartsCommand

* feat: add a command to import dashboards

* feat: API endpoint to import dashboards

* Add dispatcher

* Raise specific exception

* Fix test

* Remove print calls

* Add logging when passing
2020-11-24 22:45:35 -08:00
Erik Ritter
c0224aa928
feat: add certification info to table selector (#11785) 2020-11-24 12:13:52 -08:00
Beto Dealmeida
25345bea64
feat: add a command to import dashboards (#11749)
* feat: add a command to import dashboards

* Fix lint

* Remove print()
2020-11-23 16:28:26 -08:00
ʈᵃᵢ
fbe4a6622e
feat: SQL preview modal for Query History (#11634) 2020-11-20 16:01:06 -08:00
Beto Dealmeida
a3a2a68f01
feat: API endpoint to import charts (#11744)
* ImportChartsCommand

* feat: API endpoint to import charts

* Add dispatcher

* Fix docstring
2020-11-20 14:40:27 -08:00
Beto Dealmeida
2f4f87795d
feat: add a command to import charts (#11743)
* ImportChartsCommand

* Fix type
2020-11-20 14:20:13 -08:00
Ville Brofeldt
7ae8cd07cc
fix: do not drop calculated column on metadata sync (#11731) 2020-11-18 08:33:14 -08:00
Beto Dealmeida
cce716a821
feat: API endpoints to upload dataset/db (#11728)
* feat: API endpoints to upload dataset/db

* Fix method call
2020-11-17 14:49:33 -08:00
Rob DiCiuccio
01d15f5368
feat(templating): Safer Jinja template processing (#11704)
* Enable safer Jinja template processing

* Allow JINJA_CONTEXT_ADDONS with SAFE_JINJA_PROCESSING

* Make template processor initialization less magical, refactor classes

* Consolidat Jinja logic, remove config flag in favor of sane defaults

* Restore previous ENABLE_TEMPLATE_PROCESSING default

* Add recursive type checking, update tests

* remove erroneous config file

* Remove TableColumn models from template context

* pylint refactoring

* Add entry to UPDATING.md

* Resolve botched merge conflict

* Update docs on running single python test

* Refactor template context checking to support engine-specific methods
2020-11-17 11:55:47 -08:00
Beto Dealmeida
7bc353f8a8
feat: new import commands for dataset and databases (#11670)
* feat: commands for importing databases and datasets

* Refactor code
2020-11-16 17:11:20 -08:00
Bogdan
a16a2b87c7
chore: statds metrics for cache invalidation (#11701)
* Add more logging to the cache invalidation

* Address linter & tests

* Fix linter

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-11-16 06:48:26 -08:00
Jesse Yang
4cfcaebb61
feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG (#11509)
* feat: rename TABLE_NAMES_CACHE_CONFIG to DATA_CACHE_CONFIG

The corresponding cache will now also cache the query results.

* Slice use DATA_CACHE_CONFIG CACHE_DEFAULT_TIMEOUT

* Add test for default cache timeout

* rename FAR_FUTURE to ONE_YEAR_IN_SECS
2020-11-13 22:35:10 -08:00
Jesse Yang
ec8ccd4cf1
feat: keep modal open when saving database failed (#11618) 2020-11-12 16:10:14 -08:00
ʈᵃᵢ
432e5ab460
feat: Query History CRUD list view (#11574) 2020-11-12 11:55:13 -10:00
Daniel Vaz Gaspar
12cb27f5cb
feat: new reports models api (#11606)
* feat: new report schedule models

* lint and unique constraint

* support sqlite

* fix sqlite

* add audit mixin and minor fixes

* feat(api): alerts and reports REST API

* feat: new report schedule models

* lint and unique constraint

* support sqlite

* fix sqlite

* add audit mixin and minor fixes

* feat(api): alerts and reports REST API

* draft working version

* add tests

* test

* black

* remove copy pasta

* solve dashboard object representation being used on cache

* tests and custom filter

* fix PUT has PATCH on active field

* create feature flag

* fix lint

* address comments
2020-11-12 13:21:01 -08:00
Grace Guo
0338b48fdd
fix (dashboard): remove default filters parameters in dashboard url (#11661) 2020-11-12 13:00:27 -08:00
Jesse Yang
542d2e3b06
chore: clean up a debug line from #11653 (#11663) 2020-11-12 08:37:49 -08:00
Ville Brofeldt
2718909314
feat: add ECharts BoxPlot chart (#11199)
* feat: add ECharts BoxPlot chart

* lint

* fix cypress tests

* lint

* remove viz.py shim

* bump plugin package

* skip non-legacy plugin cypress dashboard tests

* fix cypress tests

* disable cypress tests for non-leagcy charts

* fix bad rebase

* use midpoint interpolation for quartile calculation

* bump packages and add support for no groupby

* whitespace

* whitespace

* linting

* fix tests

* xit mathjs load test

* bump mathjs to 8.0.1

* disable cypress filter test for v1 charts
2020-11-12 10:01:04 +02:00