Commit Graph

68 Commits

Author SHA1 Message Date
Karol Kostrzewa 241f380e2e
test: World bank examples (#12161)
* add world bank data fixture

* fix fixture cleanup, add fixture to dashboard_tests

* apply world bank fixtures, fix tests

* fix fixture typo, dashboard ids

* fix export dashboard metadata

* fix test_export_dashboard_command_key_order

* fix export dash tests, not add row when no orphans

* debug timeout

* fixes after merge

* fix lint

* run pre-commit

* comment test for debug

* fix save.test.js

Co-authored-by: Karol Kostrzewa <karol.kostrzewa@polidea.com>
2021-01-13 14:20:05 -08:00
Karol Kostrzewa 2ab490a101
test: birth names (#12226)
* add birth names fixture

* fix birth names related tests

* fix test_import_v0_dataset_cli_export columns order

* fix celery tests drop table

* fix mysql datetime type

* fix mysql typo in charts/api_tests

* refactor

* add licence

* fix use fixture for presto

* fix presto, hive query

* fix flaky metadata

* fix mysql bigint type

* fix run query

* fix hive datatype in metadata

* fix slice owner for cypress

* refactor num_boys num_girls

* fix is_dttm column

* debug logging

* fix query offset

* fix presto ds type in metadata

* fix presto ds type

* clean up debug logging
2021-01-11 05:57:55 -08:00
Karol Kostrzewa 1a2680d4a1
test: fix flaky Python unit tests (#12253) 2021-01-05 16:40:01 -08:00
Daniel Vaz Gaspar 2f0add3aec
fix: database alpha permissions (#12136)
* fix: database alpha permissions

* add test
2020-12-21 19:17:58 +00:00
Daniel Vaz Gaspar 5d9721e6ba
feat(queries): security perm simplification (#12072)
* feat(queries): security perm simplification

* fix mig

* update alembic down revision
2020-12-17 14:27:14 +00:00
Daniel Vaz Gaspar 790ac5ef87
feat(databases): security perm simplification (#12036)
* feat(databases): security perm simplification

* fix tests

* fix JS tests
2020-12-17 12:37:05 +00:00
Daniel Vaz Gaspar dd5cdb1b5f
feat(dashboards): security permissions simplification (#12012)
* feat(dashboards): security perm simplification

* fix tests

* fix tests

* remove unnecessary mapping

* fix JS tests

* fix cypress test and explore endpoint

* update alembic down revision

* Merge branch 'master' into feat/security-converge-dashboards

# Conflicts:
#	superset/constants.py
#	tests/security_tests.py
2020-12-17 09:53:13 +00:00
Kasia Kucharczyk 700dee6db6
feat(logs): security permissions simplification (#12061)
* Added migration for logs security converge

* Changed class permission name and method permission in LogModelView and LogRestApi

* Updated recent revision and filename

* Changed name of Log perm in manager. Updated TestRolePermission to have correct menu and permission.

* Updated latest migration revision

* Updated latest migration revision
2020-12-17 00:50:31 -08:00
Daniel Vaz Gaspar 2302adb61a
feat(datasets): security perm simplification (#12000)
* feat(datasets): security perm simplification

* feat(datasets): security perm simplification

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* include SqlMetricInlineView converge and fix JS tests

* update to current alembic revision
2020-12-16 11:49:03 +00:00
Kasia Kucharczyk 9c8b65d03f
feat(annotations): security permissions simplification (#12014)
* Changed security permissions for annotations and annotation layers

* Updated permissions in annotation layers list

* Created test for retrieving premissions info. Updated uris from f-strings to strings

* Updated annotations in security_tests and added annotations to NEW_SECURITY_CONVERGE_VIEWS

* Added migration for annotations security converge

* Updated current revision after rebase master

* Updated migration name to annotations security converge

* Updated annotations permissions names in AnnotationLayersList and updated test since 'can_write' has wider permissions

* Updated annotations migration to current
2020-12-16 09:08:06 +00:00
Daniel Vaz Gaspar f79e52f48e
feat(charts): security perm simplification (#11981)
* feat(charts): security perm simplification

* fix superset explore

* fix JS test

* fix cypress test

* fix split heads

* fix favorite permission

* fix permission

* update with new async permission

* fix new permission coming from master

* fix core permission assert

* black

* update alembic down revision
2020-12-15 11:27:06 +00:00
Kasia Kucharczyk 4da47f1635
tests: added fixture with energy usage instead of example (#11308)
* Added energy usage fixture. Updated dashboard utils and unicode fixture with new method parameters.

* Add energy fixture to tests/access_tests.py

* Add energy fixture to tests/core_tests.py

* Add energy fixture to tests/dashboard_tests.py

* Add energy fixture to tests/datasets/api_tests.py

* Add energy fixture to tests/db_engine_specs/base_engine_spec_tests.py

* Add energy fixture to tests/import_export_tests.py

* Add energy fixture to tests/model_tests.py

* Add energy fixture to tests/query_context_tests.py

* Add energy fixture to tests/security_tests.py

* Add energy fixture to tests/charts/api_tests.py

* Changed formatting of slices' parameters in energy usage fixture

* Removed loading energy udage data from test conf file

* Add energy fixture to tests/databases/api_tests.py

* Fixes after review: removed isort:skip, load_charts->load_energy_charts, removed unused import.

* Added energy fixture to tests/charts/commands_tests.py and retrieving proper Slice by name

* Fixed charts/api_tests.py to use energy_usage from fixtures

* Fixed datasets/commands_tests.py to retrieve dataset by name and use energy_usage fixture

* Changed energy usage data to generated data and fixed chart tests which was checking energy usage data
2020-12-09 12:02:29 -08:00
Daniel Vaz Gaspar 71c567bf3a
feat(css templates): security perm simplification (#11856)
* feat: security converge css templates

* fix security tests

* fix JS test

* add migration

* black and fix migration
2020-12-03 08:45:32 +00:00
Kasia Kucharczyk 32e174d4b8
test: removed unicode_test example from unit tests (#11131)
* Removed depemdency to unicode example in tests config.

* Added common methods for creating dashboards for tests.

* Added fixtures to all tests which were using unicode example.

* Added cleanup for unicode_test table

* Removed unnecessary fixture parts of unicode dashboard tests

* Parametrized creating slice for tests

* Moved fixtures for unicode test to separate file and refactored to several methods. Added param types and return types.

* Cleandup after fix

* Changed variable names to more readable

* Added cleanup for dashboards and slices

* Applied unicode fixture to charts api tests

* Update schema variable to dtype in dashboard utils

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Changed variable schema to dtype in dashboards. Replaced accessing first element with one_or_none

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-10-07 18:29:37 +03:00
Ville Brofeldt 448a41a4e7
feat(row-level-security): add base filter type and filter grouping (#10946)
* feat(row-level-security): add filter type and group key

* simplify tests and add custom list widget

* address comments

* use enum value to ensure case sensitive value is used
2020-09-22 08:54:10 +03:00
Ville Brofeldt 3ae80d3b98
test: add rls jinja extra cache key test (#10810) 2020-09-08 07:42:48 +03:00
Maxime Beauchemin 3d8857ecc8
style: rename 'Source' top nav item to 'Data' (#10715)
* style: rename 'Source' top nav item to 'Data'

* Fix tests

* tables -> datasets

* rename datasource -> dataset

* lint

* more renaming

* use npm ci
2020-08-29 18:09:47 -07:00
Daniel Vaz Gaspar 3e374dab07
fix: change public role like gamma procedure (#10674)
* fix: change public role like gamma procedure

* lint and updating UPDATING with breaking change

* fix updating text

* add test and support PUBLIC_ROLE_LIKE_GAMMA

* fix, cleanup tests

* fix, new test

* fix, public default

* Update superset/config.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* add simple public welcome page

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-08-28 10:49:10 +01:00
Erik Ritter fd2d1c58c5
Revert "chore: Cleanup database sessions (#10427)" (#10537)
This reverts commit 7645fc85c3.
2020-08-06 15:33:48 -07:00
John Bodley 7645fc85c3
chore: Cleanup database sessions (#10427)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-07-30 23:07:56 -07:00
Daniel Vaz Gaspar 562012c586
fix(permissions): alpha role has all full features (#10241)
* fix(permissions): alpha role is inconsistent

* reverse and allow Alpha to access manager menu

* Bump FAB to 3.0.1rc1 to include del permission fix

* add docs, tests and UPDATING

* EOL

* Fix query view for Alpha
2020-07-27 10:32:30 +01:00
Ville Brofeldt ac85aebe4a
chore: prefer allow/deny terminology (#10320)
* chore: prefer allow/deny terminology

* fix tests

* add PR reference
2020-07-15 19:22:46 +03:00
Bogdan 77b36783e7
s/assertEqual/assertEquals (#10195)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-29 16:38:06 -07:00
Bogdan 4e340c8368
feat: Switch from nosetest to pytest (#10177)
* Switch from nosetest to pytest

Fix schedule tests

Collect pytest coverage

Move pytest config into pytest.ini

Move cov to the pytest.ini

* Append coverage for the 2nd run

* Add coverage to all commands

* Coverage only for tests

* Get coverage from 1 place

* Rename classes to be  pytest compatible

* Test coverage for examples and tests

* Max diff to -1

* Explain how to run pytest for the whole project

* Do not append code coverage for the main run

* Do not run coverage on examples

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-29 15:36:06 -07:00
John Bodley 37777f382f
fix(security): dbs/clusters perm (#10130)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-24 10:06:25 -07:00
John Bodley aefef9ca55
chore(security): Updating assert logic (#10034)
* chore(security): Updating assert logic

* Deprecating rejected_tables

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-23 20:49:39 -07:00
Aliaksei Kushniarevich 550e78ff7c
feat: Add multiple table filters for Row Level Security (#9751)
* Add multiple table filters for Row Level Security

* Set ENABLE_ROW_LEVEL_SECURITY back to False (default)

* Merge DB migrations

* Drop table_id column and foreign key on PostgreSQL, MySQL, SQLite

* Support db records migration also

* Support downgrading from the new-fashioned formatted records

* Straighten up migrations

* Update migration's down_revision to comply master branch
2020-06-22 12:51:08 +03:00
John Bodley 9532bff48f
chore(security): Renaming access methods (#10031)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-11 13:12:23 -07:00
John Bodley 54c6ddbdb7
chore(security): Renaming schemas_accessible_by_user (#10030)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-11 11:29:43 -07:00
Aliaksei Kushniarevich ef5e11f45b
[copy] fix: Row Level Security get_rls_filters func SELECT statement (#9541)
* fix: Row Level Security get_rls_filters func SELECT statement

* More general RowLevelSecurityTests case to avoid improper ids matching
2020-04-15 16:41:54 +03:00
altef dee16de03e
[SIP-29] Add support for row-level security (#8699)
* Support and apply filters.

* Added the UI for row level security, and moved it all under SQLA in order to access the Table model more easily.

* Added a row level security filter documentation entry.

* Accidentally added two new lines to this file.

* Blacked and iSorted, hopefully.  Also, sometimes g.user may not be set.

* Another isort, and handling g not having a user attribute another way.

* Let's try this again #CI tests.

* Adjusted import order for isort; I was sure I'd already done this..

* Row level filters should be wrapped in parentheses in case one contains an OR.

* Oops, did not think that would change Black's formatting.

* Changes as per @mistercrunch.

* RLS filters are now many-to-many with Roles.

* Updated documentation to reflect RLS filters supporting multiple rows.

* Let's see what happens when I set it to the previous revision ID

* Updated from upstream.

* There was a pylint error.

* Added RLS ids to the cache keys; modified documentation; added template processing to RLS filters.

* A new migration was merged in.

* Removed RLS cache key from query_object.

* RLS added to the cache_key from query_context.

* Changes as per @etr2460.

* Updating entry for RLS pull request.

* Another migration to skip.

* Changes as per @serenajiang.

* Blacked.

* Blacked and added some attributes to check for.

* Changed to a manual query as per @mistercrunch.

* Blacked.

* Another migration in the meantime.

* Black wanted some whitespace changes.

* AttributeError: 'AnonymousUserMixin' object has no attribute 'id'.

* Oops, did hasattr backwards.

* Changes as per @mistercrunch.

* Doesn't look like text us required here anymore.

* Changes as per @dpgaspar

* Two RLS tests.

* Row level security is now disabled by default via the feature flag ENABLE_ROW_LEVEL_SECURITY.

* New head to revise.

* Changed the comment.
2020-02-22 11:21:31 +02:00
John Bodley 8a138fbd03
[tox] Allowing running of specific tests (#9097) 2020-02-06 16:45:37 -08:00
ʈᵃᵢ e5f5eed425
[chart] new, list view (react) (#8999)
* [charts] new, list view (react)

* DRY up template rendering

* fix i18n

* lint package.json
2020-02-05 12:42:42 -08:00
Daniel Vaz Gaspar 1f21bf877f
[log] fix, log model view permissions (#8993)
Limit MVC access to admin role only and limit REST API permissions
2020-01-26 12:15:57 +00:00
Maxime Beauchemin 315a11dfe2
fix: shut off unneeded endpoints (#8960)
* fix: shut off all uneeded endpoints

We recently added a new feature to FAB allowing to whitelist the needed
endpoints in ModelView and ModelRestApi.

First, we set our base wrapper class to an empty set, forcing each
class inheriting from it to explicitely turn on the endpoints that
Superset intends to use.

Second, we go ModelView by ModelView to whitelist the actual endpoints
used in the app.

Notes:
* as a result a large set of [unneeded] permissions should be cleaned up
* outside of the "private" use of endpoints in the app, people that have
  been using endpoints in their environment for other purposes may
  experience loss of functionality

* Tweaking

* Reduce the amount of endpoints using white lists

* Fix, included needed endpoints for dashboard and druid

* Drying things up

* fixes

* limiting more endpoints

* Read only on some FAB model views

* fixing some tests

* fixes

* Fixing more tests

* Addressing comments

* Drying up route_methods

* further drying

Co-authored-by: Daniel Vaz Gaspar <danielvazgaspar@gmail.com>
2020-01-23 11:25:15 -05:00
ʈᵃᵢ 7b97764dbc [dashboard] New, list view (react) (#8845)
* adds dashboard listview component

* use new api

* use json over rison

* lint

* adds seperate dashboard list view

* edit and delete actions

* fix lint ignore

* fix common_bootstrap_payload is now a function

* fix license

* fix pylint

* isort

* fix tests

* lint

* lint ts

* fix js tests

* fix double import from bad rebase

* fix indent error

* lookup permissions

* generic permission lookup

* get tslint to pass

* adds js specs

* lint

* fix rebase

* lint

* lint again

* fix type errors preventing build

* adds more specs

* fix tslint error

* fix null check

* remove unecessary code

* use translations provided by api

* more translations

* linting

* fix spec

* i18n

* fix register order
2020-01-16 00:48:55 -05:00
serenajiang 1f6f4ed879 [db migration] change datasources-clusters foreign key to cluster_id (#8576)
* [db migration] change datasources foreign key to cluster_id

* address pr comments

* address pr comment, fix ci
2020-01-13 11:02:36 -08:00
David Aaron Suddjian 016f202423 Refactor Dashboard and Slice models (#8820)
* refactor dashboard and slice models

* appease various linters

* remove shortcuts & import indirection

* appease mypy

* fix bad imports

* lint

* address various issues

* ignore type issue

* remove unused imports

* lint
2019-12-18 11:40:45 -08:00
Will Barrett ed54f6ec8d Require the SQLAlchemy URI on the database model (#8720)
* Require the SQLAlchemy URI when creating a database

* Add migration to make dbs.sqlalchemy_uri not-nullable

* Fixes for black, isort, tests

* Alter migration to use current revision from master as downgrade target

* Update tests to support new db constraint

* black
2019-12-11 14:05:23 -08:00
Bogdan e6be51953c Avoid circular dependency between superset config and superset package (#8254)
Resolve comments

Avoid circular dependency between superset config and superset package

Resolve comments
2019-12-10 09:17:34 +00:00
Bogdan 003e98c231 Support schema permissions (#8219)
* Build support for schema access in Superset and SQLab

* Lint
* Test for old and new flask
* Black formatting
* Cleanup modified files
* Support schema permissions
* Add migration
* Make it work for slices as well
* TODO and remove unused field
* Apply schema access check to dashboards as well

Cleaner schema permissions check

Fetch schema permissions in 1 query

Bugfix

Better datasource check

Use security manager for permissions checks and remove dead code

Properly handle anon user

Add database permissions on the database updates

Fix schema filtering

Lint / flake changes

Increase cypress request timeout

Run black

Resolve multiple alembic heads

* Use set comprehensions

* Fixes for the pylint
2019-12-03 14:57:24 -08:00
Will Barrett 16c0599470 Disable deprecated druid connector by default (#8512)
* Disable deprecated druid connector by default

* Add a line in UPDATING.md for the configuration change

* Remove security tests related default-disabled feature

* More test updates

* black
2019-11-27 11:28:23 -08:00
Craig Rueda e490414484 Flask App factory PR #1 (#8418)
* First cut at app factory

* Setting things back to master

* Working with new FLASK_APP

* Still need to refactor Celery

* CLI mostly working

* Working on unit tests

* Moving cli stuff around a bit

* Removing get in config

* Defaulting test config

* Adding flask-testing

* flask-testing casing

* resultsbackend property bug

* Fixing up cli

* Quick fix for KV api

* Working on save slice

* Fixed core_tests

* Fixed utils_tests

* Most tests working - still need to dig into remaining app_context issue in tests

* All tests passing locally - need to update code comments

* Fixing dashboard tests again

* Blacking

* Sorting imports

* linting

* removing envvar mangling

* blacking

* Fixing unit tests

* isorting

* licensing

* fixing mysql tests

* fixing cypress?

* fixing .flaskenv

* fixing test app_ctx

* fixing cypress

* moving manifest processor around

* moving results backend manager around

* Cleaning up __init__ a bit more

* Addressing PR comments

* Addressing PR comments

* Blacking

* Fixes for running celery worker

* Tuning isort

* Blacking
2019-11-20 15:47:06 +00:00
Will Barrett e4ca44e95f Use `config[]` not `config.get()` (#8454)
* Typo fix in CONTRIBUTING.md

* Alter references to config.get('FOO') to use preferred config['FOO']

* Set missing configuration constants in superset/config.py

* Misc. CI fixes

* Add type annotation for FEATURE_FLATGS
2019-10-30 16:19:16 -07:00
John Bodley 9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
John Bodley 05b67673c3
[metric] Adding security for restricted metrics (#8175) 2019-10-01 16:52:15 -07:00
John Bodley f7af50c399
[security] Adding docstrings and type hints (#7952) 2019-08-05 11:24:13 -07:00
Gianluca Ciccarelli e23920b8ba 7620: Start removing dependencies on requests (#7643)
* 7620: Start removing dependencies on requests

* Patch urllib.request.urlopen instead of requests.get

* Try to fix flake8

* More work on flake8 import errors

* First attempt at using urllib with cookies

* Fix pylint/flake8

* Fix test_deliver_slice_csv_attachment

* Fix test_deliver_slice_csv_inline

* Import requests and pydruid conditionally, remove dependency on prison

* Fix flake errors

* Fix load_examples

* Please flake

* Skip tests depending on optional deps

* Try to please flake

* Address review comments

* Remove Druid-related UI

* Revert "Remove Druid-related UI"

This reverts commit d7e0f166cc3f3dd2496b4a666e177f0c191aeb0f.

* Skip a few tests more

* Put imports in right order

* Apply black patch

* Please flake

* Please black, silence flake

* Use flake8 silencing the right way

* Add deps for CI
2019-08-02 10:01:28 -07:00
Daniel Vaz Gaspar b65ab51b63 [api] [database] New, migrate to new FAB API side by side (#7665)
* [api] [database] New, migrate to new FAB API side by side

* [database] view and api module reorganization

* [style] Fix, flake8

* [test] [database] Fix, tests for new database view structure

* [style] [database] black

* [database] Fix, missing apache license

* Migrate databaasync api to the new api

* flake8

* More efficient api call

* Revert querySearch to old API, test

* dummy commit

* revert config.py commit

* Remove unused import

* Remove old API view

* Remove new API view

* Add database filter

* Add database filter

* Add database filter

* Remove comments on js

* Fix JS test

* Fix pylint

* QuerySearch new databaseasync API endpoint

* Fix, query search new api endpoint

* Reintroduce old API for smooth side by side migration

* Better naming and use common hooks for view and API

* black it

* Fix, database mixin must come first

* Fix, lint
2019-07-22 21:25:16 -07:00
John Bodley 5c58fd1802
[format] Using Black (#7769) 2019-06-25 13:34:48 -07:00