* chore: bump pyhton dependencies
* bump flask-testing
* Add note to UPDATING and bump flask-testing in requirements-dev.txt
* fix old sqlparse test case that is fixed with sqlparse==0.3.1
* Fix migration and sqlparse set test
* downgrade alembic
* Downgrade sqlalchemy-utils
* pin sqlalchemy-utils due to regression in 0.36.5
* Refine comment in setup.py
* make cachelib an explicit requirement
* Make all fields optional in QueryObject and fix having_druid schema
* fix: datasource type sql to table
* lint
* Add missing fields
* Refactor tests
* Linting
* Refactor query context fixtures
* Add typing to test func
* Add OpenAPI docs to /api/v1/chart/data EP
* Add missing fields to QueryObject, fix linting and unit test errors
* Fix unit test errors
* abc
* Fix errors uncovered by schema validation and add unit test for invalid payload
* Add schema for response
* Remove unnecessary pylint disable
* Deprecate groupby from query_obj
* Fix query_object bug
* Fix histogram
* Remove groupby from legacy druid connector and fix first batch of unit tests
* Deprecate some unnecessary tests + fix a few others
* Address comments
* hide SIP-38 changes behind feature flag
* Break out further SIP-38 related tests
* Reslove test errors
* Add feature flag to QueryContext
* Resolve tests and bad rebase
* Backport recent changes from viz.py and fix broken DeckGL charts
* Fix bad rebase
* backport #9522 and address comments
* Visualize after running Ctas query (#4)
* Visualize after running Ctas query.
Stub for table viz
Work on the viz
Add JS functionality for Ctas viz
Add test for the table viz flow
Fix test
Add js test for the ctas viz
* Fix tests
* Resolve comments
* Leverate tmpSchema in the query object
* Fix i18n string
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
* Add post processing to QueryObject
* Simplify sort signature and require explicit sort order
* Add new operations and unit tests
* linting
* Address comments
* Simplify test method names
* Address comments
* Linting
* remove unnecessary logic
* Apply strict whitelisting to all getattr calls
* Add checking of rolling_type_options and add/improve docs
* [dashboards] New, tittle and slug OR filter
* Update requirements, because of prison bump
* Tests
* Fix tests
* Avoid like filter on empty string value
* merge master brings strict typing to the table
* filter owners select by text input
* use rison
* fix backend owners filter logic
* use fullname, not username on owners inputs
* fix some tests
* fixing tests
* deterministic tests
* appease linter
* add back search by username
* more comprehensive filter test
* add clarifying text
* formatting...
* [explore view] fix long query issue from Run in SQL LAB Button
* SQL Lab page needs to take the post form data, too
* fix variable names
* updated payload dict, rename hidden form
Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
* feat: [explore] don't save filters inherited from a dashboard
When navigating to explore from a dashboard context, the current
dashboard filter(s) are passed along to explore so that the context is
kept. So say you're filtering on "country=Romania", in your dashboard
and pivot to explore, that filter is still there and keep on exploring.
Now a common issue is that you'll want to make some tweak to your chart
that are unrelated to the filter, say toggling the legend off for
instance, and then save it. Now you back to your dashboard and even
though you started with an "all countries" dashboard, with a global
filter on country, now that one chart is stuck on "Romania". Typically
you notice this when filtering on something else, say "Italy" and then
that one chart now has two mutually exclusive filters, and show "No data".
Now, the fix is to flag the filter as "extra" (that's the not-so-good internal
name we use for these inherited filters) and make it clear that that
specific filter is special and won't be saved when saving the chart.
* fix build
* [dashboard] Refactor API using SIP-35
* [dashboard] Fix, import
* [dashboard] more tests
* [dashboards] a misc of improvements
* [charts] Fix, DAO and tests
* [dashboards] small exceptions refactor
* [dashboards] lint
* [dashboards] Improves comments on base classes
* [dashboards] lint
* [datasets] new, react listview
* add hidden columns to support filtering by columns not rendered
* throw exception if config is incorrect
* fix database filter
* update endpoints to point to datasets; fix translation strings
* move Link into src/components
* add add new record button to datasets listview
* fix: change database save in DatasourceEditor
This addresses the issue where pointing a datasource to another database
in the datasource editor is not reflected.
Also addresses:
- a minorcosmetic issue in the datasource editor.
- user/owners list not getting populated
* tests
* Make schema name configurable
Fixing unit tests
Fix table quoting
Mypy
Split tests out for sqlite
Grant more permissions for mysql user
Postgres doesn't support if not exists
More logging
Commit for table creation
Priviliges for postgres
Update tests
Resolve comments
Lint
No limits for the CTA queries if configures
* CTA -> CTAS and dict -> {}
* Move database creation to the .travis file
* Black
* Move tweaks to travis db setup
* Remove left over version
* Address comments
* Quote table names in the CTAS queries
* Pass tmp_schema_name for the query execution
* Rebase alembic migration
* Switch to python3 mypy
* SQLLAB_CTA_SCHEMA_NAME_FUNC -> SQLLAB_CTAS_SCHEMA_NAME_FUNC
* Black
* Returning timeseries_limit_metric in table viz get_data
* Fixing issue with include_time field
* Reformatting and adding a test
* Changing if/else structure
* Reformatting
* Support and apply filters.
* Added the UI for row level security, and moved it all under SQLA in order to access the Table model more easily.
* Added a row level security filter documentation entry.
* Accidentally added two new lines to this file.
* Blacked and iSorted, hopefully. Also, sometimes g.user may not be set.
* Another isort, and handling g not having a user attribute another way.
* Let's try this again #CI tests.
* Adjusted import order for isort; I was sure I'd already done this..
* Row level filters should be wrapped in parentheses in case one contains an OR.
* Oops, did not think that would change Black's formatting.
* Changes as per @mistercrunch.
* RLS filters are now many-to-many with Roles.
* Updated documentation to reflect RLS filters supporting multiple rows.
* Let's see what happens when I set it to the previous revision ID
* Updated from upstream.
* There was a pylint error.
* Added RLS ids to the cache keys; modified documentation; added template processing to RLS filters.
* A new migration was merged in.
* Removed RLS cache key from query_object.
* RLS added to the cache_key from query_context.
* Changes as per @etr2460.
* Updating entry for RLS pull request.
* Another migration to skip.
* Changes as per @serenajiang.
* Blacked.
* Blacked and added some attributes to check for.
* Changed to a manual query as per @mistercrunch.
* Blacked.
* Another migration in the meantime.
* Black wanted some whitespace changes.
* AttributeError: 'AnonymousUserMixin' object has no attribute 'id'.
* Oops, did hasattr backwards.
* Changes as per @mistercrunch.
* Doesn't look like text us required here anymore.
* Changes as per @dpgaspar
* Two RLS tests.
* Row level security is now disabled by default via the feature flag ENABLE_ROW_LEVEL_SECURITY.
* New head to revise.
* Changed the comment.
* Add feature flags to control query sharing, KV exposure
* Add tests, fix bug
* Skip test for kv endpoints when they are disabled
* ESLint fixes
* Remove unnecessary binds
* Fix eslint errors
* Add note to UPDATING.md RE: new feature flag options
* Use expanded version of RBAC
* Enable KV_STORE and SHARE_QUERIES_VIA_KV_STORE feature flags in the test environment
* Fix black
* Wrap tagging endpoints in a feature flag (disabled by default)
* Add tests for new feature flag
* Reuse existing feature flag
* Add TAGGING_SYSTEM to feature flags defaulted to False
* Do not show stacktraces on some intentionally-thrown errors
* Fix pylint error
* Fix JavaScript looking for the wrong key in error response from connection check
* format -> f-string
* Run black because PyCharm missed it on the last save
Migrates and completely removes: '/superset/table/<database_id>/<table_name>/<schema>/'
- Guarantees database filtering based on the user permissions
- Full OpenAPI spec
- Better error handling
* fix: shut off all uneeded endpoints
We recently added a new feature to FAB allowing to whitelist the needed
endpoints in ModelView and ModelRestApi.
First, we set our base wrapper class to an empty set, forcing each
class inheriting from it to explicitely turn on the endpoints that
Superset intends to use.
Second, we go ModelView by ModelView to whitelist the actual endpoints
used in the app.
Notes:
* as a result a large set of [unneeded] permissions should be cleaned up
* outside of the "private" use of endpoints in the app, people that have
been using endpoints in their environment for other purposes may
experience loss of functionality
* Tweaking
* Reduce the amount of endpoints using white lists
* Fix, included needed endpoints for dashboard and druid
* Drying things up
* fixes
* limiting more endpoints
* Read only on some FAB model views
* fixing some tests
* fixes
* Fixing more tests
* Addressing comments
* Drying up route_methods
* further drying
Co-authored-by: Daniel Vaz Gaspar <danielvazgaspar@gmail.com>
* [dashboards] New, API for Bulk delete
* [dashboards] Tests
* [dashboards] Fix not found on multiple delete
* [dashboards] Bulk delete partial deletes on not owned dashes
* [dashboards] Improve OpenAPI spec and tests
* [dashboards] Test for bad request
* [dashboards] i18n
* [dashboards] black
* [dashboard] make bulk all or nothing
* [dashboard] Log on sqlalchemy error
* [dashboard] Log on sqlalchemy error
* [charts] New REST API
* [charts] Small improvements
* [charts] Fix, lint
* [charts] Tests and datasource validation
* [charts] Fix, lint
* [charts] DRY post schemas
* [charts] lint and improve type declarations
* [charts] DRY owned REST APIs
* [charts] Small fixes
* [charts] More tests
* [charts] Tests and DRY
* [charts] Tests for update
* [charts] More tests
* [charts] Fix, isort
* [charts] DRY and improve quality
* [charts] DRY and more tests
* [charts] Refactor base for api and schemas
* [charts] Fix bug on partial updates for dashboards
* [charts] Fix missing apache license
* black app.py after merge
* [charts] Fix, missing imports and black
* [api] Log on sqlalchemy error
* [api] isort
* Use PyArrow Table for query result serialization
* Cleanup dev comments
* Additional cleanup
* WIP: tests
* Remove explicit dtype logic from db_engine_specs
* Remove obsolete column property
* SupersetTable column types
* Port SupersetDataFrame methods to SupersetTable
* Add test for nullable boolean columns
* Support datetime values with timezone offsets
* Black formatting
* Pylint
* More linting/formatting
* Resolve issue with timezones not appearing in results
* Types
* Enable running of tests in tests/db_engine_specs
* Resolve application context errors
* Refactor and add tests for pyodbc.Row conversion
* Appease isort, regardless of isort:skip
* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks
* Dataframe typing and nits
* Renames to reduce ambiguity
* Require the SQLAlchemy URI when creating a database
* Add migration to make dbs.sqlalchemy_uri not-nullable
* Fixes for black, isort, tests
* Alter migration to use current revision from master as downgrade target
* Update tests to support new db constraint
* black
* re-enable pylint for superset/common/query_context.py
* Re-enable pylint for superset/connectors/base/views.py
* re-enable pylint for superset/connectors/connector_registry.py
* Re-enable pylint for tests/fixtures/pyodbcRow.py
* re-enable pylint for superset/cli.py
* Fix for mypy
* Improve variable naming
* A few more naming fixes
* Build support for schema access in Superset and SQLab
* Lint
* Test for old and new flask
* Black formatting
* Cleanup modified files
* Support schema permissions
* Add migration
* Make it work for slices as well
* TODO and remove unused field
* Apply schema access check to dashboards as well
Cleaner schema permissions check
Fetch schema permissions in 1 query
Bugfix
Better datasource check
Use security manager for permissions checks and remove dead code
Properly handle anon user
Add database permissions on the database updates
Fix schema filtering
Lint / flake changes
Increase cypress request timeout
Run black
Resolve multiple alembic heads
* Use set comprehensions
* Fixes for the pylint
* Disable deprecated druid connector by default
* Add a line in UPDATING.md for the configuration change
* Remove security tests related default-disabled feature
* More test updates
* black
* First cut at app factory
* Setting things back to master
* Working with new FLASK_APP
* Still need to refactor Celery
* CLI mostly working
* Working on unit tests
* Moving cli stuff around a bit
* Removing get in config
* Defaulting test config
* Adding flask-testing
* flask-testing casing
* resultsbackend property bug
* Fixing up cli
* Quick fix for KV api
* Working on save slice
* Fixed core_tests
* Fixed utils_tests
* Most tests working - still need to dig into remaining app_context issue in tests
* All tests passing locally - need to update code comments
* Fixing dashboard tests again
* Blacking
* Sorting imports
* linting
* removing envvar mangling
* blacking
* Fixing unit tests
* isorting
* licensing
* fixing mysql tests
* fixing cypress?
* fixing .flaskenv
* fixing test app_ctx
* fixing cypress
* moving manifest processor around
* moving results backend manager around
* Cleaning up __init__ a bit more
* Addressing PR comments
* Addressing PR comments
* Blacking
* Fixes for running celery worker
* Tuning isort
* Blacking
* adds react based menu for welcome view
* bump fab
* better logic in common_bootstrap_payload
* split up files
* fix spacing and module scope styles
* lint
* render correct template for welcome
* [fix] csv upload when table metadata present
* Remove table from hive spec
* Move upload before table metadata creation
* Refine upload logic, dd unit tests and fix translations
* Use ALLOWED_EXTENSIONS from config
* Address review comments
* Fix error message grammar
* Add return type to hive csv upload and replace first with one_or_none
* Typo fix in CONTRIBUTING.md
* Alter references to config.get('FOO') to use preferred config['FOO']
* Set missing configuration constants in superset/config.py
* Misc. CI fixes
* Add type annotation for FEATURE_FLATGS
* Enable lint checking for files in db_engine_spec that have few to no
lint issues
* Enable lint and fix issue in db_engine_spec/mysql.py
* Enable pylint and fix lint for db_engine_spec/pinot.py
* Enable lint and fix issues for db_engine_specs/hive.py
* Enable lint and fix for db_engine_spec/presto.py
* Re-enable lint on base.py, fix/disable specific failures, including one
bad method signature
* Make flake8 happy after a number of pylint fixes
* Update db_engine_spec_test test cases related to Presto to support
different method naming
* automated reformatting
* One more pylint disable for druid.py
* Find the magic invocation that makes all the lint tools happy
* WIP - Add easily interpretable negative timedelta formatting
* Add unit tests for utils.core.timedelta_f
* Reformat tests.utils_tests.test_timedelta_f test code. Fixes#8274.
* Add type annotations to utils.core.timedelta_f (#8274)
* Replace recursion with absolute value in utils.core.timedelta_f (#8274)
* Rename utils.core.timedelta_f to format_timedelta (#8274)
* Fix small typo (#8274)
* Handle int64 columns with missing data in SQL Lab
* Fix docstring
* Add unit test
* Small fix
* Small fixes
* Fix cursor description update
* Better fix
* Fix unit test, black
* Fix nan comparison in unit test
* Fix regression in time grain addons
* Revert privatization of time_grain_functions
* Fix test
* Rename variable
* Fix test
* Fix typing error
* Refactor and add tests
* Add TODO
* Add support for msgpack results_backend serialization
* Serialize DataFrame with PyArrow rather than JSON
* Adjust dependencies, de-lint
* Add tests for (de)serialization methods
* Add MessagePack config info to Installation docs
* Enable msgpack/arrow serialization by default
* [Fix] Prevent msgpack serialization on synchronous queries
* Add type annotations
* Add check for calls to cache_key_wrapper to avoid unavoidable compilation of query
* Add fetch_values_predicate to check
* Only check relevant attributes
* Address nit
* Add typing to db_engine_specs
* Add more type annotations and docstrings
* Add docstrings and typing to sql_parse and db_engine_specs
* Refine select_star
* Fix execute and add more docstrings
* Revert kwargs change from execute
* Remove redundant or
* Align view and table getter schema types
* Fix return type of latest_partition
* Remove some typing from presto
* Improve docstring for __extract_from_token
* [database] Fix, Removes the limit for the page size, Bump FAB to 2.1.8
Old FAB API had no limits by default, this will keep this behaviour
but only for this endpoint
* [sqllab] Add test for database API
* [sqllab] Add test for database API
* [sqllab] Include page zero on request
* [sqllab] Fix, Black and requirements
* [sqllab] Make database API return unlimited results
* [sqllab] just a test
* [sqllab] Bump FAB to 2.1.9
* [sqllab] Remove unused import
* 7620: Start removing dependencies on requests
* Patch urllib.request.urlopen instead of requests.get
* Try to fix flake8
* More work on flake8 import errors
* First attempt at using urllib with cookies
* Fix pylint/flake8
* Fix test_deliver_slice_csv_attachment
* Fix test_deliver_slice_csv_inline
* Import requests and pydruid conditionally, remove dependency on prison
* Fix flake errors
* Fix load_examples
* Please flake
* Skip tests depending on optional deps
* Try to please flake
* Address review comments
* Remove Druid-related UI
* Revert "Remove Druid-related UI"
This reverts commit d7e0f166cc3f3dd2496b4a666e177f0c191aeb0f.
* Skip a few tests more
* Put imports in right order
* Apply black patch
* Please flake
* Please black, silence flake
* Use flake8 silencing the right way
* Add deps for CI
* [cli] New, command line option to create or set a db URI
* [tests] New, get or create db on utils
* [tests] flake8 and black
* [tests] Fix, remove dummy comments
* [cli] dummy commit to re-trigger build
* [cli] dummy commit to re-trigger build
