Migrates and completely removes: '/superset/table/<database_id>/<table_name>/<schema>/'
- Guarantees database filtering based on the user permissions
- Full OpenAPI spec
- Better error handling
* fix: shut off all uneeded endpoints
We recently added a new feature to FAB allowing to whitelist the needed
endpoints in ModelView and ModelRestApi.
First, we set our base wrapper class to an empty set, forcing each
class inheriting from it to explicitely turn on the endpoints that
Superset intends to use.
Second, we go ModelView by ModelView to whitelist the actual endpoints
used in the app.
Notes:
* as a result a large set of [unneeded] permissions should be cleaned up
* outside of the "private" use of endpoints in the app, people that have
been using endpoints in their environment for other purposes may
experience loss of functionality
* Tweaking
* Reduce the amount of endpoints using white lists
* Fix, included needed endpoints for dashboard and druid
* Drying things up
* fixes
* limiting more endpoints
* Read only on some FAB model views
* fixing some tests
* fixes
* Fixing more tests
* Addressing comments
* Drying up route_methods
* further drying
Co-authored-by: Daniel Vaz Gaspar <danielvazgaspar@gmail.com>
* Use PyArrow Table for query result serialization
* Cleanup dev comments
* Additional cleanup
* WIP: tests
* Remove explicit dtype logic from db_engine_specs
* Remove obsolete column property
* SupersetTable column types
* Port SupersetDataFrame methods to SupersetTable
* Add test for nullable boolean columns
* Support datetime values with timezone offsets
* Black formatting
* Pylint
* More linting/formatting
* Resolve issue with timezones not appearing in results
* Types
* Enable running of tests in tests/db_engine_specs
* Resolve application context errors
* Refactor and add tests for pyodbc.Row conversion
* Appease isort, regardless of isort:skip
* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks
* Dataframe typing and nits
* Renames to reduce ambiguity
* First cut at app factory
* Setting things back to master
* Working with new FLASK_APP
* Still need to refactor Celery
* CLI mostly working
* Working on unit tests
* Moving cli stuff around a bit
* Removing get in config
* Defaulting test config
* Adding flask-testing
* flask-testing casing
* resultsbackend property bug
* Fixing up cli
* Quick fix for KV api
* Working on save slice
* Fixed core_tests
* Fixed utils_tests
* Most tests working - still need to dig into remaining app_context issue in tests
* All tests passing locally - need to update code comments
* Fixing dashboard tests again
* Blacking
* Sorting imports
* linting
* removing envvar mangling
* blacking
* Fixing unit tests
* isorting
* licensing
* fixing mysql tests
* fixing cypress?
* fixing .flaskenv
* fixing test app_ctx
* fixing cypress
* moving manifest processor around
* moving results backend manager around
* Cleaning up __init__ a bit more
* Addressing PR comments
* Addressing PR comments
* Blacking
* Fixes for running celery worker
* Tuning isort
* Blacking
* adds react based menu for welcome view
* bump fab
* better logic in common_bootstrap_payload
* split up files
* fix spacing and module scope styles
* lint
* render correct template for welcome
* [fix] csv upload when table metadata present
* Remove table from hive spec
* Move upload before table metadata creation
* Refine upload logic, dd unit tests and fix translations
* Use ALLOWED_EXTENSIONS from config
* Address review comments
* Fix error message grammar
* Add return type to hive csv upload and replace first with one_or_none
* Add support for msgpack results_backend serialization
* Serialize DataFrame with PyArrow rather than JSON
* Adjust dependencies, de-lint
* Add tests for (de)serialization methods
* Add MessagePack config info to Installation docs
* Enable msgpack/arrow serialization by default
* [Fix] Prevent msgpack serialization on synchronous queries
* Add type annotations
* [api] [database] New, migrate to new FAB API side by side
* [database] view and api module reorganization
* [style] Fix, flake8
* [test] [database] Fix, tests for new database view structure
* [style] [database] black
* [database] Fix, missing apache license
* Migrate databaasync api to the new api
* flake8
* More efficient api call
* Revert querySearch to old API, test
* dummy commit
* revert config.py commit
* Remove unused import
* Remove old API view
* Remove new API view
* Add database filter
* Add database filter
* Add database filter
* Remove comments on js
* Fix JS test
* Fix pylint
* QuerySearch new databaseasync API endpoint
* Fix, query search new api endpoint
* Reintroduce old API for smooth side by side migration
* Better naming and use common hooks for view and API
* black it
* Fix, database mixin must come first
* Fix, lint
* [WiP] improve load_examples
related to #7472, longer term we will generate the examples by exporting
them into tarball as in #7472. In the meantime, we need this subset of
the features:
* allowing specifying an alternate database connection for examples
* allowing a --only-metadata flag to `load_examples` to load only
dashboard and chart definitions, no actual data is loaded
* Improve logging
* Rename data->examples
* Load only if not exist
* By default do not load, add a force flag
* fix build
* set published to true
Fixing a few failing examples, examples were referencing metrics that
were getting auto-generated in the past (like `sum__num`),
that we now prevent from getting created.
* Add ability to inject statsd client; some py test/reqs updates
- Updated the metrics logger to allow construction with an existing
statsd client, so that it can be configured by external systems or libs.
- added requirements to requirements-dev.txt which are needed to run
tests-eg coverage, nose
- removed dependency on mock lib, it is in python stdlib now
- updated tox.ini to remove the now-superfluous deps
* add license to test file, and remove blank line at EOF
Summary: We want to allow grouping by on expressions, including the
ability to select expressions (without group-by). The UI ain't good yet
.. it is not at the feature parity of say adhoc filters/metrics that
come with a nice text-box to edit the SQL. But
this suffices for my usecase for now.
(The UI would have to be redone ... but that would require some refactoring
to merge the AdhocFilter/Metric/Popover stuff such that it can be used
for this use case too.)
Also fixed a bug in the SelectControl freeForm selection.
Allow selecting all columns easily: Added a new Select-All special item
that users can select to make all the options in the selectable show
up.
The ability to group by arbitrary expressions is useful because now two
users don't need to create custom computed-fields for this.
* Use py3's f-strings instead of s.format(**locals())
In light of the bug reported here
https://github.com/apache/incubator-superset/issues/6347, which seems
like an odd `.format()` issue in py3, I greped and replaced all
instances of `.format(**locals())` using py3's f-strings
* lint
* fix tests
* [SIP-5] Open a new /api/v1/query endpoint that takes query_obj
- Introduce a new handle_superset_exception decorator to avoid repeating the logic for catching SupersetExceptions
- Create a query_obj_backfill method that takes form_data and constructs a query_obj that will be constructed in the client in the future. Use the backfill in explore_json.
- Create a new /api/v1/query endpoint that takes query_obj only and returns the payload data. Note the query_obj is constructed in the client. The endpoint currently only handles query_obj for table view viz (we'll be adding support to new viz types as we go).
- Unit test to verify the new endpoint for table view
* fix tests and lint errors
* - Move the new query endpoint into its own api.py view.
- Create QueryObject and QueryContext class to encapsulate query_object to be built from the client and additional info (e.g. datasource) needed to get the data payload for a given query
- Remove the query_obj_backfill as we'll start building the first query_object on the client so it no longer makes sense to have a short-lived backfill for the matter of days.
* Fixing lint and test errors
* Fixing additional lint error from the previous rebase.
* fixing additional lint error
* addressing additional pr comments
* Make /query accept a list of queries in the query_context object.
* fixing a lint error
* - Move time_shift based calculation and since, until check into util
- Add typing info for get_since_until
- Add new unit tests to verify time_shift calculation and the since until check
* [utils] gathering/refactoring into a "utils/" folder
Moving current utils.py into utils/core.py and moving other *util*
modules under this new "utils/" as well.
Following steps include eroding at "utils/core.py" and breaking it down
into smaller modules.
* Improve tests
* Make loading examples in scope for tests
* Remove test class attrs examples_loaded and requires_examples
* Add schema level access control on csv upload
* add db migrate merge point
* fix flake 8
* fix test
* remove unnecessary db migration
* fix flake
* nit
* fix test for test_schemas_access_for_csv_upload_endpoint
* fix test_csv_import test
* use security_manager to check whether schema is allowed to be accessed
* bring security manager to the party
* flake8 & repush to retrigger test
* address comments
* remove trailing comma
* Fixing issue with extra params in formData
* Pass in param use_slice_data to decide whether to use slice data
* Fixing core_tests to not use explore_json overrides
