* feat(database): POST, PUT, DELETE API endpoints
* post tests
* more tests
* lint
* lint
* debug ci
* fix test
* fix test
* fix test
* fix test
* fix test
* fix test
* cleanup
* handle db connection failures
* lint
* skip hive and presto for connection fail test
* fix typo
* Log cache keys to the logs
* Add tests
* Use separate table for the cache keys
* Add migration for the cache lookup table
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
* fix: change public role like gamma procedure
* lint and updating UPDATING with breaking change
* fix updating text
* add test and support PUBLIC_ROLE_LIKE_GAMMA
* fix, cleanup tests
* fix, new test
* fix, public default
* Update superset/config.py
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
* add simple public welcome page
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
* Add presto to the CI
Sample test data
Datetime conversion
Sample test data
Fix tests
* TODO to switch to timestamps
* Address feedback
* Update requirements
* Add TODOs
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
* refractored alerting to not pass sqlalchemy obj as args
* updated to pass only alert id as arg
* used object id instead of argument
* updated alerts_tests.py to reflect change
Co-authored-by: Jason Davis <@dropbox.com>
* fix(log): log crashes if expired or not authenticated
* fix lint and rison
* add tests
* more tests
* perm fix
* fix test not found
* JS lint
* fix Jest test
* implemented updates to alerting
* fixed imports and teardown
* changed unittest to pytest conventions
* add app_context to tests
Co-authored-by: Jason Davis <@dropbox.com>
* fix(permissions): alpha role is inconsistent
* reverse and allow Alpha to access manager menu
* Bump FAB to 3.0.1rc1 to include del permission fix
* add docs, tests and UPDATING
* EOL
* Fix query view for Alpha
* fix: API marshmallow3 drop utc for naive datetime fields
* fix: API marshmallow3 drop utc for naive datetime fields
* fix, tests
* isort and test
* black
* add and fix test
* fix comment
* fix(chart-data-api): case insensitive evaluation of filter op
* fix(chart-data-api): case insensitive evaluation of filter op
* mypy
* remove print statement
* add test
* feat(api): bump marshmallow and FAB to version 3
* revert query context tests changes
* obey mypy
* fix tests
* ignore types that collide with marshmallow
* preparing for RC2
* fix tests for marshmallow 3
* typing fixes for marshmallow
* fix tests and black
* fix tests
* bump to RC3 and lint
* Test RC4
* Final 3.0.0
* Address comments, fix tests, better naming, docs
* fix test
* couple of fixes, addressing comments
* bumping marshmallow
* Support more table properties for the hive upload
Refactor
Add tests, and refactor them to be pytest friendly
Use lowercase table names
Ignore isort
* Use sql params
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
* feat: add support for alerts
* Add ModelViews
* First pass at adding scheduled Alert support
* Fix syntax errors
* Add log_retention and database to model
* Improving screenshots
* Still refactoring screenshots
* Pass down thumb_size properly
* Progress on screenshot endpoint
* Add alerts.last_eval_dttm and alert query logic
* Getting ready to split compute_screenshot and screenshot/{SHA}
* split the endpoints
* Improving alerting loop
* empty methods
* Add CLI command 'superset alert' that runs a sched loop
* Getting things to work
* email delivery
* A working email\!
* Add feature flag
* Add some description_columns to AlertModelView
* Little tweaks
* Use database.get_df, eval cells for truthiness
* Migrate thumbnail/screenshot functions to use new arg pattern
* Addressing PR feedback
* Update alerts DB migration down_revision
* Resolve _deliver_email arg conflict
* Make mypy happy
* Make isort happy
* Make pylint happy
Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
* feat: add database dropdown to dashboard import
Currently, when importing a database from a JSON file, the process
looks at the database name from the source (the info is in the file)
and matches the datasources to that name. If no database by that name
exists, it simply fails.
With this PR, we add a database dropdown that allows the user to specify
which databases the datasources should target as the get upserted.
I want to stress that the code in this area is not in a great shape,
and that the challenge of serializing/deser the nested objects is
challenging, but that there should be a much better way to do this.
One of the improvement (out of scope for this PR) that would allow to
simplify those import/export would be to use UUIDs for
importable/exportable objects.
Another identified issue is the indirections between
`utils/import_expor_{model}.py` on top of `{Model}.import_object`. Not
addressing that here.
Next topic is the MVC stuff. Decided to stick with it for now as this is
more of a [obious missing feat:] than a rewrite.
* isort \!? 0%^$%Y$&?%$^?%0^?
* fix tests
* pre-committing to py3.6
* address dpgaspar's comments
* revert isort
* Switch from nosetest to pytest
Fix schedule tests
Collect pytest coverage
Move pytest config into pytest.ini
Move cov to the pytest.ini
* Append coverage for the 2nd run
* Add coverage to all commands
* Coverage only for tests
* Get coverage from 1 place
* Rename classes to be pytest compatible
* Test coverage for examples and tests
* Max diff to -1
* Explain how to run pytest for the whole project
* Do not append code coverage for the main run
* Do not run coverage on examples
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
* feat: Add new timegrains and convert_dttm to Druid engine spec
* Add TemporalType enum and fix test case
* Remove DATETIME for athena (original spec)
* Add multiple table filters for Row Level Security
* Set ENABLE_ROW_LEVEL_SECURITY back to False (default)
* Merge DB migrations
* Drop table_id column and foreign key on PostgreSQL, MySQL, SQLite
* Support db records migration also
* Support downgrading from the new-fashioned formatted records
* Straighten up migrations
* Update migration's down_revision to comply master branch
* Implement table mutator and examples how to set date defaults
Fix tests
* Fix flaky test
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
* Remove unreferenced function from views/core.py
* Remove excess constants from views/core.py
* Extract CssTemplate-related views to their own file from core.py
* Remove duplicate constant declaration and make the constant less racist
* Move free-floating functions in views/core.py to views/utils.py
* Move AccessRequestsModelView out of views/core.py into its own module
* Move health checks and KV ModelView out of core.py and into their own modules
* Move R model view to its own module
* Move after-request header setting to views/base.py from views/core.py
* black
* mypy
* isort
* Fix reference to imported app
* pylint
* Fix some imports
* Add some missing view imports
* Fix a missing import
* First draft for the slack integration
Fix slack
another typo
another typo
Fix slack
Add channels to the form
another typo
Another set of changes
Make code more transparent
Fix tests
Add logging
logging
use logger
import logging
import logging
import logging
add assert
more logging
Fix channels
Fix channels
* Address comments
* Move slack into a separate module
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
* Add owners and physical/virtual status to Datasets API list endpoint
* black
* Update type definition for DatasourceKind
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
* Fixin' tests and mypy
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
* remove word cloud from viz.py
* Fix Run in SQL Lab
* remove deprecated python tests
* break out legacy endpoint type into function
* Break out exploreChart from exportChart and implement results type
* Fix jest tests and refactor accordingly
* lint
* Rename v1 payload function
* Add dashboard id to v1 chart data request url params
* Add support for domain sharding to v1 chart data request
* Removing the logic to add timeseries_limit_metric to the data for table viz
* Also make the change in viz_sip_38
* Fix tests
Co-authored-by: michelle_thomas <michelle.thomas@airbnb.com>
* chore: bump pyhton dependencies
* bump flask-testing
* Add note to UPDATING and bump flask-testing in requirements-dev.txt
* fix old sqlparse test case that is fixed with sqlparse==0.3.1
* Fix migration and sqlparse set test
* downgrade alembic
* Downgrade sqlalchemy-utils
* pin sqlalchemy-utils due to regression in 0.36.5
* Refine comment in setup.py
* make cachelib an explicit requirement
* Make all fields optional in QueryObject and fix having_druid schema
* fix: datasource type sql to table
* lint
* Add missing fields
* Refactor tests
* Linting
* Refactor query context fixtures
* Add typing to test func
* Add OpenAPI docs to /api/v1/chart/data EP
* Add missing fields to QueryObject, fix linting and unit test errors
* Fix unit test errors
* abc
* Fix errors uncovered by schema validation and add unit test for invalid payload
* Add schema for response
* Remove unnecessary pylint disable
* Deprecate groupby from query_obj
* Fix query_object bug
* Fix histogram
* Remove groupby from legacy druid connector and fix first batch of unit tests
* Deprecate some unnecessary tests + fix a few others
* Address comments
* hide SIP-38 changes behind feature flag
* Break out further SIP-38 related tests
* Reslove test errors
* Add feature flag to QueryContext
* Resolve tests and bad rebase
* Backport recent changes from viz.py and fix broken DeckGL charts
* Fix bad rebase
* backport #9522 and address comments
* Visualize after running Ctas query (#4)
* Visualize after running Ctas query.
Stub for table viz
Work on the viz
Add JS functionality for Ctas viz
Add test for the table viz flow
Fix test
Add js test for the ctas viz
* Fix tests
* Resolve comments
* Leverate tmpSchema in the query object
* Fix i18n string
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
* Add post processing to QueryObject
* Simplify sort signature and require explicit sort order
* Add new operations and unit tests
* linting
* Address comments
* Simplify test method names
* Address comments
* Linting
* remove unnecessary logic
* Apply strict whitelisting to all getattr calls
* Add checking of rolling_type_options and add/improve docs
* [dashboards] New, tittle and slug OR filter
* Update requirements, because of prison bump
* Tests
* Fix tests
* Avoid like filter on empty string value
* merge master brings strict typing to the table
* filter owners select by text input
* use rison
* fix backend owners filter logic
* use fullname, not username on owners inputs
* fix some tests
* fixing tests
* deterministic tests
* appease linter
* add back search by username
* more comprehensive filter test
* add clarifying text
* formatting...
* [explore view] fix long query issue from Run in SQL LAB Button
* SQL Lab page needs to take the post form data, too
* fix variable names
* updated payload dict, rename hidden form
Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
* feat: [explore] don't save filters inherited from a dashboard
When navigating to explore from a dashboard context, the current
dashboard filter(s) are passed along to explore so that the context is
kept. So say you're filtering on "country=Romania", in your dashboard
and pivot to explore, that filter is still there and keep on exploring.
Now a common issue is that you'll want to make some tweak to your chart
that are unrelated to the filter, say toggling the legend off for
instance, and then save it. Now you back to your dashboard and even
though you started with an "all countries" dashboard, with a global
filter on country, now that one chart is stuck on "Romania". Typically
you notice this when filtering on something else, say "Italy" and then
that one chart now has two mutually exclusive filters, and show "No data".
Now, the fix is to flag the filter as "extra" (that's the not-so-good internal
name we use for these inherited filters) and make it clear that that
specific filter is special and won't be saved when saving the chart.
* fix build
* [dashboard] Refactor API using SIP-35
* [dashboard] Fix, import
* [dashboard] more tests
* [dashboards] a misc of improvements
* [charts] Fix, DAO and tests
* [dashboards] small exceptions refactor
* [dashboards] lint
* [dashboards] Improves comments on base classes
* [dashboards] lint
* [datasets] new, react listview
* add hidden columns to support filtering by columns not rendered
* throw exception if config is incorrect
* fix database filter
* update endpoints to point to datasets; fix translation strings
* move Link into src/components
* add add new record button to datasets listview
* fix: change database save in DatasourceEditor
This addresses the issue where pointing a datasource to another database
in the datasource editor is not reflected.
Also addresses:
- a minorcosmetic issue in the datasource editor.
- user/owners list not getting populated
* tests
* Make schema name configurable
Fixing unit tests
Fix table quoting
Mypy
Split tests out for sqlite
Grant more permissions for mysql user
Postgres doesn't support if not exists
More logging
Commit for table creation
Priviliges for postgres
Update tests
Resolve comments
Lint
No limits for the CTA queries if configures
* CTA -> CTAS and dict -> {}
* Move database creation to the .travis file
* Black
* Move tweaks to travis db setup
* Remove left over version
* Address comments
* Quote table names in the CTAS queries
* Pass tmp_schema_name for the query execution
* Rebase alembic migration
* Switch to python3 mypy
* SQLLAB_CTA_SCHEMA_NAME_FUNC -> SQLLAB_CTAS_SCHEMA_NAME_FUNC
* Black
ʈᵃᵢ
Ville Brofeldt
Daniel Vaz Gaspar
Jason Davis
John Bodley
Bogdan
Maxime Beauchemin
Lily Kuang
Craig Rueda
Erik Ritter
Jesse Yang
Hossein Torabi
Beto Dealmeida
bryanck
serenajiang
Will Barrett
Aliaksei Kushniarevich
Xiang Fu
michellethomas
dandanhub
David Aaron Suddjian
Grace Guo