Commit Graph

660 Commits

Author SHA1 Message Date
Maxime Beauchemin
318e5347bc
feat: Alerts! allowing users to set SQL-based email alerts with screenshots (#9944)
* feat: add support for alerts

* Add ModelViews

* First pass at adding scheduled Alert support

* Fix syntax errors

* Add log_retention and database to model

* Improving screenshots

* Still refactoring screenshots

* Pass down thumb_size properly

* Progress on screenshot endpoint

* Add alerts.last_eval_dttm and alert query logic

* Getting ready to split compute_screenshot and screenshot/{SHA}

* split the endpoints

* Improving alerting loop

* empty methods

* Add CLI command 'superset alert' that runs a sched loop

* Getting things to work

* email delivery

* A working email\!

* Add feature flag

* Add some description_columns to AlertModelView

* Little tweaks

* Use database.get_df, eval cells for truthiness

* Migrate thumbnail/screenshot functions to use new arg pattern

* Addressing PR feedback

* Update alerts DB migration down_revision

* Resolve _deliver_email arg conflict

* Make mypy happy

* Make isort happy

* Make pylint happy

Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
2020-07-06 13:12:27 -07:00
Bogdan
b7c45fed80
fix: downgrade sqlparse and add unit test (#10165)
* Downgrade sqlparse and add unit test

* Explain why sqlparse is pinned

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-07-06 11:10:46 -07:00
Maxime Beauchemin
2314aad450
feat: add database dropdown to dashboard import (#10118)
* feat: add database dropdown to dashboard import

Currently, when importing a database from a JSON file, the process
looks at the database name from the source (the info is in the file)
and matches the datasources to that name. If no database by that name
exists, it simply fails.

With this PR, we add a database dropdown that allows the user to specify
which databases the datasources should target as the get upserted.

I want to stress that the code in this area is not in a great shape,
and that the challenge of serializing/deser the nested objects is
challenging, but that there should be a much better way to do this.
One of the improvement (out of scope for this PR) that would allow to
simplify those import/export would be to use UUIDs for
importable/exportable objects.

Another identified issue is the indirections between
`utils/import_expor_{model}.py` on top of `{Model}.import_object`. Not
addressing that here.

Next topic is the MVC stuff. Decided to stick with it for now as this is
more of a [obious missing feat:] than a rewrite.

* isort \!? 0%^$%Y$&?%$^?%0^?

* fix tests

* pre-committing to py3.6

* address dpgaspar's comments

* revert isort
2020-07-05 15:08:37 -07:00
John Bodley
b181e48f5c
[log] Updating form-data logic (#10197)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-07-04 12:51:43 -07:00
Bogdan
77b36783e7
s/assertEqual/assertEquals (#10195)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-29 16:38:06 -07:00
Bogdan
4e340c8368
feat: Switch from nosetest to pytest (#10177)
* Switch from nosetest to pytest

Fix schedule tests

Collect pytest coverage

Move pytest config into pytest.ini

Move cov to the pytest.ini

* Append coverage for the 2nd run

* Add coverage to all commands

* Coverage only for tests

* Get coverage from 1 place

* Rename classes to be  pytest compatible

* Test coverage for examples and tests

* Max diff to -1

* Explain how to run pytest for the whole project

* Do not append code coverage for the main run

* Do not run coverage on examples

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-29 15:36:06 -07:00
Jason Davis
72fc581b15
added explore database for ctas/cvas (#10174)
Co-authored-by: Jason Davis <@dropbox.com>
2020-06-29 11:13:54 -07:00
Ville Brofeldt
26d34727d3
fix: make time grain nullable in chart data endpoint (#10187)
* fix: make time grain nullable

* add test

* lint
2020-06-29 09:54:01 +03:00
Jesse Yang
9bdfa055ac
feat(viz): add query mode switch to table chart (#10113)
1, Replace table chart rendering from jquery.DataTables to react-table: apache-superset/superset-ui#623
2. Rearrange the control panel, replace GROUP BY and NOT GROUP BY with a "Query Mode" switch: apache-superset/superset-ui#609
2020-06-28 21:37:04 -07:00
Will Barrett
4965d87505
Break some static methods out of superset.views.core.Superset (#10175) 2020-06-26 14:34:45 -07:00
Jason Davis
9de9e1c19d
feat: Added configuration to SQL Lab results "Explore" button (#10164)
* added configuration to virtual table explore button

* added description to mixin

* fixed unit tests

Co-authored-by: Jason Davis <@dropbox.com>
2020-06-26 09:54:55 -07:00
Will Barrett
df71fac1e0
refactor: Fix lint on superset/utils/core.py (#10120)
* Fix lint on superset/utils/core.py

* black

* mypy

* Fix some missing renames
2020-06-26 08:49:12 -07:00
Ville Brofeldt
410c5be2f8
fix: return slice owners for /user_slices ep (#10171)
* fix: return slice wowners for /user_slices ep

* put back created_by and changed_by

* improve comment
2020-06-26 18:10:56 +03:00
Maxime Beauchemin
1781ebbaa4
fix: [search query view] edit link is broken (#10128)
* fix: [search query view] edit link is broken

* eslint + mypy

* rever app.py changes

* addressing comments

* use api/v1/query

* fix test
2020-06-25 12:20:10 -07:00
Ville Brofeldt
b205ce32b0
feat: Add new timegrains and convert_dttm to Druid engine spec (#10160)
* feat: Add new timegrains and convert_dttm to Druid engine spec

* Add TemporalType enum and fix test case

* Remove DATETIME for athena (original spec)
2020-06-25 12:18:37 +03:00
Jason Davis
c544ebd876
fix: Fixed Attribute Error when running core_tests.py (#10154)
* fix: changed import order to fix core-test error

* reformatted using black

Co-authored-by: Jason Davis <@dropbox.com>
2020-06-24 12:46:38 -07:00
John Bodley
37777f382f
fix(security): dbs/clusters perm (#10130)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-24 10:06:25 -07:00
Bogdan
3db76c6fdc
Implement create view as functionality (#9794)
Implement create view as button in sqllab

Make CVAS configurable

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-24 09:50:41 -07:00
John Bodley
aefef9ca55
chore(security): Updating assert logic (#10034)
* chore(security): Updating assert logic

* Deprecating rejected_tables

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-23 20:49:39 -07:00
Aliaksei Kushniarevich
550e78ff7c
feat: Add multiple table filters for Row Level Security (#9751)
* Add multiple table filters for Row Level Security

* Set ENABLE_ROW_LEVEL_SECURITY back to False (default)

* Merge DB migrations

* Drop table_id column and foreign key on PostgreSQL, MySQL, SQLite

* Support db records migration also

* Support downgrading from the new-fashioned formatted records

* Straighten up migrations

* Update migration's down_revision to comply master branch
2020-06-22 12:51:08 +03:00
Daniel Vaz Gaspar
dbc43d7c7b
perf(datasets): improve datasets API performance for attr kind (#10111)
* perf(datasets): improve datasets API performance for attr kind

* fix test
2020-06-22 08:21:06 +01:00
Bogdan
be6b9b8fec
feat: implement dttm column configuration through db extra config (#9444)
* Implement table mutator and examples how to set date defaults

Fix tests

* Fix flaky test

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 13:46:45 -07:00
Will Barrett
01aede0652
refactor: Break up superset/views/core.py (#10078)
* Remove unreferenced function from views/core.py

* Remove excess constants from views/core.py

* Extract CssTemplate-related views to their own file from core.py

* Remove duplicate constant declaration and make the constant less racist

* Move free-floating functions in views/core.py to views/utils.py

* Move AccessRequestsModelView out of views/core.py into its own module

* Move health checks and KV ModelView out of core.py and into their own modules

* Move R model view to its own module

* Move after-request header setting to views/base.py from views/core.py

* black

* mypy

* isort

* Fix reference to imported app

* pylint

* Fix some imports

* Add some missing view imports

* Fix a missing import
2020-06-17 13:42:13 -07:00
Bogdan
29e9f2c70b
feat: superset report slack integration (#9810)
* First draft for the slack integration

Fix slack

another typo

another typo

Fix slack

Add channels to the form

another typo

Another set of changes

Make code more transparent

Fix tests

Add logging

logging

use logger

import logging

import logging

import logging

add assert

more logging

Fix channels

Fix channels

* Address comments

* Move slack into a separate module

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 11:01:25 -07:00
John Bodley
eba1fe2ebb
fix: Ensuring queries route accepts float or int (#10079)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-17 08:02:49 -07:00
ʈᵃᵢ
26120467af
fix: use custom int parsing over flask int parsing in sqllab queries endpoint (#10070) 2020-06-16 12:58:32 -07:00
Erik Ritter
7504b1cf20
fix: caching on viz with relative time ranges and time compare (#10061) 2020-06-16 08:47:35 -07:00
Ville Brofeldt
a3393c1bc4
fix: native annotations (#10037)
* fix: native annotations

* Add test

* Add comment to test
2020-06-12 21:53:26 +03:00
John Bodley
9532bff48f
chore(security): Renaming access methods (#10031)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-11 13:12:23 -07:00
John Bodley
54c6ddbdb7
chore(security): Renaming schemas_accessible_by_user (#10030)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-11 11:29:43 -07:00
Will Barrett
4ef26993e2
feat: Add owners and physical/virtual status to Datasets API list endpoint (#10025)
* Add owners and physical/virtual status to Datasets API list endpoint

* black

* Update type definition for DatasourceKind

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Fixin' tests and mypy

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-06-10 22:04:40 +03:00
Erik Ritter
e17da58a39
feat: add replace option to hive csv upload (#9764) 2020-06-10 08:57:33 -07:00
Ville Brofeldt
526ee3c3df
fix: flaky test_copy_dash test in dashboard_tests.py (#10021) 2020-06-09 13:29:22 +03:00
Ville Brofeldt
315518d2d2
feat: add support for query offset (#10010)
* feat: add support for query offset

* Address comments and add new tests
2020-06-09 11:46:28 +03:00
John Bodley
91517a56a3
style(mypy): Spit-and-polish pass (#10001)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-07 08:53:46 -07:00
Ville Brofeldt
5c4d4f16b3
feat: finalize Word Cloud move to new chart data endpoint (#9975)
* remove word cloud from viz.py

* Fix Run in SQL Lab

* remove deprecated python tests

* break out legacy endpoint type into function

* Break out exploreChart from exportChart and implement results type

* Fix jest tests and refactor accordingly

* lint

* Rename v1 payload function

* Add dashboard id to v1 chart data request url params

* Add support for domain sharding to v1 chart data request
2020-06-05 14:08:46 +03:00
John Bodley
244677cf5e
style(mypy): Enforcing typing for superset (#9943)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-03 15:26:12 -07:00
Xiang Fu
1d9dbcd9dc
fix: pinot select query logic (#9954)
* Update how pinot handles selection logic

* Change DATETIMECONVERT argument to use single quote for literals
2020-06-02 23:18:35 -07:00
John Bodley
1001c6d5f4
Revert "[caching] Using request context rather than globals" (#9969)
This reverts commit 90cd3889ac.

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-02 16:03:33 -07:00
Ville Brofeldt
38a6bd79da
feat: expand new chart data endpoint coverage (#9903)
* feat: implement new chart API for additional components

* Fix python tests

* Fix tests

* Fix lint

* fix camel case error in requestParams

* lint

* fix samples row limit

* Add samples row limit to config

* remove unnecessary code

* lint

* Address review comments
2020-06-02 10:47:28 +03:00
John Bodley
b296a0f250
[mypy] Enforcing typing for superset.utils (#9905)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-27 22:57:30 -07:00
Erik Ritter
c4040a2ae0
fix: annotation layer json (#9915)
* fix: annotation layer json

* attempt to add a test

* [tests] Fixing test

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-27 09:02:03 -07:00
Bogdan
3e8e441bfc
feat: implement csv upload configuration func for the schema enforcement (#9734)
* Implement csv upload func for schema enforcement

Implement function controlled csv upload schema

Refactor + fix tests

Fixing hive as well

* Add explore_db to the extras

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-05-21 13:49:53 -07:00
Ville Brofeldt
a43a1d6303
feat: Add new result formats and types to chart data API (#9841)
* feat: Add new result formats and types to chart data API

* lint

* Linting

* Add language to query payload

* Fix tests

* simplify tests
2020-05-20 21:36:14 +03:00
michellethomas
368c85de1b
fix: Removing the logic to add timeseries_limit_metric to the data for table (#9832)
* Removing the logic to add timeseries_limit_metric to the data for table viz

* Also make the change in viz_sip_38

* Fix tests

Co-authored-by: michelle_thomas <michelle.thomas@airbnb.com>
2020-05-20 10:20:54 -07:00
John Bodley
c691a07948
Revert "[sqllab] fix, strip comments before parsing statements" (#9862)
This reverts commit 949c4eae28.

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-20 09:54:17 -07:00
Erik Ritter
c7c76c6a99
build: Add python 3.8 to the build matrix (#9827) 2020-05-17 14:49:51 -07:00
Daniel Vaz Gaspar
3cc540019f
fix(mssql): reverts #9644 and displays a better error msg (#9752) 2020-05-14 17:00:02 +01:00
Daniel Vaz Gaspar
4427d65717
tests(celery): improve celery tests infra (#9775) 2020-05-14 16:58:30 +01:00
Erik Ritter
d02f2d1fa7
feat: return security errors in the SIP-40 format (#9796) 2020-05-13 17:10:37 -07:00