Commit Graph

1293 Commits

Author SHA1 Message Date
Beto Dealmeida 93bafa0e6a
feat: improve logic in is_select (#17329)
* feat: improve logic in is_select

* Add more edge cases
2021-11-02 17:30:12 -07:00
Étienne Boisseau-Sierra f0c0ef7048
fix(cli): fail CLI script on failed import/export (#16976)
* Test that failing export or import is done properly

For each CLI entry-point we will modify, we make sure that:

- a failing process exits with a non-0 exit code,
- an error is logged.

Signed-off-by: Étienne Boisseau-Sierra <etienne.boisseau-sierra@unipart.io>

* Exit process with error if export/import failed

Bubble exception up when failing import or export

During a CLI import or export of dashboards, if the process fails, the
exception it caught and a simple message is sent to the logger.
This makes that from a shell point of view, the script was successfull —
cf. #16956.

To prevent this, we want to ensure that the process exits with an error
(i.e., a non-0 exit-code) should the export or import fail mid-flight.

Signed-off-by: Étienne Boisseau-Sierra <etienne.boisseau-sierra@unipart.io>
2021-10-29 08:56:29 +01:00
Ville Brofeldt 4316fe6ae6
fix(annotataion): handle required fields properly (#17234) 2021-10-27 08:42:28 +03:00
Shiva Raisinghani ef3afbde82
refactor: Repeated boilerplate code between upload to database forms (#16756)
* abstract boilerplate code into class and rename csv to file

* add db migration

* fix some stuff

* more renaming of csv to file

* rename in translations

* update down revision

* update down revision

* bump chart version

* switch to alter column name approach in db migration

* fix db migration for MySQL

* db migration conflict
2021-10-25 13:53:06 +03:00
Yongjie Zhao 4f1d202430
chore: use official isort in pre-commmit (#17215) 2021-10-25 17:57:38 +08:00
Yongjie Zhao bedb8f4dff
fix(bigquery): calculated column cannot orderby in BigQuery (#17196)
* fix(bigquery): calculated column cannot orderby in BigQuery

* typo

* add ut

* fix lint
2021-10-22 22:42:58 +08:00
Yongjie Zhao 4c708af710
fix: avoid filters containing null value (#17168) 2021-10-21 18:20:38 +08:00
Amit Miran 029ed90afb
fix: preventing sql lab None limit value (#17155)
* fix: preventing sql lab None limit value

* test: create a test for the fix

* pylint (#17172)

* add test (#17173)

Co-authored-by: Hugh A. Miles II <hughmil3s@gmail.com>
2021-10-20 23:43:14 +03:00
Ville Brofeldt d7834f17e3
fix(filter-indicator): show filters handled by jinja as applied (#17140) 2021-10-18 19:28:05 +02:00
John Bodley 565ee2318d
chore(druid): Explicitly cast col to TIMESTAMP (#17101)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-10-18 10:06:04 -07:00
Beto Dealmeida 83a2f8346e
chore: enable BQ tests (#17017) 2021-10-14 08:07:47 -07:00
John Bodley b9ff85d9ea
chore(engine): Translate fractional time grains—requires @superset-ui bump (#17078)
* chore(engine): Translate fractional time grains

* Bump @superset-ui

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-10-12 21:40:48 -07:00
John Bodley 5e85f48515
chore(druid): Standardizing time grain transformations (#17050)
* chore(druid): Standardizing time grain transformations

* Update druid_tests.py

* Update druid_tests.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-10-12 08:34:28 -07:00
Elizabeth Thompson c2e1ab6550
add logging on successful data uploads (#17065) 2021-10-11 16:28:07 -07:00
Yongjie Zhao fd8461406d
fix: rolling and cum operator on multiple series (#16945)
* fix: rolling and cum operator on multiple series

* add UT

* updates
2021-10-07 16:42:08 +08:00
Beto Dealmeida c993c5845f
fix(BigQuery): explicitly quote columns in select_star (#16822)
* fix (BigQuery): explicitly quote columns in select_star

* Fix test

* Fix SELECT * in BQ

* Add unit tests

* Remove type changes
2021-10-06 07:43:32 -07:00
Yongjie Zhao 387ac2b6d3
fix: replace absolute difference with difference in compareOperator (#16930)
* fix: replace absolute difference with difference in compareOperator

* fix ut
2021-10-04 13:31:19 +08:00
ofekisr 0d0c759cfe
refactor: sql lab command: separate concerns into different modules (#16917)
* chore move sql_execution_context to sqllab package

* add new helper methods into base Dao

* refactor separate get existing query concern from command

* refactor separate query access validation concern

* refactor separate get query's database concern from command

* refactor separate get query rendering concern from command

* refactor sqllab_execution_context

* refactor separate creating payload for view

* chore decouple command from superset app

* fix pylint issues

* fix failed tests

* fix pylint issues

* fix failed test

* fix failed black

* fix failed black

* fix failed test
2021-10-03 11:15:46 +03:00
apurva-sigmoid 420eff42b8
feat: add Firebolt DB engine spec (#16903)
* New branch from superset for integration with firebolt sqlalchemy adapter

* Added db_engine_spec file for Firebolt

* Removed firebolt code from superset repo

* Deleted virtual env commit

* Adding time grain changes to firebolt.py

* Updated README.md

Added steps to install and run Superset with Firebolt SQLAlchemy Adapter

* Update README.md

Reduced installation steps. Using PyPi installation for adapter now

* Revert "Update README.md"

This reverts commit 5ed17c7a45.

* Revert "Updated README.md"

This reverts commit 45c5072649.

* added epoch methods, added test cases for firebolt db engine spec and edited setup.py

* Added license to files

* Added documentation for Firebolt-SQLAlchemy

* Removed trailing whitespace

Co-authored-by: raghavsharma <raghavs@sigmoidanalytics.com>
Co-authored-by: raghavSharmaSigmoid <88667094+raghavSharmaSigmoid@users.noreply.github.com>
2021-10-01 19:43:15 +03:00
Yongjie Zhao 94282b7ecd
fix: time comparison can't guarantee the accuracy (#16895)
* fix: time comparison can't guarantee the accuracy

* fix multiple series

* fix lint

* fix ut

* fix lint

* more ut

* fix typo
2021-09-30 19:59:57 +08:00
AAfghahi aa747219ad
feat: Add Private Google Sheets to dynamic form (#16628)
* first pass private gsheets

* made encrypted extra into string, refactored onParametersChanged

* private sheets working, credential_info errors

* all but test connection working

* first pass private gsheets

* made encrypted extra into string, refactored onParametersChanged

* private sheets working, credential_info errors

* all but test connection working

* Regenerate package-lock.json

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-09-28 17:08:50 -07:00
Ville Brofeldt 0f16177bde
fix(native-filters): ignore unset filter box time range (#16854) 2021-09-27 17:17:52 +03:00
ofekisr c520eb79b0
refactor: sqllab: move sqllab ralated enumns and utils to more logical place (#16843)
* refactor move QueryStatus to common

* refactor move apply_display_max_row_limit to sqllab package

* refactor move limiting_factor to sqllab package

* fix pylint issues
2021-09-26 21:15:57 +03:00
ofekisr 84f7614e97
feat(filter-set): Add filterset resource (#14015)
* Add filterset resource

* fix: fix pre-commit

* add tests

* add tests and fixes based of failures

* Fix pre-commit errors

* chore init filterset resource under ff constraint

* Fix migration conflicts

* Fix pylint and migrations issues

* Fix pylint and migrations issues

* Fix pylint and migrations issues

* Fix pylint and migrations issues

* Fix pylint and migrations issues

* Fix pylint and migrations issues

* Fix pylint and migrations issues

* Fix pylint and migrations issues

* Fix pylint and migrations issues

* Fix pylint and migrations issues

* Fix pylint and migrations issues

* add tests and fixes based of failures

* Fix missing license

* fix down revision

* update down_revision

* fix: update down_revision

* chore: add description to migration

* fix: type

* refactor: is_user_admin

* fix: use get_public_role

* fix: move import to the relevant location

* chore: add openSpec api schema

* chore: cover all openspec API

* fix: pre-commit and lint

* fix: put and post schemas

* fix: undo superset_test_config.py

* fix: limit filterSetsApi to include_route_methods = {"get_list", "put", "post", "delete"}

* renaming some params

* chore: add debug in test config

* fix: rename database to different name

* fix: try to make conftest.py harmless

* fix: pre-commit

* fix: new down_revision ref

* fix: bad ref

* fix: bad ref 2

* fix: bad ref 3

* fix: add api in initiatior

* fix: open spec

* fix: convert name to str to include int usecases

* fix: pylint

* fix: pylint

* Update superset/common/request_contexed_based.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* chore: resolve PR comments

* chore: resolve PR comments

* chore: resolve PR comments

* fix failed tests

* fix pylint

* Update conftest.py

* chore remove BaseCommand to remove abstraction

* chore remove BaseCommand to remove abstraction

* chore remove BaseCommand to remove abstraction

* chore remove BaseCommand to remove abstraction

* chore fix migration

Co-authored-by: Ofeknielsen <ofek.israel@nieslen.com>
Co-authored-by: amitmiran137 <amit.miran@nielsen.com>
Co-authored-by: Amit Miran <47772523+amitmiran137@users.noreply.github.com>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2021-09-23 11:27:59 +03:00
Phillip Kelley-Dotson a198dbb19b
feat: add certifiedby & certification details fields to the edit dataset columns fields (#16454)
* add migration

* add backend and frontend for certified

* update migration with batch

* fix integration test and update Updating.md

* Update superset-frontend/src/datasource/DatasourceEditor.jsx

Co-authored-by: Geido <60598000+geido@users.noreply.github.com>

* Update superset-frontend/src/datasource/DatasourceEditor.jsx

Co-authored-by: Geido <60598000+geido@users.noreply.github.com>

* Update superset-frontend/src/datasource/DatasourceEditor.jsx

Co-authored-by: Geido <60598000+geido@users.noreply.github.com>

* change method name

* add tooltip info

* add mixin

* merge heads

* address comments

* fix select label styles

* add extra field

* fix test?

* add extra field to put schema

Co-authored-by: Geido <60598000+geido@users.noreply.github.com>
2021-09-22 15:09:30 -07:00
Michael S. Molina 596e1cdf9b
refactor: Changes the DatabaseSelector and TableSelector to use the new Select component (#16483) 2021-09-22 07:43:46 -03:00
Beto Dealmeida 48a61bacc5
fix: handle CTEs with comments on is_select (#16769) 2021-09-21 16:23:54 -07:00
Elizabeth Thompson 2a25e2d7ca
update execution logs and states for alerts (#16736) 2021-09-20 14:26:30 -07:00
Yongjie Zhao cc1c6c1bb6
feat: add resample operator in post processing (#16607)
* feat: add resample operator in post processing

* wip

* fill zero values

* updates

* fix ut
2021-09-17 16:40:59 +08:00
Ville Brofeldt 4e3d4f6daf
feat: add global max row limit (#16683)
* feat: add global max limit

* fix lint and tests

* leave SAMPLES_ROW_LIMIT unchanged

* fix sample rowcount test

* replace max global limit with existing sql max row limit

* fix test

* make max_limit optional in util

* improve comments
2021-09-16 19:33:41 +03:00
Ville Brofeldt a839649e5c
fix(pandas-postprocessing): percentage compare to use correct column (#16716) 2021-09-16 13:29:58 +03:00
Ville Brofeldt 836b5e2c86
feat: add support for generic series limit (#16660)
* feat: add support for generic series limit

* refine series_columns logic

* update docs

* bump superset-ui

* add note to UPDATING.md

* remove default value for timeseries_limit
2021-09-16 12:09:08 +03:00
Yongjie Zhao 21f98ddc21
fix: catch exception when create connection (#16692)
* fix: catch exception when create connection

* fix lint

* added UT
2021-09-16 15:55:57 +08:00
Beto Dealmeida 092ef5bdfc
fix: set importer as owner (#16656)
* fix: set importer as owner

* Fix tests
2021-09-15 12:27:02 -07:00
Ville Brofeldt 88c09c2138
feat(jinja): improve url parameter formatting (#16711)
* feat(jinja): improve url parameter formatting

* add UPDATING.md

* fix test
2021-09-15 20:00:26 +03:00
John Bodley fb4650a6eb
chore(pylint): Remove top-level disable (#16589)
* chore(pylint): Remove top-level disable

* Update examples.py

* Update command.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-09-15 09:30:23 -07:00
aniaan 9e00e4e8cc
fix(dataset): create ES-View dataset raise exception #16623 (#16624)
* fix(dataset): create es-view dataset raise exception #16623

* fix(database): fix has_view logic

* refactor(database): fix logic

* style(lint): remove unused typing

* fix(test): add test case

* fix(test): fix test case
2021-09-14 22:29:29 +08:00
Daniel Vaz Gaspar f2bc139e35
feat: feature flag configurable custom backend (#16618)
* feat: feature flag configurable custom backend

* fix lint

* simpler approach

* fix tests

* revert dependency updates

* Update superset/utils/feature_flag_manager.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Update superset/config.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2021-09-13 14:09:53 +01:00
Jack Fragassi 4dc859f89e
fix: Ensure alerts & reports aren't schduled when flag is off (#16639)
* Don't schedule alerts & reports when flag is off

* Fix test function name

* Fix test

* Oops

* Another tweak

* Try to lint by hand

* Fix mock
2021-09-10 13:37:21 +01:00
天河 bb014b5131
fix: fix assignment in FilterBoxViz (#16662)
* Fixing assignment.

Signed-off-by: tianhe1986 <w1s2j3229@163.com>

* Adding unit test for FilterBoxViz.

Signed-off-by: tianhe1986 <w1s2j3229@163.com>

* Reformatting with black.

Signed-off-by: tianhe1986 <w1s2j3229@163.com>

* Revert format change in other test.

Signed-off-by: tianhe1986 <w1s2j3229@163.com>

* Reformatting with the same black version with pre-commit config.

Signed-off-by: tianhe1986 <w1s2j3229@163.com>
2021-09-10 15:10:55 +03:00
AAfghahi c66f278b42
feat: Backend Validation for Creation Method (#16375)
* backend creation method validation

* added tests

* Update superset/reports/dao.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/reports/dao.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update tests/integration_tests/reports/api_tests.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update tests/integration_tests/reports/api_tests.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/reports/dao.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/reports/dao.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/reports/commands/create.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/reports/commands/exceptions.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* revisions

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-09-08 16:35:18 -07:00
Ville Brofeldt 9de2196b7f
fix: TemporalWrapperType string representation (#16614)
* fix: TemporalWrapperType string representation

* fix tests
2021-09-07 13:50:24 +03:00
ofekisr e60b489867
refactor sql_json view endpoint: separate concern into ad hod method (#16595) 2021-09-05 23:18:17 +03:00
ofekisr be77ad2288
refactor: sql_json view endpoint: encapsulate ctas parameters (#16548)
* refactor sql_json view endpoint: encapsulate ctas parameters

* fix failed tests

* fix failed tests and ci issues
2021-09-05 16:34:26 +03:00
Rob DiCiuccio 4e380db3fd
fix: Support Jinja template functions in global async queries (#16412)
* Support Jinja template functions in async queries

* Pylint

* Add tests for async tasks

* Remove redundant has_request_context check
2021-09-03 14:33:29 +03:00
Ville Brofeldt 070fdbeebc
fix(tests): make parquet select deterministic with order by (#16570) 2021-09-02 15:16:33 -03:00
Shiva Raisinghani d25b0967a1
feat: Add parquet upload (#14449)
* allow csv upload to accept parquet file

* fix mypy

* fix if statement

* add test for specificying columns in CSV upload

* clean up test

* change order in test

* fix failures

* upload parquet to seperate table in test

* fix error message

* fix mypy again

* rename other extensions to columnar

* add new form for columnar upload

* add support for zip files

* undo csv form changes except usecols

* add more tests for zip

* isort & black

* pylint

* fix trailing space

* address more review comments

* pylint

* black

* resolve remaining issues
2021-08-31 10:20:25 +03:00
Erik Ritter 8adc31d14c
Revert "chore: Changes the DatabaseSelector to use the new Select component (#16334)" (#16478)
This reverts commit c768941f2f.
2021-08-26 22:28:04 -03:00
Ville Brofeldt 35864748f2
fix(explore): retain chart ownership on query context update (#16419) 2021-08-24 18:37:34 +03:00
Ville Brofeldt f6637cac7d
fix(api): return total count on related endpoint (#16397)
* fix(api): return total count on related endpoint

* update response code from 400 to 422
2021-08-24 15:07:58 +03:00
Michael S. Molina c768941f2f
chore: Changes the DatabaseSelector to use the new Select component (#16334) 2021-08-23 15:41:03 -03:00
Beto Dealmeida adebc0997b
fix: update table ID in query context on chart import (#16374)
* fix: update table ID in query context on chart import

* Fix test
2021-08-20 10:03:31 -07:00
Hugh A. Miles II 50d896f1b7
fix: Fix parsing onSaving reports toast when user hasn't saved chart (#16330)
* don't maniuplate error message

* remove extra idx reference

* u

* change print

* update with test

* add case for dashboards

* fix test
2021-08-19 14:04:23 -04:00
Beto Dealmeida afb8bd5fe6
feat: improve embedded data table in text reports (#16335)
* feat: improve HTML table in text reports

* Remove unused import

* Update tests

* Fix test
2021-08-18 17:00:29 -07:00
Beto Dealmeida ac8e54d909
fix: improve pivot post-processing (#16289)
* fix: improve pivot post-processing

* Add tests

* Trim space from column name
2021-08-17 14:41:22 -07:00
Ville Brofeldt 36abc51f90
fix(dashboard): unset empty time filter indicator (#16272) 2021-08-16 19:32:05 +03:00
Ville Brofeldt d46dc9aa45
chore: upgrade mypy and add type guards (#16227) 2021-08-14 06:31:45 +03:00
John Bodley 24b43beff9
chore(pylint): Bump Pylint to 2.9.6 (#16146)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-14 10:32:28 +12:00
Michael S. Molina 720e5b111a
chore: Shows the dataset description in the gallery dropdown (#16200)
* chore: Shows the dataset description in the gallery dropdown

* chore: Adjusts the tooltip positioning, fixes the search and removes unnecessary bootstrap data
2021-08-13 15:08:12 -03:00
Yongjie Zhao 6cd15d54a0
refactor: external metadata fetch API (#16193)
* refactor: external metadata api

* fix comments

* fix ut

* fix fe lint

* fix UT

* fix UT
2021-08-13 20:56:42 +08:00
Ville Brofeldt d6f9c48aa1
feat(dao): admin can remove self from object owners (#15149) 2021-08-13 12:42:48 +03:00
Ville Brofeldt b61c34f7c9
fix(viz): deduce metric name if empty (#16194)
* fix(viz): deduce metric name if empty

* fix unit test
2021-08-12 11:16:05 +03:00
Beto Dealmeida 7b3fce7e81
fix: revert data endpoint name (#16162) 2021-08-10 11:59:27 -07:00
Hugh A. Miles II fd80ae34a3
fix: Make sheet_name into a `ValidationInputError` (#16056)
* setup validates for name

* add error type

* fix linting

* fix test

* remove errors

* fix number

* fix test
2021-08-10 13:07:31 -04:00
Phillip Kelley-Dotson f0e3b68cc2
fix: ensure that users viewing chart does not automatically save edit data (#16077)
* add last_change_at migration

* add last_saved_by db migration

* finish rest of api migration

* run precommit

* fix name

* run precommitt

* remove unused mods

* merge migrations

* Update superset/migrations/versions/6d20ba9ecb33_add_last_saved_at_to_slice_model.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/migrations/versions/6d20ba9ecb33_add_last_saved_at_to_slice_model.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/migrations/versions/f6196627326f_update_chart_permissions.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* fix test

* precommit

* remove print

* fix test

* change test

* test commit

* test 2

* test 3

* third time the charm

* fix put req

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-08-10 10:29:49 -06:00
Ville Brofeldt 63ace7b288
feat(cross-filters): add support for temporal filters (#16139)
* feat(cross-filters): add support for temporal filters

* fix test

* make filter optional

* remove mocks

* fix more tests

* remove unnecessary optionality

* fix even more tests

* bump superset-ui

* add isExtra to schema

* address comments

* fix presto test
2021-08-10 17:18:46 +01:00
Yongjie Zhao bb1d8fe4ef
fix: boolean type into SQL 'in' operator (#16107)
* fix: boolean type into SQL 'in' operator

* fix ut

* fix ut again

* update url

* remove blank line
2021-08-10 19:21:46 +08:00
Beto Dealmeida ddb5005900
fix: test (#16137) 2021-08-08 23:13:23 -07:00
Grace Guo 85329c374e
refactor: remove unnecessary dataset queries from dashboard requests (#16110)
* refactor: remove unnecessary dataset queries from dashboard requests

* fix comments
2021-08-06 19:01:05 -07:00
Beto Dealmeida 1dbd1e9f02
chore: simplify chart permissions (#16078) 2021-08-05 08:18:29 -07:00
Jesse Yang 490890de23
fix(dashboard): 500 error caused by data_for_slices API (#16053) 2021-08-03 19:01:39 -07:00
John Bodley 41e8190575
chore: Use datetime.timedelta for defining durations in config (#16029)
* chore: use datetime.timedelta for defining durations

* Update config.py
2021-08-03 08:22:30 +12:00
John Bodley c0615c55df
fix: Ensure table uniqueness on update (#15909)
* fix: Ensure table uniqueness on update

* Update models.py

* Update slice.py

* Update datasource_tests.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-08-03 07:45:55 +12:00
Yongjie Zhao 3f57ea76db
fix: missing mulitiple metrics on pivot operator (#16026)
* fix: missing mulitiple metrics on pivot operator

* code smell
2021-08-02 11:37:10 -07:00
Elizabeth Thompson 5031a67597
feat: Self subscribe reports (#16027)
Co-authored-by: Lyndsi Kay Williams <55605634+lyndsiWilliams@users.noreply.github.com>
Co-authored-by: AAfghahi <48933336+AAfghahi@users.noreply.github.com>
2021-08-02 09:12:09 -07:00
Yongjie Zhao a786373fff
feat: auto sync table columns when change dataset (#15887)
* feat: auto sync dataset metadata when change dataset

* diablo sync button when edit mode

* handle undefine schema

* fix py UT

* fix FE UT

* improve test coverage

* fix UT
2021-08-02 16:55:31 +08:00
Yongjie Zhao c01d42fd98
fix: eliminate cartesian product columns in pivot operator (#15975)
* fix: eliminate cartesian product columns in pivot operator

* wip

* wip

* minor tip
2021-07-31 16:02:04 +08:00
Beto Dealmeida 2d61f15153
feat: send post-processed data in reports (#15953)
* feat: send post-processed data in reports

* Fix tests and lint

* Use enums

* Limit Slack message to 4k chars
2021-07-30 09:37:16 -07:00
Beto Dealmeida f4739f427e
feat: post-processing for pivot table v2 (#15879)
* feat: add pivot v2 post-processing

* Fix lint
2021-07-29 11:05:56 -07:00
Beto Dealmeida 6afa840659
feat: send report data to Slack (#15806)
* feat: send data embedded in report email

* Change post-processing to use new endpoint

* Show TEXT option only to text-based vizs

* Fix test

* feat: send data embedded in report email

* feat: send report data to Slack

* Add unit test

* trigger tests
2021-07-29 10:13:28 -07:00
Hugh A. Miles II bfe7eb9a7b
feat(dbc ui): Adding Google Sheets Dynamic Form (#15801)
* feat: Make Google Sheets Dyanmic (#15576)

* first draft

* second draft

* added tests

* first draft

* added table_catalog

* remove console.log

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/index.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* revisions

* save this for now

* working form

* save disable on public sheets

* refactor somethings

* saving this for now

* working edit

* add back query to schema

* working add

* fix styling

* fixing x

* fix linting

* prettier

* fix some type issues

* more lint fixes

* remove unused dependency

* fix linint

* fix validation

* pylint bypass

* pylint bypass

* fix this

* fix mypy

* yerp

* fix test

* fix test

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/index.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* wrap add sheets

* fix linting issues

* fix unit test

* ignore typing

* fix editting and paste issues

* remove query

* fix this

* fix test

* add test back

* fix error messaging

* update url messaging on error

* change error type

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/index.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* add errors for sheets with no name

* fix

* fix messaging for gsheets

* stop pylint

* update line

Co-authored-by: AAfghahi <48933336+AAfghahi@users.noreply.github.com>
Co-authored-by: Arash <arash.afghahi@gmail.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-07-28 15:00:27 -04:00
Beto Dealmeida 3adf8e85cd
feat: send data embedded in report email (#15805)
* feat: send data embedded in report email

* Prettify table

* Change post-processing to use new endpoint

* Show text option only for text viz

* Show TEXT option only to text-based vizs

* Fix test

* Add email test

* Add unit test
2021-07-28 09:43:04 -07:00
Yongjie Zhao 32d2aa0c40
feat: run extra query on QueryObject and add compare operator for post_processing (#15279)
* rebase master and resolve conflicts

* pylint to makefile

* fix crash when pivot operator

* fix comments

* add precision argument

* query test

* wip

* fix ut

* rename

* set time_offsets to cache key

wip

* refactor get_df_payload

wip

* extra query cache

* cache ut

* normalize df

* fix timeoffset

* fix ut

* make cache key logging sense

* resolve conflicts

* backend follow up iteration 1

wip

* rolling window type

* rebase master

* py lint and minor follow ups

* pylintrc
2021-07-28 22:34:39 +08:00
Elizabeth Thompson ea49aa3d2d
feat: add timezones to report cron (#15849)
* add timezones to report cron

* fix test
2021-07-27 18:28:24 -07:00
Beto Dealmeida 2ce676d20d
feat: call screenshot to store `query_context` (#15846)
* feat: call screenshot to store query_context

* Add unit test

* Move updateQueryContext to ExploreChartPanel

* Add error handling

* Fix code

* Fix logic
2021-07-27 14:02:27 -07:00
Hugh A. Miles II 62550db385
fix: Update Query Context on Explore loading (#15865)
* create serialize json function

* saving for now

* saving for now

* lint

* cleanup

* fix network request

* update test

* Update tests/integration_tests/charts/api_tests.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-07-26 12:43:25 -07:00
Beto Dealmeida 2f95f81be7
feat: apply post processing to chart data (#15843)
* feat: apply post processing to chart data

* Fix tests and lint

* Fix lint

* trigger tests
2021-07-26 10:58:59 -07:00
John Bodley 6d3e19d857
fix(15403): Re-enable canceling query for Hive and Presto (#15878)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-07-26 08:04:56 -07:00
John Bodley 4ba17092fa
fix: Ensure SupersetError.extra is always a dict (#15848)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-07-24 10:19:30 +12:00
Beto Dealmeida f104fba61d
feat: add `GET /api/v1/chart/{chart_id}/data/?format{format}` API (#15827)
* feat: add `GET /api/v1/chart/{chart_id}/data/?format{format}` API

* Fix test
2021-07-21 17:03:22 -07:00
Beto Dealmeida 3441182630
chore: remove unnecessary deps (#15787)
* chore: remove unneeded deps

* fix lint

* Run compile
2021-07-21 16:53:45 -07:00
Beto Dealmeida 9a79a5775b
feat: store query context when saving charts (#15824)
* WIP

* Add migration

* Fix tests
2021-07-21 13:54:39 -07:00
John Bodley ab4e3b9bf9
chore: Enforce Mypy for non-tests (#15757)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-07-22 06:46:43 +12:00
Erik Ritter e969edc451
fix: Bust chart cache when metric/column is changed (#15786) 2021-07-21 07:46:20 -07:00
AAfghahi 3a249a0090
feat: add show columns to Reports model (#15712)
* added logic for creation_method

* revisions

* added index

* Update superset/migrations/versions/3317e9248280_add_creation_method_to_reports_model.py

* filters

* search columns updated
2021-07-19 11:47:33 -07:00
Elizabeth Thompson 618a354ca1
Revert "quote column name if db requires (#15465)" (#15752)
This reverts commit 80b8df0673.
2021-07-19 10:31:05 -07:00
AAfghahi 674f234de6
feat: add logic to creation_method for reports schedule (#15685)
* migration

* added logic for creation_method

* revisions

* added index

* Update superset/migrations/versions/3317e9248280_add_creation_method_to_reports_model.py

* filters

* Update superset/models/reports.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/reports/schemas.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update tests/integration_tests/reports/api_tests.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* revisions

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-07-15 18:27:54 -07:00
u-aiaa 6b790990a8
fix: Add waiting time for chart animation when screenshot (#15610) 2021-07-15 18:25:48 -07:00
Peter Kosztolanyi 02032ee8a4
feat: cancel db query on stop (#15403)
* feat: cancel db query on stop

* fix pylint

* Add unit tests

* Do not bind multiple times

* Stop only running queries

* Postgres to cancel only the required query

* Remove extra log

* Add docstring

* Better types, docstring and naming

* Use python3 format strings

* Update superset/sql_lab.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Add cancel_query_on_windows_unload option to database

* Return cancel_query as bool

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-07-13 12:09:22 -04:00
Hugh A. Miles II f39582c900
fix: change sslmode to require for Postgres (#15642)
* change sslmode to require

* fix test
2021-07-12 15:38:07 -07:00
Amit Miran 62a8f2e193
chore(python-testing): move memoized tests to unit tests (#15507)
* chore: move memoized test into a separated file

create integration test workflow

* chore: create unit test workflow to run purely pytest

* fix: bad reference

* fix: remove pip requirements bc there aren't any yet

* temp: install unit dependencies directly

* fix: --rootdir=

* fix: try to run only unit test

* chore: decouple memoized as separated module

* fix: bring back dependencies bc superset top-level module is coupled to flask and others so no reason no to do it

* fix: reference

* fix: pre-commit

* fix: pylint
2021-07-12 10:00:18 +03:00
Beto Dealmeida 4f5f9287fc
feat: validate_parameters for GSheets (#15578)
* feat: validate_parameters for GSheets

* Move import inside fixture

* Update deps

* Rename parameter
2021-07-08 08:26:39 -07:00
Yongjie Zhao ae160f7f21
chore: add changed_on_delta_humanized field on dashboard schema (#15542)
* chore: add changed_on_delta_humanized field on dashboard schema

* fix ut
2021-07-07 14:59:49 -07:00
Beto Dealmeida 8f92618e46
fix: show all dbs in available endpoint (#15534) 2021-07-02 12:58:36 -07:00
Elizabeth Thompson 80b8df0673
quote column name if db requires (#15465)
Co-authored-by: hughhhh <hughmil3s@gmail.com>
2021-07-02 12:48:24 -05:00
Hugh A. Miles II d4480f5c9a
feat: Database Connection UI (#14881) 2021-07-01 14:40:27 -07:00
ofekisr b5119b8dff
refactor(tests): decouple unittests from integration tests (#15473)
* refactor move all tests to be under integration_tests package

* refactor decouple unittests from integration tests - commands

* add unit_tests package

* fix celery_tests.py

* fix wrong FIXTURES_DIR value
2021-07-01 18:03:07 +03:00
Beto Dealmeida e606477ec1
feat: more SIP-40 errors (#15482) 2021-06-30 16:32:59 -07:00
John Bodley ffa51753e3
refactor: Moving get_user_datasources to security manager (#15467)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-07-01 04:51:11 +12:00
Yongjie Zhao 61e6b7017b
fix: nvd3 bar chart sortby metric (#15318) 2021-06-30 19:36:05 +08:00
Yongjie Zhao 292bce5862
fix: raise unexpected error when orderby is empty (#15353)
* fix: raise unexpected error when orderby is empty

* fix ut
2021-06-30 19:34:54 +08:00
AAfghahi 743d9cc928
feat: Better Errors in SQL Lab (#15432)
* snowflake errors

* added big query

* added to setup error messages, first test

* all big query testing added

* added snowflake test

* added syntax error

* added syntax errors to most used databases
2021-06-29 16:48:27 -07:00
Ville Brofeldt ab7f31fd85
Revert "refactor(feature_flags configurations): remove redundant additional configuration for default vales (#15425)" (#15448) 2021-06-29 16:52:01 +03:00
ofekisr 486b8d911f
refactor(feature_flags configurations): remove redundant additional configuration for default vales (#15425) 2021-06-28 16:30:13 +03:00
Kamil Gabryjelski 09c44d05fd
feat(native-filters): Hide non-numeric columns in numeric range filter (#15385)
* feat(native-filters): Hide non-numeric columns in numeric range filter

* Return true if type_generic undefined

* Code review comments

* Replace any with string

* fix tests

* add missing columns to select

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2021-06-27 07:35:17 +03:00
Beto Dealmeida 241ee32f56
feat: custom error SQL Lab timeout (#15342)
* feat: custom error SQL Lab timeout

* Update test
2021-06-24 08:02:49 -07:00
Beto Dealmeida 4b00c152cc
feat: implement specific errors for SQL Lab (#15206)
* RESULTS_BACKEND_NOT_CONFIGURED_ERROR

* DML_NOT_ALLOWED_ERROR

* INVALID_CxAS_QUERY_ERROR

* Fix lint

* Add more tests
2021-06-23 07:58:20 -07:00
Beto Dealmeida 216e2b8e8e
fix: datasource payload is incorrect (#15184)
* fix: datasource payload is incorrect

* Add tests, clean code
2021-06-23 06:59:34 -07:00
Elizabeth Thompson 58cc78d2c1
fix: return query if it already exists (#15207)
* check if query exists before saving a new one

* fix test
2021-06-22 13:57:37 -07:00
Ben Reinhart ab153e66cc
feat: Synchronously return cached charts (#15157)
* feat: Synchronously return cached charts

* Fix lint issue

* Fix python lint error

* Change getChartDataRequest to return response

* Fix lint errors

* Add test

* explore_json: skip cached data check for forced refresh

Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
2021-06-22 10:00:57 -07:00
Ville Brofeldt 822eb2e27e
feat(sql): add jinja support to metrics and expressions (#15247)
* feat(sql): add jinja support to metrics and expressions

* add test
2021-06-19 08:29:04 +03:00
Daniel Wood a1027152ba
add ascend engine spec (#14682) 2021-06-18 10:35:55 -07:00
Ville Brofeldt 1269cc2f88
fix(examples): calendar chart metric should be metrics (#15173)
* fix(examples): calendar chart metric should be metrics

* fix presto test
2021-06-17 14:55:50 +03:00
Ville Brofeldt 83602923f5
feat(api): add featured datatypes to dashboard dataset ep (#15188) 2021-06-16 12:36:11 +03:00
Beto Dealmeida 75018bf99f
fix: Presto postgres test (#15163) 2021-06-15 07:10:50 -07:00
Beto Dealmeida 90d9097841
fix: validate DB-specific parameters (#15155)
* fix: validate DB-specific parameters

* Fix lint

* Update test

* Fix lint/test

* Fix lint

* Update superset/databases/api.py
2021-06-14 18:44:18 -07:00
Beto Dealmeida cc2b4fe3f4
fix: show custom errors in SQL Lab (#14959)
* fix: show custom errors in SQL Lab

* Fix lint

* Fix test

* Update superset/views/base.py

Co-authored-by: ʈᵃᵢ <tai@apache.org>

* Fix lint

* Debug failing test

* Remove print()

* Debug flaky tests

* Fix test

Co-authored-by: ʈᵃᵢ <tai@apache.org>
2021-06-10 15:20:31 -07:00
Beto Dealmeida 42cb5266fa
fix: import metrics with extra (#15047)
* fix: import metrics with extra

* Fix test
2021-06-08 18:56:55 -07:00
Daniel Vaz Gaspar 4e998e62fc
feat: add more timeout configuration on screenshots (#14868)
* feat: more timeout configuration on screenshots

* add tests
2021-06-08 15:33:42 +01:00
Beto Dealmeida 8e7f0237ab
fix: apply template_params on external_metadata (#14996)
* fix: apply template_params on external_metadata

* Fix test
2021-06-04 18:12:22 -07:00
Yongjie Zhao 723a67156c
fix: time parser truncate to first day of year/month (#14945) 2021-06-02 21:26:37 +08:00
Yongjie Zhao f4de0dcb65
fix: is_temporal should overwrite is_dttm (#14894)
* fix: is_temporal should overwrite is_dttm

* move up
2021-06-02 18:00:15 +08:00
rijojoseph07 a85f5c1344
feat(trino): add support for user impersonation (#14843)
* trino impersonation feature

* Extra options label update

* Update superset/db_engine_specs/trino.py

Co-authored-by: Đặng Minh Dũng <dungdm93@live.com>

Co-authored-by: rijojoseph01 <rijo.joseph@myntra.com>
Co-authored-by: Đặng Minh Dũng <dungdm93@live.com>
2021-05-29 09:54:18 +03:00
Hugh A. Miles II 8f81fc0ad7
fix: Redshift parameters not rendering (#14888)
* Update redshift.py

* Update api_tests.py

* fixed test

* add file back
2021-05-28 12:57:15 -04:00
Beto Dealmeida 2313e3ef4f
fix: show error on invalid import (#14851)
* fix: show error on invalid import

* Add unit test

* Remove unused imports

* Fix tests
2021-05-27 14:46:41 -07:00
Beto Dealmeida 8febd81baf
feat: validate database parameters (#14883) 2021-05-27 14:46:28 -07:00
Hugh A. Miles II ae8378a50e
fix: Big Query Edit Form (#14850)
* fix big quert edit form

* fix test

* fix api test

* fix test
2021-05-26 16:13:26 -04:00
Beto Dealmeida 8b1a117d24
feat: return parameters only for DB with default driver (#14803)
* WIP

* Fix sorting of DBs
2021-05-25 14:46:54 -07:00
Hugh A. Miles II c72894725e
fix: Fix Big Query API for POST w/ no parameters (#14822)
* Update schemas.py

* Update bigquery.py

* Fix tests

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-05-25 17:13:17 -04:00
AAfghahi 904b60e420
chore: added BasicParametersMixin to Redshift (#14752)
* redshift basic mixin and tests

* rebased

* forgot prefix
2021-05-24 17:32:53 -04:00
Hugh A. Miles II 6d33432b58
feat: Create BigQuery Parameters for DatabaseModal (#14721) 2021-05-23 12:45:48 -04:00
Ben Reinhart a06a2f30af
fix: Set g.user to anon user in Celery (#14742)
* Set g.user to anon user in Celery

* Add test

* Fix bug in logic
2021-05-21 14:33:08 -07:00
Ben Reinhart d5c008dd99
chore: Perform feature/config condition checks at request time (#14684)
* chore: conditional Home link rendering

* chore: conditional RowLevelSecurity rendering

* chore: Conditional KV rendering

* chore: Conditional TagView rendering

* chore: Conditional import dashboards link

* chore: Conditional upload csv/excel links

* chore: Conditional log api and view rendering

* chore: Conditionally render email schedules

* chore: Conditionally render alert views

* chore: Conditionally render alerts/reports

* chore: Conditionally render access requests

* chore: Conditionally render druid views

* Remove unnecessary folder

* Consistent naming

* Cleanup

* Remove object from class

* Clean up test file

* Clean up test file

* Fix lint error

* Better naming and follow conventions

* Use assertLess over assertNotEqual

* Assert less than 400

* Fix failing test
2021-05-21 14:29:52 -07:00
cccs-jc 590fe20a45
feat: Add a remove filter_flag to jinja filter_values function (#14507)
Implementation issue 13943

Co-authored-by: cccs-jc <cccs-jc@cyber.gc.ca>
2021-05-21 19:37:09 +03:00
Ville Brofeldt 7c17b1a97f
fix(pivot): default missing series to NULL_STRING (#14748) 2021-05-21 18:54:15 +03:00
AAfghahi 67e373000e
feat: Expanded Parameters for Mysql (#14680)
* added mysql form

* revisions

* Update superset/db_engine_specs/mysql.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* added ssl and mysql testing

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-05-19 08:47:33 -07:00
Beto Dealmeida 5e95d4638c
fix: import dataset with extra; Vertica URI (#14698)
* fix: import dataset with extra; Vertica URI

* Fix lint
2021-05-18 20:51:33 -07:00
Ajay M 90378ed94e
fix(explore): #10098 boolean filter not working (#14567)
* Restrict operators when column is boolean

* refactor 'isOperatorRelevant' a little bit

* Include 'BOOLEAN' to handle presto

* Update tests

* number column should show bool operators

* fix test - some dbs translate true/false to 1/0

* Fix tests and add linting

* When column type is boolean, show bool operators

* Address PR comments - simplify conditions

* Fix a linting error

* Addressing PR comment - remove unused variables
2021-05-18 14:39:19 -07:00
Beto Dealmeida 971f5883f0
feat: add SSL to new DB parameters (#14673)
* feat: add SSL to new DB parameters

* Fix test

* Raise if cls.encryption_parameters is empty
2021-05-17 21:27:43 -07:00
AAfghahi 852842028a
make config method optional (#14668) 2021-05-17 17:09:20 -04:00