Commit Graph

310 Commits

Author SHA1 Message Date
Yongjie Zhao
414cc99ca2
fix: datetime.data in series (#20618) 2022-07-07 09:33:44 +08:00
Reese
f9109583ce
fix: Allow dataset owners to explore their datasets (#20382)
* fix: Allow dataset owners to explore their datasets

* Re-order imports

* Give owners security manager permissions to their datasets

* Update test suite

* Add SqlaTable to is_owner types

* Add owners to datasource mock

* Fix VSCode import error

* Fix merge error
2022-07-06 11:27:50 -03:00
John Bodley
ad308fbde2
chore(rls): Remove passing global username (#20344)
* chore(rls): Remove passing global username

* Update manager.py

* Update manager.py

* Update manager.py

* Update manager.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2022-07-05 10:51:24 -07:00
Yongjie Zhao
b870a21eaa
refactor: remove unused flatten function (#20582) 2022-07-01 19:17:55 +08:00
Elizabeth Thompson
44f0b511dd
remove autoflush for queries during dual write (#20460) 2022-06-23 14:50:30 -07:00
Beto Dealmeida
f3b289d3c3
fix: ensure column name in description is string (#20340)
* fix: ensure column name in description is string

* Add unit test
2022-06-21 10:10:46 -03:00
Hugh A. Miles II
e3e37cb68f
chore: switching out ConnectorRegistry references for DatasourceDAO (#20380)
* rename and move dao file

* Update dao.py

* add cachekey

* Update __init__.py

* change reference in query context test

* add utils ref

* more ref changes

* add helpers

* add todo in dashboard.py

* add cachekey

* circular import error in dar.py

* push rest of refs

* fix linting

* fix more linting

* update enum

* remove references for connector registry

* big reafctor

* take value

* fix

* test to see if removing value works

* delete connectregistry

* address concerns

* address comments

* fix merge conflicts

* address concern II

* address concern II

* fix test

Co-authored-by: Phillip Kelley-Dotson <pkelleydotson@yahoo.com>
2022-06-21 13:22:39 +02:00
Daniel Vaz Gaspar
60eb1094a4
feat: add name, description and non null tables to RLS (#20432)
* feat: add name, description and non null tables to RLS

* add validation

* add and fix tests

* fix sqlite migration

* improve default value for name
2022-06-20 13:52:05 +01:00
John Bodley
8b0bee5e8b
[fbprophet] Fix frequencies (#20326) 2022-06-19 21:53:09 -07:00
mohittt8
41bbf62e58
fix(presto): use correct timespec for presto (#20333) 2022-06-16 16:43:17 -07:00
Beto Dealmeida
998624b1a5
feat: allow setting db UUID (#20412)
* WIP

* feat: allow passing UUID when creating a DB

* Test

* Fix field
2022-06-16 15:53:59 -07:00
John Bodley
b08e21efd9
[requirements] Resolve rebase conflicts (#20152)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2022-06-03 15:27:35 -07:00
Elizabeth Thompson
32bb1ce3ff
feat!: pass datasource_type and datasource_id to form_data (#19981)
* pass datasource_type and datasource_id to form_data

* add datasource_type to delete command

* add datasource_type to delete command

* fix old keys implementation

* add more tests
2022-06-02 16:48:16 -07:00
Diego Medina
8638f59b4c
fix: Box Plot Chart throws an error when the average (AVG) / SUM is being calculated on the Metrics (#20235)
* fix: Box Plot Chart throws an error when the average (AVG) / SUM is being calculated on the Metrics

* add test
2022-06-02 10:00:04 +08:00
Beto Dealmeida
05a138a191
feat: query datasets from SQL Lab (#15241)
* feat: Jinja2 macro for querying datasets

* Add docs

* Address comments
2022-06-01 15:18:40 -07:00
Ville Brofeldt
64c4226817
fix(temporary-cache): when user is anonymous (#20181)
* fix(temporary-cache): fail on anonymous user

* make exceptions generic

* fix test

* remove redundant bool return

* fix unit tests
2022-05-26 14:45:20 +03:00
Phillip Kelley-Dotson
0501ad25e8
fix: always create parameter json field (#19899)
* fix: always create parameter json field

* ensure validation for empty catalog

* check engine instead of name

* put validation in be

* fix test

* fix test

* remove test
2022-05-25 14:23:25 -07:00
Ville Brofeldt
e2f11d3680
fix(presto,trino): use correct literal dttm separator (#20123)
* fix(presto,trino): use correct literal dttm separator

* remove redundant tests
2022-05-19 20:42:31 +03:00
Hugh A. Miles II
660af409a4
feat: Save column data into json_metadata for all Query executions (#20059)
* add save_metadata function to QueryDAO

* use set_extra_json_key

* added test

* Update queries_test.py

* fix pylint

* add to session

* add to session

* refactor

* forgot the return
2022-05-18 13:11:14 -04:00
cccs-RyanS
ddc01ea781
feat(business-types): initial implementation of SIP-78 (#18794)
* add BUSINESS_TYPE_ADDONS to config with example callback

* Removing uneeded whitespace

* [Work in progress] Modifying cidr function to allow for  single ip and adding port outline

* Added test REST endpoint, added some more ports

I've thrown in a test.py script as well that will try to connect to the
business_type endpoint.

* Moving code from config.py into the business api

Very simple api is exposed that will allow someone to call a checkport
endpoint and get back a response.

* Removing commented out bits.

* Adding fucntion dict back to the config

* Moving business_type endpoint to charts

* Adding schema for get endpoint

* Removing imports, updating docstring, fixing typo

Just some small changes as described in the title.  I've updated the
test.py as well so it functions with the endpoint changes.

* Adding translation dict

* Fixing ops

* Adding check for list

* Modifying changes to add quotes where needed

Also changed BusinessTypeResponse to resp.

* Adding in some code to call the filter config

If a column starts with "cidr_" it will call the code in config.py to
try to translate the filter.  Nothing is changed in the JSON being
executed, just some information is dumped to console.

* Porting Ryan's changes

* Adding migration script (as per Ryan's PR)

* Fixing typo

* Prettier fixes

* [CLDN-1043] Adding rough version of filter changes for business types

* fix down migration

* Fixing bugs after merge

* adding functionality to appy filters in back end

* Fixing linting issues

* fix down revision

* Changing conversion callback to handle multiple values at once

* Adding string representation of values

* Code cleanup plus fixing debouce to only be called once for each entry

* Removing non needed logginh

* Changing operator list to use sting values

* Using text value operators

* Removing clear operator call

* Moving business type endpoints

* fix down revision

* Adding port functions

* update migration

* fix bad rebase and add ff

* implement validator

* dont add invalid values to response

* [CLDN-1205] Added a new exception type for a business type translation error. Added the error message in the display_value field within the business type response. Modified the IP and Port business types to populate the error message field in the response if an error occurs

* [CLDN-1205] Added meaningful error message for port translation errors

* Removing status field from businesstype Response and adding in error message

* [CLDN-1205] Added check to make sure the port business type is within the valid range of ports, if it is not, it will populate the error message

* [CLDN-1205] Fixed the if statement that checks to see if the string_value is in the valid range of port numbers. It did not corrently verify this before now.

* [CLDN-1205] Fixed an error where it was trying to use string_value in <= statements. I just casted string_value to an integer if it is numeric, which allows <= operators to be used on it

* [CLDN-1207] Added unit tests for the cidr_func and port_translation_func functions which are located in /superset/config.py

* [CLDN-1207] removed the assertRaises line as it does not work with the cidr_func and port_translation_func functions

* [CLDN-1207] Added the skeleton of the test_cidr_translate_filter_func unit test, still need to update what the expected response from the function will be.

* [CLDN-1207] Added the remainder of the back-end unit tests for the business types

* [CLDN-1207] Fixed the syntax error which caused the test_cidr_translate_filter_func_NOT_IN_double unit test to fail

* [CLDN-1207] Removed the logging that was added for debugging purposes

* [CLDN-1207] Formatted the commands_tests.py file to make it nicer to look at/read through

* [CLDN-1207] Fixed the code so that it conformed to the pylint requirements (i.e., pylint no longer complains about the code in commands_tests.py)

* [CLDN-1207] Modified some of the docstrings so they made better use of the 100 character per line, line limit

* [CLDN-1207] Added the beginnings of the unit tests for the
business types API

* [CLDN-1207] Added a comment to the top of the commands_tests.py file explaining how to run the unit tests. This prevents the next person who tries to run them from having to waste time trying the different forms of testing that Superset supports (e.g., pytest, tox, etc.)

* [CLDN-1207] Added a grammar fix to the comments describing how to run the unit tests

* [CLDN-1207] Modified the description of the business_type API endpoints as they did not represent what the API was actually doing

* [CLDN-1207] Added further instructions on how to run the unit tests that are within the business_type/api_tests.py file

* add request validation

* disable request if business type missing

* [CLDN-1207] Unit tests for the business type API are now working, however, they need to be modified to make use of @mock as we don't want to have to run the server to be able to run the unit tests

* Removing businesss types deffinitons from config

* Adding select to only show valid business types

* Fixed Enzyme tests

* Added scalfolding for selecting filter dropdown

* Adding intigration tests

* fix revision

* fix typos and unnecessary requests

* break out useBusinessTypes

* Added front-end RTL unit tests for the business type API endpoint

* Fixed error from unit tests

* Added a unit test to ensure the operator list is updated after a business type API response is received

* Removing elect compoenet for business types

* Adding feature flag and allowing saving when no business type present

* fixing useEffect hooks

* Adding feature flag to model

* Changing behavior such that an empty string returns a default response

* add form validation

* Modified comments in unit test as command to run test has changed

* Modified comments in unit test as filename to run test has changed

* Modified the api_tests.py file to conform to the linting requirements

* Changed the name of one of the tests to reflect what the test is actually testing

* Added cypress back to the package.json

* Added informative comments

* Updated comments in files as well as removed imports which were not being used

* Changes made by npm run prettier

* Fixed spelling mistakes

* Updated models.py to remove placeholder comments used in development

* Added feature flag mocking in unit test

* Fixing open api failure

* Fixing business types to pass unit tests

* Reverting unsafe connections back to false

* Removing print statement

* Adding business tpye to export test

* setting default feature flag to false for business type

* Reverting pre commit

* Reverting pre commit and running pre commit

* Reverting pre commit and running pre commit

* Fixing formatting

* Adding license

* Fixing Linting

* Protecting api enpoints

* updating model

* Fixing code path when business type exists

* Linting

* Linting

* Fixing linting

* Fixing spelling

* Fixing schemas

* Fixing app import

* fixing item render

* Added RTL test to make sure business type operator list is updated after API response

* Fixing linting

* fix migration

* Changing unit tests

* Fixing import and DB migration after rebase

* Renaming to advanced types

* Fixing Linting

* More renaming

* Removing uneeded change

* Fixing linting and test errors

* Removing unused imports

* linting

* Adding more detailed name for migration

* Moving files to plugins

* more renaming

* Fixing schema name

* Disabling feature flag that should not be enabled by default

* Adding extra cehck

* NameChange

* formatting

* Fixing equals check

* Moveing all advanced type classes and types to one file, and converting tests to functional

* Adding advanced type to test and fix linitng

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
Co-authored-by: Dan Parent <daniel.parent@cse-cst.gc.ca>
Co-authored-by: GITHUB_USERNAME <EMAIL>
Co-authored-by: cccs-Dustin <96579982+cccs-Dustin@users.noreply.github.com>
2022-05-16 17:58:21 +03:00
thinhnd2104
85315460cb
fix: Athena timestamp literal format (#19970)
* Fix Athena timestamp literal format

related: [#19969]

* Update test_athena.py

Update test

* Update athena.py

* Resolving CI/CD

* lint

Co-authored-by: Thinh Nguyen Duc <thinhnd.it@tripi.vn>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2022-05-16 09:50:20 +03:00
Hugh A. Miles II
21c5b26fc8
feat(sip-68): Add DatasourceDAO class to manage querying different datasources easier (#20030)
* restart

* update with enums

* address concerns

* remove any
2022-05-13 12:28:57 -04:00
John Bodley
449d08b25e
fix: Refactor SQL username logic (#19914)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2022-05-12 21:03:05 -07:00
Beto Dealmeida
f2881e5bbd
feat: RLS for SQL Lab (#19999)
* feat: RLS for SQL Lab

* Small fixes

* Pass username to security manager

* Update docstrings

* Add tests

* Remove type from docstring
2022-05-12 11:03:01 -07:00
Yeachan Park
8b72354654
fix(sqllab/charts): casting from timestamp[us] to timestamp[ns] would result in out of bounds timestamp (#18873)
* fix casting from timestamp[us] to timestamp[ns] would result in out of bounds timestamp from sqllab and charts

* Add unittests

* Lint changes and parameter variable rename

* Fix linting
2022-05-04 09:47:12 +03:00
Beto Dealmeida
90130e4b4f
feat: simplify SQLite time grain (#19745) 2022-05-03 07:13:33 -07:00
Ville Brofeldt
99f1f9ec61
fix(sqla): replace custom dttm type with literal_column (#19917) 2022-05-03 07:19:36 +03:00
Beto Dealmeida
1ebdaac487
fix: memoize primitives (#19930) 2022-05-02 14:50:56 -07:00
Beto Dealmeida
aff10a7fad
feat: simplify memoized_func (#19905) 2022-04-29 15:55:58 -07:00
Phillip Kelley-Dotson
ad715429f9
chore: simplify error messaging in database modal (#19165)
* testing for dbconn modal error message

* remove consoles and error alert mapping

* lint fix

* update url message

* add modal fix

* update comments

* fix err msg bugs

* fix pylint

* fix line

* fix tests

* fix assertions
2022-04-21 12:44:21 -07:00
Yongjie Zhao
3c28cd4625
feat: add renameOperator (#19776) 2022-04-20 19:48:12 +08:00
Jesse Yang
231716cb50
perf: refactor SIP-68 db migrations with INSERT SELECT FROM (#19421) 2022-04-19 18:58:18 -07:00
Yongjie Zhao
9425dd2cac
fix: drop the first level of MultiIndex (#19716) 2022-04-14 23:40:38 +08:00
Yongjie Zhao
d7dd4119d4
fix: time comparision (#19659) 2022-04-12 18:19:22 +08:00
Ville Brofeldt
9a9e3b6e3b
test(jinja): refactor to functional tests (#19606) 2022-04-08 20:32:38 +03:00
Beto Dealmeida
83c3779a6c
feat: add a where_in filter for Jinja2 (#19574) 2022-04-07 07:27:23 -07:00
Elizabeth Thompson
aa419b8119
fix: check type of url before performing string actions (#19569)
* ensure url is a string

* return url if param is a url
2022-04-06 17:32:10 -07:00
Beto Dealmeida
6828624f61
feat: improve adhoc SQL validation (#19454)
* feat: improve adhoc SQL validation

* Small changes

* Add more unit tests
2022-03-31 11:55:19 -07:00
James Turton
85e330e94b
fix(drill): specify an SA URL parm of impersonation_target for drill+sadrill (#19252)
* Update drill+sadrill to specify an SA URL parm of "impersonation_target".

Sqlalchemy-drill is being updated to support impersonation with the
drill+sadrill driver, where previously it did not.  The way that callers
should specify impersonation matches that for the drill+jdbc driver in that
a SA URL parameter of impersonation_target should be set to the username
of the user to be impersonated, while the stadard SA username and password
should be those of the proxy user.

* Remove lint.

* Address review comments.

* Use idiomatic pytest to test for a raised exception.

* Fix import statement order in drill.py.
2022-03-31 19:42:27 +03:00
Beto Dealmeida
63b5e2e4fa
perf: improve perf in SIP-68 migration (#19416)
* chore: improve perf in SIP-68 migration

* Small fixes

* Create tables referenced in SQL

* Update logic in SqlaTable as well

* Fix unit tests
2022-03-29 22:33:15 -07:00
Ville Brofeldt
a619cb4ea9
chore: upgrade black (#19410) 2022-03-29 20:03:09 +03:00
Yongjie Zhao
a39dd4493e
feat: linear imputation in Resample (#19393) 2022-03-28 22:30:45 +08:00
Ville Brofeldt
f4b71abb22
feat: introduce hashids permalink keys (#19324)
* feat: introduce hashids permalink keys

* implement dashboard permalinks

* remove shorturl notice from UPDATING.md

* lint

* fix test

* introduce KeyValueResource

* make filterState optional

* fix test

* fix resource names
2022-03-24 21:53:09 +02:00
Beto Dealmeida
c7f9060a2f
feat: import external management columns (#19315)
* feat: import flags

* Add tests
2022-03-24 10:41:22 -07:00
Yongjie Zhao
375c03e084
feat(advanced analysis): support MultiIndex column in post processing stage (#19116) 2022-03-23 13:46:28 +08:00
Beto Dealmeida
b05e7dbf2a
feat: API for asset sync (#19220)
* feat: API for asset sync

* Add unit tests.

* Improve tests

* Move files

* Add more tests
2022-03-22 08:29:24 -07:00
John Bodley
82a6811e7e
fix(presto/trino): Add TIME/TIMESTAMP WITH TIME ZONE (#19263)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2022-03-22 06:33:38 +13:00
Elizabeth Thompson
88029e21b6
fix dataset update table (#19269) 2022-03-21 09:43:51 -07:00
Ville Brofeldt
f341025d80
feat: add support for comments in adhoc clauses (#19248)
* feat: add support for comments in adhoc clauses

* sanitize remaining freeform clauses

* sanitize adhoc having in frontend

* address review comment
2022-03-18 15:08:06 -07:00
Lily Kuang
50902d51f5
fix: allow subquery in ad-hoc SQL (WIP) (#19242)
* allow adhoc subquery

* add config for allow ad hoc subquery

* default to true allow adhoc subquery

* fix test

* Update superset/errors.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset/connectors/sqla/utils.py

Co-authored-by: David Aaron Suddjian <1858430+suddjian@users.noreply.github.com>

* rename and add doc string

* fix for big query test

* Update superset/connectors/sqla/utils.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Apply suggestions from code review

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* add test

* update validate adhoc subquery

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
Co-authored-by: David Aaron Suddjian <1858430+suddjian@users.noreply.github.com>
2022-03-18 10:04:55 -07:00
Beto Dealmeida
51061f0d67
feat: import/export assets commands (#19217)
* feat: import/export assets commands

* Add overwrite test

* Fix tests
2022-03-17 10:51:17 -07:00
Craig Rueda
4b34817b19
feat(ui): Adding manifest prefix config (#19141)
* Adding manifest prefix config

* Fixing broken tests

* Fixing import

* Adding prefix for remaining assets

* Changing static prefix strategy

* Fixing DST test

* Fixing up formatting

* Fixing up async_query_manager.py types
2022-03-17 08:58:48 -07:00
Ville Brofeldt
b7a0559aaf
feat: add permalink to dashboard and explore (#19078)
* rename key_value to temporary_cache

* add migration

* create new key_value package

* add commands

* lots of new stuff

* fix schema reference

* remove redundant filter state from bootstrap data

* add missing license headers

* fix pylint

* fix dashboard permalink access

* use valid json mocks for filter state tests

* fix temporary cache tests

* add anchors to dashboard state

* lint

* fix util test

* fix url shortlink button tests

* remove legacy shortner

* remove unused imports

* fix js tests

* fix test

* add native filter state to anchor link

* add UPDATING.md section

* address comments

* address comments

* lint

* fix test

* add utils tests + other test stubs

* add key_value integration tests

* add filter box state to permalink state

* fully support persisting url parameters

* lint, add redirects and a few integration tests

* fix test + clean up trailing comma

* fix anchor bug

* change value to LargeBinary to support persisting binary values

* fix urlParams type and simplify urlencode

* lint

* add optional entry expiration

* fix incorrect chart id + add test
2022-03-17 01:15:52 +02:00
Beto Dealmeida
8234395466
feat: helper functions for RLS (#19055)
* feat: helper functions for RLS

* Add function to inject RLS

* Add UNION tests

* Add tests for schema

* Add more tests; cleanup

* has_table_query via tree traversal

* Wrap existing predicate in parenthesis

* Clean up logic

* Improve table matching
2022-03-11 14:47:11 -08:00
Ville Brofeldt
999c2c6826
fix(dashboard): import handle missing excluded charts (#19088) 2022-03-10 12:59:29 +02:00
John Bodley
26486d01c1
chore: Remove legacy SIP-15 interim logic/flags (#18936)
* chore: Remove legacy SIP-15 logic

* Update ab9a9d86e695_deprecate_time_range_endpoints.py

* Update UPDATING.md

* Update UPDATING.md

* Update UPDATING.md

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2022-03-04 09:15:36 +13:00
Beto Dealmeida
00c99c91ec
feat: new dataset/table/column models (#17543)
* feat: new models for SIP-68

* feat: new dataset models DB migration

* Add shadow write (WIP)

* Physical dataset shadow write (WIP)

* Virtual dataset shadow write (WIP)

* Update migration to populate models

* Cascade delete columns

* after_delete

* Update hook

* Add metric test

* Do not rename tables

* Small fixes

* Fix migration

* Fix tests

* Fix more tests

* Even more tests

* Addressing comments (WIP)

* Fix migration

* Rebase and update head

* Fix unit tests

* Add external management cols

* Small fixes
2022-02-24 11:02:01 -08:00
Corbin Robb
0ae1ca7e02
chore(tests-backend): migrate snowflake and sqlite to unit tests (#18854)
* migrate snowflake and sqlite

* change structure to better match the other unit tests

Co-authored-by: Corbin Robb <corbin@Corbins-MacBook-Pro.local>
2022-02-23 11:11:10 -06:00
Sujith Kumar S
7e51b200b4
fix(mssql): support top syntax for limiting queries (#18746)
* SQL-TOP Fix For Database Engines

MSSQL is not supporting LIMIT syntax in SQLs. For limiting the rows, MSSQL having a different keyword TOP. Added fixes for handling the TOP and LIMIT clauses based on the database engines.

* Teradata code for top clause handling removed from teradata.py

Teradata code for top clause handling removed from teradata.py file, since we added generic section in base engine for the same.

* Changes to handle CTE along with TOP in complex SQLs

Added changes to handle TOP command in CTEs, for DB Engines which are not supporting inline CTEs.

* Test cases for TOP unit testing in MSSQL

Added multiple unit test cases for MSSQL top command handling and also along with CTEs

* Corrected the select_keywords name key in basengine

Corrected the select_keywords name key in basengine

* Changes based on as per review.

made the required corrections based on code review to keep good code readability and code cleanliness.

* Review changes to correct lint and typo issues

Made the changes according to the review comments.

* fix linting errors

* fix teradata tests

* add coverage

* lint

* Code cleanliness

Moved the top/limit flag check from sql_lab to core.

* Changed for code cleanliness

Changes for keeping code cleanliness

* Corrected lint issue

Corrected lint issue.

* Code cleanliness

Code cleanliness

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2022-02-21 09:58:39 +02:00
Yongjie Zhao
987740aa8d
fix: contribution operator meets nan value (#18782) 2022-02-18 23:28:28 +08:00
Yongjie Zhao
38cd696981
refactor: move date_parser to unittest (#18810) 2022-02-18 20:15:31 +08:00
Yongjie Zhao
30a9d14639
refactor: postprocessing move to unit test (#18779) 2022-02-17 20:05:41 +08:00
Sujith Kumar S
b8aef10098
fix(mssql): support cte in virtual tables (#18567)
* Fix for handling regular CTE queries with MSSQL,#8074

* Moved the get_cte_query function from mssql.py to base.py for using irrespetcive of dbengine

* Fix for handling regular CTE queries with MSSQL,#8074

* Moved the get_cte_query function from mssql.py to base.py for using irrespetcive of dbengine

* Unit test added for the db engine CTE SQL parsing.

Unit test added for the db engine CTE SQL parsing.  Removed additional spaces from the CTE parsing SQL generation.

* implement in sqla model

* lint + cleanup

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2022-02-10 10:28:05 +02:00
Ville Brofeldt
fdbcbb5c84
fix(csv-export): pivot v2 with verbose names (#18633)
* fix(csv-export): pivot v2 with verbose names

* refine logic + add test

* add missing verbose_map
2022-02-09 14:01:57 +02:00
Hugh A. Miles II
7194a01040
fix: user UUIDs on export for Native Filter Configuration (#18562)
* saving work for import

* update

* move logic to update ref

* remove export updates

* remove unneeded code

* cleanup

* Update superset/dashboards/commands/importers/v1/utils.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update tests/unit_tests/dashboards/commands/importers/v1/utils_test.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2022-02-08 13:10:40 -08:00
Ville Brofeldt
7ad38d5ba1
chore(tests): migrate mssql tests to pytest (#18251)
* move test

* migrate to pytest
2022-02-02 13:19:58 +02:00
Dave McNulla
8c376548e3
fix(teradata): LIMIT syntax (#18240)
Co-authored-by: Mccush, Jack <jack.mccush@teradata.com>
Co-authored-by: Jack McCush <33156805+mccushjack@users.noreply.github.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
Co-authored-by: David McNulla <david.mcnulla@teradata.com>
2022-01-31 19:15:42 -08:00
AAfghahi
fa11a97585
fix: remove standalone (#18157)
* removed standalone

* Update tests/integration_tests/reports/commands_tests.py

* changed standalone

* added tests

* made function more generic

* Update superset/reports/notifications/email.py

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2022-01-26 13:04:26 -05:00
Michael S. Molina
f018c826b8
refactor: Moves the Explore form_data endpoint (#18151)
* refactor: Moves the Explore form_data endpoint

* Removes unused imports

* Fixes openapi schema error

* Fixes typo

* Renames and UPDATING.md
2022-01-25 11:14:50 -03:00
Michael S. Molina
959b15eeca
feat: Adds a key-value endpoint to store charts form data (#17882)
* feat: Adds a key-value endpoint to store charts form data

* Fixes linting problems

* Removes the query_params from the endpoints

* Refactors the commands

* Removes unused imports

* Changes the parameters to use dataclass

* Adds more access tests

* Gets the first dataset while testing

* Adds unit tests for the check_access function

* Changes the can_access check

* Always check for dataset access
2022-01-20 16:27:57 -03:00
Erik Ritter
14b9298ef7
fix: revert #17654 to fix subselect table name parsing (#18017) 2022-01-12 21:28:23 -08:00
John Bodley
63ca09e345
fix: Workaround for sqlparse issue #652 (#17995)
* fix: Workaround for sqlparse issue #652

* Update superset/sql_parse.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Update sql_parse.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2022-01-12 14:03:56 +13:00
Mikhail Kumachev
d2d4f8eb44
feat: Add support for Azure Data Explorer (Kusto) db engine spec (#17898)
* Add two Kusto engine specs: KQL and SQL. Some minor changes in core code to support Kusto engine specs.

* Remove redundant imports and logging.

* docs: Kusto sqlalchemy docs

* fix: Fix mypy and linting errors

* fix: Handle Black vs Pylint checks

* fix: isort problem

* refactor: Merge kustosql and kustokql in the single kusto module

* test: Add tests for Kusto db spec

* feat: Schema override does not require in KQL anymore

* Removed redundant imports.

* Added ".show" queries to readonly query determination.

* Fixed some bugs.
Added tests for convert_dttm.

* Fixed major sqlalchemy-kusto version.

* Fixed by isort.

Co-authored-by: Eugene Bikkinin <xnegxneg@gmail.com>
Co-authored-by: k.tomak <k.tomak@dodopizza.com>
Co-authored-by: Eugene Bikkinin <e.bikkinin@dodopizza.com>
2022-01-10 14:42:20 +02:00
Beto Dealmeida
30c0459808
fix: extra column in metrics (#17784) 2021-12-22 10:39:29 -08:00
Beto Dealmeida
c49545aec1
fix: dataset extra import/export (#17740)
* fix: dataset extra import/export

* Update superset/datasets/commands/importers/v1/utils.py
2021-12-22 09:41:31 -08:00
Beto Dealmeida
37cc2c4d15
fix: column extra in import/export (#17738) 2021-12-14 18:33:52 -08:00
Beto Dealmeida
2633bcccc3
fix: import dashboard stale filter_scopes (#17741) 2021-12-14 07:23:55 -08:00
Beto Dealmeida
fceabf6bc5
fix: import dash with missing immune ID (#17732) 2021-12-13 16:20:55 -08:00
Beto Dealmeida
3873cdff4d
feat: convert dataframe tests (#17655) 2021-12-09 09:47:53 -08:00
Ville Brofeldt
3a42071e0f
chore(sql): clean up invalid filter clause exception types (#17702)
* chore(sql): clean up invalid filter clause exception types

* fix lint

* rename exception
2021-12-09 17:49:32 +02:00
Beto Dealmeida
b7ae7f976a
chore: convert feature flag tests to unit tests (#17689)
* chore: convert feature flag tests to unit tests

* Fix test
2021-12-08 14:51:47 -08:00
Beto Dealmeida
8c25f2f356
fix: extract tables doesn't work with reserved keywords (#17654) 2021-12-08 05:50:42 -08:00
Ville Brofeldt
b2ffa268cd
fix(sqla): make text clause escaping optional (#17641) 2021-12-03 12:35:26 +02:00
Elizabeth Thompson
13e19291f2
fix: allow for multiple columns in pivot table report (#17636)
* allow for multiple columns in pivot table report

* fix null data issue

* Update tests/unit_tests/charts/test_post_processing.py
2021-12-02 16:42:37 -08:00
ofekisr
683d1dab13
refactor(QueryObject): decouple from superset (#17479)
* refactor: queryObject - decouple from superset

* refactor: queryObject - decouple from superset
2021-11-18 19:53:56 +02:00
Elizabeth Thompson
f10bc6d8fe
fix: add fallback and validation for report and cron timezones (#17338)
* add fallback and validation for report and cron timezones

* add logging to exception catch

* Run black

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-11-12 12:28:17 -08:00
Christian Pfarr
333b1371f7
feat: Drill ODBC/JDBC Impersonation feature (#17353)
* Added Drill ODBC Impersonation feature and necessary translations/docs

* Code Cleanup

* add jdbc impersonation_target parameter

* add unittests for DrillEngineSpec.modify_url_for_impersonation method

* reformat test_drill.py with black formatter

* run pre-commit locally

Co-authored-by: Christian Pfarr <Christian.Pfarr@deutschebahn.com>
Co-authored-by: Christian Pfarr <z0ltrix+gitlab@pm.me>
2021-11-10 08:45:16 +02:00
Beto Dealmeida
03a2c6ee8a
feat: apply d3NumberFormat to table reports (#17336) 2021-11-04 08:18:01 -07:00
Geido
2199ef2dd7
chore: Test cases for annotations and annotation layers incorrect creation through API (#17246)
* Add/Refactor tests

* Add return type

* Update api tests
2021-11-03 09:11:41 +00:00
Beto Dealmeida
93bafa0e6a
feat: improve logic in is_select (#17329)
* feat: improve logic in is_select

* Add more edge cases
2021-11-02 17:30:12 -07:00
Beto Dealmeida
83a2f8346e
chore: enable BQ tests (#17017) 2021-10-14 08:07:47 -07:00
Beto Dealmeida
c993c5845f
fix(BigQuery): explicitly quote columns in select_star (#16822)
* fix (BigQuery): explicitly quote columns in select_star

* Fix test

* Fix SELECT * in BQ

* Add unit tests

* Remove type changes
2021-10-06 07:43:32 -07:00
AAfghahi
aa747219ad
feat: Add Private Google Sheets to dynamic form (#16628)
* first pass private gsheets

* made encrypted extra into string, refactored onParametersChanged

* private sheets working, credential_info errors

* all but test connection working

* first pass private gsheets

* made encrypted extra into string, refactored onParametersChanged

* private sheets working, credential_info errors

* all but test connection working

* Regenerate package-lock.json

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-09-28 17:08:50 -07:00
Ville Brofeldt
0f16177bde
fix(native-filters): ignore unset filter box time range (#16854) 2021-09-27 17:17:52 +03:00
Beto Dealmeida
48a61bacc5
fix: handle CTEs with comments on is_select (#16769) 2021-09-21 16:23:54 -07:00
John Bodley
fb4650a6eb
chore(pylint): Remove top-level disable (#16589)
* chore(pylint): Remove top-level disable

* Update examples.py

* Update command.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-09-15 09:30:23 -07:00
Beto Dealmeida
afb8bd5fe6
feat: improve embedded data table in text reports (#16335)
* feat: improve HTML table in text reports

* Remove unused import

* Update tests

* Fix test
2021-08-18 17:00:29 -07:00
Beto Dealmeida
ac8e54d909
fix: improve pivot post-processing (#16289)
* fix: improve pivot post-processing

* Add tests

* Trim space from column name
2021-08-17 14:41:22 -07:00
Ville Brofeldt
36abc51f90
fix(dashboard): unset empty time filter indicator (#16272) 2021-08-16 19:32:05 +03:00
Ville Brofeldt
d46dc9aa45
chore: upgrade mypy and add type guards (#16227) 2021-08-14 06:31:45 +03:00
Ville Brofeldt
b61c34f7c9
fix(viz): deduce metric name if empty (#16194)
* fix(viz): deduce metric name if empty

* fix unit test
2021-08-12 11:16:05 +03:00
Hugh A. Miles II
fd80ae34a3
fix: Make sheet_name into a ValidationInputError (#16056)
* setup validates for name

* add error type

* fix linting

* fix test

* remove errors

* fix number

* fix test
2021-08-10 13:07:31 -04:00
Beto Dealmeida
2d61f15153
feat: send post-processed data in reports (#15953)
* feat: send post-processed data in reports

* Fix tests and lint

* Use enums

* Limit Slack message to 4k chars
2021-07-30 09:37:16 -07:00
Beto Dealmeida
f4739f427e
feat: post-processing for pivot table v2 (#15879)
* feat: add pivot v2 post-processing

* Fix lint
2021-07-29 11:05:56 -07:00
Hugh A. Miles II
bfe7eb9a7b
feat(dbc ui): Adding Google Sheets Dynamic Form (#15801)
* feat: Make Google Sheets Dyanmic (#15576)

* first draft

* second draft

* added tests

* first draft

* added table_catalog

* remove console.log

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/index.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* revisions

* save this for now

* working form

* save disable on public sheets

* refactor somethings

* saving this for now

* working edit

* add back query to schema

* working add

* fix styling

* fixing x

* fix linting

* prettier

* fix some type issues

* more lint fixes

* remove unused dependency

* fix linint

* fix validation

* pylint bypass

* pylint bypass

* fix this

* fix mypy

* yerp

* fix test

* fix test

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/index.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* wrap add sheets

* fix linting issues

* fix unit test

* ignore typing

* fix editting and paste issues

* remove query

* fix this

* fix test

* add test back

* fix error messaging

* update url messaging on error

* change error type

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/index.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* add errors for sheets with no name

* fix

* fix messaging for gsheets

* stop pylint

* update line

Co-authored-by: AAfghahi <48933336+AAfghahi@users.noreply.github.com>
Co-authored-by: Arash <arash.afghahi@gmail.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-07-28 15:00:27 -04:00
Elizabeth Thompson
ea49aa3d2d
feat: add timezones to report cron (#15849)
* add timezones to report cron

* fix test
2021-07-27 18:28:24 -07:00
John Bodley
ab4e3b9bf9
chore: Enforce Mypy for non-tests (#15757)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-07-22 06:46:43 +12:00
Amit Miran
62a8f2e193
chore(python-testing): move memoized tests to unit tests (#15507)
* chore: move memoized test into a separated file

create integration test workflow

* chore: create unit test workflow to run purely pytest

* fix: bad reference

* fix: remove pip requirements bc there aren't any yet

* temp: install unit dependencies directly

* fix: --rootdir=

* fix: try to run only unit test

* chore: decouple memoized as separated module

* fix: bring back dependencies bc superset top-level module is coupled to flask and others so no reason no to do it

* fix: reference

* fix: pre-commit

* fix: pylint
2021-07-12 10:00:18 +03:00
Beto Dealmeida
4f5f9287fc
feat: validate_parameters for GSheets (#15578)
* feat: validate_parameters for GSheets

* Move import inside fixture

* Update deps

* Rename parameter
2021-07-08 08:26:39 -07:00
ofekisr
b5119b8dff
refactor(tests): decouple unittests from integration tests (#15473)
* refactor move all tests to be under integration_tests package

* refactor decouple unittests from integration tests - commands

* add unit_tests package

* fix celery_tests.py

* fix wrong FIXTURES_DIR value
2021-07-01 18:03:07 +03:00