Commit Graph

55 Commits

Author SHA1 Message Date
Antonio Rivero
6657353bca
feat(database): Database Filtering via custom configuration (#24580) 2023-07-06 16:48:46 -04:00
Hugh A. Miles II
226c7f807d
fix: SSH Tunnel creation with dynamic form (#24196) 2023-07-02 23:48:48 -04:00
John Bodley
75543af550
chore(dao): Add explicit ON DELETE CASCADE when deleting datasets (#24488) 2023-06-28 16:03:25 -07:00
Daniel Vaz Gaspar
0ddc0a6738
chore: remove marshmallow-enum dependency and bump FAB (#24499) 2023-06-27 08:37:43 +01:00
John Bodley
3e76736874
chore(dao): Organize DAOs according to SIP-92 (#24331)
Co-authored-by: JUST.in DO IT <justin.park@airbnb.com>
2023-06-18 18:32:32 -07:00
Daniel Vaz Gaspar
6f145dfe36
chore: remove deprecated apis on superset, testconn, validate_sql_json, schemas_access_for_file_upload, extra_table_metadata (#24354) 2023-06-13 11:03:19 +01:00
John Bodley
a4d5d7c6b9
chore(pre-commit): Add pyupgrade and pycln hooks (#24197) 2023-06-01 12:01:10 -07:00
Beto Dealmeida
ec6318b379
fix: permission checks on import (#23200) 2023-03-15 08:31:09 -07:00
Hugh A. Miles II
9920ab3fd9
chore: Migrate /superset/schemas_access_for_file_upload to v1 (#23227)
Co-authored-by: Diego Medina <diegomedina24@gmail.com>
2023-03-14 19:21:10 -06:00
Antonio Rivero
3484e8ea7b
feat(ssh_tunnel): Import/Export Databases with SSHTunnel credentials (#23099) 2023-02-24 09:36:21 -08:00
Josh Soref
dec6ac49f6
chore(backend): Spelling (#19677)
Signed-off-by: Josh Soref <2119212+jsoref@users.noreply.github.com>
Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
Co-authored-by: Evan Rusackas <evan@preset.io>
2023-02-17 10:48:24 -07:00
Diego Medina
02cd75be8d
chore: Migrate /superset/tables/* to API v1 (#22501) 2023-02-01 12:45:57 +00:00
Antonio Rivero Martinez
d6a4a5da79
feat(ssh_tunnel): Add feature flag to SSH Tunnel API (#22805) 2023-01-26 16:53:36 -08:00
Beto Dealmeida
6f0fed77cf
feat: export/import allow_dml flag (#22806) 2023-01-24 19:16:46 -08:00
John Bodley
92cdb8c282
chore: Add explicit bidirectional performant relationships for SQLA model (#22413) 2023-01-21 10:17:56 +13:00
Antonio Rivero Martinez
ad758c0802
feat(ssh_tunnel): Rename allow_ssh_tunneling and change the default value to False (#22723) 2023-01-17 19:11:41 -08:00
Antonio Rivero Martinez
539936522f
feat(ssh_tunnel): SQLAlchemy Form UI (#22513)
Co-authored-by: hughhhh <hughmil3s@gmail.com>
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
2023-01-11 19:29:52 +00:00
Hugh A. Miles II
ebaad10d6c
feat(ssh-tunnelling): Setup SSH Tunneling Commands for Database Connections (#21912)
Co-authored-by: Antonio Rivero Martinez <38889534+Antonio-RiveroMartnez@users.noreply.github.com>
Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
2023-01-03 17:22:42 -05:00
Beto Dealmeida
481b020e9d
chore: show database UUID in API (#22411) 2022-12-13 15:55:27 -08:00
Hugh A. Miles II
e23efefc46
feat: refactor all get_sqla_engine to use contextmanager in codebase (#21943) 2022-11-15 13:45:14 -05:00
Antonio Rivero Martinez
203b289021
feat(bigquery): Custom message when Service Account doesnt have the correct Roles and Permissions (#21838) 2022-10-26 19:44:09 -04:00
Antonio Rivero Martinez
b36bd3f083
fix(databases): GSheets and Clickhouse DBs are not allowed to upload files (#21065) 2022-09-26 10:08:11 -07:00
John Bodley
eac6fdcd29
chore(sqllab): Cleanup /tables/... endpoint (#21284) 2022-09-13 08:22:12 -07:00
Beto Dealmeida
34a79add04
feat: filter parameters from DB API (#21248) 2022-09-02 11:50:04 -07:00
Beto Dealmeida
8772e2cdb3
fix: improve get_db_engine_spec_for_backend (#21171)
* fix: improve get_db_engine_spec_for_backend

* Fix tests

* Fix docs

* fix lint

* fix fallback

* Fix engine validation

* Fix test
2022-08-29 11:42:42 -07:00
Diego Medina
15b9f83150
chore: support trailing slashes in database api (#21123) 2022-08-22 15:15:04 +08:00
Jesse Yang
4d192e6e4d
feat(report): capture dashboard reports in specific states (#20552) 2022-07-26 17:01:56 -07:00
John Bodley
f0ca158989
fix: Refactor ownership checks and ensure consistency (#20499)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2022-07-07 11:04:27 -07:00
cccs-RyanS
ddc01ea781
feat(business-types): initial implementation of SIP-78 (#18794)
* add BUSINESS_TYPE_ADDONS to config with example callback

* Removing uneeded whitespace

* [Work in progress] Modifying cidr function to allow for  single ip and adding port outline

* Added test REST endpoint, added some more ports

I've thrown in a test.py script as well that will try to connect to the
business_type endpoint.

* Moving code from config.py into the business api

Very simple api is exposed that will allow someone to call a checkport
endpoint and get back a response.

* Removing commented out bits.

* Adding fucntion dict back to the config

* Moving business_type endpoint to charts

* Adding schema for get endpoint

* Removing imports, updating docstring, fixing typo

Just some small changes as described in the title.  I've updated the
test.py as well so it functions with the endpoint changes.

* Adding translation dict

* Fixing ops

* Adding check for list

* Modifying changes to add quotes where needed

Also changed BusinessTypeResponse to resp.

* Adding in some code to call the filter config

If a column starts with "cidr_" it will call the code in config.py to
try to translate the filter.  Nothing is changed in the JSON being
executed, just some information is dumped to console.

* Porting Ryan's changes

* Adding migration script (as per Ryan's PR)

* Fixing typo

* Prettier fixes

* [CLDN-1043] Adding rough version of filter changes for business types

* fix down migration

* Fixing bugs after merge

* adding functionality to appy filters in back end

* Fixing linting issues

* fix down revision

* Changing conversion callback to handle multiple values at once

* Adding string representation of values

* Code cleanup plus fixing debouce to only be called once for each entry

* Removing non needed logginh

* Changing operator list to use sting values

* Using text value operators

* Removing clear operator call

* Moving business type endpoints

* fix down revision

* Adding port functions

* update migration

* fix bad rebase and add ff

* implement validator

* dont add invalid values to response

* [CLDN-1205] Added a new exception type for a business type translation error. Added the error message in the display_value field within the business type response. Modified the IP and Port business types to populate the error message field in the response if an error occurs

* [CLDN-1205] Added meaningful error message for port translation errors

* Removing status field from businesstype Response and adding in error message

* [CLDN-1205] Added check to make sure the port business type is within the valid range of ports, if it is not, it will populate the error message

* [CLDN-1205] Fixed the if statement that checks to see if the string_value is in the valid range of port numbers. It did not corrently verify this before now.

* [CLDN-1205] Fixed an error where it was trying to use string_value in <= statements. I just casted string_value to an integer if it is numeric, which allows <= operators to be used on it

* [CLDN-1207] Added unit tests for the cidr_func and port_translation_func functions which are located in /superset/config.py

* [CLDN-1207] removed the assertRaises line as it does not work with the cidr_func and port_translation_func functions

* [CLDN-1207] Added the skeleton of the test_cidr_translate_filter_func unit test, still need to update what the expected response from the function will be.

* [CLDN-1207] Added the remainder of the back-end unit tests for the business types

* [CLDN-1207] Fixed the syntax error which caused the test_cidr_translate_filter_func_NOT_IN_double unit test to fail

* [CLDN-1207] Removed the logging that was added for debugging purposes

* [CLDN-1207] Formatted the commands_tests.py file to make it nicer to look at/read through

* [CLDN-1207] Fixed the code so that it conformed to the pylint requirements (i.e., pylint no longer complains about the code in commands_tests.py)

* [CLDN-1207] Modified some of the docstrings so they made better use of the 100 character per line, line limit

* [CLDN-1207] Added the beginnings of the unit tests for the
business types API

* [CLDN-1207] Added a comment to the top of the commands_tests.py file explaining how to run the unit tests. This prevents the next person who tries to run them from having to waste time trying the different forms of testing that Superset supports (e.g., pytest, tox, etc.)

* [CLDN-1207] Added a grammar fix to the comments describing how to run the unit tests

* [CLDN-1207] Modified the description of the business_type API endpoints as they did not represent what the API was actually doing

* [CLDN-1207] Added further instructions on how to run the unit tests that are within the business_type/api_tests.py file

* add request validation

* disable request if business type missing

* [CLDN-1207] Unit tests for the business type API are now working, however, they need to be modified to make use of @mock as we don't want to have to run the server to be able to run the unit tests

* Removing businesss types deffinitons from config

* Adding select to only show valid business types

* Fixed Enzyme tests

* Added scalfolding for selecting filter dropdown

* Adding intigration tests

* fix revision

* fix typos and unnecessary requests

* break out useBusinessTypes

* Added front-end RTL unit tests for the business type API endpoint

* Fixed error from unit tests

* Added a unit test to ensure the operator list is updated after a business type API response is received

* Removing elect compoenet for business types

* Adding feature flag and allowing saving when no business type present

* fixing useEffect hooks

* Adding feature flag to model

* Changing behavior such that an empty string returns a default response

* add form validation

* Modified comments in unit test as command to run test has changed

* Modified comments in unit test as filename to run test has changed

* Modified the api_tests.py file to conform to the linting requirements

* Changed the name of one of the tests to reflect what the test is actually testing

* Added cypress back to the package.json

* Added informative comments

* Updated comments in files as well as removed imports which were not being used

* Changes made by npm run prettier

* Fixed spelling mistakes

* Updated models.py to remove placeholder comments used in development

* Added feature flag mocking in unit test

* Fixing open api failure

* Fixing business types to pass unit tests

* Reverting unsafe connections back to false

* Removing print statement

* Adding business tpye to export test

* setting default feature flag to false for business type

* Reverting pre commit

* Reverting pre commit and running pre commit

* Reverting pre commit and running pre commit

* Fixing formatting

* Adding license

* Fixing Linting

* Protecting api enpoints

* updating model

* Fixing code path when business type exists

* Linting

* Linting

* Fixing linting

* Fixing spelling

* Fixing schemas

* Fixing app import

* fixing item render

* Added RTL test to make sure business type operator list is updated after API response

* Fixing linting

* fix migration

* Changing unit tests

* Fixing import and DB migration after rebase

* Renaming to advanced types

* Fixing Linting

* More renaming

* Removing uneeded change

* Fixing linting and test errors

* Removing unused imports

* linting

* Adding more detailed name for migration

* Moving files to plugins

* more renaming

* Fixing schema name

* Disabling feature flag that should not be enabled by default

* Adding extra cehck

* NameChange

* formatting

* Fixing equals check

* Moveing all advanced type classes and types to one file, and converting tests to functional

* Adding advanced type to test and fix linitng

Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
Co-authored-by: Dan Parent <daniel.parent@cse-cst.gc.ca>
Co-authored-by: GITHUB_USERNAME <EMAIL>
Co-authored-by: cccs-Dustin <96579982+cccs-Dustin@users.noreply.github.com>
2022-05-16 17:58:21 +03:00
Daniel Vaz Gaspar
87a4379d0a
feat: deprecate /superset/validate_sql_json migrate to api v1 (#19935)
* feat: deprecate /superset/validate_sql_json migrate to api v1

* use new error handling

* migrate SQLLAb frontend and add tests

* debug test

* debug test

* fix frontend test on sqllab

* fix tests

* fix frontend test on sqllab

* fix tests

* fix tests

* fix tests

* fix tests
2022-05-10 21:25:39 +01:00
Daniel Vaz Gaspar
4fdf230a56
feat: deprecate /superset/extra_table_metadata migrate to api v1 (#19921)
* feat: deprecate /superset/extra_table_metadata migrate to api v1

* use can_read to table_extra_metadata

* troubleshoot sqlite

* fix test

* fix test

* fix test

* fix frontend test on sqllab
2022-05-04 14:55:52 +01:00
AAfghahi
57157c8b15
fix: remove expose (#19700)
* bumping shillelagh

* remove expose
2022-04-16 22:02:23 -07:00
AAfghahi
4bf4d58423
fix: update Permissions for right nav (#19051)
* draft pr

* finished styling

* add filter

* added testing

* added tests

* added permissions tests

* Empty-Commit

* new test

* Update superset-frontend/src/views/components/MenuRight.tsx

Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>

* revisions

* added to CRUD view

Co-authored-by: Elizabeth Thompson <eschutho@gmail.com>
2022-04-11 16:50:59 -04:00
Ville Brofeldt
a619cb4ea9
chore: upgrade black (#19410) 2022-03-29 20:03:09 +03:00
cccs-Dustin
02ef9ca4cd
feat(sqllab): Add a configuration option to disable data preview (#19104) 2022-03-17 15:22:57 +02:00
Beto Dealmeida
d01fdad1d8
feat: add export_related flag (#19215)
* feat: add export_related flag

* Fix lint
2022-03-16 16:03:06 -07:00
Elizabeth Thompson
c143b37128
chore: log multiple errors (#14064)
* log all errors from db create

* return unique set of errors

* sort set for exceptions list

* run black
2022-03-08 17:31:19 -08:00
ofekisr
e53a50a155
chore(supeset.utils.core): move all database utils to database utils module (#18058) 2022-01-16 08:32:50 +02:00
Beto Dealmeida
51090c3f1b
feat: shorter timeout on test_connection (#18001)
* feat: shorter timeout on test_connection

* pip-compile-multi --no-upgrade

* Fix for SQLite

* Return 408

* Add test
2022-01-12 13:01:34 -08:00
Bogdan
afd2e9f2ca
chore: update fixtures scope that are loading data into the analytical db (#17780)
* Update db setup fixture scope

* Load data into db only once

* Update fixture scopes for data loading

* cleanup imports

* try import

* Update scope

* fix cleanup

* Import all fixtures

* Separate data log and metadata creation

* Retain table description

* fix dtype

Co-authored-by: Bogdan Kyryliuk <bogdankyryliuk@dropbox.com>
2021-12-16 16:11:47 -08:00
Geido
46cdc77ae6
fix: Redirect on 401 (#17597)
* Redirect on 401

* Bump FAB

* Format

* Update Cypress save test

* Revert Cypress change

* Bump FAB 3.4.1rc2

* Update test

* Update return statement

* Update api test

* Update datasets api test

* Update datasets api 401s to 403s

* Add typeguard

* Use Promise.resolve

* Update callApiAndParseWithhTimeout test

* Disable parseResponse test

* Try catch

* Handle npm 8 issues
2021-12-08 13:14:30 -07:00
Hugh A. Miles II
2e29f36e78
fix: RBAC for export for dashboard viewers (#17527)
* set out export

* update test

* use default dataset

* update test

* these test work

* fix test

* update

* fix

* fix test

* make the test better
2021-11-25 20:10:41 -08:00
Kamil Gabryjelski
fa51b3234e
fix(dashboard): Return columns and verbose_map for groupby values of Pivot Table v2 [ID-7] (#17287)
* fix(dashboard): Return columns and verbose_map for groupby values of Pivot Table v2

* Refactor

* Fix test and lint

* Fix test

* Refactor

* Fix lint
2021-11-05 16:05:48 +01:00
Hugh A. Miles II
e5fe18660c
fix: Show sqllab state when deleting databases (#17331)
* saving

* fix ts

* update test

* update test

* log unit

* updated test
2021-11-04 20:17:08 -04:00
Beto Dealmeida
cb34a22684
fix: import should accept old keys (#17330)
* fix: import should accept old keys

* Fix lint

* Preserve V1 schema
2021-11-03 11:25:30 -07:00
Shiva Raisinghani
ef3afbde82
refactor: Repeated boilerplate code between upload to database forms (#16756)
* abstract boilerplate code into class and rename csv to file

* add db migration

* fix some stuff

* more renaming of csv to file

* rename in translations

* update down revision

* update down revision

* bump chart version

* switch to alter column name approach in db migration

* fix db migration for MySQL

* db migration conflict
2021-10-25 13:53:06 +03:00
AAfghahi
aa747219ad
feat: Add Private Google Sheets to dynamic form (#16628)
* first pass private gsheets

* made encrypted extra into string, refactored onParametersChanged

* private sheets working, credential_info errors

* all but test connection working

* first pass private gsheets

* made encrypted extra into string, refactored onParametersChanged

* private sheets working, credential_info errors

* all but test connection working

* Regenerate package-lock.json

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-09-28 17:08:50 -07:00
Beto Dealmeida
092ef5bdfc
fix: set importer as owner (#16656)
* fix: set importer as owner

* Fix tests
2021-09-15 12:27:02 -07:00
John Bodley
fb4650a6eb
chore(pylint): Remove top-level disable (#16589)
* chore(pylint): Remove top-level disable

* Update examples.py

* Update command.py

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2021-09-15 09:30:23 -07:00
Hugh A. Miles II
bfe7eb9a7b
feat(dbc ui): Adding Google Sheets Dynamic Form (#15801)
* feat: Make Google Sheets Dyanmic (#15576)

* first draft

* second draft

* added tests

* first draft

* added table_catalog

* remove console.log

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/index.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* revisions

* save this for now

* working form

* save disable on public sheets

* refactor somethings

* saving this for now

* working edit

* add back query to schema

* working add

* fix styling

* fixing x

* fix linting

* prettier

* fix some type issues

* more lint fixes

* remove unused dependency

* fix linint

* fix validation

* pylint bypass

* pylint bypass

* fix this

* fix mypy

* yerp

* fix test

* fix test

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/index.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/DatabaseConnectionForm.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* wrap add sheets

* fix linting issues

* fix unit test

* ignore typing

* fix editting and paste issues

* remove query

* fix this

* fix test

* add test back

* fix error messaging

* update url messaging on error

* change error type

* Update superset-frontend/src/views/CRUD/data/database/DatabaseModal/index.tsx

Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>

* add errors for sheets with no name

* fix

* fix messaging for gsheets

* stop pylint

* update line

Co-authored-by: AAfghahi <48933336+AAfghahi@users.noreply.github.com>
Co-authored-by: Arash <arash.afghahi@gmail.com>
Co-authored-by: Beto Dealmeida <roberto@dealmeida.net>
2021-07-28 15:00:27 -04:00