Commit Graph

48 Commits

Author SHA1 Message Date
Bogdan 4e1fa95035
Make schema name for the CTA queries and limit configurable (#8867)
* Make schema name configurable

Fixing unit tests

Fix table quoting

Mypy

Split tests out for sqlite

Grant more permissions for mysql user

Postgres doesn't support if not exists

More logging

Commit for table creation

Priviliges for postgres

Update tests

Resolve comments

Lint

No limits for the CTA queries if configures

* CTA -> CTAS and dict -> {}

* Move database creation to the .travis file

* Black

* Move tweaks to travis db setup

* Remove left over version

* Address comments

* Quote table names in the CTAS queries

* Pass tmp_schema_name for the query execution

* Rebase alembic migration

* Switch to python3 mypy

* SQLLAB_CTA_SCHEMA_NAME_FUNC -> SQLLAB_CTAS_SCHEMA_NAME_FUNC

* Black
2020-03-03 19:52:20 +02:00
John Bodley 8a138fbd03
[tox] Allowing running of specific tests (#9097) 2020-02-06 16:45:37 -08:00
Daniel Vaz Gaspar 916d184076
[query] deprecate can_only_access_owned_queries (#9046) 2020-02-05 08:58:11 +00:00
Rob DiCiuccio 6537d5ed8c Replace pandas.DataFrame with PyArrow.Table for nullable int typing (#8733)
* Use PyArrow Table for query result serialization

* Cleanup dev comments

* Additional cleanup

* WIP: tests

* Remove explicit dtype logic from db_engine_specs

* Remove obsolete  column property

* SupersetTable column types

* Port SupersetDataFrame methods to SupersetTable

* Add test for nullable boolean columns

* Support datetime values with timezone offsets

* Black formatting

* Pylint

* More linting/formatting

* Resolve issue with timezones not appearing in results

* Types

* Enable running of tests in tests/db_engine_specs

* Resolve application context errors

* Refactor and add tests for pyodbc.Row conversion

* Appease isort, regardless of isort:skip

* Re-enable RESULTS_BACKEND_USE_MSGPACK default based on benchmarks

* Dataframe typing and nits

* Renames to reduce ambiguity
2020-01-03 11:55:39 -05:00
Craig Rueda e490414484 Flask App factory PR #1 (#8418)
* First cut at app factory

* Setting things back to master

* Working with new FLASK_APP

* Still need to refactor Celery

* CLI mostly working

* Working on unit tests

* Moving cli stuff around a bit

* Removing get in config

* Defaulting test config

* Adding flask-testing

* flask-testing casing

* resultsbackend property bug

* Fixing up cli

* Quick fix for KV api

* Working on save slice

* Fixed core_tests

* Fixed utils_tests

* Most tests working - still need to dig into remaining app_context issue in tests

* All tests passing locally - need to update code comments

* Fixing dashboard tests again

* Blacking

* Sorting imports

* linting

* removing envvar mangling

* blacking

* Fixing unit tests

* isorting

* licensing

* fixing mysql tests

* fixing cypress?

* fixing .flaskenv

* fixing test app_ctx

* fixing cypress

* moving manifest processor around

* moving results backend manager around

* Cleaning up __init__ a bit more

* Addressing PR comments

* Addressing PR comments

* Blacking

* Fixes for running celery worker

* Tuning isort

* Blacking
2019-11-20 15:47:06 +00:00
Beto Dealmeida 3b97ae3b9d
Set owner when creating SQL Lab view (#8597)
* Set owner when creating SQL Lab view

* Add unit test

* Small fix
2019-11-19 10:48:42 -08:00
Marcus 4cfd6b3ce1 [tests] Fix, removing deprecated function for future Python release compatibility. (#8353)
* removing deprecated function for future Python release compatibility.

* also removed assertNotEquals per @dpgaspar
2019-10-21 15:49:12 +01:00
John Bodley 9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
Maxime Beauchemin 68c4c3a0b9
Prevent 'main' database connection creation (#8038)
* prevent 'main' database connection creation

* fix tests

* removing get_main_database

* Kill get_main_database

* Point to examples tables
2019-09-08 10:18:09 -07:00
Daniel Vaz Gaspar 73cdad2375 [SQLLab] Fix, database api unlimited page size v2 (#8002)
* [database] Fix, Removes the limit for the page size, Bump FAB to 2.1.8

Old FAB API had no limits by default, this will keep this behaviour
but only for this endpoint

* [sqllab] Add test for database API

* [sqllab] Add test for database API

* [sqllab] Include page zero on request

* [sqllab] Fix, Black and requirements

* [sqllab] Make database API return unlimited results

* [sqllab] just a test

* [sqllab] Bump FAB to 2.1.9

* [sqllab] Remove unused import
2019-08-08 11:37:00 -07:00
Maxime Beauchemin d65b039219
Improve examples & related tests (#7773)
* [WiP] improve load_examples

related to #7472, longer term we will generate the examples by exporting
them into tarball as in #7472. In the meantime, we need this subset of
the features:

* allowing specifying an alternate database connection for examples
* allowing a --only-metadata flag to `load_examples` to load only
  dashboard and chart definitions, no actual data is loaded

* Improve logging

* Rename data->examples

* Load only if not exist

* By default do not load, add a force flag

* fix build

* set published to true
2019-07-16 21:36:56 -07:00
John Bodley 5c58fd1802
[format] Using Black (#7769) 2019-06-25 13:34:48 -07:00
John Bodley 1388452169
[schema] Updating the base column schema (#5452) 2019-04-23 18:06:21 -07:00
michellethomas 51068f007e
Adding permission for can_only_access_owned_queries (#7234)
* Adding permission for can_only_access_owned_queries

* Fixing lint adding typing to variable

* Adding test for queryview and enabling /queryview/api/read

* Fixing issues with python typing
2019-04-17 16:11:11 -07:00
Maxime Beauchemin 1dd4d7a587
Apply ASF licenses throughout the code base (#5800)
* Add license headers

* reabased

* lint

* Removing licenses from vendors folder
2019-01-15 15:53:27 -08:00
Maxime Beauchemin d427db0a8b [SQL Lab] Allow running multiple statements (#6112)
* Allow running multiple statements from SQL Lab

* fix tests

* More tests

* merge heads

* fix heads
2018-12-22 10:28:22 -08:00
Jeffrey Wang 0584e3629f Add separate limit setting for SqlLab (#4941)
* Add separate limit setting for SqlLab

Use separate param for wrap sql

Get query limit from config

unit tests for limit control rendering in sql editor

py unit test

pg tests

Add max rows limit

Remove concept of infinity, always require defined limits

consistency

Assert on validation errors instead of tooltip

fix unit tests

attempt persist state

pr comments and linting

* load configs in via common param

* default to 1k
2018-11-07 15:57:44 -08:00
Maxime Beauchemin bbfd69a138 [utils.py] gathering/refactoring into a "utils/" folder (#6095)
* [utils] gathering/refactoring into a "utils/" folder

Moving current utils.py into utils/core.py and moving other *util*
modules under this new "utils/" as well.

Following steps include eroding at "utils/core.py" and breaking it down
into smaller modules.

* Improve tests

* Make loading examples in scope for tests

* Remove test class attrs examples_loaded and requires_examples
2018-10-16 17:59:34 -07:00
timifasubaa 46c86672c8 remove utf8 declaration (#6096) 2018-10-15 11:53:24 -07:00
timifasubaa 5c5be825d9
remove future imports (#6050) 2018-10-11 09:46:25 -07:00
michellethomas 657993d609 Moving get_main_db out of tests and into superset utils (#5864) 2018-09-11 22:50:14 -07:00
michellethomas d40ded0be8 Adding simple Cypress tests (#5693)
* Adding simple Cypress tests

* Changing visitChart into multiple commands

* Adding Cypress to tox
2018-09-06 14:55:48 -07:00
Maxime Beauchemin 9331cf79b5
[sql lab] allow EXPlAIN queries (#5558)
* [sql lab] allow EXPlAIN queries

closes https://github.com/andialbrecht/sqlparse/issues/421

* typo
2018-08-03 15:33:33 -07:00
Maxime Beauchemin fe6846b8db
[sql lab] simplify the visualize flow (#5523)
* [sql lab] simplify the visualize flow

The "visualize flow" linking SQL Lab to the "explore view" has never
worked so great for people, here's a list of issues:

* it's not really clear to users that their query is wrapped as a
subquery, and the explore view runs queries on top of it

* lint + fix tests

* Addressing comments
2018-08-02 10:52:38 -07:00
John Bodley 7fcc2af68f
[sql] Correct SQL parameter formatting (#5178) 2018-07-21 12:01:26 -07:00
Maxime Beauchemin 777d876a52
Improve database type inference (#4724)
* Improve database type inference

Python's DBAPI isn't super clear and homogeneous on the
cursor.description specification, and this PR attempts to improve
inferring the datatypes returned in the cursor.

This work started around Presto's TIMESTAMP type being mishandled as
string as the database driver (pyhive) returns it as a string. The work
here fixes this bug and does a better job at inferring MySQL and Presto types.
It also creates a new method in db_engine_specs allowing for other
databases engines to implement and become more precise on type-inference
as needed.

* Fixing tests

* Adressing comments

* Using infer_objects

* Removing faulty line

* Addressing PrestoSpec redundant method comment

* Fix rebase issue

* Fix tests
2018-06-27 21:35:12 -07:00
Ville Brofeldt b391676544 Force lowercase column names for Snowflake and Oracle (#4994)
* Force lowercase column names for Snowflake and Oracle

* Force lowercase column names for Snowflake and Oracle

* Remove lowercasing of DB2 columns

* Remove DB2 lowercasing

* Fix test cases
2018-05-14 13:43:13 -05:00
Maxime Beauchemin 3b7e0a951a
[sql lab] preserve schema through visualize flow (#4742)
* [sql lab] preserve schema through visualize flow

https://github.com/apache/incubator-superset/pull/4696 got tangled
into refactoring views out of views/core.py and onto views/sql_lab.py

This is the same PR without the refactoring.

* Fix lint
2018-04-04 13:38:37 -07:00
timifasubaa 8dd052de4b [security] Refactor security code into SupersetSecurityManager (#4565)
* move access permissions methods to security manager

* consolidate all security methods into SupersetSecurityManager

* update security method calls

* update calls from tests

* move get_or_create_main_db to utils

* raise if supersetsecuritymanager is not extended

* rename sm to security_manager
2018-03-27 16:46:02 -07:00
michellethomas 5c98f5642b Fix sqllab numpy array (#4629)
* Fixing error with sqllab numpy array

* Adding tests for failing sqllab data type
2018-03-19 11:43:04 -07:00
John Bodley d57a37e341 [flake8] Adding flake8-coding (#4477) 2018-02-25 15:06:11 -08:00
Marcus Levine 4bc5fe5495 [BUGFIX]: Check datatype of results before converting to DataFrame (#4108)
* conditional check on datatype of results before converting to df

fix type checking

fix conditional checks

remove trailing whitespace and fix df_data fallback def

actually remove trailing whitespace

generalized type check to check all columns for dict

refactor dict col check

* move df conversion to helper and add unit test

add missing newlines

another missing newline

fix quotes

more quote fixes
2018-01-23 20:58:06 -08:00
John Bodley ac57780607 [flake8] Resolving Q??? errors (#3847) 2017-11-13 21:06:51 -08:00
John Bodley ec21d5af21 [flake8] Resolving E2?? errors (#3812) 2017-11-08 20:34:23 -08:00
John Bodley e2bca47421 [flake8] Resolve I??? errors (#3797) 2017-11-07 20:23:40 -08:00
Maxime Beauchemin b888802e05 [sqllab] improve Hive support (#3187)
* [sqllab] improve Hive support

* Fix "Transport not open" bug
* Getting progress bar to show
* Bump pyhive to 0.4.0
* Getting [Track Job] button to show

* Fix testzz
2017-07-27 14:00:19 -07:00
ShengyaoQian b0e2904c24 Updating permission when refreshing druid datasource (#2655)
* Updating permission when refreshing druid datasource

* Adding test

* Fix style

* Deletion view_menu after db, table, cluster, ds deletion

* Update table model

* Linting

* Override _delete instead of post_delete

* fix

* lint

* fix multi delete

* fix

* Refactoring

* Amending
2017-05-22 16:41:32 -07:00
Maxime Beauchemin 122891c29b [sql lab] allow users to save their queries (#2528)
* Allow users to save their queries

Fixing tests .

* Adding placeholder for Query Description

* initJQueryCSRF -> initJQueryAjaxCSRF
2017-04-04 20:15:19 -07:00
Maxime Beauchemin 2969cc9993 Refactoring Druid & SQLa into a proper "Connector" interface (#2362)
* Formalizing the Connector interface

* Checkpoint

* Fixing views

* Fixing tests

* Adding migrtion

* Tests

* Final

* Addressing comments
2017-03-10 09:11:51 -08:00
Maxime Beauchemin 2d866e3ffa [hotfix] fix the logging fix that broke the build (#1940)
* [hotfix] fix the logging fix that broke the build

* Fixing tests
2017-01-11 07:53:24 -08:00
Bogdan 92aa1a6124 Permissions refactoring, optimizations and unit testing. (#1798)
* Refactor and speed up superset init

* Add unit tests.

* Test fixes.

* More test updates.

* Fix read only perms

* Address comments.
2016-12-15 08:38:34 -05:00
vera-liu e1e20b8757 Sort searched queries by recency (#1735)
* Sort searched queries by recency

* Fixed sqllab tests

* Add one more tr to QueryTable spec for pagination

* Change desc to asc as we reverse queries in component
2016-12-14 10:14:54 -08:00
Bogdan 3597fdb7f8 Filter table list based on the user permissions. (#1769)
* Filter table list based on the user permissions.

* Fix tests
2016-12-06 02:18:16 -05:00
Maxime Beauchemin 324205f77a [sqllab] bugfix where a query has the same alias twice as output (#1734) 2016-12-01 19:53:23 -08:00
Maxime Beauchemin 7eef46e941 Adding links pointing to the new user profile page (#1704)
* Adding links pointing to the new user profile page

* Raising coverage

* Fixing tests
2016-12-01 15:21:18 -08:00
Maxime Beauchemin bce02e3f51 [security] improving the security scheme (#1587)
* [security] improving the security scheme

* Addressing comments

* improving docs

* Creating security module to organize things

* Moving CLI to its own module

* perms

* Materializung perms

* progrss

* Addressing comments, linting
2016-11-17 11:58:33 -08:00
Maxime Beauchemin 15b67b2c6c [WiP] rename project from Caravel to Superset (#1576)
* Change in files

* Renamin files and folders

* cleaning up a single piece of lint

* Removing boat picture from docs

* add superset word mark

* Update rename note in docs

* Fixing images

* Pinning datatables

* Fixing issues with mapbox-gl

* Forgot to rename one file

* Linting

* v0.13.0

* adding pyyaml to dev-reqs
2016-11-09 23:08:22 -08:00
vera-liu a13bed2db6 Moved sqllab tests from core_tests to sqllab_tests (#1502)
* Moved sqllab tests from core_tests to sqllab_tests

* Minor changes based on comments
2016-11-01 20:48:31 -07:00