Commit Graph

39 Commits

Author SHA1 Message Date
Maxime Beauchemin f742b9876b
Making thrift, pyhive and tableschema as extra_requires (#6696)
* Making thrift, pyhive and tableschema as extra_requires

Looking at the dependency tree for license related questions, I noticed
that tableschema had a huge tree, and only people running Hive really
need it. Making this as well as pyhive and thrift optional.

Also bumping some python dependencies

* Run pip-compile

* Removing refs to past.builtins (from future lib)

* Add thrift
2019-01-19 14:27:18 -08:00
Maxime Beauchemin 1dd4d7a587
Apply ASF licenses throughout the code base (#5800)
* Add license headers

* reabased

* lint

* Removing licenses from vendors folder
2019-01-15 15:53:27 -08:00
Maxime Beauchemin d427db0a8b [SQL Lab] Allow running multiple statements (#6112)
* Allow running multiple statements from SQL Lab

* fix tests

* More tests

* merge heads

* fix heads
2018-12-22 10:28:22 -08:00
Maxime Beauchemin bbfd69a138 [utils.py] gathering/refactoring into a "utils/" folder (#6095)
* [utils] gathering/refactoring into a "utils/" folder

Moving current utils.py into utils/core.py and moving other *util*
modules under this new "utils/" as well.

Following steps include eroding at "utils/core.py" and breaking it down
into smaller modules.

* Improve tests

* Make loading examples in scope for tests

* Remove test class attrs examples_loaded and requires_examples
2018-10-16 17:59:34 -07:00
timifasubaa 46c86672c8 remove utf8 declaration (#6096) 2018-10-15 11:53:24 -07:00
timifasubaa 5c5be825d9
remove future imports (#6050) 2018-10-11 09:46:25 -07:00
Maxime Beauchemin b9e3159f7c
Fix regression around low row limit for CSV exports (#5866)
* Fix regression around low row limit for CSV exports

* fix tests

* Still trying to fix tests
2018-09-19 13:30:25 -07:00
michellethomas 657993d609 Moving get_main_db out of tests and into superset utils (#5864) 2018-09-11 22:50:14 -07:00
michellethomas d40ded0be8 Adding simple Cypress tests (#5693)
* Adding simple Cypress tests

* Changing visitChart into multiple commands

* Adding Cypress to tox
2018-09-06 14:55:48 -07:00
Christine Chambers ae3fb04036 Bug: fixing async syntax for python 3.7 (#5759)
* Bug: fixing async syntax for python 3.7

Rename async to async_ so superset installs for python 3.7.

* Addressing PR comments. Use kwargs instead of explicitly specifying async_ so downstream engines (e.g. PyHive) that supports async can choose to use the async_ in pythonwq3.7 and async in <=python3.6

* addressing additional pr comments
2018-08-28 17:40:45 -07:00
gbates101 90decbc5db Migrated to click (#5410)
* Migrate flask_script to the Flask built-in click.

Flask 0.11 is the built-in integration of the click command line interface.
Flask-Migrate support for the new Flask CLI based on Click after Release 2.0.0.

* Resolved merge conflicts.

* Fixed issue introduced from bad merge.

* Fixed flake8 errors, added build to excluded flake8 stuff.

* * Moved the FlaskGroup declaration to the driver script.
* Moved shell context definition to cli.py
* Switched shell context definition to use decorator.
* Moved create_app definition to cli.py
* Fixed InvocationError with a wrapped function

* Added extra newlines between functions

* Removed flask-script dependency.
2018-07-20 15:26:33 -07:00
Maxime Beauchemin 777d876a52
Improve database type inference (#4724)
* Improve database type inference

Python's DBAPI isn't super clear and homogeneous on the
cursor.description specification, and this PR attempts to improve
inferring the datatypes returned in the cursor.

This work started around Presto's TIMESTAMP type being mishandled as
string as the database driver (pyhive) returns it as a string. The work
here fixes this bug and does a better job at inferring MySQL and Presto types.
It also creates a new method in db_engine_specs allowing for other
databases engines to implement and become more precise on type-inference
as needed.

* Fixing tests

* Adressing comments

* Using infer_objects

* Removing faulty line

* Addressing PrestoSpec redundant method comment

* Fix rebase issue

* Fix tests
2018-06-27 21:35:12 -07:00
Timi Fasubaa a9d7fafd9f add tests 2018-05-30 12:50:27 -07:00
Timi Fasubaa d38315a307 reuse_regex_logic 2018-05-25 15:07:27 -07:00
Timi Fasubaa 1aced9b562 force limit only when there is no existing limit 2018-05-25 14:54:11 -07:00
Maxime Beauchemin b839608c32
[sql lab] a better approach at limiting queries (#4947)
* [sql lab] a better approach at limiting queries

Currently there are two mechanisms that we use to enforce the row
limiting constraints, depending on the database engine:
1. use dbapi's `cursor.fetchmany()`
2. wrap the SQL into a limiting subquery

Method 1 isn't great as it can result in the database server storing
larger than required result sets in memory expecting another fetch
command while we know we don't need that.

Method 2 has a positive side of working with all database engines,
whether they use LIMIT, ROWNUM, TOP or whatever else since sqlalchemy
does the work as specified for the dialect. On the downside though
the query optimizer might not be able to optimize this as much as an
approach that doesn't use a subquery.

Since most modern DBs use the LIMIT syntax, this adds a regex approach
to modify the query and force a LIMIT clause without using a subquery
for the database that support this syntax and uses method 2 for all
others.

* Fixing build

* Fix lint

* Added more tests

* Fix tests
2018-05-14 14:44:05 -05:00
timifasubaa 8dd052de4b [security] Refactor security code into SupersetSecurityManager (#4565)
* move access permissions methods to security manager

* consolidate all security methods into SupersetSecurityManager

* update security method calls

* update calls from tests

* move get_or_create_main_db to utils

* raise if supersetsecuritymanager is not extended

* rename sm to security_manager
2018-03-27 16:46:02 -07:00
John Bodley d57a37e341 [flake8] Adding flake8-coding (#4477) 2018-02-25 15:06:11 -08:00
John Bodley ac57780607 [flake8] Resolving Q??? errors (#3847) 2017-11-13 21:06:51 -08:00
John Bodley 1b4f128f55 [flake8] Resolving F5?? errors (#3846) 2017-11-12 11:09:22 -08:00
John Bodley ba96984048 [flake8] Resolving E3?? errors (#3814) 2017-11-10 17:52:34 -08:00
John Bodley d03b74f754 [flake8] Resolving F4?? errors (#3811) 2017-11-08 20:34:33 -08:00
John Bodley ec21d5af21 [flake8] Resolving E2?? errors (#3812) 2017-11-08 20:34:23 -08:00
John Bodley 17623f71d4 [flake8] Resolving C??? errors (#3787) 2017-11-07 21:32:45 -08:00
John Bodley e2bca47421 [flake8] Resolve I??? errors (#3797) 2017-11-07 20:23:40 -08:00
John Bodley 02cbad59de [flake8] Resolving F8?? errors (#3778) 2017-11-06 21:15:36 -08:00
Maxime Beauchemin 122891c29b [sql lab] allow users to save their queries (#2528)
* Allow users to save their queries

Fixing tests .

* Adding placeholder for Query Description

* initJQueryCSRF -> initJQueryAjaxCSRF
2017-04-04 20:15:19 -07:00
Maxime Beauchemin 1f8e48b374 [sqllab] assign types for visualize flow (#2458)
* [sqllab] assign types for visualize flow

Somehow when using the visualize flow, the types were not
assigned at all, creating some bugs downstream. This PR attempts to get
the information required based on what pandas is knows and the types in
the data itself.

* Fixing tests

* Fixing tests

* Fixing more tests

* Fixing the last py3 tests
2017-03-24 09:23:51 -07:00
Bogdan 0779da6d24 Keep column order in .csv (#2377) 2017-03-10 14:49:11 -08:00
Maxime Beauchemin 2969cc9993 Refactoring Druid & SQLa into a proper "Connector" interface (#2362)
* Formalizing the Connector interface

* Checkpoint

* Fixing views

* Fixing tests

* Adding migrtion

* Tests

* Final

* Addressing comments
2017-03-10 09:11:51 -08:00
Bogdan 9114d86ecd Add hive to superset + monkey patch the pyhive (#2134)
* Initial hive implementation

* Fix select star query for hive.

* Exclude generated code.

* Address code coverage and linting.

* Exclude generated code from coveralls.

* Fix lint errors

* Move TCLIService to it's own repo.

* Address comments

* Implement special postgres case,
2017-03-06 16:20:55 -08:00
Maxime Beauchemin 9a62d94630 [sqllab] bugfix visualizing a query with a semi-colon (#1869)
* [sqllab] bugfix visualizing a query with a semi-colon

* Fixing tests
2017-01-06 12:24:07 -08:00
Maxime Beauchemin bce02e3f51 [security] improving the security scheme (#1587)
* [security] improving the security scheme

* Addressing comments

* improving docs

* Creating security module to organize things

* Moving CLI to its own module

* perms

* Materializung perms

* progrss

* Addressing comments, linting
2016-11-17 11:58:33 -08:00
Maxime Beauchemin 15b67b2c6c [WiP] rename project from Caravel to Superset (#1576)
* Change in files

* Renamin files and folders

* cleaning up a single piece of lint

* Removing boat picture from docs

* add superset word mark

* Update rename note in docs

* Fixing images

* Pinning datatables

* Fixing issues with mapbox-gl

* Forgot to rename one file

* Linting

* v0.13.0

* adding pyyaml to dev-reqs
2016-11-09 23:08:22 -08:00
Maxime Beauchemin 63161b11c3 [sqllab] proper, quoted, select * on the server side (#1404)
* [sqllab] proper, quoted, select * on the server side

* fixing tests
2016-10-21 16:55:37 -07:00
Bogdan 9c83b900ae Bring DB in sync with the models.py (#1172)
* Bring DB in sync with the models.py

* Make sure that migrations run on multiple dbs
2016-09-28 13:55:06 -07:00
Bogdan df89bec712 Infer types. Smart defaults for the visualize window. Basic implementation. (#1134)
* Implement smart suggestions for the visualize flow.

* Address JS comments.

* Implement caravel dataframe wrapper.
2016-09-23 11:14:38 -07:00
Maxime Beauchemin e8088d5c9a More improvements to SQL Lab (#1104)
* Handling timeouts

* Fixing timer on non-utc server

* Allowing async with results

* [bugfix] database is not selected

* Making sure the session is up and running

* Cleaning up query results and query objects

* Picking a groupby and metric field on visualize flow

* Showing local time in query history

* Using pull-left pull-right instead of grid layout for table metdata

Long column name were looking weird and icons were wrapping oddly

* Linting

* Eliminating east buttons under the sql editor

* Sort database dropdown by name

* Linting

* Allowing non-SELECT statements to run

* Adding a db config

* Making sqla checkout check cross-db
2016-09-19 15:28:10 -07:00
Maxime Beauchemin fc1e63761c Adding celery_tests.py 2016-08-29 22:11:57 -07:00