Commit Graph

263 Commits

Author SHA1 Message Date
Timi Fasubaa d38315a307 reuse_regex_logic 2018-05-25 15:07:27 -07:00
Timi Fasubaa 1aced9b562 force limit only when there is no existing limit 2018-05-25 14:54:11 -07:00
John Bodley 3207116535
Revert "[get_df] Adding support for multi-statement SQL" (#5078) 2018-05-24 14:59:34 -07:00
John Bodley 17d6464aa9
[get_df] Adding support for multi-statement SQL (#5060) 2018-05-23 11:40:25 -07:00
Beto Dealmeida 973c661501 Rename "slice" to "chart" and update translations (#5008)
* Rename slice to chart and update translations

* Fix unit tests
2018-05-21 17:49:02 -07:00
Beto Dealmeida 459cb701fb Visualization for multiple line charts (#4819)
* Initial test

* Save

* Working version

* Use since/until from payload

* Option to prefix metric name

* Rename LineMultiLayer to MultiLineViz

* Add more styles

* Explicit nulls

* Add more x controls

* Refactor to reuse nvd3_vis

* Fix x ticks

* Fix spacing

* Fix for druid datasource

* Rename file

* Small fixes and cleanup

* Fix margins

* Add proper thumbnails

* Align yaxis1 and yaxis2 ticks

* Improve code

* Trigger tests

* Move file

* Small fixes plus example

* Fix unit test

* Remove SQL and Filter sections
2018-05-21 17:47:21 -07:00
Gabe Lyons 1c9474b4ff treating floats like doubles for druid versions lower than 11.0.0 (#5030) 2018-05-21 11:50:04 -07:00
Yongjie Zhao 9f66dae328 [bugfix] Fix ZeroDivisionError and get metrics label with percent metrics (#5026)
* Fix percent_metrics ZeroDivisionError and can not get metrics with label issue

* convert iterator to list

* get percentage metrics with get_metric_label method

* Adding tests case for expression type metrics

* Simplify expression
2018-05-20 11:10:57 -05:00
Maxime Beauchemin b839608c32
[sql lab] a better approach at limiting queries (#4947)
* [sql lab] a better approach at limiting queries

Currently there are two mechanisms that we use to enforce the row
limiting constraints, depending on the database engine:
1. use dbapi's `cursor.fetchmany()`
2. wrap the SQL into a limiting subquery

Method 1 isn't great as it can result in the database server storing
larger than required result sets in memory expecting another fetch
command while we know we don't need that.

Method 2 has a positive side of working with all database engines,
whether they use LIMIT, ROWNUM, TOP or whatever else since sqlalchemy
does the work as specified for the dialect. On the downside though
the query optimizer might not be able to optimize this as much as an
approach that doesn't use a subquery.

Since most modern DBs use the LIMIT syntax, this adds a regex approach
to modify the query and force a LIMIT clause without using a subquery
for the database that support this syntax and uses method 2 for all
others.

* Fixing build

* Fix lint

* Added more tests

* Fix tests
2018-05-14 14:44:05 -05:00
Ville Brofeldt b391676544 Force lowercase column names for Snowflake and Oracle (#4994)
* Force lowercase column names for Snowflake and Oracle

* Force lowercase column names for Snowflake and Oracle

* Remove lowercasing of DB2 columns

* Remove DB2 lowercasing

* Fix test cases
2018-05-14 13:43:13 -05:00
Gabe Lyons a8514b267b [Explore] Adding Adhoc Filters (#4909)
* adding custom expressions to adhoc metrics

* adjusted transitions and made the box expandable

* adding adhoc filters

* adjusted based on feedback
2018-05-10 10:41:10 -07:00
jasnovak e29beba023 Add extraction function support for Druid queries (#4740)
* add extraction fn support for Druid queries

* bump pydruid version to get extraction fn commits

* update and add tests for druid for filters with extraction fns

* conform to flake8 rules

* fix flake8 issues

* bump pyruid version for extraction function features
2018-05-08 22:00:06 -07:00
John Bodley 440fb77df5
[druid] Updating Druid refresh metadata tests (#4887) 2018-05-03 18:14:40 -07:00
Maxime Beauchemin 3f48c005df
[bugfix] temporal columns with expression fail (#4890)
* [bugfix] temporal columns with expression fail

error msg: "local variable 'literal' referenced before assignment"

Error occurs [only] when using temporal column defined as a SQL
expression.

Also noticed that examples were using `granularity` instead of using
`granularity_sqla` as they should. Fixed that here.

* Add tests
2018-04-26 21:13:52 -07:00
Maxime Beauchemin a98c3cfdef
Fix time granularity-related issues (#4821)
* Fixing time grain

* Add tests
2018-04-18 16:17:28 -07:00
Maxime Beauchemin eac97ce9f2
[explore] proper filtering of NULLs and '' (#4651)
* [WiP] [explore] proper filtering of NULLs and ''

TODO: handling of Druid equivalents

* Unit tests

* Some refactoring

* [druid] fix 'Unorderable types' when col has nuls

Error "unorderable types: str() < int()" occurs when grouping by a
numerical Druid colummn that contains null values.

* druid/pydruid returns strings in the datafram with NAs for nulls
* Superset has custom logic around get_fillna_for_col that fills in the
NULLs based on declared column type (FLOAT here), so now we have a mixed
bag of type in the series
* pandas chokes on pivot_table or groupby operations as it cannot sorts
mixed types

The approach here is to stringify and fillna('<NULL>') to get a
consistent series.

* typo

* Fix druid_func tests

* Addressing more comments

* last touches
2018-04-17 22:26:21 -07:00
John Bodley 2900ca345d
[travis] Fixing environments (#4828) 2018-04-15 16:21:33 -07:00
Gabe Lyons 8669874ec6 [Explore] Adding custom expressions to adhoc metrics (#4736)
* adding custom expressions to adhoc metrics

* adjusted transitions and made the box expandable
2018-04-13 11:20:53 -07:00
Hugh A. Miles II 2f5cff7d9f [DeckGL] Added fixtures and Deck test (#4798)
* added fixtures and deck test

* linting

* linting

* add os.path

* remove para

* fix reference
2018-04-12 17:20:39 -07:00
John Bodley dadc0574b8
[tests] cleaning up test configuration (#4806) 2018-04-11 14:09:20 -07:00
michellethomas 725b8f94c9 Adding tests for the time table viz (#4659) 2018-04-11 13:24:51 -07:00
John Bodley 7f1d7543d0
[logs] Dropping dt column (#4587) 2018-04-10 18:34:32 -07:00
John Bodley 1627fd096b
[travis/tox] Restructuring configuration (#4552) 2018-04-10 15:59:44 -07:00
Maxime Beauchemin 627bdb2eb3
[dashboard] open in edit mode when adding a chart (#4772)
* [dashboard] open in edit mode when adding a chart

* Move dashboard unit tests to their own file

* fix tests

* Better URL management
2018-04-07 12:55:05 -07:00
John Bodley df4ff05441
[druid] Excluding refreshing verbose name (#4761) 2018-04-04 17:16:53 -07:00
Maxime Beauchemin 3b7e0a951a
[sql lab] preserve schema through visualize flow (#4742)
* [sql lab] preserve schema through visualize flow

https://github.com/apache/incubator-superset/pull/4696 got tangled
into refactoring views out of views/core.py and onto views/sql_lab.py

This is the same PR without the refactoring.

* Fix lint
2018-04-04 13:38:37 -07:00
Maxime Beauchemin f6fe11f76e
[bugfix] convert metrics to numeric in dataframe (#4726)
* [bugfix] convert metrics to numeric in dataframe

It appears sometimes the dbapi driver and pandas's read_sql fail at
returning the proper numeric types for metrics and they show up as
`object` in the dataframe. This results in "No numeric types to
aggregate" errors when trying to perform aggregations or pivoting in
pandas.

This PR looks for metrics in dataframes that are typed as "object"
and uses pandas' to_numeric to convert.

* Fix tests

* Remove all iteritems
2018-04-02 21:48:14 -07:00
Jeffrey Wang 8be0bde683 [BugFix] Allowing limit ordering by post-aggregation metrics (#4646)
* Allowing limit ordering by post-aggregation metrics

* don't overwrite og dictionaries

* update tests

* python3 compat

* code review comments, add tests, implement it in groupby as well

* python 3 compat for unittest

* more self

* Throw exception when get aggregations is called with postaggs

* Treat adhoc metrics as another aggregation
2018-04-02 21:37:00 -07:00
Gabe Lyons 68dec24542 [Explore] Streamlined metric definitions for SQLA and Druid (#4663)
* adding streamlined metric editing

* addressing lint issues on new metrics control

* enabling druid
2018-03-28 17:41:29 -07:00
timifasubaa 8dd052de4b [security] Refactor security code into SupersetSecurityManager (#4565)
* move access permissions methods to security manager

* consolidate all security methods into SupersetSecurityManager

* update security method calls

* update calls from tests

* move get_or_create_main_db to utils

* raise if supersetsecuritymanager is not extended

* rename sm to security_manager
2018-03-27 16:46:02 -07:00
John Bodley f9d85bd2e1
[druid] Updating refresh logic (#4655) 2018-03-26 18:35:43 -07:00
Riccardo Magliocchetti 76394d3f8f forms: make csv import parse dates accepts a list of columns (#4639)
Instead of a boolean which has way less chances to work. While
at it add a proper label for the "con" field.

Fixes #4637
2018-03-23 14:16:02 -07:00
Jeffrey Wang 33aa976e3d Cache the query string (#4633)
* Cache the query string

* misc linter
2018-03-21 13:13:36 -07:00
michellethomas 5c98f5642b Fix sqllab numpy array (#4629)
* Fixing error with sqllab numpy array

* Adding tests for failing sqllab data type
2018-03-19 11:43:04 -07:00
Maxime Beauchemin 93ec76f757
[sql lab] reduce the number of metadata calls when loading a table (#4593) 2018-03-15 17:53:34 -07:00
Hugh A. Miles II 2bc089ef8d Added new exception class and start of better exception/error handling (#4514)
* rebase and linting

* change back

* wip

* fixed broken test

* fix flake8

* fix test
2018-03-11 22:07:51 -07:00
Grace Guo 9edbd64c5d
[Explore] Save custom url parameters when user save slices (#4578)
* [Explore] Save url parameters when user save slices

* remove print

(cherry picked from commit bd9ecbe)

* add unit test

(cherry picked from commit 0f350ad)

* wrapping all request params into url_params

(cherry picked from commit 17197c1)
2018-03-08 13:19:41 -08:00
John Bodley ef4e5ecedf [bugfix] Fixing regression from #4500 (#4549) 2018-03-06 21:19:13 -08:00
John Bodley 48430a1918
Merge pull request #4500 from john-bodley/john-bodley-fix-pr-4396
[bugfix] Fixing regression introduced in #4396
2018-03-05 16:51:09 -08:00
John Bodley 7440d34936 [payload] Fixing regression introducted in ##4396 2018-02-28 14:23:20 -08:00
Maxime Beauchemin 849a2cecee
Add https support for Druid (#4480)
* Add https support for Druid

* addressing comment
2018-02-28 10:12:17 -08:00
Maxime Beauchemin 83524f97d7 [WiP] Cleanup & fix URL scheme for the explore view (#4490)
* Improve URLs

* Fix js tests
2018-02-27 15:08:06 -08:00
John Bodley d57a37e341 [flake8] Adding flake8-coding (#4477) 2018-02-25 15:06:11 -08:00
John Bodley e112e4417c [flake8] Adding future-import check (#4476) 2018-02-23 16:42:09 -08:00
John Bodley 8aac63e74c [flake8] Fixing additional flake8 issue w/ the presence of ignore (#4474) 2018-02-23 14:46:26 -08:00
Ville Brofeldt d6c197f8ac Remove comments from queries in SQL Lab that break Explore view (#4413)
* Remove comments from queries in SQL Lab that break Explore view

This fixes an issue where comments on the last line of the source query comment out the closing parenthesis of the subquery.

* Add test case for SqlaTable with query with comment

This test ensures that comments in the query are removed when calling SqlaTable.get_from_clause().

* Add missing blank line class definition (PEP8)
2018-02-18 16:30:11 -08:00
timifasubaa fa0aa33767 Disable user access request (#4405)
* add feature flag to config

* wrap check around a feature flag

* add flag to the model view

* remove access request from seurity tests
2018-02-14 14:49:22 -08:00
Grace Guo 342180b263
[Explore view] Use POST method for charting requests (#3993)
* [Explore view] Use POST method for charting requests

* fix per code review comments

* more code review fixes

* code review fix: remove duplicated calls for getting values from request

* [Explore view] Use POST method for charting requests

* fix per code review comments

* more code review fixes

* code review fix: remove duplicated calls for getting values from request
2018-02-13 17:21:15 -08:00
michellethomas 19a0827d1f Adding dashboard add view (#4344)
* Adding DashboardAddView and setting show_columns on SLiceAddView

* Adding tests for sliceaddview read api

* Dedupe show_columns
2018-02-06 12:38:07 -08:00
timifasubaa 6d37d97ba5 Refactor import csv (#4298)
* move helpers to utils

* make form use queryselector

* refactor exception throwing and handling

* update db_connection access point

* nits
2018-02-03 20:22:06 -08:00