Commit Graph

415 Commits

Author SHA1 Message Date
Ville Brofeldt
72d1011023
[sqllab] Fix limit parsing bug when using limit-offset comma notation (#7912)
* Fix limit parsing bug when using limit-offset comma notation

* Use native sqlparse semantics to find limit

* black
2019-07-24 08:18:39 +03:00
Ville Brofeldt
07a76f83b1
[Bugfix] Remove prequery properties from query_obj (#7896)
* Create query_obj for every filter

* Deprecate is_prequery and prequeries from query_obj

* Fix tests

* Fix typos and remove redundant ; from sql

* Add typing to namedtuples and move all query str logic to one place

* Fix unit test
2019-07-23 22:13:58 +03:00
Maxim Sukharev
648f9fa54b Set owner to dashboards and charts on import (#7894)
* Allow to pass user for dashboard import cli

Dashboard import assign current user from flask context during import.
But in case of cli import there is no flask user and imported charts
don't have an owner which prevents ability to edit them.

* Reset ownership on dashboard import

For overriding existing charts it requires `owners` property to be set.

* Add tests for reset ownership

* Use ORM to decode dashboards json

Creating instances using ORM allows to normally work with relations

* Fix test_import_dashboard_1_slice test

Previously tests used side-effect of slices import which kept id from
json on insert into db.
2019-07-22 22:30:28 -07:00
Daniel Vaz Gaspar
b65ab51b63 [api] [database] New, migrate to new FAB API side by side (#7665)
* [api] [database] New, migrate to new FAB API side by side

* [database] view and api module reorganization

* [style] Fix, flake8

* [test] [database] Fix, tests for new database view structure

* [style] [database] black

* [database] Fix, missing apache license

* Migrate databaasync api to the new api

* flake8

* More efficient api call

* Revert querySearch to old API, test

* dummy commit

* revert config.py commit

* Remove unused import

* Remove old API view

* Remove new API view

* Add database filter

* Add database filter

* Add database filter

* Remove comments on js

* Fix JS test

* Fix pylint

* QuerySearch new databaseasync API endpoint

* Fix, query search new api endpoint

* Reintroduce old API for smooth side by side migration

* Better naming and use common hooks for view and API

* black it

* Fix, database mixin must come first

* Fix, lint
2019-07-22 21:25:16 -07:00
Arpit
ee3430cce5 [Bugfix] Response header and response mismatch on explore result from sqllab (#7907)
* fix(sql-lab explore):   fixed response data and header type to be aligned

* fix(sql-lab explore):   fixed test error caused by incorrect test at master

* fix(sql-lab explore):   reformatted

* fix(sql-lab explore):   removed unused import
2019-07-22 08:47:34 +03:00
Ville Brofeldt
4568b2a532 Add cache_key_wrapper to Jinja template processor (#7816) 2019-07-20 09:12:35 -07:00
John Bodley
ccedbea506
[fix] Adding space after -- for SQL comments (#7897) 2019-07-18 13:17:26 -07:00
John Bodley
2b3e7fe4de
[sqla] Adding check for invalid filter columns (#7888) 2019-07-18 08:51:41 -07:00
Maxim Sukharev
b512502d72 Remove unnecessary fields from dashboard exported json (#7879)
* Remove unnecessary fields from dashboard exported json

This commit makes export respect export_fields and doesn't export
unnecessary relations (like users with passwords hashes) which are
ignored during the import.

* Allow to import dashboard without position_json

In case charts were added from chart-edit page and wasn't re-ordered on
the dashboard position_json field is empty and import was failing with
error.

* Update export/import tests
2019-07-17 22:22:10 -07:00
Maxime Beauchemin
d65b039219
Improve examples & related tests (#7773)
* [WiP] improve load_examples

related to #7472, longer term we will generate the examples by exporting
them into tarball as in #7472. In the meantime, we need this subset of
the features:

* allowing specifying an alternate database connection for examples
* allowing a --only-metadata flag to `load_examples` to load only
  dashboard and chart definitions, no actual data is loaded

* Improve logging

* Rename data->examples

* Load only if not exist

* By default do not load, add a force flag

* fix build

* set published to true
2019-07-16 21:36:56 -07:00
Rob DiCiuccio
df051813d5 Adds the ability to replace/extend caching backend (#7856)
* Add ability to override cache backend with custom module

* Tests for setup_cache

* Add custom CACHE_CONFIG documentation

* Fix linter errors and documentation link

* Fix black formatting errors
2019-07-12 14:06:56 -07:00
Maxime Beauchemin
87f50ba0d8
Make SHOW_STACKTRACE config apply across the board (#7845) 2019-07-12 08:58:44 -07:00
Tresdon Jones
97ffb762d0 Add "Published" feature to dashboards (#4725)
* Allow users to publish dashboards

* Rework publish dashboards feature

- The eye next to the title has been replaced with a [draft] badge
- Published status is now toggled in the Header Action Dropdown
- CRUD list shows published status

* Fix linter errors

* Update javascript tests

* Add tests and change DashboardFilter

Add some tests to make sure the published status is rendered and
Make it so that users cannot see dashboards that are published
if they don't have access to any of the slices within

* Fix some linter errors

* Remove commas from core.py

* Fix some failing tests

* More linter errors I introduced

* Fix more linter errors I introduced

* update alembic migration

* Update design of publish dash feature

* Upgrade migration version

* Secure publish endpoint

* Remove bad quotes

* Give publish span its own style

* fix publish rendering

* Add new test for publish feature

* Update migration

* update slug in test

* Update migration

* Address reviwer comments

* Fix linter errors

* Add licenses

* Remove fetchPublished(), use bootstrap data

* Update migration

* Update croniter to existing version

* Fix linter errors

* Upgrade DB Revisions

* Fix flake8 linter error

* Set all dashboards to published on migration

* Migration proper line spacing

* Fix migration to work with postgres

* UPDATE statement works with postgresql and sqlite hopefully

* Update wording to kick off travis
2019-07-10 23:14:13 -07:00
Maxime Beauchemin
ab8af536b0
Fixing the examples (#7376)
Fixing a few failing examples, examples were referencing metrics that
were getting auto-generated in the past (like `sum__num`),
that we now prevent from getting created.
2019-07-04 13:23:02 -07:00
John Bodley
9dac805e7e
[schema] Deprecating the table_columns.database_expression column (#7653) 2019-07-03 14:05:59 -07:00
John Bodley
6d7701e684
[fix] Updating Pandas resample logic (#7799) 2019-07-02 11:10:50 -07:00
John Bodley
ffa7c617ad
[travis] Adding Black tox environment (#7809) 2019-07-01 18:05:15 -07:00
John Bodley
e0d040c377
[fix] Handling of non-existent datasource (#7755) 2019-07-01 11:55:25 -07:00
Kim Truong
5994e432d4 feat: query based on all partitions (#7782)
* feat: query based on all partitions

* fix: address Beto's comments
2019-06-27 11:19:55 -07:00
John Bodley
5c58fd1802
[format] Using Black (#7769) 2019-06-25 13:34:48 -07:00
Maxime Beauchemin
859d6e7c6a
chore: taking db dependencies out of requirements-dev.txt (#7605)
* chore: taking db dependencies out of requirements-dev.txt

The deps on mysqlclient and psycopg2

* Fix unit tests

* fix tox.ini

* fix tests
2019-06-23 22:37:41 -07:00
Ville Brofeldt
1c4092c61c Add csv upload support for BigQuery (#7756)
* Add extra_require for bigquery to setup.py

* Refactor df_to_db and add df upload capability for BigQuery

* Fix unit tests and clarify kwarg logic

* Fix flake8 errors

* Add minimum versions for bigquery dependencies

* wrap to_gbq in try-catch block and raise error if pandas-gbq is missing

* Fix linting error and make error more generic
2019-06-23 14:20:09 -07:00
michellethomas
7a575ce20a
Fixing time comparison to look for past deltas (#7616)
* Fixing time comparison to look for past deltas

* Adding note to updating about time_compare
2019-06-19 10:10:18 -07:00
Ville Brofeldt
95291facff
Break up db_engine_specs (#7676)
* Refactor db_engine_specs into package

* Rename bigquery class and add epoch funcs

* Fix flake8 errors

* Dynamically load all engine specs

* Fix linting errors and unit tests

* Implement Snowflake epoch time funcs

* Implement Teradata epoch time func

* Fix presto datasource query and remove unused import

* Fix broken datasource query

* Add mypy ignore for false positive

* Add missing license files

* Make create_time_grains_tuple public

* Fix flake8 quote

* Fix incorrect licence header
2019-06-08 11:27:13 -07:00
Erik Ritter
f7812a3f27 [SQL Lab] Clarify SQL Lab query and display limits (#7641) 2019-06-07 17:16:38 -07:00
John Bodley
45b41aadcc
[fix] SQL parsing of table names (#7490) 2019-06-03 11:07:57 -07:00
Jim Horng
78c1674dc7 make filter_values() macro to properly accept value from filter box with single value setting (#7494) 2019-06-03 09:54:07 -07:00
Maxime Beauchemin
45b9880516
fix: SqlaColumn.type overflow on mysql (#7606)
* fix: SqlaColumn.type overflow on mysql

Hitting a new error when loading examples around a datatype that exceeds
the 32 chars limit on SqlaColumn.type

Type includes the 'COLLATE utf8mb4_general_ci' suffix which is too
verbose and not needed in that context.

* fix tests
2019-06-03 09:52:25 -07:00
Maxime Beauchemin
4b5931f637
Alternative fix for #7559 (#7575)
* Alternative fix for #7559

Just an idea...

* logging
2019-06-01 09:21:35 -07:00
Erik Ritter
722043c672 Allow trailing spaces in simple filter values (#7617) 2019-05-31 17:06:41 -07:00
Kim Truong
9acafd5b75 fix: handle presto columns with whitespace (#7630) 2019-05-31 13:38:05 -07:00
Kim Truong
d2967340d9 View Presto row and array objects clearly in the data grid (#7625)
* feat: rough check in for Presto rows and arrays

* fix: presto arrays

* fix: return selected and expanded columns

* fix: add helper methods and unit tests

* fix: only allow exploration of selected columns

* fix: address Beto's comments and add more unit tests
2019-05-31 11:25:07 -07:00
Ville Brofeldt
c1712e5d10
Add relative start param for time filters (#7525)
* Add relative start param for time filters

* Fix typo and add types to parse_human_datetime

* Add relative start/end to query_object

* Fix linting error
2019-05-31 16:55:26 +03:00
Ville Brofeldt
34407e8962
Make timestamp expression native SQLAlchemy element (#7131)
* Add native sqla component for time expressions

* Add unit tests and remove old tests

* Remove redundant _grains_dict method

* Clarify time_grain logic

* Add docstrings and typing

* Fix flake8 errors

* Add missing typings

* Rename to TimestampExpression

* Remove redundant tests

* Fix broken reference to db.database_name due to refactor
2019-05-30 08:28:37 +03:00
Maxime Beauchemin
fd5befee38
Bump python libs, address insecure releases (#7550)
* Bump python libs, address insecure releases

Using
https://requires.io/github/mistercrunch/superset/requirements/?branch=apache_release_improv
to identify insecure, old releases we're using and bumping.

* redis <3.0

* fix new flakes
2019-05-28 21:03:49 -07:00
Ville Brofeldt
f7d3413a50
Add support for period character in table names (#7453)
* Move schema name handling in table names from frontend to backend

* Rename all_schema_names to get_all_schema_names

* Fix js errors

* Fix additional js linting errors

* Refactor datasource getters and fix linting errors

* Update js unit tests

* Add python unit test for get_table_names method

* Add python unit test for get_table_names method

* Fix js linting error
2019-05-26 06:13:16 +03:00
John Bodley
6b9790c8b1
[ad-hoc] Fixing type for count distinct (#7573) 2019-05-22 09:41:25 -07:00
Daniel Vaz Gaspar
74704f68c7 [security] New, deprecate merge_perm, FAB method is fixed (#7355)
* [security] New, deprecate merge_perm, FAB method is fixed

* [style] Fix, flakes

* [tests] Fix, change merge_perm to add_permission_view_menu

* [security] Fix, maintain merge_perm for compatibility

* [security] New, deprecation warning on merge_perm method

* [style] Fix, flake8 C812
2019-05-20 17:30:00 -07:00
Maxime Beauchemin
d8be0a7dd5
Break line before LIMIT statement to prevent trailing comment issue (#7485)
* Break line before LIMIT statement to prevent trailing comment issue

This may not be a perfect solution but it addresses the issue in 7483

closes https://github.com/apache/incubator-superset/issues/7483

* fix tests
2019-05-13 00:34:34 -05:00
Ville Brofeldt
959c35d506 bugfix: Improve support for special characters in schema and table names (#7297)
* Bugfix to SQL Lab to support tables and schemas with characters that require quoting

* Remove debugging prints

* Add uri encoding to secondary tables call

* Quote schema names for presto

* Quote selected_schema on Snowflake, MySQL and Hive

* Remove redundant parens

* Add python unit tests

* Add js unit test

* Fix flake8 linting error
2019-05-08 00:37:44 -05:00
Alex Berghage
24970485cf feat: Add validate_sql_json endpoint for checking that a given sql query is valid for the chosen database (#7422) (#7462)
merge from lyft-release-sp8 to master
2019-05-06 10:21:02 -07:00
Kim Truong
88e6ec992c feat: view presto row objects in data grid (#7445)
* Merge lastest from master into lyft-release-sp8 (#7405)

* filter out all nan series (#7313)

* improve not rich tooltip (#7345)

* Create issue_label_bot.yaml (#7341)

* fix: do not save colors without a color scheme (#7347)

* [wtforms] Strip leading/trailing whitespace (#7084)

* [schema] Updating the datasources schema (#5451)

* limit tables/views returned if schema is not provided (#7358)

* limit tables/views returned if schema is not provided

* fix typo

* improve code performance

* handle the case when table name or view name does not present a schema

* Add type anno (#7342)

* Updated local dev instructions to include missing step

* First pass at type annotations

* [schema] Updating the base column schema (#5452)

* Update 937d04c16b64_update_datasources.py (#7361)

* Feature flag for client cache (#7348)

* Feature flag for client cache

* Fix integration test

* Revert "Fix integration test"

This reverts commit 58434ab98a.

* Feature flag for client cache

* Fix integration tests

* Add feature flag to config.py

* Add another feature check

* Fix more integration tests

* Fix raw HTML in SliceAdder (#7338)

* remove backendSync.json (#7331)

* [bubbles] issue when using duplicated metrics (#7087)

* SUPERSET-7: Docker compose config version breaks on Ubuntu 16.04 (#7359)

* SUPERSET-8: Update text in docs copyright footer (#7360)

* SUPERSET-7: Docker compose config version breaks on Ubuntu 16.04

* SUPERSET-8: Extra text in docs copyright footer

* [schema] Adding commits and removing unnecessary foreign-key definitions (#7371)

*  Store last selected dashboard in sessionStorage (#7181)

* Store last selected dashboard in sessionStorage

* Fix tests

* [schema] Updating the base metric schema (#5453)

* Fix NoneType bug & fill the test recipients with original recipients if empty (#7365)

* feat: see Presto row and array data types (#7391)

* feat: see Presto row and array data types

* fix: address PR comments

* fix: lint and build issues

* fix: add types

* Incorporate feedback from initial PR (prematurely merged to lyft-release-sp8) (#7415)

* add stronger type hints where possible

* fix: lint issues and add select_star func in Hive

* add missing pkg init

* fix: build issues

* fix: pylint issues

* fix: use logging instead of print

* feat: view presto row objects in data grid

* fix: address feedback

* fix: spacing
2019-05-03 16:29:57 -07:00
John Bodley
fb627ba376
[fix] Fixing SQL parsing issue (#7374) 2019-05-01 22:07:01 -07:00
Dave Smith
a6aabf8268 feature: see Presto row and array data types (#7413)
* Merge lastest from master into lyft-release-sp8 (#7405)

* filter out all nan series (#7313)

* improve not rich tooltip (#7345)

* Create issue_label_bot.yaml (#7341)

* fix: do not save colors without a color scheme (#7347)

* [wtforms] Strip leading/trailing whitespace (#7084)

* [schema] Updating the datasources schema (#5451)

* limit tables/views returned if schema is not provided (#7358)

* limit tables/views returned if schema is not provided

* fix typo

* improve code performance

* handle the case when table name or view name does not present a schema

* Add type anno (#7342)

* Updated local dev instructions to include missing step

* First pass at type annotations

* [schema] Updating the base column schema (#5452)

* Update 937d04c16b64_update_datasources.py (#7361)

* Feature flag for client cache (#7348)

* Feature flag for client cache

* Fix integration test

* Revert "Fix integration test"

This reverts commit 58434ab98a.

* Feature flag for client cache

* Fix integration tests

* Add feature flag to config.py

* Add another feature check

* Fix more integration tests

* Fix raw HTML in SliceAdder (#7338)

* remove backendSync.json (#7331)

* [bubbles] issue when using duplicated metrics (#7087)

* SUPERSET-7: Docker compose config version breaks on Ubuntu 16.04 (#7359)

* SUPERSET-8: Update text in docs copyright footer (#7360)

* SUPERSET-7: Docker compose config version breaks on Ubuntu 16.04

* SUPERSET-8: Extra text in docs copyright footer

* [schema] Adding commits and removing unnecessary foreign-key definitions (#7371)

*  Store last selected dashboard in sessionStorage (#7181)

* Store last selected dashboard in sessionStorage

* Fix tests

* [schema] Updating the base metric schema (#5453)

* Fix NoneType bug & fill the test recipients with original recipients if empty (#7365)

* feat: see Presto row and array data types (#7391)

* feat: see Presto row and array data types

* fix: address PR comments

* fix: lint and build issues

* fix: add types

* add stronger type hints where possible

* fix: lint issues and add select_star func in Hive

* add missing pkg init

* fix: build issues

* fix: pylint issues

* fix: use logging instead of print
2019-05-01 09:17:34 -07:00
Daniel Vaz Gaspar
06c4610e8e Bump FAB to 2.0.0 (#7323)
* Bump FAB to 2.0.0

* [tests] whitelist SecurityApi login and refresh endpoints

* [style] Fix, C812 missing trailing commas

* [security] Remove SUPERSET_UPDATE_PERMS flag

Registering sources needs to be performed after the views are
initialized on UPDATE_PERMS=False configuration

* [docs] New, FAB_UPDATE_PERMS and flask fab cli

* [docs] Fix, db upgrade needs to come first, create-admin needs a db

* [cli] New, superset init bootstraps all permissions for FAB and Superset

* [style] Fix, flakes
2019-04-30 09:01:18 -07:00
John Bodley
f488a3281d
[schema] Updating the base metric schema (#5453) 2019-04-24 21:50:14 -07:00
John Bodley
1388452169
[schema] Updating the base column schema (#5452) 2019-04-23 18:06:21 -07:00
Maxime Beauchemin
36a219da7f
Allow Gamma and Alpha to access '/users/userinfo/' (#6936)
* Allow Gamma and Alpha to access '/users/userinfo/'

closes https://github.com/apache/incubator-superset/issues/4919

* Fix unit test

* Fix test
2019-04-22 11:23:54 -07:00
Maxime Beauchemin
5f28027ce7 Merging lyftga into master 2019-04-18 22:47:47 -07:00
michellethomas
51068f007e
Adding permission for can_only_access_owned_queries (#7234)
* Adding permission for can_only_access_owned_queries

* Fixing lint adding typing to variable

* Adding test for queryview and enabling /queryview/api/read

* Fixing issues with python typing
2019-04-17 16:11:11 -07:00