Commit Graph

361 Commits

Author SHA1 Message Date
Ville Brofeldt 23e823f0c1 Bump SQLAlchemy to 1.3 (#7099)
* Bump sqla to >=1.3.1

* Refine mssql column types to only use N-prefixing when necessary

* make join explicit

* replace set with list

* Add additional test case for N-prefix

* Replace engine with dialect and fix linting error

* Remove unneeded import
2019-03-24 20:44:05 -07:00
Maxime Beauchemin b210742ad2
Fix filter_box migration PR #6523 (#7066)
* Fix filter_box migration PR #6523

* Fix druid-related bug
2019-03-20 21:41:33 -07:00
John Bodley 61add606ca
[missing values] Removing replacing missing values (#4905) 2019-03-20 17:14:15 -07:00
John Bodley 07c340cf82
[sqlparse] Fixing table name extraction for ill-defined query (#7029) 2019-03-15 09:19:30 -07:00
John Bodley 3e076cb60b
[sql-parse] Fixing LIMIT exceptions (#6963) 2019-03-05 09:36:08 -08:00
Maxime Beauchemin ed79134e7f
auto-set 'Is Temporal' to true where column_name == '__time' (#6857)
* auto-set 'Is Temporal' to true where column_name == '__time'

* Adress comments

* lint

* clarifying name and comments
2019-03-04 20:51:11 -08:00
Maxime Beauchemin 3ae02d1a54
Allow for dynamic feature flags (#6808)
* Allow for dynamic feature flags

Giving more control over feature flags, allowing administrator to define
custom logic around whether features are enabled for particular users /
roles.

The exposed function can be used for things like:
* progressive rollout of features (1%, 5%, 50%, 100%)
* experimentation
* role-based feature affectation (only admins see a particular feature)

* fix build

* Addressing comments

* Addressing @hughhh's comments
2019-02-27 15:11:38 -08:00
Ville Brofeldt ea9d22b2ec Force quotes on non-expression time grains on Postgres (#6897)
* Force quotes on non-expression time grains on postgres

* Change to or semantics

* Return lower case column name as unmutated

* Add testcases for postgres time grains

* Make expression MixedCase
2019-02-21 13:16:28 -08:00
Ville Brofeldt f5277fe684 Refactor dataframe and column name mutation logic (#6847)
* Merge dataframe and column name mutation logic, add flag for disabling column aliases and add column name length checking

* Remove custome mutate_label from oracle spec

* Move hashing from mutate_label() to make_label_compatible()

* Remove empty line

* Make label mutating and truncating more robust

* Rename variables and make proposed changes from review

* Always execute labels_expected codepath

* Fix linting error

* Add comments and fix subquery errors

* Refine column compatibility

* Simplify label assignment

* Add unit tests for BQ and Oracle

* Linting
2019-02-20 23:05:35 -08:00
michellethomas ba9523c7c4 Adding dropdown to DatasourceControl and ability to change datasource (#6816)
* Adding dropdown to DatasourceControl and ability to change datasource

* Style fixes

* Adding unit tests for datasource/get endpoint

* Fixing issue with dropdown overflow and style changes

* Fixing issues rebasing metadata button and fixing sort for datasource with no name
2019-02-20 14:32:33 -08:00
John Bodley 60d5f89faa
[main] Disable resetting main DB attributes (#6845) 2019-02-13 16:41:16 -08:00
Raphaël Merx 6e9130550d Add .standalone class to <body> when in standalone mode (#6811)
* Add .standalone class to <body> when in standalone mode

* Add test for .standalone class on <body>
2019-02-04 16:44:17 -08:00
Maxime Beauchemin 5a40f71710
[sql lab] improve table name detection in free form SQL (#6793)
* [sql lab] improve table name detection in free form SQL

* flake

* Addressing comments
2019-02-04 16:03:23 -08:00
agrawaldevesh fc4042a28b SelectControl hackery: Fixing freeform select and allowing group by and non group-by column selectors to take in arbitrary expressions including select-all semantics. (#6722)
Summary: We want to allow grouping by on expressions, including the
ability to select expressions (without group-by). The UI ain't good yet
.. it is not at the feature parity of say adhoc filters/metrics that
come with a nice text-box to edit the SQL. But
this suffices for my usecase for now.

(The UI would have to be redone ... but that would require some refactoring
to merge the AdhocFilter/Metric/Popover stuff such that it can be used
for this use case too.)

Also fixed a bug in the SelectControl freeForm selection.

Allow selecting all columns easily: Added a new Select-All special item
that users can select to make all the options in the selectable show
up.

The ability to group by arbitrary expressions is useful because now two
users don't need to create custom computed-fields for this.
2019-02-04 12:34:24 -08:00
Christine Chambers 19b3753d2c Move feature flag utility function into superset/__init__.py 2019-02-01 13:35:50 -08:00
Christine Chambers b70a9ae524 Merge default feature flags and user defined feature flags
- Rename the default feature flags key in `config.py` to DEFAULT_FEATURE_FLAGS
- Merge default feature flags with user defined ones allowing the latter to overwrite the former
- Expose feature_flags for both server and client to use
- Add a utility method for checking whether a feature flag is on on server side
2019-01-30 22:23:27 -08:00
Maxime Beauchemin 1ffee8b236
[cosmetic] remove 'List' prefix from list headers (#6725)
* [cosmetic] remove 'List' prefix from list headers

* Same change to connectors
2019-01-23 21:06:56 -08:00
bolkedebruin 1fece0d2fa Add licenses to translations (#6732)
* Add licenses

* More licenses

* Ignore messages.json as they are generated

* More licenses

* Also typescript

* Fix alignment

* Add to svg

* Many more licenses

* more licenses

* Better excludes

* Add licenses to docs and md files

* Pre-finalize licenses

* Fix newlines

* Finalize all sourde licenses

* Fix lint
2019-01-22 08:21:13 -08:00
Maxime Beauchemin f742b9876b
Making thrift, pyhive and tableschema as extra_requires (#6696)
* Making thrift, pyhive and tableschema as extra_requires

Looking at the dependency tree for license related questions, I noticed
that tableschema had a huge tree, and only people running Hive really
need it. Making this as well as pyhive and thrift optional.

Also bumping some python dependencies

* Run pip-compile

* Removing refs to past.builtins (from future lib)

* Add thrift
2019-01-19 14:27:18 -08:00
Maxime Beauchemin 75322ab12c
More ASF licenses (css + html/jinja) (#6711)
* More ASF licenses

* html

* removing from vendors
2019-01-17 22:39:00 -08:00
Maxime Beauchemin 6718588781
Add universal "New" button (#6670)
* [cosmetic] CRUD's list from 'Add Filter' to 'Filter Results'

* Add global new button

* A better viz type selector

* lint

* Move view to new module, add unit test

* 'Filter Results' -> 'Filter'

* db merge

* Filter -> 'Filter List'
2019-01-17 15:48:40 -08:00
Beto Dealmeida 99d6dc07d6
Trim query before parsing (#6703)
* Ignore empty lines

* Fix unit tests

* Use stripped

* Fix another unit test
2019-01-17 09:16:35 -08:00
Maxime Beauchemin 88964b7dfc
Deprecate auto-generated metrics (#5461)
* [WiP] deprecate auto-generated metrics & fix examples

Picking up some leftover pieces not needed anymore since the
MetricsControl

* db merge

* db merge

* fix migration

* Creating metrics required for tests
2019-01-16 22:32:13 -08:00
Maxime Beauchemin e03e276571
Bump some of the requirements-dev.txt (#6700)
* Bump some of the requirements-dev.txt

* addressing comments
2019-01-16 20:40:16 -08:00
Maxime Beauchemin 1dd4d7a587
Apply ASF licenses throughout the code base (#5800)
* Add license headers

* reabased

* lint

* Removing licenses from vendors folder
2019-01-15 15:53:27 -08:00
Maxime Beauchemin da813b7ee3
[table editor] allow selecting physical table (#6046)
* [table editor] allow selecting physical table

* Using classes for padding
2019-01-15 08:53:24 -08:00
Chinh Nguyen 284a0cccd3 Add fix for pyodbc+mssql (#6621)
* add fix for odbc+mssql

* fix for pylint/pep8
2019-01-13 09:30:05 -08:00
Maxime Beauchemin fe77b57581
[refactor] moving some datasource-related code to the frontend (#5769)
* [refactor] moving some datasource-related code to the frontend

* fix js tests

* fix tests

* fix test
2018-12-22 18:06:36 -08:00
Maxime Beauchemin d427db0a8b [SQL Lab] Allow running multiple statements (#6112)
* Allow running multiple statements from SQL Lab

* fix tests

* More tests

* merge heads

* fix heads
2018-12-22 10:28:22 -08:00
Maxime Beauchemin 3f29a1dd70
Secure unsecured views and prevent regressions (#6553)
* Secure views and prevent regressions

* Force POST on shortner

* Fix tests
2018-12-18 11:57:13 -08:00
Ville Brofeldt 5bac723df4 Refactor teradata to new time_grain_functions spec (#6539)
* Refactor teradata to new time_grain_functions spec

* Add test for time_grain_functions
2018-12-16 08:53:29 -08:00
Mahendra M 808622414c [SIP-3] Scheduled email reports for Slices / Dashboards (#5294)
* [scheduled reports] Add support for scheduled reports

* Scheduled email reports for slice and dashboard visualization
  (attachment or inline)
* Scheduled email reports for slice data (CSV attachment on inline table)
* Each schedule has a list of recipients (all of them can receive a single mail,
  or separate mails)
* All outgoing mails can have a mandatory bcc - for audit purposes.
* Each dashboard/slice can have multiple schedules.

In addition, this PR also makes a few minor improvements to the celery
infrastructure.
* Create a common celery app
* Added more celery annotations for the tasks
* Introduced celery beat
* Update docs about concurrency / pools

* [scheduled reports] - Debug mode for scheduled emails

* [scheduled reports] - Ability to send test mails

* [scheduled reports] - Test email functionality - minor improvements

* [scheduled reports] - Rebase with master. Minor fixes

* [scheduled reports] - Add warning messages

* [scheduled reports] - flake8

* [scheduled reports] - fix rebase

* [scheduled reports] - fix rebase

* [scheduled reports] - fix flake8

* [scheduled reports] Rebase in prep for merge

* Fixed alembic tree after rebase
* Updated requirements to latest version of packages (and tested)
* Removed py2 stuff

* [scheduled reports] - fix flake8

* [scheduled reports] - address review comments

* [scheduled reports] - rebase with master
2018-12-10 22:29:29 -08:00
Maxime Beauchemin cc3a625a4b
Use py3's f-strings instead of s.format(**locals()) (#6448)
* Use py3's f-strings instead of s.format(**locals())

In light of the bug reported here
https://github.com/apache/incubator-superset/issues/6347, which seems
like an odd `.format()` issue in py3, I greped and replaced all
instances of `.format(**locals())` using py3's f-strings

* lint

* fix tests
2018-12-02 13:50:49 -08:00
Junda Yang f1cae2ecdd override get_view_names in PrestoEngineSpec (#6459)
* override get_view_names in PrestoEngineSpec

* add test

* flake 8

* flake 8
2018-11-28 15:13:38 -08:00
Jiaji Zhou 1eb0d0d835 Remove coordinator in Druid Cluster config (#5164) (#6443) 2018-11-27 09:41:16 -08:00
timifasubaa 91b758f3bc
Reduce data loaded before loading tests (#6298)
* reduce data loaded before loading tests

* make cypress only load needed tests
2018-11-19 15:27:25 -08:00
michellethomas d1a3ba85dd
Adding cypress tests for showing errors, time range filter, and verbose name (#6407)
* Adding cypress tests for showing erros, time range filter, and verbose name

* Adding tests for time range without ago

* Adding test to verify error message showing
2018-11-19 11:29:46 -08:00
John Bodley 0873abde12
[404] Aborting for views with invalid dashboard/slice IDs (#6355) 2018-11-12 10:08:20 -08:00
Christine Chambers 8c0551ed46 [SIP-5] Open a new /api/v1/query endpoint that takes query_obj (#6220)
* [SIP-5] Open a new /api/v1/query endpoint that takes query_obj

- Introduce a new handle_superset_exception decorator to avoid repeating the logic for catching SupersetExceptions
- Create a query_obj_backfill method that takes form_data and constructs a query_obj that will be constructed in the client in the future. Use the backfill in explore_json.
- Create a new /api/v1/query endpoint that takes query_obj only and returns the payload data. Note the query_obj is constructed in the client. The endpoint currently only handles query_obj for table view viz (we'll be adding support to new viz types as we go).
- Unit test to verify the new endpoint for table view

* fix tests and lint errors

* - Move the new query endpoint into its own api.py view.
- Create QueryObject and QueryContext class to encapsulate query_object to be built from the client and additional info (e.g. datasource) needed to get the data payload for a given query
- Remove the query_obj_backfill as we'll start building the first query_object on the client so it no longer makes sense to have a short-lived backfill for the matter of days.

* Fixing lint and test errors

* Fixing additional lint error from the previous rebase.

* fixing additional lint error

* addressing additional pr comments

* Make /query accept a list of queries in the query_context object.

* fixing a lint error

* - Move time_shift based calculation and since, until check into util
- Add typing info for get_since_until
- Add new unit tests to verify time_shift calculation and the since until check
2018-11-07 17:23:52 -08:00
Jeffrey Wang 0584e3629f Add separate limit setting for SqlLab (#4941)
* Add separate limit setting for SqlLab

Use separate param for wrap sql

Get query limit from config

unit tests for limit control rendering in sql editor

py unit test

pg tests

Add max rows limit

Remove concept of infinity, always require defined limits

consistency

Assert on validation errors instead of tooltip

fix unit tests

attempt persist state

pr comments and linting

* load configs in via common param

* default to 1k
2018-11-07 15:57:44 -08:00
Maxime Beauchemin 7b3095d6ff
Fix examples charts/dashboards and refactor (#5881)
* Fix examples charts/dashboards and refactor

* pylinting

* Fix pylint

* Lint the refactor

* Rebased
2018-10-31 15:29:04 -07:00
Junda Yang c552c125d7 Move metadata cache one layer up (#6153)
* Update wording

* nit update for api endpoint url

* move metadata cache one layer up

* refactor cache

* fix flake8 and DatabaseTablesAsync

* nit

* remove logging for cache

* only fetch for all tables that allows cross schema fetch

* default allow_multi_schema_metadata_fetch to False

* address comments

* remove unused defaultdict

* flake 8
2018-10-31 13:23:26 -07:00
Maxime Beauchemin bbfd69a138 [utils.py] gathering/refactoring into a "utils/" folder (#6095)
* [utils] gathering/refactoring into a "utils/" folder

Moving current utils.py into utils/core.py and moving other *util*
modules under this new "utils/" as well.

Following steps include eroding at "utils/core.py" and breaking it down
into smaller modules.

* Improve tests

* Make loading examples in scope for tests

* Remove test class attrs examples_loaded and requires_examples
2018-10-16 17:59:34 -07:00
timifasubaa 46c86672c8 remove utf8 declaration (#6096) 2018-10-15 11:53:24 -07:00
timifasubaa 047c8b0943
remove six dependency (#6091) 2018-10-13 09:38:46 -07:00
timifasubaa 5c5be825d9
remove future imports (#6050) 2018-10-11 09:46:25 -07:00
John Bodley 1ee08fc216
[select-star] Adding optional schema to view (#6051) 2018-10-08 10:32:40 -07:00
Grace Guo a9ef0aeaf5
Revert #5991 (#6035) 2018-10-04 13:43:01 -07:00
yamyamyuo ab5c910801 [bugfix] Fix Tableviz metrics column disorder (#5566)
* fix metrics disorder

* add tableviz metric order tests

* lint code

* use OrderedDict to avoid metrics disorder

* fix unit test
2018-10-04 08:53:26 -07:00
Arpit 73882945bf feat: import/export dashboards via cli (#5991)
* feat: import/export dashboards via cli

* style: fixed lint error

* test: added test for import and export util

* test: removing import test as it is causing integrity issues

Import is a wrapper around exist functionality so we can go ahead without a test or mock the actual db operation using https://docs.python.org/3/library/unittest.mock.html

And validate the wrapper operations only.

* test: remove test data file

* test: removed usage of reserved keyword id
2018-10-01 14:02:16 -07:00