Commit Graph

87 Commits

Author SHA1 Message Date
Ville Brofeldt 375797f649
feat(native-filters): add timegrain and column filter (#13484)
* feat(native-filters): add timegrain and column filter

* add fetch values predicate

* bump deps

* lint

* fix test

* add python test for legacy merge

* fix default value and isInitialized to not check strict equality

* Address comments

* add FilterValue type

* address review comments
2021-03-09 17:27:46 +02:00
Ville Brofeldt 70e12ed27d
chore: bump pyarrow and pandas (#12882)
* bump pyarrow and pandas

* remove df copy
2021-03-02 19:44:53 +02:00
Ville Brofeldt d8c32b8097
fix(chart-data-api): support numeric temporal columns (#13138) 2021-02-16 09:51:22 +02:00
simcha90 d6fc720f4c
feat(native-filters): Time native filter (#12992)
* Add Time Filter component

* Improve Time Filter component

* Fix import errors

* Display Time Filter

* Remove console logs

* Change Control Panel

* Remove unnecessary files

* Use time range override

* test: fix tests

* feat: re run pipeline

* fix: fix some case for Time filter

* fix: merge with master

* use original time range

* fix height

* add cross filter behavior

* apply filters on initialization

* add applied filter to overrides

* add unit tests for merge_extra_form_data

Co-authored-by: Agata Stawarz-Pastewska <agata.stawarz-pastewska@polidea.com>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2021-02-13 12:09:54 +02:00
Ville Brofeldt ac73991913
fix(viz): improve dtype inference logic (#12933) 2021-02-04 10:47:27 -08:00
Duy Nguyen Hoang 77da4ada99
fix: error while parsing invalid json form_data (#12586)
* Fix error while parsing invalid json form_data

* Refine error returned
2021-01-19 16:15:16 +00:00
Karol Kostrzewa 241f380e2e
test: World bank examples (#12161)
* add world bank data fixture

* fix fixture cleanup, add fixture to dashboard_tests

* apply world bank fixtures, fix tests

* fix fixture typo, dashboard ids

* fix export dashboard metadata

* fix test_export_dashboard_command_key_order

* fix export dash tests, not add row when no orphans

* debug timeout

* fixes after merge

* fix lint

* run pre-commit

* comment test for debug

* fix save.test.js

Co-authored-by: Karol Kostrzewa <karol.kostrzewa@polidea.com>
2021-01-13 14:20:05 -08:00
Yongjie Zhao 9b0e6d0cc0
refactor: from superset.utils.core break down date_parser (#12408) 2021-01-11 14:16:42 -08:00
Karol Kostrzewa 2ab490a101
test: birth names (#12226)
* add birth names fixture

* fix birth names related tests

* fix test_import_v0_dataset_cli_export columns order

* fix celery tests drop table

* fix mysql datetime type

* fix mysql typo in charts/api_tests

* refactor

* add licence

* fix use fixture for presto

* fix presto, hive query

* fix flaky metadata

* fix mysql bigint type

* fix run query

* fix hive datatype in metadata

* fix slice owner for cypress

* refactor num_boys num_girls

* fix is_dttm column

* debug logging

* fix query offset

* fix presto ds type in metadata

* fix presto ds type

* clean up debug logging
2021-01-11 05:57:55 -08:00
Yongjie Zhao 9eb911dca3
fix(explore): refine previous calendar range (#12308) 2021-01-06 10:09:13 -08:00
Yongjie Zhao 1f8347228e
fix(explore): timepicker show human readable string in pill (#12300)
* fix(explore): timepicker show human readable string in pill

* fix lint
2021-01-06 22:37:35 +08:00
David Aaron Suddjian 865beae3d8
feat(dashboard): Dashboard-Native Filters (#11814)
* wip: filter create modal

* add a feature flag

* automatic changes to package lock

* wip

* filter sidebar and basic state management

* move create button to the sidebar

* first step for edit filterconfig

* partially fix tests...

* edits to types and comments

* respect feature flag on the filter sidebar

* add filterconfig form

* get input state working

* feat: tree filter scopes UI

* fix: turn on flag

* sticky filter bar

* stop preferring default export

* feat: finish filter scoping

* fix: under toggle

* fix: title

* fix: add licence

* refactor: update TS

* fix: fix on reopen modal + validation

* new filter bar menu

* adding, but commenting out, bulk scoping action

* adding some placeholder buttons and styles therefor

* feat: add filter chart

* add relative path to package.json

* update modal

* a little input styling... just getting warmed up

* Revert "feat: add filter chart"

This reverts commit b1302d35b6.

* Revert "add relative path to package.json"

This reverts commit 26a7b40e18.

* https package lock idk

* feat: add filter chart

* add relative path to package.json

* flexboxes all the way down

* dynamically generate groupby and datasource in select control

* big wip

* fix target column name

* no importing nonexistent things

* styles and name editing

* Add hook for retrieval of all filter states

* start with a new filter when clicking add filter

* handle removed filters gracefully

* fix incorrect default filter configuration

* add fields to useAllFilterState

* add redux for filterconfigs

* add support for native_filters

* remove consoles

* improve filter removal

* unbreak infinite loop

* basic sidebar toggling working!

* collapsing and menu working more smoothly

* linting

* make dataset and column inputs work

* save filter values properly

* add dashboard event for filter updates

* guarded

* apply filters properly

* fix schema

* making New Filter button a link

* gridunits ftw

* centering modal

* tis not a button anymore! nixing type.

* plus and collapse buttons instead of "more" menu

* updating full size filter icons

* adding icons to filter collapsing/expanding

* turning off animation, but leaving class-based animation css

* fix linting error

* fix native filters for legacy charts

* updates test

* no individual apply buttons

* fix bugs with filter config modal

* remove redundant code

* switch to the filter with validation errors on submit

* separate form validation

* switch config button from add to edit

* update tests

* oops forgot to add the fancy new useChangeEffect hook

* comments and code reorganization

* rename native_filters to extr_form_data and move hook

* disable native filters in viz selector

* add cascading

* implement new extra form data api

* cleanup

* updates tests

* bump npm packages

* fix bad merge on package.json + lock

* lint

* replace in and not in with uppercase

* lint

* lint

* lint

* lint

* bulk test fix

* Sort select input alphabetically

* Change type for sorting elements

* fix rest of unit tests

* make filter operators all uppercase

* Hide Filter bar when there are no filters

* Show edit button for dashboard owners only

* Add visible argument to filters toggle function to avoid future regression

* Improve Toggle filters bar function

* lint

* fix js lint + set createNewOnOpen

* Handle setting extra form data in Filter Bar instead of Filter Control

* Add Handle apply filter function to Apply button

* Allow applying changes instantly

* Fix types

* remove console logs

* Add Error Boundary component to Filter bar and Filter Config Modal

* fix jest tests

* update native filters tests to pass

* reset cypress baseUrl

* remove unnecessary field

* cleanup: remove unused state fields

* move unrelated types to an appropriate location

* remove misplaced resource fetch error logic

* fix cascadeParentIds error

* fix cypress password

* initial attempt at fixing scope issue

* fix bad merge

* fix lint

* trying out makeApi for saving filters

* remove unused import

* fix test

* silence bad test

* add native-filter feat flag config

* oops fix here

* remove space

* Update superset-frontend/src/common/components/index.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/cypress-base/cypress/integration/dashboard/nativeFilters.test.ts

Co-authored-by: Evan Rusackas <evan@preset.io>

* use styledMount in tests

* comment

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/nativeFilters/FilterBar.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/nativeFilters/FilterConfigForm.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/nativeFilters/FilterConfigModal.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/nativeFilters/FilterConfigurationLink.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* address PR feedback

* fix package lock

* null guards

* Fix charts resizing

* fix cypress tests

* add in nativefilters to form data

* fix lint and test

Co-authored-by: Phillip Kelley-Dotson <pkelleydotson@yahoo.com>
Co-authored-by: Simcha Shats <simcha.shats@nielsen.com>
Co-authored-by: amitNielsen <amit.miran@nielsen.com>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
Co-authored-by: Evan Rusackas <evan@preset.io>
Co-authored-by: Agata Stawarz-Pastewska <agata.stawarz-pastewska@polidea.com>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-12-18 17:06:37 -08:00
Yongjie Zhao b592cc7e73
feat(explore): time picker enhancement (#11418) 2020-12-17 18:27:21 -08:00
David Aaron Suddjian 18658f45be
feat(dashboards): Filter status indicators (#10936)
* Initial commit of new filters badge.

* refactor applied/rejected filters code

* finished filter indicators

* filter badge tested

* unnecessary imports

* formatting and types

* fixes

* license

* code quality tweaks

* state management for showing focused filter scope

* clean up filter key extraction code

* remove unnecessary styles

* temp css to demonstrate highlighting

* fix focused filter logic

* no more color badges

* new toys for highlighting dash components (#11144)

* tweak style for the filter chart when filter is focused

* style: Filters p0 css2 (#11151)

* nixing background tweak

* src paths

* another quick theme color

* src paths, adjusting pill icon color, changing icons, showing applied/busted counts

* linting stuff

* fixing and tweaking tests

* show filter indicator when filters are not active

* chart title bar cleanup

* open the right panel when popover opens

* unused import

* fix EditableTitle tests

* margin on dashboard header

* show the chart dropdown menu

* fix blur filter breaking dropdowns

* style tweak - no pointer events when irrelevant charts are blurred

* fix box shadow on filter highlight

* it's an array

* attempt fixing e2e

* style: filters p0 icon churn (#11215)

* new filters icon

* icon styling

* bigger icons in list views

* better sizing of table actions and favStars

* more icon sizing...

* fixing more button size jankiness

* linting

* Filters performance (#11255)

* fixing time filter "ok" button

* making unset filter menu collapsible

* sort alphabetically

* fix highlighting when removing items

* try a flex layout (for browser render perf)

* more specific transitioning

* temp: comment out some code as a test

* temp: comment out more code

* temp: remove possibly expensive computations from ChartHolder

* Revert "temp: comment out some code as a test"

This reverts commit 309b880e90.

* Revert "temp: comment out more code"

This reverts commit 64c88b2cba.

* Revert "temp: remove possibly expensive computations from ChartHolder"

This reverts commit 37ce0214f0.

* experiment: upgrade react-select to v3

* Revert "experiment: upgrade react-select to v3"

This reverts commit c3972ba486.

* fix the damn problem

* remove code used for testing purposes

* awful hack to avoid adding a class to a container

* approaching infinity... and not beyond!

* fix ref forwarding

* add theme to tests as necessary

* fix(extra-filters): add logic for identifying applied extra filters (#11325)

* fix: use dashboard id for stable cache key (#11293)

* fix: button translations missing (#11187)

* button translations missing

* blank space before text

* feat: update time_compare description and choices (#11294)

* feat: update time_compare description and choices

* Update sections.jsx

* fix(extra-filters): add logic for identifying applied extra filters

* lint

Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
Co-authored-by: rubenSastre <ruben.sastre@decathlon.com>
Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>

* address design feedback

* slight tweak to panel logic, keep panels open that user has opened

* rearrange code to be more graceful

* fix: bump superset-ui/core (#11385)

* use is_dttm instead of is_temporal

* types, names

* only show unset filter panel if there are unset filters

* fix highlighting the filter control

* fix filterbox layout

* translations

* fix cypress

* actually add the test attribute

* Update superset-frontend/src/dashboard/components/DashboardBuilder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/DashboardBuilder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* formatting

* add link comment to hack

* Update superset-frontend/src/dashboard/components/gridComponents/ChartHolder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* stop importing lodash

* Update superset-frontend/src/dashboard/components/gridComponents/ChartHolder.jsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* skip broken test

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* Update superset-frontend/src/dashboard/components/FiltersBadge/Styles.tsx

Co-authored-by: Evan Rusackas <evan@preset.io>

* adjust colors of titles

* linting

* no indicators when chart is loading

* support all time fields

* fix lock file

Co-authored-by: Natalie Ruhe <natalie@preset.io>
Co-authored-by: Evan Rusackas <evan@preset.io>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
Co-authored-by: Jesse Yang <jesse.yang@airbnb.com>
Co-authored-by: rubenSastre <ruben.sastre@decathlon.com>
Co-authored-by: Erik Ritter <erik.ritter@airbnb.com>
Co-authored-by: Ville Brofeldt <ville.v.brofeldt@gmail.com>
2020-10-28 15:46:24 -07:00
Grace Guo a10e86ab31
fix: revert eTag cache feature for dashboard (#11203)
* revert #11137

* revert #10963
2020-10-08 12:15:08 -07:00
Grace Guo 6633409a15
feat: enable ETag header for dashboard GET requests (#10963)
* feat: add etag for dashboard load requests

* fix review comments
2020-09-29 10:57:16 -07:00
John Bodley b48dd4b7d9
chore: Using cache factory method (#10887)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-09-15 12:48:19 -07:00
John Bodley 1ee87cc4d1
fix: dashboard extra filters (#10692)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-09-02 16:03:25 -07:00
Ville Brofeldt 85c0cad816
fix: add None checking to cast_to_num (#10584)
* fix: add None checking to cast_to_num

* fix doctest
2020-08-12 19:58:14 +03:00
Ville Brofeldt 363abfa1a5
fix: remove unnecessary exception when exploring non-legacy viz plugins (#10538)
* fix: remove unnecessary exception when exploring non-legacy viz plugins

* lint
2020-08-07 08:41:39 +03:00
Ville Brofeldt b316f723a1
fix(chart-data-api): case insensitive evaluation of filter op (#10299)
* fix(chart-data-api): case insensitive evaluation of filter op

* fix(chart-data-api): case insensitive evaluation of filter op

* mypy

* remove print statement

* add test
2020-07-13 17:21:02 +03:00
John Bodley b181e48f5c
[log] Updating form-data logic (#10197)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-07-04 12:51:43 -07:00
Bogdan 77b36783e7
s/assertEqual/assertEquals (#10195)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-29 16:38:06 -07:00
Bogdan 4e340c8368
feat: Switch from nosetest to pytest (#10177)
* Switch from nosetest to pytest

Fix schedule tests

Collect pytest coverage

Move pytest config into pytest.ini

Move cov to the pytest.ini

* Append coverage for the 2nd run

* Add coverage to all commands

* Coverage only for tests

* Get coverage from 1 place

* Rename classes to be  pytest compatible

* Test coverage for examples and tests

* Max diff to -1

* Explain how to run pytest for the whole project

* Do not append code coverage for the main run

* Do not run coverage on examples

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-29 15:36:06 -07:00
John Bodley 1001c6d5f4
Revert "[caching] Using request context rather than globals" (#9969)
This reverts commit 90cd3889ac.

Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-06-02 16:03:33 -07:00
John Bodley b296a0f250
[mypy] Enforcing typing for superset.utils (#9905)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-27 22:57:30 -07:00
John Bodley 04e3a4a72d
[caching] Using request context rather than globals (#9715)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-05-11 15:55:05 -07:00
John Bodley 7f89f12c4a
[debug] Debugging caching issue (#9665)
Co-authored-by: John Bodley <john.bodley@airbnb.com>
2020-04-29 12:16:47 -07:00
Bogdan 3574a3a4fb
Make email parsing more robust (#9523)
* Make email parsing more robust

* Address comment

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-04-15 16:58:03 +03:00
Ville Brofeldt 499f9c8fca
feat: add SSL certificate validation for Druid (#9396)
* feat: add SSL certificate feature

* Address comments

* don't mutate extras

* Address comments and add polish

* Add further polish
2020-03-27 19:07:07 +02:00
Grace Guo adebd40d30
[cache warm_up] warm_up slice with dashboard default_filters (#9311)
* [cache warm_up] warm_up slice with dashboard default_filters

* update Celery warmup tasks

* fix code review comments

* add try catch and type checking for parsed dash metadata

* extra code review fix
2020-03-18 08:21:10 -07:00
Erik Ritter f80fadff0e
Reduce dashboard bootstrap payload (#9284) 2020-03-13 12:18:22 -07:00
John Bodley 8a138fbd03
[tox] Allowing running of specific tests (#9097) 2020-02-06 16:45:37 -08:00
Erik Ritter 68e85ab1b6 [Caching] Ensure cache is always created (#9020)
* [Caching] Ensure cache is always created

* Update cache_manager.py

* Refactoring cache typing
2020-01-25 22:49:05 +00:00
Rob DiCiuccio 7ff18f060c Ensure proper JSON serialization of numpy.ndarray (#8946) 2020-01-10 13:24:03 -05:00
Bogdan 003e98c231 Support schema permissions (#8219)
* Build support for schema access in Superset and SQLab

* Lint
* Test for old and new flask
* Black formatting
* Cleanup modified files
* Support schema permissions
* Add migration
* Make it work for slices as well
* TODO and remove unused field
* Apply schema access check to dashboards as well

Cleaner schema permissions check

Fetch schema permissions in 1 query

Bugfix

Better datasource check

Use security manager for permissions checks and remove dead code

Properly handle anon user

Add database permissions on the database updates

Fix schema filtering

Lint / flake changes

Increase cypress request timeout

Run black

Resolve multiple alembic heads

* Use set comprehensions

* Fixes for the pylint
2019-12-03 14:57:24 -08:00
Ville Brofeldt 7104b04817
Pass url parameters from dashboard to charts (#8536)
* Pass url_params from dashboard to charts

* Update params to form_data instead of overwriting

* Add cypress tests

* Add python test

* Add docs

* Move reserved url params to utils

* Bump cypress
2019-11-21 07:07:08 +02:00
Craig Rueda e490414484 Flask App factory PR #1 (#8418)
* First cut at app factory

* Setting things back to master

* Working with new FLASK_APP

* Still need to refactor Celery

* CLI mostly working

* Working on unit tests

* Moving cli stuff around a bit

* Removing get in config

* Defaulting test config

* Adding flask-testing

* flask-testing casing

* resultsbackend property bug

* Fixing up cli

* Quick fix for KV api

* Working on save slice

* Fixed core_tests

* Fixed utils_tests

* Most tests working - still need to dig into remaining app_context issue in tests

* All tests passing locally - need to update code comments

* Fixing dashboard tests again

* Blacking

* Sorting imports

* linting

* removing envvar mangling

* blacking

* Fixing unit tests

* isorting

* licensing

* fixing mysql tests

* fixing cypress?

* fixing .flaskenv

* fixing test app_ctx

* fixing cypress

* moving manifest processor around

* moving results backend manager around

* Cleaning up __init__ a bit more

* Addressing PR comments

* Addressing PR comments

* Blacking

* Fixes for running celery worker

* Tuning isort

* Blacking
2019-11-20 15:47:06 +00:00
John Bodley 1e3d833b02
[sip-15] Fixing time range endpoints from dashboards (#8513) 2019-11-06 09:15:58 -08:00
John Bodley 5dba2f4b56
[SIP-15] Adding grace period (#8490) 2019-11-04 12:00:41 -08:00
John Bodley eb150eba1f
[sip-15] Fix time range endpoints encoding (#8481) 2019-10-31 07:11:48 -07:00
John Bodley 786d7706e7
[fix] Updating parse_human_timedelta typing (#8436) 2019-10-23 16:04:46 -07:00
Marcus 4cfd6b3ce1 [tests] Fix, removing deprecated function for future Python release compatibility. (#8353)
* removing deprecated function for future Python release compatibility.

* also removed assertNotEquals per @dpgaspar
2019-10-21 15:49:12 +01:00
John Bodley 9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
Ben bc83b5fbe6 More intuitive display of negative time deltas (#8274) (#8276)
* WIP - Add easily interpretable negative timedelta formatting

* Add unit tests for utils.core.timedelta_f

* Reformat tests.utils_tests.test_timedelta_f test code. Fixes #8274.

* Add type annotations to utils.core.timedelta_f (#8274)

* Replace recursion with absolute value in utils.core.timedelta_f (#8274)

* Rename utils.core.timedelta_f to format_timedelta (#8274)

* Fix small typo (#8274)
2019-09-22 19:37:43 +03:00
Beto Dealmeida 4132d8fb0f
Simplify and improve `expand_data` in Presto (#8233)
* WIP

* Working version, needs cleanup

* Add unit tests to split

* Small fixes

* Dedupe array columns

* Fix lint
2019-09-17 14:04:14 -07:00
Rob DiCiuccio 7595d9e5fd [SQL Lab] Async query results serialization with MessagePack and PyArrow (#8069)
* Add support for msgpack results_backend serialization

* Serialize DataFrame with PyArrow rather than JSON

* Adjust dependencies, de-lint

* Add tests for (de)serialization methods

* Add MessagePack config info to Installation docs

* Enable msgpack/arrow serialization by default

* [Fix] Prevent msgpack serialization on synchronous queries

* Add type annotations
2019-08-27 14:23:40 -07:00
serenajiang 17f0740692 Fix bug where lists in queries cannot contain multiple types (#8032)
* use set instead of sorted to check equality of lists

* run black
2019-08-13 23:06:44 +03:00
Daniel Vaz Gaspar 994ac04c1f [cli] New, command line option to create or set a db URI (#7918)
* [cli] New, command line option to create or set a db URI

* [tests] New, get or create db on utils

* [tests] flake8 and black

* [tests] Fix, remove dummy comments

* [cli] dummy commit to re-trigger build

* [cli] dummy commit to re-trigger build
2019-07-27 17:43:39 -07:00
Rob DiCiuccio df051813d5 Adds the ability to replace/extend caching backend (#7856)
* Add ability to override cache backend with custom module

* Tests for setup_cache

* Add custom CACHE_CONFIG documentation

* Fix linter errors and documentation link

* Fix black formatting errors
2019-07-12 14:06:56 -07:00