Commit Graph

19 Commits

Author SHA1 Message Date
Bogdan ac2937a6c5
fix: use nullpool in the celery workers (#10819)
* Use nullpool in the celery workers

* Address comments

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-09-10 13:29:57 -07:00
Craig Rueda 2aaa4d92d9
chore: Migrating reports to AuthWebdriverProxy (#10567)
* Migrating reports to AuthWebdriverProxy

* Extracting out webdriver proxy / Adding thumbnail tests to CI

* Adding license

* Adding license again

* Empty commit

* Adding thumbnail tests to CI

* Switching thumbnail test to Postgres

* Linting

* Adding mypy:ignore / removing thumbnail tests from CI

* Putting ignore statement back

* Updating docs

* First cut at authprovider

* First cut at authprovider mostly working - still needs more tests

* Auth provider tests added

* Linting

* Linting again...

* Linting again...

* Busting CI cache

* Reverting workflow change

* Fixing dataclasses

* Reverting back to master

* linting?

* Reverting installation.rst

* Reverting package-lock.json

* Addressing feedback

* Blacking

* Lazy logging strings

* UPDATING.md note
2020-08-12 13:28:41 -07:00
Bogdan 0071d374da
chore: move load examples to the fixture (#10545)
* Move load examples to the fixture

* Update tests/celery_tests.py

Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>

* Address feedback

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
Co-authored-by: Ville Brofeldt <33317356+villebro@users.noreply.github.com>
2020-08-10 13:20:19 -07:00
Jesse Yang ce10e13ec3
test: schedules test refactor (#10492) 2020-07-31 12:05:39 -07:00
Jesse Yang 7ff1757448
build: optimize GitHub Actions for building speed and stability (#10485) 2020-07-30 22:28:56 -07:00
Bogdan 7075c2ff69
Pin slack client, breaking change (#10345)
Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-07-20 09:20:05 -07:00
Maxime Beauchemin 318e5347bc
feat: Alerts! allowing users to set SQL-based email alerts with screenshots (#9944)
* feat: add support for alerts

* Add ModelViews

* First pass at adding scheduled Alert support

* Fix syntax errors

* Add log_retention and database to model

* Improving screenshots

* Still refactoring screenshots

* Pass down thumb_size properly

* Progress on screenshot endpoint

* Add alerts.last_eval_dttm and alert query logic

* Getting ready to split compute_screenshot and screenshot/{SHA}

* split the endpoints

* Improving alerting loop

* empty methods

* Add CLI command 'superset alert' that runs a sched loop

* Getting things to work

* email delivery

* A working email\!

* Add feature flag

* Add some description_columns to AlertModelView

* Little tweaks

* Use database.get_df, eval cells for truthiness

* Migrate thumbnail/screenshot functions to use new arg pattern

* Addressing PR feedback

* Update alerts DB migration down_revision

* Resolve _deliver_email arg conflict

* Make mypy happy

* Make isort happy

* Make pylint happy

Co-authored-by: Rob DiCiuccio <rob.diciuccio@gmail.com>
2020-07-06 13:12:27 -07:00
Bogdan 4e340c8368
feat: Switch from nosetest to pytest (#10177)
* Switch from nosetest to pytest

Fix schedule tests

Collect pytest coverage

Move pytest config into pytest.ini

Move cov to the pytest.ini

* Append coverage for the 2nd run

* Add coverage to all commands

* Coverage only for tests

* Get coverage from 1 place

* Rename classes to be  pytest compatible

* Test coverage for examples and tests

* Max diff to -1

* Explain how to run pytest for the whole project

* Do not append code coverage for the main run

* Do not run coverage on examples

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-29 15:36:06 -07:00
Bogdan 29e9f2c70b
feat: superset report slack integration (#9810)
* First draft for the slack integration

Fix slack

another typo

another typo

Fix slack

Add channels to the form

another typo

Another set of changes

Make code more transparent

Fix tests

Add logging

logging

use logger

import logging

import logging

import logging

add assert

more logging

Fix channels

Fix channels

* Address comments

* Move slack into a separate module

Co-authored-by: bogdan kyryliuk <bogdankyryliuk@dropbox.com>
2020-06-17 11:01:25 -07:00
David Aaron Suddjian 016f202423 Refactor Dashboard and Slice models (#8820)
* refactor dashboard and slice models

* appease various linters

* remove shortcuts & import indirection

* appease mypy

* fix bad imports

* lint

* address various issues

* ignore type issue

* remove unused imports

* lint
2019-12-18 11:40:45 -08:00
Hossein Torabi b053d66ea7 fix urlib opener bug (#8788) 2019-12-07 16:07:09 -08:00
Craig Rueda e490414484 Flask App factory PR #1 (#8418)
* First cut at app factory

* Setting things back to master

* Working with new FLASK_APP

* Still need to refactor Celery

* CLI mostly working

* Working on unit tests

* Moving cli stuff around a bit

* Removing get in config

* Defaulting test config

* Adding flask-testing

* flask-testing casing

* resultsbackend property bug

* Fixing up cli

* Quick fix for KV api

* Working on save slice

* Fixed core_tests

* Fixed utils_tests

* Most tests working - still need to dig into remaining app_context issue in tests

* All tests passing locally - need to update code comments

* Fixing dashboard tests again

* Blacking

* Sorting imports

* linting

* removing envvar mangling

* blacking

* Fixing unit tests

* isorting

* licensing

* fixing mysql tests

* fixing cypress?

* fixing .flaskenv

* fixing test app_ctx

* fixing cypress

* moving manifest processor around

* moving results backend manager around

* Cleaning up __init__ a bit more

* Addressing PR comments

* Addressing PR comments

* Blacking

* Fixes for running celery worker

* Tuning isort

* Blacking
2019-11-20 15:47:06 +00:00
Marcus 4cfd6b3ce1 [tests] Fix, removing deprecated function for future Python release compatibility. (#8353)
* removing deprecated function for future Python release compatibility.

* also removed assertNotEquals per @dpgaspar
2019-10-21 15:49:12 +01:00
John Bodley 9fc37ea9f1 [ci] Deprecate flake8 (#8409)
* [ci] Deprecate flake8

* Addressing @villebro's comments
2019-10-18 14:44:27 -07:00
Gianluca Ciccarelli e23920b8ba 7620: Start removing dependencies on requests (#7643)
* 7620: Start removing dependencies on requests

* Patch urllib.request.urlopen instead of requests.get

* Try to fix flake8

* More work on flake8 import errors

* First attempt at using urllib with cookies

* Fix pylint/flake8

* Fix test_deliver_slice_csv_attachment

* Fix test_deliver_slice_csv_inline

* Import requests and pydruid conditionally, remove dependency on prison

* Fix flake errors

* Fix load_examples

* Please flake

* Skip tests depending on optional deps

* Try to please flake

* Address review comments

* Remove Druid-related UI

* Revert "Remove Druid-related UI"

This reverts commit d7e0f166cc3f3dd2496b4a666e177f0c191aeb0f.

* Skip a few tests more

* Put imports in right order

* Apply black patch

* Please flake

* Please black, silence flake

* Use flake8 silencing the right way

* Add deps for CI
2019-08-02 10:01:28 -07:00
John Bodley 5c58fd1802
[format] Using Black (#7769) 2019-06-25 13:34:48 -07:00
Dave Smith ba19a62412 Injectable statsd client (#7138)
* Add ability to inject statsd client; some py test/reqs updates

- Updated the metrics logger to allow construction with an existing
statsd client, so that it can be configured by external systems or libs.
- added requirements to requirements-dev.txt which are needed to run
  tests-eg coverage, nose
- removed dependency on mock lib, it is in python stdlib now
- updated tox.ini to remove the now-superfluous deps

* add license to test file, and remove blank line at EOF
2019-03-27 14:08:36 -07:00
Maxime Beauchemin 1dd4d7a587
Apply ASF licenses throughout the code base (#5800)
* Add license headers

* reabased

* lint

* Removing licenses from vendors folder
2019-01-15 15:53:27 -08:00
Mahendra M 808622414c [SIP-3] Scheduled email reports for Slices / Dashboards (#5294)
* [scheduled reports] Add support for scheduled reports

* Scheduled email reports for slice and dashboard visualization
  (attachment or inline)
* Scheduled email reports for slice data (CSV attachment on inline table)
* Each schedule has a list of recipients (all of them can receive a single mail,
  or separate mails)
* All outgoing mails can have a mandatory bcc - for audit purposes.
* Each dashboard/slice can have multiple schedules.

In addition, this PR also makes a few minor improvements to the celery
infrastructure.
* Create a common celery app
* Added more celery annotations for the tasks
* Introduced celery beat
* Update docs about concurrency / pools

* [scheduled reports] - Debug mode for scheduled emails

* [scheduled reports] - Ability to send test mails

* [scheduled reports] - Test email functionality - minor improvements

* [scheduled reports] - Rebase with master. Minor fixes

* [scheduled reports] - Add warning messages

* [scheduled reports] - flake8

* [scheduled reports] - fix rebase

* [scheduled reports] - fix rebase

* [scheduled reports] - fix flake8

* [scheduled reports] Rebase in prep for merge

* Fixed alembic tree after rebase
* Updated requirements to latest version of packages (and tested)
* Removed py2 stuff

* [scheduled reports] - fix flake8

* [scheduled reports] - address review comments

* [scheduled reports] - rebase with master
2018-12-10 22:29:29 -08:00