superset/contrib/docker/superset_config.py

49 lines
1.6 KiB
Python
Raw Normal View History

Init docker for local development environment. (#4193) This commit will try to dockerize superset in local development environment. The basic design is: - Enable superset, redis and postgres service instead of using sqlite, just want to simulate production environment settings - Use environment variables to config various app settings. It's easy to run and config superset to any environment if we use environment than traditional config files - For local development environment, we just expose postgres and redis to local host machine thus you can connect local port via `psql` or `redis-cli` - Wrap start up command in a standard `docker-entrypoint.sh`, and use `tail -f /dev/null` combined with manually `superset runserver -d` to make sure that code error didn't cause the container to fail. - Use volumes to share code between host and container, thus you can use your favourite tools to modify code and your code will run in containerized environment - Use volumes to persistent postgres and redis data, and also `node_modules` data. - If we don't cache `node_modules` in docker volume, then every time run docker build, the `node_modules` directory, will is about 500 MB large, will be sent to docker daemon, and make the build quite slow. - Wrap initialization commands to a single script `docker-init.sh` After this dockerize setup, any developers who want to contribute to superset, just follow three easy steps: ``` git clone https://github.com/apache/incubator-superset/ cd incubator-superset cp contrib/docker/{docker-build.sh,docker-compose.yml,docker-entrypoint.sh,docker-init.sh,Dockerfile} . cp contrib/docker/superset_config.py superset/ bash -x docker-build.sh docker-compose up -d docker-compose exec superset bash bash docker-init.sh ```
2018-06-10 00:26:41 -04:00
# -*- coding: utf-8 -*-
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
from __future__ import unicode_literals
import os
def get_env_variable(var_name, default=None):
"""Get the environment variable or raise exception."""
try:
return os.environ[var_name]
except KeyError:
if default is not None:
return default
else:
error_msg = 'The environment variable {} was missing, abort...'\
.format(var_name)
raise EnvironmentError(error_msg)
POSTGRES_USER = get_env_variable('POSTGRES_USER')
POSTGRES_PASSWORD = get_env_variable('POSTGRES_PASSWORD')
POSTGRES_HOST = get_env_variable('POSTGRES_HOST')
POSTGRES_PORT = get_env_variable('POSTGRES_PORT')
POSTGRES_DB = get_env_variable('POSTGRES_DB')
# The SQLAlchemy connection string.
SQLALCHEMY_DATABASE_URI = 'postgresql://%s:%s@%s:%s/%s' % (POSTGRES_USER,
POSTGRES_PASSWORD,
POSTGRES_HOST,
POSTGRES_PORT,
POSTGRES_DB)
REDIS_HOST = get_env_variable('REDIS_HOST')
REDIS_PORT = get_env_variable('REDIS_PORT')
class CeleryConfig(object):
BROKER_URL = 'redis://%s:%s/0' % (REDIS_HOST, REDIS_PORT)
CELERY_IMPORTS = ('superset.sql_lab', )
CELERY_RESULT_BACKEND = 'redis://%s:%s/1' % (REDIS_HOST, REDIS_PORT)
CELERY_ANNOTATIONS = {'tasks.add': {'rate_limit': '10/s'}}
CELERY_TASK_PROTOCOL = 1
CELERY_CONFIG = CeleryConfig