superset/contrib/docker/superset_config.py

59 lines
2.2 KiB
Python
Raw Normal View History

# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
Init docker for local development environment. (#4193) This commit will try to dockerize superset in local development environment. The basic design is: - Enable superset, redis and postgres service instead of using sqlite, just want to simulate production environment settings - Use environment variables to config various app settings. It's easy to run and config superset to any environment if we use environment than traditional config files - For local development environment, we just expose postgres and redis to local host machine thus you can connect local port via `psql` or `redis-cli` - Wrap start up command in a standard `docker-entrypoint.sh`, and use `tail -f /dev/null` combined with manually `superset runserver -d` to make sure that code error didn't cause the container to fail. - Use volumes to share code between host and container, thus you can use your favourite tools to modify code and your code will run in containerized environment - Use volumes to persistent postgres and redis data, and also `node_modules` data. - If we don't cache `node_modules` in docker volume, then every time run docker build, the `node_modules` directory, will is about 500 MB large, will be sent to docker daemon, and make the build quite slow. - Wrap initialization commands to a single script `docker-init.sh` After this dockerize setup, any developers who want to contribute to superset, just follow three easy steps: ``` git clone https://github.com/apache/incubator-superset/ cd incubator-superset cp contrib/docker/{docker-build.sh,docker-compose.yml,docker-entrypoint.sh,docker-init.sh,Dockerfile} . cp contrib/docker/superset_config.py superset/ bash -x docker-build.sh docker-compose up -d docker-compose exec superset bash bash docker-init.sh ```
2018-06-10 00:26:41 -04:00
import os
def get_env_variable(var_name, default=None):
"""Get the environment variable or raise exception."""
try:
return os.environ[var_name]
except KeyError:
if default is not None:
return default
else:
error_msg = 'The environment variable {} was missing, abort...'\
.format(var_name)
raise EnvironmentError(error_msg)
POSTGRES_USER = get_env_variable('POSTGRES_USER')
POSTGRES_PASSWORD = get_env_variable('POSTGRES_PASSWORD')
POSTGRES_HOST = get_env_variable('POSTGRES_HOST')
POSTGRES_PORT = get_env_variable('POSTGRES_PORT')
POSTGRES_DB = get_env_variable('POSTGRES_DB')
# The SQLAlchemy connection string.
SQLALCHEMY_DATABASE_URI = 'postgresql://%s:%s@%s:%s/%s' % (POSTGRES_USER,
POSTGRES_PASSWORD,
POSTGRES_HOST,
POSTGRES_PORT,
POSTGRES_DB)
REDIS_HOST = get_env_variable('REDIS_HOST')
REDIS_PORT = get_env_variable('REDIS_PORT')
class CeleryConfig(object):
BROKER_URL = 'redis://%s:%s/0' % (REDIS_HOST, REDIS_PORT)
CELERY_IMPORTS = ('superset.sql_lab', )
CELERY_RESULT_BACKEND = 'redis://%s:%s/1' % (REDIS_HOST, REDIS_PORT)
CELERY_ANNOTATIONS = {'tasks.add': {'rate_limit': '10/s'}}
CELERY_TASK_PROTOCOL = 1
CELERY_CONFIG = CeleryConfig