2019-01-22 11:21:13 -05:00
|
|
|
<!--
|
|
|
|
Licensed to the Apache Software Foundation (ASF) under one
|
|
|
|
or more contributor license agreements. See the NOTICE file
|
|
|
|
distributed with this work for additional information
|
|
|
|
regarding copyright ownership. The ASF licenses this file
|
|
|
|
to you under the Apache License, Version 2.0 (the
|
|
|
|
"License"); you may not use this file except in compliance
|
|
|
|
with the License. You may obtain a copy of the License at
|
|
|
|
|
|
|
|
http://www.apache.org/licenses/LICENSE-2.0
|
|
|
|
|
|
|
|
Unless required by applicable law or agreed to in writing,
|
|
|
|
software distributed under the License is distributed on an
|
|
|
|
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
|
|
|
|
KIND, either express or implied. See the License for the
|
|
|
|
specific language governing permissions and limitations
|
|
|
|
under the License.
|
|
|
|
-->
|
2018-09-19 13:01:51 -04:00
|
|
|
|
2019-03-25 19:21:26 -04:00
|
|
|
# Apache Releases
|
2018-09-19 13:01:51 -04:00
|
|
|
|
2019-10-30 13:24:18 -04:00
|
|
|
Until things settle and we create scripts that streamline this,
|
|
|
|
you'll probably want to run these commands manually and understand what
|
2019-03-25 19:21:26 -04:00
|
|
|
they do prior to doing so.
|
2018-09-19 13:01:51 -04:00
|
|
|
|
2019-10-30 13:24:18 -04:00
|
|
|
For coordinating on releases, on operational topics that require more
|
|
|
|
synchronous communications, we recommend using the `#apache-releases` channel
|
2019-10-23 00:42:12 -04:00
|
|
|
on the Superset Slack. People crafting releases and those interested in
|
|
|
|
partaking in the process should join the channel.
|
|
|
|
|
2019-10-23 09:43:04 -04:00
|
|
|
## Release setup (First Time Only)
|
2019-04-20 13:26:01 -04:00
|
|
|
|
2019-03-25 19:21:26 -04:00
|
|
|
First you need to setup a few things. This is a one-off and doesn't
|
|
|
|
need to be done at every release.
|
2018-09-19 13:01:51 -04:00
|
|
|
|
|
|
|
```bash
|
2019-07-04 01:33:14 -04:00
|
|
|
# Create PGP Key, and use your @apache.org email address
|
2019-03-25 19:21:26 -04:00
|
|
|
gpg --gen-key
|
2019-08-12 17:01:15 -04:00
|
|
|
|
2019-03-25 19:21:26 -04:00
|
|
|
# Checkout ASF dist repo
|
2019-04-15 15:48:51 -04:00
|
|
|
|
|
|
|
svn checkout https://dist.apache.org/repos/dist/dev/incubator/superset/ ~/svn/superset_dev
|
|
|
|
|
2019-07-04 01:33:14 -04:00
|
|
|
svn checkout https://dist.apache.org/repos/dist/release/incubator/superset/ ~/svn/superset
|
2019-03-25 19:21:26 -04:00
|
|
|
cd ~/svn/superset
|
2019-08-12 17:01:15 -04:00
|
|
|
|
|
|
|
|
2019-03-25 19:21:26 -04:00
|
|
|
# Add your GPG pub key to KEYS file. Replace "Maxime Beauchemin" with your name
|
2019-10-28 06:17:10 -04:00
|
|
|
export SUPERSET_PGP_FULLNAME="Maxime Beauchemin"
|
|
|
|
(gpg --list-sigs "${SUPERSET_PGP_FULLNAME}" && gpg --armor --export "${SUPERSET_PGP_FULLNAME}" ) >> KEYS
|
2019-08-12 17:01:15 -04:00
|
|
|
|
|
|
|
|
2019-03-25 19:21:26 -04:00
|
|
|
# Commit the changes
|
|
|
|
svn commit -m "Add PGP keys of new Superset committer"
|
2018-09-19 13:01:51 -04:00
|
|
|
```
|
|
|
|
|
2020-02-18 00:14:31 -05:00
|
|
|
## Setting up the release environment (do every time)
|
|
|
|
|
|
|
|
As the vote process takes a minimum of 72h (community vote) + 72h (IPMC) vote,
|
|
|
|
often stretching over several weeks calendar time if votes don't pass, chances are
|
|
|
|
the same terminal session won't be used for crafting the release candidate and the
|
|
|
|
final release. Therefore, it's a good idea to do the following every time you
|
|
|
|
work on a new phase of the release process to make sure you aren't releasing
|
|
|
|
the wrong files/using wrong names. There's a script to help you set correctly all the
|
|
|
|
necessary environment variables. Change your current directory to `superset/RELEASING`
|
|
|
|
|
|
|
|
```bash
|
2020-04-06 02:21:31 -04:00
|
|
|
# usage (BASH): . set_release_env.sh <SUPERSET_RC_VERSION> <PGP_KEY_FULLNAME>
|
|
|
|
# usage (ZSH): source set_release_env.sh <SUPERSET_RC_VERSION> <PGP_KEY_FULLNAME>
|
|
|
|
#
|
|
|
|
# example: source set_relese_env.sh 0.36.0rc3 myid@apache.org
|
2020-02-18 00:14:31 -05:00
|
|
|
```
|
|
|
|
|
2020-04-06 02:21:31 -04:00
|
|
|
The script will output the exported variables. Here's example for 0.36.0rc3:
|
2020-02-18 00:14:31 -05:00
|
|
|
|
|
|
|
```
|
|
|
|
-------------------------------
|
|
|
|
Set Release env variables
|
2020-04-06 02:21:31 -04:00
|
|
|
SUPERSET_VERSION=0.36.0
|
|
|
|
SUPERSET_RC=3
|
|
|
|
SUPERSET_GITHUB_BRANCH=0.36
|
2020-02-18 00:14:31 -05:00
|
|
|
SUPERSET_PGP_FULLNAME=myid@apache.org
|
2020-04-06 02:21:31 -04:00
|
|
|
SUPERSET_VERSION_RC=0.36.0rc3
|
|
|
|
SUPERSET_RELEASE=apache-superset-incubating-0.36.0
|
|
|
|
SUPERSET_RELEASE_RC=apache-superset-incubating-0.36.0rc3
|
|
|
|
SUPERSET_RELEASE_TARBALL=apache-superset-incubating-0.36.0-source.tar.gz
|
|
|
|
SUPERSET_RELEASE_RC_TARBALL=apache-superset-incubating-0.36.0rc3-source.tar.gz
|
|
|
|
SUPERSET_TMP_ASF_SITE_PATH=/tmp/incubator-superset-site-0.36.0
|
2020-02-18 00:14:31 -05:00
|
|
|
-------------------------------
|
|
|
|
```
|
|
|
|
|
2019-10-28 06:17:10 -04:00
|
|
|
## Crafting a source release
|
|
|
|
|
2020-02-09 20:53:56 -05:00
|
|
|
When crafting a new minor or major release we create
|
2020-02-18 00:14:31 -05:00
|
|
|
a branch named with the release MAJOR.MINOR version (on this example 0.35).
|
2020-02-09 20:53:56 -05:00
|
|
|
This new branch will hold all PATCH and release candidates
|
2019-10-28 06:17:10 -04:00
|
|
|
that belong to the MAJOR.MINOR version.
|
|
|
|
|
|
|
|
The MAJOR.MINOR branch is normally a "cut" from a specific point in time from the master branch.
|
|
|
|
Then (if needed) apply all cherries that will make the PATCH
|
|
|
|
|
2019-11-19 10:44:38 -05:00
|
|
|
Next update the `CHANGELOG.md` with all the changes that are included in the release. Make sure you have
|
|
|
|
set your GITHUB_TOKEN environment variable.
|
|
|
|
|
|
|
|
```bash
|
|
|
|
# will overwrites the local CHANGELOG.md, somehow you need to merge it in
|
2020-02-18 00:14:31 -05:00
|
|
|
github-changes -o apache -r incubator-superset --token $GITHUB_TOKEN -b $SUPERSET_GITHUB_BRANCH
|
2019-11-19 10:44:38 -05:00
|
|
|
```
|
|
|
|
|
2019-11-27 10:28:37 -05:00
|
|
|
Then, in `UPDATING.md`, a file that contains a list of notifications around
|
|
|
|
deprecations and upgrading-related topics,
|
|
|
|
make sure to move the content now under the `Next Version` section under a new
|
|
|
|
section for the new release.
|
|
|
|
|
2020-02-18 00:14:31 -05:00
|
|
|
Finally bump the version number on `superset-frontend/package.json` (replace with whichever version is being released excluding the RC version):
|
2019-10-28 06:17:10 -04:00
|
|
|
|
2019-11-15 08:44:07 -05:00
|
|
|
```json
|
2020-04-06 02:21:31 -04:00
|
|
|
"version": "0.36.0"
|
2019-11-15 08:44:07 -05:00
|
|
|
```
|
2019-10-28 06:17:10 -04:00
|
|
|
|
2020-04-06 02:21:31 -04:00
|
|
|
Commit the change with the version number, then git tag the version with the release candidate and push to the branch:
|
|
|
|
|
|
|
|
```
|
|
|
|
# add changed files and commit
|
|
|
|
git add ...
|
|
|
|
git commit ...
|
|
|
|
# push new tag
|
|
|
|
git tag ${SUPERSET_VERSION_RC}
|
|
|
|
git push upstream ${SUPERSET_VERSION_RC}
|
|
|
|
```
|
2019-10-28 06:17:10 -04:00
|
|
|
|
2019-10-23 09:43:04 -04:00
|
|
|
## Preparing the release candidate
|
|
|
|
|
|
|
|
The first step of preparing an Apache Release is packaging a release candidate
|
2019-11-19 10:44:38 -05:00
|
|
|
to be voted on. Make sure you have correctly prepared and tagged the ready to ship
|
2020-02-09 20:53:56 -05:00
|
|
|
release on Superset's repo (MAJOR.MINOR branch), the following script will clone
|
2019-11-19 10:44:38 -05:00
|
|
|
the tag and create a signed source tarball from it:
|
2019-10-23 09:43:04 -04:00
|
|
|
|
|
|
|
```bash
|
2019-11-25 05:01:31 -05:00
|
|
|
# make_tarball will use the previously set environment variables
|
2019-11-19 10:44:38 -05:00
|
|
|
# you can override by passing arguments: make_tarball.sh <SUPERSET_VERSION> <SUPERSET_VERSION_RC> "<PGP_KEY_FULLNAME>"
|
|
|
|
./make_tarball.sh
|
2019-10-23 09:43:04 -04:00
|
|
|
```
|
2019-03-25 19:21:26 -04:00
|
|
|
|
2019-11-19 10:44:38 -05:00
|
|
|
Note that `make_tarball.sh`:
|
2019-10-23 09:43:04 -04:00
|
|
|
|
2020-02-09 20:53:56 -05:00
|
|
|
- By default assumes you have already executed an SVN checkout to `$HOME/svn/superset_dev`.
|
|
|
|
This can be overriden by setting `SUPERSET_SVN_DEV_PATH` environment var to a different svn dev directory
|
2019-11-19 10:44:38 -05:00
|
|
|
- Will refuse to craft a new release candidate if a release already exists on your local svn dev directory
|
|
|
|
- Will check `package.json` version number and fails if it's not correctly set
|
2019-10-23 09:43:04 -04:00
|
|
|
|
2019-11-19 10:44:38 -05:00
|
|
|
### Build and test the created source tarball
|
2019-10-23 09:43:04 -04:00
|
|
|
|
2020-02-18 00:14:31 -05:00
|
|
|
To build and run the **local copy** of the recently created tarball:
|
2019-10-23 09:43:04 -04:00
|
|
|
```bash
|
2019-11-19 10:44:38 -05:00
|
|
|
# Build and run a release candidate tarball
|
|
|
|
./test_run_tarball.sh local
|
|
|
|
# you should be able to access localhost:5001 on your browser
|
|
|
|
# login using admin/admin
|
2019-03-25 19:21:26 -04:00
|
|
|
```
|
|
|
|
|
2019-10-23 09:43:04 -04:00
|
|
|
### Shipping to SVN
|
2019-04-20 13:26:01 -04:00
|
|
|
|
2019-04-15 15:48:51 -04:00
|
|
|
Now let's ship this RC into svn's dev folder
|
2019-03-25 19:21:26 -04:00
|
|
|
|
|
|
|
```bash
|
2019-04-20 13:26:01 -04:00
|
|
|
cd ~/svn/superset_dev/
|
2019-10-23 09:43:04 -04:00
|
|
|
svn add ${SUPERSET_VERSION_RC}
|
|
|
|
svn commit -m "Release ${SUPERSET_VERSION_RC}"
|
2019-03-25 19:21:26 -04:00
|
|
|
```
|
|
|
|
|
2019-11-19 10:44:38 -05:00
|
|
|
### Build and test from SVN source tarball
|
2019-11-15 08:44:07 -05:00
|
|
|
|
2020-02-18 00:14:31 -05:00
|
|
|
To build and run the recently created tarball **from SVN**:
|
2019-11-15 08:44:07 -05:00
|
|
|
```bash
|
2019-11-19 10:44:38 -05:00
|
|
|
# Build and run a release candidate tarball
|
|
|
|
./test_run_tarball.sh
|
|
|
|
# you should be able to access localhost:5001 on your browser
|
|
|
|
# login using admin/admin
|
2019-11-15 08:44:07 -05:00
|
|
|
```
|
|
|
|
|
2019-10-30 13:24:18 -04:00
|
|
|
### Voting
|
|
|
|
Now you're ready to start the [VOTE] thread. Here's an example of a
|
|
|
|
previous release vote thread:
|
|
|
|
https://lists.apache.org/thread.html/e60f080ebdda26896214f7d3d5be1ccadfab95d48fbe813252762879@<dev.superset.apache.org>
|
|
|
|
|
2019-11-25 05:01:31 -05:00
|
|
|
To easily send a voting request to Superset community, still on the `superset/RELEASING` directory:
|
|
|
|
|
|
|
|
```bash
|
2020-02-09 20:53:56 -05:00
|
|
|
# Note: use Superset's virtualenv
|
2019-11-25 05:01:31 -05:00
|
|
|
(venv)$ python send_email.py vote_pmc
|
|
|
|
```
|
|
|
|
|
|
|
|
The script will interactively ask for extra information so it can authenticate on the Apache Email Relay.
|
2020-02-09 20:53:56 -05:00
|
|
|
The release version and release candidate number are fetched from the previously set environment variables.
|
2019-11-25 05:01:31 -05:00
|
|
|
|
|
|
|
```bash
|
|
|
|
Sender email (ex: user@apache.org): your_apache_email@apache.org
|
|
|
|
Apache username: your_apache_user
|
|
|
|
Apache password: your_apache_password
|
|
|
|
```
|
|
|
|
|
2019-10-30 13:24:18 -04:00
|
|
|
Once 3+ binding votes (by PMC members) have been cast and at
|
|
|
|
least 72 hours have past, you can post a [RESULT] thread:
|
|
|
|
https://lists.apache.org/thread.html/50a6b134d66b86b237d5d7bc89df1b567246d125a71394d78b45f9a8@%3Cdev.superset.apache.org%3E
|
|
|
|
|
2019-11-25 05:01:31 -05:00
|
|
|
To easily send the result email, still on the `superset/RELEASING` directory:
|
|
|
|
|
|
|
|
```bash
|
2020-02-09 20:53:56 -05:00
|
|
|
# Note: use Superset's virtualenv
|
2019-11-25 05:01:31 -05:00
|
|
|
(venv)$ python send_email.py result_pmc
|
|
|
|
```
|
|
|
|
|
|
|
|
The script will interactively ask for extra information needed to fill out the email template. Based on the
|
2020-02-09 20:53:56 -05:00
|
|
|
voting description, it will generate a passing, non passing or non conclusive email.
|
2019-11-25 05:01:31 -05:00
|
|
|
here's an example:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
Sender email (ex: user@apache.org): your_apache_email@apache.org
|
|
|
|
Apache username: your_apache_user
|
|
|
|
Apache password: your_apache_password
|
|
|
|
A List of people with +1 binding vote (ex: Max,Grace,Krist): Daniel,Alan,Max,Grace
|
|
|
|
A List of people with +1 non binding vote (ex: Ville): Ville
|
|
|
|
A List of people with -1 vote (ex: John):
|
|
|
|
```
|
|
|
|
|
2019-10-30 13:24:18 -04:00
|
|
|
Following the result thread, yet another [VOTE] thread should be
|
|
|
|
started at general@incubator.apache.org.
|
|
|
|
|
2019-11-25 05:01:31 -05:00
|
|
|
To easily send the voting request to Apache community, still on the `superset/RELEASING` directory:
|
|
|
|
|
|
|
|
```bash
|
2020-02-09 20:53:56 -05:00
|
|
|
# Note: use Superset's virtualenv
|
2019-11-25 05:01:31 -05:00
|
|
|
(venv)$ python send_email.py vote_ipmc
|
|
|
|
```
|
|
|
|
|
2019-11-27 10:28:37 -05:00
|
|
|
Once 3+ binding votes (by IPMC members) have been cast and at
|
|
|
|
least 72 hours have past, you can post a [RESULT] thread
|
2019-10-30 13:24:18 -04:00
|
|
|
|
2019-11-27 10:28:37 -05:00
|
|
|
To easily send the result email, still on the `superset/RELEASING` directory:
|
2019-04-15 15:48:51 -04:00
|
|
|
|
2019-11-25 05:01:31 -05:00
|
|
|
```bash
|
2020-02-09 20:53:56 -05:00
|
|
|
# Note: use Superset's virtualenv
|
2019-11-27 10:28:37 -05:00
|
|
|
(venv)$ python send_email.py result_ipmc
|
|
|
|
```
|
|
|
|
|
|
|
|
Again, the script will interactively ask for extra information needed to fill out the email template. Based on the
|
2020-02-09 20:53:56 -05:00
|
|
|
voting description, it will generate a passing, non passing or non conclusive email.
|
2019-11-27 10:28:37 -05:00
|
|
|
here's an example:
|
|
|
|
|
|
|
|
```bash
|
|
|
|
Sender email (ex: user@apache.org): your_apache_email@apache.org
|
|
|
|
Apache username: your_apache_user
|
|
|
|
Apache password: your_apache_password
|
|
|
|
A List of people with +1 binding vote (ex: Alan, Justin): Alan,Jeff,
|
|
|
|
A List of people with +1 non binding vote (ex: Ville):
|
|
|
|
A List of people with -1 vote (ex: John):
|
2019-11-25 05:01:31 -05:00
|
|
|
```
|
|
|
|
|
2019-11-27 10:28:37 -05:00
|
|
|
|
2019-10-23 09:43:04 -04:00
|
|
|
### Validating a release
|
2019-04-20 13:26:01 -04:00
|
|
|
|
|
|
|
https://www.apache.org/info/verification.html
|
|
|
|
|
|
|
|
## Publishing a successful release
|
|
|
|
|
2019-10-23 09:43:04 -04:00
|
|
|
Upon a successful vote (community AND IPMC), you'll have to copy the folder
|
|
|
|
into the non-"dev/" folder.
|
2019-04-15 15:48:51 -04:00
|
|
|
```bash
|
2019-10-23 09:43:04 -04:00
|
|
|
cp -r ~/svn/superset_dev/${SUPERSET_VERSION_RC}/ ~/svn/superset/${SUPERSET_VERSION}/
|
2019-04-15 15:48:51 -04:00
|
|
|
cd ~/svn/superset/
|
2019-10-23 09:43:04 -04:00
|
|
|
# Rename the RC (0.34.1rc1) to the actual version being released (0.34.1)
|
|
|
|
for f in ${SUPERSET_VERSION}/*; do mv "$f" "${f/${SUPERSET_VERSION_RC}/${SUPERSET_VERSION}}"; done
|
|
|
|
svn add ${SUPERSET_VERSION}
|
|
|
|
svn commit -m "Release ${SUPERSET_VERSION}"
|
|
|
|
```
|
|
|
|
|
|
|
|
Then tag the final release:
|
|
|
|
```bash
|
|
|
|
# Go to the root directory of the repo, e.g. `~/src/incubator-superset`
|
|
|
|
cd ~/src/incubator-superset/
|
|
|
|
# make sure you're on the correct branch (e.g. 0.34)
|
|
|
|
git branch
|
|
|
|
# Create the release tag
|
|
|
|
git tag -f ${SUPERSET_VERSION}
|
2019-04-15 15:48:51 -04:00
|
|
|
```
|
|
|
|
|
2019-11-27 10:28:37 -05:00
|
|
|
### Update CHANGELOG and UPDATING on superset
|
|
|
|
|
|
|
|
Now that we have a final Apache source release we need to open a pull request on Superset
|
|
|
|
with the changes on `CHANGELOG.md` and `UPDATING.md`.
|
2019-08-12 17:01:15 -04:00
|
|
|
|
2019-11-05 03:44:06 -05:00
|
|
|
### Publishing a Convenience Release to PyPI
|
2019-11-27 10:28:37 -05:00
|
|
|
|
2020-04-17 11:28:10 -04:00
|
|
|
Using the final release tarball, unpack it and run `./pypi_push.sh`.
|
|
|
|
This script will build the Javascript bundle and echo the twine command
|
|
|
|
allowing you to publish to PyPI. You may need to ask a fellow committer to grant
|
2019-11-05 03:44:06 -05:00
|
|
|
you access to it if you don't have access already. Make sure to create
|
|
|
|
an account first if you don't have one, and reference your username
|
|
|
|
while requesting access to push packages.
|
|
|
|
|
2019-11-27 10:28:37 -05:00
|
|
|
### Announcing
|
2019-09-10 13:36:07 -04:00
|
|
|
|
2019-11-27 10:28:37 -05:00
|
|
|
Once it's all done, an [ANNOUNCE] thread announcing the release to the dev@ mailing list is the final step.
|
|
|
|
|
|
|
|
```bash
|
2020-02-09 20:53:56 -05:00
|
|
|
# Note use Superset's virtualenv
|
2019-11-27 10:28:37 -05:00
|
|
|
(venv)$ python send_email.py announce
|
|
|
|
```
|
|
|
|
|
|
|
|
## Post release
|
2019-09-10 13:36:07 -04:00
|
|
|
|
2019-11-27 10:28:37 -05:00
|
|
|
#### Refresh documentation website
|
2019-08-12 17:01:15 -04:00
|
|
|
|
|
|
|
Every once in a while we want to compile the documentation and publish it.
|
|
|
|
Here's how to do it.
|
|
|
|
|
|
|
|
```bash
|
2019-12-03 04:37:33 -05:00
|
|
|
./make_docs.sh
|
|
|
|
```
|
2019-08-12 17:01:15 -04:00
|
|
|
|
2019-12-03 04:37:33 -05:00
|
|
|
Superset documentation site is ready at http://localhost:5002
|
2019-08-12 17:01:15 -04:00
|
|
|
|
2019-12-03 04:37:33 -05:00
|
|
|
```
|
|
|
|
$ cd /tmp/incubator-superset-site-${SUPERSET_VERSION}
|
|
|
|
$ git add .
|
|
|
|
$ git commit -a -m "New doc version ${SUPERSET_VERSION}"
|
|
|
|
$ git push origin asf-site
|
2019-08-12 17:01:15 -04:00
|
|
|
```
|