From 0d2e42229e4664c2f7a870dcdacea800f1fb674f Mon Sep 17 00:00:00 2001 From: Ville Brofeldt <33317356+villebro@users.noreply.github.com> Date: Fri, 27 May 2022 16:09:12 +0300 Subject: [PATCH] docs: update release instructions (#20210) --- RELEASING/README.md | 188 ++++++++++++++++++++++++++++++------- RELEASING/changelog.py | 6 ++ RELEASING/requirements.txt | 19 ++++ 3 files changed, 180 insertions(+), 33 deletions(-) create mode 100644 RELEASING/requirements.txt diff --git a/RELEASING/README.md b/RELEASING/README.md index 46913d55ec..e48728d1b8 100644 --- a/RELEASING/README.md +++ b/RELEASING/README.md @@ -67,6 +67,33 @@ need to be done at every release. svn update ``` +To minimize the risk of mixing up your local development environment, it's recommended to work on the +release in a different directory than where the devenv is located. In this example, we'll clone +the repo directly from the main `apache/superset` repo to a new directory `superset-release`: + +```bash +cd +git clone git@github.com:apache/superset.git superset-release +cd superset-release +``` + +We recommend setting up a virtual environment to isolate the python dependencies from your main +setup: + +```bash +virtualenv venv +source venv/bin/activate +``` + + +In addition, we recommend using the [`cherrytree`](https://pypi.org/project/cherrytree/) tool for +automating cherry picking, as it will help speed up the release process. To install `cherrytree` +and other dependencies that are required for the release process, run the following commands: + +```bash +pip install -r RELEASING/requirements.txt +``` + ## Setting up the release environment (do every time) As the vote process takes a minimum of 72h, sometimes stretching over several weeks @@ -78,35 +105,39 @@ the wrong files/using wrong names. There's a script to help you set correctly al necessary environment variables. Change your current directory to `superset/RELEASING` and execute the `set_release_env.sh` script with the relevant parameters: + +Usage (MacOS/ZSH): +```bash +cd RELEASING +source set_release_env.sh +``` + Usage (BASH): ```bash . set_release_env.sh ``` -Usage (ZSH): -```bash -source set_release_env.sh -``` - Example: ```bash -source set_release_env.sh 0.38.0rc1 myid@apache.org +source set_release_env.sh 1.5.1rc1 myid@apache.org ``` -The script will output the exported variables. Here's example for 0.38.0rc1: +The script will output the exported variables. Here's example for 1.5.1rc1: ``` +------------------------------- Set Release env variables -SUPERSET_VERSION=0.38.0 +SUPERSET_VERSION=1.5.1 SUPERSET_RC=1 -SUPERSET_GITHUB_BRANCH=0.38 -SUPERSET_PGP_FULLNAME=myid@apache.org -SUPERSET_VERSION_RC=0.38.0rc1 -SUPERSET_RELEASE=apache-superset-0.38.0 -SUPERSET_RELEASE_RC=apache-superset-0.38.0rc1 -SUPERSET_RELEASE_TARBALL=apache-superset-0.38.0-source.tar.gz -SUPERSET_RELEASE_RC_TARBALL=apache-superset-0.38.0rc1-source.tar.gz -SUPERSET_TMP_ASF_SITE_PATH=/tmp/superset-site-0.38.0 +SUPERSET_GITHUB_BRANCH=1.5 +SUPERSET_PGP_FULLNAME=villebro@apache.org +SUPERSET_VERSION_RC=1.5.1rc1 +SUPERSET_RELEASE=apache-superset-1.5.1 +SUPERSET_RELEASE_RC=apache-superset-1.5.1rc1 +SUPERSET_RELEASE_TARBALL=apache-superset-1.5.1-source.tar.gz +SUPERSET_RELEASE_RC_TARBALL=apache-superset-1.5.1rc1-source.tar.gz +SUPERSET_TMP_ASF_SITE_PATH=/tmp/incubator-superset-site-1.5.1 +------------------------------- ``` ## Crafting a source release @@ -116,23 +147,101 @@ a branch named with the release MAJOR.MINOR version (on this example 0.37). This new branch will hold all PATCH and release candidates that belong to the MAJOR.MINOR version. -The MAJOR.MINOR branch is normally a "cut" from a specific point in time from the master branch. -Then (if needed) apply all cherries that will make the PATCH. +### Creating an initial minor release (e.g. 1.5.0) +The MAJOR.MINOR branch is normally a "cut" from a specific point in time from the master branch. +When creating the initial minor release (e.g. 1.5.0), create a new branch: ```bash -git checkout -b $SUPERSET_GITHUB_BRANCH -git push upstream $SUPERSET_GITHUB_BRANCH +git checkout master +git pull +git checkout -b ${SUPERSET_GITHUB_BRANCH} +git push origin $SUPERSET_GITHUB_BRANCH ``` -Next, update the `CHANGELOG.md` with all the changes that are included in the release. -Make sure the branch has been pushed to `upstream` to ensure the changelog generator -can pick up changes since the previous release. -Change log script requires a github token and will try to use your env var GITHUB_TOKEN. -you can also pass the token using the parameter `--access_token`. +Note that this initializes a new "release cut", and is NOT needed when creating a patch release +(e.g. 1.5.1). + +### Creating a patch release (e.g. 1.5.1) + +When getting ready to bake a patch release, simply checkout the relevant branch: -Example: ```bash -python changelog.py --previous_version 0.37 --current_version 0.38 changelog +git checkout master +git pull +git checkout ${SUPERSET_GITHUB_BRANCH} +``` + +### Cherry picking + +It is customary to label PRs that have been introduced after the cut with the label +`v.`. For example, for any PRs that should be included in the 1.5 branch, the +label `v1.5` should be added. + +To see how well the labelled PRs would apply to the current branch, run the following command: + +```bash +cherrytree bake -r apache/superset -m master -l v${SUPERSET_GITHUB_BRANCH} ${SUPERSET_GITHUB_BRANCH} +``` + +This requires the presence of an environment variable `GITHUB_TOKEN`. Alternatively, +you can pass the token directly via the `--access-token` parameter (`-at` for short). + +#### Happy path: no conflicts + +This will show how many cherries will apply cleanly. If there are no conflicts, you can simply apply all cherries +by adding the `--no-dry-run` flag (`-nd` for short): + +```bash +cherrytree bake -r apache/superset -m master -l v${SUPERSET_GITHUB_BRANCH} -nd ${SUPERSET_GITHUB_BRANCH} +``` + + +#### Resolving conflicts + +If there are conflicts, you can issue the following command to apply all cherries up until the conflict automatically, and then +break by adding the `-error-mode break` flag (`-e break` for short): + +```bash +cherrytree bake -r apache/superset -m master -l v${SUPERSET_GITHUB_BRANCH} -nd -e break ${SUPERSET_GITHUB_BRANCH} +``` + +After applying the cleanly merged cherries, `cherrytree` will specify the SHA of the conflicted cherry. To resolve the conflict, +simply issue the following command: + +```bash +git cherry-pick +``` + +Then fix all conflicts, followed by + +```bash +git add -u # add all changes +git cherry-pick --continue +``` + +After this, rerun all the above steps until all cherries have been picked, finally pushing all new commits to the release branch +on the main repo: + +```bash +git push +``` + +### Updating changelog + +Next, update the `CHANGELOG.md` with all the changes that are included in the release. +Make sure the branch has been pushed to `origin` to ensure the changelog generator +can pick up changes since the previous release. +Similar to `cherrytree`, the change log script requires a github token, either as an env var +(`GITHUB_TOKEN`) or as the parameter `--access_token`. + +#### Initial release (e.g. 1.5.0) + +When generating the changelog for an initial minor relese, you should compare with +the previous release (in the example, the previous release branch is `1.4`, so remember to +update it accordingly): + +```bash +python changelog.py --previous_version 1.4 --current_version ${SUPERSET_GITHUB_BRANCH} changelog ``` You can get a list of pull requests with labels started with blocking, risk, hold, revert and security by using the parameter `--risk`. @@ -141,16 +250,29 @@ Example: python changelog.py --previous_version 0.37 --current_version 0.38 changelog --access_token {GITHUB_TOKEN} --risk ``` -The script will checkout both branches and compare all the PR's, copy the output and paste it on the `CHANGELOG.md` +The script will checkout both branches, compare all the PRs, and output the lines that are needed to be added to the +`CHANGELOG.md` file in the root of the repo. Remember to also make sure to update the branch id (with the above command +`1.5` needs to be changed to `1.5.0`) Then, in `UPDATING.md`, a file that contains a list of notifications around deprecations and upgrading-related topics, make sure to move the content now under the `Next Version` section under a new section for the new release. -Finally bump the version number on `superset-frontend/package.json` (replace with whichever version is being released excluding the RC version): +#### Patch release (e.g. 1.5.1) -```json +To compare the forthcoming patch release with the latest release from the same branch, set +`--previous_version` as the tag of the previous release (in this example `1.5.0`; remember to update accordingly) + +```bash +python changelog.py --previous_version 1.5.0 --current_version ${SUPERSET_GITHUB_BRANCH} changelog +``` + +### Set version number + +Finally, bump the version number on `superset-frontend/package.json` (replace with whichever version is being released excluding the RC version): + +``` "version": "0.38.0" ``` @@ -162,7 +284,7 @@ git add ... git commit ... # push new tag git tag ${SUPERSET_VERSION_RC} -git push upstream ${SUPERSET_VERSION_RC} +git push origin ${SUPERSET_VERSION_RC} ``` ## Preparing the release candidate @@ -180,7 +302,7 @@ the tag and create a signed source tarball from it: Note that `make_tarball.sh`: -- By default assumes you have already executed an SVN checkout to `$HOME/svn/superset_dev`. +- By default, the script assumes you have already executed an SVN checkout to `$HOME/svn/superset_dev`. This can be overridden by setting `SUPERSET_SVN_DEV_PATH` environment var to a different svn dev directory - Will refuse to craft a new release candidate if a release already exists on your local svn dev directory - Will check `package.json` version number and fails if it's not correctly set @@ -289,7 +411,7 @@ git branch # Create the release tag git tag -f ${SUPERSET_VERSION} # push the tag to the remote -git push upstream ${SUPERSET_VERSION} +git push origin ${SUPERSET_VERSION} ``` ### Update CHANGELOG and UPDATING on superset diff --git a/RELEASING/changelog.py b/RELEASING/changelog.py index 5d4f346c8e..0729853ba5 100644 --- a/RELEASING/changelog.py +++ b/RELEASING/changelog.py @@ -164,6 +164,12 @@ class GitChangeLog: return False def _get_changelog_version_head(self) -> str: + if not len(self._logs): + print( + f"No changes found between revisions. " + f"Make sure your branch is up to date." + ) + sys.exit(1) return f"### {self._version} ({self._logs[0].time})" def _parse_change_log( diff --git a/RELEASING/requirements.txt b/RELEASING/requirements.txt new file mode 100644 index 0000000000..bd3586e04c --- /dev/null +++ b/RELEASING/requirements.txt @@ -0,0 +1,19 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +cherrytree +jinja2