Integrating auto with bump2version

2021-02-11T00:00:00-05:00

auto by Intuit lets you set up automatic creation of tags & releases and population of changelogs in a GitHub project. It takes care of determining the version number for new releases, but, by default, it does not set the new version number in your code. This isn’t a problem if your project uses something like setuptools_scm or versioneer to fetch the version number from Git, but if your project’s version number is hardcoded in your code, you’ll need another solution. bump2version is that solution, and it can be integrated into auto as follows.

Set up `bump2version`

To make your project’s repository usable with bump2version, create a .bumpversion.cfg (no “2”!) file as follows:

[bumpversion]
# Replace the value here with your project's current version number:
current_version = 0.1.0

commit = True

# If integrating with GitHub Actions, you'll need to include "[skip ci]" in
# the bump2version commit message in order to prevent auto from running
# unnecessarily:
message = [skip ci] Bump version: {current_version} → {new_version}

# auto will be taking care of the tagging for us, so set `tag` to False:
tag = False

# For each file in your code that contains your project's version number,
# add a `[bumpversion:file:PATH]` section header, like so:
[bumpversion:file:src/myproject/__init__.py]

With this file in place and bump2version installed, you can automatically update your project’s version number by running bump2version $PART, where $PART is the part of the version number to increase (major, minor, or patch).

See the bump2version documentation for more information.

Integrate with `auto`

I assume you’ve already set up auto for your repository. To get auto to run bump2version automatically at the right time when creating a new release, use the exec plugin to register an afterChangelog hook. Add the following to the "plugins" field in your repository’s .autorc file:

{
    ...
    "plugins": [
        ...
        [
            "exec",
            {
                "afterChangelog": "bump2version \"$(printf '%s\n' \"$ARG_0\" | jq -r .bump)\""
            }
        ]
    ]
}

Tip

Despite what the exec documentation may imply, the $ARG_0 environment variable is a JSON object containing the payload passed to the respective hook. This means we must extract the semantic version part to bump using jq.

Warning

Do not use echo in place of printf here! The $ARG_0 variable contains JSON containing strings with \n escape sequences, which echo would convert into newlines, resulting in invalid JSON.

With this setting in place, auto will run bump2version on each new release after populating the changelog, and the commit created by bump2version will be the commit to receive the new version tag.

Integrate with GitHub Actions

Assuming you’ve already set up a GitHub Actions workflow for running auto (see here for an example), the only addition needed to support bump2version is to install it before running auto:

- name: Set up Python
  uses: actions/setup-python@v2
  with:
    python-version: '^3.6'

- name: Install bump2version
  run: python -m pip install bump2version

Running Extra Steps after Releasing with auto in GitHub Actions

2020-10-26T00:00:00-04:00

Let’s say you’ve set up auto for your project via a GitHub Actions workflow, and now you want that workflow to carry out additional steps — such as building & uploading assets — whenever auto creates a new release. Let’s also say that none of the available plugins for auto covers your use-case and you’re not a JavaScript programmer, so you won’t be writing a new plugin to do what you want. How do you adjust your GitHub Actions workflow to run these extra steps at the right time? Read on to find out.

So you’ve got a GitHub Actions workflow for auto already set up. Presumably, it looks something liks this:

name: Auto-release on PR merge

on:
  # This is the closest thing to triggering on a PR merge, as long as you
  # don't push directly to master.
  push:
    branches:
      - master

jobs:
  auto-release:
    runs-on: ubuntu-latest
    if: "!contains(github.event.head_commit.message, 'ci skip') && !contains(github.event.head_commit.message, 'skip ci')"
    steps:
      - name: Checkout source
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Download latest auto
        run: |
          auto_download_url="$(curl -fsSL https://api.github.com/repos/intuit/auto/releases/latest | jq -r '.assets[] | select(.name == "auto-linux.gz") | .browser_download_url')"
          wget -O- "$auto_download_url" | gunzip > ~/auto
          chmod a+x ~/auto

      - name: Create release
        run: ~/auto shipit
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Now you want to insert further steps that will be triggered whenever auto shipit successfully creates a new release.

For simple cases — where the commands to run can be expressed in a single shell command — we can use auto’s exec plugin to run the desired commands via the afterRelease hook. For example, building & uploading a Python package for PyPI can be integrated into auto by adding the following item to the "plugins" field in the repository’s .autorc file:

{
    ...
    "plugins": [
        ...
        [
            "exec",
            {
                "afterRelease": "python -m build && twine upload/*"
            }
        ]
    ]
}

(This particular example assumes that the appropriate Python dependencies are already set up earlier in the workflow and that the twine username & password are passed as environment variables to the ~/auto shipit step.)

For more complex post-release activity which can only be implemented as GitHub Actions steps, we need something else. Just adding the steps and nothing else directly to the workflow won’t work, as auto shipit doesn’t always create a new release, such as when a pull request with a skip-release label is merged, or when a pull request without a release label is merged while onlyPublishWithReleaseLabel is set to true. So we need some logic to test whether there’s a new release.

Theoretically, one option would be to create a separate workflow that runs whenever a new tag is pushed or a new GitHub release is created, but this won’t work with an out-of-the-box setup; auto uses GITHUB_TOKEN to create the tag & release, and GitHub Actions workflows specifically do not trigger on events performed with a GITHUB_TOKEN. You could get this approach to work by passing auto a personal access token instead of GITHUB_TOKEN, but there’s a more direct way to make this work instead that also lets you keep your release-related workflow steps in a single file.

Here’s the key trick: By running auto version before auto shipit, you find out whether auto is about to create a new release, and you can save the output from auto version to use to trigger extra steps later. If auto is about to create a new release, auto version will output the release level (”patch”, “minor”, or “major”); otherwise, if auto shipit would do nothing, auto version just outputs an empty line.

Hence, insert the following step before the auto shipit step:

- name: Check whether a release is due
  id: auto-version
  run: |
    version="$(~/auto version)"
    echo "version=$version" >> "$GITHUB_OUTPUT"
  env:
    GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Here, we use a workflow command to make the output from auto version available to subsequent steps. Later steps can then be configured to only run if a new release is being made by adding an if field to them, like so:

- name: Build asset for new release
  run: ...
  if: steps.auto-version.outputs.version != ''

If you have multiple steps that you want to run after a release, adding an if field to all of them can become excessive; isn’t there a way to apply an if to a whole block of steps? There is indeed; you can split off the extra steps into a separate job in the same workflow and have that entire job be guarded by a single if.

First, in order to make the output from the auto version step available to other jobs in the workflow, you need to add an outputs field to the original auto job (the one that in the example above is named “auto-release”), at the same level as the runs-on and steps keys. This outputs field should contain a YAML object mapping a name for the output value to a ${{ steps.….outputs.… }} expression that evaluates to the output from the auto version step. For the example workflow configurations shown so far, this would mean a configuration like the following:

jobs:
  auto-release:
    runs-on: ubuntu-latest
    if: "!contains(github.event.head_commit.message, 'ci skip') && !contains(github.event.head_commit.message, 'skip ci')"
    ### vv Add this bit vv ###
    outputs:
      auto-version: ${{ steps.auto-version.outputs.version }}
    ### ^^ Add this bit ^^ ###
    steps:
      # ...

With this in place, a new job can be added to the workflow containing all the steps you want to run after a new release is made. This new job needs two special fields (at the same level as runs-on and steps):

A needs field containing the job ID of the auto job (so needs: auto-release for the examples given here) to declare a dependency on it
An if field containing an expression of the form needs.AUTO_JOB_NAME.outputs.AUTO_VERSION_OUTPUT_NAME != '' (so if: needs.auto-release.outputs.auto-version != '' for the examples given here); this causes the job to be skipped if auto version outputs nothing, i.e., if no release is made

The configuration for this new job would then look like:

jobs:

  # `auto release` job from above omitted
  # ...

  build-and-publish:
    runs-on: ubuntu-latest
    needs: auto-release
    if: needs.auto-release.outputs.auto-version != ''
    steps:
      # ...

There’s one more thing to be aware of: If you check out your repository in this new job, by default the HEAD will be the commit that triggered the workflow originally and will not include the changelog commit or tag created by auto. If you need the commit or tag (say, because your project uses setuptools_scm or the like to derive its version number from Git tags at build time), you can tell the actions/checkout action to check out the latest commit from the repository by passing the default branch as the ref input like so:

- name: Checkout source
  uses: actions/checkout@v4
  with:
    ref: master  # or `main` or whatever your default branch is
    # This setting is needed to fetch tags:
    fetch-depth: 0

This does come with a caveat, though: in the event that multiple commits or merges to the default branch were made in quick succession, you may end up checking out a later commit than the tag that auto created. If this is a problem, one way to deal with it is to specifically check out the tag for the latest GitHub release, like so:

- name: Get tag of latest release
  id: latest-release
  run: |
    latest_tag="$(curl -fsSL https://api.github.com/repos/$GITHUB_REPOSITORY/releases/latest | jq -r .tag_name)"
    echo "tag=$latest_tag" >> "$GITHUB_OUTPUT"

- name: Checkout source
  uses: actions/checkout@v4
  with:
    ref: ${{ steps.latest-release.outputs.tag }}
    fetch-depth: 0

This, of course, fails if auto creates multiple tags in quick succession. I’m not aware of a decent way to deal with this eventuality; how about listening to the docs and just not running auto that often in the first place?

Using all of these tricks, your final workflow configuration should look something like this:

name: Auto-release on PR merge

on:
  # This is the closest thing to triggering on a PR merge, as long as you
  # don't push directly to master.
  push:
    branches:
      - master

jobs:
  auto-release:
    runs-on: ubuntu-latest
    if: "!contains(github.event.head_commit.message, 'ci skip') && !contains(github.event.head_commit.message, 'skip ci')"
    outputs:
      auto-version: ${{ steps.auto-version.outputs.version }}
    steps:
      - name: Checkout source
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Download latest auto
        run: |
          auto_download_url="$(curl -fsSL https://api.github.com/repos/intuit/auto/releases/latest | jq -r '.assets[] | select(.name == "auto-linux.gz") | .browser_download_url')"
          wget -O- "$auto_download_url" | gunzip > ~/auto
          chmod a+x ~/auto

      - name: Check whether a release is due
        id: auto-version
        run: |
          version="$(~/auto version)"
          echo "version=$version" >> "$GITHUB_OUTPUT"
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

      - name: Create release
        run: ~/auto shipit
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}

  build-and-publish:
    runs-on: ubuntu-latest
    needs: auto-release
    if: needs.auto-release.outputs.auto-version != ''
    steps:
      - name: Get tag of latest release
        id: latest-release
        run: |
          latest_tag="$(curl -fsSL https://api.github.com/repos/$GITHUB_REPOSITORY/releases/latest | jq -r .tag_name)"
          echo "tag=$latest_tag" >> "$GITHUB_OUTPUT"

      - name: Checkout source
        uses: actions/checkout@v4
        with:
          ref: ${{ steps.latest-release.outputs.tag }}
          fetch-depth: 0

      # Remaining steps go here
      # ...

Enjoy!

Knowledge Bits - auto

Integrating auto with bump2version

Set up bump2version

Integrate with auto

Integrate with GitHub Actions

Running Extra Steps after Releasing with auto in GitHub Actions

Set up `bump2version`

Integrate with `auto`