Blog post/reflection on semantic-release

[rarkins]

Renovate is rapidly approaching 5,000 releases (GitHub already rounds up to "5k"). Could be interesting to do a case study/reflection on using semantic-release and the pros/cons, especially if those in the wider community beyond Renovate maintainers would be interested in contributing/participating to the discussion? Also relevant these days to discuss the increased risk of automation without 2FA etc.

@JamieMagee @viceice @hutson @gr2m @pvdlg @travi @JustinBeckwith

[viceice]

Idea: do weekly releases with cron and manual releases on demand, eg Bugfix. Can be easily configured with actions. 🤔

[HonkingGoose]

Renovate is rapidly approaching 5,000 releases (GitHub already rounds up to "5k"). Could be interesting to do a case study/reflection on using semantic-release and the pros/cons, especially if those in the wider community beyond Renovate maintainers would be interested in contributing/participating to the discussion? Also relevant these days to discuss the increased risk of automation without 2FA etc.

I really like this idea! I'm very impressed with the level of "creating a release" automation that we have on this repository (and on the related ones). I would read a good post about working with semantic-release and the pros/cons from the perspective of a power-user. 😄

Would be even cooler if you could get help with creating the post from the semantic-release maintainers. That way you could really make a nice and correct showcase post.

---

Idea: do weekly releases with cron and manual releases on demand, eg Bugfix. Can be easily configured with actions. 🤔

I'm sorry @viceice but I don't like this idea much. With your proposal we re-introduce error prone manual steps into the release process, where right now it's "set the correct PR title, and let the semantic-release bot worry about making the release". I get where you're coming from, but your proposal does not fix the underlying problem.

My view of the real problem with frequent releases

The main problem is that we're releasing so frequently that each changelog might only have one fix/feature. This makes it hard to get a good overview of all changes. The GitHub user interface does not make it easy to grab a "diff" of all changelogs between say version 24.0.0 and version 25.3.2.

The answer is not to slow down on our releases, but instead make it easier to get a "changelog-diff" between the current version of Renovate and the version to which the user is upgrading.

User can get a full changelog on demand

I'm thinking of having a section on our Renovate docs website, where a user can select what version they have now, and what version they're upgrading to. Via some code-magic we transform the separate changelogs into one big changelog.

It would look something like this:

List of changes between Renovate version `24.0.0` and `25.3.2`

## Breaking changes

- Breaking change 1
- Breaking change 2

## Features

- Feature 1
- Feature 2

## Bugfixes

- Bugfix 1
- Bugfix 2

## Documentation

- Documentation change 1
- Documentation change 2

etc, etc.

[Chumper]

User can get a full changelog on demand

I'm thinking of having a section on our Renovate docs website, where a user can select what version they have now, and what version they're upgrading to. Via some code-magic we transform the separate changelogs into one big changelog.

It would look something like this:

List of changes between Renovate version `24.0.0` and `25.3.2`

## Breaking changes

- Breaking change 1
- Breaking change 2

## Features

- Feature 1
- Feature 2

## Bugfixes

- Bugfix 1
- Bugfix 2

## Documentation

- Documentation change 1
- Documentation change 2

etc, etc.

I like this idea, you could then add that to the documentation page.
When you also list the version the hosted app currently has, then it would be even more useful.
I would keep the semantic release workflow, release all documentation on main immediately and provide the mentioned select fields.

In that case documentation is up to date with main, but people see what version is the hosted one using (could be selected by default) and change it if they want.

[travi]

Count me in. I'm happy to be part of the discussion.

[rarkins]

Related: https://github.com/Belco90/octoclairvoyant by @Belco90, who has volunteered in #8177 to advise/assist on building merged release notes into Renovate.

Example changelog: https://octoclairvoyant.vercel.app/?repo=renovatebot%2Frenovate&from=23.95.0&to=24.22.3

[hutson]

I would read a good post about working with semantic-release and the pros/cons from the perspective of a power-user.

I would also appreciate an in-depth assessment of Renovate's experience using an automated release process over an extended period of time. For what it is worth, a few points I would look for: - What were the original pain points that led to the adoption of an automated release workflow? - To what extent were those issues addressed? - Were any found to be none-issues after the fact? - What downsides are there to an automated release process? - How have those been mitigated, and where do we still need help? - Did Renovate observe a change in the perception of the Renovate project as a result if adopting automated releases? One suggestion I would like to put out there is to do the initial assessment without the input of the `semantic-release` team. I believe the `semantic-release` team will have valuable input on the process taken by Renovate, but I worry that may skew the assessment. I feel the community could benefit from an original perspective on automated releases, followed by a separate constructive response from the `semantic-release` team on the experiences of Renovate.

[hutson]

Also relevant these days to discuss the increased risk of automation without 2FA etc.

@rarkins if it's possible, I would like to see a review of automated release processes from your unique dual perspective. You have experience setting up an automatic release process as an Open Source developer, while now having the perspective of what it's like working in the security field living with the reality of near constant supply chain attacks.

[HonkingGoose]

Example changelog: https://octoclairvoyant.vercel.app/?repo=renovatebot%2Frenovate&from=23.95.0&to=24.22.3

Wow, that octoclairvoyant page is great! 🎉 ✨ That's very close to what I had in mind!

I do have some ideas for tweaks to make it more readable, but maybe that discussion should be moved into the linked issue (#8177)?

[Belco90]

Related: https://github.com/Belco90/octoclairvoyant by @Belco90, who has volunteered in #8177 to advise/assist on building merged release notes into Renovate.

Example changelog: https://octoclairvoyant.vercel.app/?repo=renovatebot%2Frenovate&from=23.95.0&to=24.22.3

Hey! I'm happy to be part of the discussion too if necessary, regarding octoclairvoyant or semantic-release in general.

Wow, that octoclairvoyant page is great! 🎉 ✨ That's very close to what I had in mind!

I do have some ideas for tweaks to make it more readable, but maybe that discussion should be moved into the linked issue (#8177)?

Nice! I still have tons of ideas too, but I have been really busy lately with eslint-plugin-testing-library. I hope I can go back to octoclairvoyant soon, it's my favourite side project so far! Anyway, probably better to keep that discussion in the other issue indeed.

[JustinBeckwith]

I have so many thoughts here :) On my personal OSS, I happily use semantic-release, and I believe pretty strongly in the benefits. At Google though, we need more control and customizability over our processes, and support for many languages at the same time. The whole experience led to @bcoe bootstrapping the release-please ecosystem:

• https://github.com/googleapis/release-please
• https://github.com/google-github-actions/release-please-action

release-please takes a very similar approach to semantic-release, but instead of automatically cutting releases it creates a Release PR with the proposed changelog changes in a rolling fashion. This lets the project owner decide when it's time to cut the release, so multiple changes can be lumped together.

I'd be happy to collaborate here!

[JamieMagee]

I like the idea of octoclairvoyant. It could be useful to add a link to it to the release notes, with the to version filled out automatically. However, I can also see a few downsides:

• Adding a new external dependency
• I assume that most users aren't very aware of what version they're running, so it's an extra step to find the from version

release-please also seems to solve the problem, and more fits @viceice's idea of a 'manual' process.

Another thought: Could we also slow down the releases by extracting semantic-release from the main branch build, and run it in a separate GitHub Action on a cron job with optional manual trigger? I think that's the exact workflow @viceice was describing (Correct me if I'm wrong). What do you think @HonkingGoose?

[viceice]

Yes, i don't want to change the release workflow in general. Just do not run semantic release every main commit. Instead run it via cron or manually. 😉

[rarkins]

Good to identify "what are the problems we are trying to solve?".

We know there are multiple advantages to per-commit releasing:

• Minimal maintainer work
• Allows regressions to be identified to the commit (release)
• Users gets features/fixes immediately
• If the app is updated and a regression is identified in that latest release, we only roll that one feature/fix back and not potentially many

Next is to collect the disadvantages and work out:

• Are there alternative solutions than "not semantic releasing"? (e.g. bundling of release notes)
• Are these disadvantages enough that it's worth the trade-off of losing the advantages?

Something I'd like to do better is fix regression problems in the same feature stream which introduced them. e.g. if the addition of feature X in 25.2.0 causes a bug in feature Y, I would prefer that users don't need to update to 25.3.x or higher to get that fix. The reason being that I don't want people to have to keep "doubling down" on even more feature releases just to fix something. However that could be quite challenging, especially for an open source project. It would need something like:

• Ability for more dynamic releasing (e.g. without predefined/configure branch names for every stream)
• Ability to specify "cherry pick back to..." a certain stream and the system automatic backports it as far back as there (if it can)

In other words the promised land where stream improves its stability.

[HonkingGoose]

release-please also seems to solve the problem, and more fits @viceice's idea of a 'manual' process.

Another thought: Could we also slow down the releases by extracting semantic-release from the main branch build, and run it in a separate GitHub Action on a cron job with optional manual trigger? I think that's the exact workflow @viceice was describing (Correct me if I'm wrong). What do you think @HonkingGoose?

I don't like the idea of a cron job, as now you run the risk that the cron job decides to release right when you're busy fixing a regression/egregious bug, just because "it's time to release now".

If we want to go the "manual/semi-automated" route, I'd much prefer that a human decides when it's time to release. Say by merging the release-please PR or pushing the "release button" on the action.

---

Something I'd like to do better is fix regression problems in the same feature stream which introduced them. e.g. if the addition of feature X in 25.2.0 causes a bug in feature Y, I would prefer that users don't need to update to 25.3.x or higher to get that fix. The reason being that I don't want people to have to keep "doubling down" on even more feature releases just to fix something. However that could be quite challenging, especially for an open source project.

I worry this will get complicated very fast for people like me who want to contribute to the documentation or make a quick fix to the current Renovate behavior. Right now it's very easy: target main. If we have multiple branches, it might get confusing on which branch you need to target with your proposed changes.

Docusaurus v2 has multiple branches for their documentation for various releases, and it's confusing to me to find out which branch(es) I should target with documentation fixes. For me, this means I don't contribute to their documentation, even though I use and like Docusaurus v2, and want to give back to the project! 😞

---

As we're talking about "Methods of releasing source code to production", I strongly recommend that you read/skim this excellent article that Martin Fowler wrote on "Patterns for Managing Source Code Branches", especially the section on "The path from mainline to production release".

https://martinfowler.com/articles/branching-patterns.html

https://martinfowler.com/articles/branching-patterns.html#path-to-production

This might help clarify what branching workflows there are in general, and the pros and cons of each approach.

I'd vote: "Keep things simple". So either release each time we commit to main (like we do now), or follow the "Release-Ready Mainline" pattern, and use release-please to automate the cutting of the release:

Keep mainline sufficiently healthy that the head of mainline can always be put directly into production

[hutson]

Did Renovate observe a change in the perception of the Renovate project as a result if adopting automated releases? - @hutson
(e.g. bundling of release notes) - @rarkins

While not a change in perception of Renovate's product direction or quality, I have felt that Renovate's release cadence has been a point of contention (The issue that came to mind was #444).

This thread appears to validate that point as several posts have touched on how to answer what has changed between the distant releases, or how to reduce the cognitive overhead required to mentally group changes together into an accurate, and meaningful, view of what changes the downstream user will consume.

[hutson]

We know there are multiple advantages to per-commit releasing:

I would like to add my own point to that list. I want my upstream products to release following a predictable process. semantic-release is one such tool to achieve predictable releases.

As a technical team lead I often find myself with a buggy upstream product and the responsibility to decide: do I roll forward to a new feature release in hopes that addresses the bug, do I wait until the upstream product publishes a patch, or do I allocate my team's time to code a workaround. I only have finite resources (Team members and a tight schedule) and a responsibility to downstream users. Answering these questions is exhaustive. Products that have a predictable release cadence tend to be the easier products to reason about.

[hutson]

Something I'd like to do better is fix regression problems in the same feature stream which introduced them. - @rarkins

I would ideally like this as well. I'm responsible for supporting enterprise workflows, and that usually means optimizing for mean-time-to-recovery.

If we find 25.2.0 contains a bug that disrupts our company's employees, a clock starts counting the time our company is not generating as much value as it could be if employees weren't disrupted by the bug. If a fix is introduced as a patch, such as 25.2.1, it's easy to reason about the time it will take to recover from the current incident. If the answer is to roll forward, we are now introducing more variables such as, will the new feature contain a new/different bug, etc.. That only increases the risk of addressing the original problem in a given time.

Ability to specify "cherry pick back to..." a certain stream and the system automatic backports it as far back as there (if it can) - @rarkins

Unless the bug only exists in a previous release. Though, I guess if someone reports a bug in an earlier release, you patch that specific version and then attempt to automatically apply the patch to all feature releases before and after the reported version.

[gr2m]

Thanks for opening this discussion! There are some pain points that were already mentioned that @travi and I hope to address in semantic-release. Please keep 'em coming :)

One thing I'd be interested in is the composability of semantic-release. Do y'all have any opinions of the shared config setup of semantic-release, versus alternative approaches to composability?

[rarkins]

I think the compsability is a very good thing, certainly it's power-user friendly. No opinion on the shared config approach