get changelog from npm

[k-yle]

What would you like Renovate to be able to do?

To be able to fetch the changelog from the npm package itself, if renovate can't find it on GitHub or GitLab.

If npm sees a CHANGELOG file, it will always publish it. This makes it potentially a more reliable than searching through the repo for tags, releases or changelog-like files.

This had advantages over the current approach because:

• It works even if renovate can't find the source repo
• It works even if renovate can't guess the release/tag version
• It works if the git release versions don't match the npm versions
• It works for packages in monorepos (see Improve changelog detection for monorepos #2926), even if they don't specify repository.directory in package.json (like @aws-amplify/auth)
• It works for public packages that live in private repos
• It works packages that don't live on GitHub or GitLab

typedoc-plugin-markdown and @aws-amplify/auth are some examples where the current approach fails, but fetching from NPM would work.

Did you already have any implementation ideas?

If getting the changelog from GitHub or GitLab fails, use

https://unpkg.com/{{package}}@{{version}}/CHANGELOG.md

[rarkins]

I think this has enough positives that it's worth doing. Some downsides to note:

1. Releases are immutable, so the CHANGELOG entry for a release will never be added (if missing) or updated (if incorrect or incomplete) after publication
2. We are relying on yet-another-external-service in the form of unpkg
3. It's npm-only, naturally

In my experience CHANGELOG entries often are updated post-release.

Given these points, I wouldn't replace any of our existing logic with this, but instead supplement it just specifically for the npm case. I think we could use it as a fallback for fetching the changelog file if we can't fetch it through our usual means (e.g. it's on a supported platform, and we are able to locate the file).

What do you think?

[k-yle]

Sounds great @rarkins!

As you said, I think it would be valuable to have npm as a fallback source if the better options (GitHub/GitLab) fail.

1. Releases are immutable, so the CHANGELOG entry for a release will never be added (if missing) or updated (if incorrect or incomplete) after publication

That's definitely a drawback, but if this is the last resort then I think showing something is better than nothing. And if the package has a CI process to generate a changelog alongside the release then this won't be an issue.

[rarkins]

Yes, I agree. There are still a few unknowns to explore here - for example we usually try to avoid datasource-specific logic in lib/workers/* files - but I think this is acceptable to move ahead to development with, if you or someone else is willing.

[marcodejongh]

This will also fix changelog generation for packages in public npm, published from a private repo.
I wish there was an alternative to unpkg though, as we also have private npm packages (in our own registry) that we'd like changelog generation for. But just fixing the public usecase would be a huge win.

I'd be willing to help out. Are you suggesting @rarkins that we just add fetch changelog last attempt in the global changelog fetching logic? Which I think is in: lib/workers/pr/changelog/index.ts ?

[viceice]

Yes, i think here

renovate/lib/workers/pr/changelog/index.ts

Line 39 in 74d7691

return res;

something like:

if (!res) {
  res = await sourceUnpkg.getChangeLogJSON({ ...args, releases });
}

[github-actions[bot]]

Hi there,

Help us by making a minimal reproduction repository.

Before we can start work on your issue we first need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction to understand what is needed.

We may close the issue if you (or someone else) have not provided a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

[rarkins]

A reproduction is still needed which demonstrates an update where the changelog is not found by Renovate today but is available via npm/unpkg

[cjpearson]

I have one possible example. We publish several packages publicly out of a larger, private repository. The changelog files are included in the npm package and can be download from unpkg.

This is how the renovate updates look: https://github.com/cjpearson/renovate-demo/pull/11
And the changelog: https://unpkg.com/@scayle/storefront-nuxt/CHANGELOG.md

If renovate is able to able to pull the changes from the npm package it would be a nice improvement for us.

[khitrenovich]

Quote from now-closed #28007 -

Some packages are public, but not fully open-sourced - meaning that they are available to fetch from NPM, but their source is not open to public. A good example is @stytch/react - their CHANGELOG.md is available on https://www.npmjs.com/package/@stytch/react?activeTab=code and can be seen in the code diff, but does not appear in Renovate PRs. It would be great to have it fetched!

[thomaswinckell]

I have the same need but not for the same reason. In a closed environment where Renovate only has access to an internal npm registry that is a proxy to npmjs.org (no access to internet, so no access to Github or Gitlab), this could be the solution for retrieving changelogs.

Maybe we could have something like "if no github.com token has been configured, then retrieve changelogs from npm" ?