feat: extend conda data source #34681
Conversation
| @@ -43,12 +47,22 @@ export class CondaDatasource extends Datasource { | |||
| return null; | |||
| } | |||
|
|
|||
| if (registryUrl.startsWith('https://prefix.dev/')) { | |||
| // the registryUrl here will at least contains 3 `/` , | |||
| // therefore channel won't be undefined in any case. | |||
There was a problem hiding this comment.
you can't be sure, a user could made a mistake. so catch it and add a test.
There was a problem hiding this comment.
I'm sure becasue this is in the if(registryUrl.startsWith('https://prefix.dev/')), jf user made a mistake with registryUrl='' it won't go into this code path.
It's impossible that a registryUrl starts with https://prefix.dev/ and it doesn't have 3 /, right?
Sorry, this is the previous conversation about this PR, I should put in the PR description. short version: prefix.dev is also a conda repo and it has a graphql API, so when the registry URL is prefix.dev we use graphql to query it. |
| const files = await this.getPrefixPagedResponse( | ||
| ` | ||
| query search($channel: String!, $package: String!, $page: Int = 0) { | ||
| data: package(channelName: $channel, name: $package) { | ||
| data: variants(limit: 500, page: $page) { | ||
| pages | ||
| page { | ||
| version | ||
| createdAt | ||
| yankedReason | ||
| } | ||
| } | ||
| } | ||
| } | ||
| `, |
There was a problem hiding this comment.
Why are you not pulling this in the first API call?
There was a problem hiding this comment.
like the schema suggest, this is listing all files of a package, so it has different total page count with first api call. One version will contains multiple files (called variants here), and package/version doesn't have createdAt and yankedReason, only package/variant have it, so we need to pull variants and get the yankedReason and createdAt. This is same logic with api.anaconda.
There was a problem hiding this comment.
It's possible to merge these 2 queries but I don't want to spend that mental energy to do it 😅
I think current implementation is much more easy to understand, let's refactor it when it cause trouble.
| 'lookup package from prefix.dev graphql API', | ||
| ); | ||
|
|
||
| const versions = await getPagedResponse( |
There was a problem hiding this comment.
prettier will be possible to format graphql if we have gql tag here, but graphql-tag is currently not in our dep tree so I didn't add it.
|
It wil fetch 500 records of versions and 500 records of files from prefix.dev graphql API. the files contribute to the It should be able to fetch all version of most package(I don't know if there is a package has more than 500 versions). But many package have more than 500 files, for example, numpy currently have 2800 files and 500 files will cover from 2.2.3 to 1.26.3. If we only select platform "linux-64" it has 692 records and will cover from 2.2.3 to 1.14.2. |
Changes
add prefix.dev support
add releaseTimestamp support
Context
Documentation (please check one with an [x])
How I've tested my work (please select one)
I have verified these changes via: