Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only process maven repository once if it's the same #2982

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

akerimsenol
Copy link
Contributor

This should improve import performance for large projects

@nskvortsov nskvortsov self-assigned this Mar 7, 2025
@nskvortsov nskvortsov self-requested a review March 7, 2025 16:51
@@ -15,6 +15,9 @@ import java.util.*


class MavenRepositoriesProjectResolver: AbstractProjectResolverExtension() {

val processedRepositoryModels = IdentityHashMap<RepositoryModels, Unit>()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also storing state here will cause memory leak, as this class is a long-living service. It is not re-created on each sync.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I didn't realize that. Thanks! What would be the alternative? I've seen a pattern where this sort of cache was stored inside DataNode, is that what I should do?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clearing the map in an import finished callback should be fine, too, right?

@@ -15,6 +15,9 @@ import java.util.*


class MavenRepositoriesProjectResolver: AbstractProjectResolverExtension() {

val processedRepositoryModels = IdentityHashMap<RepositoryModels, Unit>()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the IdentityHashMap work as expected here?

At first glance, different instances of RepositoryModels can contain duplicates - different instances of MavenRepositoryModel that are logically equal, but have different references.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was my experience, but I don't fully understand how it ends up the same. I will double check and update here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For reference, I ran the debugger with a sample project with multiple modules logging the object ids and contents each time it runs and here it is showing they are all the same.
identity-debugger-output.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants