Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Way to trigger rescan on historical builds for a new pattern #12

Open
ezyang opened this issue Jun 11, 2019 · 2 comments
Open
Labels
enhancement New feature or request

Comments

@ezyang
Copy link

ezyang commented Jun 11, 2019

Go to https://circle.pytorch.org/pattern-details.html?pattern_id=59 It has only scanned 3.4% of builds. I want it to scan everything (updating our statuses accordingly). Is there a way to do that?

@kostmo kostmo added the enhancement New feature or request label Jun 11, 2019
@kostmo
Copy link
Owner

kostmo commented Jun 13, 2019

Since this can be a long-running task, I think I'll need to implement some logic to prevent multiple simultaneous scans from being started and have a way to view the realtime progress on the web UI

@kostmo
Copy link
Owner

kostmo commented Oct 14, 2019

Fleshing this out a bit more:

  • A web UI should be created that can queue the historical scan.
  • The UI should specify how far into the past that builds should be rescanned.
  • Currently SQS is being used for queuing contemporary build scans. A separate SQS queue should be created for historical scans.
    • In the long term, each time a new pattern is created, a new historical scan could be automatically initiated (with some cooldown delay to allow for the authoring of multiple patterns in a work session; it's more efficient to scan logs for multiple patterns at once).
  • A unique identifier for the scan action should be stored in the database and used to display in real time the amount of work done by this scan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants