Added notebook example for virtual try-on using Gemini 2.0 and Imagen 3

[NSTiwari]

Added notebook example for virtual try-on using Gemini 2.0's 2D spatial understanding and Imagen 3's image generation/editing capabilities.

@Giom-V: Could you please review?

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[Giom-V]

I'll review it tomorrow

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

markmcd commented on 2025-03-05T09:00:09Z
----------------------------------------------------------------

Can you make sure embedded images are supplied in the cookbook repo too? If needed they can go in an assets/ or files/ dir.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

markmcd commented on 2025-03-05T09:00:10Z
----------------------------------------------------------------

Can you include a Step 0 or "Before you begin" section that includes everything the user needs to do?

It looks like they need a Gemini API key (a fair assumption for this repo) but also a GCP account with certain features turned on (billing, vertex APIs, etc).

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

markmcd commented on 2025-03-05T09:00:11Z
----------------------------------------------------------------

Small nit, but if you re-run this at all, can you clear the outputs for the pip install cells? It's pretty noisy & we don't need it.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

markmcd commented on 2025-03-05T09:00:11Z
----------------------------------------------------------------

Line #2.    %cd /content/segment-anything-2

There's a few Colab-specific things in this notebook. Can you either fix them (might be a lot of work) or just put a note up top saying that this only works on Google Colab?

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

markmcd commented on 2025-03-05T09:00:12Z
----------------------------------------------------------------

By default the user is downloading all of the checkpoints, but the text indicates that they only need 1. Can you set this up to only download one, but still make it clear that they can switch to a different model? (e.g. a drop-down or enum or something)

Edit: actually maybe this is fine if the download is quick enough on Colab.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

markmcd commented on 2025-03-05T09:00:13Z
----------------------------------------------------------------

Did you write all of this code from scratch? or was it taken from another source?

I see some of the comments match code used in other notebooks. We just need to make sure it's properly attributed if you didn't write it.

markmcd commented on 2025-03-06T09:40:38Z
----------------------------------------------------------------

Ah I see - it's fine if it's code that is under the same license/copyright as the rest of this repo. We just need to make sure anything from an external source is properly attributed. Thanks for adding the comments.

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

markmcd commented on 2025-03-05T09:00:14Z
----------------------------------------------------------------

Line #80.        image = PILImage.fromarray(image)

The last lines of the draw_bounding_boxes function assume there is one image and one coordinates, but the code runs in a loop over bounding_boxes.

If this is only meant to return the first bbox, then put return inside the loop, otherwise you'll need to figure out how to handle multiple (and hoist coordinates up to the right scope)

[review-notebook-app]

View / edit / reply to this conversation on ReviewNB

markmcd commented on 2025-03-05T09:00:15Z
----------------------------------------------------------------

Line #1.    input_image = 'image_01.png' # @param {type : 'string'}

This image doesn't exist in a basic runthrough. Can you provide a default image so "run all" works? If you're contributing and donating the license for the image embedded in the output, you could use that here too.

And can you make it clear what dir the image is in? It's not /content/, and you can't use a path in the input box or image saving fails.

[markmcd]

Great tutorial! Will be great to move it to Gemini's Imagen when it's widely available.

Added a few comments from a first pass. I couldn't get it to run fully so might have more to say later. Thanks again :)

[NSTiwari]

@markmcd: Thank you for reviewing and providing the feedback. The image I'm using in the tutorial is mine. I'm happy to donate the license for it. Do I need to do anything specific for that or is it enough if I just put it in the assets folder?

I'll make the necessary changes as suggested, and commit them for your further review.

[NSTiwari]

Hi @markmcd, @Giom-V. Pushed the commits with suggested changes.

✅ Added a sample image in assets folder and downloading it using wget to use further in the notebook
✅ Added a "Before you begin" section for prerequisites
✅ Cleaned the output of the pip install cells
✅ Added a note at the top of the notebook indicating the notebook is designed for Colab environment
✅ Gave reference of the functions used in the code
✅ Added a break statement in draw_bounding_boxes function to get the first bounding box

Could you please review?

[markmcd]

Ah I see - it's fine if it's code that is under the same license/copyright as the rest of this repo. We just need to make sure anything from an external source is properly attributed. Thanks for adding the comments.

---

View entire conversation on ReviewNB

[markmcd]

Awesome, thanks for the quick updates. This LGTM, but you'll need to address the failing checks first. @Giom-V did you want to add anything?

[Giom-V]

No, I trust your judgement (and my lack of time).
Thanks again @NSTiwari !

[NSTiwari]

Thank you @markmcd, @Giom-V! Just fixing the notebook by addressing the formatting checks.
Question: One of the checks fails as the Colab notebook URL isn't matched (it points to the main branch) and since it's not merged to main yet, it cannot find the notebook either. Shall I temporarily use my GitHub repo's URL in that case?

[tensorflow::button_colab] Colab button URL doesn't match: https://colab.research.google.com/github/google-gemini/cookbook/blob/main/examples/Virtual_Try_On.ipynb

[Giom-V]

Question: One of the checks fails as the Colab notebook URL isn't matched (it points to the main branch) and since it's not merged to main yet, it cannot find the notebook either. Shall I temporarily use my GitHub repo's URL in that case?

You should use the one in the error message:

https://colab.research.google.com/github/google-gemini/cookbook/blob/main/examples/Virtual_Try_On.ipynb

[NSTiwari]

https://colab.research.google.com/github/google-gemini/cookbook/blob/main/examples/Virtual_Try_On.ipynb

Yes, that's what I've used.

[NSTiwari]

@Giom-V, @markmcd: I've created a fresh PR #525 as I messed up with some commits in this one. I'll close this, and request you to please review #525. All the changes have been incorporated as discussed earlier, including the checks for notebook formatting.

Sorry again, much appreciated!