Nomic-embeddings-support

[I8dNLo]

Fix of nomic-embeddings from #204

Also moved jina/miniLM models to refactored classes: PooledEmbedding and PooledNormalizedEmbedding, which implements the logic of embeddings from sentance-transformers lib for corresponding models

Todo:

• move Jina embeddings to PooledNormalizedEmbedding class
• Fix quantized model nomic-ai/nomic-embed-text-v1.5-Q canonical vector

[Anush008]

Hey.

Do we not want to add aliases with the old class names to avoid breaking changes?

Never mind. Someone would not be accessing those classes directly. Hopefully.

[joein]

Were these embeddings obtained differently from nomic-ai/nomic-embed-text-v1 ?

Why do they have more digits after the point?

[twellck]

Hi @I8dNLo,

I'm currently working on adding support for Matryoshka Representation Learning embedding models, the main one being nomic-ai/nomic-embed-text-v1.5.

From the rest of the library as well as Nomic's documentation it seems we typically do normalize embeddings.
However, it seems that from this PR the embeddings for nomic models went from normalized to "nonnormalized", while this is not an issue when using cosine similarity it could have caused issues (or might cause issues to revert back) for people that have been using dot product.

I haven't been able to find the logic from sentence-transformers you are referring to when it comes to normalization, it appears that normalization is not tied to the model in sentence-transformers. Were you mainly referring to the mean_pooling method?

Ultimately, I am able to implement for variable dimensionality without normalization (even if it isn't optimal and not recommended for nomic-ai/nomic-embed-text-v1.5), I would like to avoid going down this path and have inconsistencies in the way different models are handled when it isn't related to the model's requirements.

I'd be happy to your opinion as well as other contributors on whether to go ahead with my current changes:
I.e. merge the normalized_pooled & the pooled embedding classes & normalization on my new matryoshka embedding classes) OR instead keep nomic models' embeddings nonnormalized?