The docs don't really cover how to make a decision re: the `dimensions` and `metric` parameters when

The docs don't really cover how to make a decision re: the

dimensions

dimensions

dimensions

dimensions and

metric

metric

metric

metric parameters when creating an index. The RAG tutorial suggests 768 and cosine, while the Get started suggests a mere 32 and Euclidian. Could anyone help me understand how to choose here? Is it a trade-off of some kind? Thank you.

MMitya The docs don't really cover how to make a decision re: the `dimensions` and `met...

Isaac McFadyen•12/13/24, 6:17 PM

Sure. So dimension is the number of numbers (say that 5 times fast haha) in your embedding and metric is how they're compared.

So for example, OpenAI has the text-embedding-3-largetext-embedding-3-large model which outputs dimension 3,0723,072, or text-embedding-3-smalltext-embedding-3-small which outputs 1,5361,536, etc. When you embed 1 text they will give you back an array of <n> numbers where <n> is the dimension. With OpenAI you can request smaller embeddings by specifying the dimension you want, but they'll lose some accuracy as a result.

Metric is how to match embeddings. Generally you want cosinecosine unless the model/provider tells you otherwise (OpenAI recommends cosinecosine for their models). Euclidian also works and will give identical numbers to cosinecosine (if the embeddings are normalized which they often are) but cosinecosine is a bit faster and has some other benefits.

MityaOP•12/13/24, 6:20 PM

Thanks @Isaac McFadyen, much appreciated. So it's cosine for metric, then. But in terms of dimension, is it just a case of choosing a bigger number for more accurate (similar) results? Just not sure what I should be choosing here. Basically we have a customer feedback platform and want to pull out similar bits of feedback for clients.

MMitya Thanks @Isaac McFadyen, much appreciated. So it's cosine for metric, then. But i...

Isaac McFadyen•12/13/24, 6:23 PM

So you'll need to match the number to the model. Some models can output lower dimensions (in the case of OpenAI) but they can't ever output larger dimensions. It's really a cost (storage space/query time) versus accuracy trade-off.

Isaac McFadyen•12/13/24, 6:23 PM

Vectorize also has a limit on the dimension size.

Isaac McFadyen•12/13/24, 6:23 PM

https://developers.cloudflare.com/vectorize/platform/limits/ looks like it's limited to 15361536 for dimension.

Isaac McFadyen•12/13/24, 6:24 PM

I'd take a look at the documentation for your model. If it's higher than 15361536 then you'll obviously have to somehow reduce it to 15361536 but if it's lower then you just have to use the lower dimension. 512512 or 10241024 is common for open source models.

IIsaac McFadyen I'd take a look at the documentation for your model. If it's higher than `1536` ...

MityaOP•12/13/24, 8:46 PM

Thanks very much for the info. I'll do some reading.

MityaOP•12/13/24, 9:17 PM

Just checking some terminology. When the limits page says a maximum of 5,000,000 "vectors", that's akin to "rows" in a conventional DB, right? So vectors = rows, and embeddings = the numbers that make up that vector, right?

MMitya Just checking some terminology. When the limits page says a maximum of 5,000,000...

Isaac McFadyen•12/13/24, 9:26 PM

Correct, yeah.
1 embedding = <n> numbers in an array where <n> is the dimension
5 000 000 embeddings = 5 million arrays each with <n> numbers
An embedding is a type of vector, but not all vectors are embeddings. In the case of Machine Learning they usually are though.

MityaOP•12/13/24, 9:28 PM

OK great, so if I wanted to store a "row" in my Vectorize DB for every product in my relational DB, I'd be limited to 5million.

Isaac McFadyen•12/13/24, 9:29 PM

Correct, yeah.

MityaOP•12/13/24, 9:31 PM

I really hope they give you a job at CF for all your expertise/help!

MityaOP•12/13/24, 9:37 PM

Can Vectorize handle embeddings on non-text content e.g. images? This page suggests not as the "use case" column for its single embeddings model says "text"?

MMitya Can Vectorize handle embeddings on non-text content e.g. images? [This page](htt...

Isaac McFadyen•12/13/24, 9:41 PM

Vectorize absolutely can, it can use vectors from anything (I've even thrown facial recognition embeddings in and they worked well). What you need is an embedding model that can generate vectors from images.

Isaac McFadyen•12/13/24, 9:42 PM

Vectorize just stores the embeddings, it doesn't generate them. That's the purpose of a model from Workers AI or OpenAI or similar (Workers AI doesn't accept images but some other ones do).

Isaac McFadyen•12/13/24, 9:42 PM

Specifically you're looking for "multi-modal embedding models"

MityaOP•12/13/24, 9:46 PM

Oh - I was thinking that to use Vectorize I had to use CF's embeddings model, but if I understand you right I can use any? i.e. some that can handle images.

MityaOP•12/13/24, 9:48 PM

Lastly for tonight: if I store a bunch of user feedback strings in my Vectorize (the embeddings thereof), can I query it with something like "give me all the items about toilets", or would I need to feed it an example piece of feedback so it could find similar ones?

MMitya Lastly for tonight: if I store a bunch of user feedback strings in my Vectorize ...

Isaac McFadyen•12/13/24, 9:54 PM

Usually the model is designed for this use-case so it would work. You often embed the items directly, and then when you're embedding your question you add a special prefix that allows the embedding model to match it more closely.

Isaac McFadyen•12/13/24, 9:54 PM

It's often along the lines of Represent this sentence for searching relevant passages:Represent this sentence for searching relevant passages:

Isaac McFadyen•12/13/24, 9:55 PM

So you'd embed + store your feedback strings, and then embed Represent this sentence for searching relevant passages: give me all the items about toiletsRepresent this sentence for searching relevant passages: give me all the items about toilets and when you query Vectorize it should give you all the toilet-related feedback items

MityaOP•12/13/24, 9:56 PM

Ah interesting - but I'm not sure I understand the need for the "Represent this..." prefix?

MityaOP•12/13/24, 9:56 PM

What would happen if you just queried with an embedded "give me all the feedback about toilets" without that prefix?

MMitya Ah interesting - but I'm not sure I understand the need for the "Represent this....

Isaac McFadyen•12/13/24, 9:56 PM

It's just how the model was trained. It was trained to take a question + a prefix.

MMitya What would happen if you just queried with an embedded "give me all the feedback...

Isaac McFadyen•12/13/24, 9:57 PM

It would still work but likely a bit degraded because it wasn't trained that way.

MityaOP•12/13/24, 9:57 PM

Oh I see, so that's just a fixed piece of text I need to use?

Isaac McFadyen•12/13/24, 9:57 PM

I took that prefix directly from https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1 (a popular open-source model) by the way, it'll be in the model description, yeah.

MityaOP•12/13/24, 9:57 PM

Right! Would never have spotted that. Appreciated.

ggu•12/19/24, 3:55 PM

Vectorize update:
Added support for range queries in metadata filters. Merry Christmas!

https://developers.cloudflare.com/vectorize/platform/changelog/

Cloudflare Docs

Changelog · Vectorize

Vero•12/19/24, 4:00 PM

@Vectorize

VVero @Vectorize⤴️

M.A.K•12/19/24, 4:02 PM

alright

Rahul gangotri•12/19/24, 4:03 PM

Can we also expect

Metadata filters without creating indexes for each
Create dynamic vector indexes programatically without the manual bindings

Andrew Brown•12/20/24, 7:29 PM

I attempted to record a video on how to use Vectorize but I failed at the following:

figuring out binding VECTORIZE

The documentation needs work in these areas.

EndiM•12/22/24, 9:31 PM

EndiM•12/22/24, 9:32 PM

Is anyone of the vectorize team Working today?

EndiM•12/22/24, 9:34 PM

I need someone to help me with an issue that was previously resolved by cloudflare, and is happening again. I do have all the information that it needs, but is something that needs to be resolved.

EndiM•12/23/24, 9:38 AM

Still awaiting!

EndiM•12/23/24, 9:39 AM

I see @Torianna @Tanmay Kapoor online, is one of you guys available?

itsmatteomanf•12/23/24, 9:40 AM

?pings

SuperHelpflare•12/23/24, 9:40 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

itsmatteomanf•12/23/24, 9:40 AM

This applies to Cloudflare employees, too. This server is not support, it's community support.

Iitsmatteomanf ?pings

EndiM•12/23/24, 9:40 AM

Hi matt

Torianna•12/23/24, 9:40 AM

https://tenor.com/view/rrane-battal-rrane-battal-k%C4%B1zg%C4%B1n-battal-k%C4%B1zg%C4%B1n-gif-25355985

Tenor

The docs don't really cover how to make a decision re: the `dimensions` and `metric` parameters when

Similar Threads

Similar Threads

Similar Threads