Cloudflare Developers•13mo ago

Ah interesting - but I'm not sure I understand the need for the "Represent this..." prefix?

MityaOP•12/13/24, 9:56 PM

What would happen if you just queried with an embedded "give me all the feedback about toilets" without that prefix?

MMitya Ah interesting - but I'm not sure I understand the need for the "Represent this....

Isaac McFadyen•12/13/24, 9:56 PM

It's just how the model was trained. It was trained to take a question + a prefix.

MMitya What would happen if you just queried with an embedded "give me all the feedback...

Isaac McFadyen•12/13/24, 9:57 PM

It would still work but likely a bit degraded because it wasn't trained that way.

MityaOP•12/13/24, 9:57 PM

Oh I see, so that's just a fixed piece of text I need to use?

Isaac McFadyen•12/13/24, 9:57 PM

I took that prefix directly from https://huggingface.co/mixedbread-ai/mxbai-embed-large-v1 (a popular open-source model) by the way, it'll be in the model description, yeah.

MityaOP•12/13/24, 9:57 PM

Right! Would never have spotted that. Appreciated.

ggu•12/19/24, 3:55 PM

Vectorize update:
Added support for range queries in metadata filters. Merry Christmas!

https://developers.cloudflare.com/vectorize/platform/changelog/

Cloudflare Docs

Changelog · Vectorize

Vero•12/19/24, 4:00 PM

@Vectorize

VVero @Vectorize⤴️

M.A.K•12/19/24, 4:02 PM

alright

Rahul gangotri•12/19/24, 4:03 PM

Can we also expect

Metadata filters without creating indexes for each
Create dynamic vector indexes programatically without the manual bindings

Andrew Brown•12/20/24, 7:29 PM

I attempted to record a video on how to use Vectorize but I failed at the following:

figuring out binding VECTORIZE

The documentation needs work in these areas.

EndiM•12/22/24, 9:31 PM

EndiM•12/22/24, 9:32 PM

Is anyone of the vectorize team Working today?

EndiM•12/22/24, 9:34 PM

I need someone to help me with an issue that was previously resolved by cloudflare, and is happening again. I do have all the information that it needs, but is something that needs to be resolved.

EndiM•12/23/24, 9:38 AM

Still awaiting!

EndiM•12/23/24, 9:39 AM

I see @Torianna @Tanmay Kapoor online, is one of you guys available?

itsmatteomanf•12/23/24, 9:40 AM

?pings

SuperHelpflare•12/23/24, 9:40 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

itsmatteomanf•12/23/24, 9:40 AM

This applies to Cloudflare employees, too. This server is not support, it's community support.

Iitsmatteomanf ?pings

EndiM•12/23/24, 9:40 AM

Hi matt

Torianna•12/23/24, 9:40 AM

https://tenor.com/view/rrane-battal-rrane-battal-k%C4%B1zg%C4%B1n-battal-k%C4%B1zg%C4%B1n-gif-25355985

Tenor

Iitsmatteomanf This applies to Cloudflare employees, too. This server is not support, it's comm...

EndiM•12/23/24, 9:43 AM

Aha

EndiM•12/23/24, 9:43 AM

EndiM•12/23/24, 9:45 AM

I thought I tagged cf employees, sorry

EEndiM I thought I tagged cf employees, sorry

itsmatteomanf•12/23/24, 9:52 AM

Even then, do not ping them unless you are told to do so.

Iitsmatteomanf Even then, do not ping them unless you are told to do so.

EndiM•12/23/24, 9:53 AM

Yeah, i was told to do so by cloudflare support on the web since yesterday.

Tanmay Kapoor•12/23/24, 10:00 AM

Hi @EndiM, is there a thread on dev community or detailed query you posted earlier? Can you share the link please?

Ashley•1/5/25, 7:28 PM

Are there any plans to increase the current dimension limit? OpenAI’s latest embedding model is over 3000, and while it’s possible to lower it, if you want more accuracy then it simply won’t be possible to use Vectorize

ajgeiss0702•1/8/25, 11:23 PM

can the vectorize http api docs be updated? I basically had to try random things then give up and searched in this channel before i saw somebody with the correct format

Moiz_08•1/9/25, 7:00 AM

I would really appreciate if somebody assit me to create an api for inserting the embeddings into vectorize.

Idle•1/10/25, 7:39 AM

@Community Champion

MMoiz_08 I would really appreciate if somebody assit me to create an api for inserting th...

ggu•1/11/25, 1:45 AM

Yeah what’s up?

Johnny•1/13/25, 2:43 AM

Hi, are there plan to increase the topk limit of 20 with metadata filtering? With chunked data a limit of 20 is severely limiting, especially for paginated search experiences.

makrdev•1/13/25, 11:58 AM

Hey guys, checked but couldn't find. What is the best approach to update metadata? Should we use upsert for that?

Mmakrdev Hey guys, checked but couldn't find. What is the best approach to update metadat...

yevgen•1/13/25, 1:50 PM

correct.

Utkarsh Saxena•1/15/25, 12:25 PM

Hi, I am having issue in inserting vector with correct vector format, anyone else experiencing same issue, there is no error but it is returning null. Any idea

Paige•1/15/25, 5:21 PM

Hello, I'm wondering one thing before getting into cloudflare vectorize, it seems many vector databases charge based on how much data there is to scan through, but I don't find the documentation for vectorize clear, does "queried dimensions" refer to that same thing, or only to the actual dimensions of the vector you're searching by (please ping when responding

)

daguang•1/16/25, 1:56 AM

Vectorize If I filter using Metadata filter, will I be billed for the quantity before filtering or after filtering?

daguang•1/16/25, 1:56 AM

I went through all the documentation but couldn't find a corresponding description

Bob•1/17/25, 6:55 PM

Hello

Is there no UI option to create vector index? I am getting this screen. the UI comes for a split second and goes away

Bob•1/17/25, 6:58 PM

ok. I can create via cli. please ignore my question. :blob_sweat:

Bob•1/19/25, 1:08 PM

Hi. I found this in the vectorize docs. I have 2 questions -

How long do I need to wait before sending next 5000embeddings after 1st batch so that I don't get rate limited. I am trying to insert and later query via HTTP API.
The global limit says 1200req/5min which is too low. It means, on avg only 4 queries can be done on the vectorize index per second right? This limit seems too low or I am not understanding it properly. If I query the index via worker, does this limit not apply in that case?

Ah interesting - but I'm not sure I understand the need for the "Represent this..." prefix?

Similar Threads