Like why is querying those 10k vecotrs once vs ten times not close to ten times the price? Something

Ddivby0 Like why is querying those 10k vecotrs once vs ten times not close to ten times ...

M

It’s just storage (consistent) and then your queries. It is additive. It would be horrendously expensive if it was multiplicative.

D

divby0OP•11/25/23, 9:49 PM

yeah if it's storage that makes sense. But it says stored vectors are computed by multiplying stored vectors multiplied by dimensions. And seperately, queried vectors are 10k + 100

D

divby0OP•11/25/23, 9:50 PM

just the point "queried vector dimensions" has

your total queried vector dimensions would be 3.878million

your total queried vector dimensions would be 3.878million

which is wrong, isn't it?

F

Frank•11/25/23, 11:16 PM

is the 1000 namespace limit going to be lifted post beta, or what sort of an increase are we expecting?

D

deo time•11/26/23, 1:32 AM

Having the same issue

FFrank is the 1000 namespace limit going to be lifted post beta, or what sort of an inc...

H

huw•11/26/23, 5:30 AM

To your first question: Yes, the 1,000 namespace limit is listed as a beta limitation in the Limits page.
To your second question: I am going to heavily speculate here, but I think Vectorize is a layer on top of Durable Objects based on the old million vector demo. It looks like cluster centroids are stored on different Durable Objects, which to me would imply that namespace segmentation is possibly occurring by segmenting each namespace into its own ‘cluster’ of DOs (which would make it faster than metadata filtering, which we have confirmation of above). So the would theoretically be no limit here, or at least the limit would be constrained by how the namespace indexation is occurring.

00xWW Figured out what it was; my requests were exceeding the token limit, as I was us...

H

huw•11/27/23, 1:06 AM

I tried computing token lengths for BAAI but I don’t think we fully support transformers.js yet—would love to know if I’m wrong on that

D

divby0OP•11/27/23, 11:37 AM

After Vectorize bindings are not working for Pages Functions, I had to deal with the API of it which is really not documented.

Why are you hiding how the body looks like? Why is the example

@/path/to/vectors.ndjson

@/path/to/vectors.ndjson

not clickable? I have no clue how to continue:

D

divby0OP•11/27/23, 11:38 AM

Screenshot is taken from here: https://developers.cloudflare.com/api/operations/vectorize-insert-vector#request-body

I've read all docs of the workers API and the HTTP API now and I think combining the knowledge, I tried sending a POST request with the correct authorization to the insert endpoint.

The body is

{"id":"1","values":[32.4,74.1,3.2],"metadata":{"path":"r2://bucket-name/path/to/image.png","format":"png","category":"profile_image"}}
{"id":"1","values":[32.4,74.1,3.2],"metadata":{"path":"r2://bucket-name/path/to/image.png","format":"png","category":"profile_image"}}

{"id":"1","values":[32.4,74.1,3.2],"metadata":{"path":"r2://bucket-name/path/to/image.png","format":"png","category":"profile_image"}}
{"id":"1","values":[32.4,74.1,3.2],"metadata":{"path":"r2://bucket-name/path/to/image.png","format":"png","category":"profile_image"}}

{"id":"1","values":[32.4,74.1,3.2],"metadata":{"path":"r2://bucket-name/path/to/image.png","format":"png","category":"profile_image"}}
{"id":"1","values":[32.4,74.1,3.2],"metadata":{"path":"r2://bucket-name/path/to/image.png","format":"png","category":"profile_image"}}

{"id":"1","values":[32.4,74.1,3.2],"metadata":{"path":"r2://bucket-name/path/to/image.png","format":"png","category":"profile_image"}}
{"id":"1","values":[32.4,74.1,3.2],"metadata":{"path":"r2://bucket-name/path/to/image.png","format":"png","category":"profile_image"}}

as taken from one of the docs examples. The

Content-Type

Content-Type

headers have been set to

application/x-ndjson

application/x-ndjson

. The server returns "unexpected - Internal server error" with status code 500.

I guess I am still missing something that would be in place when this would be a real file that I was sending in the post body.

Update: I have tested adding it as a file in Postman just to try it out. Still not working :/

Update 2: The 500 internal error was due to me testing with 3 dimensional vectors but my actual index had way more. Solution: Test with the vectors of the same dimension of course...

S

scotto•11/27/23, 12:31 PM

hi, namespace filtering is not working for me, have a short code on how to reproduce

Indexing and searching, if i remove the namespace param from the search file, results are there, but using namespace filtering i'm not getting any results (with the same vector)

indexing.js33.04KB

Ccosbgn That would be amazing, it's quite complicated now to deal with situation where a...

D

divby0OP•11/27/23, 12:54 PM

This. It's so important to be able to delete again, as there is the 1000 namespace limit (each of my users creates several namespaces but usually deletes the related objects in the DB again).

Like the 1000 namespace thing wouldn't be such an issue if we could delete some again.

D

divby0OP•11/27/23, 1:24 PM

Idea: It would be nice to be able to query vectors and get the metadata but NOT the vectors (since my metadata is way shorter than the 1536 dimensional vector)

Ddivby0 Idea: It would be nice to be able to query vectors and get the metadata but NOT...

Z

zegevlier•11/27/23, 1:27 PM

This is now possible with the

returnMetadata: true

returnMetadata: true

parameter, as listed in https://developers.cloudflare.com/vectorize/platform/client-api/#query-vectors

D

divby0OP•11/27/23, 1:28 PM

Ah awesome! That definitely needs to be added to https://developers.cloudflare.com/api/operations/vectorize-query-vector doc for HTTP

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

D

divby0OP•11/27/23, 2:22 PM

I have 1536 dimension openai embeddings and I can't insert 284 of them, but it works if I only insert the first up to 100 vectors. 101 doesn't work.

I looked up the limitations and didn't see anything relevant. Is this a bug?

Update: I implemented chunking and now all the request fail (they have 90 vectors each, so I have like 4 of them). If I try a few times, sometimes several of the 4 request succeed, but never all and it's not reliably of course

Update 2: Still trying to triangulate the issue. Tested by slicing only the first 3 dimensions and then I can insert all at once without a problem or batch them into 4 batches without a problem. Weird

Update 3: Batching into maxSize 10 batches (29 of them) did work on all 29 insert requests. Is there any metadata or index name limit per batch maybe?

Ddivby0 I have 1536 dimension openai embeddings and I can't insert 284 of them, but it w...

M

Matt Silverlock•11/27/23, 3:12 PM

Do you have metadata or index names that exceed the limits? Are you positive they don’t?

MMatt Silverlock Do you have metadata or index names that exceed the limits? Are you positive the...

D

divby0OP•11/27/23, 3:13 PM

Otherwise batching them into 29 individual batches wouldn't have 100% success rate

D

divby0OP•11/27/23, 3:13 PM

Let me programmatically check again

D

divby0OP•11/27/23, 3:15 PM

id is a uuid-v4 (36 characters)
and the metadata is always below 8000 characters for all 284 vectors

Ddivby0 id is a uuid-v4 (36 characters) and the metadata is always below 8000 characters...

M

Matt Silverlock•11/27/23, 4:15 PM

How are you counting metadata length? The limit is enforced on all bytes (keys, etc included), not just values. Our error handling needs to be better here but would be good to rule that out.

Ddivby0 Otherwise batching them into 29 individual batches wouldn't have 100% success ra...

D

deo time•11/27/23, 4:31 PM

I had to write a helper that would batch them into groups of 100 as well as retry if a 500 error happened, worked for the most part

D

deo time•11/27/23, 4:34 PM

If you're inserting a lot though itll take quite a while

MMatt Silverlock How are you counting metadata length? The limit is enforced on all bytes (keys, ...

D

divby0OP•11/27/23, 6:32 PM

Yes, I have a static key (and only one). It's basically

{"content":"..."}

{"content":"..."}

with the

...

...

being less than 8'000 for all elements. Which should guarantee that it's less than 10KiB, right?

D

divby0OP•11/27/23, 6:46 PM

Also it works when I batch the individual vectors into small enough amounts so this is also a proof that the metadata is not too big (or any other single field)

Ddeo time I had to write a helper that would batch them into groups of 100 as well as retr...

D

divby0OP•11/27/23, 6:47 PM

I already was a bit mad that I need to write batching, but I'd hate to write a retry mechanism for inserting some vectors.

But many thanks, this might be my only option until they support the bindings version for pages functions. Which can't be too far away I guess

D

divby0OP•11/28/23, 10:28 AM

I am getting error code 2007. What does that mean and where can I look it up? There must be a list of error codes somewhere, otherwise you guys wouldn't have exposed one, right?

Ccfnathan I believe I see your error: `Bad Request: the vector length is incorrect for th...

D

divby0OP•11/28/23, 10:30 AM

If I recall correct, the reported error improvements that are coming very soon are still not there after 2 months. At least I happened to send the wrong amoutn of dimensions yesterday as well and it showed me a non-useful internal error.

Ddivby0 I am getting error code 2007. What does that mean and where can I look it up? Th...

D

divby0OP•11/28/23, 10:47 AM

I changed nothing and now, 18mins later the same request went through. I didn't redeploy (so env vars haven't changed), I merely had different vectors that were sent maybe (same openai 1536 embeddings, just for different content).

I am not sure if I am supposed to write so much here, but I'd figure the cloudflare team would be interested in seeing exactly what's not working.

Update: it has failed a few times again and now works a few times again. Not sure what I should learn from this

Ddivby0 I changed nothing and now, 18mins later the same request went through. I didn't ...

J

Jerome•11/28/23, 11:34 AM

Code 2007 is an error whose internals are not exposed; for an upsert it means that there were issues while trying to make the change durable. We are seeing this elevated error rate when write rate on an index reaches some threshold. This is something we are aware of, and we are working on improving the situation.

J

Jerome•11/28/23, 11:36 AM

Thank you for reporting!

D

divby0OP•11/28/23, 11:37 AM

This specific issue was with the query endpoint in fact. Still something you have on the radar?

J

Jerome•11/28/23, 11:38 AM

Yes, for query operations it means that the durable storage access failed while reading, despite retries

D

divby0OP•11/28/23, 11:38 AM

I see, okay. So if you guys have those things on your radar and we can expect improvements, that's all I can ask for. I just thought I may be the only one experiencing this and there is a specific bug causing this for me

J

Jerome•11/28/23, 11:38 AM

We have a view on these; we can DM to exchange specifics on your index to help you mitigate if you can

JJerome We have a view on these; we can DM to exchange specifics on your index to help y...

D

divby0OP•11/28/23, 11:39 AM

that would be awesome. I'll DM you the details in a few minutes

E

Esked•11/29/23, 5:58 PM

Hello! I'm having some issues with Vectorize's query function.
When I run

await env.<INDEX>.query(vector, { topK: 1})

await env.<INDEX>.query(vector, { topK: 1})

await env.<INDEX>.query(vector, { topK: 1})

await env.<INDEX>.query(vector, { topK: 1}), in some cases it simply fails and returns the following error message:

TypeError: Cannot read properties of undefined (reading '0')

TypeError: Cannot read properties of undefined (reading '0')

TypeError: Cannot read properties of undefined (reading '0')

TypeError: Cannot read properties of undefined (reading '0')
It's completely unpredictable, sometimes it works perfectly after a couple of calls, sometimes it fails on the very first one. The

reading '0'

reading '0'

reading '0'

reading '0' isn't constant either, sometimes the number is different, and different userQueries result in different numbers.
Here is the code for the function

export async function getEmbedResponse(env: Environment, userQuery: string, data: string[]): Promise<{ score: number, result: string }> {
  const ai = new Ai(env.AI)

  const queryVector: EmbeddingResponse = await ai.run('@cf/baai/bge-base-en-v1.5', {
    text: [userQuery],
  })

  let matches = await env.<INDEX>.query(queryVector.data[0], { topK: 1 })

  const embedAiResponse = { score: matches.matches[0].score, result: data[parseInt(matches.matches[0].vectorId)] }

  return embedAiResponse
}

export async function getEmbedResponse(env: Environment, userQuery: string, data: string[]): Promise<{ score: number, result: string }> {
  const ai = new Ai(env.AI)

  const queryVector: EmbeddingResponse = await ai.run('@cf/baai/bge-base-en-v1.5', {
    text: [userQuery],
  })

  let matches = await env.<INDEX>.query(queryVector.data[0], { topK: 1 })

  const embedAiResponse = { score: matches.matches[0].score, result: data[parseInt(matches.matches[0].vectorId)] }

  return embedAiResponse
}

export async function getEmbedResponse(env: Environment, userQuery: string, data: string[]): Promise<{ score: number, result: string }> {
  const ai = new Ai(env.AI)

  const queryVector: EmbeddingResponse = await ai.run('@cf/baai/bge-base-en-v1.5', {
    text: [userQuery],
  })

  let matches = await env.<INDEX>.query(queryVector.data[0], { topK: 1 })

  const embedAiResponse = { score: matches.matches[0].score, result: data[parseInt(matches.matches[0].vectorId)] }

  return embedAiResponse
}

export async function getEmbedResponse(env: Environment, userQuery: string, data: string[]): Promise<{ score: number, result: string }> {
  const ai = new Ai(env.AI)

  const queryVector: EmbeddingResponse = await ai.run('@cf/baai/bge-base-en-v1.5', {
    text: [userQuery],
  })

  let matches = await env.<INDEX>.query(queryVector.data[0], { topK: 1 })

  const embedAiResponse = { score: matches.matches[0].score, result: data[parseInt(matches.matches[0].vectorId)] }

  return embedAiResponse
}

EmbeddingResponse is an interface based on the examples on the website

interface EmbeddingResponse {
    shape: number[];
    data: number[][];
}

interface EmbeddingResponse {
    shape: number[];
    data: number[][];
}

interface EmbeddingResponse {
    shape: number[];
    data: number[][];
}

interface EmbeddingResponse {
    shape: number[];
    data: number[][];
}

E

Esked•11/29/23, 5:58 PM

I run the vector upserting before calling this function

E

Esked•11/29/23, 8:46 PM

I can see the error message after the query call too

E

Esked•11/29/23, 8:49 PM

E

Esked•11/29/23, 8:49 PM

(edited image to cover information, oops

)

E

Esked•11/29/23, 9:06 PM

Ok I've tested a bit more and it seems like the problem is on my end.
The

data

data

array is sometimes

undefined

undefined

, not sure why but that should be my problem

E

Esked•11/29/23, 9:18 PM

Yep, the problem is definitely on my end, sorry

D

divby0OP•11/30/23, 8:46 AM

@Jerome have you gotten any info on what the plans for vectorize bindings inside pages functions are? Asking here instead of DM because others are probably looking ofr that info as well

Ddivby0 @Jerome have you gotten any info on what the plans for vectorize bindings inside...

M

Matt Silverlock•11/30/23, 3:33 PM

Soon! Ideally before the next holiday period. Prioritizing some stability/scalability work and metadata filtering first, but should be a very fast follow.

Like why is querying those 10k vecotrs once vs ten times not close to ten times the price? Something

Similar Threads

Similar Threads

Similar Threads