ah, is that intentional or might change?

C

charl.devOP•10/27/23, 12:33 PM

with a limit of 100ish indexes, chances of collision atm are not a worry if I make them all lowercase - but seems that capital letters should be allowed?

Ccharl.dev ah, is that intentional or might change?

M

Matt Silverlock•10/27/23, 12:43 PM

Might change. Again, keep in mind metadata filtering and (in the future) increased total index limits will both land.

MMatt Silverlock Might change. Again, keep in mind metadata filtering and (in the future) increas...

Z

zegevlier•10/27/23, 12:45 PM

I was just making a PR to the docs to clarify this on the getting started pages. Should I still make it or hold off for now?

Zzegevlier I was just making a PR to the docs to clarify this on the getting started pages....

M

Matt Silverlock•10/27/23, 12:51 PM

I would still PR it

Zzegevlier I was just making a PR to the docs to clarify this on the getting started pages....

M

Matt Silverlock•10/27/23, 12:51 PM

(And thanks!)

C

charl.devOP•10/27/23, 1:14 PM

and not sure if this is a thing, the delete index is only returning 204, no body. From the docs, is says response would be 200 with:

{
  "errors": [],
  "messages": [],
  "result": {},
  "success": true
}

{
  "errors": [],
  "messages": [],
  "result": {},
  "success": true
}

{
  "errors": [],
  "messages": [],
  "result": {},
  "success": true
}

{
  "errors": [],
  "messages": [],
  "result": {},
  "success": true
}

F

futuristick•10/28/23, 12:45 AM

Might be missing something but filtering by namespace doesn't seem to be working for me. I'm inserting vectors as ndjson via Wrangler, formatted as

{ id: "id", values: [], metadata: {}, namespace: "namespace" }

{ id: "id", values: [], metadata: {}, namespace: "namespace" }

{ id: "id", values: [], metadata: {}, namespace: "namespace" }

{ id: "id", values: [], metadata: {}, namespace: "namespace" }

then trying to query it via

INDEX.query(embeddings, { returnVectors: true, namespace: "namespace" });

INDEX.query(embeddings, { returnVectors: true, namespace: "namespace" });

INDEX.query(embeddings, { returnVectors: true, namespace: "namespace" });

INDEX.query(embeddings, { returnVectors: true, namespace: "namespace" });

but introducing namespace results in 0 matches

Ffuturistick Might be missing something but filtering by namespace doesn't seem to be working...

C

charl.devOP•10/28/23, 6:14 AM

where did you see that namespace is an allowed prop?

Ccharl.dev where did you see that namespace is an allowed prop?

H

huw•10/28/23, 7:58 AM

It’s in the TypeScript types

H

huw•10/28/23, 7:58 AM

It’s not documented tho

Ffuturistick Might be missing something but filtering by namespace doesn't seem to be working...

H

huw•10/28/23, 1:47 PM

Oh yep, looks like

query

query

doesn’t return namespace info in the vectors

F

futuristick•10/28/23, 1:48 PM

Right, I knew it isn't returned, but I figured filtering by namespace at least worked

Ffuturistick Right, I knew it isn't returned, but I figured filtering by namespace at least w...

O

Old Man Maker•10/28/23, 6:16 PM

Not yet - Its very close by all accounts and hopefully will be released soon.

F

futuristick•10/28/23, 6:17 PM

Gotcha, thanks

Zzach @Matt 200K vectors per index is the real limitation. Index count limit not reall...

M

Matt Silverlock•10/30/23, 1:58 PM

What do you need per index / do you have 200k vectors today elsewhere, or is this forward-looking?

Zzach @futuristick Namespace filtering works for me with a traditional/in-code insert ...

F

futuristick•10/30/23, 2:04 PM

Interesting, thanks for the pointer!

Zzach Forward-looking bc I want to offer users the ability to index and query various ...

M

Matt Silverlock•10/30/23, 10:10 PM

Got it. Per the docs, 200k is a beta limitation and one that will grow over time. Metadata filtering will solve the other part for you and we expect that to land this month.

Zzach Metadata filtering comes *after* the vector search, whereas Namespace filtering ...

M

Matt Silverlock•10/30/23, 10:45 PM

Metadata is before.

S

Sam•10/31/23, 2:27 PM

Hey guys. Getting this error trying to upload my vectors using the cli:

✘ [ERROR] Received a malformed response from the API

POST /accounts/ab18c413a5487e485eb510e76de1e6de/vectorize/indexes/coaching-index/insert -> 200 OK

Any clues why?

Ccfnathan I can see the successful POSTs for your index and it looks like the data from yo...

S

Sam•10/31/23, 4:43 PM

Thanks Nathan!

Ccfnathan I can see the successful POSTs for your index and it looks like the data from yo...

S

Sam•10/31/23, 5:21 PM

Hey Nathan. Tried querying the index. Doesn't seem to contain any data.

Ccfnathan I can see the successful POSTs for your index and it looks like the data from yo...

S

Sam•10/31/23, 5:27 PM

D

DeviAI•10/31/23, 5:52 PM

Guys how to upsert or deleteByIds in to vector index. I did not see this in the documentation

Thank you for any help.

https://developers.cloudflare.com/api/operations/vectorize-list-vectorize-indexes

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

D

DeviAI•10/31/23, 6:03 PM

Thank you, looking forward. Biggest fan of vectorize already. Congrats to team. Cheers.

H

huw•11/1/23, 8:52 AM

Just FYI if you get a

this index is disabled

this index is disabled

this index is disabled

this index is disabled error after deleting and re-creating an index, you just need to re-bind that index to the Worker. A redeploy will do it.

H

huw•11/1/23, 8:54 AM

Just to put this one out there for CF—I am storing vectors with partially random keys (like

${uniqueId}:${randomId}

${uniqueId}:${randomId}

${uniqueId}:${randomId}

${uniqueId}:${randomId}, for chunking a piece of content under the token limit). I’d like to be able to, at some point, delete those items without having to guess where they are by supplying nearby vectors. I’m not fussy about how that would be fulfilled, but I like the syntax from KV where

get

get

just bulk-loads every item and allows a

prefix

prefix

.

Hhuw Just to put this one out there for CF—I am storing vectors with partially random...

H

huw•11/1/23, 11:05 PM

Also gonna request the ability to paginate a search query. If I find results in my search that are out of date, I’d like to exclude them and re-run the search to find the next K results, without having to optimistically search for a large K and filter it down post-hoc

L

LemonJelloo•11/2/23, 1:55 AM

Hi, everyone, this is my first time using vectorize and worker. I am following this tutorial https://developers.cloudflare.com/vectorize/get-started/intro/#2-create-an-index to create a worker and access my index created. However when I tried to run part 4 of the tutorial locally, it gives me an error and I found out that my env is null although I followed exactly how the tutorial set everything up. I used two other devices to try to run the code locally but both gave me same error. Does anyone know why this is the case? Thank you so much.

Introduction to Vectorize · Vectorize

Vectorize is currently in public beta and is enabled for all developers on Workers Paid plans. During the beta, you may run into bugs and instability …

LLemonJelloo Hi, everyone, this is my first time using vectorize and worker. I am following t...

S

Sam•11/2/23, 12:05 PM

Yeah I don't think the bindings work for local testing. You'll probably need to deploy it first and then test it.

A

Ajay•11/2/23, 12:59 PM

how do you bind multiple vector databases? I tried what's documented here
https://developers.cloudflare.com/workers/wrangler/configuration/#vectorize-indexes
but that didn't work

Configuration - Wrangler · Cloudflare Workers docs

Use a

wrangler.toml

wrangler.toml

configuration file to customize the development and deployment setup for your Worker project and other Developer Platform …

Configuration - Wrangler · Cloudflare Workers docs

A

Ajay•11/2/23, 1:01 PM

also, there's a typo in the docs

Z

zegevlier•11/2/23, 1:02 PM

If you want you can submit a PR to fix it: https://github.com/cloudflare/cloudflare-docs, if not I can do that

K

kian•11/2/23, 1:03 PM

There's one open already from September

Z

zegevlier•11/2/23, 1:03 PM

alright then NotLikeThis

A

Ajay•11/2/23, 1:06 PM

aside from that, can either of you "assign an array of the below object to the vectorize key". it works for me with just one DB. Using binding and index_name keys.

AAjay aside from that, can either of you "assign an array of the below object to the v...

H

huw•11/2/23, 11:35 PM

Here’s what I have

vectorize = [
  { binding = "embeddings-development", index_name="embeddings-development" },
  { binding = "embeddings-staging", index_name="embeddings-staging" },
  { binding = "embeddings-production", index_name="embeddings-production" }
]

vectorize = [
  { binding = "embeddings-development", index_name="embeddings-development" },
  { binding = "embeddings-staging", index_name="embeddings-staging" },
  { binding = "embeddings-production", index_name="embeddings-production" }
]

vectorize = [
  { binding = "embeddings-development", index_name="embeddings-development" },
  { binding = "embeddings-staging", index_name="embeddings-staging" },
  { binding = "embeddings-production", index_name="embeddings-production" }
]

vectorize = [
  { binding = "embeddings-development", index_name="embeddings-development" },
  { binding = "embeddings-staging", index_name="embeddings-staging" },
  { binding = "embeddings-production", index_name="embeddings-production" }
]

(I think that’s what you mean?)

ah, is that intentional or might change?

Similar Threads

Similar Threads

Similar Threads