Cloudflare Developers•3y ago

This quarter, on the nearer side.

Wwadefletch If I was only planning to query data via vector similarity, is there a downside ...

Matt SilverlockOP•10/23/23, 7:20 PM

Ultimately up to your data access patterns & structure. Impossible to otherwise advise.

MMatt Silverlock This quarter, on the nearer side.

wadefletch•10/23/23, 7:43 PM

Thanks! I am building a multi-tenant RAG application, in which users can upload their own documents. Is the better way to isolate these, a) hybird queries on some namespace property or b) separate indexes for each user? I imagine it would depend on the size of the indicies (aka the # of embeddings per user)? Do you expect the index limit will increase sufficiently to even make this possible?

Victor•10/23/23, 9:11 PM

On the limits page, I don't see betabeta flag next to Maximum index name lengthMaximum index name length, so is this a infrastructure limitation or can this be increased later?

VVictor On the limits page, I don't see `beta` flag next to `Maximum index name length`,...

Matt SilverlockOP•10/23/23, 9:45 PM

What kind of length do you need and why? 63 characters per name is common across most Cloudflare (and cloud provider) naming - would love to understand the need for longer names

Matt SilverlockOP•10/23/23, 9:45 PM

(63 = max length of a DNS label)

MMatt Silverlock What kind of length do you need and why? 63 characters per name is common across...

Victor•10/23/23, 9:55 PM

I'm looking for a way to structure the organization of the data in vectorize. Like a SaaS platform would have the tenant id/data space id/source id/resource idtenant id/data space id/source id/resource id since a new vectorize index is created per resource (each vector inside = each line in the resource). If each id is a uuid, that's 16 (uuid length) * 4ids + 3 separators = 67 characters.

Victor•10/23/23, 9:56 PM

That way when queried through code (rest not workers bindings), we can jump to the right vectorize store directly

Victor•10/23/23, 9:57 PM

I mean an alternative is to have just a resource uuid and a lookup table that has the other info in it, but that's an extra pre-call before the vectorize call

VVictor I'm looking for a way to structure the organization of the data in vectorize. Li...

Matt SilverlockOP•10/23/23, 9:58 PM

Note that Vectorize supports namespaces per index and (soon) metadata filtering. How many indexes overall do you expect? If we bumped one limit (name length) I don’t want you to just run into another. Better to understand what you’re trying to architect here.

Cc42alyst Thanks, with postgres databases that support vector datatype we can store the da...

Old Man Maker•10/25/23, 12:19 PM

Yep - I get what you're saying, and I can see a positive to that. I guess with Vectorize not being opinionated on data store adds to the flexibility. I've got a few cases where I have data in different sources and it really works well for my architecture and use cases. But I certainly see the benefit to the postgres solution as you mention

cosbgn•10/25/23, 1:51 PM

currently when querying a vector db, the result has only the id and score.
So I need an extra 5 calls to get the metadata (one for each, if topK=5).

Is it planned to return also metadata on query or it will never be available?

cosbgn•10/25/23, 2:02 PM

Great! Thanks!

Ccosbgn currently when querying a vector db, the result has only the id and score. So I ...

Matt SilverlockOP•10/25/23, 5:52 PM

To be extra clear, you can set returnVectors: truereturnVectors: true to get the metadata today (alongside the vectors themselves). The API updates will allow you to just get the metadata on its own, but you don't need to make 5 API calls to do that today.

Ccosbgn currently when querying a vector db, the result has only the id and score. So I ...

Matt SilverlockOP•10/25/23, 5:53 PM

See https://developers.cloudflare.com/vectorize/learning/query-vectors/

Query vectors · Vectorize

Querying an index, or vector search, enables you to search an index by providing an input vector and returning the nearest vectors based on the …

cosbgn•10/25/23, 8:36 PM

Oh, I thought returnVectors would return the values. Anyways the new API returnValues and returnMetadata is much cleared! Thanks!

Wwadefletch Thanks! I am building a multi-tenant RAG application, in which users can upload ...

charl.dev•10/26/23, 3:42 PM

You can create a index per client via api

Wwadefletch If I was only planning to query data via vector similarity, is there a downside ...

charl.dev•10/26/23, 3:43 PM

I was doing this with kv, the downside is you need to list each kv, fetch it and then do the similarity search. This was pre workers ai etc

Ccharl.dev You can create a index per client via api

charl.dev•10/26/23, 3:44 PM

So when creating a client, create their index and a db of you want, store a ref somewhere

charl.dev•10/26/23, 3:45 PM

Cf now allows you to create like 50000dbs, so each client can have an isolated db. It’s pretty epic

Ccharl.dev You can create a index per client via api

wadefletch•10/26/23, 4:15 PM

Only 100 indexes of 200,000 vectors rn though, right? That's a mighty small user count for a startup.

wesleyyue•10/26/23, 4:57 PM

Is there any bench mark on Vectorize? Something like this? https://supabase.com/blog/increase-performance-pgvector-hnsw

Supabase

pgvector v0.5.0: Faster semantic search with HNSW indexes

Increase performance in pgvector using HNSW indexes

wadefletch•10/26/23, 5:29 PM

@Matt Any flexibility on the 100 indexes limit? Is that something exceptions could be granted for within the beta? In what ballpark do you expect the final (paid) limit to land?

Wwadefletch @Matt Any flexibility on the 100 indexes limit? Is that something exceptions cou...

Matt SilverlockOP•10/26/23, 5:30 PM

We expect to land metadata filtering soon, which is going to be the better solution here. We will increase the index limit too but not immediately.

Wwadefletch @Matt Any flexibility on the 100 indexes limit? Is that something exceptions cou...

Matt SilverlockOP•10/26/23, 5:31 PM

(Do you have a need for more than 100 indexes right now or within the next few weeks, or is this forward looking?)

MMatt Silverlock (Do you have a need for more than 100 indexes _right now_ or within the next few...

wadefletch•10/26/23, 5:32 PM

Very forward looking, just spec'ing infra options.

MMatt Silverlock We expect to land metadata filtering soon, which is going to be the better solut...

wadefletch•10/26/23, 5:32 PM

Ok, great! Thanks!

Wwadefletch Very forward looking, just spec'ing infra options.

Matt SilverlockOP•10/26/23, 5:34 PM

Got it. Right now our focus is:

1. Metadata filtering
2. Increasing overall performance and per-index vector limits
3. User-facing analytics

Wwadefletch Only 100 indexes of 200,000 vectors rn though, right? That's a mighty small user...

charl.dev•10/26/23, 9:49 PM

Ment per tenant, not user

Wwadefletch Very forward looking, just spec'ing infra options.

charl.dev•10/26/23, 9:52 PM

Cf limits are mostly soft limits that can be increased if you ask nicely, so I wouldn’t let the limits deter you

charl.dev•10/27/23, 12:14 PM

when creating a vector index, im getting:

"message": "vectorize.index.invalid_name - Index name \"user_29w83sxmDNGwOuEthce5gg56FcC\""

"message": "vectorize.index.invalid_name - Index name \"user_29w83sxmDNGwOuEthce5gg56FcC\""

Its within the char limit, not sure what other restrictions there are?

charl.dev•10/27/23, 12:19 PM

also tried without the underscore

zegevlier•10/27/23, 12:20 PM

Perhaps it's too long?

zegevlier•10/27/23, 12:21 PM

From https://developers.cloudflare.com/vectorize/platform/limits/#limits

Maximum index name length: 63 bytes

zegevlier•10/27/23, 12:21 PM

This is only 32, so that's not it. hmm

zegevlier•10/27/23, 12:24 PM

Ah I found it, you aren't allowed to use capital letters in the name

charl.dev•10/27/23, 12:29 PM

ah, is that intentional or might change?

charl.dev•10/27/23, 12:33 PM

with a limit of 100ish indexes, chances of collision atm are not a worry if I make them all lowercase - but seems that capital letters should be allowed?

Ccharl.dev ah, is that intentional or might change?

Matt SilverlockOP•10/27/23, 12:43 PM

Might change. Again, keep in mind metadata filtering and (in the future) increased total index limits will both land.

MMatt Silverlock Might change. Again, keep in mind metadata filtering and (in the future) increas...

zegevlier•10/27/23, 12:45 PM

I was just making a PR to the docs to clarify this on the getting started pages. Should I still make it or hold off for now?

Zzegevlier I was just making a PR to the docs to clarify this on the getting started pages....

Matt SilverlockOP•10/27/23, 12:51 PM

I would still PR it

Zzegevlier I was just making a PR to the docs to clarify this on the getting started pages....

Matt SilverlockOP•10/27/23, 12:51 PM

(And thanks!)

charl.dev•10/27/23, 1:14 PM

and not sure if this is a thing, the delete index is only returning 204, no body. From the docs, is says response would be 200 with:

{
  "errors": [],
  "messages": [],
  "result": {},
  "success": true
}

{
  "errors": [],
  "messages": [],
  "result": {},
  "success": true
}

futuristick•10/28/23, 12:45 AM

Might be missing something but filtering by namespace doesn't seem to be working for me. I'm inserting vectors as ndjson via Wrangler, formatted as

{ id: "id", values: [], metadata: {}, namespace: "namespace" }

{ id: "id", values: [], metadata: {}, namespace: "namespace" }

then trying to query it via

INDEX.query(embeddings, { returnVectors: true, namespace: "namespace" });

INDEX.query(embeddings, { returnVectors: true, namespace: "namespace" });

but introducing namespace results in 0 matches

Ffuturistick Might be missing something but filtering by namespace doesn't seem to be working...

charl.dev•10/28/23, 6:14 AM

where did you see that namespace is an allowed prop?

Ccharl.dev where did you see that namespace is an allowed prop?

huw•10/28/23, 7:58 AM

It’s in the TypeScript types

This quarter, on the nearer side.

Similar Threads

Similar Threads

Similar Threads