Cloudflare Developers•3y ago

Hi @Vy, yes, that'd be what I'm interested in

Ddbk Hi @Vy, yes, that'd be what I'm interested in

Vy•12/19/23, 11:56 PM

thanks, we should be able to tackle GET same time as DELETE

natanparmigiano•12/20/23, 11:59 PM

Hey all!
I have a question regarding billing, if using metadata filters, would total queried vector dimensions be calculated towards the entire index or only the vectors matching the metadata filter?

natanparmigiano•12/21/23, 12:00 AM

I am currently studying the use case of partitioning user documents using filters, but don't want to be billed on a search through all vectors for a question regarding a single document

natanparmigiano•12/21/23, 1:23 AM

Sorry everyone, dumb question, it seems I misunderstood the pricing, a query is billed as 1 queried vector regardless of how many vectors are in the index, right? Now it makes sense why my estimations were coming out so absurd

Nnatanparmigiano Sorry everyone, dumb question, it seems I misunderstood the pricing, a query is ...

Vy•12/21/23, 4:21 PM

no, number of vectors in your index does impact billing since that's the total search space for a query. See https://developers.cloudflare.com/vectorize/platform/pricing/#calculating-vector-dimensions for some examples.

Pricing · Vectorize

Vectorize bills based on:

natanparmigiano•12/21/23, 4:25 PM

But that would affect the "stored vectors" cost, correct?
My understanding is that the "per query" cost is not affected by the stored vectors count (as in, if I have 100000 vectors stored, the cost per query is the same as if I have 200000, but the stored vectors bill would increase)

Is my understanding correct in that sense?

Nnatanparmigiano But that would affect the "stored vectors" cost, correct? My understanding is th...

Matt Silverlock•12/21/23, 4:47 PM

Nnatanparmigiano But that would affect the "stored vectors" cost, correct? My understanding is th...

Matt Silverlock•12/21/23, 4:48 PM

If this doesn't make sense, help us understand where you're stuck. There's also some examples further down:

Matt Silverlock•12/21/23, 4:49 PM

Put another way:
- If you store 10k vectors of 1536 dimension each and make ZERO queries, it's

(10000)*1536*(0.040/1000000)

(10000)*1536*(0.040/1000000)

= $0.6144 for the month
- If you query that index 50k times that month, it's

(10000+50000)*1536*(0.040/1000000)

(10000+50000)*1536*(0.040/1000000)

= $3.68. The query component is $3.072. You just add them up.

natanparmigiano•12/21/23, 4:56 PM

In the end it does make sense!
The docs are quite clear, I just got it wrong at first, since the provided formula includes both the query and storage costs, I at first assumed that index size would affect per query costs

I think it would be clearer to newcomers if the pricing examples had the cost components (stored and queried vectors) shown (and calculated) separately.

natanparmigiano•12/21/23, 4:58 PM

It was just a misunderstanding in my part, as clearly there are two unrelated cost components defined on the top of the article, but the calculation including both factors kinda threw me off!

Thanks a lot for your help, the costs are clear to me now

Nnatanparmigiano Hey all! I have a question regarding billing, if using metadata filters, would t...

Jeffrey Konowitch•12/29/23, 4:25 PM

Was this question answered? Will the queried vector formula be affected when using metadata filters?

JJeffrey Konowitch Was this question answered? Will the queried vector formula be affected when usi...

Jeffrey Konowitch•12/29/23, 5:44 PM

In other words, can we save money by smartly partitioning and querying with metadata. It seems namespaces are the only way to do this at the moment, but are limited to 1000.

xeon06•1/5/24, 6:27 AM

Hey folks, I notice that the scores are get are either in the 0.8 for good results, or 0.7 for bad ones. Is it normal to not have that much of a range of scores?

MMatt Silverlock If this doesn't make sense, help us understand where you're stuck. There's also ...

Frank•1/5/24, 8:49 PM

Does querying with a namespace filter change any of this calc? I could be wrong, but seems quite expensive as is.

FFrank Does querying with a namespace filter change any of this calc? I could be wrong,...

Vy•1/5/24, 8:51 PM

currently no, but we're open to feedback/specifics for your workload!

VVy currently no, but we're open to feedback/specifics for your workload!

Frank•1/5/24, 9:11 PM

Ie. Searching thru vectors of contacts vs. vectors of docs/notes. Latter may be super dense.
Also wish there was no limit on # of namespace once hits GA. Would greatly reduce search space. We can namespace it using a composite key. Can just be concat of user id+ type of data or something

Frank•1/5/24, 9:17 PM

Be great if price is based on actual search space instead of total stored. Assuming namespace and metadata filtering reduces search space

divby0•1/7/24, 8:32 AM

We still can only add vectors, never delete them again though. Together with beta-limits that are relatively low is slowly getting an issue. Had to do a manual reset of the whole index now and I hope it will be enough until we finally get either pages-bindings or delete via http api

divby0•1/7/24, 8:32 AM

Not really sure how this whole product can exist for "so long" without just the option to delete stuff again, kind of boggles my mind why stuff isn't on fire for other people

JJerome @cosbgn vector deletion on the REST api is on the backlog. Upsert will ship soon...

Yongyi•1/7/24, 11:43 AM

I am considering on trying out ai vectorize. it seems like the rest api is there no? https://developers.cloudflare.com/api/operations/vectorize-delete-vectorize-index

or am I missing something?

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

YYongyi I am considering on trying out ai vectorize. it seems like the rest api is there...

divby0•1/7/24, 11:50 AM

That's only the whole index, you cant delete vectors you inserted

Ddivby0 That's only the whole index, you cant delete vectors you inserted

Yongyi•1/7/24, 2:53 PM

Ohhh

Ddivby0 That's only the whole index, you cant delete vectors you inserted

Yongyi•1/7/24, 4:00 PM

correct me if im wrong, you can still delete vectors with cloudflare workers? https://developers.cloudflare.com/vectorize/platform/client-api/#delete-vectors-by-id

Workers API · Vectorize

This page covers the Vectorize API available within Cloudflare Workers, including usage examples.

Yongyi•1/7/24, 4:06 PM

ok I see even if I use workers, you are bound by a vector from environment such as env.(vectorIndex). If vector index surpasses 200,000 vectors per index, you cannot dynamically create a new index inside a worker

Yongyi•1/7/24, 4:06 PM

that's going to be annoying as you scale up

YYongyi correct me if im wrong, you can still delete vectors with cloudflare workers? ht...

divby0•1/7/24, 4:24 PM

This is not the rest http api, but the workers api

Ddivby0 This is not the rest http api, but the workers api

Yongyi•1/7/24, 4:28 PM

Ok got it. Thanks for explaining. I do see the bottleneck and annoyance when vector index reach its limit

Yongyi•1/7/24, 4:28 PM

Not scalable

Ccfnathan Stay tuned! Get & delete by id should land in http api the next few days

Yongyi•1/9/24, 12:35 AM

That would be great!!

Old Man Maker•1/9/24, 4:58 PM

If not already available (I couldn't find anything for Workers) or on the roadmap, the ability to update metadata by id would be helpful

OOld Man Maker If not already available (I couldn't find anything for Workers) or on the roadma...

huw•1/9/24, 10:45 PM

As in, a metadata-only upsert? I am currently working around it by doing a get by ID + upsert but it means loading the whole vector which I’m not editing at all

kingmesal•1/9/24, 11:01 PM

I'm finding it kind of interesting that

namespace

namespace

is buried so low in the structure.

I feel like the

namespace

namespace

should be up leveled.

If I have a record with id

and I insert that into namespace

netflix

netflix

, but then I have the same object id, perhaps different metadata and certainly different vectors, and it has a namespace

hulu

hulu

the vectorize database will overwrite the first record, or fail depending on if I called

upsert

upsert

insert

insert

If the namespace were moved up then the actions would happen on the namespace and this type of use case would not be an issue.

I bring this up, regardless of whether this example seems logical or not because it would be possible to have ID conflicts in the same vector table even though numerous namespaces could be used.

Jerome•1/10/24, 9:42 AM

@King Mesal noted thx for the feedback! If we were to allow same id in different namespaces, queries by ID would really be by id/namespace.

Btw you can achieve this id-per-namespace in several ways:

* by having the namespace part of the id
* by having the vendor ID in the metadata of the vector instead, using a different ID scheme, and still use namespaces for partitioning the vector set
* by using several indexes, one per "namespace" (netflix, hulu, ...) as we don't bill per number of index but per usage. That way it's much more similar to what Pinecone does with namespaces for instance, where namespaces are "higher up" in the structure, but there's no way to query across all namespaces in a single query.

Hhuw As in, a metadata-only upsert? I am currently working around it by doing a get b...

Old Man Maker•1/10/24, 3:51 PM

I'm looking to update the metadata values without rewriting the entire vector

JJerome @King Mesal noted thx for the feedback! If we were to allow same id in different...

kingmesal•1/10/24, 5:13 PM

Combining data to ensure a unique key is certainly an option, but that is generally not a favorable option because then all users must know to split the ids or concat to get the ids.

As it is right now, the namespace is really just a piece of metadata from an api perspective.

The problem with the suggestion of several indexes, is that I store data for multiple types of data across multiple customers. As of now I have an index per type of data (because they have the same shape). Ideally the namespace should be all I need to use the same index for all customers, but without prefixing the key, I cannot ensure uniqueness.

If I created an index per customer I would be afraid of running into Cloudflare limits on the number of indexes I create.

Oh, but, if namespace were upleveled, an empty (default) namespace could still be a valid namespace...

kingmesal•1/10/24, 6:07 PM

If the guidance is, Just go ahead and create the extra indexes, I'm good with that, so long as I don't have to be concerned about running out of total indexes allowed ... e.g. there (may be still) used to be limits on number of KV namespaces, and D1 databases, etc... so if that is a non-concern I'm good with that....

actually, after thinking about that, this approach is quite ineffective from a worker because I would need to have a binding to every index...

kingmesal•1/10/24, 6:17 PM

Of course I just went and looked at limits and see

Indexes per account    100 indexes beta

Indexes per account    100 indexes beta

kingmesal•1/10/24, 6:26 PM

Separate topic:

1. I was trying to get a list of all the IDs I had stored in my index and found no way to do that.
2. I built a way to do that, but found there is a

Maximum returned results (topK)    20 beta

Maximum returned results (topK)    20 beta

. Any chance the dashboard will get some updates to show the vector count for the index?
3. If I lose track of an ID that is in my index I have no way of removing it without deleting and recreating the index. It may be helpful to have a

removeAll()

removeAll()

... which is on many of the vector db's out there.

Kkingmesal Combining data to ensure a unique key is certainly an option, but that is genera...

huw•1/10/24, 10:55 PM

If you’re really worried about user access, you could define a wrapper class around the index that takes (namespace, id) as arguments to put and get

huw•1/10/24, 10:56 PM

It’s pretty common practice—for example in KV—to require users to segment their keys like this if they want a uniqueness guarantee

Hhuw If you’re really worried about user access, you could define a wrapper class aro...

kingmesal•1/10/24, 11:52 PM

I fully understand HOW to work around it... the idea generally speaking is to NOT have to work around things.... which is why I'm bringing these things up here.

Kkingmesal I fully understand HOW to work around it... the idea generally speaking is to NO...

Old Man Maker•1/12/24, 12:43 PM

For the moment, you're going to find (which you may have already spotted) that you're going to dodge one bullet, only to be hit by the next in terms of limits.
In some ways, this has actually been a good process, because it made me really think about the data in a much deeper way than otherwise - in order to get some scale out of it, I've got a rotator system in, which binds N indexes to a worker, and I have N workers doing that. With 100 indexes and 200K vectors/index limits, that's about the only way to really push it at the moment - I would suspect/hope we are due for an update soon including limit increases.

Ccfnathan In case you missed it, these went live yesterday 🙂 Get By IDs: https://develop...

Yongyi•1/12/24, 3:02 PM

Oh wow

Yongyi•1/12/24, 3:02 PM

Thank you

Kkingmesal I fully understand HOW to work around it... the idea generally speaking is to NO...

Vy•1/12/24, 3:55 PM

@King Mesal thanks for flagging these, we are evaluating as the team keeps improving Vectorize

Hi @Vy, yes, that'd be what I'm interested in

Similar Threads