This is not the rest http api, but the workers api

Ddivby0 This is not the rest http api, but the workers api

Y

Yongyi•1/7/24, 4:28 PM

Ok got it. Thanks for explaining. I do see the bottleneck and annoyance when vector index reach its limit

Y

Yongyi•1/7/24, 4:28 PM

Not scalable

Ccfnathan Stay tuned! Get & delete by id should land in http api the next few days

Y

Yongyi•1/9/24, 12:35 AM

That would be great!!

O

Old Man Maker•1/9/24, 4:58 PM

If not already available (I couldn't find anything for Workers) or on the roadmap, the ability to update metadata by id would be helpful

OOld Man Maker If not already available (I couldn't find anything for Workers) or on the roadma...

H

huw•1/9/24, 10:45 PM

As in, a metadata-only upsert? I am currently working around it by doing a get by ID + upsert but it means loading the whole vector which I’m not editing at all

K

kingmesal•1/9/24, 11:01 PM

I'm finding it kind of interesting that

namespace

namespace

namespace

namespace is buried so low in the structure.

I feel like the

namespace

namespace

namespace

namespace should be up leveled.

If I have a record with id

123

123 and I insert that into namespace

netflix

netflix

netflix

netflix, but then I have the same object id, perhaps different metadata and certainly different vectors, and it has a namespace

hulu

hulu

hulu

hulu the vectorize database will overwrite the first record, or fail depending on if I called

upsert

upsert

upsert

upsert or

insert

insert

insert

insert

If the namespace were moved up then the actions would happen on the namespace and this type of use case would not be an issue.

I bring this up, regardless of whether this example seems logical or not because it would be possible to have ID conflicts in the same vector table even though numerous namespaces could be used.

J

Jerome•1/10/24, 9:42 AM

@King Mesal noted thx for the feedback! If we were to allow same id in different namespaces, queries by ID would really be by id/namespace.

Btw you can achieve this id-per-namespace in several ways:

* by having the namespace part of the id
* by having the vendor ID in the metadata of the vector instead, using a different ID scheme, and still use namespaces for partitioning the vector set
* by using several indexes, one per "namespace" (netflix, hulu, ...) as we don't bill per number of index but per usage. That way it's much more similar to what Pinecone does with namespaces for instance, where namespaces are "higher up" in the structure, but there's no way to query across all namespaces in a single query.

Hhuw As in, a metadata-only upsert? I am currently working around it by doing a get b...

O

Old Man Maker•1/10/24, 3:51 PM

I'm looking to update the metadata values without rewriting the entire vector

JJerome @King Mesal noted thx for the feedback! If we were to allow same id in different...

K

kingmesal•1/10/24, 5:13 PM

Combining data to ensure a unique key is certainly an option, but that is generally not a favorable option because then all users must know to split the ids or concat to get the ids.

As it is right now, the namespace is really just a piece of metadata from an api perspective.

The problem with the suggestion of several indexes, is that I store data for multiple types of data across multiple customers. As of now I have an index per type of data (because they have the same shape). Ideally the namespace should be all I need to use the same index for all customers, but without prefixing the key, I cannot ensure uniqueness.

If I created an index per customer I would be afraid of running into Cloudflare limits on the number of indexes I create.

Oh, but, if namespace were upleveled, an empty (default) namespace could still be a valid namespace...

K

kingmesal•1/10/24, 6:07 PM

If the guidance is, Just go ahead and create the extra indexes, I'm good with that, so long as I don't have to be concerned about running out of total indexes allowed ... e.g. there (may be still) used to be limits on number of KV namespaces, and D1 databases, etc... so if that is a non-concern I'm good with that....

actually, after thinking about that, this approach is quite ineffective from a worker because I would need to have a binding to every index...

K

kingmesal•1/10/24, 6:17 PM

Of course I just went and looked at limits and see

Indexes per account    100 indexes beta

Indexes per account 100 indexes beta

Indexes per account    100 indexes beta

Indexes per account 100 indexes beta

K

kingmesal•1/10/24, 6:26 PM

Separate topic:

1. I was trying to get a list of all the IDs I had stored in my index and found no way to do that.
2. I built a way to do that, but found there is a

Maximum returned results (topK)    20 beta

Maximum returned results (topK) 20 beta

Maximum returned results (topK)    20 beta

Maximum returned results (topK) 20 beta. Any chance the dashboard will get some updates to show the vector count for the index?
3. If I lose track of an ID that is in my index I have no way of removing it without deleting and recreating the index. It may be helpful to have a

removeAll()

removeAll()

removeAll()

removeAll() ... which is on many of the vector db's out there.

Kkingmesal Combining data to ensure a unique key is certainly an option, but that is genera...

H

huw•1/10/24, 10:55 PM

If you’re really worried about user access, you could define a wrapper class around the index that takes (namespace, id) as arguments to put and get

H

huw•1/10/24, 10:56 PM

It’s pretty common practice—for example in KV—to require users to segment their keys like this if they want a uniqueness guarantee

Hhuw If you’re really worried about user access, you could define a wrapper class aro...

K

kingmesal•1/10/24, 11:52 PM

I fully understand HOW to work around it... the idea generally speaking is to NOT have to work around things.... which is why I'm bringing these things up here.

Kkingmesal I fully understand HOW to work around it... the idea generally speaking is to NO...

O

Old Man Maker•1/12/24, 12:43 PM

For the moment, you're going to find (which you may have already spotted) that you're going to dodge one bullet, only to be hit by the next in terms of limits.
In some ways, this has actually been a good process, because it made me really think about the data in a much deeper way than otherwise - in order to get some scale out of it, I've got a rotator system in, which binds N indexes to a worker, and I have N workers doing that. With 100 indexes and 200K vectors/index limits, that's about the only way to really push it at the moment - I would suspect/hope we are due for an update soon including limit increases.

Ccfnathan In case you missed it, these went live yesterday 🙂 Get By IDs: https://develop...

Y

Yongyi•1/12/24, 3:02 PM

Oh wow

Y

Yongyi•1/12/24, 3:02 PM

Thank you

Kkingmesal I fully understand HOW to work around it... the idea generally speaking is to NO...

V

Vy•1/12/24, 3:55 PM

@King Mesal thanks for flagging these, we are evaluating as the team keeps improving Vectorize

O

Old Man Maker•1/12/24, 4:46 PM

I'm getting a response from my index insert that indicates nothing is happening. There are no errors, but this is the response:

"{"count":0,"ids":[]}"

This is my insert object:

let phase1TempVec:VectorizeVector ={
  id:contentData.d1Key,
  values: phase1Data.data[0],
  namespace:'content_items',
  metadata:metaRecord
}

let phase1TempVec:VectorizeVector ={
  id:contentData.d1Key,
  values: phase1Data.data[0],
  namespace:'content_items',
  metadata:metaRecord
}

let phase1TempVec:VectorizeVector ={
  id:contentData.d1Key,
  values: phase1Data.data[0],
  namespace:'content_items',
  metadata:metaRecord
}

let phase1TempVec:VectorizeVector ={
  id:contentData.d1Key,
  values: phase1Data.data[0],
  namespace:'content_items',
  metadata:metaRecord
}

Any thoughts?

O

Old Man Maker•1/12/24, 5:12 PM

That appears to be due to the insert, rather than an upsert needed.

O

Old Man Maker•1/12/24, 5:27 PM

Just to be super clear here on the limits -

When I have a string of say 150 words and I generate embeddings for that entire string - when I insert that into a vectorize index, is that counted as 1 vector against the 200K limit - or is the number of dimensions taken into consideration against the capacity?

W

wadefletch•1/13/24, 10:11 AM

Is there a predicted timeline for additional metadata filtering operators? Being able to filter queries to vectors from a date range (represented as ints, don’t have to be fancy dates) would be mega.

B

Bob•1/15/24, 9:05 AM

I'm using langchain and vectorize with ada embedding from openAI. When I do a similarity search I'm getting repeated entries returned. What am I missing?

B

Bob•1/15/24, 11:20 AM

I split into paragraphs instead of pages and it's working far better.

OOld Man Maker Just to be super clear here on the limits - When I have a string of say 150 wo...

J

Jerome•1/16/24, 9:41 AM

1 vector inserted is counted as 1 vector against the 200K limit yes; this 200K vectors limit is not counted against vector dimensions

V

Vy•1/17/24, 12:08 AM

Hi folks, Vectorize team is planning to make the following backwards incompatible change to the

/query

/query

HTTP endpoint to bring request/response format in line with Workers API. See https://github.com/cloudflare/cloudflare-docs/pull/12546

GitHub

Vectorize query HTTP endpoint changes by vy-ton · Pull Request #125...

D

divby0OP•1/20/24, 8:36 AM

I keep getting 502 errors if my index has 1536 vectors (another test-index with 3 vectors works fine. I also tried creating a fresh, empty and new 1536 vector size index which still yields 502)

D

divby0OP•1/20/24, 8:43 AM

If I do the req via postman I get the html 502 error that explains that my browser and cloudflare frankfurt are working but not the host server (api-gateway.api.cloudflare.com). The ray-id (if helpful) is: 848604d8446d3a86

D

divby0OP•1/20/24, 8:48 AM

So the 502 code likely isn't what your api-gateway responded with, but I can't see more unfortunately. I've had this before once, and I can't recall if the issue was solved at all, maybe it's just still there.

My guess is that somewhere before my request gets to vectorize (maybe the api gateway?) the big post body gets rejected (cause 1536 vectors that take up much place in a string that contains

0.018004786

0.018004786

0.018004786

0.018004786 format numbers. The post body is 6kb big.

KKarl Hey everyone, i've come back to some old code and updated the types for CF and i...

K

kingmesal•1/20/24, 5:55 PM

I have a question on this exact same thing... which is...

Is it supposed to be id or vectorId ?
wrangler:

compatibility_date = "2024-01-20"

compatibility_date = "2024-01-20"

compatibility_date = "2024-01-20"

compatibility_date = "2024-01-20"
and using the latest data types:

"@cloudflare/workers-types": "^4.20240117.0",

"@cloudflare/workers-types": "^4.20240117.0",

"@cloudflare/workers-types": "^4.20240117.0",

"@cloudflare/workers-types": "^4.20240117.0", shows this:

declare interface VectorizeMatches {
  matches: VectorizeMatch[];
  count: number;
}
declare interface VectorizeMatch {
  /** The ID for the vector. This can be user-defined, and must be unique. It should uniquely identify the object, and is best set based on the ID of what the vector represents. */
  vectorId: string;
  /** The score or rank for similarity, when returned as a result */
  score: number;
  /** Vector data for the match. Included only if the user specified they want it returned (via {@link VectorizeQueryOptions}). */
  vector?: VectorizeVector;
}

declare interface VectorizeMatches {
  matches: VectorizeMatch[];
  count: number;
}
declare interface VectorizeMatch {
  /** The ID for the vector. This can be user-defined, and must be unique. It should uniquely identify the object, and is best set based on the ID of what the vector represents. */
  vectorId: string;
  /** The score or rank for similarity, when returned as a result */
  score: number;
  /** Vector data for the match. Included only if the user specified they want it returned (via {@link VectorizeQueryOptions}). */
  vector?: VectorizeVector;
}

declare interface VectorizeMatches {
  matches: VectorizeMatch[];
  count: number;
}
declare interface VectorizeMatch {
  /** The ID for the vector. This can be user-defined, and must be unique. It should uniquely identify the object, and is best set based on the ID of what the vector represents. */
  vectorId: string;
  /** The score or rank for similarity, when returned as a result */
  score: number;
  /** Vector data for the match. Included only if the user specified they want it returned (via {@link VectorizeQueryOptions}). */
  vector?: VectorizeVector;
}

declare interface VectorizeMatches {
  matches: VectorizeMatch[];
  count: number;
}
declare interface VectorizeMatch {
  /** The ID for the vector. This can be user-defined, and must be unique. It should uniquely identify the object, and is best set based on the ID of what the vector represents. */
  vectorId: string;
  /** The score or rank for similarity, when returned as a result */
  score: number;
  /** Vector data for the match. Included only if the user specified they want it returned (via {@link VectorizeQueryOptions}). */
  vector?: VectorizeVector;
}

When I query the records from the data store I get this data structure back:

{
  "count": 3,
  "matches": [
    {
      "id": "uuid-2",
      "namespace": "site:ecb",
      "score": null
    },
    {
      "id": "uuid-3",
      "namespace": "site:ecb",
      "score": null
    },
    {
      "id": "uuid-1",
      "namespace": "site:ecb",
      "score": null
    }
  ]
}

{
  "count": 3,
  "matches": [
    {
      "id": "uuid-2",
      "namespace": "site:ecb",
      "score": null
    },
    {
      "id": "uuid-3",
      "namespace": "site:ecb",
      "score": null
    },
    {
      "id": "uuid-1",
      "namespace": "site:ecb",
      "score": null
    }
  ]
}

{
  "count": 3,
  "matches": [
    {
      "id": "uuid-2",
      "namespace": "site:ecb",
      "score": null
    },
    {
      "id": "uuid-3",
      "namespace": "site:ecb",
      "score": null
    },
    {
      "id": "uuid-1",
      "namespace": "site:ecb",
      "score": null
    }
  ]
}

{
  "count": 3,
  "matches": [
    {
      "id": "uuid-2",
      "namespace": "site:ecb",
      "score": null
    },
    {
      "id": "uuid-3",
      "namespace": "site:ecb",
      "score": null
    },
    {
      "id": "uuid-1",
      "namespace": "site:ecb",
      "score": null
    }
  ]
}

The supplied interfaces do NOT match what is being returned by the API calls. namespace is not listed as an optional field in

VectorizeMatch

VectorizeMatch

VectorizeMatch

VectorizeMatch

K

kingmesal•1/20/24, 6:07 PM

I will also +

this vectorize

As being able to do

>

>=

>=

>=

>=

<

<=

<=

<=

<= and boolean

true

true

false

false

is the only thing stopping me from moving forward with Vectorize ...

It would still be super convenient to also have an API:

  public queryById(
    id: string,
    options: VectorizeQueryOptions
  ): Promise<VectorizeMatches>;

  public queryById(
    id: string,
    options: VectorizeQueryOptions
  ): Promise<VectorizeMatches>;

  public queryById(
    id: string,
    options: VectorizeQueryOptions
  ): Promise<VectorizeMatches>;

  public queryById(
    id: string,
    options: VectorizeQueryOptions
  ): Promise<VectorizeMatches>;

I 100% know I can write this on my own, but it is one of the most used queries I have.

Ccfnathan Are you asking specifically if the namespace field is optional in the new format...

K

kingmesal•1/21/24, 7:17 PM

The tsconfig only has

"types": ["@cloudflare/workers-types"],

"types": ["@cloudflare/workers-types"],

"types": ["@cloudflare/workers-types"],

"types": ["@cloudflare/workers-types"],

I did see that notice, but this vecordId / id thing has been an issue for atleast 3 weeks (that is when I started building my use case w/ vectorize). I'll wait until Tuesday before coming back to this topic.

For the namespace thing, if I go browsing into the downloaded workers-types package -> experimental folder I see what you are referring to. I'm curious why the latest changes are still listed as experimental and not in a dated folder.... not sure how that is setup... but I can see the new api layout in the experimental folder.

Kkingmesal The tsconfig only has `"types": ["@cloudflare/workers-types"],` I did see that ...

D

DaniFoldi•1/21/24, 7:50 PM

Experimental always contains the newest types - including ones that end-users like us can't yet access, since the features behind them are experimental. I also recommend using the

2023-07-01

2023-07-01

2023-07-01

2023-07-01 entrypoint, as suggested above by Nathan, if your worker compatibility_date is set to something newer.

In general, the recommendation is to use the newest dated folder (so never experimental) that is older than your worker compat date. So if you have

compatibility_date = '2024-01-01'

compatibility_date = '2024-01-01'

compatibility_date = '2024-01-01'

compatibility_date = '2024-01-01'

you'd use

"types": ["@cloudflare/workers-types/2023-07-01"]

"types": ["@cloudflare/workers-types/2023-07-01"]

"types": ["@cloudflare/workers-types/2023-07-01"]

"types": ["@cloudflare/workers-types/2023-07-01"]. Not all changes requiring a compatibility flag/date update require a new type update, for example if they fix previously buggy behaviour or introduce a change that doesn't change types, just some runtime quirks, that's why there are fewer type entrypoints than entries in the changelog.

DDaniFoldi Experimental always contains the newest types - including ones that end-users li...

K

kingmesal•1/21/24, 7:53 PM

That is helpful information, thanks!

Given the new API updates landing this week... will there be a new compat date added so the data types match up ?

This is not the rest http api, but the workers api

Similar Threads

Similar Threads

Similar Threads