it's possible to see the amount of vectors inside a index? - Cloudflare Developers

© 2026 Hedgehog Software, LLC

More

Communities Docs About Terms Privacy

scotto

Sscotto it's possible to see the amount of vectors inside a index?

thipperz

T

thipperz•4/25/24, 8:06 PM

such basic stuff right

thipperz

T

thipperz•4/25/24, 8:06 PM

i'm hoping to see a lot of things when Vectorize goes GA

Bram Adams

B

Bram Adams•4/29/24, 3:38 PM

would really love any kind of way to introspect anything about the vector db. a ui similar to supabase where i could see the metadata at least would go a really long way!

Sweetbasil

S

Sweetbasil•4/29/24, 5:42 PM

Hi Cloudflare team
Is Vectorize database HIPPA compliant?

antonioballs

A

antonioballs•4/29/24, 11:55 PM

Any update on the unreliability of querying Vectorize via REST API, @cfnathan ? Discussed here: Can you provide more info about the. Can provide more info you need. Cheers.

Discord

Discord - A New Way to Chat with Friends & Communities

Discord is the easiest way to communicate over voice, video, and text. Chat, hang out, and stay close with your friends and communities.

Bonadio

B

Bonadio•5/1/24, 5:07 AM

Vectorize seems very slow. I have a simple RAG application that uses vectorize with an index of size 1536 with cosine, with a single document of 5 pages , I was timing the code and vectorize takes almost 2 seconds to respond.
Here are my timing
(log) Embedding time 378
(log) Vectorize time 1844
(log) Document time 171
(log) LLM time 1518

the code is this

        const vectorizeStartTime: any = new Date()
        const queryVec = await c.env.VECTORIZE_TESTE_1536.query(embQuery, { 
            topK: 3, 
            returnValues: false, 
            returnMetadata: true, 
            namespace: data.customerId.toString()+"_"+data.repoId.toString()
        });
        const vectorizeEndtime: any= new Date()
        console.log("Vectorize time", vectorizeEndtime - vectorizeStartTime)

        const vectorizeStartTime: any = new Date()
        const queryVec = await c.env.VECTORIZE_TESTE_1536.query(embQuery, { 
            topK: 3, 
            returnValues: false, 
            returnMetadata: true, 
            namespace: data.customerId.toString()+"_"+data.repoId.toString()
        });
        const vectorizeEndtime: any= new Date()
        console.log("Vectorize time", vectorizeEndtime - vectorizeStartTime)

Is this the expected speed?

Bonadio

B

Bonadio•5/1/24, 5:11 AM

If I run the same code using chromadb in my local machine it only takes 102 milliseconds to respond

nuno

N

nuno•5/2/24, 11:24 AM

Hi guys, is it possible to query a vector and filter by multiple namespaces?

cfnathan

Ccfnathan This is a symptom of how we have the beta deployed. As mentioned in the above th...

Bonadio

B

Bonadio•5/3/24, 1:23 AM

Hi @cfnathan, thanks for the feedback.

cfnathan

Ccfnathan This is a symptom of how we have the beta deployed. As mentioned in the above th...

scotto

S

scottoOP•5/3/24, 10:29 PM

any ETA for this new version?

notorious_tom

N

notorious_tom•5/3/24, 11:55 PM

hey @George Portillo just wanted to confirm I'm seeing the same issue with code that was working before :\

Jens

J

Jens•5/6/24, 7:40 PM

The pageContent text is not stored and is only used for embedding to create the vectors, right?

So that I have to save the text separately and cannot simply get it back later from the vector db.

const store = new CloudflareVectorizeStore(embeddings, {
      index: env.VECTORIZE_INDEX,
});
await store.addDocuments(
  [
    {
      pageContent: "hello",
      metadata: {},
    },
    {
      pageContent: "world",
      metadata: {},
    },
    {
      pageContent: "hi",
      metadata: {},
    },
  ],
  { ids: ["id1", "id2", "id3"] }
);

const store = new CloudflareVectorizeStore(embeddings, {
      index: env.VECTORIZE_INDEX,
});
await store.addDocuments(
  [
    {
      pageContent: "hello",
      metadata: {},
    },
    {
      pageContent: "world",
      metadata: {},
    },
    {
      pageContent: "hi",
      metadata: {},
    },
  ],
  { ids: ["id1", "id2", "id3"] }
);

cfnathan

Ccfnathan This is a symptom of how we have the beta deployed. As mentioned in the above th...

Jens

J

Jens•5/6/24, 8:38 PM

Where is the datacenter?

Original message was deleted

Jens

J

Jens•5/7/24, 10:03 AM

Yes thanks, I thought it was probably in the USA. I am Europa and the performance wasn't great. Now I tried something else it was faster.

Aim Salam

A

Aim Salam•5/10/24, 1:47 AM

Quick question:

Is there any chance 'Maximum vectors per index' will be unlimited.

Wang

W

Wang•5/15/24, 2:19 AM

When can the GA version be released? :bibicat:

antonioballs

A

antonioballs•5/16/24, 10:49 AM

@cfnathan had a chance to try querying vectorize again and it now works about 30% of the time when testing from location outside US. The rest of the time the error is 'Request failed with status code 502'. statusText: 'Bad Gateway'. And some HTML is returned which includes a Cloudflare Ray ID: 884acb211000270d if that helps. Better but still unreliable

zensin

Z

zensin•5/22/24, 1:01 PM

needs better embedding models once stable

antonioballs

Aantonioballs @cfnathan had a chance to try querying vectorize again and it now works about 30...

zensin

Z

zensin•5/22/24, 1:01 PM

I'm curious if the issue you mentioned is still happening

zensin

Zzensin I'm curious if the issue you mentioned is still happening

antonioballs

A

antonioballs•5/23/24, 10:29 AM

Yes it's unusable at the moment. I've switched to Pinecone

scotto

S

scottoOP•5/25/24, 7:08 PM

Any news on stabilization from outside US?

imnotgoodatchoosingusernames

I

imnotgoodatchoosingusernames•5/27/24, 8:06 PM

Hi! Just a quick question, but can we use namespaces with the LangChain Cloudflare Vectorize integration?

Isaac McFadyen

I

Isaac McFadyen•5/27/24, 10:33 PM

(for context: the message was removed)

scotto

Sscotto Any news on stabilization from outside US?

yevgen

Y

yevgen•5/28/24, 10:01 AM

The team is focused to release next version of Vectorize with 2 MM per index support end of June in private beta. In the meantime for folks experiencing issues with REST query API outside of US we recommend using Workers bindings which doesn't need to go through API Gateway service.

antonioballs

A

antonioballs•5/28/24, 2:35 PM

Hi @yevgen, do you mean service bindings, where we make a HTTP request to one worker which calls another worker that handles the requests to Vectorize?

Because I've tried that approach and still no data is returned from Vectorize. Let me know if I'm missing something.

antonioballs

Aantonioballs Hi @yevgen, do you mean service bindings, where we make a HTTP request to one w...

yevgen

Y

yevgen•5/28/24, 3:08 PM

yes, this approach removes API Gateway from the call sequence. I don't think there is anything specific with query Vectorize outside of US if done through workers bindings.

matt

M

matt•5/29/24, 2:46 AM

anyone else getting "VECTOR_QUERY_ERROR (code = 1005): Internal Error: Internal error" ? was working yesterday.

matt

M

matt•5/29/24, 4:06 AM

working now. must've been related to the D1 issues.

yevgen

Yyevgen yes, this approach removes API Gateway from the call sequence. I don't think the...

antonioballs

A

antonioballs•5/29/24, 2:52 PM

Tried this approach and still doesnt work (from outside the US at least). No data is returned from Vectorize.

russw_

R

russw_•5/30/24, 1:08 PM

All, I'm wondering if anyone knows when/if the vector limit will be increased from 200k. That limit is preventing me from using Vectorize for my application.

russw_

Rrussw_All, I'm wondering if anyone knows when/if the vector limit will be increased fr...

yevgen

Y

yevgen•5/30/24, 2:09 PM

We are focusing to start onboarding first customers on new version of Vectorize in second half of June, early July. Please let me know if you want to be on the waiting list for new version of Vectorize.

matt

Mmatt working now. must've been related to the D1 issues.

yevgen

Y

yevgen•5/30/24, 2:12 PM

Vectorize doesn't use D1 internally.

Wang

W

Wang•5/31/24, 3:05 PM

@yevgen I want join the waiting list, thanks

Maurice

M

Maurice•6/3/24, 1:58 PM

Hi, I want to report a problem:

I'm using the HTTP API for Vectorize. I found that when I make a query using this endpoint:
https://developers.cloudflare.com/api/operations/vectorize-query-vector

If I try with a test vector of a small size (eg. 10) it works, but if I pass in the full vector (1368) it returns a 502 error.

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

Maurice

M

Maurice•6/3/24, 1:58 PM

   const body = {
        'returnMetadata': true,
        'topK': 10,
        'returnValues': false,
        'vector': vector.slice(0, 10),
    };

   const body = {
        'returnMetadata': true,
        'topK': 10,
        'returnValues': false,
        'vector': vector.slice(0, 10),
    };

yevgen

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

russw_

R

russw_•6/7/24, 7:42 PM

Yes, I would love to be on the waiting list for the new version of Vectorize!

gauravmandall

G

gauravmandall•6/11/24, 4:32 PM

I'm entering this command but somehow it's not working:

bunx wrangler vectorize create --dimensions=1536 ers-v1 --metric=cosine

bunx wrangler vectorize create --dimensions=1536 ers-v1 --metric=cosine

showing this error:

🚧 Creating index: 'ers-v1'

✘ [ERROR] A request to the Cloudflare API (/accounts/53e0c2270158721ff328e572f56950ea/vectorize/indexes) failed.

  vectorize.not_entitled [code: 1005]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

🚧 Creating index: 'ers-v1'

✘ [ERROR] A request to the Cloudflare API (/accounts/53e0c2270158721ff328e572f56950ea/vectorize/indexes) failed.

  vectorize.not_entitled [code: 1005]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

#cloudflare-typescript #wrangler #vectorize-beta

gauravmandall

Ggauravmandall I'm entering this command but somehow it's not working: ```bunx wrangler vectori...

DaniFoldi

D

DaniFoldi•6/11/24, 6:45 PM

Vectorize is currently only available for accounts subscribed to paid workers tier. In the future they will allow free accounts, however there's no timeline, this is the note in the docs:

Vectorize will be available to developers on the Workers Free plan in the future.

ömer

Ö

ömer•6/12/24, 1:04 PM

Hello, is it possible to use Vectorize locally? For some reason vectorize binding is undefined even though I declared it in wrangler.toml

yigit

Y

yigit•6/12/24, 10:25 PM

Any plan to support million scale per namesace? The current limitations hold us back for large amount of vector (but low dim like 256) querying on CF.

yevgen

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

yigit

Y

yigit•6/12/24, 10:26 PM

I want to be part of it

ömer

Öömer Hello, is it possible to use Vectorize locally? For some reason vectorize bindin...

Joey

J

Joey•6/13/24, 1:42 PM

you have to add the --remote flag to your wrangler command as vectorize is currently not supported locally,

zwily

Z

zwily•6/17/24, 11:37 PM

I just started using Vectorize, and the query result shape is different than what's in the @cloudflare/workers-types@4.20240614.0:

{
  "count": 1,
  "matches": [
    {
      "vectorId": "145",
      "score": 0.582596078,
      "vector": {
        "id": "145",
        "values": [],
        "metadata": {
          "content": "foo"
        }
      }
    }
  ]
}

{
  "count": 1,
  "matches": [
    {
      "vectorId": "145",
      "score": 0.582596078,
      "vector": {
        "id": "145",
        "values": [],
        "metadata": {
          "content": "foo"
        }
      }
    }
  ]
}

Note the new vectorvector object in each result object. This also doesn't match what's in the docs: https://developers.cloudflare.com/vectorize/reference/metadata-filtering/

Did this change recently?

Cloudflare Docs

Metadata filtering · Vectorize

In addition to providing an input vector to your query, you can also filter by vector metadata associated with every vector. Query results only …

Metadata filtering · Vectorize

yevgen

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

zwily

Z

zwily•6/18/24, 12:42 AM

I am interested

        const vectorizeStartTime: any = new Date()
        const queryVec = await c.env.VECTORIZE_TESTE_1536.query(embQuery, { 
            topK: 3, 
            returnValues: false, 
            returnMetadata: true, 
            namespace: data.customerId.toString()+"_"+data.repoId.toString()
        });
        const vectorizeEndtime: any= new Date()
        console.log("Vectorize time", vectorizeEndtime - vectorizeStartTime)

        const vectorizeStartTime: any = new Date()
        const queryVec = await c.env.VECTORIZE_TESTE_1536.query(embQuery, { 
            topK: 3, 
            returnValues: false, 
            returnMetadata: true, 
            namespace: data.customerId.toString()+"_"+data.repoId.toString()
        });
        const vectorizeEndtime: any= new Date()
        console.log("Vectorize time", vectorizeEndtime - vectorizeStartTime)

🚧 Creating index: 'ers-v1'

✘ [ERROR] A request to the Cloudflare API (/accounts/53e0c2270158721ff328e572f56950ea/vectorize/indexes) failed.

  vectorize.not_entitled [code: 1005]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

🚧 Creating index: 'ers-v1'

✘ [ERROR] A request to the Cloudflare API (/accounts/53e0c2270158721ff328e572f56950ea/vectorize/indexes) failed.

  vectorize.not_entitled [code: 1005]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

{
  "count": 1,
  "matches": [
    {
      "vectorId": "145",
      "score": 0.582596078,
      "vector": {
        "id": "145",
        "values": [],
        "metadata": {
          "content": "foo"
        }
      }
    }
  ]
}

{
  "count": 1,
  "matches": [
    {
      "vectorId": "145",
      "score": 0.582596078,
      "vector": {
        "id": "145",
        "values": [],
        "metadata": {
          "content": "foo"
        }
      }
    }
  ]
}

const store = new CloudflareVectorizeStore(embeddings, {
      index: env.VECTORIZE_INDEX,
});
await store.addDocuments(
  [
    {
      pageContent: "hello",
      metadata: {},
    },
    {
      pageContent: "world",
      metadata: {},
    },
    {
      pageContent: "hi",
      metadata: {},
    },
  ],
  { ids: ["id1", "id2", "id3"] }
);

const store = new CloudflareVectorizeStore(embeddings, {
      index: env.VECTORIZE_INDEX,
});
await store.addDocuments(
  [
    {
      pageContent: "hello",
      metadata: {},
    },
    {
      pageContent: "world",
      metadata: {},
    },
    {
      pageContent: "hi",
      metadata: {},
    },
  ],
  { ids: ["id1", "id2", "id3"] }
);