@cfnathan had a chance to try querying vectorize again and it now works about 30% of the time when t

@cfnathan had a chance to try querying vectorize again and it now works about 30% of the time when testing from location outside US. The rest of the time the error is 'Request failed with status code 502'. statusText: 'Bad Gateway'. And some HTML is returned which includes a Cloudflare Ray ID: 884acb211000270d if that helps. Better but still unreliable

zensin•5/22/24, 1:01 PM

needs better embedding models once stable

Aantonioballs @cfnathan had a chance to try querying vectorize again and it now works about 30...

zensin•5/22/24, 1:01 PM

I'm curious if the issue you mentioned is still happening

Zzensin I'm curious if the issue you mentioned is still happening

antonioballsOP•5/23/24, 10:29 AM

Yes it's unusable at the moment. I've switched to Pinecone

scotto•5/25/24, 7:08 PM

Any news on stabilization from outside US?

imnotgoodatchoosingusernames•5/27/24, 8:06 PM

Hi! Just a quick question, but can we use namespaces with the LangChain Cloudflare Vectorize integration?

Isaac McFadyen•5/27/24, 10:33 PM

(for context: the message was removed)

Sscotto Any news on stabilization from outside US?

yevgen•5/28/24, 10:01 AM

The team is focused to release next version of Vectorize with 2 MM per index support end of June in private beta. In the meantime for folks experiencing issues with REST query API outside of US we recommend using Workers bindings which doesn't need to go through API Gateway service.

antonioballsOP•5/28/24, 2:35 PM

Hi @yevgen, do you mean service bindings, where we make a HTTP request to one worker which calls another worker that handles the requests to Vectorize?

Because I've tried that approach and still no data is returned from Vectorize. Let me know if I'm missing something.

Aantonioballs Hi @yevgen, do you mean service bindings, where we make a HTTP request to one w...

yevgen•5/28/24, 3:08 PM

yes, this approach removes API Gateway from the call sequence. I don't think there is anything specific with query Vectorize outside of US if done through workers bindings.

matt•5/29/24, 2:46 AM

anyone else getting "VECTOR_QUERY_ERROR (code = 1005): Internal Error: Internal error" ? was working yesterday.

matt•5/29/24, 4:06 AM

working now. must've been related to the D1 issues.

Yyevgen yes, this approach removes API Gateway from the call sequence. I don't think the...

antonioballsOP•5/29/24, 2:52 PM

Tried this approach and still doesnt work (from outside the US at least). No data is returned from Vectorize.

russw_•5/30/24, 1:08 PM

All, I'm wondering if anyone knows when/if the vector limit will be increased from 200k. That limit is preventing me from using Vectorize for my application.

Rrussw_All, I'm wondering if anyone knows when/if the vector limit will be increased fr...

yevgen•5/30/24, 2:09 PM

We are focusing to start onboarding first customers on new version of Vectorize in second half of June, early July. Please let me know if you want to be on the waiting list for new version of Vectorize.

Mmatt working now. must've been related to the D1 issues.

yevgen•5/30/24, 2:12 PM

Vectorize doesn't use D1 internally.

Wang•5/31/24, 3:05 PM

@yevgen I want join the waiting list, thanks

Maurice•6/3/24, 1:58 PM

Hi, I want to report a problem:

I'm using the HTTP API for Vectorize. I found that when I make a query using this endpoint:
https://developers.cloudflare.com/api/operations/vectorize-query-vector

If I try with a test vector of a small size (eg. 10) it works, but if I pass in the full vector (1368) it returns a 502 error.

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

Maurice•6/3/24, 1:58 PM

   const body = {
        'returnMetadata': true,
        'topK': 10,
        'returnValues': false,
        'vector': vector.slice(0, 10),
    };

   const body = {
        'returnMetadata': true,
        'topK': 10,
        'returnValues': false,
        'vector': vector.slice(0, 10),
    };

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

russw_•6/7/24, 7:42 PM

Yes, I would love to be on the waiting list for the new version of Vectorize!

gauravmandall•6/11/24, 4:32 PM

I'm entering this command but somehow it's not working:

bunx wrangler vectorize create --dimensions=1536 ers-v1 --metric=cosine

bunx wrangler vectorize create --dimensions=1536 ers-v1 --metric=cosine

showing this error:

🚧 Creating index: 'ers-v1'

✘ [ERROR] A request to the Cloudflare API (/accounts/53e0c2270158721ff328e572f56950ea/vectorize/indexes) failed.

  vectorize.not_entitled [code: 1005]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

🚧 Creating index: 'ers-v1'

✘ [ERROR] A request to the Cloudflare API (/accounts/53e0c2270158721ff328e572f56950ea/vectorize/indexes) failed.

  vectorize.not_entitled [code: 1005]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

#cloudflare-typescript #wrangler #vectorize-beta

Ggauravmandall I'm entering this command but somehow it's not working: ```bunx wrangler vectori...

DaniFoldi•6/11/24, 6:45 PM

Vectorize is currently only available for accounts subscribed to paid workers tier. In the future they will allow free accounts, however there's no timeline, this is the note in the docs:

Vectorize will be available to developers on the Workers Free plan in the future.

ömer•6/12/24, 1:04 PM

Hello, is it possible to use Vectorize locally? For some reason vectorize binding is undefined even though I declared it in wrangler.toml

yigit•6/12/24, 10:25 PM

Any plan to support million scale per namesace? The current limitations hold us back for large amount of vector (but low dim like 256) querying on CF.

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

yigit•6/12/24, 10:26 PM

I want to be part of it

Öömer Hello, is it possible to use Vectorize locally? For some reason vectorize bindin...

Joey•6/13/24, 1:42 PM

you have to add the --remote flag to your wrangler command as vectorize is currently not supported locally,

zwily•6/17/24, 11:37 PM

I just started using Vectorize, and the query result shape is different than what's in the @cloudflare/workers-types@4.20240614.0:

{
  "count": 1,
  "matches": [
    {
      "vectorId": "145",
      "score": 0.582596078,
      "vector": {
        "id": "145",
        "values": [],
        "metadata": {
          "content": "foo"
        }
      }
    }
  ]
}

{
  "count": 1,
  "matches": [
    {
      "vectorId": "145",
      "score": 0.582596078,
      "vector": {
        "id": "145",
        "values": [],
        "metadata": {
          "content": "foo"
        }
      }
    }
  ]
}

Note the new vectorvector object in each result object. This also doesn't match what's in the docs: https://developers.cloudflare.com/vectorize/reference/metadata-filtering/

Did this change recently?

Cloudflare Docs

Metadata filtering · Vectorize

In addition to providing an input vector to your query, you can also filter by vector metadata associated with every vector. Query results only …

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

zwily•6/18/24, 12:42 AM

I am interested

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

thipperz•6/18/24, 9:14 PM

I'm also quite interested!

Rihan•6/19/24, 6:06 PM

Does Vectorize support local development the same way as D1, or does it run on Cloudflare when locally bound too (like Workers AI)?

Rihan•6/19/24, 6:25 PM

Looks like it's not :(
- https://developers.cloudflare.com/workers/testing/local-development/#supported-resource-bindings-in-different-environments
- https://github.com/cloudflare/workers-sdk/issues/4360

aiGeek•6/25/24, 12:52 AM

Are there any more business representative datasets for vectorize that have more complex JSON metadata available or planned? The number of elements need not be large. This would really help the community to easily try out querying and other API features. It also promotes the building of a knowledge base from a baseline dataset from which common questions will come about.

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

chain swanging•6/27/24, 9:46 AM

Hey man I'd really like to be on the waiting list, using vectorise a lot right now

Rihan•6/27/24, 3:50 PM

Is there any way to query for all record IDs stored within a Vectorize index? I'd like to know to build a way of viewing records (metadata mainly) like you can with D1, and to build a script to easily purge an index without fully recreating it.

RRihan Is there any way to query for all record IDs stored within a Vectorize index? I'...

zwily•6/28/24, 10:06 AM

No. It’s a tough omission.

KorigamiK•6/28/24, 12:52 PM

Can you add me to the vectorize beta?

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

cosbgn•6/29/24, 11:03 AM

Hey @yevgen I'm exploring vectorize, could you tell me briefly what will be the limits/new features coming soon on v2?

cosbgn•6/29/24, 11:06 AM

One of my biggest concerns is 1K namespaces, if I want to add a namespace for each "workspace" my users might have I can only have a max of 100 users (if each user has 10 workspaces) which makes it really restrictive.

Alternatively I could do one namespace per users and use metadata to filter the correct workspace but seems inefficient

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

hydroxide•7/1/24, 4:54 AM

I'd like to trial this

Vinodh•7/1/24, 5:01 AM

Vectorize is not available for local development ?

VVinodh Vectorize is not available for local development ?

Vero•7/1/24, 8:37 AM

It's not supported yet

Ccosbgn One of my biggest concerns is 1K namespaces, if I want to add a namespace for ea...

ulysses•7/2/24, 1:48 AM

I have this question as well–what can we expect the limits to be?

thipperz•7/3/24, 5:29 PM

Please Vectorize V2 come soon

thipperz•7/3/24, 5:30 PM

my main concern is the topK

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

Derpenstein69•7/4/24, 7:58 PM

Please add me to the list, thank you

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

kingmesal•7/6/24, 6:52 PM

I'd like in.

Bierman•7/9/24, 2:02 PM

Is there a rough ETA on being able to run vectorize locally?

For context: I have a worker using a lot of cf services and it seems I have to use the api for vectorize as the —remote flag points all services to remote dev when really my goal is to run as many services locally as possible, then use remote for the unsupported ones. Is there a better way to handle this?

Ppdwittig Hey all 👋 - New product launch - Vectorize V2 In the next week or two, we'll s...

Fokke•7/9/24, 4:36 PM

Are you also increasing the number of vectors in an index or just the number of indexes? I am working on a use case where we need something close to 20-40MM vectors. We could store those all over multiple indexes, but a single index would be much preferred.

Or do you perhaps mean the number of vectors per index is increased to 2MM and the number of indexes stays the same?

Ppdwittig Hey all 👋 - New product launch - Vectorize V2 In the next week or two, we'll s...

yigit•7/9/24, 6:28 PM

hey @pdwittig we have a unique case with 500m docs but each of them are 256 dimensions. do you think that we can try vectorize v2 for our usecase?

leesus•7/12/24, 12:46 PM

Is there any page/documentation around Vectorize error codes?

@cfnathan had a chance to try querying vectorize again and it now works about 30% of the time when t

Similar Threads

Similar Threads

Similar Threads