Hello, is it possible to use Vectorize locally? For some reason vectorize binding is undefined even

Hello, is it possible to use Vectorize locally? For some reason vectorize binding is undefined even though I declared it in wrangler.toml

yigit•6/12/24, 10:25 PM

Any plan to support million scale per namesace? The current limitations hold us back for large amount of vector (but low dim like 256) querying on CF.

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

yigit•6/12/24, 10:26 PM

I want to be part of it

Öömer Hello, is it possible to use Vectorize locally? For some reason vectorize bindin...

Joey•6/13/24, 1:42 PM

you have to add the --remote flag to your wrangler command as vectorize is currently not supported locally,

zwily•6/17/24, 11:37 PM

I just started using Vectorize, and the query result shape is different than what's in the @cloudflare/workers-types@4.20240614.0:

{
  "count": 1,
  "matches": [
    {
      "vectorId": "145",
      "score": 0.582596078,
      "vector": {
        "id": "145",
        "values": [],
        "metadata": {
          "content": "foo"
        }
      }
    }
  ]
}

{
  "count": 1,
  "matches": [
    {
      "vectorId": "145",
      "score": 0.582596078,
      "vector": {
        "id": "145",
        "values": [],
        "metadata": {
          "content": "foo"
        }
      }
    }
  ]
}

Note the new vectorvector object in each result object. This also doesn't match what's in the docs: https://developers.cloudflare.com/vectorize/reference/metadata-filtering/

Did this change recently?

Cloudflare Docs

Metadata filtering · Vectorize

In addition to providing an input vector to your query, you can also filter by vector metadata associated with every vector. Query results only …

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

zwily•6/18/24, 12:42 AM

I am interested

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

thipperz•6/18/24, 9:14 PM

I'm also quite interested!

Rihan•6/19/24, 6:06 PM

Does Vectorize support local development the same way as D1, or does it run on Cloudflare when locally bound too (like Workers AI)?

Rihan•6/19/24, 6:25 PM

Looks like it's not :(
- https://developers.cloudflare.com/workers/testing/local-development/#supported-resource-bindings-in-different-environments
- https://github.com/cloudflare/workers-sdk/issues/4360

aiGeek•6/25/24, 12:52 AM

Are there any more business representative datasets for vectorize that have more complex JSON metadata available or planned? The number of elements need not be large. This would really help the community to easily try out querying and other API features. It also promotes the building of a knowledge base from a baseline dataset from which common questions will come about.

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

chain swanging•6/27/24, 9:46 AM

Hey man I'd really like to be on the waiting list, using vectorise a lot right now

Rihan•6/27/24, 3:50 PM

Is there any way to query for all record IDs stored within a Vectorize index? I'd like to know to build a way of viewing records (metadata mainly) like you can with D1, and to build a script to easily purge an index without fully recreating it.

RRihan Is there any way to query for all record IDs stored within a Vectorize index? I'...

zwily•6/28/24, 10:06 AM

No. It’s a tough omission.

KorigamiK•6/28/24, 12:52 PM

Can you add me to the vectorize beta?

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

cosbgn•6/29/24, 11:03 AM

Hey @yevgen I'm exploring vectorize, could you tell me briefly what will be the limits/new features coming soon on v2?

cosbgn•6/29/24, 11:06 AM

One of my biggest concerns is 1K namespaces, if I want to add a namespace for each "workspace" my users might have I can only have a max of 100 users (if each user has 10 workspaces) which makes it really restrictive.

Alternatively I could do one namespace per users and use metadata to filter the correct workspace but seems inefficient

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

hydroxide•7/1/24, 4:54 AM

I'd like to trial this

Vinodh•7/1/24, 5:01 AM

Vectorize is not available for local development ?

VVinodh Vectorize is not available for local development ?

Vero•7/1/24, 8:37 AM

It's not supported yet

Ccosbgn One of my biggest concerns is 1K namespaces, if I want to add a namespace for ea...

ulysses•7/2/24, 1:48 AM

I have this question as well–what can we expect the limits to be?

thipperz•7/3/24, 5:29 PM

Please Vectorize V2 come soon

thipperz•7/3/24, 5:30 PM

my main concern is the topK

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

Derpenstein69•7/4/24, 7:58 PM

Please add me to the list, thank you

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

kingmesal•7/6/24, 6:52 PM

I'd like in.

Bierman•7/9/24, 2:02 PM

Is there a rough ETA on being able to run vectorize locally?

For context: I have a worker using a lot of cf services and it seems I have to use the api for vectorize as the —remote flag points all services to remote dev when really my goal is to run as many services locally as possible, then use remote for the unsupported ones. Is there a better way to handle this?

Ppdwittig Hey all 👋 - New product launch - Vectorize V2 In the next week or two, we'll s...

Fokke•7/9/24, 4:36 PM

Are you also increasing the number of vectors in an index or just the number of indexes? I am working on a use case where we need something close to 20-40MM vectors. We could store those all over multiple indexes, but a single index would be much preferred.

Or do you perhaps mean the number of vectors per index is increased to 2MM and the number of indexes stays the same?

Ppdwittig Hey all 👋 - New product launch - Vectorize V2 In the next week or two, we'll s...

yigit•7/9/24, 6:28 PM

hey @pdwittig we have a unique case with 500m docs but each of them are 256 dimensions. do you think that we can try vectorize v2 for our usecase?

leesus•7/12/24, 12:46 PM

Is there any page/documentation around Vectorize error codes?

leesus•7/12/24, 12:47 PM

Am seeing a handful of upsert/delete/query errors in sentry with codes 2006 and 2008 and "internal server error" messaging but struggling to find any indication of what the actual issue is

Maltmax•7/14/24, 3:49 PM

Hi! I don't suppose there is a way to get all vectors stored in a vectorize database without knowing the vector IDs, right?

akazwz•7/16/24, 3:09 PM

@pdwittig hello team. my account id is a275d70ec37a6046e6778c77d65dd38b. and i have problems in dashboard vectorize. it is empty now.

Idle•7/17/24, 10:56 AM

?pings

Flare•7/17/24, 10:56 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

SteffTek•7/18/24, 7:48 AM

vectorize ui not loading :pain:

SteffTek•7/18/24, 7:48 AM

akazwz•7/19/24, 3:50 AM

me too

yigit•7/20/24, 12:25 AM

Is there anyone with access to vectorize v2? I'm eagerly waiting.

Zeeshan•7/24/24, 4:22 PM

Hi folks, just to confirm here once again, if i want to convert data in my d1 db to vectors, i can reach out to vectorise right? and it is paid right?

zensin•7/26/24, 2:08 AM

I keep putting off part of a project hoping this will come out of beta. hows she lookin?

Zzensin I keep putting off part of a project hoping this will come out of beta. hows she...

DaniFoldi•7/26/24, 11:43 AM

:soontm: is the most accurate answer you'll probably get

yevgen•7/31/24, 5:58 PM

I'm happy to announce that all customers who requested access to Vectorize V2 https://forms.gle/jtSJAqz478DsULN86 have been onboarded.

Google Docs

Vectorize V2 - Waitlist

In the next week or two, we'll start rolling out the next generation of Vectorize, which has been completely re-architected from the ground up. This new version supports 2M+ vector indexes, delivers significantly improved query latency, and much more.

Over the coming weeks, we'll be gradually inviting new teams to onboard. If you're interested ...

LordQ•8/1/24, 2:05 PM

Can we expect a visualisation window for Cloudflare Vectorize to manage or monitor our embedding data? Like what we can use in D1 Database

Yyevgen I'm happy to announce that all customers who requested access to Vectorize V2 ht...

zwily•8/3/24, 1:38 PM

I’m 90% sure i filled out the form when you posted it, and never got an email. I just filled it out again.

Zzwily I’m 90% sure i filled out the form when you posted it, and never got an email. I...

Victor•8/3/24, 3:15 PM

I didn't get an email other, but if you hit the /V2 API endpoint either they'll work or you'll get a 403 operation forbidden

VVictor I didn't get an email other, but if you hit the /V2 API endpoint either they'll ...

zwily•8/3/24, 3:16 PM

Where is the documentation?

Zzwily Where is the documentation?

Victor•8/3/24, 3:17 PM

https://developers.cloudflare.com/api/operations/vectorize-list-vectorize-indexes

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

thipperz•8/5/24, 11:34 PM

Hi there,

I filled out the form but I'm getting vectorize.operation_forbidden [code: 1011]vectorize.operation_forbidden [code: 1011] when trying to create a V2 index. I've installed the required Wrangler prerelease version.

The full message also includes

✘ [ERROR] A request to the Cloudflare API (/accounts/***/vectorize/v2/indexes) failed.✘ [ERROR] A request to the Cloudflare API (/accounts/***/vectorize/v2/indexes) failed.

BlockTxn•8/10/24, 2:15 PM

I'm trying to delete an index, i can confirm that the token and account id are 100% correct, followed the doc yet

  curl --request DELETE \
  --url https://api.cloudflare.com/client/v4/accounts/c9XXXXXXXXXX/vectorize/v2/indexes/vector-index \
  --header 'Authorization: Bearer h1IXXXXXXXXX' \
  --header 'Content-Type: application/json'

  curl --request DELETE \
  --url https://api.cloudflare.com/client/v4/accounts/c9XXXXXXXXXX/vectorize/v2/indexes/vector-index \
  --header 'Authorization: Bearer h1IXXXXXXXXX' \
  --header 'Content-Type: application/json'

got

{"success":false,"errors":[{"code":10000,"message":"Authentication error"}]}

{"success":false,"errors":[{"code":10000,"message":"Authentication error"}]}

any ideas ? thanks

BBlockTxn I'm trying to delete an index, i can confirm that the token and account id are 1...

Isaac McFadyen•8/10/24, 2:17 PM

You're using tokens correct? Not global API key?

IIsaac McFadyen You're using tokens correct? Not global API key?

BlockTxn•8/10/24, 2:19 PM

Not sure, I went to https://dash.cloudflare.com/profile/api-tokens
Created api key
Tested it with

curl -X GET "https://api.cloudflare.com/client/v4/user/tokens/verify" \
-H "Authorization: Bearer h1XXX" \
-H "Content-Type:application/json"

then followed on with the tutorials no problems but cannot delete, maybe i'm wrong

Hello, is it possible to use Vectorize locally? For some reason vectorize binding is undefined even

Similar Threads

Similar Threads

Similar Threads