All, I'm wondering if anyone knows when/if the vector limit will be increased from 200k. That limit

Rrussw_All, I'm wondering if anyone knows when/if the vector limit will be increased fr...

Y

yevgen•5/30/24, 2:09 PM

We are focusing to start onboarding first customers on new version of Vectorize in second half of June, early July. Please let me know if you want to be on the waiting list for new version of Vectorize.

Mmatt working now. must've been related to the D1 issues.

Y

yevgen•5/30/24, 2:12 PM

Vectorize doesn't use D1 internally.

W

Wang•5/31/24, 3:05 PM

@yevgen I want join the waiting list, thanks

M

Maurice•6/3/24, 1:58 PM

Hi, I want to report a problem:

I'm using the HTTP API for Vectorize. I found that when I make a query using this endpoint:
https://developers.cloudflare.com/api/operations/vectorize-query-vector

If I try with a test vector of a small size (eg. 10) it works, but if I pass in the full vector (1368) it returns a 502 error.

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

M

Maurice•6/3/24, 1:58 PM

   const body = {
        'returnMetadata': true,
        'topK': 10,
        'returnValues': false,
        'vector': vector.slice(0, 10),
    };

   const body = {
        'returnMetadata': true,
        'topK': 10,
        'returnValues': false,
        'vector': vector.slice(0, 10),
    };

   const body = {
        'returnMetadata': true,
        'topK': 10,
        'returnValues': false,
        'vector': vector.slice(0, 10),
    };

   const body = {
        'returnMetadata': true,
        'topK': 10,
        'returnValues': false,
        'vector': vector.slice(0, 10),
    };

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

R

russw_OP•6/7/24, 7:42 PM

Yes, I would love to be on the waiting list for the new version of Vectorize!

G

gauravmandall•6/11/24, 4:32 PM

I'm entering this command but somehow it's not working:

bunx wrangler vectorize create --dimensions=1536 ers-v1 --metric=cosine

bunx wrangler vectorize create --dimensions=1536 ers-v1 --metric=cosine

bunx wrangler vectorize create --dimensions=1536 ers-v1 --metric=cosine

bunx wrangler vectorize create --dimensions=1536 ers-v1 --metric=cosine

showing this error:

🚧 Creating index: 'ers-v1'

✘ [ERROR] A request to the Cloudflare API (/accounts/53e0c2270158721ff328e572f56950ea/vectorize/indexes) failed.

  vectorize.not_entitled [code: 1005]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

🚧 Creating index: 'ers-v1'

✘ [ERROR] A request to the Cloudflare API (/accounts/53e0c2270158721ff328e572f56950ea/vectorize/indexes) failed.

  vectorize.not_entitled [code: 1005]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

🚧 Creating index: 'ers-v1'

✘ [ERROR] A request to the Cloudflare API (/accounts/53e0c2270158721ff328e572f56950ea/vectorize/indexes) failed.

  vectorize.not_entitled [code: 1005]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

🚧 Creating index: 'ers-v1'

✘ [ERROR] A request to the Cloudflare API (/accounts/53e0c2270158721ff328e572f56950ea/vectorize/indexes) failed.

  vectorize.not_entitled [code: 1005]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/workers-sdk/issues/new/choose

#cloudflare-typescript #wrangler #vectorize-beta

Ggauravmandall I'm entering this command but somehow it's not working: ```bunx wrangler vectori...

D

DaniFoldi•6/11/24, 6:45 PM

Vectorize is currently only available for accounts subscribed to paid workers tier. In the future they will allow free accounts, however there's no timeline, this is the note in the docs:

Vectorize will be available to developers on the Workers Free plan in the future.

Ö

ömer•6/12/24, 1:04 PM

Hello, is it possible to use Vectorize locally? For some reason vectorize binding is undefined even though I declared it in wrangler.toml

Y

yigit•6/12/24, 10:25 PM

Any plan to support million scale per namesace? The current limitations hold us back for large amount of vector (but low dim like 256) querying on CF.

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

Y

yigit•6/12/24, 10:26 PM

I want to be part of it

Öömer Hello, is it possible to use Vectorize locally? For some reason vectorize bindin...

J

Joey•6/13/24, 1:42 PM

you have to add the --remote flag to your wrangler command as vectorize is currently not supported locally,

Z

zwily•6/17/24, 11:37 PM

I just started using Vectorize, and the query result shape is different than what's in the @cloudflare/workers-types@4.20240614.0:

{
  "count": 1,
  "matches": [
    {
      "vectorId": "145",
      "score": 0.582596078,
      "vector": {
        "id": "145",
        "values": [],
        "metadata": {
          "content": "foo"
        }
      }
    }
  ]
}

{
  "count": 1,
  "matches": [
    {
      "vectorId": "145",
      "score": 0.582596078,
      "vector": {
        "id": "145",
        "values": [],
        "metadata": {
          "content": "foo"
        }
      }
    }
  ]
}

{
  "count": 1,
  "matches": [
    {
      "vectorId": "145",
      "score": 0.582596078,
      "vector": {
        "id": "145",
        "values": [],
        "metadata": {
          "content": "foo"
        }
      }
    }
  ]
}

{
  "count": 1,
  "matches": [
    {
      "vectorId": "145",
      "score": 0.582596078,
      "vector": {
        "id": "145",
        "values": [],
        "metadata": {
          "content": "foo"
        }
      }
    }
  ]
}

Note the new

vector

vector

object in each result object. This also doesn't match what's in the docs: https://developers.cloudflare.com/vectorize/reference/metadata-filtering/

Did this change recently?

Cloudflare Docs

Metadata filtering · Vectorize

In addition to providing an input vector to your query, you can also filter by vector metadata associated with every vector. Query results only …

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

Z

zwily•6/18/24, 12:42 AM

I am interested

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

T

thipperz•6/18/24, 9:14 PM

I'm also quite interested!

R

Rihan•6/19/24, 6:06 PM

Does Vectorize support local development the same way as D1, or does it run on Cloudflare when locally bound too (like Workers AI)?

R

Rihan•6/19/24, 6:25 PM

Looks like it's not :(
- https://developers.cloudflare.com/workers/testing/local-development/#supported-resource-bindings-in-different-environments
- https://github.com/cloudflare/workers-sdk/issues/4360

A

aiGeek•6/25/24, 12:52 AM

Are there any more business representative datasets for vectorize that have more complex JSON metadata available or planned? The number of elements need not be large. This would really help the community to easily try out querying and other API features. It also promotes the building of a knowledge base from a baseline dataset from which common questions will come about.

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

C

chain swanging•6/27/24, 9:46 AM

Hey man I'd really like to be on the waiting list, using vectorise a lot right now

R

Rihan•6/27/24, 3:50 PM

Is there any way to query for all record IDs stored within a Vectorize index? I'd like to know to build a way of viewing records (metadata mainly) like you can with D1, and to build a script to easily purge an index without fully recreating it.

RRihan Is there any way to query for all record IDs stored within a Vectorize index? I'...

Z

zwily•6/28/24, 10:06 AM

No. It’s a tough omission.

K

KorigamiK•6/28/24, 12:52 PM

Can you add me to the vectorize beta?

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

C

cosbgn•6/29/24, 11:03 AM

Hey @yevgen I'm exploring vectorize, could you tell me briefly what will be the limits/new features coming soon on v2?

C

cosbgn•6/29/24, 11:06 AM

One of my biggest concerns is 1K namespaces, if I want to add a namespace for each "workspace" my users might have I can only have a max of 100 users (if each user has 10 workspaces) which makes it really restrictive.

Alternatively I could do one namespace per users and use metadata to filter the correct workspace but seems inefficient

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

H

hydroxide•7/1/24, 4:54 AM

I'd like to trial this

V

Vinodh•7/1/24, 5:01 AM

Vectorize is not available for local development ?

VVinodh Vectorize is not available for local development ?

V

Vero•7/1/24, 8:37 AM

It's not supported yet

Ccosbgn One of my biggest concerns is 1K namespaces, if I want to add a namespace for ea...

U

ulysses•7/2/24, 1:48 AM

I have this question as well–what can we expect the limits to be?

T

thipperz•7/3/24, 5:29 PM

Please Vectorize V2 come soon

T

thipperz•7/3/24, 5:30 PM

my main concern is the topK

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

D

Derpenstein69•7/4/24, 7:58 PM

Please add me to the list, thank you

Yyevgen We are focusing to start onboarding first customers on new version of Vectorize ...

K

kingmesal•7/6/24, 6:52 PM

I'd like in.

B

Bierman•7/9/24, 2:02 PM

Is there a rough ETA on being able to run vectorize locally?

For context: I have a worker using a lot of cf services and it seems I have to use the api for vectorize as the —remote flag points all services to remote dev when really my goal is to run as many services locally as possible, then use remote for the unsupported ones. Is there a better way to handle this?

Ppdwittig Hey all 👋 - New product launch - Vectorize V2 In the next week or two, we'll s...

F

Fokke•7/9/24, 4:36 PM

Are you also increasing the number of vectors in an index or just the number of indexes? I am working on a use case where we need something close to 20-40MM vectors. We could store those all over multiple indexes, but a single index would be much preferred.

Or do you perhaps mean the number of vectors per index is increased to 2MM and the number of indexes stays the same?

Ppdwittig Hey all 👋 - New product launch - Vectorize V2 In the next week or two, we'll s...

Y

yigit•7/9/24, 6:28 PM

hey @pdwittig we have a unique case with 500m docs but each of them are 256 dimensions. do you think that we can try vectorize v2 for our usecase?

L

leesus•7/12/24, 12:46 PM

Is there any page/documentation around Vectorize error codes?

L

leesus•7/12/24, 12:47 PM

Am seeing a handful of upsert/delete/query errors in sentry with codes 2006 and 2008 and "internal server error" messaging but struggling to find any indication of what the actual issue is

M

Maltmax•7/14/24, 3:49 PM

Hi! I don't suppose there is a way to get all vectors stored in a vectorize database without knowing the vector IDs, right?

A

akazwz•7/16/24, 3:09 PM

@pdwittig hello team. my account id is a275d70ec37a6046e6778c77d65dd38b. and i have problems in dashboard vectorize. it is empty now.

I

Idle•7/17/24, 10:56 AM

?pings

F

Flare•7/17/24, 10:56 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

S

SteffTek•7/18/24, 7:48 AM

vectorize ui not loading :pain:

S

SteffTek•7/18/24, 7:48 AM

A

akazwz•7/19/24, 3:50 AM

me too

Y

yigit•7/20/24, 12:25 AM

Is there anyone with access to vectorize v2? I'm eagerly waiting.

Z

Zeeshan•7/24/24, 4:22 PM

Hi folks, just to confirm here once again, if i want to convert data in my d1 db to vectors, i can reach out to vectorise right? and it is paid right?

Z

zensin•7/26/24, 2:08 AM

I keep putting off part of a project hoping this will come out of beta. hows she lookin?

Zzensin I keep putting off part of a project hoping this will come out of beta. hows she...

D

DaniFoldi•7/26/24, 11:43 AM

:soontm: is the most accurate answer you'll probably get

Y

yevgen•7/31/24, 5:58 PM

I'm happy to announce that all customers who requested access to Vectorize V2 https://forms.gle/jtSJAqz478DsULN86 have been onboarded.

Google Docs

Vectorize V2 - Waitlist

In the next week or two, we'll start rolling out the next generation of Vectorize, which has been completely re-architected from the ground up. This new version supports 2M+ vector indexes, delivers significantly improved query latency, and much more.

Over the coming weeks, we'll be gradually inviting new teams to onboard. If you're interested ...

All, I'm wondering if anyone knows when/if the vector limit will be increased from 200k. That limit

Similar Threads

Similar Threads

Similar Threads