Hey! Whats the limit on insert/upsert. for some reasons, my vector inserts/upsers have stoped at 12

E

EndiMOP•10/30/24, 9:10 PM

@yevgen

Y

Yota•10/30/24, 9:25 PM

Is there a way to dynamically add Vectorize bindings programatically?

YYota Is there a way to dynamically add Vectorize bindings programatically?

Y

Yota•10/30/24, 9:27 PM

ah i found it https://developers.cloudflare.com/vectorize/best-practices/create-indexes/

Y

Yota•10/30/24, 9:30 PM

Hmmm, but these don't seem to be enough number for my use-case
Indexes per account 50,000 (Workers Paid)

E

EndiMOP•10/31/24, 12:26 AM

This issue is happening from : 2024-10-25T14:36:55.873Z
whatever improvement was made on this date, caused this to break.
Triggering it from binding or api, even tho the call return succees true, it actually does nothing on the index. No delete, no insert, no upset

EEndiM Hey! Whats the limit on insert/upsert. for some reasons, my vector inserts/upse...

Y

yevgen•10/31/24, 8:52 AM

Nothing should have caused such issues, let me check with the team, it could be delayed as most of the team is in US time zone.

Yyevgen Nothing should have caused such issues, let me check with the team, it could be ...

E

EndiMOP•10/31/24, 8:52 AM

Thanks for getting back! Okay thank you.

Y

yevgen•11/1/24, 1:16 PM

Vectorize update:

2024-10-30: Vectorize has local mode support in Wrangler @3.84.0 https://github.com/cloudflare/workers-sdk/releases/tag/wrangler%403.84.0

GitHub

Release wrangler@3.84.0 · cloudflare/workers-sdk

Minor Changes

#6999 0111edb Thanks @garvit-gupta! - docs: Vectorize GA Announcement Banner

#6916 a33a133 Thanks @garrettgu10! - Local development now supports Vectorize bindings

#7004 15ef013...

S

Samko•11/1/24, 2:53 PM

api idea - It would be great to have method for get/delete (all)vectors by

namespace

namespace

namespace

namespace mainly delete use-case would be very helpful - in my case I have namespace for every "project" and I would like an easy way to clean-up.

A

ac•11/1/24, 9:09 PM

I am noticing a large number of

AiError: 3001: Unknown internal error

AiError: 3001: Unknown internal error

AiError: 3001: Unknown internal error

AiError: 3001: Unknown internal error s when doing some Vectorize queries. Is there any way to get more insight into what these are? I have no clue if it's not finding a vector by ID, if I'm getting rate limited, or if it's something I can't control

T

thousandmiles•11/5/24, 1:23 AM

I've been browsing through messages in the past and documentations on vectorize, looks like now it's a great time to jump in, considering the beta release of workflow as well. though my concern is full RAG pipeline still needs a long way to go, so I'd prefer develop locally using llamaindex+chromadb(or equivalent)+litellm(embedding, inference, rerank, whisper, etc) first, before switching to cloudflare edge side completely, which I do need at a certain point because the product I am building requires global coverage.

may I ask guy who already had previous experiences here, that, is this a viable plan, or anything I should keep in mind for choosing tech stack or develop or doing gitops? thanks a million!

R

Rubi•11/7/24, 8:44 AM

Hi, is Cloudflare Vectorize now support $in metadata filter ?

Y

yevgen•11/13/24, 8:19 PM

Vectorize update:
Added support for

$in

$in

$in

$in and

$nin

$nin

$nin

$nin metadata filters
https://developers.cloudflare.com/vectorize/platform/changelog/

Cloudflare Docs

Changelog | Vectorize

L

leesus•11/15/24, 11:30 AM

Is supporting arrays with metadata filtering on the roadmap? I'm migrating a v1 index to v2, and looking at adding a metadata index while I'm at it, but my metadata has string array properties. According to the docs this isn't possible now, but will it be in the near future? If not, I'll need to rethink how my metadata is stored before the migration.

C

CedricHadjian•11/18/24, 4:43 PM

How can I see what vectors I have in my index? I checked the commands I could run, and none mentioned showing the vectors I got in my index. I want to be able to delete and test stuff before I run the actual thing. Any guidance would be appreciated.

L

leesus•11/21/24, 1:27 PM

Are there any tips and tricks to improve query performance against Vectorize?

We make use of (per-project) namespaces for limiting the scope of similarity searches, and are seeing very slow performance on initial query against a namespace, and subsequent queries are much faster. One example has roughly 100 vectors and is taking ~3s for first query vs ~150ms for subsequent queries. Are cold-starts a thing with Vectorize?

M

momegas•11/22/24, 5:44 PM

Hello, Quick question, I get really slow response times in Vectorise. Like seconds of response time. My code is very simple, so nothing else should be slowing it down. I am in Greece btw, so not sure if that plays a role. I get the same speeds when deploying the worker.

  }),
  async (c) => {
    const text = "XXX";
    const e = await embedText(c, text);
    const r = await c.env.VECTORIZE.query(e, {
      topK: 5,
      returnValues: false,
      returnMetadata: "none",
    });

    return c.json({ matches: r.matches });
  }
);

  }),
  async (c) => {
    const text = "XXX";
    const e = await embedText(c, text);
    const r = await c.env.VECTORIZE.query(e, {
      topK: 5,
      returnValues: false,
      returnMetadata: "none",
    });

    return c.json({ matches: r.matches });
  }
);

  }),
  async (c) => {
    const text = "XXX";
    const e = await embedText(c, text);
    const r = await c.env.VECTORIZE.query(e, {
      topK: 5,
      returnValues: false,
      returnMetadata: "none",
    });

    return c.json({ matches: r.matches });
  }
);

  }),
  async (c) => {
    const text = "XXX";
    const e = await embedText(c, text);
    const r = await c.env.VECTORIZE.query(e, {
      topK: 5,
      returnValues: false,
      returnMetadata: "none",
    });

    return c.json({ matches: r.matches });
  }
);

I

Isaac McFadyen•11/27/24, 12:06 AM

@Kingsley Michael I removed your message as this is the channel for #vectorize, please use #off-topic for anything unrelated to Cloudflare products

A

Alex Karp•11/27/24, 2:01 AM

The limits page says that there's a maximum of 5,000,000 vectors per index, but I'm getting a 4002 error when attempting to upsert past 240,000. any idea why or things I should change?

EDIT: RESOLVED, I was using V1 instead of V2

C

CedricHadjian•11/27/24, 9:08 PM

TypeError: vectors.map is not a function

What does this mean when it's triggered from this:
await vectorIndex.insert({
id: String(data[0][dataId]),
values: vectorArray,
metadata
});

EDIT:
I tested it on a small function:

            if (request.method === 'GET' && pathname === `/embed`) {
                try {
                    const text = "Liam Marshall"

                    const response = await env.AI.run("@cf/baai/bge-small-en-v1.5", {
                        text,
                    });
    
                    await env.TESTVECTORS.insert({
                        id: 1,
                        values: response.data[0],
                        metadata: {
                            key: "value"
                        }
                    })
                    
                    return new Response(JSON.stringify(response), {
                        headers: {
                            'content-type': 'text/plain;charset=UTF-8',
                            'Access-Control-Allow-Origin': '*',
                        },
                    });
                } catch (error) {
                    console.log(error);
                }

            }

            if (request.method === 'GET' && pathname === `/embed`) {
                try {
                    const text = "Liam Marshall"

                    const response = await env.AI.run("@cf/baai/bge-small-en-v1.5", {
                        text,
                    });
    
                    await env.TESTVECTORS.insert({
                        id: 1,
                        values: response.data[0],
                        metadata: {
                            key: "value"
                        }
                    })
                    
                    return new Response(JSON.stringify(response), {
                        headers: {
                            'content-type': 'text/plain;charset=UTF-8',
                            'Access-Control-Allow-Origin': '*',
                        },
                    });
                } catch (error) {
                    console.log(error);
                }

            }

            if (request.method === 'GET' && pathname === `/embed`) {
                try {
                    const text = "Liam Marshall"

                    const response = await env.AI.run("@cf/baai/bge-small-en-v1.5", {
                        text,
                    });
    
                    await env.TESTVECTORS.insert({
                        id: 1,
                        values: response.data[0],
                        metadata: {
                            key: "value"
                        }
                    })
                    
                    return new Response(JSON.stringify(response), {
                        headers: {
                            'content-type': 'text/plain;charset=UTF-8',
                            'Access-Control-Allow-Origin': '*',
                        },
                    });
                } catch (error) {
                    console.log(error);
                }

            }

            if (request.method === 'GET' && pathname === `/embed`) {
                try {
                    const text = "Liam Marshall"

                    const response = await env.AI.run("@cf/baai/bge-small-en-v1.5", {
                        text,
                    });
    
                    await env.TESTVECTORS.insert({
                        id: 1,
                        values: response.data[0],
                        metadata: {
                            key: "value"
                        }
                    })
                    
                    return new Response(JSON.stringify(response), {
                        headers: {
                            'content-type': 'text/plain;charset=UTF-8',
                            'Access-Control-Allow-Origin': '*',
                        },
                    });
                } catch (error) {
                    console.log(error);
                }

            }

I got the same error: vectors.map is not a function...

CCedricHadjian TypeError: vectors.map is not a function What does this mean when it's triggere...

C

CedricHadjian•11/27/24, 10:25 PM

@yevgen Any help would be appreciated, we're a high tier client and we want to implement vectorize over our app. But we're facing vectors.map is not a function on simple calls as well.

CCedricHadjian @yevgen Any help would be appreciated, we're a high tier client and we want to i...

G

ggu•11/29/24, 6:57 PM

Does this happen just locally or also when deployed?

Gggu Does this happen just locally or also when deployed?

C

CedricHadjian•11/29/24, 6:57 PM

Both

C

CedricHadjian•11/29/24, 6:58 PM

This wasn't an issue before, it suddenly became an issue. I also created a fresh new worker and the issue still happened.

G

garvitg•11/30/24, 1:23 AM

Hi @CedricHadjian. Thanks for reaching out! I was able to tweak the code you have shared a bit and was able to get it to work. This is a codeblock that works as expected.

            try {
                const text = 'Liam Marshall';

                const ai_response = await env.AI.run('@cf/baai/bge-small-en-v1.5', {
                    text,
                });

                const vectorize_response = await env.TESTVECTORS.insert([
                    {
                        id: '1',
                        values: ai_response.data[0],
                        metadata: {
                            key: 'value',
                        },
                    },
                ]);

                return new Response(JSON.stringify(vectorize_response), {
                    headers: {
                        'content-type': 'text/plain;charset=UTF-8',
                        'Access-Control-Allow-Origin': '*',
                    },
                });
            } catch (error) {
                console.log(error);
            }

            try {
                const text = 'Liam Marshall';

                const ai_response = await env.AI.run('@cf/baai/bge-small-en-v1.5', {
                    text,
                });

                const vectorize_response = await env.TESTVECTORS.insert([
                    {
                        id: '1',
                        values: ai_response.data[0],
                        metadata: {
                            key: 'value',
                        },
                    },
                ]);

                return new Response(JSON.stringify(vectorize_response), {
                    headers: {
                        'content-type': 'text/plain;charset=UTF-8',
                        'Access-Control-Allow-Origin': '*',
                    },
                });
            } catch (error) {
                console.log(error);
            }

            try {
                const text = 'Liam Marshall';

                const ai_response = await env.AI.run('@cf/baai/bge-small-en-v1.5', {
                    text,
                });

                const vectorize_response = await env.TESTVECTORS.insert([
                    {
                        id: '1',
                        values: ai_response.data[0],
                        metadata: {
                            key: 'value',
                        },
                    },
                ]);

                return new Response(JSON.stringify(vectorize_response), {
                    headers: {
                        'content-type': 'text/plain;charset=UTF-8',
                        'Access-Control-Allow-Origin': '*',
                    },
                });
            } catch (error) {
                console.log(error);
            }

            try {
                const text = 'Liam Marshall';

                const ai_response = await env.AI.run('@cf/baai/bge-small-en-v1.5', {
                    text,
                });

                const vectorize_response = await env.TESTVECTORS.insert([
                    {
                        id: '1',
                        values: ai_response.data[0],
                        metadata: {
                            key: 'value',
                        },
                    },
                ]);

                return new Response(JSON.stringify(vectorize_response), {
                    headers: {
                        'content-type': 'text/plain;charset=UTF-8',
                        'Access-Control-Allow-Origin': '*',
                    },
                });
            } catch (error) {
                console.log(error);
            }

Summary of the fixes:

I enclosed the object being passed to the Vectorize insert operation in square brackets. This is needed because the insert function requires the input to be of an Array type.
I converted the id from 1 to '1'. This is needed because Vectorize vector identifiers need to be of a String type.

Feel free to reach out if you need additional support for Vectorize!

Ggarvitg Hi @CedricHadjian. Thanks for reaching out! I was able to tweak the code you hav...

C

CedricHadjian•11/30/24, 11:44 AM

Thank you so much, I spent 6 hours trying to figure out what the issue is and it turns out to be the brackets, I also remember trying stringifying the id but it still never worked.

CCedricHadjian Thank you so much, I spent 6 hours trying to figure out what the issue is and it...

G

garvitg•11/30/24, 5:22 PM

Happens to the best of us, I am glad we could help you out with this! Something that helps us when we develop Worker code is to setup the Worker locally as an npm project. Then you can load the project into your preferred IDE which would almost certainly display relevant packages and method signatures from

node_modules

node_modules

. This project code can also be pushed to your preferred Version Control System if it is a collaborative project. You can refer to https://developers.cloudflare.com/vectorize/get-started/embeddings/ for an example on setting up the project from scratch.

Cloudflare Docs

Vectorize and Workers AI · Vectorize

Vectorize allows you to generate vector embeddings using a machine-learning model, including the models available in Workers AI.

Ggarvitg Happens to the best of us, I am glad we could help you out with this! Something ...

C

CedricHadjian•11/30/24, 5:58 PM

Thank you, I had it working on another test worker that I was playing around with and when it came to production, I got stuck at this because I didn't notice the brackets

C

CedricHadjian•11/30/24, 6:02 PM

Is it possible to insert 5000 at the same time using insert function? As far as I know it's possible to do it through embeddings.ndjson file, but would 5000 work or is there another limit?

CCedricHadjian Is it possible to insert 5000 at the same time using insert function? As far as ...

G

garvitg•12/1/24, 2:41 AM

The current batch size limit is 1000 vectors, when you insert/upsert using a Worker: https://developers.cloudflare.com/vectorize/platform/limits/. A higher limit of 5000 vectors is available for inserts/upserts via the HTTP API.

Cloudflare Docs

Limits · Vectorize

The following limits apply to accounts, indexes and vectors (as specified):

G

gilxng•12/1/24, 2:33 PM

Hi @garvitg , I got question regarding the workers ai and vectorize, I have an app running Resnet50 to vectorize image, I'm planning to switch over to cloudflare since you guys support Resnet50, however, I saw the limitiations of the Vectorize it says it has maximum dimensions per vector 1536 dimensions, which is not high enough for my app since it generate 2048 dimensions using Resnet50, any idea how can I reduce the dimensions that is generated from Resnet50?

I'm really new to this field, so any help would be awesome, thanks!

D

DCsan•12/2/24, 6:36 PM

is it possible to load stuff into a prod vectorize index from the local machine? or is the data running in the miniflare / simulator only?

M

Mitya•12/4/24, 3:05 PM

Hi everyone. I wonder if someone could help me fill some gaps in my knowledge relating to using Vectorize (and vectorisation generally) to help with AI data analysis on potentially large datasets.

Without Vectorize, I can send data to an LLM for analysis/categorisation etc., but I'm of course limited by tokens as to how much data I can send. I have done some reading and I understand that getting my data into a Vectorize DB, then querying that as a preliminary step to sending data off to the LLM for analysis, might be the answer here.

But I'm hazy on the exact steps/rationale. Specifically, how does this help limit the amount of data I ultimately send to the LLM? Is the idea that Vectorize reduces it to a subset, so duplicates or near-duplicates are merged/omitted, or...? (Sorry for the long message - just looking for some high-level guidance!)

MMitya Hi everyone. I wonder if someone could help me fill some gaps in my knowledge re...

I

Isaac McFadyen•12/4/24, 3:43 PM

So the idea with RAG is this:

you embed all your documents and store them in a vector DB (vectorize or otherwise)
when a user question comes in, you embed the user question
you do a search in the vector DB for "top <n> items" where
```
n
```
n
```
n
```
n
is 3, or 5, or 10, or higher depending on your use-case
you give the results to the LLM along with the query

The idea is that, because embeddings between related items are similar, when you query the vector DB with the embedding of the user question, you're likely to get back things in the vector DB related to the user question. Then you give that to the LLM to actually answer the question based on the gathered knowledge.

IIsaac McFadyen So the idea with RAG is this: - you embed all your documents and store them in a...

M

Mitya•12/4/24, 3:53 PM

Thank you, that's really helpful. So this model depends on a particular question as a filter, and is less useful if, say, I simply want to run sentiment analysis on 100,000 user messages. Right?

MMitya Thank you, that's really helpful. So this model depends on a particular question...

I

Isaac McFadyen•12/4/24, 3:55 PM

So you can definitely run sentiment analysis using an LLM, but you wouldn't really need a RAG pipeline for that unless your sentiment depends on some external knowledge you want to give the LLM.

M

Mitya•12/4/24, 3:56 PM

Right. My thinking for introducing RAG was to get round the issue of there being a limit on how much data I can send to an LLM in one go. Perhaps that requires a different solution i.e. piecemeal analysis on 1000 messages at a time and then derive an average of all the sentiments or something.

MMitya Right. My thinking for introducing RAG was to get round the issue of there being...

I

Isaac McFadyen•12/4/24, 3:57 PM

Do the sentiments relate to one another?

I

Isaac McFadyen•12/4/24, 3:57 PM

As in, usually you'd only send a few messages at a time, but do you need the context of all the messages at once to derive overall sentiment?

M

Mitya•12/4/24, 3:59 PM

Probably yes, because the user messages are written freely. A real example: Users might feed back to the question "What do you think of our show?". And we'd like to do two things: 1) give sentiment analysis e.g. people are mostly happy; 2) have the LLM give suggestions e.g. "People want more matinée shows"

MMitya Probably yes, because the user messages are written freely. A real example: User...

I

Isaac McFadyen•12/4/24, 3:59 PM

I see. So usually sending all the messages at once is going to significantly degrade your result and cost a lot.

I

Isaac McFadyen•12/4/24, 4:00 PM

Instead you'd analyze each message individually, and then do what you want with the statistics.

I

Isaac McFadyen•12/4/24, 4:00 PM

That also allows different kinds of reporting: "10 users disliked it but the majority thought it was great" or "1% of users gave <x> feedback about the show" etc.

I

Isaac McFadyen•12/4/24, 4:01 PM

The only scenario where you'd want to send all the messages at once to an LLM would be if each message depended on each other.

I

Isaac McFadyen•12/4/24, 4:02 PM

In general the more data you send to an LLM the worse the quality gets as well. For example, Gemini advertises 1M context length but as soon as you get past about 100k the quality plummets (based on multiple reports from different sources)

M

Mitya•12/4/24, 4:03 PM

Interesting - didn't realise that. There's obvious cost/latency issues with sending more data but didn't realise the quality dropped off. OK so you're suggesting (if I understand correctly) analysing each message as it comes in and storing that analysis locally, for later aggregation/insights.

MMitya Probably yes, because the user messages are written freely. A real example: User...

I

Isaac McFadyen•12/4/24, 4:05 PM

Yeah, basically. So in this specific scenario I'd do this:

grab a message, ask the LLM something like:
Analyze this comment about this show <give context on the show here>. Respond with an overall sentiment, such as "positive" or "negative", and the most prominent piece of feedback in the user review
throw that into a DB somewhere
then you can analyze "10k reviews were positive, 10k were negative" and then also "100 feedbacks said that the show should add <x> part"

I

Isaac McFadyen•12/4/24, 4:06 PM

The trickiest part here is going to be matching user feedback so that you can say "100 users gave this feedback" but you can actually maybe do that with a vector DB because similar feedbacks will have similar embeddings.

Hey! Whats the limit on insert/upsert. for some reasons, my vector inserts/upsers have stoped at 12

Similar Threads

Similar Threads

Similar Threads