when will `--experimental-vectorize-bind

when will --experimental-vectorize-bind-to-prod flag get stable, it fails 4/5 times resulting in 10000 Auth error

7 Replies

Hi @Dev . Could you please share any details about how you're running into that issue? What steps lead to that error?

yevgen•2w ago

We can't reliably reproduce issue you described. Without a way to reliably reproduce it we can't chase Wrangler team about the fix or assistance of root cause analysis.

DevOP•2w ago

Hi, i have this dev script in package.json of my backend app. "dev": "bun run format && npx wrangler dev --experimental-vectorize-bind-to-prod --port=3000" so when i call the vectorize upsert procedure, it gives the following 10000 Auth Error when generating embeddings as this is the first point of contact to the binding.

 InferenceUpstreamError: 10000: Authentication error
    at Ai._parseError (cloudflare-internal:ai-api:181:24)
    at async Ai.run (cloudflare-internal:ai-api:154:19)
    at async generateEmbeddings (file:///home/dsrao/Programs/react-router/airtheon-new/apps/server/src/lib/generate-embeddings.ts:17:20)

 InferenceUpstreamError: 10000: Authentication error
    at Ai._parseError (cloudflare-internal:ai-api:181:24)
    at async Ai.run (cloudflare-internal:ai-api:154:19)
    at async generateEmbeddings (file:///home/dsrao/Programs/react-router/airtheon-new/apps/server/src/lib/generate-embeddings.ts:17:20)

it eventually works after multiple tries. I have checked all the necessary api tokens and I'm always logged in through wrangler >generate-embeddings.ts

interface AiResponse {
    data: number[][];
}

export async function generateEmbeddings(
    chunks: string[],
    env: Env,
    userId: string,
    memoryId: string,
    idPrefix: string,
    workspaceId: string
) {
    const vectors: VectorizeVector[] = [];
    const today = new Date();
    for (let i = 0; i < chunks.length; i++) {
        // biome-ignore lint/nursery/noAwaitInLoop: false positive
        const response = await env.AI.run("@cf/baai/bge-m3", {
            text: [chunks[i]],
        });
        vectors.push({
            id: `${idPrefix}_${memoryId}_chunk_${i}`,
            values: (response as AiResponse).data[0],
            metadata: {
                chunk_index: i,
                user_id: userId,
                memory_id: memoryId,
                workspace_id: workspaceId,
                timestamp: today.toISOString(),
            },
        });
    }
    return vectors;
}

interface AiResponse {
    data: number[][];
}

export async function generateEmbeddings(
    chunks: string[],
    env: Env,
    userId: string,
    memoryId: string,
    idPrefix: string,
    workspaceId: string
) {
    const vectors: VectorizeVector[] = [];
    const today = new Date();
    for (let i = 0; i < chunks.length; i++) {
        // biome-ignore lint/nursery/noAwaitInLoop: false positive
        const response = await env.AI.run("@cf/baai/bge-m3", {
            text: [chunks[i]],
        });
        vectors.push({
            id: `${idPrefix}_${memoryId}_chunk_${i}`,
            values: (response as AiResponse).data[0],
            metadata: {
                chunk_index: i,
                user_id: userId,
                memory_id: memoryId,
                workspace_id: workspaceId,
                timestamp: today.toISOString(),
            },
        });
    }
    return vectors;
}

>upsert-procedure

const vectors = await generateEmbeddings(
    chunks,
    ctx.env,
    ctx.session.user.id,
    memoryId,
    "note",
    ctx.session.session.activeOrganizationId
);

await ctx.env.VECTORIZE_NOTES.upsert(vectors);

const vectors = await generateEmbeddings(
    chunks,
    ctx.env,
    ctx.session.user.id,
    memoryId,
    "note",
    ctx.session.session.activeOrganizationId
);

await ctx.env.VECTORIZE_NOTES.upsert(vectors);

garvitg•2w ago

Thank you for the details. We are investigating this issue. I am still unable to reproduce this. Do you get this error when you leave the dev mode running for some time and come back to it? Or is there any other usage pattern that makes the error more likely to happen? Wait, the stacktrace here seems to indicate that the error is generated by the Ai.run command within generateEmbeddings. If this is the stacktrace you observe each time, then it may not be a Vectorize issue.

DevOP•2w ago

its completely random, one time it will work and gives error on next I haven't received this error in prod, only happens in dev

garvitg•2w ago

Workers AI also runs a mixed mode setting in the dev more afaik. The stacktrace indicates that the error comes from the 'AI.run' command.

DevOP•2w ago

Okay, got it Thanks!

Gaming

Programming

when will `--experimental-vectorize-bind

Did you find this page helpful?