Any chance Claude Haiku or Sonnet will come to workers ai?

Aanywhichway Any chance Claude Haiku or Sonnet will come to workers ai?

I

Workers AI models are hosted by Cloudflare (all are open-source models), they're not external providers like Claude or OpenAI, so no they won't be coming.

I

Isaac McFadyen•12/4/24, 7:12 PM

You can use AI Gateway to proxy requests to Anthropic's Claude, though.

A

anywhichwayOP•12/4/24, 8:23 PM

Cluade licenses to Google, Amazon, and others. I was hoping Cloudflare would also strike a deal with them.

Aanywhichway Cluade licenses to Google, Amazon, and others. I was hoping Cloudflare would als...

I

Isaac McFadyen•12/4/24, 9:10 PM

While that might be possible, it would have to be a whole new product - Workers AI isn't designed for that. Plus, there's really not much point. The reason cloud providers license models like Claude is so that you can have unified billing if you already use that provider.

C

codebam•12/4/24, 10:53 PM

can someone help me with @cf/black-forest-labs/flux-1-schnell

C

codebam•12/4/24, 10:53 PM

when I save the file it doesn't open

C

codebam•12/4/24, 10:53 PM

I used

atob()

atob()

on it to convert the string to binary

C

codebam•12/4/24, 10:53 PM

am I supposed to do something else?

C

codebam•12/4/24, 10:54 PM

oh wait there's an example

C

codebam•12/4/24, 10:56 PM

the example doesn't work

Ccodebam can someone help me with @cf/black-forest-labs/flux-1-schnell

K

kyle•12/5/24, 1:45 AM

you have to convert from base64 to binary. in js I do

Buffer.from(base64Image, 'base64')

Buffer.from(base64Image, 'base64')

C

codebam•12/5/24, 1:46 AM

oh cool. that's a lot better than the thing cloudflare's example did

C

codebam•12/5/24, 1:46 AM

thanks

C

codebam•12/5/24, 1:48 AM

Buffer isn't in workers

Ccodebam Buffer isn't in workers

K

kyle•12/5/24, 1:49 AM

might have to do something here https://developers.cloudflare.com/workers/runtime-apis/nodejs/buffer/

Cloudflare Docs

Buffer · Cloudflare Workers docs

The Buffer API in Node.js is one of the most commonly used Node.js APIs for manipulating binary data. Every Buffer instance extends from the standard Uint8Array

class, but adds a range of unique capabilities such as built-in base64 and hex encoding/decoding, byte-order manipulation, and encoding-aware substring searching.

C

codebam•12/5/24, 1:50 AM

nah I'll just keep using

Uint8Array.from(binaryString, (m) => m.codePointAt(0));

Uint8Array.from(binaryString, (m) => m.codePointAt(0));

Uint8Array.from(binaryString, (m) => m.codePointAt(0));

Uint8Array.from(binaryString, (m) => m.codePointAt(0));

C

codebam•12/5/24, 1:50 AM

thanks tho

J

James•12/5/24, 1:53 AM

I asked this in the wrangler chat, but am I the only one who has to re-auth/re-allow wrangler to access workers AI locally every hour or two?

C

codebam•12/5/24, 1:54 AM

I don't have that issue

C

codebam•12/5/24, 1:54 AM

oh wait locally?

C

codebam•12/5/24, 1:54 AM

I didn't even know it worked locally. never used to

I

Isaac McFadyen•12/5/24, 2:00 AM

It now calls out to the remote Workers AI endpoint when developing locally, yeah.

J

James•12/5/24, 2:16 AM

Yeah it works, but it's irritating how I gave to re-auth it a few times a day

A

A3 Studio•12/5/24, 8:50 AM

Why can't I use workers ai with auth and api gateway?

    const response = await config.ai.run('@cf/black-forest-labs/flux-1-schnell',
      {
        prompt,
        steps: DEFAULT_STEPS
      },
      {
        gateway: { id: 'misu-ai' },
        extraHeaders: {
          Authorization: `Bearer ${config.aiGateway.apiToken}`,
          'cf-aig-authorization': `Bearer ${config.aiGateway.apiToken}`,
          'cf-aig-token': `Bearer ${config.aiGateway.apiToken}`
        }
      }
    )

    const response = await config.ai.run('@cf/black-forest-labs/flux-1-schnell',
      {
        prompt,
        steps: DEFAULT_STEPS
      },
      {
        gateway: { id: 'misu-ai' },
        extraHeaders: {
          Authorization: `Bearer ${config.aiGateway.apiToken}`,
          'cf-aig-authorization': `Bearer ${config.aiGateway.apiToken}`,
          'cf-aig-token': `Bearer ${config.aiGateway.apiToken}`
        }
      }
    )

    const response = await config.ai.run('@cf/black-forest-labs/flux-1-schnell',
      {
        prompt,
        steps: DEFAULT_STEPS
      },
      {
        gateway: { id: 'misu-ai' },
        extraHeaders: {
          Authorization: `Bearer ${config.aiGateway.apiToken}`,
          'cf-aig-authorization': `Bearer ${config.aiGateway.apiToken}`,
          'cf-aig-token': `Bearer ${config.aiGateway.apiToken}`
        }
      }
    )

    const response = await config.ai.run('@cf/black-forest-labs/flux-1-schnell',
      {
        prompt,
        steps: DEFAULT_STEPS
      },
      {
        gateway: { id: 'misu-ai' },
        extraHeaders: {
          Authorization: `Bearer ${config.aiGateway.apiToken}`,
          'cf-aig-authorization': `Bearer ${config.aiGateway.apiToken}`,
          'cf-aig-token': `Bearer ${config.aiGateway.apiToken}`
        }
      }
    )

works if i do it without api gateway. with api gateway everytime i get an unathorised error. i have done it with just this header cf-aig-authorization and all the other headers on their own. I am doing open ai requests through my gateway its just workers ai. 0 examples in the dcos doing config.ai.run with an auth tokens. i just assumed i would add the auth to the extra header or something

LLogan Grasby Updates like @cf/openai/whisper-large-v3 with inputs: ``` { "task": "translat...

M

mr.niko.la•12/5/24, 6:57 PM

Whisper v3 turbo is on cf workers ???

C

Clark Olson-Smith•12/7/24, 1:46 AM

Need help with Cloudflare Vectorize + Workers AI embedding query. Getting 'invalid query vector, expected 768 dimensions, and got 0 dimensions' despite vector data being present. Here's my code:

const embedding = await env.AI.run('@cf/baai/bge-base-en-v1.5', {
  text: query
});

const values = Array.from(embedding.data[0]);
const results = await vectorIndex.query({
  topK: 5,
  values: values
});

const embedding = await env.AI.run('@cf/baai/bge-base-en-v1.5', {
  text: query
});

const values = Array.from(embedding.data[0]);
const results = await vectorIndex.query({
  topK: 5,
  values: values
});

const embedding = await env.AI.run('@cf/baai/bge-base-en-v1.5', {
  text: query
});

const values = Array.from(embedding.data[0]);
const results = await vectorIndex.query({
  topK: 5,
  values: values
});

const embedding = await env.AI.run('@cf/baai/bge-base-en-v1.5', {
  text: query
});

const values = Array.from(embedding.data[0]);
const results = await vectorIndex.query({
  topK: 5,
  values: values
});

Logs show embedding.data[0] contains 768 values but query fails. Tried multiple query formats. Any suggestions?

P

Paige•12/7/24, 4:00 PM

Is hosting of custom models something that is expected to be a feature relatively soon for ai workers?

I

Isaac McFadyen•12/7/24, 4:25 PM

?workers-ai-models

S

SuperHelpflare•12/7/24, 4:25 PM

Workers AI currently only supports popular open-source models provided by the Cloudflare team, as well as your own LoRAs that can be applied on top of the Cloudflare-provided models. You cannot currently upload your own models or use a model from HuggingFace. See the documentation for the list of Cloudflare-provided models: https://developers.cloudflare.com/workers-ai/models/

I

Isaac McFadyen•12/7/24, 4:25 PM

^ TLDR: probably not, no

B

Ben-xD•12/7/24, 10:29 PM

I noticed my friend has a workers AI page which shows usage, but mine doesn't. My friend did find a bug in the usage values though. It just shows "Welcome to Workers AI" on https://dash.cloudflare.com/$accountId/ai/workers-ai

Is the detailed workers page a feature being AB tested?

A

Aaron•12/7/24, 10:54 PM

is the workers ai limit charged per worker

A

Aaron•12/7/24, 10:54 PM

or is it global

A

Aaron•12/7/24, 10:55 PM

in other words i have 10 workers

A

Aaron•12/7/24, 10:55 PM

is the effective limit 3200 per minute

A

Aaron•12/7/24, 10:55 PM

vs 320 per minute across 10 workers

A

Aaron•12/7/24, 10:55 PM

320 per minute seemed pretty low

A

Aaron•12/7/24, 10:55 PM

for text generation

A

Aaron•12/7/24, 10:58 PM

sorry it's actually 300

B

Ben-xD•12/8/24, 12:39 AM

Has anyone tried to run tests (vitest) with a project that uses Workers AI? I'm getting:

orkerd/server/workerd-api.c++:759: error: wrapped binding module can't be resolved (internal modules only); moduleName = miniflare-internal:wrapped:__WRANGLER_EXTERNAL_AI_WORKER
workerd/jsg/util.c++:274: error: e = workerd/server/workerd-api.c++:789: failed: expected !value.IsEmpty(); global did not produce v8::Value
stack: 102abb063 10302858b 10300fe4b 102a679a7 102a6d647 102a70d7b 102a6f58b 102a6124f 104f28b03 104f28e13 104f277ff 104f27583 102a507c7 18c208273; sentryErrorContext = jsgInternalError
service core:user:vitest-pool-workers-runner-: Uncaught Error: internal error

orkerd/server/workerd-api.c++:759: error: wrapped binding module can't be resolved (internal modules only); moduleName = miniflare-internal:wrapped:__WRANGLER_EXTERNAL_AI_WORKER
workerd/jsg/util.c++:274: error: e = workerd/server/workerd-api.c++:789: failed: expected !value.IsEmpty(); global did not produce v8::Value
stack: 102abb063 10302858b 10300fe4b 102a679a7 102a6d647 102a70d7b 102a6f58b 102a6124f 104f28b03 104f28e13 104f277ff 104f27583 102a507c7 18c208273; sentryErrorContext = jsgInternalError
service core:user:vitest-pool-workers-runner-: Uncaught Error: internal error

orkerd/server/workerd-api.c++:759: error: wrapped binding module can't be resolved (internal modules only); moduleName = miniflare-internal:wrapped:__WRANGLER_EXTERNAL_AI_WORKER
workerd/jsg/util.c++:274: error: e = workerd/server/workerd-api.c++:789: failed: expected !value.IsEmpty(); global did not produce v8::Value
stack: 102abb063 10302858b 10300fe4b 102a679a7 102a6d647 102a70d7b 102a6f58b 102a6124f 104f28b03 104f28e13 104f277ff 104f27583 102a507c7 18c208273; sentryErrorContext = jsgInternalError
service core:user:vitest-pool-workers-runner-: Uncaught Error: internal error

orkerd/server/workerd-api.c++:759: error: wrapped binding module can't be resolved (internal modules only); moduleName = miniflare-internal:wrapped:__WRANGLER_EXTERNAL_AI_WORKER
workerd/jsg/util.c++:274: error: e = workerd/server/workerd-api.c++:789: failed: expected !value.IsEmpty(); global did not produce v8::Value
stack: 102abb063 10302858b 10300fe4b 102a679a7 102a6d647 102a70d7b 102a6f58b 102a6124f 104f28b03 104f28e13 104f277ff 104f27583 102a507c7 18c208273; sentryErrorContext = jsgInternalError
service core:user:vitest-pool-workers-runner-: Uncaught Error: internal error

BBen-xD Has anyone tried to run tests (vitest) with a project that uses Workers AI? I'm ...

A

Aaron•12/8/24, 1:40 AM

hody jones?

BBen-xD I noticed my friend has a workers AI page which shows usage, but mine doesn't. M...

I

Isaac McFadyen•12/8/24, 3:48 AM

If you're only using beta models they won't show usage.

B

Ben-xD•12/8/24, 3:49 AM

I've been using non-beta models (identical code/commit to my friend, sitting next to them) but mine doesn't show usage. We've been using non-beta models the entire day

AAaron hody jones?

B

Ben-xD•12/8/24, 3:50 AM

What does hody jones mean?

S

Sadath•12/9/24, 2:49 PM

Hi , lets say I don't use the ai models hosted by cloudfare and instead import it from somewhere else using workers ai python-sdk . Will the ai inference at edge still happen ?

S

snoob•12/9/24, 10:08 PM

Has anyone experienced this type error issue for the new models? Im following the tutorial pretty much to a tee thats at https://github.com/craigsdennis/punderful-workflows/tree/main.

It seems i only have access to the old models in autocomplete. Im using the latest @cloudflare/workers-types: ^4.20241205.0.

Tsconfig.json is the same as the punderful tutorial linked above:

{
  "compilerOptions": {
    "target": "ESNext",
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "strict": true,
    "skipLibCheck": true,
    "lib": [
      "ESNext"
    ],
    "types": [
        "@cloudflare/workers-types/experimental"
    ],
    "jsx": "react-jsx",
    "jsxImportSource": "hono/jsx"
  },
}

{
  "compilerOptions": {
    "target": "ESNext",
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "strict": true,
    "skipLibCheck": true,
    "lib": [
      "ESNext"
    ],
    "types": [
        "@cloudflare/workers-types/experimental"
    ],
    "jsx": "react-jsx",
    "jsxImportSource": "hono/jsx"
  },
}

{
  "compilerOptions": {
    "target": "ESNext",
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "strict": true,
    "skipLibCheck": true,
    "lib": [
      "ESNext"
    ],
    "types": [
        "@cloudflare/workers-types/experimental"
    ],
    "jsx": "react-jsx",
    "jsxImportSource": "hono/jsx"
  },
}

{
  "compilerOptions": {
    "target": "ESNext",
    "module": "ESNext",
    "moduleResolution": "Bundler",
    "strict": true,
    "skipLibCheck": true,
    "lib": [
      "ESNext"
    ],
    "types": [
        "@cloudflare/workers-types/experimental"
    ],
    "jsx": "react-jsx",
    "jsxImportSource": "hono/jsx"
  },
}

Is this an issue with the Workflows not having the right type definitions?

S

snoob•12/9/24, 10:46 PM

Im able to run the workflow no problem, Im in a turbo repo so probably messing something up

S

snoob•12/9/24, 11:01 PM

Oh interesting, the new models dont even work with typescript in that example either. LOL

J

James•12/10/24, 8:15 PM

Yeah the Workers AI types are months out of date, but hopefully we'll see an update soon: https://x.com/_mchenco/status/1865865722913841282

michelle (@_mchenco) on X

@CherryJimbo @CloudflareDev its coming! figured out a more sustainable way to do it

Twitter

•

12/8/24, 9:07 PM

Z

zcpbx•12/12/24, 3:33 AM

( I’m new to clourflare AI. ) if the models I am interested in running are not listed is there a way to package/distribute weights for them? Thank you

Any chance Claude Haiku or Sonnet will come to workers ai?

Similar Threads

Similar Threads

Similar Threads