No documentation I can find and only this community entry -> https://community.cloudflare.com/t/infe

Rrichw Hi all. I've got my Worker running AI models, which is great. I can see on https...

K

Kristian•10/30/24, 1:55 PM

Hey there! I have seen the same issue, I'll ask around internally and see what I can find out for you. Thanks for surfacing this

Ffry69 No documentation I can find and only this community entry -> https://community.c...

K

Kristian•10/30/24, 1:57 PM

Will work on improving documentation here. Thanks!

KKristian Will work on improving documentation here. Thanks!

M

Martijn•10/30/24, 2:04 PM

Do you know what the limit is? Specifically for the image AIs

MMartijn Do you know what the limit is? Specifically for the image AIs

K

Kristian•10/30/24, 2:04 PM

Not sure! Working on figuring that out today for ya

M

Martijn•10/30/24, 2:05 PM

Awesome thanks! Also let me know if there is a way to increase this limit

P

pigri•10/30/24, 3:29 PM

If anyone else needs an AI mock service, we've created one! You can check it out on GitHub: https://github.com/openshieldai/mockai or access it directly here: https://mockai.openshield.ai. Currently, OpenAI is supported, but we will add more providers.

Ppigri If anyone else needs an AI mock service, we've created one! You can check it out...

F

fry69OP•10/30/24, 3:30 PM

GitHub link is 404

Ffry69 GitHub link is 404

P

pigri•10/30/24, 3:38 PM

Thank you! Public now

Ppigri Thank you! Public now

F

fry69OP•10/30/24, 3:43 PM

Looks great at a quick glance. I'd also announce this in -> #what-i-built

MMartijn Do you know what the limit is? Specifically for the image AIs

R

Raylight•10/30/24, 4:09 PM

The limit is "fuzzy". Around 2.5-3MB depending on the content. Seems to apply to all models. (

JSON.stringify(inputs).length

JSON.stringify(inputs).length

JSON.stringify(inputs).length

JSON.stringify(inputs).length needs to be less than 9.5 MB, to be precise. I.e. the input to AI.run is stringified at one point internally and must not exceed ~10 million bytes or you'll get "request to large". cc @Kristian)

Rrichw Hi all. I've got my Worker running AI models, which is great. I can see on https...

K

kian•10/30/24, 4:16 PM

Your error mentions

8b

8b

8b

8b, but there is no

8b

8b

8b

8b Llama 3.2 model

K

kian•10/30/24, 4:17 PM

Sounds like you typo'd the model name, as you mentioned it's

@cf/meta/llama-3.2-3b-instruct

@cf/meta/llama-3.2-3b-instruct

@cf/meta/llama-3.2-3b-instruct

@cf/meta/llama-3.2-3b-instruct

RRaylight The limit is "fuzzy". Around 2.5-3MB depending on the content. Seems to apply to...

M

Martijn•10/30/24, 4:36 PM

The image is around 3,5mb so you recommend compressing them?

MMartijn The image is around 3,5mb so you recommend compressing them?

R

Raylight•10/30/24, 5:11 PM

Yeah, you could try that. The limit for compressed/random data is roughly 2.6MB. In my experience, sending smaller images (file size) has resulted in a significant improvement in latency.

RRaylight Yeah, you could try that. The limit for compressed/random data is roughly 2.6MB....

F

fry69OP•10/30/24, 5:14 PM

These are still gigantic image sizes for LLMs, it should be possible to downscale them to ~100-300kb size and get similar results.

F

fry69OP•10/30/24, 5:15 PM

Most likely the AI API will scale those images down anyway before processing.

M

Martijn•10/30/24, 5:37 PM

What format do you recommend to compress them to jpg? With what setting

MMartijn What format do you recommend to compress them to jpg? With what setting

I

Isaac McFadyen•10/30/24, 5:38 PM

JPEG would work well. Keep in mind that LLMs actually use very small sizes internally anyway - a size of 640 pixels is good enough, no need for 4k images.

M

Martijn•10/30/24, 5:38 PM

With the object classification from fb i feel like if you resize the box dimensions dont work anymore

R

Raylight•10/30/24, 5:39 PM

Either way will work. Compressing or downscaling. The main thing is to get the file size down. When I tested I could shave of half a second of latency simply by sending the (1024x1024) image as a ~200kb jpg instead of a 2MB png.

MMartijn With the object classification from fb i feel like if you resize the box dimensi...

I

Isaac McFadyen•10/30/24, 5:39 PM

The box dimensions should be based on the pixels you give, so it should be consistent?

M

Martijn•10/30/24, 5:39 PM

Yeah but if I want to then visualize the boxes on the original image it wouldnt match anymore right

RRaylight Either way will work. Compressing or downscaling. The main thing is to get the f...

I

Isaac McFadyen•10/30/24, 5:39 PM

I'd recommend resizing over compressing. A compressed 4k image is still going to be resized internally by the model anyway, and by aggressively compressing big images you're going to lose some of the details.

MMartijn Yeah but if I want to then visualize the boxes on the original image it wouldnt ...

I

Isaac McFadyen•10/30/24, 5:40 PM

Just make sure to calculate the resize ratio.

M

Martijn•10/30/24, 5:40 PM

I guess yeah

I

Isaac McFadyen•10/30/24, 5:40 PM

If you have an image that is 3840 and resize to 640 then just multiple the returned coordinates by 6 (3840 divided by 640)

I

Isaac McFadyen•10/30/24, 5:41 PM

Just an FYI that I'm not picking 640 pixels arbitrarily - that's the actual size used by lots of models (ex. YOLO object detection)

M

Martijn•10/30/24, 5:42 PM

Kind of difficult since the images I want to classify are user input and could differ but I’m sure ill figure it out

MMartijn Kind of difficult since the images I want to classify are user input and could d...

I

Isaac McFadyen•10/30/24, 5:43 PM

Hmmm yeah. So depending on what your input pipeline looks like, I'd take the longest dimension (height or width) and then divide that by 640, and then resize to 640, send the image, and multiple all results by that ratio

I

Isaac McFadyen•10/30/24, 5:43 PM

Might be tricky if your pipeline is browser based but there's still options even then

M

Martijn•10/30/24, 5:44 PM

Users upload through an app, it gets stored in R2 and then i’ll likely run some automation on it for it to be classified

I

Isaac McFadyen•10/30/24, 5:44 PM

If you're using a Worker for that automation you can try

sharp

sharp

sharp

sharp, I believe they have a WASM backend now that might work in Workers

M

Martijn•10/30/24, 5:45 PM

Probably be a job on our api thatll send it to the worker and update the db with the response

IIsaac McFadyen I'd recommend resizing over compressing. A compressed 4k image is still going to...

R

Raylight•10/30/24, 6:04 PM

Jpeg blocks are 8x8 pixels, so you're unlikely to lose details if the image is resized by a large factor. I believe resnet expects 224x224 as input, which would amount to ~150kb of uncompressed image data. If you're going to handle the resizing yourself you still have to decide on which kernel to use and in which color space to do the operation - e.g. lancoz, linear gamma. Worst case scenario is if you happen to resize to image dimensions that doesn't quite match the model, since the low res image will be resized once more. So idk, I'd probably go with lossy compression. Although if the image dimensions are huge, it could make sense to bring them down a bit.

H

hNico21•10/30/24, 9:21 PM

Hey, when I send

{
  "messages": [
    {
      "role": "system",
      "content": "You are a bot that parses user input into recipe ingredients..."
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "[\"2 tbsp minced cilantro, leaves and stems\"]"
        }
      ]
    }
  ],
  "return_citations": true,
  "return_images": true
}

{
  "messages": [
    {
      "role": "system",
      "content": "You are a bot that parses user input into recipe ingredients..."
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "[\"2 tbsp minced cilantro, leaves and stems\"]"
        }
      ]
    }
  ],
  "return_citations": true,
  "return_images": true
}

{
  "messages": [
    {
      "role": "system",
      "content": "You are a bot that parses user input into recipe ingredients..."
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "[\"2 tbsp minced cilantro, leaves and stems\"]"
        }
      ]
    }
  ],
  "return_citations": true,
  "return_images": true
}

{
  "messages": [
    {
      "role": "system",
      "content": "You are a bot that parses user input into recipe ingredients..."
    },
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "[\"2 tbsp minced cilantro, leaves and stems\"]"
        }
      ]
    }
  ],
  "return_citations": true,
  "return_images": true
}

I'll get: Answer HTTP 400:

{
  "errors": [
    {
      "message": "AiError: Bad input: must have required property 'prompt', 'messages1content' must be string, must match exactly one schema in oneOf",
      "code": 5006
    }
  ],
  "success": false,
  "result": {},
  "messages": []
}

{
  "errors": [
    {
      "message": "AiError: Bad input: must have required property 'prompt', 'messages1content' must be string, must match exactly one schema in oneOf",
      "code": 5006
    }
  ],
  "success": false,
  "result": {},
  "messages": []
}

{
  "errors": [
    {
      "message": "AiError: Bad input: must have required property 'prompt', 'messages1content' must be string, must match exactly one schema in oneOf",
      "code": 5006
    }
  ],
  "success": false,
  "result": {},
  "messages": []
}

{
  "errors": [
    {
      "message": "AiError: Bad input: must have required property 'prompt', 'messages1content' must be string, must match exactly one schema in oneOf",
      "code": 5006
    }
  ],
  "success": false,
  "result": {},
  "messages": []
}

H

hNico21•10/30/24, 9:24 PM

so it says content must be a string, but Openai-API says string or Array https://platform.openai.com/docs/api-reference/messages/createMessage#messages-createmessage-content

A

ajgeiss0702•10/31/24, 3:31 AM

any idea how I can get more useful error messages? No matter what, I only get

AiError: AiError: Unknown internal error

AiError: AiError: Unknown internal error

AiError: AiError: Unknown internal error

AiError: AiError: Unknown internal error

A

ajgeiss0702•10/31/24, 3:31 AM

im trying to use a lora adapter

A

ajgeiss0702•10/31/24, 3:37 AM

please ping me if you reply

T

Tanis Wolfbane•10/31/24, 10:42 AM

Any one manage to get flux to work? When I run the exactly example in the documentation it errors with NSFW:

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/black-forest-labs/flux-1-schnell    -X POST    -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN"    -d '{ "prompt": "cyberpunk cat" }'
{"errors":[{"message":"AiError: AiError: Input prompt contains NSFW content.","code":3030}],"success":false,"result":{},"messages":[]}

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/black-forest-labs/flux-1-schnell    -X POST    -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN"    -d '{ "prompt": "cyberpunk cat" }'
{"errors":[{"message":"AiError: AiError: Input prompt contains NSFW content.","code":3030}],"success":false,"result":{},"messages":[]}

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/black-forest-labs/flux-1-schnell    -X POST    -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN"    -d '{ "prompt": "cyberpunk cat" }'
{"errors":[{"message":"AiError: AiError: Input prompt contains NSFW content.","code":3030}],"success":false,"result":{},"messages":[]}

curl https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/ai/run/@cf/black-forest-labs/flux-1-schnell    -X POST    -H "Authorization: Bearer $CLOUDFLARE_API_TOKEN"    -d '{ "prompt": "cyberpunk cat" }'
{"errors":[{"message":"AiError: AiError: Input prompt contains NSFW content.","code":3030}],"success":false,"result":{},"messages":[]}

F

fry69OP•10/31/24, 2:32 PM

FYI -> https://discord.com/channels/595317990191398933/895794943182909470/1301548773620715563

F

Fanitullen•10/31/24, 6:59 PM

Am I the only one that can't get

@cf/black-forest-labs/flux-1-schnell

@cf/black-forest-labs/flux-1-schnell

@cf/black-forest-labs/flux-1-schnell

@cf/black-forest-labs/flux-1-schnell to work? Even the short example at https://developers.cloudflare.com/workers-ai/models/flux-1-schnell/ is broken for me.

TTanis Wolfbane Any one manage to get flux to work? When I run the exactly example in the docume...

D

DaniFoldi•10/31/24, 7:02 PM

Hey, is anyone else still seeing this with flux?

I

Isaac McFadyen•10/31/24, 7:08 PM

We're going to try escalating it

D

DaniFoldi•10/31/24, 7:10 PM

Isaac has escalated this, we'll let you know if we have any updates

C

Chaika•10/31/24, 7:13 PM

Looks like the example is categorized as nsfw but most other inputs aren't

FFanitullen Am I the only one that can't get `@cf/black-forest-labs/flux-1-schnell` to work?...

R

Raylight•10/31/24, 7:14 PM

The example code doesn't work. Flux returns a json object with the image encoded as base64. So you need to decode it.

RRaylight The example code doesn't work. Flux returns a json object with the image encoded...

F

Fanitullen•10/31/24, 7:17 PM

Thank you, got it working now.

R

Raylight•10/31/24, 7:31 PM

A lot of prompts triggers the NSFW error. "cyber", "bar", "toe", "arm", "nose", "drill", "boot", "spy", "cow", "chicken" just from trying a few random one word prompts. "a cyberpunk cat" is ok, "cyberpunk cat" isn't.

RRaylight A *lot* of prompts triggers the NSFW error. "cyber", "bar", "toe", "arm", "nose"...

D

DaniFoldi•10/31/24, 9:45 PM

The team is looking into the filter, for the time being they recommend using longer prompts as they are less likely to be caught - they recommend at least 4-5 words

No documentation I can find and only this community entry -> https://community.cloudflare.com/t/infe

Similar Threads