Which way should I go if I want to receive text from a user and turn it into a predefined schema? li

Which way should I go if I want to receive text from a user and turn it into a predefined schema? like, parameter extracting of text using LLM or what's the best practice for that? Anyone got a clue

BBrinamite I do believe that the warning is a bug. I am only 93.8 neurons in for today 🧐

ale•4/18/25, 6:29 PM

hi @Brinamite, thanks for reporting this. The team is working on it

Clint•4/18/25, 7:07 PM

why might I see working tool calling locally but not when deployed to workers.dev?

Alexander•4/19/25, 1:20 PM

Hey

Is there any way to increase AutoRAG pipeline limit during beta? I am building a RAG app that requires separation of data between users and since there is no way to add metadata before / during retrieval the only way I can enforce the above is 1 user == 1 rag pipeline

I'd be happy to explore other options - maybe I am missing something?

1) ideally - increase rag pipeline limit
2) or allow query / search endpoint to use metadata filters so that I can exclude data belonging to different users

michaelthatsit•4/20/25, 12:59 AM

I'm getting this error when trying to use streamObject. Is there plans to support JSON Schema mode with stream mode?

InferenceUpstreamError [AiError]: 5023: JSON Schema mode is not supported with stream mode.

InferenceUpstreamError [AiError]: 5023: JSON Schema mode is not supported with stream mode.

Alexander•4/20/25, 4:03 AM

Hey, is it possible to trigger AutoRAG to re-index from the API? Couldn't find any way to do this?

AAlexander Hey, is it possible to trigger AutoRAG to re-index from the API? Couldn't find a...

Viktor•4/20/25, 5:22 PM

AutoRAG does indexing when data is uploaded to R2. So if you need to re-index, maybe you could override data in R2

RRapid Which way should I go if I want to receive text from a user and turn it into a p...

RapidOP•4/20/25, 10:12 PM

Anyone?

mr.niko.la•4/21/25, 5:23 AM

What is the whsiper turbo file
Size limit ? 100mb

Also requesting the documentation or repo to show sample code to run it on workers ai.

Current documentation is not clear. Thx

mr.niko.la•4/21/25, 8:08 AM

wait this is crazy. my code has failover using whisper turbo. does it show up as regular whisper on this dashboard? im very confused as i never refernce @cf/openai/whisper@cf/openai/whisper is this a bug on the dashboard. tbh i dont even know this was being run anywhere as i never really had the failover ever trigger....

TTait Hey all. If I wanted to extract audio from a video before hitting a workers AI ...

面

面条•4/21/25, 1:15 PM

I just solved this problem https://kai.bi/post/cloudflare-audio-concat

Mmr.niko.la wait this is crazy. my code has failover using whisper turbo. does it show up a...

Isaac McFadyen•4/21/25, 4:13 PM

Whisper and Whisper Turbo are different models, if you're using turbo it should appear as Turbo not base

yinxingmaiming6409•4/22/25, 1:26 AM

Isn't the Rag size limit 4M? Why does my xlsx file 2.1M still prompt that it exceeds the size limit?

Kasjmier Leela•4/22/25, 4:18 AM

i have a possibly silly question. forgive me if it is.

i want to use an email worker to intercept incoming emails, extract email content, run some workers-ai logic on the content to gather context and send the context out as a json response.

is this possible with an email worker?

IIsaac McFadyen Whisper and Whisper Turbo are different models, if you're using turbo it should ...

mr.niko.la•4/22/25, 5:49 AM

this is strange i looked at my code and only

this.env.AI.run('@cf/openai/whisper-large-v3-turbo

this.env.AI.run('@cf/openai/whisper-large-v3-turbo

. i cant find any reference to the othe rone yet i see it in my dashboard. ...

is there any other way to debug it further?

Puru Dahal•4/22/25, 6:38 AM

@Cloudflare Not sure which one to tag specifically, but when can we expect Deepseek - V3?

SuperHelpflare•4/22/25, 7:06 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

mr.niko.la•4/22/25, 7:36 AM

where is this playground the one i got doesnt have grok in it....

mr.niko.la•4/22/25, 7:40 AM

https://playground.ai.cloudflare.com/ this doesnt have grok...

RRapid Anyone?

Viktor•4/22/25, 11:03 AM

What exactly are you trying to reach? Do you want something you can parse afterwards? You can always ask the llm to output data in different formats, I noticed json works well with llama

Viktor•4/22/25, 11:06 AM

like this, "Output as valid json, dont write additional text or markdown"

RapidOP•4/22/25, 11:48 AM

I have a function that I wanna call using LLM. So basically imagine a user writes I want to buy 5 NVDA stock.

So I want it to interpt to a JSON and then send the user a message with the parameters so he clicks "confirm" and then execute it. So basically just parsing text into predefined schema.

VViktor like this, "Output as valid json, dont write additional text or markdown"

RapidOP•4/22/25, 12:03 PM

I will try it, was wondering if there is something that is meant for that though

RRapid I have a function that I wanna call using LLM. So basically imagine a user write...

Viktor•4/22/25, 12:37 PM

You can use functions / "tools" for that

Viktor•4/22/25, 12:39 PM

See this: https://developers.cloudflare.com/workers-ai/features/function-calling/
Basicly you enter your prompt, and pass additional functions that the LLM can call. If the LLM wants to call it, it'll pass information about the function, you execute the function, and add the result as

assistant

assistant

message.
There is a

cloudflare/ai-utils

cloudflare/ai-utils

making this simpler, but I'm struggling with that one right now (see my thread from sunday)

Cloudflare Docs

Function calling · Cloudflare Workers AI docs

Function calling enables people to take Large Language Models (LLMs) and use the model response to execute functions or interact with external APIs. The developer usually defines a set of functions and the required input schema for each function, which we call tools. The model then intelligently understands when it needs to do a tool call, and i...

Viktor•4/22/25, 12:42 PM

Also there is the "human in the loop", which can be used to actually intercept an ai workflow. But I didn't work with those ones yet, so I can't give any futher information if that is just a function waiting for confirmation, or has some other additional things
https://developers.cloudflare.com/agents/concepts/human-in-the-loop/

Cloudflare Docs

Human in the Loop · Cloudflare Agents docs

Human-in-the-Loop (HITL) workflows integrate human judgment and oversight into automated processes. These workflows pause at critical points for human review, validation, or decision-making before proceeding. This approach combines the efficiency of automation with human expertise and oversight where it matters most.

Viktor•4/22/25, 12:46 PM

Those are the issues I am having with ai-utils right now: #Is module @cloudflare/ai-utils stable?. If I don't get an answer here in the near future, I'll look for my access data for Github again and push it in as an issue, but so far it's not in a hurry and not a priority, so I'll wait here relaxed for an answer

RapidOP•4/22/25, 1:15 PM

Thanks Viktor, I will check function-calling, as I said, I need to see the response before passing it to the function, as I want the user confirmation for the requested action. It just a faster possible way to interact with a bot, instead of clicking buttons and configuring settings

VViktor Also there is the "human in the loop", which can be used to actually intercept a...

RapidOP•4/22/25, 1:16 PM

Thanks dude!

zido•4/22/25, 6:02 PM

Can we add Molmo by Allenai please

Mmr.niko.la where is this playground the one i got doesnt have grok in it....

mr.niko.la•4/22/25, 7:09 PM

Ok I searched for this playground can’t fit in. I guess cf devs don’t follow this discord ?

Chris•4/22/25, 9:24 PM

Hi folks! What's the best way to do embedded function calling through an AI Gateway (e.g. to enable caching)?
https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/

It looks like cloudflare/ai-utils doesn't let us pass in a

gateway

gateway

(as shown in the AI Gateway docs: https://developers.cloudflare.com/ai-gateway/providers/workersai/#workers-binding)

Though I'm not sure where to find detailed documentation on the AI binding — the docs don't say anything useful about the second parameter object: https://developers.cloudflare.com/workers-ai/configuration/bindings/#methods

The TypeScript types seem to have the best information. But still, it doesn't seem like AI binding / gateway /

runWithTools

runWithTools

are very composable.. or am I missing something?

jackoyam•4/22/25, 9:33 PM

Hi all, I keep getting AiError:5024 with @cf/meta/llama-3.1-8b-instruct-fast trying to use JSON Mode. Switching to @cf/meta/llama-4-scout-17b-16e-instruct works fine, but I'd like to use the cheaper model for certain tasks. I include this is the ai.run: response_format: { "type": "json_schema", "json_schema": schema },

Not sure why 3.1 keeps failing and whether there's an alternative?

Also: Docs are inconsistent with API Schema on each model's page, and the available models on the 'JSON Mode' page

JoaoCupido•4/23/25, 1:53 PM

Apparently, dreamshaper-8-lcm is giving an internal server error on my side. I don't know if it's only me or someone else has the same issue

VViktor AutoRAG does indexing when data is uploaded to R2. So if you need to re-index, m...

Alexander•4/24/25, 9:22 AM

Hey, thx for the tip - hmmm...... for some reason I may have thought otherwises. Isn't there a 4 hour window instead?

https://developers.cloudflare.com/autorag/configuration/indexing/

I will test it today to check but I was left with the impression that indexing is not on demand / on upload but on regular interval?

Cloudflare Docs

Indexing · AutoRAG

AutoRAG automatically indexes your data into vector embeddings optimized for semantic search. Once a data source is connected, indexing runs continuously in the background to keep your knowledge base fresh and queryable.

云

云•4/27/25, 1:26 AM

@cf/myshell-ai/melotts

@cf/myshell-ai/melotts

does not support chinese which the original model does support

akazwz•4/27/25, 7:55 AM

I find this problem too, will u support Chinese language ?

mounty•4/28/25, 10:25 AM

Is it possible to run HuggingFaceTB/SmolLM2-135M-Instruct on workersai?

Mmounty Is it possible to run HuggingFaceTB/SmolLM2-135M-Instruct on workersai?

Isaac McFadyen•4/28/25, 4:10 PM

?workers-ai-models TLDR: no.

SuperHelpflare•4/28/25, 4:10 PM

Workers AI currently only supports popular open-source models provided by the Cloudflare team, as well as your own LoRAs that can be applied on top of the Cloudflare-provided models. You cannot currently upload your own models or use a model from HuggingFace. See the documentation for the list of Cloudflare-provided models: https://developers.cloudflare.com/workers-ai/models/

ByteBeast•4/29/25, 5:14 AM

I have build an agent using agent package but I am stuck at one thing, I want to get URL params or query params in tools of MCP server like agentId so I dont have to pass agentId in tools props (paramsSchema).

nick•4/30/25, 4:20 AM

For the text-to-image models such as flux-1-schnell the current output is simply:

{ image: string }

{ image: string }

is there a way to have it output the usage data, similar to the

{ usage: { prompt_tokens: number, completion_tokens: number, total_tokens: number }}

{ usage: { prompt_tokens: number, completion_tokens: number, total_tokens: number }}

output for text generation models? Based on the pricing, I'd assume it would be something like

{ usage: { tiles: number, steps: number }}

{ usage: { tiles: number, steps: number }}

? Or does this just need to be calculated on each usage on our end using the default step of 4 if none is provided as input, and measuring the output image size?

Regarding output image size. Is there or will there be a way to specify the desired image output resolution? The image size i'm getting back is 1024x1024, so four 512x512 tiles. Just want to confirm this will always be the case?

Cloudflare Docs

flux-1-schnell

FLUX.1 [schnell] is a 12 billion parameter rectified flow transformer capable of generating images from text descriptions.

antim8•4/30/25, 12:54 PM

It seems like some models are not available in the worker types. For example https://developers.cloudflare.com/workers-ai/models/llama-3.1-8b-instruct-fast/

or did i miss something?

Cloudflare Docs

llama-3.1-8b-instruct-fast

[Fast version] The Meta Llama 3.1 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction tuned generative models. The Llama 3.1 instruction tuned text only models are optimized for multilingual dialogue use cases and outperform many of the available open source and closed chat models on common indus...

antim8•4/30/25, 12:58 PM

those are available in the code. Seems like there are a bunch of model missing

Rabid•4/30/25, 7:12 PM

Anyone have a simple example for scoring NSFW images? I tried resnet-50 with a very NWFW image, and it just adds labels like "bath towel"

mr.niko.la•4/30/25, 8:35 PM

[ai]
binding = "AI"

[ai]
binding = "AI"

However, the

401 Unauthorized

401 Unauthorized

errors in my logs indicate that the authentication for Workers AI is failing, despite the correct binding. This suggests issues with account permissions, missing API tokens, or a misconfiguration in the Workers AI setup.

Do i need to add any token for

whisper turbo large v3

whisper turbo large v3

in the

vars

vars

section of my

wrangler.toml

wrangler.toml

file?

mr.niko.la•4/30/25, 8:50 PM

WORKERS_AI_TOKEN = "gdhd"

WORKERS_AI_TOKEN = "gdhd"

. i suppose i need to add the

workers ai

workers ai

api token too?

Mmr.niko.la `WORKERS_AI_TOKEN = "gdhd"`. i suppose i need to add the `workers ai` api token ...

Isaac McFadyen•4/30/25, 9:11 PM

You shouldn't need a token when you're using the binding, but how are you calling the model?

Isaac McFadyen•4/30/25, 9:11 PM

The binding is separate from the API, and if you're using the binding you shouldn't be calling the API at all

IIsaac McFadyen You shouldn't need a token when you're using the binding, but how are you callin...

mr.niko.la•4/30/25, 11:25 PM

Hi Isaac, thanks for the clarification! I'm using the AI binding to call the @cf/openai/whisper-large-v3-turbo model via env.AI.run in my Worker. Here’s a sample:

          try {
            console.log(`[BYPASS] Attempt 3: Using AI binding with raw bytes`);
            const aiTranscription = await this.env.AI.run('@cf/openai/whisper-large-v3-turbo', {
              audio: [...audioBytes], // Convert Uint8Array to regular array
              language: 'en',
            });

          try {
            console.log(`[BYPASS] Attempt 3: Using AI binding with raw bytes`);
            const aiTranscription = await this.env.AI.run('@cf/openai/whisper-large-v3-turbo', {
              audio: [...audioBytes], // Convert Uint8Array to regular array
              language: 'en',
            });

          try {
            console.log(`[BYPASS] Attempt 2: Using AI binding with base64`);
            const aiTranscription = await this.env.AI.run('@cf/openai/whisper-large-v3-turbo', {
              audio: base64Audio,
              language: 'en',
            });

          try {
            console.log(`[BYPASS] Attempt 2: Using AI binding with base64`);
            const aiTranscription = await this.env.AI.run('@cf/openai/whisper-large-v3-turbo', {
              audio: base64Audio,
              language: 'en',
            });

Despite the binding in wrangler.toml ([ai] binding = "AI"), I’m getting 401 Unauthorized errors. Could this be an account permission issue or a binding misconfiguration? Any suggestions to debug?

Which way should I go if I want to receive text from a user and turn it into a predefined schema? li

Similar Threads

Similar Threads

Similar Threads