I'm running into an issue where the response is being cut short. I've looked into the discord histor

I'm running into an issue where the response is being cut short. I've looked into the discord history here and can't seem to find an answer. any ideas?

Mmichaelthatsit I'm running into an issue where the response is being cut short. I've looked int...

Isaac McFadyen•4/16/25, 12:56 AM

max_tokens

max_tokens

defaults to 256 tokens, try setting that higher in your request to the model.

michaelthatsitOP•4/16/25, 1:06 AM

it's unclear where to set that.

michaelthatsitOP•4/16/25, 1:15 AM

nvm I'm dumb

IIsaac McFadyen It looks like this is one line in which case you could just remove any lines tha...

cosbgn•4/16/25, 12:11 PM

In this case yes, but files are user submitted, I don't have control on how the metadata line will be. Is it always a single line? not sure

Muhammadreza Haghiri•4/16/25, 9:15 PM

Hello everyone. Is there a chance of having Flux Dev as well?

Brinamite•4/17/25, 8:59 PM

I do believe that the warning is a bug. I am only 93.8 neurons in for today

Mmewfree hey team, seeing this scary banner even though I did not even have any requests ...

Brinamite•4/17/25, 9:00 PM

ah, seeing that even others are posting this

BBrinamite I do believe that the warning is a bug. I am only 93.8 neurons in for today 🧐

Isaac McFadyen•4/17/25, 9:27 PM

Do you know how much you're using? Could that mean 93.8k? Not sure of the unit there.

Rapid•4/18/25, 12:41 PM

Which way should I go if I want to receive text from a user and turn it into a predefined schema? like, parameter extracting of text using LLM or what's the best practice for that? Anyone got a clue

BBrinamite I do believe that the warning is a bug. I am only 93.8 neurons in for today 🧐

ale•4/18/25, 6:29 PM

hi @Brinamite, thanks for reporting this. The team is working on it

Clint•4/18/25, 7:07 PM

why might I see working tool calling locally but not when deployed to workers.dev?

Alexander•4/19/25, 1:20 PM

Hey

Is there any way to increase AutoRAG pipeline limit during beta? I am building a RAG app that requires separation of data between users and since there is no way to add metadata before / during retrieval the only way I can enforce the above is 1 user == 1 rag pipeline

I'd be happy to explore other options - maybe I am missing something?

1) ideally - increase rag pipeline limit
2) or allow query / search endpoint to use metadata filters so that I can exclude data belonging to different users

michaelthatsitOP•4/20/25, 12:59 AM

I'm getting this error when trying to use streamObject. Is there plans to support JSON Schema mode with stream mode?

InferenceUpstreamError [AiError]: 5023: JSON Schema mode is not supported with stream mode.

InferenceUpstreamError [AiError]: 5023: JSON Schema mode is not supported with stream mode.

Alexander•4/20/25, 4:03 AM

Hey, is it possible to trigger AutoRAG to re-index from the API? Couldn't find any way to do this?

AAlexander Hey, is it possible to trigger AutoRAG to re-index from the API? Couldn't find a...

Viktor•4/20/25, 5:22 PM

AutoRAG does indexing when data is uploaded to R2. So if you need to re-index, maybe you could override data in R2

RRapid Which way should I go if I want to receive text from a user and turn it into a p...

Rapid•4/20/25, 10:12 PM

Anyone?

mr.niko.la•4/21/25, 5:23 AM

What is the whsiper turbo file
Size limit ? 100mb

Also requesting the documentation or repo to show sample code to run it on workers ai.

Current documentation is not clear. Thx

mr.niko.la•4/21/25, 8:08 AM

wait this is crazy. my code has failover using whisper turbo. does it show up as regular whisper on this dashboard? im very confused as i never refernce

@cf/openai/whisper

@cf/openai/whisper

is this a bug on the dashboard. tbh i dont even know this was being run anywhere as i never really had the failover ever trigger....

TTait Hey all. If I wanted to extract audio from a video before hitting a workers AI ...

面

面条•4/21/25, 1:15 PM

I just solved this problem https://kai.bi/post/cloudflare-audio-concat

Mmr.niko.la wait this is crazy. my code has failover using whisper turbo. does it show up a...

Isaac McFadyen•4/21/25, 4:13 PM

Whisper and Whisper Turbo are different models, if you're using turbo it should appear as Turbo not base

yinxingmaiming6409•4/22/25, 1:26 AM

Isn't the Rag size limit 4M? Why does my xlsx file 2.1M still prompt that it exceeds the size limit?

Kasjmier Leela•4/22/25, 4:18 AM

i have a possibly silly question. forgive me if it is.

i want to use an email worker to intercept incoming emails, extract email content, run some workers-ai logic on the content to gather context and send the context out as a json response.

is this possible with an email worker?

IIsaac McFadyen Whisper and Whisper Turbo are different models, if you're using turbo it should ...

mr.niko.la•4/22/25, 5:49 AM

this is strange i looked at my code and only

this.env.AI.run('@cf/openai/whisper-large-v3-turbo

this.env.AI.run('@cf/openai/whisper-large-v3-turbo

. i cant find any reference to the othe rone yet i see it in my dashboard. ...

is there any other way to debug it further?

Puru Dahal•4/22/25, 6:38 AM

@Cloudflare Not sure which one to tag specifically, but when can we expect Deepseek - V3?

SuperHelpflare•4/22/25, 7:06 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

mr.niko.la•4/22/25, 7:36 AM

where is this playground the one i got doesnt have grok in it....

mr.niko.la•4/22/25, 7:40 AM

https://playground.ai.cloudflare.com/ this doesnt have grok...

RRapid Anyone?

Viktor•4/22/25, 11:03 AM

What exactly are you trying to reach? Do you want something you can parse afterwards? You can always ask the llm to output data in different formats, I noticed json works well with llama

Viktor•4/22/25, 11:06 AM

like this, "Output as valid json, dont write additional text or markdown"

Rapid•4/22/25, 11:48 AM

I have a function that I wanna call using LLM. So basically imagine a user writes I want to buy 5 NVDA stock.

So I want it to interpt to a JSON and then send the user a message with the parameters so he clicks "confirm" and then execute it. So basically just parsing text into predefined schema.

VViktor like this, "Output as valid json, dont write additional text or markdown"

Rapid•4/22/25, 12:03 PM

I will try it, was wondering if there is something that is meant for that though

RRapid I have a function that I wanna call using LLM. So basically imagine a user write...

Viktor•4/22/25, 12:37 PM

You can use functions / "tools" for that

Viktor•4/22/25, 12:39 PM

See this: https://developers.cloudflare.com/workers-ai/features/function-calling/
Basicly you enter your prompt, and pass additional functions that the LLM can call. If the LLM wants to call it, it'll pass information about the function, you execute the function, and add the result as

assistant

assistant

message.
There is a

cloudflare/ai-utils

cloudflare/ai-utils

making this simpler, but I'm struggling with that one right now (see my thread from sunday)

Cloudflare Docs

Function calling · Cloudflare Workers AI docs

Function calling enables people to take Large Language Models (LLMs) and use the model response to execute functions or interact with external APIs. The developer usually defines a set of functions and the required input schema for each function, which we call tools. The model then intelligently understands when it needs to do a tool call, and i...

Viktor•4/22/25, 12:42 PM

Also there is the "human in the loop", which can be used to actually intercept an ai workflow. But I didn't work with those ones yet, so I can't give any futher information if that is just a function waiting for confirmation, or has some other additional things
https://developers.cloudflare.com/agents/concepts/human-in-the-loop/

Cloudflare Docs

Human in the Loop · Cloudflare Agents docs

Human-in-the-Loop (HITL) workflows integrate human judgment and oversight into automated processes. These workflows pause at critical points for human review, validation, or decision-making before proceeding. This approach combines the efficiency of automation with human expertise and oversight where it matters most.

Viktor•4/22/25, 12:46 PM

Those are the issues I am having with ai-utils right now: #Is module @cloudflare/ai-utils stable?. If I don't get an answer here in the near future, I'll look for my access data for Github again and push it in as an issue, but so far it's not in a hurry and not a priority, so I'll wait here relaxed for an answer

Rapid•4/22/25, 1:15 PM

Thanks Viktor, I will check function-calling, as I said, I need to see the response before passing it to the function, as I want the user confirmation for the requested action. It just a faster possible way to interact with a bot, instead of clicking buttons and configuring settings

VViktor Also there is the "human in the loop", which can be used to actually intercept a...

Rapid•4/22/25, 1:16 PM

Thanks dude!

zido•4/22/25, 6:02 PM

Can we add Molmo by Allenai please

Mmr.niko.la where is this playground the one i got doesnt have grok in it....

mr.niko.la•4/22/25, 7:09 PM

Ok I searched for this playground can’t fit in. I guess cf devs don’t follow this discord ?

Chris•4/22/25, 9:24 PM

Hi folks! What's the best way to do embedded function calling through an AI Gateway (e.g. to enable caching)?
https://developers.cloudflare.com/workers-ai/features/function-calling/embedded/api-reference/

It looks like cloudflare/ai-utils doesn't let us pass in a gatewaygateway (as shown in the AI Gateway docs: https://developers.cloudflare.com/ai-gateway/providers/workersai/#workers-binding)

Though I'm not sure where to find detailed documentation on the AI binding — the docs don't say anything useful about the second parameter object: https://developers.cloudflare.com/workers-ai/configuration/bindings/#methods

The TypeScript types seem to have the best information. But still, it doesn't seem like AI binding / gateway / runWithToolsrunWithTools are very composable.. or am I missing something?

jackoyam•4/22/25, 9:33 PM

Hi all, I keep getting AiError:5024 with @cf/meta/llama-3.1-8b-instruct-fast trying to use JSON Mode. Switching to @cf/meta/llama-4-scout-17b-16e-instruct works fine, but I'd like to use the cheaper model for certain tasks. I include this is the ai.run: response_format: { "type": "json_schema", "json_schema": schema },

Not sure why 3.1 keeps failing and whether there's an alternative?

Also: Docs are inconsistent with API Schema on each model's page, and the available models on the 'JSON Mode' page

JoaoCupido•4/23/25, 1:53 PM

Apparently, dreamshaper-8-lcm is giving an internal server error on my side. I don't know if it's only me or someone else has the same issue

VViktor AutoRAG does indexing when data is uploaded to R2. So if you need to re-index, m...

Alexander•4/24/25, 9:22 AM

Hey, thx for the tip - hmmm...... for some reason I may have thought otherwises. Isn't there a 4 hour window instead?

https://developers.cloudflare.com/autorag/configuration/indexing/

I will test it today to check but I was left with the impression that indexing is not on demand / on upload but on regular interval?

Cloudflare Docs

Indexing · AutoRAG

AutoRAG automatically indexes your data into vector embeddings optimized for semantic search. Once a data source is connected, indexing runs continuously in the background to keep your knowledge base fresh and queryable.

云

云•4/27/25, 1:26 AM

@cf/myshell-ai/melotts@cf/myshell-ai/melotts does not support chinese which the original model does support

akazwz•4/27/25, 7:55 AM

I find this problem too, will u support Chinese language ?

mounty•4/28/25, 10:25 AM

Is it possible to run HuggingFaceTB/SmolLM2-135M-Instruct on workersai?

Mmounty Is it possible to run HuggingFaceTB/SmolLM2-135M-Instruct on workersai?

Isaac McFadyen•4/28/25, 4:10 PM

?workers-ai-models TLDR: no.

SuperHelpflare•4/28/25, 4:10 PM

Workers AI currently only supports popular open-source models provided by the Cloudflare team, as well as your own LoRAs that can be applied on top of the Cloudflare-provided models. You cannot currently upload your own models or use a model from HuggingFace. See the documentation for the list of Cloudflare-provided models: https://developers.cloudflare.com/workers-ai/models/

ByteBeast•4/29/25, 5:14 AM

I have build an agent using agent package but I am stuck at one thing, I want to get URL params or query params in tools of MCP server like agentId so I dont have to pass agentId in tools props (paramsSchema).

I'm running into an issue where the response is being cut short. I've looked into the discord histor

Similar Threads