Is there any place to watch out for when beta models go out of beta
Is there any place to watch out for when beta models go out of beta
OHi everyone,
I'm currently working with the Cloudflare-hosted BAAI model (@cf/baai/bge-base-en-v1.5) for generating vector embeddings in a Hono application. However, I'm encountering an issue where the AI service throws an 3001: Unknown internal error.
Here's a brief overview of what I'm doing:
I send a request to the AI model with a text array containing a single string.
const { data } = await c.env.AI.run('@cf/baai/bge-base-en-v1.5', { text: [text] });
The error occurs at this point, and the response from the AI service contains the 3001 with the message "Unknown internal error."
I've double-checked the input data, and everything seems to be in order. However, I'm still unsure what's causing this issue.
Has anyone else encountered this error, or does anyone have insights into what might be causing it? Any guidance on how to resolve this would be greatly appreciated.
PS: This error seems to occur infrequently and intermittently. Most of requests to the embedding model successfully complete.
I'm currently working with the Cloudflare-hosted BAAI model (@cf/baai/bge-base-en-v1.5) for generating vector embeddings in a Hono application. However, I'm encountering an issue where the AI service throws an 3001: Unknown internal error.
Here's a brief overview of what I'm doing:
I send a request to the AI model with a text array containing a single string.
const { data } = await c.env.AI.run('@cf/baai/bge-base-en-v1.5', { text: [text] });
The error occurs at this point, and the response from the AI service contains the 3001 with the message "Unknown internal error."
I've double-checked the input data, and everything seems to be in order. However, I'm still unsure what's causing this issue.
Has anyone else encountered this error, or does anyone have insights into what might be causing it? Any guidance on how to resolve this would be greatly appreciated.
PS: This error seems to occur infrequently and intermittently. Most of requests to the embedding model successfully complete.
SI believe pricing shoud be reviewed as it's order of magnitude more expensive than any compettipl
OIs there a way to measure how many input and output tokens(or neurons) are being used when using workers AI? For context, other models return token usage details along with response via API but workers AI just seems to return just the response.
VIs there any plan for supporting FLUX.1 text-to-image on Workers AI?
XIs there any plan to launch the QWen2 model
A^^
BI believe the text-generation models in Cloudflare AI should at least return the number of tokens counted in the prompt (input) and also generated (output), so we can track the end user's usage and properly rate-limit for future requests like other AI providers.
Cwill flux.1 be added to cloudflare ai libraries...?
RI don't think so , I thought flux only
Partnered with few providers , possibly only certain companies can monetize it ? . maybe the "open" flux dev model but even that the license says no commercialize so idk
Partnered with few providers , possibly only certain companies can monetize it ? . maybe the "open" flux dev model but even that the license says no commercialize so idk
Twhere can i find information about how much does it cost to generate one image with stable diffusion?
TIt only says about neuron which seems to only related to LLM space.
IAll models on Workers AI use neurons, it's not an LLM thing it's just a standardized unit to measure the computational cost of the model.
IBut all the image models are currently free, any models marked
Beta on this page are free until they come out of beta. https://developers.cloudflare.com/workers-ai/models/#text-to-imageTHi, Issac. Is there a way to get rough estimation of neuron usage for a single stable diffusion inference?
TFor a100, I think it takes 1.x second for sd v1.5 512 x 512 and for sdxl lightening 1024 x 1024 around 3 to 4s.
TPlease leave comments if you have any information about the pricing model for stable diffusion later.
II don't believe there is currently, no. There is a neuron pricing calculator but I'm not sure whether the text-to-image models have been added yet.
I
Rhttps://huggingface.co/black-forest-labs/FLUX.1-schnell can be used for commercial purposes. flux.1 dev would require some kind of arrangement, though.
VMaybe you can refer to other online text-to-image generation provider price, for commercial concerns, I think cloudflare won't make it much more expensive or cheaper than other providers. As far as i know, sdxl 1024*1024 20 steps cost about 0.2~0.3 cents per image.
Rhey folks, where can I request the addition of new models? The 
cloudflare is perfect for running those guardrails models which should run as fast as possible
@hf/thebloke/llamaguard-7b-awq there is a bit outdated, still uses llama 2 and it's not very smart, what about upgrading it to https://huggingface.co/meta-llama/Llama-Guard-3-8B which uses llama 3.1? And while at it perhaps also add https://huggingface.co/meta-llama/Prompt-Guard-86M cloudflare is perfect for running those guardrails models which should run as fast as possible
P
KI created custom LoRA models, but they all stopped working. I'm wondering if anyone else might have had such an issue. I'm not sure if there's a problem with my account or with Cloudflare's LoRA in general.
CHello everyone! 
I'm currently working on a personal project and have hit a bit of a roadblock. I'm using text generation models to create a chat interface, but I've noticed that I need to specify the system role and user prompts with each query. Unfortunately, there's no option to store a large dataset in the model's memory that it can reference each time it generates an output.
What I'd like to do is store a large set of data in one place, which the model can then access whenever I ask it a question. However, I'm relatively new to the Cloudflare ecosystem and could really use some guidance. Is there a way to achieve this using Cloudflare's functionalities or products?
Thanks in advance for your help!
I'm currently working on a personal project and have hit a bit of a roadblock. I'm using text generation models to create a chat interface, but I've noticed that I need to specify the system role and user prompts with each query. Unfortunately, there's no option to store a large dataset in the model's memory that it can reference each time it generates an output.
What I'd like to do is store a large set of data in one place, which the model can then access whenever I ask it a question. However, I'm relatively new to the Cloudflare ecosystem and could really use some guidance. Is there a way to achieve this using Cloudflare's functionalities or products?
Thanks in advance for your help!
CTo elaborate, I'm working with the GitHub API and fetching large amount of json data, which I'd then like my model to reference everytime it generates an output.
VI'm not sure i understand you correctly and I'm not very familiar with LLM models, it sounds like you are trying to build a RAG (before LLM gives responding, it refers to some data set you have prepared)? If that's what you are try to do, you can refer to https://developers.cloudflare.com/workers-ai/tutorials/build-a-retrieval-augmented-generation-ai .
In short, RAG is roughly like you need do a simple search by yourself, distill the reference data and make the reference data become much smaller before calling LLM models
As my understanding of the article, I think you can split the json data into many text segments based one some methods(maybe split by git project), using embedding models on workers ai to convert each text to vector, store the origin text to cloudflare D1 and get an ID, store (ID,vector) pair into cloudflare Vectorize. When you run LLM model, you can retrive relevant data through embedding the input and searching the input vector in Vectorize and get some IDs of relevant vectors, retrieve relevant text from D1 by these IDs, and use the relevant data as part of the prompt.
In short, RAG is roughly like you need do a simple search by yourself, distill the reference data and make the reference data become much smaller before calling LLM models
As my understanding of the article, I think you can split the json data into many text segments based one some methods(maybe split by git project), using embedding models on workers ai to convert each text to vector, store the origin text to cloudflare D1 and get an ID, store (ID,vector) pair into cloudflare Vectorize. When you run LLM model, you can retrive relevant data through embedding the input and searching the input vector in Vectorize and get some IDs of relevant vectors, retrieve relevant text from D1 by these IDs, and use the relevant data as part of the prompt.
Cloudflare Docs
This guide will instruct you through setting up and deploying your first application with Cloudflare AI. You will build a fully-featured AI-powered application, using tools like Workers AI, Vectorize, D1, and Cloudflare Workers.
SHello, where i can find more templates that are working with workers-ai, im looking something for text-to-image
DFor folks in London, UK - we’re hosting the first AI Engineer Meetup at Cloudflare offices on 12th Sep https://lu.ma/ynbdcv1d
The Age of the AI Engineer has begun
We're excited to launch the London chapter of AI Engineer meet-ups series. We're bringing a slice of the AI Engineer…
We're excited to launch the London chapter of AI Engineer meet-ups series. We're bringing a slice of the AI Engineer…
CThank your for the response! Yes, I think you're understanding it correctly. I reached a similar conclusion after tinkering around for a bit that I need to set up an external store, break down data in chunks and store them separately and then contextually retrieve them based on the prompt.
I took a look at Cloudflare KV store (https://developers.cloudflare.com/kv/), which is a key-value data storage. I was thinking I could extract relevant data from the github api and store each of them in their particular keys and retrieve them based on some condition. But I think building a RAG like you mentioned here might be more relevant and built for this specific use-case, although it does sound complex though. I'll take a look, thanks! :)
I took a look at Cloudflare KV store (https://developers.cloudflare.com/kv/), which is a key-value data storage. I was thinking I could extract relevant data from the github api and store each of them in their particular keys and retrieve them based on some condition. But I think building a RAG like you mentioned here might be more relevant and built for this specific use-case, although it does sound complex though. I'll take a look, thanks! :)
Bwhats the cost to run whisper-tiny-en?
I only see pricing for normal whisper
I only see pricing for normal whisper
Boh nice ty!
VHow to get the token consumed by a
For example, the below code. I want to know the total token consumed by this request so that we charge to customer accordingly.
ai.run() request?For example, the below code. I want to know the total token consumed by this request so that we charge to customer accordingly.
Awill the translation model eventually be updated? m2m100 by meta's own standard is outdated and should be replaced with something like https://ai.meta.com/blog/nllb-200-high-quality-machine-translation/
Meta AI has built a single AI model, NLLB-200, that is the first to translate across 200 different languages with state-of-the-art quality that has been validated through extensive evaluations for each of them.
ONone of the questions regarding the token usage gets answered I see. Still awaiting any update on this 
VI see CF use another metric called Neurons, but can’t find how it’s calculated on docs or any way to see that in ai.run() response on runtime to calculate for each user using the app.
Ohttps://ai.cloudflare.com/#pricing-calculator You can check that here but there's no definite way to measure the usage of neurons or tokens based on individual queries. All we have is an estimate but no clear usage based metrics. Maybe in the birthday week next month they might change that or it'll just get ignored like most of the messages about token usage.
Beta models are free for now so they wont show up on pricing calculator
Beta models are free for now so they wont show up on pricing calculator
�response should be Unit8Array not Response class, but problem it's read once (I guess), which causing
TypeError: The ReadableStream has been locked to a reader. prevent updateRecentImages worksdo you have any idea?
1Im checking back soon! Any news on TTS models?
Vcan anyone please tell me what is the limit of worker ai for free tier it says 10000 neurons , what does it mean and i have no idea i have made few request to try the llama model and the request were successful but on the dashboard i cant see my request
help plz
help plz
Vhere is a calculator https://ai.cloudflare.com/#pricing-calculator , you can consider it as amount of computation, but frankly it's still a pretty confused concept...
AAnyone know where to lookup error codes?
Ahas anything changed?
Acan we deploy our own pretrained models
RHi,
Not sure if this is the right place for this, but the docs directed me to this server
I was playing around with Workers AI text generation models and have found that setting
If we are using streaming then we directly get the
if non-streaming then it is a
As per the docs the minimum allowed values for
Not sure if this is the right place for this, but the docs directed me to this server
I was playing around with Workers AI text generation models and have found that setting
top_p or repetition_penalty value to 0Cloudflare API ErrorIf we are using streaming then we directly get the
data: [DONE] event, if non-streaming then it is a
500 "Cloudflare API error" As per the docs the minimum allowed values for
top_p & repetition_penalty is 0, so ideally we shouldn't get any error..R
OAre there some hidden output token limits? My response seems to be cut short after 2000 words regardless of whether I'm streaming or not on beta models
XHey, I'm using llama-3.1-8b-instruct to classify and reject spam/inappropriate messages, it works mostly fine, but I'm struggling to reliably return a structured output (eg a json {"appropriate": boolean}).
Xis there a trick beside finetuning the prompt and hope for the best? so far, asking to classify messages in other languages than english seems to trip the model (eg. it sometimes returns "approprié" as the key if the message is in french)
XI'm aware there are more specific models, eg. https://huggingface.co/predibase/jigsaw my question is how to tell the workers AI to answer following a specific structure
