VNopes i have not set that up. When i was setting the worker, CF gave me python + wrangler as a template
HOk, that’s how I set my headers. Not sure exactly how to do it with python functions but read this, it should help https://developers.cloudflare.com/pages/functions/middleware/
Cloudflare Docs
Middleware is reusable logic that can be run before your onRequest function. Middlewares are typically utility functions. Error handling, user …
Vokay so should i replace python with js ?
HI doubt you need to do that, I just haven’t worked with python on CF so I don’t know how to implement middleware. I’m sure the info is in the link I gave you
QHello I use spring AI framework for development and I found a problem
QIn this document I saw that it supports OPENAI’s API format
https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/
https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/
Cloudflare Docs
Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions) and text embedding models (/v1/embeddings). This allows you …
QI noticed that the interface did not return Usage
RToken counts aren't included in the response when running Cloudflare's OpenAI compatible endpoint. Perhaps the docs could use some information about supported parameters and the level of compatibility? Tested it with code similar to the example and couldn't get max_tokens to have any effect.
FUnable to add a fine-tune. Error:
https://developers.cloudflare.com/workers-ai/tutorials/fine-tune-models-with-autotrain
Receiving this when executing the following command:
Folder contains adapter_config.json and adapter_model.safetensors.
I used autotrain on Huggingface to build these files: https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/colabs/AutoTrain_LLM.ipynb
Safetensor file size is 80mb. I have another one 100.1 mb. Both fails
https://developers.cloudflare.com/workers-ai/tutorials/fine-tune-models-with-autotrain
Receiving this when executing the following command:
Folder contains adapter_config.json and adapter_model.safetensors.
I used autotrain on Huggingface to build these files: https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/colabs/AutoTrain_LLM.ipynb
Safetensor file size is 80mb. I have another one 100.1 mb. Both fails
Cloudflare Docs
Fine tuning an AI model gives you the opportunity to add additional training data to the model. Workers AI allows for Low-Rank Adaptation, LoRA, …
FFull error:
Ferror_messages
message.txt3.29KB
Crequest.body.query not working and request.text()??
Cnvm it changed
TI understand that currently, developers can only supply LoRAs, rather than arbitrary full models — presumably because, in a multitenant inference setting, you want a few base models everyone is using that can stay always-loaded-in-VRAM, with just the much-smaller LoRAs being loaded/unloaded between calls
but fine-tunes processed with the BitDelta technique (https://www.together.ai/blog/bitdelta; https://arxiv.org/html/2402.10193v2) seem to offer another alternative; the approach has basically the same benefits as LoRAs for multi-tenant settings — keeping the base model(s) always-loaded, and just loading/unloading fine-tune deltas. (In fact, I believe these models would be loaded as adapters by inference frameworks, exactly as LoRAs are — just with these adapter models' layers having the same rank as the original model layers, rather than being low-rank.)
would the Workers AI team potentially consider allowing the use of arbitrary developer-provided fine-tunes (of the fixed set of base models), by computing and caching each fine-tune's BitDelta against the base-model?
(or by uploading pre-computed BitDeltas — but I don't think there's a standard format/encoding for these yet, or even open-source tooling to generate them, as these are a performance/resource optimization mostly relevant only to multi-tenant inference platforms. so the developer pointing at the fine-tune's URL, and CF fetching it + computing a BitDelta from it, is probably better UX for now... unless your team also wants to be the one to open-source the format and tooling for doing this
)
but fine-tunes processed with the BitDelta technique (https://www.together.ai/blog/bitdelta; https://arxiv.org/html/2402.10193v2) seem to offer another alternative; the approach has basically the same benefits as LoRAs for multi-tenant settings — keeping the base model(s) always-loaded, and just loading/unloading fine-tune deltas. (In fact, I believe these models would be loaded as adapters by inference frameworks, exactly as LoRAs are — just with these adapter models' layers having the same rank as the original model layers, rather than being low-rank.)
would the Workers AI team potentially consider allowing the use of arbitrary developer-provided fine-tunes (of the fixed set of base models), by computing and caching each fine-tune's BitDelta against the base-model?
(or by uploading pre-computed BitDeltas — but I don't think there's a standard format/encoding for these yet, or even open-source tooling to generate them, as these are a performance/resource optimization mostly relevant only to multi-tenant inference platforms. so the developer pointing at the fine-tune's URL, and CF fetching it + computing a BitDelta from it, is probably better UX for now... unless your team also wants to be the one to open-source the format and tooling for doing this
)
TIs it possible to know by when the text-images model will be out of beta? Mainly curious about the pricing per generation will be.
FHello, Stability.ai updated SD3 license. Do you plan to add it to your catalog? https://stability.ai/news/license-update?utm_source=www.therundown.ai&utm_medium=newsletter&utm_campaign=openai-s-secret-hack-exposed
Stability AI
Our new Community License is now free for research, non-commercial, and commercial use. You only need a paid Enterprise license if your yearly revenues exceed USD$1M and you use Stability AI models in commercial products or services.
UIs there a way to set
max_tokens on runWithTools?
NWant to run resnet50 but for image embeddings, not classifications. Is this possible?
PDoes Cloudflare run some kind of moderation check on input to models?
I'm looking into using Workers AI to identify content which may be CSAM related, as unfortunately OpenAI's free moderation API was not sufficient for my use case and mostly gave me false negatives.
Hopefully the user-generated input I'll give to models won't trigger some kind of flag/check, so just want to make sure that my use case is alright.
I'm looking into using Workers AI to identify content which may be CSAM related, as unfortunately OpenAI's free moderation API was not sufficient for my use case and mostly gave me false negatives.
Hopefully the user-generated input I'll give to models won't trigger some kind of flag/check, so just want to make sure that my use case is alright.
AHello, I build a LLMs/AI chat platform build and design for Cloudflare worker AI. with many features like function calling, image to text, text to image, text to speech. all build with Cloudflare worker. https://github.com/akazwz/antonai
GitHub
a LLMs/AI chat platform build and design for Cloudflare worker AI. with many features like function calling, image to text, text to image, text to speech - akazwz/antonai
CIf you are looking for an Image-To-Text app, here is one built with Llava + SvelteKit https://floor-is-llava.pages.dev/ (code in the footer)
CWe do offer the LlamaGuard model: https://ai.meta.com/research/publications/llama-guard-llm-based-input-output-safeguard-for-human-ai-conversations/ Does this look like it might assist with what you are after?
We introduce Llama Guard, an LLM-based input-output safeguard model geared towards Human-AI conversation use cases. Our model incorporates a safety risk...
U?
RNo. You would have to modify runWithTools to be able to do that, e.g. https://github.com/cloudflare/ai-utils/compare/main...Raylight-PWL:ai-utils:Raylight-patch-maxTokens-option.
U:frown:
UI think a good idea would be to have either max_tokens be the max tokens for each call, or have the user able to configure the max tokens for each call
Uthe second option being something like initial_max_tokens, tool_max_tokens, and possibly trim_max_tokens (very scuffed index names, but you get the point)
UIts been really good
UUon normal CF ai i had to guess so much 
Uai-utils is definitely awesome in comparison
UxD
Ui once tried to read the actual AI library on npm to figure out unknowns
UNever trying that again!
Ucompiled code :salute:
Unp!! :D
UI have a quick question
Uconsidering you do ai-utils shenanigans
Uare there any plans on tool usage being supported on the qwen models?
Ufrom what I read, the actual model itself supports it, but I'm obviously not entirely sure on it
