Unable to add a fine-tune. Error: https://developers.cloudflare.com/workers-ai/tutorials/fine-tune-m

Unable to add a fine-tune. Error:
https://developers.cloudflare.com/workers-ai/tutorials/fine-tune-models-with-autotrain

FILE_PARSE_ERROR: 'file' should be of valid safetensors type [code: 1000], quiting...

FILE_PARSE_ERROR: 'file' should be of valid safetensors type [code: 1000], quiting...

Receiving this when executing the following command:

npx wrangler ai finetune create @cf/mistral/mistral-7b-instruct-v0.2-lora warp-finetune-mistral folder_location

npx wrangler ai finetune create @cf/mistral/mistral-7b-instruct-v0.2-lora warp-finetune-mistral folder_location

Folder contains adapter_config.json and adapter_model.safetensors.
I used autotrain on Huggingface to build these files: https://colab.research.google.com/github/huggingface/autotrain-advanced/blob/main/colabs/AutoTrain_LLM.ipynb

Safetensor file size is 80mb. I have another one 100.1 mb. Both fails

Cloudflare Docs

Fine Tune Models With AutoTrain from HuggingFace · Cloudflare Worke...

Fine tuning an AI model gives you the opportunity to add additional training data to the model. Workers AI allows for Low-Rank Adaptation, LoRA, …

Google Colab

Ffifaaaron88 Unable to add a fine-tune. Error: https://developers.cloudflare.com/workers-ai/t...

fifaaaron88OP•7/3/24, 6:23 PM

Full error:

--- 2024-07-03T18:20:15.757Z debug
🪵  Writing logs to "/Users/agdfdf/.wrangler/logs/wrangler-2024-07-03_18-20-15_678.log"
---

--- 2024-07-03T18:20:15.757Z debug
Failed to load .env file ".env": Error: ENOENT: no such file or directory, open '.env'
    at Object.openSync (node:fs:582:18)
    at Object.readFileSync (node:fs:461:35)
    at tryLoadDotEnv (/Users/ad/node_modules/wrangler/wrangler-dist/cli.js:159106:72)
    at loadDotEnv (/Users/ad/node_modules/wrangler/wrangler-dist/cli.js:159115:12)
    at Array.reduce (<anonymous>) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '.env'
}
---

--- 2024-07-03T18:20:15.764Z debug
Retrieving cached values for account from ../../../node_modules/.cache/wrangler
---

--- 2024-07-03T18:20:15.764Z log
🌀 Creating new finetune "warp-mistral-new" for model "@cf/mistral/mistral-7b-instruct-v0.2-lora"...
---
--- 2024-07-03T18:20:15.765Z debug
-- START CF API REQUEST: POST https://api.cloudflare.com/client/v4/accounts/<account_id_masqueraded>/ai/finetunes
---
--- 2024-07-03T18:20:15.765Z debug
HEADERS: omitted; set WRANGLER_LOG_SANITIZE=false to include sanitized data
---
--- 2024-07-03T18:20:15.765Z debug
INIT: omitted; set WRANGLER_LOG_SANITIZE=false to include sanitized data
---
--- 2024-07-03T18:20:15.765Z debug
-- END CF API REQUEST
---

--- 2024-07-03T18:20:17.189Z debug
-- START CF API RESPONSE: Bad Request 400
---

--- 2024-07-03T18:20:17.190Z debug
HEADERS: omitted; set WRANGLER_LOG_SANITIZE=false to include sanitized data
---

--- 2024-07-03T18:20:17.190Z debug
RESPONSE: omitted; set WRANGLER_LOG_SANITIZE=false to include sanitized data
---

--- 2024-07-03T18:20:17.190Z debug
-- END CF API RESPONSE
---

--- 2024-07-03T18:20:17.229Z error
✘ [ERROR] 🚨 Finetune couldn't be created: A request to the Cloudflare API (/accounts/<account_id_masqueraded>/ai/finetunes) failed. FINETUNE_EXISTS_ERROR: finetune with the name 'warp-mistral-new' already exists [code: 1000]

--- 2024-07-03T18:20:15.757Z debug
🪵  Writing logs to "/Users/agdfdf/.wrangler/logs/wrangler-2024-07-03_18-20-15_678.log"
---

--- 2024-07-03T18:20:15.757Z debug
Failed to load .env file ".env": Error: ENOENT: no such file or directory, open '.env'
    at Object.openSync (node:fs:582:18)
    at Object.readFileSync (node:fs:461:35)
    at tryLoadDotEnv (/Users/ad/node_modules/wrangler/wrangler-dist/cli.js:159106:72)
    at loadDotEnv (/Users/ad/node_modules/wrangler/wrangler-dist/cli.js:159115:12)
    at Array.reduce (<anonymous>) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '.env'
}
---

--- 2024-07-03T18:20:15.764Z debug
Retrieving cached values for account from ../../../node_modules/.cache/wrangler
---

--- 2024-07-03T18:20:15.764Z log
🌀 Creating new finetune "warp-mistral-new" for model "@cf/mistral/mistral-7b-instruct-v0.2-lora"...
---
--- 2024-07-03T18:20:15.765Z debug
-- START CF API REQUEST: POST https://api.cloudflare.com/client/v4/accounts/<account_id_masqueraded>/ai/finetunes
---
--- 2024-07-03T18:20:15.765Z debug
HEADERS: omitted; set WRANGLER_LOG_SANITIZE=false to include sanitized data
---
--- 2024-07-03T18:20:15.765Z debug
INIT: omitted; set WRANGLER_LOG_SANITIZE=false to include sanitized data
---
--- 2024-07-03T18:20:15.765Z debug
-- END CF API REQUEST
---

--- 2024-07-03T18:20:17.189Z debug
-- START CF API RESPONSE: Bad Request 400
---

--- 2024-07-03T18:20:17.190Z debug
HEADERS: omitted; set WRANGLER_LOG_SANITIZE=false to include sanitized data
---

--- 2024-07-03T18:20:17.190Z debug
RESPONSE: omitted; set WRANGLER_LOG_SANITIZE=false to include sanitized data
---

--- 2024-07-03T18:20:17.190Z debug
-- END CF API RESPONSE
---

--- 2024-07-03T18:20:17.229Z error
✘ [ERROR] 🚨 Finetune couldn't be created: A request to the Cloudflare API (/accounts/<account_id_masqueraded>/ai/finetunes) failed. FINETUNE_EXISTS_ERROR: finetune with the name 'warp-mistral-new' already exists [code: 1000]

fifaaaron88OP•7/3/24, 6:53 PM

error_messages

message.txt3.29KB

Chy•7/5/24, 3:03 PM

request.body.query not working and request.text()??

Chy•7/5/24, 3:22 PM

nvm it changed

tsutsu•7/5/24, 4:59 PM

I understand that currently, developers can only supply LoRAs, rather than arbitrary full models — presumably because, in a multitenant inference setting, you want a few base models everyone is using that can stay always-loaded-in-VRAM, with just the much-smaller LoRAs being loaded/unloaded between calls

but fine-tunes processed with the BitDelta technique (https://www.together.ai/blog/bitdelta; https://arxiv.org/html/2402.10193v2) seem to offer another alternative; the approach has basically the same benefits as LoRAs for multi-tenant settings — keeping the base model(s) always-loaded, and just loading/unloading fine-tune deltas. (In fact, I believe these models would be loaded as adapters by inference frameworks, exactly as LoRAs are — just with these adapter models' layers having the same rank as the original model layers, rather than being low-rank.)

would the Workers AI team potentially consider allowing the use of arbitrary developer-provided fine-tunes (of the fixed set of base models), by computing and caching each fine-tune's BitDelta against the base-model?

(or by uploading pre-computed BitDeltas — but I don't think there's a standard format/encoding for these yet, or even open-source tooling to generate them, as these are a performance/resource optimization mostly relevant only to multi-tenant inference platforms. so the developer pointing at the fine-tune's URL, and CF fetching it + computing a BitDelta from it, is probably better UX for now... unless your team also wants to be the one to open-source the format and tooling for doing this

)

trace•7/8/24, 10:18 AM

Is it possible to know by when the text-images model will be out of beta? Mainly curious about the pricing per generation will be.

Ferdi KIZILTOPRAK•7/8/24, 2:03 PM

Hello, Stability.ai updated SD3 license. Do you plan to add it to your catalog? https://stability.ai/news/license-update?utm_source=www.therundown.ai&utm_medium=newsletter&utm_campaign=openai-s-secret-hack-exposed

Stability AI

Community License — Stability AI

Our new Community License is now free for research, non-commercial, and commercial use. You only need a paid Enterprise license if your yearly revenues exceed USD$1M and you use Stability AI models in commercial products or services.

user•7/10/24, 11:31 AM

Is there a way to set

max_tokens

max_tokens

runWithTools

runWithTools

Name With Exactly 32 Characters!•7/11/24, 1:26 PM

Want to run resnet50 but for image embeddings, not classifications. Is this possible?

Peps•7/14/24, 3:43 PM

Does Cloudflare run some kind of moderation check on input to models?
I'm looking into using Workers AI to identify content which may be CSAM related, as unfortunately OpenAI's free moderation API was not sufficient for my use case and mostly gave me false negatives.
Hopefully the user-generated input I'll give to models won't trigger some kind of flag/check, so just want to make sure that my use case is alright.

akazwz•7/14/24, 4:07 PM

Hello, I build a LLMs/AI chat platform build and design for Cloudflare worker AI. with many features like function calling, image to text, text to image, text to speech. all build with Cloudflare worker. https://github.com/akazwz/antonai

GitHub

GitHub - akazwz/antonai: a LLMs/AI chat platform build and design f...

a LLMs/AI chat platform build and design for Cloudflare worker AI. with many features like function calling, image to text, text to image, text to speech - akazwz/antonai

RRahul is it possible to upload image like chatgpt and ask llama to analyse and answer ...

Craig Dennis•7/15/24, 2:46 PM

If you are looking for an Image-To-Text app, here is one built with Llava + SvelteKit https://floor-is-llava.pages.dev/ (code in the footer)

PPeps Does Cloudflare run some kind of moderation check on input to models? I'm lookin...

Craig Dennis•7/15/24, 2:49 PM

We do offer the LlamaGuard model: https://ai.meta.com/research/publications/llama-guard-llm-based-input-output-safeguard-for-human-ai-conversations/ Does this look like it might assist with what you are after?

Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversa...

We introduce Llama Guard, an LLM-based input-output safeguard model geared towards Human-AI conversation use cases. Our model incorporates a safety risk...

Uuser Is there a way to set `max_tokens` on `runWithTools`?

user•7/16/24, 2:41 PM

Uuser ?

Raylight•7/16/24, 3:47 PM

No. You would have to modify runWithTools to be able to do that, e.g. https://github.com/cloudflare/ai-utils/compare/main...Raylight-PWL:ai-utils:Raylight-patch-maxTokens-option.

user•7/16/24, 3:58 PM

:frown:

DDhravya Shah not right now - not sure how that would work though, Because runwithtools make...

user•7/17/24, 5:56 PM

I think a good idea would be to have either max_tokens be the max tokens for each call, or have the user able to configure the max tokens for each call

user•7/17/24, 5:57 PM

the second option being something like initial_max_tokens, tool_max_tokens, and possibly trim_max_tokens (very scuffed index names, but you get the point)

user•7/17/24, 6:02 PM

Its been really good

user•7/17/24, 6:02 PM

~~its nice to actually have types available unlike on normal cf-AI~~

user•7/17/24, 6:02 PM

on normal CF ai i had to guess so much

user•7/17/24, 6:03 PM

ai-utils is definitely awesome in comparison

user•7/17/24, 6:04 PM

i once tried to read the actual AI library on npm to figure out unknowns

user•7/17/24, 6:04 PM

Never trying that again!

user•7/17/24, 6:04 PM

compiled code :salute:

user•7/17/24, 6:05 PM

np!! :D

user•7/17/24, 6:05 PM

I have a quick question

user•7/17/24, 6:05 PM

considering you do ai-utils shenanigans

user•7/17/24, 6:05 PM

are there any plans on tool usage being supported on the qwen models?

user•7/17/24, 6:06 PM

from what I read, the actual model itself supports it, but I'm obviously not entirely sure on it

DDhravya Shah they _should_ already work with qwen models actually

user•7/17/24, 6:06 PM

It's not documented as so, and when I tried I could not get it to work haha

user•7/17/24, 6:06 PM

then again I could also have been doing it wrong.

user•7/17/24, 6:06 PM

No, it just never actually ran the tool code

user•7/17/24, 6:07 PM

but I also couldn't get it to work on the documented model so I was probably doing it wrong

user•7/17/24, 6:07 PM

one secaroo

user•7/17/24, 6:07 PM

if i can find the project within the mess of my projects folder...

user•7/17/24, 6:09 PM

--- 2024-07-03T18:20:15.757Z debug 🪵 Writing logs to "/Users/agdfdf/.wrangler/logs/wrangler-2024-07-03_18-20-15_678.log" --- --- 2024-07-03T18:20:15.757Z debug Failed to load .env file ".env": Error: ENOENT: no such file or directory, open '.env' at Object.openSync (node:fs:582:18) at Object.readFileSync (node:fs:461:35) at tryLoadDotEnv (/Users/ad/node_modules/wrangler/wrangler-dist/cli.js:159106:72) at loadDotEnv (/Users/ad/node_modules/wrangler/wrangler-dist/cli.js:159115:12) at Array.reduce (<anonymous>) { errno: -2, code: 'ENOENT', syscall: 'open', path: '.env' } --- --- 2024-07-03T18:20:15.764Z debug Retrieving cached values for account from ../../../node_modules/.cache/wrangler --- --- 2024-07-03T18:20:15.764Z log 🌀 Creating new finetune "warp-mistral-new" for model "@cf/mistral/mistral-7b-instruct-v0.2-lora"... --- --- 2024-07-03T18:20:15.765Z debug -- START CF API REQUEST: POST https://api.cloudflare.com/client/v4/accounts/<account_id_masqueraded>/ai/finetunes --- --- 2024-07-03T18:20:15.765Z debug HEADERS: omitted; set WRANGLER_LOG_SANITIZE=false to include sanitized data --- --- 2024-07-03T18:20:15.765Z debug INIT: omitted; set WRANGLER_LOG_SANITIZE=false to include sanitized data --- --- 2024-07-03T18:20:15.765Z debug -- END CF API REQUEST --- --- 2024-07-03T18:20:17.189Z debug -- START CF API RESPONSE: Bad Request 400 --- --- 2024-07-03T18:20:17.190Z debug HEADERS: omitted; set WRANGLER_LOG_SANITIZE=false to include sanitized data --- --- 2024-07-03T18:20:17.190Z debug RESPONSE: omitted; set WRANGLER_LOG_SANITIZE=false to include sanitized data --- --- 2024-07-03T18:20:17.190Z debug -- END CF API RESPONSE --- --- 2024-07-03T18:20:17.229Z error ✘ [ERROR] 🚨 Finetune couldn't be created: A request to the Cloudflare API (/accounts/<account_id_masqueraded>/ai/finetunes) failed. FINETUNE_EXISTS_ERROR: finetune with the name 'warp-mistral-new' already exists [code: 1000]

Unable to add a fine-tune. Error: https://developers.cloudflare.com/workers-ai/tutorials/fine-tune-m

Similar Threads