@Ali_hat what type of tasks are you trying to get from it ? or more so what's the use case
@Ali_hat what type of tasks are you trying to get from it ? or more so what's the use case
RI have something I made for this exctly, for getting models to return your structured data properly DM me if you want a link
I@rob Without logit-level constraints it's not possible to have a guarantee that it will return structured data every time.
Roh yea for sure. it'll either retry and error but it's a drastic increase over other available methods or having to prompt engineer every single model
AHi @rob, I basically use it for Q&A based on customer data. I don't need structured data as I only need plain text. Here's the code in a nutshell.
Rremembering context from previous HTTP calls
what do you mean there?
what do you mean there?
℠I’d bet my bald head that he means the model keeps training on chat-data instead of starting every chat-session fresh (from base model) .
IWhich is not the case - the model only remembers history/messages you give it (if anything) and is not automatically trained or tuned over time.
Hhas anyone successfully used their own fine-tuned lora? I can't successfully upload a
gemma-7b-it finetune at all ('file' should be of valid safetensors type), and I can successfully upload/run inference for mistral/mistral-7b-instruct-v0.2-lora, but the results are worse than the vanilla modelHthe lora works fine outside cloudflare
Hand per other comments, I've restricted the modules to
q_proj and v_projHI finetuned
mistralai/mistral-7b-instruct-v0.2 using the AutoTrain_LLM notebook , then cURL to https://api.cloudflare.com/client/v4/accounts/${account_id}/ai/run/@cf/mistral/mistral-7b-instruct-v0.2-lora
AYes, exactly. Which is not what I'd expect. Of course it's just an impression but it's a bit weird for me when I call the endpoint the first time (a decent response) and a second/third time with the same parameters (now it's a much shorter and less informative response).
℠That’s the randomness at play
℠Maybe you want to fine tune it and train it to provide more consistent answers in the form that you need?
℠Asking GPT the exact same question would also yield me sometimes very different answers - don’t know if that’s changed, as I haven’t been working too much with the chat models
℠In SD you can provide a fixed seed, which means you can keep reloading the same prompt until the style details are some you like. Then you can lock that seed in and continue working on variants/changes but instead of entirely different outputs, it stays in the vicinity of the original image
℠So you can make alts with a similar looking character in the same landscape and style, etc
℠Maybe you can do something akin to that with yours
AYes, that's eventually what I'd like to do. And getting different answers is usually fine. I had hoped to squeeze out a bit more without finetuning but I'm maybe being too naive.
AThat's a good hint, I haven't experimented with using seed yet.
℠As it sounds like you don’t really need a dynamic bot for long conversation, but just one that does single a Q&A at a time
PIs anyone running langchain on workers? Curious to get some insights into examples of costs
HSure, feel free to DM me
SIs there any plan to support an image embedding model in the api?
Vanyone know of an open source model better than whisper? If not anyone tried training it?
IDepends. Nvidia's Parakeet is quite good at noisy audio (far better than Whisper in my experience). Haven't tried training/finetuning it but the base model is quite good.
IParakeet has two versions - one with no punctuation that is best for accuracy (but is missing punctuation) and one that has punctuation like Whisper that has a little less accuracy.
IIf you go with the one that doesn't have punctuation there are other models designed to take text as input and add punctuation that you can run on the text after.
GWhat's the highest token count LLM available on workers ai?
C
CArgument of type '"@hf/thebloke/deepseek-coder-6.7b-instruct-awq"' is not assignable to parameter of type 'BaseAiImageToTextModels'. typescript (2769) [84, 37]Cthe code still works if I
// @ts-expect-error
KWhat version of workers-types?
C4.20240620.0
RToken count as in context window? qwen1.5-7b-chat-awq can handle a context of ~20k tokens, qwen1.5-14b-chat-awq ~7.5k, and llama-3-8b-instruct-awq ~8k. (Based on my testing.) Those are also limits for the total number of tokens, including output.
RHmm.. interesting..
@cf/mistral/mistral-7b-instruct-v0.2-lora
云huh i tried to pass number array as image to dreamshaper
云but got the error
YHey folks,
I am getting authentication errors when using the workers AI, the curl and python requests that is auto generated also is failing.
{'success': False, 'errors': [{'code': 10000, 'message': 'Authentication error'}]}
I am getting authentication errors when using the workers AI, the curl and python requests that is auto generated also is failing.
{'success': False, 'errors': [{'code': 10000, 'message': 'Authentication error'}]}
R@cf/lykon/dreamshaper-8-lcm@cf/runwayml/stable-diffusion-v1-5-img2img