Sure! Thanks... I was just double checking if theres any list out there that i miss out...

W

WallacyOP•5/2/24, 6:29 PM

Anyway, just to know... I just deployed one @cf/llava test and worked very fast... good job!

WWallacy Sure! Thanks... I was just double checking if theres any list out there that i m...

C

Chaika•5/2/24, 6:29 PM

there's the list in the dashboard which includes llava

C

Chaika•5/2/24, 6:29 PM

C

Chaika•5/2/24, 6:30 PM

iirc the docs build from that list so I assume the dash will always be more updated/latest if not containing everything

CChaika Click to see attachment

W

WallacyOP•5/2/24, 6:32 PM

Thanks you so much! For some reason i didn't bother to look on the dashboard.

Mmichelle i wish 😢 docs doesn't build dynamically - we have to manually PR it. moving tow...

C

Chaika•5/2/24, 6:32 PM

yea I just meant that afaik this script https://github.com/craigsdennis/tmp-model-contentator/blob/main/scripts/gen-simple-docs.ts is used to build those ai docs which is pulling from the same api as the dash

W

WallacyOP•5/2/24, 6:32 PM

Im the RTFM guy... i aways go to the docs first. Now i know! Thanks.

Original message was deleted

I

Isaac McFadyen•5/2/24, 6:39 PM

@944890 Please don't post phishing links in this Discord (or in this channel, it's unrelated to Workers AI).

I

Isaac McFadyen•5/2/24, 6:39 PM

I'm removing both your messages so people don't click on them.

W

WallacyOP•5/2/24, 7:41 PM

Hey, i have one more question... Just to know better what to expect on the future. Theres any reasonable. limit for memory or other metrict that we can expect to run on workers-ai? Like, will 70b models be a thing on future?

C

Chaika•5/3/24, 1:20 AM

Am I confusing something or as of sometime today was @cf/bytedance/stable-diffusion-xl-lightning and @cf/stabilityai/stable-diffusion-xl-base-1.0 changed from returning a ReadableStream to an object containing a int array of the output? The example doesn't seem to work anymore either because of it

L

lui•5/3/24, 3:53 PM

Hi! I'm evaluating the usage of Whisper from a cloudflare worker for the company and wanted to know if there's any other argument besides the

audio

audio

audio

audio in the input that can be used? looking at this example:

    const input = {
      audio: [...new Uint8Array(blob)],
      // what are the other input options?
    };

    const response = await env.AI.run(
      "@cf/openai/whisper",
      input
    );

    const input = {
      audio: [...new Uint8Array(blob)],
      // what are the other input options?
    };

    const response = await env.AI.run(
      "@cf/openai/whisper",
      input
    );

    const input = {
      audio: [...new Uint8Array(blob)],
      // what are the other input options?
    };

    const response = await env.AI.run(
      "@cf/openai/whisper",
      input
    );

    const input = {
      audio: [...new Uint8Array(blob)],
      // what are the other input options?
    };

    const response = await env.AI.run(
      "@cf/openai/whisper",
      input
    );

anything from this OpenAI doc?

V

Vitaliy Shevchenko•5/4/24, 8:49 AM

How to fix error:

No such model @cf/meta/llama-3-8b-instruct or task

No such model @cf/meta/llama-3-8b-instruct or task

No such model @cf/meta/llama-3-8b-instruct or task

No such model @cf/meta/llama-3-8b-instruct or task ? I want to launch a text chat bot as an online assistant. It seems to me that this model is better suited, but it cannot be used due to an error.

VVitaliy Shevchenko How to fix error: `No such model @cf/meta/llama-3-8b-instruct or task` ? I want ...

R

Raylight•5/4/24, 8:58 AM

Sounds like you're using the

@cloudflare/ai

@cloudflare/ai

@cloudflare/ai

@cloudflare/ai npm package. It's depreciated, so use the native binding to be able to access newly added models. See https://developers.cloudflare.com/workers-ai/changelog/.

M

Mrinank•5/4/24, 6:05 PM

import { CheerioWebBaseLoader } from 'langchain/document_loaders/web/cheerio';
import { MemoryVectorStore } from 'langchain/vectorstores/memory';
import { OpenAI } from 'langchain/llms/openai';
import { OpenAIEmbeddings } from 'langchain/embeddings/openai';
import { RetrievalQAChain } from 'langchain/chains';

export default {
        async fetch(request, env, ctx) {
                const loader = new CheerioWebBaseLoader('https://en.wikipedia.org/wiki/Brooklyn');
                const docs = await loader.loadAndSplit();
                //console.log(docs);

                const store = await MemoryVectorStore.fromDocuments(docs, new OpenAIEmbeddings({ openAIApiKey: env.OPENAI_API_KEY }));

                const model = new OpenAI({ openAIApiKey: env.OPENAI_API_KEY });
                const chain = RetrievalQAChain.fromLLM(model, store.asRetriever());

                const { searchParams } = new URL(request.url);
                const question = searchParams.get('question') ?? 'What is this article about? Can you give me 3 facts about it?';

                const res = await chain.call({
                        query: question,
                });
                console.log(res.text);

                return new Response(res.text);
        },
};

import { CheerioWebBaseLoader } from 'langchain/document_loaders/web/cheerio';
import { MemoryVectorStore } from 'langchain/vectorstores/memory';
import { OpenAI } from 'langchain/llms/openai';
import { OpenAIEmbeddings } from 'langchain/embeddings/openai';
import { RetrievalQAChain } from 'langchain/chains';

export default {
        async fetch(request, env, ctx) {
                const loader = new CheerioWebBaseLoader('https://en.wikipedia.org/wiki/Brooklyn');
                const docs = await loader.loadAndSplit();
                //console.log(docs);

                const store = await MemoryVectorStore.fromDocuments(docs, new OpenAIEmbeddings({ openAIApiKey: env.OPENAI_API_KEY }));

                const model = new OpenAI({ openAIApiKey: env.OPENAI_API_KEY });
                const chain = RetrievalQAChain.fromLLM(model, store.asRetriever());

                const { searchParams } = new URL(request.url);
                const question = searchParams.get('question') ?? 'What is this article about? Can you give me 3 facts about it?';

                const res = await chain.call({
                        query: question,
                });
                console.log(res.text);

                return new Response(res.text);
        },
};

import { CheerioWebBaseLoader } from 'langchain/document_loaders/web/cheerio';
import { MemoryVectorStore } from 'langchain/vectorstores/memory';
import { OpenAI } from 'langchain/llms/openai';
import { OpenAIEmbeddings } from 'langchain/embeddings/openai';
import { RetrievalQAChain } from 'langchain/chains';

export default {
        async fetch(request, env, ctx) {
                const loader = new CheerioWebBaseLoader('https://en.wikipedia.org/wiki/Brooklyn');
                const docs = await loader.loadAndSplit();
                //console.log(docs);

                const store = await MemoryVectorStore.fromDocuments(docs, new OpenAIEmbeddings({ openAIApiKey: env.OPENAI_API_KEY }));

                const model = new OpenAI({ openAIApiKey: env.OPENAI_API_KEY });
                const chain = RetrievalQAChain.fromLLM(model, store.asRetriever());

                const { searchParams } = new URL(request.url);
                const question = searchParams.get('question') ?? 'What is this article about? Can you give me 3 facts about it?';

                const res = await chain.call({
                        query: question,
                });
                console.log(res.text);

                return new Response(res.text);
        },
};

import { CheerioWebBaseLoader } from 'langchain/document_loaders/web/cheerio';
import { MemoryVectorStore } from 'langchain/vectorstores/memory';
import { OpenAI } from 'langchain/llms/openai';
import { OpenAIEmbeddings } from 'langchain/embeddings/openai';
import { RetrievalQAChain } from 'langchain/chains';

export default {
        async fetch(request, env, ctx) {
                const loader = new CheerioWebBaseLoader('https://en.wikipedia.org/wiki/Brooklyn');
                const docs = await loader.loadAndSplit();
                //console.log(docs);

                const store = await MemoryVectorStore.fromDocuments(docs, new OpenAIEmbeddings({ openAIApiKey: env.OPENAI_API_KEY }));

                const model = new OpenAI({ openAIApiKey: env.OPENAI_API_KEY });
                const chain = RetrievalQAChain.fromLLM(model, store.asRetriever());

                const { searchParams } = new URL(request.url);
                const question = searchParams.get('question') ?? 'What is this article about? Can you give me 3 facts about it?';

                const res = await chain.call({
                        query: question,
                });
                console.log(res.text);

                return new Response(res.text);
        },
};

how to make this work with cf embeddings instead of openai? any blog or guide?

i got this code from https://blog.cloudflare.com/langchain-and-cloudflare

K

KTibow•5/4/24, 7:13 PM

has CLIP already been requested

V

Vitaliy Shevchenko•5/5/24, 1:36 AM

How to avoid this error for llama-3-8b-instruct?

must have required property 'prompt', must NOT have more than 6144 characters, must match exactly one schema in oneOf

must have required property 'prompt', must NOT have more than 6144 characters, must match exactly one schema in oneOf

must have required property 'prompt', must NOT have more than 6144 characters, must match exactly one schema in oneOf

must have required property 'prompt', must NOT have more than 6144 characters, must match exactly one schema in oneOf

Can anyone provide an example of the correct typescript code to check the prompt size and trim it? I use the messages array, which I pass to the llm model call; in this array there is an arbitrary number of nested arrays with the keys “user/system/assistant” and “content”.

V

Vitaliy Shevchenko•5/5/24, 9:49 AM

https://developers.cloudflare.com/workers-ai/fine-tunes/

Any information or predictions on when LoRA Fine Tuning for llama-3-8b-instruct will be available?

Cloudflare Docs

Fine-tunes · Cloudflare Workers AI docs

Learn how to use Workers AI to get fine-tuned inference.

VVitaliy Shevchenko How to avoid this error for llama-3-8b-instruct? `must have required property '...

R

Raylight•5/5/24, 11:13 AM

afaik it's not documented, but I run into that error if any single message in the messages array has more than ~6144 characters. There's also a total token limit where the model (at least with llama-3-8b-instruct) will stop generating. Seems to be somewhere around 2700-2800 tokens.

MMrinank ```typescript import { CheerioWebBaseLoader } from 'langchain/document_loaders/w...

R

Raylight•5/5/24, 12:38 PM

You can swap out OpenAIEmbeddings with CloudflareWorkersAIEmbeddings. Definitely not a guide, but.. well, here's an example: https://gist.github.com/Raylight-PWL/50d7a20193874c6c784759001bb26e5c.

P

preetramsha•5/5/24, 1:20 PM

Hello I am thinking of publishing an app with workers-ai API. but I am concerned about rate limits.
can my app make thousands of requests per minute?

P

preetramsha•5/5/24, 1:21 PM

please help

RRaylight You can swap out OpenAIEmbeddings with CloudflareWorkersAIEmbeddings. Definitely...

M

Mrinank•5/5/24, 2:11 PM

Thanks it worked :MeowHeartCloudflare:

MMrinank Thanks it worked :MeowHeartCloudflare:

R

Raylight•5/5/24, 4:12 PM

Great! Beware that the example with the Brooklyn article consumes a lot of neurons.

RRaylight Great! Beware that the example with the Brooklyn article consumes a *lot* of neu...

M

Mrinank•5/5/24, 6:48 PM

yeah it took 550+ neurons but i am in free tier

M

Mrinank•5/5/24, 6:48 PM

were those from llm or while generating embeddings?

Ppreetramsha Hello I am thinking of publishing an app with workers-ai API. but I am concerned...

R

rob•5/6/24, 1:06 PM

Yes

Rrob Yes

P

preetramsha•5/6/24, 1:11 PM

but how do I handle more requests with than the minits specified?

J

Jens•5/6/24, 1:28 PM

hi, I try to implement the Langchain pdf loader into my application. But if I call

loader.load()

loader.load()

loader.load()

loader.load() I always get the error message:

RangeError: Maximum call stack size exceeded

RangeError: Maximum call stack size exceeded

RangeError: Maximum call stack size exceeded

RangeError: Maximum call stack size exceeded Does someone know what could be the cause? I have already tried it with small pdf files, but that didn't help either

notes.post('/', async (c: CustomContext) => {
    // Need to polyfill a method that Cloudflare Workers is missing for the PDF loader
  globalThis.setImmediate = ((fn: () => {}) => setTimeout(fn, 0)) as any
    const cloudflareFetchResponse = await fetch(
        'https://www.cloudflare.com/resources/assets/slt3lc6tev37/3HWObubm6fybC0FWUdFYAJ/5d5e3b0a4d9c5a7619984ed6076f01fe/Cloudflare_for_Campaigns_Security_Guide.pdf'
    )
    const cloudflarePdfBlob = await cloudflareFetchResponse.blob()
    const laoder = new WebPDFLoader(cloudflarePdfBlob)
    const docs = await loader.load() //RangeError: Maximum call stack size exceeded

notes.post('/', async (c: CustomContext) => {
    // Need to polyfill a method that Cloudflare Workers is missing for the PDF loader
  globalThis.setImmediate = ((fn: () => {}) => setTimeout(fn, 0)) as any
    const cloudflareFetchResponse = await fetch(
        'https://www.cloudflare.com/resources/assets/slt3lc6tev37/3HWObubm6fybC0FWUdFYAJ/5d5e3b0a4d9c5a7619984ed6076f01fe/Cloudflare_for_Campaigns_Security_Guide.pdf'
    )
    const cloudflarePdfBlob = await cloudflareFetchResponse.blob()
    const laoder = new WebPDFLoader(cloudflarePdfBlob)
    const docs = await loader.load() //RangeError: Maximum call stack size exceeded

notes.post('/', async (c: CustomContext) => {
    // Need to polyfill a method that Cloudflare Workers is missing for the PDF loader
  globalThis.setImmediate = ((fn: () => {}) => setTimeout(fn, 0)) as any
    const cloudflareFetchResponse = await fetch(
        'https://www.cloudflare.com/resources/assets/slt3lc6tev37/3HWObubm6fybC0FWUdFYAJ/5d5e3b0a4d9c5a7619984ed6076f01fe/Cloudflare_for_Campaigns_Security_Guide.pdf'
    )
    const cloudflarePdfBlob = await cloudflareFetchResponse.blob()
    const laoder = new WebPDFLoader(cloudflarePdfBlob)
    const docs = await loader.load() //RangeError: Maximum call stack size exceeded

notes.post('/', async (c: CustomContext) => {
    // Need to polyfill a method that Cloudflare Workers is missing for the PDF loader
  globalThis.setImmediate = ((fn: () => {}) => setTimeout(fn, 0)) as any
    const cloudflareFetchResponse = await fetch(
        'https://www.cloudflare.com/resources/assets/slt3lc6tev37/3HWObubm6fybC0FWUdFYAJ/5d5e3b0a4d9c5a7619984ed6076f01fe/Cloudflare_for_Campaigns_Security_Guide.pdf'
    )
    const cloudflarePdfBlob = await cloudflareFetchResponse.blob()
    const laoder = new WebPDFLoader(cloudflarePdfBlob)
    const docs = await loader.load() //RangeError: Maximum call stack size exceeded

The part of this code is from the langchain repository https://github.com/jacoblee93/langchain-nuxt-cloudflare-template

Thanks

GitHub

GitHub - jacoblee93/langchain-nuxt-cloudflare-template

Contribute to jacoblee93/langchain-nuxt-cloudflare-template development by creating an account on GitHub.

JJens hi, I try to implement the Langchain pdf loader into my application. But if I ca...

J

Jens•5/6/24, 3:55 PM

with this loader it works

:

  const loader = new CheerioWebBaseLoader('https://developers.cloudflare.com/workers-ai/models/', {
        selector: 'body',
    })

    const docs = await loader.load()

  const loader = new CheerioWebBaseLoader('https://developers.cloudflare.com/workers-ai/models/', {
        selector: 'body',
    })

    const docs = await loader.load()

  const loader = new CheerioWebBaseLoader('https://developers.cloudflare.com/workers-ai/models/', {
        selector: 'body',
    })

    const docs = await loader.load()

  const loader = new CheerioWebBaseLoader('https://developers.cloudflare.com/workers-ai/models/', {
        selector: 'body',
    })

    const docs = await loader.load()

U

Unsmart•5/6/24, 4:04 PM

Whats up with the workers ai usage tab? Cost summary fails to load, daily usage always says 0, the "all models" section says I have only used it on April 11th, and the model specific ones say I have only used them on April 10th. (im a time traveler I guess mustve went ahead a few days used it and broke something my bad)

U

Unsmart•5/6/24, 4:06 PM

oh wait I just realized april was last month I was thinking that wouldve been this month :lul:

UUnsmart Whats up with the workers ai usage tab? Cost summary fails to load, daily usage ...

C

Chaika•5/6/24, 4:06 PM

this month is May lol

C

Chaika•5/6/24, 4:06 PM

only GA models bill you, which is some of the confusion maybe

C

Chaika•5/6/24, 4:06 PM

mine looks sane

U

Unsmart•5/6/24, 4:07 PM

hmm

U

Unsmart•5/6/24, 4:08 PM

Yeah I guess I only do use the beta models atm kinda stupid to not have usage data on them

RRaylight Sounds like you're using the `@cloudflare/ai` npm package. It's depreciated, so ...

K

kingmesal•5/7/24, 12:10 AM

Regarding this https://developers.cloudflare.com/workers-ai/changelog/.
What is the type for TypeScript? I cannot find anything on this

Cloudflare Docs

Changelog · Cloudflare Workers AI docs

Review recent changes to Cloudflare Workers AI.

Kkingmesal Regarding this https://developers.cloudflare.com/workers-ai/changelog/. What is ...

R

Raylight•5/7/24, 5:36 AM

They should be in

@cloudflare/workers-types

@cloudflare/workers-types

@cloudflare/workers-types

@cloudflare/workers-types. https://github.com/cloudflare/workerd/blob/main/types/defines/ai.d.ts if you want to look at them. That's the best I got.

R

Raven•5/8/24, 8:45 AM

Can we serve custom models on Workers AI or do we have to wait for them to get added?

R

Raven•5/8/24, 8:46 AM

And if it is not currently possible to serve custom models via Cloudflare, and it's not prohibited to discuss, what are some good options? I have a list and I'm checking it twice, but curious what people think and/or use

RRaven Can we serve custom models on Workers AI or do we have to wait for them to get a...

I

Isaac McFadyen•5/8/24, 12:54 PM

?workers-ai-models

Sure! Thanks... I was just double checking if theres any list out there that i miss out...

Similar Threads