Hey, I'm using llama-3.1-8b-instruct to classify and reject spam/inappropriate messages, it works mo

X

xavOP•9/3/24, 12:39 PM

is there a trick beside finetuning the prompt and hope for the best? so far, asking to classify messages in other languages than english seems to trip the model (eg. it sometimes returns "approprié" as the key if the message is in french)

X

xavOP•9/3/24, 12:41 PM

I'm aware there are more specific models, eg. https://huggingface.co/predibase/jigsaw my question is how to tell the workers AI to answer following a specific structure

predibase/jigsaw · Hugging Face

M

mr.niko.la•9/3/24, 2:49 PM

Can we add Parler-TTS to ai workers ?

Parler-TTS is the Hugging Face open source model

Is there any that has implement TTS on cloudflare edge ?

OOreki Are there some hidden output token limits? My response seems to be cut short aft...

R

Raylight•9/3/24, 3:00 PM

Yes, and the limit depends on the model. See workers-ai

RRaylight Yes, and the limit depends on the model. See https://discord.com/channels/59531...

O

Oreki•9/3/24, 4:01 PM

I don't think

@cf/meta/llama-3.1-8b-instruct-fp8

@cf/meta/llama-3.1-8b-instruct-fp8

@cf/meta/llama-3.1-8b-instruct-fp8

@cf/meta/llama-3.1-8b-instruct-fp8 has 32k token limit

O

Oreki•9/3/24, 4:01 PM

I tried it as well, and it somewhat stopped at 3000 characters

Xxav Hey, I'm using llama-3.1-8b-instruct to classify and reject spam/inappropriate m...

R

rob•9/3/24, 8:59 PM

try feeding a asisstant response starting of "{" in the mesg hustoy and itll try. I have a few triks that yield 99% success or so if you want to DM me

V

Vikash•9/6/24, 8:59 AM

How to bind Vector index to Pages project, the button show "Get started" and redirect to /vectorize page while I aleady have an index.

VVikash How to bind Vector index to Pages project, the button show "Get started" and red...

V

Vikash•9/6/24, 4:38 PM

I used the API directly to update bindings, the dashboard is broken I think

V

Vikash•9/6/24, 4:41 PM

I am getting dimensions error in indexing. Any suggestion?

VECTOR_UPSERT_ERROR (code = 40012): invalid vector for id="sdfksd", expected 768 dimensions, and got 1024 dimensions

VECTOR_UPSERT_ERROR (code = 40012): invalid vector for id="sdfksd", expected 768 dimensions, and got 1024 dimensions

const embeddingResult = await env.AI.run('@cf/baai/bge-large-en-v1.5', {
      text:  value,      
    });
    const embeddingBatch: number[][] = embeddingResult.data;

    await env.VECTORIZE.upsert(
      embeddingBatch.map((embedding, index) => ({
        id: sourceId,
        values: embedding,
        namespace: 'default',
        metadata: {
          id: sessionId
        },
      }))
    );

const embeddingResult = await env.AI.run('@cf/baai/bge-large-en-v1.5', {
      text:  value,      
    });
    const embeddingBatch: number[][] = embeddingResult.data;

    await env.VECTORIZE.upsert(
      embeddingBatch.map((embedding, index) => ({
        id: sourceId,
        values: embedding,
        namespace: 'default',
        metadata: {
          id: sessionId
        },
      }))
    );

const embeddingResult = await env.AI.run('@cf/baai/bge-large-en-v1.5', {
      text:  value,      
    });
    const embeddingBatch: number[][] = embeddingResult.data;

    await env.VECTORIZE.upsert(
      embeddingBatch.map((embedding, index) => ({
        id: sourceId,
        values: embedding,
        namespace: 'default',
        metadata: {
          id: sessionId
        },
      }))
    );

const embeddingResult = await env.AI.run('@cf/baai/bge-large-en-v1.5', {
      text:  value,      
    });
    const embeddingBatch: number[][] = embeddingResult.data;

    await env.VECTORIZE.upsert(
      embeddingBatch.map((embedding, index) => ({
        id: sourceId,
        values: embedding,
        namespace: 'default',
        metadata: {
          id: sessionId
        },
      }))
    );

VVikash I am getting dimensions error in indexing. Any suggestion? ``` VECTOR_UPSERT_E...

A

ac•9/6/24, 5:03 PM

bge-large is a 1024-dimension model, so the vectorize index needs to be 1024 dimension as well. Looks like it's 768. bge-base is a 768-dimension model

V

Vikash•9/6/24, 5:17 PM

Ok thanks, I will use the base model

V

Vikash•9/6/24, 5:18 PM

Also, anyway to alter index to change it to 1024?

J

JustinNoel•9/7/24, 12:37 PM

I'm getting really horrible results with all of the text to image AI models (I've tried all of the ones CF has). I'm using the prompts below and getting garbage responses like those in this image.

Does anyone have suggestions on how to actually get what I want?

export default {
  async fetch(request, env) {
    const inputs = {
      prompt: "create an image  that is 512x512. the background should be a solid, plain, yellow color. text over the background should say 'Learn How to Pronounce MySQL' in English. Text should be red and use an Arial font. ",
      negative_prompt: "There shOuld not be any other effects or images.",
      height: 512,
      width: 1024
    };

    const response = await env.AI.run(
      "@cf/bytedance/stable-diffusion-xl-lightning",
      inputs
    );

    return new Response(response, {
      headers: {
        "content-type": "image/png",
      },
    });

  },
};

export default {
  async fetch(request, env) {
    const inputs = {
      prompt: "create an image  that is 512x512. the background should be a solid, plain, yellow color. text over the background should say 'Learn How to Pronounce MySQL' in English. Text should be red and use an Arial font. ",
      negative_prompt: "There shOuld not be any other effects or images.",
      height: 512,
      width: 1024
    };

    const response = await env.AI.run(
      "@cf/bytedance/stable-diffusion-xl-lightning",
      inputs
    );

    return new Response(response, {
      headers: {
        "content-type": "image/png",
      },
    });

  },
};

export default {
  async fetch(request, env) {
    const inputs = {
      prompt: "create an image  that is 512x512. the background should be a solid, plain, yellow color. text over the background should say 'Learn How to Pronounce MySQL' in English. Text should be red and use an Arial font. ",
      negative_prompt: "There shOuld not be any other effects or images.",
      height: 512,
      width: 1024
    };

    const response = await env.AI.run(
      "@cf/bytedance/stable-diffusion-xl-lightning",
      inputs
    );

    return new Response(response, {
      headers: {
        "content-type": "image/png",
      },
    });

  },
};

export default {
  async fetch(request, env) {
    const inputs = {
      prompt: "create an image  that is 512x512. the background should be a solid, plain, yellow color. text over the background should say 'Learn How to Pronounce MySQL' in English. Text should be red and use an Arial font. ",
      negative_prompt: "There shOuld not be any other effects or images.",
      height: 512,
      width: 1024
    };

    const response = await env.AI.run(
      "@cf/bytedance/stable-diffusion-xl-lightning",
      inputs
    );

    return new Response(response, {
      headers: {
        "content-type": "image/png",
      },
    });

  },
};

K

katopz•9/7/24, 2:03 PM

SD is bad at text, Should be better when CF support FLUX i think

K

katopz•9/7/24, 2:06 PM

Text Embeddings also useless to me, only en no multi-language model

K

katopz•9/7/24, 2:10 PM

Other model also outdate, LoRA stuck at

llama-2

llama-2

, CF ai seem to be useless to me atm.

Kkatopz SD is bad at text, Should be better when CF support FLUX i think 🤔

I

Isaac McFadyen•9/7/24, 5:07 PM

This ^ plus SD uses CLIP for prompting which doesn't like natural language very much... comma seperated tags work better, for example:
Bad:

create me an image that is 512x512, with text that says "my text here" and a red background, with yellow text

create me an image that is 512x512, with text that says "my text here" and a red background, with yellow text

Good:

red background, text saying "my text here", arial font, yellow color

red background, text saying "my text here", arial font, yellow color

K

katopz•9/8/24, 4:16 AM

yeah, that too.

U

user•9/8/24, 10:14 AM

is the documentation for stable-diffusion-xl-base-1.0 incorrect?

Cloudflare Docs

stable-diffusion-xl-base-1.0 | Cloudflare Workers AI docs

Diffusion-based text-to-image generative model by Stability AI. Generates and modify images based on text prompts.

U

user•9/8/24, 10:14 AM

it says the response is a binary string but its an object

U

user•9/8/24, 10:15 AM

or is it a blob?

U

user•9/8/24, 10:23 AM

response, not input

U

user•9/8/24, 10:24 AM

https://user.you-sk.id/XMGRn78KexWR

uwu

owo

U

user•9/8/24, 10:25 AM

it claims its a binary string but the type is an object with zero keys

U

user•9/8/24, 10:26 AM

sending a request to the api directly returns an image correctly

U

user•9/8/24, 10:27 AM

but using env.AI does not

U

user•9/8/24, 10:31 AM

ah

U

user•9/8/24, 10:35 AM

Then how come doing

New blob([res])

New blob([res])

returns a blob with raw content of

[object Object]

[object Object]

?

U

user•9/8/24, 10:36 AM

logging JSON.stringify() on the response returns

{}

{}

U

user•9/8/24, 10:41 AM

    const input: TTIInput = {
        prompt: prompt,
        strength: strength,
    };

    const res = await env.AI.run("@cf/stabilityai/stable-diffusion-xl-base-1.0", input);

    const blob = new Blob([res]);

    console.log(await blob.text());

    const input: TTIInput = {
        prompt: prompt,
        strength: strength,
    };

    const res = await env.AI.run("@cf/stabilityai/stable-diffusion-xl-base-1.0", input);

    const blob = new Blob([res]);

    console.log(await blob.text());

    const input: TTIInput = {
        prompt: prompt,
        strength: strength,
    };

    const res = await env.AI.run("@cf/stabilityai/stable-diffusion-xl-base-1.0", input);

    const blob = new Blob([res]);

    console.log(await blob.text());

    const input: TTIInput = {
        prompt: prompt,
        strength: strength,
    };

    const res = await env.AI.run("@cf/stabilityai/stable-diffusion-xl-base-1.0", input);

    const blob = new Blob([res]);

    console.log(await blob.text());

U

user•9/8/24, 10:41 AM

this logs [object Object]

U

user•9/8/24, 10:52 AM

well, when I log JSON.stringify(res) it prints "{}"

U

user•9/8/24, 10:52 AM

so...

U

user•9/8/24, 10:57 AM

well i'd figure it would throw or return nil, or at the bare minimum return "[]"

U

user•9/8/24, 11:01 AM

? wdym

U

user•9/8/24, 11:03 AM

if i do

let a = new Uint8Array(1)
console.log(JSON.stringify(a))

let a = new Uint8Array(1)
console.log(JSON.stringify(a))

let a = new Uint8Array(1)
console.log(JSON.stringify(a))

let a = new Uint8Array(1)
console.log(JSON.stringify(a))

it logs

{"0":0}

{"0":0}

, but when i log the response from ai.run it's just

{}

{}

U

user•9/8/24, 11:06 AM

also, doing

console.log(res instanceof Uint8Array)

console.log(res instanceof Uint8Array)

logs "false", so

U

user•9/8/24, 11:07 AM

:UShrug:

U

user•9/8/24, 11:08 AM

im fairly certain it's an object

L

landon•9/8/24, 11:28 PM

with the run method

const resp = await this.ai.run(model, input);

const resp = await this.ai.run(model, input);

it is returning a

ReadableStream<Uint8Array>

ReadableStream<Uint8Array>

rather than a

Uint8Array

Uint8Array

like the types are saying. I'm having to coerce it into the correct type before being able to manipulate it

here is some working code:

      const resp = (await this.ai.run(model, input)) as unknown;

      let buffer: Uint8Array;

      if (resp instanceof ReadableStream) {
        const chunks: Uint8Array[] = [];
        // eslint-disable-next-line no-restricted-syntax
        for await (const chunk of resp as ReadableStream<Uint8Array>) {
          chunks.push(chunk);
        }
        buffer = new Uint8Array(chunks.reduce((acc, chunk) => acc + chunk.length, 0));
        let offset = 0;
        // eslint-disable-next-line no-restricted-syntax
        for (const chunk of chunks) {
          buffer.set(chunk, offset);
          offset += chunk.length;
        }
      }
      else if (resp instanceof Uint8Array) {
        buffer = resp;
      }
      else {
        throw new Error("Unknown return type for the ai run");
      }

      const base64Image = fromUint8Array(buffer);

      const resp = (await this.ai.run(model, input)) as unknown;

      let buffer: Uint8Array;

      if (resp instanceof ReadableStream) {
        const chunks: Uint8Array[] = [];
        // eslint-disable-next-line no-restricted-syntax
        for await (const chunk of resp as ReadableStream<Uint8Array>) {
          chunks.push(chunk);
        }
        buffer = new Uint8Array(chunks.reduce((acc, chunk) => acc + chunk.length, 0));
        let offset = 0;
        // eslint-disable-next-line no-restricted-syntax
        for (const chunk of chunks) {
          buffer.set(chunk, offset);
          offset += chunk.length;
        }
      }
      else if (resp instanceof Uint8Array) {
        buffer = resp;
      }
      else {
        throw new Error("Unknown return type for the ai run");
      }

      const base64Image = fromUint8Array(buffer);

      const resp = (await this.ai.run(model, input)) as unknown;

      let buffer: Uint8Array;

      if (resp instanceof ReadableStream) {
        const chunks: Uint8Array[] = [];
        // eslint-disable-next-line no-restricted-syntax
        for await (const chunk of resp as ReadableStream<Uint8Array>) {
          chunks.push(chunk);
        }
        buffer = new Uint8Array(chunks.reduce((acc, chunk) => acc + chunk.length, 0));
        let offset = 0;
        // eslint-disable-next-line no-restricted-syntax
        for (const chunk of chunks) {
          buffer.set(chunk, offset);
          offset += chunk.length;
        }
      }
      else if (resp instanceof Uint8Array) {
        buffer = resp;
      }
      else {
        throw new Error("Unknown return type for the ai run");
      }

      const base64Image = fromUint8Array(buffer);

      const resp = (await this.ai.run(model, input)) as unknown;

      let buffer: Uint8Array;

      if (resp instanceof ReadableStream) {
        const chunks: Uint8Array[] = [];
        // eslint-disable-next-line no-restricted-syntax
        for await (const chunk of resp as ReadableStream<Uint8Array>) {
          chunks.push(chunk);
        }
        buffer = new Uint8Array(chunks.reduce((acc, chunk) => acc + chunk.length, 0));
        let offset = 0;
        // eslint-disable-next-line no-restricted-syntax
        for (const chunk of chunks) {
          buffer.set(chunk, offset);
          offset += chunk.length;
        }
      }
      else if (resp instanceof Uint8Array) {
        buffer = resp;
      }
      else {
        throw new Error("Unknown return type for the ai run");
      }

      const base64Image = fromUint8Array(buffer);

L

landon•9/8/24, 11:34 PM

fromUint8Array(buffer)

fromUint8Array(buffer)

is from

js-base64

js-base64

L

landon•9/8/24, 11:38 PM

so you can use that to convert your response to an actual Uint8Array. hopefully they fix the types, or the implementation to return a Uint8Array soon. its gonna break my code when they do though...

L

landon•9/8/24, 11:43 PM

editing it to make it resilient against the inevitable bug fix

Llandon editing it to make it resilient against the inevitable bug fix

U

user•9/9/24, 2:27 AM

U

user•9/9/24, 2:27 AM

thank you much

U

user•9/9/24, 2:28 AM

should this be a bug report?

R

rob•9/9/24, 11:14 AM

Slow af

Llandon with the run method `const resp = await this.ai.run(model, input);` it is return...

R

Raylight•9/9/24, 2:16 PM

Cool! An alternative way to read the stream to completion is

const buffer = await new Response(resp).arrayBuffer();

const buffer = await new Response(resp).arrayBuffer();

.

IIsaac McFadyen This ^ plus SD uses CLIP for prompting which doesn't like natural language very ...

J

JustinNoel•9/10/24, 11:42 AM

Thanks for the replies @katopz and Isaac. I was so hopeful about "Good: red background, text saying "my text here", arial font, yellow color".

Hey, I'm using llama-3.1-8b-instruct to classify and reject spam/inappropriate messages, it works mo

Similar Threads

Similar Threads

Similar Threads