API call to the cf: https://developers.cloudflare.com/api/operations/workers-ai-search-model If you

V

VictorOP•6/28/24, 7:30 PM

You could also turn it into a ci build step that runs on a schedule and include it in your release
I did it here in GA in bash: https://github.com/demosjarco/matt-ai/blob/production/.github/workflows/ai-catalog.yml
End result (json): https://github.com/demosjarco/matt-ai/blob/production/shared/workers-ai-catalog.json
End result (ts): https://github.com/demosjarco/matt-ai/blob/production/shared/workers-ai-catalog.ts

GitHub

matt-ai/.github/workflows/ai-catalog.yml at production · demosjarco...

Magically All The Things AI. Contribute to demosjarco/matt-ai development by creating an account on GitHub.

GitHub

matt-ai/shared/workers-ai-catalog.json at production · demosjarco/m...

Magically All The Things AI. Contribute to demosjarco/matt-ai development by creating an account on GitHub.

GitHub

matt-ai/shared/workers-ai-catalog.ts at production · demosjarco/mat...

Magically All The Things AI. Contribute to demosjarco/matt-ai development by creating an account on GitHub.

VVictor API call to the cf: https://developers.cloudflare.com/api/operations/workers-ai-...

V

VictorOP•6/28/24, 7:32 PM

@Perry Sahnow because these count towards the global rate limit (1200/5m - https://developers.cloudflare.com/fundamentals/api/reference/limits/), you could cache the api results or go the build step method

PPerry Sahnow Yeah they presumably don't change more often than daily

V

VictorOP•6/28/24, 7:40 PM

I personally have mine set to update weekly and just watch and see how big of a commit (if any) happens in a given week - adjust from there. I did ask michelle about scheduled update cycle of models going forward, but they don't have any planned yet

云

云•6/28/24, 11:42 PM

huh i think the ai isn't recognizing my tools?

云

云•6/28/24, 11:43 PM

云

云•6/28/24, 11:43 PM

export const factorization = tool({
    description: 'Prime factorize any number even if they are very large',
    parameters: {
        type: 'object',
        properties: {
            number: {
                type: 'string',
                description: 'the number to factorize'
            }
        },
        required: ['number']
    },
    name: 'factorize-num',

    async function({ number }) {
        console.log('huh', number);
        return await factorize(number);
    }
});

export const factorization = tool({
    description: 'Prime factorize any number even if they are very large',
    parameters: {
        type: 'object',
        properties: {
            number: {
                type: 'string',
                description: 'the number to factorize'
            }
        },
        required: ['number']
    },
    name: 'factorize-num',

    async function({ number }) {
        console.log('huh', number);
        return await factorize(number);
    }
});

export const factorization = tool({
    description: 'Prime factorize any number even if they are very large',
    parameters: {
        type: 'object',
        properties: {
            number: {
                type: 'string',
                description: 'the number to factorize'
            }
        },
        required: ['number']
    },
    name: 'factorize-num',

    async function({ number }) {
        console.log('huh', number);
        return await factorize(number);
    }
});

export const factorization = tool({
    description: 'Prime factorize any number even if they are very large',
    parameters: {
        type: 'object',
        properties: {
            number: {
                type: 'string',
                description: 'the number to factorize'
            }
        },
        required: ['number']
    },
    name: 'factorize-num',

    async function({ number }) {
        console.log('huh', number);
        return await factorize(number);
    }
});

云

云•6/28/24, 11:43 PM

the function is not being called at all

云

云•6/28/24, 11:43 PM

{
    "response": "The number you've provided, 1203210203982198283, is a large number and appears to be an integer. However, finding the prime factorization of such a large number is not practical to do by hand, as it typically involves the use of specialized algorithms and computational resources. \n\nIn general, the factorization of large numbers is often done using numerical methods or mathematical software, such as the quadratic sieve, trial division, or Lenstra elliptic curve factorization, methods typically employed in cryptography or number theory research.\n\nFor all practical purposes, we would use computational tools to find the prime factorization of this number. To get an approximate factorization, you might use an online calculator or a programming language like Python with a library that supports number theory, such as `sympy`:\n\n\`\`\`python\nimport sympy as sp\n\nnumber = 1203210203982198283\n_factors = sp.factorint(number)\n_factors\n\`\`\`\nYou would need to run this code in an environment where you have the `sympy` library installed.\n\nIf you just want to know if it's a prime number, you should also consider that finding the factorization of a very large number to determine primality is computationally expensive, and for such a large number, it's highly unlikely to be a prime.\n\nIf you're seeking a prime factorization for educational purposes or a specific reason, please provide more context, and I can try to help further. Otherwise, it's best to use specialized software or an online tool to get the factorization."
}

{
    "response": "The number you've provided, 1203210203982198283, is a large number and appears to be an integer. However, finding the prime factorization of such a large number is not practical to do by hand, as it typically involves the use of specialized algorithms and computational resources. \n\nIn general, the factorization of large numbers is often done using numerical methods or mathematical software, such as the quadratic sieve, trial division, or Lenstra elliptic curve factorization, methods typically employed in cryptography or number theory research.\n\nFor all practical purposes, we would use computational tools to find the prime factorization of this number. To get an approximate factorization, you might use an online calculator or a programming language like Python with a library that supports number theory, such as `sympy`:\n\n\`\`\`python\nimport sympy as sp\n\nnumber = 1203210203982198283\n_factors = sp.factorint(number)\n_factors\n\`\`\`\nYou would need to run this code in an environment where you have the `sympy` library installed.\n\nIf you just want to know if it's a prime number, you should also consider that finding the factorization of a very large number to determine primality is computationally expensive, and for such a large number, it's highly unlikely to be a prime.\n\nIf you're seeking a prime factorization for educational purposes or a specific reason, please provide more context, and I can try to help further. Otherwise, it's best to use specialized software or an online tool to get the factorization."
}

{
    "response": "The number you've provided, 1203210203982198283, is a large number and appears to be an integer. However, finding the prime factorization of such a large number is not practical to do by hand, as it typically involves the use of specialized algorithms and computational resources. \n\nIn general, the factorization of large numbers is often done using numerical methods or mathematical software, such as the quadratic sieve, trial division, or Lenstra elliptic curve factorization, methods typically employed in cryptography or number theory research.\n\nFor all practical purposes, we would use computational tools to find the prime factorization of this number. To get an approximate factorization, you might use an online calculator or a programming language like Python with a library that supports number theory, such as `sympy`:\n\n\`\`\`python\nimport sympy as sp\n\nnumber = 1203210203982198283\n_factors = sp.factorint(number)\n_factors\n\`\`\`\nYou would need to run this code in an environment where you have the `sympy` library installed.\n\nIf you just want to know if it's a prime number, you should also consider that finding the factorization of a very large number to determine primality is computationally expensive, and for such a large number, it's highly unlikely to be a prime.\n\nIf you're seeking a prime factorization for educational purposes or a specific reason, please provide more context, and I can try to help further. Otherwise, it's best to use specialized software or an online tool to get the factorization."
}

{
    "response": "The number you've provided, 1203210203982198283, is a large number and appears to be an integer. However, finding the prime factorization of such a large number is not practical to do by hand, as it typically involves the use of specialized algorithms and computational resources. \n\nIn general, the factorization of large numbers is often done using numerical methods or mathematical software, such as the quadratic sieve, trial division, or Lenstra elliptic curve factorization, methods typically employed in cryptography or number theory research.\n\nFor all practical purposes, we would use computational tools to find the prime factorization of this number. To get an approximate factorization, you might use an online calculator or a programming language like Python with a library that supports number theory, such as `sympy`:\n\n\`\`\`python\nimport sympy as sp\n\nnumber = 1203210203982198283\n_factors = sp.factorint(number)\n_factors\n\`\`\`\nYou would need to run this code in an environment where you have the `sympy` library installed.\n\nIf you just want to know if it's a prime number, you should also consider that finding the factorization of a very large number to determine primality is computationally expensive, and for such a large number, it's highly unlikely to be a prime.\n\nIf you're seeking a prime factorization for educational purposes or a specific reason, please provide more context, and I can try to help further. Otherwise, it's best to use specialized software or an online tool to get the factorization."
}

云

云•6/28/24, 11:45 PM

didn't work even if i change the type of number to "number"

云

云•6/28/24, 11:51 PM

nvm i got it fixed

DDhravya Shah Hi everyone! we just made making AI agents 10x easier with a new open source pac...

M

MissS•6/30/24, 3:50 AM

Can this be done from durable objects?

S

scotto•6/30/24, 5:27 PM

News for multilingual embedding model ?

C

csilk•7/1/24, 4:41 AM

qq; is there a limit on the response length from the LLM?

C

csilk•7/1/24, 6:23 AM

And, a "less quick" question
Is there a way to do a

json_format

json_format

json_format

json_format response with workers AI? (like what OpenAI supports)
Specifically trying to get an LLM to take an input like "dogs" with a system message of "generate a trivia question based on the topic and respond in json format of

{ question: "string", answer: "string" }

{ question: "string", answer: "string" }

{ question: "string", answer: "string" }

{ question: "string", answer: "string" }
Do the worker ai response is

{question: "what is the best dog", answer: "all of them"}

{question: "what is the best dog", answer: "all of them"}

{question: "what is the best dog", answer: "all of them"}

{question: "what is the best dog", answer: "all of them"} (made this up

)

Ccsilk qq; is there a limit on the response length from the LLM?

R

rob•7/1/24, 10:48 AM

Ya each model has diff max token output

DDhravya Shah which ones do you wanna see? 👀

S

scotto•7/1/24, 12:23 PM

BAAI/bge-m3 is ok, from the same family of the already supported

A

AngusMa•7/1/24, 2:05 PM

Why didn’t Workers AI Playground work?

AAngusMa Why didn’t Workers AI Playground work?

P

Ping for toast•7/1/24, 4:00 PM

homepage demo is also broken

DDhravya Shah fixing

P

Ping for toast•7/1/24, 4:23 PM

did you also see my video

PPing for toast homepage demo is also broken

P

Ping for toast•7/1/24, 4:23 PM

here

P

Ping for toast•7/1/24, 4:24 PM

maybe related issues?

P

Ping for toast•7/1/24, 4:24 PM

but

DDhravya Shah Yeah should be fixed (gradually rolling out)

P

Ping for toast•7/1/24, 4:25 PM

sweet, I was finna say "its still not working" but nice ninja edit and nice quick fix!

A

AngusMa•7/1/24, 5:22 PM

I logged all data I had received from a streaming Workers AI call and got a key called

p

p. What is that?

R

rob•7/1/24, 5:23 PM

https://blog.cloudflare.com/ai-side-channel-attack-mitigated

The Cloudflare Blog

Mitigating a token-length side-channel attack in our AI products

The Workers AI and AI Gateway team recently collaborated closely with security researchers at Ben Gurion University regarding a report submitted through our Public Bug Bounty program. Through this process, we discovered and fully patched a vulnerability affecting all LLM providers. Here’s the story

R

rob•7/1/24, 5:39 PM

thats the p thing ^

DDhravya Shah How did you fix it?

云

云•7/1/24, 9:18 PM

messed around with the descriptions until it used my tool

云

云•7/1/24, 9:18 PM

~~:lul: learning how to satisfy our future ai overlords~~

V

vyastj•7/2/24, 9:31 AM

Hello, I want to use CF worker routes to deploy few security headers on all the incoming traffic except many subdomains (huge list). Hence i wrote a worker in

wrangler.toml

name = "security-headers-prod-us"
main = "index.py"
compatibility_date = "2024-03-20"
account_id = "your-account-id"
route = notexcluded.domain.com/*"
zone_id = "domain.com"

name = "security-headers-prod-us"
main = "index.py"
compatibility_date = "2024-03-20"
account_id = "your-account-id"
route = notexcluded.domain.com/*"
zone_id = "domain.com"

name = "security-headers-prod-us"
main = "index.py"
compatibility_date = "2024-03-20"
account_id = "your-account-id"
route = notexcluded.domain.com/*"
zone_id = "domain.com"

name = "security-headers-prod-us"
main = "index.py"
compatibility_date = "2024-03-20"
account_id = "your-account-id"
route = notexcluded.domain.com/*"
zone_id = "domain.com"

with index.py

from js import Response, fetch

async def on_fetch(request):
    # List of excluded domains
    excluded_domains = [
        "example1.domain.com",
        "example2.comain.com",
        "...",
        "example3.domain.com"
 ]

   # Get the host from the request URL
    url = request.url
    hostname = url.split("/")[2]

    # Check if the hostname should be excluded
    if any(hostname.endswith(domain) for domain in excluded_domains):
        response = await fetch(request)
    else:
        response = await fetch(request)
        new_response = Response.new(response.body, response)
        
        new_response.headers.append(
        "x-workers-hello",
        "Hello from Cloudflare Workers"
        )

        new_response.headers.set("x-frame-options", "SAMEORIGIN")
        new_response.headers.set("x-content-type-options", "nosniff")
        new_response.headers.set("referrer-policy", "strict-origin-when-cross-origin")
        new_response.headers.set("content-security-policy", "object-src 'none' blob:; base-uri 'self'; report-uri https://cspappdirect.report-uri.com/r/d/csp/enforce; worker-src 'self' blob:")

        response = new_response

    return new_response

from js import Response, fetch

async def on_fetch(request):
    # List of excluded domains
    excluded_domains = [
        "example1.domain.com",
        "example2.comain.com",
        "...",
        "example3.domain.com"
 ]

   # Get the host from the request URL
    url = request.url
    hostname = url.split("/")[2]

    # Check if the hostname should be excluded
    if any(hostname.endswith(domain) for domain in excluded_domains):
        response = await fetch(request)
    else:
        response = await fetch(request)
        new_response = Response.new(response.body, response)
        
        new_response.headers.append(
        "x-workers-hello",
        "Hello from Cloudflare Workers"
        )

        new_response.headers.set("x-frame-options", "SAMEORIGIN")
        new_response.headers.set("x-content-type-options", "nosniff")
        new_response.headers.set("referrer-policy", "strict-origin-when-cross-origin")
        new_response.headers.set("content-security-policy", "object-src 'none' blob:; base-uri 'self'; report-uri https://cspappdirect.report-uri.com/r/d/csp/enforce; worker-src 'self' blob:")

        response = new_response

    return new_response

from js import Response, fetch

async def on_fetch(request):
    # List of excluded domains
    excluded_domains = [
        "example1.domain.com",
        "example2.comain.com",
        "...",
        "example3.domain.com"
 ]

   # Get the host from the request URL
    url = request.url
    hostname = url.split("/")[2]

    # Check if the hostname should be excluded
    if any(hostname.endswith(domain) for domain in excluded_domains):
        response = await fetch(request)
    else:
        response = await fetch(request)
        new_response = Response.new(response.body, response)
        
        new_response.headers.append(
        "x-workers-hello",
        "Hello from Cloudflare Workers"
        )

        new_response.headers.set("x-frame-options", "SAMEORIGIN")
        new_response.headers.set("x-content-type-options", "nosniff")
        new_response.headers.set("referrer-policy", "strict-origin-when-cross-origin")
        new_response.headers.set("content-security-policy", "object-src 'none' blob:; base-uri 'self'; report-uri https://cspappdirect.report-uri.com/r/d/csp/enforce; worker-src 'self' blob:")

        response = new_response

    return new_response

from js import Response, fetch

async def on_fetch(request):
    # List of excluded domains
    excluded_domains = [
        "example1.domain.com",
        "example2.comain.com",
        "...",
        "example3.domain.com"
 ]

   # Get the host from the request URL
    url = request.url
    hostname = url.split("/")[2]

    # Check if the hostname should be excluded
    if any(hostname.endswith(domain) for domain in excluded_domains):
        response = await fetch(request)
    else:
        response = await fetch(request)
        new_response = Response.new(response.body, response)
        
        new_response.headers.append(
        "x-workers-hello",
        "Hello from Cloudflare Workers"
        )

        new_response.headers.set("x-frame-options", "SAMEORIGIN")
        new_response.headers.set("x-content-type-options", "nosniff")
        new_response.headers.set("referrer-policy", "strict-origin-when-cross-origin")
        new_response.headers.set("content-security-policy", "object-src 'none' blob:; base-uri 'self'; report-uri https://cspappdirect.report-uri.com/r/d/csp/enforce; worker-src 'self' blob:")

        response = new_response

    return new_response

V

vyastj•7/2/24, 9:36 AM

But this does not seem to work when i do a curl to check for added headers

curl -I https://notexcluded.domain.com

curl -I https://notexcluded.domain.com

curl -I https://notexcluded.domain.com

curl -I https://notexcluded.domain.com

Vvyastj Hello, I want to use CF worker routes to deploy few security headers on all the ...

H

hydroxide•7/2/24, 10:27 AM

Do you have a middleware.js?

Hhydroxide Do you have a middleware.js?

V

vyastj•7/2/24, 10:38 AM

Nopes i have not set that up. When i was setting the worker, CF gave me python + wrangler as a template

Vvyastj Nopes i have not set that up. When i was setting the worker, CF gave me python +...

H

hydroxide•7/2/24, 10:50 AM

Ok, that’s how I set my headers. Not sure exactly how to do it with python functions but read this, it should help https://developers.cloudflare.com/pages/functions/middleware/

Cloudflare Docs

Middleware · Cloudflare Pages docs

Middleware is reusable logic that can be run before your onRequest function. Middlewares are typically utility functions. Error handling, user …

Hhydroxide Ok, that’s how I set my headers. Not sure exactly how to do it with python funct...

V

vyastj•7/2/24, 10:54 AM

okay so should i replace python with js ?

Vvyastj okay so should i replace python with js ?

H

hydroxide•7/2/24, 12:40 PM

I doubt you need to do that, I just haven’t worked with python on CF so I don’t know how to implement middleware. I’m sure the info is in the link I gave you

Q

qinfengge•7/2/24, 1:35 PM

Hello I use spring AI framework for development and I found a problem

Q

qinfengge•7/2/24, 1:36 PM

In this document I saw that it supports OPENAI’s API format
https://developers.cloudflare.com/workers-ai/configuration/open-ai-compatibility/

Cloudflare Docs

OpenAI compatible API endpoints · Cloudflare Workers AI docs

Workers AI supports OpenAI compatible endpoints for text generation (/v1/chat/completions) and text embedding models (/v1/embeddings). This allows you …

API call to the cf: https://developers.cloudflare.com/api/operations/workers-ai-search-model If you

Similar Threads