I have an app which would benefit from a simple KV solution

P

pippin_moleOP•4/29/24, 2:09 PM

But the actual python project is too big to sit in a worker - and also isnt compatible with webassembly

Ppippin_mole I have an app which would benefit from a simple KV solution

C

Chaika•4/29/24, 2:10 PM

your options are API or Worker Proxy

API is easier, but has a global rate limit 1200 requests per 5 minutes, shared with your account including dashboard, and all request have to go back to core/no edge caching

Worker is harder and you pay per request but you get edge caching

CChaika your options are API or Worker Proxy API is easier, but has a global rate limit...

P

pippin_moleOP•4/29/24, 2:10 PM

Is this for KV or the actual python project

Ppippin_mole Is this for KV or the actual python project

C

Chaika•4/29/24, 2:12 PM

what do you mean?

C

Chaika•4/29/24, 2:12 PM

Your python project would either use the CF API directly, or you use a CF Worker to proxy requests to the KV binding, bound to that worker

P

pippin_moleOP•4/29/24, 2:13 PM

I'm confused as to what you're talking about regarding 'your options are API or Worker Proxy' - do you mean as a replacement for KV or where I should host my Python project

CChaika Your python project would either use the CF API directly, or you use a CF Worker...

P

pippin_moleOP•4/29/24, 2:14 PM

Again, the CF Workers require a build to webassembly - some of the packages I'm using cannot compile to webassembly

Ppippin_mole I'm confused as to what you're talking about regarding 'your options are API or ...

C

Chaika•4/29/24, 2:14 PM

I mean as a way to access Cloudflare KV

C

Chaika•4/29/24, 2:14 PM

you host your python project wherever, and either you call the CF API to use KV, or you call a worker you make to access KV, both have different benefits/downsides

P

pippin_moleOP•4/29/24, 2:15 PM

Am I not able to just use something like cfkv

PyPI

cfkv

A Python wrapper for the Cloudflare KV Store API

C

Chaika•4/29/24, 2:15 PM

That's using the CF API, so you have those downsides as noted above

global rate limit 1200 requests per 5 minutes, shared with your account including dashboard, and all request have to go back to core/no edge caching

P

pippin_moleOP•4/29/24, 2:15 PM

I dont care so much about edge compatibility - it's only ever going to run on a single instance

P

pippin_moleOP•4/29/24, 2:16 PM

The python project is only ever going to run once per 10-15minutes

C

Chaika•4/29/24, 2:16 PM

then sounds like that could work for you

P

pippin_moleOP•4/29/24, 2:16 PM

I dont really see how workers can ever be used for anything more than returning 'hello world'

P

pippin_moleOP•4/29/24, 2:16 PM

Literally every package I've tried to use requires me to 'request availability from cloudflare team'

Ppippin_mole Literally every package I've tried to use requires me to 'request availability f...

C

Chaika•4/29/24, 2:17 PM

talking about python workers? Yea they're still very much in beta, JS workers are recommended and probably will always be the most supported

P

pippin_moleOP•4/29/24, 2:17 PM

Well yeah I tried using Rust workers, which tbh is in even more of a beta

P

pippin_moleOP•4/29/24, 2:18 PM

Both Rust + Python gave me a bad experience

C

Chaika•4/29/24, 2:18 PM

Yea, JS/TS is just best for workers. I mean it all runs on V8 JS Engine - same as Chrome, Both Rust and Python are just running as WASM and binding back to javascript methods/it's all still JS at the start and finish

BBeny It means your script can use 30s of CPU time for each request to the worker, whi...

R

rootalien•4/29/24, 4:33 PM

I have max 10mb files but what is the bad usage?

Original message was deleted

I

Isaac McFadyen•4/29/24, 4:38 PM

?channel-crossposting

F

Flare•4/29/24, 4:38 PM

Please do not post your question in multiple channels per the rules at #

welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

Original message was deleted

R

rootalien•4/29/24, 4:38 PM

#stream-and-calls

I

Isaac McFadyen•4/29/24, 4:38 PM

You probably want to move to #stream-and-calls yeah.

I

Isaac McFadyen•4/29/24, 4:38 PM

I'll delete your message here.

E

EvanPiro•4/29/24, 4:39 PM

Is sending multi part form data with files supported in workers? I can't seem to get a trivial example working:

      const fileContent = "This is a simple text file.";
      const file = new File([fileContent], "simple_text_file.txt", {
        type: "text/plain",
      });

      formData.set("file", file);
      formData.append("purpose", "fine-tune")
      const fetchOptions: RequestInit = {
        method: "POST",
        headers: {
          Authorization: `Bearer ${apiKey}`,
        },
        body: formData,
      };

      // Make the POST request to OpenAI
      const response = await fetch("https://api.openai.com/v1/files", fetchOptions);

      const fileContent = "This is a simple text file.";
      const file = new File([fileContent], "simple_text_file.txt", {
        type: "text/plain",
      });

      formData.set("file", file);
      formData.append("purpose", "fine-tune")
      const fetchOptions: RequestInit = {
        method: "POST",
        headers: {
          Authorization: `Bearer ${apiKey}`,
        },
        body: formData,
      };

      // Make the POST request to OpenAI
      const response = await fetch("https://api.openai.com/v1/files", fetchOptions);

      const fileContent = "This is a simple text file.";
      const file = new File([fileContent], "simple_text_file.txt", {
        type: "text/plain",
      });

      formData.set("file", file);
      formData.append("purpose", "fine-tune")
      const fetchOptions: RequestInit = {
        method: "POST",
        headers: {
          Authorization: `Bearer ${apiKey}`,
        },
        body: formData,
      };

      // Make the POST request to OpenAI
      const response = await fetch("https://api.openai.com/v1/files", fetchOptions);

      const fileContent = "This is a simple text file.";
      const file = new File([fileContent], "simple_text_file.txt", {
        type: "text/plain",
      });

      formData.set("file", file);
      formData.append("purpose", "fine-tune")
      const fetchOptions: RequestInit = {
        method: "POST",
        headers: {
          Authorization: `Bearer ${apiKey}`,
        },
        body: formData,
      };

      // Make the POST request to OpenAI
      const response = await fetch("https://api.openai.com/v1/files", fetchOptions);

L

Louis•4/29/24, 4:46 PM

I'm working on a project where I need to upload videos to Cloudflare using JavaScript. I found a code that works fine for normal-sized videos, but I've run into an issue when trying to upload videos larger than 200 MB.

I've tried several solutions, but I haven't yet found an effective way to handle the uploading of large videos. Does anyone have an example or can you guide me on what I might be missing or how I can approach this issue?

ERROR CONNECTION

R

rootalien•4/29/24, 4:53 PM

#r2

A

an•4/29/24, 5:47 PM

Is there any way to measure cpu time? Since Date.now() doesn't work as it mentioned in this wiki https://developers.cloudflare.com/workers/runtime-apis/performance
My worker is run heavily on cpu with wasm like processing image , measure only database fetches or api fetches is not enough for request tracing, didn't give me enough info to visual the request timeline

Cloudflare Docs

Performance and timers · Cloudflare Workers docs

Measure timing, performance, and timing of subrequests and other operations.

H

hob8yte•4/29/24, 10:50 PM

Anyone know of resources/projects running ffmpeg.wasm on workers? I want to transcribe some pretty short videos (i think under the 10MB limit around workers size) using workers ai and know very little about both ffmpeg and wasm

Rrootalien I have max 10mb files but what is the bad usage?

B

Beny•4/30/24, 12:57 AM

For example loading the entire file into memory just to send it somewhere else, using streams is better for that

T

tonytheonly•4/30/24, 1:56 AM

I have a worker that I want to call an external api on, but I want to return a response without awaiting the the external api call. (the external api call is to just save time writing to the db on another service).

        const promise = fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        });
        c.executionCtx.waitUntil(promise.then())

        flow.params = { campaignId: campaignId }
        return flow;

        const promise = fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        });
        c.executionCtx.waitUntil(promise.then())

        flow.params = { campaignId: campaignId }
        return flow;

        const promise = fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        });
        c.executionCtx.waitUntil(promise.then())

        flow.params = { campaignId: campaignId }
        return flow;

        const promise = fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        });
        c.executionCtx.waitUntil(promise.then())

        flow.params = { campaignId: campaignId }
        return flow;

T

tonytheonly•4/30/24, 1:57 AM

however when I am testing it local and it is not hitting the service

T

tonytheonly•4/30/24, 1:58 AM

I am using hono https://hono.dev/api/context#executionctx

Context - Hono

Ultrafast web framework for Cloudflare Workers, Fastly Compute, Deno, Bun, Vercel, Node.js, and others. Fast, but not only fast.

T

tonytheonly•4/30/24, 2:00 AM

the worker is on localhost:8787 and the write service for the impression is on localhost:3000 I am able to create the impression successully calling it directly but the waitUntil is not hitting it when calling the localhost:8787 endpoint

T

tonytheonly•4/30/24, 12:15 PM

If not waitUntil what method would I use? I believe I still have to await a durable object call correct?

I

ItsWendell•4/30/24, 12:20 PM

@tony Since this is quite a general question about waitUntil I think it's actually fine do to this in here. Do you see any other logs in your console? Are both endpoints ran through the workers sdk / wrangler?

Give this a try:

        c.executionCtx.waitUntil(fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        }).catch((e) => console.error("Error doing impressions", e))
);

        c.executionCtx.waitUntil(fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        }).catch((e) => console.error("Error doing impressions", e))
);

        c.executionCtx.waitUntil(fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        }).catch((e) => console.error("Error doing impressions", e))
);

        c.executionCtx.waitUntil(fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        }).catch((e) => console.error("Error doing impressions", e))
);

D

Danny Zhao•4/30/24, 12:22 PM

If I have a worker like the official example

export default {
  async fetch(request: Request): Promise<Response> {
    /**
     * Replace `remote` with the host you wish to send requests to
     */
    const remote = "https://example.com";

    return await fetch(remote, request);
  },
};

export default {
  async fetch(request: Request): Promise<Response> {
    /**
     * Replace `remote` with the host you wish to send requests to
     */
    const remote = "https://example.com";

    return await fetch(remote, request);
  },
};

export default {
  async fetch(request: Request): Promise<Response> {
    /**
     * Replace `remote` with the host you wish to send requests to
     */
    const remote = "https://example.com";

    return await fetch(remote, request);
  },
};

export default {
  async fetch(request: Request): Promise<Response> {
    /**
     * Replace `remote` with the host you wish to send requests to
     */
    const remote = "https://example.com";

    return await fetch(remote, request);
  },
};

And the response is about 100MB which takes a user 1 minute to download. How much money may I be charged for each request according to the pricing model?

DDanny Zhao If I have a worker like the official example ```typescript export default { a...

I

ItsWendell•4/30/24, 12:28 PM

You'll be charged for CPU time and amount of requests, not waiting for IO, since this is IO and you're not doing/awaiting for any CPU actions it shouldn't be too much. This will incur probably at most <10ms (probably less) of CPU time per request. I don't think you pay for the data transferred at all, but not 100% sure about that in terms of ingress.

IItsWendell You'll be charged for CPU time and amount of requests, not waiting for IO, since...

D

Danny Zhao•4/30/24, 12:39 PM

Thanks for responding!

I

ItsWendell•4/30/24, 12:47 PM

Also good to monitor this as you go in the dashboard!

IItsWendell @tony Since this is quite a general question about waitUntil I think it's actual...

T

tonytheonly•4/30/24, 12:55 PM

one is a worker the other is a node docker image. Does waitUntil only work on when calling other workers?

I have an app which would benefit from a simple KV solution

Similar Threads

Similar Threads

Similar Threads