I have max 10mb files but what is the bad usage?

Original message was deleted

I

Isaac McFadyen•4/29/24, 4:38 PM

?channel-crossposting

F

Flare•4/29/24, 4:38 PM

Please do not post your question in multiple channels per the rules at #

welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

Original message was deleted

R

rootalienOP•4/29/24, 4:38 PM

#stream-and-calls

I

Isaac McFadyen•4/29/24, 4:38 PM

You probably want to move to #stream-and-calls yeah.

I

Isaac McFadyen•4/29/24, 4:38 PM

I'll delete your message here.

E

EvanPiro•4/29/24, 4:39 PM

Is sending multi part form data with files supported in workers? I can't seem to get a trivial example working:

      const fileContent = "This is a simple text file.";
      const file = new File([fileContent], "simple_text_file.txt", {
        type: "text/plain",
      });

      formData.set("file", file);
      formData.append("purpose", "fine-tune")
      const fetchOptions: RequestInit = {
        method: "POST",
        headers: {
          Authorization: `Bearer ${apiKey}`,
        },
        body: formData,
      };

      // Make the POST request to OpenAI
      const response = await fetch("https://api.openai.com/v1/files", fetchOptions);

      const fileContent = "This is a simple text file.";
      const file = new File([fileContent], "simple_text_file.txt", {
        type: "text/plain",
      });

      formData.set("file", file);
      formData.append("purpose", "fine-tune")
      const fetchOptions: RequestInit = {
        method: "POST",
        headers: {
          Authorization: `Bearer ${apiKey}`,
        },
        body: formData,
      };

      // Make the POST request to OpenAI
      const response = await fetch("https://api.openai.com/v1/files", fetchOptions);

      const fileContent = "This is a simple text file.";
      const file = new File([fileContent], "simple_text_file.txt", {
        type: "text/plain",
      });

      formData.set("file", file);
      formData.append("purpose", "fine-tune")
      const fetchOptions: RequestInit = {
        method: "POST",
        headers: {
          Authorization: `Bearer ${apiKey}`,
        },
        body: formData,
      };

      // Make the POST request to OpenAI
      const response = await fetch("https://api.openai.com/v1/files", fetchOptions);

      const fileContent = "This is a simple text file.";
      const file = new File([fileContent], "simple_text_file.txt", {
        type: "text/plain",
      });

      formData.set("file", file);
      formData.append("purpose", "fine-tune")
      const fetchOptions: RequestInit = {
        method: "POST",
        headers: {
          Authorization: `Bearer ${apiKey}`,
        },
        body: formData,
      };

      // Make the POST request to OpenAI
      const response = await fetch("https://api.openai.com/v1/files", fetchOptions);

L

Louis•4/29/24, 4:46 PM

I'm working on a project where I need to upload videos to Cloudflare using JavaScript. I found a code that works fine for normal-sized videos, but I've run into an issue when trying to upload videos larger than 200 MB.

I've tried several solutions, but I haven't yet found an effective way to handle the uploading of large videos. Does anyone have an example or can you guide me on what I might be missing or how I can approach this issue?

ERROR CONNECTION

R

rootalienOP•4/29/24, 4:53 PM

#r2

A

an•4/29/24, 5:47 PM

Is there any way to measure cpu time? Since Date.now() doesn't work as it mentioned in this wiki https://developers.cloudflare.com/workers/runtime-apis/performance
My worker is run heavily on cpu with wasm like processing image , measure only database fetches or api fetches is not enough for request tracing, didn't give me enough info to visual the request timeline

Cloudflare Docs

Performance and timers · Cloudflare Workers docs

Measure timing, performance, and timing of subrequests and other operations.

H

hob8yte•4/29/24, 10:50 PM

Anyone know of resources/projects running ffmpeg.wasm on workers? I want to transcribe some pretty short videos (i think under the 10MB limit around workers size) using workers ai and know very little about both ffmpeg and wasm

Rrootalien I have max 10mb files but what is the bad usage?

B

Beny•4/30/24, 12:57 AM

For example loading the entire file into memory just to send it somewhere else, using streams is better for that

T

tonytheonly•4/30/24, 1:56 AM

I have a worker that I want to call an external api on, but I want to return a response without awaiting the the external api call. (the external api call is to just save time writing to the db on another service).

        const promise = fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        });
        c.executionCtx.waitUntil(promise.then())

        flow.params = { campaignId: campaignId }
        return flow;

        const promise = fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        });
        c.executionCtx.waitUntil(promise.then())

        flow.params = { campaignId: campaignId }
        return flow;

        const promise = fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        });
        c.executionCtx.waitUntil(promise.then())

        flow.params = { campaignId: campaignId }
        return flow;

        const promise = fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        });
        c.executionCtx.waitUntil(promise.then())

        flow.params = { campaignId: campaignId }
        return flow;

T

tonytheonly•4/30/24, 1:57 AM

however when I am testing it local and it is not hitting the service

T

tonytheonly•4/30/24, 1:58 AM

I am using hono https://hono.dev/api/context#executionctx

Context - Hono

Ultrafast web framework for Cloudflare Workers, Fastly Compute, Deno, Bun, Vercel, Node.js, and others. Fast, but not only fast.

T

tonytheonly•4/30/24, 2:00 AM

the worker is on localhost:8787 and the write service for the impression is on localhost:3000 I am able to create the impression successully calling it directly but the waitUntil is not hitting it when calling the localhost:8787 endpoint

T

tonytheonly•4/30/24, 12:15 PM

If not waitUntil what method would I use? I believe I still have to await a durable object call correct?

I

ItsWendell•4/30/24, 12:20 PM

@tony Since this is quite a general question about waitUntil I think it's actually fine do to this in here. Do you see any other logs in your console? Are both endpoints ran through the workers sdk / wrangler?

Give this a try:

        c.executionCtx.waitUntil(fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        }).catch((e) => console.error("Error doing impressions", e))
);

        c.executionCtx.waitUntil(fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        }).catch((e) => console.error("Error doing impressions", e))
);

        c.executionCtx.waitUntil(fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        }).catch((e) => console.error("Error doing impressions", e))
);

        c.executionCtx.waitUntil(fetch('http://localhost:3000/impression', {
            method: "POST",
            body: JSON.stringify(data),
            headers: {
                "content-type": "application/json",
            }
        }).catch((e) => console.error("Error doing impressions", e))
);

D

Danny Zhao•4/30/24, 12:22 PM

If I have a worker like the official example

export default {
  async fetch(request: Request): Promise<Response> {
    /**
     * Replace `remote` with the host you wish to send requests to
     */
    const remote = "https://example.com";

    return await fetch(remote, request);
  },
};

export default {
  async fetch(request: Request): Promise<Response> {
    /**
     * Replace `remote` with the host you wish to send requests to
     */
    const remote = "https://example.com";

    return await fetch(remote, request);
  },
};

export default {
  async fetch(request: Request): Promise<Response> {
    /**
     * Replace `remote` with the host you wish to send requests to
     */
    const remote = "https://example.com";

    return await fetch(remote, request);
  },
};

export default {
  async fetch(request: Request): Promise<Response> {
    /**
     * Replace `remote` with the host you wish to send requests to
     */
    const remote = "https://example.com";

    return await fetch(remote, request);
  },
};

And the response is about 100MB which takes a user 1 minute to download. How much money may I be charged for each request according to the pricing model?

DDanny Zhao If I have a worker like the official example ```typescript export default { a...

I

ItsWendell•4/30/24, 12:28 PM

You'll be charged for CPU time and amount of requests, not waiting for IO, since this is IO and you're not doing/awaiting for any CPU actions it shouldn't be too much. This will incur probably at most <10ms (probably less) of CPU time per request. I don't think you pay for the data transferred at all, but not 100% sure about that in terms of ingress.

IItsWendell You'll be charged for CPU time and amount of requests, not waiting for IO, since...

D

Danny Zhao•4/30/24, 12:39 PM

Thanks for responding!

I

ItsWendell•4/30/24, 12:47 PM

Also good to monitor this as you go in the dashboard!

IItsWendell @tony Since this is quite a general question about waitUntil I think it's actual...

T

tonytheonly•4/30/24, 12:55 PM

one is a worker the other is a node docker image. Does waitUntil only work on when calling other workers?

IItsWendell @tony Since this is quite a general question about waitUntil I think it's actual...

T

tonytheonly•4/30/24, 12:58 PM

The entire function is wrapped in a try catch so any error would be logged and thrown.

T

tonytheonly•4/30/24, 12:59 PM

That being said I get now errors, and when looking at the logs of the localhost:3000 i see no console log of the request. However when I post the same information in postman to localhost:3000 I get a console log

G

George Boot•4/30/24, 1:10 PM

I am getting

Cannot read properties of undefined (reading 'writeDataPoint')

Cannot read properties of undefined (reading 'writeDataPoint')

Cannot read properties of undefined (reading 'writeDataPoint')

Cannot read properties of undefined (reading 'writeDataPoint') when trying analytics engine. I've added the binding to my toml and deployed. Is there anything else I need to to to start using analytics engine?

G

George Boot•4/30/24, 1:11 PM

It seems like my analytics_engine_datasets isnt actually bound to the worker. A console.log of the env doesn't show the binding

M

matiasjMusante•4/30/24, 1:15 PM

Does someone know something about function usage on static page rendering in next on pages, as disccused above:

https://community.cloudflare.com/t/loading-static-page-counts-towards-function-request/535651

Is this really like this? Is this situation shared among all other frameworks/ssgenerators? I mean, do they all incur in function usage when rendering static pages? I was reading about Astro, and it seems nice but I dont know if may be it´s the same with all frameworks.

MmatiasjMusante Does someone know something about function usage on static page rendering in nex...

I

Isaac McFadyen•4/30/24, 1:20 PM

?channel-crossposting

F

Flare•4/30/24, 1:20 PM

Please do not post your question in multiple channels per the rules at #

welcome-and-rules. It creates confusion for people trying to help you and doesn't get your issue or question solved any faster.

M

matiasjMusante•4/30/24, 1:23 PM

Oh sorry I didn´t mean to!! I just didn´t know exactly where to ask,

BBeny For example loading the entire file into memory just to send it somewhere else, ...

R

rootalienOP•4/30/24, 3:29 PM

The company I work for doesn't want to do this. because they have their own software.

R

Rich•4/30/24, 3:47 PM

Does anybody know if the Cache API supports Cache Reserve?

R

Rich•4/30/24, 3:54 PM

do you know if Cache Reserve / R2 is global or per colo?

C

Chaika•4/30/24, 3:58 PM

it's an r2 bucket, r2 buckets are offered in a few locations but it's just like s3 in the sense that it's persistent object storage

C

Chaika•4/30/24, 3:59 PM

you have three options for cache
Local Colo Cache <- only one Cache API will ever touch
Tiered Cache <- can vary depending on setup, either Smart (one) or Generic (a few upper tiers), optionally with middle tier (a few in the middle)
Cache Reserve <- R2, last
So Requests could go
Local -> Tiered Cache Middle -> Tiered Cache Upper -> Cache Reserve -> Origin

C

Chaika•4/30/24, 3:59 PM

https://blog.cloudflare.com/introducing-cache-reserve

R

Rich•4/30/24, 4:05 PM

Ok thanks -- I guess rolling our own cache reserve with R2 directly would be a bit clumsy as CF cache purging obv wouldn't work.

I wonder, does CF intend to add support for Cache Reserve and Tired Caching when using the Cache API eventually?

RRich Ok thanks -- I guess rolling our own cache reserve with R2 directly would be a b...

C

Chaika•4/30/24, 4:08 PM

don't think I've heard anything about it, and it's been this way since Cache API came out in 2019. You can use the fetch() api to use tiered caching/cache reserve, but would have to have an endpoint to fetch of course

CChaika don't think I've heard anything about it, and it's been this way since Cache API...

R

Rich•4/30/24, 4:10 PM

yeah we need the Cache API as we're caching POST requests (graphql). It'd be interesting to know why the Cache API can't take advantage of it but fetch can...

RRich yeah we need the Cache API as we're caching POST requests (graphql). It'd be int...

C

Chaika•4/30/24, 4:12 PM

well I imagine some of it is just simple limitations

Tiered Caching Smart requires an IP/geolocation to use, cache api doesn't give it one.
I know Cache Reserve is also part of "Pingora" which their connecting to origin software, so a step outside of the normal stack/where workers run
plus Cache API is completely free

Fetch() goes through the normal outgoing flow

IItsWendell @tony Since this is quite a general question about waitUntil I think it's actual...

T

tonytheonly•4/30/24, 4:13 PM

workers-and-pages-helpexecutionCtx.waitUntil on write service

CChaika well I imagine some of it is just simple limitations Tiered Caching Smart requi...

R

Rich•4/30/24, 4:29 PM

hmm I see. I guess the Cache API could accept an IP as a header -- but yeah I suppose the fundamental difference between fetch and the Cache API is the Request passed to the latter doesn't necessarily point to an actual origin -- could be

new Request("https://this.doesnt.exist.at/all")

new Request("https://this.doesnt.exist.at/all")

new Request("https://this.doesnt.exist.at/all")

new Request("https://this.doesnt.exist.at/all") .

Given our cache key isn't really a

Request

Request

, I wonder if we're better off ditching the Cache API and implementing a more standard key / value cache with R2 or KV

B

bene2591•4/30/24, 7:13 PM

hi, if I store assets into cache using the cache api using a worker and purge the zone cache where this worker has a route, will these files also be purged?

I have max 10mb files but what is the bad usage?

Similar Threads

Similar Threads

Similar Threads