Advised memory is 1600MB, but a Cloudflare Worker has a limit of 128MB. In my tests with Lambda, the

Advised memory is 1600MB, but a Cloudflare Worker has a limit of 128MB. In my tests with Lambda, the average memory usage is 500-900MB, so how is it even possible that Puppeteer runs fine in Worker?

dexter•9/20/24, 2:32 PM

Is there a way (maybe with wrangler.toml) to set a different subdomain.workers.dev than the Worker name? For example use the name "foo-worker" but with foo.workers.devfoo.workers.dev as the domain?

Ddexter Is there a way (maybe with wrangler.toml) to set a different subdomain.workers.d...

Chaika•9/20/24, 2:34 PM

Don't think so, if you want any customization like that, Custom Domain is the answer

LLordSilver Advised memory is 1600MB, but a Cloudflare Worker has a limit of 128MB. In my te...

Chaika•9/20/24, 2:35 PM

because it's not running in the worker, your worker merely connects and proxies commands/etc to a puppeteer instance running externally on a cf machine. iirc they all still run in ORD unless that changed

LLordSilver There's no such thing in AWS, I just run Playwright inside a Lambda function and...

Chaika•9/20/24, 2:37 PM

I think you've mostly figured it out now, but yea Playwright indeed would be way too big in Worker's limits, Browser Rendering is Puppeteer but running on CF separate from your worker instance and using a modified puppeteer lib to make it all work, it only supports some stuff though. There is a channel for it #browser-rendering-api
Some more details in blog post https://blog.cloudflare.com/browser-rendering-open-beta/

CChaika because it's not running in the worker, your worker merely connects and proxies ...

LordSilverOP•9/20/24, 2:54 PM

What does ORD mean?

LordSilverOP•9/20/24, 2:54 PM

Are yo usaying that when using the puppeteer module specifically, Cloudflare would run the code in a separate machine from your worker?

LLordSilver What does ORD mean?

Hard@Work•9/20/24, 2:55 PM

https://where.durableobjects.live/colo/ORD

Where Durable Objects Live

Chicago, Illinois, United States (colo)

Tracking where Durable Objects are created, wherever you are in the world.

HHard@Work https://where.durableobjects.live/colo/ORD

LordSilverOP•9/20/24, 2:55 PM

The first example doesn't use DO.

LordSilverOP•9/20/24, 2:55 PM

They aren't mandatory to use this feature.

LLordSilver The first example doesn't use DO.

Hard@Work•9/20/24, 2:56 PM

I was referring to the fact that ORD means Chicago

Hard@Work•9/20/24, 2:56 PM

Not that it uses DOs

LordSilverOP•9/20/24, 2:57 PM

What if you use playwright instead of puppeteer? Then the worker is unable to understand that it has to run the code in the separate machine I guess?

LordSilverOP•9/20/24, 2:57 PM

Ah nvm, you cannot.

LordSilverOP•9/20/24, 2:58 PM

I've just noticed the lib isn't puppeteerpuppeteer but rather @cloudflare/puppeteer@cloudflare/puppeteer so a custom version.

LordSilverOP•9/20/24, 2:59 PM

Now I understand what's this "Browser Rendering API". Basically proxying all requests of the puppeteer agent to a different beefier machine.

CChaika I think you've mostly figured it out now, but yea Playwright indeed would be way...

LordSilverOP•9/20/24, 3:03 PM

Well, if you compare puppeteer-core with playwright-core, it's less than 1MB difference.

LLordSilver Well, if you compare [puppeteer-core](https://www.npmjs.com/package/puppeteer-co...

Chaika•9/20/24, 3:04 PM

neither of those include the core browser

CChaika neither of those include the core browser

LordSilverOP•9/20/24, 3:04 PM

Yeah, you import the browser, there's optimized one just for serverless.

LordSilverOP•9/20/24, 3:04 PM

And it shouldn't matter. The size of the browser would be the same.

LLordSilver And it shouldn't matter. The size of the browser would be the same.

Chaika•9/20/24, 3:05 PM

Workers size limits are 1 MB free 10 MB paid, is what I was referring to there

Chaika•9/20/24, 3:05 PM

trying to run either inside of the Worker itself wouldn't work

LordSilverOP•9/20/24, 3:05 PM

Yeah I know, Cloudflare would need to support it with their own custom module.

LordSilverOP•9/20/24, 3:06 PM

So you don't pay for any resource used by this separate machine that actually runs puppeteer?

LLordSilver So you don't pay for any resource used by this separate machine that actually ru...

Chaika•9/20/24, 3:07 PM

You will eventually, right now it's just

Browser Rendering is now available to all customers with a paid Workers plan. Each account is limited to running two new browsers per minute and two concurrent browsers at no cost during this period.

LordSilverOP•9/20/24, 3:07 PM

Ah, seemed too good to be true.

CChaika You will eventually, right now it's just > Browser Rendering is now available to...

LordSilverOP•9/20/24, 3:08 PM

Two new browsers per minute, but what if you keep the browser open indefinitely with a DO?

LLordSilver Two new browsers per minute, but what if you keep the browser open indefinitely ...

Chaika•9/20/24, 3:09 PM

can't do indefinitely as eventually the DO will sleep/browser session will expire but using a DO with a queue/etc is one way around it yea

Chaika•9/20/24, 3:09 PM

https://developers.cloudflare.com/browser-rendering/get-started/browser-rendering-with-do/ there's a guide there for that

Cloudflare Docs

Deploy a Browser Rendering Worker with Durable Objects | Browser Re...

By following this guide, you will create a Worker that uses the Browser Rendering API along with Durable Objects to take screenshots from web pages and store them in R2.

LordSilverOP•9/20/24, 3:09 PM

There's a hard limit to how long DO can hang after a Worker finished?

LordSilverOP•9/20/24, 3:10 PM

The example in the docs uses 60s

LLordSilver There's a hard limit to how long DO can hang after a Worker finished?

Chaika•9/20/24, 3:11 PM

In that example they're using an alarm to keep the DO alive it looks like. But otherwise yea, DOs will be evicted from memory a bit after finishing processing the last request

LordSilverOP•9/20/24, 3:12 PM

So they use a trick. And cannot set the KEEP_BROWSER_ALIVE_IN_SECONDSKEEP_BROWSER_ALIVE_IN_SECONDS to something like 24h?

LordSilverOP•9/20/24, 3:13 PM

It's expensive since you actually pay for the DO, but Cloudflare pays even more for the machine running the browser.

LLordSilver It's expensive since you actually pay for the DO, but Cloudflare pays even more ...

Chaika•9/20/24, 3:13 PM

You would be paying for the constant alarm invocations yea, and even besides that there's a hard limit on the browser session for 10 minutes of inactivity before closure: https://developers.cloudflare.com/browser-rendering/platform/puppeteer/#keep-alive

Cloudflare Docs

Puppeteer | Browser Rendering docs

Puppeteer

is one of the most popular libraries that abstract the lower-level DevTools protocol from developers and provides a high-level API that you can use to easily instrument Chrome/Chromium and automate browsing sessions. Puppeteer is used for tasks like creating screenshots, crawling pages, and testing web applications.

CChaika You would be paying for the constant alarm invocations yea, and even besides th...

LordSilverOP•9/20/24, 3:21 PM

Wait, so you don't actually need the alarm to keep the session open.

LLordSilver Wait, so you don't actually need the alarm to keep the session open.

Chaika•9/20/24, 3:22 PM

If you don't have that, your Durable Object will be evicted and you'll lose your handle to the session

Chaika•9/20/24, 3:23 PM

You should be able to combine that DO Code with the session code: https://developers.cloudflare.com/browser-rendering/get-started/reuse-sessions/ to regrab a session though

CChaika If you don't have that, your Durable Object will be evicted and you'll lose your...

LordSilverOP•9/20/24, 3:24 PM

Ok, so basically the browser could potentially stay instanciated up to 10m, but the next time you connect to it, it would be in a new session, unless you keep a DO linked to it.

LLordSilver Ok, so basically the browser could potentially stay instanciated up to 10m, but ...

Chaika•9/20/24, 3:25 PM

Browser and session are the same thing

Chaika•9/20/24, 3:25 PM

That DO Code was written before the sessions api came

LordSilverOP•9/20/24, 3:26 PM

Then I'm confused.

Chaika•9/20/24, 3:26 PM

it always make a new session/browser on each startup. So if you didn't have the alarm, you'd have an orphaned session

Chaika•9/20/24, 3:26 PM

If users omit the browser.close() statement, it will stay open, ready to be connected to again and re-used but it will, by default, close automatically after 1 minute of inactivity. Users can optionally extend this idle time up to 10 minutes, by using the keep_alive option, set in milliseconds:

LordSilverOP•9/20/24, 3:26 PM

If I use this code, but I don't have any DO, what happens?

Chaika•9/20/24, 3:26 PM

You get a new session

LordSilverOP•9/20/24, 3:26 PM

Yeah, that's what I said.

LordSilverOP•9/20/24, 3:26 PM

So the browser in the puppeteer machine stays open, but you get a new session.

LLordSilver So the browser in the puppeteer machine stays open, but you get a new session.

Chaika•9/20/24, 3:27 PM

no, the browser and the session are the same thing lol

Chaika•9/20/24, 3:27 PM

It'd be a brand new browser/session/instance of remote puppeteer

Chaika•9/20/24, 3:27 PM

if you do launch without a session id every time you'd be running up against that 2 created per minute and not reusing them so more inefficient

Advised memory is 1600MB, but a Cloudflare Worker has a limit of 128MB. In my tests with Lambda, the

Similar Threads

Similar Threads

Similar Threads