because it's not running in the worker, your worker merely connects and proxies commands/etc to a pu

because it's not running in the worker, your worker merely connects and proxies commands/etc to a puppeteer instance running externally on a cf machine. iirc they all still run in ORD unless that changed

LLordSilver There's no such thing in AWS, I just run Playwright inside a Lambda function and...

ChaikaOP•9/20/24, 2:37 PM

I think you've mostly figured it out now, but yea Playwright indeed would be way too big in Worker's limits, Browser Rendering is Puppeteer but running on CF separate from your worker instance and using a modified puppeteer lib to make it all work, it only supports some stuff though. There is a channel for it #browser-rendering-api
Some more details in blog post https://blog.cloudflare.com/browser-rendering-open-beta/

CChaika because it's not running in the worker, your worker merely connects and proxies ...

LordSilver•9/20/24, 2:54 PM

What does ORD mean?

LordSilver•9/20/24, 2:54 PM

Are yo usaying that when using the puppeteer module specifically, Cloudflare would run the code in a separate machine from your worker?

LLordSilver What does ORD mean?

Hard@Work•9/20/24, 2:55 PM

https://where.durableobjects.live/colo/ORD

Where Durable Objects Live

Chicago, Illinois, United States (colo)

Tracking where Durable Objects are created, wherever you are in the world.

HHard@Work https://where.durableobjects.live/colo/ORD

LordSilver•9/20/24, 2:55 PM

The first example doesn't use DO.

LordSilver•9/20/24, 2:55 PM

They aren't mandatory to use this feature.

LLordSilver The first example doesn't use DO.

Hard@Work•9/20/24, 2:56 PM

I was referring to the fact that ORD means Chicago

Hard@Work•9/20/24, 2:56 PM

Not that it uses DOs

LordSilver•9/20/24, 2:57 PM

What if you use playwright instead of puppeteer? Then the worker is unable to understand that it has to run the code in the separate machine I guess?

LordSilver•9/20/24, 2:57 PM

Ah nvm, you cannot.

LordSilver•9/20/24, 2:58 PM

I've just noticed the lib isn't

puppeteer

puppeteer

but rather

@cloudflare/puppeteer

@cloudflare/puppeteer

so a custom version.

LordSilver•9/20/24, 2:59 PM

Now I understand what's this "Browser Rendering API". Basically proxying all requests of the puppeteer agent to a different beefier machine.

CChaika I think you've mostly figured it out now, but yea Playwright indeed would be way...

LordSilver•9/20/24, 3:03 PM

Well, if you compare puppeteer-core with playwright-core, it's less than 1MB difference.

LLordSilver Well, if you compare [puppeteer-core](https://www.npmjs.com/package/puppeteer-co...

ChaikaOP•9/20/24, 3:04 PM

neither of those include the core browser

CChaika neither of those include the core browser

LordSilver•9/20/24, 3:04 PM

Yeah, you import the browser, there's optimized one just for serverless.

LordSilver•9/20/24, 3:04 PM

And it shouldn't matter. The size of the browser would be the same.

LLordSilver And it shouldn't matter. The size of the browser would be the same.

ChaikaOP•9/20/24, 3:05 PM

Workers size limits are 1 MB free 10 MB paid, is what I was referring to there

ChaikaOP•9/20/24, 3:05 PM

trying to run either inside of the Worker itself wouldn't work

LordSilver•9/20/24, 3:05 PM

Yeah I know, Cloudflare would need to support it with their own custom module.

LordSilver•9/20/24, 3:06 PM

So you don't pay for any resource used by this separate machine that actually runs puppeteer?

LLordSilver So you don't pay for any resource used by this separate machine that actually ru...

ChaikaOP•9/20/24, 3:07 PM

You will eventually, right now it's just

Browser Rendering is now available to all customers with a paid Workers plan. Each account is limited to running two new browsers per minute and two concurrent browsers at no cost during this period.

LordSilver•9/20/24, 3:07 PM

Ah, seemed too good to be true.

CChaika You will eventually, right now it's just > Browser Rendering is now available to...

LordSilver•9/20/24, 3:08 PM

Two new browsers per minute, but what if you keep the browser open indefinitely with a DO?

LLordSilver Two new browsers per minute, but what if you keep the browser open indefinitely ...

ChaikaOP•9/20/24, 3:09 PM

can't do indefinitely as eventually the DO will sleep/browser session will expire but using a DO with a queue/etc is one way around it yea

ChaikaOP•9/20/24, 3:09 PM

https://developers.cloudflare.com/browser-rendering/get-started/browser-rendering-with-do/ there's a guide there for that

Cloudflare Docs

Deploy a Browser Rendering Worker with Durable Objects | Browser Re...

By following this guide, you will create a Worker that uses the Browser Rendering API along with Durable Objects to take screenshots from web pages and store them in R2.

LordSilver•9/20/24, 3:09 PM

There's a hard limit to how long DO can hang after a Worker finished?

LordSilver•9/20/24, 3:10 PM

The example in the docs uses 60s

LLordSilver There's a hard limit to how long DO can hang after a Worker finished?

ChaikaOP•9/20/24, 3:11 PM

In that example they're using an alarm to keep the DO alive it looks like. But otherwise yea, DOs will be evicted from memory a bit after finishing processing the last request

LordSilver•9/20/24, 3:12 PM

So they use a trick. And cannot set the

KEEP_BROWSER_ALIVE_IN_SECONDS

KEEP_BROWSER_ALIVE_IN_SECONDS

to something like 24h?

LordSilver•9/20/24, 3:13 PM

It's expensive since you actually pay for the DO, but Cloudflare pays even more for the machine running the browser.

LLordSilver It's expensive since you actually pay for the DO, but Cloudflare pays even more ...

ChaikaOP•9/20/24, 3:13 PM

You would be paying for the constant alarm invocations yea, and even besides that there's a hard limit on the browser session for 10 minutes of inactivity before closure: https://developers.cloudflare.com/browser-rendering/platform/puppeteer/#keep-alive

Cloudflare Docs

Puppeteer | Browser Rendering docs

Puppeteer

is one of the most popular libraries that abstract the lower-level DevTools protocol from developers and provides a high-level API that you can use to easily instrument Chrome/Chromium and automate browsing sessions. Puppeteer is used for tasks like creating screenshots, crawling pages, and testing web applications.

CChaika You would be paying for the constant alarm invocations yea, and even besides th...

LordSilver•9/20/24, 3:21 PM

Wait, so you don't actually need the alarm to keep the session open.

LLordSilver Wait, so you don't actually need the alarm to keep the session open.

ChaikaOP•9/20/24, 3:22 PM

If you don't have that, your Durable Object will be evicted and you'll lose your handle to the session

ChaikaOP•9/20/24, 3:23 PM

You should be able to combine that DO Code with the session code: https://developers.cloudflare.com/browser-rendering/get-started/reuse-sessions/ to regrab a session though

CChaika If you don't have that, your Durable Object will be evicted and you'll lose your...

LordSilver•9/20/24, 3:24 PM

Ok, so basically the browser could potentially stay instanciated up to 10m, but the next time you connect to it, it would be in a new session, unless you keep a DO linked to it.

LLordSilver Ok, so basically the browser could potentially stay instanciated up to 10m, but ...

ChaikaOP•9/20/24, 3:25 PM

Browser and session are the same thing

ChaikaOP•9/20/24, 3:25 PM

That DO Code was written before the sessions api came

LordSilver•9/20/24, 3:26 PM

Then I'm confused.

ChaikaOP•9/20/24, 3:26 PM

it always make a new session/browser on each startup. So if you didn't have the alarm, you'd have an orphaned session

ChaikaOP•9/20/24, 3:26 PM

If users omit the browser.close() statement, it will stay open, ready to be connected to again and re-used but it will, by default, close automatically after 1 minute of inactivity. Users can optionally extend this idle time up to 10 minutes, by using the keep_alive option, set in milliseconds:

LordSilver•9/20/24, 3:26 PM

If I use this code, but I don't have any DO, what happens?

ChaikaOP•9/20/24, 3:26 PM

You get a new session

LordSilver•9/20/24, 3:26 PM

Yeah, that's what I said.

LordSilver•9/20/24, 3:26 PM

So the browser in the puppeteer machine stays open, but you get a new session.

LLordSilver So the browser in the puppeteer machine stays open, but you get a new session.

ChaikaOP•9/20/24, 3:27 PM

no, the browser and the session are the same thing lol

ChaikaOP•9/20/24, 3:27 PM

It'd be a brand new browser/session/instance of remote puppeteer

ChaikaOP•9/20/24, 3:27 PM

if you do launch without a session id every time you'd be running up against that 2 created per minute and not reusing them so more inefficient

ChaikaOP•9/20/24, 3:27 PM

You can see this code here: https://developers.cloudflare.com/browser-rendering/get-started/reuse-sessions/ on how to reuse sessions

Cloudflare Docs

Reuse sessions | Browser Rendering docs

The best way to improve the performance of your browser rendering worker is to reuse sessions. One way to do that is via Durable Objects, which allows you to keep a long running connection from a worker to a browser. Another way is to keep the browser open after you’ve finished with it, and connect to that session each time you have a new reques...

ChaikaOP•9/20/24, 3:28 PM

namely,

        browser = await puppeteer.connect(env.MYBROWSER, sessionId);

        browser = await puppeteer.connect(env.MYBROWSER, sessionId);

instead of pupeeteer.launch`

LordSilver•9/20/24, 3:28 PM

So what's the point of this code without a DO? workers-and-pages-discussions

because it's not running in the worker, your worker merely connects and proxies commands/etc to a pu

Similar Threads

Similar Threads

Similar Threads