Two new browsers per minute, but what if you keep the browser open indefinitely with a DO?

LLordSilver Two new browsers per minute, but what if you keep the browser open indefinitely ...

Chaika•9/20/24, 3:09 PM

can't do indefinitely as eventually the DO will sleep/browser session will expire but using a DO with a queue/etc is one way around it yea

Chaika•9/20/24, 3:09 PM

https://developers.cloudflare.com/browser-rendering/get-started/browser-rendering-with-do/ there's a guide there for that

Cloudflare Docs

Deploy a Browser Rendering Worker with Durable Objects | Browser Re...

By following this guide, you will create a Worker that uses the Browser Rendering API along with Durable Objects to take screenshots from web pages and store them in R2.

LordSilverOP•9/20/24, 3:09 PM

There's a hard limit to how long DO can hang after a Worker finished?

LordSilverOP•9/20/24, 3:10 PM

The example in the docs uses 60s

LLordSilver There's a hard limit to how long DO can hang after a Worker finished?

Chaika•9/20/24, 3:11 PM

In that example they're using an alarm to keep the DO alive it looks like. But otherwise yea, DOs will be evicted from memory a bit after finishing processing the last request

LordSilverOP•9/20/24, 3:12 PM

So they use a trick. And cannot set the KEEP_BROWSER_ALIVE_IN_SECONDSKEEP_BROWSER_ALIVE_IN_SECONDS to something like 24h?

LordSilverOP•9/20/24, 3:13 PM

It's expensive since you actually pay for the DO, but Cloudflare pays even more for the machine running the browser.

LLordSilver It's expensive since you actually pay for the DO, but Cloudflare pays even more ...

Chaika•9/20/24, 3:13 PM

You would be paying for the constant alarm invocations yea, and even besides that there's a hard limit on the browser session for 10 minutes of inactivity before closure: https://developers.cloudflare.com/browser-rendering/platform/puppeteer/#keep-alive

Cloudflare Docs

Puppeteer | Browser Rendering docs

Puppeteer

is one of the most popular libraries that abstract the lower-level DevTools protocol from developers and provides a high-level API that you can use to easily instrument Chrome/Chromium and automate browsing sessions. Puppeteer is used for tasks like creating screenshots, crawling pages, and testing web applications.

CChaika You would be paying for the constant alarm invocations yea, and even besides th...

LordSilverOP•9/20/24, 3:21 PM

Wait, so you don't actually need the alarm to keep the session open.

LLordSilver Wait, so you don't actually need the alarm to keep the session open.

Chaika•9/20/24, 3:22 PM

If you don't have that, your Durable Object will be evicted and you'll lose your handle to the session

Chaika•9/20/24, 3:23 PM

You should be able to combine that DO Code with the session code: https://developers.cloudflare.com/browser-rendering/get-started/reuse-sessions/ to regrab a session though

CChaika If you don't have that, your Durable Object will be evicted and you'll lose your...

LordSilverOP•9/20/24, 3:24 PM

Ok, so basically the browser could potentially stay instanciated up to 10m, but the next time you connect to it, it would be in a new session, unless you keep a DO linked to it.

LLordSilver Ok, so basically the browser could potentially stay instanciated up to 10m, but ...

Chaika•9/20/24, 3:25 PM

Browser and session are the same thing

Chaika•9/20/24, 3:25 PM

That DO Code was written before the sessions api came

LordSilverOP•9/20/24, 3:26 PM

Then I'm confused.

Chaika•9/20/24, 3:26 PM

it always make a new session/browser on each startup. So if you didn't have the alarm, you'd have an orphaned session

Chaika•9/20/24, 3:26 PM

If users omit the browser.close() statement, it will stay open, ready to be connected to again and re-used but it will, by default, close automatically after 1 minute of inactivity. Users can optionally extend this idle time up to 10 minutes, by using the keep_alive option, set in milliseconds:

LordSilverOP•9/20/24, 3:26 PM

If I use this code, but I don't have any DO, what happens?

Chaika•9/20/24, 3:26 PM

You get a new session

LordSilverOP•9/20/24, 3:26 PM

Yeah, that's what I said.

LordSilverOP•9/20/24, 3:26 PM

So the browser in the puppeteer machine stays open, but you get a new session.

LLordSilver So the browser in the puppeteer machine stays open, but you get a new session.

Chaika•9/20/24, 3:27 PM

no, the browser and the session are the same thing lol

Chaika•9/20/24, 3:27 PM

It'd be a brand new browser/session/instance of remote puppeteer

Chaika•9/20/24, 3:27 PM

if you do launch without a session id every time you'd be running up against that 2 created per minute and not reusing them so more inefficient

Chaika•9/20/24, 3:27 PM

You can see this code here: https://developers.cloudflare.com/browser-rendering/get-started/reuse-sessions/ on how to reuse sessions

Cloudflare Docs

Reuse sessions | Browser Rendering docs

The best way to improve the performance of your browser rendering worker is to reuse sessions. One way to do that is via Durable Objects, which allows you to keep a long running connection from a worker to a browser. Another way is to keep the browser open after you’ve finished with it, and connect to that session each time you have a new reques...

Chaika•9/20/24, 3:28 PM

namely,

        browser = await puppeteer.connect(env.MYBROWSER, sessionId);

        browser = await puppeteer.connect(env.MYBROWSER, sessionId);

instead of pupeeteer.launch`

LordSilverOP•9/20/24, 3:28 PM

So what's the point of this code without a DO? workers-and-pages-discussions

LLordSilver So what's the point of this code without a DO? https://discord.com/channels/5953...

Chaika•9/20/24, 3:28 PM

you get a session/browser instance/puppeteer instance you can use

LordSilverOP•9/20/24, 3:28 PM

But you would get it even without the keepalive

Chaika•9/20/24, 3:29 PM

the keepalive is useless in that case because you'd never be reusing the session

Chaika•9/20/24, 3:29 PM

If you browser.close()'d, it'd go away. If you just disconnected or died, you'd just have an orphaned session

LordSilverOP•9/20/24, 3:29 PM

So you are saying the puppeteer machine would kill the browser even if you specifically told it to keep the browser open for 10m?

LLordSilver So you are saying the puppeteer machine would kill the browser even if you speci...

Chaika•9/20/24, 3:30 PM

no but it'd be useless to you lol

LordSilverOP•9/20/24, 3:30 PM

Yeah I know, that's what I was saying.

LLordSilver If I use this code, but I don't have any DO, what happens?

Chaika•9/20/24, 3:30 PM

if you run this over and over you'd just keep getting new

LordSilverOP•9/20/24, 3:30 PM

So the browser keeps running for 10m, but you have no way to reconnect to the same session.

LordSilverOP•9/20/24, 3:30 PM

It's more like wasted resources for Cloudflare.

LLordSilver So the browser keeps running for 10m, but you have no way to reconnect to the sa...

Chaika•9/20/24, 3:30 PM

You do have a way to reconnect, you're not just using it in that example

Chaika•9/20/24, 3:30 PM

see https://developers.cloudflare.com/browser-rendering/get-started/reuse-sessions/ for how to reuse sessions

Cloudflare Docs

Reuse sessions | Browser Rendering docs

LordSilverOP•9/20/24, 3:31 PM

Yeah I mean with a DO.

Chaika•9/20/24, 3:31 PM

You don't need a DO to reuse sessions although one session is tied to one execution so limited in usage vs a DO can just keep processing/queue

Chaika•9/20/24, 3:32 PM

side note: I agree that all that manual management is silly and in a serverless platform CF should really just do all that for you and let you just use the api without worrying about sessions or instances. We've brought that up to them before, and the session APIs were a bit of their response to it, still annoying you have to manually manage
Using the DO to reuse your session and the sessions API are two entirely separate ways of extending the life of a session/browser instance to reduce cold starts/limits

LordSilverOP•9/20/24, 5:10 PM

What sessions API?

LordSilverOP•9/20/24, 5:12 PM

But yeah, this looks way more complex than just being able to run puppeteer directly on your worker. If you are making high-level abstraction, at least make them right so that there's a good DX that truly hides stuff happening behind the scenes.

Patrick M•9/20/24, 5:14 PM

With

nodejs_compat_v2

nodejs_compat_v2

you can use

nodemailer

nodemailer

now! SMTP transport does not work but you can import the library, build an email payload, and probably use other transports like to Amazon SES too. Example | Source

LLordSilver What sessions API?

Chaika•9/20/24, 5:45 PM

You can do

puppeteer.sessions(env.YourBrowserBinding)

puppeteer.sessions(env.YourBrowserBinding)

like in this example here https://developers.cloudflare.com/browser-rendering/get-started/reuse-sessions/

Cloudflare Docs

Reuse sessions | Browser Rendering docs

Chaika•9/20/24, 5:46 PM

then you can filter out active sessions, grab the session id, and .connect rather then .launch to reuse it

LordSilverOP•9/20/24, 6:10 PM

Why it has to be so complicated.

Dean•9/20/24, 9:56 PM

anyone else notice you can't Deploy through CF workers via the dashboard? If you modify any code, the Deploy button remains disabled.

orangeboi•9/21/24, 1:49 AM

i'm getting this error all of a sudden in my deploys and I can't explain it.

I can run npm run dev, npm run build, npm run serve no problem

Two new browsers per minute, but what if you keep the browser open indefinitely with a DO?

Similar Threads

Similar Threads

Similar Threads