This morning, the problem is no longer happening. So whatever happened got cleaned up by itself.
This morning, the problem is no longer happening. So whatever happened got cleaned up by itself.
PThanks for reporting this, we're working on a way to provide you with your open sessions, so you can reuse them if you want to
Swhat is the best method to maintain a browser session without it auto closing?
A browser instance gets killed if it does not get any command for 60 seconds, freeing one instance.Sahh i got it nvm:
YI'm trying to hack on @Cloudflare/puppeteer locally. I can't get anything to install this is driving me crazy.
YI pull the repo, run
npm install with node 18 and it fails with "Cannot use import statement outside a module", even though, according to the GitHub Actions workflows, that is supposed to work fine.YI tried converting the offending files to
.mts but that only moves the problem somwhere else.YI tried with the upstream puppeteer on the commit that was forked and the issue is the same 
DHmm, if the package is esm-only (which I haven't looked into), could it be that you have to be in a module for it to work? Wild guess, all this cjs vs esm nonsense is driving me crazy too, but could you try adding
"type": "module" to your package.json?
YThe problem is that only the TypeScript files are in ESM. All the rest is in CJS.
YSorry to ping you @celso but I didn't manage to run @Cloudflare/puppeteer locally because of the problem I ran into above.
(Context: I am attempting to reproduce the service that manages chromium instances so that I can use the Browser Rendering API locally with workerd)
(Context: I am attempting to reproduce the service that manages chromium instances so that I can use the Browser Rendering API locally with workerd)
JI'm not sure where that error is coming from, but it may be coming from within your browser context. Buffer isn't defined in the browser's javascript. Maybe the
extractData method is trying to use it?JCan you provide the stacktrace of the error?
JAdd
console.log(e.stack) after the "page on response error"
CI'm not too familiar with puppeter's APIs, but node support with workers is like this:
node_compat=true in your wrangler.toml polyfills some of the nodejs functionality like buffer, ex
nodejs_compat flag would only expose it via
What version of cloudflare/puppeter are you using? The latest should work with nodejs_compat flag, but before 0.0.4 release it needed node_compat
node_compat=true in your wrangler.toml polyfills some of the nodejs functionality like buffer, ex
import { Buffer } from 'buffer' would just worknodejs_compat flag would only expose it via
node:buffer, have to specifically use it, ex import { Buffer } from 'node:buffer'.What version of cloudflare/puppeter are you using? The latest should work with nodejs_compat flag, but before 0.0.4 release it needed node_compat
DHi - Whats the best way to bypass when websites block us from taking a screenshot?
J1) Request the wishes of the website and don't take the screenshot
2) How exactly are they blocking you?
2) How exactly are they blocking you?
DCloudflare antibody firewall
JI'm not sure you are going to get many tips about how to circumvent Cloudflare's Firewall on the Cloudflare Discord... 
YThere's no new code. I just pull the repo and did
npm install. I checked that I had a compatible version of node set, but it still didn't work.JCan someone explain how this works in CF's distributed network? When a browser is started, is it only in a single location? If I wanted browsers in multiple locations, how would I handle that?
PHey jason! Currently it is started on a single location, we'll be adding support for Geolocation, so that for example the content rendered might change according to the location where connection is initiated
JThanks as always @Pedro
J@Pedro Another question... one of the things that excites me most about this project is the hope of scalability. When I've run puppeteer myself on my servers, the challenge that I've always run into it trying to handle scaling it up. Trying to throttle requests to not start too many processes and run the server out of memory, etc. Is one of the goals of this project actually to be able to solve that?
Is this eventually going to work like other Workers do, where I don't need to care about how many requests I throw at it? Or am I going to have to implement all of the throttling code anyways - but now with the limitation that I can't see what's going on inside the server?
Is this eventually going to work like other Workers do, where I don't need to care about how many requests I throw at it? Or am I going to have to implement all of the throttling code anyways - but now with the limitation that I can't see what's going on inside the server?
PGreat question! We'll be providing a simple way to reuse sessions and keeping them open, so you can focus on the task at hand. We'll still be setting default limits, mostly to prevent abuse and malicious behaviour, but those can be raised according to you use case
JThanks. Another way to ask the question. Let's say that I have 1000 urls to crawl. Am I going to need to do the work to manually distribute those requests over multiple sessions, throttle them to not send too many at a time, etc? With a normal worker that doesn't use the browser rendering, I would just POST all 1000 requests as fast as possible to my worker and not worry about it. But I suspect with the browser rendering, I wouldn't be able to do that.
PYou can choose to reuse the same session (until expired) to crawls the URLs sequentially or you can choose to create concurrent sessions. It's basically up to you. We're providing an easier way to reuse the sessions, but the "puppeteer" behaviour is still up to you to define, so it's flexible enough to accommodate most use cases. (You can already reuse the same session for multiple URLs using Durable Objects as per the tutorial, we're making it easier)
JAre you saying that a single session can handle one request at a time? So if I wanted to do 1000 requests in parallel, I would need to start 1000 separate sessions?
PYou can use the same session for multiple tasks like crawling multiple URLs. But if you want to do things in parallel, you just create new concurrent sessions. It's like running a Browser and doing the tasks or Multiple Browsers at the same time and performing multiple task in each one at the same time. Hope I was clear enough 
JYep, I think I understand. 
And I think it means that it's not going to have the worker-level scalability that I hoped it would. At least not without costing a fortune. Thanks for the information!
And I think it means that it's not going to have the worker-level scalability that I hoped it would. At least not without costing a fortune. Thanks for the information!PWould love to learn more about your use case. Please feel free to ping me and let's make sure we have an offer that also suits more specific and demanding use cases such as yours
JDid you post in #Browser Rendering API Beta Access ? It’s usually within 24 hours.
KDoes this API work in dev? I'm not seeing any indication that it shouldn't but when calling a method that works in prod I'm getting
Could not start browser instance. Error: TypeError: Cannot read properties of undefined (reading 'fetch')
Cuhm, does this api work for Pages? 
JYou need to use
wrangler dev --remoteY@sc Hello! Did you have the chance to run the Puppeeteer fork locally? Thank you. 
YThank you!
PHey! You can use DOs, regardless of using one session or multiple concurrent sessions. Basically, using DOs allows you to reuse the same session by keeping it open longer, thus if you don't have the need to crawl or execute commands in parallel, one session should be enough. If you have the need to execute commands in parallel, then you can open new "concurrent" sessions
EYou can request access in browser-rendering-apiBrowser Rendering API Beta Access
EAnd you can see in the chat history that it doesnt take too long
EI don't see a message from you in that thread
