Are there plans to allow more than 2 browser per minute for the rendering API?

M

Mr. TechnicianOP•8/12/24, 2:03 PM

it's an unfortunate limitation that means I can't use it for my project

MMr. Technician Are there plans to allow more than 2 browser per minute for the rendering API?

P

Peps•8/12/24, 2:30 PM

Hm out of curiosity, is there a reason you can't re-use sessions?

P

Peps•8/12/24, 2:31 PM

There's a guide on how to re-use session at https://developers.cloudflare.com/browser-rendering/get-started/reuse-sessions/

Cloudflare Docs

Reuse sessions · Browser Rendering docs

The best way to improve the performance of your browser rendering worker is to reuse sessions. One way to do that is via Durable Objects, which allows …

M

Mr. TechnicianOP•8/12/24, 2:31 PM

hmm I was not aware of that as an option...

M

Mr. TechnicianOP•8/12/24, 2:31 PM

but I would still need to run tasks in serial, no?

M

Mr. TechnicianOP•8/12/24, 2:33 PM

I have site that currently uses Cloudflare pages for the React piece but uses Azure Functions coupled with Browserless in a VM for web scraping

M

Mr. TechnicianOP•8/12/24, 2:52 PM

the issue is I need to spin up a new browser each time a user makes a new request. Reuse is on the table but not if I need many concurrent browsers

P

Peps•8/12/24, 3:21 PM

I need to spin up a new browser each time

why?

PPeps > I need to spin up a new browser each time why?

M

Mr. TechnicianOP•8/12/24, 7:32 PM

I guess I am using the wrong term. What I need is multiple discrete puppeteer sessions if there happen to be multiple concurrent users running different requests

M

Mr. TechnicianOP•8/12/24, 8:08 PM

https://developers.cloudflare.com/browser-rendering/platform/limits/

Cloudflare Docs

Limits · Browser Rendering docs

Control and interact with a headless browser instance programmatically.

M

Mr. TechnicianOP•8/12/24, 8:08 PM

"two concurrent browsers"

M

Mr. TechnicianOP•8/12/24, 8:09 PM

so does browser = session?

M

Mr. TechnicianOP•8/12/24, 8:34 PM

I'm confused as to how this is useful to much of anyone if it's so limited

P

Peps•8/12/24, 9:34 PM

hm you could create multiple browser contexts per browser I believe

P

Peps•8/12/24, 9:34 PM

and then have a context per-user

PPeps hm you could create multiple browser contexts per browser I believe

M

Mr. TechnicianOP•8/12/24, 11:19 PM

I was inferring that possibility from people saying that taking screenshots freezes the entire browser context

M

Mr. TechnicianOP•8/12/24, 11:19 PM

I am not taking screenshots so that should be fine

.

.gwapes•8/13/24, 2:20 PM

Hi, im following this with ts https://developers.cloudflare.com/browser-rendering/get-started/screenshots/, but im getting this error? Idk what's wrong

Argument of type 'Fetcher' is not assignable to parameter of type 'BrowserWorker'.
  Types of property 'fetch' are incompatible.
    Type '(input: RequestInfo<unknown, CfProperties<unknown>>, init?: RequestInit<CfProperties<unknown>> | undefined) => Promise<Response>' is not assignable to type '{ (input: RequestInfo | URL, init?: RequestInit | undefined): Promise<Response>; (input: RequestInfo, init?: RequestInit | undefined): Promise<...>; }'.

//env 
import { Fetcher } from "@cloudflare/workers-types";

export interface env {
    MYBROWSER: Fetcher
}

Argument of type 'Fetcher' is not assignable to parameter of type 'BrowserWorker'.
  Types of property 'fetch' are incompatible.
    Type '(input: RequestInfo<unknown, CfProperties<unknown>>, init?: RequestInit<CfProperties<unknown>> | undefined) => Promise<Response>' is not assignable to type '{ (input: RequestInfo | URL, init?: RequestInit | undefined): Promise<Response>; (input: RequestInfo, init?: RequestInit | undefined): Promise<...>; }'.

//env 
import { Fetcher } from "@cloudflare/workers-types";

export interface env {
    MYBROWSER: Fetcher
}

Argument of type 'Fetcher' is not assignable to parameter of type 'BrowserWorker'.
  Types of property 'fetch' are incompatible.
    Type '(input: RequestInfo<unknown, CfProperties<unknown>>, init?: RequestInit<CfProperties<unknown>> | undefined) => Promise<Response>' is not assignable to type '{ (input: RequestInfo | URL, init?: RequestInit | undefined): Promise<Response>; (input: RequestInfo, init?: RequestInit | undefined): Promise<...>; }'.

//env 
import { Fetcher } from "@cloudflare/workers-types";

export interface env {
    MYBROWSER: Fetcher
}

Argument of type 'Fetcher' is not assignable to parameter of type 'BrowserWorker'.
  Types of property 'fetch' are incompatible.
    Type '(input: RequestInfo<unknown, CfProperties<unknown>>, init?: RequestInit<CfProperties<unknown>> | undefined) => Promise<Response>' is not assignable to type '{ (input: RequestInfo | URL, init?: RequestInit | undefined): Promise<Response>; (input: RequestInfo, init?: RequestInit | undefined): Promise<...>; }'.

//env 
import { Fetcher } from "@cloudflare/workers-types";

export interface env {
    MYBROWSER: Fetcher
}

Sscythe Could you be hitting this https://github.com/cloudflare/workerd/issues/1298 ? Ot...

.

.gwapes•8/15/24, 9:23 PM

Tbh I think the docs are wrong. https://developers.cloudflare.com/browser-rendering/get-started/screenshots/ the typescript code here says to type MYBROWSER as Fetcher, but await puppeteer.launch takes BrowserWorker

Cloudflare Docs

Deploy a Browser Rendering Worker | Browser Rendering docs

By following this guide, you will create a Worker that uses the Browser Rendering API to take screenshots from web pages. This is a common use case for browser automation.

T

tony.l•8/19/24, 4:22 AM

Hi, I'm getting

failed: Code generation from strings disallowed for this context

failed: Code generation from strings disallowed for this context

failed: Code generation from strings disallowed for this context

failed: Code generation from strings disallowed for this context whenever an xpath selector is used on @Cloudflare/puppeteer. I understand it's a security measure to restrict the use of

createFunction

createFunction

createFunction

createFunction in the worker, but it's necessary for xpaths to work in puppeteer. does anyone have a clue how to work around this?

call site-

page.waitForXPath(xPathSelector, { visible: true })

page.waitForXPath(xPathSelector, { visible: true })

page.waitForXPath(xPathSelector, { visible: true })

page.waitForXPath(xPathSelector, { visible: true })
extracted stacktrace-

  stack: EvalError: Code generation from strings disallowed for this context
    at new Function (<anonymous>)
    at createFunction (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93195:9)
    at interpolateFunction (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93214:10)
    at get _querySelector (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93288:33)
    at XPathQueryHandler.waitFor (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93374:34)
    at Frame2.waitForSelector (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:102047:32)
    at Frame2.waitForXPath (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:102075:17)
    at CDPPage.waitForXPath (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:100304:29)

  stack: EvalError: Code generation from strings disallowed for this context
    at new Function (<anonymous>)
    at createFunction (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93195:9)
    at interpolateFunction (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93214:10)
    at get _querySelector (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93288:33)
    at XPathQueryHandler.waitFor (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93374:34)
    at Frame2.waitForSelector (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:102047:32)
    at Frame2.waitForXPath (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:102075:17)
    at CDPPage.waitForXPath (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:100304:29)

  stack: EvalError: Code generation from strings disallowed for this context
    at new Function (<anonymous>)
    at createFunction (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93195:9)
    at interpolateFunction (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93214:10)
    at get _querySelector (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93288:33)
    at XPathQueryHandler.waitFor (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93374:34)
    at Frame2.waitForSelector (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:102047:32)
    at Frame2.waitForXPath (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:102075:17)
    at CDPPage.waitForXPath (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:100304:29)

  stack: EvalError: Code generation from strings disallowed for this context
    at new Function (<anonymous>)
    at createFunction (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93195:9)
    at interpolateFunction (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93214:10)
    at get _querySelector (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93288:33)
    at XPathQueryHandler.waitFor (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:93374:34)
    at Frame2.waitForSelector (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:102047:32)
    at Frame2.waitForXPath (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:102075:17)
    at CDPPage.waitForXPath (file://<app-root>/.wrangler/tmp/dev-Ol4QPo/index.js:100304:29)

J

Jorge•8/22/24, 7:15 AM

Hii, does anyone know where can I find the pricing for the browser rendering?

C

ceddy•8/23/24, 3:02 AM

@scythe i was trying to buy the workers paid plan but i keep getting 500s

Cceddy @scythe i was trying to buy the workers paid plan but i keep getting 500s

I

Isaac McFadyen•8/23/24, 3:05 AM

?pings

F

Flare•8/23/24, 3:05 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

N

NoGameNoFun•8/27/24, 9:46 AM

Hi, what is the ip rotation logic in browser rendering api? is there any rotation or configure setting to it?

M

Mr. TechnicianOP•9/1/24, 1:08 AM

i'm looking for some guidance on with the browser rendering API

M

Mr. TechnicianOP•9/1/24, 1:13 AM

I need to perform an arbitrary number of browser rendering requests at once. I was hoping I could reuse sessions so that I don't go over the two session limit

M

Mr. TechnicianOP•9/1/24, 1:13 AM

for example, by using multiple tabs in the same session

M

Mr. TechnicianOP•9/1/24, 1:15 AM

Right now I just copied the example here https://developers.cloudflare.com/browser-rendering/get-started/reuse-sessions/
and tried running it on more than two tabs. This doesn't work unfortunately as I can't use the same session in more than one worker

Cloudflare Docs

Reuse sessions | Browser Rendering docs

The best way to improve the performance of your browser rendering worker is to reuse sessions. One way to do that is via Durable Objects, which allows you to keep a long running connection from a worker to a browser. Another way is to keep the browser open after you’ve finished with it, and connect to that session each time you have a new reques...

M

Mr. TechnicianOP•9/1/24, 1:16 AM

Maybe the durable objects example would work instead?

M

Mr. TechnicianOP•9/1/24, 1:50 AM

so i'm looking at the durable object example and adapted it for my own test. it seems like only one worker instance is allowed to use the browser session the DO is storing at once

M

Mr. TechnicianOP•9/2/24, 11:18 PM

can anyone shed some light on this?

M

Mr. TechnicianOP•9/2/24, 11:19 PM

TLDR: I need multiple worker instances (possibly in the same isolate, possibly not) to access the same browser rendering session

M

Mr. TechnicianOP•9/2/24, 11:19 PM

this is merely due to the 2 browser limit

MMr. Technician this is merely due to the 2 browser limit

J

Julio Ribeiro•9/3/24, 12:26 AM

I need to perform an arbitrary number of browser rendering requests at once.

Maybe you can use a queue? Your queue producer will send the rendering requests and the consumer will make these requests as time permits (2 sessions per minute)...
There is an example using queues here: https://developers.cloudflare.com/queues/tutorials/web-crawler-with-browser-rendering/

M

Mr. TechnicianOP•9/3/24, 12:27 AM

the problem with that approach is the user would need to wait longer (assuming others are ahead of them)

M

Mr. TechnicianOP•9/3/24, 12:32 AM

cloudflare has to understand that this limitation is a big issue

M

Mr. TechnicianOP•9/3/24, 12:32 AM

granted it’s in beta

M

Mr. TechnicianOP•9/3/24, 4:38 AM

i think i will abandon this for now and stick with browserless as my remote browser provider

Ssc Setting the geographical location of the browser is in the roadmap. With DO hint...

M

maguayo•9/3/24, 10:24 AM

Any news on the Browser geo location? The websites are different in france or USA. We need to control where the browser spawns.

M

Mr. TechnicianOP•9/3/24, 6:46 PM

I imagine you could also use a third-party proxy via puppeteer as well

M

Mr. TechnicianOP•9/3/24, 6:48 PM

that is my intended use-case anyway

C

ceddy•9/6/24, 10:52 PM

How do you get logs for the browser rendering? like each session?

C

ceddy•9/6/24, 10:53 PM

@Walshy

C

ceddy•9/6/24, 10:55 PM

https://tenor.com/view/calculating-puzzled-math-confused-confused-look-gif-14677181

Tenor

C

ceddy•9/6/24, 10:55 PM

omg i think queue's and browsers logs cannot be stored

Cceddy @Walshy

I

Isaac McFadyen•9/7/24, 3:17 AM

?pings

Are there plans to allow more than 2 browser per minute for the rendering API?

Similar Threads

Similar Threads

Similar Threads