Misunderstood that, so you can keep it alive for months on end and only pay for the durable object?

BBram Misunderstood that, so you can keep it alive for months on end and only pay for ...

M

Mackenly•11/15/23, 6:53 AM

Maybe, or maybe not it seems. It depends if the DO gets reset or runs out of memory. That question is really what we've been asking since the pricing was announced. So, maybe, but probably not in real world production (at least not for months).

UUnsmart Main difference there is you sort of have to manage the load of the durable obje...

B

BramOP•11/15/23, 7:02 AM

Which imo is weird for a cloud product, it should be cost optimised behind the scenes and have competitive per usage pricing imo, if you have a special use case that needs full control over the browser instance lifetimes, they could expose an additional API for lower level control?

BBram Which imo is weird for a cloud product, it should be cost optimised behind the s...

P

Pedro•11/15/23, 11:27 AM

Thank you for your feedback. While our current beta has specific limits and procedures, we're open to evolving them based on feedback and common use cases

B

BramOP•11/15/23, 8:43 PM

Hi, let's say I want to set the html programitcally with

setContent

setContent

and that html runs some javascript that wants to access a large local json file, can puppeteer somehow access the local filestystem of the worker? Or how would this work?

BBram Hi, let's say I want to set the html programitcally with `setContent` and that h...

B

BramOP•11/16/23, 10:30 AM

I ended up just embedding the large json directly into a script tag, not sure if it is the most efficient way, but it works

Ssc You can't access the local file system. You could save the file in R2 (they can ...

B

BramOP•11/16/23, 11:23 AM

yea, in fact the json blob is inside r2 already, but it has to stay private, was thinking about using a signed url to generate temporary access, but was wondering if there was a better way. For now directly embedding it into the html (like so:

<script> const json = JSON.parse(${rawJson};  ...

<script> const json = JSON.parse(${rawJson};  ...

) and then use

page.setContents

page.setContents

works, but not sure if this is a performant way to do it.

J

jo32•11/18/23, 3:24 PM

How do I disable CORS when launching a browser?

GGustavo I think it's clear already the current pricing is at least very confusing and sh...

P

Pedro•11/20/23, 9:43 AM

Hey @gustavopch, we're still in beta and are actively learning from all the use-cases and great feedback we're getting. This is really helpful for us as we fine-tune our pricing to better suit your needs. Thanks for bearing with us!

Eevo4ce Hi Pedro, great to hear you’re receiving great feedback from the community. In...

P

Pedro•11/21/23, 11:00 AM

Hey @evo4ce, sure, as an update I can tell you that we're considering alternative approaches so that the pricing concerns are addressed, namely, keeping sessions open without the need for the users to manage Durable Objects/state

PPedro Hey @evo4ce, sure, as an update I can tell you that we're considering alternativ...

V

Vikash•11/21/23, 12:02 PM

At-least add some reports so that we can calculate how many sessions, requests we are consuming in Beta for better planning the and costs.

Ssc fyi we updated the code in the Durable Object example https://developers.cloudfl...

V

Vikash•11/21/23, 4:46 PM

Still the page is not closed/re-used.

So we will end up creating too many stalled pages after few requests.

Add ‘await page.close()’

Ssc fyi we updated the code in the Durable Object example https://developers.cloudfl...

T

Tsopic•11/25/23, 8:37 AM

Would love to test out this Browser rendering API.

Currently using Google cloud for puppeteer lambdas for email preview renders etc, but dis too slow for meh. Joined the beta access waitlist

TTsopic Would love to test out this Browser rendering API. Currently using Google clou...

V

Vero•11/26/23, 1:55 PM

To get access tomorrow, please drop your account ID in the thread: Browser Rendering API Beta Access

B

barretlee•11/30/23, 1:40 AM

Is there a way to quickly get browser rendering access in worker?

B

barretlee•11/30/23, 1:41 AM

Thanks you @Vero

, got it.

D

daodao97.•11/30/23, 8:02 AM

Hi, could I get access for 14438683d1c010b14269b6c993f8c048 Thank you in advance!

Ddaodao97.Hi, could I get access for 14438683d1c010b14269b6c993f8c048 Thank you in advance...

Z

zegevlier•11/30/23, 10:56 AM

Post your account ID in this thread: Browser Rendering API Beta Access

P

Paige•12/4/23, 3:16 PM

It seems like the @\cloudflare/puppeteer npm package is not working entirely as intended as the regular puppeteer does - for example, taking a screenshot of a youtube page, 1st image is using puppeteer on my pc (same result when using a vps I have), and the 2nd image is puppeteer hosted using the browser rendering api on cloudflare using @\cloudflare/puppeteer npm package

For reference, I did use almost exactly the same code, the puppeteer one:

    const browser = await pup.launch({
        headless: `new`,
        defaultViewport: { width: 1920, height: 1080 },
    })

    const getPageSnapshot = (url, fullPage) => new Promise(async (resolve, reject) => { // this returns a promise resolving to a buffer
        const page = await browser.newPage().catch(reject)
        await page.goto(url).catch(reject)

        resolve(await page.screenshot({
            fullPage
        }).then(async (r) => { await page.close(); return r }).catch(reject))
    })

    const browser = await pup.launch({
        headless: `new`,
        defaultViewport: { width: 1920, height: 1080 },
    })

    const getPageSnapshot = (url, fullPage) => new Promise(async (resolve, reject) => { // this returns a promise resolving to a buffer
        const page = await browser.newPage().catch(reject)
        await page.goto(url).catch(reject)

        resolve(await page.screenshot({
            fullPage
        }).then(async (r) => { await page.close(); return r }).catch(reject))
    })

    const browser = await pup.launch({
        headless: `new`,
        defaultViewport: { width: 1920, height: 1080 },
    })

    const getPageSnapshot = (url, fullPage) => new Promise(async (resolve, reject) => { // this returns a promise resolving to a buffer
        const page = await browser.newPage().catch(reject)
        await page.goto(url).catch(reject)

        resolve(await page.screenshot({
            fullPage
        }).then(async (r) => { await page.close(); return r }).catch(reject))
    })

    const browser = await pup.launch({
        headless: `new`,
        defaultViewport: { width: 1920, height: 1080 },
    })

    const getPageSnapshot = (url, fullPage) => new Promise(async (resolve, reject) => { // this returns a promise resolving to a buffer
        const page = await browser.newPage().catch(reject)
        await page.goto(url).catch(reject)

        resolve(await page.screenshot({
            fullPage
        }).then(async (r) => { await page.close(); return r }).catch(reject))
    })

And the cloudflare one:

    async handle(
        request: Request,
        env: any,
        context: any,
        data: Record<string, any>
    ) : Promise<Response> {
                // some url handling
        
        const browser = await puppeteer.launch(env.MYBROWSER);
        const page = await browser.newPage();
        await page.setViewport({ width: 1920, height: 1080 })
        await page.goto(url);
        const screenshot = (await page.screenshot({
            fullPage: false,
        })) as Buffer;
        await browser.close();

        return new Response(screenshot, {
            headers: {
                'Content-Type': `image/jpeg`,
            },
        });
    }

    async handle(
        request: Request,
        env: any,
        context: any,
        data: Record<string, any>
    ) : Promise<Response> {
                // some url handling
        
        const browser = await puppeteer.launch(env.MYBROWSER);
        const page = await browser.newPage();
        await page.setViewport({ width: 1920, height: 1080 })
        await page.goto(url);
        const screenshot = (await page.screenshot({
            fullPage: false,
        })) as Buffer;
        await browser.close();

        return new Response(screenshot, {
            headers: {
                'Content-Type': `image/jpeg`,
            },
        });
    }

    async handle(
        request: Request,
        env: any,
        context: any,
        data: Record<string, any>
    ) : Promise<Response> {
                // some url handling
        
        const browser = await puppeteer.launch(env.MYBROWSER);
        const page = await browser.newPage();
        await page.setViewport({ width: 1920, height: 1080 })
        await page.goto(url);
        const screenshot = (await page.screenshot({
            fullPage: false,
        })) as Buffer;
        await browser.close();

        return new Response(screenshot, {
            headers: {
                'Content-Type': `image/jpeg`,
            },
        });
    }

    async handle(
        request: Request,
        env: any,
        context: any,
        data: Record<string, any>
    ) : Promise<Response> {
                // some url handling
        
        const browser = await puppeteer.launch(env.MYBROWSER);
        const page = await browser.newPage();
        await page.setViewport({ width: 1920, height: 1080 })
        await page.goto(url);
        const screenshot = (await page.screenshot({
            fullPage: false,
        })) as Buffer;
        await browser.close();

        return new Response(screenshot, {
            headers: {
                'Content-Type': `image/jpeg`,
            },
        });
    }

PPaige It seems like the @\cloudflare/puppeteer npm package is not working entirely as ...

U

Unsmart•12/4/23, 3:31 PM

try adding a waitUntil "load" on the page goto

UUnsmart try adding a waitUntil "load" on the page goto

P

Paige•12/4/23, 3:39 PM

using

        await page.goto(url, {
            waitUntil: `load`
        });

        await page.goto(url, {
            waitUntil: `load`
        });

        await page.goto(url, {
            waitUntil: `load`
        });

        await page.goto(url, {
            waitUntil: `load`
        });

gives an identical result

J

Jürgen B•12/5/23, 9:52 AM

I'm getting the following error "Exceeded Resources" in my logs for my scheduled execs, however it seems my script works just great, any ideas?

P

Paige•12/5/23, 7:59 PM

will try

P

Paige•12/6/23, 6:39 AM

In the docs there was a screenshot hinting at some additional pricing per opened browser session, but I cannot find any actual pricing page for this, does it cost me anything extra to open browser instances, and where can I find how much?

D

dj_urgarden•12/6/23, 1:19 PM

Hey hello there, can I please get an access for

291b861f09df80526646856f248a8776

291b861f09df80526646856f248a8776

PPaige In the docs there was a screenshot hinting at some additional pricing per opened...

M

Mackenly•12/6/23, 2:59 PM

The pricing is a big problem with this service. It was announced here: https://blog.cloudflare.com/running-serverless-puppeteer-workers-durable-objects/

The Cloudflare Blog

Running Serverless Puppeteer with Workers and Durable Objects

We’ve heard from developers that configuring and maintaining their own serverless browser automation systems can be quite painful. The Workers Browser Rendering API solves this

MMackenly The pricing is a big problem with this service. It was announced here: https://b...

P

Paige•12/6/23, 3:48 PM

So in theory I could just keep 10 concurrent sessions as durable objects and use those together for all of my requests?

P

Paige•12/6/23, 3:48 PM

for just the 5$/month

PPaige So in theory I could just keep 10 concurrent sessions as durable objects and use...

M

Mackenly•12/6/23, 3:52 PM

Yeah in theory. There are reasons they'll get shut down (updates and stuff) but in theory, yes.

MMackenly Yeah in theory. There are reasons they'll get shut down (updates and stuff) but ...

P

Paige•12/6/23, 3:57 PM

do you coincidentally know an estimate of how often that'd typically be?

PPaige do you coincidentally know an estimate of how often that'd typically be?

M

Mackenly•12/6/23, 10:45 PM

I'm too hesitant to use it in prod because of the pricing risk (it's beta still anyways) so don't have any real prod use to compare the price.

P

Paige•12/7/23, 6:39 AM

Yeah I came to the same conclusion

J

Jiamin•12/9/23, 3:14 AM

Hello Cloudflare teams

J

Jiamin•12/9/23, 3:15 AM

Can I have the ability to access the Cloudflare Browser Rendering API? I want to use OpenAI GPTs to call a worker and achieve a cool functionality.

J

Jiamin•12/9/23, 3:18 AM

Can I get access? emma

afe2cf41c6b92b459a3b7e502156ca9b

E

Erisa•12/9/23, 3:26 AM

You'll need this thread browser-rendering-apiBrowser Rendering API Beta Access

EErisa You'll need this thread https://discord.com/channels/595317990191398933/11126750...

J

Jiamin•12/9/23, 3:30 AM

D

Dragon•12/9/23, 8:25 AM

Hi Cloudflare folks, I am the founder of MultiOn AI (https://www.multion.ai/) - building general-purpose web AI agents, we are looking to scale our browser workloads and will love to try out the serverless pupeteer with Cloudflare for our production loads

.

Let me know if some one from the team will be available to get us set up and get us early access! I am at div@multion.ai

MultiOn - Your Personal AI Agent

Your personal AI agent

MMackenly I'm too hesitant to use it in prod because of the pricing risk (it's beta still ...

P

Pedro•12/11/23, 1:59 PM

Hey @Mackenly Jones, we mentioned here previously, but we're working on addressing the pricing concerns by making the session management easier

DDragon Hi Cloudflare folks, I am the founder of MultiOn AI (https://www.multion.ai/) -...

P

Pedro•12/11/23, 2:00 PM

Hey @dragon, Looking forward to learning more. Please let us know if there anything we can help out with

V

Vikash•12/12/23, 8:26 AM

I was testing to capture screenshot for top 5k top domains from CF - https://radar.cloudflare.com/domains to test if we can migrate our Screenshot API from AWS (https://agenty.com/tools/webpage-to-screenshot)

I created a queue to send request to D0 to capture screenshot and upload

But the queue is getting error randomly :

Unabled to create new browser:

Unabled to create new browser:

  {
      "message": [
        "Browser DO: Could not start browser instance. Error: Error: Unabled to create new browser: code: 429: message: Too Many Requests",
        "Error: Unabled to create new browser: code: 429: message: Too Many Requests\n    at PuppeteerWorkers.launch (index.js:19201:13)\n    at async Browser2.fetch (index.js:43967:24)"
      ],
      "level": "error",
      "timestamp": 1702366528655
    }

  {
      "message": [
        "Browser DO: Could not start browser instance. Error: Error: Unabled to create new browser: code: 429: message: Too Many Requests",
        "Error: Unabled to create new browser: code: 429: message: Too Many Requests\n    at PuppeteerWorkers.launch (index.js:19201:13)\n    at async Browser2.fetch (index.js:43967:24)"
      ],
      "level": "error",
      "timestamp": 1702366528655
    }

  {
      "message": [
        "Browser DO: Could not start browser instance. Error: Error: Unabled to create new browser: code: 429: message: Too Many Requests",
        "Error: Unabled to create new browser: code: 429: message: Too Many Requests\n    at PuppeteerWorkers.launch (index.js:19201:13)\n    at async Browser2.fetch (index.js:43967:24)"
      ],
      "level": "error",
      "timestamp": 1702366528655
    }

  {
      "message": [
        "Browser DO: Could not start browser instance. Error: Error: Unabled to create new browser: code: 429: message: Too Many Requests",
        "Error: Unabled to create new browser: code: 429: message: Too Many Requests\n    at PuppeteerWorkers.launch (index.js:19201:13)\n    at async Browser2.fetch (index.js:43967:24)"
      ],
      "level": "error",
      "timestamp": 1702366528655
    }

So it sems the D1 browser reset automatically and the new launch fails to create the browser.

DO Code

  async fetch(url: string) {
    //if there's a browser session open, re-use it
    if (!this.browser || !this.browser.isConnected()) {
      console.log(`Browser DO: Starting new instance`);
      try {
        this.browser = await puppeteer.launch(this.env.CHROMIUM);
      } catch (e) {
        console.error(
          `Browser DO: Could not start browser instance. Error: ${e}`,
          e.stack
        );
        throw e;
      }
    }
   .......
  }

  async fetch(url: string) {
    //if there's a browser session open, re-use it
    if (!this.browser || !this.browser.isConnected()) {
      console.log(`Browser DO: Starting new instance`);
      try {
        this.browser = await puppeteer.launch(this.env.CHROMIUM);
      } catch (e) {
        console.error(
          `Browser DO: Could not start browser instance. Error: ${e}`,
          e.stack
        );
        throw e;
      }
    }
   .......
  }

  async fetch(url: string) {
    //if there's a browser session open, re-use it
    if (!this.browser || !this.browser.isConnected()) {
      console.log(`Browser DO: Starting new instance`);
      try {
        this.browser = await puppeteer.launch(this.env.CHROMIUM);
      } catch (e) {
        console.error(
          `Browser DO: Could not start browser instance. Error: ${e}`,
          e.stack
        );
        throw e;
      }
    }
   .......
  }

  async fetch(url: string) {
    //if there's a browser session open, re-use it
    if (!this.browser || !this.browser.isConnected()) {
      console.log(`Browser DO: Starting new instance`);
      try {
        this.browser = await puppeteer.launch(this.env.CHROMIUM);
      } catch (e) {
        console.error(
          `Browser DO: Could not start browser instance. Error: ${e}`,
          e.stack
        );
        throw e;
      }
    }
   .......
  }

Cloudflare Radar

Up to date Internet trends and insight

Agenty

Agenty - No-Code Web Automation Software

Automate your business with advanced, fully-featured agents on Agenty. Fast, scalable and no-code web automation tool for web scraping, change monitoring, browser automation and more...

Ssc Looks like your browser is being closed under you and so you're hitting rate lim...

V

Vikash•12/12/23, 9:52 AM

You mean here?

 async alarm() {
    this.keptAliveInSeconds += 10;

    // Extend browser DO life
    if (this.keptAliveInSeconds < KEEP_BROWSER_ALIVE_IN_SECONDS) {
      console.log(
        `Browser DO: has been kept alive for ${this.keptAliveInSeconds} seconds. Extending lifespan.`
      );
      await this.storage.setAlarm(Date.now() + 10 * 1000);
      await this.browser.version();
    } else
      console.log(
        `Browser DO: exceeded life of ${KEEP_BROWSER_ALIVE_IN_SECONDS}. Browser DO will be shut down in 10 seconds.`
      );
  }

 async alarm() {
    this.keptAliveInSeconds += 10;

    // Extend browser DO life
    if (this.keptAliveInSeconds < KEEP_BROWSER_ALIVE_IN_SECONDS) {
      console.log(
        `Browser DO: has been kept alive for ${this.keptAliveInSeconds} seconds. Extending lifespan.`
      );
      await this.storage.setAlarm(Date.now() + 10 * 1000);
      await this.browser.version();
    } else
      console.log(
        `Browser DO: exceeded life of ${KEEP_BROWSER_ALIVE_IN_SECONDS}. Browser DO will be shut down in 10 seconds.`
      );
  }

 async alarm() {
    this.keptAliveInSeconds += 10;

    // Extend browser DO life
    if (this.keptAliveInSeconds < KEEP_BROWSER_ALIVE_IN_SECONDS) {
      console.log(
        `Browser DO: has been kept alive for ${this.keptAliveInSeconds} seconds. Extending lifespan.`
      );
      await this.storage.setAlarm(Date.now() + 10 * 1000);
      await this.browser.version();
    } else
      console.log(
        `Browser DO: exceeded life of ${KEEP_BROWSER_ALIVE_IN_SECONDS}. Browser DO will be shut down in 10 seconds.`
      );
  }

 async alarm() {
    this.keptAliveInSeconds += 10;

    // Extend browser DO life
    if (this.keptAliveInSeconds < KEEP_BROWSER_ALIVE_IN_SECONDS) {
      console.log(
        `Browser DO: has been kept alive for ${this.keptAliveInSeconds} seconds. Extending lifespan.`
      );
      await this.storage.setAlarm(Date.now() + 10 * 1000);
      await this.browser.version();
    } else
      console.log(
        `Browser DO: exceeded life of ${KEEP_BROWSER_ALIVE_IN_SECONDS}. Browser DO will be shut down in 10 seconds.`
      );
  }

Original message was deleted

Z

zegevlier•12/14/23, 4:02 PM

Browser Rendering API Beta Access

V

vikiival | koda.art•12/21/23, 12:24 PM

Hello @zegevlier @sc can you please help me with debugging?

Misunderstood that, so you can keep it alive for months on end and only pay for the durable object?

Similar Threads

Similar Threads

Similar Threads