Would love to test out this Browser rendering API. Currently using Google cloud for puppeteer lamb

TTsopic Would love to test out this Browser rendering API. Currently using Google clou...

V

Vero•11/26/23, 1:55 PM

To get access tomorrow, please drop your account ID in the thread: Browser Rendering API Beta Access

B

barretlee•11/30/23, 1:40 AM

Is there a way to quickly get browser rendering access in worker?

B

barretlee•11/30/23, 1:41 AM

Thanks you @Vero

, got it.

D

daodao97.•11/30/23, 8:02 AM

Hi, could I get access for 14438683d1c010b14269b6c993f8c048 Thank you in advance!

Ddaodao97.Hi, could I get access for 14438683d1c010b14269b6c993f8c048 Thank you in advance...

Z

zegevlier•11/30/23, 10:56 AM

Post your account ID in this thread: Browser Rendering API Beta Access

P

Paige•12/4/23, 3:16 PM

It seems like the @\cloudflare/puppeteer npm package is not working entirely as intended as the regular puppeteer does - for example, taking a screenshot of a youtube page, 1st image is using puppeteer on my pc (same result when using a vps I have), and the 2nd image is puppeteer hosted using the browser rendering api on cloudflare using @\cloudflare/puppeteer npm package

For reference, I did use almost exactly the same code, the puppeteer one:

    const browser = await pup.launch({
        headless: `new`,
        defaultViewport: { width: 1920, height: 1080 },
    })

    const getPageSnapshot = (url, fullPage) => new Promise(async (resolve, reject) => { // this returns a promise resolving to a buffer
        const page = await browser.newPage().catch(reject)
        await page.goto(url).catch(reject)

        resolve(await page.screenshot({
            fullPage
        }).then(async (r) => { await page.close(); return r }).catch(reject))
    })

    const browser = await pup.launch({
        headless: `new`,
        defaultViewport: { width: 1920, height: 1080 },
    })

    const getPageSnapshot = (url, fullPage) => new Promise(async (resolve, reject) => { // this returns a promise resolving to a buffer
        const page = await browser.newPage().catch(reject)
        await page.goto(url).catch(reject)

        resolve(await page.screenshot({
            fullPage
        }).then(async (r) => { await page.close(); return r }).catch(reject))
    })

    const browser = await pup.launch({
        headless: `new`,
        defaultViewport: { width: 1920, height: 1080 },
    })

    const getPageSnapshot = (url, fullPage) => new Promise(async (resolve, reject) => { // this returns a promise resolving to a buffer
        const page = await browser.newPage().catch(reject)
        await page.goto(url).catch(reject)

        resolve(await page.screenshot({
            fullPage
        }).then(async (r) => { await page.close(); return r }).catch(reject))
    })

    const browser = await pup.launch({
        headless: `new`,
        defaultViewport: { width: 1920, height: 1080 },
    })

    const getPageSnapshot = (url, fullPage) => new Promise(async (resolve, reject) => { // this returns a promise resolving to a buffer
        const page = await browser.newPage().catch(reject)
        await page.goto(url).catch(reject)

        resolve(await page.screenshot({
            fullPage
        }).then(async (r) => { await page.close(); return r }).catch(reject))
    })

And the cloudflare one:

    async handle(
        request: Request,
        env: any,
        context: any,
        data: Record<string, any>
    ) : Promise<Response> {
                // some url handling
        
        const browser = await puppeteer.launch(env.MYBROWSER);
        const page = await browser.newPage();
        await page.setViewport({ width: 1920, height: 1080 })
        await page.goto(url);
        const screenshot = (await page.screenshot({
            fullPage: false,
        })) as Buffer;
        await browser.close();

        return new Response(screenshot, {
            headers: {
                'Content-Type': `image/jpeg`,
            },
        });
    }

    async handle(
        request: Request,
        env: any,
        context: any,
        data: Record<string, any>
    ) : Promise<Response> {
                // some url handling
        
        const browser = await puppeteer.launch(env.MYBROWSER);
        const page = await browser.newPage();
        await page.setViewport({ width: 1920, height: 1080 })
        await page.goto(url);
        const screenshot = (await page.screenshot({
            fullPage: false,
        })) as Buffer;
        await browser.close();

        return new Response(screenshot, {
            headers: {
                'Content-Type': `image/jpeg`,
            },
        });
    }

    async handle(
        request: Request,
        env: any,
        context: any,
        data: Record<string, any>
    ) : Promise<Response> {
                // some url handling
        
        const browser = await puppeteer.launch(env.MYBROWSER);
        const page = await browser.newPage();
        await page.setViewport({ width: 1920, height: 1080 })
        await page.goto(url);
        const screenshot = (await page.screenshot({
            fullPage: false,
        })) as Buffer;
        await browser.close();

        return new Response(screenshot, {
            headers: {
                'Content-Type': `image/jpeg`,
            },
        });
    }

    async handle(
        request: Request,
        env: any,
        context: any,
        data: Record<string, any>
    ) : Promise<Response> {
                // some url handling
        
        const browser = await puppeteer.launch(env.MYBROWSER);
        const page = await browser.newPage();
        await page.setViewport({ width: 1920, height: 1080 })
        await page.goto(url);
        const screenshot = (await page.screenshot({
            fullPage: false,
        })) as Buffer;
        await browser.close();

        return new Response(screenshot, {
            headers: {
                'Content-Type': `image/jpeg`,
            },
        });
    }

PPaige It seems like the @\cloudflare/puppeteer npm package is not working entirely as ...

U

Unsmart•12/4/23, 3:31 PM

try adding a waitUntil "load" on the page goto

UUnsmart try adding a waitUntil "load" on the page goto

P

Paige•12/4/23, 3:39 PM

using

        await page.goto(url, {
            waitUntil: `load`
        });

        await page.goto(url, {
            waitUntil: `load`
        });

        await page.goto(url, {
            waitUntil: `load`
        });

        await page.goto(url, {
            waitUntil: `load`
        });

gives an identical result

J

Jürgen B•12/5/23, 9:52 AM

I'm getting the following error "Exceeded Resources" in my logs for my scheduled execs, however it seems my script works just great, any ideas?

P

Paige•12/5/23, 7:59 PM

will try

P

Paige•12/6/23, 6:39 AM

In the docs there was a screenshot hinting at some additional pricing per opened browser session, but I cannot find any actual pricing page for this, does it cost me anything extra to open browser instances, and where can I find how much?

D

dj_urgarden•12/6/23, 1:19 PM

Hey hello there, can I please get an access for

291b861f09df80526646856f248a8776

291b861f09df80526646856f248a8776

291b861f09df80526646856f248a8776

291b861f09df80526646856f248a8776

PPaige In the docs there was a screenshot hinting at some additional pricing per opened...

M

Mackenly•12/6/23, 2:59 PM

The pricing is a big problem with this service. It was announced here: https://blog.cloudflare.com/running-serverless-puppeteer-workers-durable-objects/

The Cloudflare Blog

Running Serverless Puppeteer with Workers and Durable Objects

We’ve heard from developers that configuring and maintaining their own serverless browser automation systems can be quite painful. The Workers Browser Rendering API solves this

MMackenly The pricing is a big problem with this service. It was announced here: https://b...

P

Paige•12/6/23, 3:48 PM

So in theory I could just keep 10 concurrent sessions as durable objects and use those together for all of my requests?

P

Paige•12/6/23, 3:48 PM

for just the 5$/month

PPaige So in theory I could just keep 10 concurrent sessions as durable objects and use...

M

Mackenly•12/6/23, 3:52 PM

Yeah in theory. There are reasons they'll get shut down (updates and stuff) but in theory, yes.

MMackenly Yeah in theory. There are reasons they'll get shut down (updates and stuff) but ...

P

Paige•12/6/23, 3:57 PM

do you coincidentally know an estimate of how often that'd typically be?

PPaige do you coincidentally know an estimate of how often that'd typically be?

M

Mackenly•12/6/23, 10:45 PM

I'm too hesitant to use it in prod because of the pricing risk (it's beta still anyways) so don't have any real prod use to compare the price.

P

Paige•12/7/23, 6:39 AM

Yeah I came to the same conclusion

J

Jiamin•12/9/23, 3:14 AM

Hello Cloudflare teams

J

Jiamin•12/9/23, 3:15 AM

Can I have the ability to access the Cloudflare Browser Rendering API? I want to use OpenAI GPTs to call a worker and achieve a cool functionality.

J

Jiamin•12/9/23, 3:18 AM

Can I get access? emma

afe2cf41c6b92b459a3b7e502156ca9b

E

Erisa•12/9/23, 3:26 AM

You'll need this thread browser-rendering-apiBrowser Rendering API Beta Access

EErisa You'll need this thread https://discord.com/channels/595317990191398933/11126750...

J

Jiamin•12/9/23, 3:30 AM

D

Dragon•12/9/23, 8:25 AM

Hi Cloudflare folks, I am the founder of MultiOn AI (https://www.multion.ai/) - building general-purpose web AI agents, we are looking to scale our browser workloads and will love to try out the serverless pupeteer with Cloudflare for our production loads

.

Let me know if some one from the team will be available to get us set up and get us early access! I am at div@multion.ai

MultiOn - Your Personal AI Agent

Your personal AI agent

MMackenly I'm too hesitant to use it in prod because of the pricing risk (it's beta still ...

P

Pedro•12/11/23, 1:59 PM

Hey @Mackenly Jones, we mentioned here previously, but we're working on addressing the pricing concerns by making the session management easier

DDragon Hi Cloudflare folks, I am the founder of MultiOn AI (https://www.multion.ai/) -...

P

Pedro•12/11/23, 2:00 PM

Hey @dragon, Looking forward to learning more. Please let us know if there anything we can help out with

V

Vikash•12/12/23, 8:26 AM

I was testing to capture screenshot for top 5k top domains from CF - https://radar.cloudflare.com/domains to test if we can migrate our Screenshot API from AWS (https://agenty.com/tools/webpage-to-screenshot)

I created a queue to send request to D0 to capture screenshot and upload
But the queue is getting error randomly :
```
Unabled to create new browser:
```
Unabled to create new browser:
```
Unabled to create new browser:
```
Unabled to create new browser:

  {
      "message": [
        "Browser DO: Could not start browser instance. Error: Error: Unabled to create new browser: code: 429: message: Too Many Requests",
        "Error: Unabled to create new browser: code: 429: message: Too Many Requests\n    at PuppeteerWorkers.launch (index.js:19201:13)\n    at async Browser2.fetch (index.js:43967:24)"
      ],
      "level": "error",
      "timestamp": 1702366528655
    }

  {
      "message": [
        "Browser DO: Could not start browser instance. Error: Error: Unabled to create new browser: code: 429: message: Too Many Requests",
        "Error: Unabled to create new browser: code: 429: message: Too Many Requests\n    at PuppeteerWorkers.launch (index.js:19201:13)\n    at async Browser2.fetch (index.js:43967:24)"
      ],
      "level": "error",
      "timestamp": 1702366528655
    }

  {
      "message": [
        "Browser DO: Could not start browser instance. Error: Error: Unabled to create new browser: code: 429: message: Too Many Requests",
        "Error: Unabled to create new browser: code: 429: message: Too Many Requests\n    at PuppeteerWorkers.launch (index.js:19201:13)\n    at async Browser2.fetch (index.js:43967:24)"
      ],
      "level": "error",
      "timestamp": 1702366528655
    }

  {
      "message": [
        "Browser DO: Could not start browser instance. Error: Error: Unabled to create new browser: code: 429: message: Too Many Requests",
        "Error: Unabled to create new browser: code: 429: message: Too Many Requests\n    at PuppeteerWorkers.launch (index.js:19201:13)\n    at async Browser2.fetch (index.js:43967:24)"
      ],
      "level": "error",
      "timestamp": 1702366528655
    }

So it sems the D1 browser reset automatically and the new launch fails to create the browser.

DO Code

  async fetch(url: string) {
    //if there's a browser session open, re-use it
    if (!this.browser || !this.browser.isConnected()) {
      console.log(`Browser DO: Starting new instance`);
      try {
        this.browser = await puppeteer.launch(this.env.CHROMIUM);
      } catch (e) {
        console.error(
          `Browser DO: Could not start browser instance. Error: ${e}`,
          e.stack
        );
        throw e;
      }
    }
   .......
  }

  async fetch(url: string) {
    //if there's a browser session open, re-use it
    if (!this.browser || !this.browser.isConnected()) {
      console.log(`Browser DO: Starting new instance`);
      try {
        this.browser = await puppeteer.launch(this.env.CHROMIUM);
      } catch (e) {
        console.error(
          `Browser DO: Could not start browser instance. Error: ${e}`,
          e.stack
        );
        throw e;
      }
    }
   .......
  }

  async fetch(url: string) {
    //if there's a browser session open, re-use it
    if (!this.browser || !this.browser.isConnected()) {
      console.log(`Browser DO: Starting new instance`);
      try {
        this.browser = await puppeteer.launch(this.env.CHROMIUM);
      } catch (e) {
        console.error(
          `Browser DO: Could not start browser instance. Error: ${e}`,
          e.stack
        );
        throw e;
      }
    }
   .......
  }

  async fetch(url: string) {
    //if there's a browser session open, re-use it
    if (!this.browser || !this.browser.isConnected()) {
      console.log(`Browser DO: Starting new instance`);
      try {
        this.browser = await puppeteer.launch(this.env.CHROMIUM);
      } catch (e) {
        console.error(
          `Browser DO: Could not start browser instance. Error: ${e}`,
          e.stack
        );
        throw e;
      }
    }
   .......
  }

Cloudflare Radar

Up to date Internet trends and insight

Agenty

Agenty - No-Code Web Automation Software

Automate your business with advanced, fully-featured agents on Agenty. Fast, scalable and no-code web automation tool for web scraping, change monitoring, browser automation and more...

Ssc Looks like your browser is being closed under you and so you're hitting rate lim...

V

Vikash•12/12/23, 9:52 AM

You mean here?

 async alarm() {
    this.keptAliveInSeconds += 10;

    // Extend browser DO life
    if (this.keptAliveInSeconds < KEEP_BROWSER_ALIVE_IN_SECONDS) {
      console.log(
        `Browser DO: has been kept alive for ${this.keptAliveInSeconds} seconds. Extending lifespan.`
      );
      await this.storage.setAlarm(Date.now() + 10 * 1000);
      await this.browser.version();
    } else
      console.log(
        `Browser DO: exceeded life of ${KEEP_BROWSER_ALIVE_IN_SECONDS}. Browser DO will be shut down in 10 seconds.`
      );
  }

 async alarm() {
    this.keptAliveInSeconds += 10;

    // Extend browser DO life
    if (this.keptAliveInSeconds < KEEP_BROWSER_ALIVE_IN_SECONDS) {
      console.log(
        `Browser DO: has been kept alive for ${this.keptAliveInSeconds} seconds. Extending lifespan.`
      );
      await this.storage.setAlarm(Date.now() + 10 * 1000);
      await this.browser.version();
    } else
      console.log(
        `Browser DO: exceeded life of ${KEEP_BROWSER_ALIVE_IN_SECONDS}. Browser DO will be shut down in 10 seconds.`
      );
  }

 async alarm() {
    this.keptAliveInSeconds += 10;

    // Extend browser DO life
    if (this.keptAliveInSeconds < KEEP_BROWSER_ALIVE_IN_SECONDS) {
      console.log(
        `Browser DO: has been kept alive for ${this.keptAliveInSeconds} seconds. Extending lifespan.`
      );
      await this.storage.setAlarm(Date.now() + 10 * 1000);
      await this.browser.version();
    } else
      console.log(
        `Browser DO: exceeded life of ${KEEP_BROWSER_ALIVE_IN_SECONDS}. Browser DO will be shut down in 10 seconds.`
      );
  }

 async alarm() {
    this.keptAliveInSeconds += 10;

    // Extend browser DO life
    if (this.keptAliveInSeconds < KEEP_BROWSER_ALIVE_IN_SECONDS) {
      console.log(
        `Browser DO: has been kept alive for ${this.keptAliveInSeconds} seconds. Extending lifespan.`
      );
      await this.storage.setAlarm(Date.now() + 10 * 1000);
      await this.browser.version();
    } else
      console.log(
        `Browser DO: exceeded life of ${KEEP_BROWSER_ALIVE_IN_SECONDS}. Browser DO will be shut down in 10 seconds.`
      );
  }

Original message was deleted

Z

zegevlier•12/14/23, 4:02 PM

Browser Rendering API Beta Access

V

vikiival | koda.art•12/21/23, 12:24 PM

Hello @zegevlier @sc can you please help me with debugging?

V

vikiival | koda.art•12/21/23, 12:24 PM

Seems my original message was deleted

Z

zegevlier•12/21/23, 12:24 PM

?pings

F

Flare•12/21/23, 12:24 PM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

Zzegevlier ?pings

V

vikiival | koda.art•12/21/23, 12:25 PM

Just want to know why my message was deleted

Z

zegevlier•12/21/23, 12:26 PM

I wasn't the one who deleted it, but I see that you advertised giving money for someone to help you. That's not allowed here.

Zzegevlier I wasn't the one who deleted it, but I see that you advertised giving money for ...

V

vikiival | koda.art•12/21/23, 1:02 PM

"motivating someone to help me ;D"

But noted.
Still are there some limitations on the Browser rendering ?

Vvikiival | koda.art "motivating someone to help me ;D" But noted. Still are there some limitations...

P

Pedro•12/22/23, 3:39 PM

Hi! You can check the current limits here: https://developers.cloudflare.com/browser-rendering/platform/limits/ . Would love to learn more about your use-case and your feedback on the current offer.

Limits · Browser Rendering docs

The Workers Browser Rendering API is currently in beta. Add yourself to the waitlist to reserve your spot, and as we expand availability, we will …

K

kian•12/30/23, 10:34 PM

Seems like Node/Undici types instead of workers-types

J

James•12/30/23, 10:42 PM

You might be hitting this: https://github.com/cloudflare/workerd/issues/1298

X

xeon06•12/30/23, 11:07 PM

Oops, didn't realize this was in beta, I wrote all the code already and realize I can't run it

d85a0daa3e2452f1bcc7ba56d59ceb5e

d85a0daa3e2452f1bcc7ba56d59ceb5e

d85a0daa3e2452f1bcc7ba56d59ceb5e

d85a0daa3e2452f1bcc7ba56d59ceb5e in case anyone is feeling generous with the beta access

K

kian•12/30/23, 11:09 PM

Post it in Browser Rendering API Beta Access and it'll likely be seen quicker

X

xeon06•12/30/23, 11:10 PM

Missed that thank you!

PPedro Hi! You can check the current limits here: https://developers.cloudflare.com/bro...

V

vikiival | koda.art•1/2/24, 10:33 AM

I am trying to make this via cloudflare workers. But sadly WebGL does not work in cloudflare

https://dev.to/rainerselvet/serverless-3d-webgl-rendering-with-threejs-4o11

DEV Community

Serverless 3D WebGL rendering with ThreeJS

This image above was rendered in a serverless function on page load (not kidding, check the image...

Vvikiival | koda.art I am trying to make this via cloudflare workers. But sadly WebGL does not work i...

P

Pedro•1/2/24, 11:42 AM

Hey @vikiival | KodaDot , I believe you're right, but since that's not directly related to Browser Rendering, I would encourage you to post your question on the Workers channel in order to get a better answer regarding what you're trying to accomplish with WebGL

J

jason•1/8/24, 7:36 AM

@Pedro You had mentioned that the team is working on updates to Session handling. Can you give us an idea on when those updates might be available? I understand that you probably can't be very specific, but are we thinking weeks? Months? Q1? Q2?. Can you give any sort of estimate?

P

Pedro•1/8/24, 8:30 AM

Hey

Would love to test out this Browser rendering API. Currently using Google cloud for puppeteer lamb

Similar Threads

Similar Threads

Similar Threads