@Walshy | Deploying - Cloudflare Developers

AAmirkhan @Walshy | Deploying

I

Isaac McFadyen•5/13/24, 1:01 PM

?pings

F

Flare•5/13/24, 1:01 PM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

Original message was deleted

A

Advany•5/15/24, 10:36 AM

compatibility_date = "2023-03-14"
node_compat = true

thats the fix

WWalshy Yeah you need Workers Paid for browser bindings

M

Maltmax•5/16/24, 7:38 AM

Hi! I'm also new to the browser rendering API (and new to workers too... excited to be here!) If I upgrade to a paid workers account should the browser rendering work with the tutorial provided?

A

akazwz•5/21/24, 10:53 AM

@sc Hello, I found that the type in the code is not quite right and I have submitted a pull request, could you review it? https://github.com/cloudflare/puppeteer/pull/44

GitHub

fix: update activeSessions type by akazwz · Pull Request #44 · clou...

What kind of change does this PR introduce?
fix type

Did you add tests for your changes?
no need
If relevant, did you update the documentation?
no need
Summary
fix type

Does this PR introduce a ...

C

Conner•5/22/24, 6:00 PM

Does cloudflare browser rendering support aspx pages? I'm getting a

__doPostBack is not defined

__doPostBack is not defined

error when running puppeteer in browser rendering. I tried the same code in local puppeteer on node, and it works just fine. The page loads fine on both, a screenshot of the page looks the same on both. I also used puppeteer verison 17 locally, since that's the version that @cloudflare/puppeteer is forked from. Still not working. Any thoughts?

C

Conner•5/22/24, 6:16 PM

Would appreciate any help here. Tried a bunch of different methods in cloudflare puppeteer before realizing it works perfectly fine in a local puppeteer.

CConner Does cloudflare browser rendering support aspx pages? I'm getting a `__doPostBac...

I

Ian•5/22/24, 9:09 PM

This most likely means there aren't any components on the page that can trigger a postback. https://stackoverflow.com/a/3480384/46914

Stack Overflow

__doPostBack is not defined

Im getting that error when try to call a doPostBack on one of my pages, every page that i have in the project use doPostBack function but in this particular page im getting that Javascript erro...

I

Ian•5/22/24, 9:10 PM

@Conner If there are components, then this could be a race condition. try adding a wait to your puppeteer script.

IIan @Conner If there are components, then this could be a race condition. try addin...

C

Conner•5/22/24, 9:49 PM

Awesome thanks... what should I wait for?
Because I screenshot the page before doing __doPostBack and everything looks loaded in

I

Ian•5/22/24, 9:50 PM

there might be a "scripts loaded" event in puppeteer, or maybe you need to turn on JS support. not 100% sure.

I

Ian•5/22/24, 10:13 PM

@Conner You could use

Page.waitForFunction()

Page.waitForFunction()

to return a truthy value https://pptr.dev/api/puppeteer.page.waitforfunction

Page.waitForFunction() method | Puppeteer

Waits for the provided function, pageFunction, to return a truthy value when evaluated in the page's context.

IIan @Conner You could use `Page.waitForFunction()` to return a truthy value https://...

C

Conner•5/22/24, 10:13 PM

I'm evaling other code before that to be clear
Basic querySelector for elements on the page

C

Conner•5/22/24, 10:13 PM

Just seems to be that function that's not available

I

Ian•5/22/24, 10:15 PM

the elements will exist on thet page from the start since they're rendered on the server, however, the JS that adds

__doPostBack

__doPostBack

will not be instantaneous.

C

Conner•5/22/24, 10:15 PM

Ah that makes sense

C

Conner•5/22/24, 10:15 PM

I'll try a waitForFunction thanks

I

Ian•5/22/24, 10:15 PM

you could also just try a 5 second timeout to see if that works since it will confirm it's a race condition.

I

Ian•5/22/24, 10:16 PM

it most likely works locally because the page network response times are faster locally.

I

Ian•5/22/24, 10:16 PM

wait, did you try locally but pointing at the remote page?

C

Conner•5/22/24, 10:16 PM

Yes locally pointing at the remote page

C

Conner•5/22/24, 10:16 PM

Same code locally and in cloudflare, a remote site I'm scraping

I

Ian•5/22/24, 10:17 PM

okay.

I

Ian•5/22/24, 10:17 PM

Ensure you have

waitUntil

waitUntil

set to

load

load

await page.goto('https://google.com', { waitUntil: 'load' });

await page.goto('https://google.com', { waitUntil: 'load' });

await page.goto('https://google.com', { waitUntil: 'load' });

await page.goto('https://google.com', { waitUntil: 'load' });

C

Conner•5/22/24, 10:18 PM

I do yeah
I'll try the waitForFunction gimme a couple minutes and I'll lyk

C

Conner•5/22/24, 10:18 PM

await page.waitForFunction(() => typeof window.__doPostBack === 'function');

await page.waitForFunction(() => typeof window.__doPostBack === 'function');

I

Ian•5/22/24, 10:18 PM

https://www.webshare.io/academy-article/puppeteer-wait-for-page-to-load this article explains the various waitUntil options. You can wait for network idle, for example.

C

Conner•5/22/24, 10:20 PM

I do have

networkidle0

networkidle0

actually yeah

C

Conner•5/22/24, 10:23 PM

@Ian The waitForFunction didn't work,

TimeoutError: waiting for function failed: timeout 30000ms exceeded

TimeoutError: waiting for function failed: timeout 30000ms exceeded

C

Conner•5/22/24, 10:23 PM

Any chance cloudflare's browser might not load the aspx?

I

Ian•5/22/24, 10:23 PM

okay, so the JS is never running.

CConner Any chance cloudflare's browser might not load the aspx?

I

Ian•5/22/24, 10:24 PM

nothing about ASPX pages that make them special.

I

Ian•5/22/24, 10:24 PM

It could be a config difference.

C

Conner•5/22/24, 10:24 PM

Could it be an https thing?

C

Conner•5/22/24, 10:25 PM

Does the ASPX page load the __doPostBack over https or http

C

Conner•5/22/24, 10:25 PM

async custom_launch(endpoint: BrowserWorker): Promise<Browser> {
        const res = await endpoint.fetch('/v1/acquire');
        const status = res.status;
        const text = await res.text();
        if (status !== 200) {
            throw new Error(
                `Unabled to create new browser: code: ${status}: message: ${text}`
            );
        }
        // Got a 200, so response text is actually an AcquireResponse
        const response: { sessionId: string } = JSON.parse(text);
        const transport = await WorkersWebSocketTransport.create(
            endpoint,
            response.sessionId
        );
        return puppeteer.connect({ transport, sessionId: response.sessionId, ignoreHTTPSErrors: true, defaultViewport: { width: 1920, height: 1080 } });
    }

async custom_launch(endpoint: BrowserWorker): Promise<Browser> {
        const res = await endpoint.fetch('/v1/acquire');
        const status = res.status;
        const text = await res.text();
        if (status !== 200) {
            throw new Error(
                `Unabled to create new browser: code: ${status}: message: ${text}`
            );
        }
        // Got a 200, so response text is actually an AcquireResponse
        const response: { sessionId: string } = JSON.parse(text);
        const transport = await WorkersWebSocketTransport.create(
            endpoint,
            response.sessionId
        );
        return puppeteer.connect({ transport, sessionId: response.sessionId, ignoreHTTPSErrors: true, defaultViewport: { width: 1920, height: 1080 } });
    }

async custom_launch(endpoint: BrowserWorker): Promise<Browser> {
        const res = await endpoint.fetch('/v1/acquire');
        const status = res.status;
        const text = await res.text();
        if (status !== 200) {
            throw new Error(
                `Unabled to create new browser: code: ${status}: message: ${text}`
            );
        }
        // Got a 200, so response text is actually an AcquireResponse
        const response: { sessionId: string } = JSON.parse(text);
        const transport = await WorkersWebSocketTransport.create(
            endpoint,
            response.sessionId
        );
        return puppeteer.connect({ transport, sessionId: response.sessionId, ignoreHTTPSErrors: true, defaultViewport: { width: 1920, height: 1080 } });
    }

async custom_launch(endpoint: BrowserWorker): Promise<Browser> {
        const res = await endpoint.fetch('/v1/acquire');
        const status = res.status;
        const text = await res.text();
        if (status !== 200) {
            throw new Error(
                `Unabled to create new browser: code: ${status}: message: ${text}`
            );
        }
        // Got a 200, so response text is actually an AcquireResponse
        const response: { sessionId: string } = JSON.parse(text);
        const transport = await WorkersWebSocketTransport.create(
            endpoint,
            response.sessionId
        );
        return puppeteer.connect({ transport, sessionId: response.sessionId, ignoreHTTPSErrors: true, defaultViewport: { width: 1920, height: 1080 } });
    }

I had to do this a while back for an unrelated page. Added the

ignoreHTTPSErrors: true

ignoreHTTPSErrors: true

ignoreHTTPSErrors: true

ignoreHTTPSErrors: true. Instead of default

puppeteer.launch

puppeteer.launch

puppeteer.launch

puppeteer.launch

I

Ian•5/22/24, 10:25 PM

if you've got mixed https and http resources, it might be refusing to load cross origin.

I

Ian•5/22/24, 10:26 PM

yeah, you might need

ignoreHTTPSErrors

ignoreHTTPSErrors

ignoreHTTPSErrors

ignoreHTTPSErrors enabled.

C

Conner•5/22/24, 10:26 PM

Makes sense alright I'll give the

custom_launch

custom_launch

custom_launch

custom_launch a try

I

Ian•5/22/24, 10:34 PM

You don't need that whole function, just when you call

puppeteer.connect

puppeteer.connect

puppeteer.connect

puppeteer.connect pass it the

ignoreHTTPSErrors: true

ignoreHTTPSErrors: true

ignoreHTTPSErrors: true

ignoreHTTPSErrors: true

IIan You don't need that whole function, just when you call `puppeteer.connect` pass ...

C

Conner•5/22/24, 10:51 PM

Still fails with

TimeoutError: waiting for function failed: timeout 30000ms exceeded

TimeoutError: waiting for function failed: timeout 30000ms exceeded

C

Conner•5/22/24, 10:51 PM

Any other ideas
Odd it works from a local puppeteer but not in cloudflare

C

Conner•5/22/24, 10:58 PM

Also @Ian it looks like

__doPostBack

__doPostBack

is defined inline in the javascript of the page, not loaded externally

<script type="text/javascript">
    //<![CDATA[
    var theForm = document.forms['form1'];
    if (!theForm) {
        theForm = document.form1;
    }
    function __doPostBack(eventTarget, eventArgument) {
        if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
            theForm.__EVENTTARGET.value = eventTarget;
            theForm.__EVENTARGUMENT.value = eventArgument;
            theForm.submit();
        }
    }
    //]]>
</script>

<script type="text/javascript">
    //<![CDATA[
    var theForm = document.forms['form1'];
    if (!theForm) {
        theForm = document.form1;
    }
    function __doPostBack(eventTarget, eventArgument) {
        if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
            theForm.__EVENTTARGET.value = eventTarget;
            theForm.__EVENTARGUMENT.value = eventArgument;
            theForm.submit();
        }
    }
    //]]>
</script>

<script type="text/javascript">
    //<![CDATA[
    var theForm = document.forms['form1'];
    if (!theForm) {
        theForm = document.form1;
    }
    function __doPostBack(eventTarget, eventArgument) {
        if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
            theForm.__EVENTTARGET.value = eventTarget;
            theForm.__EVENTARGUMENT.value = eventArgument;
            theForm.submit();
        }
    }
    //]]>
</script>

<script type="text/javascript">
    //<![CDATA[
    var theForm = document.forms['form1'];
    if (!theForm) {
        theForm = document.form1;
    }
    function __doPostBack(eventTarget, eventArgument) {
        if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
            theForm.__EVENTTARGET.value = eventTarget;
            theForm.__EVENTARGUMENT.value = eventArgument;
            theForm.submit();
        }
    }
    //]]>
</script>

C

Conner•5/22/24, 11:00 PM

Could it be some anti scraping thing? It's an old website so I wouldn't imagine so

I

Ian•5/22/24, 11:17 PM

okay then that script is there right from the start; no load time.

@Walshy | Deploying

Similar Threads

Similar Threads

Similar Threads