Any chance cloudflare's browser might not load the aspx? - Cloudflare Developers

© 2026 Hedgehog Software, LLC

More

Communities Docs About Terms Privacy

Ian

I

Ian•5/22/24, 10:23 PM

okay, so the JS is never running.

Conner

CConner Any chance cloudflare's browser might not load the aspx?

Ian

I

Ian•5/22/24, 10:24 PM

nothing about ASPX pages that make them special.

Ian

I

Ian•5/22/24, 10:24 PM

It could be a config difference.

Conner

C

ConnerOP•5/22/24, 10:24 PM

Could it be an https thing?

Conner

C

ConnerOP•5/22/24, 10:25 PM

Does the ASPX page load the __doPostBack over https or http

Conner

C

ConnerOP•5/22/24, 10:25 PM

async custom_launch(endpoint: BrowserWorker): Promise<Browser> {
        const res = await endpoint.fetch('/v1/acquire');
        const status = res.status;
        const text = await res.text();
        if (status !== 200) {
            throw new Error(
                `Unabled to create new browser: code: ${status}: message: ${text}`
            );
        }
        // Got a 200, so response text is actually an AcquireResponse
        const response: { sessionId: string } = JSON.parse(text);
        const transport = await WorkersWebSocketTransport.create(
            endpoint,
            response.sessionId
        );
        return puppeteer.connect({ transport, sessionId: response.sessionId, ignoreHTTPSErrors: true, defaultViewport: { width: 1920, height: 1080 } });
    }

async custom_launch(endpoint: BrowserWorker): Promise<Browser> {
        const res = await endpoint.fetch('/v1/acquire');
        const status = res.status;
        const text = await res.text();
        if (status !== 200) {
            throw new Error(
                `Unabled to create new browser: code: ${status}: message: ${text}`
            );
        }
        // Got a 200, so response text is actually an AcquireResponse
        const response: { sessionId: string } = JSON.parse(text);
        const transport = await WorkersWebSocketTransport.create(
            endpoint,
            response.sessionId
        );
        return puppeteer.connect({ transport, sessionId: response.sessionId, ignoreHTTPSErrors: true, defaultViewport: { width: 1920, height: 1080 } });
    }

I had to do this a while back for an unrelated page. Added the ignoreHTTPSErrors: trueignoreHTTPSErrors: true. Instead of default puppeteer.launchpuppeteer.launch

Ian

I

Ian•5/22/24, 10:25 PM

if you've got mixed https and http resources, it might be refusing to load cross origin.

Ian

I

Ian•5/22/24, 10:26 PM

yeah, you might need ignoreHTTPSErrorsignoreHTTPSErrors enabled.

Conner

C

ConnerOP•5/22/24, 10:26 PM

Makes sense alright I'll give the custom_launchcustom_launch a try

Ian

I

Ian•5/22/24, 10:34 PM

You don't need that whole function, just when you call puppeteer.connectpuppeteer.connect pass it the ignoreHTTPSErrors: trueignoreHTTPSErrors: true

Ian

IIan You don't need that whole function, just when you call `puppeteer.connect` pass ...

Conner

C

ConnerOP•5/22/24, 10:51 PM

Still fails with TimeoutError: waiting for function failed: timeout 30000ms exceededTimeoutError: waiting for function failed: timeout 30000ms exceeded

Conner

C

ConnerOP•5/22/24, 10:51 PM

Any other ideas
Odd it works from a local puppeteer but not in cloudflare

Conner

C

ConnerOP•5/22/24, 10:58 PM

Also @Ian it looks like __doPostBack__doPostBack is defined inline in the javascript of the page, not loaded externally

<script type="text/javascript">
    //<![CDATA[
    var theForm = document.forms['form1'];
    if (!theForm) {
        theForm = document.form1;
    }
    function __doPostBack(eventTarget, eventArgument) {
        if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
            theForm.__EVENTTARGET.value = eventTarget;
            theForm.__EVENTARGUMENT.value = eventArgument;
            theForm.submit();
        }
    }
    //]]>
</script>

<script type="text/javascript">
    //<![CDATA[
    var theForm = document.forms['form1'];
    if (!theForm) {
        theForm = document.form1;
    }
    function __doPostBack(eventTarget, eventArgument) {
        if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
            theForm.__EVENTTARGET.value = eventTarget;
            theForm.__EVENTARGUMENT.value = eventArgument;
            theForm.submit();
        }
    }
    //]]>
</script>

Conner

C

ConnerOP•5/22/24, 11:00 PM

Could it be some anti scraping thing? It's an old website so I wouldn't imagine so

Ian

I

Ian•5/22/24, 11:17 PM

okay then that script is there right from the start; no load time.

Ian

I

Ian•5/22/24, 11:18 PM

This makes me wonder if scripts are disabled, so that JS never gets evaluated.

Ian

I

Ian•5/22/24, 11:23 PM

@Conner could it be this https://stackoverflow.com/a/55766824/46914 ?

Stack Overflow

How can I execute a __doPostBack function in puppeteer?

I am trying to scrape a web page that requires clicking on a page button through a doPostBack function. I have tried the following code in the chrome console.

javascript:doPostBack('ctl00$

Ian

IIan @Conner could it be this https://stackoverflow.com/a/55766824/46914 ?

Conner

C

ConnerOP•5/23/24, 5:05 AM

Yeah I tried calling .click on the a tag, but it didn't do anything - I'm assuming because it's href is javascript:__doPostBack(...)javascript:__doPostBack(...), which is still undefined

Ian

IIan This makes me wonder if scripts are disabled, so that JS never gets evaluated.

Conner

C

ConnerOP•5/23/24, 5:06 AM

If scripts were disabled, wouldn't my other .evaluate calls fail?

Conner

C

ConnerOP•5/23/24, 5:06 AM

It has to be some difference between cloudflare puppeteer/chromium and normal puppeteer/chromium right? Because that's the only difference when I run it locally

Conner

CConner If scripts were disabled, wouldn't my other .evaluate calls fail?

Ian

I

Ian•5/23/24, 5:06 AM

Not necessarily. it literally might be ignoring

<script>

<script>

tag contents, but the javascript interpreter is still running, hence why .evaluate().evaluate() would work.

Conner

CConner It has to be some difference between cloudflare puppeteer/chromium and normal pu...

Ian

I

Ian•5/23/24, 5:07 AM

Yes, that definitely could be it. are you sure you're running the same version?

Ian

I

Ian•5/23/24, 5:08 AM

also, are there any flags that cloudflare might enable/disable?

Conner

C

ConnerOP•5/23/24, 5:09 AM

The script for this is in the body if that makes any difference

<body>
    <form method="post" action="./action.aspx" onkeypress="javascript:return WebForm_FireDefaultButton(event, 'criteria_basic_btn_search')" id="form1">
        <div class="aspNetHidden">
            <input type="hidden" name="__EVENTTARGET" id="__EVENTTARGET" value=""/>
            <input type="hidden" name="__EVENTARGUMENT" id="__EVENTARGUMENT" value=""/>
            <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="...e"/>
        </div>
        <script type="text/javascript">
            //<![CDATA[
            var theForm = document.forms['form1'];
            if (!theForm) {
                theForm = document.form1;
            }
            function __doPostBack(eventTarget, eventArgument) {
                if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
                    theForm.__EVENTTARGET.value = eventTarget;
                    theForm.__EVENTARGUMENT.value = eventArgument;
                    theForm.submit();
                }
            }
            //]]>
        </script>
        ...

<body>
    <form method="post" action="./action.aspx" onkeypress="javascript:return WebForm_FireDefaultButton(event, 'criteria_basic_btn_search')" id="form1">
        <div class="aspNetHidden">
            <input type="hidden" name="__EVENTTARGET" id="__EVENTTARGET" value=""/>
            <input type="hidden" name="__EVENTARGUMENT" id="__EVENTARGUMENT" value=""/>
            <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="...e"/>
        </div>
        <script type="text/javascript">
            //<![CDATA[
            var theForm = document.forms['form1'];
            if (!theForm) {
                theForm = document.form1;
            }
            function __doPostBack(eventTarget, eventArgument) {
                if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
                    theForm.__EVENTTARGET.value = eventTarget;
                    theForm.__EVENTARGUMENT.value = eventArgument;
                    theForm.submit();
                }
            }
            //]]>
        </script>
        ...

Ian

IIan Yes, that definitely could be it. are you sure you're running the same version?

Conner

C

ConnerOP•5/23/24, 5:10 AM

Yep, 17, just like @cloudflare/puppeteer github pages says they use

Ian

IIan also, are there any flags that cloudflare might enable/disable?

Conner

C

ConnerOP•5/23/24, 5:10 AM

And not sure, I couldn't find it online anywhere... any team members have any idea in here?

Ian

I

Ian•5/23/24, 5:12 AM

Oh I wonder if the //<![CDATA[//<![CDATA[ makes some kind of difference here? I don't think it does.

Ian

I

Ian•5/23/24, 5:16 AM

@Conner If you visit the site directly, are you able to access __doPostBack__doPostBack ?

Ian

I

Ian•5/23/24, 5:17 AM

Also, what is the actual puppeteer code you're using for this?

Conner

C

ConnerOP•5/23/24, 5:23 AM

This times out await page.waitForFunction(() => typeof window.__doPostBack === 'function', { timeout: 30000, polling: 100 });await page.waitForFunction(() => typeof window.__doPostBack === 'function', { timeout: 30000, polling: 100 });

In chrome devtools if I do __doPostBack.toString()__doPostBack.toString() I get

function __doPostBack(eventTarget, eventArgument) {
    if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
        theForm.__EVENTTARGET.value = eventTarget;
        theForm.__EVENTARGUMENT.value = eventArgument;
        theForm.submit();
    }
}

function __doPostBack(eventTarget, eventArgument) {
    if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
        theForm.__EVENTTARGET.value = eventTarget;
        theForm.__EVENTARGUMENT.value = eventArgument;
        theForm.submit();
    }
}

As expected

Ian

IIan Oh I wonder if the `//<![CDATA[` makes some kind of difference here? I don't thi...

Conner

C

ConnerOP•5/23/24, 5:24 AM

I don't believe so, I'm scraping the same page on local puppeteer as cloudflare puppeteer haha

Ian

I

Ian•5/23/24, 5:25 AM

Yeah, I'm just trying to figure out what's different; obviously it's something to do with the config cloudflare is using, or the network itself.

Ian

I

Ian•5/23/24, 5:26 AM

You might want to try pointing the cloudflare puppeteer at one of those user agent inspectors to see what it says https://www.whatsmyua.info/

What's my user agent?

Detect user-agent, operating system, browser, and device using several libraries, including ua-parser, ua-parser-js, and platform.

Ian

I

Ian•5/23/24, 5:27 AM

take screenshots of both and compare them to see what the differences are.

Conner

C

ConnerOP•5/23/24, 5:29 AM

I'm already setting user agent

await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36');

await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36');

and I've screenshotted both and they're the same

Ian

IIan Yeah, I'm just trying to figure out what's different; obviously it's something t...

Conner

C

ConnerOP•5/23/24, 5:30 AM

I couldn't find anything on any special config cloudflare has but if you know of anything that'd be awesome

Ian

I

Ian•5/23/24, 5:30 AM

not that I can see. what about console logs or errors, are you gathering those?

Conner

C

ConnerOP•5/23/24, 5:31 AM

Not gathering any console logs, that's a good idea

Conner

C

ConnerOP•5/23/24, 5:31 AM

My other next thought is maybe inspect the page network requests? Making sure they line up with what I see in devtools in a normal browser?

Ian

I

Ian•5/23/24, 5:31 AM

https://blog.adamcameron.me/2021/01/listening-to-console-log-of-page-loaded.html

Listening to the console log of a page loaded with Puppeteer

Adam Cameron works out how to listen for a Puppeteer page's log activities.

Conner

CConner My other next thought is maybe inspect the page network requests? Making sure th...

Ian

I

Ian•5/23/24, 5:32 AM

yeah, though, you already have the JS you need right there baked into the page.

Ian

I

Ian•5/23/24, 5:32 AM

My current guess is something is crashing and blocking that function from being created.

Ian

I

Ian•5/23/24, 5:33 AM

@Conner quick question, are you literally calling __doCallBack__doCallBack ? Or clicking the submit button?

Ian

IIan @Conner quick question, are you literally calling `__doCallBack` ? Or clicking t...

Conner

C

ConnerOP•5/23/24, 5:34 AM

I've tried both, neither worked

Conner

C

ConnerOP•5/23/24, 5:34 AM

I also tried running the code inside __doCallBack

Conner

C

ConnerOP•5/23/24, 5:35 AM

function __doPostBack(eventTarget, eventArgument) {
    if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
        theForm.__EVENTTARGET.value = eventTarget;
        theForm.__EVENTARGUMENT.value = eventArgument;
        theForm.submit();
    }
}

function __doPostBack(eventTarget, eventArgument) {
    if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
        theForm.__EVENTTARGET.value = eventTarget;
        theForm.__EVENTARGUMENT.value = eventArgument;
        theForm.submit();
    }
}

When I tried running that code inside, I got something like cannot set property value of undefined

Conner

C

ConnerOP•5/23/24, 5:35 AM

So theForm.__EVENTTARGET is undefined? Which again is weird because the whole page visually loads in fine

Ian

I

Ian•5/23/24, 5:36 AM

Right because theFormtheForm isn't defined in there, it's defined just outside by

var theForm = document.forms['form1'];

var theForm = document.forms['form1'];

Conner

CConner The script for this is in the body if that makes any difference ``` <body> ...

Conner

C

ConnerOP•5/23/24, 5:36 AM

It's a global variable but yeah I also tried querySelector #form1

Ian

I

Ian•5/23/24, 5:39 AM

so what's supposed to happen is when a button on the aspx page is clicked it will call __doPostBack__doPostBack and fill in the eventTarget and any eventArgs. These then get filled in as those hidden __EVENTTARGET__EVENTTARGET and __EVENTARGUMENT__EVENTARGUMENT input fields defined above. Those get submitted to the server and that's how it knows what button you pressed so it can run the appropriate event handler code on the server, then send you back a new page render.

async custom_launch(endpoint: BrowserWorker): Promise<Browser> {
        const res = await endpoint.fetch('/v1/acquire');
        const status = res.status;
        const text = await res.text();
        if (status !== 200) {
            throw new Error(
                `Unabled to create new browser: code: ${status}: message: ${text}`
            );
        }
        // Got a 200, so response text is actually an AcquireResponse
        const response: { sessionId: string } = JSON.parse(text);
        const transport = await WorkersWebSocketTransport.create(
            endpoint,
            response.sessionId
        );
        return puppeteer.connect({ transport, sessionId: response.sessionId, ignoreHTTPSErrors: true, defaultViewport: { width: 1920, height: 1080 } });
    }

async custom_launch(endpoint: BrowserWorker): Promise<Browser> {
        const res = await endpoint.fetch('/v1/acquire');
        const status = res.status;
        const text = await res.text();
        if (status !== 200) {
            throw new Error(
                `Unabled to create new browser: code: ${status}: message: ${text}`
            );
        }
        // Got a 200, so response text is actually an AcquireResponse
        const response: { sessionId: string } = JSON.parse(text);
        const transport = await WorkersWebSocketTransport.create(
            endpoint,
            response.sessionId
        );
        return puppeteer.connect({ transport, sessionId: response.sessionId, ignoreHTTPSErrors: true, defaultViewport: { width: 1920, height: 1080 } });
    }

<script type="text/javascript">
    //<![CDATA[
    var theForm = document.forms['form1'];
    if (!theForm) {
        theForm = document.form1;
    }
    function __doPostBack(eventTarget, eventArgument) {
        if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
            theForm.__EVENTTARGET.value = eventTarget;
            theForm.__EVENTARGUMENT.value = eventArgument;
            theForm.submit();
        }
    }
    //]]>
</script>

<script type="text/javascript">
    //<![CDATA[
    var theForm = document.forms['form1'];
    if (!theForm) {
        theForm = document.form1;
    }
    function __doPostBack(eventTarget, eventArgument) {
        if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
            theForm.__EVENTTARGET.value = eventTarget;
            theForm.__EVENTARGUMENT.value = eventArgument;
            theForm.submit();
        }
    }
    //]]>
</script>

<body>
    <form method="post" action="./action.aspx" onkeypress="javascript:return WebForm_FireDefaultButton(event, 'criteria_basic_btn_search')" id="form1">
        <div class="aspNetHidden">
            <input type="hidden" name="__EVENTTARGET" id="__EVENTTARGET" value=""/>
            <input type="hidden" name="__EVENTARGUMENT" id="__EVENTARGUMENT" value=""/>
            <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="...e"/>
        </div>
        <script type="text/javascript">
            //<![CDATA[
            var theForm = document.forms['form1'];
            if (!theForm) {
                theForm = document.form1;
            }
            function __doPostBack(eventTarget, eventArgument) {
                if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
                    theForm.__EVENTTARGET.value = eventTarget;
                    theForm.__EVENTARGUMENT.value = eventArgument;
                    theForm.submit();
                }
            }
            //]]>
        </script>
        ...

<body>
    <form method="post" action="./action.aspx" onkeypress="javascript:return WebForm_FireDefaultButton(event, 'criteria_basic_btn_search')" id="form1">
        <div class="aspNetHidden">
            <input type="hidden" name="__EVENTTARGET" id="__EVENTTARGET" value=""/>
            <input type="hidden" name="__EVENTARGUMENT" id="__EVENTARGUMENT" value=""/>
            <input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="...e"/>
        </div>
        <script type="text/javascript">
            //<![CDATA[
            var theForm = document.forms['form1'];
            if (!theForm) {
                theForm = document.form1;
            }
            function __doPostBack(eventTarget, eventArgument) {
                if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
                    theForm.__EVENTTARGET.value = eventTarget;
                    theForm.__EVENTARGUMENT.value = eventArgument;
                    theForm.submit();
                }
            }
            //]]>
        </script>
        ...

function __doPostBack(eventTarget, eventArgument) {
    if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
        theForm.__EVENTTARGET.value = eventTarget;
        theForm.__EVENTARGUMENT.value = eventArgument;
        theForm.submit();
    }
}

function __doPostBack(eventTarget, eventArgument) {
    if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
        theForm.__EVENTTARGET.value = eventTarget;
        theForm.__EVENTARGUMENT.value = eventArgument;
        theForm.submit();
    }
}

function __doPostBack(eventTarget, eventArgument) {
    if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
        theForm.__EVENTTARGET.value = eventTarget;
        theForm.__EVENTARGUMENT.value = eventArgument;
        theForm.submit();
    }
}

function __doPostBack(eventTarget, eventArgument) {
    if (!theForm.onsubmit || (theForm.onsubmit() != false)) {
        theForm.__EVENTTARGET.value = eventTarget;
        theForm.__EVENTARGUMENT.value = eventArgument;
        theForm.submit();
    }
}