What’s the main driver for that? - Cloudflare Developers

K

KiwiOP•4/9/25, 11:02 PM

*driver=reason

KKiwi What’s the main driver for that?

A

Avi•4/9/25, 11:02 PM

theyre deprecating pages in favor of assets

K

KiwiOP•4/9/25, 11:02 PM

Sweet

A

Avi•4/9/25, 11:03 PM

they dont want to freak people out so they're being very vague on timelines but theyve made it clear that workers+assets is the future and pages is not

A

Avi•4/9/25, 11:03 PM

makes sense

A

Avi•4/9/25, 11:03 PM

should have been like this from the start

K

KiwiOP•4/9/25, 11:03 PM

100%

CCoder75 Hi. I have this website which is https://oldschoolstylecomputerbyrajneet.pages.d...

G

gruntlord6•4/9/25, 11:52 PM

just use the free tier of r2

G

gruntlord6•4/9/25, 11:53 PM

you get a certain amount of free image transformation with the images platform per month but you always have to pay to host them in there

G

gruntlord6•4/9/25, 11:54 PM

you also shouldn't have issues with pages

L

LordSilver•4/10/25, 1:35 AM

Does anyone know why the fetch html api is crazy slow? Are there ways to improve it?

L

LordSilver•4/10/25, 1:37 AM

Crazy slow and you even get rate-limited, nice...

❯ bun index
24 |     body: JSON.stringify(payload),
25 |   });
26 | 
27 |   // Check if the response was successful
28 |   if (!response.ok) {
29 |     throw new Error(
               ^
error: Failed to fetch HTML content: 429 Too Many Requests

❯ bun index
24 |     body: JSON.stringify(payload),
25 |   });
26 | 
27 |   // Check if the response was successful
28 |   if (!response.ok) {
29 |     throw new Error(
               ^
error: Failed to fetch HTML content: 429 Too Many Requests

❯ bun index
24 |     body: JSON.stringify(payload),
25 |   });
26 | 
27 |   // Check if the response was successful
28 |   if (!response.ok) {
29 |     throw new Error(
               ^
error: Failed to fetch HTML content: 429 Too Many Requests

❯ bun index
24 |     body: JSON.stringify(payload),
25 |   });
26 | 
27 |   // Check if the response was successful
28 |   if (!response.ok) {
29 |     throw new Error(
               ^
error: Failed to fetch HTML content: 429 Too Many Requests

J

James•4/10/25, 1:44 AM

The API for what exactly? What endpoint?

JJames The API for what exactly? What endpoint?

L

LordSilver•4/10/25, 1:57 AM

https://developers.cloudflare.com/browser-rendering/rest-api/content-endpoint/

Cloudflare Docs

Fetch HTML · Browser Rendering docs

The /content endpoint instructs the browser to navigate to a website and capture the fully rendered HTML of a page, including the head section, after JavaScript execution. This is ideal for capturing content from JavaScript-heavy or interactive websites.

L

LordSilver•4/10/25, 1:58 AM

it takes 5-6s to return the html page, whereas if i do a fetch on my own it takes ms

L

LordSilver•4/10/25, 2:00 AM

my theory is that Cloudflare doesn't have a fleet of Puppeteer workers ready at all times to immediately process your fetch request, so they need to instantiate a Chrome browser from scratch every time.

LLordSilver my theory is that Cloudflare doesn't have a fleet of Puppeteer workers ready at ...

C

Chaika•4/10/25, 2:17 AM

Pretty much all that. Actual fetch within your worker comes out of the same machine. Puppeteer is expensive, need to spin up an environment/isolate each user from each other, and rendering a page via Puppeteer in general is going to be way more expensive then a fetch of course, even locally it takes ~2s to get the html of google.com via puppeteer. iirc there's some interesting bits about knowing when rendering is done too which involve waiting for no activity. Also last I knew only Chicago was running browser rendering, at least back when it was just a workers binding.

It's just not really comparable, a single http request vs rendering a web page with all its resources

L

LordSilver•4/10/25, 2:37 AM

But when you use the Browser Rendering API from your worker, it spins up another special VM that runs Puppeteer and connects through it via websocket right?

L

LordSilver•4/10/25, 2:38 AM

Then your worker sends commands to that VM running Puppeteer.

CChaika Pretty much all that. Actual fetch within your worker comes out of the same mach...

L

LordSilver•4/10/25, 2:39 AM

Through my tests, I could see the IP used by the Browser Rendering API was sometimes from Spain, sometimes France.

L

LordSilver•4/10/25, 2:40 AM

Also I can render the full page with js included using happy-dom in less than 1s.

L

LordSilver•4/10/25, 2:40 AM

and that's still a browser, just way smaller than Chrome.

L

LordSilver•4/10/25, 2:48 AM

Just did a request now and got response from MRS location.

L

LordSilver•4/10/25, 2:49 AM

But I'm pretty sure Puppeteer isn't running in a DO since that would be impossible given the constraints.

L

LordSilver•4/10/25, 2:52 AM

Maybe I remember wrong. So a Worker is 3-10MB max in size, while a DO is 5-10GB max in size?

L

LordSilver•4/10/25, 2:56 AM

Playwright+Chrome is around 100-165MB in size. I wonder if Cloudflare runs the headless version of Chrome.

Downloading Chromium 134.0.6998.35 (playwright build v1161) from https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1161/chromium-linux.zip
164.9 MiB [====================] 100% 0.0s
Downloading Chromium Headless Shell 134.0.6998.35 (playwright build v1161) from https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1161/chromium-headless-shell-linux.zip
100.9 MiB [====================] 100% 0.0s

Downloading Chromium 134.0.6998.35 (playwright build v1161) from https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1161/chromium-linux.zip
164.9 MiB [====================] 100% 0.0s
Downloading Chromium Headless Shell 134.0.6998.35 (playwright build v1161) from https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1161/chromium-headless-shell-linux.zip
100.9 MiB [====================] 100% 0.0s

Downloading Chromium 134.0.6998.35 (playwright build v1161) from https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1161/chromium-linux.zip
164.9 MiB [====================] 100% 0.0s
Downloading Chromium Headless Shell 134.0.6998.35 (playwright build v1161) from https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1161/chromium-headless-shell-linux.zip
100.9 MiB [====================] 100% 0.0s

Downloading Chromium 134.0.6998.35 (playwright build v1161) from https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1161/chromium-linux.zip
164.9 MiB [====================] 100% 0.0s
Downloading Chromium Headless Shell 134.0.6998.35 (playwright build v1161) from https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1161/chromium-headless-shell-linux.zip
100.9 MiB [====================] 100% 0.0s

LLordSilver Crazy slow and you even get rate-limited, nice... ``` ❯ bun index 24 | body...

L

LordSilver•4/10/25, 3:03 AM

found answer to this at least.

LLordSilver Maybe I remember wrong. So a Worker is 3-10MB max in size, while a DO is 5-10GB ...

C

Chaika•4/10/25, 3:12 AM

Both have same script size (10 mb max paid), since durable objects are defined in worker scripts, and same memory (128 mb max). the 10 gb just sqlite storage limit inside of the durable object
But yea, way too small to run puppeteer, they've documented before they run it in their container platform https://blog.cloudflare.com/container-platform-preview/

CChaika Both have same script size (10 mb max paid), since durable objects are defined i...

L

LordSilver•4/10/25, 3:30 AM

Ok, so I remembered right. Then these Playwright/Puppeteer instances are essentially in a k8s cluster, which Cloudflare calls "container platform".

L

LordSilver•4/10/25, 3:36 AM

I'm now unsure whether the response I’m seeing is from where Puppeteer actually runs or from the DO location that responded to my client after the request was rerouted through Cloudflare's network infrastructure.

L

LordSilver•4/10/25, 3:36 AM

If it's true that Browser Rendering API runs only in Chicago, that would explain the 6s latency in response.

LLordSilver If it's true that Browser Rendering API runs only in Chicago, that would explain...

C

Chaika•4/10/25, 4:15 AM

I tried the browser rendering rest api, fetching cloudflare.com/cdn-cgi/trace and it runs in Europe and other locations. I'd trust that output, espec since Cloudflares container platform is global. It's been a while since I tested the workers API, stuff may have changed/could be different from rest api in that aspect, etc.

C

Cоle•4/10/25, 5:04 AM

I've been digging through the Worker docs for a bit, but I haven't found anything related to DIY ML/AI models. I'm working on a Pages site, and I need to do some super basic linear-regression-adjacent inferences (maybe like 20-100 per page load, cache-able). At first, I was thinking about writing a basic XGBoost output JS parser... but now I'm thinking about scalability. Does CF Workers' platform have a feature loosely applicable to this?

Ggruntlord6 just use the free tier of r2

C

Coder75•4/10/25, 5:05 AM

OK. Thank you for sharing your experience and knowledge

R

Ricky U•4/10/25, 5:13 AM

If i create a worker within the functions directory, will pages functions pick it up?

RRicky U If i create a worker within the functions directory, will pages functions pick i...

C

Cоle•4/10/25, 5:26 AM

not within

functions/

functions/

functions/

functions/ but https://developers.cloudflare.com/workers/static-assets/migrate-from-pages/#pages-functions-with-an-advanced-mode-_workerjs-file

3

39dylan•4/10/25, 5:55 AM

i have a static site that is too large for pages/workers (20k file limit) can i use cloudflare snippets + R2 storage to replicate this stack? it is simply a static site thats built on my local machine then uploaded old school way

L

LordSilver•4/10/25, 6:15 AM

just noticed playwright is now almost tripled puppeteer in just a year on npm. hopefully it becomes GA soon.

CChaika I tried the browser rendering rest api, fetching cloudflare.com/cdn-cgi/trace an...

L

LordSilver•4/10/25, 6:16 AM

makes sense. i wish we could get also the response headers alongside the html with the

/content

/content

/content

/content endpoint. :/

M

Mitya•4/10/25, 10:57 AM

Has anyone noticed that certain

console.log()

console.log()

console.log()

console.log() calls don't show up when working locally (in my case, with the

--remote

--remote

--remote

--remote flag?) This happens all the time for me - and it's 100% not because the line of code doesn't run. Sometimes if I restart the worker it then fixes itself and logs show. Seemingly I can also fix it by just modifying an arbitrary line of code (or adding a space) so the worker gets freshly uploaded to the edge. Any one from CF know why this would happen? It's happened on multiple versions of Wrangler, too.

D

datacenterdude•4/10/25, 2:12 PM

Is there any documentation/BP on migrating a Pages site/repo to operate natively within a worker? So for example I have a worker running an API backend, but the swaggerUI is in a separate repo subfolder and Pages gets built from that. Can I just serve both from the worker efficiently? Just put /swagger in the same folder as the worker and wrangler deploy? I haven’t tried it yet but curious if other have and what stumbles they run into.

Ccardyet I'm a bit confused migrating from Pages to Workers for a NextJs project which ha...

W

Wouter J•4/10/25, 3:41 PM

Migrating from Pages to Workers, and away from @Cloudflare/workers-types but at this point i'm slowly starting to lose my mind.

D

datacenterdude•4/10/25, 5:57 PM

Moving my swagger page from Pages into Assets within the worker, and running into a new one I haven't seen where

wrangler deploy

wrangler deploy

is upset about the numbering of the arrays in the

wrangler.jsonc

wrangler.jsonc

wrangler.jsonc

wrangler.jsonc file. Feel like I'm missing something dumb...

But.... I've got the

"directory"

"directory"

"directory"

"directory" property defined.

{
  "name": "api",
  "main": "v1/index.ts",
  "compatibility_date": "2025-04-05",
  "d1_databases": [
    {
      "binding": "...",
      "database_name": "...",
      "database_id": "..."
    }
  ],
  "assets": [
    {
      "binding": "SWAGGER_JS",
      "directory": "./swagger",
      "pattern": "*.js"
    },
    {
      "binding": "SWAGGER_CSS",
      "directory": "./swagger",
      "pattern": "*.css"
    },
    {
      "binding": "SWAGGER_HTML",
      "directory": "./swagger",
      "pattern": "*.html"
    },
    {
      "binding": "SWAGGER_YAML",
      "directory": "./swagger",
      "pattern": "*.yaml"
    }
  ]
}

{
  "name": "api",
  "main": "v1/index.ts",
  "compatibility_date": "2025-04-05",
  "d1_databases": [
    {
      "binding": "...",
      "database_name": "...",
      "database_id": "..."
    }
  ],
  "assets": [
    {
      "binding": "SWAGGER_JS",
      "directory": "./swagger",
      "pattern": "*.js"
    },
    {
      "binding": "SWAGGER_CSS",
      "directory": "./swagger",
      "pattern": "*.css"
    },
    {
      "binding": "SWAGGER_HTML",
      "directory": "./swagger",
      "pattern": "*.html"
    },
    {
      "binding": "SWAGGER_YAML",
      "directory": "./swagger",
      "pattern": "*.yaml"
    }
  ]
}

{
  "name": "api",
  "main": "v1/index.ts",
  "compatibility_date": "2025-04-05",
  "d1_databases": [
    {
      "binding": "...",
      "database_name": "...",
      "database_id": "..."
    }
  ],
  "assets": [
    {
      "binding": "SWAGGER_JS",
      "directory": "./swagger",
      "pattern": "*.js"
    },
    {
      "binding": "SWAGGER_CSS",
      "directory": "./swagger",
      "pattern": "*.css"
    },
    {
      "binding": "SWAGGER_HTML",
      "directory": "./swagger",
      "pattern": "*.html"
    },
    {
      "binding": "SWAGGER_YAML",
      "directory": "./swagger",
      "pattern": "*.yaml"
    }
  ]
}

{
  "name": "api",
  "main": "v1/index.ts",
  "compatibility_date": "2025-04-05",
  "d1_databases": [
    {
      "binding": "...",
      "database_name": "...",
      "database_id": "..."
    }
  ],
  "assets": [
    {
      "binding": "SWAGGER_JS",
      "directory": "./swagger",
      "pattern": "*.js"
    },
    {
      "binding": "SWAGGER_CSS",
      "directory": "./swagger",
      "pattern": "*.css"
    },
    {
      "binding": "SWAGGER_HTML",
      "directory": "./swagger",
      "pattern": "*.html"
    },
    {
      "binding": "SWAGGER_YAML",
      "directory": "./swagger",
      "pattern": "*.yaml"
    }
  ]
}

D

datacenterdude•4/10/25, 6:02 PM

Aw, I think I see it now. It needs assets > bindings > binding1 > binding2 > binding*

D

datacenterdude•4/10/25, 6:05 PM

Like so...

"assets": {
    "bindings": [
      {
        "binding": "SWAGGER_JS",
        "directory": "./swagger",
        "pattern": "*.js"

"assets": {
    "bindings": [
      {
        "binding": "SWAGGER_JS",
        "directory": "./swagger",
        "pattern": "*.js"

"assets": {
    "bindings": [
      {
        "binding": "SWAGGER_JS",
        "directory": "./swagger",
        "pattern": "*.js"

"assets": {
    "bindings": [
      {
        "binding": "SWAGGER_JS",
        "directory": "./swagger",
        "pattern": "*.js"

And nope, that didn't get it done either. Stuck here.

D

datacenterdude•4/10/25, 6:16 PM

Figured it out. Had to be in this format:

 "assets": {
    "directory": "./swagger",
    "binding": "SWAGGER"
  }

 "assets": {
    "directory": "./swagger",
    "binding": "SWAGGER"
  }

 "assets": {
    "directory": "./swagger",
    "binding": "SWAGGER"
  }

 "assets": {
    "directory": "./swagger",
    "binding": "SWAGGER"
  }

D

datacenterdude•4/10/25, 6:19 PM

Turns out I was just overcomplicating things. Doesn't sound like me at all.

S

Samantas5855•4/10/25, 9:55 PM

are there issues with the caching in cloudflare pages?

K

KiwiOP•4/10/25, 10:29 PM

Has anyone got a public repo they can share which shows a good Workers + Hono + BetterAuth implementation with a React Client?