like, when you parse it, how are you compressing it? - Cloudflare Developers

© 2026 Hedgehog Software, LLC

More

Communities Docs About Terms Privacy

kian

Kkian like, when you parse it, how are you compressing it?

dave

D

dave•12/29/23, 4:16 AM

uh it's pretty bad code tbh..

const operation_json = await operation_result.json() as any;

if(operation_json.result === "win") {
  const filename = cacheKey.pathname.split("/").pop() ?? "undefined.pdf";
  const filename_json = filename.replace(/\.[^/.]+$/, "") + ".json";

  const operation_json_str = JSON.stringify(operation_json, null, 2);
  const custom_res = new Response(operation_json_str);

  const compressedReadableStream = custom_res.body!.pipeThrough(
    new CompressionStream('gzip')
  );

  // ...
}

const operation_json = await operation_result.json() as any;

if(operation_json.result === "win") {
  const filename = cacheKey.pathname.split("/").pop() ?? "undefined.pdf";
  const filename_json = filename.replace(/\.[^/.]+$/, "") + ".json";

  const operation_json_str = JSON.stringify(operation_json, null, 2);
  const custom_res = new Response(operation_json_str);

  const compressedReadableStream = custom_res.body!.pipeThrough(
    new CompressionStream('gzip')
  );

  // ...
}

kian

K

kianOP•12/29/23, 4:18 AM

gotcha - I didn’t know how it’d work without stringifying before compression but that’s what you’re doing

dave

D

dave•12/29/23, 4:18 AM

the thing is I'd need to know the size of the compressed gzip object to be able to upload it via S3 anyway

dave

D

dave•12/29/23, 4:18 AM

so gzip for huge objects is just never gonna work for me here

kian

K

kianOP•12/29/23, 4:19 AM

Multipart is pretty much all you’d be able to do

dave

D

dave•12/29/23, 4:19 AM

I wanna go to bed so I just made a separate path where anything over >16MB is not compressed at all

kian

Kkian Multipart is pretty much all you’d be able to do

dave

D

dave•12/29/23, 4:21 AM

wouldn't that get super messy because I want to return the same thing to the client?

Andrew Murphy

A

Andrew Murphy•12/29/23, 4:37 AM

is there some kind of limitation on making http requests from a worker? I have nextjs app that i am using

next-on-pages

next-on-pages

for; it has an api that gets deployed to workers that calls a graphql endpoint. it works fine locally, and using wrangler, but when deployed i get an error ApolloError: Unexpected token '<', "<html><hea"... is not valid JSONApolloError: Unexpected token '<', "<html><hea"... is not valid JSON

kian

K

kianOP•12/29/23, 4:38 AM

What’s the URL you’re hitting?

kian

K

kianOP•12/29/23, 4:38 AM

It can’t be an IP or use custom ports.

dave

D

dave•12/29/23, 4:39 AM

cloudflared tunnels might work for @andrewmurphy.io, assuming he controls where the server stuff is run.

Andrew Murphy

A

Andrew Murphy•12/29/23, 4:41 AM

It doesn't use an IP or custom port. https://courses.techleaderslaunchpad.com/graphqlhttps://courses.techleaderslaunchpad.com/graphql

dave

Ddave cloudflared tunnels might work for @andrewmurphy.io, _assuming_ he controls wher...

Andrew Murphy

A

Andrew Murphy•12/29/23, 4:41 AM

why should I need to tunnel to a publicly accessible URL?

Andrew Murphy

AAndrew Murphy why should I need to tunnel to a publicly accessible URL?

dave

D

dave•12/29/23, 4:41 AM

if you were running on a custom port or IP

dave

Ddave if you were running on a custom port or IP

Andrew Murphy

A

Andrew Murphy•12/29/23, 4:41 AM

ah okay, i am not

Andrew Murphy

A

Andrew Murphy•12/29/23, 4:43 AM

oh, it also works fine on the build. the same API endpoint is called on the cloudflare build

Andrew Murphy

A

Andrew Murphy•12/29/23, 4:44 AM

so cloudflare can access that API fine on its build servers, just not from a worker

Andrew Murphy

AAndrew Murphy so cloudflare can access that API fine on its build servers, just not from a wor...

kian

K

kianOP•12/29/23, 4:49 AM

The build runs outside of Cloudflare’s infrastructure and from different IPs.

kian

K

kianOP•12/29/23, 4:49 AM

It’s possible that the website blocks Workers

Original message was deleted

kian

K

kianOP•12/29/23, 4:49 AM

Given that it’s only talking to Cloudflare, probably not.

kian

Kkian It’s possible that the website blocks Workers

Andrew Murphy

A

Andrew Murphy•12/29/23, 4:51 AM

it's just a wordpress instance that i host, so I doubt it

kian

K

kianOP•12/29/23, 4:53 AM

Fetch it without a client and see what await response.text()await response.text() says.

kian

Kkian Fetch it without a client and see what `await response.text()` says.

Andrew Murphy

A

Andrew Murphy•12/29/23, 4:53 AM

what do you mean "without a client" ?

Andrew Murphy

A

Andrew Murphy•12/29/23, 4:54 AM

oh, you mean without apollo, just a fetch request

Andrew Murphy

A

Andrew Murphy•12/29/23, 4:54 AM

good idea, now i need to learn how to use fetch to make a graphql call

kian

K

kianOP•12/29/23, 4:55 AM

kian

K

kianOP•12/29/23, 4:56 AM

Personally, it just tries to take me to a captcha - which a Worker can't do.

Andrew Murphy

A

Andrew Murphy•12/29/23, 5:00 AM

ohhh, it'll be my hosting provider doing "smart" stuff. let me see if i can turn it off

kian

Kkian Personally, it just tries to take me to a captcha - which a Worker can't do.

Andrew Murphy

A

Andrew Murphy•12/29/23, 5:02 AM

just confirmed you're right

<html><head><link rel="icon" href="data:;"><meta http-equiv="refresh" content="0;/.well-known/sgcaptcha/?r=%2Fgraphql&y=ipr:2a06:98c0:3600::103:1703826118.917"></meta></head></html>

<html><head><link rel="icon" href="data:;"><meta http-equiv="refresh" content="0;/.well-known/sgcaptcha/?r=%2Fgraphql&y=ipr:2a06:98c0:3600::103:1703826118.917"></meta></head></html>

bny doing the raw fetch thing. now to work out how to get my hosting provider to turn that off

Andrew Murphy

A

Andrew Murphy•12/29/23, 5:06 AM

as a rubbish work around I could at build time generate a static file with data from the endpoint and use that from the worker, but that's a bad solution

Erisa

EErisa Smart + Regional is a very powerful combination if your goal is to maximise cach...

dave

D

dave•12/29/23, 6:58 AM

woot woot, I see it working!

All the requests I just did in IAD (Google Cloud Run), are returning cached responses (just like they did for me in MIA).

dave

D

dave•12/29/23, 7:12 AM

noice

Andrew Murphy

AAndrew Murphy ohhh, it'll be my hosting provider doing "smart" stuff. let me see if i can turn...

sathoro

S

sathoro•12/29/23, 10:35 AM

Siteground blocks all Workers and you can only work around it by going through a proxy. They will not whitelist it for you

sathoro

S

sathoro•12/29/23, 10:37 AM

Btw save yourself the headache of trying to get them to unblock it for you, they are absolutely worthless

kian

K

kianOP•12/29/23, 6:44 PM

ctx.waitUntil, ctx being the third arg to the fetch handler

kian

K

kianOP•12/29/23, 6:44 PM

it’s non-blocking (pass it a promise, no await) and will extend the lifetime of the Worker by 30s after the response is returned

kian

K

kianOP•12/29/23, 6:45 PM

do it before

kian

K

kianOP•12/29/23, 6:45 PM

it’s basically fire and forget

kian

K

kianOP•12/29/23, 6:45 PM

don’t await waitUntil and don’t await anything you pass to it

dave

D

dave•12/30/23, 12:11 AM

POST https://sup.ai.moda/analyze - Canceled @ 12/29/2023, 5:56:24 PM
  (log)   <-- POST /analyze
  (debug) cf-cache-status: HIT [PDF]
  (debug) md5: 263b609960b7b33c154cf0607b207b73
  (debug) cf-cache-status: HIT [JSON]
  (log)   --> POST /analyze 200 24ms

POST https://sup.ai.moda/analyze - Canceled @ 12/29/2023, 5:56:24 PM
  (log)   <-- POST /analyze
  (debug) cf-cache-status: HIT [PDF]
  (debug) md5: 263b609960b7b33c154cf0607b207b73
  (debug) cf-cache-status: HIT [JSON]
  (log)   --> POST /analyze 200 24ms

Is 24ms the time it took for the cancel to be issued, or for my waitUntil to be finished?

- -

-

- -•12/30/23, 8:37 AM

does deploying workers cost you anything?

Enzo

E

Enzo•12/30/23, 1:35 PM

hi there! I'm coming from Vercel, where I can set s-max-age: {seconds}s-max-age: {seconds} and my functions are cached, so my quota is not fully used..

I deployed the same API on CloudfarePages/Workers and it didn't take effect.. my quota is on 90%

I also tried to setup a Cache Rule at domain level, but don't take effect.. I'm using Hono.. can someone share an example of Functions Cache?

thanks in advance!

Enzo

E

Enzo•12/30/23, 1:38 PM

thanks! so i'd like to setup on Worker's side.. I'm using Hono.. should I use s-max-ages-max-age? or maxagemaxage?

Enzo

E

Enzo•12/30/23, 1:38 PM

(on Vercel it was s-max-ages-max-age)

Enzo

E

Enzo•12/30/23, 1:38 PM

ups sorry now I understand

Enzo

E

Enzo•12/30/23, 1:39 PM

so there is no option like Funcions Caching on Vercel?

Enzo

E

Enzo•12/30/23, 1:39 PM

yeah, I want to prevent that.. ok thanks so much!

Pepper

P

Pepper•12/30/23, 3:18 PM

On the quick edit page, does anyone know how to stop it from automatically running every time you make a change?

sorewwa

S

sorewwa•12/30/23, 4:27 PM

please someone help me

POST https://sup.ai.moda/analyze - Canceled @ 12/29/2023, 5:56:24 PM
  (log)   <-- POST /analyze
  (debug) cf-cache-status: HIT [PDF]
  (debug) md5: 263b609960b7b33c154cf0607b207b73
  (debug) cf-cache-status: HIT [JSON]
  (log)   --> POST /analyze 200 24ms

POST https://sup.ai.moda/analyze - Canceled @ 12/29/2023, 5:56:24 PM
  (log)   <-- POST /analyze
  (debug) cf-cache-status: HIT [PDF]
  (debug) md5: 263b609960b7b33c154cf0607b207b73
  (debug) cf-cache-status: HIT [JSON]
  (log)   --> POST /analyze 200 24ms

const operation_json = await operation_result.json() as any;

if(operation_json.result === "win") {
  const filename = cacheKey.pathname.split("/").pop() ?? "undefined.pdf";
  const filename_json = filename.replace(/\.[^/.]+$/, "") + ".json";

  const operation_json_str = JSON.stringify(operation_json, null, 2);
  const custom_res = new Response(operation_json_str);

  const compressedReadableStream = custom_res.body!.pipeThrough(
    new CompressionStream('gzip')
  );

  // ...
}

const operation_json = await operation_result.json() as any;

if(operation_json.result === "win") {
  const filename = cacheKey.pathname.split("/").pop() ?? "undefined.pdf";
  const filename_json = filename.replace(/\.[^/.]+$/, "") + ".json";

  const operation_json_str = JSON.stringify(operation_json, null, 2);
  const custom_res = new Response(operation_json_str);

  const compressedReadableStream = custom_res.body!.pipeThrough(
    new CompressionStream('gzip')
  );

  // ...
}