Did you read the body before you transform it?

HHello, I’m Allie!You can use a bundler with Functions, right?

I

Isaac McFadyen•5/31/22, 7:33 PM

You can but it negates all of what Functions does, and the command for it is pretty undocumented.

I

Isaac McFadyen•5/31/22, 7:34 PM

(Advanced Mode +

wrangler pages build

wrangler pages build

wrangler pages build

wrangler pages build I think? And then you have to call

next()

next()

next()

next() yourself and it's a mess rn)

HHello, I’m Allie!Did you read the body before you transform it?

A

amorfati•5/31/22, 7:34 PM

  const oldResponse = await next()
  const contentType = oldResponse.headers.get('Content-Type')
  if (contentType && contentType.startsWith('text/html')) {
    const newResponse = handler.transform(oldResponse)
    return oldResponse
  }

  const oldResponse = await next()
  const contentType = oldResponse.headers.get('Content-Type')
  if (contentType && contentType.startsWith('text/html')) {
    const newResponse = handler.transform(oldResponse)
    return oldResponse
  }

  const oldResponse = await next()
  const contentType = oldResponse.headers.get('Content-Type')
  if (contentType && contentType.startsWith('text/html')) {
    const newResponse = handler.transform(oldResponse)
    return oldResponse
  }

  const oldResponse = await next()
  const contentType = oldResponse.headers.get('Content-Type')
  if (contentType && contentType.startsWith('text/html')) {
    const newResponse = handler.transform(oldResponse)
    return oldResponse
  }

A

amorfati•5/31/22, 7:35 PM

It happens when returning the old response. I'm just using new response for logging

Aamorfati It happens when returning the old response. I'm just using new response for logg...

H

Hello, I’m Allie!OP•5/31/22, 7:35 PM

Once a Response is read, you cannot read it again. If you transform oldResponse, you cannot return it

A

amorfati•5/31/22, 7:36 PM

gotcha

EElfhild Is D1 planned to work on functions too? do you know?

W

Walshy•5/31/22, 7:37 PM

Of course. We will always support all the bindings Workers do. It's a little more complex so there is a delay but we will add it

A

amorfati•5/31/22, 7:38 PM

my end goal is to read the html, find a specific tag with id "article", inject new element under it. When I read the html, it always says "undefined" for the innerHTML even when I use *.

A

amorfati•5/31/22, 7:38 PM

Line 39 here always logs undefined:

K

kian•5/31/22, 7:40 PM

https://developers.cloudflare.com/workers/runtime-apis/html-rewriter/#element

HTMLRewriter · Cloudflare Workers docs

Documentation for Cloudflare Workers, a serverless execution environment that allows you to create entirely new applications or augment existing ones without configuring or maintaining infrastructure.

A

amorfati•5/31/22, 7:40 PM

when I log

console.log('Incoming element: ', JSON.stringify(element, null, 2))

console.log('Incoming element: ', JSON.stringify(element, null, 2))

console.log('Incoming element: ', JSON.stringify(element, null, 2))

console.log('Incoming element: ', JSON.stringify(element, null, 2))

I get:

Incoming element:  {
  "ptr": 1298176
}

Incoming element:  {
  "ptr": 1298176
}

Incoming element:  {
  "ptr": 1298176
}

Incoming element:  {
  "ptr": 1298176
}

K

kian•5/31/22, 7:40 PM

innerHTML doesn’t seem to be a property of element

A

amorfati•5/31/22, 7:43 PM

the first argument of

on()

on()

on()

on() is what the rewriter will read, everything else ignored. So if I do

HTMLRewriter().on('*', new ElementHandler())

HTMLRewriter().on('*', new ElementHandler())

HTMLRewriter().on('*', new ElementHandler())

HTMLRewriter().on('*', new ElementHandler()) then filter for element with tagName "article" inside the

ElementHandler

ElementHandler

ElementHandler

ElementHandler class, it's the same as

HTMLRewriter().on('article', new ElementHandler())

HTMLRewriter().on('article', new ElementHandler())

HTMLRewriter().on('article', new ElementHandler())

HTMLRewriter().on('article', new ElementHandler())?

A

amorfati•5/31/22, 7:44 PM

it's more computation passing

*

* then filtering (is my understanding)

Kkian innerHTML doesn’t seem to be a property of element

A

amorfati•5/31/22, 7:49 PM

thanks, this helps. I think the rest of my problems is the fact that the middleware reader reads the html before the content has been fully loaded.

N

nitsky•5/31/22, 8:07 PM

Hi, is there any support for Rust+WASM in functions?

Ggarrison My function is somehow causing an Error 1101: Worker threw exception, but I don'...

J

James•6/1/22, 2:00 AM

Unfortunately there aren’t any error logs for Functions yet, sorry!

Does your function work locally with pages dev? If so, could you possibly share it here so we can try and help you diagnose why it’s not working in production?

Ggarrison Thanks for the info! I actually don't even know how to test it locally, but I wi...

J

James•6/1/22, 2:04 AM

I’d definitely recommend testing locally first then: https://developers.cloudflare.com/pages/platform/functions/#develop-and-preview-locally
That should give you a pretty good representation of how things will run and should let you diagnose any major issues

Functions (beta) · Cloudflare Pages docs

Documentation for Cloudflare Pages, the best way to deploy your static and JAMstack sites

Ggarrison I'm running it locally. Where should I expect an error log to appear? (In the ...

J

James•6/1/22, 2:13 AM

Console, if there are any errors

J

James•6/1/22, 2:13 AM

Is it working fine locally?

J

James•6/1/22, 2:15 AM

Can you share the function code/repo? That seems odd

Ggarrison https://github.com/garrison/cloudflare-pages-shared-password/blob/main/functions...

J

James•6/1/22, 2:22 AM

Thanks. Couple of immediate things: the errors you're throwing aren't really errors, so they're being logged weirdly - that's why you're seeing the

object Object

object Object

object Object

object Object logs

J

James•6/1/22, 2:22 AM

Replace your errors with something like this (haven't solved any typescript things):

class UnauthorizedException extends Error {
  constructor(reason) {
    super(reason);
    this.status = 401;
    this.statusText = 'Unauthorized';
    this.reason = reason;
  }
}

class BadRequestException extends Error {
  constructor(reason) {
    super(reason);
    this.status = 400;
    this.statusText = 'Bad Request';
    this.reason = reason;
  }
}

class UnauthorizedException extends Error {
  constructor(reason) {
    super(reason);
    this.status = 401;
    this.statusText = 'Unauthorized';
    this.reason = reason;
  }
}

class BadRequestException extends Error {
  constructor(reason) {
    super(reason);
    this.status = 400;
    this.statusText = 'Bad Request';
    this.reason = reason;
  }
}

class UnauthorizedException extends Error {
  constructor(reason) {
    super(reason);
    this.status = 401;
    this.statusText = 'Unauthorized';
    this.reason = reason;
  }
}

class BadRequestException extends Error {
  constructor(reason) {
    super(reason);
    this.status = 400;
    this.statusText = 'Bad Request';
    this.reason = reason;
  }
}

class UnauthorizedException extends Error {
  constructor(reason) {
    super(reason);
    this.status = 401;
    this.statusText = 'Unauthorized';
    this.reason = reason;
  }
}

class BadRequestException extends Error {
  constructor(reason) {
    super(reason);
    this.status = 400;
    this.statusText = 'Bad Request';
    this.reason = reason;
  }
}

J

James•6/1/22, 2:23 AM

Then you'll get more expected errors I imagine, like I see

[pages:err] GET /: Error: Please use a HTTPS connection.

[pages:err] GET /: Error: Please use a HTTPS connection.

[pages:err] GET /: Error: Please use a HTTPS connection.

[pages:err] GET /: Error: Please use a HTTPS connection. for example.

J

James•6/1/22, 2:24 AM

If I comment out the

https

https

https

https detection stuff real quick, I can get through the basic auth without issue

J

James•6/1/22, 2:26 AM

Otherwise this looks good to me!

J

James•6/1/22, 2:26 AM

Oh wait, I see another potential issue that's super nuanced, one sec

J

James•6/1/22, 2:28 AM

const response = await next();
response.headers.set('Cache-Control', 'no-store');
return response;

const response = await next();
response.headers.set('Cache-Control', 'no-store');
return response;

const response = await next();
response.headers.set('Cache-Control', 'no-store');
return response;

const response = await next();
response.headers.set('Cache-Control', 'no-store');
return response;

^ This code will likely work in dev, but not in prod. In dev, headers from effective

fetch

fetch

requests can be mutable, whereas in prod they're not. There's various issues to resolve this (and I think it's mostly resolved now), but for safety you may want to clone the response before mutating the headers, like this:

const response = await next();
const newResponse = response.clone();
newResponse.headers.set('Cache-Control', 'no-store');
return newResponse;

const response = await next();
const newResponse = response.clone();
newResponse.headers.set('Cache-Control', 'no-store');
return newResponse;

const response = await next();
const newResponse = response.clone();
newResponse.headers.set('Cache-Control', 'no-store');
return newResponse;

const response = await next();
const newResponse = response.clone();
newResponse.headers.set('Cache-Control', 'no-store');
return newResponse;

J

James•6/1/22, 2:28 AM

Lemme know if all of that gets you any further along!

JJames ```js const response = await next(); response.headers.set('Cache-Control', 'no-s...

I

Isaac McFadyen•6/1/22, 2:29 AM

I don't think clone works, I think you need to do

new Response(response.body, response)

new Response(response.body, response)

new Response(response.body, response)

new Response(response.body, response)?

I

Isaac McFadyen•6/1/22, 2:29 AM

Maybe that's just for Request though.

J

James•6/1/22, 2:31 AM

Clone should work afaik, but that's another great tip if it doesn't

J

James•6/1/22, 2:32 AM

Once https://github.com/cloudflare/miniflare/commit/1599c75c087f04fcc1a969b93588d802052cf629 lands in wrangler2 though, things should hopefully be more consistent in prod vs dev

Ggarrison Yup, replaced clone() with what was suggested above, and now it works.

J

James•6/1/22, 3:07 AM

Glad to hear it

A

amorfati•6/1/22, 6:49 AM

in the code sample in this screenshot, if we were to apply the KV in the shown KV namespace bindings, would it be:

npx wrangler pages dev dist --kv appleData

npx wrangler pages dev dist --kv appleData

npx wrangler pages dev dist --kv appleData

npx wrangler pages dev dist --kv appleData

or

npx wrangler pages dev dist --kv APPLE_DISTRIBUTION

npx wrangler pages dev dist --kv APPLE_DISTRIBUTION

npx wrangler pages dev dist --kv APPLE_DISTRIBUTION

npx wrangler pages dev dist --kv APPLE_DISTRIBUTION

then:

await env.appleData.get('APPLE_DISTRIBUTION');

await env.appleData.get('APPLE_DISTRIBUTION');

await env.appleData.get('APPLE_DISTRIBUTION');

await env.appleData.get('APPLE_DISTRIBUTION');

or

await env.APPLE_DISTRIBUTION.get('...');

await env.APPLE_DISTRIBUTION.get('...');

await env.APPLE_DISTRIBUTION.get('...');

await env.APPLE_DISTRIBUTION.get('...');

or something else?

W

Walshy•6/1/22, 9:08 AM

number 2 and number 2

I

Isaac McFadyen•6/1/22, 3:03 PM

@jbergstroem are you seeing the same issue with any other colos?

J

jbergstroem•6/1/22, 3:05 PM

So far, I've tried from

7148cf4558a76f2f-SCL

7148cf4558a76f2f-SCL

7148cf4558a76f2f-SCL

7148cf4558a76f2f-SCL (boo),

7148c9e2de03ab38-EZE

7148c9e2de03ab38-EZE

7148c9e2de03ab38-EZE

7148c9e2de03ab38-EZE (yay) and

7148c2165e46694b-FRA

7148c2165e46694b-FRA

7148c2165e46694b-FRA

7148c2165e46694b-FRA (yay)

J

jbergstroem•6/1/22, 3:06 PM

We've also seen worker errors from

-SCL

-SCL

-SCL

-SCL; here's a ray id that threw a worker error (1101):

7148a5d07dc62ded

7148a5d07dc62ded

7148a5d07dc62ded

7148a5d07dc62ded

I

Isaac McFadyen•6/1/22, 3:06 PM

CC @walshydev a

_worker.js

_worker.js

_worker.js

_worker.js seems to be returning an empty 200 response in certain colos only (same deployment), Ray IDs are above.

Did you read the body before you transform it?

Similar Threads

Similar Threads

Similar Threads