FYI requests are fully buffered by default. The request is not sent to your server before the client

Aasp FYI requests are fully buffered by default. The request is not sent to your serv...

D

dave•5/26/23, 7:44 PM

oh interesting, didn't know uploads were different than downloads!

Ddave oh interesting, didn't know uploads were different than downloads!

A

aspOP•5/26/23, 7:45 PM

Downloads aren't buffered unless you are on Enterprise and enable Response Buffering.

A

aspOP•5/26/23, 7:45 PM

The reason for buffering requests is so WAF can run on the request body.

Aasp The reason for buffering requests is so WAF can run on the request body.

D

dave•5/26/23, 7:45 PM

Ahhh yes, that makes sense!

D

dave•5/26/23, 7:45 PM

that also sorta makes sense why there's a 100MB limit

A

aspOP•5/26/23, 7:50 PM

Usually, disabling request buffering is only done for Enterprise customers, but I believe - though don't quote me on this - that PAYGO customers can get it disabled if they ask nicely and are lucky. There's nothing about unbuffered uploads that inherently makes it Enterprise only. If anything, it reduces load on Cloudflare.

H

Harris•5/26/23, 9:17 PM

I'd like to create multiple workers within the same project, is the wrangler.toml a 1:1 relationship per worker?

J

James•5/26/23, 9:21 PM

It is, yes. You'd almost certainly want a separate wangler.toml per worker

H

Harris•5/26/23, 9:24 PM

Gotcha! thanks

H

Harris•5/26/23, 9:27 PM

Is it possible to have a worker explicitly execute in 3 different locations at the same time? I'm creating a dashboard that's monitoring a public API and would like to account for geo differences

J

James•5/26/23, 9:30 PM

Not really, no. Workers deploy everywhere. If you have 3 VPSes in those regions, you could hit the worker from those locations.

With Smart Placement (https://developers.cloudflare.com/workers/platform/smart-placement/), the Worker will move to a specific region if it makes sense (due to hitting a regional DB multiple timese for example), but that's also not really what you want it sounds like.

J

James•5/26/23, 9:30 PM

Cloudflare has a specific healthchecks product though that you can use to hit stuff from specific regions - perhaps that would be more suited?

HHarris Is it possible to have a worker explicitly execute in 3 different locations at t...

H

Hello, I’m Allie!•5/26/23, 9:30 PM

You can also use a DO(with some trial and error), or a Load Balancer Health Check(might be Enterprise only)

H

Hello, I’m Allie!•5/26/23, 9:31 PM

What James said

H

Harris•5/26/23, 9:31 PM

ah so i'm just doing this as a personal project, and it's not actually my API

H

Harris•5/26/23, 9:31 PM

I

Ian•5/26/23, 9:32 PM

Any way to point dynamic subdomains to a worker ?

C

Chaika•5/26/23, 9:32 PM

You can use Durable Object locationHints as well, but the regions are pretty large

IIan Any way to point dynamic subdomains to a worker ?

C

Chaika•5/26/23, 9:33 PM

You mean like wildcard? You can use HTTP Routes and a proxied wildcard, (ex:

*.example.com/*

*.example.com/*

or

*/*

*/*

worker route,

AAAA

AAAA

*.example.com

*.example.com

100::

100::

Proxied

Proxied

record)

JJames Not really, no. Workers deploy everywhere. If you have 3 VPSes in those regions,...

H

Harris•5/26/23, 9:33 PM

Smart placement isn't available for cron trigger i believe

CChaika You mean like wildcard? You can use HTTP Routes and a proxied wildcard, (ex: `*....

I

Ian•5/26/23, 9:33 PM

it doesn't seem possible for workers

HHarris Smart placement isn't available for cron trigger i believe

J

James•5/26/23, 9:33 PM

Yeah they don't work together yet unfortunately

IIan it doesn't seem possible for workers

C

Chaika•5/26/23, 9:33 PM

For Workers? 100% it is, I have a worker just like that. You just have to use Http Routes and not Custom Domains

I

Ian•5/26/23, 9:34 PM

Interesting, let me check

C

Chaika•5/26/23, 9:34 PM

Also for each domain you don't want to be controlled by the worker, you need to add a seperate route with a service of none

C

Chaika•5/26/23, 9:35 PM

You'd end up with something like:

api.example.com/*

api.example.com/*

: service none (for normal origins)

*/*

*/*

to match all or

*.example.com/*

*.example.com/*

to match all subdomains besides apex, service: Worker

C

Chaika•5/26/23, 9:36 PM

(It may be worth keeping in mind as well the default universal cert you get only covers first level subdomains, while your worker could respond on gifhigfohgfd.workers.example.com, your cert won't cover it unless you use ACM for

*.workers.example.com

*.workers.example.com

)

H

Harris•5/26/23, 9:47 PM

Question on limits:

I'm using the Unbounded Usage Model, but my worker sends a HTTPS request that has a streaming body, and can take longer than 30 seconds, however my worker runs every 5 minutes. How do these limits interact?

K

kian•5/26/23, 9:49 PM

30 seconds is CPU time

K

kian•5/26/23, 9:49 PM

Streaming a body from a

fetch

fetch

subrequest isn't going to use up any CPU

H

Harris•5/26/23, 9:49 PM

even if I am using the main function to log something like time to first byte?

H

Harris•5/26/23, 9:52 PM

    const response = await fetch(url, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${env.OPENAI_API_KEY}`,
      },
      body,
    })
      .then((res) => response.body)
      .then((body) => {
        const reader = body.getReader()

        // do stuff with reader
      })

    const response = await fetch(url, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${env.OPENAI_API_KEY}`,
      },
      body,
    })
      .then((res) => response.body)
      .then((body) => {
        const reader = body.getReader()

        // do stuff with reader
      })

    const response = await fetch(url, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${env.OPENAI_API_KEY}`,
      },
      body,
    })
      .then((res) => response.body)
      .then((body) => {
        const reader = body.getReader()

        // do stuff with reader
      })

    const response = await fetch(url, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${env.OPENAI_API_KEY}`,
      },
      body,
    })
      .then((res) => response.body)
      .then((body) => {
        const reader = body.getReader()

        // do stuff with reader
      })

K

kian•5/26/23, 9:56 PM

What is happening in

do stuff with reader

do stuff with reader

?

K

kian•5/26/23, 9:57 PM

It'll probably use up CPU time, but I doubt it's anywhere near 30 seconds

H

Harris•5/26/23, 10:03 PM

const startTime = Date.now()

    const response = await fetch(url, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${env.OPENAI_API_KEY}`,
      },
      body,
    })

    if (!response.ok) {
      console.log(
        `Unexpected response from OpenAI API: ${response.status} ${response.statusText}`
      )
      return
    }

    if (!response.body) {
      console.log("No response body")
      return
    }
    const reader = response.body.getReader()

    // Get first byte
    let chunk = await reader.read()

    // Log first byte time
    const firstByteTime = Date.now() - startTime

    // Read the rest of the response
    while (!chunk.done) {
      chunk = await reader.read()
    }

const startTime = Date.now()

    const response = await fetch(url, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${env.OPENAI_API_KEY}`,
      },
      body,
    })

    if (!response.ok) {
      console.log(
        `Unexpected response from OpenAI API: ${response.status} ${response.statusText}`
      )
      return
    }

    if (!response.body) {
      console.log("No response body")
      return
    }
    const reader = response.body.getReader()

    // Get first byte
    let chunk = await reader.read()

    // Log first byte time
    const firstByteTime = Date.now() - startTime

    // Read the rest of the response
    while (!chunk.done) {
      chunk = await reader.read()
    }

const startTime = Date.now()

    const response = await fetch(url, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${env.OPENAI_API_KEY}`,
      },
      body,
    })

    if (!response.ok) {
      console.log(
        `Unexpected response from OpenAI API: ${response.status} ${response.statusText}`
      )
      return
    }

    if (!response.body) {
      console.log("No response body")
      return
    }
    const reader = response.body.getReader()

    // Get first byte
    let chunk = await reader.read()

    // Log first byte time
    const firstByteTime = Date.now() - startTime

    // Read the rest of the response
    while (!chunk.done) {
      chunk = await reader.read()
    }

const startTime = Date.now()

    const response = await fetch(url, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
        Authorization: `Bearer ${env.OPENAI_API_KEY}`,
      },
      body,
    })

    if (!response.ok) {
      console.log(
        `Unexpected response from OpenAI API: ${response.status} ${response.statusText}`
      )
      return
    }

    if (!response.body) {
      console.log("No response body")
      return
    }
    const reader = response.body.getReader()

    // Get first byte
    let chunk = await reader.read()

    // Log first byte time
    const firstByteTime = Date.now() - startTime

    // Read the rest of the response
    while (!chunk.done) {
      chunk = await reader.read()
    }

H

Harris•5/26/23, 10:03 PM

the event stream might not finish within 30s, and this code would be executing on the main thread, wouldn't it?

K

kian•5/26/23, 10:04 PM

the event stream might not finish within 30s

It's 30 seconds of CPU time, none of this is particularly heavy compute.

H

Harris•5/26/23, 10:04 PM

oooh i see

H

Harris•5/26/23, 10:04 PM

basically CPU time = time to process

K

kian•5/26/23, 10:04 PM

You could just do your TTFB after the

const response = await ...

const response = await ...

and then

return response

return response

, couldn't you?

H

Harris•5/26/23, 10:04 PM

there's more code underneath

H

Harris•5/26/23, 10:05 PM

 const totalTime = Date.now() - startTime

    console.log("Time to first byte: " + firstByteTime + "ms")
    console.log("Total response time: " + totalTime + "ms")

    // Log response time
    const client = new Client(env.DB_URL + "?sslmode=require")
    await client.connect()

    const text =
      "INSERT INTO response_times(model, date, ttfb, duration) VALUES($1, $2, $3, $4) RETURNING *"
    const values = [
      "gpt-4",
      new Date(event.scheduledTime),
      firstByteTime,
      totalTime,
    ]

    try {
      const res = await client.query(text, values)
      console.log(res.rows[0])
    } catch (err) {
      console.error(err)
    } finally {
      ctx.waitUntil(client.end())
    }

 const totalTime = Date.now() - startTime

    console.log("Time to first byte: " + firstByteTime + "ms")
    console.log("Total response time: " + totalTime + "ms")

    // Log response time
    const client = new Client(env.DB_URL + "?sslmode=require")
    await client.connect()

    const text =
      "INSERT INTO response_times(model, date, ttfb, duration) VALUES($1, $2, $3, $4) RETURNING *"
    const values = [
      "gpt-4",
      new Date(event.scheduledTime),
      firstByteTime,
      totalTime,
    ]

    try {
      const res = await client.query(text, values)
      console.log(res.rows[0])
    } catch (err) {
      console.error(err)
    } finally {
      ctx.waitUntil(client.end())
    }

 const totalTime = Date.now() - startTime

    console.log("Time to first byte: " + firstByteTime + "ms")
    console.log("Total response time: " + totalTime + "ms")

    // Log response time
    const client = new Client(env.DB_URL + "?sslmode=require")
    await client.connect()

    const text =
      "INSERT INTO response_times(model, date, ttfb, duration) VALUES($1, $2, $3, $4) RETURNING *"
    const values = [
      "gpt-4",
      new Date(event.scheduledTime),
      firstByteTime,
      totalTime,
    ]

    try {
      const res = await client.query(text, values)
      console.log(res.rows[0])
    } catch (err) {
      console.error(err)
    } finally {
      ctx.waitUntil(client.end())
    }

 const totalTime = Date.now() - startTime

    console.log("Time to first byte: " + firstByteTime + "ms")
    console.log("Total response time: " + totalTime + "ms")

    // Log response time
    const client = new Client(env.DB_URL + "?sslmode=require")
    await client.connect()

    const text =
      "INSERT INTO response_times(model, date, ttfb, duration) VALUES($1, $2, $3, $4) RETURNING *"
    const values = [
      "gpt-4",
      new Date(event.scheduledTime),
      firstByteTime,
      totalTime,
    ]

    try {
      const res = await client.query(text, values)
      console.log(res.rows[0])
    } catch (err) {
      console.error(err)
    } finally {
      ctx.waitUntil(client.end())
    }

HHarris basically CPU time = time to process

I

Isaac McFadyen•5/26/23, 10:09 PM

CPU time = the time an operation takes on the CPU

I

Isaac McFadyen•5/26/23, 10:09 PM

It's not related to real time at all, really

I

Isaac McFadyen•5/26/23, 10:10 PM

A CPU intensive operation taking 1 second could use up to 1 second of CPU time (ignoring Workers limits of course), but that would mean it's using 100% of the CPU for 100% of the time which never happens, so it's usually far less

I

Isaac McFadyen•5/26/23, 10:10 PM

Fetch requests take a few ms of CPU time, for example, even though they might wait far longer for a response from wherever you're fetching from

H

Harris•5/26/23, 10:12 PM

makes sense, thank you

N

nikiv.dev•5/26/23, 11:01 PM

does this mean my variables i set in dashboard will be overriden?

N

nikiv.dev•5/26/23, 11:01 PM

my config is just this

N

nikiv.dev•5/26/23, 11:03 PM

FYI requests are fully buffered by default. The request is not sent to your server before the client

Similar Threads

Similar Threads

Similar Threads