2 replies

Caching NextJS app on worker has long KV cache put times

Hey everyone, we're implementing a custom HTML caching layer in a Worker and running into an unexpected blocking issue that seems to contradict the documentation.

We're using

ctx.waitUntil()

ctx.waitUntil()

for a background KV write, but the operation is blocking the client response.

The Problem

We're seeing two main issues in our traces:
1. High Latency Write: Our

kv_put

kv_put

operation takes about 900ms to write a large HTML document into the cache.
2. Blocking Request: This 900ms latency is blocking the total request time for the client, delaying the

return out

return out

statement.

Our Code & Confusion

My understanding is that ctx.waitUntil(Promise)
ctx.waitUntil(Promise)
should allow the main

fetch

fetch

handler to return immediately while the Promise resolves in the background.

    // we forward the request to the actual nextjs worker since its not cached
    let res = await nextWorker.fetch(req, env, ctx)

    if (res.ok && res.body) {
      // First, check for compression and decompress if necessary
      if (res.headers.get("Content-Encoding")) {
        res = new Response(res.body, res)
        res.headers.delete("Content-Encoding")
      }
      // since nextjs streams, we are tee-ing the response to return one part and cache the other copy
      const [toClient, toCache] = res.body.tee()

// We are constructing the response to the client
const out = new Response(toClient, {
    status: res.status,
    statusText: res.statusText,
    headers: clientHeaders,
})

// We perform the background cache insert
ctx.waitUntil(
    this.cacheResponse(env, cacheKey, toCache, res.status, cacheHeaders)
)

// We expect to return to the client immediately here
return out

    // we forward the request to the actual nextjs worker since its not cached
    let res = await nextWorker.fetch(req, env, ctx)

    if (res.ok && res.body) {
      // First, check for compression and decompress if necessary
      if (res.headers.get("Content-Encoding")) {
        res = new Response(res.body, res)
        res.headers.delete("Content-Encoding")
      }
      // since nextjs streams, we are tee-ing the response to return one part and cache the other copy
      const [toClient, toCache] = res.body.tee()

// We are constructing the response to the client
const out = new Response(toClient, {
    status: res.status,
    statusText: res.statusText,
    headers: clientHeaders,
})

// We perform the background cache insert
ctx.waitUntil(
    this.cacheResponse(env, cacheKey, toCache, res.status, cacheHeaders)
)

// We expect to return to the client immediately here
return out

I think the main issue is how

tee

tee

works and that it awaits both streamed forks of the reply to be finished (or awaited) before it returns. That is why we see these crazy latencies because our caching system then puts it into the KV cache but it seems like it's awaited before we return to the client

Cloudflare Developers•4mo ago•

2 replies

josula

Caching NextJS app on worker has long KV cache put times

Hey everyone, we're implementing a custom HTML caching layer in a Worker and running into an unexpected blocking issue that seems to contradict the documentation.

We're using

ctx.waitUntil()

ctx.waitUntil()

for a background KV write, but the operation is blocking the client response.

The Problem

We're seeing two main issues in our traces:
1. High Latency Write: Our

kv_put

kv_put

operation takes about 900ms to write a large HTML document into the cache.
2. Blocking Request: This 900ms latency is blocking the total request time for the client, delaying the

return out

return out

statement.

Our Code & Confusion

My understanding is that ctx.waitUntil(Promise)
ctx.waitUntil(Promise)
should allow the main

fetch

fetch

handler to return immediately while the Promise resolves in the background.

    // we forward the request to the actual nextjs worker since its not cached
    let res = await nextWorker.fetch(req, env, ctx)

    if (res.ok && res.body) {
      // First, check for compression and decompress if necessary
      if (res.headers.get("Content-Encoding")) {
        res = new Response(res.body, res)
        res.headers.delete("Content-Encoding")
      }
      // since nextjs streams, we are tee-ing the response to return one part and cache the other copy
      const [toClient, toCache] = res.body.tee()

// We are constructing the response to the client
const out = new Response(toClient, {
    status: res.status,
    statusText: res.statusText,
    headers: clientHeaders,
})

// We perform the background cache insert
ctx.waitUntil(
    this.cacheResponse(env, cacheKey, toCache, res.status, cacheHeaders)
)

// We expect to return to the client immediately here
return out

    // we forward the request to the actual nextjs worker since its not cached
    let res = await nextWorker.fetch(req, env, ctx)

    if (res.ok && res.body) {
      // First, check for compression and decompress if necessary
      if (res.headers.get("Content-Encoding")) {
        res = new Response(res.body, res)
        res.headers.delete("Content-Encoding")
      }
      // since nextjs streams, we are tee-ing the response to return one part and cache the other copy
      const [toClient, toCache] = res.body.tee()

// We are constructing the response to the client
const out = new Response(toClient, {
    status: res.status,
    statusText: res.statusText,
    headers: clientHeaders,
})

// We perform the background cache insert
ctx.waitUntil(
    this.cacheResponse(env, cacheKey, toCache, res.status, cacheHeaders)
)

// We expect to return to the client immediately here
return out

I think the main issue is how

tee

tee

Caching NextJS app on worker has long KV cache put times

The Problem

Our Code & Confusion

Caching NextJS app on worker has long KV cache put times

The Problem

Our Code & Confusion

Similar Threads

Similar Threads

Similar Threads