It’s the only part of our stack that isn’t multi-region either.

Ddave It’s the only part of our stack that isn’t multi-region either.

S

sathoro•5/31/23, 3:18 AM

yeah but doesn't really matter since auth code is rarely called compared to everything else

S

sathoro•5/31/23, 3:18 AM

we allow indefinite logins

A

Advany•5/31/23, 8:54 AM

I need to do 10k fetches to api end point. Responses can take up to 15 seconds. What would be the cost efficient way to do it? Add them to the queue and process one by one?

Or startup a DO and let it handle it using alarms after each fetch?

What would be the most cost efficient way?

AAdvany I need to do 10k fetches to api end point. Responses can take up to 15 seconds. ...

S

sathoro•5/31/23, 10:06 AM

using DO as a queue should work well. you would need to do the pricing calculations though. I think we are talking less than a few dollars though (probably much less)...

Z

zegevlier•5/31/23, 10:23 AM

I'm fairly sure 10k requests will fit in the included usage

Ssathoro using DO as a queue should work well. you would need to do the pricing calculati...

A

Advany•5/31/23, 10:58 AM

I just saw that queues require workers to be on the unbound plan. So that means costs of using queue + workers or DO's would be the same

AAdvany I just saw that queues require workers to be on the unbound plan. So that means ...

S

sathoro•5/31/23, 10:59 AM

I suggested DOs based on developer experience. we are talking about pennies anyways so I wasn't too worried about cost

A

Advany•5/31/23, 11:00 AM

yeah, the 10k was only a sample. It will be more like 50 million in production

AAdvany yeah, the 10k was only a sample. It will be more like 50 million in production

S

sathoro•5/31/23, 11:01 AM

yeah then I would definitely use DOs. Queues is a beta product that has low concurrency and additional latency. with DOs you could create unlimited queues for extra concurrency

S

sathoro•5/31/23, 11:02 AM

this is what I do in prod. a DO for each user as a queue. works great

Ssathoro yeah then I would definitely use DOs. Queues is a beta product that has low conc...

A

Advany•5/31/23, 11:02 AM

yeah. I might even go for a non serverless solution. Seems serverless is really expensive for this use case.

AAdvany yeah. I might even go for a non serverless solution. Seems serverless is really ...

S

sathoro•5/31/23, 11:02 AM

I was thinking about suggesting that since you seemed to be hung up on cost and this is something where a $5 VPS would be the most cost effective thing. just depends on your needs

Ssathoro I was thinking about suggesting that since you seemed to be hung up on cost and ...

A

Advany•5/31/23, 11:09 AM

yeah. the client needs a cheaper solution in this case... my calculations seem to amount to around $600 for 25 million requests queued and handled

DO seconds 25m * 15 sec = 375 million seconds
DO price = 375 million seconds * 128mb/1gb * $12.50 = $603

+ storage costs + request costs

I think a $100 vps might do a better job handling the use case, unless my calculations are wrong

AAdvany yeah. the client needs a cheaper solution in this case... my calculations seem t...

S

sathoro•5/31/23, 11:11 AM

that sounds like a lot but that is 4,340 days in processing time lol. that is a heck of a lot of requests especially when they take so long...

A

Advany•5/31/23, 11:12 AM

yeah. It is... need it to prerender sitemaps... and chrome isn't that fast lol

AAdvany yeah. It is... need it to prerender sitemaps... and chrome isn't that fast lol

S

sathoro•5/31/23, 11:13 AM

ah...

Ssirupsen At what point of monthly bandwidth to the cache API does someone from Cloudflare...

H

Hello, I’m Allie!•5/31/23, 11:29 AM

Yeah, CF doesn’t provide an explicit public limit for various reasons. If you are a non-profit/OSS provider, then you might qualify for a discount. Pricing is something you need to contact sales for

H

Hello, I’m Allie!•5/31/23, 11:29 AM

For anything above PAYGO, it won’t be published

Ssathoro ah...

A

Advany•5/31/23, 11:57 AM

maybe I can do 4 requests at a time inside a do or worker customer with parallel primoses... didnt think of that. That should lower the cost to $150 a month...

AAdvany maybe I can do 4 requests at a time inside a do or worker customer with parallel...

S

sathoro•5/31/23, 12:09 PM

yeah you can do like 8 at a time I think

S

sathoro•5/31/23, 12:28 PM

that reminds me, is it normal not to get any replies on limit increase requests? I've done several spanning several weeks and never heard anything

S

sathoro•5/31/23, 12:28 PM

(I'm not spamming them, they are for different limits)

Ssirupsen 👍 I'll try to get on a call with someone from sales

K

kian•5/31/23, 12:37 PM

@rickyrobinett might be able to get you in touch with the right people. I suspect it'll differ based on if you're just using the CDN (i.e not using any other Cloudflare product) or if you're paying for Workers/R2/etc as what's generating that cache(able) content but they can confirm.

R

rickyrobinett•5/31/23, 12:40 PM

Yes, @sirupsen. Happy to help navigate. I’ll DM you.

S

sathoro•5/31/23, 12:51 PM

is the built-in fetch caching supposed to work locally? I'm doing this and it doesn't seem to be caching it (with

--local

--local

--local

--local)

const resp = await fetch(`...`, {
  cf: {
    cacheTtl: 60 * 10,
  },
});

const resp = await fetch(`...`, {
  cf: {
    cacheTtl: 60 * 10,
  },
});

const resp = await fetch(`...`, {
  cf: {
    cacheTtl: 60 * 10,
  },
});

const resp = await fetch(`...`, {
  cf: {
    cacheTtl: 60 * 10,
  },
});

S

sathoro•5/31/23, 1:04 PM

just tested on a real worker and it isn't working either, what am I missing? Hmm

KKartik How many emails can i send using email workers?🧐

S

sathoro•5/31/23, 1:10 PM

aren't those for receiving emails? if you are sending emails you will need to use a third-party service like Postmark (unless you don't care about them all going to spam) and your limits would be between you and them

KKartik I think we can send emails also🤫

D

DaniFoldi•5/31/23, 1:17 PM

you can only send to verified addresses, which should really be just you/team members, to give you notifications of eg. critical errors, or whatever you set it up as

D

DaniFoldi•5/31/23, 1:17 PM

a good example I have is a contact form, you can receive an email to the address you verified, and you can then reply if you'd like to

KKartik Umm, but that depends on the address i have provided in the binding, but if i do...

D

DaniFoldi•5/31/23, 1:21 PM

you can then send to all verified addresses of the account

L

lokiwind•5/31/23, 1:58 PM

catch (error) {
          console.error(`Error while inserting row into accesslog table: ${error}`);
          return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
        }

catch (error) {
          console.error(`Error while inserting row into accesslog table: ${error}`);
          return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
        }

catch (error) {
          console.error(`Error while inserting row into accesslog table: ${error}`);
          return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
        }

catch (error) {
          console.error(`Error while inserting row into accesslog table: ${error}`);
          return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
        }

'error' of type 'unknown'.

L

lokiwind•5/31/23, 1:59 PM

What am I doing wrong?

Llokiwind What am I doing wrong?

H

Hello, I’m Allie!•5/31/23, 2:00 PM

Try

try {
  // ...
} catch (error: any) {
  console.error(`Error while inserting row into accesslog table: ${error}`);
  return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
}

try {
  // ...
} catch (error: any) {
  console.error(`Error while inserting row into accesslog table: ${error}`);
  return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
}

try {
  // ...
} catch (error: any) {
  console.error(`Error while inserting row into accesslog table: ${error}`);
  return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
}

try {
  // ...
} catch (error: any) {
  console.error(`Error while inserting row into accesslog table: ${error}`);
  return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
}

L

lokiwind•5/31/23, 2:01 PM

try {
          await env.DB.prepare(
            "INSERT INTO accesslog (ip, time, path) VALUES (?1, ?2, ?3)"
          ).bind(ip, time, pathname).run();
        } catch (error) {
          console.error(`Error while inserting row into accesslog table: ${error}`);
          return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
        }
      }

try {
          await env.DB.prepare(
            "INSERT INTO accesslog (ip, time, path) VALUES (?1, ?2, ?3)"
          ).bind(ip, time, pathname).run();
        } catch (error) {
          console.error(`Error while inserting row into accesslog table: ${error}`);
          return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
        }
      }

try {
          await env.DB.prepare(
            "INSERT INTO accesslog (ip, time, path) VALUES (?1, ?2, ?3)"
          ).bind(ip, time, pathname).run();
        } catch (error) {
          console.error(`Error while inserting row into accesslog table: ${error}`);
          return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
        }
      }

try {
          await env.DB.prepare(
            "INSERT INTO accesslog (ip, time, path) VALUES (?1, ?2, ?3)"
          ).bind(ip, time, pathname).run();
        } catch (error) {
          console.error(`Error while inserting row into accesslog table: ${error}`);
          return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
        }
      }

@HardAtWork

Llokiwind ```js try { await env.DB.prepare( "INSERT INTO accesslog (...

H

Hello, I’m Allie!•5/31/23, 2:01 PM

Yeah, so add the

any

any

any

any type, and see if it works.

Llokiwind ```js try { await env.DB.prepare( "INSERT INTO accesslog (...

L

lokiwind•5/31/23, 2:02 PM

The code is now like this

L

lokiwind•5/31/23, 2:02 PM

But error :'error' of type 'unknown'.

Llokiwind The code is now like this

H

Hello, I’m Allie!•5/31/23, 2:03 PM

Yeah, so add the

any

any

any

any type to the catch block like so, and see if it stops throwing that error.

try {
    await env.DB.prepare(
      "INSERT INTO accesslog (ip, time, path) VALUES (?1, ?2, ?3)"
    ).bind(ip, time, pathname).run();
  } catch (error: any) {
    console.error(`Error while inserting row into accesslog table: ${error}`);
    return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
  }
}

try {
    await env.DB.prepare(
      "INSERT INTO accesslog (ip, time, path) VALUES (?1, ?2, ?3)"
    ).bind(ip, time, pathname).run();
  } catch (error: any) {
    console.error(`Error while inserting row into accesslog table: ${error}`);
    return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
  }
}

try {
    await env.DB.prepare(
      "INSERT INTO accesslog (ip, time, path) VALUES (?1, ?2, ?3)"
    ).bind(ip, time, pathname).run();
  } catch (error: any) {
    console.error(`Error while inserting row into accesslog table: ${error}`);
    return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
  }
}

try {
    await env.DB.prepare(
      "INSERT INTO accesslog (ip, time, path) VALUES (?1, ?2, ?3)"
    ).bind(ip, time, pathname).run();
  } catch (error: any) {
    console.error(`Error while inserting row into accesslog table: ${error}`);
    return Response.json({ message: `Error while inserting row into accesslog table: ${error.message}` });
  }
}

L

lokiwind•5/31/23, 2:04 PM

Thank you I will try

L

lokiwind•5/31/23, 2:07 PM

Problem solved thanks @HardAtWork

M

Murder Chicken•5/31/23, 4:04 PM

Random question. Has anyone found an HTML minifier that works in a CF worker? Most of them fail with missing dependencies like:

  node_modules/clean-css/lib/options/format.js:1:30: ERROR: Could not resolve "os"
  node_modules/clean-css/lib/options/inline-request.js:1:18: ERROR: Could not resolve "url"
  node_modules/clean-css/lib/options/rebase-to.js:1:19: ERROR: Could not resolve "path"
  node_modules/clean-css/lib/reader/apply-source-maps.js:1:17: ERROR: Could not resolve "fs"
  node_modules/clean-css/lib/reader/apply-source-maps.js:2:19: ERROR: Could not resolve "path"

  node_modules/clean-css/lib/options/format.js:1:30: ERROR: Could not resolve "os"
  node_modules/clean-css/lib/options/inline-request.js:1:18: ERROR: Could not resolve "url"
  node_modules/clean-css/lib/options/rebase-to.js:1:19: ERROR: Could not resolve "path"
  node_modules/clean-css/lib/reader/apply-source-maps.js:1:17: ERROR: Could not resolve "fs"
  node_modules/clean-css/lib/reader/apply-source-maps.js:2:19: ERROR: Could not resolve "path"

  node_modules/clean-css/lib/options/format.js:1:30: ERROR: Could not resolve "os"
  node_modules/clean-css/lib/options/inline-request.js:1:18: ERROR: Could not resolve "url"
  node_modules/clean-css/lib/options/rebase-to.js:1:19: ERROR: Could not resolve "path"
  node_modules/clean-css/lib/reader/apply-source-maps.js:1:17: ERROR: Could not resolve "fs"
  node_modules/clean-css/lib/reader/apply-source-maps.js:2:19: ERROR: Could not resolve "path"

  node_modules/clean-css/lib/options/format.js:1:30: ERROR: Could not resolve "os"
  node_modules/clean-css/lib/options/inline-request.js:1:18: ERROR: Could not resolve "url"
  node_modules/clean-css/lib/options/rebase-to.js:1:19: ERROR: Could not resolve "path"
  node_modules/clean-css/lib/reader/apply-source-maps.js:1:17: ERROR: Could not resolve "fs"
  node_modules/clean-css/lib/reader/apply-source-maps.js:2:19: ERROR: Could not resolve "path"

It’s the only part of our stack that isn’t multi-region either.

Similar Threads