I am trying to test locally getting a file from R2 bucket which i added via dashboard

I created a worker cron job and am testing locally via wrangler dev --test-scheduled. calling await env.MY_BUCKET.get('myfile.json'); returns null even though the file exists in my R2 bucket dashboard. Why is this the case?

Ff8 I created a worker cron job and am testing locally via wrangler dev --test-sched...

Chaika•5/31/24, 6:28 PM

default dev is local/all simulated, have to use --remote if you want to use remote resources

CChaika default dev is local/all simulated, have to use --remote if you want to use remo...

f8OP•5/31/24, 6:34 PM

Is there way to manually add files to r2 simulated bucket via dashboard?

It wont let me use production one:

In development, you should use a separate r2 bucket than the one you'd use in production. Please create a new r2 bucket with "wrangler r2 bucket create <name>" and add its name as preview_bucket_name to the r2_buckets "MY_BUCKET" in your wrangler.toml

Ff8 Is there way to manually add files to r2 simulated bucket via dashboard? It wo...

Chaika•5/31/24, 6:38 PM

The whole thing with the simulated bucket is that it's all local

Chaika•5/31/24, 6:38 PM

so nothing in the dashboard is going to be able to touch your local resources

f8OP•5/31/24, 6:39 PM

Thanks

CChaika The whole thing with the simulated bucket is that it's all local

f8OP•6/1/24, 12:27 AM

Im getting a 504 Gateway Time-out cloudflare workers for my cron job at 120214ms. I have no cpu limit set and paying for the standard plan

Walshy•6/1/24, 12:29 AM

What are you doing in the Worker?

WWalshy What are you doing in the Worker?

f8OP•6/1/24, 12:30 AM

Making hundreds of requests to a third party api which im using that fetched data from those api requests and putting them in a json which at end i will push to R2

Walshy•6/1/24, 12:30 AM

and where you seeing the 504?

i'm not surprised you see it if you fetch anything - after 100 seconds (which you're doing 120 total) it'll return that

Walshy•6/1/24, 12:30 AM

yeah so my guess - that third party api is timing out

Walshy•6/1/24, 12:31 AM

you can debug to confirm but that's my theory

f8OP•6/1/24, 12:31 AM

Getting 504 in console after running npx wrangler dev --test-scheduled --remote

f8OP•6/1/24, 12:31 AM

I ran this script locally though and it worked fine

f8OP•6/1/24, 12:32 AM

Would cloudflare not timeout ever in this case?

Walshy•6/1/24, 12:33 AM

I'm not sure what you mean

f8OP•6/1/24, 12:34 AM

Say the third party api wasn't timing out, what else could cause the 504 gateway timeout on cloudflare's side for this cronjob

Walshy•6/1/24, 12:35 AM

I wonder if --test-scheduled is essentially doing a http request and that's why it's timing out too

Walshy•6/1/24, 12:35 AM

since your Worker isn't responding in 100 seconds - timeout

f8OP•6/1/24, 12:35 AM

Gotcha. It's always timing out around 122411ms as well

Walshy•6/1/24, 12:36 AM

I'd need to check the code and see what --test-scheduled is doing

Walshy•6/1/24, 12:36 AM

but 504 in cf == no response for 100 seconds

Walshy•6/1/24, 12:36 AM

(or whatever your configured limit is - for you it'll very likely be 100 seconds)

f8OP•6/1/24, 12:37 AM

But it seems to give error after 122 seconds of running

Walshy•6/1/24, 12:37 AM

yeah the 20 seconds is not that relevant, bunch of reasons there could be 20 more seconds

Walshy•6/1/24, 12:37 AM

hell you could have 650 seconds and it still be this limit

Walshy•6/1/24, 12:37 AM

it's a long explanation for why

f8OP•6/1/24, 12:38 AM

Gotcha, how would i go about debugging this then

f8OP•6/1/24, 12:38 AM

would this theoretically work in prod withou --test-scheduled flag?

Walshy•6/1/24, 12:38 AM

Well, see how long your fetches and things are taking to return

Ff8 would this theoretically work in prod withou --test-scheduled flag?

Walshy•6/1/24, 12:39 AM

if --test-scheduled was doing a http request then maybe

Walshy•6/1/24, 12:39 AM

that theory is that we're timing out on the worker response

Walshy•6/1/24, 12:39 AM

rather than the third party response

Walshy•6/1/24, 12:39 AM

i'd debug the third party first as that's easier than throwing in prod and waiting

Walshy•6/1/24, 12:40 AM

Test scheduled events by visiting /__scheduled in browser

f8OP•6/1/24, 12:40 AM

the third party response isn't timing out. I know that because we're looping over the same api request multiple times with diff params

f8OP•6/1/24, 12:41 AM

and randomly on a certain loop it times out

Walshy•6/1/24, 12:41 AM

oh yeah you're doing a http requesthere so very possible we're timing out that way

Walshy•6/1/24, 12:42 AM

this is a fun little not even bug but behaviour, you test crons over a request but request timeout is a thing that way -- however that isn't the case for proper crons

WWalshy > Test scheduled events by visiting /__scheduled in browser

f8OP•6/1/24, 12:42 AM

will check

f8OP•6/1/24, 12:46 AM

Times out after

Screen_Shot_2024-05-31_at_8.46.12_PM.png

f8OP•6/1/24, 12:46 AM

Screen_Shot_2024-05-31_at_8.46.47_PM.png

WWalshy this is a fun little not even bug but behaviour, you test crons over a request b...

f8OP•6/1/24, 12:51 AM

How is a cpu time limit exceeded possible i have no timelimit set

Walshy•6/1/24, 12:52 AM

there's still a 30 second cpu time limit

Walshy•6/1/24, 12:52 AM

you can set a lower one but standard still has a limit

f8OP•6/1/24, 12:55 AM

so is there no solution besides make script run shorter?

Walshy•6/1/24, 12:57 AM

i mean if you're hitting cpu limit then you can try to optimise that

Walshy•6/1/24, 12:57 AM

otherwise push to a queue and process in shorter amounts

f8OP•6/1/24, 12:59 AM

Ill try pushing to a queue. Thanks for help

WWalshy otherwise push to a queue and process in shorter amounts

f8OP•6/3/24, 12:57 AM

Instead of a cron job for calling a third party api to get the inidividual point score for an array of user account id's to have a live leaderboard, what if i used durable objects (one for each user account id) and the alarms api? Would that make sense

paulc•8/16/24, 4:13 PM

@Walshy I'm running into a similar timeout issue that @f8 mentioned above. I'm running browser rendering to scrape pdfs. Running wrangler dev locally ($ npm run dev -- --remote -e stage --test-scheduled) and triggering scheduled event with $ curl "http://localhost:8787/__scheduled", once the worker execution time hits around 80 seconds, it starts executing all over again as if the curl was called again - but it wasn't called again. Is this by design in workers?

I am trying to test locally getting a file from R2 bucket which i added via dashboard

Similar Threads

Similar Threads

Similar Threads