Nope, we are in the CF purgatory when it comes to vitest + workflows. :NotLikeThis:

D

When architecting workflows and steps, what are the pros/cons of using a queue in conjunction with a workflow vs. just running a loop and performing a step. Example: I read from the db a list of URLs that need to be moved over to R2. What would be better architecture: Read in the list, loop through and perform batch steps for each? Or add each one to the queue which then triggers a new workflow for each. Here is what I have come up with, curious to hear what else I should be thinking about.

No Queue (stay within workflow) approach:
Pros - avoid queue pricing; can aggregate results of all of them and then do more processing once they are all done.
Cons - may hit the 1024 steps limit depending on size of list and number of inner steps; all share same workflow storage so may hit 1GB storage limit

Use Queue (run new workflow per item) approach:
Pros - can rely on queue retries; starting new workflow reduces number of inner steps to avoid 1024 limit; allow for 1GB storage for each
Cons - consumes more workflow requests, increasing price; not able to easily process after all files in list are processed. added cost due to using queue.

Anything I'm missing or misinterpreting? Curious how others architect your workflows. thanks

JJehu Nope, we are in the CF purgatory when it comes to vitest + workflows. :NotLikeTh...

A

Advany•6/14/25, 6:01 PM

to bad... wont use workflows untill this is sovled

S

scook•6/15/25, 1:49 AM

Has anyone run into issues locally when invoking Workflows from async
waitUntil
waitUntil
waitUntil
waitUntil code?
I have a Worker that attempts to trigger a Workflow like so:

// Worker
ctx.waitUntil(
  nonBlockingFunc({ ... })
);

async nonBlockingFunc() {
  const workflowInstance = await env.MY_WORKFLOW.create();
  // await new Promise((resolve) => setTimeout(resolve, 2000)); // sleep fixes???
  return workflowInstance.id;
}

// Worker
ctx.waitUntil(
  nonBlockingFunc({ ... })
);

async nonBlockingFunc() {
  const workflowInstance = await env.MY_WORKFLOW.create();
  // await new Promise((resolve) => setTimeout(resolve, 2000)); // sleep fixes???
  return workflowInstance.id;
}

// Worker
ctx.waitUntil(
  nonBlockingFunc({ ... })
);

async nonBlockingFunc() {
  const workflowInstance = await env.MY_WORKFLOW.create();
  // await new Promise((resolve) => setTimeout(resolve, 2000)); // sleep fixes???
  return workflowInstance.id;
}

// Worker
ctx.waitUntil(
  nonBlockingFunc({ ... })
);

async nonBlockingFunc() {
  const workflowInstance = await env.MY_WORKFLOW.create();
  // await new Promise((resolve) => setTimeout(resolve, 2000)); // sleep fixes???
  return workflowInstance.id;
}

On my local dev, it appears to successfully create the Workflow instance (I can see the instance id), but the Workflow itself is never actually run. However, if I insert a sleep after the invocation, then the Workflow does actually get run. Afaict this is only happening on my local and not in production.

My friendly ChatGPT claims that this is a shortcoming of Cloudflare local development / Miniflare not fully respecting the async tasks in the

waitUntil

waitUntil

waitUntil

waitUntil, because it garbage collects the state and tears down the execution environment after the Workflow instance is registered but BEFORE the workflow is actually run. The sleep prevents the premature shutdown of the Worker execution context.

Can anyone verify if this is true?

I

IvoDev•6/15/25, 11:49 PM

Can i use durable objects inside workflows and viceversa?

J

JehuOP•6/16/25, 12:02 AM

Yes, bindings all the way

I

IvoDev•6/16/25, 12:04 AM

amazing ! ultimate durability

A

alex (he/him)•6/16/25, 7:10 AM

Can I put everything that doesn't need an external service directly inside the

run

run

method without wrapping it inside a

step.do

step.do

step.do

step.do. My examples are

zod

zod

zod

zod validation of params or something like

const ai = new GoogleGenAI()

const ai = new GoogleGenAI()

const ai = new GoogleGenAI()

const ai = new GoogleGenAI().. I theory I don't want to persist those, I'm fine if they run again on a new try. So would it be best to put them in the global

run

run

scope?

I

ivanovm•6/16/25, 11:39 PM

hey cloudflare team, i love the platform otherwise but it is genuinely unacceptable to have 524 timeout on outgoing requests at 100 seconds. almost every major ai model can take over 100 seconds to respond, cloudflare becomes unusable by default for ai applications

Aalex (he/him)Can I put everything that doesn't need an external service directly inside the `...

H

Hello, I’m Allie!•6/17/25, 3:05 AM

Sure, that’s fine. Though, even if it doesn’t need to hit an external service, if it performs a lot of work, I would still persist it

S

SODaLC•6/17/25, 3:36 PM

Is the first message in a workflow log suppose to be an error, or have I done something wrong? The rest of the code executes just fine.

DDan (Openside)When architecting workflows and steps, what are the pros/cons of using a queue i...

D

Dan (Openside)•6/17/25, 10:20 PM

curious if any Cloudflare team members have thoughts on this.

G

gamarra2042•6/18/25, 11:10 AM

anyone having problem with workflows today? instances all on queue and not running

C

Cristian Lipinski•6/18/25, 12:30 PM

Hello guys

Im facing a problem where my workflow instances get queued but never starts

This started may 15th, on that time, I noticed that when we manually paused some wf queued instance, all instances above it were triggered

then I publised a worker that basically start, pause, and finish a wf instance so my routines could be triggered

That worked until today (June 18th); this morning, none of my instances are being triggered, even with that start/pause “quick fix”

There are some crucial routines to our company, i'm even thinking about trying to migrate those routines to other account or even service

We're using a lot of cloudflare services and investing around 4.5k usd monthly

Could anyone help me with that matter?

HHello, I’m Allie!Sure, that’s fine. Though, even if it doesn’t need to hit an external service, i...

A

alex (he/him)•6/18/25, 2:01 PM

True. But an sdk client or a zod validator isn’t serializiable anyway, right?

TTobias Is it possible to set concurrency limits on a workflow? Kind of like how Trigger...

I

ItsWendell•6/18/25, 6:42 PM

Do we have any updates on this? I'm building a complicated queue with durable workers now as a workaround but this would make it easier

G

Gabriel Francisco•6/18/25, 7:59 PM

hey, how cloudflare workers with multiple environments deal with workflows?

G

Gabriel Francisco•6/18/25, 8:00 PM

should I put one workflow at the root of the wrangler.toml or should I create a workflow for each env?

G

Gabriel Francisco•6/18/25, 8:00 PM

the first one seems more correct but it produces a warning

Aalex (he/him)True. But an sdk client or a zod validator isn’t serializiable anyway, right?

H

Hello, I’m Allie!•6/19/25, 3:28 AM

I would think they are? Since they are still JS objects? Might be a case of try and see?

GGabriel Francisco should I put one workflow at the root of the wrangler.toml or should I create a ...

D

ddsgadget•6/19/25, 10:53 AM

You should have a separate workflow for each environment. Just change the name of the specific workflow in the wrangler.toml for each environment and leave everythng else the same in the workflow definition in the toml and it will work. Cloudflare will build a separate workflow for each environment using the same exact class. hpe that makese sense

Dddsgadget You should have a separate workflow for each environment. Just change the name o...

G

Gabriel Francisco•6/19/25, 2:07 PM

thank you! very helpful!

N

nicoritschel•6/19/25, 2:29 PM

Are containers still landing this month?

M

Marco Aleixo•6/19/25, 5:37 PM

hey team - I have almost 500 workflows stuck as running status ( without steps ) for almost 5 days

G

Gabriel Francisco•6/20/25, 3:15 PM

have anyone got "An RPC result was not disposed properly" when calling a workflow from a hono route?

G

Gabriel Francisco•6/20/25, 3:15 PM

there is no disposables at this flow, but I keep getting this error

GGabriel Francisco have anyone got "An RPC result was not disposed properly" when calling a workflo...

K

koraytaylan•6/20/25, 7:26 PM

could it be that you do not await the create call to finish before exiting the hono route?

Kkoraytaylan could it be that you do not await the create call to finish before exiting the h...

G

Gabriel Francisco•6/20/25, 9:19 PM

good hypothesis, but its not that

G

Gabriel Francisco•6/20/25, 9:19 PM

I use the instance id, right after too, ensuring the create is awaited

G

Gabriel Francisco•6/20/25, 9:20 PM

it seems like a bug on hono or workflows

E

Elsa k🦮•6/20/25, 9:20 PM

Yes

R

rbravo•6/21/25, 1:55 AM

Hey, If I have a WF instance that has a sleepUntil, and I use .terminate it should cancel all the following executions that are inside this workflow, right? Right now even if I terminate the instance it keeps the execution.

GGabriel Francisco have anyone got "An RPC result was not disposed properly" when calling a workflo...

C

Caio•6/23/25, 9:44 AM

Hi Gabriel. Could you provide more details here? Is this reoccurring for you? Which route are you hitting?

CCaio Hi Gabriel. Could you provide more details here? Is this reoccurring for you? Wh...

G

Gabriel Francisco•6/23/25, 1:14 PM

It is recurring, at least locally, when creating a workflow. My code is very simple:

    .post('/:integration/fetch', zValidator('param', z.object({ integration: z.string().min(1) })), async (ctx) => {
        const { integration } = ctx.req.valid('param')
        const instance = await ctx.env.HistoryIngestion.create({
            id: integration,
            params: { integrationId: integration },
        })
        await ctx.env.Jobs.put(
            `workflow:${integration}`,
            JSON.stringify({ instance: instance.id, status: 'pending', progress: 0, startedAt: Date.now() }),
        )
        return ctx.json({ success: true }, 202)
    })

    .post('/:integration/fetch', zValidator('param', z.object({ integration: z.string().min(1) })), async (ctx) => {
        const { integration } = ctx.req.valid('param')
        const instance = await ctx.env.HistoryIngestion.create({
            id: integration,
            params: { integrationId: integration },
        })
        await ctx.env.Jobs.put(
            `workflow:${integration}`,
            JSON.stringify({ instance: instance.id, status: 'pending', progress: 0, startedAt: Date.now() }),
        )
        return ctx.json({ success: true }, 202)
    })

    .post('/:integration/fetch', zValidator('param', z.object({ integration: z.string().min(1) })), async (ctx) => {
        const { integration } = ctx.req.valid('param')
        const instance = await ctx.env.HistoryIngestion.create({
            id: integration,
            params: { integrationId: integration },
        })
        await ctx.env.Jobs.put(
            `workflow:${integration}`,
            JSON.stringify({ instance: instance.id, status: 'pending', progress: 0, startedAt: Date.now() }),
        )
        return ctx.json({ success: true }, 202)
    })

    .post('/:integration/fetch', zValidator('param', z.object({ integration: z.string().min(1) })), async (ctx) => {
        const { integration } = ctx.req.valid('param')
        const instance = await ctx.env.HistoryIngestion.create({
            id: integration,
            params: { integrationId: integration },
        })
        await ctx.env.Jobs.put(
            `workflow:${integration}`,
            JSON.stringify({ instance: instance.id, status: 'pending', progress: 0, startedAt: Date.now() }),
        )
        return ctx.json({ success: true }, 202)
    })

M

meow•6/25/25, 4:40 AM

Hi, is there anyway to turn on verbose logging. I'm having some problem while building a crawler using workflows. While the links expand, it somehow leads to memory leak. Couldn't find the root cause for this.

entryErrorContext = jsgInternalError; wdErrId = cbiok85ihdsus98r7iao99nq
workerd/jsg/util.c++:320: error: e = workerd/util/sqlite.c++:499: failed: expected _ec ==
SQLITE_OK [7 == 0]; out of memory: SQLITE_NOMEM
stack:
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@2667f9d
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@2667c9d
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@1af50ab
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@1af4cee
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@1af49b8

entryErrorContext = jsgInternalError; wdErrId = cbiok85ihdsus98r7iao99nq
workerd/jsg/util.c++:320: error: e = workerd/util/sqlite.c++:499: failed: expected _ec ==
SQLITE_OK [7 == 0]; out of memory: SQLITE_NOMEM
stack:
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@2667f9d
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@2667c9d
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@1af50ab
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@1af4cee
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@1af49b8

entryErrorContext = jsgInternalError; wdErrId = cbiok85ihdsus98r7iao99nq
workerd/jsg/util.c++:320: error: e = workerd/util/sqlite.c++:499: failed: expected _ec ==
SQLITE_OK [7 == 0]; out of memory: SQLITE_NOMEM
stack:
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@2667f9d
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@2667c9d
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@1af50ab
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@1af4cee
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@1af49b8

entryErrorContext = jsgInternalError; wdErrId = cbiok85ihdsus98r7iao99nq
workerd/jsg/util.c++:320: error: e = workerd/util/sqlite.c++:499: failed: expected _ec ==
SQLITE_OK [7 == 0]; out of memory: SQLITE_NOMEM
stack:
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@2667f9d
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@2667c9d
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@1af50ab
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@1af4cee
./node_modules/@cloudflare/workerd-linux-64/bin/workerd@1af49b8

Original message was deleted

I

Idle•6/25/25, 9:51 AM

?pings

S

SuperHelpflare•6/25/25, 9:51 AM

Please do not ping community members for non-moderation reasons. Doing so will not solve your issue faster and will make people less likely to want to help you.

M

meow•6/25/25, 9:52 AM

Sorry

G

Gabriel Francisco•6/27/25, 11:31 PM

has anyone got "Worker exceeded CPU time limit." from workflows even that it is only waiting for TCP data and not performing any cpu task? (node:net socket)

GGabriel Francisco has anyone got "Worker exceeded CPU time limit." from workflows even that it is ...

K

koraytaylan•6/28/25, 9:59 AM

i believe you try to do some http calls not inside a step but within the main scope.

Kkoraytaylan i believe you try to do some http calls not inside a step but within the main sc...

G

Gabriel Francisco•6/28/25, 12:41 PM

its not the case. why would you think that?

GGabriel Francisco its not the case. why would you think that?

K

koraytaylan•6/28/25, 12:45 PM

because the whole point workflows (at least for me) is to not have any time limits. code running inside a step not supposed to have any such limits i believe. then i guess i would try splitting that specific step into multiple smaller ones to see it will still have this issue assuming the step being too big could be the reason.

Kkoraytaylan because the whole point workflows (at least for me) is to not have any time limi...

G

Gabriel Francisco•6/28/25, 1:15 PM

the problem is that the step is very small in code. but I receive tons of data via tcp

G

Gabriel Francisco•6/28/25, 1:16 PM

that was my assuption too, but my experience has been totally different

G

Gabriel Francisco•6/28/25, 1:16 PM

lots of silly limits

G

Gabriel Francisco•6/28/25, 1:17 PM

there is a very low memory limit (128mb), cpu time limit, http requests limit, even the quentity of active timers (setTimeouts)

G

Gabriel Francisco•6/28/25, 1:17 PM

very bad dx

G

Gabriel Francisco•6/28/25, 1:18 PM

and the cpu time limit I think its a bug on the node_compat of "node:net", no way I'm getting this limit from my code, it's probably internal

GGabriel Francisco there is a very low memory limit (128mb), cpu time limit, http requests limit, e...

K

koraytaylan•6/28/25, 1:20 PM

but receiving tons of data into memory is never a good idea anyway. for a similar case i stream the incoming data into an r2 object and then process it afterwards in a similar streaming fashion

GGabriel Francisco and the cpu time limit I think its a bug on the node_compat of "node:net", no wa...

K

koraytaylan•6/28/25, 1:21 PM

there is this new containers offering which is similarly in beta currently. from what you describe it seems like a better fit to me than workflows.

Kkoraytaylan but receiving tons of data into memory is never a good idea anyway. for a simila...

G

Gabriel Francisco•6/28/25, 1:26 PM

even if I stream the data to a r2 bucket I feel I will get the cpu limit

Nope, we are in the CF purgatory when it comes to vitest + workflows. :NotLikeThis:

Similar Threads