Again, if `workflows list` shows nothing, you haven't registered your Workflow yet by deploying it.

MMatt Silverlock Again, if `workflows list` shows nothing, you haven't registered your Workflow y...

T

telliks•11/20/24, 6:30 PM

I understand that my workflow is not deployed, I just don’t understand what I am supposed to do extra apart from the config I showed and then deploying the whole worker with “wrangler deploy”. From my understanding, having the config is enough and then wrangler should pick up that it should create the configured workflow. This is what my whole question is about. It is not deploying my configured workflow when I deploy the worker, even though the deployment runs successfully.

Original message was deleted

T

telliks•11/20/24, 7:05 PM

Sure, here is the whole file with some redactions.

#:schema node_modules/wrangler/config-schema.json
name = "my-app"
compatibility_date = "2024-10-22"
compatibility_flags = ["nodejs_compat"]
main = "dist/index.js"
minify = true

upload_source_maps = true

tail_consumers = [{ service = "my-app-tail" }]

[build]
command = "pnpm run build"

[observability]
enabled = true

[ai]
binding = "AI"

[[kv_namespaces]]
binding = "INCOMING_EMAILS_KV"
id = "<redacted>"

[[kv_namespaces]]
binding = "DLQ_KV"
id = "<redacted>"

[triggers]
crons = ["* * * * *", "*/30 * * * *"]

[[queues.consumers]]
queue = "account-emails"
dead_letter_queue = "dlq"
max_batch_size = 20       
max_batch_timeout = 10    
max_retries = 10              

[[queues.consumers]]
queue = "incoming-emails"
dead_letter_queue = "dlq"
max_batch_size = 10       
max_batch_timeout = 10    
max_retries = 10          

[[queues.consumers]]
queue = "dlq"

[[queues.producers]]
binding = "ACCOUNT_EMAILS_QUEUE"
queue = "account-emails"

[[queues.producers]]
binding = "INCOMING_EMAILS_QUEUE"
queue = "incoming-emails"

[[workflows]]
name = "process-incoming-emails"
binding = "PROCESS_INCOMING_EMAILS_WORKFLOW"
class_name = "ProcessIncomingEmailsWorkflow"

#:schema node_modules/wrangler/config-schema.json
name = "my-app"
compatibility_date = "2024-10-22"
compatibility_flags = ["nodejs_compat"]
main = "dist/index.js"
minify = true

upload_source_maps = true

tail_consumers = [{ service = "my-app-tail" }]

[build]
command = "pnpm run build"

[observability]
enabled = true

[ai]
binding = "AI"

[[kv_namespaces]]
binding = "INCOMING_EMAILS_KV"
id = "<redacted>"

[[kv_namespaces]]
binding = "DLQ_KV"
id = "<redacted>"

[triggers]
crons = ["* * * * *", "*/30 * * * *"]

[[queues.consumers]]
queue = "account-emails"
dead_letter_queue = "dlq"
max_batch_size = 20       
max_batch_timeout = 10    
max_retries = 10              

[[queues.consumers]]
queue = "incoming-emails"
dead_letter_queue = "dlq"
max_batch_size = 10       
max_batch_timeout = 10    
max_retries = 10          

[[queues.consumers]]
queue = "dlq"

[[queues.producers]]
binding = "ACCOUNT_EMAILS_QUEUE"
queue = "account-emails"

[[queues.producers]]
binding = "INCOMING_EMAILS_QUEUE"
queue = "incoming-emails"

[[workflows]]
name = "process-incoming-emails"
binding = "PROCESS_INCOMING_EMAILS_WORKFLOW"
class_name = "ProcessIncomingEmailsWorkflow"

#:schema node_modules/wrangler/config-schema.json
name = "my-app"
compatibility_date = "2024-10-22"
compatibility_flags = ["nodejs_compat"]
main = "dist/index.js"
minify = true

upload_source_maps = true

tail_consumers = [{ service = "my-app-tail" }]

[build]
command = "pnpm run build"

[observability]
enabled = true

[ai]
binding = "AI"

[[kv_namespaces]]
binding = "INCOMING_EMAILS_KV"
id = "<redacted>"

[[kv_namespaces]]
binding = "DLQ_KV"
id = "<redacted>"

[triggers]
crons = ["* * * * *", "*/30 * * * *"]

[[queues.consumers]]
queue = "account-emails"
dead_letter_queue = "dlq"
max_batch_size = 20       
max_batch_timeout = 10    
max_retries = 10              

[[queues.consumers]]
queue = "incoming-emails"
dead_letter_queue = "dlq"
max_batch_size = 10       
max_batch_timeout = 10    
max_retries = 10          

[[queues.consumers]]
queue = "dlq"

[[queues.producers]]
binding = "ACCOUNT_EMAILS_QUEUE"
queue = "account-emails"

[[queues.producers]]
binding = "INCOMING_EMAILS_QUEUE"
queue = "incoming-emails"

[[workflows]]
name = "process-incoming-emails"
binding = "PROCESS_INCOMING_EMAILS_WORKFLOW"
class_name = "ProcessIncomingEmailsWorkflow"

#:schema node_modules/wrangler/config-schema.json
name = "my-app"
compatibility_date = "2024-10-22"
compatibility_flags = ["nodejs_compat"]
main = "dist/index.js"
minify = true

upload_source_maps = true

tail_consumers = [{ service = "my-app-tail" }]

[build]
command = "pnpm run build"

[observability]
enabled = true

[ai]
binding = "AI"

[[kv_namespaces]]
binding = "INCOMING_EMAILS_KV"
id = "<redacted>"

[[kv_namespaces]]
binding = "DLQ_KV"
id = "<redacted>"

[triggers]
crons = ["* * * * *", "*/30 * * * *"]

[[queues.consumers]]
queue = "account-emails"
dead_letter_queue = "dlq"
max_batch_size = 20       
max_batch_timeout = 10    
max_retries = 10              

[[queues.consumers]]
queue = "incoming-emails"
dead_letter_queue = "dlq"
max_batch_size = 10       
max_batch_timeout = 10    
max_retries = 10          

[[queues.consumers]]
queue = "dlq"

[[queues.producers]]
binding = "ACCOUNT_EMAILS_QUEUE"
queue = "account-emails"

[[queues.producers]]
binding = "INCOMING_EMAILS_QUEUE"
queue = "incoming-emails"

[[workflows]]
name = "process-incoming-emails"
binding = "PROCESS_INCOMING_EMAILS_WORKFLOW"
class_name = "ProcessIncomingEmailsWorkflow"

T

telliks•11/20/24, 7:27 PM

@Skye If it helps, I could successfully create a brand new workflow with the

npm create cloudflare@latest workflows-starter -- --template "cloudflare/workflows-starter"

npm create cloudflare@latest workflows-starter -- --template "cloudflare/workflows-starter"

command (apart from the empty object issue when

env.MY_WORKFLOW.create()

env.MY_WORKFLOW.create()

is called locally). Does this command do more than simply scaffolding the worker project on my local file system?

For my existing worker, I just added the workflow config to the toml file. Should I create a workflow with a wrangler CLI command first? Based on all the discussions, I'm under the impression that the wrangler.toml config should be enough and that during deployment wrangler will create a workflow if it doesn't exist yet. Please confirm if this is correct. I know this is not how other cloudflare products like KV or Queues work. You have to create them first and then reference them in the toml file.

T

telliks•11/20/24, 7:44 PM

Okay, finally got to the bottom of this. My local wrangler version was on the latest version but the one running in CI was on v3.80.0, so it never deployed the workflow

Thanks Skye, Matt, Diogo Ferreira and sdr for trying to help me

P

pao ramen•11/20/24, 9:41 PM

ok, in the end I gave up. I faced too many durable execution errors, and queues started to get clogged because

Workflow.create

Workflow.create

was taking a whole minute. I will keep an eye when workflows get more mature.

M

makisuo•11/20/24, 10:48 PM

Is it possible to have multiple workflows in one worker? If I try define multiple workflows wrangler crashes without any error logs.

Mmakisuo Is it possible to have multiple workflows in one worker? If I try define multipl...

A

arch•11/20/24, 11:25 PM

A fix was recently merged and will probably be tagged as a release soon https://github.com/cloudflare/workers-sdk/commit/563439bd02c450921b28d721d36be5a70897690d

GitHub

Fix multiple workflows and add proper engine persistance (#7286) · ...

fix: allow multiple workflow definitions
feat: make engine persist logs and status over multiple runs
chore: add persistance test and remove redundant env types
chore: change chang...

Original message was deleted

P

pao ramen•11/21/24, 8:55 AM

awesome! let me know when those improvements land so I can test them out.

Ttelliks @Skye If it helps, I could successfully create a brand new workflow with the `np...

D

Diogo Ferreira•11/21/24, 11:34 AM

^apart from the empty object issue when env.MY_WORKFLOW.create() is called locally

wrangler@3.88.0 fixes that

A

alex (he/him)•11/21/24, 4:31 PM

I have two things which are a bit annoying, but which I can workaround. However any ETA on fixes?

a) The Dashboard trigger doesn't work. The params object get's stringified strangely, so my logic never works. Triggering from the CLI or a Binding with the same params object works without issues
b) The option to delete workflows in the UI & CLI. I changed the name of the workflow, now I have a stale workflow with the old name in the UI

K

kchro3•11/21/24, 8:50 PM

Hi team, I'm seeing an issue when reading & writing from KV in a Workflow.

I am developing with

wrangler dev --remote --env stage

wrangler dev --remote --env stage

& i have a workflow & a KV in a stage env.

I am testing a workflow where it reads from a KV & then writes back to the same KV.

My assumption was that if I stage my worker => trigger the workflow, it will run the stage workflow => read & write from the preview KV in the stage environment because the worker has access to the preview KV.

However, it looks like the stage workflow is actually reading & writing against the non-preview KV in the stage environment, which is inconsistent with the worker behavior. Is this a bug?

I

irvyn•11/22/24, 4:03 PM

Hey based on the limits for workflows (based on workers) does this mean that if I had to do processing of a 2MB file -> 4MB file.

It wont work within just 1 step but I would be able to pass small amounts through multiple steps as long as it stays below the entire limit of "Maximum state that can be persisted per Workflow instance" ?

https://developers.cloudflare.com/workflows/reference/limits/

Maximum persisted state per step    1MiB (2^20 bytes)    1MiB (2^20 bytes)
Maximum state that can be persisted per Workflow instance    100MB    1GB

Maximum persisted state per step    1MiB (2^20 bytes)    1MiB (2^20 bytes)
Maximum state that can be persisted per Workflow instance    100MB    1GB

Cloudflare Docs

Limits | Cloudflare Workflows docs

Limits that apply to authoring, deploying, and running Workflows are detailed below.

森

森@VerseLink (RW)•11/22/24, 11:49 PM

is there a way to check if an workflow id exists?

森

森@VerseLink (RW)•11/22/24, 11:49 PM

say for example if workflow.get(id) and id doesn't exist what is being returned? the documentation wasn't clear about this

森森@VerseLink (RW)say for example if workflow.get(id) and id doesn't exist what is being returned?...

M

Matt SilverlockOP•11/23/24, 12:05 PM

It will throw an exception.

森

森@VerseLink (RW)•11/23/24, 12:07 PM

any specific detail on what type of exception it would throw? since catch all is fine, but what if the exception was a different reason

森森@VerseLink (RW)any specific detail on what type of exception it would throw? since catch all is...

M

Matt SilverlockOP•11/23/24, 12:14 PM

There’s not a specific exception type yet. We should add.

M

Matt SilverlockOP•11/23/24, 12:14 PM

Docs PR: https://github.com/cloudflare/cloudflare-docs/pull/18364

GitHub

workflows: clarify get throws by elithrar · Pull Request #18364 · c...

Clarify that get throws and add an example of catching the error.

森

森@VerseLink (RW)•11/23/24, 12:15 PM

Thank you very much!

森

森@VerseLink (RW)•11/23/24, 12:15 PM

btw i found some minor errors on the document

森

森@VerseLink (RW)•11/23/24, 12:15 PM

https://developers.cloudflare.com/workflows/build/workers-api/#status
this should return

Promise<InstanceStatus>

Promise<InstanceStatus>

Cloudflare Docs

Workers API | Cloudflare Workflows docs

This guide details the Workflows API within Cloudflare Workers, including methods, types, and usage examples.

森

森@VerseLink (RW)•11/23/24, 12:16 PM

https://developers.cloudflare.com/workflows/build/workers-api/#instancestatus

Cloudflare Docs

Workers API | Cloudflare Workflows docs

This guide details the Workflows API within Cloudflare Workers, including methods, types, and usage examples.

森

森@VerseLink (RW)•11/23/24, 12:16 PM

output should return

object[]

object[]

instead of

object

object

;

森森@VerseLink (RW)https://developers.cloudflare.com/workflows/build/workers-api/#status this shoul...

M

Matt SilverlockOP•11/23/24, 12:17 PM

Open to adding a PR? I’m about to step out as it’s the weekend

MMatt Silverlock Open to adding a PR? I’m about to step out as it’s the weekend

森

森@VerseLink (RW)•11/23/24, 12:17 PM

Will do

森

森@VerseLink (RW)•11/23/24, 12:18 PM

finally is there a way to know in the Workflow itself if it is being terminated / restarted, and allow itself to clean up some resources?

If my understanding is correct, termination only happens between each step/sleep, but there doesn't seem to be way to know if the workflow is being terminated.

森森@VerseLink (RW)finally is there a way to know in the Workflow itself if it is being terminated ...

M

Matt SilverlockOP•11/23/24, 12:23 PM

No. That also poses a challenge: how long would we allow? What if termination needs to happen faster or is due to machine failure (which would restart your workflow from the last good step elsewhere), or exceeding CPU or memory?

“Finalizers” are challenging as they set an expectation of 100% but not always the case.

M

Matt SilverlockOP•11/23/24, 12:24 PM

We might revisit this but not straightforward today.

MMatt Silverlock No. That also poses a challenge: how long would we allow? What if termination ne...

森

森@VerseLink (RW)•11/23/24, 12:36 PM

I see, thank you for your reply.

Alternatively, allow a workflow to "stop" (so we have a separate call

workflow.get(id).stop()

workflow.get(id).stop()

) instead of terminate as a means to softly shutdown the workflow. So the user could gracefully close the workflow (for example the workflow could be a long running polling workflow that doesn't have an end state).

User could decide when the stop process should timeout, and call

terminate()

terminate()

to kill the workflow and abruptly kill the workflow in case it's stuck in a inf-loop and we want to kill it for sure.

森

森@VerseLink (RW)•11/23/24, 12:38 PM

If I'm not mistaken,

terminate

terminate

is the only way to stop a long running / no end-state workflow, so it would be nice to gracefully or signal the workflow to transition to a potential end-state or cleanup

A

an•11/23/24, 2:23 PM

Hi, currently the cpu time limit on each step is 30s, does that mean the step can live up to 30s?

My step can run up to 10 minutes due to a sleep function in my dependency, but since it's not using cpu I'm not sure I can use step for this

Migrating library to workflow logic is not doable, though.

Aan Hi, currently the cpu time limit on each step is 30s, does that mean the step ca...

S

Seekerdasbatatas•11/23/24, 4:00 PM

the step can live indefinetly (sleep, wait for I/O, etc..) - those 30 seconds regard actual CPU time: think of it as 30 seconds worth of cpu cycles

S

Seekerdasbatatas•11/23/24, 4:01 PM

if you are sleeping or waiting for I/O, you are not consuming cpu time

A

an•11/23/24, 4:39 PM

got it, thank you

L

laurynas•11/23/24, 9:58 PM

Any thoughts or plans about adding a websocket/SSE or similar reporting endpoint that we could subscribe to when running a Workflow? To my mind I currently need to either poll the

.status()

.status()

method (which should be fine for now) or set some outside state from inside the step callback (seems like a code smell).

森

森@VerseLink (RW)•11/24/24, 5:15 AM

btw,

Maximum steps per Workflow    256

Maximum steps per Workflow    256

, does this mean only for Workers Free? what about the limitation for Workers Paid

森

森@VerseLink (RW)•11/24/24, 5:16 AM

because I just realized this would prohibit the idea of using workflow for a long running no end-state polling process

森

森@VerseLink (RW)•11/24/24, 5:16 AM

which workflow could be used for

Llaurynas Any thoughts or plans about adding a websocket/SSE or similar reporting endpoint...

森

森@VerseLink (RW)•11/24/24, 5:24 AM

I really liked the concept of workflow; since workflow should be implemented on top of durable object (correct me if i'm wrong).
It feels like websocket implementation should be possible.
Looking through workflows documentation, it did gave me inspiration to refactor some of my internal codes so that it's a drop in replacement of workflow for Durable Objects.
(and once workflow is matured with more signals or queues then switch back)
The only down side is that the billing would be DO (so wall time and all) but it should give you all the granular steps you could ask for
(start, pause, restart, graceful stop, termination)
Let me know if you are interested!

森

森@VerseLink (RW)•11/24/24, 5:44 AM

Currently I think workflow lacks a few functionality, cf team let me know if there's a better place to post suggestions:

Signal: Signal for the workflow itself to pause / resume / restart / stop; currently there's no way in a workflow to observe any signal natively, my prototype implementation involves using KV to write and read signals between each steps; still a hassle, and terminated workflow has no way to clear KV flags, so a cron job or expiring KV flags are needed.
Long running task: It looks like the workflow has maximum steps limitation, along with the tendency to store all the results in the status, which is good for finite states; but for long, non-end state (unless interrupted) workflow, it might hit the limitation of max steps or max storage eventually since there's no way of clearing or discarding old results.
Events: There's no way for a workflow to natively send events to other processes or back to the client, like SSE or Websocket. The workaround would be to implement would be using Queue or call upon a DO to monitor the results. (which kind of defeats the purpose of workflow).

## Proposal

Signals:

First proposal: Allow the
```
WorkflowEvent<T>
```
WorkflowEvent<T>
```
WorkflowEvent<T>
```
WorkflowEvent<T>
to change; the workflow would accept the original WorkflowEvent as some of sort constructor (so it could be accessed via
```
this.initEvent
```
this.initEvent
```
this.initEvent
```
this.initEvent
etc; then the arguments in the
```
run
```
run
```
run
```
run
indicates a modified arguments that the workflow can compare to; a change of value can indicate a signal
Second proposal: Alternatively, allow
```
WorkflowEvent<T>
```
WorkflowEvent<T>
```
WorkflowEvent<T>
```
WorkflowEvent<T>
to have
```
signal
```
signal
```
signal
```
signal
property, this property has special value that could be checked and changed unlike
```
payload
```
payload
```
payload
```
payload
indicating the original value; the
```
signal
```
signal
```
signal
```
signal
property can have built-in signals like
```
shouldStop
```
shouldStop
```
shouldStop
```
shouldStop
to indicate a built-in option for the workflow to be informed that it should gracefully transition into a stop or cancelled state with proper clean up.

### Long running task

First proposal: Allow a previous state results to be discarded or overwritten if the same steps repeat itself (indicating a loop), this should be an opt-in behaviour and
```
WorkflowStepConfig
```
WorkflowStepConfig
```
WorkflowStepConfig
```
WorkflowStepConfig
can accept a option to
```
overrideResult
```
overrideResult
```
overrideResult
```
overrideResult
; this may be harder to implement then the second porposal.
Second proposal: Allow discard of staled state results, only preserve the newest 256 results and any older results are discarded. This would transfer the responsibility to preserve old data to new results to the user end; it's easier to be implemented, and requires minimal breaking change, where first proposal can be implementated later.

### Events
Allow a workflow to accept websocket, just like DO.
I don't think there's a good way to persist Response for SSE over

step.sleep

step.sleep

step.sleep

step.sleep

DDaniel Jakobsen Hey is there typescript support for the param in the workflow? Right now `Workf...

C

can•11/25/24, 8:16 AM

isn't params something you pass to the workflow? i believe we have to define it and pass it to the WorkflowEntrypoint generics
https://developers.cloudflare.com/workflows/get-started/guide/

D

ddsgadget•11/25/24, 1:12 PM

We have a workflow that is stuck in Queued status. We can't do anything to get it out of that status. Pausing it or Terminating the instance in the dashboard does nothing (the UI just hangs). Trying to pause terminate it via the API also throws errors saying that it can moved to a different state because it is in the queed state. How do you delete or terminate a queued state? Seems like you can't get out of it.

DDaniel Jakobsen Hey is there typescript support for the param in the workflow? Right now `Workf...

M

Matt SilverlockOP•11/25/24, 1:23 PM

unknown

unknown

unknown

unknown is correct - you pass your own type: https://developers.cloudflare.com/workflows/build/workers-api/#workflowinstancecreateoptions

Cloudflare Docs

Workers API | Cloudflare Workflows docs

This guide details the Workflows API within Cloudflare Workers, including methods, types, and usage examples.

Dddsgadget We have a workflow that is stuck in Queued status. We can't do anything to get i...

M

Matt SilverlockOP•11/25/24, 1:23 PM

Possibly a bug (it’s a beta!). Can you share your account ID & instance ID here?

Ccan isn't params something you pass to the workflow? i believe we have to define it ...

M

Matt SilverlockOP•11/25/24, 1:26 PM

You can also see more here: https://developers.cloudflare.com/workflows/build/events-and-parameters/#typescript-and-type-parameters

What you pass to

create({params: YourType})

create({params: YourType})

create({params: YourType})

create({params: YourType}) is the

payload

payload

payload

payload property of WorkflowEvent.

You don’t need to cast it with as - you’re passing it, and

as

as

as

as doesn’t tell Workflows anything.

In your Workflow you type the

WorkflowEvent

WorkflowEvent

WorkflowEvent

WorkflowEvent by passing your type as the type parameter as per the docs I linked.

Cloudflare Docs

Events and parameters | Cloudflare Workflows docs

When a Workflow is triggered, it can receive an optional event. This event can include data that your Workflow can act on, including request details, user data fetched from your database (such as D1 or KV) or from a webhook, or messages from a Queue consumer.

MMatt Silverlock Possibly a bug (it’s a beta!). Can you share your account ID & instance ID here?

D

ddsgadget•11/25/24, 1:58 PM

instance is: 079dd27c-cb60-45c9-a0f1-8bad4b169c97

Again, if `workflows list` shows nothing, you haven't registered your Workflow yet by deploying it.

Signals:

Similar Threads

Similar Threads

Signals:

Similar Threads