I thought you could potentially return from a step also, but not sure if that would work as it would
I thought you could potentially return from a step also, but not sure if that would work as it would be returning a connection not a normal value.
SMaybe someone from CF can clarify, I am using hyperdrive also as the connection string.
SAlso it seems there is an undocumented payload limit on params for workflows.
When I try to pass a base64 in params it will fail to start the workflow every time but will not throw an error when I instantiate the workflow.
When I try to pass a base64 in params it will fail to start the workflow every time but will not throw an error when I instantiate the workflow.
AGm! Loving workflows in general but feels like i'm missing the right way to track errors. I get the exception message when i go to the flow instance page and wrangler tail but no stack trace so it takes a bit longer to debug. Any way to get the full stack trace?
IValues that are returned need to be serializable, so yes, you'll need to recreate the DB conn every step.
IAdditionally, your Workflow may be paused/move machines/etc between steps so putting a DB conn in the global scope isn't advised either
SCheers for confirming yeah thought I read or heard words to the latter.
Hyperdrive should hopefully minimise the additional latency and reuse the connection I guess!
Hyperdrive should hopefully minimise the additional latency and reuse the connection I guess!
IYup - and realistically a DB connection is only on the order of several hundreds of ms/single-digit seconds at the most so unless your Workflow is very time-sensitive it probably won't matter that much.
SIs it possible To have a workflow that uses a websocket? Example. Let's say a user posts a comment on my site and I want that shot off to a workflow and the web client to listen to know when that workflow is done (via a websocket). The workflow does its steps needed on the post data and then the last step in the flow I want it to shoot off a websocket message saying it is done and then the web client can pick up that message and do what needs to be done after that. Is this something possible? Or does the client just have to keep polling the workflow for its status till it is done? Thanks!
SAh ok. Well, that could be something you can add to workflow to have the ability to use websockets within workflow.
JYou can make workflow push updates to Durable Object, which has websocket support
SAny plan on adding option in
cloudflare:test
PHi there! I've been porting some python code I had using inngest, and some other built with queues, and I have a bunch of feedback that might be useful.
For reference, the code that I've migrated to workflows usually does something like this:
Pao
For reference, the code that I've migrated to workflows usually does something like this:
- fetches a bunch of resources from the DB
- (fanout) for each resource it does an http call to an API and then it saves some data to the DB
- (fanin) once everything is done, it makes more changes to the DB
- The 256 step limit means that I can't build this as a single workflow. I have to add a queue to implement the "fanout". The alternative is to do a
, but then you don't get to retry failed workflow creation.Promise.allSettled - The http call has a rate limit, and unlike queues, there is no way to specify concurrency/batch size. Since I moved the "fanout" to a queue, this is where I implement the rate limit, but it would be great to be able to describe the whole workflow without queues.
- Because of moving the fanout to queues, there is no way to do the fanin part since it can't be known when all the queues have been processed.
- There is no way to compose workflows. This means that if you have a piece of workflow that is used in several places, you have to abstract away the steps. It would be great if workflows could return values so a workflow could call another workflow like a step.
Pao
POne last thing... Is there a workaround to the "inserted row already exists in table while trying to run multiple workflows locally"? I can't have more than one workflow locally
Jare you using the same workflow id/name?
PGitHub
Which Cloudflare product(s) does this pertain to? Workers Runtime, Wrangler, Other What version(s) of the tool(s) are you using? 3.83.0 What version of Node are you using? No response What operatin...
TI'm not sure if I missed to see the answer to the following question anywhere: is an instantiated workflow guaranteed to run on a dedicated worker?
Asking to understand how the worker-specific limits (e.g., subrequests) possibly apply.
Asking to understand how the worker-specific limits (e.g., subrequests) possibly apply.
EAny pointers to docs that describe how workflow can be used with workers for platforms?
edit: looks like this is not currently supported, but would love to understand what the timeline looks like. workflow + workers for platforms is exactly what I need and I'll unfortunately have to forgo workflow until the two can compose together and continue to work on my own workflow layer.
edit: looks like this is not currently supported, but would love to understand what the timeline looks like. workflow + workers for platforms is exactly what I need and I'll unfortunately have to forgo workflow until the two can compose together and continue to work on my own workflow layer.
PMore feedback:
- bug: when triggering a workflow from the UI, the json payload gets stringified (so it does not work at all)
- it would be great to be able to retry workflows/steps from the UI
MRemote isn’t possible while doing dev locally. I.e, can’t access remote R2 while running wrangler dev
So workers with assets is convenient but because it’s not fully fledged out, it depends on how “serious” your project requirements.
The eta for doing running wrangler remotely is this quarter, but I wouldn’t trust CF and assume it would be available sometime early next year
So workers with assets is convenient but because it’s not fully fledged out, it depends on how “serious” your project requirements.
The eta for doing running wrangler remotely is this quarter, but I wouldn’t trust CF and assume it would be available sometime early next year
MApparently, they are working on making workflows have a built in rate limiter. I wouldn’t assume this to be release anytime soon
Without specific numbers than giving advice is difficult but you could potentially do this with breaking up your fanout into multiple workflows. It’s also possible to have a “always on” workflow that manages other workflows although this solution might be less elegant than just using a queue.
Without specific numbers than giving advice is difficult but you could potentially do this with breaking up your fanout into multiple workflows. It’s also possible to have a “always on” workflow that manages other workflows although this solution might be less elegant than just using a queue.
MCan’t you start another workflow from an existing workflow?
PYes, but unlike steps, you don't get the result
KI have a question about workflow IDs. It supports setting the ID in case there is some external ID like a user ID, but it sounds like you cannot reuse IDs. Doesn't that defeat the purpose of being able to reference a workflow?
Is the recommendation to append a timestamp with the external ID or something for a recurring workflow per user ID?
Is the recommendation to append a timestamp with the external ID or something for a recurring workflow per user ID?
SThank MissS :). What's the best way to stay up to date on the progress on Cloudflares side?
PI will unify all workflows in a single one with a switch case, like queues.
DIt is on the Roadmap
SExporting multiple workflows from wrangler.toml seems to cause errors with miniflare locally (and works fine on remote/deployed)
This is similar syntax that I use for queues, but just to double check is this the correct way to add multiple workflows?
I can send you the debug logs, but nothing useful apart from some stuff about LLVM symbolizer. Have validated the issue only occurs when I add the additional workflows, does not occur with just one workflow.
edit: I assume its the same as the issue above https://github.com/cloudflare/workers-sdk/issues/7127 let me know if you need anything from me.
This is similar syntax that I use for queues, but just to double check is this the correct way to add multiple workflows?
I can send you the debug logs, but nothing useful apart from some stuff about LLVM symbolizer. Have validated the issue only occurs when I add the additional workflows, does not occur with just one workflow.
edit: I assume its the same as the issue above https://github.com/cloudflare/workers-sdk/issues/7127 let me know if you need anything from me.
GitHub
Which Cloudflare product(s) does this pertain to? Workers Runtime, Wrangler, Other What version(s) of the tool(s) are you using? 3.83.0 What version of Node are you using? No response What operatin...
MGuys is there any public information on the roadmap? I mean like what to expect when. (I know it’s still in early but I’m very excited that’s why I wanted to learn more)
As far as I understand from the community these are the most expected features or common issues etc.
1) Concurrency settings per workflow or step (maybe other flow control features like throttling, rate limiting etc.)
2) Event based flows like waitUntil API
3) Returning value from workflow and accessing them inside another workflows
4) Local development issues(multiple workflows, id, observability etc)
5) Workflow level clean-up functions such as ‘onFailure’
6) Prioritization on workflow based on key or something similar
7) Limit increase (concurrency, step count etc.)
As far as I understand from the community these are the most expected features or common issues etc.
1) Concurrency settings per workflow or step (maybe other flow control features like throttling, rate limiting etc.)
2) Event based flows like waitUntil API
3) Returning value from workflow and accessing them inside another workflows
4) Local development issues(multiple workflows, id, observability etc)
5) Workflow level clean-up functions such as ‘onFailure’
6) Prioritization on workflow based on key or something similar
7) Limit increase (concurrency, step count etc.)
DUnfortunately I don't think you'll get answers to when type questions, besides ":soontm:"
Dbut trust me you're not alone, I'm also excited about Workflows and eagerly waiting for some of these 
MYeah I know there are a lot of people like us and setting expectations with clear communication would be great in the early stage. Let’s hope that we get answers at least some of them 
DI don't disagree, just saying what I think will be realistic
priorities change all the time, and even if they were to commit to a date/timeline, it's not a 100% guarantee, as a few examples have shown
priorities change all the time, and even if they were to commit to a date/timeline, it's not a 100% guarantee, as a few examples have shown
MYes, you are right but I think nobody expect commitment on a date or something like 100% guarantee on these features/issues. People want to understand the direction and the level of commitment to the product itself. These are not huge things when you have a clear communication. (Not blaming anybody on the team, just saying things can be improved 
)
)
MPeople do interpret dates as commitments 
MHey Matt, let’s forget about the date then.
It would be great if you could inform us on these topics in order to clarify the direction you guys want to go.
Should we expect these features or not? (or any other feature you want to add) Instead of a date, maybe you can share the timeline relative to other features. (like “we are prioritizing this one over this one” etc.)
It would be great if you could inform us on these topics in order to clarify the direction you guys want to go.
Should we expect these features or not? (or any other feature you want to add) Instead of a date, maybe you can share the timeline relative to other features. (like “we are prioritizing this one over this one” etc.)
MI think we've laid these out in this chat - but prioritization does change. We're super early in the beta (only a month in!) and fixing the prioritization means we'd be ignoring folks / the ability to solve problems as they come up.
MMost of these (bar sharing state across Workflows) will likely land in the next 3-4 months. There's not a strict ordering - some of these are easier than others, others need design time/discussion, etc.
I know that's not the answer you want, but it's the answer I'm giving.
I know that's not the answer you want, but it's the answer I'm giving.
MNot quite, but it’s close enough to the answer I was looking for. I appreciate you taking the time to address it. Now, anyone searching for this can find it too. 
MI'm unsure of the best way to implement a fan-out pattern using Workflows. I fetch a list of files, and want to process each one. Should I loop over the files and call step.do, awaiting them all with await Promise.all? Or should I trigger a new "file-process" workflow instance for each of them?
1I've only just started looking at Workflows, I was wondering if it would be suitable for a process where one step requires collecting data from dozens of endpoints (DOs in this case) and waiting until all are returned before going to the next step. Would this be possible?
MYou can do both actually and both have different trade-offs. There is a hard limit (256) for maximum number of steps in a single run. So if the list you want to process is bigger than it you can’t. There is no hard limit for workflows (you can run 100 workflows concurrently tho) however you won’t be able to share state between them without external storage (kv etc.)
MYes, you can do. It’s recommended to make a single fetch call in a step in order not to lose its retry logic and persistency.
Mthanks, didn't immediately see this hard limit in the docs - but it is there, I just missed it. I have a few thousand, so need to trigger a workflow for each.
Punluckily, workflows have limited amount of steps. So only do step.do for each file if you are 100% sure you are not going to have more than 256 files. Regarding triggering a new process for each, it can be done but it does not support batching nor concurrency. What I do to workaround this is one step, and inside I chunk and send batches to a queue. That queue then triggers the workflows.
Also, since having multiple workflows locally does not work, I recommend having a single workflow that dispatches work.
Also, since having multiple workflows locally does not work, I recommend having a single workflow that dispatches work.
PThis is how I connected the pieces myself. Basically a queue endpoint that will forward events to a workflow. The workflow then dispatches with a given id. It's based on convention (each workflow has an id, and the queue name is that id + '-queue') and it allows me to have some type safety. I then pass an object around, so I can compose workflows as you expect.
EAny comment on whether or not workflows will ever support workers for platforms?
SBug in the workflow list versions docs, result should be a collection
https://developers.cloudflare.com/api/operations/wor-list-workflow-versions
https://developers.cloudflare.com/api/operations/wor-list-workflow-versions
Interact with Cloudflare's products and services via the Cloudflare API
CCurious about this as well. Seems a bit of overlap between queues and workflows. Would be nice if there was a guide which outlined reasons to use either