Also when running more than 2 workflows per second im getting throttled. Sometimes responses from

Also when running more than 2 workflows per second im getting throttled.

Sometimes responses from workflows rest API endpoint to start a new workflow is taking up to 40seconds

SSam Is workflows down, some of my requests ahve been stuck in queue for a few minute...

Diogo Ferreira•11/11/24, 2:54 PM

How many Workflows do you have running?
We are aware of the some performance issue under heavy load (per account) and we are working on fixes for it

SSam Also when running more than 2 workflows per second im getting throttled. Somet...

Diogo Ferreira•11/11/24, 2:56 PM

Is this something new for you? I mean, did you see it in the past?

DDiogo Ferreira Is this something new for you? I mean, did you see it in the past?

SamOP•11/11/24, 2:57 PM

First time I’ve ran a large workload

Focux•11/11/24, 3:28 PM

Is there a way to re-run a workflow after sleep when the workflow has finished running?

Focux•11/11/24, 3:30 PM

My use case is that I'm writing a workflow to do a visual ping to a website every X minutes and would love to automatically re-run the workflow after it's finished running

Focux•11/11/24, 3:30 PM

But not sure if this is even possible

FFocux Is there a way to re-run a workflow after sleep when the workflow has finished r...

Diogo Ferreira•11/11/24, 3:35 PM

Not for the moment. A restart feature is on our roadmap

Focux•11/11/24, 4:27 PM

Roger

Focux•11/11/24, 4:28 PM

If you need some API ideas, Restate has a good API for scheduling workflow execution

FFocux If you need some API ideas, Restate has a good API for scheduling workflow execu...

Diogo Ferreira•11/11/24, 4:29 PM

this?
https://restate.dev/

Focux•11/11/24, 4:30 PM

Yeah, that's it!

Focux•11/11/24, 4:31 PM

They have a feature called delayed calls:

ctx.serviceSendClient(MyService).myHandler(payload, { delay: 5000 });

ctx.serviceSendClient(MyService).myHandler(payload, { delay: 5000 });

Focux•11/11/24, 4:35 PM

They have other cool ideas as well that could be useful in workflows, like access to storage (similar to Durable Objects), the ability to have more handlers inside a workflow to return statuses, or whatever you want to return about the current workflow instance, signals, etc.

Focux•11/11/24, 4:36 PM

https://docs.restate.dev/use-cases/workflows/

LordSilver•11/12/24, 12:33 PM

Is there no TS support? How do you write usable code then?

LordSilver•11/12/24, 12:33 PM

I'm not DHH.

LLordSilver Is there no TS support? How do you write usable code then?

Matt Silverlock•11/12/24, 1:45 PM

All of the examples are in TypeScript.

LLordSilver Is there no TS support? How do you write usable code then?

Matt Silverlock•11/12/24, 1:46 PM

I’d recommend starting with the guide and then coming back here if you get stuck: https://developers.cloudflare.com/workflows/get-started/guide/

Cloudflare Docs

Guide | Cloudflare Workflows docs

Workflows allow you to build durable, multi-step applications using the Workers platform. A Workflow can automatically retry, persist state, run for hours or days, and coordinate between third-party APIs.

ctrlalt.defeat•11/12/24, 2:32 PM

Is the "Cross-script calls" feature supposed to work for Workflows local development (

wrangler dev

wrangler dev

)? (See https://developers.cloudflare.com/workflows/build/workers-api/#cross-script-calls)

[[workflows]]
name = "test-item-workflow"
binding = "MY_WORKFLOW"
class_name = "TestItemWorkflow"
# THIS >>>>>>
script_name = "my-app-workflows"

[[workflows]]
name = "test-item-workflow"
binding = "MY_WORKFLOW"
class_name = "TestItemWorkflow"
# THIS >>>>>>
script_name = "my-app-workflows"

I get

Worker "workflows:test-item-workflow"'s binding "USER_WORKFLOW" refers to a service "core:user:my-app-workflows", but no such service is defined.

Worker "workflows:test-item-workflow"'s binding "USER_WORKFLOW" refers to a service "core:user:my-app-workflows", but no such service is defined.

, even though I am setting the name of the other worker with

wrangler dev --name my-app-workflows

wrangler dev --name my-app-workflows

MMatt Silverlock All of the examples are in TypeScript.

LordSilver•11/12/24, 3:20 PM

were they updated or it was always like that?

LLordSilver were they updated or it was always like that?

Matt Silverlock•11/12/24, 3:21 PM

They were always like that, from the beginning. You can check the commit history: https://github.com/cloudflare/cloudflare-docs/commits/production/src/content/docs/workflows/get-started/guide.mdx

skip•11/13/24, 12:48 AM

Hi @Cayter

. Would you recommend sticking with Cloudflare for Remix developers?

Should we look to moving away from CF Pages and using a Workers + Static assets?

(the diff between Pages and Workers is a very confusing one for me. And Workflows has highlighted how difficult it is [for me] to grasp - but yes I know it's beta!)

Asking bc you seem to have leaned heavily into Remix+CF in the past... but I wonder if you would do so again with so many platform and framework options? workflows

SamOP•11/13/24, 11:27 AM

Am I correct in thinking that I should establish variables each time in a given step as if I define them outside of a step it may not work correctly?

e.g. is this the correct approach establishing my DB client in each step as opposed to one client variable in the parent scope?

SSam Am I correct in thinking that I should establish variables each time in a given ...

jordanfinners•11/13/24, 12:48 PM

I've defined mine in the run function and used it in the steps

it's working, whether it's best practice I don't know though

jordanfinners•11/13/24, 12:49 PM

Is there a way to alert on failed workflows?

SamOP•11/13/24, 12:56 PM

I thought you could potentially return from a step also, but not sure if that would work as it would be returning a connection not a normal value.

SamOP•11/13/24, 12:56 PM

Maybe someone from CF can clarify, I am using hyperdrive also as the connection string.

SamOP•11/13/24, 12:59 PM

Also it seems there is an undocumented payload limit on params for workflows.

When I try to pass a base64 in params it will fail to start the workflow every time but will not throw an error when I instantiate the workflow.

akhilravidas•11/13/24, 5:19 PM

Gm! Loving workflows in general but feels like i'm missing the right way to track errors. I get the exception message when i go to the flow instance page and wrangler tail but no stack trace so it takes a bit longer to debug. Any way to get the full stack trace?

SSam I thought you could potentially return from a step also, but not sure if that wo...

Isaac McFadyen•11/13/24, 7:02 PM

Values that are returned need to be serializable, so yes, you'll need to recreate the DB conn every step.

Isaac McFadyen•11/13/24, 7:02 PM

Additionally, your Workflow may be paused/move machines/etc between steps so putting a DB conn in the global scope isn't advised either

SamOP•11/13/24, 7:42 PM

Cheers for confirming yeah thought I read or heard words to the latter.

Hyperdrive should hopefully minimise the additional latency and reuse the connection I guess!

SSam Cheers for confirming yeah thought I read or heard words to the latter. Hyperd...

Isaac McFadyen•11/13/24, 8:13 PM

Yup - and realistically a DB connection is only on the order of several hundreds of ms/single-digit seconds at the most so unless your Workflow is very time-sensitive it probably won't matter that much.

surgery18•11/14/24, 7:05 PM

Is it possible To have a workflow that uses a websocket? Example. Let's say a user posts a comment on my site and I want that shot off to a workflow and the web client to listen to know when that workflow is done (via a websocket). The workflow does its steps needed on the post data and then the last step in the flow I want it to shoot off a websocket message saying it is done and then the web client can pick up that message and do what needs to be done after that. Is this something possible? Or does the client just have to keep polling the workflow for its status till it is done? Thanks!

Original message was deleted

surgery18•11/14/24, 7:26 PM

Ah ok. Well, that could be something you can add to workflow to have the ability to use websockets within workflow.

Jani•11/14/24, 7:59 PM

You can make workflow push updates to Durable Object, which has websocket support

Sameer Ali•11/14/24, 10:33 PM

Any plan on adding option in

cloudflare:test

cloudflare:test

to write test cases? or there is any other way to writing integeration test for steps?

pao ramen•11/15/24, 8:58 AM

Hi there! I've been porting some python code I had using inngest, and some other built with queues, and I have a bunch of feedback that might be useful.

For reference, the code that I've migrated to workflows usually does something like this:

fetches a bunch of resources from the DB
(fanout) for each resource it does an http call to an API and then it saves some data to the DB
(fanin) once everything is done, it makes more changes to the DB

I think this is the more straightforward kind of workflow you can imagine, and yet, I've found some shortcomings implementing them on Cloudflare:

The 256 step limit means that I can't build this as a single workflow. I have to add a queue to implement the "fanout". The alternative is to do a
```
Promise.allSettled
```
```
Promise.allSettled
```
, but then you don't get to retry failed workflow creation.
The http call has a rate limit, and unlike queues, there is no way to specify concurrency/batch size. Since I moved the "fanout" to a queue, this is where I implement the rate limit, but it would be great to be able to describe the whole workflow without queues.
Because of moving the fanout to queues, there is no way to do the fanin part since it can't be known when all the queues have been processed.
There is no way to compose workflows. This means that if you have a piece of workflow that is used in several places, you have to abstract away the steps. It would be great if workflows could return values so a workflow could call another workflow like a step.

Cheers,
Pao

pao ramen•11/15/24, 11:55 AM

One last thing... Is there a workaround to the "inserted row already exists in table while trying to run multiple workflows locally"? I can't have more than one workflow locally

Jeff•11/15/24, 5:55 PM

are you using the same workflow id/name?

JJeff are you using the same workflow id/name?

pao ramen•11/15/24, 10:05 PM

No. It's this issue: https://github.com/cloudflare/workers-sdk/issues/7127#issuecomment-2478815654

GitHub

🐛 BUG: Fatal uncaught kj::Exception: kj/table.c++:49: failed: inser...

Which Cloudflare product(s) does this pertain to? Workers Runtime, Wrangler, Other What version(s) of the tool(s) are you using? 3.83.0 What version of Node are you using? No response What operatin...

Timo Reimann•11/15/24, 10:57 PM

I'm not sure if I missed to see the answer to the following question anywhere: is an instantiated workflow guaranteed to run on a dedicated worker?

Asking to understand how the worker-specific limits (e.g., subrequests) possibly apply.

Evan•11/15/24, 11:52 PM

Any pointers to docs that describe how workflow can be used with workers for platforms?

edit: looks like this is not currently supported, but would love to understand what the timeline looks like. workflow + workers for platforms is exactly what I need and I'll unfortunately have to forgo workflow until the two can compose together and continue to work on my own workflow layer.

pao ramen•11/16/24, 10:06 AM

More feedback:

bug: when triggering a workflow from the UI, the json payload gets stringified (so it does not work at all)
it would be great to be able to retry workflows/steps from the UI

Sskip Hi @Cayter 👋. Would you recommend sticking with Cloudflare for Remix develope...

MissS•11/17/24, 8:08 PM

Remote isn’t possible while doing dev locally. I.e, can’t access remote R2 while running wrangler dev

So workers with assets is convenient but because it’s not fully fledged out, it depends on how “serious” your project requirements.

The eta for doing running wrangler remotely is this quarter, but I wouldn’t trust CF and assume it would be available sometime early next year

Ppao ramen Hi there! I've been porting some python code I had using inngest, and some other...

MissS•11/17/24, 8:21 PM

Apparently, they are working on making workflows have a built in rate limiter. I wouldn’t assume this to be release anytime soon

Without specific numbers than giving advice is difficult but you could potentially do this with breaking up your fanout into multiple workflows. It’s also possible to have a “always on” workflow that manages other workflows although this solution might be less elegant than just using a queue.

Ppao ramen Hi there! I've been porting some python code I had using inngest, and some other...

MissS•11/17/24, 8:22 PM

Can’t you start another workflow from an existing workflow?

MMissS Can’t you start another workflow from an existing workflow?

pao ramen•11/17/24, 9:29 PM

Yes, but unlike steps, you don't get the result

kchro3•11/18/24, 2:21 AM

I have a question about workflow IDs. It supports setting the ID in case there is some external ID like a user ID, but it sounds like you cannot reuse IDs. Doesn't that defeat the purpose of being able to reference a workflow?

Is the recommendation to append a timestamp with the external ID or something for a recurring workflow per user ID?

MMissS Remote isn’t possible while doing dev locally. I.e, can’t access remote R2 while...

skip•11/18/24, 2:58 AM

Thank MissS :). What's the best way to stay up to date on the progress on Cloudflares side?

Also when running more than 2 workflows per second im getting throttled. Sometimes responses from

Similar Threads

Similar Threads

Similar Threads