It’s hard without knowing the underlying data, but the step name as you linked is the cache key. So

AAshley It’s hard without knowing the underlying data, but the step name as you linked i...

M

I can provide a little more context. I'm the developer of https://railforless.us/ which scrapes prices for Amtrak's trains. They use a bucket pricing system and I am working on a workflow that will find every current bucket (I know there are 8). I am scraping 14 days of prices per step, so I would loop until I have found all 8 buckets (or give up after x tries). It sounds like I could use the date range as the key and it would correctly cache the state of that step?

RailForLess.us

Taking the train shouldn't be a pain. Make your next Amtrak trip a reality with flexible scheduling and the tools you need to plan the perfect trip.

A

AshleyOP•1/18/25, 5:39 PM

Yes, I think so, as long as the date is consistent (e.g. not using “now” functions where the time changes etc so make sure it’s just YYYY-MM-DD or something)

M

Mr. Technician•1/18/25, 11:29 PM

it would be, yeah

MMatt Silverlock Can you share your account ID here? (it’s safe to share publicly)

R

Rakhim•1/19/25, 10:28 AM

any updates? I had to reduce the pace of creating new workflow instances in an attempt to reduce the backlog, it works but very slowly. I'm still never seeing anything close to 100 concurrent workflows

MMr. Technician I can provide a little more context. I'm the developer of https://railforless.us...

T

thousandmiles•1/20/25, 5:39 AM

great tool you built. so is your website purely serverless on cf right now?

Tthousandmiles great tool you built. so is your website purely serverless on cf right now?

M

Mr. Technician•1/20/25, 5:41 AM

thanks! My buddy takes credit for the interface while i handled all of the API pieces. But yes, it’s purely serverless on cf with the exception of our remote puppeteer service, since the browser rendering API in workers is missing features we need

M

Mr. Technician•1/20/25, 5:43 AM

i use websockets, durable objects, queues, d1, and cron jobs

M

Mr. Technician•1/20/25, 5:43 AM

and kv

MMr. Technician thanks! My buddy takes credit for the interface while i handled all of the API p...

T

thousandmiles•1/20/25, 5:45 AM

impressive! though i am not quite proficient on cf worker myself, but i did manage some heavy workload using aws lambda/step functions back in 2 years.

M

Mr. Technician•1/20/25, 5:46 AM

i originally wrote the backend in .net to run in azure functions, then rewrote it all in typescript to leverage workers

MMr. Technician i use websockets, durable objects, queues, d1, and cron jobs

T

thousandmiles•1/20/25, 5:47 AM

are you planning to opensource (even parts of) this backend as well?

MMr. Technician i originally wrote the backend in .net to run in azure functions, then rewrote i...

T

thousandmiles•1/20/25, 5:48 AM

way to go. this is exactly i am doing now, u know, think serverless. even LLM/agentic workflow is going serverless

M

Mr. Technician•1/20/25, 5:48 AM

possibly. the react front-end is open-source presently

M

Mr. Technician•1/20/25, 5:49 AM

i hope my “secret” is kept safe from amtrak who would probably rather this service didn’t exist

M

Mr. Technician•1/20/25, 5:50 AM

it’s quite frankly only possible because of how horribly designed their booking system is

J

Jikyu•1/20/25, 6:06 AM

Can a workflow call itself? Like how a worker can call itself via service binding

MMr. Technician i hope my “secret” is kept safe from amtrak who would probably rather this servi...

T

thousandmiles•1/20/25, 6:08 AM

haha, that makes sense.

L

Luca (steebchen)•1/20/25, 9:50 AM

it would be awesome to have full text search on the params on all workflows as well

E

eduardo | Latitude.sh•1/20/25, 12:42 PM

Error handling seems quite complex. I have a tail worker that should post to the bug tracking app, but it doesn't seem to catch exceptions at all

tail worker excerpt:

  async tail(events, env, ctx) {
    // Check if events array exists and has items
    if (!Array.isArray(events) || events.length === 0) {
      return new Response("No events found", { status: 200 });
    }

    // Only proceed if there's an exception AND exceptions array is not empty
    const event = events[0];
    if (event.outcome !== "exception" || !event.exceptions?.length) {
      return new Response("No exception found", { status: 200 });
    }

    // Log the event structure to help us understand the data
    console.log("Tail worker event:", JSON.stringify(events, null, 2));

...

  async tail(events, env, ctx) {
    // Check if events array exists and has items
    if (!Array.isArray(events) || events.length === 0) {
      return new Response("No events found", { status: 200 });
    }

    // Only proceed if there's an exception AND exceptions array is not empty
    const event = events[0];
    if (event.outcome !== "exception" || !event.exceptions?.length) {
      return new Response("No exception found", { status: 200 });
    }

    // Log the event structure to help us understand the data
    console.log("Tail worker event:", JSON.stringify(events, null, 2));

...

  async tail(events, env, ctx) {
    // Check if events array exists and has items
    if (!Array.isArray(events) || events.length === 0) {
      return new Response("No events found", { status: 200 });
    }

    // Only proceed if there's an exception AND exceptions array is not empty
    const event = events[0];
    if (event.outcome !== "exception" || !event.exceptions?.length) {
      return new Response("No exception found", { status: 200 });
    }

    // Log the event structure to help us understand the data
    console.log("Tail worker event:", JSON.stringify(events, null, 2));

...

  async tail(events, env, ctx) {
    // Check if events array exists and has items
    if (!Array.isArray(events) || events.length === 0) {
      return new Response("No events found", { status: 200 });
    }

    // Only proceed if there's an exception AND exceptions array is not empty
    const event = events[0];
    if (event.outcome !== "exception" || !event.exceptions?.length) {
      return new Response("No exception found", { status: 200 });
    }

    // Log the event structure to help us understand the data
    console.log("Tail worker event:", JSON.stringify(events, null, 2));

...

workflow step

        const payload = await step.do('prepares payload', async () => {
            const isDev = this.env.ENVIRONMENT === 'dev';
            const data = isDev ? mockData : event.payload;
            
            console.log("Received payload:", JSON.stringify(data, null, 2));
            throw new Error("Testing error handling with payload data");
            
            return data;
        });

        const payload = await step.do('prepares payload', async () => {
            const isDev = this.env.ENVIRONMENT === 'dev';
            const data = isDev ? mockData : event.payload;
            
            console.log("Received payload:", JSON.stringify(data, null, 2));
            throw new Error("Testing error handling with payload data");
            
            return data;
        });

        const payload = await step.do('prepares payload', async () => {
            const isDev = this.env.ENVIRONMENT === 'dev';
            const data = isDev ? mockData : event.payload;
            
            console.log("Received payload:", JSON.stringify(data, null, 2));
            throw new Error("Testing error handling with payload data");
            
            return data;
        });

        const payload = await step.do('prepares payload', async () => {
            const isDev = this.env.ENVIRONMENT === 'dev';
            const data = isDev ? mockData : event.payload;
            
            console.log("Received payload:", JSON.stringify(data, null, 2));
            throw new Error("Testing error handling with payload data");
            
            return data;
        });

What am I missing? It never throws the error

L

Lila Rest•1/20/25, 3:58 PM

Hey there! Is there any possibility of running some of the steps of a workflow in parallel? For example, in different worker invocations to minimize the overall execution speed?

LLila Rest Hey there! Is there any possibility of running some of the steps of a workflow i...

M

Matt Silverlock•1/21/25, 4:05 PM

You can always fire off another Workflow, even from within a Workflow. Workflows are the way to run tasks concurrently, or in parallel.

B

Bonadio•1/21/25, 6:36 PM

Hi, what is the use of the return of the run function in the workflow?
I saw in the twillio example that it returns a value.
https://github.com/craigsdennis/twilio-cloudflare-workflow/blob/main/src/index.ts

export class NeverGonnaWorkflow extends WorkflowEntrypoint<Env, NeverGonnaParams> {
    async run(event: WorkflowEvent<NeverGonnaParams>, step: WorkflowStep) {
        const { to, host, content } = event.payload;
        await step.sleep('wait for the right moment', '135 seconds');
        const callSid = await step.do('call person back', async () => {
            const client = new Twilio(this.env.TWILIO_ACCOUNT_SID, this.env.TWILIO_AUTH_TOKEN);
            // Say verb here does text to speech
            // Play allows you to play media.
            // The mp3 file is hosted on Cloudflare Workers using dynamic assets
            const twiml = `
            <Response>
                <Say>Hello from a Cloudflare Workflow!</Say>
                <Say>You said "${content}".</Say>
                <Say>Check out this classic:</Say>
                <Play>https://${host}/classic.mp3</Play>
            </Response>`;
            const call = await client.calls.create({
                to,
                from: this.env.TWILIO_PHONE_NUMBER,
                twiml,
            });
            return call.sid;
        });

        return { success: true, callSid };   // <------ How to use this?

    }
}

export class NeverGonnaWorkflow extends WorkflowEntrypoint<Env, NeverGonnaParams> {
    async run(event: WorkflowEvent<NeverGonnaParams>, step: WorkflowStep) {
        const { to, host, content } = event.payload;
        await step.sleep('wait for the right moment', '135 seconds');
        const callSid = await step.do('call person back', async () => {
            const client = new Twilio(this.env.TWILIO_ACCOUNT_SID, this.env.TWILIO_AUTH_TOKEN);
            // Say verb here does text to speech
            // Play allows you to play media.
            // The mp3 file is hosted on Cloudflare Workers using dynamic assets
            const twiml = `
            <Response>
                <Say>Hello from a Cloudflare Workflow!</Say>
                <Say>You said "${content}".</Say>
                <Say>Check out this classic:</Say>
                <Play>https://${host}/classic.mp3</Play>
            </Response>`;
            const call = await client.calls.create({
                to,
                from: this.env.TWILIO_PHONE_NUMBER,
                twiml,
            });
            return call.sid;
        });

        return { success: true, callSid };   // <------ How to use this?

    }
}

export class NeverGonnaWorkflow extends WorkflowEntrypoint<Env, NeverGonnaParams> {
    async run(event: WorkflowEvent<NeverGonnaParams>, step: WorkflowStep) {
        const { to, host, content } = event.payload;
        await step.sleep('wait for the right moment', '135 seconds');
        const callSid = await step.do('call person back', async () => {
            const client = new Twilio(this.env.TWILIO_ACCOUNT_SID, this.env.TWILIO_AUTH_TOKEN);
            // Say verb here does text to speech
            // Play allows you to play media.
            // The mp3 file is hosted on Cloudflare Workers using dynamic assets
            const twiml = `
            <Response>
                <Say>Hello from a Cloudflare Workflow!</Say>
                <Say>You said "${content}".</Say>
                <Say>Check out this classic:</Say>
                <Play>https://${host}/classic.mp3</Play>
            </Response>`;
            const call = await client.calls.create({
                to,
                from: this.env.TWILIO_PHONE_NUMBER,
                twiml,
            });
            return call.sid;
        });

        return { success: true, callSid };   // <------ How to use this?

    }
}

export class NeverGonnaWorkflow extends WorkflowEntrypoint<Env, NeverGonnaParams> {
    async run(event: WorkflowEvent<NeverGonnaParams>, step: WorkflowStep) {
        const { to, host, content } = event.payload;
        await step.sleep('wait for the right moment', '135 seconds');
        const callSid = await step.do('call person back', async () => {
            const client = new Twilio(this.env.TWILIO_ACCOUNT_SID, this.env.TWILIO_AUTH_TOKEN);
            // Say verb here does text to speech
            // Play allows you to play media.
            // The mp3 file is hosted on Cloudflare Workers using dynamic assets
            const twiml = `
            <Response>
                <Say>Hello from a Cloudflare Workflow!</Say>
                <Say>You said "${content}".</Say>
                <Say>Check out this classic:</Say>
                <Play>https://${host}/classic.mp3</Play>
            </Response>`;
            const call = await client.calls.create({
                to,
                from: this.env.TWILIO_PHONE_NUMBER,
                twiml,
            });
            return call.sid;
        });

        return { success: true, callSid };   // <------ How to use this?

    }
}

GitHub

twilio-cloudflare-workflow/src/index.ts at main · craigsdennis/twil...

Use Cloudflare Workers to create a scheduled durable execution with your Twilio applications - craigsdennis/twilio-cloudflare-workflow

BBonadio Hi, what is the use of the return of the run function in the workflow? I saw in...

M

Matt Silverlock•1/21/25, 6:51 PM

You can capture that value when you inspect an instances’ status. This can be useful if your Workflow is computing some kind of result, storing data against a key, or generating some kind of identifier

MMatt Silverlock You can always fire off another Workflow, even from within a Workflow. Workflows...

L

Lila Rest•1/22/25, 8:31 AM

Thanks Matt! And will this new workflow run in the same thread or in another?
I know that workers called from other workers will run on the same resources. We are executing a huge tree of operations and I’m trying to maximize the speed of execution.

LLila Rest Thanks Matt! And will this new workflow run in the same thread or in another? I ...

M

Matt Silverlock•1/22/25, 11:03 AM

That wont affect it here.

MMatt Silverlock That wont affect it here.

L

Lila Rest•1/22/25, 3:31 PM

To be 100% sure you're saying that calling a workflow from another will happen in two separate threads?

LLila Rest To be 100% sure you're saying that calling a workflow from another will happen i...

M

Matt Silverlock•1/22/25, 3:39 PM

I'm saying it's no different from any other Worker invocation. If the Workflow is a different Workflow (script) then it is likely to spawn a new isolate. We don't make any more guarantees than that.

MMark Hey all. Does anyone know what token permission is required to create a new wor...

M

Matt Silverlock•1/23/25, 5:59 PM

Workers scripts edit + read:

MMatt Silverlock I'm saying it's no different from any other Worker invocation. If the Workflow i...

L

Lila Rest•1/24/25, 9:08 AM

Ah I see, all clear!

MMatt Silverlock Can you share your account ID here? (it’s safe to share publicly)

M

m.muiznieks•1/26/25, 3:08 PM

Yes, I've noticed the same thing for me as well.
Also the response that request is queued comes after 60-80s.

The time until START of the step since receiving is 40+ minutes. :))

(thats the reason I came here for looking is this an issue for anyone else).

My account ID: 5e3dea7b807381cba36b9292bde14920

Additionally I can say that even sending the requests to my workflow worker gets even 60 to 90 seconds to receive any response from them. Is this a feature? I was not aware of this before.

A

Aslak Hellesoy•1/26/25, 10:12 PM

Hi all. Is it possible to pause/resume workflow steps and wait for user input?

We're using workflows for order processing. For new users, we require that the user verifies their identity with an external KYC provider. Is there a way to wrap this in a step that will block until the user has verified their identity? Some kind of signal mechanism similar to what Temporal has?

Thanks in advance.

Mm.muiznieks Yes, I've noticed the same thing for me as well. Also the response that request...

M

m.muiznieks•1/27/25, 2:30 AM

Also... around 60 instances are marked as queued. But none shows up at any time window. Are there any possibilities to "restart" the workflow?

C

Chris Jayden•1/27/25, 8:13 AM

I'm running a workflow to do some R2 work. However, when I'm not running

--remote

--remote

, no real operations are taking place on my buckets. So calling

list

list

doesn't return anything.

however, when i do run

--remote

--remote

and call my worfklow, i get a

workflow.not_found

workflow.not_found

.

Is there something I'm missing?

CChris Jayden I'm running a workflow to do some R2 work. However, when I'm not running `--remo...

H

Hello, I’m Allie!•1/27/25, 8:17 AM

Afaik, remote mode doesn’t work for Workflows yet

HHello, I’m Allie!Afaik, remote mode doesn’t work for Workflows yet

C

Chris Jayden•1/27/25, 8:18 AM

Thanks for chiming in. Any other way to test my workflows w/ R2?

CChris Jayden Thanks for chiming in. Any other way to test my workflows w/ R2?

H

Hello, I’m Allie!•1/27/25, 8:21 AM

You should be able to do lists in local mode, via the CLI, if that is what you mean?

HHello, I’m Allie!You should be able to do lists in local mode, via the CLI, if that is what you m...

C

Chris Jayden•1/27/25, 10:02 AM

I'm trying to query my file in my worker during a workflow. So env.MY_BUCKET.get() or env.MY_BUCKET.list(). But it won't return actual files since I'm developing locally, but i can't set it to

--remote

--remote

because workflows are not supported in remote

CChris Jayden I'm trying to query my file in my worker during a workflow. So env.MY_BUCKET.get...

H

Hello, I’m Allie!•1/27/25, 10:42 AM

Yes, you would need to seed the local store with data

HHello, I’m Allie!Yes, you would need to seed the local store with data

C

Chris Jayden•1/27/25, 10:55 AM

Ohhhhh you can do that? Cool. i didn't know. Thanks that helps

C

Chris Jayden•1/27/25, 11:00 AM

Another question, is it fine to map promises like this with Cloudflare workflows?

const uploads = await Promise.all(
    batch.map(async (image, index) => {
        // Add 1 second sleep between uploads, but not before the first one
        if (index > 0) {
            await step.sleep(`delay-upload-${i}-${index}`, "1 second");
        }

        return step.do(
            `process-image-${i}`,
            {
                retries: {
                    limit: 3,
                    delay: "5 seconds",
                    backoff: "exponential",
                },
            },
            () => this.downloadAndUploadImage(image, orgId)
        );
    })
);

const uploads = await Promise.all(
    batch.map(async (image, index) => {
        // Add 1 second sleep between uploads, but not before the first one
        if (index > 0) {
            await step.sleep(`delay-upload-${i}-${index}`, "1 second");
        }

        return step.do(
            `process-image-${i}`,
            {
                retries: {
                    limit: 3,
                    delay: "5 seconds",
                    backoff: "exponential",
                },
            },
            () => this.downloadAndUploadImage(image, orgId)
        );
    })
);

const uploads = await Promise.all(
    batch.map(async (image, index) => {
        // Add 1 second sleep between uploads, but not before the first one
        if (index > 0) {
            await step.sleep(`delay-upload-${i}-${index}`, "1 second");
        }

        return step.do(
            `process-image-${i}`,
            {
                retries: {
                    limit: 3,
                    delay: "5 seconds",
                    backoff: "exponential",
                },
            },
            () => this.downloadAndUploadImage(image, orgId)
        );
    })
);

const uploads = await Promise.all(
    batch.map(async (image, index) => {
        // Add 1 second sleep between uploads, but not before the first one
        if (index > 0) {
            await step.sleep(`delay-upload-${i}-${index}`, "1 second");
        }

        return step.do(
            `process-image-${i}`,
            {
                retries: {
                    limit: 3,
                    delay: "5 seconds",
                    backoff: "exponential",
                },
            },
            () => this.downloadAndUploadImage(image, orgId)
        );
    })
);

F

Flogert Çiku•1/28/25, 5:40 PM

Hey there ? Is there possible to add workflows inside an existing worker ?

FFlogert Çiku Hey there ? Is there possible to add workflows inside an existing worker ?

A

AshleyOP•1/28/25, 5:43 PM

Yes, you just need to export the class that implements your workflow and set the appropriate bindings in wrangler.toml as per the docs

F

Flogert Çiku•1/28/25, 5:45 PM

@Ashley thanks, the idea is that some functionalities of the worker to be executed as workflow steps.
and also the existing worker is in js, so far there is only documentation in .ts

D

D Trombett•1/29/25, 6:22 PM

For how long after a workflow is finished,

WORKFLOW.get(id)

WORKFLOW.get(id)

will return the finished instance? Also, does

.restart()

.restart()

work for finished workflows too?

Mm.muiznieks Yes, I've noticed the same thing for me as well. Also the response that request...

M

m.muiznieks•1/29/25, 7:46 PM

I am on the Workers paid plan.

For at least 3 - 4 days I see that the workflows are flowing... not as expected.

They build up a massive queue (which seems to grow bigger day by day) and have long inactivity periods (as visible on the screen—before that, there was no activity for 1h+). Meanwhile, usually I've not seen more than 28 instances running, in rare cases they hit at one moment higher, but havent seen near 100.

The workflow itself contains just one simple request (to Meta conversion API), so there is nothing resource-demanding (because I'm testing if and how to scale the workflow usage).

At the error logs I see that the requests might fail due to "Durable Object restarting," because of some code updates. Thats cool, but I'm just waiting when this queue will be "errored" when the wrong timing hits.

So my concerns and question is - are there some other limits that must be taken in mind when building the Workflows? Or is this some issue related to my account or what?

I understand that it's low possibility to get an answer here, but I still hope on that. :))

Thank you .

Mm.muiznieks I am on the Workers paid plan. For at least 3 - 4 days I see that the workflow...

M

m.muiznieks•1/29/25, 7:58 PM

Addition to previous post - here is the situation on another Flow - queue just getting bigger, but nothing is processed. It feels like that only one workflow connected to the worker is running. Did I miss this limit somewhere if its true?

Mm.muiznieks I am on the Workers paid plan. For at least 3 - 4 days I see that the workflow...

S

Seekerdasbatatas•1/29/25, 9:07 PM

Are there any other workflows in your account? Note that the concurrency limit is account wide

SSeekerdasbatatas Are there any other workflows in your account? Note that the concurrency limit i...

M

m.muiznieks•1/29/25, 9:41 PM

No. These are two working workflows, which basically do the same thing.
By looking into possible solutions I divided the requests in half.

That helped a bit, as the active one started to process something and now the queue is slowly decreasing, but the other one - does not do anything, and now is up to 2k in the queue (both waiting queue sums are the same as before and keeps increasing).

For wider context - they are receiving the requests from the Zaraz custom-managed component. But I don't know if it has anything to do with this unless that affects some other underlying processes.

Mm.muiznieks No. These are two working workflows, which basically do the same thing. By look...

S

Seekerdasbatatas•1/29/25, 9:42 PM

Huh weird, thanks for letting us know - can you send your account id for us to investigate?

It’s hard without knowing the underlying data, but the step name as you linked is the cache key. So

Similar Threads