im trying to find how to split the file into multipart without hiting the limit

T

TunaOP•2/21/25, 4:18 PM

step 1: init multipart upload command
step 2: creating an array of steps, each step does. promise.all([promises of uploading a part]) -> maximum 2 concurent uploads
step 3. complete request
if failed abort reqeust

TTuna im trying to find how to split the file into multipart without hiting the limit

S

Seekerdasbatatas•2/21/25, 5:59 PM

if you mean R2's multipart upload, you could calculate the part size by dividing the total file size by 990 sub-requests (if you have the file length, i also don't do a 1000 to give you some leeway for other stuff ): https://developers.cloudflare.com/r2/objects/multipart-objects/

Cloudflare Docs

Multipart upload · Cloudflare R2 docs

R2 supports S3 API's Multipart Upload with some limitations.

S

Seekerdasbatatas•2/21/25, 6:00 PM

the maximum is 5GiB so using this method you could upload 5TiB (ish)

F

fforres•2/22/25, 7:14 PM

Hey folks, been using workflows is a lot more lately, mostly as encapsulation of agentic flows.

Wondering if there are either plans or existing strategies to visualize AI Gateway calls/entries related to workflow runs?

(same w/ sub-request or triggering sub-workflows?)

So far, thinking of just adding some metadata to the LLM calls... and then manual filter, but was thinking as you folks have control over the gateway + knowldge of the workflow/worker Instances there was something better.

(Would be awesome to be able to shove metadata/custom data to workflow metrics)

Ffforres Hey folks, been using workflows is a lot more lately, mostly as encapsulation of...

M

Matt Silverlock•2/23/25, 12:38 AM

What kind of metadata?

T

TunaOP•2/23/25, 7:09 AM

Hey everyone what is the road map of the workflows? is it something i can start using on production ?

TTuna Hey everyone what is the road map of the workflows? is it something i can start ...

T

tadhglewis•2/23/25, 7:15 AM

It's not public. It's beta so inherently no, it's not business critical ready. Yes some people probably use it for prod systems, depends on risk appetite

SSeekerdasbatatas if you mean R2's multipart upload, you could calculate the part size by dividing...

T

TunaOP•2/23/25, 7:18 AM

Im using one workflow to for multipart upload, and with current limitation the largest file i can support is 15 GiB and not 5TiB file.
taking into consideration 15 minutes time limit, and if every upload part in worst case will retry.
can you take a look at my calculator https://rb.gy/tdpydr, maybe im wrong ? i have seen a lot of question regards limits, it would be nice to create a helper so developers can use this to manage their quota limits.

Free link shortener: Fast & simple short URLs

Need to shorten a URL? Use RB.GY for free, memorable shortened links that are easy to track. Ideal for social media, emails, and ads.

Ttadhglewis It's not public. It's beta so inherently no, it's not business critical ready. Y...

T

TunaOP•2/23/25, 7:24 AM

Thanks for the quick answer!

My use case is to move every uploaded file to R2 to another storage provider. For files larger than 100 MiB, I’m using the workflow for multipart upload (start multipart, upload parts, complete, or abort on failure).

Do you think the workflow beta version can handle this in production at 100 TiB of files per day, or am I way off? If this beta isn’t suited for such scale, is there another Cloudflare service you’d recommend that could handle this load and make it easier to transition my architecture from the workflow?

TTuna Thanks for the quick answer! 😊 My use case is to move every uploaded file to R...

M

Matt Silverlock•2/23/25, 9:38 PM

For clarity: it's not the volume of data, it's that a beta service might experience the occasional disruption or bug as we continue to iterate on it. We have teams and other customers using Workflows in production, but we ask folks to decide for themselves here.

S

steyblind•2/24/25, 1:12 AM

Hi all, just started using workflows. Overall, extremely rad.

Something baffling me is the __LOCAL_DEV_STEP_OUTPUTS and what the equivalent would be from a deployed workflow? obvs I can trial-and-error this, but I wanted to ID documentation on this vs. "production" outputs

MMatt Silverlock What kind of metadata?

F

fforres•2/24/25, 5:47 AM

Oh, like attaching the workflow-id to the

cf-aig-metadata

cf-aig-metadata

And then filter in gateway view for it.

A

ANBU•2/24/25, 9:15 AM

Hey, I'm working on a next js app that is served via workers (not pages) with open nextjs
Now, in the same app, I'm trying to use a workflow but getting this error

Worker "workflows:test"'s binding "USER_WORKFLOW" refers to service "core:user:" with a named entrypoint "TestWorkflow", but "core:user:" has no such named entrypoint.

Worker "workflows:test"'s binding "USER_WORKFLOW" refers to service "core:user:" with a named entrypoint "TestWorkflow", but "core:user:" has no such named entrypoint.

Worker "workflows:test"'s binding "USER_WORKFLOW" refers to service "core:user:" with a named entrypoint "TestWorkflow", but "core:user:" has no such named entrypoint.

Worker "workflows:test"'s binding "USER_WORKFLOW" refers to service "core:user:" with a named entrypoint "TestWorkflow", but "core:user:" has no such named entrypoint.

I can see some mentions of this error in this channel, but they are related to pages deployment, which I'm not using.
Any tips on how to move forward?

AANBU Hey, I'm working on a next js app that is served via workers (not pages) with op...

M

Matt Silverlock•2/24/25, 10:57 AM

Can you show your code? Are you exporting the Workflow definition?

MMatt Silverlock Can you show your code? Are you exporting the Workflow definition?

A

ANBU•2/24/25, 4:01 PM

I'm using the code mentioned here https://developers.cloudflare.com/workflows/get-started/cli-quick-start/

This is what my directory structure looks like

.
├── src
│   ├── app (next js app)
│   │   ├── api
│   │   │   └── [[...route]]
│   │   │       └── route.ts
│   │   ├── page.tsx
│   │   └── layout.tsx
│   └── server (hono server)
│       ├── routes
│       │   └── workflow-route.ts
│       ├── workflows
│       │   └── my-workflow.ts
│       └── _app.ts 
├── wrangler.jsonc
└── package.json

.
├── src
│   ├── app (next js app)
│   │   ├── api
│   │   │   └── [[...route]]
│   │   │       └── route.ts
│   │   ├── page.tsx
│   │   └── layout.tsx
│   └── server (hono server)
│       ├── routes
│       │   └── workflow-route.ts
│       ├── workflows
│       │   └── my-workflow.ts
│       └── _app.ts 
├── wrangler.jsonc
└── package.json

.
├── src
│   ├── app (next js app)
│   │   ├── api
│   │   │   └── [[...route]]
│   │   │       └── route.ts
│   │   ├── page.tsx
│   │   └── layout.tsx
│   └── server (hono server)
│       ├── routes
│       │   └── workflow-route.ts
│       ├── workflows
│       │   └── my-workflow.ts
│       └── _app.ts 
├── wrangler.jsonc
└── package.json

.
├── src
│   ├── app (next js app)
│   │   ├── api
│   │   │   └── [[...route]]
│   │   │       └── route.ts
│   │   ├── page.tsx
│   │   └── layout.tsx
│   └── server (hono server)
│       ├── routes
│       │   └── workflow-route.ts
│       ├── workflows
│       │   └── my-workflow.ts
│       └── _app.ts 
├── wrangler.jsonc
└── package.json

Cloudflare Docs

CLI quick start · Cloudflare Workflows docs

Workflows allow you to build durable, multi-step applications using the Workers platform. A Workflow can automatically retry, persist state, run for hours or days, and coordinate between third-party APIs.

A

ANBU•2/24/25, 4:04 PM

I'm exporting the workflow in my-workflow.ts

A

ANBU•2/24/25, 4:23 PM

I also tinkered with the given example on a separate repo created using

npm create cloudflare@latest workflows-starter -- --template "cloudflare/workflows-starter"

npm create cloudflare@latest workflows-starter -- --template "cloudflare/workflows-starter"

and noticed that, even if I define the workflow in other file, I still need to export it from index.ts, I think this is somewhat related, but not able to figure this out in next js

AANBU I also tinkered with the given example on a separate repo created using `npm cre...

M

Matt Silverlock•2/24/25, 4:57 PM

TestWorkflow

TestWorkflow

isn't from the example (from your error message) so I suspect you changed a binding or the Workflow name itself?

M

Matt Silverlock•2/24/25, 4:57 PM

with a named entrypoint "TestWorkflow"

with a named entrypoint "TestWorkflow"

If you didn't follow the guide as-is, it won't work

MMatt Silverlock `TestWorkflow` isn't from the example (from your error message) so I suspect you...

A

ANBU•2/24/25, 5:10 PM

Yup I changed the name in the bindings and code but rest all is same

A

ANBU•2/24/25, 5:15 PM

Just to be extra sure, created another workflow with the same name and bindings
Still getting the same error with MyWorkflow

A

ANBU•2/24/25, 5:16 PM

A

ANBU•2/24/25, 5:18 PM

AANBU Click to see attachment

M

Matt Silverlock•2/24/25, 5:51 PM

@Luís Duarte

@Diogo Ferreira - this one seems off?

D

dan•2/24/25, 7:07 PM

I see this message: workflows
but didn't see a response to it - is there a way to listen to workflow state via a stream?

AANBU Hey, I'm working on a next js app that is served via workers (not pages) with op...

S

Seekerdasbatatas•2/24/25, 7:37 PM

hey AFAICT, OpenNext uses their own entrypoint shim to be able to interoperate with Next: https://github.com/opennextjs/opennextjs-cloudflare/blob/main/packages/cloudflare/src/cli/templates/worker.ts

you will probably need to extend this worker template to export your workflow in the worker entrypoint file

GitHub

opennextjs-cloudflare/packages/cloudflare/src/cli/templates/worker....

Open Next.js adapter for Cloudflare. Contribute to opennextjs/opennextjs-cloudflare development by creating an account on GitHub.

S

Seekerdasbatatas•2/24/25, 7:38 PM

(this also applies to DOs, for example)

S

Seekerdasbatatas•2/24/25, 7:55 PM

or you can create another script/worker that just exports the workflow directly

Ddan I see this message: https://discord.com/channels/595317990191398933/129692370779...

M

Matt Silverlock•2/25/25, 11:32 AM

Not at the moment.

SSeekerdasbatatas or you can create another script/worker that just exports the workflow directly

A

ANBU•2/25/25, 12:21 PM

Sorry, I didn't get this one, it sounds simpler. How can this be achieved?

SSeekerdasbatatas hey AFAICT, OpenNext uses their own entrypoint shim to be able to interoperate w...

A

ANBU•2/25/25, 1:39 PM

Tried this by copying this file, exporting my workflow from there and changing main file in wrangler.jsonc
But still getting the same error
Is there anything else thats needed apart from this?

MMatt Silverlock Not at the moment.

D

dan•2/25/25, 4:28 PM

It'd be cool to see that at some point - it's a blocking feature for me to use this for my AI workflows

W

WILDE•2/25/25, 4:53 PM

Hi there! I am trying to replace some queues with workflows and I've realized that there is a known issue

Workflows does not support npx wrangler dev --remote.

Workflows does not support npx wrangler dev --remote.

I'd like to know if this is something that will be fixed in the future or will remain as it is. I have some workers that are connected to some external services such as clickhouse, upstash and some others. Build them locally with Docker for

yarn dev

yarn dev

to work will be crazy

A

ajay | dreamlit.ai•2/26/25, 1:37 AM

Hi, big fan of workflows, excited to start using it in our app.

Quick question, whenever Wrangler detects a change, it hot reloads the server, which kills any currently running workflows. Is there any way to either 1) keep them alive or 2) have some kind of startup script/hook that runs so we can run relaunch them ourselves?

D

DanGamble•2/26/25, 9:47 AM

abstract class BaseWorkflow extends WorkflowEntrypoint<Env, Params> {
  async run(event: WorkflowEvent<Params>, step: WorkflowStep) {
    const something = await step.do("do something", async () => {
      // ...
    })

    await this.execute(event, step)
  }

  abstract execute(event: WorkflowEvent<Params>, step: WorkflowStep): Promise<void>
}

class TestWorkflow extends BaseWorkflow {
  async execute(event: WorkflowEvent<Params>, step: WorkflowStep) {
    const something = await step.do("do something", async () => {
      // ...
    })
    
    await someStepsIWantToShare(event, step)
  }
}

async function someStepsIWantToShare (event: WorkflowEvent<Params>, step: WorkflowStep) {
  const anotherStep = await step.do("do something else", async () => {
    // ...
  })
}

abstract class BaseWorkflow extends WorkflowEntrypoint<Env, Params> {
  async run(event: WorkflowEvent<Params>, step: WorkflowStep) {
    const something = await step.do("do something", async () => {
      // ...
    })

    await this.execute(event, step)
  }

  abstract execute(event: WorkflowEvent<Params>, step: WorkflowStep): Promise<void>
}

class TestWorkflow extends BaseWorkflow {
  async execute(event: WorkflowEvent<Params>, step: WorkflowStep) {
    const something = await step.do("do something", async () => {
      // ...
    })
    
    await someStepsIWantToShare(event, step)
  }
}

async function someStepsIWantToShare (event: WorkflowEvent<Params>, step: WorkflowStep) {
  const anotherStep = await step.do("do something else", async () => {
    // ...
  })
}

abstract class BaseWorkflow extends WorkflowEntrypoint<Env, Params> {
  async run(event: WorkflowEvent<Params>, step: WorkflowStep) {
    const something = await step.do("do something", async () => {
      // ...
    })

    await this.execute(event, step)
  }

  abstract execute(event: WorkflowEvent<Params>, step: WorkflowStep): Promise<void>
}

class TestWorkflow extends BaseWorkflow {
  async execute(event: WorkflowEvent<Params>, step: WorkflowStep) {
    const something = await step.do("do something", async () => {
      // ...
    })
    
    await someStepsIWantToShare(event, step)
  }
}

async function someStepsIWantToShare (event: WorkflowEvent<Params>, step: WorkflowStep) {
  const anotherStep = await step.do("do something else", async () => {
    // ...
  })
}

abstract class BaseWorkflow extends WorkflowEntrypoint<Env, Params> {
  async run(event: WorkflowEvent<Params>, step: WorkflowStep) {
    const something = await step.do("do something", async () => {
      // ...
    })

    await this.execute(event, step)
  }

  abstract execute(event: WorkflowEvent<Params>, step: WorkflowStep): Promise<void>
}

class TestWorkflow extends BaseWorkflow {
  async execute(event: WorkflowEvent<Params>, step: WorkflowStep) {
    const something = await step.do("do something", async () => {
      // ...
    })
    
    await someStepsIWantToShare(event, step)
  }
}

async function someStepsIWantToShare (event: WorkflowEvent<Params>, step: WorkflowStep) {
  const anotherStep = await step.do("do something else", async () => {
    // ...
  })
}

Is this a valid way for work with Workflows?

M

Matt Silverlock•2/26/25, 7:41 PM

FYI: we've just raised the # of concurrent Workflows from ~100 to ~1000 (briefly) to (now) ~4,500 per Workflow definition: https://x.com/chatsidhartha/status/1894834426552459433

(Plans are to get to near ~10k or more per Workflow as we move forward)

Sid (@chatsidhartha) on X

Yesterday, we shipped an (minor) increase in concurrency limits for @CloudflareDev Workflows to 1,500 (from 100)

Today, we shipped a major improvement and concurrency limits are now up (AGAIN) to 4,500!

Happy building

https://t.co/kKXv4nPVtl

X

•

2/26/25, 7:38 PM

A

ajay | dreamlit.ai•2/26/25, 8:39 PM

What happens when there's an active workflow running and there's a new version deployed? My understanding is that it will continue to run on the older version... but what if it invokes another workflow via its service bindings? Will that run in the old version or the new?

Aajay | dreamlit.ai What happens when there's an active workflow running and there's a new version d...

M

Matt Silverlock•2/26/25, 9:15 PM

Any running Workflow runs as-is today. Looking at options to provide config for this (e.g. terminate/restart all old instances on a deploy).

If it invokes another Workflow via a binding, that is still going to be the "latest" version. The old script isn't kept around. Workflows is no different here - if Worker A calls Worker B over a Service Binding, and you re-deploy Worker B, Worker A will always (only) call the latest code.

R

rodrigo•2/26/25, 9:19 PM

Running into (what I believe) are some rate limit errors when scheduling a workflow. I count around ~80k requests but only around 10k got scheduled/queued. I believe this error is related:

Error: Durable Object is overloaded. Too many requests queued.

Error: Durable Object is overloaded. Too many requests queued.

Error: Durable Object is overloaded. Too many requests queued.

Error: Durable Object is overloaded. Too many requests queued.

What's the rate limit for scheduling workflows? The

Maximum number of queued instances

Maximum number of queued instances

Maximum number of queued instances

Maximum number of queued instances looks like 100k per the docs

Rrichw Hi all. With workflows, what's the best practice for pausing for user input? Let...

D

DanB•2/26/25, 9:51 PM

Maybe have 2 separate workflows.
One for everything up to the email (including), and the other is triggered by the user responding to the email.

wf1 --> send email --> sleep (max_time_to_wait_for_user_to_respond) --> handle user not responding
wf2 --> cancel wf1 --> handle user email response

D

DanB•2/26/25, 10:09 PM

I run a workflow from a worker. The logs of the worker go to cloudflare logs. In the logs, I see an error line with a single word "run". This is happening on every execution of the workflow, and it is not coming from my code. The workflow itself runs correctly and completes all steps normally. Any clue?

DDanB I run a workflow from a worker. The logs of the worker go to cloudflare logs. In...

K

kchro3•2/26/25, 11:06 PM

i'm seeing this happening too, would love support / explanation on what these internal errors are

Nnoman yall, i'm trying to to use whisper to transcribe some audio as part of a workflo...

M

Matt Silverlock•2/26/25, 11:27 PM

How big is the file? You will likely need to stream it in / stream the response out as otherwise the file + transcription will have to remain buffered in memory, in full, the whole time.

Nnoman the `AI.run` method doesn't like receiving a `ReadableStream` for the audio, oth...

I

Isaac McFadyen•2/27/25, 12:56 AM

Unfortunately I don't believe you can stream using the AI binding, because internally it stringifies all inputs to JSON and that can't be streamed: https://github.com/cloudflare/workerd/blob/main/src/cloudflare/internal/ai-api.ts#L118

im trying to find how to split the file into multipart without hiting the limit

Similar Threads

Similar Threads

Similar Threads