saw this blog post (https://blog.cloudflare.com/workers-vitest-integration/) which states: "Test any

Sskorfmann saw this blog post (https://blog.cloudflare.com/workers-vitest-integration/) whi...

S

skorfmannOP•12/11/24, 8:57 PM

ok, just saw this issue https://github.com/cloudflare/workers-sdk/issues/7414

GitHub

🐛 BUG: Vitest does not work with workflows · Issue #7414 · cloudfla...

Which Cloudflare product(s) does this pertain to? Miniflare, Other What version(s) of the tool(s) are you using? 0.5.32 [@cloudflare/vitest-pool-workers], 3.91.0 [wrangler] What version of Node are...

O

oof2win2•12/12/24, 12:37 AM

is there some way to temporarily fix/workaround the issue of instances getting into the stuck state, where they don't have any input and aren't triggered? is it like a thing with how many instances are created on a per-second basis?

O

oof2win2•12/12/24, 1:58 AM

seems it isn't a creation per second issue. have it in a queue, between each item there is a 1 second delay.

R

Rakhim•12/12/24, 11:10 AM

There're several messages about

An RPC stub was not disposed properly

An RPC stub was not disposed properly

, but I still couldn't figure out the issue. I made a minimal reproducible example. My workflow has 1 step and does nothing except for logging:

export class AddOrUpdateItemWorkflow extends WorkflowEntrypoint<Bindings, Params> {
    async run(event: WorkflowEvent<Params>, step: WorkflowStep) {
        await step.do('assign item sqid', async () => {
            console.log(event.payload.item.title);
        });
    }
}

export class AddOrUpdateItemWorkflow extends WorkflowEntrypoint<Bindings, Params> {
    async run(event: WorkflowEvent<Params>, step: WorkflowStep) {
        await step.do('assign item sqid', async () => {
            console.log(event.payload.item.title);
        });
    }
}

export class AddOrUpdateItemWorkflow extends WorkflowEntrypoint<Bindings, Params> {
    async run(event: WorkflowEvent<Params>, step: WorkflowStep) {
        await step.do('assign item sqid', async () => {
            console.log(event.payload.item.title);
        });
    }
}

export class AddOrUpdateItemWorkflow extends WorkflowEntrypoint<Bindings, Params> {
    async run(event: WorkflowEvent<Params>, step: WorkflowStep) {
        await step.do('assign item sqid', async () => {
            console.log(event.payload.item.title);
        });
    }
}

My worker calls it like so

const instance = await env.ADD_UPDATE_ITEM_WORKFLOW.create({params: { item }});
console.log(instance.id);
console.log(await instance.status());

const instance = await env.ADD_UPDATE_ITEM_WORKFLOW.create({params: { item }});
console.log(instance.id);
console.log(await instance.status());

const instance = await env.ADD_UPDATE_ITEM_WORKFLOW.create({params: { item }});
console.log(instance.id);
console.log(await instance.status());

const instance = await env.ADD_UPDATE_ITEM_WORKFLOW.create({params: { item }});
console.log(instance.id);
console.log(await instance.status());

When testing locally, in about 25% of cases I get:

[ERROR] An RPC stub was not disposed properly. You must call dispose() on all stubs in order to let the other side know that you are no longer using them. You cannot rely on the garbage collector for this because it may take arbitrarily long before actually collecting unreachable objects. As a shortcut, calling dispose() on the result of an RPC call disposes all stubs within it.

[ERROR] An RPC stub was not disposed properly. You must call dispose() on all stubs in order to let the other side know that you are no longer using them. You cannot rely on the garbage collector for this because it may take arbitrarily long before actually collecting unreachable objects. As a shortcut, calling dispose() on the result of an RPC call disposes all stubs within it.

[ERROR] An RPC stub was not disposed properly. You must call dispose() on all stubs in order to let the other side know that you are no longer using them. You cannot rely on the garbage collector for this because it may take arbitrarily long before actually collecting unreachable objects. As a shortcut, calling dispose() on the result of an RPC call disposes all stubs within it.

[ERROR] An RPC stub was not disposed properly. You must call dispose() on all stubs in order to let the other side know that you are no longer using them. You cannot rely on the garbage collector for this because it may take arbitrarily long before actually collecting unreachable objects. As a shortcut, calling dispose() on the result of an RPC call disposes all stubs within it.

Is this a Miniflare issue or I'm missing something fundamental? The workflows seemingly finish correctly without errors.

RRakhim There're several messages about `An RPC stub was not disposed properly`, but I s...

M

Matt Silverlock•12/12/24, 11:22 AM

Miniflare/local dev. What version of wrangler are you on?

MMatt Silverlock Miniflare/local dev. What version of wrangler are you on?

R

Rakhim•12/12/24, 11:23 AM

Latest wrangler

3.95.0

3.95.0

M

Matt Silverlock•12/12/24, 11:23 AM

Got it. We’ll take a look!

RRakhim There're several messages about `An RPC stub was not disposed properly`, but I s...

M

Matt Silverlock•12/12/24, 11:24 AM

@Diogo Ferreira FYI

R

Rakhim•12/12/24, 3:45 PM

I'm now using Workflows in production. Dozens of runs, everything works great! The ability to view each instance in the dashboard is very cool and helps a lot.
I have noticed, however, that each instance run produced an error log like this:

{
    "$cloudflare": {
      "diagnosticsChannelEvents": [],
      "truncated": false,
      "event": {
        "rpcMethod": "run"
      },
      "scriptName": "minifeed",
      "outcome": "exception",
      "eventType": "rpc",
      "entrypoint": "AddOrUpdateItemWorkflow",
      "scriptVersion": {
        "id": "44488eac-<REDACTED>-b1db278bc4cc"
      },
      "executionModel": "stateless",
      "$metadata": {
        "requestId": "L7NQ<REDACTED>U6AF",
        "id": "01JE<REDACTED>VKMK8",
        "type": "cf-worker-event",
        "error": "run"
      }
    },
    "level": "error",
    "message": "run"
  }

{
    "$cloudflare": {
      "diagnosticsChannelEvents": [],
      "truncated": false,
      "event": {
        "rpcMethod": "run"
      },
      "scriptName": "minifeed",
      "outcome": "exception",
      "eventType": "rpc",
      "entrypoint": "AddOrUpdateItemWorkflow",
      "scriptVersion": {
        "id": "44488eac-<REDACTED>-b1db278bc4cc"
      },
      "executionModel": "stateless",
      "$metadata": {
        "requestId": "L7NQ<REDACTED>U6AF",
        "id": "01JE<REDACTED>VKMK8",
        "type": "cf-worker-event",
        "error": "run"
      }
    },
    "level": "error",
    "message": "run"
  }

{
    "$cloudflare": {
      "diagnosticsChannelEvents": [],
      "truncated": false,
      "event": {
        "rpcMethod": "run"
      },
      "scriptName": "minifeed",
      "outcome": "exception",
      "eventType": "rpc",
      "entrypoint": "AddOrUpdateItemWorkflow",
      "scriptVersion": {
        "id": "44488eac-<REDACTED>-b1db278bc4cc"
      },
      "executionModel": "stateless",
      "$metadata": {
        "requestId": "L7NQ<REDACTED>U6AF",
        "id": "01JE<REDACTED>VKMK8",
        "type": "cf-worker-event",
        "error": "run"
      }
    },
    "level": "error",
    "message": "run"
  }

{
    "$cloudflare": {
      "diagnosticsChannelEvents": [],
      "truncated": false,
      "event": {
        "rpcMethod": "run"
      },
      "scriptName": "minifeed",
      "outcome": "exception",
      "eventType": "rpc",
      "entrypoint": "AddOrUpdateItemWorkflow",
      "scriptVersion": {
        "id": "44488eac-<REDACTED>-b1db278bc4cc"
      },
      "executionModel": "stateless",
      "$metadata": {
        "requestId": "L7NQ<REDACTED>U6AF",
        "id": "01JE<REDACTED>VKMK8",
        "type": "cf-worker-event",
        "error": "run"
      }
    },
    "level": "error",
    "message": "run"
  }

But each run's status is

Successful

Successful

nevertheless.

VVinz Follow up questions: a) if a workflow is completed, can I then create a new one ...

M

Matt Silverlock•12/12/24, 6:16 PM

Workflow instance IDs are unique per Workflow. The ID is the "the" identifier for finding the logs, metrics and status of an instance, even after completion. Re-using IDs would make it very hard to know if something was kicked off before.
Why not pass the
```
userId
```
```
userId
```
in the params, generate a composite ID e.g.
```
${userId}-${randomHex}
```
```
${userId}-${randomHex}
```
- and store that ID in D1? What if you had more Workflows you wanted to run for a given userId in the future? Emails? Cleanup? Different message campaigns? etc
Error

A

alex (he/him)•12/13/24, 7:09 AM

Is there a way to get the ID inside of the workflow, maybe inside of the

event

event

param?

Aalex (he/him)Is there a way to get the ID inside of the workflow, maybe inside of the `event`...

M

Matt Silverlock•12/13/24, 11:42 AM

Not yet but should be easy something we can add as a top-level prop on the

WorkflowEvent

WorkflowEvent

@Seekerdasbatatas?

S

skorfmannOP•12/13/24, 2:34 PM

a question regarding pricing https://developers.cloudflare.com/workflows/reference/pricing/

Requests (invocations): the number of Workflow invocations. Subrequests made from a Workflow do not incur additional request costs.

A Request in this context means the initial creation of the Workflow instance, or each individual step?

Cloudflare Docs

Pricing · Cloudflare Workflows docs

Workflows pricing is identical to Workers Standard pricing and are billed on two dimensions:

Sskorfmann a question regarding pricing https://developers.cloudflare.com/workflows/referen...

D

D Trombett•12/13/24, 3:09 PM

The creation of workflow. You are allowed to run a maximum of 512 steps per workflow

Sskorfmann a question regarding pricing https://developers.cloudflare.com/workflows/referen...

M

Matt Silverlock•12/13/24, 4:10 PM

Added additional clarity here:

MMatt Silverlock Not yet but should be easy something we can add as a top-level prop on the `Work...

S

Seekerdasbatatas•12/13/24, 7:04 PM

I'll check it out next week

D

D Trombett•12/13/24, 11:11 PM

One thing that imho could be very cool too is a method to list all the running workflows

DD Trombett One thing that imho could be very cool too is a method to list all the running w...

M

Matt Silverlock•12/14/24, 1:56 PM

Right now you can also use the REST API (but something we can look into on the binding)

DD Trombett One thing that imho could be very cool too is a method to list all the running w...

M

Matt Silverlock•12/14/24, 1:56 PM

https://developers.cloudflare.com/api-next/resources/workflows/subresources/instances/

the JS SDK

Cloudflare API | Workflows › Instances

Interact with Cloudflare's products and services via the Cloudflare API

M

Matt Silverlock•12/14/24, 1:59 PM

import Cloudflare from 'cloudflare';

const client = new Cloudflare({
  apiToken: this.env.CLOUDFLARE_API_TOKEN
});

for await (const instance of client.workflows.instances.list('my-workflow', { account_id: this.env.ACCOUNT_ID, status: "running"})) {
  console.log(instance.id);
}

import Cloudflare from 'cloudflare';

const client = new Cloudflare({
  apiToken: this.env.CLOUDFLARE_API_TOKEN
});

for await (const instance of client.workflows.instances.list('my-workflow', { account_id: this.env.ACCOUNT_ID, status: "running"})) {
  console.log(instance.id);
}

import Cloudflare from 'cloudflare';

const client = new Cloudflare({
  apiToken: this.env.CLOUDFLARE_API_TOKEN
});

for await (const instance of client.workflows.instances.list('my-workflow', { account_id: this.env.ACCOUNT_ID, status: "running"})) {
  console.log(instance.id);
}

import Cloudflare from 'cloudflare';

const client = new Cloudflare({
  apiToken: this.env.CLOUDFLARE_API_TOKEN
});

for await (const instance of client.workflows.instances.list('my-workflow', { account_id: this.env.ACCOUNT_ID, status: "running"})) {
  console.log(instance.id);
}

MMatt Silverlock ```ts import Cloudflare from 'cloudflare'; const client = new Cloudflare({ ap...

D

D Trombett•12/14/24, 3:10 PM

Thanks, this looks perfect!

M

Mozzy•12/14/24, 6:29 PM

I have a non-Workflow function that fetches, validates with Zod, and then puts data to KV.
I was thinking I could break up each step into a Workflow step, but unfortunately, the data is well above the limit:

Serialized RPC arguments or return values are limited to 1MiB, but the size of this value was: 4972045 bytes.

Serialized RPC arguments or return values are limited to 1MiB, but the size of this value was: 4972045 bytes.

Do I just need to put this all into 1 step to make it work?

M

Mozzy•12/14/24, 6:33 PM

But then I also feel like I lose parts of what Workflows are

M

Mozzy•12/14/24, 7:02 PM

Would it make sense to store the data in an array outside the steps like so:

Declare variable
In Step 1: fetch data and store in variable
In Step 2: use the data from the variable

MMozzy Would it make sense to store the data in an array outside the steps like so: 1. ...

I

Isaac McFadyen•12/14/24, 7:04 PM

No because state outside a step may be cleared at any time: https://developers.cloudflare.com/workflows/build/rules-of-workflows/#do-not-rely-on-state-outside-of-a-step

MMozzy But then I also feel like I lose parts of what Workflows are

I

Isaac McFadyen•12/14/24, 7:06 PM

Not really, because that concept is idempotent so can be put in one step:

fetch fails, so both ways would fail anyway
zod validation fails, which causes a failure in both scenarios
insert into KV fails in which case you can safely fetch again

M

Mozzy•12/14/24, 7:07 PM

It's more that I would like to have retries on the fetch part only

M

Mozzy•12/14/24, 7:09 PM

Like, if the KV or validation fails, I don't want to redo the fetching
I know I can implement this myself within the step, but I'm looking to learn Workflows and that would feel unintuitive

MMozzy It's more that I would like to have retries on the fetch part only

I

Isaac McFadyen•12/14/24, 7:12 PM

How expensive is the fetch?

IIsaac McFadyen How expensive is the fetch?

M

Mozzy•12/14/24, 7:14 PM

It's to a resource I don't own so I'd rather not do any more than I need to. It's also several fetches per run

C

Carl•12/14/24, 7:20 PM

is there a recommended number of workflows per worker. the indefinite queued state is becoming an issue so i'm exploring have each workflow with it's own worker (not an ideal situation and not sure if it might improve things).

P

peachneo•12/15/24, 4:55 AM

is the concurrent execution limit likely to be raised significantly (like over 100k) once the beta is complete? (Trying to figure out if I'm better off using DOs or workflows)

Y

yzziK•12/15/24, 3:35 PM

Hey so i have a rag workflow pipeline for a hackathon the issue is the steps are failing with
"Too many subrequests." for generating embeddings and storing it into neon db(its a requirement thats why i cant use service binding to D1). are there any subrequest limits in workflows ??

-free tier account

Y

yzziK•12/15/24, 4:34 PM

Ok found it 50 subrequests only

YyzziK Hey so i have a rag workflow pipeline for a hackathon the issue is the steps are...

M

Matt Silverlock•12/15/24, 5:03 PM

Same as Workers: 50 on free, 1000 on Paid.

YyzziK Hey so i have a rag workflow pipeline for a hackathon the issue is the steps are...

M

Matt Silverlock•12/15/24, 5:04 PM

https://developers.cloudflare.com/workflows/reference/limits/

Cloudflare Docs

Limits · Cloudflare Workflows docs

Limits that apply to authoring, deploying, and running Workflows are detailed below.

Y

yzziK•12/15/24, 5:04 PM

Thats so bad

Y

yzziK•12/15/24, 5:04 PM

I think I'll have to invoke other workflows from the current one to bypass it

Y

yzziK•12/15/24, 5:04 PM

I hope ?

YyzziK I think I'll have to invoke other workflows from the current one to bypass it

M

Matt Silverlock•12/15/24, 5:05 PM

You still have a daily limit on total invocations (it's a free plan). 50 subrequests per Workflow instance is still a fair few.

Y

yzziK•12/15/24, 5:07 PM

It's 100 invocations per day right. I need to process 50~200 documents at max so maybe send them in a batch of 25-40 to other workflows

YyzziK It's 100 invocations per day right. I need to process 50~200 documents at max so...

M

Matt Silverlock•12/15/24, 5:08 PM

It’s 100,000 per day (see the docs I linked)

Y

yzziK•12/15/24, 5:09 PM

Nice

Y

yzziK•12/15/24, 9:29 PM

Can I call the worker's endpoint from the workflow in the same application? How do I use service binding in the same worker.

Does using a service binding like this, I can avoid reaching 50 subrequests limit ?

P

peachneo•12/16/24, 3:21 AM

Keen to understand what kind of limit is envisioned longer term for "Concurrent Workflow instances" if anyone from CF can speak to that?

O

oof2win2•12/16/24, 9:58 AM

i don't mean to be rude in any way, but is there any ETA on the ghost executions? i really find workflows to be the best thing for my usecase, but i can't use them if a certain amount of executions just get killed and not even retried

O

oof2win2•12/16/24, 10:00 AM

or if there is anything on my side that i can do to resolve the issue i'd be happy to do so

saw this blog post (https://blog.cloudflare.com/workers-vitest-integration/) which states: "Test any

Similar Threads

Similar Threads

Similar Threads