So it is handled but its by CF and not me

Ssdr So it is handled but its by CF and not me

I

itsmatteomanf•10/27/24, 11:11 PM

I'd add a "[...] and it was not handled by user's Worklow code, so the Workflow failed.", that should explain it better.

S

sdrOP•10/27/24, 11:20 PM

Also spotted a bug in the docs / return type: workflows

S

sdrOP•10/27/24, 11:27 PM

Also whats the right way to setup local development of shared resources? If I have a pages & workers / workflows how do I share wrangler state between them?

I see a suggestion of getPlatformProxy but the docs are a bit confusing
https://github.com/cloudflare/cloudflare-docs/issues/16572

GitHub

How to share state of D1 and other products locally (workers and pa...

Proposed changes https://developers.cloudflare.com/workers/wrangler/api/#parameters-1 https://developers.cloudflare.com/d1/build-with-d1/local-development/#persist-data Subject Matter How to share ...

S

sdrOP•10/27/24, 11:49 PM

Also I notice that the step name listed in the history have an int appended to them. Im not adding that so wondering why its there / where its documented

SSid Chatterjee Hello world. So what did we think? Catching up on feedback

M

MissS•10/28/24, 4:45 AM

Did we ever hear confirmation on how pricing works?

I.e. separate steps don’t count as a separate invocation? What about retries? Are they a separate invocation etc

J

Jani•10/28/24, 9:51 AM

Couldn't find this in docs: by default, how does id reuse work in practice? As example see temporal id reuse policy

JJani Couldn't find this in docs: by default, how does id reuse work in practice? As e...

S

Seekerdasbatatas•10/28/24, 10:08 AM

All IDs are unique at the moment - why do you need ID reuse in workflows?

SSeekerdasbatatas All IDs are unique at the moment - why do you need ID reuse in workflows?

F

fry69•10/28/24, 10:14 AM

My guess would be e.g. automatic "Terminate if Running", restart a workflow with updated information and kill any running one? This can be done with other existing means of course if I understand the docs correctly, just not automatic.

Ffry69 My guess would be e.g. automatic "Terminate if Running", restart a workflow with...

S

Seekerdasbatatas•10/28/24, 10:15 AM

hum that's an interesting use case, I'll note it ^^

M

MissS•10/28/24, 10:39 AM

What happens if I just use a constant for all of my workflow IDs? Will it ensure that only one workflow is valid or will it programatically make deriviative IDs?

MMissS What happens if I just use a constant for all of my workflow IDs? Will it ensure...

D

Diogo Ferreira•10/28/24, 10:48 AM

You will not be able to create multiple Workflows with the same ID

T

Timo Reimann•10/28/24, 11:24 AM

Heya

quick question to confirm my understanding: do nested steps behave identical to non-nested ones with regards to cache usage after failures?

Specifically, if a nested step fails, would the cached results from all parent steps be reused? Or is the caching of parent steps tied to the success of the (most) inner step(s) somehow?

MMissS Did we ever hear confirmation on how pricing works? I.e. separate steps don’t c...

S

Seekerdasbatatas•10/28/24, 11:25 AM

You are only charged for CPU cycles not wall time

about the invocations, it depends on how you write your workflow (your retry policy, how much time do you sleep) but providing you don't wait for too long it should be 1 invocation

SSeekerdasbatatas You are only charged for CPU cycles **not wall time** - about the invocations, i...

F

fry69•10/28/24, 11:31 AM

After how much sleep time does a retry count as another invocation? Or this information not publicly available? Can you give rough guidance (or best practices) to avoid too many invocations per workflow?

Ffry69 After how much sleep time does a retry count as another invocation? Or this info...

S

Seekerdasbatatas•10/28/24, 11:34 AM

Not sure if I can disclose the sleep time, but it's long enough to not worry about the invocations you get per month for free

SSeekerdasbatatas You are only charged for CPU cycles **not wall time** - about the invocations, i...

M

MissS•10/28/24, 11:43 AM

Are retries a separate invocation?

I’m wondering because if it is, then in some capacity, it encourages a dev to handle retries ourselves as much as possible instead of using the system

Based on your answer, it sounds like the answer is “sometimes”

Ffry69 My guess would be e.g. automatic "Terminate if Running", restart a workflow with...

J

Jani•10/28/24, 11:43 AM

Exactly - for interval based workflows this would be very useful. In some other cases we'd want to reuse only if workflow failed, using the same id (lets say some part of id is assembled by entity id - meaning 1:1 mapping to entity) but with updated data

JJani Exactly - for interval based workflows this would be very useful. In some other ...

F

fry69•10/28/24, 11:45 AM

You should be able to write an abstraction layer for this using available primitives, I think.

MMissS Are retries a separate invocation? I’m wondering because if it is, then in som...

S

Seekerdasbatatas•10/28/24, 11:46 AM

It depends on much time you need to wait between retries, the system is the same as sleeps

SSeekerdasbatatas It depends on much time you need to wait between retries, the system is the same...

M

MissS•10/28/24, 11:47 AM

Are we walking in the range of, seconds, minutes, hours or days/weeks? is this possible to know?

MMissS Are we walking in the range of, seconds, minutes, hours or days/weeks? is this p...

S

Seekerdasbatatas•10/28/24, 11:47 AM

minutes, it's long enough that if you need to wait for that time or higher it's actually good that you get another invocation ahhaha

M

MissS•10/28/24, 11:48 AM

That’s very helpful, thank you, since the main use case IMO is downstream api is overwhelmed. So a few minutes is probably most obvious use cases

SSeekerdasbatatas minutes, it's long enough that if you need to wait for that time or higher it's ...

M

MissS•10/28/24, 11:49 AM

Is it a few minutes from the last retry or the original invocation?

MMissS Is it a few minutes from the last retry or the original invocation?

S

Seekerdasbatatas•10/28/24, 11:49 AM

from the last unit of work (last retry then)

J

jordanfinners•10/28/24, 12:03 PM

Hey, been using workflows over the weekend, it's pretty great! I have some feedback/questions please

The output that is displayed in the UI for each step is awesome. But is it possible to copy a non-truncated version of it? Also, would it be possible to mark some as secret so it's not available?

Where do the console logs go? It looks like they appear in whatever triggered the workflows but a little hard to tie it all together

Is there any advice for unit testing?

Is it possible to increase the size limit returned by each step?

Are there any limits to the code running as part of the workflows outside of a step?

Happy to go through any of this in more detail

D

DiamondDragon•10/28/24, 3:28 PM

If I need to fetch a list of 500 items and then loop over each item and do some additional fetching to transform the data, and then run another few steps inside each loop, this can be handled in workflows correct? And in this use case would it be ok to use a dynamic key inside the workflow name argument since the items id would be deterministic if the workflow needed to retry

DDiamondDragon If I need to fetch a list of 500 items and then loop over each item and do some ...

D

Diogo Ferreira•10/28/24, 6:42 PM

yes, it can be handled in a Workflow

C

Craig Dennis•10/28/24, 11:32 PM

New Sleeping and Scheduling video just dropped https://youtu.be/L6gR4Yr3UW8

YouTubeCloudflare Developers

Schedule and Sleep For Your Apps with Durable Workflow Execution a....

Cloudflare Workflows allows you to initiate sleep as an explicit step, which can be useful when you want a Workflow to wait, schedule work ahead, or pause until an input or other external state is ready.

So.. imagine you run a social media tool that allows for liking of posts and you need to reliably handle a like: incrementing the count, loggi...

S

sdrOP•10/29/24, 12:28 AM

Also noticed that the instance id is not included in the response for its details endpoint. Is that expected?
https://developers.cloudflare.com/api/operations/wor-describe-workflow-instance

Cloudflare API Documentation

Interact with Cloudflare's products and services via the Cloudflare API

M

MissS•10/29/24, 11:12 AM

Maybe I missed this but can I have conditional steps?

I.e. can I have steps inside of a condition?

MMissS Maybe I missed this but can I have conditional steps? I.e. can I have steps ins...

D

Diogo Ferreira•10/29/24, 11:27 AM

Yes you can!

Ssdr Also noticed that the instance id is not included in the response for its detail...

D

Diogo Ferreira•10/29/24, 11:33 AM

In fact, it is not included on the response. We can return it in a future release.

MMissS Maybe I missed this but can I have conditional steps? I.e. can I have steps ins...

S

Seekerdasbatatas•10/29/24, 11:36 AM

rule of thumb: you can do any control-flow logic like you would normally in JS as long as it's deterministic

SSeekerdasbatatas rule of thumb: you can do any control-flow logic like you would normally in JS a...

M

MissS•10/29/24, 11:38 AM

So can I have a step inside of an if condition based on an api call result?

MMissS So can I have a step inside of an if condition based on an api call result?

S

Seekerdasbatatas•10/29/24, 11:38 AM

as long the api call is a step of itself yes - otherwise you would break this "rule" https://developers.cloudflare.com/workflows/build/rules-of-workflows/#do-not-rely-on-state-outside-of-a-step

A

Advany•10/29/24, 11:56 AM

I am trying to extract some data from a html page using html rewriter:

var htmlRewriter = new HTMLRewriter();

                var title = '';
                var description = '';
                var favicon = '';
                var ogImage = '';

                htmlRewriter.on('title', {
                    text: (text) => {
                        title = text;
                    }
                });

                htmlRewriter.on('meta[name="description"]', {
                    element: (element) => {
                        description = element.getAttribute('content');
                    }
                });

                htmlRewriter.on('link[rel="icon"]', {
                    element: (element) => {
                        favicon = element.getAttribute('href');
                    }
                });

                htmlRewriter.on('meta[property="og:image"]', {
                    element: (element) => {
                        ogImage = element.getAttribute('content');
                    }
                });

                var content_response = new Response(html, {
                    headers: {
                        'Content-Type': 'text/html',
                    },
                });

                await (await htmlRewriter.transform(content_response)).text();

                console.log('meta data', { title, description, favicon, ogImage });

                return { title, description, favicon, ogImage };

var htmlRewriter = new HTMLRewriter();

                var title = '';
                var description = '';
                var favicon = '';
                var ogImage = '';

                htmlRewriter.on('title', {
                    text: (text) => {
                        title = text;
                    }
                });

                htmlRewriter.on('meta[name="description"]', {
                    element: (element) => {
                        description = element.getAttribute('content');
                    }
                });

                htmlRewriter.on('link[rel="icon"]', {
                    element: (element) => {
                        favicon = element.getAttribute('href');
                    }
                });

                htmlRewriter.on('meta[property="og:image"]', {
                    element: (element) => {
                        ogImage = element.getAttribute('content');
                    }
                });

                var content_response = new Response(html, {
                    headers: {
                        'Content-Type': 'text/html',
                    },
                });

                await (await htmlRewriter.transform(content_response)).text();

                console.log('meta data', { title, description, favicon, ogImage });

                return { title, description, favicon, ogImage };

var htmlRewriter = new HTMLRewriter();

                var title = '';
                var description = '';
                var favicon = '';
                var ogImage = '';

                htmlRewriter.on('title', {
                    text: (text) => {
                        title = text;
                    }
                });

                htmlRewriter.on('meta[name="description"]', {
                    element: (element) => {
                        description = element.getAttribute('content');
                    }
                });

                htmlRewriter.on('link[rel="icon"]', {
                    element: (element) => {
                        favicon = element.getAttribute('href');
                    }
                });

                htmlRewriter.on('meta[property="og:image"]', {
                    element: (element) => {
                        ogImage = element.getAttribute('content');
                    }
                });

                var content_response = new Response(html, {
                    headers: {
                        'Content-Type': 'text/html',
                    },
                });

                await (await htmlRewriter.transform(content_response)).text();

                console.log('meta data', { title, description, favicon, ogImage });

                return { title, description, favicon, ogImage };

var htmlRewriter = new HTMLRewriter();

                var title = '';
                var description = '';
                var favicon = '';
                var ogImage = '';

                htmlRewriter.on('title', {
                    text: (text) => {
                        title = text;
                    }
                });

                htmlRewriter.on('meta[name="description"]', {
                    element: (element) => {
                        description = element.getAttribute('content');
                    }
                });

                htmlRewriter.on('link[rel="icon"]', {
                    element: (element) => {
                        favicon = element.getAttribute('href');
                    }
                });

                htmlRewriter.on('meta[property="og:image"]', {
                    element: (element) => {
                        ogImage = element.getAttribute('content');
                    }
                });

                var content_response = new Response(html, {
                    headers: {
                        'Content-Type': 'text/html',
                    },
                });

                await (await htmlRewriter.transform(content_response)).text();

                console.log('meta data', { title, description, favicon, ogImage });

                return { title, description, favicon, ogImage };

I keep geting the error: An RPC stub was not disposed properly. You must call dispose() on all stubs in order to let the other side know that you are no longer using them. You cannot rely on the garbage collector for this because it may take arbitrarily long before actually collecting unreachable objects. As a shortcut, calling dispose() on the result of an RPC call disposes all stubs within it.

A

Advany•10/29/24, 12:00 PM

btw, this is inside a step, if I remove wait .text(), it works but I don't get the response...

AAdvany I am trying to extract some data from a html page using html rewriter: ``` var ...

A

Advany•10/29/24, 12:14 PM

rewrote it using cheerio

M

makrdev•10/29/24, 1:06 PM

'create' method doesn't return the 'id' of created instance. Is this expected?

const {id} = await WORKFLOW.create();
console.log(id) // RpcProperty {}

const {id} = await WORKFLOW.create();
console.log(id) // RpcProperty {}

This issue exist in local env. It works on prod.

J

Jani•10/29/24, 1:43 PM

I’ve been testing out long-running workflows and wanted to share some observations:

Testing persistence outside of an execution loop: my main goal was to see if workflows keep their state properly when running in a loop (storing a reference of a step result outside of a loop) with longer sleep intervals (around 30 minutes). This seemed to work - state reference was not lost even when it most definitely hibernated during those long waits.
Switching versions mid-sleep: during the second sleep cycle, I deployed a new version of the workflow. Surprisingly, it moved the execution to the new version instead of sticking with the original one and even disregarded the current active sleep cycle. Shouldn’t it use the version it was invoked with? (source)
Dashboard wall time: the workflow mostly sat in sleep mode, but the dashboard showed a total wall time of almost 64 minutes, which doesn’t seem right given the actual activity.

Adding more details here in a thread.

Mmakrdev 'create' method doesn't return the 'id' of created instance. Is this expected? ...

D

Diogo Ferreira•10/29/24, 1:45 PM

Yes, this is a known issue on local env. We are working on a fix for it

G

Gyurmatag•10/29/24, 4:44 PM

Hey guys!
I want to get all my meetings from Google calendar API from 2022. I want to fetch them week by week. For example like this:

for (const { weekStart, weekEnd } of weeklyIntervals) {
      const events = await step.do(
        `fetchMeetings for ${weekStart} - ${weekEnd}`,
        { retries: { limit: 3, delay: 2000 } },
        async () => {
          const response = await fetch(
            `https://www.googleapis.com/calendar/v3/calendars/primary/events?timeMin=${weekStart}&timeMax=${weekEnd}&orderBy=startTime&singleEvents=true`,
            {
              headers: { Authorization: `Bearer ${accessToken}` },
            }
          );

          if (!response.ok) throw new Error("Failed to fetch events");
          const data = await response.json();
          return data.items.map((event: { summary: string }) => event.summary);
        }
      );

      // Log summaries for this week
      events.forEach((summary: string, index: number) =>
        console.log(`Event ${index + 1} on week ${weekStart} - ${weekEnd}: ${summary}`)
      );
    }

for (const { weekStart, weekEnd } of weeklyIntervals) {
      const events = await step.do(
        `fetchMeetings for ${weekStart} - ${weekEnd}`,
        { retries: { limit: 3, delay: 2000 } },
        async () => {
          const response = await fetch(
            `https://www.googleapis.com/calendar/v3/calendars/primary/events?timeMin=${weekStart}&timeMax=${weekEnd}&orderBy=startTime&singleEvents=true`,
            {
              headers: { Authorization: `Bearer ${accessToken}` },
            }
          );

          if (!response.ok) throw new Error("Failed to fetch events");
          const data = await response.json();
          return data.items.map((event: { summary: string }) => event.summary);
        }
      );

      // Log summaries for this week
      events.forEach((summary: string, index: number) =>
        console.log(`Event ${index + 1} on week ${weekStart} - ${weekEnd}: ${summary}`)
      );
    }

for (const { weekStart, weekEnd } of weeklyIntervals) {
      const events = await step.do(
        `fetchMeetings for ${weekStart} - ${weekEnd}`,
        { retries: { limit: 3, delay: 2000 } },
        async () => {
          const response = await fetch(
            `https://www.googleapis.com/calendar/v3/calendars/primary/events?timeMin=${weekStart}&timeMax=${weekEnd}&orderBy=startTime&singleEvents=true`,
            {
              headers: { Authorization: `Bearer ${accessToken}` },
            }
          );

          if (!response.ok) throw new Error("Failed to fetch events");
          const data = await response.json();
          return data.items.map((event: { summary: string }) => event.summary);
        }
      );

      // Log summaries for this week
      events.forEach((summary: string, index: number) =>
        console.log(`Event ${index + 1} on week ${weekStart} - ${weekEnd}: ${summary}`)
      );
    }

for (const { weekStart, weekEnd } of weeklyIntervals) {
      const events = await step.do(
        `fetchMeetings for ${weekStart} - ${weekEnd}`,
        { retries: { limit: 3, delay: 2000 } },
        async () => {
          const response = await fetch(
            `https://www.googleapis.com/calendar/v3/calendars/primary/events?timeMin=${weekStart}&timeMax=${weekEnd}&orderBy=startTime&singleEvents=true`,
            {
              headers: { Authorization: `Bearer ${accessToken}` },
            }
          );

          if (!response.ok) throw new Error("Failed to fetch events");
          const data = await response.json();
          return data.items.map((event: { summary: string }) => event.summary);
        }
      );

      // Log summaries for this week
      events.forEach((summary: string, index: number) =>
        console.log(`Event ${index + 1} on week ${weekStart} - ${weekEnd}: ${summary}`)
      );
    }

Will this solution work? Should I include the loop inside the step? (I am in free tier) Does the paid plan have unlimited steps?

M

marbemac•10/29/24, 5:21 PM

Man what a solid foundation you guys have put together with the workflows beta! One questions so far:

Any creative ways you can think of to handle the "human in the loop" pattern with workflows? Basically where at some point in the workflow we want to "sleep" until a human has taken some sort of action (approve / deny, etc). Inngest manages this with their

waitForEvent

waitForEvent

function (https://www.inngest.com/docs/features/inngest-functions/steps-workflows/wait-for-event), which I know CF doesn't have, but wondering if there are other creative ways that are not too hacky to support this kind of functionality with CF.

Mmarbemac Man what a solid foundation you guys have put together with the workflows beta! ...

H

Hello, I’m Allie!•10/29/24, 5:45 PM

Break it into two separate Workflows maybe? Not entirely sure on the logistics there(would probably depend on your setup)

Mmarbemac Man what a solid foundation you guys have put together with the workflows beta! ...

D

Diogo Ferreira•10/29/24, 5:52 PM

For the moment, we do not support

waitForEvent

waitForEvent

steps or events as entrypoint (focus on the "for the moment"

). One way you could achieve the intended behaviour, would be store the humam action in a KV or D1, and periodically check for that action, sleeping between checks

DDiogo Ferreira For the moment, we do not support `waitForEvent` steps or events as entrypoint (...

N

nclevenger•10/29/24, 5:56 PM

I was just experimenting with this myself - I was about to try opening a websocket to the Slack API in Socket Mode and try to wrap the

onmessage

onmessage

handler with a promise that could be awaited in the

step

step

https://api.slack.com/apis/socket-mode

Slack API

Socket Mode

How to use Socket Mode with SDKs.

GGyurmatag Hey guys! I want to get all my meetings from Google calendar API from 2022. I wa...

D

Diogo Ferreira•10/29/24, 5:57 PM

That approach should work. The for loop should be outside the step, that way you will benefit from the persistancy of previous successful step executions, sparing on the API calls.

Recommendation. Give it a spin!

So it is handled but its by CF and not me

Similar Threads

Similar Threads

Similar Threads