Rivet slower than vanilla durable objects

I'm testing moving my current workload from vanilla durable objects to rivet + durable objects. I am seeing an extra 500-1000ms second of startup latency. Is that surprising to you? Am I doing something wrong?
No description
77 Replies
maxbittker
maxbittkerOP4w ago
>hey – appreciate it! that number seems high. can you share a bit more about your testing setup: where your code is running, if you're using rivet cloud, and the client code you're setting up with? >regardless – there are a handful of optimizations that we have coming up in the next month that will bring this down regardless. the most obvious being right now Rivet requires a round trip to getOrCreate the actor then another round trip to connect. >combining those will make a big improvement. we're not using rivet cloud, only cloudflare right now I think I see this round trip in the tracing
maxbittker
maxbittkerOP4w ago
if I am reasonable confident the actor doesn't exist, can i save time with .create?
No description
Nathan
Nathan4w ago
ah ok – that number is definitely caused by the second round trip. RivetKit is meant to be a lightweight layer on top of DO otherwise. we don't have a timeline on fixing this just yet, but it will be soon since we're in optimization phase. if you need a partial optimization, you can bypass the round trip for subsequent requests with something like:
const myActor = client.myActor.getOrCreate("my-key");
const actorId = await myActor.resolve();
// save actr id to localstorage

// ...user reloads page...

const actorId = /* load from localstorage */;
const myActor = client.myActor.getForId(actorId);
const myActor = client.myActor.getOrCreate("my-key");
const actorId = await myActor.resolve();
// save actr id to localstorage

// ...user reloads page...

const actorId = /* load from localstorage */;
const myActor = client.myActor.getForId(actorId);
if you're using react can give the equivalent curious to see results after you change that. if there's still a discrepancy, we'll get that sorted. definitely going to ship this optimization soon, though.
maxbittker
maxbittkerOP4w ago
I'm calling this from a cloudflare worker fwiw, not directly from the client yes let me try oh sorry, this doesn't help, I mostly only care about the cold start perf
Nathan
Nathan4w ago
gotcha i'm not sure how that could add a whole second
maxbittker
maxbittkerOP4w ago
also i'm seeing the round trip but I'm also seeing the chunky 897ms POST
Nathan
Nathan4w ago
in this ss it shows partykit. is that rivetkit?
maxbittker
maxbittkerOP4w ago
it's possible this is on us and we're doing something bad in our DO startup yes it is, migrating from partykit
Nathan
Nathan4w ago
ok cool was confused let me see if i can reprdouce on my end
maxbittker
maxbittkerOP4w ago
partykit was faster but less reliable
Nathan
Nathan4w ago
what did you run in to with partykit?
maxbittker
maxbittkerOP4w ago
durable objects randomly blowing up / OOMing which really seems to be fixed with rivetkit!
Nathan
Nathan4w ago
interesting. can't say i know why we fix it haha i think i see the root issue this should be a pretty quick fix give me a few min
Nathan
Nathan4w ago
a few min turned in to a whole-day refactoring to add support for destroying durable objects but i present:
pnpm add https://pkg.pr.new/rivet-dev/rivet/rivetkit@3462
pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/cloudflare-workers@3462
pnpm add https://pkg.pr.new/rivet-dev/rivet/rivetkit@3462
pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/cloudflare-workers@3462
the rest of the packages – https://github.com/rivet-dev/rivet/pull/3462#issuecomment-3525107624
Nathan
Nathan4w ago
there is still an extra round trip to the DO so i expect it might be a smidge slower than partykit, let me know what you find. the root issue is we were storing some metadata in global kv and that was adding the 1s
maxbittker
maxbittkerOP4w ago
Failed to wake up for generation: ActorError: HTTP request error: Not Found (404):

Not Found (RivetKit)
at ActorHandleRaw.action
(file:///Users/max/websim/node_modules/.pnpm/rivetkit@https+++pkg.pr.new+rivet-dev+rivet+rivetkit@3462_@hono+node-server@1.19.4_hono_e895edce39348cb5cc1a98451e93e650/node_modules/rivetkit/src/client/actor-handle.ts:159:10)
at null.<anonymous> (async
file:///Users/max/websim/party/.wrangler/tmp/dev-3CzduL/source.js:276304:13)
__type: 'ActorError',
group: 'internal',
code: 'internal_error',
metadata: undefined
}


Failed to start generation: ActorError: HTTP request error: Not Found (404):

Not Found (RivetKit)
at ActorHandleRaw.action
(file:///Users/max/websim/node_modules/.pnpm/rivetkit@https+++pkg.pr.new+rivet-dev+rivet+rivetkit@3462_@hono+node-server@1.19.4_hono_e895edce39348cb5cc1a98451e93e650/node_modules/rivetkit/src/client/actor-handle.ts:159:10)
at null.<anonymous> (async
file:///Users/max/websim/party/.wrangler/tmp/dev-3CzduL/source.js:276278:13)
at async handler (file:///Users/max/websim/packages/core/src/utils/metrics.ts:54:16)
at async Timing.run (file:///Users/max/websim/packages/core/src/utils/metrics.ts:69:14) {
__type: 'ActorError',
group: 'internal',
code: 'internal_error',
metadata: undefined
}


error - 0c233f74-6463-40a5-8cb3-e01a1715e084 - HTTP request error: Not Found (404):
Not Found (RivetKit): Error: HTTP request error: Not Found (404):
Not Found (RivetKit)
at ActorHandleRaw.action (file:///Users/max/websim/node_modules/.pnpm/rivetkit@https+++pkg.pr.new+rivet-dev+rivet+rivetkit@3462_@hono+node-server@1.19.4_hono_e895edce39348cb5cc1a98451e93e650/node_modules/rivetkit/src/client/actor-handle.ts:159:10)
at null.<anonymous> (async file:///Users/max/websim/party/.wrangler/tmp/dev-3CzduL/source.js:276278:13)
Failed to wake up for generation: ActorError: HTTP request error: Not Found (404):

Not Found (RivetKit)
at ActorHandleRaw.action
(file:///Users/max/websim/node_modules/.pnpm/rivetkit@https+++pkg.pr.new+rivet-dev+rivet+rivetkit@3462_@hono+node-server@1.19.4_hono_e895edce39348cb5cc1a98451e93e650/node_modules/rivetkit/src/client/actor-handle.ts:159:10)
at null.<anonymous> (async
file:///Users/max/websim/party/.wrangler/tmp/dev-3CzduL/source.js:276304:13)
__type: 'ActorError',
group: 'internal',
code: 'internal_error',
metadata: undefined
}


Failed to start generation: ActorError: HTTP request error: Not Found (404):

Not Found (RivetKit)
at ActorHandleRaw.action
(file:///Users/max/websim/node_modules/.pnpm/rivetkit@https+++pkg.pr.new+rivet-dev+rivet+rivetkit@3462_@hono+node-server@1.19.4_hono_e895edce39348cb5cc1a98451e93e650/node_modules/rivetkit/src/client/actor-handle.ts:159:10)
at null.<anonymous> (async
file:///Users/max/websim/party/.wrangler/tmp/dev-3CzduL/source.js:276278:13)
at async handler (file:///Users/max/websim/packages/core/src/utils/metrics.ts:54:16)
at async Timing.run (file:///Users/max/websim/packages/core/src/utils/metrics.ts:69:14) {
__type: 'ActorError',
group: 'internal',
code: 'internal_error',
metadata: undefined
}


error - 0c233f74-6463-40a5-8cb3-e01a1715e084 - HTTP request error: Not Found (404):
Not Found (RivetKit): Error: HTTP request error: Not Found (404):
Not Found (RivetKit)
at ActorHandleRaw.action (file:///Users/max/websim/node_modules/.pnpm/rivetkit@https+++pkg.pr.new+rivet-dev+rivet+rivetkit@3462_@hono+node-server@1.19.4_hono_e895edce39348cb5cc1a98451e93e650/node_modules/rivetkit/src/client/actor-handle.ts:159:10)
at null.<anonymous> (async file:///Users/max/websim/party/.wrangler/tmp/dev-3CzduL/source.js:276278:13)
after running
pnpm add https://pkg.pr.new/rivet-dev/rivet/rivetkit@3462
pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/cloudflare-workers@3462
pnpm add https://pkg.pr.new/rivet-dev/rivet/rivetkit@3462
pnpm add https://pkg.pr.new/rivet-dev/rivet/@rivetkit/cloudflare-workers@3462
oops, hold that thought
Nathan
Nathan4w ago
holding it is bleeding edge so let me know if you run in anything
maxbittker
maxbittkerOP4w ago
(having some type errors, working on them in order to test this) onStart became onWake right? and did onWebSocket change? no opts?
Nathan
Nathan4w ago
yes, apologies this update is bleeding edge we're cleaning up some naming one last time, working on documenting breaking changes
maxbittker
maxbittkerOP4w ago
works, looking into the perf now
No description
maxbittker
maxbittkerOP4w ago
the relevent stuff is probably what you see in the trigger_generation span looks like we're maybe down to 300ms? need to look at more examples but you can see there is a PUT, then a POST, and the DO starts running midway through the POST (worth noting that I didn't yet correctly impliment the getForId advice you had)
Nathan
Nathan4w ago
oh if i’m understanding the traces correctly — this might mean it’s using the HTTP client and taking an extra hop through the HTTP endpoints instead of using the native CF apis. taking a quick break, i’ll send you some code snippets when i get back.
Nathan
Nathan4w ago
are you accessing the client like this?
import { createHandler, type Client } from "@rivetkit/cloudflare-workers";
import { Hono } from "hono";
import { registry } from "./registry";

const app = new Hono<{ Bindings: { RIVET: Client<typeof registry> } }>();

app.post("/increment/:name", async (c) => {
const client = c.env.RIVET;
const name = c.req.param("name");

// Get or create actor and call action
const counter = client.counter.getOrCreate(name);
const newCount = await counter.increment(1);

return c.json({ count: newCount });
});

const { handler, ActorHandler } = createHandler(registry, { fetch: app.fetch });
export { handler as default, ActorHandler };
import { createHandler, type Client } from "@rivetkit/cloudflare-workers";
import { Hono } from "hono";
import { registry } from "./registry";

const app = new Hono<{ Bindings: { RIVET: Client<typeof registry> } }>();

app.post("/increment/:name", async (c) => {
const client = c.env.RIVET;
const name = c.req.param("name");

// Get or create actor and call action
const counter = client.counter.getOrCreate(name);
const newCount = await counter.increment(1);

return c.json({ count: newCount });
});

const { handler, ActorHandler } = createHandler(registry, { fetch: app.fetch });
export { handler as default, ActorHandler };
https://www.rivet.dev/docs/actors/quickstart/cloudflare-workers/
Rivet
Cloudflare Workers Quickstart - Rivet
Get started with Rivet Actors on Cloudflare Workers with Durable Objects
Nathan
Nathan4w ago
there's also the No Router option here what i think might be happening is you might be using createClient instead of the one provided via env.RIVET. that one would make HTTP request to inself instead of talking directly to the Durable Object. side tangent – is this jaeger? curious on your experience with otel/tracing on cloudflare, esp with their newer observability stuff
maxbittker
maxbittkerOP4w ago
we are using createClient! where does env.RIVET come from in the no router option? I don't see RIVET in the docs wrangler.json example do I need createHandler? we are doing crossworker RPC
Nathan
Nathan4w ago
ah to clarify: worker A is calling in to worker B which uses RivetKit?
Nathan
Nathan4w ago
GitHub
rivet/rivetkit-typescript/packages/cloudflare-workers/src/handler.t...
An open-source library for long-lived processes with realtime, persistence, and hibernation - rivet-dev/rivet
maxbittker
maxbittkerOP4w ago
Where does RIVET get its durable object binding to a different worker? or only same-worker?
Nathan
Nathan4w ago
ah ok. i'm surprised this adds 300 ms latency, even without worker bindings. do you have access to what path the requests go to in your traces? also having a hard time seeing which requests are ours from the trace. i need to read up on worker bindings again, it's been a minute. give me a moment is the idea that worker A would talk directly to the RivetKit DO?
maxbittker
maxbittkerOP4w ago
we could potentially put the DO in the same worker
Nathan
Nathan4w ago
ok, so it sounds like what you need is the equivalnet of createClient but using the Cloudflare worker driver that should be pretty easy
maxbittker
maxbittkerOP4w ago
or even call it directly from the browser without the hop, potentially
Nathan
Nathan4w ago
are you able to send me your wrangler.json over dm just so i know what i'm looking at + so i can reprdouce this in an example?
maxbittker
maxbittkerOP4w ago
that would be awesome
Nathan
Nathan4w ago
rivetkit is built to support both
maxbittker
maxbittkerOP4w ago
>do you have access to what path the requests go to in your traces? mb-rivet-bleeding-ed.generation-preview.websim.com/rivet/actors?actor_ids=XXX%3A0&namespace=default mb-rivet-bleeding-ed.generation-preview.websim.com/rivet/gateway/xxx:0/action/startPageGeneration paths like this I had a few versions of this code so if those look weird I can them check again and make sure I grabbed the right ones we don't use the new cloudflare observability stuff yet mostly just axiom, and I export to the jaeger viewer because the axiom viewer doesn't support zooming really helpful having these cloudflare bindings so we can build more confidence without a big migration btw
Nathan
Nathan4w ago
give me a few more minutes, shipping a proper api & example for this
Nathan
Nathan4w ago
that'll let you talk directly to your DO binding without overriding your fetch handler. lmk if that fixes the issue or if there's other binding issues i'm not aware of. if you need to rename the binding, i can also expose that.
maxbittker
maxbittkerOP4w ago
>if you need to rename the binding, i can also expose that. no not a big deal I should point my rivetkit version to this PR to test?
Nathan
Nathan4w ago
this link is to a pkg.prp.new comment which gives you the package names to use
maxbittker
maxbittkerOP4w ago
pnpm add https://pkg.pr.new/rivet-dev/rivet/rivetkit@3466
Nathan
Nathan4w ago
yep
maxbittker
maxbittkerOP4w ago
I see! hadn't seen pr.new before, neat
Nathan
Nathan4w ago
it's a life saver for us
maxbittker
maxbittkerOP4w ago
do you know if I can follow this example when calling the actor from a different worker? I am trying to use the createInlineClient, but it needs the registry like in your example, and the registry needs the actor
maxbittker
maxbittkerOP4w ago
(previously we were just sharing a type between the two)
No description
maxbittker
maxbittkerOP4w ago
I'm trying a dummy object in the registry
Nathan
Nathan4w ago
i need to take a look. it's something that's been on our radar but haven't actually tested it. how do you do what you're describing with a normal DO? do you bind the DO directly or do you bind to another service's fetch handler
maxbittker
maxbittkerOP4w ago
honestly now that we're looking at it, we should just go straight from browser to rivet
Nathan
Nathan4w ago
let me know how that works. there's still an extra round trip with that one for now. if you're concerned about startup time: 1. pass disableMetadataLookup = true to your client 2. try the .resolve() caching i mentioned earlier (this will be resolved in due time)
Nathan
Nathan3w ago
@maxbittker did you get this figure out?
maxbittker
maxbittkerOP3w ago
no, moved everything back into my same worker and then got bogged down with getting the types and routes working correctly
Nathan
Nathan3w ago
bah can you share what type of type erorrs you ran in to when you have a chance? to see if there's anything we can make better
maxbittker
maxbittkerOP3w ago
The inferred type of 'client' cannot be named without a reference to '../../../node_modules/.pnpm/rivetkit@https+++pkg.pr.new+rivet-dev+rivet+rivetkit@22eaf35b4f42ed64791bc6c89a7cfe5ca7_4db5a8f09ef885aaf48a361b01828ff9/node_modules/rivetkit/dist/tsup/config-BSxqEtUB'. This is likely not portable. A type annotation is necessary. type shit...
Nathan
Nathan3w ago
yikes usually that comes from using different versions of a package or bundler issues i'll have to look in to how to make that more lax. what bit of code did that error on?
maxbittker
maxbittkerOP3w ago
this is my registry.ts file:
import { createInlineClient } from "@rivetkit/cloudflare-workers";
import { setup } from "rivetkit";
import { siteActor } from "./actors/site_actor";

/**
* Actor Registry
*
* Registers all actors for use in the application.
* Uses `satisfies` to assert conformance to the Registry type spec.
*/
export const registry = setup({
use: {
site: siteActor,
},
});
const inlineClient = createInlineClient(registry);
const { client, fetch, ActorHandler } = inlineClient;
export { ActorHandler, client, fetch };
import { createInlineClient } from "@rivetkit/cloudflare-workers";
import { setup } from "rivetkit";
import { siteActor } from "./actors/site_actor";

/**
* Actor Registry
*
* Registers all actors for use in the application.
* Uses `satisfies` to assert conformance to the Registry type spec.
*/
export const registry = setup({
use: {
site: siteActor,
},
});
const inlineClient = createInlineClient(registry);
const { client, fetch, ActorHandler } = inlineClient;
export { ActorHandler, client, fetch };
Nathan
Nathan3w ago
which line was it erroring on?
maxbittker
maxbittkerOP3w ago
src/generation/actors/site_actor.ts:233:14 - error TS2742: The inferred type of 'siteActor' cannot be named without a reference to '../../../../node_modules/.pnpm/rivetkit@https+++pkg.pr.new+rivet-dev+rivet+rivetkit@22eaf35b4f42ed64791bc6c89a7cfe5ca7_4db5a8f09ef885aaf48a361b01828ff9/node_modules/rivetkit/dist/tsup/config-BSxqEtUB'. This is likely not portable. A type annotation is necessary. 233 export const siteActor = actor({ src/generation/registry.ts:11:14 - error TS2742: The inferred type of 'registry' cannot be named without a reference to '../../../node_modules/.pnpm/rivetkit@https+++pkg.pr.new+rivet-dev+rivet+rivetkit@22eaf35b4f42ed64791bc6c89a7cfe5ca7_4db5a8f09ef885aaf48a361b01828ff9/node_modules/rivetkit/dist/tsup/config-BSxqEtUB'. This is likely not portable. A type annotation is necessary. 11 export const registry = setup({ src/generation/registry.ts:17:9 - error TS2742: The inferred type of 'client' cannot be named without a reference to '../../../node_modules/.pnpm/rivetkit@https+++pkg.pr.new+rivet-dev+rivet+rivetkit@22eaf35b4f42ed64791bc6c89a7cfe5ca7_4db5a8f09ef885aaf48a361b01828ff9/node_modules/rivetkit/dist/tsup/config-BSxqEtUB'. This is likely not portable. A type annotation is necessary. 17 const { client, fetch, ActorHandler } = inlineClient; another question - i moved my DO into my same worker do I still need to use createInlineClient? / bleeding edge? (if I want to benefit from the fast worker->rivet call)
Nathan
Nathan3w ago
setup is just a thin wrapper around createInlineClient. it depends on which one looks more like your code: - custom backend with on publicly exposed Rivet API: https://github.com/rivet-dev/rivet/blob/11-13-feat_cloudflare-workers_add_createinlineclient_/examples/cloudflare-workers-inline-client/src/index.ts - you want a publicly exposed Rivet API (you can also add your own router): https://github.com/rivet-dev/rivet/blob/11-13-feat_cloudflare-workers_add_createinlineclient_/examples/cloudflare-workers/src/registry.ts
GitHub
rivet/examples/cloudflare-workers-inline-client/src/index.ts at 11-...
An open-source library for long-lived processes with realtime, persistence, and hibernation - rivet-dev/rivet
GitHub
rivet/examples/cloudflare-workers/src/registry.ts at 11-13-feat_clo...
An open-source library for long-lived processes with realtime, persistence, and hibernation - rivet-dev/rivet
Nathan
Nathan3w ago
ig the question is are you going to connect to rivet directly from your browser or are you writing your own glue code in the worker? lmk if that doesn't make sense
maxbittker
maxbittkerOP3w ago
i'm back to glue code in worker path
maxbittker
maxbittkerOP3w ago
so I have this call: (working)
No description
Nathan
Nathan3w ago
gotcha. sounds like that's the ideal arch for you?
maxbittker
maxbittkerOP3w ago
but i also do a websocket connection from the frontend that i'm having trouble with URL http://localhost:8787/gateway/b9fe887d87264147802a1e0cb1d79e41dad7db99b494ecbe8db1a0114001abfe:0/websocket/client?token=[...]level=ERROR msg="onWebSocket error" actor=site key=NPlwBnjq1yB43mZGf actorId=b9fe887d87264147802a1e0cb1d79e41dad7db99b494ecbe8db1a0114001abfe:0 error="TypeError: Invalid URL string."
Nathan
Nathan3w ago
let me look in to this one, we've made a couple changes around it recently
maxbittker
maxbittkerOP3w ago
const { client, fetch, ActorHandler } = inlineClient;

export default {
async fetch(request, cfEnv, ctx) {
const url = new URL(request.url);
// Inject Rivet env
const env = Object.assign({ RIVET: client }, cfEnv);


if (url.pathname.startsWith("/rivet")) {
const strippedPath = url.pathname.substring("/rivet".length);
url.pathname = strippedPath;
console.log("URL", url.toString());
const modifiedRequest = new Request(url.toString(), request);
return fetch(modifiedRequest, env, ctx);
}

const handler = await getInstrumentedHandler();
invariant(handler, "instrumentedHandler not found");
invariant(handler.fetch, "fetch method not found");
return handler.fetch(request, cfEnv, ctx);
},
} satisfies ExportedHandler<Env>;

export { ActorHandler };
const { client, fetch, ActorHandler } = inlineClient;

export default {
async fetch(request, cfEnv, ctx) {
const url = new URL(request.url);
// Inject Rivet env
const env = Object.assign({ RIVET: client }, cfEnv);


if (url.pathname.startsWith("/rivet")) {
const strippedPath = url.pathname.substring("/rivet".length);
url.pathname = strippedPath;
console.log("URL", url.toString());
const modifiedRequest = new Request(url.toString(), request);
return fetch(modifiedRequest, env, ctx);
}

const handler = await getInstrumentedHandler();
invariant(handler, "instrumentedHandler not found");
invariant(handler.fetch, "fetch method not found");
return handler.fetch(request, cfEnv, ctx);
},
} satisfies ExportedHandler<Env>;

export { ActorHandler };
should I not strip the path?
maxbittker
maxbittkerOP3w ago
i kind of vendored that code here💀 because i wanted to be able to call inline client myself might be making this overly complicated on myself
Nathan
Nathan3w ago
i see. there's a chance this is a bug in the code i shipped give me a moment
maxbittker
maxbittkerOP3w ago
` "rivetkit": "https://pkg.pr.new/rivet-dev/rivet/rivetkit@3466", still on this fyi
Nathan
Nathan3w ago
realizing i never got back on this i'll test this out rq
maxbittker
maxbittkerOP3w ago
thanks nathan
Nathan
Nathan3w ago
https://github.com/rivet-dev/rivet/blob/c4298c442265d1740d468f287548eeff4ff385dc/examples/cloudflare-workers-inline-client/src/index.ts#L33 i've verified this works e2e on the latest branch. moving further convo over to #Actor not found, will drop the new release in a second

Did you find this page helpful?