Optimistic updates / syncing with AshEvents
hello! I am prototyping a new project using Ash, AshEvents, and AshStateMachine. It fulfills all of the requirements, save a big one: we need our application to support mutations when the client is offline, with eventual consistency.
this leads to two problems:
- is there any way to export / separately compile ash resource definitions in some way to be run on a client for optimistic edits, without duplicating the transitions / actions?
- when a client returns online, is there a way to retroactively add events to the event log and then replay them? i.e. if an online client performs action A, then the offline client performs B, then the online client performs C, we'd like to generate the resource state in the order of A -> B -> C rather than A -> C -> B once the client syncs. It's not entirely clear from the documentation what happens during replay if the event log now contains events where applying them causes errors, such as illegal state transitions.
22 Replies
I don't think that
AshEvents is going to help you much here.
@Torkan do you have any thoughts here?
FWIW I don't think there is anything on earth that can magically handle this
You need some kind of algorithm sync, operational transform/CRDTs etc.Yeah, not much that AshEvents can help out with in that department unfortunately 😅
If you build up a queue of actions for the client to execute once back offline, it will happily process those actions, but there's no way to guarantee that those actions will actually be accepted by your system anymore
It's insanely complex to handle that stuff, and so much to take into consideration
alright, scratch part 2, do you have any suggestions for how to handle part 1? it seems like a natural fit given Ash resources being data layer agnostic, so I could use postgres on the backend and an in-memory store for offline access - I just have no idea where to start on maybe running this on the client as well
I can get away with replaying the actions on top of the current state, for now, rather than backdating
So: there are strategies for doing what you want to do, but not generically really.
Or at least, we don't provide a way to do it generically
But you could write a script that does some codegen based off of specific actions that you want to run for example
What type of clients are we talking about here btw?
javascript (service workers for offline support)
Right, I don't think I've seen anyone else come up with any more clever solutions than to just reimplement the behaviour of the optimistic updates on the locally stored data
gotcha. I got so hopeful when I saw that Ash kept the actions, data layer, and side effects all nicely separate!
Haha yes, I feel you, offline support is a bitch 😅
out of curiosity...... I'm an elixir novice, would it be at all possible to define resource actions in Gleam, which can then compile to BEAM and javascript? I'm not sure yet how all the metaprogramming works in resources
Depending on how complicated your use case and business logic are, it doesn't have to be too bad to replicate that behaviour
You can probably wrap some Gleam code in a resource action, which the action can just execute, but then you lose out on everything Ash offers really
You can't "take the Ash resource with you" client-side
Yeah, if you're going to get anything out of Ash for that sort of thing its a data-driven generation approach for generating a client library w/ the same interface
Which I would wager would take a couple of years to nail down and get in a stable state, at least 😅
BTW: if you actually find a full stack framework that offers offline-mode and magically converging state: LMK
If the app is something simple like a to-do list and there's rarely conflicting changes going on across different users, you can just manipulate some client-side state and firing off actions later on will most of the time work nicely.. If you're building a financial transaction system, it gets a bit more hairy 😅
😂 the magically converging state is impossible, but I'd love offline mode. yes, the use case for this has low conflict probability, which is why I'm happy to discard conflicting events for now
I wonder if it's possible to compile only the action code to javascript, and then write a native JS library to interop with it? there are other projects that attempt similar things in a different context, like Hologram for client-side code compiled from elixir sources
It's not impossible, but it's quite the undertaking
You are probably better off using some sort of client side data storage, and manipulating that data by replicating what the backend would do if the action succeeded.. Then when the client comes back online, have it phone home with a queue of actions, execute those in a transaction, and revert to the current state if any of those actions fail
yep, that's the obvious / current best way to do it.
I'm not sure if I'm insane for even considering trying it the other way? The way Ash is structured (thank you all very much for this library, it's a joy so far to work with and the documentation is excellent!) already very neatly separates out the bits you'd need in order to do something like this.
Online: AshJsonApi + core ash resources and actions
Offline: (generate a service worker artifact which replicates AshJsonApi responses, using resource actions)
Sync: replay actions when online, either error or trigger custom behavior on action failure
it is technically possible to do, right? not easy at all, but not impossible
Possibly. You may be better off looking for some kind of local-first sync/storage engine
and using Ash to implement the corresponding backend.
i.e some sort of CRDT/OT/sync system
Phoenix.Sync with electric-sql & tanstack-db or something like that if that fits your use-case, or just storing data in localstorage/browser db and then manipulating that while offline
thank you! more resources to read 🙂
Np!