is tanstack db an appropriate choice for a large paginated table?
I'm testing Tanstack db and feel like I'm going in circles on this issue. We run a larger Admin application that uses lots of tables (Tanstack table in fact). These tables have filters for the order status, who they are assigned to, the order type etc. This historically has been server-side paginated due to its size.
This is where my ignorance with Tanstack db is shining... if I use something like
useLiveInfiniteQuery it works great, but all my searches/filters/etc (my where clauses) would only be invoking the data in the collections cache. This is not ideal if an user wants to search for an order older than what is in the cache. If I ignore pagination and use the default eager it takes 15-20 seconds to load a 150mb table (not ideal either).
Hopefully this question makes sense. For reference, I am using tRPC and Prisma.16 Replies
optimistic-gold•5d ago
hey yeah we just released query-driven sync example for this sort of use case — see https://tanstack.com/blog/tanstack-db-0.5-query-driven-sync
TanStack DB 0.5 — Query-Driven Sync | TanStack Blog
You don't need a new API for every component. With 0.5, the component's query is the API call. tsx // Your component's query... const { data: projectTodos } = useLiveQuery((q) = q .from({ todos }) .jo...
ratty-blushOP•5d ago
yeah interesting, I saw this, but not sure how this translates to trpc then?
optimistic-gold•5d ago
did you see https://tanstack.com/blog/tanstack-db-0.5-query-driven-sync#how-query-collection-predicate-mapping-works ?
TanStack DB 0.5 — Query-Driven Sync | TanStack Blog
You don't need a new API for every component. With 0.5, the component's query is the API call. tsx // Your component's query... const { data: projectTodos } = useLiveQuery((q) = q .from({ todos }) .jo...
ratty-blushOP•4d ago
these patterns are tough to wrap my head around ; )
I get it, but feels like Im writing all my queries and mutations twice, then feels weird to write duplicate code to map between them. Skill issue of course...
sensitive-blue•4d ago
It got surprisingly easy after I started actually writing the code, so I can't recommend you enough to just try it!
It can seem definitely intimidating at first, but if you're comfortable with TanStack Query, you basically just have to learn useLiveQuery's API and how to translate the query it builds for you to the input your tRPC endpoint accepts.
In the new-coming linearlarge demo repo, you can find some example of paginated data:
https://github.com/TanStack/db/pull/891/files#diff-55f5fdc347618b4a01d964095e3fe8c89e438a67661ee0b889b6d48e92127a33
It can be a bit challenging to wrap your head around what's going on inside the
queryFnof createIssuesQueryCollection, but if you can find the time to test out by yourself TanStack DB, you'll see it's actually pretty intuitive.optimistic-gold•4d ago
The goal is that the translation to the trpc call and then backend call is generic so the only query you write is the frontend one
Think of it as you're just passing through arguments
ratty-blushOP•4d ago
yeah so thats the other side of the coin... I have lots of trpc functions to mutate this data, so the
onUpdate will need some kind of switch statement to determine which data changed and which trpc function to call? For example, I have trpc functions for things like updateOrderStatus... my onUpdate will need a switch statement or similar to determine if the order status in the collection has been updated to then fire that trpc function?sensitive-blue•4d ago
you can create a transaction specific to a mutation, and have as many custom transactions as you need
let me find the docs for it
optimistic-gold•4d ago
Yeah the onUpdate stuff is a convenience but not very powerful. See https://tanstack.com/db/latest/docs/guides/mutations for how to make actions or manual transactions
Mutations | TanStack DB Docs
TanStack DB Mutations TanStack DB provides a powerful mutation system that enables optimistic updates with automatic state management. This system is built around a pattern of optimistic mutation → ba...
ratty-blushOP•4d ago
okay got it, that makes a lot more sense.... thank you!
i dont have offset in
ctx.meta?.loadSubsetOptions?ratty-blushOP•4d ago
optimistic-gold•4d ago
Yeah that's a bug (this stuff is quite new still). That'll be fixed next week
ratty-blushOP•4d ago
got it
optimistic-gold•4d ago
Added an issue you can follow
https://github.com/TanStack/db/issues/892
GitHub
offset isn't passed to loadSubsetOptions · Issue #892 · TanStack/db
I've validated the bug against the latest version of DB packages Describe the bug A clear and concise description of what the bug is. To Reproduce Steps to reproduce the behavior: Go to '.....
ratty-blushOP•4d ago
is it just a type issue or no go on it at all?
optimistic-gold•4d ago
I think it's not getting passed in but you can log it out to see