"live" time series query
I'm not currently using TanStack Query, but I would like to! and want to know if it could fit my current use case. I'll try to be as concise as possible.
I have a database table called
case_data with columns like so
which is unique on timestamp+case_id
And another table called cases with columns like so
which is unique on case_id only
A user on the frontend (vite react) selects a specific case, and updates its status to indicate a request to be processed.
Another server sees the status update, and begins appending records to case_data starting with an earliest timestamp and marching forward monotonically. The process takes ~10 mins let's say, and results in a total number expected_num_data_records of records being appended to case_data.
I would like the frontend to show the case's data as soon as it's available, instead of pinwheeling for ~10 mins; so meanwhile the frontend incrementally queries the case_data table in chunks, starting from an earliest expected timestamp. Currently the structure of the requests are:
{
case_id: number,
starting_data_index: number,
num_results: number
}
And the backend's response is like:
{
results: [ data, data, ...],
actual_num_results: number
}
The response may not include the total num_results as the other server may not have inserted those yet. However the results are guaranteed "coherent" between starting_data_index and going for actual_num_results; the frontend appends the results to a buffer, and repeats, setting a new starting index to starting_data_index + actual_num_results, continuing until expected_num_data_records have been received.
Can something similar be achieved in TanStack query? I can change the backend functionality, but I can't make the "other server" insert records faster, and I'd like to keep the incremental data loading if possible.
Sorry for being long winded, and thanks!6 Replies
national-gold•9mo ago
You could take a look at websockets per https://tkdodo.eu/blog/using-web-sockets-with-react-query
Using WebSockets with React Query
A step-by-step guide on how to make real-time notifications work with react-query
national-gold•9mo ago
And if you can use trpc see https://trpc.io/docs/server/subscriptions
Subscriptions | tRPC
Introduction
national-gold•9mo ago
Basically, you need the api to notify the frontend when any tasks complete so the query cache can be invalidated
eastern-cyanOP•9mo ago
oh awesome - that all makes sense! I'll give this a shot
unwilling-turquoise•9mo ago
Maybe you could also use the new streamedQuery with an async iterable with next results instead of invalidating the cache, it was just released but it’s pretty cool
other-emerald•8mo ago
do you have an example of using streamedQuery with websockets/socketio/SSE?