Pagination vs Streams vs Other approaches
When do you decide it's better to use pagination in this case vs streams vs another approach when you're fetching records from the db?
Is it like if it's only a few thousand records, use pagination. If it's hundreds of thousands of records or more use streams? Or do you think about something else here for your decision?
14 Replies
⌛ This post has been reserved for your question.
Hey @Kale Vivi! Please useTIP: Narrow down your issue to simple and precise questions to maximize the chance that others will reply in here./closeor theClose Postbutton above when your problem is solved. Please remember to follow the help guidelines. This post will be automatically marked as dormant after 300 minutes of inactivity.
depends on what you want to do 😱
what do you mean by what to do?
well depends what you want to do with the data from the db
if u just wanna show it on the frontend like 100 entries on page 6 then pagination
if u wanna fetch everything use stream
at least thats what i know
I want to fetch the records and call a set of apis to post the data somewhere
Is it always better to use streams for this?
even if it's just a few thousand records?
idk
if ur sending it by like batch
then u can probably load it to ur memory
but then the app will have less memory to do stuff somewhere else
yeah that's a good point
If I want to keep the memory footprint low, I would need to stream it right
i mean when i had to like
for example load all the logs from a db
and it was like 5 million
i would just use a stream to fetch it from database and process each log
without pagination or something
Yeah I guess I just want to make sure I have the thought process, strategy right
Like when to use one approach versus the other
Besides pagination, streams, are there other ways to fetch from db that are efficient?
i got no clue
fetching list is inefficient for sure
from what i saw pagination is mostly used to show data on the frontend not like processing so yea
theres also batch jdbc
One other question I have on this is if I have an list of entities and I convert it to list of domain object then wrap that into a stream, does the conversion from entities to domain negate the benefits of stream here? Like should I have used stream from the getgo here but what if I want to adhere to separation between entity objects and domain objects in different layers of my application?
yea i think it does affects lazy evaluation but the best alternative is to perform pagination at the DB level, before loading into memory? so the DB returns only the paged result and you convert only those records so the rest of the stream remains lazy
^ ^ Exactly. Trying to stream domain objects is a classic N+1 trap.
💤 Post marked as dormant
This post has been inactive for over 300 minutes, thus, it has been archived.
If your question was not answered yet, feel free to re-open this post or create a new one.
In case your post is not getting any attention, you can try to use /help ping.
Warning: abusing this will result in moderative actions taken against you.