Is there a best-practice for super large libraries on android? My wife has 50k+ photos/videos
My wife has to sync a huge library and every time i try to do a manual sync the app crashes. The background sync consistently crashes as well but does appear to make progress.
Any idea how to make this better? Or do I need to just wait out the background sync and itll start working after?
I'm hosting this in docker, and the server itself appears to work fine. My library with only 10k photos syncs great.
16 Replies
:wave: Hey @Untiltable,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: read applicable release notes.
3. :ballot_box_with_check: reviewed the FAQs for known issues.
4. :ballot_box_with_check: reviewed Github for known issues.
5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
6. :ballot_box_with_check: uploaded the relevant information (see below).
7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.
Successfully submitted, a tag has been added to inform contributors. :white_check_mark:Honestly 50k is nothing, it should not crash
If they’re all on device that’s quite different
Immich can handle millions of remote assets
Ah, true that
Does the logs show anything? If not, can you use an app like LogFox and grab the crash logs when the app crashes?
Lots of asterisks for that. The app sync in particular doesn't scale well
The initial sync is the most painful part for large libraries. Partial syncs after that should be better. We're reworking the sync in general so it should be smoother in the future. There's also the issue of checksumming all the files on device which is quite slow and intensive, so we might need to optimize that as well
Doesn't it work ok when they are remote? I thought it was much better than if they are local
It's faster, but still slow as hell
There are lots of aspects to it that contribute to the slowness
But 50k remote assets should generally be okay. It's really when you add another zero that the sync gets bad
And then the app in general isn't optimized for large libraries, lots of things that are done sequentially, or scale exponentially
sorry, yes, they are all on device
she has a 2tb microsd
i got it to finally sync by letting it crash a lot and only background sync actually worked
I’m glad they got synced eventually, but if you could get hold of the logs, it’ll be much appreciated. We can see where it is failing / crashing and check what can be done to address it
sure, i actually am going to move it to a strong server so i gotta restart it all again. Was also trying out photoprism to see if that'd work better since I like the NSFW AI filtering and whatnot, but you can't really beat the immich UI for what I'm trying to do.
so just looking at the phone itself i didn't see any logs from the logs screen at least about it not working, but the app becomes 100% unresponsive and android kills it
@shenlong-tanwen is that what you're looking for?
oh this view only contains the logs that we can catch, OOM or other crashes from outside the app will not be available here. You need to use something like this https://github.com/F0x1d/LogFox to capture them
ok, ill try to get that set up at some point to see if i can catch those. Do i need to root my phone for it?
No, you can give it the required permissions through adb