Seeding library from iCloud and beta sync issues
I've been using Immich since the beginning of the year on Android and Web with great success! Thanks to all the contributors for creating a wonderful tool!
Initially, I didn't have much luck with the IOS app on my wife's phone. I did use Immich-Go to import her 50k assets from iCloud, but the app itself just wasn't working with a library that large.
So I was super excited when you recently released the new beta timeline! The IOS app is now 1000 percent better! Unfortunately, I'm still having issues. Here's what I'm doing:
1) Request latest iCloud Photo Archive from Apple
2) Download and import using Immich-Go
3) Uninstall/Re-install Immich on iPhone and Enable beta timeline
4) Keep phone on and watch sync process
Issue #1: Once the app is done sync'ing, I'm left with 4000+ assets that it says needs to be uploaded. But I just imported all the latest from iCloud, so at most there should be a few dozen missing assets. How can I troubleshoot this?
Issue #2: If I decide to go ahead and let Immich upload the "missing" assets, it uploads at a snails pace... like KB/s. I'll keep it open for 10-15 minutes, and it will have uploaded maybe 5-10 assets. Then my wife wants her phone back. What could be causing this? The phone does get warm, so maybe IOS is throttling Immich?
And in multiple cases, I've noticed it was uploading an asset that was already uploaded (same size and filename) through iCloud/Immich-Go... why would that happen?
Server: v1.137.3
IOS version: Latest as of today
Any help/feedback would be greatly appreciated!!
20 Replies
:wave: Hey @EricH,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
2. :blue_square: read applicable release notes.
3. :blue_square: reviewed the FAQs for known issues.
4. :blue_square: reviewed Github for known issues.
5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.
My hunch is that all the assets are on iCloud, so that it would need to download from iCloud first before uploading to Immich
Hey Alex, thanks for the reply! Are you talking about the IOS setting "optimize storage" that offloads assets to iCloud? I didn't think Immich supported that feature, so we don't have it enabled. All assets should be on the phone.
can you help with getting the screenshot of the Beta Sync Stats in the App Settings page?
Sure!
Looks good
For issue 1, I am not sure how did you import the iCloud assets. It could be that the way the files moved around aren’t bringing the original file content
For issue 2, check the server I/O, now that the upload is done in parallel, it could it the bottleneck of the disk write speed on the server
I used this command: .\immich-go upload from-icloud ".\Photos" --server=http://192.168.0.6:2283 --api-key=<removed>. Is there a better way?
Regarding issue 2, is there a specific metric (or command) you are looking for? Something like iotop? My gut is that this isn't related to server I/O, because the immich-go command above did 50k files in a few hours. When I upload from my Android, it is also super fast.
As an aside, I am happy to blow away my Immich library and start over, if there's a better way to do the import. I'm still in the "experimenting" phase.
Where is that Photos folder coming from?
that's the folder extracted from the iCloud backup:
I think the files in that folder isn't the original files that are on the phone, that is why you see the mismatches, the query to get the upload candidate use the file hash content, so if it is altered by anymeans, they will be seen as different files
Okay thanks Alex, I've probably been confused, as I thought iCloud got you all of the originals. It must give you most of them, as it "worked" for 44k assets. Anyway, I'll do a few quick tests with iotop regarding issue #2.
you can pause the jobs to test as well 🙂
I don't know... it seems all roads lead to the iPhone. On my Android phone, I did a test where I uploaded 100 assets (all photos). It took ~75 seconds. I then created a 200 MB video on the Android and Immich backed it up at 12 MB/s. Orders of magnitude faster than the iPhone. Doing another test on the iPhone now... will post the result shortly.
Can you check if you are connecting to the right Wifi channel?
I know 2.4Ghz can be slow
I'm connected to our 5 GHz Wifi network, but when I run a Google Speedtest, the download is ~400 Mbps and the upload is 0.1 Mbps. I tried 2.4 GHz, and the upload is the same. My Android phone is more like ~200 Mbps down and 50 Mbps up. Definitely something weird going on with this phone. Anyway, this explains my painfully slow Immich uploads! Thank you for steering me in the right direction!
Back to issue #1. Is there any officially supported methods of seeding an iPhone library into Immich? Based on your comments above, it makes me think the only "right" way is to just let Immich upload them? Is that correct? Once I fix this weird Wifi issue, should I just start over (letting Immich upload everything)?
I've concluded there is some kind of hardware problem with this phone. I tried every setting on the phone itself and nothing worked. I even tried a different access point and that didn't help either. I then did a painful binary search of access point settings, and finally found a couple of things that improved the speed. Settings for 802.11b/g/n (not ax) and channel width of 20MHz (not 20/40) made the biggest difference. I went from uploading at 0.1-0.2 Mbps to 5-10 Mbps. It isn't fast, but it is functional!
Do you have a different AP that you can try?
With my upload speeds semi fixed, I've been babysitting the iPhone sync for the last couple of days, and I've found a few oddities. First, if I leave Backup enabled, when I come back to the Immich App, sometimes I see unexpected things:
Hi Alex, I did try a different access point and no difference, unfortunately,
You don't have to worry about the red tile, basically the queue cancel those items and they will be put into the queue once the current one is finished