Does hashing on mobile need to finish prior to enabling backup
As the title.
I am asking because I want to know if the app now finally does not duplicate images anymore on sync, but if I have to wait for the hashing it will take about 48 hours
Can I enable backup before it finishes hashing? Is it smart enough to handle both?
If this has not been tested, please let me know.
cheers.
53 Replies
:wave: Hey @smileBeda,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: read applicable release notes.
3. :ballot_box_with_check: reviewed the FAQs for known issues.
4. :ballot_box_with_check: reviewed Github for known issues.
5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
6. :ballot_box_with_check: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.
Successfully submitted, a tag has been added to inform contributors. :white_check_mark:no, the query to get the backup candidate will get already hashed assets for backup
Thanks
Can I quikcly ask, @Alex Tran, the little clouds on the images in the mobile app, what do they signify>
I have cloud with line thorhg > not backed up, that is clear
cloud with checkmark vs cloud without checkmark thou confuses me (and those started to change randonly last time so I want to be sure to understand them)
Ok then I do not understand haha
I have assets they say the cloud of
Asset is only available on the server and was uploaded from some other device (like the web client) or was deleted from this device after upload
yet, they 100% are both on mobile and remote
And in fact... as I observe the images... they suddenly change to the cloud Asset was uploaded from this device and is now backed up to the server; the original file is still on the device
This is all still while back up is not enabled, but the thing is hashing along
Is it due to the hashing? Not hashed (yet) = has no knowledge if it is the same or not?
If the latter, I guess all good.. otherwise, not, because then it probably will start backing them up thinking they are not the same as the remote 🙁Not hashed (yet) = has no knowledge if it is the same or not?This
Great, so I am taking the feared step again > enable backup.
I amn 99% positive it backs up stuff again that was alreaedy sybnced with the cli, and this time, I made sure the files I imported with CLI where identical to what is in the library
1. I see thousands of items in the "view details" of the backup (already bad because if antying there should be maybe 3 new pics) but those are red-background, i suppose it did not upload them. Does not help that the filename on mobile is not identical to the filename reported in the web, and searching by it yelds no results...
2. I see at least 2 assets being uploaded now, which I am 100% sure they are there already, but will see if the actual asset count on the web actually increeases once these are done.
I dont think this is actually possible to resolve seeing all this.
it means that the source file where you import from the CLI is not the same as the one you have on your phone
mismached hash content
yeah but how is that possible
I mean, what else can you do but export what you have in the photos on mac, making sure you are NOT writing any exif to the file but produce sidecars instead.
Confirmed btw, duplicates start to pop up.
It re-uploads each and every single file.
Good questions for iCloud/Photos application on MacOS
pretty sure they are not the original files stored on iCloud
But these files are not on icloud. they are on disk. they are original.
mismatched hash information indicates otherwise
That is exactly what "do not optimize icloud" does, and then you export them with osxphotos
Anyway, be it how it is, clearly this is not suitable for the entire apple world
- the app does not sync thousands of photos well (or at all)
- we cannot reliably export> import
- we cannot run the app on more than one device (because then same happens: everything gets duplicated)
I am just struggling to understand how then we should use it.
sorry about the experience, I am not sure why you are facing so much struggle
😢
I cant be the only one who runs into these issues?
Or are you saying everyone else with real-live iphone libs just has no issue?
the experience is varied based on the state of your library on iCloud
Do you want me to check the hash info of the two assets, one import from the CLI and one uploaded from iOS?
Im trying to understand right now why about 5k of the assets would be recognized as synced, while 3k not.
This doesn’t make any sense. If it’s due to iCloud/photos not storing originals on Mac (which it does by definition of the setting) all of these assets should be async.
What does the red signify here?
The „more details“ doesn’t tell me if there is an error or such.
Are you on 1.139.2?
This is the original from Photos app
On immich, the image got duplicated, each with a new name!
once 18238.HEIC (has details like camera etc), 484.3KiB, once IMG_2984.HEIC (has no further details like camera etc). THe latter is how it is in my photos app, by name, but there, it has meta data, on immich, not. the first is from the export done with osxphotos which yes changes the filename. is the filename included in the hashing??
This not a live foto, it is a still
The first is marked as cloud > not on device. The second is marked as not synced. On web interface, only the first is present.
Are you on 1.139.2?No, these updates are a bit fast. Updating the system now...
is the filename included in the hashing?Nope. looks like iCloud shenanigant to me lol
Well, the file on iphone is NOT on the device of course.
I bet that is the issue.
can you send the two files that are duplicate to me?
you can download from the web, zip them up and send them here
It hashes the file in iphone, which is a bastardized version (much smaller)
Confirmed:
on mac, the file is called 18238.HEIC and has all data and is full size
on iPhone, it is called IMG_2984.HEIC, has no data, and is small (like the one immich tries to sync from phone
Sending files now.
But from what I see it is clear: the immich phone does not get the original file, it tries to sync the skimmed optimized stuff.
THey are not both on the web, as said. One is synced, the other not, but on the phone app both appear in the timeline
can you check in the Photos app on iOS what is the file name called?
Confirmed: on mac, the file is called 18238.HEIC and has all data and is full size on iPhone, it is called IMG_2984.HEIC, has no data, and is small (like the one immich tries to sync from phone)
I mean check within the Photos app, not in the Immich app
see above
that is the photos app data.
and in Immich app, the unsynced file is also called IMG_2984?
The unsynced is the small one yes, IMG_2984
if you select the file manually in Immich and press upload, what happen?
The one on the web is 18238 and on phone immich says that file does not exist localy
Trying upload now...
1 queued for upload...
Can you check on the web? see if that image get synced and see if it has metadata like location, camera model...etc
nothing happens, it does not upload it. In the upload progess, I cannot see if it even queued it, becaus the names there are all hashed
is there anything in the server logs?
checking, will need to press btton again,, one moment
can't make sense of this sorry
On mobile
Error getting file for asset 77CA11A9-A4FB-44BF-87EF-556A535EB846/L0/001
I assume that is the file, although the name is non-tellingyou are having debug/verbose log on the server so it prints a lot of things
you can chang it back to info
I have
log enabled, and I also restarted the container, this is still the same - an avalanche of data being logged.
info as such I have no option in the settings
are you looking at the correct server?
did you hit save?
I even saved it to fatal now, same.
and yes, I have only one immich install - so I am sure it is the right server
Just found another one, 🤣 instead of 😢 is my only choice here:
- In the web, first pic in the timeline is a pic I took years ago
/data/upload/8b0b26f4-6b09-4b08-9c41-eb327117cdce/8d/2e/8d2e1cb1-aea8-4f90-882b-85185a69ffe6.jpg
- The very exact same pic is of course also in the timeline where it should be:
/data/upload/8b0b26f4-6b09-4b08-9c41-eb327117cdce/93/d8/93d8dd3e-77a0-4f8f-b265-40a276619776.JPG
- On the "good" file, facial recognition works (it appears under my name in "explore")
- On the "bad" file, facial recognition fails and it does not recognise me
- The "bad" file plain and simple does not exist on my Photos lib (iphone or mac)
- it exists on immich thou, and in the immich app it labels the "bad" as both synced and existing locally, while the good file it says it does not exist locally lol
🤷♂️
... I need to step back from this, it makes mit all tigglysound like a good solution, probably start fresh on a brand new instance/database will help clear your head
... this is the 5th clear scratch this week lol
I see
I do not think that this is working as it is intended to. I mean, I make a lot of mistakes, but something is massively messing around here that is out of a user control
The last thing I will try now is sync a tiny winy album
I think iCloud's shenanigan is a sensible conclusion
Here we go:
It is failing to upload the "duplicate" which it grabbed from iPhone photos app. Because well, it is already online
But the iPhone immich app showed it as not synced, added it twice in the timeline as described in the message I am replying to.
That SQL error should be telling enough.
so that is good, in a sense that it detects the file are on the server so it is not uploading
however, the hash content on iOS seems to be incorrect, so it doesn't show the cloud with checkmark
bring in the expert @shenlong-tanwen
Yes, I thought the same. "Good" as in at least it does not succeed in pushing it ip. But a nightmare to "rely" on, because now multiply this problem by 10'000 and try to "trust" your images are in immich, and not duplicated 😐
I would be ready to bet that this simply escapes most users. Because.... the funniest of all things is that this does not happen with all images.
In my case it seems to be about a third of the assets (this time, last time it was less, once more, etc etc)
I now just tried to disconnect the app, then reconnected it and synced only a tiny album
There, it does not fail (until now, and I made sure to include some of the same pics that previously failed)
This kind of indicates it has some issues with scaling?
I don't think so, again simply with syncing iCloud asset, it needs to first download then upload to Immich
I do think there is a bug somewhere though
about the hashing stuff
need to investigate more for iCloud assets
I wish apple's photo app had a setting to "automatically move all to xyz album"
This way, I could sync only one album which would have only new images, and sync the rest (older) using immich CLI
As far as I was able to tell, as long as it adds images "one by one" (as in not hundreds or thousands in a bunch) it does not produce hiccups
Unfortunately that is not an option, and if manual moving is needed, I could as well just run a cli backup every then and when, and not sync on mobile at all.
Just that this defies the idea a bit.
On the other hand, one cannot get rid of photo app anyway. they do not let you delete it, ultimately 😐