Backup is uploading duplicates
To save data, I copied all files from my Android device to a Macbook and backed them up with the Ente MacOS app.
I then set up the backup on the Android app, but it seems like Ente doesn't recognize the images that are already in the cloud as the same ones on the phone.
The result is that the Android app now uploads the same images again. The images are not detected as duplicates (Backup -> Free up space -> duplicate files).
Did I miss something? Does MacOS mess with the files? I'm pretty sure I did the exact same thing on with Windows/Android on a different device/account before.
15 Replies
Ente automatically deduplicates files during uploads, but the duplicate detection works slightly differently on each platform.
On laptops (using the web or desktop app), Ente checks both the file name and hash. A file will be considered a duplicate and skipped if a file with the same name and hash already exists in the album. If you're uploading to a different album and the same file with the same name already exists in another album, a symlink to the existing file will be created. More details are available here: https://help.ente.io/photos/features/deduplicate#web-and-desktop
On Android, Ente also uses a hash check. Unlike iOS, the native Android filesystem behavior is to keep physical copies if the same photo is in different albums. So Ente does the same: duplicates to the same album will be skipped, duplicates when going to separate albums will create copies. More details are available here: https://help.ente.io/photos/features/deduplicate#mobile
You can also manually remove duplicates in Settings → Backup → Free up space → Remove duplicates. More details are available here: https://help.ente.io/photos/features/deduplicate#manual-deduplication. Ente also provides a tool for manual removal of similar images using private ML in Settings → Backup → Free up space → Similar images: https://help.ente.io/photos/features/deduplicate#filtering-similar-images.
-# If your issue is resolved, feel free to use the /solved command to close this thread. If you'd like to ask me another question use /docsearch
Deduplicate | Ente Help
Removing duplicates photos using Ente Photos
Could you confirm if you're seeing multiple entries in the home screen?
Yes, I do. There are identical copies of the images on the home screen.
At first I thought it's because I uploaded the images (from MacOS) without adding them to an album, since the Android app automatically created the 'camera' album (the folder where the images are stored is called camera). But after moving all images in the cloud to the camera album, the issue is the same.
and these are not popping up as separate items within Settings > Backup > Free up space > Duplicates?
No. As mentioned in my original post, they don't show there. Neither do they show in 'deduplicate files' on the MacOS app
Should I just try to start over again? It's roughly 100GB of data, so I was hoping to solve this without starting from scratch again
That's very odd
If (and only if) possible, could you please add a few such duplicates to an album and share the album link with support@ente.io?
Odd indeed.
If there are no other suggestions, I guess I'll just try to start over again.
If that still doesn't help, I may consider sharing an album. However, I just did a diff on 2 duplicates that I downloaded from Ente and they are 1:1 identical.
If you could share these with support@ente.io before you start over, that'd be great
I took a closer look again and realize that I missed a change in the metadata between the duplicates.
GPSVersionID, GPSImgDirectionRef, and GPSImgDirection are changed from their original values when the files are uploaded with the Ente Android app.
What you see on the right is the original data, which is preserved when uploading the images with the Ente MacOS app.
Android app version: 1.2.4(1269), downloaded via Aurora
Thanks for confirming!
You could then prune these items from Settings > Backup > Free up space > Similar images
Is this a known bug? I checked again and those three EXIF tags are altered every single time I backup an image through the Android app (also reinstalled from a different source, F-Droid, and the issue remains). I need to retain this metadata.
Pruning is not an option as
1) this would mean uploading 100GB of files over the phone again and
2) I don't have that option. As mentioned before Backup > Free up space > Remove duplicates comes up empty. If you're referring to the ML detection of similar images, I haven't checked that yet since ML isn't active.
Also checked on a different phone and account now. The altering of EXIF data does not happen there (same version of the app on Android).
EXIF tags are altered every single timeSounds like the device is updating EXIF on its own?
you're referring to the ML detection of similar imagesYes I am
The EXIF is only changed after uploading the data with the Ente app. It remains unchanged on the phone.
This seems odd, we do not modify items on disk
Could you share logs with support@ente.io? Settings > Support > Report bug