Duplicates and deduplication?
Hello Immich experts! 👋
I have been using Immich for almost a year, have it setup on an OMV and using it to view and backup photos and movies from my Iphone and my wife's Iphone.
One thing I've noticed when viewing photos is that we have soo many duplicates, essentially I cannot view any photos from a few years without scrolling through duplicates, it's like everywhere! 😩
When I go to utilities, I often see duplicates and then I click deduplicate all and they disappear. I often go to my wif's account and check if she has duplicates to remove which she sometimes do, and then I remove them.
Then time passes and I yet again notice duplicates and I feel am I crazy!?! 🤯
What is happening with duplicates? Is there some edge case that utilities cannot get rid of e.g. cannot see duplicates between my photos and my wife's? Or is something recreating duplicates e.g. our apps freak out and upload photos again?
Any suggestion on workarounds or behaviour changes that can resolve this?
17 Replies
:wave: Hey @oscarnx,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: read applicable release notes.
3. :ballot_box_with_check: reviewed the FAQs for known issues.
4. :ballot_box_with_check: reviewed Github for known issues.
5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
6. :ballot_box_with_check: uploaded the relevant information (see below).
7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.
Successfully submitted, a tag has been added to inform contributors. :white_check_mark:The deduplicate utility only works for one user, yes. If you have partner sharing with your partner's assets also showing up in the timeline the dedup utility won't fix that
Feels like a pretty important feature though, especially in family situations like ours where my wife often wants me to send photos that I take so she can share on insta etc. Or are there reasons why deduplication should not remove duplicates across users?
I mean a pretty obvious one is that you need to decide on which side you want to delete pictures
Partner sharing is one way (you need to enable it on both sides) so this can get really weird
Whatever user A is doing shouldn't ever affect user B's library
True, but I guess it also depends on how you view the library. For us, we have a family library of all photos, we don't necessarily distinguish if I took a photo of my wife of vice versa, it's photos for the family. Sure, we have photos that are not necessarily shared, but also not necessarily secret/private.
Then maybe just use one account?
Good point, I'll look into that. Feels like it could come with a whole bunch of special consideration but maybe not. 😊
I'll also create a feature request for it, must be others in same situation as me. 😊
Thank you for the fast replies @Daniel! 🙏
We have this covered already in feature requests under Better Sharing
Not really though
At least I personally don't include dedup across accounts in that
I would. It’s about cross account permissions
You mean this one? https://github.com/immich-app/immich/discussions/9441
#12614
Sure that one is fine I guess. It’s already open
Oh but yeah that one would fit for your use case
I think I would agree with deleting assets of other users, that's fair
They are related yes, I actually made a post to 9441 back in August 😊 Having read through 12614 I think a lot of it is covered there, some refactoring is desirable IMHO. The family use case is probably very common and there is always one party who is either more technically skilled or using Immich more that becomes family-admin. "Full transparency mode" would be good! 😊
There’s a lot of room for changes , but also a lot of different opinions and needs
So we need to figure out a best path forward