nkdf.
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
I don't mind, utilize it however you wish
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
how were you planning to identify duplicate photos? Part of the reason why my script ended so big was that the iphone xr and my canon eos wrote the same filenames
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
I don't know anything about managing a github repo, maybe one of the regular contributors would be a better person to ask
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
Here is my python script that I used to detect duplicate images that resulted from uploading the contents of a google takeout where the images were stored in 'storage saver quality' when I already had original quality images from my device. I have tested the script on my installation of 40k assets, approximately 2800 were duplicates. Usual disclaimers apply, no guarantees are made. I have no coding skills, and I cannot hello world, I made this with the assistance of chatgpt.
It requires the output of getAllAssets from immich api: https://immich.app/docs/api/get-all-assets stored into a file called output.json The script then compares the originalFileName to identify duplicates. It then further ensures the duplicate filenames share the same exifInfo properties make, model, and dateTimeOriginal (in case you have multiple devices that duplicate filenames). Then it compares the fileSizeInBytes, and returns the smaller files. The final output is a comma delimited list of ids that you can feed into the deleteAsset api endpoint to remove these files from immich.
It requires the output of getAllAssets from immich api: https://immich.app/docs/api/get-all-assets stored into a file called output.json The script then compares the originalFileName to identify duplicates. It then further ensures the duplicate filenames share the same exifInfo properties make, model, and dateTimeOriginal (in case you have multiple devices that duplicate filenames). Then it compares the fileSizeInBytes, and returns the smaller files. The final output is a comma delimited list of ids that you can feed into the deleteAsset api endpoint to remove these files from immich.
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
it's too long.. discord truncates it.. do we have a perferred place or will a pastebin do?
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
task completed. happy to share the script if anyone else runs into the issue
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
nevermind.. filesizeinbytes
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
I might need to somehow take the assets and query the filesystem or something...
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
I just realized I can't compare the results of getallassets since 'resolution' returned is all based on exif data, and there is no information on actual file size
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
40k assets? its a small server
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
is using 'searchAsset' the only way to return an asset id by searching the filename?
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
instead of cleaning out the filesystem first
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
Maybe I'll try to enumerate a list of files to be deleted and pass them into the api that way
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
interesting.. ok I'll read up
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
although I don't know if I have the scripting skills to attempt.. but may be worth a read
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
is the db schema somewhere I can reference?
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
and was going to go full commit lol
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
unfortunately I'm somewhat OG... been on it since the project started
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
If I remove the duplicate from the filesystem, I assume I would have to clean up the database and artifacts somehow?
39 replies
IImmich
•Created by nkdf. on 6/12/2023 in #help-desk-support
Duplicate detection not working as expected
oh crap.. I screwed up then
39 replies