Somewhere during the last releases, tons of pics lost geolocation metadata
I’m absolutely certain the metadata was there - I exported it from Apple as sidecar XML, spent more than a week ensuring the import was correct, and even documented the whole process in a GitHub discussion. I did the manual import specifically because the standard phone import failed to include the data. Afterward, I double-checked: the GEO data was present. For instance, over 1,300 photos in the USA appeared correctly on the map.
But now, I’ve noticed that every asset which had issues during the standard phone import - and which I had fixed via the Immich CLI import using osxphotos exports with sidecar metadata - has lost its GEO data again. My suspicion is that this happened when the new filter for “assets without geodata” was introduced, though I can’t confirm that.
Looking in the folders now, the sidecar metadata files for those assets are simply gone.
So my question is: under what conditions does Immich delete sidecar metadata files? I know it sounds unlikely, but I’m 100% sure those assets had geolocation data - I did that process several times until I got it right and it was the entire reason I did the manual import. This affected over 5,000 assets in total, and now they’ve all lost that data again. If I have to redo the import, I need to understand what caused the deletion to prevent it from happening once more. Because something is removing those files - I’m not imagining this one, I could miss an image or two, not 5k.
Imports where done with past versions but always actual to what was up to date at that time.
61 Replies
:wave: Hey @smileBeda,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: read applicable release notes.
3. :ballot_box_with_check: reviewed the FAQs for known issues.
4. :ballot_box_with_check: reviewed Github for known issues.
5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
6. :ballot_box_with_check: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.
Successfully submitted, a tag has been added to inform contributors. :white_check_mark:My suspicion is that this happened when the new filter for “assets without geodata” was introduced, though I can’t confirm that.Sounds plausible, do you mind adding a reproduciable file and its metadata file along with the command that you use to import? Let's see if we can reproduce to identify the bugs
The approach I follow is this:
https://github.com/immich-app/immich/discussions/21251 (see
Actual Import process for import commands and processes)
As for problematic file + sidecar, I will have to re-run osxphotos on my assets, spot out the file, then I will share (a few, of the 5+k)
This will take a "moment"
Note, all those assets that have still valid GPS data also still have a valid XML file!
So, something explicitly deletes certain files. According (probably unreliable) GPT, it is because when a file has no GPS data embedded (the case for all those culprit files), immich treats the XML as a one-off databasis, imports the GPS data and then removes the XML
I couldn't fathom thou why it would do that
Is there code in immich that does something like that, or more generally, does delete sidecars?
If there is no such code... then it's a mistery. If there is, that code would be the code to look at I would say 😄[Discussion] Successfully sync an iCloud Library to self hosted Immich in an airtight env. (immich-app/immich#21251)
no problem, having reproduciable steps will help
What would I do as in terms of fixing this?
Can immich cli just import sidecars? I assume not… so it’s a full wipe + reimport I guess?
if there is an issue with the server, we should fix it. but we need to identify where the issue is at first
Right, I understand this, what I’m asking is how I can fix my assets. I mean, that geodata is obviously gone.
So I think a full re-import is the only way (even if a future fix would avoid it to happen again, lost metadata isn’t recoverable. Unless you suspect that data to be there in the db of course)
I’ll share sample assets on about 2hrs
yeah probably don't do a full re-import until we figure out the issue to help saving you time and effort
I am sorry but sharing an export exactly as I had it is impossible right now
Either due to whatever, osxphotos today fails (last macOS update perhaps???)
HOWEVER, I randomly checked some images, and i even found some that actually have location data embedded, so, we can be 100% sure the info is there not only because I had it in the xml, but because it is in the photo!
Yet immich reports no location on those
An example zipped up here below
So not only there is an issue where it deletes sidecar data and losing the data previously imported from it, but it also does not see the actual data in side that photo (it is stripped or something)
maybe something is up with your instance?
try run metadata extraction on that photo
maybe the companion xml file you added is the problem
since it will prioritise the information from the sidecar file
try run metadata extraction on that photo- Did that twice already - I mean the bulk job and the single photo "refresh metadata" thingy
maybe the companion xml file you added is the problem- The xml is gone on that pic too (and it was there, 100%, plus, this pic (and other ajdacents, where properly mapped, I recall those because they are exactly from a location I used for spot check back then) (above is the folder on my system containing that photo) And keep in mind this is not just this photo. As said, a few missing, corrupt, etc... but we are talking about more than 5k photos that where properly appearing on map because I added sidecars - and now those sidecars are gone, and the geodata too I think the question we need to ask is not how to replicate, but does immich delete sidecars under some given circumstances?
Downloaded the same asset from immich - lat long is gone
This is very clear pattern:
- images that where not working for whatever reason with standard phone import (messed up date and locations)
- then imported with an osxphotos generated export holding sidecar
- then all good in immich!
- then all of a sudden, sidecar gone and with it, the data in the photo too (which probably it never added, because as said for some reason, these photos are also not added properly when directly importing from phone, something is different about them, but never figured out what... since I fixed it with sidecar xml
And what a bit puzzles me too, is, it is only this area
Like, everything else in the world I have is mapped correctly
Again, I think the main concern is under what circumstances does immich delete sidecar files
That is the whole and only reason for this to happen in the first place, and it not doing it, fixing the problem at hand.
@Alex - I have backups
Result of backup query below:
The sidecar was there:
/data/upload/.../00290fea-9612-...xmp <-- sidecar XMP file
Somewhere between then and now, immich deleted that sidecar file
I am now checking newer backups to see if I can spot the date. This is worrying me a little.I think there are a lot of moving parts with your case. It'd be best to find a reproducible steps to identify the bug
I think I know what is going on here
Image asset:
/data/upload/55b79ac5-6ec5-4d1d-8875-cf769347dd3b/d4/d4/d4d4242d-5769-4098-8773-408551f07f41.jpg
Sidecar asset: /data/upload/55b79ac5-6ec5-4d1d-8875-cf769347dd3b/00/29/00290fea-9612-4838-8d2e-0d12a2485ce4.xmp
different paths! Different names too
Immich does for some reason move some and rename of the sidecars, but not others... duh, I understand nothing no more
Note that all working assets have their sidecar right next to them in the same folder
Like for example:
this image has OK geodata, because xml is inside the same folder.Did you recently run metadata extraction for all your images again or something like that?
Also, what version of the server are you on?
yes
Up to 2025-09-08 (v1.140.1 backups) > all good, sidecar IS IN DATABASE
From 2025-09-09 (v1.140.1 backup) > Sidecar gone
I analysed all backups
I'm guessing you ran into this problem: #13897, which was fixed in #21312, and released in 1.141.1
[Issue] immich-cli does not properly upload sidecar file (XMP) (immich-app/immich#13897)
[Pull Request] fix: sidecar check job (immich-app/immich#21312)
Timeline of 9622.jpg
• Up to 2025-09-08 (v1.140.1 backups)
• sidecarPath = /data/upload/.../00290fea-...xmp
• exifInfoId = 0198d8bf-71fc-7673-afb4-5094c3abdd33
• From 2025-09-09 (v1.140.1 backup)
• sidecarPath = \N (gone)
• exifInfoId = 019929ed-b3ca-78a7-a13b-4bcfea02deb7 (new UUID)
• 2025-09-10 (v1.141.1 backup)
• Same as 09 Sep: no sidecar, new EXIF UUID.
but my sidecars where uploaded way before that date/release
In any case.. what do I translate this to?
start all over again?
From your backups you can figure out which assets had sidecar files (that were deleted) and you also have a filesystem backup with those same sidecar files?
Well, all assets had sidecars 🙂
and no, of course there is no file backup from that time, I mean... that is not really how backups work (incremental data backup, not snapshots) - but I would assume the files did not change factually, only news have been added.
what do you mean that's not how backups work
you are unable to access/retrieve the deleted xmp files from your backups?
The sidecars are not deleted
as you can see above
Let me check something real quick
they are in different folders, which is probably even expected due to sharding
They are however deleted from the db
Exactly as the ticket you shared describes at the last comment:
It looks like there was a bug in the sidecar sync job that would ignore the asset.sidecarPath if it wasn't next to the asset with the same name. This should be fixed with the linked PR though.Sidecar files wont be in the same folder all the time I presume due to sharding. That why the DB entry pointing to it. But... that DB entry got nuked, becayse it thought it is "next to the file" Thats how I understand it
that's my comment btw lol
OK, the bug in the code was just limited to the database column
sidecarPath getting set to null
If you simply restored those values, everything should start working againexactly what happened here:
Timeline of 9622.jpg
• Up to 2025-09-08 (v1.140.1 backups)
• sidecarPath = /data/upload/.../00290fea-...xmp
• exifInfoId = 0198d8bf-71fc-7673-afb4-5094c3abdd33
• From 2025-09-09 (v1.140.1 backup)
• sidecarPath = \N (gone)
• exifInfoId = 019929ed-b3ca-78a7-a13b-4bcfea02deb7 (new UUID)
• 2025-09-10 (v1.141.1 backup)
• Same as 09 Sep: no sidecar, new EXIF UUID.
How would I restore only the affected values?
In the interim, 20gb more photos where added 😄
You are not using the storage template, right?
no
So the values for
sidecarPath should never be changing
We can test this for a single asset if you wantgladly
Can you try running
UPDATE "asset" SET "sidecarPath" = '<old path>' WHERE "id" = '<asset id>'; ?<old path> needs to be the one in the DB or the one on disk?
I notice they are different, I mean... docker vs disk path
from the database
give me a "minute" 😄
I think you can just run some SQL to fix/restore the broken sidecarPath values, since they are still on disk and you have a backup which tells you what values they used to be. The only caveat is, you need to run the sidecar jobs again, but on
v1.141.1+ otherwise they'll just be set to null again 😅
If you aren't on 1.141.1 or later, you might want to upgrade to at least that version, take another database dump, and then try running SQL to fix stuff.I have run the command, refreshed metdata on that single pic, the location is back in immich
It is not in the DB, but that is probably expected?
Now the big question is... how do I do this for only the problematic files
I mean, I think I have to somehow parse backup into a loop, compare it against a filtered list of items from current immich instance, which lists only "bad" files, then do the comand for each of those.
Let me pick some gpt brain here, unless you have a better idea
I don't really think it matters if you update this for all assets tbh
since the value should either be the same or have been set to null
this looks great btw. is latitude/longitude set now or is it still null? if it is in the UI it should be here as well now, it might have taken a second for the job to save it back to the database
right, after I did run metadata thing, the same is now populated:
great, so this record is now "fixed", right?
how many assets do you have?
5000 affected assets at least
20k I shall not touch with a 10 foot pole because not mine (wife will eat me alive), and where imported after that issue anyway
another 10k my assets which if broken does not matter that much
May I ask what you say to this:
1. On a backup that still has sidecars (e.g. immich-db-backup-20250908...):
2. Load into a temp table in live DB
3. Bulk update only missing ones
4. refresh metadata
I have no idea what I am about to do apart of that I can read and understand what the commands do in a general sense... but it was fully suggested by GPT, thus why asking what you think about it
You are on 1.141.1 or later now, right?
indeed, up-to-date:
v1.142.1 rn
Great
Idk what step 1 is doing, but the general idea after that makes sense. Basically, you want to export id,sidecarPath, import it into a new table in your production database, and then use it in a SQL UPDATE statement.
Step one extracts all existing sidecar paths from my backup:
-P "\t/data/upload/.+.xmp" finds asset rows with sidecar path. • cut -f1,16 extracts: • field 1 = id • field 16 = sidecarPath (check column index if needed, but in your dump the sidecar was the 16th). Result: sidecars.tsv with two columns: id<TAB>sidecarPath.Ok... going to do that, hold my beer haha
Yeah, looks like that could work
If you just did a normal restore, you could just run the SQL to generate SQL that you could then run on your production database:
Yeah rofl...
(check column index if needed, but in your dump the sidecar was the 16th).
I should have checked, because it wasn't
Anyway... after a heart attack and a half and fixing the above mess (it put filename in the place of sidebar path into the DB lol)...
All assets have location data again!oh, that's great news
I really can't believe that issue has been opened since last November, and it was just fixed like a few weeks ago.
(the metadata job is still running, but I can already see hundreds of images back on the map, that where missing, so all good so far)
Awesome
This thread has been closed. To re-open, use the button below.
Thanks for the awesome help!