Immich sorts by the wrong date metadata - Should use Date Taken exif before anything else
Immich is taking the modified or created date and using that for the sorting, instead of using the actual Date Taken metadata. This means that photos exported today which were taken a year ago come up as from today, for example.
How can I fix this? Surely there's a setting for it that I've just missed? As-is makes everything date-related – including the main timeline – wrong.
52 Replies
:wave: Hey @EmSixTeen,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: read applicable release notes.
3. :ballot_box_with_check: reviewed the FAQs for known issues.
4. :ballot_box_with_check: reviewed Github for known issues.
5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
6. :ballot_box_with_check: uploaded the relevant information (see below).
7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.
What are the actual EXIF fields in the file, as read by
exiftool?
That's not exiftool
..
I'm not just being annoying for the sake of it. exiftool outputs the exif fields exactly as they are in the file, and is also what Immich uses on the backend, so it's the only good way to know what's actually going on for troubleshooting purposes.
Right, you know it's correct though. Anyway here's direct from exiftool:
Has the metadata extraction job finished yet?
Immich reads a few fields in priority order, the first one it should be picking up is this one
[Composite] SubSecDateTimeOriginal : 2024:09:07 14:52:13.23+01:00
I see XMP in those fields
Did the XMP file get uploaded with your asset?
How would I know that?
Immich is literally just pointed at the processed files folder
? Like an external library you mean
Yes
I don't upload photos to Immich
Okay, well XMPs in external libraries should be supported, so you could check the library whether the XMP file is there and named aptly
Is that the actual problem here, or just something else which is also an issue?
Also, fwiw, there are no .xmp files in the Processed folders
Or anywhere that I can see in the raw file folders for that matter
Alright so it's put in the file then somehow, I wonder if the way they do that is 'correct'. Could you zip up an example and upload it here? Zipping part is important because discord strips all metadata
ehh
I think this is a sidequest
These are files exported from Lightroom, Lightroom embeds XMP metadata and doesn't create sidecar files
This is basic troubleshooting
It's usually the case that all thumbnails aren't generated when Immich picks up the files in the processed (external library) folder, and I've realised that I need to go and manually run the job (ie Jobs > Generate thumbnails > Missing) - it doesn't even say there's any jobs Active or Waiting though. I'm imagining here that this is another symptom of the same thing?
If I run one of the other jobs do you think it'll pick up those other date fields, and sort appropriately?
So it finds the files and then doesn't do any thumb generation, metadata extraction... ?
After exporting photos from Lightroom to the folder that Immich uses as an external library those files show up in Immich, but most files have thumbnails and some don't. I realised running the job manually generates the missing ones, despite it not thinking there's any pending. Don't know about the metadata extraction - What should I expect to see, and where?
The external library is only updated once a day by default, are these pictures all exported at the same time?
I'm unsure what you mean, but after exporting a collection of photos they all come up in Immich, some with and some without thumbnails, all exported at the same time, with the missing thumbnails being sporadic and seemingly random.
And if you do "missing" on metadata extraction job for instance do all of them get fixed or are some still broken?
I wonder if there's a couple of images breaking things
The 'Extract metadata' one? Let's run it and see.
In any case, there should be some output in the docker container logs
Took about 2 seconds and nothing changes
I'm running on unraid with:
imagegenius/immich
PostgreSQL_Immich
If I go to the docker logs for the immich container I don't really see anything.
Did you restart docker here?
Extract metadata did nothing, but doing 'Rescan' on 'External libraries' seems to have picked up the correct dates and now I see camera info in the photos' info pane
Now? Did before
at 10:57 AM whenever that was
It was an hour ago while trying to troubleshoot this, before either of you had yet replied, and nothing changed before or after that.
☝️
And did that produce any logs? 🙂
This is what's in there when I open the logs now, after manually running 'Rescan'
All I see is "36 updated" but not why/how they weren't before. Out of ideas for now, but do check the logs if you "upload" a new batch
I see that I have 'Library watching' turned on in the settings, with default settings for that, so that'll be why they're showing up right away at least - even if it's not 'right'. I wonder if it's viable for me to just reduce this scanning interval massively and if this scan's the same one as the external library rescan
The watching should also see xmp files
er
They aren't xmp files
Ah but perhaps it sees the original and not the edited one
nah the originals are in another folder not available to Immich. I'm imagining it might be that Immich is a little too eager to add the files, and they're not fully written or something at the time they're picked up
That's what I meant, it picks up a version, uploads, then ignores any ongoing edits
Yeah! Think that might actually be the case, just an educated guess more than anything
Easy to test 😛
AFAIK it should handle this and wait for a file to settle down before it pulls it in
Oh nvm, seems like we don't set
awaitWriteFinish. @etnoy is there a reason for that?(I figure I need to wait on that response :D)
Where is this?
I really should've included that huh 🤦
Iirc in the opts for the file watcher
Telling it to wait for a file to stop changing before it emits an event
Ok, I don't know, the file watcher is the ugly stepchild of external libraries
Maybe it happened in my performance refactor
I could have a dig through the git history tomorrow if necessary
It's code that's very hard to test
Maybe it's possible with e2e now but idk
So, searching through the whole history, the only mention of
awaitWriteFinish is in #6192 but only in the form of a key at the top of server/src/infra/entities/system-config.entity.ts that, as far as I can tell, didn't actually get used anywhere