I
Immich5mo ago
raf

Error loading images after storage template migration

Hello everyone, I posted on Reddit and was directed from a maintainer to post here as well: After enabling the following storage template {{y}}/{{y}}-{{MM}}/{{filename}} , I ran the "Storage template migration" job. After that, all images are show as "Error loading image". The photos are not gone, as I can see them stored in the hard drive in the following structure: /immich-upload/library/user/[photos]. Also, the server stats shows the correct photos count. I also ran all jobs to see if it would fix it, but no success. Anyone would have an idea what is going on and what I should do to solve this? Details: Immich version: 1.123.0 Installation: Docker compose in Debian server Directory mounted from OMV NAS as SMB/CIFS .env file:
UPLOAD_LOCATION=/mnt/nas/photos/immich-upload
DB_DATA_LOCATION=./data/pgdata
IMMICH_VERSION=v1.123.0
DB_PASSWORD=[redacted]
DB_USERNAME=[redacted]
DB_DATABASE_NAME=[redacted]
UPLOAD_LOCATION=/mnt/nas/photos/immich-upload
DB_DATA_LOCATION=./data/pgdata
IMMICH_VERSION=v1.123.0
DB_PASSWORD=[redacted]
DB_USERNAME=[redacted]
DB_DATABASE_NAME=[redacted]
Docker compose:
name: immich
services:
immich-server:
container_name: immich_server
image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
- /etc/localtime:/etc/localtime:ro
env_file:
- .env
ports:
- 3007:2283
depends_on:
- redis
- database
restart: always
healthcheck:
disable: false
name: immich
services:
immich-server:
container_name: immich_server
image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
volumes:
- ${UPLOAD_LOCATION}:/usr/src/app/upload
- /etc/localtime:/etc/localtime:ro
env_file:
- .env
ports:
- 3007:2283
depends_on:
- redis
- database
restart: always
healthcheck:
disable: false
No description
No description
54 Replies
Immich
Immich5mo ago
:wave: Hey @raf, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time). 2. :ballot_box_with_check: read applicable release notes. 3. :blue_square: reviewed the FAQs for known issues. 4. :blue_square: reviewed Github for known issues. 5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy). 6. :ballot_box_with_check: uploaded the relevant information (see below). 7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed.
Zeus
Zeus5mo ago
What’s in the logs?
raf
rafOP5mo ago
Thank you for your help! This is repeating for each photo:
[Nest] 17 - 12/21/2024, 3:31:14 PM ERROR [Api:GlobalExceptionFilter~5oxv0yao] Unknown error: Error: ENOENT: no such file or directory, access 'upload/thumbs/a733e9fc-3103-4ba3-9c25-12b32c213bc6/e1/74/e174f2ca-43c6-4758-b1a9-1b3ae44f9f42-thumbnail.webp'
Error: ENOENT: no such file or directory, access 'upload/thumbs/a733e9fc-3103-4ba3-9c25-12b32c213bc6/e1/74/e174f2ca-43c6-4758-b1a9-1b3ae44f9f42-thumbnail.webp'
at async access (node:internal/fs/promises:605:10)
at async sendFile (/usr/src/app/dist/utils/file.js:54:9)
at async AssetMediaController.viewAsset (/usr/src/app/dist/controllers/asset-media.controller.js:58:9)
[Nest] 17 - 12/21/2024, 3:31:14 PM ERROR [Api:GlobalExceptionFilter~5oxv0yao] Unknown error: Error: ENOENT: no such file or directory, access 'upload/thumbs/a733e9fc-3103-4ba3-9c25-12b32c213bc6/e1/74/e174f2ca-43c6-4758-b1a9-1b3ae44f9f42-thumbnail.webp'
Error: ENOENT: no such file or directory, access 'upload/thumbs/a733e9fc-3103-4ba3-9c25-12b32c213bc6/e1/74/e174f2ca-43c6-4758-b1a9-1b3ae44f9f42-thumbnail.webp'
at async access (node:internal/fs/promises:605:10)
at async sendFile (/usr/src/app/dist/utils/file.js:54:9)
at async AssetMediaController.viewAsset (/usr/src/app/dist/controllers/asset-media.controller.js:58:9)
bo0tzz
bo0tzz5mo ago
If you open up one of these assets and run the thumbnail job from the menu, what happens?
Zeus
Zeus5mo ago
What’s at /mnt/nas/photos/immich-upload/thumbs?
raf
rafOP5mo ago
This is the tree structure, level 3
tree -d -L 3 /mnt/nas/photos
/mnt/nas/photos
|-- immich-library
`-- immich-upload
|-- backups
|-- encoded-video
|-- library
| `-- rafael
|-- profile
|-- thumbs
| `-- a733e9fc-3103-4ba3-9c25-12b32c213bc6
`-- upload
`-- a733e9fc-3103-4ba3-9c25-12b32c213bc6
tree -d -L 3 /mnt/nas/photos
/mnt/nas/photos
|-- immich-library
`-- immich-upload
|-- backups
|-- encoded-video
|-- library
| `-- rafael
|-- profile
|-- thumbs
| `-- a733e9fc-3103-4ba3-9c25-12b32c213bc6
`-- upload
`-- a733e9fc-3103-4ba3-9c25-12b32c213bc6
The pictures are inside immich-upload/library/rafael/*
Zeus
Zeus5mo ago
Try this
raf
rafOP5mo ago
Opening them in Immich shows the same error. I also ran the thumbnail job, but didn't fix it. This error appears in the log, when running the thumbnails job:
[Nest] 7 - 12/21/2024, 3:42:03 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: Input file is missing: upload/library/rafael/2020/2020-02/IMG_4216.HEIC
[Nest] 7 - 12/21/2024, 3:42:03 PM ERROR [Microservices:JobService] Error: Input file is missing: upload/library/rafael/2020/2020-02/IMG_4216.HEIC
[Nest] 7 - 12/21/2024, 3:42:03 PM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: Input file is missing: upload/library/rafael/2020/2020-02/IMG_4216.HEIC
[Nest] 7 - 12/21/2024, 3:42:03 PM ERROR [Microservices:JobService] Error: Input file is missing: upload/library/rafael/2020/2020-02/IMG_4216.HEIC
The thumbnails are there.
Zeus
Zeus5mo ago
Please post the output of docker inspect immich_server
raf
rafOP5mo ago
docker inspect immich-server
[]
Error: No such object: immich-server
docker inspect immich-server
[]
Error: No such object: immich-server
Mraedis
Mraedis5mo ago
_ not -
Zeus
Zeus5mo ago
Underscore
raf
rafOP5mo ago
Sorry, here it goes
Mraedis
Mraedis5mo ago
But this just sounds like the nas was not mounted yet when the container was started to me
raf
rafOP5mo ago
I'll restart the container
Mraedis
Mraedis5mo ago
Can you download the original through immich? yeah restarting should fix that
raf
rafOP5mo ago
No, download doesn't work.
Mraedis
Mraedis5mo ago
Just confirming my suspicions there 😛
raf
rafOP5mo ago
Restarted the container, same error, pictures show as "Error loading image".
Zeus
Zeus5mo ago
Why did it start without the .immich files though..? Please run docker exec -it immich_server ls /usr/src/app/upload
raf
rafOP5mo ago
docker exec -it immich_server ls /usr/src/app/upload
backups encoded-video library profile thumbs upload
docker exec -it immich_server ls /usr/src/app/upload
backups encoded-video library profile thumbs upload
Zeus
Zeus5mo ago
Now add on library/rafael/2020 to that command
raf
rafOP5mo ago
2020 doesn't exist.
docker exec -it immich_server ls /usr/src/app/upload/library/rafael
2024
docker exec -it immich_server ls /usr/src/app/upload/library/rafael
2024
Zeus
Zeus5mo ago
Do a docker compose down then docker compose up -d
raf
rafOP5mo ago
Same error.
[Nest] 17 - 12/21/2024, 3:54:25 PM ERROR [Api:GlobalExceptionFilter~wl6gw9om] Unknown error: Error: ENOENT: no such file or directory, access 'upload/thumbs/a733e9fc-3103-4ba3-9c25-12b32c213bc6/ea/ba/eaba140c-ec2d-43ba-983c-119e2818d803-thumbnail.webp'
[Nest] 17 - 12/21/2024, 3:54:25 PM ERROR [Api:GlobalExceptionFilter~wl6gw9om] Unknown error: Error: ENOENT: no such file or directory, access 'upload/thumbs/a733e9fc-3103-4ba3-9c25-12b32c213bc6/ea/ba/eaba140c-ec2d-43ba-983c-119e2818d803-thumbnail.webp'
Zeus
Zeus5mo ago
Also whats in the library/rafael folder on the NAS?
raf
rafOP5mo ago
/mnt/nas/photos
|-- immich-library
`-- immich-upload
|-- backups
|-- encoded-video
|-- library
| `-- rafael
| `-- 2024
/mnt/nas/photos
|-- immich-library
`-- immich-upload
|-- backups
|-- encoded-video
|-- library
| `-- rafael
| `-- 2024
Zeus
Zeus5mo ago
please stop immich docker, unmount the nas mount and see what is in the underlying folder
raf
rafOP5mo ago
Okay, something is off. Debain server only sees 2024, but NAS has all the other years.
Zeus
Zeus5mo ago
perhaps your NAS is not mounted at all
raf
rafOP5mo ago
No description
raf
rafOP5mo ago
let me ls the mount in debian server
ls /mnt/nas/photos/immich-upload/library/rafael/
2024
ls /mnt/nas/photos/immich-upload/library/rafael/
2024
Ok, looks like the error is that the mount only shows one folder, for some reason.
Zeus
Zeus5mo ago
I think it's more likely that it's not mounted at all and you're seeing your local filesystem can you run df -h
raf
rafOP5mo ago
df -h
Filesystem Size Used Avail Use% Mounted on
udev 7.3G 0 7.3G 0% /dev
tmpfs 1.5G 3.3M 1.5G 1% /run
/dev/sda2 218G 50G 157G 25% /
tmpfs 7.3G 0 7.3G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/sda1 511M 5.9M 506M 2% /boot/efi
tmpfs 1.5G 0 1.5G 0% /run/user/1000
df -h
Filesystem Size Used Avail Use% Mounted on
udev 7.3G 0 7.3G 0% /dev
tmpfs 1.5G 3.3M 1.5G 1% /run
/dev/sda2 218G 50G 157G 25% /
tmpfs 7.3G 0 7.3G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/sda1 511M 5.9M 506M 2% /boot/efi
tmpfs 1.5G 0 1.5G 0% /run/user/1000
Looks like it's not mounted I will stop the container and mount. Ok, mounted back:
df -h
Filesystem Size Used Avail Use% Mounted on
udev 7.3G 0 7.3G 0% /dev
tmpfs 1.5G 3.0M 1.5G 1% /run
/dev/sda2 218G 50G 157G 25% /
tmpfs 7.3G 0 7.3G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/sda1 511M 5.9M 506M 2% /boot/efi
tmpfs 1.5G 0 1.5G 0% /run/user/1000
//192.168.178.49/photos 3.6T 564G 3.1T 16% /mnt/nas/photos
df -h
Filesystem Size Used Avail Use% Mounted on
udev 7.3G 0 7.3G 0% /dev
tmpfs 1.5G 3.0M 1.5G 1% /run
/dev/sda2 218G 50G 157G 25% /
tmpfs 7.3G 0 7.3G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
/dev/sda1 511M 5.9M 506M 2% /boot/efi
tmpfs 1.5G 0 1.5G 0% /run/user/1000
//192.168.178.49/photos 3.6T 564G 3.1T 16% /mnt/nas/photos
Restarted the container, but it is not starting:
[Nest] 7 - 12/21/2024, 4:05:15 PM ERROR [Microservices:StorageService] Failed to read upload/backups/.immich: Error: ENOENT: no such file or directory, open 'upload/backups/.immich'
microservices worker error: Error: Failed to read "<UPLOAD_LOCATION>/backups/.immich - Please see https://immich.app/docs/administration/system-integrity#folder-checks for more information."
[Nest] 17 - 12/21/2024, 4:05:15 PM LOG [Api:EventRepository] Initialized websocket server
microservices worker exited with code 1
Killing api process
Container stopped
[Nest] 7 - 12/21/2024, 4:05:15 PM ERROR [Microservices:StorageService] Failed to read upload/backups/.immich: Error: ENOENT: no such file or directory, open 'upload/backups/.immich'
microservices worker error: Error: Failed to read "<UPLOAD_LOCATION>/backups/.immich - Please see https://immich.app/docs/administration/system-integrity#folder-checks for more information."
[Nest] 17 - 12/21/2024, 4:05:15 PM LOG [Api:EventRepository] Initialized websocket server
microservices worker exited with code 1
Killing api process
Container stopped
Zeus
Zeus5mo ago
you will need to create the .immich file in the backups folder of UPLOAD_LOCATION it seems that for some time possible your NAS has been unmounted so you may have a split immich install
raf
rafOP5mo ago
Oh, that sounds bad. Should I create an empty file?
Zeus
Zeus5mo ago
yes just touch .immich
raf
rafOP5mo ago
I'm sorry, but what is the backup folder inside the UPLOAD_LOCATION? I have these:
ls -la
total 4
drwxr-xr-x 2 rafaelfe rafaelfe 0 Oct 10 17:18 .
drwxr-xr-x 2 rafaelfe rafaelfe 0 Nov 6 00:13 ..
drwxr-xr-x 2 rafaelfe rafaelfe 0 Oct 10 17:18 encoded-video
drwxr-xr-x 2 rafaelfe rafaelfe 0 Oct 17 19:10 library
drwxr-xr-x 2 rafaelfe rafaelfe 0 Oct 10 17:18 profile
drwxr-xr-x 2 rafaelfe rafaelfe 0 Oct 10 17:18 thumbs
drwxr-xr-x 2 rafaelfe rafaelfe 0 Oct 10 17:18 upload
ls -la
total 4
drwxr-xr-x 2 rafaelfe rafaelfe 0 Oct 10 17:18 .
drwxr-xr-x 2 rafaelfe rafaelfe 0 Nov 6 00:13 ..
drwxr-xr-x 2 rafaelfe rafaelfe 0 Oct 10 17:18 encoded-video
drwxr-xr-x 2 rafaelfe rafaelfe 0 Oct 17 19:10 library
drwxr-xr-x 2 rafaelfe rafaelfe 0 Oct 10 17:18 profile
drwxr-xr-x 2 rafaelfe rafaelfe 0 Oct 10 17:18 thumbs
drwxr-xr-x 2 rafaelfe rafaelfe 0 Oct 10 17:18 upload
Zeus
Zeus5mo ago
I guess you need to create it. out of curiosity what is the size of the UPLOAD_LOCATION that is NOT on the NAS? the one that just had 2024 (both upload and library folders)
raf
rafOP5mo ago
The UPLOAD_LOCATION is pointing to the mount, that is in the NAS. I mounted the NAS back, and I assume it merged to what was in the local folder before.
Zeus
Zeus5mo ago
nope, no merging whatever was there is still there if you unmount the nas
raf
rafOP5mo ago
I see, so the local files are disabled (or hidden) once the mount is active?
Zeus
Zeus5mo ago
yes
raf
rafOP5mo ago
du -h --max-depth=1 | sort -hr
53G .
9.9G ./2021
9.1G ./2023
9.1G ./2020
9.0G ./2022
8.0G ./2024
7.9G ./2019
4.3M ./2018
3.6M ./2015
1.4M ./2017
du -h --max-depth=1 | sort -hr
53G .
9.9G ./2021
9.1G ./2023
9.1G ./2020
9.0G ./2022
8.0G ./2024
7.9G ./2019
4.3M ./2018
3.6M ./2015
1.4M ./2017
I assume those are sizes for the mount
Zeus
Zeus5mo ago
yes
raf
rafOP5mo ago
Should I unmount and check?
Zeus
Zeus5mo ago
yep
raf
rafOP5mo ago
By the way, all the photos are shown correctly now, with correctly mounting the NAS directory and adding the .immich file to a /backups folder. I'm very grateful for your help. But let me get back to the local directory size.
Zeus
Zeus5mo ago
oh good
raf
rafOP5mo ago
Output of local directory size:
du -h --max-depth=1 | sort -hr
629M ./2024
629M .
du -h --max-depth=1 | sort -hr
629M ./2024
629M .
This is funny.
Zeus
Zeus5mo ago
ok, and how about the upload folder? you might be able to piece things together if you put those files back in the correct spot in the NAS folder
raf
rafOP5mo ago
du -h --max-depth=1 | sort -hr
705M .
629M ./library
75M ./thumbs
1.4M ./upload
8.0K ./profile
8.0K ./encoded-video
8.0K ./backups
du -h --max-depth=1 | sort -hr
705M .
629M ./library
75M ./thumbs
1.4M ./upload
8.0K ./profile
8.0K ./encoded-video
8.0K ./backups
I see. Those are probably files uploaded from my phone. Do you think if I delete everything from the local directory, it would reupload again?
Zeus
Zeus5mo ago
no, you have to delete them through the immich UI
raf
rafOP5mo ago
So copying those files to the NAS, as you suggested, should be a better option. Is it enough to just copy the originals, or should I copy the contents of the thumbs folder as well? Despite the following outcome, I'd like to thank you and Mraedis for the great help. Not surprising to anyone, it was my mistake, nothing related to Immich itself. Thank you for your support.

Did you find this page helpful?