Immich•2y ago

Folder and Storage Migration Mess-up

I use Immich since ~ August 2022. Immich Version: 1.57.1 I wanted to use the new storage label feature and realized after the migration job, I got files in the upload folder as well as in library. And in library there are some assets that apparently could not be migrated from the uuid to the storage template. upload folder:

root@docker:/data/immich/upload# ls -l
insgesamt 44
drwxr-xr-x  7 adminuser adminuser 4096 27. Mai 15:10 1c62e351-3435-4eb7-a2b5-d35b6035c0d2
drwx------  5 adminuser adminuser 4096 27. Mai 13:00 6cc0efad-939f-46f6-8aa5-8d0aeafc906d
drwxr-xr-x 10 root      root      4096 24. Mai 03:47 encoded-video
drwxr-xr-x 14 root      root      4096 27. Mai 14:14 library
drwxr-xr-x  9 root      root      4096 21. Mai 16:43 thumbs
drwxr-xr-x  4 root      root      4096 27. Mai 15:50 upload

root@docker:/data/immich/upload# ls -l
insgesamt 44
drwxr-xr-x  7 adminuser adminuser 4096 27. Mai 15:10 1c62e351-3435-4eb7-a2b5-d35b6035c0d2
drwx------  5 adminuser adminuser 4096 27. Mai 13:00 6cc0efad-939f-46f6-8aa5-8d0aeafc906d
drwxr-xr-x 10 root      root      4096 24. Mai 03:47 encoded-video
drwxr-xr-x 14 root      root      4096 27. Mai 14:14 library
drwxr-xr-x  9 root      root      4096 21. Mai 16:43 thumbs
drwxr-xr-x  4 root      root      4096 27. Mai 15:50 upload

root@docker:/data/immich/upload/library# ls -l
insgesamt 48
drwxr-xr-x  5 root root 4096 27. Mai 14:14 1c62e351-3435-4eb7-a2b5-d35b6035c0d2 (this is the uuid for lars)
drwxr-xr-x  3 root root 4096 27. Mai 14:14 6cc0efad-939f-46f6-8aa5-8d0aeafc906d (this is the uuid for userN)
drwxr-xr-x 11 root root 4096 27. Mai 14:10 lars
drwxr-xr-x 11 root root 4096 27. Mai 14:10 userN

root@docker:/data/immich/upload/library# ls -l
insgesamt 48
drwxr-xr-x  5 root root 4096 27. Mai 14:14 1c62e351-3435-4eb7-a2b5-d35b6035c0d2 (this is the uuid for lars)
drwxr-xr-x  3 root root 4096 27. Mai 14:14 6cc0efad-939f-46f6-8aa5-8d0aeafc906d (this is the uuid for userN)
drwxr-xr-x 11 root root 4096 27. Mai 14:10 lars
drwxr-xr-x 11 root root 4096 27. Mai 14:10 userN

The storage migration job in the microservice container is not throwing any error. If I search for m:FILENAME i can find all files as far as I can tell. I'd like to now, how to deal with this assets now.

12 Replies

koyaxOP•2y ago

This is how the Folders with the uuid in upload look like:

root@docker:/data/immich/upload/1c62e351-3435-4eb7-a2b5-d35b6035c0d2# ls -la
insgesamt 104
drwxr-xr-x  7 adminuser adminuser  4096 27. Mai 15:10 .
drwxr-xr-x 12 adminuser adminuser  4096 27. Mai 16:10 ..
drwxr-xr-x  3 adminuser adminuser  4096 27. Mai 13:00 2016
drwx------  3 adminuser adminuser  4096 27. Mai 13:00 2023
drwx------  2 adminuser adminuser 81920  7. Apr 11:03 encoded-video
drwx------  3 adminuser adminuser  4096 27. Mai 13:00 original
drwx------  6 adminuser adminuser  4096 13. Dez 17:16 thumb

root@docker:/data/immich/upload/1c62e351-3435-4eb7-a2b5-d35b6035c0d2# ls -la
insgesamt 104
drwxr-xr-x  7 adminuser adminuser  4096 27. Mai 15:10 .
drwxr-xr-x 12 adminuser adminuser  4096 27. Mai 16:10 ..
drwxr-xr-x  3 adminuser adminuser  4096 27. Mai 13:00 2016
drwx------  3 adminuser adminuser  4096 27. Mai 13:00 2023
drwx------  2 adminuser adminuser 81920  7. Apr 11:03 encoded-video
drwx------  3 adminuser adminuser  4096 27. Mai 13:00 original
drwx------  6 adminuser adminuser  4096 13. Dez 17:16 thumb

encoded-video also still contains encoded videos, that are already replaced by new ones, that I created through the video encoder job. Could it be, that I should have run the storage migration job after some specific updates?

Alex Tran•2y ago

https://github.com/immich-app/immich/releases/tag/v1.52.0

GitHub

Release v1.52.0 · immich-app/immich

v1.52.0 Highlights This release includes many bug fixes and hardening of the recent new features in Immich. We are making Immich more robust as we try to move into the stable territory of the appli...

Alex Tran•2y ago

There was this release that move the structure around

koyaxOP•2y ago

Thanks. Totally forgot about this update. I remember not doing the migration yet, because I wanted to wait for bitrate limiting for encoded videos. The storage migration job did not move all originals to library for me though. And running it again doesn't move the files left behind. They definitely exist in the database though. Did the folder structure change multiple times? look like There are original files in the folder sorted by Date (like 2016 and 2023) as well in original, where they are stored with their uuid as filename.

Alex Tran•2y ago

No it changes very infrequent, we only make the changes if we must, rather do this in the development phases than in when the app steps into the stable stage

koyaxOP•2y ago

@Alex Are transcode and thumbnail jobs that are started for missing ones checking if files are actually present or are they just looking into the database and check if there is a path set?

Alex Tran•2y ago

They will check if there is a path set for the thumbnail

koyaxOP•2y ago

So if I delete the thumbnail file on the file system after creation, it will not be recreated if I run the job for missing thumbnails?

Alex Tran•2y ago

that is correct

koyaxOP•2y ago

How "safe" is it, to remove assets from the database and to reupload them manually?

Alex Tran•2y ago

Pretty safe

koyaxOP•2y ago

Okay. I have some assets, that for whatever reason, are not migrated to the new library folder. I'd now go and delete these assets from the database, reupload the files via the webui and then delete the files in the old location. Okay, moved failed storage migration assets manually now. In the Database I saw a few video assets where encodedVideoPath still is <null> but the Transcode Job doesn't start the transcode job for these assets. Anything else, that the transcode Job considers for assets, where the transcode is "missing"?

Gaming

Programming

Folder and Storage Migration Mess-up

Did you find this page helpful?