Manual upgrade from 1.116 to 1.123 not working
What am I doing wrong? I am trying to migrate Immich from 1.116 to 1.123. Because 1.116 is under supervision of Cosmos cloud (which is a Docker container manager among other things) and automatic update does not work, I started a completely new instance of Immich using the Getting Started documentation and doing everything manually (ie creating a new folder, copying the example compose and env files, editing them so that the UPLOAD_LOCATION points to the same place as the old instance of Immich is using). I then took the latest Postgres database dump from the old Immich instance and loaded restored it into the new destination. Library path is identical for both Immich instances. When starting the new instance, I was asked to create a new (admin) user. I did, and I used the same e-mail address and password that I had for the old instance. I was expecting to see all the photos from the old instance, but it is blank. Where did I go wrong?
79 Replies
:wave: Hey @Kvickis,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
2. :blue_square: read applicable release notes.
3. :blue_square: reviewed the FAQs for known issues.
4. :blue_square: reviewed Github for known issues.
5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed..env file:
You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables
The location where your uploaded files are stored
UPLOAD_LOCATION=/usr/immich-photos/library
The location where your database files are stored
DB_DATA_LOCATION=./postgres
To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
TZ=Etc/UTC
The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release
Connection secret for postgres. You should change it to a random password
Please use only the characters
A-Za-z0-9, without special characters or spaces
DB_PASSWORD=U8RROEFOTYJENM6O17T1UNZI
The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immichWhen starting the new instance, I was asked to create a new (admin) user.This suggests that the postgres dump didn't restore properly
OK, but the output from the restore command just showed words like "ADDED... ", "CREATED...", "CHANGED..." etc. No indication whatsoever of something going wrong. Could it be that the database password needs enclosing in quotation marks or something? (I used the same password as for the 1.116 instance of Immich)
Here is the restore command I used:
gunzip < *.sql.gz | sed "s/SELECT pg_catalog.set_config('search_path', '', false);/SELECT pg_catalog.set_config('search_path', 'public, pg_catalog', true);/g" | docker exec -i immich_postgres psql --username=postgres
Should it have included a --password= also?
Do you still have the output it gave? The command is straight from the docs I assume
Sorry, no, it went off the buffer I am afraid. I have deleted everything related to postgresql now and trying to recreate the steps. I tried to add a parameter --password but that didn't seem to work. I will post back when I am a bit further
OK, here is the output from the restore command (I did not get a chance to input the database password however):
Please run this:
docker exec -it immich_postgres psql -U postgres -d immich
Then once you’re in, type \droot@cosmos:/home/puke/docker/immich# docker exec -it immich_postgres psql -U postgres -d immich
psql (14.10 (Debian 14.10-1.pgdg120+1))
Type "help" for help.
immich=# \d
Did not find any relations.
immich=#
\l ?immich=# \l
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges
-----------+----------+----------+------------+------------+-----------------------
immich | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 |
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +
| | | | | postgres=CTc/postgres
(4 rows)
\l+ please
In code blocksList of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges | Size | Tablespace |
Description
-----------+----------+----------+------------+------------+-----------------------+---------+------------+--------------------------------------------
immich | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | 8569 kB | pg_default |
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | 1168 MB | pg_default | default administrative connection database
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +| 8417 kB | pg_default | unmodifiable empty database
| | | | | postgres=CTc/postgres | | |
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +| 8417 kB | pg_default | default template for new databases
| | | | | postgres=CTc/postgres | | |
(4 rows)
Sorry, I don't think that was code-blocks. Please remind me on how to enter them
Triple backtick `
Ah
List of databases
Name | Owner | Encoding | Collate | Ctype | Access privileges | Size | Tablespace |
Description
-----------+----------+----------+------------+------------+-----------------------+---------+------------+--------------------------------------------
immich | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | 8569 kB | pg_default |
postgres | postgres | UTF8 | en_US.utf8 | en_US.utf8 | | 1168 MB | pg_default | default administrative connection database
template0 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +| 8417 kB | pg_default | unmodifiable empty database
| | | | | postgres=CTc/postgres | | |
template1 | postgres | UTF8 | en_US.utf8 | en_US.utf8 | =c/postgres +| 8417 kB | pg_default | default template for new databases
| | | | | postgres=CTc/postgres | | |
(4 rows)
Any better?
Your restore went into the Postgres database
How did you create this backup?
The automatic function in Immich (?)
Question mark because I didn't do anything myself
I dont think so..
What folder was it in?
Is
DB_DATABASE_NAME here the same as on your old setup?/usr/immich-backup/daily/immich-latest.sql.gz
Let me check on the DB_DATABASE_NAME
That doesn't smell like Immich's built in backups
You had the backup database dumper container, with a very old set of settings
That is correct
Can you please check in your UPLOAD_LOCATION/backups folder
OK, the folder exists but is empty
That was from 1.120 Zeus, he's coming from 1.116
Oh yeah
Ok, you’ll need to do the restore again and add
--dbname=immich to the psql commandOK, will have to down the containers first I suppose. (I also checked on the DATABASE_NAME and it is "immich" in both instances)
You’ll have to down it and empty the DB_DATA folder
OK, restoring with the --dbname=immich parameter now
Done. Should I start all containers now?
Yes
I guess Postgresql is still working as the "du -sh postgres/" gives increasing values
(and I can hear the fan)
so better to stay calm for a while, right?
Uh, the restore was done?
The restore command finished. But I guess Postgreqsl is doing something
Indexes and stuff probably
It is definitely doing somthing. "top" says it uses 392 %. (Out of 400%)
So I'll wait to startup the other containers until it calms down.
It is still using 200%. I will report back when everything has settled down
And that would be now.
Will start up the other containers now.
One step closer to a solution. The database restore seemed to have worked in the sense, that the users are preserved. Also I can login. After logging in, pictures are replaced with "broken image" place-holders.
Your UPLOAD_LOCATION is identical in both installs? What’s in the logs?
UPLOAD_LOCATION is identical. I will have a look at the log.
I have a log file available. How do I post it here?
If you just saved the library part of the UPLOAD_LOCATION then all your thumbnails (and thus web views) are broken
Check if the actual image is working by downloading the original
If that works, you can fix thumbs by running the job on "ALL"
Yeah but it looks like they’re using the same files on disk
Not a restore
He did say "library"
The UPLOAD_LOCATION in the .env file points to the exact same location in the file-system as the corresponding variable in the JSON-file describing the 1.116 install of Immich did.
Right that should have worked then 🤔
Can’t you just paste the logs or something? Idk
Well, not unless you re-used the exact location but the location according to cosmos was /mnt/media but it actually was something entirely else, which would not surprise me
How can I clarify the situation?
The logs
The log file contains entries like these:
immich_server | [Nest] 17 - 12/21/2024, 2:32:06 PM ERROR [Api:LoggerRepository~52o8l3vj] Unable to send file: Error
immich_server | Error: ENOENT: no such file or directory, access 'upload/thumbs/10268a84-ba9f-4861-ae74-404214ae435c/5d/2f/5d2f49e2-8ba3-4062-9142-2b649a3a6452-thumbnail.jpeg'
immich_server | at async access (node:internal/fs/promises:605:10)
immich_server | at async sendFile (/usr/src/app/dist/utils/file.js:54:9)
immich_server | at async AssetMediaController.viewAsset (/usr/src/app/dist/controllers/asset-media.controller.js:58:9)
immich_server | [Nest] 17 - 12/21/2024, 2:32:06 PM ERROR [Api:GlobalExceptionFilter~52o8l3vj] Unknown error: Error: ENOENT: no such file or directory, access 'upload/thumbs/10268a84-ba9f-4861-ae74-404214ae435c/5d/2f/5d2f49e2-8ba3-4062-9142-2b649a3a6452-thumbnail.jpeg'
Ok, so what is at /usr/immich-photos/library/thumbs?
On the host
root@cosmos:/home/puke/docker/immich# ls -l /usr/immich-photos/library/thumbs/
totalt 4
drwxr-xr-x 3 root root 4096 21 dec 15.38 10268a84-ba9f-4861-ae74-404214ae435c
Ok start going into the folder and see if you can find the thumbnail it’s looking for
root@cosmos:/home/puke/docker/immich# ls -l /usr/immich-photos/library/thumbs/10268a84-ba9f-4861-ae74-404214ae435c/de/0b/
totalt 0
Total 0 ?
No you want
ls -l /usr/immich-photos/library/thumbs/10268a84-ba9f-4861-ae74-404214ae435c/5d/2f for this oneThat does not exist. However, let me try something else: I am looking at a particular photo from the 1.116 installation. It is in upload/library/hgp/2024/2024-12-18/IMG_20241218_195647.jpg
Looking at the same (broken) photo in the 1.123 installation, I see
upload/library/hgp/2024/2024-12-18/IMG_20241218_195647.jpg
So it is pointing to the same file.
And does that file exist in /usr/immich-photos/library/library/hgp/2024…
Also can you download the original of the broken one ? Does that work?
It does exist in the path listed above, yes (not with ../library/library/... as you wrote). I can download it from the 1.116 install, but now from the 1.123 install (it says downloading but the size is only 79 bytes and the download never starts/finishes)
The path I wrote is the correct one
if its not, your UPLOAD_LOCATION is wrong
What’s in /usr/immich-photos?
Please run tree -L2 in that folder
.
├── backups
├── encoded-video
│ ├── 10268a84-ba9f-4861-ae74-404214ae435c
│ ├── 390e7685-f3d8-4c0a-b0cb-d873035e6bec
│ └── 6ca69d8e-d23d-4859-8ab7-dcf7151e8d58
├── library
│ ├── amelie
│ ├── backups
│ ├── encoded-video
│ ├── hgp
│ ├── library
│ ├── martha
│ ├── profile
│ ├── thumbs
│ └── upload
├── profile
├── thumbs
│ ├── 10268a84-ba9f-4861-ae74-404214ae435c
│ ├── 390e7685-f3d8-4c0a-b0cb-d873035e6bec
│ └── 6ca69d8e-d23d-4859-8ab7-dcf7151e8d58
└── upload
├── 10268a84-ba9f-4861-ae74-404214ae435c
├── 390e7685-f3d8-4c0a-b0cb-d873035e6bec
└── 6ca69d8e-d23d-4859-8ab7-dcf7151e8d58
25 directories, 0 files
Your UPLOAD_LOCATION is wrong
You told us you used the same one as the other install but that’s clearly not correct
It should be /usr/immich-photos
The folders in library/ should be empty (profile, upload, thumbs, library, encoded video, backups)
Sorry. So I just down the containers, change from UPLOAD_LOCATION=/usr/immich-photos/library to
UPLOAD_LOCATION=/usr/immich-photos
in the .env file and restart?
Yes, there may be other issues but we’ll cross as they come
Can you do tree -L2 /usr/immich-photos/library as well?
root@cosmos:/usr/immich-photos/library# tree -L 2
.
├── amelie
│ ├── 2012
│ ├── 2017
│ ├── 2019
│ ├── 2020
│ ├── 2022
│ ├── 2023
│ └── 2024
├── backups
├── encoded-video
├── hgp
│ ├── 1978
│ ├── 1989
│ ├── 2015
│ ├── 2017
│ ├── 2018
│ ├── 2019
│ ├── 2020
│ ├── 2021
│ ├── 2022
│ ├── 2023
│ └── 2024
├── library
├── martha
│ ├── 2019
│ ├── 2020
│ ├── 2021
│ ├── 2022
│ ├── 2023
│ └── 2024
├── profile
├── thumbs
│ └── 10268a84-ba9f-4861-ae74-404214ae435c
└── upload
Is the thumbs folder 1026.. empty as well?
Or very small, only a few files?
root@cosmos:/usr/immich-photos/library/thumbs# tree
.
└── 10268a84-ba9f-4861-ae74-404214ae435c
└── de
└── 0b
4 directories, 0 files
I’d just delete all those folders in library/ that are empty
Because you have a nested structure now on accident from starting immich in that folder
And yes do this
I am vary about deleting stuff without having a proper backup... What could happen if I restart, change the env-variable on up the containser again without deleting them first?
It should be fine to just restart with the change. You don’t already have backups? 🥲
I do have backups of the photos, and I do have backups of the database (as you are aware). However, I don't have a separate backup of the immich directory structure, and restoring a delete that went wrong may take a very long time... So I propose that I restart now that I have made the change to the .env file.
THAT WORKED!
Thanks A LOT!
Awesome!
If you’re not going to delete those empty folders at least make a note or something please because if you come back for help we are going to be very confused
Got it! I will make a note.
So just one more question, which is strictly not related to this upgrade issue, if I may: Viewing some old pictures in my external library, When I first click it, it is blurry. Using the scroll-wheel of the mouse, one step, and the picture gets clear (full resolution) as the image is zoomed in. Further zooming in preserves that full resolution, but zooming back out again to the original size gets me that blurry picture. Refreshing thumbnails does not seem to make any difference.
How blurry? You might need to share an example
Are we talking like unrecognizable or just not perfect quality?
Neither. It is low-res (like in perhaps 480p or something)
I can share a picture
The same picture zoomed in one step:
web preview loads the generated thumb which is low res
zooming then loads the original
(as I understand it, might be wrong 😛 )
So this is expected behaviour, and nothing I can change? (It annoys me)
there's a checkmark somewhere to always load originals
I think 🤔 let me find it
user settings -> app settings (top) -> prefer original
Excellent! Thank you soo much! (It's time to BUY Immich, I think)