Immich•5mo ago

Some kind of database screwup/microservices issue

I've been running immich successfully for quite a while now. It's been going great. But I stupidly set up a job to update containers on OMV last week and forgot to exclude Immich. Still, it seemed to be running fine for viewing photos. Then I went to upload yesterday, and no previews or thumbnails would generate, and I couldn't get ANY jobs to run from the admin section. The images I uploaded would download fine uncorrupted.

So I went through the official upgrade process, and changed out the compose file, mainly what changed were the images used for the services. Big mistake. Now I got errors as soon as I got past login. I have plenty of backups, so tried the recovery process multiple times. No dice. All the containers except the immich server start apparently healthy. That one gives microprocessor errors.

OpenMediaVault v. 7 running Immich via docker compose plugin GUI. Run on public-facing domain behind Authelia and NGINX proxy manager.

Immich•7/31/25, 11:50 AM

:wave: Hey @oliphauntsightings,

Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.

References

Container Logs:
```
docker compose logs
```
```
docker compose logs
```
docs
Container Status:
```
docker ps -a
```
```
docker ps -a
```
docs
Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA

Immich•7/31/25, 11:50 AM

Checklist

I have...

:ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
:ballot_box_with_check: read applicable release notes.
:ballot_box_with_check: reviewed the FAQs for known issues.
:ballot_box_with_check: reviewed Github for known issues.
:ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
:ballot_box_with_check: uploaded the relevant information (see below).
:ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

Information

In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:

Your docker-compose.yml and .env files.
Logs from all the containers and their status (see above).
All the troubleshooting steps you've tried so far.
Any recent changes you've made to Immich or your system.
Details about your system (both software/OS and hardware).
Details about your storage (filesystems, type of disks, output of commands like
```
fdisk -l
```
```
fdisk -l
```
and
```
df -h
```
```
df -h
```
).
The version of the Immich server, mobile app, and other relevant pieces.
Any other information that you think might be relevant.

Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)

If this ticket can be closed you can use the

/close

/close

command, and re-open it later if needed.

IImmich :wave: Hey @oliphauntsightings, Thanks for reaching out to us. Please carefully...

oliphauntsightingsOP•7/31/25, 11:53 AM

1 -- I tried with the latest release compose, but it wouldn't start at all.

IImmich :wave: Hey @oliphauntsightings, Thanks for reaching out to us. Please carefully...

oliphauntsightingsOP•7/31/25, 11:55 AM

Here's a paste with the beginning and latest of the logs. They're still running, and ran all night, so too long to post the whole thing. Although if needed I can post all 2MB: https://pastebin.com/bs55zMPH

Pastebin

Immich logs July 31 2025 - Pastebin.com

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

oliphauntsightingsOP•7/31/25, 11:56 AM

immich.env1.13KB

oliphauntsightingsOP•7/31/25, 11:56 AM

immich.yml2.39KB

bo0tzz•7/31/25, 11:57 AM

The postgres container image is wrong

IImmich :wave: Hey @oliphauntsightings, Thanks for reaching out to us. Please carefully...

oliphauntsightingsOP•7/31/25, 11:59 AM

fdisk_-l3.09KB

Bbo0tzz The postgres container image is wrong

oliphauntsightingsOP•7/31/25, 12:00 PM

Yes. The one I was using is no longer available.

IImmich ## Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest rel...

ImmichAPP•7/31/25, 12:02 PM

Successfully submitted, a tag has been added to inform contributors. :white_check_mark:

bo0tzz•7/31/25, 12:02 PM

idk which one you were using before, but either way this one is wrong and won't work

oliphauntsightingsOP•7/31/25, 12:03 PM

I tried with the one listed in the version's compose example, but the container wouldn't start that way

Bbo0tzz idk which one you were using before, but either way this one is wrong and won't ...

oliphauntsightingsOP•7/31/25, 12:04 PM

I can try a fourth restore and change it out again if that might help.

Bbo0tzz idk which one you were using before, but either way this one is wrong and won't ...

oliphauntsightingsOP•7/31/25, 12:09 PM

This was the repository the image was pulled from originally and before I tried to update things after everything went to hell: ghcr.io/immich-app/postgres

bo0tzz•7/31/25, 12:12 PM

That should be right

Mraedis•7/31/25, 12:13 PM

ghcr.io/immich-app/postgres

ghcr.io/immich-app/postgres

contains a bunch of different postgres versions, mixed with a bunch of different extension versions, you probably switched one of them or all of them around

oliphauntsightingsOP•7/31/25, 12:15 PM

Well, I'll try again with that one. Which version of Immich should I use? I have db backups going back a couple weeks. None before I tried to switch to the latest release are labeled with a release tag.

Mraedis•7/31/25, 12:15 PM

I can't tell you that, it should be the same version you used in the past

Mraedis•7/31/25, 12:15 PM

When did you start using immich

oliphauntsightingsOP•7/31/25, 12:16 PM

Looks like March 18

Mraedis•7/31/25, 12:19 PM

That'd be v1.129.0 which has

docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:739cdd626151ff1f796dc95a6591b55a714f341c737e27f045019ceabf8e8c52

docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:739cdd626151ff1f796dc95a6591b55a714f341c737e27f045019ceabf8e8c52

If you upgrade you should now use

14-vectorchord0.4.3-pgvectors0.2.1

14-vectorchord0.4.3-pgvectors0.2.1

Mraedis•7/31/25, 12:19 PM

ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.1

ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.1

oliphauntsightingsOP•7/31/25, 12:20 PM

Thank you. Will changing the postgres image and the version tag in the .env be enough to do the restart properly using the docker compose files?

oliphauntsightingsOP•7/31/25, 12:21 PM

Or I should say attempt the restore

Mraedis•7/31/25, 12:21 PM

You already tried to restore so your DB folder won't be in working condition, no?

Mraedis•7/31/25, 12:21 PM

TL;DR yes, restore

oliphauntsightingsOP•7/31/25, 12:22 PM

I backed up a copy. Plus have the dump backups

Mraedis•7/31/25, 12:22 PM

Go ham, just check the logs

oliphauntsightingsOP•7/31/25, 12:23 PM

Thanks. I'll just try not to go spam.

oliphauntsightingsOP•7/31/25, 1:11 PM

is it normal for it to take me through onboarding when it's up again?

bo0tzz•7/31/25, 1:14 PM

That suggests the restore didn't work

oliphauntsightingsOP•7/31/25, 1:51 PM

Can you tell me exactly what these two commands do? I understand all the others that are part of the process:

oliphauntsightingsOP•7/31/25, 1:51 PM

docker exec -t immich_postgres pg_dumpall --clean --if-exists --username=<DB_USERNAME> | gzip > "/path/to/backup/dump.sql.gz"

oliphauntsightingsOP•7/31/25, 1:52 PM

gunzip --stdout "/path/to/backup/dump.sql.gz" \
| sed "s/SELECT pg_catalog.set_config('search_path', '', false);/SELECT pg_catalog.set_config('search_path', 'public, pg_catalog', true);/g" \
| docker exec -i immich_postgres psql --dbname=postgres --username=<DB_USERNAME>

oliphauntsightingsOP•7/31/25, 1:52 PM

obviously I replaced the variables with actual values

Mraedis•7/31/25, 1:56 PM

docker exec -t immich_postgres pg_dumpall --clean --if-exists --username=<DB_USERNAME> | gzip > "/path/to/backup/dump.sql.gz"

This makes the dump

gunzip --stdout "/path/to/backup/dump.sql.gz" \
| sed "s/SELECT pg_catalog.set_config('search_path', '', false);/SELECT pg_catalog.set_config('search_path', 'public, pg_catalog', true);/g" \
| docker exec -i immich_postgres psql --dbname=postgres --username=<DB_USERNAME>

This unzips the dump, sends the stream to stdout, corrects a wrong on the fly and sends the output to the immich docker container for the restore command

Mraedis•7/31/25, 1:56 PM

Are you hosting on windows?

oliphauntsightingsOP•7/31/25, 2:07 PM

No. It's on OMV, which is kind of a modification on top of linux

Mraedis•7/31/25, 2:08 PM

Ah right OMV

Mraedis•7/31/25, 2:08 PM

So what are the logs for the postgres container

oliphauntsightingsOP•7/31/25, 2:09 PM

new_postgres.log1.68KB

Mraedis•7/31/25, 2:22 PM

That looks like the old postgres log

Mraedis•7/31/25, 2:22 PM

What image are you using

oliphauntsightingsOP•7/31/25, 2:25 PM

I was using these:

oliphauntsightingsOP•7/31/25, 2:26 PM

1immich.env1.12KB

1immich.yml3.19KB

oliphauntsightingsOP•7/31/25, 2:26 PM

I put the one in front in order to upload here to differentiate from the yaml I showed you earlier

Mraedis•7/31/25, 2:28 PM

I wouldn't pin

v1.129.0

v1.129.0

unless you're certain that was the version before upgrading

Mraedis•7/31/25, 2:28 PM

It should be fine to use the very latest compose and restore to that

oliphauntsightingsOP•7/31/25, 2:29 PM

Thanks. So the latest compose and .env, but with the old images?

Some kind of database screwup/microservices issue

References

Checklist

Information

Similar Threads

Similar Threads

References

Checklist

Information

Similar Threads