microservices worker exited with code 1

Hello, not sure what I've done. I've seen a few posts on Github and Discord that seem similar, but can't see any solutions that might assist to fix. I've been running Immich for a few weeks without issue. It is currently running on verison v1.131.3 Today I restarted the Postgres container, didn't think much of it, and no idea if that caused the issue, but now the Immich server continually displays this in a loop:
Initializing Immich v1.131.3
Detected CPU Cores: 16
Starting api worker
Starting microservices worker
[Nest] 7 - 04/12/2025, 7:42:22 AM LOG [Microservices:EventRepository] Initialized websocket server
microservices worker error: TypeError: Cannot read properties of undefined (reading 'replace'), stack: TypeError: Cannot read properties of undefined (reading 'replace')
at queryError (/usr/src/app/node_modules/postgres/cjs/src/connection.js:389:48)
at errored (/usr/src/app/node_modules/postgres/cjs/src/connection.js:384:17)
at Socket.error (/usr/src/app/node_modules/postgres/cjs/src/connection.js:376:5)
at Socket.emit (node:events:518:28)
at emitErrorNT (node:internal/streams/destroy:170:8)
at emitErrorCloseNT (node:internal/streams/destroy:129:3)
at process.processTicksAndRejections (node:internal/process/task_queues:90:21)
microservices worker exited with code 1
Killing api process
Initializing Immich v1.131.3
Detected CPU Cores: 16
Starting api worker
Starting microservices worker
[Nest] 7 - 04/12/2025, 7:42:22 AM LOG [Microservices:EventRepository] Initialized websocket server
microservices worker error: TypeError: Cannot read properties of undefined (reading 'replace'), stack: TypeError: Cannot read properties of undefined (reading 'replace')
at queryError (/usr/src/app/node_modules/postgres/cjs/src/connection.js:389:48)
at errored (/usr/src/app/node_modules/postgres/cjs/src/connection.js:384:17)
at Socket.error (/usr/src/app/node_modules/postgres/cjs/src/connection.js:376:5)
at Socket.emit (node:events:518:28)
at emitErrorNT (node:internal/streams/destroy:170:8)
at emitErrorCloseNT (node:internal/streams/destroy:129:3)
at process.processTicksAndRejections (node:internal/process/task_queues:90:21)
microservices worker exited with code 1
Killing api process
I've now restarted the whole docker server just to rule that out. But still same issue. The Postgres container logs say the following:
PostgreSQL Database directory appears to contain a database; Skipping initialization
2025-04-12 07:24:45.672 UTC [1] LOG: redirecting log output to logging collector process
2025-04-12 07:24:45.672 UTC [1] HINT: Future log output will appear in directory "log".
PostgreSQL Database directory appears to contain a database; Skipping initialization
2025-04-12 07:24:45.672 UTC [1] LOG: redirecting log output to logging collector process
2025-04-12 07:24:45.672 UTC [1] HINT: Future log output will appear in directory "log".
16 Replies
Immich
Immich3w ago
:wave: Hey @Zoltrix, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time). 2. :ballot_box_with_check: read applicable release notes. 3. :ballot_box_with_check: reviewed the FAQs for known issues. 4. :ballot_box_with_check: reviewed Github for known issues. 5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy). 6. :ballot_box_with_check: uploaded the relevant information (see below). 7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed. Successfully submitted, a tag has been added to inform contributors. :white_check_mark:
Zoltrix
ZoltrixOP3w ago
Not sure if the above helps? Assuming that confirms the database has content?
bo0tzz
bo0tzz3w ago
We'll need your compose and .env, detail about the system, etc
2025-04-12 07:24:45.672 UTC [1] HINT: Future log output will appear in directory "log".
And ideally those logs from postgres
Zoltrix
ZoltrixOP3w ago
No description
Zoltrix
ZoltrixOP3w ago
From latest postgres logs:
2025-04-12 07:24:45.672 UTC [1] LOG: starting PostgreSQL 14.17 (Debian 14.17-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
2025-04-12 07:24:45.672 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2025-04-12 07:24:45.672 UTC [1] LOG: listening on IPv6 address "::", port 5432
2025-04-12 07:24:45.674 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2025-04-12 07:24:45.678 UTC [28] LOG: database system was shut down at 2025-04-12 07:24:45 UTC
[2025-04-12T07:24:45Z INFO service::utils::clean] Find directory "pg_vectors/indexes/17575".
[2025-04-12T07:24:45Z INFO service::utils::clean] Find directory "pg_vectors/indexes/17331".
[2025-04-12T07:24:45Z INFO service::utils::clean] Find directory "pg_vectors/indexes/17575/segments/d8d491f2-5cb4-4485-a400-7a7966e3fa2b".
2025-04-12 07:24:45.685 UTC [1] LOG: database system is ready to accept connections
[2025-04-12T07:24:45Z INFO service::utils::clean] Find directory "pg_vectors/indexes/17331/segments/a079820e-c1bc-41f7-b040-0f65f1a08e42".
2025-04-12 07:24:45.672 UTC [1] LOG: starting PostgreSQL 14.17 (Debian 14.17-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
2025-04-12 07:24:45.672 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
2025-04-12 07:24:45.672 UTC [1] LOG: listening on IPv6 address "::", port 5432
2025-04-12 07:24:45.674 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2025-04-12 07:24:45.678 UTC [28] LOG: database system was shut down at 2025-04-12 07:24:45 UTC
[2025-04-12T07:24:45Z INFO service::utils::clean] Find directory "pg_vectors/indexes/17575".
[2025-04-12T07:24:45Z INFO service::utils::clean] Find directory "pg_vectors/indexes/17331".
[2025-04-12T07:24:45Z INFO service::utils::clean] Find directory "pg_vectors/indexes/17575/segments/d8d491f2-5cb4-4485-a400-7a7966e3fa2b".
2025-04-12 07:24:45.685 UTC [1] LOG: database system is ready to accept connections
[2025-04-12T07:24:45Z INFO service::utils::clean] Find directory "pg_vectors/indexes/17331/segments/a079820e-c1bc-41f7-b040-0f65f1a08e42".
I noticed this thread mentions permission issues with group and user ID https://discord.com/channels/979116623879368755/1049703391762321418/threads/1359070399916146729 My env doesn't appear to have PUID and PGID variables, but it never has, and has worked. They weren't listed in the instructions I followed here https://immich.app/docs/install/portainer/ Of course I might be on the wrong track 🤷‍♂️
Zeus
Zeus3w ago
Immich doesn’t even support PUID or PGID. Thats not related at all
Zoltrix
ZoltrixOP3w ago
Users here reporting same issue: https://github.com/immich-app/immich/issues/16054 Some reported rolling back to v1.125.7 fixed it. I tried that, now getting the following:
Zoltrix
ZoltrixOP3w ago
Might have to try a restore... am concerned though that following the instructions https://immich.app/docs/administration/backup-and-restore/ and running a docker compose down -v command might delete the non Immich containers as well... Just confirming that @Alex does mention it is required here: https://discord.com/channels/979116623879368755/1049703391762321418/threads/1359070399916146729
bo0tzz
bo0tzz3w ago
I think he just meant "Could it be a permissions issue"
Zeus
Zeus3w ago
https://immich.app/docs/install/environment-variables/ You can review the code but as far as I know the PUID/PGID are not loaded or referenced in our codebase in any way it will affect the containers in that docker-compose.yml file
Zoltrix
ZoltrixOP3w ago
Restored from backup to a second docker server... was all running ok... once I tried to exclude the containers from auto updating with Watchtower (eg, apply the "com.centurylinklabs.watchtower.enable" label to the Immich containers) it shit the bed again and broke... incidentally this is what must have broke it on my first docker server, as I was trying to exclude them from Watchtower, I didn't think it could have been the cause when it first happened... so no idea if this is a bug or not? 🤷‍♂️
Zeus
Zeus3w ago
It sounds like you’re not applying labels correctly and breaking your compose.
Zoltrix
ZoltrixOP3w ago
Perhaps me not understanding things correctly, but I thought I could just add the label via Portainer and redeploy it?
No description
Zoltrix
ZoltrixOP3w ago
Ok from doing some research you can apply a label to an individual container that is built from a stack... the change will not persist once that stack is re-deployed, but it should not catastrophically break things like it has done with Immich

Did you find this page helpful?