I
Immich•3mo ago
rtwo_

Server fails to fetch all assets after recent 1.125 update

Hello and thank you for your briiliant work with this app, it works like magic! After the recent update (I can't pinpoint exactly when - I update regularly, but I didn't have time to use Immich much recently; it's happening since at least 1.125.5) the server fails to fetch both all assets and the memory lane. Postgres is crashing the whole server with what I think is a segfault error.

verbose-logs.txt

logs.txt

env

docker-compose.yml

11 Replies
Immich
Immich•3mo ago
:wave: Hey @rtwo_, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Immich
Immich•3mo ago
Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time). 2. :ballot_box_with_check: read applicable release notes. 3. :ballot_box_with_check: reviewed the FAQs for known issues. 4. :ballot_box_with_check: reviewed Github for known issues. 5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy). 6. :ballot_box_with_check: uploaded the relevant information (see below). 7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed.
GitHub
immich-app immich · Discussions
Explore the GitHub Discussions forum for immich-app immich. Discuss code, ask questions & collaborate with the developer community.
FAQ | Immich
User
GitHub
Issues · immich-app/immich
High performance self-hosted photo and video management solution. - Issues · immich-app/immich
Immich
Immich•3mo ago
Successfully submitted, a tag has been added to inform contributors. :white_check_mark:
rtwo_
rtwo_OP•3mo ago
Some more info: I am running Debian 12 with a CasaOS frontend (installed from their iso) on a Raspberry Pi 4 with 4 GB of RAM. Output of df -h: 
Filesystem      Size  Used Avail Use% Mounted on
udev            1.6G     0  1.6G   0% /dev
tmpfs           380M  4.5M  375M   2% /run
/dev/mmcblk0p2  111G   65G   41G  62% /
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           5.0M   16K  5.0M   1% /run/lock
/dev/mmcblk0p1  510M   65M  446M  13% /boot/firmware
/dev/sda1       1.8T  660G  1.1T  38% /media/devmon/BACKUPCASA
tmpfs           380M     0  380M   0% /run/user/1000
Filesystem      Size  Used Avail Use% Mounted on
udev            1.6G     0  1.6G   0% /dev
tmpfs           380M  4.5M  375M   2% /run
/dev/mmcblk0p2  111G   65G   41G  62% /
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           5.0M   16K  5.0M   1% /run/lock
/dev/mmcblk0p1  510M   65M  446M  13% /boot/firmware
/dev/sda1       1.8T  660G  1.1T  38% /media/devmon/BACKUPCASA
tmpfs           380M     0  380M   0% /run/user/1000
Filesystem      Size  Used Avail Use% Mounted on
udev            1.6G     0  1.6G   0% /dev
tmpfs           380M  4.5M  375M   2% /run
/dev/mmcblk0p2  111G   65G   41G  62% /
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           5.0M   16K  5.0M   1% /run/lock
/dev/mmcblk0p1  510M   65M  446M  13% /boot/firmware
/dev/sda1       1.8T  660G  1.1T  38% /media/devmon/BACKUPCASA
tmpfs           380M     0  380M   0% /run/user/1000
Filesystem      Size  Used Avail Use% Mounted on
udev            1.6G     0  1.6G   0% /dev
tmpfs           380M  4.5M  375M   2% /run
/dev/mmcblk0p2  111G   65G   41G  62% /
tmpfs           1.9G     0  1.9G   0% /dev/shm
tmpfs           5.0M   16K  5.0M   1% /run/lock
/dev/mmcblk0p1  510M   65M  446M  13% /boot/firmware
/dev/sda1       1.8T  660G  1.1T  38% /media/devmon/BACKUPCASA
tmpfs           380M     0  380M   0% /run/user/1000
Zeus
Zeus•3mo ago
Hello, a few thoughts 1) a docker container should not be able to crash the whole server. This is an issue beyond immich/postgres 2) please run these tests: https://immich.app/docs/FAQ/#how-can-i-verify-the-integrity-of-my-database
rtwo_
rtwo_OP•3mo ago
Sorry for the wording, meant to say it's crashing the immich-server container. I'll try the tests
Zeus
Zeus•3mo ago
Oh. That’s much better
rtwo_
rtwo_OP•3mo ago
DB checksums are off apparently The file structure scan also doesn't return any errors Have you checked the logs? At times it returns a timebucket error of some sort (not in the logs, it's rare); the failing sql query also has something to do with time
Alex Tran
Alex Tran•3mo ago
Can you try run the follow command from the terminal>? 
docker exec -it immich_postgres psql --dbname=immich --username=postgres
docker exec -it immich_postgres psql --dbname=immich --username=postgres
select a.id, a."fileCreatedAt" from assets a where a."fileCreatedAt" < '1800-01-01 00:00:00';
select a.id, a."fileCreatedAt" from assets a where a."fileCreatedAt" < '1800-01-01 00:00:00';
select a.id, a."fileModifiedAt" from assets a where a."fileModifiedAt" < '1800-01-01 00:00:00';
select a.id, a."fileModifiedAt" from assets a where a."fileModifiedAt" < '1800-01-01 00:00:00';
rtwo_
rtwo_OP•3mo ago
rtw0@raspberrypi:~/immich-app $ docker exec -it immich_postgres psql --dbname=immich --username=postgres
psql (14.10 (Debian 14.10-1.pgdg120+1))
Type "help" for help.

immich=# select a.id, a."fileCreatedAt" from assets a where a."fileCreatedAt" < '1800-01-01 00:00:00';
 id | fileCreatedAt 
----+---------------
(0 rows)

immich=# select a.id, a."fileModifiedAt" from assets a where a."fileModifiedAt" < '1800-01-01 00:00:00';
 id | fileModifiedAt 
----+----------------
(0 rows)

immich=#
rtw0@raspberrypi:~/immich-app $ docker exec -it immich_postgres psql --dbname=immich --username=postgres
psql (14.10 (Debian 14.10-1.pgdg120+1))
Type "help" for help.

immich=# select a.id, a."fileCreatedAt" from assets a where a."fileCreatedAt" < '1800-01-01 00:00:00';
 id | fileCreatedAt 
----+---------------
(0 rows)

immich=# select a.id, a."fileModifiedAt" from assets a where a."fileModifiedAt" < '1800-01-01 00:00:00';
 id | fileModifiedAt 
----+----------------
(0 rows)

immich=#
Sorry for the delay. Should I just restore the database from a backup?
rtwo_
rtwo_OP•3mo ago
Some more interesting stuff, with the timebucket error. 
/usr/src/app/node_modules/postgres/cjs/src/connection.js:250
    const x = socket.write(chunk, fn)
                     ^

TypeError: Cannot read properties of null (reading 'write')
    at Immediate.nextWrite (/usr/src/app/node_modules/postgres/cjs/src/connection.js:250:22)
    at process.processImmediate (node:internal/timers:491:21)
    at process.callbackTrampoline (node:internal/async_hooks:130:17)

Node.js v22.13.1
api worker exited with code 1
/usr/src/app/node_modules/postgres/cjs/src/connection.js:250
    const x = socket.write(chunk, fn)
                     ^

TypeError: Cannot read properties of null (reading 'write')
    at Immediate.nextWrite (/usr/src/app/node_modules/postgres/cjs/src/connection.js:250:22)
    at process.processImmediate (node:internal/timers:491:21)
    at process.callbackTrampoline (node:internal/async_hooks:130:17)

Node.js v22.13.1
api worker exited with code 1

morelogs.txt

Did you find this page helpful?