Immich causes Crash (Unraid Server)
Hey there,
i am not 100% sure, but like 99,9%, that Immich is causing my server to crash from time to time (but quite often).
I did Memtest already, which passed.
I let Server run a few days without immich and with load (Parity Check etc) and it was ok.
Yesterday evening i started Immich again, wife did start upload from iOS (about 500-1000 Photos) and when i checked during night, Server did crash again.
I did experience this especially when putting "Upload-Load" to immich (like uploading a big bunch of pictures via Browser etc), but also while doing nothing (actively, maybe background tasks on iOS app causing it then?!).
As log-server didnt work (also crashing), i put a monitor on the Unraid-Server and got the output (see below), which is cycling (every few seconds the screen runs again these lines).
What Infos etc you need to help? And pleeeaaase help, my wife is killing me for not having the pictures/videos available
Hardware:
Gigabyte Technology Co., Ltd. Z790 GAMING X AX , Version x.x
American Megatrends International, LLC., Version F9
BIOS dated: Thu 14 Dec 2023 12:00:00 AM CET
Intel® Core™ i9-14900K @ 3200 MHz
Memory: 64 GiB DDR5
Nvidia Driver Version: 550.135
Open Source Kernel Module: No
NVIDIA GeForce RTX 3050
Parity ST12000VN0008-2YS101_ZRT192BW - 12 TB (sdg)
Parity 2 ST12000VN0008-2YS101_ZRT19247 - 12 TB (sdd) Disk 1 ST12000VN0008-2YS101_ZRT18LQG - 12 TB (sdb)
Disk 2 ST12000VN0008-2YS101_ZRT190N5 - 12 TB (sde) Disk 3 ST8000VN004-2M2101_WSD6G4KF - 8 TB (sdc) Disk 1-3 BTRFS
Parity 2 ST12000VN0008-2YS101_ZRT19247 - 12 TB (sdd) Disk 1 ST12000VN0008-2YS101_ZRT18LQG - 12 TB (sdb)
Disk 2 ST12000VN0008-2YS101_ZRT190N5 - 12 TB (sde) Disk 3 ST8000VN004-2M2101_WSD6G4KF - 8 TB (sdc) Disk 1-3 BTRFS
22 Replies
:wave: Hey @[SAAR]Rippchen,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJAChecklist
I have...
1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: read applicable release notes.
3. :ballot_box_with_check: reviewed the FAQs for known issues.
4. :ballot_box_with_check: reviewed Github for known issues.
5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
6. :ballot_box_with_check: uploaded the relevant information (see below).
7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like
fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.GitHub
immich-app immich · Discussions
Explore the GitHub Discussions forum for immich-app immich. Discuss code, ask questions & collaborate with the developer community.
FAQ | Immich
User
GitHub
Issues · immich-app/immich
High performance self-hosted photo and video management solution. - Issues · immich-app/immich
it looks like that not all of these points apply to the Unraid-Version. Like i cant run the docker compose logs command
Maybe you guys can tell me which info you need in this case (using Unraid Server) and how to obtain. Otherwise ill need to ask Unraid-Support maybe
Successfully submitted, a tag has been added to inform contributors. :white_check_mark:
Immich isn't really capable of crashing your server by doing anything other than overloading it
It could be that prolonged load causes things to overheat or such, for example
The best suggestion I have is to turn down the concurrency of all the jobs to 1, and maybe even limit the container's resources
I solved this on my underpowered crashing server by limiting container resources to at max half the CPU
i think my server has quite some power. Apparently its not crashing 100% but the Unraid GUI is crashing as well as all containers/vms are down then
But the Display connected to the server is showing something and also "moving" (/see screenshot)
that looks like a kernel panic🤔
You have a 14th gen Intel cpu and an old bios version, I hope you didn’t damage your cpu
btw whats your unraid version?
Oof, good spot
came like this. id assume they ship it in good consition (came assembled and tested)
up2date
means for me as a noob? panic usually isnt good i think 😊
Don't you know the intel drama from last summer?
im afraid no (or i forgot) 🙈 but ill check for bios updates asap
IIRC the TLDR; is 13th and 14th generation i7 and i9 can be permanently damaged if used during heavy workloads.
and only a recent bios update can mitigate that issue. But once it has been damaged, it is too late
Is your server still working or is it just a crash?
crazy, but i remember something
Server works like a charm
But in case of crash it has a high load (UPS has somewhat about 200+ Watts and it converts into a heater; usually i have 90-120W)
crash seem to affect everything except what display shows. Means he Unraid UI, the Unraid-VMS and the Dockers are all unreachable
server itself seem to run - as the display cycles the shown content
not sure what the kernel panic is, but i hope that i can troubleshoot somehow whats going on. every day without pictures available is not that good for me regarding wife 😄
Anyone can help? 😦
Hello, this is not an immich issue. We can’t assist further
Your system seems unstable since it is kernel panic and crashing
ok guys. I am now on PhotoPrism since about 2 weeks and Immich is turned off (container stopped). No more crashes. So its Immich-Related
How can we find out what exactly is causing it or lets say to find out how to fix
Photoprism has less features than immich. I’ve tried both. So it probably uses less resources
You should also reach out to unraid. A docker container has/should have no way to crash a system
If it does, that’s a bug in docker or the host, not the container
ok
i will
thx
Did you try limiting the jobs in immich like we said?
I had the bug too as well. I even upgraded to a Ryzen 5900x and it did the same thing. What I did was limit everything down to 1 and so far so good. I really wanna get down to the issue because I even set it to only use 4 cores and it locked up my server still.