Immich can't run with external HDD
I'm new to immich, trying to set it up on a Raspberry-Pi 5 (the 4 GB variant). I'm using docker compose for installation, following the official setup guide. I configured the $UPLOAD_DIR path to be on my 4TB external HDD (with NTFS filesystem), which always stays mounted on the pi. However, docker fails to run with this setup in the following way:
The 'immich_server' service keeps restarting with an error that it can't find the .immich file inside $UPLOAD_DIR/library. The same .immich file is also not there in the other folders in $UPLAOD_DIR (such as 'backup', 'encoded-video', 'thumbs' etc). It seems immich is unable to create these 'dot' files. I'm also unable to open the webpage (at port 2283) because the service 'immich_server' crashes before I can do anything after startup.
I've tried the following to no effect:
1) Changing the permissions in that folder - chmod 777
2) Changing mount options for the HDD - setting umask=0000
3) Using a different disk that's not NTFS - a USB drive with ext4 filesystem
There are 2 cases when it all starts working:
1) I change $UPLOAD_DIR path from the external drive to an internal microSD (somewhere on the pi's home directory)
2) I manually create a .immich file in all the folders it's supposed to be in (using a small shell script).
I even tried creating the .immich on the hard disk from within the 'immich_server' container (with a "docker exec -it" command ), and the file was created without issue! Then it can't be a permission or filesystem error. It's just that immich setup is not doing the same when it's run for the first time.
So, what in god's name could be going on?
It's a bit demotivating to find such a vexing issue before I even uploaded a single photo.
Big thanks if anyone can help!
40 Replies
:wave: Hey @Athul,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJAChecklist
I have...
1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
2. :blue_square: read applicable release notes.
3. :blue_square: reviewed the FAQs for known issues.
4. :blue_square: reviewed Github for known issues.
5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like
fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.GitHub
immich-app immich · Discussions
Explore the GitHub Discussions forum for immich-app immich. Discuss code, ask questions & collaborate with the developer community.
FAQ | Immich
User
GitHub
Issues · immich-app/immich
High performance self-hosted photo and video management solution. - Issues · immich-app/immich
Windows is fine for UPLOAD_LOCATION. If you’re changing folders without moving the old folder, you need to reset the database. You haven’t done that
Immich can tell that the files it made earlier are missing
there's no windows, I'm running a debian-based linux on a raspberry pi.
The filesystem is NTFS though.
And this is happening to the library folder, not the postgres DB. I haven't moved anything either, just a clean install and the error appears.
Steps to re-create the issue:
Mount an external drive connected to a debian-based linux system (raspberry pi 5 in this case)
1) Follow the docker compose installation guide on the immich website.
2) Set $UPLOAD_DIR as a path pointing to somewhere inside the external drive
3) Set $DB_DATA_LOCATION to a docker named volume.
4) Run the app using "docker compose up -d"
5) Get the error: file /usr/src/app/upload/library/.immich does not exist
6) The container keeps restarting now until you run "docker compose down"
Zeus gave you the answer
If you’re changing folders without moving the old folder, you need to reset the database. You haven’t done that Immich can tell that the files it made earlier are missing
wait, i didn't understand Zeus.
I'm not on windows and I didn't change any folder.
You must've changed the upload location after you started it initially
hmm.. the error was there on my first ever start-up..
also, the error is not there anymore if i change the folder again to somewhere in my home dir. It's just the external drive that's causing the issue.
"Somewhere in your home dir" means any location, or one location specifically?
like i said.. immich didn't make those files on the first run, as far as i can see.
Immich most definitely does make those files on a first run
a temp folder i made in my home_dir to test it out... initalyy i was trying on the hard disk, which kept erroring.
Which folders are next to your compose file?
in my homedir is immich-test directory.
in there, alongside docker-compose.yml and .env is a symlink 'upload_dir' -> '/mnt/mydrive/immich_library'
In .env file, UPLOAD_LOCATION=./upload_dir
Oh and also you should share your compose and env file
I want to know which files and folders are there overall. All of them, not just what you did yourself
Just a screenshot of
ls -la or similarcontents of the folder with docker-compose.yml
Your compose and .env file as well please?
my .env and docker-compose.yml files
Ok, and what have you changed exactly that made it work?
changed UPLOAD_LOCATION to ./library (inside the immich docker folder, which means it's not on the drive)
FWIW I'm not sure if symlinks actually work
it also failed when i changed thepath to another external drive
Well that should've created a
library folder, no?i tried after resolving the symlink, which still had teh same exact error.
the second external drive was ext4, so it's not related to ntfs or permissions as I initally had thought
you can see some of these in the commented-out lines in .env file
I am just trying to figure out where those files ended up in
?
oh, as in it might have been created elsewhere?
Yes that's definitely the issue
I am just confused how it could've started up then
They certainly do not
also Daniel, how do I reset the db so that I can try afresh? I haven't set up any photos anyway, so just want to wipe it clean.
docker compose down -v
If there aren't any files that's safe to do
But first get rid of the symlink
Well, before you start it up againhmm I could give it a shot.
But i tried changing the symlink to no effect, see that commented line in .env
And in general, symlinks can't be such a problem right... after all linux is designed with all this in mind from my experience.
I primarily just want to try to remove as many possible issues as possible
And there's no reason to use a symlink here in the first place
okay hmm
Symlinks categorically do not (and should not) work across a container boundary
Agreed, Symlinks will not work here
Unless you mount the target 🤣🤣
(And even then probably not)
so I tried it once again after resetting the database and it worked!
Deleted the database volume, Wiped the $UPLOAD_LOCATION folder, now it's running.
The env and compose files are attached.. basically it's working fine now (symlink is still there btw)
I guess maybe I've been moving it around and getting these errors like Zeus said, without resetting the db. But the very first time also it did error, as far as I remember.
I see, I'll look into that
I guess maybe I've been moving it around and getting these errors like Zeus said, without resetting the db. But the very first time also it did error, as far as I remember.That is the only possible explanation to me, yeah If you do that you have to move the folders and its content too
thanks for the help guys!
a few learning on symlinks:
1)
A symlink value for $UPLOAD_LOCATION does work. I'm using it now and photo uploads, albums, users etc are working fine in immich.
Btw, I'm just testing now and I'll switch to a no-symlink policy when I'm done.
It seems that the link is resolved while the container is being set up and the link target is actually getting mounted. So no problems should be there after that, since the symlink is then out of the picture.
2)
Any symlinks inside the mounted directory won't work as expected. The symlinks can be seen inside the container, but docker assumes that their targets are in the container file system, not the hosts's file system.
So, like others have hinted at, all such symlinks won't work. Plus if they did, you could unintentionally give the container access to unauthorized files through such symlnks.