Can't upload images - k8s deployment
Hi, I have deployed immich in kubernetes. Some notes in my setup:
- Using longhorn as storage provisioning
- Using postgresql (with all required extensions, including vectors) deployed as a separate helm chart
- immich ingress exposed via nodePort and nginx ingress controller
- Enabled redis + machine_learning microservices as well
- immich, redis and postgresql all have persistent volumes attached
- UPLOAD_LOCATION env var set to /usr/src/app/upload
the UI seems to be working fine, except.. when I try to upload images it shows "Unable to upload file" (screenshot). All pods running, no errors in "immich-server" pod logs either, I'm confused what could be the issue.. please help. : ))
8 Replies
:wave: Hey @skull,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
2. :blue_square: read applicable release notes.
3. :blue_square: reviewed the FAQs for known issues.
4. :blue_square: reviewed Github for known issues.
5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.Could be a mis configured reverse proxy like missing upload limit. Please connect direct to :2283
@Zeus I have nodeport exposed in my kubernetes which is 30080 (http traffic) and 30443 (https traffic). So my traffic is as follows:
Pod (2283) -> service (clusterIP 2283) -> ingress nginx -> node port (30080). It's hosted in my machine at home, not cloud so i can't expose it other than nodeport
Im using http://immich.home.k8s:30080 to open it, immich.home.k8s is defined in my /etc/hosts file on the vm running the k8s
I don’t understand. Why can’t you expose it locally?
Either way the issue is 99% an issue with your reverse proxy , not immich
You can try a small file upload, 1-200KB to test
kubectl port-forward
But the fix is that you need the body size annotation on your ingress
Pretty sure the helm chart suggests this tooi will try now thanks!
I feel like I did say that 😅
yes I just did not know I can set it with annotation, I was about to try set ENV var for immich for upload limit 😁
that fixed the issue...
nginx.ingress.kubernetes.io/client-body-buffer-size: 512M
nginx.ingress.kubernetes.io/proxy-body-size: 512M
on ingress rule
thank you so much