Failing to upgrade to V1.131.0 on docker

Krispy · 2025-03-31T19:55:06.723Z

I'm getting a manifest error when trying to pull the image. (sanitized) Compose and env file to follow.

I

Immich•3/31/25, 7:55 PM

:wave: Hey @Krispy,

Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.

References

Container Logs:
```
docker compose logs
```
```
docker compose logs
```
docs
Container Status:
```
docker ps -a
```
```
docker ps -a
```
docs
Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA

I

Immich•3/31/25, 7:55 PM

Checklist

I have...

:blue_square: verified I'm on the latest release(note that mobile app releases may take some time).
:blue_square: read applicable release notes.
:blue_square: reviewed the FAQs for known issues.
:blue_square: reviewed Github for known issues.
:blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
:blue_square: uploaded the relevant information (see below).
:blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable

(an item can be marked as "complete" by reacting with the appropriate number)

Information

In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:

Your docker-compose.yml and .env files.
Logs from all the containers and their status (see above).
All the troubleshooting steps you've tried so far.
Any recent changes you've made to Immich or your system.
Details about your system (both software/OS and hardware).
Details about your storage (filesystems, type of disks, output of commands like
```
fdisk -l
```
```
fdisk -l
```
and
```
df -h
```
```
df -h
```
).
The version of the Immich server, mobile app, and other relevant pieces.
Any other information that you think might be relevant.

Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)

If this ticket can be closed you can use the

/close

/close

command, and re-open it later if needed.

Z

Zeus•3/31/25, 7:55 PM

Allow 30 minutes for new images to propagate and check again please

ZZeus Allow 30 minutes for new images to propagate and check again please

K

KrispyOP•3/31/25, 7:56 PM

will do. I'm sourcing from ghcr.io

A

Alex Tran•3/31/25, 7:56 PM

this is a bug, the team is looking into the fix

K

KrispyOP•3/31/25, 7:57 PM

WUD also didn't discover an updated image for the CUDA machine learning image.

AAlex Tran this is a bug, the team is looking into the fix

B

bo0tzz•3/31/25, 7:58 PM

This is a different issue actually

KKrispy WUD also didn't discover an updated image for the CUDA machine learning image.

B

bo0tzz•3/31/25, 7:58 PM

The ML build is delayed as there's some other jobs running that are holding it up

K

KrispyOP•3/31/25, 7:59 PM

gotcha, i'll hold updating for a while

B

bo0tzz•3/31/25, 7:59 PM

If you want, you can monitor https://github.com/immich-app/immich/actions/runs/14179298526/job/39721496184 to see when it gets going

B

bo0tzz•3/31/25, 8:01 PM

There it goes

T

The Bird•3/31/25, 8:01 PM

Just got unstuck

B

bo0tzz•3/31/25, 8:01 PM

Should be available now @Krispy

K

KrispyOP•3/31/25, 8:22 PM

trying now

K

KrispyOP•3/31/25, 8:42 PM

machine learning container is unhealthy. Is this me, or a bug? I can start posting logs if needed

K

KrispyOP•3/31/25, 8:46 PM

compose file

message.txt4.09KB

K

KrispyOP•3/31/25, 8:47 PM

.env file

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/TankPool/media/photo_backup
# The location where your database files are stored
DB_DATA_LOCATION=/mnt/AppPool/docker/immich/postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=v1.131.0

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=mysecurepw

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/TankPool/media/photo_backup
# The location where your database files are stored
DB_DATA_LOCATION=/mnt/AppPool/docker/immich/postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=v1.131.0

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=mysecurepw

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/TankPool/media/photo_backup
# The location where your database files are stored
DB_DATA_LOCATION=/mnt/AppPool/docker/immich/postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=v1.131.0

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=mysecurepw

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/TankPool/media/photo_backup
# The location where your database files are stored
DB_DATA_LOCATION=/mnt/AppPool/docker/immich/postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=v1.131.0

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=mysecurepw

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

K

KrispyOP•3/31/25, 8:48 PM

hwacell.ml.yml

# Configurations for hardware-accelerated machine learning

# If using Unraid or another platform that doesn't allow multiple Compose files,
# you can inline the config for a backend by copying its contents
# into the immich-machine-learning service in the docker-compose.yml file.

# See https://immich.app/docs/features/ml-hardware-acceleration for info on usage.

services:
  armnn:
    devices:
      - /dev/mali0:/dev/mali0
    volumes:
      - /lib/firmware/mali_csffw.bin:/lib/firmware/mali_csffw.bin:ro # Mali firmware for your chipset (not always required depending on the driver)
      - /usr/lib/libmali.so:/usr/lib/libmali.so:ro # Mali driver for your chipset (always required)

  cpu: {}

  cuda:
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu

  openvino:
    device_cgroup_rules:
      - 'c 189:* rmw'
    devices:
      - /dev/dri:/dev/dri
    volumes:
      - /dev/bus/usb:/dev/bus/usb

  openvino-wsl:
    devices:
      - /dev/dri:/dev/dri
      - /dev/dxg:/dev/dxg
    volumes:
      - /dev/bus/usb:/dev/bus/usb
      - /usr/lib/wsl:/usr/lib/wsl

# Configurations for hardware-accelerated machine learning

# If using Unraid or another platform that doesn't allow multiple Compose files,
# you can inline the config for a backend by copying its contents
# into the immich-machine-learning service in the docker-compose.yml file.

# See https://immich.app/docs/features/ml-hardware-acceleration for info on usage.

services:
  armnn:
    devices:
      - /dev/mali0:/dev/mali0
    volumes:
      - /lib/firmware/mali_csffw.bin:/lib/firmware/mali_csffw.bin:ro # Mali firmware for your chipset (not always required depending on the driver)
      - /usr/lib/libmali.so:/usr/lib/libmali.so:ro # Mali driver for your chipset (always required)

  cpu: {}

  cuda:
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu

  openvino:
    device_cgroup_rules:
      - 'c 189:* rmw'
    devices:
      - /dev/dri:/dev/dri
    volumes:
      - /dev/bus/usb:/dev/bus/usb

  openvino-wsl:
    devices:
      - /dev/dri:/dev/dri
      - /dev/dxg:/dev/dxg
    volumes:
      - /dev/bus/usb:/dev/bus/usb
      - /usr/lib/wsl:/usr/lib/wsl

# Configurations for hardware-accelerated machine learning

# If using Unraid or another platform that doesn't allow multiple Compose files,
# you can inline the config for a backend by copying its contents
# into the immich-machine-learning service in the docker-compose.yml file.

# See https://immich.app/docs/features/ml-hardware-acceleration for info on usage.

services:
  armnn:
    devices:
      - /dev/mali0:/dev/mali0
    volumes:
      - /lib/firmware/mali_csffw.bin:/lib/firmware/mali_csffw.bin:ro # Mali firmware for your chipset (not always required depending on the driver)
      - /usr/lib/libmali.so:/usr/lib/libmali.so:ro # Mali driver for your chipset (always required)

  cpu: {}

  cuda:
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu

  openvino:
    device_cgroup_rules:
      - 'c 189:* rmw'
    devices:
      - /dev/dri:/dev/dri
    volumes:
      - /dev/bus/usb:/dev/bus/usb

  openvino-wsl:
    devices:
      - /dev/dri:/dev/dri
      - /dev/dxg:/dev/dxg
    volumes:
      - /dev/bus/usb:/dev/bus/usb
      - /usr/lib/wsl:/usr/lib/wsl

# Configurations for hardware-accelerated machine learning

# If using Unraid or another platform that doesn't allow multiple Compose files,
# you can inline the config for a backend by copying its contents
# into the immich-machine-learning service in the docker-compose.yml file.

# See https://immich.app/docs/features/ml-hardware-acceleration for info on usage.

services:
  armnn:
    devices:
      - /dev/mali0:/dev/mali0
    volumes:
      - /lib/firmware/mali_csffw.bin:/lib/firmware/mali_csffw.bin:ro # Mali firmware for your chipset (not always required depending on the driver)
      - /usr/lib/libmali.so:/usr/lib/libmali.so:ro # Mali driver for your chipset (always required)

  cpu: {}

  cuda:
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu

  openvino:
    device_cgroup_rules:
      - 'c 189:* rmw'
    devices:
      - /dev/dri:/dev/dri
    volumes:
      - /dev/bus/usb:/dev/bus/usb

  openvino-wsl:
    devices:
      - /dev/dri:/dev/dri
      - /dev/dxg:/dev/dxg
    volumes:
      - /dev/bus/usb:/dev/bus/usb
      - /usr/lib/wsl:/usr/lib/wsl

K

KrispyOP•3/31/25, 8:54 PM

docker ps output:

K

KrispyOP•3/31/25, 8:55 PM

message.txt21.78KB

K

KrispyOP•3/31/25, 8:57 PM

logs

message.txt80.5KB

K

KrispyOP•3/31/25, 8:58 PM

Or logs seperately instead of via the compose logs command...
machine learning:

root@truenas[/mnt/AppPool/docker/stacks/immich]# docker logs immich_machine_learning
[03/31/25 20:22:26] INFO     Starting gunicorn 23.0.0
[03/31/25 20:22:26] INFO     Listening at: http://[::]:3003 (8)
[03/31/25 20:22:26] INFO     Using worker: immich_ml.config.CustomUvicornWorker
[03/31/25 20:22:26] INFO     Booting worker with pid: 9
[03/31/25 20:22:28] INFO     Started server process [9]
[03/31/25 20:22:28] INFO     Waiting for application startup.
[03/31/25 20:22:28] INFO     Created in-memory cache with unloading after 300s
                             of inactivity.
[03/31/25 20:22:28] INFO     Initialized request thread pool with 8 threads.
[03/31/25 20:22:28] INFO     Application startup complete.
[03/31/25 20:40:15] INFO     Starting gunicorn 23.0.0
[03/31/25 20:40:15] INFO     Listening at: http://[::]:3003 (8)
[03/31/25 20:40:15] INFO     Using worker: immich_ml.config.CustomUvicornWorker
[03/31/25 20:40:15] INFO     Booting worker with pid: 9
[03/31/25 20:40:17] INFO     Started server process [9]
[03/31/25 20:40:17] INFO     Waiting for application startup.
[03/31/25 20:40:17] INFO     Created in-memory cache with unloading after 300s
                             of inactivity.
[03/31/25 20:40:17] INFO     Initialized request thread pool with 8 threads.
[03/31/25 20:40:17] INFO     Application startup complete.

root@truenas[/mnt/AppPool/docker/stacks/immich]# docker logs immich_machine_learning
[03/31/25 20:22:26] INFO     Starting gunicorn 23.0.0
[03/31/25 20:22:26] INFO     Listening at: http://[::]:3003 (8)
[03/31/25 20:22:26] INFO     Using worker: immich_ml.config.CustomUvicornWorker
[03/31/25 20:22:26] INFO     Booting worker with pid: 9
[03/31/25 20:22:28] INFO     Started server process [9]
[03/31/25 20:22:28] INFO     Waiting for application startup.
[03/31/25 20:22:28] INFO     Created in-memory cache with unloading after 300s
                             of inactivity.
[03/31/25 20:22:28] INFO     Initialized request thread pool with 8 threads.
[03/31/25 20:22:28] INFO     Application startup complete.
[03/31/25 20:40:15] INFO     Starting gunicorn 23.0.0
[03/31/25 20:40:15] INFO     Listening at: http://[::]:3003 (8)
[03/31/25 20:40:15] INFO     Using worker: immich_ml.config.CustomUvicornWorker
[03/31/25 20:40:15] INFO     Booting worker with pid: 9
[03/31/25 20:40:17] INFO     Started server process [9]
[03/31/25 20:40:17] INFO     Waiting for application startup.
[03/31/25 20:40:17] INFO     Created in-memory cache with unloading after 300s
                             of inactivity.
[03/31/25 20:40:17] INFO     Initialized request thread pool with 8 threads.
[03/31/25 20:40:17] INFO     Application startup complete.

root@truenas[/mnt/AppPool/docker/stacks/immich]# docker logs immich_machine_learning
[03/31/25 20:22:26] INFO     Starting gunicorn 23.0.0
[03/31/25 20:22:26] INFO     Listening at: http://[::]:3003 (8)
[03/31/25 20:22:26] INFO     Using worker: immich_ml.config.CustomUvicornWorker
[03/31/25 20:22:26] INFO     Booting worker with pid: 9
[03/31/25 20:22:28] INFO     Started server process [9]
[03/31/25 20:22:28] INFO     Waiting for application startup.
[03/31/25 20:22:28] INFO     Created in-memory cache with unloading after 300s
                             of inactivity.
[03/31/25 20:22:28] INFO     Initialized request thread pool with 8 threads.
[03/31/25 20:22:28] INFO     Application startup complete.
[03/31/25 20:40:15] INFO     Starting gunicorn 23.0.0
[03/31/25 20:40:15] INFO     Listening at: http://[::]:3003 (8)
[03/31/25 20:40:15] INFO     Using worker: immich_ml.config.CustomUvicornWorker
[03/31/25 20:40:15] INFO     Booting worker with pid: 9
[03/31/25 20:40:17] INFO     Started server process [9]
[03/31/25 20:40:17] INFO     Waiting for application startup.
[03/31/25 20:40:17] INFO     Created in-memory cache with unloading after 300s
                             of inactivity.
[03/31/25 20:40:17] INFO     Initialized request thread pool with 8 threads.
[03/31/25 20:40:17] INFO     Application startup complete.

root@truenas[/mnt/AppPool/docker/stacks/immich]# docker logs immich_machine_learning
[03/31/25 20:22:26] INFO     Starting gunicorn 23.0.0
[03/31/25 20:22:26] INFO     Listening at: http://[::]:3003 (8)
[03/31/25 20:22:26] INFO     Using worker: immich_ml.config.CustomUvicornWorker
[03/31/25 20:22:26] INFO     Booting worker with pid: 9
[03/31/25 20:22:28] INFO     Started server process [9]
[03/31/25 20:22:28] INFO     Waiting for application startup.
[03/31/25 20:22:28] INFO     Created in-memory cache with unloading after 300s
                             of inactivity.
[03/31/25 20:22:28] INFO     Initialized request thread pool with 8 threads.
[03/31/25 20:22:28] INFO     Application startup complete.
[03/31/25 20:40:15] INFO     Starting gunicorn 23.0.0
[03/31/25 20:40:15] INFO     Listening at: http://[::]:3003 (8)
[03/31/25 20:40:15] INFO     Using worker: immich_ml.config.CustomUvicornWorker
[03/31/25 20:40:15] INFO     Booting worker with pid: 9
[03/31/25 20:40:17] INFO     Started server process [9]
[03/31/25 20:40:17] INFO     Waiting for application startup.
[03/31/25 20:40:17] INFO     Created in-memory cache with unloading after 300s
                             of inactivity.
[03/31/25 20:40:17] INFO     Initialized request thread pool with 8 threads.
[03/31/25 20:40:17] INFO     Application startup complete.

KKrispy machine learning container is unhealthy. Is this me, or a bug? I can start posti...

K

KnightElm•3/31/25, 9:12 PM

ML container is unhealthy for me too

K

KnightElm•3/31/25, 9:13 PM

Basically the same log

K

KrispyOP•3/31/25, 9:13 PM

I see a .1 just got published

K

KnightElm•3/31/25, 9:14 PM

I updated to 1.131.1 and it still has the issue for me

K

KnightElm•3/31/25, 9:15 PM

Honestly it's weird that it's showing unhealthy

K

KnightElm•3/31/25, 9:15 PM

Logs look okay

M

Mraedis•3/31/25, 9:18 PM

Issue has been found

M

Mraedis•3/31/25, 9:19 PM

It's literally the file that does the healthcheck inside the container that's missing

K

KnightElm•3/31/25, 9:19 PM

Ah gotcha! Thank you for your hard work!

MMraedis It's literally the file that does the healthcheck inside the container that's mi...

K

KrispyOP•3/31/25, 9:22 PM

So, .2 on the way?

M

Mraedis•3/31/25, 9:23 PM

Probably not that fast considering it's not breaking anything other than being annoying

M

Mraedis•3/31/25, 9:23 PM

But I imagine the wait won't be too long, no promises though

I'm not a dev

MMraedis Probably not that fast considering it's not breaking anything other than being a...

K

KnightElm•3/31/25, 9:23 PM

Totallyy fine!

Failing to upgrade to V1.131.0 on docker

References

Checklist

Information

Similar Threads

Similar Threads

Similar Threads

References

Checklist

Information