I
Immich•3d ago
blackfox1337

Issue upgrading to 1.133.0 from 1.132.2

docker-compose.yml 
#version: "3.8"

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding
      file: hwaccel.transcoding.yml
      service: quicksync # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    #command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /mnt/tank/Home/Photos:/home/photos:ro
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:2283
#      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    #extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #  file: hwaccel.ml.yml
    #  service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:51d6c56749a4243096327e3fb964a48ed92254357108449cb6e23999c37773c5
    restart: always

  database:
    container_name: immich_postgres
#    image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    image: ghcr.io/immich-app/postgres:14-vectorchord0.3.0-pgvectors0.2.0
    environment:
      POSTGRES_PASSWORD: postgres #${DB_PASSWORD}
      POSTGRES_USER: postgres #${DB_USERNAME}
      POSTGRES_DB: immich #${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
     # Uncomment the DB_STORAGE_TYPE: 'HDD' var if your database isn't stored on SSDs
     # DB_STORAGE_TYPE: 'HDD'
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:
#version: "3.8"

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding
      file: hwaccel.transcoding.yml
      service: quicksync # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    #command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /mnt/tank/Home/Photos:/home/photos:ro
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:2283
#      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    #extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #  file: hwaccel.ml.yml
    #  service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:51d6c56749a4243096327e3fb964a48ed92254357108449cb6e23999c37773c5
    restart: always

  database:
    container_name: immich_postgres
#    image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    image: ghcr.io/immich-app/postgres:14-vectorchord0.3.0-pgvectors0.2.0
    environment:
      POSTGRES_PASSWORD: postgres #${DB_PASSWORD}
      POSTGRES_USER: postgres #${DB_USERNAME}
      POSTGRES_DB: immich #${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
     # Uncomment the DB_STORAGE_TYPE: 'HDD' var if your database isn't stored on SSDs
     # DB_STORAGE_TYPE: 'HDD'
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:
12 Replies
Immich
Immich•3d ago
:wave: Hey @blackfox1337, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time). 2. :ballot_box_with_check: read applicable release notes. 3. :ballot_box_with_check: reviewed the FAQs for known issues. 4. :ballot_box_with_check: reviewed Github for known issues. 5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy). 6. :ballot_box_with_check: uploaded the relevant information (see below). 7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed. Successfully submitted, a tag has been added to inform contributors. :white_check_mark:
blackfox1337
blackfox1337OP•3d ago
server logs 
Postgres notice: {

  severity_local: 'WARNING',

  severity: 'WARNING',

  code: '57P02',

  message: 'terminating connection because of crash of another server process',

  detail: 'The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.',

  hint: 'In a moment you should be able to reconnect to the database and repeat your command.',

  file: 'postgres.c',

  line: '2916',

  routine: 'quickdie'

}

Postgres notice: {

  severity_local: 'WARNING',

  severity: 'WARNING',

  code: '57P02',

  message: 'terminating connection because of crash of another server process',

  detail: 'The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.',

  hint: 'In a moment you should be able to reconnect to the database and repeat your command.',

  file: 'postgres.c',

  line: '2916',

  routine: 'quickdie'

}

Postgres notice: {

  severity_local: 'WARNING',

  severity: 'WARNING',

  code: '57P02',

  message: 'terminating connection because of crash of another server process',

  detail: 'The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.',

  hint: 'In a moment you should be able to reconnect to the database and repeat your command.',

  file: 'postgres.c',

  line: '2916',

  routine: 'quickdie'

}

Postgres notice: {

  severity_local: 'WARNING',

  severity: 'WARNING',

  code: '57P02',

  message: 'terminating connection because of crash of another server process',

  detail: 'The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.',

  hint: 'In a moment you should be able to reconnect to the database and repeat your command.',

  file: 'postgres.c',

  line: '2916',

  routine: 'quickdie'

}

Query failed : {

  durationMs: 309189.720595,

  error: Error: write CONNECTION_CLOSED immich_postgres:5432

      at Socket.closed (/usr/src/app/node_modules/postgres/cjs/src/connection.js:440:57)

      at Socket.emit (node:events:518:28)

      at TCP.<anonymous> (node:net:351:12) {

    code: 'CONNECTION_CLOSED',

    errno: 'CONNECTION_CLOSED',

    address: [ 'immich_postgres' ],

    port: [ 5432 ]

  },

  sql: 'SELECT pg_advisory_lock($1)',

  params: [ 200 ]

}

Query failed : {

  durationMs: 309014.87378399994,

  error: Error: write CONNECTION_CLOSED immich_postgres:5432

      at Socket.closed (/usr/src/app/node_modules/postgres/cjs/src/connection.js:440:57)

      at Socket.emit (node:events:518:28)

      at TCP.<anonymous> (node:net:351:12) {

    code: 'CONNECTION_CLOSED',

    errno: 'CONNECTION_CLOSED',

    address: [ 'immich_postgres' ],

    port: [ 5432 ]

  },

  sql: 'ALTER TABLE face_search ALTER COLUMN embedding SET DATA TYPE real[]',


Postgres notice: {

  severity_local: 'WARNING',

  severity: 'WARNING',

  code: '57P02',

  message: 'terminating connection because of crash of another server process',

  detail: 'The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.',

  hint: 'In a moment you should be able to reconnect to the database and repeat your command.',

  file: 'postgres.c',

  line: '2916',

  routine: 'quickdie'

}

Postgres notice: {

  severity_local: 'WARNING',

  severity: 'WARNING',

  code: '57P02',

  message: 'terminating connection because of crash of another server process',

  detail: 'The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.',

  hint: 'In a moment you should be able to reconnect to the database and repeat your command.',

  file: 'postgres.c',

  line: '2916',

  routine: 'quickdie'

}

Postgres notice: {

  severity_local: 'WARNING',

  severity: 'WARNING',

  code: '57P02',

  message: 'terminating connection because of crash of another server process',

  detail: 'The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.',

  hint: 'In a moment you should be able to reconnect to the database and repeat your command.',

  file: 'postgres.c',

  line: '2916',

  routine: 'quickdie'

}

Postgres notice: {

  severity_local: 'WARNING',

  severity: 'WARNING',

  code: '57P02',

  message: 'terminating connection because of crash of another server process',

  detail: 'The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.',

  hint: 'In a moment you should be able to reconnect to the database and repeat your command.',

  file: 'postgres.c',

  line: '2916',

  routine: 'quickdie'

}

Query failed : {

  durationMs: 309189.720595,

  error: Error: write CONNECTION_CLOSED immich_postgres:5432

      at Socket.closed (/usr/src/app/node_modules/postgres/cjs/src/connection.js:440:57)

      at Socket.emit (node:events:518:28)

      at TCP.<anonymous> (node:net:351:12) {

    code: 'CONNECTION_CLOSED',

    errno: 'CONNECTION_CLOSED',

    address: [ 'immich_postgres' ],

    port: [ 5432 ]

  },

  sql: 'SELECT pg_advisory_lock($1)',

  params: [ 200 ]

}

Query failed : {

  durationMs: 309014.87378399994,

  error: Error: write CONNECTION_CLOSED immich_postgres:5432

      at Socket.closed (/usr/src/app/node_modules/postgres/cjs/src/connection.js:440:57)

      at Socket.emit (node:events:518:28)

      at TCP.<anonymous> (node:net:351:12) {

    code: 'CONNECTION_CLOSED',

    errno: 'CONNECTION_CLOSED',

    address: [ 'immich_postgres' ],

    port: [ 5432 ]

  },

  sql: 'ALTER TABLE face_search ALTER COLUMN embedding SET DATA TYPE real[]',


blackfox1337
blackfox1337OP•3d ago
The server container restarts and the same repeats, The disk is kept busy in this time though
No description
Zeus
Zeus•3d ago
Does PG have a memory limit set? We need to see the database logs which are in the database folder
blackfox1337
blackfox1337OP•3d ago
No 
Using SSD storage

PostgreSQL Database directory appears to contain a database; Skipping initialization

2025-05-23 18:56:47.331 GMT [1] LOG:  skipping missing configuration file "/var/lib/postgresql/data/postgresql.override.conf"

2025-05-23 18:56:47.332 GMT [1] LOG:  skipping missing configuration file "/var/lib/postgresql/data/postgresql.override.conf"

2025-05-23 18:56:48.437 UTC [1] LOG:  starting PostgreSQL 14.17 (Debian 14.17-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit

2025-05-23 18:56:48.438 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432

2025-05-23 18:56:48.438 UTC [1] LOG:  listening on IPv6 address "::", port 5432

2025-05-23 18:56:48.440 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"

2025-05-23 18:56:48.446 UTC [28] LOG:  database system was interrupted; last known up at 2025-05-23 18:51:54 UTC

[2025-05-23T18:56:48Z INFO  service::utils::clean] Find directory "pg_vectors/indexes/270143".

[2025-05-23T18:56:48Z INFO  service::utils::clean] Find directory "pg_vectors/indexes/130020".

[2025-05-23T18:56:48Z INFO  service::utils::clean] Find directory "pg_vectors/indexes/270143/segments/9b1ddac7-37c4-4ebd-a837-c6309f305ade".

[2025-05-23T18:56:49Z INFO  service::utils::clean] Find directory "pg_vectors/indexes/130020/segments/686b2e4b-c83d-4047-9fc1-08072fe5628e".

2025-05-23 18:56:49.746 UTC [28] LOG:  database system was not properly shut down; automatic recovery in progress

2025-05-23 18:56:49.751 UTC [28] LOG:  redo starts at 6/DF0840C0

2025-05-23 18:56:50.247 UTC [28] LOG:  unexpected pageaddr 6/D24FC000 in log segment 0000000100000006000000E5, offset 5226496

2025-05-23 18:56:50.247 UTC [28] LOG:  redo done at 6/E54FBDB0 system usage: CPU: user: 0.15 s, system: 0.15 s, elapsed: 0.49 s

2025-05-23 18:56:52.093 UTC [1] LOG:  database system is ready to accept connections

2025-05-23 19:02:56.521 UTC [74] LOG:  could not send data to client: Broken pipe

2025-05-23 19:02:56.521 UTC [74] FATAL:  connection to client lost

2025-05-23 19:08:32.590 UTC [1] LOG:  server process (PID 97) was terminated by signal 13: Broken pipe

2025-05-23 19:08:32.590 UTC [1] LOG:  terminating any other active server processes

2025-05-23 19:08:34.340 UTC [1] LOG:  all server processes terminated; reinitializing

2025-05-23 19:08:37.454 UTC [99] LOG:  database system was interrupted; last known up at 2025-05-23 19:07:09 UTC

2025-05-23 19:10:07.523 UTC [99] LOG:  database system was not properly shut down; automatic recovery in progress

2025-05-23 19:10:07.531 UTC [99] LOG:  redo starts at 6/FEDFF340
Using SSD storage

PostgreSQL Database directory appears to contain a database; Skipping initialization

2025-05-23 18:56:47.331 GMT [1] LOG:  skipping missing configuration file "/var/lib/postgresql/data/postgresql.override.conf"

2025-05-23 18:56:47.332 GMT [1] LOG:  skipping missing configuration file "/var/lib/postgresql/data/postgresql.override.conf"

2025-05-23 18:56:48.437 UTC [1] LOG:  starting PostgreSQL 14.17 (Debian 14.17-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit

2025-05-23 18:56:48.438 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432

2025-05-23 18:56:48.438 UTC [1] LOG:  listening on IPv6 address "::", port 5432

2025-05-23 18:56:48.440 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"

2025-05-23 18:56:48.446 UTC [28] LOG:  database system was interrupted; last known up at 2025-05-23 18:51:54 UTC

[2025-05-23T18:56:48Z INFO  service::utils::clean] Find directory "pg_vectors/indexes/270143".

[2025-05-23T18:56:48Z INFO  service::utils::clean] Find directory "pg_vectors/indexes/130020".

[2025-05-23T18:56:48Z INFO  service::utils::clean] Find directory "pg_vectors/indexes/270143/segments/9b1ddac7-37c4-4ebd-a837-c6309f305ade".

[2025-05-23T18:56:49Z INFO  service::utils::clean] Find directory "pg_vectors/indexes/130020/segments/686b2e4b-c83d-4047-9fc1-08072fe5628e".

2025-05-23 18:56:49.746 UTC [28] LOG:  database system was not properly shut down; automatic recovery in progress

2025-05-23 18:56:49.751 UTC [28] LOG:  redo starts at 6/DF0840C0

2025-05-23 18:56:50.247 UTC [28] LOG:  unexpected pageaddr 6/D24FC000 in log segment 0000000100000006000000E5, offset 5226496

2025-05-23 18:56:50.247 UTC [28] LOG:  redo done at 6/E54FBDB0 system usage: CPU: user: 0.15 s, system: 0.15 s, elapsed: 0.49 s

2025-05-23 18:56:52.093 UTC [1] LOG:  database system is ready to accept connections

2025-05-23 19:02:56.521 UTC [74] LOG:  could not send data to client: Broken pipe

2025-05-23 19:02:56.521 UTC [74] FATAL:  connection to client lost

2025-05-23 19:08:32.590 UTC [1] LOG:  server process (PID 97) was terminated by signal 13: Broken pipe

2025-05-23 19:08:32.590 UTC [1] LOG:  terminating any other active server processes

2025-05-23 19:08:34.340 UTC [1] LOG:  all server processes terminated; reinitializing

2025-05-23 19:08:37.454 UTC [99] LOG:  database system was interrupted; last known up at 2025-05-23 19:07:09 UTC

2025-05-23 19:10:07.523 UTC [99] LOG:  database system was not properly shut down; automatic recovery in progress

2025-05-23 19:10:07.531 UTC [99] LOG:  redo starts at 6/FEDFF340
blackfox1337
blackfox1337OP•3d ago
Zeus
Zeus•3d ago
Your database looks corrupted At least that’s my guess
blackfox1337
blackfox1337OP•3d ago
😢
blackfox1337
blackfox1337OP•3d ago
blackfox1337
blackfox1337OP•3d ago
latest db logs immich-db-backup-1747965600082.sql.gz Would this be enough to restore? UPDATE: I rolled back to v1.132.3, and everything works fine. No DB corruption
bo0tzz
bo0tzz•3d ago
"no corruption" is hard to say with certainty, it could be that 1.132.3 just happens to not poke the sore spots
blackfox1337
blackfox1337OP•3d ago
Aah Should I try upgrading again, or is there anything I can do to avoid corruption Took the chance and went ahead with upgrade, happy to inform that it worked without any issues. Seems like I did not bring down the stack when I tried to upgrade for the first time.

Did you find this page helpful?