Immich•2mo ago

(another) question about missing memories.

Hi everybody. I want to thank everybody developing immich. I switched from nextcloud-photos/memories to immich and its so much faster to use. I am hosting this for me and my parents since june 2025 (not sure which version i started with) and we imported all of our photos (47k going back to 2009). My first user is kind of an admin user and all other users are created via Authentik/OIDC. Its hosted on an Unraid Server. (Not sure what i need to post for the setup if needed) My issues is that i never have seen the memories feature. I read a lot of github issues and reddit post but most of them seems to be solved by waiting. But i am waiting for over 2 months now so i guess there is something wrong. Things i have done: - checked that its enabled -> yes - checked the database with: select from system_metadata where key like 'memories-state'; -> says 0 rows - activated the verbose log -> tried to generate the job manually: there is a line memory-create but nothing happens. (see attached screenshot, also checked the database again and still comes back with 0 rows) I am not sure what else to do. Thanks in advance for tips and help. Rico

21 Replies

Immich•2mo ago

:wave: Hey @RicoB, Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:. References - Container Logs: docker compose logs docs - Container Status: docker ps -a docs - Reverse Proxy: https://immich.app/docs/administration/reverse-proxy - Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA Checklist I have... 1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time). 2. :ballot_box_with_check: read applicable release notes. 3. :ballot_box_with_check: reviewed the FAQs for known issues. 4. :ballot_box_with_check: reviewed Github for known issues. 5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy). 6. :blue_square: uploaded the relevant information (see below). 7. :ballot_box_with_check: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable (an item can be marked as "complete" by reacting with the appropriate number) Information In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider: - Your docker-compose.yml and .env files. - Logs from all the containers and their status (see above). - All the troubleshooting steps you've tried so far. - Any recent changes you've made to Immich or your system. - Details about your system (both software/OS and hardware). - Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h). - The version of the Immich server, mobile app, and other relevant pieces. - Any other information that you think might be relevant. Please paste files and logs with proper code formatting, and especially avoid blurry screenshots. Without the right information we can't work out what the problem is. Help us help you ;) If this ticket can be closed you can use the /close command, and re-open it later if needed.

Sergey Katsubo•2mo ago

Hey Rico, thanks for reaching out. From your description it looks like memory generation jobs never succeeded. First, let's check if all memory-related tables/records in the database are empty:

select * from system_metadata where key = 'memories-state';
select * from memory limit 1;
select * from memory_audit limit 1;
select * from memory_asset limit 1;

select * from system_metadata where key = 'memories-state';
select * from memory limit 1;
select * from memory_audit limit 1;
select * from memory_asset limit 1;

After checking the database I suggest to check Redis: it stores jobs (BullMQ). As your screenshot confirms, the memories generation job was requested through API. So let's see its lifecycle in Redis. Fetch last 100 events for the backgroundTask queue. (The command is quite large because it transforms Redis output into something readable)

docker exec immich_redis redis-cli --raw XREVRANGE "immich_bull:backgroundTask:events" + - COUNT 100 | \
  awk '/^[0-9]+-[0-9]+$/ { if (NR > 1) print output ; output = "ID=" $0 ; next }
  { if (!key) { key = $0 } else { output = output ", " key "=" $0 ; key = "" } }
  END { print output }'

docker exec immich_redis redis-cli --raw XREVRANGE "immich_bull:backgroundTask:events" + - COUNT 100 | \
  awk '/^[0-9]+-[0-9]+$/ { if (NR > 1) print output ; output = "ID=" $0 ; next }
  { if (!key) { key = $0 } else { output = output ", " key "=" $0 ; key = "" } }
  END { print output }'

RicoBOP•2mo ago

Hi Sergey, i checked the database: everything is emtpy:

`immich=# select * from system_metadata where key = 'memories-state';
 key | value 
-----+-------
(0 rows)
immich=# select * from memory limit 1;
 id | createdAt | updatedAt | deletedAt | ownerId | type | data | isSaved | memoryAt | seenAt | showAt | hideAt | updateId 
----+-----------+-----------+-----------+---------+------+------+---------+----------+--------+--------+--------+----------
(0 rows)
immich=# select * from memory_audit limit 1;
 id | memoryId | userId | deletedAt 
----+----------+--------+-----------
(0 rows)
immich=# select * from memory_asset limit 1;
 memoriesId | assetsId | createdAt | updatedAt | updateId 
------------+----------+-----------+-----------+----------
(0 rows)`

`immich=# select * from system_metadata where key = 'memories-state';
 key | value 
-----+-------
(0 rows)
immich=# select * from memory limit 1;
 id | createdAt | updatedAt | deletedAt | ownerId | type | data | isSaved | memoryAt | seenAt | showAt | hideAt | updateId 
----+-----------+-----------+-----------+---------+------+------+---------+----------+--------+--------+--------+----------
(0 rows)
immich=# select * from memory_audit limit 1;
 id | memoryId | userId | deletedAt 
----+----------+--------+-----------
(0 rows)
immich=# select * from memory_asset limit 1;
 memoriesId | assetsId | createdAt | updatedAt | updateId 
------------+----------+-----------+-----------+----------
(0 rows)`

The output of the redis command was very long (as expected the 100jobs) I looked for things with memory and found some:

ID=1756582908089-0, event=added, jobId=3209, name=MemoryGenerate
ID=1756582898924-1, event=waiting, jobId=3208
ID=1756582898924-0, event=added, jobId=3208, name=MemoryCleanup
ID=1756581348983-1, event=waiting, jobId=3207
ID=1756581348983-0, event=added, jobId=3207, name=VersionCheck
ID=1756577748983-1, event=waiting, jobId=3206
ID=1756577748983-0, event=added, jobId=3206, name=VersionCheck
ID=1756574748752-1, event=waiting, jobId=3205
ID=1756574748752-0, event=added, jobId=3205, name=MemoryGenerate
ID=1756572633630-1, event=waiting, jobId=3204

(the event is always waiting or added, should there be something like completed?) Stupid question: i noticed that i reused the redis container/image (bitnami/redis:latest) which i already had for some other services on my server. Is the immich-redis something special (like the postgres with that extensions).

Sergey Katsubo•2mo ago

Interesting. So the job is added but stays "waiting". As you mentioned, it should go through waiting -> active -> completed. It seems the worker (job processor) is not picking it up for some reason. Immich does not use anything special from Redis afaik. Are there other Redis containers on your host / or in the same docker network? (There was a recent issue with 2 Redis containers, so Immich was balanced by docker DNS between them, causing issues with thumbnails generation job) Then, if there is only one Redis for Immich, I suggest to purge backgroundTask records and retry memories generation. 1. Dump all existing keys from Redis, just in case, out of curiosity and for possible future investigation. Use the long multiline command from https://github.com/immich-app/immich/discussions/20548#discussioncomment-14052954 It saves output to redis.txt in the current working directory. You can search for immich_bull:backgroundTask in the output. Apart from events there should be per-job records as well. 2. Purge backgroundTask:* keys from Redis.

docker exec immich_redis sh -c 'redis-cli --scan --pattern "immich_bull:backgroundTask:*" | xargs redis-cli DEL'

docker exec immich_redis sh -c 'redis-cli --scan --pattern "immich_bull:backgroundTask:*" | xargs redis-cli DEL'

3. Re-run the memories generation job. 4. See if it helps. Check DB tables and Redis events.

Immich•2mo ago

[Discussion] Error loading images (No thumbnail generation after April) (immich-app/immich#20548)

RicoBOP•2mo ago

There is only one "big" redis container on my network but i read this is not a "good" practice. So after my post yesterday i created a immichredis-container. reconfigured immich to point to the new redis container (different name so dns is not a problem), generarted the memory task and this morning i was greeted by the memories. So this had the same effect as purging the jobs from the "big" redis container. I think i will keep the immich-redis-container. So basically my problem is solved now and we could close this. But if the "stale" jobs is worth to investigate further i am happy to help.

Sergey Katsubo•2mo ago

So basically my problem is solved now

Sounds great!

this had the same effect as purging the jobs from the "big" redis container

Yep, makes sense. I appreciate if you could save backgroundTask keys from the big Redis:

CONTAINER=big-redis-container
docker exec -i $CONTAINER bash << 'EOF' > redis.txt
KEYS="immich_bull:backgroundTask:*"
two_lines_to_kv='{ sub(/^[0-9]+\) /, "") ; if (NR % 2) { key = $0 } else { print key ": " $0 } }'
xrange_to_csv='/^[0-9]+-[0-9]+$/ { if (NR > 1) print output ; output = "id=" $0 ; next } { if (!key) { key = $0 } else { output = output ", " key "=" $0 ; key = "" } } END { print output }'
redis-cli --scan --pattern "$KEYS" | sort | while read -r key; do
  type=$(redis-cli type "$key")
  echo -e "\n==== $key ($type) ===="  
  case $type in
    "string") redis-cli get "$key" ;;
    "list")   redis-cli lrange "$key" 0 -1 ;;
    "set")    redis-cli smembers "$key" ;;
    "zset")   redis-cli zrange "$key" 0 -1 withscores | awk "$two_lines_to_kv" ;;
    "hash")   redis-cli hgetall "$key" | awk "$two_lines_to_kv" ;;
    "stream") redis-cli xrange "$key" - + | awk "$xrange_to_csv" ;;
    *)        echo "Unsupported: $type" ;;
  esac
done
EOF

CONTAINER=big-redis-container
docker exec -i $CONTAINER bash << 'EOF' > redis.txt
KEYS="immich_bull:backgroundTask:*"
two_lines_to_kv='{ sub(/^[0-9]+\) /, "") ; if (NR % 2) { key = $0 } else { print key ": " $0 } }'
xrange_to_csv='/^[0-9]+-[0-9]+$/ { if (NR > 1) print output ; output = "id=" $0 ; next } { if (!key) { key = $0 } else { output = output ", " key "=" $0 ; key = "" } } END { print output }'
redis-cli --scan --pattern "$KEYS" | sort | while read -r key; do
  type=$(redis-cli type "$key")
  echo -e "\n==== $key ($type) ===="  
  case $type in
    "string") redis-cli get "$key" ;;
    "list")   redis-cli lrange "$key" 0 -1 ;;
    "set")    redis-cli smembers "$key" ;;
    "zset")   redis-cli zrange "$key" 0 -1 withscores | awk "$two_lines_to_kv" ;;
    "hash")   redis-cli hgetall "$key" | awk "$two_lines_to_kv" ;;
    "stream") redis-cli xrange "$key" - + | awk "$xrange_to_csv" ;;
    *)        echo "Unsupported: $type" ;;
  esac
done
EOF

Wondering if it gives us any hint about the root cause. This weird rare bug is haunting me 🙂

RicoBOP•2mo ago

Got the file. Is it save to attach the file here? I looked into it and i found a few "paused" events. I am not sure if i remember this correct: when i imported all the pictures with immich-go: i had to provide a user-api-key and an admin-api-key and i read somewhere something that the background jobs get paused during the import and will be enabled again after the import. Also during my first try my PC crashed during the import. (afterwards i switch to Linux for that). Maybe this screwed something up and just did not notice....

id=1749984914705-0, event=completed, jobId=35, returnvalue=null, prev=active
id=1749984914705-1, event=drained
id=1749988506552-0, event=paused
id=1749988514561-0, event=added, jobId=36, name=version-check
id=1749988514561-1, event=waiting, jobId=36
id=1749989606160-0, event=added, jobId=37, name=user-deletion
id=1749989606160-1, event=waiting, jobId=37

id=1749984914705-0, event=completed, jobId=35, returnvalue=null, prev=active
id=1749984914705-1, event=drained
id=1749988506552-0, event=paused
id=1749988514561-0, event=added, jobId=36, name=version-check
id=1749988514561-1, event=waiting, jobId=36
id=1749989606160-0, event=added, jobId=37, name=user-deletion
id=1749989606160-1, event=waiting, jobId=37

after that paused there are only added/waiting

Sergey Katsubo•2mo ago

Thanks. If backgroundTask was paused that will explain the issue. Could you also post a few lines before id=1749984914705-0, event=completed, jobId=35, returnvalue=null, prev=active?

RicoBOP•2mo ago

I just attach the complete file.

redis.txt

Sergey Katsubo•2mo ago

Awesome So you've already explained how it happened: immich-go/PC crashed after pausing some jobs/queues. On subsequent invocations immich-go does not resume those jobs because it thinks they were paused deliberately by someone https://github.com/simulot/immich-go/blob/main/app/cmd/upload/run.go#L116

Immich•2mo ago

        UpCmd.shouldResumeJobs[name] = !job.QueueStatus.IsPaused

        UpCmd.shouldResumeJobs[name] = !job.QueueStatus.IsPaused

RicoBOP•2mo ago

Maybe it would be good to see the state somewhere?

Sergey Katsubo•2mo ago

Good point. Unlike other jobs/queues, backgroundTask is not displayed in Jobs UI 🤔

RicoBOP•4w ago

Thanks for the help again. If you need something else please let me know.

Immich•4w ago

This thread has been closed. To re-open, use the button below.

Sergey Katsubo•4w ago

jfyi Next version will have a button to resume all paused jobs 👍 https://github.com/immich-app/immich/pull/21685

Immich•4w ago

[Pull Request] feat: add button to unpause all queues (immich-app/immich#21685)

RicoBOP•4w ago

Nice. I hope it helps more people than just me 🙂

Sergey Katsubo•4w ago

I hope it helps more people

Definitely. Like these issues with dozens of comments: - Empty Trash Not Deleting the Assets in Trash #19582 - Memories not generating despite having photos from past years #19728

Immich•4w ago

[Issue] Empty Trash Not Deleting the Assets in Trash (immich-app/immich#19582) [Issue] Memories not generating despite having photos from past years (immich-app/immich#19728)

Gaming

Programming

(another) question about missing memories.

Did you find this page helpful?