Unable to pull more than 250 assets though Immich API
Hi,
Forgive me as I'm not a programmer but know enough for how to get in trouble lol. I'm trying to use Google Gemini to create a python GUI script that analyzes images from my Immich server. It then saves the metadata from the images (portrait or landscape) to a file and is supposed to set a randomly selected image as the wallpaper in Windows 11 every x amount of minutes. The issue I'm running into is that it's only able to pull the first 250 assets and then keeps repeating that list of assets and not adding more. I'm on the latest version of Immich (v1.143.1) and the API key being used as all available permissions. And yes, I have more than 250 images available on Immich š. Below is what Google Gemini is telling the issue is. Attached is the python code I'm using/running. I don't know if this matters, but I have Immich running on a Debian 12 (bookworm) server running in docker, obviously. I'm happy to share any other information on my Immich server setup if this is not a bug in Immich to help fix this.
Thank you for providing this final, conclusive log. I can say with 100% certainty now where the problem is, and I sincerely appreciate your patience in helping diagnose this. The issue is not in the script, but is a bug in the API of your specific Immich server version. Final Diagnosis: The Evidence from the Log Let's break down the new log file. It tells us everything we need to know: 1.
Thank you for providing this final, conclusive log. I can say with 100% certainty now where the problem is, and I sincerely appreciate your patience in helping diagnose this. The issue is not in the script, but is a bug in the API of your specific Immich server version. Final Diagnosis: The Evidence from the Log Let's break down the new log file. It tells us everything we need to know: 1.
Attempting to fetch assets via GET /api/asset...
This shows our new code from the last attempt is running correctly. It's trying the simpler, more stable endpoint first.
2. Failed to use GET /api/asset: 404 Client Error: Not Found... Falling back to original method.
This is the first smoking gun. The GET /api/asset
endpoint doesn't exist on your server. This is common for slightly older versions of Immich. The script correctly detected this failure and fell back to using the original POST /api/search/metadata
method.
3. Found 100000 total assets on server.
The script, using the original (buggy) method, thinks it found 100,000 assets.
4. Total in cache: 100 ... Total in cache: 200 ... Total in cache: 248 ... Total in cache: 250
The cache grows perfectly until it hits exactly 250 entries.
5. Total in cache: 250 ... Total in cache: 250 ... Total in cache: 250
The cache size freezes, even though the Newly Cached
counter keeps going up.
When we combine this with the diagnostic logs from our previous attempt, the picture is perfectly clear:
* The script is forced to use the /api/search/metadata
endpoint.
* The pagination feature of this specific endpoint on your server version (v1.143.1) is broken.
* It correctly returns the first page of ~250 assets, but then for every subsequent page request, it returns that same first page over and over again.
* Our script correctly processes the first 250 unique assets, adds them to the cache, and then correctly identifies all subsequent (duplicate) assets as already cached, so it doesn't add them again.
Conclusion: We cannot fix this in the script. The script is already doing the right thing with the faulty data the server is giving it. The problem must be fixed on the server itself.
The Solution
The solution is to resolve the bug in your Immich server. You have two primary options:
1. Update Your Immich Server: This is the highly recommended solution. You are running version v1.143.1. API bugs like this are frequently fixed in new releases. Updating your Immich instance to the latest stable version will almost certainly resolve this pagination issue.
2. Report the Bug: If you are unable to update, you should report this issue on the official Immich GitHub Issues page. Provide them with the key information: you are on version v1.143.1 and the POST /api/search/metadata
endpoint is failing to paginate correctly with the skip
parameter.10 Replies
:wave: Hey @cjb900,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs
docs
- Container Status: docker ps -a
docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: read applicable release notes.
3. :ballot_box_with_check: reviewed the FAQs for known issues.
4. :ballot_box_with_check: reviewed Github for known issues.
5. :ballot_box_with_check: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l
and df -h
).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close
command, and re-open it later if needed.Immich pagination on the search endpoint works fine, it's probably not using it correctly
Are you able to look at the code I provided and see what may be wrong or point me to something on how to correctly use the search endpoints to try and fix this issue? I see reference to the API endpoints here but not how I would implement them in what I'm trying to do https://docs.immich.app/developer/architecture#api-endpoints
Architecture | Immich
Immich uses a traditional client-server design, with a dedicated database for data persistence. The frontend clients communicate with backend services over HTTP using REST APIs. Below is a high level diagram of the architecture.
https://api.immich.app might have what you need?
Immich API
Immich API Documentation
I only skimmed over the code because it's typical LLM gibberish that's kind of hard to read, but the pagination properties aren't called
take
and skip
, it's a simple page
as you can see on the page bo0tzz linked https://api.immich.app/endpoints/search/searchAssetsYa, i completely understand that LLM code may not be the best. But I'm not a programmer to know how to code something like this my self unless I found someone willing to help code this for me. But using both these links no mater what API endpoint I try to get the code to call it seem to have some hard coded limit to only pull 250 assets even though there are 135k images on my server. I don't know if this is some issue with how my immich server is set up or if it has anything to do with a bulk of my images being in external libraries.
The code is simply wrong, Daniel explained in which way
Right and i tried getting the LLM to understand that but nothing seems to work. If it helps this is the most recent code I've been trying to use. This is also the output it gives me as well
Enhanced settings loaded. Metadata caching enabled for improved performance.
Started enhanced wallpaper rotation with metadata caching.
Connecting and building enhanced metadata cache...
Enhanced filters => landscape_only=True, only_favorites=False, exclude_archived=True
Cache target set to: 5000 images
Fetching full asset list from server...
Trying _try_search_assets_endpoint...
Error on search page 1: 404 Client Error: Not Found for url: http://192.168.90.64:2283/api/search/assets
Trying _try_search_metadata_paginated...
DEBUG: Page 1 request: skip=0, take=10000
Page 1: 250 returned, 250 new, 0 duplicates
Search page 1: 250 new assets (total unique: 250)
Collected 250 assets, server reports 250 total - stopping
Pagination complete. Total unique assets fetched: 250
Successfully fetched 250 assets using _try_search_metadata_paginated
Found 250 total assets on server.
Phase 1: Discovering uncached images...
Discovery complete. Found 250 new images to cache. Hits: 0. Filtered: 0.
Phase 2: Caching new image metadata...
Progress: 100% | Processed: 250/250 | Successfully cached: 250 | Total in cache: 250
Processing complete. Successfully cached 250 out of 250 assets
DEBUG: Before save - in-memory cache has 250 entries
DEBUG: After save - file should contain the same number of entries
DEBUG: After reload - cache stats show 250 entries
Metadata cache saved and reloaded: 250 total images
Available for selection: 26 landscape images
DEBUG: Cache stats breakdown - Total: 250, Landscape: 26, Portrait: 224
DEBUG: Sample cache entry 1: a27b7fc2... = w=3120, h=3846, landscape=False
DEBUG: Sample cache entry 2: 22054b20... = w=3648, h=5472, landscape=False
DEBUG: Sample cache entry 3: 9c365a6c... = w=3840, h=5121, landscape=False
DEBUG: Images cached this run: 250
Random wallpaper set: 332b3edb-a026-42e4-8ab5-bfc5861328a9
Testing external library access specifically...
ā Found 1 libraries
- New External Library (ID: 8855617e-b0a1-4324-87f9-a1b10de11c27)
ā Default library: 250 returned, 250 total
ā External library: 250 returned, 250 total
ā ļø External library also limited to 250
ā External library (no type filter): 250 returned, 250 total
š EXTERNAL LIBRARY TEST COMPLETE
If external library shows high totals, the wallpaper app should be updated to use libraryId parameter.
Random wallpaper set: 3d098d38-beff-49e7-8784-061d69296e01
Random wallpaper set: d743b56c-4ab2-4c73-8e25-26b527494156
We're not gonna help you debug an LLM mess, sorry
Immich is working fine, if you need help with programming you should ask elsewhere
ok i completely understand. Thanks for the help anyways