Unable to run job handler (Ocr) error
I have been running OCR under v2.2.1 and now v2.2.2 over the weekend. Of the 20k+ jobs processed, there have been about 100 entries into the logs of the following:
ERROR [Microservices:{"id":"ddfccd00-7c24-48cf-8fa4-dee5bf2c17f4"}] Unable to run job handler (Ocr): Error: Machine learning request '{"ocr":{"detection":{"modelName":"PP-OCRv5_server","options":{"minScore":0.5,"maxResolution":736}},"recognition":{"modelName":"PP-OCRv5_server","options":{"minScore":0.8}}}}' failed for all URLs
immich_server | Error: Machine learning request '{"ocr":{"detection":{"modelName":"PP-OCRv5_server","options":{"minScore":0.5,"maxResolution":736}},"recognition":{"modelName":"PP-OCRv5_server","options":{"minScore":0.8}}}}' failed for all URLs
immich_server | at MachineLearningRepository.predict (/usr/src/app/server/dist/repositories/machine-learning.repository.js:117:15)
The processing never completely stopped, these would just show up in the logs from time to time. I paused the OCR job at one point with about 11k jobs to go, upgraded from v2.2.1 to v2.2.2 and then kicked off the Missing OCR jobs task. I have been watching immich_server go through all of the previously encountered errors shown above, 4 at a time (concurrency is set to 4). The waiting count would hold until the failure messages were logged and 4 more jobs were dispatched. I expected that after these previously encountered errors had been run through, the processing would continue with jobs that have not been attempted yet. That is what happened. Jobs appear to be completing.
Trying to figure out how to trace back to a corrupted image or what might be happening. Any help would be appreciated.
7 Replies
:wave: Hey @mikejaegerster,
Thanks for reaching out to us. Please carefully read this message and follow the recommended actions. This will help us be more effective in our support effort and leave more time for building Immich :immich:.
References
- Container Logs:
docker compose logs docs
- Container Status: docker ps -a docs
- Reverse Proxy: https://immich.app/docs/administration/reverse-proxy
- Code Formatting https://support.discord.com/hc/en-us/articles/210298617-Markdown-Text-101-Chat-Formatting-Bold-Italic-Underline#h_01GY0DAKGXDEHE263BCAYEGFJA
Checklist
I have...
1. :ballot_box_with_check: verified I'm on the latest release(note that mobile app releases may take some time).
2. :ballot_box_with_check: read applicable release notes.
3. :ballot_box_with_check: reviewed the FAQs for known issues.
4. :ballot_box_with_check: reviewed Github for known issues.
5. :blue_square: tried accessing Immich via local ip (without a custom reverse proxy).
6. :blue_square: uploaded the relevant information (see below).
7. :blue_square: tried an incognito window, disabled extensions, cleared mobile app cache, logged out and back in, different browsers, etc. as applicable
(an item can be marked as "complete" by reacting with the appropriate number)
Information
In order to be able to effectively help you, we need you to provide clear information to show what the problem is. The exact details needed vary per case, but here is a list of things to consider:
- Your docker-compose.yml and .env files.
- Logs from all the containers and their status (see above).
- All the troubleshooting steps you've tried so far.
- Any recent changes you've made to Immich or your system.
- Details about your system (both software/OS and hardware).
- Details about your storage (filesystems, type of disks, output of commands like fdisk -l and df -h).
- The version of the Immich server, mobile app, and other relevant pieces.
- Any other information that you think might be relevant.
Please paste files and logs with proper code formatting, and especially avoid blurry screenshots.
Without the right information we can't work out what the problem is. Help us help you ;)
If this ticket can be closed you can use the /close command, and re-open it later if needed.The asset ID is in the error:
ddfccd00-7c24-48cf-8fa4-dee5bf2c17f4Can I search them directly or go through API?
it's accessible right in the UI with /photos/ddfccd00-7c24-48cf-8fa4-dee5bf2c17f4
So https://my.immich.app/photos/ddfccd00-7c24-48cf-8fa4-dee5bf2c17f4
The images don't appear to be corrupted but are almost all, if not all, text. Should I consider mobile model first? Reduced resolution? Thanks for the quick help.
Just FYI the server model is pretty heavy, even the mobile one is already loaded
Especially if the images are high res right now
There will be a bugfix release to address this because it wasn't intended to be parsed at that res 😛
I see. Okay, I'll just let them run for now, and let it finish the rest of the images. Once future fixes get released, I'll rerun things and see how they change. Glad to know my images are good. Thanks for the help. I love this software!