Unable to run job handler (Ocr) error
I have been running OCR under v2.2.1 and now v2.2.2 over the weekend. Of the 20k+ jobs processed, there have been about 100 entries into the logs of the following:
ERROR [Microservices:{"id":"ddfccd00-7c24-48cf-8fa4-dee5bf2c17f4"}] Unable to run job handler (Ocr): Error: Machine learning request '{"ocr":{"detection":{"modelName":"PP-OCRv5_server","options":{"minScore":0.5,"maxResolution":736}},"recognition":{"modelName":"PP-OCRv5_server","options":{"minScore":0.8}}}}' failed for all URLs
immich_server | Error: Machine learning request '{"ocr":{"detection":{"modelName":"PP-OCRv5_server","options":{"minScore":0.5,"maxResolution":736}},"recognition":{"modelName":"PP-OCRv5_server","options":{"minScore":0.8}}}}' failed for all URLs
immich_server | at MachineLearningRepository.predict (/usr/src/app/server/dist/repositories/machine-learning.repository.js:117:15)
The processing never completely stopped, these would just show up in the logs from time to time. I paused the OCR job at one point with about 11k jobs to go, upgraded from v2.2.1 to v2.2.2 and then kicked off the Missing OCR jobs task. I have been watching immich_server go through all of the previously encountered errors shown above, 4 at a time (concurrency is set to 4). The waiting count would hold until the failure messages were logged and 4 more jobs were dispatched. I expected that after these previously encountered errors had been run through, the processing would continue with jobs that have not been attempted yet. That is what happened. Jobs appear to be completing.
Trying to figure out how to trace back to a corrupted image or what might be happening. Any help would be appreciated.
ERROR [Microservices:{"id":"ddfccd00-7c24-48cf-8fa4-dee5bf2c17f4"}] Unable to run job handler (Ocr): Error: Machine learning request '{"ocr":{"detection":{"modelName":"PP-OCRv5_server","options":{"minScore":0.5,"maxResolution":736}},"recognition":{"modelName":"PP-OCRv5_server","options":{"minScore":0.8}}}}' failed for all URLs
immich_server | Error: Machine learning request '{"ocr":{"detection":{"modelName":"PP-OCRv5_server","options":{"minScore":0.5,"maxResolution":736}},"recognition":{"modelName":"PP-OCRv5_server","options":{"minScore":0.8}}}}' failed for all URLs
immich_server | at MachineLearningRepository.predict (/usr/src/app/server/dist/repositories/machine-learning.repository.js:117:15)
The processing never completely stopped, these would just show up in the logs from time to time. I paused the OCR job at one point with about 11k jobs to go, upgraded from v2.2.1 to v2.2.2 and then kicked off the Missing OCR jobs task. I have been watching immich_server go through all of the previously encountered errors shown above, 4 at a time (concurrency is set to 4). The waiting count would hold until the failure messages were logged and 4 more jobs were dispatched. I expected that after these previously encountered errors had been run through, the processing would continue with jobs that have not been attempted yet. That is what happened. Jobs appear to be completing.
Trying to figure out how to trace back to a corrupted image or what might be happening. Any help would be appreciated.
