Machine learning gives no results
When running machine learning all photo's get processed but i get no results. The logs of the microservice shows:
[Nest] 7 - 09/24/2023, 7:06:21 AM ERROR [JobService] Unable to run job handler (recognizeFaces/recognize-faces): Error: Request for facial recognition failed with status 404: Not Found
[Nest] 7 - 09/24/2023, 7:06:21 AM ERROR [JobService] Error: Request for facial recognition failed with status 404: Not Found
at MachineLearningRepository.post (/usr/src/app/dist/infra/repositories/machine-learning.repository.js:29:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async FacialRecognitionService.handleRecognizeFaces (/usr/src/app/dist/domain/facial-recognition/facial-recognition.services.js:105:23)
at async /usr/src/app/dist/domain/job/job.service.js:107:37
at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:346:28)
at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:531:24)
[Nest] 7 - 09/24/2023, 7:06:21 AM ERROR [JobService] Object:
{
"id": "3efc13af-1720-4bbf-b972-4a56a210366d"
}
Machinelearning logs show:
INFO: 172.18.0.8:54032 - "POST /predict HTTP/1.1" 404 Not Found
INFO: 172.18.0.8:53870 - "POST /predict HTTP/1.1" 404 Not Found
INFO: 172.18.0.8:54032 - "POST /predict HTTP/1.1" 404 Not Found
INFO: 172.18.0.8:54032 - "POST /predict HTTP/1.1" 404 Not Found
INFO: 172.18.0.8:54032 - "POST /predict HTTP/1.1" 404 Not Found
Immich server logs show:
[Nest] 7 - 09/23/2023, 8:55:01 PM ERROR [ExceptionsHandler] getaddrinfo ENOTFOUND immich_postgres
Error: getaddrinfo ENOTFOUND immich_postgres
at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:108:26)
I thought i had a default installation, but somehow it's not working.
16 Replies
Make sure all your containers are properly up to date
Thx for your reply. As far as i can tell everything is up to date. How can i verify?
docker compose pull should pull all the latest imagesI'm using portainer. Is that the same as using 'update stack' with re-pull image?
I think so
I already did that, but I'll try again
Now iget this error in portainer:
failed to pull images of the stack: immich-microservices Skipped - Image is already being pulled by immich-server immich-proxy Pulling immich-web Pulling redis Pulling immich-machine-learning Pulling typesense Pulling database Pulling immich-server Pulling database Pulled redis Pulled typesense Pulled immich-proxy Pulled immich-server Pulled immich-web Pulled
failed to pull images of the stack: immich-microservices Skipped - Image is already being pulled by immich-server immich-proxy Pulling immich-web Pulling redis Pulling immich-machine-learning Pulling typesense Pulling database Pulling immich-server Pulling database Pulled redis Pulled typesense Pulled immich-proxy Pulled immich-server Pulled immich-web Pulled
Is it actually erroring out? That message doesn't indicate a real problem
It says "failure" and then gives the error message.
Strange
Does it have a way to pull the images one by one?
I increased the storage for docker and retried, now i didn't get an error message and the stack updated. Retrying recognize faces.
Yes, i tried this yesterday.
It finished recognizing faces. Still no faces detected when i open the explore tab.
What about the logs?
They look the same, but not sure which logs you want to see
Did you restart the containers after pulling the new images?
Yes, it does so automatically. I can try manually though
Restarted all and running the recognition again
No results - are there any logs you want to see?
If the logs are still giving that same error, the only reason I can think of is that the ml container is not properly up to date
I'll try and see if i can manually force an update there
I deleted the container and image en redeployed the stack. The log for ML gives:
[09/24/23 08:36:57] INFO Starting gunicorn 21.2.0
[09/24/23 08:36:57] INFO Listening at: http://0.0.0.0:3003 (10)
[09/24/23 08:36:57] INFO Using worker: uvicorn.workers.UvicornWorker
[09/24/23 08:36:57] INFO Booting worker with pid: 11
[09/24/23 08:37:07] INFO Created in-memory cache with unloading disabled.
[09/24/23 08:37:07] INFO Initialized request thread pool with 4 threads. I think i fixed it. Probably the issue was that the ML container wasn't updated properly due to storage limitations. After manually deleting and updating the container all i had to do was reset the ML settings in the administration panel. Now the job is running and the Microservice error is gone. Thx for the support!
[09/24/23 08:36:57] INFO Listening at: http://0.0.0.0:3003 (10)
[09/24/23 08:36:57] INFO Using worker: uvicorn.workers.UvicornWorker
[09/24/23 08:36:57] INFO Booting worker with pid: 11
[09/24/23 08:37:07] INFO Created in-memory cache with unloading disabled.
[09/24/23 08:37:07] INFO Initialized request thread pool with 4 threads. I think i fixed it. Probably the issue was that the ML container wasn't updated properly due to storage limitations. After manually deleting and updating the container all i had to do was reset the ML settings in the administration panel. Now the job is running and the Microservice error is gone. Thx for the support!