Open-WebUI 404 Error
When using the Better Ollama CUDA 12 template, and following the instructions found here: blog.runpod.io/run-llama-3-1-405b-with-ollama-a-step-by-step-guide, getting an error when posting a query using open-webui: Ollama: 404, message='Not Found', url='https://<snip>-11434.proxy.runpod.net/api/chat'
Interestingly enough, replacing the open-webui localhost URL with the above URL works well with cURL using network diagnostics.
Wanted to replicate the issue on a less expensive server, but can no longer find the template....
Can someone help me fix my tensorflow installation to see the gpu?
I've been trying to fix this for over a week.
Running the official template with pytorch 2.1.0, cuda 118...
There's inconsistency in performance ( POD )
Hello. I rent and operate 20 RTX4090 GPUs all day long.
However, there are significant differences in inference speeds.
Each line in the table in the attached image represents 2 RTX 4090 GPUs.
One processes 150 images in 3 minutes. However, the rest only process 50-80 images. On my own RTX4090 2-way server that I purchased directly, the throughput is 180 images processed in 3 minutes. I haven't been able to figure out why these speed differences are occurring.
The inference task is generating one image....
