Llama 3.1 + Serveless

I´m trying to use this tutorial:

Llama 3.1 via Ollama

Tried to use: pooyaharatian/runpod-ollama:0.0.8 and override the default start with llama3.1

but getting this error:

{
"delayTime": 16752,
"error": "model "llama3.1" not found, try pulling it first",
"executionTime": 156,
"id": "f3687a15-700f-4acf-856a-d7df024ad304-u1",
"status": "FAILED"
}

into the logs:

2024-09-02 14:52:09.063
[info]
The model you are attempting to pull requires a newer version of Ollama.

Tried to update to pooyaharatian/runpod-ollama:0.0.9 but getting some JSON decoded errors.

GGustavo Monti I´m trying to use this tutorial: https://discord.com/channels/91282980641508559...

Madiator2011 (Work)•9/3/24, 1:17 PM

you are using outdated version latest one is
pooyaharatian/runpod-ollama:0.0.9

MMadiator2011 (Work)you are using outdated version latest one is pooyaharatian/runpod-ollama:0.0.9

Gustavo MontiOP•9/3/24, 1:35 PM

Tried to update the version but getting some JsonDecoded errors using llama3.1:8b

Madiator2011 (Work)•9/3/24, 4:08 PM

what is your input?

Madiator2011 (Work)•9/3/24, 4:14 PM

@Gustavo Monti

Madiator2011 (Work)•9/3/24, 4:15 PM

Some user reported that the request should look like this:

{
 "input": {
   "method_name": "api/generate",
   "input": {
     "prompt": "why the sky is blue?"
   }
 }
}

{
 "input": {
   "method_name": "api/generate",
   "input": {
     "prompt": "why the sky is blue?"
   }
 }
}

MMadiator2011 (Work)what is your input?

Gustavo MontiOP•9/3/24, 4:29 PM

@Madiator2011 (Work) with api/generate I´m getting this error:

{
"delayTime": 8104,
"error": "model "llama3.1:8b" not found, try pulling it first",
"executionTime": 3919,
"id": "fd7e2a5e-6ec7-4b98-a6c4-ec5cd23abf6d-u1",
"status": "FAILED"
}

Madiator2011 (Work)•9/3/24, 4:34 PM

do you use network storage?

Madiator2011 (Work)•9/3/24, 4:34 PM

also what do you see in worker logs

MMadiator2011 (Work)do you use network storage?

Gustavo MontiOP•9/3/24, 5:06 PM

No. Only container disk!

MMadiator2011 (Work)also what do you see in worker logs

Gustavo MontiOP•9/3/24, 5:06 PM

Gustavo MontiOP•9/3/24, 5:10 PM

I´ve found this: https://github.com/pooyahrtn/RunpodOllama/issues/12

Maybe need to create another version 0.0.10 with an llama version updated

GitHub

unable to pull llama 3.1 on 0.09 · Issue #12 · pooyahrtn/RunpodOlla...

I've updated my container image to latest version 0.09 but when i try to use llama3.1 Runpod serverless Container image: pooyaharatian/runpod-ollama:0.0.9 Container start command: llama3.1:8b I...

Llama 3.1 + Serveless

Similar Threads

Similar Threads

Similar Threads