R
Runpod•17mo ago
🆁🅰🅻🅻🅴

TheBloke/goliath-120b-GPTQ with RunPod Kobold AI United

Hi! I got goliath-120b-GPTQ running with 3 A40. But the text generation speed is extremely slow. What is the best option for GPU config and settings to run this model? Thank you in advance!
6 Replies
🆁🅰🅻🅻🅴
🆁🅰🅻🅻🅴OP•17mo ago
No description
🆁🅰🅻🅻🅴
🆁🅰🅻🅻🅴OP•17mo ago
No description
🆁🅰🅻🅻🅴
🆁🅰🅻🅻🅴OP•17mo ago
It only uses one GPU?
Unknown User
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View
🆁🅰🅻🅻🅴
🆁🅰🅻🅻🅴OP•17mo ago
3 x A40 Why does it only uses 1 gpu for me? Some setting i missed?
Unknown User
Unknown User•17mo ago
Message Not Public
Sign In & Join Server To View

Did you find this page helpful?