Head size 160 is not supported by PagedAttention

Hello, I hope everyone is doing great! I am stuck with this error: ValueError: Head size 160 is not supported by PagedAttention. Supported head sizes are: [64, 80, 96, 112, 128, 256] Does it mean I have to RETRAIN MY MODEL? Full logs are in attachment

message.txt

1 Reply

Unknown User•16mo ago

Message Not Public

Runpod Join

We're a community of enthusiasts, engineers, and enterprises, all sharing insights on AI, Machine Learning and GPUs!

20KMembers

View on Discord

Gaming

Programming

Head size 160 is not supported by PagedAttention

Did you find this page helpful?