Prompt formatting is weird for my model
Tried making a serverless endpoint with my model from huggingface but its stuck with this prompt template. The correct one should be the alpaca prompt template which is like "instruction", "input", "output". How do I change it to this? It's only giving me errors when I change it in this window and try to send a request that way.
Solution:
That field doesn't change based on the container or model you're using on the endpoint. It's just a placeholder. To test it via the RunPod UI like you're trying, which allows only non-openAI paths
/run or runsync, the payload should be:
```json
{
"input": {
"messages": [...
Default stable diffusion preset doesn't work on rtx 5090 serverless
Hi! I've created SDXL from preset on runpod serverless, made one simple request using requests windows, and closed the window, and returned to runpod like two day later to see 5 dollars taken from my account, 5 gpus in workers windows, and logs showing cards trying to start few times for no reason like 15-20 hours after the only request that I have sent
Can't deploy Qwen/Qwen2.5-14B-Instruct-1M on serverless
Steps to reproduce:
1. Use Serverless vLLM quick deploy for Qwen/Qwen2.5-14B-Instruct-1M (image attached)
2. Proceed with default config.
3. Try and send a request....
