R
RunPod•4mo ago
iNF3Rnus

Are stream endpoints not working?

This is a temp endpoint just to show you all. /stream isn't available, what's up?
No description
1 Reply
iNF3Rnus
iNF3Rnus•4mo ago
Potentially nevermind, I think I understand now... You start an async job with run, then get the id from that, and retrieve the results with stream/{job_id} Based off of testing this in Postman, I don't think this supports actual streaming, or am I mistaken? I need these endpoints for LLMs for use with a FE application and I want to literally stream tokens text/event-stream style, is that supported?
Want results from more Discord servers?
Add your server
More Posts
Cuda - Out of Memory error when the 2nd GPU not utilizedI have a pod with 2 x 80 GB PCIe and I am trying to load and run Smaug-72B-v0.1 LLM. The problem is,Postman returns either 401 Unauthorized, or when the request can be sent it returns as Failed, errorPostman reads the following, when I send runsync request from runpod tutorial (from generativelabs) Backdrop Build V3 Credits missingHi team, I hope this message finds you well. I am writing to follow up on the recent offer I receivText-generation-inference on serverless endpointsHi, I don't have much experience neither with llms nor with python, so I always just use this image When on 4000 ADA, it's RANDOMLY NOT DETECTING GPU!When on 4000 ADA, it's RANDOMLY NOT DETECTING GPU! Yesterday I set it up and it's okay. Today I set Cold Start Time is too longWhen i test a HelloWorld project, run , it take too much time. Worker Configuration as attachment, I