R
Runpod15mo ago
ohno

FastAPI RunPod serverless request format

Hi everyone, Second post from me. I am (still) trying to deploy a FastAPI app in a Docker container hosted on a serverless RunPod. Here is a toy example:
import uvicorn

from fastapi import FastAPI
from pydantic import BaseModel


app = FastAPI()

class PingResponse(BaseModel):
message: str

@app.post(path="/ping", response_model=PingResponse)
async def ping() -> dict[str, str]:

response = {"message": "ok"}

return response

if __name__ == "__main__":
uvicorn.run(app=app, host="0.0.0.0", port=8080)
import uvicorn

from fastapi import FastAPI
from pydantic import BaseModel


app = FastAPI()

class PingResponse(BaseModel):
message: str

@app.post(path="/ping", response_model=PingResponse)
async def ping() -> dict[str, str]:

response = {"message": "ok"}

return response

if __name__ == "__main__":
uvicorn.run(app=app, host="0.0.0.0", port=8080)
The code is deployed as a Docker image with the port 8080 exposed. My problem is: I just can't figure out how to format my requests so that the payload hits the FastAPI path. This is the format I started with:
import requests

URL = "https://api.runpod.ai/v2/<POD_ID>/runsync"
HEADERS = {"Authorization" : "Bearer <TOKEN>"}

body = {
"input": {
"api": {
"method": "POST",
"endpoint": "/ping",
},
"payload": {}, # Empty for example, but holds parameters in real case
},
}

response = requests.post(URL, json=body, headers=HEADERS)
import requests

URL = "https://api.runpod.ai/v2/<POD_ID>/runsync"
HEADERS = {"Authorization" : "Bearer <TOKEN>"}

body = {
"input": {
"api": {
"method": "POST",
"endpoint": "/ping",
},
"payload": {}, # Empty for example, but holds parameters in real case
},
}

response = requests.post(URL, json=body, headers=HEADERS)
The request does get to the serverless worker, but then just hangs without being forwarded to the FastAPI path method. Am I missing something here? P.S.: last question from me I swear.
9 Replies
Unknown User
Unknown User15mo ago
Message Not Public
Sign In & Join Server To View
ohno
ohnoOP15mo ago
Her @nerdylive , thanks for answering so fast. No handler code: I created the serverless through the web GUI. Here is my template, with some fields obfuscated for safety
No description
Unknown User
Unknown User15mo ago
Message Not Public
Sign In & Join Server To View
ohno
ohnoOP15mo ago
Yeah I know for no Docker-in-Docker, I'm just launching the base image used to make the template just like RunPod would with any pre-built image. Stuck scratching my head at the moment - I've been following the docs to the letter, and I am having the same problem with a normal Pod rather than serverless. Deploying the same image to a Pod and running the same request gives:
{'error': 'endpoint not found'}
{'error': 'endpoint not found'}
Unknown User
Unknown User15mo ago
Message Not Public
Sign In & Join Server To View
ohno
ohnoOP15mo ago
Exactly
Unknown User
Unknown User15mo ago
Message Not Public
Sign In & Join Server To View
ohno
ohnoOP15mo ago
Tried just that - found this one, which looks very similar to my use case https://github.com/dannysemi/runpod-serverless-proxy/tree/main And I can't find anything they do that I don't. Starting to think I may be a little dense.
GitHub
GitHub - dannysemi/runpod-serverless-proxy
Contribute to dannysemi/runpod-serverless-proxy development by creating an account on GitHub.
Unknown User
Unknown User15mo ago
Message Not Public
Sign In & Join Server To View

Did you find this page helpful?