Serverless Text Embedding - 400
I'm using a text embedding serverless endpoint to run an instance of "sentence-transformers/all-MiniLM-L6-v2". I keep getting a bad request 400 error. The old code I had (using openAI SDK) stopped working and I've tried to configure based on new documentation without any luck. Would greatly appreciate any help!
New
---------
runpod.api_key = os.getenv("RUNPOD_API_KEY")
cleaned_text = {"prompt": "Hello, World!"}
endpoint = runpod.Endpoint("i10xxxxxxxxvp")
run_request = endpoint.run("Your text to embed here")
Old (worked before)
-------
cleaned_text = 'This is clean text.'
Initialize OpenAI client with RunPod configuration
api_key = os.getenv("RUNPOD_API_KEY")
client = OpenAI(
api_key=api_key,
base_url= os.getenv("https://api.runpod.ai/v2/i10rxxxxxxxxxxxp/openai/v1")
)
Get embedding using OpenAI-compatible endpoint
response = client.embeddings.create(
model="sentence-transformers/all-MiniLM-L6-v2",
input=cleaned_text
)
embedding = response.data[0].embedding
7 Replies
Unknown User•6mo ago
Message Not Public
Sign In & Join Server To View
With runpod.endpoint
---------------------
Job output: {'code': 400, 'message': "Invalid input: {'delayTime': 17100, 'id': '31304f73-9a33-4c1d-865d-bad4496705b5-e1', 'input': {'prompt': 'Hello, World!'}, 'status': 'IN_PROGRESS'}", 'object': 'error', 'param': None, 'type': 'BadRequestError'}
With OpenAI SDK:
openai.AuthenticationError: Error code: 401 - {'error': {'message': 'Incorrect API key provided: rpa_1KJN**bk9q. You can find your API key at https://platform.openai.com/account/api-keys.', 'type': 'invalid_request_error', 'param': None, 'code': 'invalid_api_key'}}
Unknown User•6mo ago
Message Not Public
Sign In & Join Server To View
Tx Jason. The env var RUNPOD_API_KEY is set properly and getting read into the code. I believe the issue could be because I'm running an embedding model vs. chatbot and the input format : {'prompt':'Hello World'} isn't what the endpoint expects. I've looked at the runpod documentation to no avail. Any more thoughts?
Unknown User•6mo ago
Message Not Public
Sign In & Join Server To View
hi Jason. finally, got it working with endpoint passing this json: {"input":{"model":"...", "input": "..."}}. The OpenAI SDK method kept failing after multiple T&E, it kept giving "invalid API" even with a new Key. Thanks for your help!
Unknown User•6mo ago
Message Not Public
Sign In & Join Server To View