drained of my funds somehow. HELP??
hey guys i dont know who would be able to help me out here but i had set up a serverless endpoint with a custom template. all it does is generate a custom image when the user clicks to generate one. it runs me less than $0.20 a day, usually less. But on one particular day, I was charged me entire account funds ($24) and i truly dont know why that's happened. how could the worker be running all day? how didn't it time out? and also, im pretty sure it wasn't on my end because i have an idle timeout set to 5 minutes maximum so i truly don't know what's going on. can someone help me?
attached is the screenshot of average usage + the time i was charged everything:
it's funny because the day before, i reloaded funds (Nov 22, $25), and then the next day i was essentially drained of all my funds (Nov 23, a little more than $24).
does any one know what could possibly be going wrong?
for context: i have only 1 serverless endpoint running a custom a1111 docker image with 5 serverless workers configured on RTX A4000 and RTX A4500 machiens
37 Replies
im assuming there was a job that just kept running and was never killed? is there a work around for this?
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
@nerdylive
Endpoint: grrjm4di7nvl75
It just happened again over night
I changed my code and everything so that it was near impossible to keep a working running for long but it still happened
@chee
Escalated To Zendesk
The thread has been escalated to Zendesk!
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
Ok I’ll do that in a sec not on my laptop rn
Send the ticket too
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
@nerdylive
so it was only 5 requests it seems like and none of them were constantly running if im reading this right
somehow still went from $16 to 0
so there were only 5 requests on dec 11 and it said they were all completed but my account was still drained
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
wait a second:
there was one running until 4am
how is this even possible
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
so basically it's jsut a simple image generation. it's running a custom a1111 stable diffusion image generation
so each worker that's running should just generate an image then just turn off
trying to figure that out now
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
uhhh, i believe so? what i did to get here was that i was running the a1111 ui client on my own pc but i wanted that specific, customized version to be hosted on a server for my app and cost the least amount possible for genreations so i found out how to make it into a docker image/template and made it serverless
does that answer the question
also when should i expect a response from zendesk
also i super appreciate your help @nerdylive
just dont want the account to get drained like this when my app is live, you know?
unless im doing it wrong, i cant find any logs for yesterday
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
hmm ok ill take your word for it.
would zendesk or anyone be able to take a look at my code to help me out with it?
it shouldnt be too complicated
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
of course im using cursor to help but i dojnt know if its understanding the xontext well
ok
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
no, why would that help
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
ok, yeah ill look at the logs while its running htis time but is there a way to check the logs from yesterday that im not seeing?
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
this is all i see
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
also for more context, i have each worker set to 1 second idle timeout but the execution of each worker to run before being terminated set to 600 seconds (10 minutes)
dont know if that has to do with anything but just thought id put it in here
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
hmm, i would do that normally but it took me so long to set up that whole thing
it probably IS a code problem like you're saying
no i just see that one
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View
@nerdylive is it possible to get on a call with you and show you my code
Unknown User•11mo ago
Message Not Public
Sign In & Join Server To View