frogsbody
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
Yeah will do, I'll post any findings here
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
I may sleep for a bit and get back to this, interseting problem to solve
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
Not enough money in the account lol
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
I'm going to have to refund this account lol, it won't let me start another pod
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
So I messed something up with whatever we tried
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
I can't run the default torch script here
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
My environment is messed up now
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
Interesting
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
I actually can't even ping between each machine now
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
And vLLM uses that same ray cluster?
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
Issue is that I'm not sure if that's something we can even fix under the hood with vLLM... I just don't know enough about how vLLM works
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
We use eth1 I think here
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
And never actually loads any of the weights in
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
I suspect that it's actually only loading the pytorch stuff
245 replies
RRunPod
•Created by frogsbody on 4/15/2025 in #⛅｜pods-clusters
SGLang DeepSeek-V3-0324
But then we have that weird bug where it hangs model loading at 1% lol
245 replies