Loading models from network volume cache is taking too long.
Hello all,
I'm loading my model like following so that I can use the cache from my network volume.
model = AutoModel.from_pretrained(
os.getenv("MODEL_NAME"),
cache_dir=os.getenv("/runpod-volume/models"),
local_files_only=True,
)
And recently, loading models start taking really long time. Originally it was taking 3-4 seconds, now I'm experiencing 40 secs during the daytime. How can I resolve this issue?
I'm using US-OR-1 for my network volume
I'm loading my model like following so that I can use the cache from my network volume.
model = AutoModel.from_pretrained(
os.getenv("MODEL_NAME"),
cache_dir=os.getenv("/runpod-volume/models"),
local_files_only=True,
)
And recently, loading models start taking really long time. Originally it was taking 3-4 seconds, now I'm experiencing 40 secs during the daytime. How can I resolve this issue?
I'm using US-OR-1 for my network volume
