Serverless Package Build Failed - Disk Quota Exceeded
TL;DR - Runpod deploy failed with Disk Quota Exceeded error even though volume has sufficient disk space.
Details
1. I pushed a new release from my repository to submit a new build to runpod severless.
2. When building the package, runpod failed with the following error
3. I verified that my disk have sufficient space. I also increased it but the issue persist.
4. Here is the build ID for reference ->
e5a888dd-a3a8-479e-862f-7d816cf61096
5. This was never an issue before, we did not add new packages to requirements.txt.
Qusetion: What is the issue here? This is during the build phase, when runpod detects a new release from my repo.16 Replies
do you embed models to your docker image?
No I don't embed model in docker image. My model is stored in Volume and loaded in directly.
do you know the size of the resulting docker image?
like about 10 gigs
How can I check that? Do I have to build it locally?
yeah you have to build it if you wanna check
disk quota exceeded: unknown - Runpod
I've been routinely pushing updates to my serverless, and everything has been working fine until today. Now, I’m encountering the following error during the image push:
```
error writing layer blob: rpc error: code = Unknown desc = close /runpod-volume/registry.runpod.net/[redacted-project-path]/ingest/[hash...]/ref: disk quota exceeded: unkn...
its on runpod's side ig
make a ticket at the support page @hangry6890
https://contact.runpod.io/hc/en-us/requests/new
I got the same out of disk space error when I build locally, and the error seems related to the base image provided by Runpod.
runpod/pytorch:2.8.0-py3.11-cuda12.8.1-cudnn-devel-ubuntu22.04 <- This image is almost 10GB large
Note: I still have tons of disk space left but still see this error.
I use that image as the base image to build on-top of, and that takes up the most space
Error Message
might be the default container size limit
@hangry6890
Escalated To Zendesk
The thread has been escalated to Zendesk!
Unknown User•4d ago
Message Not Public
Sign In & Join Server To View
I mean the local build failing
default docker desktop container size limit exists
Unknown User•4d ago
Message Not Public
Sign In & Join Server To View
TL;DR - We're still facing the same problem on Runpod deploy after image build works locally.
Details
- Updated base image to use
FROM pytorch/pytorch:2.8.0-cuda12.8-cudnn9-runtime
- Verified build works locally and it has a size of 8.24GB
- However, Runpod still have the same issue
- This is the build id -> 82c83ef1-b44c-4529-96b7-410a7e169c30
- Opened zendesk ticket #27234
Error message on Runpod
@Jason , @riverfog7Its on runpod side