Slow I/O

Hey, I am trying to download a 7GB file and run a ffmpeg process to extract an audio from that file (its a video). Locally it takes on average around 5 minutes, but when I try it on the cloud (I chose the CPU, general purpose since a GPU doesn't seem to give any advantage here) and it looks like the I/O is SUPER SLOW. Is there anything I can do to speed up the Disk I/O?
4 Replies
nerdylive
nerdylive2mo ago
Don't use network volumes
Itay Elgazar
Itay Elgazar2mo ago
What do you mean? I didn't configure any network volume - I simply use an URL (of an acceleerated S3 bucket) to extract the audio from that video It looks like that:
def video_to_audio_url(video_url, output):
# Use ffmpeg to extract audio from the video
if os.path.exists(output):
os.remove(output)
process = (
ffmpeg
.input(video_url)
.output(output, format='mp3', acodec='libmp3lame')
.overwrite_output()
.run_async(pipe_stdin=True, pipe_stdout=True, pipe_stderr=True)
)

stdout, stderr = process.communicate()

if os.path.exists(output):
print("Audio extracted successfully")
return True
else:
print(f"Failed to extract audio: {stderr.decode('utf-8')}")
return False
def video_to_audio_url(video_url, output):
# Use ffmpeg to extract audio from the video
if os.path.exists(output):
os.remove(output)
process = (
ffmpeg
.input(video_url)
.output(output, format='mp3', acodec='libmp3lame')
.overwrite_output()
.run_async(pipe_stdin=True, pipe_stdout=True, pipe_stderr=True)
)

stdout, stderr = process.communicate()

if os.path.exists(output):
print("Audio extracted successfully")
return True
else:
print(f"Failed to extract audio: {stderr.decode('utf-8')}")
return False
nerdylive
nerdylive2mo ago
Hmm I'm not sure about that then, but if you think it's an hardware issue, make a support ticket on the website then
Encyrption
Encyrption2mo ago
Have you tried in a region outside of EU-RO-1? It's the only region with CPU but from personal experience it is broken with a capital B. My app was failing to download the checkpoints files, timing out after 10 minutes. I moved my app into CA-MTL-1 and the checkpoints downloaded in under a minute without any issues. You'll have to put your work on a GPU instance but I would bet money on that fixing your problem. If you do find that fixes your problem please open a ticket. Maybe if they get enough people complaining / moving out of EU-RO-1 they will actually fix it.