[S3/Rclone] Slow Transfer between EU-RO-1 Volumes
Context:
I am transferring the Llama-3.1-70B model (~150GB) between two RunPod Network Volumes (S3) located in the same Data Center (EU-RO-1).
Setup:
- Method: Using an intermediary Pod in EU-RO-1 running
- Config: Both remotes configured with
The Issue:
The transfer initiates correctly but drops to effectively zero (~2 Bytes/s) when processing the
Current Command:
I followed this article https://contact.runpod.io/hc/en-us/articles/38651285638547-Knowledge-Article-Moving-Data-Between-Data-Centers
Logs
I am transferring the Llama-3.1-70B model (~150GB) between two RunPod Network Volumes (S3) located in the same Data Center (EU-RO-1).
Setup:
- Method: Using an intermediary Pod in EU-RO-1 running
rclone- Config: Both remotes configured with
provider: Otherendpoint: https://s3api-eu-ro-1.runpod.ioregion: eu-ro-1The Issue:
The transfer initiates correctly but drops to effectively zero (~2 Bytes/s) when processing the
llm_env/Current Command:
I followed this article https://contact.runpod.io/hc/en-us/articles/38651285638547-Knowledge-Article-Moving-Data-Between-Data-Centers
Logs
Questions
Is there a better way to copy files between network volumes?Runpod Help
OverviewThis article provides a step-by-step guide on how to move data between data centers using network volumes, SSH, and the
rsync
