serverless - lora from network storage
Hi, I have a flux generation serverless setup that's working pretty well.
I bake all models into the docker image so even though the docker image size is pretty large, cold start is pretty reasonable and generations are fast enough.
Now the issue arises with a new workflow where I will train more lora and need to set these available to the serverless workflow.
-if I bake new loras into the docker image, it needs to rebuild and download the entire docker image which takes a long time for active workers and causes a down time until it loads
-I was thinking putting the loras on network storage (lets say there are 100 loras on network storage each is around 20mb), whenever worker is up it copies the lora requested (only 1 lora) from network storage to the serverless storage and runs it from there. would this increase execution time significantly?
basically it will run ALL models on docker baked image + single 20mb lora from network storage
is there any other alternative or best practice for doing this?
I bake all models into the docker image so even though the docker image size is pretty large, cold start is pretty reasonable and generations are fast enough.
Now the issue arises with a new workflow where I will train more lora and need to set these available to the serverless workflow.
-if I bake new loras into the docker image, it needs to rebuild and download the entire docker image which takes a long time for active workers and causes a down time until it loads
-I was thinking putting the loras on network storage (lets say there are 100 loras on network storage each is around 20mb), whenever worker is up it copies the lora requested (only 1 lora) from network storage to the serverless storage and runs it from there. would this increase execution time significantly?
basically it will run ALL models on docker baked image + single 20mb lora from network storage
is there any other alternative or best practice for doing this?
