Can I use the option RTX A6000 NVLink with 2gpus and run 2 trainings at once?
Can I use the option RTX A6000 NVLink with 2gpus and run 2 trainings at once?
Fyes for lora
Ffor fine tune i didnt test probably OOM
�OOM? it's 48 GB each
Fyes
Ffine tuning need 80 gb gpus
Fwhen you do multi gpu
�I'm talking two separate trainings
�on same VM
�I already did that with 2 a6000 before, so that works, but I'm unsure what the NVlink means
Fseperate works
Fnvlink used when multiple gpu does a thing together
�CHat GPT O1-preview gave this answer: "Yes, you can run the two GPUs separately, treating each as its own device. Even though your RTX A6000 GPUs are connected via NVLink—which enables high-bandwidth peer-to-peer communication and memory access between them—they still present themselves as two distinct GPU devices to the system and to frameworks like PyTorch or TensorFlow"
Fyep
�If I have 1.jpg 1.txt 2.jpg 2.txt, the captions will automatically be read from the txt files in kohya, yes?
�is it possible to convert an sdxl lora to a flux lora?
ZCan you make a LoRA from a fine-tuned model? That is, extract "ohwx man" from it to run tests? Is there any comparison in quality?
FPatreon
Get more from SECourses: FLUX, Tutorials, Guides, Resources, Training, Scripts on Patreon
�doing sdxl training on kaggle free, if your computer goes to sleep, will that stop the training? 
Fkaggle page has to stay open
Fand selected
�oh wow, has to be active tab too? insane 
Fi would leave that way :d
Ahey guys ! ANy tips on fastest way to copy my 46GB safetensor files from masscompute server to my local drive. I want to use in comfyui
Ci upload to hf or any cloud. then download them later
https://medium.com/@yushantripleseven/uploading-from-jupyter-to-huggingface-ccf352f1e049
https://medium.com/@yushantripleseven/uploading-from-jupyter-to-huggingface-ccf352f1e049
MHey Guys, Doctor… When training for a person or character (LoRA or finetune), what to do of training images where our subject is very well visible, nice resolution, nice lighting, etc… but… there's another person next to them?
1. Do not use the image for training
2. "Censor" the other person with a colored rectangle/circle using a color that is not in the general palette of the image
3. Crop the image, even if it means having a very weird form factor (e.g. very narrow and tall picture)
1. Do not use the image for training
2. "Censor" the other person with a colored rectangle/circle using a color that is not in the general palette of the image
3. Crop the image, even if it means having a very weird form factor (e.g. very narrow and tall picture)
Fhugging face
Fdont use
Fi dont use 2 person unless you are training 2 person at the same time
MThank you doc… May I also ask if you or anyone else here have ever resumed a training from Hugging Face in Kohya_SS? My training died as the disk went out of space, but I do have the saved states on HF
Fi never tried
MAnyways, I've been lucky enough to get a few credits on TensorDock and I've been trying your presets for Dreambooth finetuning on 1 and 2x H100's
MBasically, you get 1.68 s/it on 1 and 2.36 s/it on the 2x system [fixed it/s → s/it]
MCorrect me if I'm wrong but this is better-than-linear scaling, right? I mean, it takes far less than 2x 1.68s (3.36 s) to do a step
MBTW I don't know if it happens to you too, but Kohya_ss ends training with a bunch of errors even when everything went fine… Pretty scary…
Fno it didnt scale linearly
Fare the pci ex?
Fyou got like only 30% speed up
MI noticed VRAM load is disproportionately higher when running the finetune on the 2x GPU: 100% on both GPUs vs. 45% on the single system. [CPU → GPU]
MThat is a lot of overhead, possibly a bug in the script?
Fi reported this already
Fkohya doesnt know
Ffrom 28 gb to 79 gb it goes
MAh yes now I remember!
-My error remains.... https://www.youtube.com/watch?v=AY6DMBCIZ3A&t=400s ... I tried 19 times, but not easy to catch up.. sorry to my knoledge and my brain bad..... : ( and bad my english...
YouTubeSECourses
Updated for SDXL 1.0. How to install #Kohya SS GUI trainer and do #LoRA training with Stable Diffusion XL (#SDXL) this is the video you are looking for. I have shown how to install Kohya from scratch. The best parameters to do LoRA training with SDXL. How to use Kohya SDXL LoRAs with ComfyUI. How to do checkpoint comparison with SDXL LoRAs and m...
-message.txt3.44KB
Flooks like kohya install error