its inner text encoder alike attention
its inner text encoder alike attention
F4x gpu results are great as well
Falmost linear speed increase
Fgonna record runpod and massed compute tutorial hopefully today
Dyes but it worked for the base model training, celebrities are women and men and no end up mixed
Dsomething is missing, may be something is wrong with the clip_l te training, or it needs higher LR
CI still get the same error even after uploading all files to the folder
Dthe other thing that can be is that is a distilled model and something is missed during the distillation and cant link new tokens to a new unique concept like a especific person without overwriting the entire class
Ftrue i thinkn key might be fine tuning
Fyou want to install swarmui?
Fwhat you want to do?
Fgood point
Fi added 2 new research results
FPatreon
Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon
CPatreon
Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon
F
Ffirst you need to install swarmui there
FYouTubeSECourses
In this video, I demonstrate how to install and use #SwarmUI on cloud services. If you lack a powerful GPU or wish to harness more GPU power, this video is essential. You'll learn how to install and utilize SwarmUI, one of the most powerful Generative AI interfaces, on Massed Compute, RunPod, and Kaggle (which offers free dual T4 GPU access for ...
F15:22 How to install latest SwarmUI on RunPod with accurate template selection
FPatreon
Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon
Fthere is installer file on patreon post
Fhopefully i am recording flux training on runpod and massed compute today
CSorry i didnt understand i needed to follow other steps first. wil ltry to get swarmui installed
Fyes first install swarmui
Fthen it will work
FHello Dr Furkan, We agree that the latest patreon video and update is not about Massed compute training at the moment?
F@Dr. Furkan Gözükara
Fi am recording it
Fand trained 4x on massed compute
A4090
MI recreated config from your files and still the same, I tested also on a 8go linix pc with A4000 and there is also OOM. Did you test with card with <= 10go on linux? I will test your script on windows same config this weekend, windows as shared vram but not Linux could that make it work on windows because at some point it uses system ram if it goes over gpu limit ?
Tbecause of FLUX i'm considering upgrading from 32 GB to 96 GB RAM
Fi watch vram usage and i tested on windows on rtx 3060
Fwould be a good choice. try to get actually 2x 64 gb ram
MI will try on windows by the way my 3080 as only 10gb ram and the 3060 have 12
T64GB DDR5 stick exist?
Fi tested and lowest config uses 7.5 gb
Fyou are right 
Fnot yet i guess
Mok thank you I will try on windows and try to understand the differences
Fsure let me know results
Frecording cloud tutorial
PHi, Dr Furkan, I am interested in the Flux controlnet, have you also conducted experiments on it too? Is there any chance for us to train or finetune the controlnet within 24GB VRAM like the LoRAs do?
Fnope i didnt. actually i never trained a controlnet. it is not an easy task
UHey guys, trying to install the whole thing on my 2nd PC. Getting an error in step 1 (installing Kohya and requirements):
"Traceback (most recent call last):
File "C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Program Files\0 - KOHYA\KohyaTraining\kohya_ss\venv\Scripts\accelerate.exe__main.py", line 4, in <module>
File "C:\Program Files\0 - KOHYA\KohyaTraining\kohya_ss\venv\lib\site-packages\accelerate__init.py", line 16, in <module>
from .accelerator import Accelerator
File "C:\Program Files\0 - KOHYA\KohyaTraining\kohya_ss\venv\lib\site-packages\accelerate\accelerator.py", line 32, in <module>
import torch
File "C:\Program Files\0 - KOHYA\KohyaTraining\kohya_ss\venv\lib\site-packages\torch__init__.py", line 148, in <module>
raise err
OSError: [WinError 126] No se puede encontrar el módulo especificado. Error loading "C:\Program Files\0 - KOHYA\KohyaTraining\kohya_ss\venv\lib\site-packages\torch\lib\fbgemm.dll" or one of its dependencies.
Kohya_ss setup menu:..."
What could be going wrong?
"Traceback (most recent call last):
File "C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Usuario\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Program Files\0 - KOHYA\KohyaTraining\kohya_ss\venv\Scripts\accelerate.exe__main.py", line 4, in <module>
File "C:\Program Files\0 - KOHYA\KohyaTraining\kohya_ss\venv\lib\site-packages\accelerate__init.py", line 16, in <module>
from .accelerator import Accelerator
File "C:\Program Files\0 - KOHYA\KohyaTraining\kohya_ss\venv\lib\site-packages\accelerate\accelerator.py", line 32, in <module>
import torch
File "C:\Program Files\0 - KOHYA\KohyaTraining\kohya_ss\venv\lib\site-packages\torch__init__.py", line 148, in <module>
raise err
OSError: [WinError 126] No se puede encontrar el módulo especificado. Error loading "C:\Program Files\0 - KOHYA\KohyaTraining\kohya_ss\venv\lib\site-packages\torch\lib\fbgemm.dll" or one of its dependencies.
Kohya_ss setup menu:..."
What could be going wrong?
Tif you go to C:\Program Files\0 - KOHYA\KohyaTraining\kohya_ss\venv\lib\site-packages\torch\lib\ do you see fbgemm.dll ?
UYes, its there.
Fi replied you solution
Fthis is torch 2.4 error on win 11
UWhat solution? Haven't seen it.
Fopen this post