Software Engineering Courses (SECourses)•15mo ago

he said he finetune with 3 million high quality data

wardensc2OP•10/9/24, 5:34 PM

I bet he need to have a company like facebook or amazon

wardensc2OP•10/9/24, 5:34 PM

to do that

wardensc2OP•10/9/24, 5:34 PM

= )))

wardensc2OP•10/9/24, 5:35 PM

I dont buy it at all

VVigilence Maybe it needs different config since the data is distilled.

dsienra•10/9/24, 5:35 PM

the config is the same for training, is different for inferring it uses real cfg rather than Distilled CFG Scale so is slower to generate pictures

wardensc2OP•10/9/24, 5:35 PM

base on how long to train just 30 images for a checkpoint

dsienra•10/9/24, 5:36 PM

for that reason I use the trained lora on regular flux-dev

Vigilence•10/9/24, 5:36 PM

Yeah I would like to extract the lora.

Wwardensc2 he said he finetune with 3 million high quality data

dsienra•10/9/24, 5:37 PM

that model is not de-distilled try this one https://huggingface.co/nyanko7/flux-dev-de-distill

nyanko7/flux-dev-de-distill · Hugging Face

VVigilence Yeah I would like to extract the lora.

wardensc2OP•10/9/24, 5:37 PM

training checkpoint of flux is require much better hardware than lora, so training a checkpoint then extract a lora is not an efficient way to get a lora. I rather train a lora directly

Vigilence•10/9/24, 5:37 PM

@wardensc2 I tried the 4090 config the Dr. had an it said about 4000 hours for 100 images.

Vigilence•10/9/24, 5:38 PM

96gb ram

VVigilence @wardensc2 I tried the 4090 config the Dr. had an it said about 4000 hours for 1...

wardensc2OP•10/9/24, 5:38 PM

it's true

Vigilence•10/9/24, 5:38 PM

I had to edit his lora config as well, was too slow.

wardensc2OP•10/9/24, 5:38 PM

I train an rank 128 lora fp16, I need 4 a6000 with 48gb Vram and nearly 5 hours

wardensc2OP•10/9/24, 5:38 PM

but I get the best quality lora I ever have

Vigilence•10/9/24, 5:39 PM

I can only do 64 rank since I need to work as well.

Vigilence•10/9/24, 5:39 PM

2.61 s/it

VVigilence I can only do 64 rank since I need to work as well.

wardensc2OP•10/9/24, 5:39 PM

that why we use runpod or mass compute

wardensc2OP•10/9/24, 5:40 PM

because we can get something like H100 or A100 which speed is double or quadruple of the 4090

Vigilence•10/9/24, 5:40 PM

Yeah I plan on switching but wanted to learn how to do everything before being charged $$$ for failures.

VVigilence @wardensc2 I tried the 4090 config the Dr. had an it said about 4000 hours for 1...

daroth•10/9/24, 5:40 PM

On my 4090 it took 2.5 hours to train a lora

wardensc2OP•10/9/24, 5:41 PM

when I use 4 a6000 basically I have a vga with 4x speed of a 3090 and 48gb Vram

Vigilence•10/9/24, 5:41 PM

@daroth How many epochs?

wardensc2OP•10/9/24, 5:41 PM

cost me only 4 usd for a lora

Vigilence•10/9/24, 5:41 PM

I train slow and go for around 20.

Ddaroth On my 4090 it took 2.5 hours to train a lora

wardensc2OP•10/9/24, 5:41 PM

low rank lora as well

Ddsienra that model is not de-distilled try this one https://huggingface.co/nyanko7/flux-...

wardensc2OP•10/9/24, 5:43 PM

yes I know but I need best settings from you = ))))

Hhamuz Also, has anyone here rented and used an H100? Is there really a significant spe...

wardensc2OP•10/9/24, 5:45 PM

H100 = 2xA100 in term of speed

VVigilence @daroth How many epochs?

daroth•10/9/24, 5:47 PM

dunno. limited it to 2500 steps. rank doesn't affect speed really

Wwardensc2 yes I know but I need best settings from you = ))))

dsienra•10/9/24, 5:47 PM

I will test a full train with regularization images and see the result, other than that is the same config, now I'm using higher learning rate to speed up the training and get results faster, but use the Dr. Furkan setting for best quality

dsienra•10/9/24, 5:49 PM

caption for the subjects very simple "name man" "name woman" etc

Ddsienra caption for the subjects very simple "name man" "name woman" etc

wardensc2OP•10/9/24, 5:56 PM

You dont use keyword prompt in Kohya don't you ?

wardensc2OP•10/9/24, 5:56 PM

dsienra•10/9/24, 5:58 PM

I use the name of the folder for captions and the repetitions if needed for regularization or for dataset balancing

dsienra•10/9/24, 5:59 PM

folder names "1_Name man" "1_Name woman" in that case 1 repetition

uisato•10/9/24, 6:00 PM

New LORA freely-available! - https://www.youtube.com/shorts/VZTt2e3wcvI

dsienra•10/9/24, 6:00 PM

remember Flux is case sensitive

dsienra•10/9/24, 6:01 PM

so for the Name I use capital letters

FFurkan Gözükara SECourses Click to see attachment

xlads•10/9/24, 6:06 PM

this looks amamzing

xlads•10/9/24, 6:07 PM

Vigilence•10/9/24, 6:58 PM

@Dr. Furkan Gözükara Have you tried flux dora via onetrainer? Supposedly its comparable to extracting a lora from a full finetune.

Wwardensc2 H100 = 2xA100 in term of speed

hamuz•10/9/24, 7:27 PM

thanks

daroth•10/9/24, 7:35 PM

if we're doing a full finetune (then extract) instead of regular lora training, is the ~2000-2500 steps still valid?

Alittle Snack•10/9/24, 7:42 PM

Tried searching but can't find it (I assume it's in the video tutorial but I can't watch video from work) Can any SwarmUI user tell me where I need to put the T5xxl model so SwarmUI will detect it? Trying to figure out how to use Swarm for Flux and this is the only bit I can't see anywhere

daroth•10/9/24, 7:43 PM

models\clip

Ddaroth is there a cmd arg or setting that can be used so Supir auto-loads a preset? nam...