Software Engineering Courses (SECourses)•17mo ago

Using the flux swarmui provided, normal flux generates an image using dev-fp8 in 20 seconds. If I ad

Using the flux swarmui provided, normal flux generates an image using dev-fp8 in 20 seconds. If I add a lora, trained on flux.dev it takes several minutes per 5 steps. Any idea what causes this or if a full dev Lora can be quantized to work faster with the fp8 model?

Thanks!

WWebsteria Using the flux swarmui provided, normal flux generates an image using dev-fp8 in...

nicearoni•8/27/24, 4:44 PM

what GPU do you have? maybe adding lora causes you to have to use shared vram?

WWebsteria Using the flux swarmui provided, normal flux generates an image using dev-fp8 in...

Furkan Gözükara SECourses•8/27/24, 5:04 PM

not it doesnt have to be

Furkan Gözükara SECourses•8/27/24, 5:04 PM

what is your gpu

WebsteriaOP•8/27/24, 5:04 PM

4090

Furkan Gözükara SECourses•8/27/24, 5:04 PM

lora may be overloading your vram

Furkan Gözükara SECourses•8/27/24, 5:04 PM

can you see how much vram it uses with lora?

WebsteriaOP•8/27/24, 5:05 PM

Yes. I am not at home yet but when I get there in about 30 mins I will check.

WebsteriaOP•8/27/24, 5:05 PM

So the version a Lora is trained on doesn’t matter?

WWebsteria So the version a Lora is trained on doesn’t matter?

Furkan Gözükara SECourses•8/27/24, 5:31 PM

you need to use fp16 as base model during training

Furkan Gözükara SECourses•8/27/24, 5:31 PM

you can use lora on fp16 or fp8 base model after trained

Mattheus Chediak•8/27/24, 5:33 PM

Hey @Dr. Furkan Gözükara can i follow your instructions to runpod and use aws instead ?

Mattheus Chediak•8/27/24, 5:33 PM

create a notebook there and just follow the rest of the instructions ?

MMattheus Chediak Hey @Dr. Furkan Gözükara can i follow your instructions to runpod and use aws in...

Furkan Gözükara SECourses•8/27/24, 5:36 PM

if your aws has python 3.10 cuda and c++ yes

Furkan Gözükara SECourses•8/27/24, 5:36 PM

and nvidia drivers

Mattheus Chediak•8/27/24, 5:39 PM

thanks

Mattheus Chediak•8/27/24, 5:52 PM

it just started crashing

Mattheus Chediak•8/27/24, 5:54 PM

maybe its the port

Mattheus Chediak•8/27/24, 5:54 PM

ill try setting the port

Maxivi•8/27/24, 6:08 PM

@Dr. Furkan Gözükara bmaltis is updated now

Maxivi•8/27/24, 6:08 PM

text encoder training works

Maxivi•8/27/24, 6:08 PM

vram is using much less now

MMaxivi @Dr. Furkan Gözükara bmaltis is updated now

Furkan Gözükara SECourses•8/27/24, 6:19 PM

nice let me check

Vs12•8/27/24, 6:20 PM

Succesfully trained a model with clip l text encoder, results are better now and so are prompting

Furkan Gözükara SECourses•8/27/24, 6:20 PM

vram usage shouldnt be changed

VVs12 Succesfully trained a model with clip l text encoder, results are better now and...

Furkan Gözükara SECourses•8/27/24, 6:20 PM

awesome

Furkan Gözükara SECourses•8/27/24, 6:20 PM

you used same LR?

Vs12•8/27/24, 6:20 PM

you can use the fp8 t5

Vs12•8/27/24, 6:20 PM

use half learning rate for text encoder

Vs12•8/27/24, 6:20 PM

or else you will cook your model

VVs12 you can use the fp8 t5

Furkan Gözükara SECourses•8/27/24, 6:20 PM

this shouldnt make difference

Furkan Gözükara SECourses•8/27/24, 6:21 PM

i think it auto cast

Furkan Gözükara SECourses•8/27/24, 6:21 PM

and we train only clip L

Vs12•8/27/24, 6:21 PM

im getting less vram than before so im not sure whats changed

Furkan Gözükara SECourses•8/27/24, 6:21 PM

let me check vram too

Furkan Gözükara SECourses•8/27/24, 6:21 PM

i will still record video waiting clip l training results would delay further

Vs12•8/27/24, 6:21 PM

it may be this

Vs12•8/27/24, 6:21 PM

In FLUX.1 LoRA training, when --fp8_base--fp8_base is specified, the FLUX.1 model file with fp8 (float8_e4m3fnfloat8_e4m3fn type) can be loaded directly. Also, in flux_minimal_inference.pyflux_minimal_inference.py, it is possible to load it by specifying fp8 (float8_e4m3fn)fp8 (float8_e4m3fn) in --flux_dtype--flux_dtype.

VVs12 In FLUX.1 LoRA training, when `--fp8_base` is specified, the FLUX.1 model file w...

Furkan Gözükara SECourses•8/27/24, 6:22 PM

ye it just let you load fp8

Furkan Gözükara SECourses•8/27/24, 6:22 PM

fp16 was still casted to fp8

Vs12•8/27/24, 6:22 PM

i used this here --fp8_base_unet

Vs12•8/27/24, 6:23 PM

and clip is trained at bf16

Vs12•8/27/24, 6:23 PM

im using sd scripts directly

Furkan Gözükara SECourses•8/27/24, 6:24 PM