Software Engineering Courses (SECourses)•16mo ago

what kohya said If you specify fp8_base for LoRA training, the flux will be cast to fp8 from bf16,

what kohya said

If you specify fp8_base for LoRA training, the flux will be cast to fp8 from bf16, so the VRAM usage will be the same even with full (bf16) base model.**

FFurkan Gözükara SECourses fused back pass reduced VRAM usage by far

dxqb•9/15/24, 7:55 AM

kohya now has fused back pass?

dxqb•9/15/24, 10:21 AM

what did you have for Lora? I'm getting 5.7 s/it using OT, bs1 adafactor fused backpass

FluffyQuack•9/15/24, 11:30 AM

Results from trying to train a character + a puppet mascot character

20240915_1328_Oil_painting_of_a_MSFio_woman_with_orang_flux1-dev_2084106653.png

MMaxivi what kohya said If you specify fp8_base for LoRA training, the flux will be cas...

Furkan Gözükara SECourses•9/15/24, 11:36 AM

he said fp8 doesnt work

Ddxqb kohya now has fused back pass?

Furkan Gözükara SECourses•9/15/24, 11:36 AM

for fine tuning yes

Furkan Gözükara SECourses•9/15/24, 11:36 AM

works amazing

Furkan Gözükara SECourses•9/15/24, 11:44 AM

looks like my checkpoints totally cooked

Furkan Gözükara SECourses•9/15/24, 11:46 AM

Furkan Gözükara SECourses•9/15/24, 11:47 AM

i guess i need to lower LR a lot

Dazzastrous [4090]•9/15/24, 11:47 AM

getting shredded

FluffyQuack•9/15/24, 11:53 AM

Has anyone done tests regarding interactions between multiple LoRAs, or is there documentation somewhere how it works in detail (assuming it doesn't get so technical it goes over my head)

For instance, does LoRA load order matter? Is it better to train a model with two characters (like the one I did above with a character + mascot) compared to training two different LoRAs and using them both at the same time?

FFluffyQuack Has anyone done tests regarding interactions between multiple LoRAs, or is there...

Furkan Gözükara SECourses•9/15/24, 12:08 PM

it doenst work well

Furkan Gözükara SECourses•9/15/24, 12:08 PM

you have to train both in same image usually

FFluffyQuack Results from trying to train a character + a puppet mascot character

Aikage•9/15/24, 2:31 PM

Metal slug

atlas170•9/15/24, 2:35 PM

Is there any concensus on what is causing the grid lines/scan lines in some Flux LoRA's? I've read anything from: VAE issue, CUDA version issue, a bug in kohya, anti-AI watermark in training dataset. Has anyone successfully narrowed down what was causing it?

dxqb•9/15/24, 3:06 PM

I don't see anything here, but I've seen grid lines when I've tried to use a Lora at very high weights, like 2

Aatlas170 Is there any concensus on what is causing the grid lines/scan lines in some Flux...

Furkan Gözükara SECourses•9/15/24, 3:18 PM

it happens to me when i generate high res images

virginia•9/15/24, 3:29 PM

@Dr. Furkan Gözükara good afternoon, as to your liveportrait, could I put a video and image in a ratio 3:4, and get generated video also in 3:4?

druhl•9/15/24, 3:38 PM

There was so much fuss about the GGUF models/ people running out of VRAM/ Flux being too large, that I spent the entire day today with my home PC (12GB 4070 Super) & ComfyUI, trying various models. To my surprise, I found:

1) ComfyUI could load and generate with the original 23GB Flux Dev Model (reasonable speed) on my PC & FP8 T5XXL. I do not understand how people are running out of VRAM? It seems both swarm UI and comfy ui have some resource management going on inside which lets you use the dev model. Am I missing something? Since it was working, I loaded Florence2 and SAM2 as well, ended up generating + inpainting with relative ease.

2) As for the GGUF models, I actually found that they took much, much longer than the 23GB model to generate images!! I tried Q6_K and Q8_O and was able to do the above operations with them as well. The quality was good, but obviously not better than the full size dev model.

3) I found that Swarm UI is somehow doing a much better job at generating with the LoRAs+Img2Img. On the other hand, it seems SwarmUI can't work with Controlnets+LoRAs at all. The results are laughable tbh. The inpainting on ComfyUI is the best!

TitanTaliwacker•9/15/24, 5:21 PM

So I am pulling my hair out (and I am already bald). I really appreciate all the work put in to the tutorials and I have followed each step. I am now getting an error that shows "13:11:50-486153 ERROR LoRA type must be set to 'Flux1' or 'Flux1 OFT' if Flux1 checkbox is checked." Flux is auto checked as the model is in there and loaded. Please let me know what I can do to fix this.

TitanTaliwacker•9/15/24, 5:24 PM

nvm! I think I got it.

Vvirginia @Dr. Furkan Gözükara good afternoon, as to your liveportrait, could I put a vide...

Furkan Gözükara SECourses•9/15/24, 6:01 PM

yes it should work

Ddruhl There was so much fuss about the GGUF models/ people running out of VRAM/ Flux b...

Furkan Gözükara SECourses•9/15/24, 6:01 PM

1 , 2 , 3 are all correct :d

TTitanTaliwacker nvm! I think I got it.

Furkan Gözükara SECourses•9/15/24, 6:02 PM

did you solve, are you using our config auto installer?

Furkan Gözükara SECourses•9/15/24, 6:12 PM

waiting new flux trainings to be completed

Furkan Gözükara SECourses•9/15/24, 6:12 PM

Dazzastrous [4090]•9/15/24, 6:13 PM

more? lol

DDazzastrous [4090]more? lol

Furkan Gözükara SECourses•9/15/24, 6:13 PM

yep

Furkan Gözükara SECourses•9/15/24, 6:13 PM

lots of more trainings i need to do

Furkan Gözükara SECourses•9/15/24, 6:59 PM

man i cant imagine back in days how i was programming. now i am fully depended on LLMs

Furkan Gözükara SECourses•9/15/24, 7:00 PM

Dazzastrous [4090]•9/15/24, 7:15 PM

if only they had this in your day huh Doc?

Dazzastrous [4090]•9/15/24, 7:17 PM

must have changed the way institutions teach programming massively

FFurkan Gözükara SECourses did you solve, are you using our config auto installer?

TitanTaliwacker•9/15/24, 7:26 PM

I am! I am running my first Flux Lora now. Thanks for all the work you put into this!

xlads•9/15/24, 7:28 PM

Has any made a lora on an item? like a coca cola can or something?

DDazzastrous [4090]if only they had this in your day huh Doc?

Furkan Gözükara SECourses•9/15/24, 7:31 PM

yep

TTitanTaliwacker I am! I am running my first Flux Lora now. Thanks for all the work you put into ...

Furkan Gözükara SECourses•9/15/24, 7:31 PM

awesome

Xxlads Has any made a lora on an item? like a coca cola can or something?

Furkan Gözükara SECourses•9/15/24, 7:32 PM

not yet only style i tested

Furkan Gözükara SECourses•9/15/24, 7:35 PM

lol

Furkan Gözükara SECourses•9/15/24, 7:36 PM

it was 0 kb

Furkan Gözükara SECourses•9/15/24, 7:36 PM

i had to delete to take screenshot

virginia•9/15/24, 7:37 PM

@Dr. Furkan Gözükara have you tried this kaggle notebook? SUPIR_v52 i have started , but sever is too small even to draw an image

MaxiviOP•9/15/24, 7:39 PM

https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct/tree/main

Qwen/Qwen2-VL-7B-Instruct at main

Vvirginia @Dr. Furkan Gözükara have you tried this kaggle notebook? SUPIR_v52 i have start...

Furkan Gözükara SECourses•9/15/24, 7:39 PM

ye you cant upscale much there

xlads•9/15/24, 7:49 PM

Dr. can i follow the same steps as training a ohwx man from this video https://www.youtube.com/watch?v=nySGu12Y05k&t=1463s to train a prouduct?

YouTubeSECourses

FLUX LoRA Training Simplified: From Zero to Hero with Kohya SS GUI ...

Ultimate Kohya GUI FLUX LoRA training tutorial. This tutorial is product of non-stop 9 days research and training. I have trained over 73 FLUX LoRA models and analyzed all to prepare this tutorial video. The research still going on and hopefully the results will be significantly improved and latest configs and findings will be shared. Please wat...

Furkan Gözükara SECourses•9/15/24, 8:02 PM

Xxlads Dr. can i follow the same steps as training a ohwx man from this video https://w...

Furkan Gözükara SECourses•9/15/24, 8:02 PM

yep

Furkan Gözükara SECourses•9/15/24, 8:02 PM

ohwx product_class

AstraBat•9/15/24, 8:20 PM

I've happily made a successful lora with Flux. Your instructions on Patreon really helped.

AstraBat•9/15/24, 8:23 PM

I had to run one of the .bat files to avoid running out of memory on a 4090 - the file for installing Torch 2.5.

what kohya said If you specify fp8_base for LoRA training, the flux will be cast to fp8 from bf16,

Similar Threads