Software Engineering Courses (SECourses)•17mo ago

fp16 accelerate vs bf16 accelerate the difference is humongous - early testing

FFurkan Gözükara SECourses fp16 accelerate vs bf16 accelerate the difference is humongous - early testing

Sipriyani•8/19/24, 11:34 PM

what I dont get, the ones on the left are still, same trained ohwx man but nothing like you, at all,. ? which makes fp16 unusable? I remember you had an article of this, but its out of my knowlege Doc!.
(https://medium.com/@furkangozukara/what-is-the-difference-between-fp16-and-bf16-here-a-good-explanation-for-you-d75ac7ec30fa)

Medium

What is the difference between FP16 and BF16? Here a good explanati...

Updated and improved 29 July 2024

SSipriyani what I dont get, the ones on the left are still, same trained ohwx man but noth...

Furkan Gözükara SECoursesOP•8/19/24, 11:40 PM

ye according to fp16 it is supposed to work better

Furkan Gözükara SECoursesOP•8/19/24, 11:40 PM

but in sdxl it was also inferior

johnasdf.•8/19/24, 11:42 PM

Should I switch that lora config to bf16 instead of fp16? I'm getting OOM with 46GB

Jjohnasdf.Should I switch that lora config to bf16 instead of fp16? I'm getting OOM with 4...

Furkan Gözükara SECoursesOP•8/19/24, 11:43 PM

ye BF16 better

Furkan Gözükara SECoursesOP•8/19/24, 11:43 PM

i am getting results will update configs :d

Furkan Gözükara SECoursesOP•8/19/24, 11:44 PM

where do you train on massed compute?

johnasdf.•8/19/24, 11:44 PM

ok.. I dont really understand this, but adjust maximum image sized from 2048 to 1644 allow me to get it to fit, but now only using 31GB

johnasdf.•8/19/24, 11:47 PM

Right now I'm on Runpod. I'll use massed about 25% of the time, but I haven't found a way to download the models I trained in any reasonable timeframe

Jjohnasdf.Right now I'm on Runpod. I'll use massed about 25% of the time, but I haven't fo...

Furkan Gözükara SECoursesOP•8/19/24, 11:47 PM

upload hugging face :d

Furkan Gözükara SECoursesOP•8/19/24, 11:47 PM

which i am doing

Furkan Gözükara SECoursesOP•8/19/24, 11:47 PM

i am training at 1024x1024 no bucketing

Furkan Gözükara SECoursesOP•8/19/24, 11:47 PM

128 rank works fine

Furkan Gözükara SECoursesOP•8/19/24, 11:47 PM

bf16 training not fp8

johnasdf.•8/19/24, 11:48 PM

I'll give that a try... Is there anyway to install runpodctl on massed instance?

Jjohnasdf.I'll give that a try... Is there anyway to install runpodctl on massed instance?

Furkan Gözükara SECoursesOP•8/19/24, 11:48 PM

yep you can do

Furkan Gözükara SECoursesOP•8/19/24, 11:56 PM

looks like running model in FP16 precision instead of FP8 makes minimalistic difference

johnasdf.•8/20/24, 12:20 AM

I got runpodctl setup on massed computer. I'll see how my download speeds are for models. That was really my only complaint

johnasdf.•8/20/24, 12:31 AM

on massed computer, when we're on the desktop, open the terminal, git check out, activate venv, then load kohya_gui.py?

johnasdf.•8/20/24, 12:32 AM

When i did that I got the old version, but when I go back to the terminal it says im up to date on the date on the sd3.flux branch

Jjohnasdf.When i did that I got the old version, but when I go back to the terminal it say...

Furkan Gözükara SECoursesOP•8/20/24, 1:06 AM

you need to do this

Furkan Gözükara SECoursesOP•8/20/24, 1:07 AM

find kohya ss in apps

johnasdf.•8/20/24, 1:07 AM

I got it

Furkan Gözükara SECoursesOP•8/20/24, 1:07 AM

open a terminal

Furkan Gözükara SECoursesOP•8/20/24, 1:07 AM

git pull

Furkan Gözükara SECoursesOP•8/20/24, 1:07 AM

ah ok

johnasdf.•8/20/24, 1:07 AM

thank you

Furkan Gözükara SECoursesOP•8/20/24, 1:08 AM

you are welcome

johnasdf.•8/20/24, 1:09 AM

I definitely think it's better to save off steps then epochs though, on that first runpod machine It's taking 1.5 hours to get through 5 epochs... needless to say, I changed that to steps on the massed instance

Jjohnasdf.I definitely think it's better to save off steps then epochs though, on that fir...

Furkan Gözükara SECoursesOP•8/20/24, 1:27 AM

you can do either way

Furkan Gözükara SECoursesOP•8/20/24, 1:27 AM

since i made repet 1 atm i save based on epochs