Software Engineering Courses (SECourses)•17mo ago

I did step 2, so my libraries are updated and i'm only using 17.5gb vram

Furkan Gözükara SECourses•8/22/24, 11:16 PM

it prevents gui updating back to older libraries

Nnicearoni I did step 2, so my libraries are updated and i'm only using 17.5gb vram

Furkan Gözükara SECourses•8/22/24, 11:16 PM

oh nice

Furkan Gözükara SECourses•8/22/24, 11:16 PM

8.8 second is slow though let me test on my pc

nicearoniOP•8/22/24, 11:20 PM

I noticed on your earlier configs that used 23.5gb that sampling had to be turned off, otherwise after the first sample, GPU mem usage went up over 30gb, so it has to use shared vram, essentially killing the training. did this happen with you as well which is why you don't have samples included in your configs? now that vram usage is down in the 17gb range, might sampling during training be possible? just curious

Nnicearoni I noticed on your earlier configs that used 23.5gb that sampling had to be turne...

Furkan Gözükara SECourses•8/22/24, 11:25 PM

you can use sampling but i dont prefer it because with my experience

Furkan Gözükara SECourses•8/22/24, 11:25 PM

sampling usually broken

Furkan Gözükara SECourses•8/22/24, 11:25 PM

hurts training

nicearoniOP•8/22/24, 11:26 PM

good to know - I will skip it. much less of an issue with how fast flux trains. thank you

Nnicearoni good to know - I will skip it. much less of an issue with how fast flux trains. ...

Furkan Gözükara SECourses•8/22/24, 11:26 PM

you are welcome

Furkan Gözükara SECourses•8/22/24, 11:26 PM

i am testing new 16 gb config right now

Furkan Gözükara SECourses•8/22/24, 11:27 PM

3.71 second it on rtx 6000 - should be similar for rtx 3090

Furkan Gözükara SECourses•8/22/24, 11:27 PM

ah my rtx 3090 also getting 7.79 second it weird

Furkan Gözükara SECourses•8/22/24, 11:27 PM

i wonder if reason is windows vs ubuntu

Furkan Gözükara SECourses•8/22/24, 11:27 PM

need to check this on runpod

FFurkan Gözükara SECourses i wonder if reason is windows vs ubuntu

nicearoniOP•8/22/24, 11:28 PM

wouldn't surprise me - gotta love\hate windows sometimes haha

FFurkan Gözükara SECourses 24 gb config uses 17gb :

nicearoniOP•8/22/24, 11:32 PM

driver version 535 is rather old yes? i'm on 560.81. probably not going to try and revert back and risk screwing things up but makes me wonder

Darth Teethius•8/22/24, 11:32 PM

What speed did you get on the 3060 10gb test?

Nnicearoni wouldn't surprise me - gotta love\hate windows sometimes haha

Furkan Gözükara SECourses•8/22/24, 11:33 PM

testing now

Nnicearoni driver version 535 is rather old yes? i'm on 560.81. probably not going to try a...

Furkan Gözükara SECourses•8/22/24, 11:33 PM

i think newest driver may be better

DDarth Teethius What speed did you get on the 3060 10gb test?

Furkan Gözükara SECourses•8/22/24, 11:34 PM

around 22 second / it

Furkan Gözükara SECourses•8/22/24, 11:34 PM

20-22 second

Furkan Gözükara SECourses•8/22/24, 11:34 PM

you can reduce to 512px it speed up like 2.5x but reduces quality slightly

Darth Teethius•8/22/24, 11:34 PM

Ok that makes sense. Im getting about 25 s/it

DDarth Teethius Ok that makes sense. Im getting about 25 s/it

Furkan Gözükara SECourses•8/22/24, 11:37 PM

ye seems about right

Furkan Gözükara SECourses•8/22/24, 11:37 PM

slow but quality is almost same as 48 gb config

Furkan Gözükara SECourses•8/22/24, 11:37 PM

i didnt notice much diff

Furkan Gözükara SECourses•8/22/24, 11:37 PM

it is mind blowing this is 12b model

nicearoniOP•8/22/24, 11:40 PM

I'm currently running Best_v1_5e_5_max_grad_norm_0 from v3 of the config scripts - loras are coming out at about 2.3gb each, does that sound right?

Nnicearoni I'm currently running Best_v1_5e_5_max_grad_norm_0 from v3 of the config scripts...

Furkan Gözükara SECourses•8/22/24, 11:40 PM

yep sounds accurate

Furkan Gözükara SECourses•8/22/24, 11:40 PM

since we are saving as float

Furkan Gözükara SECourses•8/22/24, 11:41 PM

you can save as bfloat too - reduces size by half

Furkan Gözükara SECourses•8/23/24, 12:35 AM

on runpod initial loading for flux training takes forever

Furkan Gözükara SECourses•8/23/24, 12:36 AM

definitely go with massed compute guys

Furkan Gözükara SECourses•8/23/24, 12:41 AM

dmn rtx 3090 gets 5.75 second / it on linux and i get 7.78 second it on windows rtx 3090 ti

Furkan Gözükara SECourses•8/23/24, 12:44 AM

i think i found the reason why windows slow

Furkan Gözükara SECourses•8/23/24, 12:44 AM

testing

AIGambino•8/23/24, 12:46 AM

dr did you have an autocrop thing we can use?

AAIGambino dr did you have an autocrop thing we can use?

Furkan Gözükara SECourses•8/23/24, 12:46 AM

yep here : https://www.patreon.com/posts/sota-subject-and-88391247

Patreon

SOTA Subject Cropper and Face Focused Image Resizer Scripts Do Bett...

Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon

Furkan Gözükara SECourses•8/23/24, 12:46 AM

12_Jan_2024_auto_cropper

Furkan Gözükara SECourses•8/23/24, 12:46 AM

use 12_Jan_2024_auto_cropper then resizer to get exact size