Software Engineering Courses (SECourses)•3y ago

VRAM usage is a bit hectic

Aashley VRAM usage is a bit hectic

felixsanz•6/5/23, 11:13 PM

that's a lot lol ,worth it?

Ffelixsanz that's a lot lol ,worth it?

ashleyOP•6/5/23, 11:14 PM

Probably not, I am still training, it/sec is almost 1/3 of using Lion, so very slow

FFurkan Gözükara SECourses yes but they may bleed. likely to bleed if they are same class

Nom•6/5/23, 11:14 PM

but if they are different concept, then it doesnt really matter if you did 2 concepts at once or just one training session after another right

Ffelixsanz you use buckets? or train for that resolution?

Furkan Gözükara SECourses•6/5/23, 11:16 PM

no i use single bucket. all images are 768x1024

NNom but if they are different concept, then it doesnt really matter if you did 2 con...

Furkan Gözükara SECourses•6/5/23, 11:17 PM

it matters since you are also training unet

Furkan Gözükara SECourses•6/5/23, 11:17 PM

the more you train unet = bad

FFurkan Gözükara SECourses it matters since you are also training unet

Nom•6/5/23, 11:18 PM

I see so in multiple concepts, it doesnt train the unet so theres some advantage

FFurkan Gözükara SECourses no i use single bucket. all images are 768x1024

felixsanz•6/5/23, 11:19 PM

so you train that resolution? SD 1.5 model?

Ffelixsanz so you train that resolution? SD 1.5 model?

Furkan Gözükara SECourses•6/5/23, 11:19 PM

realistic vision yes

felixsanz•6/5/23, 11:20 PM

can you share some parameters like LR, steps, etc? so i give it a try too

Ffelixsanz can you share some parameters like LR, steps, etc? so i give it a try too

Furkan Gözükara SECourses•6/5/23, 11:20 PM

same as in my video

felixsanz•6/5/23, 11:20 PM

how many training pictures, how many reg images, that kind of stuff

Furkan Gözükara SECourses•6/5/23, 11:20 PM

1e-7 lion

felixsanz•6/5/23, 11:20 PM

you like lion

Ffelixsanz how many training pictures, how many reg images, that kind of stuff

Furkan Gözükara SECourses•6/5/23, 11:20 PM

tried 90, 85 class images

Furkan Gözükara SECourses•6/5/23, 11:20 PM

30 training images

felixsanz•6/5/23, 11:20 PM

that's a lot of images

felixsanz•6/5/23, 11:21 PM

and how many vram does this need?

Ffelixsanz and how many vram does this need?

Furkan Gözükara SECourses•6/5/23, 11:22 PM

it is very slow

Furkan Gözükara SECourses•6/5/23, 11:22 PM

you need 24 gb card

Furkan Gözükara SECourses•6/5/23, 11:22 PM

because i dont use xformers

Furkan Gözükara SECourses•6/5/23, 11:22 PM

or gradient checkpointing

felixsanz•6/5/23, 11:22 PM

so the a5000

felixsanz•6/5/23, 11:22 PM

is what i normally use anyway

Furkan Gözükara SECourses•6/5/23, 11:22 PM

no any 24 gb can work

Furkan Gözükara SECourses•6/5/23, 11:22 PM

like rtx 3090

felixsanz•6/5/23, 11:23 PM

but a5000 is cheapest i think

felixsanz•6/5/23, 11:23 PM

i'm gonna try but my training images are already shitty at 512x512 xD

Furkan Gözükara SECourses•6/5/23, 11:24 PM

FFurkan Gözükara SECourses like rtx 3090

ashleyOP•6/5/23, 11:31 PM

RTX 3090 doesn't work for me on RunPod, always can't find the GPU

Aashley RTX 3090 doesn't work for me on RunPod, always can't find the GPU

Furkan Gözükara SECourses•6/5/23, 11:32 PM

ye looks like prefer started to pefer

Ffelixsanz but a5000 is cheapest i think

ashleyOP•6/5/23, 11:33 PM

On RunPod, 3090 is cheaper

Aashley On RunPod, 3090 is cheaper

Furkan Gözükara SECourses•6/5/23, 11:39 PM

ye i use community

Furkan Gözükara SECourses•6/5/23, 11:39 PM

0.29$

Furkan Gözükara SECourses•6/5/23, 11:39 PM

it is probably cheaper than electricity it would use in some countries :d

FFurkan Gözükara SECourses it is probably cheaper than electricity it would use in some countries :d

ashleyOP•6/5/23, 11:40 PM

Yes, thats why they say they are going to get rid of the current spot pricing structure because it costs more in energy than what they are charging for spot. They said they might keep it but make it a fixed rate and not have the bidding system anymore.

ashleyOP•6/5/23, 11:41 PM

In some countries like mine, we are lucky to even have electricity. They cut out our power in rotation for several hours a day.

ashleyOP•6/5/23, 11:41 PM

The joys of living in Africa