Software Engineering Courses (SECourses)•3y ago

hmmm i havent used his ones, cause some of the images there you don't want to use. So i'd just pul

hmmm i havent used his ones, cause some of the images there you don't want to use.

So i'd just pull out the best ones from there, take about 1000>

get nice clear shot images

Yorth - 夜土•12/9/23, 8:01 PM

ehm, hey everyone, does anyone have experience with kohya ss?

FFiveSAI hmmm i havent used his ones, cause some of the images there you don't want to us...

Lorelai AI•12/9/23, 8:01 PM

Awesome, what should the network Dimm and Alpha be?

FiveSAIOP•12/9/23, 8:02 PM

Theres a few you can try

128 32

96 32

64 32

i mainly go with 128 32

FiveSAIOP•12/9/23, 8:02 PM

Yes i know some videos and tutorials say to use alpha of 1, but it takes longer to converge and sometimes doesnt work to well

YYorth - 夜土 ehm, hey everyone, does anyone have experience with kohya ss?

FiveSAIOP•12/9/23, 8:02 PM

Yep whats up

LLorelai AI Awesome, what should the network Dimm and Alpha be?

FiveSAIOP•12/9/23, 8:03 PM

Also if you feel like constant learning rate is cooking your image, switch to cosine

Yorth - 夜土•12/9/23, 8:03 PM

I'm trying to see if its possible to just pass in the link of a huggingface dataset instead of making an image folder and linking it with captions

Yorth - 夜土•12/9/23, 8:03 PM

especially on the gui side

FFiveSAI Also if you feel like constant learning rate is cooking your image, switch to co...

Lorelai AI•12/9/23, 8:04 PM

You know what, I’ll bet that’s the biggest issue on time. Been using 1 on Alpha

LLorelai AI You know what, I’ll bet that’s the biggest issue on time. Been using 1 on Alpha

FiveSAIOP•12/9/23, 8:04 PM

It shouldnt be hmmm, are you usingMemory efficient attention >

Lorelai AI•12/9/23, 8:05 PM

I don’t think so

YYorth - 夜土 I'm trying to see if its possible to just pass in the link of a huggingface data...

FiveSAIOP•12/9/23, 8:05 PM

are you training locally or runpod

Yorth - 夜土•12/9/23, 8:05 PM

on google colab

LLorelai AI I don’t think so

FiveSAIOP•12/9/23, 8:05 PM

how about gaming? noticing any issues with the 4090?

YYorth - 夜土 on google colab

FiveSAIOP•12/9/23, 8:06 PM

ahh never used google collab, you should try kaggle or runpod

FFiveSAI how about gaming? noticing any issues with the 4090?

Lorelai AI•12/9/23, 8:06 PM

Not a bit, crushes Baldur’s 3 on max

Yorth - 夜土•12/9/23, 8:06 PM

they should work the same I think

Yorth - 夜土•12/9/23, 8:06 PM

did work with kaggle before and it's pretty similar

LLorelai AI Not a bit, crushes Baldur’s 3 on max

FiveSAIOP•12/9/23, 8:07 PM

you should be good then, if training is still slow lemme know

FiveSAIOP•12/9/23, 8:07 PM

also do you have harware-acclerated gpu scheduling turned on?

YYorth - 夜土 they should work the same I think

FiveSAIOP•12/9/23, 8:07 PM

damn might have to wait till somene gets back to you

idk google collab

Lorelai AI•12/9/23, 8:08 PM

Do you change the LR scheduler?

Lorelai AI•12/9/23, 8:08 PM

And B16 or F16?

FFiveSAI damn might have to wait till somene gets back to you 😦 idk google collab

Yorth - 夜土•12/9/23, 8:08 PM

ah okay, honestly I'm only looking at this

LLorelai AI Do you change the LR scheduler?

FiveSAIOP•12/9/23, 8:08 PM

Yep metioned it above ^ Also if you feel like constant learning rate is cooking your image, switch to cosine

LLorelai AI And B16 or F16?

FiveSAIOP•12/9/23, 8:08 PM

bf16 since you're on a 4090

YYorth - 夜土 ah okay, honestly I'm only looking at this

FiveSAIOP•12/9/23, 8:09 PM

try putting it like this /stabilityai/stable-diffusion-xl-base-1.0

Yorth - 夜土•12/9/23, 8:10 PM

ah, that's chill, I think it wants me to download it and give it the full path. I can do that

Yorth - 夜土•12/9/23, 8:11 PM

the issue is honestly with the dataset, if I just give it this Yorth/dalleTestDataFiltered2, it will make life much easier than downloading the dataset, separating the image folder, finding a way to link them with the captions without messing things up

Yorth - 夜土•12/9/23, 8:11 PM

:sigh:

FFiveSAI bf16 since you're on a 4090

Lorelai AI•12/9/23, 8:20 PM

I’m playing around and the numbers are dropping. For reference are your settings based on 512 by 512? I’m using 768 by 1344

FiveSAIOP•12/9/23, 8:22 PM

Mine is 1024x1024, and on 1536x1536 it takes around 2 and half hours

LLorelai AI I’m playing around and the numbers are dropping. For reference are your settings...

FiveSAIOP•12/9/23, 8:23 PM

can you share me your cmd, like a secreenshot of it

FiveSAIOP•12/9/23, 8:23 PM

you can dm if you want

Lorelai AI•12/9/23, 8:24 PM

What’s a cmd?

LLorelai AI What’s a cmd?

FiveSAIOP•12/9/23, 8:24 PM

the terminal where the kohay gui is

FiveSAIOP•12/9/23, 8:24 PM

where it tells you the progress of training

Lorelai AI•12/9/23, 8:25 PM

Sure

FiveSAIOP•12/9/23, 8:26 PM

also your kohya could be bad, so maybe reinstall it as well, and reconfigure accelerate

Lorelai AI•12/9/23, 8:26 PM

Lorelai AI•12/9/23, 8:26 PM

It’s much better with your suggestions

FiveSAIOP•12/9/23, 8:27 PM

whats your vram usage?

Lorelai AI•12/9/23, 8:27 PM

How do I check that?

FiveSAIOP•12/9/23, 8:27 PM

search for task manager

FiveSAIOP•12/9/23, 8:27 PM

then go to performance and then to gpu

Lorelai AI•12/9/23, 8:28 PM

It’s tinkering at 100 percent is that normal

FiveSAIOP•12/9/23, 8:28 PM

okay cancel the training, and enable gradient checkpointing

FiveSAIOP•12/9/23, 8:29 PM

also 2080 steps is a little low, so maybe up your repeats by 10 or epochs to 14

Lorelai AI•12/9/23, 8:30 PM

This Konya version doesn’t have all the older one has, is there a hidden spot for gradient checkpoints

FiveSAIOP•12/9/23, 8:30 PM

which version are you on?

hmmm i havent used his ones, cause some of the images there you don't want to use. So i'd just pul

Similar Threads