Software Engineering Courses (SECourses)•3y ago

can you share your json file, also are you using the latest nvidia drivers with Sysmem fallback poli

can you share your json file, also are you using the latest nvidia drivers with Sysmem fallback policy

FFiveSAI can you share your json file, also are you using the latest nvidia drivers with ...

Lorelai AI•12/9/23, 7:33 PM

That’s a good idea to check the drivers. I’ll grab the json when I have a moment

FiveSAIOP•12/9/23, 7:33 PM

i also have a 4090 my trainings are usually 35 mins to 2 hours depending

Lorelai AI•12/9/23, 7:34 PM

What settings are you using? I know that’s a large request. I’m all over the place even with the trainings and I’m still new.

FiveSAIOP•12/9/23, 7:35 PM

depends i use different settings, but usually adafactor and the extra arguments with the gradient checkpointing and sometimes xformers not always.

FFiveSAI depends i use different settings, but usually adafactor and the extra arguments ...

Lorelai AI•12/9/23, 7:36 PM

Do you think adafactor is better than AdamW (not the 8 bit)

FiveSAIOP•12/9/23, 7:38 PM

For me for faster training with lora i always use adamw8bit, dreambooth still testing but best is adafactor

FiveSAIOP•12/9/23, 7:39 PM

and for sdxl i use the noise ofset 0.0357

FiveSAIOP•12/9/23, 7:40 PM

also using a older version of kohya uses less vram i think 21.8.4

FiveSAIOP•12/9/23, 7:42 PM

also if you want me to train your data set on my system i can. so we canm compare training times

Lorelai AI•12/9/23, 7:44 PM

I’m just playing around at the moment I’m going to look at the items you mentioned. What about the basic settings like images, epochs, repeats? What I’m struggling with real people. I followed the doctors videos and did the first 2 and got just ok results.

FiveSAIOP•12/9/23, 7:48 PM

So far people, I use 15-30 Images, no harsh angles , usually front, 3/4 and side shots.

20 Repeats seems to be the golden amount.

Also i found using too much class images affects my training,

and usually 8 epochs, I generate samples as well, so I usually stop the training early.

FiveSAIOP•12/9/23, 7:48 PM

Also you may want to try using min snr gamma at the value of 10

Lorelai AI•12/9/23, 7:50 PM

How many class images roughly?

FiveSAIOP•12/9/23, 7:51 PM

I have my own set of 2000, but it only uses around 500

the more repeats you have the more class images it uses

Lorelai AI•12/9/23, 7:51 PM

And what SDXL base model do you use?

FiveSAIOP•12/9/23, 7:52 PM

Its a bit tricky so far, i use the base sdxl model 1.0, But usually loras dont work too well with other checkpoints.

So im using a custom merge of base xl juggernaught xl dreamshaper and realvis,

FiveSAIOP•12/9/23, 7:53 PM

but if you're only use base xl, then yeah base xl is the best

FiveSAIOP•12/9/23, 7:54 PM

will probably make my own tutorial soon, still testing for best settings

Lorelai AI•12/9/23, 7:54 PM

I’m setting this up that the moment thanks so much for the help. Learning rates?

FiveSAIOP•12/9/23, 7:54 PM

Are you training lora?

Lorelai AI•12/9/23, 7:54 PM

Yes

FiveSAIOP•12/9/23, 7:55 PM

If you're using adafactor

0.0001
0.0001
0.0001

for all 3

FiveSAIOP•12/9/23, 7:56 PM

Extra arguments

scale_parameter=False relative_step=False warmup_init=False

FiveSAIOP•12/9/23, 7:56 PM

you can also try one with weight decay

scale_parameter=False relative_step=False warmup_init=False weight_decay=0.01

Lorelai AI•12/9/23, 7:56 PM

Ok I think I have almost everything. Question though. The Doctor proves 5200 class images. Do I use the whole folder or pull some out.

FiveSAIOP•12/9/23, 8:00 PM

hmmm i havent used his ones, cause some of the images there you don't want to use.

So i'd just pull out the best ones from there, take about 1000>

get nice clear shot images