Software Engineering Courses (SECourses)•6mo ago

also for batch size, what number do you recommend? I'm renting A100 gpu. I'm testing with a batch si

also for batch size, what number do you recommend? I'm renting A100 gpu. I'm testing with a batch size of 1 for 15 images. I want to increase it but I'm afraid the model might overfit. I can test them myself but I'm paying by the hour so testing all of them is going to burn me out

Uunderdog also for batch size, what number do you recommend? I'm renting A100 gpu. I'm tes...

Furkan Gözükara SECourses•7/11/25, 9:22 PM

i recommend bs 1

Uunderdog are you able to share the results for learning purposes?

Furkan Gözükara SECourses•7/11/25, 9:23 PM

i planned the test but later didnt have time

underdogOP•7/11/25, 9:24 PM

Thank you very much. Your testing evaluation is saving my time

underdogOP•7/11/25, 9:28 PM

one more question
you used constant LR scheduler, why not cosine or cosine with restarts?

With the constant scheduler, the model doesn't learn. The training loss doesn't decrease. Is this okay?

Uunderdog one more question you used constant LR scheduler, why not cosine or cosine with ...

Furkan Gözükara SECourses•7/11/25, 9:33 PM

training loss useless

Furkan Gözükara SECourses•7/11/25, 9:33 PM

constant also gives you chance to continue perfectly fine

Konstantin•7/12/25, 5:59 PM

What could be the problem, I tried the kohya-ss/sd-scripts branch for lumina 2 (feature-lumina-image) for lumina_train.py. There were a couple of typos img instead of noisy_model_input, but after correcting them, the training started, the loss even goes down, but at the first generation of samples I get noise. If I add "sample_at_first = true" then I see that before training the image is generated normally, but from the very beginning of training there is noise. Now I have
max_timestep = 1000
guidance_scale = 4
timestep_sampling = "nextdit_shift"
discrete_flow_shift = 6.0
model_prediction_type = "raw"
optimizer_args = [ "weight_decay=0.00",]
optimizer_type = "PagedLion8bit"

I tried changing the optimizer, timestep_sampling, discrete_flow_shift, all the same, no results... Already at the 30th step of training there is noise.

KKonstantin What could be the problem, I tried the kohya-ss/sd-scripts branch for lumina 2 (...

Furkan Gözükara SECourses•7/12/25, 9:03 PM

never tried lumina 2

K4bronsky•7/14/25, 2:17 AM

@Furkan Gözükara SECourses is theres a way to understand how to improve speed? i followed all the steps from the videos on and rtx 5090

KK4bronsky <@205854764540362752> is theres a way to understand how to improve speed? i foll...

Furkan Gözükara SECourses•7/14/25, 2:50 AM

which config?

FFurkan Gözükara SECourses which config?

K4bronsky•7/14/25, 2:53 AM

test_24GB_GPU_22900MB_7.3_second_it_Tier_1.json

KK4bronsky test_24GB_GPU_22900MB_7.3_second_it_Tier_1.json

Furkan Gözükara SECourses•7/14/25, 2:56 AM

use 48 gb config

Furkan Gözükara SECourses•7/14/25, 2:56 AM

rtx 5090 can do it since full model uses like 28.5 gb

K4bronsky•7/14/25, 2:56 AM

ok thanks

Furkan Gözükara SECourses•7/14/25, 2:59 AM

you are welcome i got to sleep

FFurkan Gözükara SECourses you are welcome i got to sleep

K4bronsky•7/14/25, 11:38 AM

i changed to 48 but stills runs at the same speed.
ae = "X:/AI/Kohya/ae.safetensors" do you see anything wrong with the toml?
I really appreciate your help
blocks_to_swap = 0
bucket_no_upscale = true
bucket_reso_steps = 64
cache_latents = true
cache_latents_to_disk = true
cache_text_encoder_outputs = true
cache_text_encoder_outputs_to_disk = true
caption_dropout_every_n_epochs = 0
caption_dropout_rate = 0

K4bronsky•7/14/25, 11:38 AM

single_blocks_to_swap = 0
t5xxl = "X:/AI/Kohya/t5xxl_fp16.safetensors"
t5xxl_max_token_length = 512
timestep_sampling = "sigmoid"
train_batch_size = 1
train_blocks = "all"
train_data_dir = "X:\AI\Kohya\SwarmUI_Model_Downloader_v54\SwarmUI\Models\diffusion_models\img"
vae_batch_size = 4
wandb_run_name = "miakillamodel"
xformers = true

KK4bronsky i changed to 48 but stills runs at the same speed. ae = "X:/AI/Kohya/ae.safetens...

Furkan Gözükara SECourses•7/14/25, 11:39 AM

wait at least 100 steps

Furkan Gözükara SECourses•7/14/25, 11:39 AM

also show me nvitop output

FFurkan Gözükara SECourses wait at least 100 steps

K4bronsky•7/14/25, 11:42 AM

KK4bronsky Click to see attachment

Furkan Gözükara SECourses•7/14/25, 11:52 AM

yes looking accurate

Furkan Gözükara SECourses•7/14/25, 11:52 AM

what is speed?

FFurkan Gözükara SECourses yes looking accurate

K4bronsky•7/14/25, 11:54 AM

4.22

KK4bronsky 4.22

Furkan Gözükara SECourses•7/14/25, 12:05 PM

yes a little bit slow

Furkan Gözükara SECourses•7/14/25, 12:05 PM

i dont know what causes exactly but kohya working slower than before

Furkan Gözükara SECourses•7/14/25, 12:05 PM

he made something worse

Furkan Gözükara SECourses•7/14/25, 12:05 PM

we cant fully utilize gpu for some reason

Furkan Gözükara SECourses•7/14/25, 12:05 PM

FFurkan Gözükara SECourses yes a little bit slow

K4bronsky•7/14/25, 12:11 PM

ok, at least i know Im on the right path, thanks so much

KK4bronsky ok, at least i know Im on the right path, thanks so much

Furkan Gözükara SECourses•7/14/25, 12:15 PM

you are welcome

KK4bronsky ok, at least i know Im on the right path, thanks so much

Furkan Gözükara SECourses•7/14/25, 2:55 PM

10 epoch is too low

Furkan Gözükara SECourses•7/14/25, 2:55 PM

how many images you have in dataset?

FFurkan Gözükara SECourses you are welcome

K4bronsky•7/14/25, 2:56 PM

yeah i paused just to test, i have 171 faces with same hairstyle different emotions, 53 body different poses, a bit of different hair styles

KK4bronsky yeah i paused just to test, i have 171 faces with same hairstyle different emoti...

Furkan Gözükara SECourses•7/14/25, 2:56 PM

what you want to teach?

FFurkan Gözükara SECourses what you want to teach?

K4bronsky•7/14/25, 2:58 PM

a person, trying to keep the same hairstyle, but also the option to change the hairstyle with prompts

KK4bronsky a person, trying to keep the same hairstyle, but also the option to change the h...

Furkan Gözükara SECourses•7/14/25, 2:59 PM

ok it is easy. 171 images are all from same person right?

Furkan Gözükara SECourses•7/14/25, 2:59 PM

a bit of different hair styles are ok

Furkan Gözükara SECourses•7/14/25, 2:59 PM

so what you should do is

K4bronsky•7/14/25, 2:59 PM

yeah mostly same hair cut, the ones with full body have different haircuts

Furkan Gözükara SECourses•7/14/25, 2:59 PM

just train with ohwx woman

Furkan Gözükara SECourses•7/14/25, 2:59 PM

then when prompting

Furkan Gözükara SECourses•7/14/25, 2:59 PM

describe the hair style in details

Furkan Gözükara SECourses•7/14/25, 2:59 PM

and it should work 95%+

FFurkan Gözükara SECourses just train with ohwx woman

K4bronsky•7/14/25, 3:00 PM

ok, ill keep training from epoch 10 forward, and ill try that in a bit, thanks so much

KK4bronsky ok, ill keep training from epoch 10 forward, and ill try that in a bit, thanks s...

Furkan Gözükara SECourses•7/14/25, 3:01 PM

you are welcome

Furkan Gözükara SECourses•7/14/25, 3:01 PM

i trained with 256 images

Furkan Gözükara SECourses•7/14/25, 3:01 PM

and 70 epoch was best

Furkan Gözükara SECourses•7/14/25, 3:01 PM

and it is excellent quality

K4bronsky•7/14/25, 3:32 PM

excelent, yeah thats what ill try to do

Shyam Nimmana•7/16/25, 5:28 AM

For LoRA training with 15 images and an epoch of 200 on RTXA6000, it takes around 7 hours. Shall we change the epoch value to 15 or 10? Is the caption mandatory?

also for batch size, what number do you recommend? I'm renting A100 gpu. I'm testing with a batch si

Similar Threads