while generating images in a grid with multiple models, how does inpainting work? which image does
while generating images in a grid with multiple models, how does inpainting work? which image does it take as reference?
Fwhat kind of inpainting?
Fsegment?
Uyes, to make the face look more like the original character and to make it more clear
Fit inpaints every image
Fit generates
Fit is part of prompt
Uyes that I understand. While generating one image with one model it takes the first image that we generated as reference and inpaints the face right? but in grid generation, I didnt have a first image and I directly started generating images. My doubt is what reference would it take for inpainting if we dont have our first image generated?
Fit takes every generated image
Fand then inpaint face
UOh okay. I thought "1" in 1, 0.7, 0.5 meant take the first image as reference for inpainting the face <segment:yolo-face_yolov9c.pt-1,0.7,0.5//cid=11> that is why I was confused and wanted to know what would happen for the first image generation if we didn't have our first image generated.
F1 means first face in image
Uoh 
I completely misunderstood. Thanks for the clarification.
I completely misunderstood. Thanks for the clarification.Ucan we use ipadapter to use a reference image/face so the generated images are consistent?
similar to segment, do we have anything for ipadapter like this - <ipadapter:test_image.jpg:0.5> ?
similar to segment, do we have anything for ipadapter like this - <ipadapter:test_image.jpg:0.5> ?
Fwe have this but there could be other ways too
FPatreon
Get more from SECourses: FLUX, Tutorials, Guides, Resources, Training, Scripts on Patreon
UThank you 
I will take a look!
I will take a look!UI noticed you are using adafactor, what is your take on prodigy? the learning rate will adapt accordingly right? it might consume more vram but what do you think the result quality would be with prodigy?
I might be wrong but because the learning rate adapts, I think we might eliminate the risk of overfitting. Your advise would be helpful.
I might be wrong but because the learning rate adapts, I think we might eliminate the risk of overfitting. Your advise would be helpful.
Fi tested them and none of the others yielded as realistic as adafactor
Fi did a big test recently
UThank you for 15 image dataset, were you able to figure what the unet learning rate was ideal?
for 15 image dataset, were you able to figure what the unet learning rate was ideal?Fyes our rate is ideal
Fjust do at least 200 epoch
Fbut you can do up to like 3000 and compare
Uare you able to share the results for learning purposes?
Ualso for batch size, what number do you recommend? I'm renting A100 gpu. I'm testing with a batch size of 1 for 15 images. I want to increase it but I'm afraid the model might overfit. I can test them myself but I'm paying by the hour so testing all of them is going to burn me out 
Fi recommend bs 1
Fi planned the test but later didnt have time 
UThank you very much. Your testing evaluation is saving my time
Uone more question
you used constant LR scheduler, why not cosine or cosine with restarts?
With the constant scheduler, the model doesn't learn. The training loss doesn't decrease. Is this okay?
you used constant LR scheduler, why not cosine or cosine with restarts?
With the constant scheduler, the model doesn't learn. The training loss doesn't decrease. Is this okay?
Ftraining loss useless
Fconstant also gives you chance to continue perfectly fine
KWhat could be the problem, I tried the kohya-ss/sd-scripts branch for lumina 2 (feature-lumina-image) for lumina_train.py. There were a couple of typos img instead of noisy_model_input, but after correcting them, the training started, the loss even goes down, but at the first generation of samples I get noise. If I add "sample_at_first = true" then I see that before training the image is generated normally, but from the very beginning of training there is noise. Now I have
max_timestep = 1000
guidance_scale = 4
timestep_sampling = "nextdit_shift"
discrete_flow_shift = 6.0
model_prediction_type = "raw"
optimizer_args = [ "weight_decay=0.00",]
optimizer_type = "PagedLion8bit"
I tried changing the optimizer, timestep_sampling, discrete_flow_shift, all the same, no results... Already at the 30th step of training there is noise.
max_timestep = 1000
guidance_scale = 4
timestep_sampling = "nextdit_shift"
discrete_flow_shift = 6.0
model_prediction_type = "raw"
optimizer_args = [ "weight_decay=0.00",]
optimizer_type = "PagedLion8bit"
I tried changing the optimizer, timestep_sampling, discrete_flow_shift, all the same, no results... Already at the 30th step of training there is noise.
Fnever tried lumina 2
K@Furkan Gözükara SECourses is theres a way to understand how to improve speed? i followed all the steps from the videos on and rtx 5090
Fwhich config?
Ktest_24GB_GPU_22900MB_7.3_second_it_Tier_1.json
Fuse 48 gb config
Frtx 5090 can do it since full model uses like 28.5 gb
Kok thanks
Fyou are welcome i got to sleep
Ki changed to 48 but stills runs at the same speed.
ae = "X:/AI/Kohya/ae.safetensors" do you see anything wrong with the toml?
I really appreciate your help
blocks_to_swap = 0
bucket_no_upscale = true
bucket_reso_steps = 64
cache_latents = true
cache_latents_to_disk = true
cache_text_encoder_outputs = true
cache_text_encoder_outputs_to_disk = true
caption_dropout_every_n_epochs = 0
caption_dropout_rate = 0
ae = "X:/AI/Kohya/ae.safetensors" do you see anything wrong with the toml?
I really appreciate your help
blocks_to_swap = 0
bucket_no_upscale = true
bucket_reso_steps = 64
cache_latents = true
cache_latents_to_disk = true
cache_text_encoder_outputs = true
cache_text_encoder_outputs_to_disk = true
caption_dropout_every_n_epochs = 0
caption_dropout_rate = 0
Ksingle_blocks_to_swap = 0
t5xxl = "X:/AI/Kohya/t5xxl_fp16.safetensors"
t5xxl_max_token_length = 512
timestep_sampling = "sigmoid"
train_batch_size = 1
train_blocks = "all"
train_data_dir = "X:\AI\Kohya\SwarmUI_Model_Downloader_v54\SwarmUI\Models\diffusion_models\img"
vae_batch_size = 4
wandb_run_name = "miakillamodel"
xformers = true
t5xxl = "X:/AI/Kohya/t5xxl_fp16.safetensors"
t5xxl_max_token_length = 512
timestep_sampling = "sigmoid"
train_batch_size = 1
train_blocks = "all"
train_data_dir = "X:\AI\Kohya\SwarmUI_Model_Downloader_v54\SwarmUI\Models\diffusion_models\img"
vae_batch_size = 4
wandb_run_name = "miakillamodel"
xformers = true
Fwait at least 100 steps
Falso show me nvitop output
K
Fyes looking accurate
Fwhat is speed?
K4.22
Fyes a little bit slow
Fi dont know what causes exactly but kohya working slower than before