Software Engineering Courses (SECourses)•13mo ago

Btw, I still don't understand what's the point of batching if you have 100% CUDA cores utilisation o

Btw, I still don't understand what's the point of batching if you have 100% CUDA cores utilisation on batch size 1?

DD4UNTLESS Can I use batch 1 48gb on A6000? Will it take longer?

Furkan Gözükara SECourses•12/22/24, 2:15 PM

yes you can use

Furkan Gözükara SECourses•12/22/24, 2:15 PM

it will take a little bit longer

TTimson Btw, I still don't understand what's the point of batching if you have 100% CUDA...

Furkan Gözükara SECourses•12/22/24, 2:15 PM

it still slightly improves speed

Furkan Gözükara SECourses•12/22/24, 2:16 PM

but for quality it reduces a little bit

Furkan Gözükara SECourses•12/22/24, 2:16 PM

you can calculate

Furkan Gözükara SECourses•12/22/24, 2:16 PM

batch size 1 = 6.4 second / it = 7*6.4 second for 7 samples

Furkan Gözükara SECourses•12/22/24, 2:16 PM

batch size 7 = 29.5 second / it

Furkan Gözükara SECourses•12/22/24, 2:16 PM

also i recommend now L40 with our coupon now only 49 cents

D4UNTLESS•12/22/24, 3:35 PM

I'm using cropping + padding to bring all my images to 1024x1024
So I'm not gonna use bucketing. Should I go with batch size 6?

DD4UNTLESS I'm using cropping + padding to bring all my images to 1024x1024 So I'm not gonn...

Furkan Gözükara SECourses•12/22/24, 4:16 PM

you shouldnt get

Furkan Gözükara SECourses•12/22/24, 4:16 PM

i tested many times

Furkan Gözükara SECourses•12/22/24, 4:16 PM

batch size 7 should work no bucket @ 1024 px

DD4UNTLESS Click to see attachment

D4UNTLESS•12/22/24, 4:36 PM

So this error was because of different resolutions? If I use all images with 1024x1024 now (no bucketing) I will not get errors at batch size 7? Just need to get confirmation from you

DD4UNTLESS So this error was because of different resolutions? If I use all images with 102...

Furkan Gözükara SECourses•12/22/24, 4:39 PM

yes

Furkan Gözükara SECourses•12/22/24, 4:39 PM

it was using more vram

FFurkan Gözükara SECourses yes

D4UNTLESS•12/22/24, 6:18 PM

ok thanks! I'll try again after a few hours

johnasdf.•12/22/24, 6:54 PM

Has anyone gotten this message when fine-tuning a Flux model? ValueError: mutable default <class 'timm.models.maxxvit.MaxxVitConvCfg'> for field conv_cfg is not allowed: use default_factory

Jjohnasdf.Has anyone gotten this message when fine-tuning a Flux model? ValueError: mutab...

Furkan Gözükara SECourses•12/22/24, 6:58 PM

which model ?

johnasdf.•12/22/24, 6:59 PM

Initially I fine-tuned with AGFlux_v21, a de-distilled model. Now I am trying to take a second pass on it.

johnasdf.•12/22/24, 7:02 PM

ChatGPT thinks it's either timm, Torch, or Kohya

johnasdf.•12/22/24, 7:04 PM

I noticed there are two venvs created when I ran the kohya setup v17... venv, and venv3. Is that what was intended to happen?

Jjohnasdf.Initially I fine-tuned with AGFlux_v21, a de-distilled model. Now I am trying to...

Furkan Gözükara SECourses•12/22/24, 7:06 PM

report to kohya

Furkan Gözükara SECourses•12/22/24, 7:06 PM

some models are not supported

johnasdf.•12/22/24, 7:07 PM

But I fine-tuned the model a few weeks ago, and then I just downloaded it from my huggingface account.

Jjohnasdf.But I fine-tuned the model a few weeks ago, and then I just downloaded it from m...

Furkan Gözükara SECourses•12/22/24, 7:07 PM

in that case

Furkan Gözükara SECourses•12/22/24, 7:08 PM

try model on swarmui

Furkan Gözükara SECourses•12/22/24, 7:08 PM

see if working

johnasdf.•12/22/24, 7:38 PM

Yeah. it works

johnasdf.•12/22/24, 7:52 PM

I got it to work. I just needed to upgrade timm to the latest version, which is 1.0.12. It had installed 0.6.12. If you want to update the Runpod installer.

johnasdf.•12/22/24, 7:54 PM

When you trained the de-distilled model did you use guidance > 1?

Jjohnasdf.When you trained the de-distilled model did you use guidance > 1?

Furkan Gözükara SECourses•12/22/24, 10:23 PM

yes

Furkan Gözükara SECourses•12/22/24, 10:23 PM

use 3.5 i think it was best

johnasdf.•12/22/24, 10:35 PM

Thank you

FFurkan Gözükara SECourses yes

D4UNTLESS•12/22/24, 11:53 PM

Thanks for the help! Last time I got an error at 32 steps.
I combined padding & cropping to bring all my images to 1014x1024. I used batch 7 config and last time I checked (1 hour ago), it was already at 250 steps

DD4UNTLESS Thanks for the help! Last time I got an error at 32 steps. I combined padding & ...

Furkan Gözükara SECourses•12/23/24, 12:35 AM

great

Furkan Gözükara SECourses•12/23/24, 12:35 AM

250 steps = 250 *7 samples

Furkan Gözükara SECourses•12/23/24, 12:36 AM

1750 steps at batch size 1

ctz00•12/23/24, 6:26 AM

hi, i am doing character training in sdxl dreambooth in kohya but the results are like this:

ctz00•12/23/24, 6:26 AM

ctz00•12/23/24, 6:27 AM

--save_model_as="safetensors" \
--full_bf16 \
--output_name="momo_sdxl_dreambooth" \
--lr_scheduler_num_cycles=8 \
--max_data_loader_n_workers=0 \
--learning_rate=1e-04 \
--lr_scheduler="constant" \
--train_batch_size=1 \
--max_train_steps=10000 \
--save_every_n_steps=500 \
--mixed_precision="bf16" \
--save_precision="bf16" \
--cache_latents \
--cache_latents_to_disk  \
--optimizer_type="Adafactor" \
--optimizer_args \
--bucket_reso_steps=64 \
--noise_offset=0.0 \
--gradient_checkpointing \
--bucket_no_upscale \
--train_text_encoder \
--sample_every_n_epochs=1 \
--sample_every_n_steps=500 \
--sample_sampler="euler" \

--save_model_as="safetensors" \
--full_bf16 \
--output_name="momo_sdxl_dreambooth" \
--lr_scheduler_num_cycles=8 \
--max_data_loader_n_workers=0 \
--learning_rate=1e-04 \
--lr_scheduler="constant" \
--train_batch_size=1 \
--max_train_steps=10000 \
--save_every_n_steps=500 \
--mixed_precision="bf16" \
--save_precision="bf16" \
--cache_latents \
--cache_latents_to_disk  \
--optimizer_type="Adafactor" \
--optimizer_args \
--bucket_reso_steps=64 \
--noise_offset=0.0 \
--gradient_checkpointing \
--bucket_no_upscale \
--train_text_encoder \
--sample_every_n_epochs=1 \
--sample_every_n_steps=500 \
--sample_sampler="euler" \

ctz00•12/23/24, 6:27 AM

how to cure?

Cctz00 hi, i am doing character training in sdxl dreambooth in kohya but the results ar...

Furkan Gözükara SECourses•12/23/24, 8:49 AM

hi follow this video every step : https://youtu.be/0t5l6CP9eBg

YouTubeSECourses

Full Stable Diffusion SD & XL Fine Tuning Tutorial With OneTrainer ...

In this tutorial, I am going to show you how to install OneTrainer from scratch on your computer and do a Stable Diffusion SDXL (Full Fine-Tuning 10.3 GB VRAM) and SD 1.5 (Full Fine-Tuning 7GB VRAM) based models training on your computer and also do the same training on a very cheap cloud machine from MassedCompute if you don't have such compute...

Furkan Gözükara SECourses•12/23/24, 8:49 AM

it shows configs

TimsonOP•12/23/24, 11:29 AM

@Furkan Gözükara SECourses how does new L40 compares to A6000 on training tasks?

TTimson <@205854764540362752> how does new L40 compares to A6000 on training tasks?

Furkan Gözükara SECourses•12/23/24, 12:04 PM

little bit faster

Furkan Gözükara SECourses•12/23/24, 12:04 PM

more disk space

Furkan Gözükara SECourses•12/23/24, 12:04 PM

disk space really worths it

Yobin•12/23/24, 2:12 PM

@Furkan Gözükara SECourses so the dreambooth tab under the kohya gui for flux is not actually dreambooth but it is just native fine tune?

Yobin•12/23/24, 2:17 PM

also it seems not using captioner (like joycaption) is ok with flux, any upside of using captioner still?

TheToday•12/23/24, 3:02 PM

Is there any news on Lora's training for Hunyuan?

--save_model_as="safetensors" \ --full_bf16 \ --output_name="momo_sdxl_dreambooth" \ --lr_scheduler_num_cycles=8 \ --max_data_loader_n_workers=0 \ --learning_rate=1e-04 \ --lr_scheduler="constant" \ --train_batch_size=1 \ --max_train_steps=10000 \ --save_every_n_steps=500 \ --mixed_precision="bf16" \ --save_precision="bf16" \ --cache_latents \ --cache_latents_to_disk \ --optimizer_type="Adafactor" \ --optimizer_args \ --bucket_reso_steps=64 \ --noise_offset=0.0 \ --gradient_checkpointing \ --bucket_no_upscale \ --train_text_encoder \ --sample_every_n_epochs=1 \ --sample_every_n_steps=500 \ --sample_sampler="euler" \

Btw, I still don't understand what's the point of batching if you have 100% CUDA cores utilisation o

Similar Threads

Similar Threads

Similar Threads