Software Engineering Courses (SECourses)•13mo ago

Adafactor best for memory

Furkan Gözükara SECoursesOP•12/19/24, 12:54 AM

each one requires a new research

QQM is there a particular reason behind using ohwx man?

Furkan Gözükara SECoursesOP•12/19/24, 12:54 AM

ohwx is a rare token for clip

Furkan Gözükara SECoursesOP•12/19/24, 12:54 AM

so i still use it

Hiramos_•12/19/24, 4:13 AM

@Furkan Gözükara SECourses Is it possible to perform DreamBooth training on a custom Flux checkpoint?

HHiramos_<@205854764540362752> Is it possible to perform DreamBooth training on a custom ...

Furkan Gözükara SECoursesOP•12/19/24, 9:44 AM

yes

Furkan Gözükara SECoursesOP•12/19/24, 9:44 AM

https://medium.com/@furkangozukara/thoroughly-experimented-with-fine-tuning-dreambooth-training-of-flux-dev-de-distill-pixelwave-13fab8c625bf

Medium

Thoroughly experimented with Fine-Tuning / DreamBooth training of F...

Thoroughly experimented with Fine-Tuning / DreamBooth training of Flux-dev-de-distill, PixelWave v03, Verus Vision and base FLUX Dev model…

Zhigarev•12/19/24, 11:38 AM

Does anyone here have experience creating the same DreamBooth character with different outfits? The problem is that when training in DreamBooth without captions, it's necessary to specify every clothing element to achieve the desired look, which heavily consumes tokens in the prompt and reduces adherence to the prompt during generation. Do you have any experience with this?

D4UNTLESS•12/19/24, 12:09 PM

@Furkan Gözükara SECourses hi
I'm about to do flux fine tuning on massed compute by following your tutorial but I have two questions:

1. What's the difference between a model fine tuned by using batch size of 1 and a model fine tuned by using a batch size of 7? How much (by percentage) is the 1 batch size better than the 7 batch size?
2. I'm fine tuning the model to train a realistic character. I have 410 training images. How many epochs should I go with if I use a batch size of 7? I need your best guess. In your tutorial, you mentioned that for 256 training images, 40 epoch is best for a batch size of 7

DD4UNTLESS <@205854764540362752> hi I'm about to do flux fine tuning on massed compute by f...

Furkan Gözükara SECoursesOP•12/19/24, 12:45 PM

very small differnce

Furkan Gözükara SECoursesOP•12/19/24, 12:45 PM

go with up to 50 epochs

Furkan Gözükara SECoursesOP•12/19/24, 12:45 PM

and compare

FFurkan Gözükara SECourses Adafactor best for memory

Renzo•12/19/24, 12:52 PM

Interesting! I will experiment with AdamW today.

DD4UNTLESS <@205854764540362752> hi I'm about to do flux fine tuning on massed compute by f...

Timson•12/19/24, 2:33 PM

in my experience the more images are in the training set the more training steps is required to reach the same level of training.
For example 15 character image set is already good at 1800 steps (120 epochs), but with 50 images in training set the same level of training is achieved around 3200 steps (64 epochs)

Timson•12/19/24, 2:38 PM

SO the more images there are - the more generalised concept the model is learning - the more training steps required

Timson•12/19/24, 2:42 PM

IMO the sweet spot for character training is 25 images. With more images the training gets exponentially slower and the end result is not necessarily better

TTimson in my experience the more images are in the training set the more training steps...

Furkan Gözükara SECoursesOP•12/19/24, 3:35 PM

true

TTimson IMO the sweet spot for character training is 25 images. With more images the tra...

Furkan Gözükara SECoursesOP•12/19/24, 3:35 PM

it depends your aims

FFurkan Gözükara SECourses it depends your aims

D4UNTLESS•12/19/24, 4:27 PM

Thanks for the reply earlier. I previously trained a flux lora following your tutorial and used one of your configs with 100 training images and 31 epochs. It achieved (to some extent) what I was looking for (a slightly different face but same body) but only missed the mark at body consistency. So, this time I'm looking to get something better by finetuning instead of lora. I want a slight variation in face but 90-100% body consistency. As said before, I'm using 410 images. So will I achieve my targets with 50 epochs? Thanks!

FFurkan Gözükara SECourses yes

Hiramos_•12/19/24, 6:30 PM

Thank you!

DD4UNTLESS Thanks for the reply earlier. I previously trained a flux lora following your tu...

Furkan Gözükara SECoursesOP•12/19/24, 9:11 PM

yes should be sufficient

Furkan Gözükara SECoursesOP•12/19/24, 9:11 PM

410 images really a lot

FFurkan Gözükara SECourses yes should be sufficient

D4UNTLESS•12/19/24, 9:18 PM

Thanks! One final question. My images are in different resolutions and cannot be cropped. You can see some of the resolutions in the image. Can I train on these resolutions or just 1024x1024?

DD4UNTLESS Thanks! One final question. My images are in different resolutions and cannot be...

Furkan Gözükara SECoursesOP•12/19/24, 9:43 PM

this is not much recommended

Furkan Gözükara SECoursesOP•12/19/24, 9:43 PM

flux works best with 1 mega pixel

Furkan Gözükara SECoursesOP•12/19/24, 9:43 PM

if i were you

Furkan Gözükara SECoursesOP•12/19/24, 9:43 PM

i would compare bucketing enabled vs

Furkan Gözükara SECoursesOP•12/19/24, 9:43 PM

cropping all into 1024x1024 training

Jassim•12/19/24, 9:54 PM

im following step by step when trying to finetune sdxl on kohya ss and keep getting meta lat json error

JJassim im following step by step when trying to finetune sdxl on kohya ss and keep gett...

Furkan Gözükara SECoursesOP•12/19/24, 9:54 PM

hi i think you should use flux 1 branch

Furkan Gözükara SECoursesOP•12/19/24, 9:54 PM

he didnt update main branch for a while now

Furkan Gözükara SECoursesOP•12/19/24, 9:54 PM

flux 1 branch was working when i last time tested

Jassim•12/19/24, 9:56 PM

sd3-flux.1?

Jassim•12/19/24, 9:57 PM

Opera_Snapshot_2024-12-20_015714_github.com.png

JJassim sd3-flux.1?

Furkan Gözükara SECoursesOP•12/19/24, 10:15 PM

sd3-flux.1

Furkan Gözükara SECoursesOP•12/19/24, 10:15 PM

you can use my fine tuning flux installer

Renzo•12/19/24, 11:02 PM

@Furkan Gözükara SECourses , have you fine-tuned (Dreambooth) the FP8 or GGUF model? If so, how were the results?

RRenzo <@205854764540362752> , have you fine-tuned (Dreambooth) the FP8 or GGUF model? ...

Furkan Gözükara SECoursesOP•12/19/24, 11:11 PM

not possible for flux

Furkan Gözükara SECoursesOP•12/19/24, 11:11 PM

only fp16 can be trained

FFurkan Gözükara SECourses cropping all into 1024x1024 training

D4UNTLESS•12/19/24, 11:21 PM

As most of my training images are below 1024x1024, can I do padding to bring all of them to 1024x1024?

DD4UNTLESS Thanks! One final question. My images are in different resolutions and cannot be...

Timson•12/20/24, 7:50 AM

training on different resolutions in the dataset seems to be working fine for me

DD4UNTLESS As most of my training images are below 1024x1024, can I do padding to bring all...

Furkan Gözükara SECoursesOP•12/20/24, 10:51 AM

it may work yes

TTimson training on different resolutions in the dataset seems to be working fine for me

Furkan Gözükara SECoursesOP•12/20/24, 10:51 AM

yes i also say compare both method and see working good or not

ArtChicken•12/20/24, 11:16 AM

I'm retraining some human models that I trained between May and June on onetrainer (with the presets available at that time), and I have to say that in terms of similarity, the new trainings (based on SDXL and some realistic PDXL models that gave me great results) can't even come close to the ones I got a few months ago. So I asked myself: has the workflow on onetrainer changed a lot in the last months? Are there any special or different precautions to take into account when using the updated presets? I'm using fast tier v4... and I almost felt like trying to use fast tier v2 again to see if I can get the excellent models of the past months again...

TTimson training on different resolutions in the dataset seems to be working fine for me

D4UNTLESS•12/20/24, 11:57 AM

Thanks, Timson! I'll give it a go then (without padding)

AArtChicken I'm retraining some human models that I trained between May and June on onetrain...

Furkan Gözükara SECoursesOP•12/20/24, 12:43 PM

they made so many updates

Furkan Gözükara SECoursesOP•12/20/24, 12:43 PM

maybe broken

FFurkan Gözükara SECourses they made so many updates

ArtChicken•12/20/24, 2:06 PM

Can you suggest me what is the best guide currently on Patreon for doing SDXL/PDXL training (primarily... and maybe also LORA Flux)?

AArtChicken Can you suggest me what is the best guide currently on Patreon for doing SDXL/PD...

Furkan Gözükara SECoursesOP•12/20/24, 2:50 PM

lora flux : https://youtu.be/nySGu12Y05k

YouTubeSECourses

FLUX LoRA Training Simplified: From Zero to Hero with Kohya SS GUI ...

Ultimate Kohya GUI FLUX LoRA training tutorial. This tutorial is product of non-stop 9 days research and training. I have trained over 73 FLUX LoRA models and analyzed all to prepare this tutorial video. The research still going on and hopefully the results will be significantly improved and latest configs and findings will be shared. Please wat...

Furkan Gözükara SECoursesOP•12/20/24, 2:50 PM

Full Instructions and Links Written Post (the one used in the tutorial)

https://www.patreon.com/posts/click-to-open-post-used-in-tutorial-110879657

Furkan Gözükara SECoursesOP•12/20/24, 2:51 PM

sdxl : https://youtu.be/0t5l6CP9eBg

YouTubeSECourses

Full Stable Diffusion SD & XL Fine Tuning Tutorial With OneTrainer ...

In this tutorial, I am going to show you how to install OneTrainer from scratch on your computer and do a Stable Diffusion SDXL (Full Fine-Tuning 10.3 GB VRAM) and SD 1.5 (Full Fine-Tuning 7GB VRAM) based models training on your computer and also do the same training on a very cheap cloud machine from MassedCompute if you don't have such compute...

Adafactor best for memory

Similar Threads

Similar Threads

Similar Threads