The extracted lora is actually better than the finetune!
The extracted lora is actually better than the finetune!
Fthat means your model overtrained :d
FPatreon
Get more from SECourses: FLUX, Tutorials, Guides, Resources, Training, Scripts on Patreon
Fdifferent model what you mean?
Fwhich different model
VI picked the wrong base model by accident.
VI mean the colors are nicer and the textures more pronounced over the trained model when using dev paired with lora.
Fyes it is very likely model was overtrained and LoRA extraction reduced this impact
VMaybe so, I used the 200th epoch but at 150 the textures are slighly more pronounced.
Fi also find that epoch good for my 15 images dataset
VYeah it was nice with image prompt, composition, texture quality, layout and overall amount of detail (items placed nicely).
V@Furkan Gözükara SECourses I asked bmatis and he added a feature to the gui that was in the code, the ability to set the max amount of save-state folders to what ever we want. If set to 1 it will be overwritten with the newest save-state made.
Atrying to train on a Flux model but getting this error with it... File "D:\kohya_ss\sd-scripts\library\flux_utils.py", line 81, in analyze_checkpoint_state
max_double_block_index = max(
ValueError: max() arg is an empty sequence
max_double_block_index = max(
ValueError: max() arg is an empty sequence
AIt works fine with the default flux dev, but not this other finetuned dev ccheckpoint
�Does this mean we can now resume training where we left or if I want to train again with more epochs can I just add it and start from where it last trained?
Fyou need to report to kohya
Fyes you can
Fyou can also use latest checkpoint
Sdr Furkan i hope everything is ok , the full tutorial about make drambooth with flux is in process right?
Fyep editing
F
Show this process make dreambooth models intead of loral model make better models thna make the loras? only to undertand how is work
Ffine tuning is training full model
Flora is optimization
Fthus fine tuning better
D@Furkan Gözükara SECourses not sure if kohya and/or you still see this on a closed PR, so here too:
@bmaltais can we add this to gui so many people want to test
all my proof-of-concept samples were at weight 1.
Yes, loss of reg steps is then very low compared to train steps, but this is a function of the reg step prediction already being very close to the target.
you can try at weight 10 or even 1000, but I'd expect that the regularization then overwhelms the training steps, and the model doesn't learn anything anymore.
@bmaltais can we add this to gui so many people want to test
- Specify a large value for --prior_loss_weight option (not dataset config). We recommend 10-1000.
- Set the loss in the training without using the regularization image to be close to the loss in the training using DOP.
all my proof-of-concept samples were at weight 1.
Yes, loss of reg steps is then very low compared to train steps, but this is a function of the reg step prediction already being very close to the target.
you can try at weight 10 or even 1000, but I'd expect that the regularization then overwhelms the training steps, and the model doesn't learn anything anymore.
Vnope it works, just tested it thoroughly, weight 10 works fine just takes longer to train
Deven at 1 I think it dampens training of the concept you do want to change, so even if 10 works I'd be careful and/or try different values
DI was more thinking in the other direction, doing less reg steps than 1:1, because even at 1:1 with weight 1, regularisation is strong
Dbut currently working on sth else so I'd probably not get around to more tests any time soon
Fi am waiting bmaltais to add
Dit's merged in sd3 branch
VYou could always do this if you used the save-state options. You would then paste the path of the folder in the resume from state box. However, the issue is that if you trained for 100 epochs, and the setting was set for 200 epochs, resuming from the trained state of 100 epochs will then train for another 200 epochs, so 300 in total. The epochs in the setting would have to be adjusted if you want max 200.
The new setting is to change the maximum amount of save-state folders you can have. Normally it would save either at the end, which is not great if something bad happens, or at every time you save a copy of the model. This can be a lot of big files, so with this option you can have it set to say a max saved-state amount of 1, so that each new saved-state folder overwrites the older one. Great for saving HD space.
The new setting is to change the maximum amount of save-state folders you can have. Normally it would save either at the end, which is not great if something bad happens, or at every time you save a copy of the model. This can be a lot of big files, so with this option you can have it set to say a max saved-state amount of 1, so that each new saved-state folder overwrites the older one. Great for saving HD space.
WHi @Furkan Gözükara SECourses I try to use your Quality_1_23100MB_14_12_Second_IT.json to train checkpoint but it's very difficult to get 23.1 GB Vram. Windows mostly eat 1.2 - 1.3 GB VRam. How do you limit VRAM use by windows. Since I have 24gb VRAM choosing Quality_1_15500MB_32_Second_IT.json is double the training times and not ultilize my VRAM.
CHello, I want to try to finetune flux dev on a style rather than a character? Any advice on what is different? For example the captioning, I only captioned with my trigger word + class for the character, is that the same for style? If you have any documentation or training (even if not from you) that's be very much appreciated!! love
VFor styles you can either go with a trigger word for the entire image, fully caption everything, or no captions. Characters require captions for everything except the things about the character, like their hair, clothing etc. Things that are captioned can be changed later, things not captioned are static and will be reproduced when prompted.
Fyes in that case
Fjust increase number of block swaps
Fshown in coming video
Fyes i have a full guide and research
Fplease read fully
CThanks mate
Shi, do you know when the Fine Tune tutorial video will be online?
Fhopefully today
Fcurrently generating subtitles
F
Ccant wait, already prepared 4 sets of photos for training
SDr Furkan do you have some flux base model favorites for training?