Software Engineering Courses (SECourses)•15mo ago

almost finished rendering but an error (

Vigilence•10/17/24, 11:06 AM

@Furkan Gözükara SECourses The use of save-states is much better as its actually continuing the training of the model. Starting off from the last saved checkpoint is essentially a whole different training of the model and the results will be different.

Vigilence•10/17/24, 11:09 AM

Anyway, I figured out how to use saved state and it worked, you just have to enable it at the start and it saves a special folder called saved-state. If for some reason you have to continue training due to crash or no HD space you can continue by inputting the folder patch into this field before you start. If you get an error then the path is not correct. If the folder exists with what is seen in the photo then it will definitely continue training with the same state it left off at as well as same result of training, just a continuation.

Vigilence•10/17/24, 11:26 AM

I forgot to mention, you would also have to edit the number of epochs, to be what you did not initially finish. So if you have a training for 200 epochs, and the saved state is from the 150 epoch, you have to change the epoch to 50 in the settings, or else it will continue training from the 150th epoch save-state for another 200 epochs. You can increase the epochs what ever you like, but it will be start from epoch 1 to what you set, since you are continuing the training using the saved-state folder. Its confusing if you don't understand this part.

Will Scarlett Ohara•10/17/24, 12:38 PM

Is there any point in modifying LR with adafactor? 22 hours of training and 4400 steps later, I have the impression that my final samples are still undertrained.

Will Scarlett Ohara•10/17/24, 12:46 PM

If I choose the last checkpoint, can I resume training from there for more steps?

CCarlos Quick question, does fine tuning require less steps than LORA? My 3,000 steps fi...

Furkan Gözükara SECourses•10/17/24, 12:59 PM

learning rates

Furkan Gözükara SECourses•10/17/24, 12:59 PM

and step count depends on dataset

Wwantedrbk almost finished rendering but an error (

Furkan Gözükara SECourses•10/17/24, 12:59 PM

your yolo file is not properly downloaded

VVigilence <@205854764540362752> The use of save-states is much better as its actually cont...

Furkan Gözükara SECourses•10/17/24, 1:00 PM

nice but that is extra space :d

WWill Scarlett Ohara Is there any point in modifying LR with adafactor? 22 hours of training and 4400...

Furkan Gözükara SECourses•10/17/24, 1:00 PM

sure you can increase

Furkan Gözükara SECourses•10/17/24, 1:00 PM

to get faster

WWill Scarlett Ohara If I choose the last checkpoint, can I resume training from there for more steps...

Furkan Gözükara SECourses•10/17/24, 1:01 PM

sure you can

Furkan Gözükara SECourses•10/17/24, 1:01 PM

it will continue

FFurkan Gözükara SECourses sure you can increase

Will Scarlett Ohara•10/17/24, 1:31 PM

I thought adafactor had a kind of auto LR. Thanks

WWill Scarlett Ohara I thought adafactor had a kind of auto LR. Thanks

Furkan Gözükara SECourses•10/17/24, 1:37 PM

we use static

Furkan Gözükara SECourses•10/17/24, 1:37 PM

auto lr works good for LLMs where you can truly calculate meaningfull accuracy

Vigilence•10/17/24, 1:59 PM

@Furkan Gözükara SECourses Yeah, that is the only downside. The provided drive is not suited for save-states unless you constantly download and delete older files.

Vigilence•10/17/24, 2:25 PM

I think there is an option to only have a single saved-state folder, with each new saved-state overwriting the older one. I am asking kohya and bmaltis how to do this.

mostlyhuman•10/17/24, 7:17 PM

@Furkan Gözükara SECourses I want to do some dreambooth training on Flux.1 to teach it more about musculature and anatomy, ChatGPT suggested creating training imagesets that use segmentation masks to isolate the various muscles . Can Flux.1 use segmented images for training like this? Any information is sincerely appreciated!

Vigilence•10/17/24, 7:39 PM

@mostlyhuman I recommend checking this out. https://www.reddit.com/r/DreamBooth/comments/1fz539o/training_on_body_parts_rather_than_the_person_as/

From the DreamBooth community on Reddit

Explore this post and more from the DreamBooth community

VVigilence <@243094858074554370> I recommend checking this out. https://www.reddit.com/r/D...

mostlyhuman•10/17/24, 7:57 PM

Thank you, that is exactly what I need, hopefully it works, it seemed like others were having trouble replicating the results. I am a little confused on some of the details I reached out to the OP, hopefully he can shed some light. Thanks again!

Vigilence•10/17/24, 7:58 PM

Check the link and download the lora for research purposes. Keep in mind the lora is not SFW.

VVigilence Check the link and download the lora for research purposes. Keep in mind the lo...

mostlyhuman•10/17/24, 7:59 PM

I didnt see a link, it was in his post?

Vigilence•10/17/24, 8:00 PM

Its in the reply to the OP.

Vigilence•10/17/24, 8:01 PM

The lora using this method can be found here

https://www.seaart.ai/models/detail/7d73c6a5f56ade946171cd1b8ddf00e2

VVigilence The lora using this method can be found here 🔞 - https://www.seaart.ai/models...

mostlyhuman•10/17/24, 8:05 PM

perfect, thanks!

Vigilence•10/17/24, 8:06 PM

Segmentation such as SAM2 could be used, but you would have to test to see how accurate it is.

VVigilence Segmentation such as SAM2 could be used, but you would have to test to see how a...

mostlyhuman•10/17/24, 8:12 PM

So is that just a link to his finished model? I was hoping to see a few more examples of how he structures his datasets

Mmostlyhuman <@205854764540362752> I want to do some dreambooth training on Flux.1 to teach i...

Furkan Gözükara SECourses•10/17/24, 8:44 PM

i dont think you need and probably wouldnt work

Vigilence•10/17/24, 9:03 PM

@Furkan Gözükara SECourses It works, just check the unlocked versions with proper nudity. New flux models now know what bobs, vajajays, boobs, rears, etc look like.

Vigilence•10/17/24, 9:06 PM

@mostlyhuman I think he was teased away from explaining it further since the data he was working on was nudity. But you can message him, he can explain in better detail. Maybe even share more examples.

VVigilence <@205854764540362752> It works, just check the unlocked versions with proper nud...

mostlyhuman•10/17/24, 9:07 PM

ahh okay, that tracks

Vigilence•10/17/24, 11:23 PM

@Furkan Gözükara SECourses When can we see the newest comparison between dora, finetune, finetune lora and lycoris?

VVigilence <@205854764540362752> When can we see the newest comparison between dora, finetu...

Furkan Gözükara SECourses•10/17/24, 11:31 PM

later

Furkan Gözükara SECourses•10/17/24, 11:31 PM

by lycoris already inferior in quality

Furkan Gözükara SECourses•10/17/24, 11:31 PM

dora maybe

Vigilence•10/17/24, 11:31 PM

We must all see its inferiority!

Furkan Gözükara SECourses•10/17/24, 11:31 PM

i agree i plan to research them too

Vigilence•10/17/24, 11:32 PM

@Furkan Gözükara SECourses BTW, I wanted to ask, why do you not have buckets enabled in the training config?

VVigilence <@205854764540362752> BTW, I wanted to ask, why do you not have buckets enabled ...

Furkan Gözükara SECourses•10/17/24, 11:34 PM

i dont recommend

Vigilence•10/17/24, 11:34 PM

Why is that?

Furkan Gözükara SECourses•10/17/24, 11:34 PM

its impact being unpredictable

Vigilence•10/17/24, 11:34 PM

I see.

Furkan Gözükara SECourses•10/17/24, 11:34 PM

so train at 1024x1024 first

Furkan Gözükara SECourses•10/17/24, 11:34 PM

then train with bucket

Furkan Gözükara SECourses•10/17/24, 11:34 PM

and compare

Vigilence•10/17/24, 11:35 PM

Can't you control it by resizing the images to be either 1024px by longest or shortest side? You still keep the max resolution to 1024x1024, but the images are then bucketed more naturally.

Furkan Gözükara SECourses•10/17/24, 11:40 PM

i think it is best to crop to 1024x1024

Furkan Gözükara SECourses•10/17/24, 11:40 PM

then again you can just enable bucket and let kohya handle

Furkan Gözükara SECourses•10/17/24, 11:40 PM

i really recommend comparing results of both

almost finished rendering but an error (

Similar Threads