Software Engineering Courses (SECourses)•15mo ago

quick question, how is everyone captioning for flux training? single trigger word, joy caption long

quick question, how is everyone captioning for flux training? single trigger word, joy caption long with trigger word, or joy caption without trigger word? or something else

CNOC•10/7/24, 6:22 AM

For training a person just the instance/class and no captions seem to get the job done very well so probably not worth over thinking. For style most people will parrot that it should be captioned natural language but some of the most popular flux lora creators on civit will use only basic trigger or tags and yield great results and some have tested with no captions and still had good results. Broadly it seemed the best to worst options were VLM/LLM/Joycaption, tagging, trigger only, no caption but with Flux I struggled to find any conclusive info but could find conflicting 'evidence' of various methods so it appears the annoying answer is its probably case by case depending on a review of results for your specific purpose. I would love to know the answer myself too to save that effort of trial and error so if your google skills are better than mine please share if you find something!

FFurkan Gözükara SECourses https://www.patreon.com/posts/112099700

liorftw•10/7/24, 6:31 AM

can i train it with flux 1.1? someone has the link for flux 1.1?

liorftw•10/7/24, 6:43 AM

or i need to do it after extraction of my lora? im little confused

CNOC•10/7/24, 6:48 AM

you cannot train flux1.1, the kohya configs on patreon include a script that will download the flux dev model and required files, then you can choose to train a lora or do a fine tune and extract a lora or use the fine tune as is in place of flux

Darth TeethiusOP•10/7/24, 7:09 AM

@Dr. Furkan Gözükara im trying the new flux training scripts. Using the json for 12gb (i have a 3060) and im getting a Cuda oom error. My virtual memory is set to 65gb per the patreon description. Any idea what might be wrong?

Darth TeethiusOP•10/7/24, 7:09 AM

Darth TeethiusOP•10/7/24, 7:10 AM

this is the error

DDarth Teethius this is the error

crystalwizard•10/7/24, 7:11 AM

did you search google for that error?

Darth TeethiusOP•10/7/24, 7:13 AM

i searched the cuda out of memory

DDarth Teethius i searched the cuda out of memory

crystalwizard•10/7/24, 7:13 AM

search the "might be complied" message

crystalwizard•10/7/24, 7:13 AM

and make sure you ahve enough paging space

Darth TeethiusOP•10/7/24, 7:14 AM

i have pagefile set at 65gb

Darth TeethiusOP•10/7/24, 7:14 AM

i have that space

Darth TeethiusOP•10/7/24, 7:15 AM

im not seeing the complied error

DDarth Teethius im not seeing the complied error

crystalwizard•10/7/24, 7:16 AM

right under the first out of memory error, look for Cuda kernal errors might be asynchronously reported at some other API call

crystalwizard•10/7/24, 7:16 AM

it's a pretty common cuda error.

Darth TeethiusOP•10/7/24, 7:19 AM

it seems to be --disable-cuda-malloc related

Darth TeethiusOP•10/7/24, 7:19 AM

from what im reading,

Darth TeethiusOP•10/7/24, 7:20 AM

if im getting it right, but im unsure where to insert that

Darth TeethiusOP•10/7/24, 7:20 AM

looks like it goes into the .bat

CCNOC you cannot train flux1.1, the kohya configs on patreon include a script that wil...

liorftw•10/7/24, 7:23 AM

so I need to extract a lora that i like? and then to do a dreambooth training on it?

Darth TeethiusOP•10/7/24, 7:26 AM

say you want to make the best quality lora, you would do a fine tune, then there is a function in kohya that looks at the base file(flux) and your fine tune safetensor and it extracts the difference between them. That becomes your final lora

liorftw•10/7/24, 7:28 AM

Fine tune is to train a dreambooth?

liorftw•10/7/24, 7:29 AM

Sorry I'm just started learning

Darth TeethiusOP•10/7/24, 7:29 AM

yes

liorftw•10/7/24, 7:29 AM

When I train dreambooth I need to train big data set of man pics again with also my pics?

Darth TeethiusOP•10/7/24, 7:30 AM

for flux?

liorftw•10/7/24, 7:30 AM

Yes

liorftw•10/7/24, 7:30 AM

Because I remember that what you needed for sdxl

Darth TeethiusOP•10/7/24, 7:30 AM

no man pics. reg images don't work with flux

liorftw•10/7/24, 7:30 AM

Oh cool

liorftw•10/7/24, 7:31 AM

So Im taking my lora and my dreambooth and doing extraction

liorftw•10/7/24, 7:31 AM

And it will do the best result

CNOC•10/7/24, 7:31 AM

you only need to do one or the other

CNOC•10/7/24, 7:31 AM

You can train a lora directly, or you can do a finetune and extract a lora which will be better quality

liorftw•10/7/24, 7:32 AM

Ok thank you

liorftw•10/7/24, 7:32 AM

Thank you both

CNOC•10/7/24, 7:33 AM

No problem. Are you training locally or on massed compute or something?

liorftw•10/7/24, 7:33 AM

There will be support for 1.1?

Lliorftw There will be support for 1.1?

crystalwizard•10/7/24, 7:34 AM

1.1 is flux pro. flux pro, all versions, are api only

CNOC•10/7/24, 7:34 AM

I personally doubt it, I don't see the reason for them releasing it

liorftw•10/7/24, 7:34 AM

I'm doing local with 4090

DDarth Teethius it seems to be --disable-cuda-malloc related

crystalwizard•10/7/24, 7:35 AM

i'm going to suggest you go to the technical support channel on the stable diffusion discord and look for CS1o, and ask him about that. that question comes up in there all the time

Darth TeethiusOP•10/7/24, 7:36 AM

thank you\

liorftw•10/7/24, 7:37 AM

Maybe there is better checkpoints instead of training flux? Like people not fine tune the model to do more realistic? Like sdxl models

CNOC•10/7/24, 7:37 AM

If you are doing the fine tune just be aware when it saves checkpoints they are huge, if you are wanting to just tinker around for fun I think a lora is a decent first option. Just get like 15 pics that are half decent and see how it goes

liorftw•10/7/24, 7:38 AM

I already did lora and it did very well

CNOC•10/7/24, 7:39 AM

If you are wanting a better checkpoint for the people realism (eg skin) part I think you will likely find that your own training/lora will already address that enough as is probably

liorftw•10/7/24, 7:42 AM

Hmm it interesting

quick question, how is everyone captioning for flux training? single trigger word, joy caption long

Similar Threads

Similar Threads

Similar Threads