Software Engineering Courses (SECourses)•16mo ago

Is there a tutorial to combine two different LORA models together that have different trigger words.

Is there a tutorial to combine two different LORA models together that have different trigger words. For example, an ohwx model of myself and a bbuk model of my wife. I'd like to combine them to make pictures including both of us. When I select both LORA's while generating a picture, each with a weight of 1, I mostly get the person from the first LORA selected

MMarrvelous Is there a tutorial to combine two different LORA models together that have diff...

Furkan Gözükara SECourses•9/19/24, 8:41 PM

wont work with flux

Furkan Gözükara SECourses•9/19/24, 8:41 PM

flux fully internally captions

Furkan Gözükara SECourses•9/19/24, 8:41 PM

no way to seperate

Kallamamran•9/19/24, 8:46 PM

When will Joycaption get Pixtral support?

FFurkan Gözükara SECourses wont work with flux

MarrvelousOP•9/19/24, 8:47 PM

Thanks for the answer

KKallamamran When will Joycaption get Pixtral support? 😛

Furkan Gözükara SECourses•9/19/24, 8:52 PM

any gradio demo?

Furkan Gözükara SECourses•9/19/24, 8:53 PM

Furkan Gözükara SECourses•9/19/24, 8:54 PM

Furkan Gözükara SECourses•9/19/24, 8:54 PM

preparing dataset of fine tuning will publish captions ngrams too

Kallamamran•9/19/24, 8:54 PM

You can run Pixtral in LM Studio and connect to it with API, so I guess running Pixtral separately should be possible

KKallamamran You can run Pixtral in LM Studio and connect to it with API, so I guess running ...

Furkan Gözükara SECourses•9/19/24, 8:55 PM

well i really need proper model loading pipeline in a single app

Furkan Gözükara SECourses•9/19/24, 8:55 PM

to include it

Furkan Gözükara SECourses•9/19/24, 8:55 PM

otherwise super hard to implement

Furkan Gözükara SECourses•9/19/24, 8:55 PM

especially since we have multi gpu pipe

Kallamamran•9/19/24, 9:00 PM

I see

Furkan Gözükara SECourses•9/19/24, 9:11 PM

caption editor updated

Furkan Gözükara SECourses•9/19/24, 9:11 PM

https://www.patreon.com/posts/108992085

Patreon

Image Captioning Editor Gradio APP - Edit Your Captions Super Easy ...

Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon

Furkan Gözükara SECourses•9/19/24, 9:57 PM

general fine tuning of FLUX

Furkan Gözükara SECourses•9/19/24, 9:57 PM

Furkan Gözükara SECourses•9/19/24, 10:10 PM

check this out : https://www.linkedin.com/posts/furkangozukara_started-my-flux-fine-tuning-project-with-activity-7242656065597702145-W_3B?utm_source=share&utm_medium=member_desktop

Furkan Gözükara on LinkedIn: Started my FLUX Fine Tuning project wi...

Started my FLUX Fine Tuning project with ours previously manually collected ultra HD human images dataset. 5200 images for each man and woman. I am training…

Furkan Gözükara SECourses•9/19/24, 10:34 PM

6k images cached so far

MarrvelousOP•9/19/24, 10:39 PM

We need to get NVIDIA to sponsor you Dr Furkan!

MMarrvelous We need to get NVIDIA to sponsor you Dr Furkan!

Furkan Gözükara SECourses•9/19/24, 11:00 PM

yep

Furkan Gözükara SECourses•9/19/24, 11:09 PM

i just messaged nvidia to get early rtx 5090 for review :d

Pelolisu•9/19/24, 11:46 PM

haha i want rtx 5090 too

MarrvelousOP•9/20/24, 12:13 AM

I asked copilot how to merge two flux lora models and it provided me with some python code. It is saving the merged model right now... I'll see if it works well or not and report back

MarrvelousOP•9/20/24, 12:14 AM

what would be the easy way to segment the faces of both triggers like ohwx and bbuk?

MarrvelousOP•9/20/24, 12:33 AM

Still didn't work. Should have just trusted the Dr and not asked copilot. Merged model still favors one trigger over the other

MMarrvelous I asked copilot how to merge two flux lora models and it provided me with some p...

Furkan Gözükara SECourses•9/20/24, 12:49 AM

kohya gui has merge

Furkan Gözükara SECourses•9/20/24, 12:49 AM

just published another amazing article : https://www.patreon.com/posts/112399667

Patreon

Single Block / Layer FLUX LoRA Training Research Results and LoRA N...

Get more from SECourses: Tutorials, Guides, Resources, Training, FLUX, MidJourney, Voice Clone, TTS, ChatGPT, GPT, LLM, Scripts by Furkan Gözü on Patreon

Furkan Gözükara SECourses•9/20/24, 12:49 AM

public one

FFurkan Gözükara SECourses public one

Sonate•9/20/24, 1:13 AM

Hi, does Flux dev perform well on Kaggle?

SSonate Hi, does Flux dev perform well on Kaggle?

Furkan Gözükara SECourses•9/20/24, 1:34 AM

yes works there

Furkan Gözükara SECourses•9/20/24, 1:34 AM

but slow

MarrvelousOP•9/20/24, 2:20 AM

Would more steps improve accuracy of words with flux? For example, putting your company name and then a team name on a second line on a polo shirt. The company name is displaying just fine, but the 2nd line of text isn't (either blurry or misspelled).

MMarrvelous Would more steps improve accuracy of words with flux? For example, putting your...

Furkan Gözükara SECourses•9/20/24, 2:25 AM

people say that train T5

Furkan Gözükara SECourses•9/20/24, 2:25 AM

XXL for words

MarrvelousOP•9/20/24, 2:26 AM

Thanks

Furkan Gözükara SECourses•9/20/24, 3:05 AM

WetOnTheWater•9/20/24, 4:16 AM

In the flux lora provided configs, do we know how the discrete flow shift value was chosen? Default is 3, config specifies 3.1XXXX?

FFurkan Gözükara SECourses but it is lesser overfit

dxqb•9/20/24, 5:52 AM

agree on 128. Are you using dropout for Lora?

WWetOnTheWater In the flux lora provided configs, do we know how the discrete flow shift value ...

dxqb•9/20/24, 5:55 AM

what does flow shift do? OT doesn't have that

mikemenders•9/20/24, 11:40 AM

While experimenting with Flux, I'm looking at the Pony models because the realistic models are getting better. However, not all models like Lora's and they are unstable.
I have trained a few Lora's on the ponyRealism_v21VAE model, and it seems most realistic Pony models can use the Lora trained on this model.
I have a total of 23 Pony-based models in my system. I have checked 6 seeds on each of them with the same prompt and three different samplers (Euler_a, DPM++ 2M Karras and DPM++ 3M SDE Karras) to see which one best reproduces the original face. I used a CompreFace script to check the final result against the image I used to train the model.
The best models was realDream_pony10 with DPM++ 2M Karras and cyberrealisticPony_v63 with Euler_a. This also gave me a result of 0.95, but I got results above 0.9 instead, and that too in 50% of the cases (so 3 out of 6 seeds were above 0.9).
So it's really hard to train a good person Lora for Pony models, but it's not impossible, and even if I don't get as good results with it as I did with SD 1.5 (where I often had 0.98-0.99 identity!), I'm slowly getting the results I've been looking for. The plain SDXL and Flux are out of the question for now, because I need the NSFW pose and camera knowledge of the Pony in these models, which they don't have at the moment. They take nice pictures, but they are dumb in NSFW. And when I use stacked Lora, the face is distorted in most cases.
As a bonus, here's a prompt that I've used to achieve a nearly similar effect on models weaker than Flux if you want to take nice photos:
'cinematic film still, shallow depth of field, vignette, highly detailed, high budget, bokeh, cinemascope, moody, epic, gorgeous, film grain, grainy'

druhl•9/20/24, 12:47 PM

Do you guys still check the 'bucketing' option? I don't see it being checked in the configs. Also, do you guys try to maintain aspect ratio in style training as well or dataset* images with varying aspect ratios are acceptable with Flux?

SnicksterJay•9/20/24, 2:12 PM

this is a very noob question but I downloaded the flux safetensors to do flux training. I see the safetensors in my file explorer, but once I try to select them within Kohya GUI i can't find it for the life of me. It doesn't even show up when i browse within GUI. It is the flux1-dev safetensors you download using the patreon one click. What can i do that will get Kohya GUI to recognize or see the flux model safetensors?

SSnicksterJay this is a very noob question but I downloaded the flux safetensors to do flux tr...

�

𝗡𝗶𝘁𝗶𝗻 𝗣𝗮𝗿𝗲𝘃𝗮•9/20/24, 2:15 PM

make sure all file type is selected at the bottom right when you are in the file browsing window

Ddruhl Do you guys still check the 'bucketing' option? I don't see it being checked in ...

dxqb•9/20/24, 2:40 PM

sure. bucketing just got a lot more interesting, because Flux is able to learn from low resolutions well

druhl•9/20/24, 3:49 PM

Well, I do have tons of low res images on the subject lol

Alittle Snack•9/20/24, 4:18 PM

What LoRA file sizes are folks getting with Dim/Alpha 128/128 - mine are coming in somewhat huge at almost 3Gb - is this just a Flux thing? any way to reduce them down to something more readably sharable?

mikemenders•9/20/24, 4:47 PM

I am trying JoyCaption. I see this error message in the console after opening the image:
"Too little data for declared Content-Length"
What's my problem?

Is there a tutorial to combine two different LORA models together that have different trigger words.

Similar Threads

Similar Threads

Similar Threads