Software Engineering Courses (SECourses)•11mo ago

If I want to further finetune a lora or a model, what is the best approach ? Do i need the original

If I want to further finetune a lora or a model, what is the best approach ? Do i need the original dataset ? or can i give the model or lora new images without the old ones and continue further tuning ?

�

𝗡𝗶𝘁𝗶𝗻 𝗣𝗮𝗿𝗲𝘃𝗮•2/1/25, 12:05 PM

@Dr. Furkan Gözükara Is it possible to train a style and an object in the same Lora? You mention something about having two folders in the dataset preparation part and having a specific number of repeats for each of them.

rob•2/1/25, 2:43 PM

What are the fundamental differences between kohya, onetrainer and simpletuner? I've only used ai-toolkit (kohya) because there are many online resources, but I've seen information about SimpleTuner being better.. however, I am not sure why/what the differences could be

AiInfluence•2/1/25, 7:24 PM

AiInfluence•2/1/25, 7:24 PM

Some stuff I made with rank 128

RockerxOP•2/2/25, 4:49 AM

@Dr. Furkan Gözükara can you suggest ?
If I want to further finetune a lora or a model, what is the best approach ? Do i need the original dataset ? or can i give the model or lora new images without the old ones and continue further tuning ?

mmercuri•2/2/25, 7:05 AM

Whats the recommended ui/ config to use flux with controlnet?

MMeLL Can anyone help me with this error?

Vitium•2/2/25, 8:30 AM

Are you, by any chance, using swarmUI through StabilityMatrix? I had a similar issue ( couldn't load any flux model whatsoever), and I highly suspect it's StabilityMatrix being wonky

Karandeep Singh•2/2/25, 9:48 AM

Hi everyone, I need advice on setting trigger tokens and classes for two training datasets:

Hands Interaction

Dataset: ~150 diverse photos of peoples showing interactions like handshakes or couples holding hands etc.
What trigger token would best represent these interactions, and what class should I assign?

Physical Product

Dataset: ~25 photos of a physical product (e.g., a cooker - utensil).
What trigger token and class would you recommend for this category?

AlexGilSeg•2/2/25, 3:53 PM

@Dr. Furkan Gözükara Hey.. Thank you for all the info you have on training models.. Its truly amazing.. I read in one of your posts about rank and flux loras but I didnt get the point really.. I am trying to prevent overfitting to use characters as comicbook characters.. The loras are amazing for realistic but I find that they are difficult to convert to comicbook characters.. You wrot a post about less paramters not sure what you meant.. Do you suggest a low rank/alpha to prevent overfitting or what did you mean ? Thanks

mmercuri•2/3/25, 12:56 AM

@Dr. Furkan Gözükara What's your recommendation for training with a minimum number of small number pictures? Any experinece with using 1 or 2? Effectively training a lora based on an AI generated character created with Flux.

Have you had experience with ControlNet for posing working successfully on Flux yet? I see Black Forest labs has released some files to assist with this, but was curious if you had worked with them.

wardensc2•2/3/25, 1:57 AM

@Dr. Furkan Gözükara the newest of your ComfyUI install include DeepSpeed, does it improve inference speed or something ?

Ruairi Robinson•2/3/25, 3:20 AM

Hey I made a music video using veo 2, some of you might find it of interest, anyway its all AI (except the music). Hope its ok to post here.
https://www.youtube.com/watch?v=XICNVp7yhg0&t=1s

YouTubeRuairi Robinson

The Blue Stone - Steve Moore

Music video for Steve Moore's "The Blue Stone"
Directed/edited by Ruairi Robinson
Visuals generated using Google Veo 2

.tarkan•2/3/25, 8:38 AM

@Dr. Furkan Gözükara Here a gen from a custom fine tune on large images I’ve tiled and cut into 1024x1024 pixel chunk separate pngs to capture every tiny possible detail from the up to 6K large images. It was an experiment and it went well with your config. Original image count was 15 for those large images and then after filing there were over 400 images. So this seems to be the trick to be able to train on any resolution images without losing any details.

RRuairi Robinson Hey I made a music video using veo 2, some of you might find it of interest, any...

roosterpolice•2/3/25, 1:40 PM

This is absolutely stunning mate!

..tarkan @Dr. Furkan Gözükara Here a gen from a custom fine tune on large images I’ve til...

julius•2/3/25, 7:01 PM

interesting! how does the model know the tiles fit together when training?

julius•2/3/25, 7:03 PM

made a new flux fine tuning model of my friend with 58 images, also gave great and trustworthy results. im wondering how low it can go. my first attempt with 15 images was too glass-skin like. but yea at least now i know that 58 images works well!

Jjulius interesting! how does the model know the tiles fit together when training?

.tarkan•2/3/25, 7:57 PM

I've written a python script with chatGPT that will tile them with overlap and is also including the original full size image. Plus I'm using flux dev de-distilled.

.tarkan•2/3/25, 8:03 PM

@julius so yeah the fine details the model can't train on when only the full image is trained on it's own since the image gets automatically downscaled to 1024x1024 so the training will never see the full details in your dataset. I've taken a step further and have 3 different resolutions for the original image in 2x increments so that it will have 1024x1024 which is the full image downscaled to give the training the full context of the subject then a 2048x2048 version which will serve for 2x Ultimate SD Upscale training on 1024x tiles and then a 4096x4096 size tiled in 1024x1024 tiles to serve as data for training for 4x Ultimate SD Upscale and lastly the original resolution whatever it is to be tiled into 1024x1024 tiles with overlap to not miss any detail from the original dataset.

.tarkan•2/3/25, 8:03 PM

Here some more outputs.

.tarkan•2/3/25, 8:03 PM

.tarkan•2/3/25, 8:04 PM

.tarkan•2/3/25, 8:04 PM

FFurkan Gözükara SECourses Click to see attachment

Neuromasters•2/3/25, 9:22 PM

Wow, there are so many neural nets that you've taken apart and assembled

Darth Teethius•2/3/25, 10:35 PM

Has anyone used the nee deep face similarity app? It works great but I'm not sure what sort of numbers are considered very similar in terms of comparison. I know lower is better but is .3 good or still very off for instance?

Jjulius made a new flux fine tuning model of my friend with 58 images, also gave great a...