The Neverending history haha
The Neverending history haha
F
Nwhats the best way to character flux caption? replicate / use joy captioning or simply only use an instance prompt + class prompt ?
DMy experience of training with de_distill:
1) It adds a sharpness to the image. The images are more noisier, better realism, and harsh reality kind. TBH, that is not always a good thing. If you like or do not like an image, it is subjective>
2) The ability to modify CFG can give you vastly varying results for the same seed. Its CFG scale is much, much tamer than dev2pro and produces linear effects (as you would expect) if you increase or decrease it.
3) The additional time it adds to inference is very frustrating. Original dev said 60+ steps, and he is right. I got good results at Step 70. You can get the generation much faster on flux_dev with 25 to 42 steps. Adding steps adds time to an already slower generation speed.
4) You need to use extra settings during generations like thresholding, which adds additional complexity to the traditional step-cfg system.
1) It adds a sharpness to the image. The images are more noisier, better realism, and harsh reality kind. TBH, that is not always a good thing. If you like or do not like an image, it is subjective>
2) The ability to modify CFG can give you vastly varying results for the same seed. Its CFG scale is much, much tamer than dev2pro and produces linear effects (as you would expect) if you increase or decrease it.
3) The additional time it adds to inference is very frustrating. Original dev said 60+ steps, and he is right. I got good results at Step 70. You can get the generation much faster on flux_dev with 25 to 42 steps. Adding steps adds time to an already slower generation speed.
4) You need to use extra settings during generations like thresholding, which adds additional complexity to the traditional step-cfg system.
DDreambooth on flux_dev:
DDreambooth on flux_dev_dedistill:
Nalso lets say im training to make a consistent yet diverse race (ie different robots as opposed to a consistent singular character robot like Optimus Prime) for a flux lora / DB . its best to only use the instance prompt: "ohwx" + no captions + instance : "robot"
Awhat do i fill for class prompt and instance prompt for finetuning on photography images for realism style, not like a single character
VWe all know from Dr. Furkan Gözükara's work that it typically takes 15 images, 3000 steps, and a learning rate of 4e-6 to train a model to capture a face. However, training something as complex as an alternate futuristic reality is an entirely different challenge. The dataset size, steps, and learning rate required for this are much more demanding.
In my case, with a dataset of 500 images for stage 1 and 1000 images for stage 2, I initially started with a low learning rate (4e-6), fearing overfitting. However, after running into issues like underfitting and wasting valuable resources, I learned an important lesson: starting with a high learning rate (like 4e-5) allows you to capture the core of the concept much faster, even if it overfits. After that, you can restart with a lower learning rate to fine-tune the details.
In fact, starting with a low learning rate to avoid overfitting ended up costing me both time and money. It takes longer to realize you're underfitting, and by the time you do, you've already invested significant resources. Allowing the model to overfit initially and then restarting with a lower LR is much more cost-effective and efficient.
That said, I’m still open to learning if this insight can be further optimized. I tend to favor restarting rather than fine-tuning because of my perfectionist tendencies, but there may be a balance that I’m missing. For now, the strategy of beginning with a high LR and adjusting downwards has proven to be the most effective approach for me.
In my case, with a dataset of 500 images for stage 1 and 1000 images for stage 2, I initially started with a low learning rate (4e-6), fearing overfitting. However, after running into issues like underfitting and wasting valuable resources, I learned an important lesson: starting with a high learning rate (like 4e-5) allows you to capture the core of the concept much faster, even if it overfits. After that, you can restart with a lower learning rate to fine-tune the details.
In fact, starting with a low learning rate to avoid overfitting ended up costing me both time and money. It takes longer to realize you're underfitting, and by the time you do, you've already invested significant resources. Allowing the model to overfit initially and then restarting with a lower LR is much more cost-effective and efficient.
That said, I’m still open to learning if this insight can be further optimized. I tend to favor restarting rather than fine-tuning because of my perfectionist tendencies, but there may be a balance that I’m missing. For now, the strategy of beginning with a high LR and adjusting downwards has proven to be the most effective approach for me.
DMakes sense.
DWith overfit, you have a bad product. But at least you have a product.
Fprompt + class prompt
Fyes but try to find a class prompt that will match better
Fyou can test different class prompts on dev model and see which one generates more resembaling images to yours
Fit is true every dataset is different, but i tested my LR on both 15 and 256 images worked very good. are you doing fine tuning or lora? your LR difference is huge 10 times
Aye I just testing with something for now, but my biggest question is how many steps and epoch should it need with 10k images.
F10k images very likely not work
Fi tested 10k images here with our reg images : https://huggingface.co/MonsterMMORPG/Big_FLUX_Fine_Tuning_Experiments/tree/main
Faltough my attempt was general fine tuning
Fstyle may work
Adid it work ?
Fwell i had to use a very low LR
Fi did 5 epochs of 10k images
Fdidnt see significant improvement or anything
Fregular LR destroyed the model
Amaybe needs more
Awith very low lr
Alike 100 epoch
Fyep
Fpossibly
Awhat was the reason you stopped at 5 ?
Fi had to stop to do trainings for fine tuning tutorial
Aah haha
Ais there much benefit to using H100 vs A6000 for this
Alike is the speed improvement worth it you think
Fwell
Aits almost 10x the cost
Fi think L40S and batch size 7 most sense making
Fbut on runpod speed was
Fway lower than I expect
Fmaybe it was that pod
Fi didnt test h100 speed
Fyou should test effective step speed
Aill test it later today and report back
VI do fine-tuning with dreambooth batch-7, I'm currently testing with 2e-5 and 1e-5 (4e-5 was indeed too high, but better too high than too low), with 4000 steps, I also activated buckets for multiple aspect ratio's (where each bucket must have at least 7 images). The first 10% of steps are warmup and the LR decreases with a cosine. It's looking like 2e-5 is working the best for my major concept.
Ahow many images total ?
VI'm currently doing stage 1 that has 556 images. The images in stage 1 has high concept and low details, most of these images are AI generated to really get the concept. For stage 2, I have 2551 images, these images have less of the concept but are high in detail and quality, which are mostly production grade photographs and marketing material for movies and such.
VI don't know what the config will be for stage 2.
Ahow are you ensuring each bucket has at least 7 ?
VI made a script. You want?
Ayeah sure if you dont mind sharing that would be great
