Software Engineering Courses (SECourses)•11mo ago

I tried switching to prodigy optimizer setting learning rate to 1 and cosine. With 60 images using b

I tried switching to prodigy optimizer setting learning rate to 1 and cosine. With 60 images using basic tag captions, 150 epoch, batch 4 the result was very amazing and fast.

AikageOP•2/23/25, 4:44 PM

It actually overtrained

AikageOP•2/23/25, 4:44 PM

I have never used it before though so I will need to fine tune the settings and educate myself on the differences between it and adafactor or Adamw

AAikage I have never used it before though so I will need to fine tune the settings and ...

Furkan Gözükara SECourses•2/23/25, 4:52 PM

nice give it a try

Furkan Gözükara SECourses•2/23/25, 4:53 PM

so far i see that optimizer makes minimal difference as long as you are using most optimal parameters. my new workflow will be only slightly better

snowmonkey•2/23/25, 9:42 PM

@Furkan Gözükara SECourses Hi Furkan, would you like to try Veo 2 with image to video ?

snowmonkey•2/23/25, 9:42 PM

We have it enabled at mitte.ai already before other platforms and thought you might want to try it first before others

Ssnowmonkey <@205854764540362752> Hi Furkan, would you like to try Veo 2 with image to video...

Furkan Gözükara SECourses•2/23/25, 10:19 PM

sure add my account some credits i can ttest

FFurkan Gözükara SECourses sure add my account some credits i can ttest

snowmonkey•2/23/25, 10:24 PM

adding!

snowmonkey•2/23/25, 10:24 PM

done

Ssnowmonkey done 🙂

Furkan Gözükara SECourses•2/23/25, 10:42 PM

AAikage I tried switching to prodigy optimizer setting learning rate to 1 and cosine. Wi...

Renzo•2/24/25, 5:07 PM

What GPU did you use?

AikageOP•2/24/25, 5:32 PM

4090ti

AikageOP•2/24/25, 5:32 PM

I am having some trouble tbh it is like 80% there but I cannot push it to 100% if that makes sense

AikageOP•2/24/25, 5:32 PM

I have tried to increase steps and decrease d coef

AikageOP•2/24/25, 5:33 PM

decreasing d coef helped with theh ands and eyes a little but still getting heavy artifacting in many pictures

AikageOP•2/24/25, 5:33 PM

well, not HEAVY, but visible here and there

AikageOP•2/24/25, 5:33 PM

enough that it is irritating

AikageOP•2/24/25, 5:33 PM

Maybe not for me after all

Renzo•2/24/25, 5:34 PM

Interesting. Would you say that Furkan's configuration based on Adafactor still gives higher quality results than with prodigy?

Renzo•2/24/25, 5:39 PM

Or what do you mean by 100%?

RRenzo Interesting. Would you say that Furkan's configuration based on Adafactor still ...

AikageOP•2/26/25, 2:16 PM

sorry for delayed response, still playing around with it. So far, yes absolutely

AikageOP•2/26/25, 2:16 PM

but the speed increase is undeniable so I will continue to experiment

AikageOP•2/26/25, 2:18 PM

there are other parameters I am testing now (dropout, weight decay, further refining d coef) as well as changing batch size/repeats/epochs to see if I can arrive at a good setup

AikageOP•2/26/25, 2:18 PM

https://github.com/derrian-distro/LoRA_Easy_Training_Scripts Also found some other program (not that it will be helpful to train but something to play with)

AikageOP•2/26/25, 2:19 PM

I believe it is Khoya based

AikageOP•2/26/25, 2:19 PM

but a different front end similar to how swarm is to comfy

AikageOP•2/26/25, 2:21 PM

but I could be wrong

I'll take a look and see if I like it better than onetrainer. Khoya UI I didn't like much so this might be a nice solution

Renzo•2/26/25, 5:59 PM

Thank you for expanding on your experiments. Are you using an LLM to guide you through tunning parameters such as dropout or weight decay? I wonder if SOTA LLMs can help us arrive at the optimal ones faster than doing it on our own.

Renzo•2/26/25, 6:19 PM

Are you planning to share your findings? I once tried to fine-tune with prodigy on Kohya but I could not make it work.

AikageOP•2/26/25, 10:32 PM

Sure if course

AikageOP•2/26/25, 10:32 PM

I am not using llm just researching the old fashioned way

AikageOP•2/26/25, 10:32 PM

And trial and error

AikageOP•2/27/25, 12:58 PM

getting close - in fact I think i have it the only problem I have now is the problem I always have which is with unique features getting lost (moles on specific sides of faces, or heterochromia, etc.)

AikageOP•2/27/25, 12:59 PM

I am going to go back and try to weight images which highlight those features more heavily in my training dataset

AikageOP•2/27/25, 1:38 PM

No Access I'll show some results and the issue I have

AikageOP•2/27/25, 1:41 PM

But it might be a day or two actually not feeling well today

Renzo•2/27/25, 2:03 PM

Nice. I appreciate it!

AAikage 4090ti

lazyboy2011•3/1/25, 4:33 AM

4090ti prototype?

sorry that catching for it but I was looking ti without success, as I know they decided not to Release it but I can be in mistake

AikageOP•3/1/25, 4:37 AM

yeah i thought for sure i had ti

AikageOP•3/1/25, 4:38 AM

but you're right

AikageOP•3/1/25, 4:38 AM

i do not

AikageOP•3/1/25, 4:38 AM

lol all this time people probably like what a douche hahahaha

lazyboy2011•3/1/25, 4:38 AM

Prototype photo

AikageOP•3/1/25, 4:39 AM

it was canceled

AikageOP•3/1/25, 4:39 AM

so just a regular 4090 I guess it is time to upgrade to 5090 then

AikageOP•3/1/25, 4:39 AM

maybe they will actually release a 5090 ti

lazyboy2011•3/1/25, 4:39 AM

Yes you are right

lazyboy2011•3/1/25, 4:40 AM

In my case i will buy 5090 and together with 4090

lazyboy2011•3/1/25, 4:41 AM

Actually what is happening now on market with 5000 series, stories that 5080=4070ti,

lazyboy2011•3/1/25, 4:42 AM

Waiting for stabilisation

I tried switching to prodigy optimizer setting learning rate to 1 and cosine. With 60 images using b

Similar Threads