Software Engineering Courses (SECourses)•11mo ago

Interesting. Would you say that Furkan's configuration based on Adafactor still gives higher quality

Interesting. Would you say that Furkan's configuration based on Adafactor still gives higher quality results than with prodigy?

RenzoOP•2/24/25, 5:39 PM

Or what do you mean by 100%?

RRenzo Interesting. Would you say that Furkan's configuration based on Adafactor still ...

Aikage•2/26/25, 2:16 PM

sorry for delayed response, still playing around with it. So far, yes absolutely

Aikage•2/26/25, 2:16 PM

but the speed increase is undeniable so I will continue to experiment

Aikage•2/26/25, 2:18 PM

there are other parameters I am testing now (dropout, weight decay, further refining d coef) as well as changing batch size/repeats/epochs to see if I can arrive at a good setup

Aikage•2/26/25, 2:18 PM

https://github.com/derrian-distro/LoRA_Easy_Training_Scripts Also found some other program (not that it will be helpful to train but something to play with)

Aikage•2/26/25, 2:19 PM

I believe it is Khoya based

Aikage•2/26/25, 2:19 PM

but a different front end similar to how swarm is to comfy

Aikage•2/26/25, 2:21 PM

but I could be wrong

I'll take a look and see if I like it better than onetrainer. Khoya UI I didn't like much so this might be a nice solution

RenzoOP•2/26/25, 5:59 PM

Thank you for expanding on your experiments. Are you using an LLM to guide you through tunning parameters such as dropout or weight decay? I wonder if SOTA LLMs can help us arrive at the optimal ones faster than doing it on our own.

RenzoOP•2/26/25, 6:19 PM

Are you planning to share your findings? I once tried to fine-tune with prodigy on Kohya but I could not make it work.

Aikage•2/26/25, 10:32 PM

Sure if course

Aikage•2/26/25, 10:32 PM

I am not using llm just researching the old fashioned way

Aikage•2/26/25, 10:32 PM

And trial and error

Aikage•2/27/25, 12:58 PM

getting close - in fact I think i have it the only problem I have now is the problem I always have which is with unique features getting lost (moles on specific sides of faces, or heterochromia, etc.)

Aikage•2/27/25, 12:59 PM

I am going to go back and try to weight images which highlight those features more heavily in my training dataset

Aikage•2/27/25, 1:38 PM

No Access I'll show some results and the issue I have

Aikage•2/27/25, 1:41 PM

But it might be a day or two actually not feeling well today

RenzoOP•2/27/25, 2:03 PM

Nice. I appreciate it!

AAikage 4090ti

lazyboy2011•3/1/25, 4:33 AM

4090ti prototype?

sorry that catching for it but I was looking ti without success, as I know they decided not to Release it but I can be in mistake

Aikage•3/1/25, 4:37 AM

yeah i thought for sure i had ti

Aikage•3/1/25, 4:38 AM

but you're right

Aikage•3/1/25, 4:38 AM

i do not

Aikage•3/1/25, 4:38 AM

lol all this time people probably like what a douche hahahaha

lazyboy2011•3/1/25, 4:38 AM

Prototype photo

Aikage•3/1/25, 4:39 AM

it was canceled

Aikage•3/1/25, 4:39 AM

so just a regular 4090 I guess it is time to upgrade to 5090 then

Aikage•3/1/25, 4:39 AM

maybe they will actually release a 5090 ti

lazyboy2011•3/1/25, 4:39 AM

Yes you are right

lazyboy2011•3/1/25, 4:40 AM

In my case i will buy 5090 and together with 4090

lazyboy2011•3/1/25, 4:41 AM

Actually what is happening now on market with 5000 series, stories that 5080=4070ti,

lazyboy2011•3/1/25, 4:42 AM

Waiting for stabilisation

cyleap•3/1/25, 9:56 AM

@Furkan Gözükara SECourses hey im using the gradio app for WAN. im using it in RTX A6000 GPU. i set the vram 48GB. using 480p model with IMG2VID. this is regular speed right? or am i doing something wrong? Cause I put the 4k quality img (which is like 30MB) in the input, do i also need to put the smaller image?

Ccyleap <@205854764540362752> hey im using the gradio app for WAN. im using it in RTX A6...

Furkan Gözükara SECourses•3/1/25, 10:04 AM

hi this looks normal

Furkan Gözükara SECourses•3/1/25, 10:04 AM

RTX a6000 is not fast just big vram

FFurkan Gözükara SECourses RTX a6000 is not fast just big vram 😄

cyleap•3/1/25, 10:04 AM

yeah, you reckon 4090 is faster?

Ccyleap yeah, you reckon 4090 is faster?

Furkan Gözükara SECourses•3/1/25, 10:04 AM

yes

Furkan Gözükara SECourses•3/1/25, 10:04 AM

but for this model i didnt compare

Furkan Gözükara SECourses•3/1/25, 10:04 AM

5090 faster though i tested

cyleap•3/1/25, 10:05 AM

okk. ill give it a shot.

Furkan Gözükara SECourses•3/1/25, 10:05 AM

make sure to get high ram machine

Furkan Gözükara SECourses•3/1/25, 10:05 AM

if on runpod

FFurkan Gözükara SECourses yes

cyleap•3/1/25, 10:05 AM

another question. does fp8 makes it faster?

FFurkan Gözükara SECourses make sure to get high ram machine

cyleap•3/1/25, 10:05 AM

okk

Ccyleap another question. does fp8 makes it faster?

Furkan Gözükara SECourses•3/1/25, 10:05 AM

nope only reduces RAM and VRAM

Furkan Gözükara SECourses•3/1/25, 10:05 AM

because operations are still made in bf16

Furkan Gözükara SECourses•3/1/25, 10:05 AM

only model weights loaded as fp8

cyleap•3/1/25, 10:05 AM

i see.

cyleap•3/1/25, 10:06 AM

thanks !

Aikage•3/1/25, 2:55 PM

I made a mistake and used the 720 image to video

1 hour later i'm only 65% done

Aikage•3/1/25, 3:06 PM

Does having a large base image resolution or the output resolution being different than the incoming image resolution make a difference in terms of speed for Wan i2v?

Interesting. Would you say that Furkan's configuration based on Adafactor still gives higher quality

Similar Threads