Software Engineering Courses (SECourses)•15mo ago

so H100, batch size 12 I getting 16s/it

ar.1OP•10/20/24, 2:04 PM

65GB GPU memory usage

Aar.1 so H100, batch size 12 I getting 16s/it

Furkan Gözükara SECourses•10/20/24, 2:08 PM

wow nice

Furkan Gözükara SECourses•10/20/24, 2:08 PM

almost 1 second / it per image

ar.1OP•10/20/24, 2:09 PM

I can probably fit 14 batch size

FFurkan Gözükara SECourses almost 1 second / it per image

ar.1OP•10/20/24, 2:12 PM

so with H100 at $3 per hour, 10k images with 15 epochs (batch size 12) is gonna cost around $170

Aar.1 so with H100 at $3 per hour, 10k images with 15 epochs (batch size 12) is gonna ...

Furkan Gözükara SECourses•10/20/24, 2:29 PM

looks like way expensive compared to A6000

FFurkan Gözükara SECourses looks like way expensive compared to A6000

ar.1OP•10/20/24, 2:30 PM

yeah thats what im thinking

Furkan Gözükara SECourses•10/20/24, 2:30 PM

10k images 15 epochs would take 53$

Furkan Gözükara SECourses•10/20/24, 2:30 PM

0.31 cent per hour

ar.1OP•10/20/24, 2:30 PM

much better

Furkan Gözükara SECourses•10/20/24, 2:31 PM

yep

FFurkan Gözükara SECourses yep

ar.1OP•10/20/24, 2:32 PM

we cant do multi gpu with A6000 dreambooth finetuning right?

ar.1OP•10/20/24, 2:33 PM

I wish there was option to add more SSD in massed compute

ar.1OP•10/20/24, 2:34 PM

256GB on the A6000 isnt that great

Aar.1 we cant do multi gpu with A6000 dreambooth finetuning right?

Furkan Gözükara SECourses•10/20/24, 2:34 PM

yes you need 80 gb gpus

Aar.1 I wish there was option to add more SSD in massed compute

Vinch•10/20/24, 2:34 PM

I keep having to upload checkpoints and delete them to make room for the next checkpoint

VVinch I keep having to upload checkpoints and delete them to make room for the next ch...

ar.1OP•10/20/24, 2:34 PM

yeah its a pain haha

ar.1OP•10/20/24, 2:34 PM

Has anyone tried A100

ar.1OP•10/20/24, 2:35 PM

I wonder its speed against the H100 as its almost half the price

VVinch I keep having to upload checkpoints and delete them to make room for the next ch...

Furkan Gözükara SECourses•10/20/24, 2:39 PM

you can delete existing apps and models

Furkan Gözükara SECourses•10/20/24, 2:39 PM

shown in new coming video

Furkan Gözükara SECourses•10/20/24, 2:39 PM

also yes i agree 256 GB disk space is very limiting for flux fine tuning :/

FFurkan Gözükara SECourses shown in new coming video

ar.1OP•10/20/24, 2:40 PM

very eager to watch it, whats the ETA ?

dsienra•10/20/24, 2:40 PM

@Dr. Furkan Gözükara this is cool!!!

Aar.1 very eager to watch it, whats the ETA ?

Furkan Gözükara SECourses•10/20/24, 2:40 PM

still editing

Ddsienra @Dr. Furkan Gözükara this is cool!!!

Furkan Gözükara SECourses•10/20/24, 2:40 PM

this is next hopefully after i published video to test

dsienra•10/20/24, 2:42 PM

sounds very promising!!!

snowmonkey•10/20/24, 2:51 PM

Does anyone have experience creating serverless Comfy API endpoint on RunPod ?
I keep getting "Max retries reached while waiting for image generation" with fresh installation of https://github.com/blib-la/runpod-worker-comfy

dsienra•10/20/24, 4:44 PM

@Dr. Furkan Gözükara Update https://huggingface.co/nyanko7/flux-dev-de-distill Lora training. 3 people same class in 1 lora with regularization, regularization captions just the class, lr 0.00005 unet and Te, very interesting results, high diversity on the class on inference on regular flux-dev, promoting for the trained subjects high resemblance, one curious thing a . on the prompt produces a total different image, for example same seed "Name class" different image than "Name class." is like the . works like a separation in the prompt, more to test, if you will use regularization use only the class for the captions, on previous tests I used captioned regularization and it produces unpredictable and unconcistent results. more things to test

nyanko7/flux-dev-de-distill · Hugging Face

FFurkan Gözükara SECourses this is next hopefully after i published video to test

dsienra•10/20/24, 5:02 PM

Now training De-distilled same dataset with no regularization for testing new output preservation script

Ssnowmonkey Does anyone have experience creating serverless Comfy API endpoint on RunPod ? ...

Furkan Gözükara SECourses•10/20/24, 5:44 PM

this is totally pro stuff. only runpod team may help you for free

Ddsienra @Dr. Furkan Gözükara Update https://huggingface.co/nyanko7/flux-dev-de-distill L...

Furkan Gözükara SECourses•10/20/24, 5:45 PM

thanks for the info

Ddsienra Now training De-distilled same dataset with no regularization for testing new ou...

Furkan Gözükara SECourses•10/20/24, 5:45 PM

great way to find all

FFurkan Gözükara SECourses 10k images 15 epochs would take 53$

ar.1OP•10/20/24, 6:09 PM

and 160 hours !!!

ar.1OP•10/20/24, 6:09 PM

thats the only problem

ar.1OP•10/20/24, 6:11 PM

I can get A6000 ada or L40S which may be a bit faster, although 3x the price

ar.1OP•10/20/24, 6:11 PM

I doubt they will have 3x the speed up

deepblue_1337•10/20/24, 6:28 PM

what would be the current way for a face swap? Existing image, existing face.

Ddeepblue_1337 what would be the current way for a face swap? Existing image, existing face.

Furkan Gözükara SECourses•10/20/24, 6:32 PM

rope

Furkan Gözükara SECourses•10/20/24, 6:32 PM

face fusion

deepblue_1337•10/20/24, 6:39 PM

rope seems not really to be in develeopment any more, would face fusion be the safer bet?

ar.1OP•10/20/24, 6:59 PM

Does anyone have any info about optimal dataset sizes ?

I'm trying to retain to make images look more realistic as if they were taken from a camera. How many images should it need for that ?

FFurkan Gözükara SECourses yes but try to find a class prompt that will match better

Nick44•10/20/24, 7:18 PM

unfortunately that didnt work for me too well

Im trying to make a flux lora with varied yet consistent outputs like this with 20 images ( can use more)

I had the best but not optimal results with 1_ohwx Head and body + joy captioning

when I use only instance + class prompt + no captions its way under trained

not too sure what to try next