Software Engineering Courses (SECourses)•14mo ago

And also another thing that I would like to know is: Can I close my Browser or even shut down my PC

And also another thing that I would like to know is:

Can I close my Browser or even shut down my PC while the training is running on massed Compute? Because I saw that the the best training for about 250 images dataset will take about 30-31 hours per training

NNesshom And also another thing that I would like to know is: Can I close my Browser or ...

NicB•11/28/24, 6:20 PM

Just don't shutdown the VM instance in massed compute itself. It is like a separate computer for yours so everything will run as long as you don't shut it down

NNicB Just don't shutdown the VM instance in massed compute itself. It is like a separ...

NesshomOP•11/28/24, 6:24 PM

Okey nice

NNicB Just don't shutdown the VM instance in massed compute itself. It is like a separ...

NesshomOP•11/28/24, 6:25 PM

What about the other things like billing and so on?

NNesshom What about the other things like billing and so on?

NicB•11/28/24, 6:28 PM

Our billing by default is set for automatic charges. You only get charged for instance that are on the running instance page. So you can set thresholds as low as $5 for recharge amount and minimum balance amount.

NesshomOP•11/28/24, 7:18 PM

Nur how Home I couldn’t deploy the gpu? It said you don’t have credits or something like that

Konstantinas•11/28/24, 8:03 PM

I was finetuning Flux, and at some point made prompt larger, and suddenly it failed when trying to generate test image after saving checkpoint. I wonder now, new prompt was only 88 words (356 characters), seems like all within limit. Maybe it was something else?

Konstantinas•11/28/24, 8:08 PM

or maybe its just some random memory error?

ZZhigarev Alright, then what are the settings for training through DreamBooth? As I unders...

Furkan Gözükara SECourses•11/28/24, 10:27 PM

nope not at all. my configs are designed to train any concept or even do fine tuning with large dataset

ZZhigarev And is it possible, for example, to use widescreen images for training so that i...

Furkan Gözükara SECourses•11/28/24, 10:27 PM

you should compare

NNesshom <@1167168099838074940> @SpecialHelper <@205854764540362752> Hey guys I am curre...

Furkan Gözükara SECourses•11/28/24, 10:28 PM

2x wont make difference

Furkan Gözükara SECourses•11/28/24, 10:28 PM

during fine tuning

Furkan Gözükara SECourses•11/28/24, 10:28 PM

but if you do 2x gpu lora it will be

KKonstantinas I was finetuning Flux, and at some point made prompt larger, and suddenly it fai...

Furkan Gözükara SECourses•11/28/24, 10:28 PM

ye that is why i dont recommend samples during training

Furkan Gözükara SECourses•11/28/24, 10:28 PM

they can also cause out of vram or ram error

wardensc2•11/29/24, 2:08 AM

I try to train new character checkpoint using pixelwave and failed miserably, the new trained checkpoint is so much noise and blur seem like beside of Flux Dev original model, mix or trained model is very difficult to use as a training base

gio3000.•11/29/24, 5:41 AM

hi @Furkan Gözükara SECourses what is the best way to train multiple person or is it best to create a model for each person ? If second
option is better what is the best aproach in confy ui to generate a single image with multiple model?

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭•11/29/24, 9:28 AM

If I have 15 great 2048x2048 face pics, should I just these for training, or must I downsize to 1024 first? Any chance the 2048 pics will give better result or not?

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭•11/29/24, 11:01 AM

Another thing, if I want the best possible lora, I won't loose details by first making fine tuned checkpoints, then converting? For this character I really want both, so was thinking of just doing fine tune, then convert the best epochs to loras.

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭•11/29/24, 11:12 AM

I now get error when using your kohya_v16 install script, can't find the gui.sh.

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭•11/29/24, 11:13 AM

massedcompute

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭•11/29/24, 11:14 AM

I figured out my error, I deleted the kohya ss folder, didn't think I needed that when I had your script

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭•11/29/24, 11:17 AM

Foud kohya_ss on git and did a git clone inside apps folder, hope that works

Wwardensc2 I try to train new character checkpoint using pixelwave and failed miserably, th...

Furkan Gözükara SECourses•11/29/24, 12:24 PM

i used it

Furkan Gözükara SECourses•11/29/24, 12:24 PM

but pixelwave is overfit model

Ggio3000.hi <@205854764540362752> what is the best way to train multiple person or is i...

Furkan Gözükara SECourses•11/29/24, 12:24 PM

yes a model for each person

Furkan Gözükara SECourses•11/29/24, 12:24 PM

second question you can try regional prompting in swarmui

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭If I have 15 great 2048x2048 face pics, should I just these for training, or mus...

Furkan Gözükara SECourses•11/29/24, 12:24 PM

for face i think 1024 works best

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭I figured out my error, I deleted the kohya ss folder, didn't think I needed tha...

Furkan Gözükara SECourses•11/29/24, 12:25 PM

ye script uses base installation

Konstantinas•11/29/24, 1:47 PM

if training on de-destilled flux model in kohya, I guess prompts will simply fail to work?

KKonstantinas if training on de-destilled flux model in kohya, I guess prompts will simply fai...

Furkan Gözükara SECourses•11/29/24, 1:48 PM

what do you mean?

Furkan Gözükara SECourses•11/29/24, 1:48 PM

i trained on de-distilled

Furkan Gözükara SECourses•11/29/24, 1:48 PM

https://medium.com/@furkangozukara/thoroughly-experimented-with-fine-tuning-dreambooth-training-of-flux-dev-de-distill-pixelwave-13fab8c625bf

Medium

Thoroughly experimented with Fine-Tuning / DreamBooth training of F...

Thoroughly experimented with Fine-Tuning / DreamBooth training of Flux-dev-de-distill, PixelWave v03, Verus Vision and base FLUX Dev model…

Konstantinas•11/29/24, 1:49 PM

I mean, if I use "sample prompts" in kohya ss, it will think this is Flux model, but as I have de-distilled, this needs different CGF strategy? Its like, in Comfy UI, I need different workflow, if I use de-distilled model?

Konstantinas•11/29/24, 1:50 PM

you probably simply did not use sample prompting when training?

KKonstantinas I mean, if I use "sample prompts" in kohya ss, it will think this is Flux model,...

Furkan Gözükara SECourses•11/29/24, 2:04 PM

what changes is

Furkan Gözükara SECourses•11/29/24, 2:04 PM

instead of cfg 1

Furkan Gözükara SECourses•11/29/24, 2:04 PM

we use cfg 3.5

Furkan Gözükara SECourses•11/29/24, 2:04 PM

both training and inference

Konstantinas•11/29/24, 2:05 PM

I see, thanks. I'll disable "sample prompting" for now, seems like not yet way to supply it to kohya ss

Konstantinas•11/29/24, 2:06 PM

as only has "cfg scale" as an option there

KKonstantinas I see, thanks. I'll disable "sample prompting" for now, seems like not yet way t...

Furkan Gözükara SECourses•11/29/24, 2:14 PM

ye i dont recommend samples

Konstantinas•11/29/24, 2:15 PM

I just like to sneak-peak early on how it goes

but I guess I can just stop training at cehckpoints, render previews manually, and resume from state if needed

Zhigarev•11/29/24, 2:54 PM

Hello everyone! Owners of the RTX 3090 and CPUs (WITHOUT integrated graphics), has anyone managed to achieve the training speed 7.3 s/it stated in the config by @Furkan Gözükara SECourses ? Even when I don't turn anything off, I still end up using 1000GB of VRAM on Windows 11, even if absolutely no third-party programs are running. How it possible ? i got onaly 11.16 s/it with 24GB config file...

Konstantinas•11/29/24, 3:07 PM

about VRAM, I notice one thing. I have RTX 4090, on 24GB config, I have 6s/it; Then I swap blocks to 12GB config, and have 6.7s/it, almost no difference, but now I have lots of free VRAM and can use my PC normally alongside training. Have no idea why is that so

Konstantinas•11/29/24, 3:08 PM

maybe its because I have 128GB of CPU ram

ZZhigarev Hello everyone! Owners of the RTX 3090 and CPUs (WITHOUT integrated graphics), h...

Furkan Gözükara SECourses•11/29/24, 3:10 PM

something wrong

Furkan Gözükara SECourses•11/29/24, 3:10 PM

did you do a fresh kohya install?

KKonstantinas about VRAM, I notice one thing. I have RTX 4090, on 24GB config, I have 6s/it; T...

Furkan Gözükara SECourses•11/29/24, 3:11 PM

you should get better than 6s actually but still not too bad

RebelPuppy•11/29/24, 4:25 PM

oppions on this aws instance type https://instances.vantage.sh/aws/ec2/g6.2xlarge

g6.2xlarge pricing and specs - Vantage

The g6.2xlarge instance is in the gpu instance family with 8 vCPUs, 32.0 GiB of memory and up to 10 Gibps of bandwidth starting at $0.9776 per hour.

RebelPuppy•11/29/24, 4:35 PM

https://instances.vantage.sh/aws/ec2/g5.xlarge

g5.xlarge pricing and specs - Vantage

The g5.xlarge instance is in the gpu instance family with 4 vCPUs, 16.0 GiB of memory and up to 10 Gibps of bandwidth starting at $1.006 per hour.

And also another thing that I would like to know is: Can I close my Browser or even shut down my PC

Similar Threads