Software Engineering Courses (SECourses)•17mo ago

https://one.one.one.one/

1.1.1.1 — The free app that makes your Internet faster.

Install the free app that makes your phone’s Internet more fast, private, and reliable.

Furkan Gözükara SECoursesOP•8/31/24, 10:26 AM

so activate warp vpn and do a fresh install

FFurkan Gözükara SECourses you can use both fp8 or fp16

orpheus_reup•8/31/24, 10:33 AM

Yes... but fp8 just not working with Lora. I don't know why. I just switch models, everything else stay the same.

Maybe I'll risk a forge update

FFurkan Gözükara SECourses https://one.one.one.one/

LW24Ai•8/31/24, 10:35 AM

The problem is not with the internet, I installed the original one and it works https://github.com/bmaltais/kohya_ss and your build is failing

GitHub

GitHub - bmaltais/kohya_ss

Contribute to bmaltais/kohya_ss development by creating an account on GitHub.

LLW24Ai The problem is not with the internet, I installed the original one and it works ...

Furkan Gözükara SECoursesOP•8/31/24, 10:37 AM

You can see my install by editing file it is nothing special it installs that with accurate branch

Oorpheus_reup Yes... but fp8 just not working with Lora. I don't know why. I just switch model...

Furkan Gözükara SECoursesOP•8/31/24, 10:38 AM

Ah could be forge thing

Furkan Gözükara SECoursesOP•8/31/24, 10:38 AM

I used only Fp16 on forge

Sharinel•8/31/24, 10:42 AM

Just a quick FYI for anyone interested, here's how long it took me to do a 20img/150 epoch training session on a 4090 this morning. Used 'Rank 4 best config better colours' script

SSharinel Just a quick FYI for anyone interested, here's how long it took me to do a 20img...

Furkan Gözükara SECoursesOP•8/31/24, 10:43 AM

Very nice speed

FFurkan Gözükara SECourses You can see my install by editing file it is nothing special it installs that wi...

LW24Ai•8/31/24, 10:50 AM

Is this error going to cause me problems down the road?

LLW24Ai Is this error going to cause me problems down the road?

Sharinel•8/31/24, 10:52 AM

https://discord.com/channels/772774097734074388/1248477696443416656/1279388443486716037

SSharinel https://discord.com/channels/772774097734074388/1248477696443416656/127938844348...

LW24Ai•8/31/24, 10:52 AM

Thanks

LLW24Ai Is this error going to cause me problems down the road?

Furkan Gözükara SECoursesOP•8/31/24, 11:00 AM

Nope

Furkan Gözükara SECoursesOP•8/31/24, 11:00 AM

We don't use xformers with FLUX

FFurkan Gözükara SECourses I used only Fp16 on forge 🙂

orpheus_reup•8/31/24, 11:02 AM

The fp16 works really well.

I used no captioning, no regs. Best Lora i ever made

Dragoy•8/31/24, 12:18 PM

@Dr. Furkan Gözükara Can you please tell me if this is considered a good learning curve for my 3090?

DDragoy @Dr. Furkan Gözükara Can you please tell me if this is considered a good learnin...

Furkan Gözükara SECoursesOP•8/31/24, 12:19 PM

loss rate is meaningless

Furkan Gözükara SECoursesOP•8/31/24, 12:19 PM

speed is looking ok not very bad

Furkan Gözükara SECoursesOP•8/31/24, 12:19 PM

i think your model will be really good

Furkan Gözükara SECoursesOP•8/31/24, 12:20 PM

trained a lora for a client i think it turned out to be perfect

Furkan Gözükara SECoursesOP•8/31/24, 12:20 PM

can do smiling very well since we included that :d

mikemenders•8/31/24, 12:36 PM

While you guys are training Flux, I'm experiencing the interesting features of PonyRealism 2.1 under Prodigy. (I see that the Flux training is still being tweaked at Kohya, so I'll wait.) It turns out that the SDXL rule of only training Unet doesn't apply here, because then the finished Lora will only look a little like the person. However, if I train normally (i.e. without the --network_train_unet_only switch), I'm already on target. By using the 'fp8 base training' switch I can weaken the strength of the model a bit, i.e. sometimes it can come in handy in achieving a more accurate Lora. Without fp8, the Lora is stronger, so the d_coef switch needs to be reduced.

Mmikemenders While you guys are training Flux, I'm experiencing the interesting features of P...

Furkan Gözükara SECoursesOP•8/31/24, 12:39 PM

for sdxl i suggest unet + text encoder 1

Furkan Gözükara SECoursesOP•8/31/24, 12:39 PM

also for flux i am testing training clip large text encoder atm

dxqb•8/31/24, 12:48 PM

RealVisXL5 was released

Ddxqb RealVisXL5 was released

Furkan Gözükara SECoursesOP•8/31/24, 12:49 PM

wow i should test

Furkan Gözükara SECoursesOP•8/31/24, 12:50 PM

testing flux clip large trainings

FFurkan Gözükara SECourses wow i should test

dxqb•8/31/24, 12:59 PM

seems to be a big training effort, not just remerge: "Hello everyone! An early version of RealVisXL V5.0 (560k steps) is now available on Hugging Face. The model is still in training, and the final version will be released on Hugging Face and CivitAI at the end of August. You can leave your feedback on this version in the comments below."

GomezBro•8/31/24, 12:59 PM

@Dr. Furkan Gözükara Bro, I am going to sleep now its 6am haha, training loras all night, I fixed all my machines now. Anyway, I want to Train 1000 images (a style of art) I want to train that but doing it on 1 A6000 it would take like a week right? What system you recommend and how much would it cost me on Mass compute? 4-8 GPU A6000? If its $3-5 an HR I am down to pay it. Please do recommend the correct VM to use on there and do you have a video setting one up as I only use local, never rented hardware before. Let me know thanks and gn brother! Totally worth the Patreon fee!

Ddxqb seems to be a big training effort, not just remerge: "Hello everyone! An early v...

Furkan Gözükara SECoursesOP•8/31/24, 1:27 PM

nice

GGomezBro @Dr. Furkan Gözükara Bro, I am going to sleep now its 6am haha, training loras a...

Furkan Gözükara SECoursesOP•8/31/24, 1:28 PM

get 4x A6000 on massed compute

Furkan Gözükara SECoursesOP•8/31/24, 1:28 PM

it is 1.25$ per hour with our coupon

Furkan Gözükara SECoursesOP•8/31/24, 1:28 PM

i made the multi gpu training tutorial so it will be published today or tomorrow hopefully

virginia•8/31/24, 1:32 PM

@Dr. Furkan Gözükara two models have been changed by the author, %cd /kaggle/temp/VAE
!wget "https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/ae.safetensors" -O "ae.safetensors" !wget "https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/flux1-schnell.safetensors" -O "flux1-schnell.safetensors" am I right? another question it seems to take a few mins to get 1 image from flux model on kaggle, right?

virginia•8/31/24, 1:35 PM

@Dr. Furkan Gözükara another question, if I want to use custom nodes in swarm UI, I did not see the folder, do I have to create one? Autocompletions Users-log.ldb clip_vision tensorrt
Embeddings Users.ldb comfy-auto-model.yaml unet
Lora VAE controlnet upscale_models
Stable-Diffusion clip diffusion_models, or where are the custom node folder?

m3m3x•8/31/24, 1:38 PM

@Dr. Furkan Gözükara I've test on windows on my PC with A4000 (8go) and it worked, however as I suspected it use shared memory during the step "move text encoder to gpu" as there is not enough memory on the GPU, that's why it OOM on linux, because it can't use shared memory, then it's passed back to the cpu freeing memory and the rest is loaded that require less than 8go so during training it's stay under 8go. Is there a way to not move the text encoder to gpu, like having it calculated on cpu directly, having it use shared memory or making it use less VRAM, so it will work also on linux ?

Vvirginia @Dr. Furkan Gözükara two models have been changed by the author, %cd /kaggle/tem...

Furkan Gözükara SECoursesOP•8/31/24, 1:38 PM

yes links changed

Mm3m3x @Dr. Furkan Gözükara I've test on windows on my PC with A4000 (8go) and it worke...

Furkan Gözükara SECoursesOP•8/31/24, 1:38 PM

i see what you mean now

Furkan Gözükara SECoursesOP•8/31/24, 1:38 PM

so you get error during cache

Furkan Gözükara SECoursesOP•8/31/24, 1:38 PM

you need to ask this to kohya

Furkan Gözükara SECoursesOP•8/31/24, 1:38 PM

cant linux has shared vram?

m3m3x•8/31/24, 1:39 PM

Ok I could maybe create a ticket on it's git

FFurkan Gözükara SECourses cant linux has shared vram?

m3m3x•8/31/24, 1:39 PM

I search but didn't find a way to do that it seem not implemented in linux nvidia driver

Mm3m3x Ok I could maybe create a ticket on it's git

Furkan Gözükara SECoursesOP•8/31/24, 1:39 PM

please make a detailed reply here and tag @kohya-ss : https://github.com/kohya-ss/sd-scripts/pull/1374

GitHub

support SD3 by kohya-ss · Pull Request #1374 · kohya-ss/sd-scripts

Replace SD3Tokenizer with the original CLIP-L/G/T5 tokenizers.
Extend the max token length to 256 for T5XXL.
Refactor caching for latents.
Refactor caching for Text Encoder outputs
Extract arch...