Software Engineering Courses (SECourses)•2y ago

whats with the different naming across platforms? these supposed to be the same parameters, no?

Ddaroth whats with the different naming across platforms? these supposed to be the same ...

Furkan Gözükara SECourses•7/23/24, 10:52 AM

yes all same

Furkan Gözükara SECourses•7/23/24, 10:53 AM

good catch you have

darothOP•7/23/24, 11:41 AM

btw a 128/128 extracted Lora will be like a gb per piece, right?

Ddaroth btw a 128/128 extracted Lora will be like a gb per piece, right?

Furkan Gözükara SECourses•7/23/24, 11:48 AM

it totally depends rank and precision

Furkan Gözükara SECourses•7/23/24, 11:48 AM

let me check

Furkan Gözükara SECourses•7/23/24, 11:48 AM

128 rank float precision is 2 gb

Furkan Gözükara SECourses•7/23/24, 11:48 AM

128 rank bf16 would be 1 gb

Furkan Gözükara SECourses•7/23/24, 11:48 AM

64 rank bf16 would be 500 mb

darothOP•7/23/24, 11:52 AM

most of the time I see it recommended that alpha is half of rank

Ddaroth most of the time I see it recommended that alpha is half of rank

Furkan Gözükara SECourses•7/23/24, 12:43 PM

alpha is just multiplier of the LR

Furkan Gözükara SECourses•7/23/24, 12:43 PM

i think it wont matter in this case

darothOP•7/23/24, 4:32 PM

I'm curious, did you do experiments with training images with transparent pngs? no background at all, just purely the subject

Ddaroth I'm curious, did you do experiments with training images with transparent pngs? ...

Furkan Gözükara SECourses•7/23/24, 4:53 PM

i didnt for person

Furkan Gözükara SECourses•7/23/24, 4:53 PM

also tranmsparent becomes white or black

Furkan Gözükara SECourses•7/23/24, 4:53 PM

depending on script i think

Furkan Gözükara SECourses•7/23/24, 4:53 PM

no transparent level exists in stable diffusion

Théo•7/23/24, 7:07 PM

actually there is a transparent concept in Forge with layer diffusion, but it is a layer calculation by generation, working great but impossible to import that kind of technique in a lora (for now)

Théo•7/23/24, 7:08 PM

TThéo actually there is a transparent concept in Forge with layer diffusion, but it is...

Furkan Gözükara SECourses•7/23/24, 8:44 PM

i know there is 1 model

Furkan Gözükara SECourses•7/23/24, 8:44 PM

specifically trained and designed for this

kraussian•7/25/24, 3:57 AM

Hi @Furkan Gözükara SECourses I've messaged you on Patreon, and you've suggested enabling

xformers

xformers

on OneTrainer to improve speed. How do I do that?

Kkraussian Hi <@205854764540362752> I've messaged you on Patreon, and you've suggested enab...

Maxivi•7/25/24, 7:28 AM

go here

MMaxivi go here

kraussian•7/25/24, 7:53 AM

Found it. Thanks!

MMaxivi go here

Furkan Gözükara SECourses•7/25/24, 9:34 AM

thanks for reply before i did :d

Maxivi•7/25/24, 9:44 AM

no problem

devinthenull•7/26/24, 2:09 AM

I've seen mixed advice, what's the latest thoughts on xformers for lora/fine tuning

Jay•7/26/24, 4:06 AM

Hi all, a total noob to training here and just have experience with Automatic1111 on RunPod. Any advice on the best training option for realistic people using RunPod? Dreambooth, Kohya, OneTrainer etc?

kraussian•7/26/24, 2:52 PM

@Furkan Gözükara SECourses after running OneTrainer for ~6 hours, I was finally excited to see the results of my DreamBooth training. But when I try to load the model in A1111 WebUI, I'm getting this error:

changing setting sd_model_checkpoint to elwq_v0_1.safetensors [a5db6c23b6]: NotImplementedError
Traceback (most recent call last):
  ...
  File "C:\sd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    module._apply(fn)
  [Previous line repeated 3 more times]
  File "C:\sd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 833, in _apply
    param_applied = fn(param)
  File "C:\sd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1158, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
NotImplementedError: Cannot copy out of meta tensor; no data!

changing setting sd_model_checkpoint to elwq_v0_1.safetensors [a5db6c23b6]: NotImplementedError
Traceback (most recent call last):
  ...
  File "C:\sd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 810, in _apply
    module._apply(fn)
  [Previous line repeated 3 more times]
  File "C:\sd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 833, in _apply
    param_applied = fn(param)
  File "C:\sd\stable-diffusion-webui\venv\lib\site-packages\torch\nn\modules\module.py", line 1158, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
NotImplementedError: Cannot copy out of meta tensor; no data!

I've trained using the slow preset, 100 epochs. Only change I've made (apart from folder locations) is to enable xformers. What am I doing wrong?

Ddevinthenull I've seen mixed advice, what's the latest thoughts on xformers for lora/fine tun...

Furkan Gözükara SECourses•7/26/24, 11:24 PM

try to avoid if possible

Furkan Gözükara SECourses•7/26/24, 11:24 PM

but recent versions are much better than previous

JJay Hi all, a total noob to training here and just have experience with Automatic111...

Furkan Gözükara SECourses•7/26/24, 11:24 PM

Kohya dreambooth on runpod

Furkan Gözükara SECourses•7/26/24, 11:25 PM

i think i replied your message on patreon?

Kkraussian <@205854764540362752> after running OneTrainer for ~6 hours, I was finally excit...

Furkan Gözükara SECourses•7/26/24, 11:25 PM

did you take any backups_?

Furkan Gözükara SECourses•7/26/24, 11:25 PM

checkpoints?

Furkan Gözükara SECourses•7/26/24, 11:25 PM

what is your file size?

Furkan Gözükara SECourses•7/26/24, 11:26 PM

i think it couldnt understand your model class like sdxl or sd 1.5 but not sure

FFurkan Gözükara SECourses did you take any backups_?

kraussian•7/27/24, 1:08 AM

Yes, I’ve taken backups every 10 epochs, so let me see if those work.
File sizes are 6.7GB for backups and the final file

Kkraussian Yes, I’ve taken backups every 10 epochs, so let me see if those work. File sizes...

Furkan Gözükara SECourses•7/27/24, 1:23 AM

looking good

Furkan Gözükara SECourses•7/27/24, 1:23 AM

ye try backup

FFurkan Gözükara SECourses i think i replied your message on patreon?

Jay•7/27/24, 1:37 AM

yes thanks!

FFurkan Gözükara SECourses looking good

kraussian•7/27/24, 9:59 AM

Managed to find a working backup at 90 epochs, but the images generated look nothing like the training images. I've used 33 images which I feel are of relatively good quality, and the 5,400 regularization images from your Patreon.
Should I be increasing the learning rate?

gohar•7/27/24, 10:08 AM

sir, when will u work on RVC or any other better voice cloning tool

Kkraussian Managed to find a working backup at 90 epochs, but the images generated look not...

Furkan Gözükara SECourses•7/27/24, 10:54 AM

can you share screenshots of your concepts setups

Ggohar sir, when will u work on RVC or any other better voice cloning tool

Furkan Gözükara SECourses•7/27/24, 10:54 AM

it is on my list but these keep coming up new ai models giving no time

gohar•7/27/24, 11:00 AM

on point sir

gohar•7/27/24, 1:20 PM

sir SUPIR is taking too much time on kaggle while video upscaling.. is there any solution to this?