anyone have got some good flux-dev-de-destill OneTrainer finetuning configurations ?
anyone have got some good flux-dev-de-destill OneTrainer finetuning configurations ?
T@Furkan Gözükara SECourses Could you check v9 configs for 8GB again.. with Torch 2.5, i get out of memory errors, With 2.4 - it works but very slow. It was working on an earlier version of your 8GB configs. File "E:\StabilityMatrix-RAID\Kohya_Flux\kohya_ss\sd-scripts\library\flux_models.py", line 830, in _forward
attn = attention(q, k, v, pe=pe, attn_mask=attn_mask)
File "E:\StabilityMatrix-RAID\Kohya_Flux\kohya_ss\sd-scripts\library\flux_models.py", line 449, in attention
x = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=attn_mask)
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1.90 GiB. GPU 0 has a total capacity of 8.00 GiB of which 0 bytes is free. Of the allocated memory 8.05 GiB is allocated by PyTorch, and 2.07 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
steps: 0%|
attn = attention(q, k, v, pe=pe, attn_mask=attn_mask)
File "E:\StabilityMatrix-RAID\Kohya_Flux\kohya_ss\sd-scripts\library\flux_models.py", line 449, in attention
x = torch.nn.functional.scaled_dot_product_attention(q, k, v, attn_mask=attn_mask)
torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 1.90 GiB. GPU 0 has a total capacity of 8.00 GiB of which 0 bytes is free. Of the allocated memory 8.05 GiB is allocated by PyTorch, and 2.07 GiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
steps: 0%|
Fcan you try with SDPA?
Fi am testing on linux now to test
Fexact vram
Fyou have shared vram enabled right?
Fye it still uses 7gb vram
TI think it has to do with Torch 2.5? Does it work on windows? Shared VRAM should be enabled, system has 64GB, and Task Manager shows 40GB for GPU Memory (8+32), shared GPU memory shows 32GB
Fyes it works
Ftorch 2.5 and xformers is now official
TWhat else can i try? It worked before, but not after using latest torch and configs.. clean SS install using your scripts
Fye clean and let me know
Fi will also now test on my RTX 3060
TI did try Clean.. This is a clean install.
Fdid you try 6 gb config?
Tyes. also gives out of memory
Fso something wrong in your case
Fthe only changed config is SPDA to xformers
Twith SPDA setting.. RuntimeError: CUDA error: out of memory
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with
TORCH_USE_CUDA_DSA to enable device-side assertions.Fyour computer is the issue in that case
Fbecause we use torch 2.5 for a month like now
Fi am doing a fresh install
Fnormally you shouldnt get OOM
Fwhen shared VRAM enabled
Twhere do you enable shared VRAM?
Fit is by default unless you turned off
Ftry a fresh nvidia driver install
Freset all settings
TCuda installed a driver - should i install cuda first, then the latest studio driver?
Fi think first install latest game ready driver
Fthen dont touch cuda
Fcuda is 12.4 right?
Tyes cuda 12.4
Tokay I'll uninstall all drivers. Install latest studio.. Then Cuda 12.4, then fresh install using v9 SS_kohya, then torch 2.5 upgrade
Tif I don't do torch 2.5 upgrade, it works on 2.4 but at 300s/IT
Tbut no cuda error
Fi think dont reinstall cuda
Fye totally wrong
Fi am about to test on rtx 3060
Ffresh install
Fwhat was your gpu?
Ftest starting
Fye 12 gb config used max 10 gb
F
T2080super 8GB
Fi see
Fi think you should get speed between 50 second to 100 second it cant tell exact
F3060 gets 40 second / it
F
TI was getting 52 approx
F
