Software Engineering Courses (SECourses)•17mo ago

yes, but i also tested with 1024, and 512 is much better

VVs12 yes, but i also tested with 1024, and 512 is much better

Furkan Gözükara SECourses•8/22/24, 1:12 AM

i am gonna test thanks and weird :d

Vs12OP•8/22/24, 1:12 AM

very weird yes

Vs12OP•8/22/24, 1:12 AM

but flux for some reason loves 512 better

FFurkan Gözükara SECourses i am gonna test thanks and weird :d

Vs12OP•8/22/24, 1:12 AM

all good

Vs12OP•8/22/24, 1:13 AM

also have you tried clip skip none vs clips skip 1?

VVs12 also have you tried clip skip none vs clips skip 1?

Furkan Gözükara SECourses•8/22/24, 1:13 AM

not yet

Furkan Gözükara SECourses•8/22/24, 1:13 AM

still testing other params :d

Vs12OP•8/22/24, 1:13 AM

try clip skip 0 when you get a chance, i don't like kohya clip skip 1

Furkan Gözükara SECourses•8/22/24, 1:13 AM

each training is like 7 hours

FFurkan Gözükara SECourses each training is like 7 hours

Vs12OP•8/22/24, 1:13 AM

very long mine are usually less than 1 hour

Furkan Gözükara SECourses•8/22/24, 1:20 AM

i am updating versions on the server now

Furkan Gözükara SECourses•8/22/24, 1:26 AM

To not wait an entire day I am going to train on 2x RTX 4080 , 1024x1024 and 512x512 resolutions
Results will be in 8 - 9 hours hopefully published - got to sleep

Furkan Gözükara SECourses•8/22/24, 1:51 AM

dmn huge vram drop and speed increase with newest driver

Furkan Gözükara SECourses•8/22/24, 1:51 AM

Furkan Gözükara SECourses•8/22/24, 1:51 AM

from 44 gb to 27 staggering

Furkan Gözükara SECourses•8/22/24, 1:51 AM

from 8.4 second to 4.4 second

Furkan Gözükara SECourses•8/22/24, 1:55 AM

testing 24 gb config

Furkan Gözükara SECourses•8/22/24, 2:03 AM

huge news : https://www.reddit.com/r/StableDiffusion/comments/1ey7nwy/flux_lora_rank_128_kohya_ss_gui_training_for_8bit/

From the StableDiffusion community on Reddit: FLUX LoRA Rank 128 Ko...

Explore this post and more from the StableDiffusion community

AIGambino•8/22/24, 2:03 AM

upvoted

AAIGambino upvoted

Furkan Gözükara SECourses•8/22/24, 2:05 AM

thanks. testing right now

Furkan Gözükara SECourses•8/22/24, 2:08 AM

512px uses only 2gb lesser vram

Furkan Gözükara SECourses•8/22/24, 2:08 AM

but 2x faster

FFurkan Gözükara SECourses dmn huge vram drop and speed increase with newest driver

KGTachi•8/22/24, 2:28 AM

Did you update driver manually on massed compute?

KKGTachi Did you update driver manually on massed compute?

Furkan Gözükara SECourses•8/22/24, 2:37 AM

yep

Furkan Gözükara SECourses•8/22/24, 2:37 AM

also i had to prevent kohya to overwrite it

Furkan Gözükara SECourses•8/22/24, 2:37 AM

message.txt4.22KB

Furkan Gözükara SECourses•8/22/24, 2:38 AM

gui.sh file

ruyanisme•8/22/24, 2:46 AM

Hi @Dr. Furkan Gözükara, if today is your birthday and you have unlimited budget, which GPU would you buy?

Rruyanisme Hi @Dr. Furkan Gözükara, if today is your birthday and you have unlimited budget...

Furkan Gözükara SECourses•8/22/24, 2:49 AM

H100

Furkan Gözükara SECourses•8/22/24, 2:49 AM

from consumer grade rtx 4090

GomezBro•8/22/24, 4:43 AM

Guys I am training on my 3090 ti 24GB,(I also own a 48GB A6000 should I use that or its the same result?) Do I have to change anything on instance prompts or parameters? Those are all blank.

All i do is run kohya, load the 24gb json settings, change directories to my tensors, images and then hit train, i get errors:

\Desktop\Kohya\kohya_ss\venv\lib\site-packages\gradio\queueing.py", line 532, in process_events response = await route_utils.call_process_api(

\Desktop\Kohya\kohya_ss\venv\lib\site-packages\gradio\route_utils.py", line 276, in call_process_api
output = await app.get_blocks().process_api(

\Desktop\Kohya\kohya_ss\venv\lib\site-packages\gradio\blocks.py", line 1928, in process_api
result = await self.call_function(

\Kohya\kohya_ss\venv\lib\site-packages\gradio\blocks.py", line 1514, in call_function
prediction = await anyio.to_thread.run_sync(

\Desktop\Kohya\kohya_ss\venv\lib\site-packages\anyio_backends_asyncio.py", line 2177, in run_sync_in_worker_thread
return await future

\Desktop\Kohya\kohya_ss\venv\lib\site-packages\anyio_backends_asyncio.py", line 859, in run
result = context.run(func, args)

\Desktop\Kohya\kohya_ss\venv\lib\site-packages\gradio\utils.py", line 832, in wrapper
response = f(args, **kwargs)

\Desktop\Kohya\kohya_ss\kohya_gui\lora_gui.py", line 1171, in train_model
"network_module": network_module,

\Desktop\Kohya\kohya_ss\kohya_gui\lora_gui.py", line 1171, in train_model
"network_module": network_module,
UnboundLocalError: local variable 'network_module' referenced before assignment

GomezBro•8/22/24, 4:44 AM

Any other settings I need to change before I hit train button?

FFurkan Gözükara SECourses dmn huge vram drop and speed increase with newest driver

protector131090•8/22/24, 6:59 AM

are you using KohyaSS or SD scripts without GUI ? i cant find Checkbox for Flux in Kohya SD3-FLux branch. Only XL and SD 3 ones.

Pprotector131090 are you using KohyaSS or SD scripts without GUI ? i cant find Checkbox for Flux ...

Maxivi•8/22/24, 7:20 AM

loaad the flux model fist, and make sure you're in the sd3 flux branch

virginia•8/22/24, 9:26 AM

@Dr. Furkan Gözükara i forgot I could extract lora from ckpt

virginia•8/22/24, 10:18 AM

@Dr. Furkan Gözükara Traceback (most recent call last):
File "/opt/conda/bin/accelerate", line 8, in <module>
sys.exit(main())
File "/opt/conda/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 47, in main
args.func(args)
File "/opt/conda/lib/python3.10/site-packages/accelerate/commands/launch.py", line 1008, in launch_command
multi_gpu_launcher(args)
File "/opt/conda/lib/python3.10/site-packages/accelerate/commands/launch.py", line 666, in multi_gpu_launcher
distrib_run.run(args)
File "/opt/conda/lib/python3.10/site-packages/torch/distributed/run.py", line 797, in run
elastic_launch(
File "/opt/conda/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/opt/conda/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
/kaggle/working/kohya_ss/sd-scripts/sdxl_train.py FAILED
------------------------------------------------------------
Failures:
<NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
time : 2024-08-22_10:17:12
host : 979028f82d51
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 1222)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================
10:17:13-143688 INFO Training has ended. , what is wrong, i use P100

virginia•8/22/24, 10:47 AM

another question i am training sdxl lora, directly, but why the progress is like this