i have images that ar 3000pixels
i have images that ar 3000pixels
Vand results are great
Fwhat do you mean
Vlike i train very high quality images
Vthey bucket down to 512
Vand results are amazing
Fso they are basically 512
Fdownscaled to
Vyes, but i also tested with 1024, and 512 is much better
Fi am gonna test thanks and weird :d
Vvery weird yes
Vbut flux for some reason loves 512 better
Vall good
Valso have you tried clip skip none vs clips skip 1?
Fnot yet
Fstill testing other params :d
Vtry clip skip 0 when you get a chance, i don't like kohya clip skip 1
Feach training is like 7 hours
V
Vvery long mine are usually less than 1 hour
F
Fi am updating versions on the server now
FTo not wait an entire day I am going to train on 2x RTX 4080 , 1024x1024 and 512x512 resolutions
Results will be in 8 - 9 hours hopefully published - got to sleep
Results will be in 8 - 9 hours hopefully published - got to sleep
Fdmn huge vram drop and speed increase with newest driver
F
Ffrom 44 gb to 27 staggering
Ffrom 8.4 second to 4.4 second
Ftesting 24 gb config
Fhuge news : https://www.reddit.com/r/StableDiffusion/comments/1ey7nwy/flux_lora_rank_128_kohya_ss_gui_training_for_8bit/
Reddit
Explore this post and more from the StableDiffusion community
Aupvoted
Fthanks. testing right now
F512px uses only 2gb lesser vram
Fbut 2x faster
KDid you update driver manually on massed compute?
Fyep
Falso i had to prevent kohya to overwrite it
Fmessage.txt4.22KB
Fgui.sh file
RHi @Dr. Furkan Gözükara, if today is your birthday and you have unlimited budget, which GPU would you buy?
Ffrom consumer grade rtx 4090
GGuys I am training on my 3090 ti 24GB,(I also own a 48GB A6000 should I use that or its the same result?) Do I have to change anything on instance prompts or parameters? Those are all blank.
All i do is run kohya, load the 24gb json settings, change directories to my tensors, images and then hit train, i get errors:
\Desktop\Kohya\kohya_ss\venv\lib\site-packages\gradio\queueing.py", line 532, in process_events response = await route_utils.call_process_api(
\Desktop\Kohya\kohya_ss\venv\lib\site-packages\gradio\route_utils.py", line 276, in call_process_api
output = await app.get_blocks().process_api(
\Desktop\Kohya\kohya_ss\venv\lib\site-packages\gradio\blocks.py", line 1928, in process_api
result = await self.call_function(
\Kohya\kohya_ss\venv\lib\site-packages\gradio\blocks.py", line 1514, in call_function
prediction = await anyio.to_thread.run_sync(
\Desktop\Kohya\kohya_ss\venv\lib\site-packages\anyio_backends_asyncio.py", line 2177, in run_sync_in_worker_thread
return await future
\Desktop\Kohya\kohya_ss\venv\lib\site-packages\anyio_backends_asyncio.py", line 859, in run
result = context.run(func, args)
\Desktop\Kohya\kohya_ss\venv\lib\site-packages\gradio\utils.py", line 832, in wrapper
response = f(args, **kwargs)
\Desktop\Kohya\kohya_ss\kohya_gui\lora_gui.py", line 1171, in train_model
"network_module": network_module,
\Desktop\Kohya\kohya_ss\kohya_gui\lora_gui.py", line 1171, in train_model
"network_module": network_module,
UnboundLocalError: local variable 'network_module' referenced before assignment
All i do is run kohya, load the 24gb json settings, change directories to my tensors, images and then hit train, i get errors:
\Desktop\Kohya\kohya_ss\venv\lib\site-packages\gradio\queueing.py", line 532, in process_events response = await route_utils.call_process_api(
\Desktop\Kohya\kohya_ss\venv\lib\site-packages\gradio\route_utils.py", line 276, in call_process_api
output = await app.get_blocks().process_api(
\Desktop\Kohya\kohya_ss\venv\lib\site-packages\gradio\blocks.py", line 1928, in process_api
result = await self.call_function(
\Kohya\kohya_ss\venv\lib\site-packages\gradio\blocks.py", line 1514, in call_function
prediction = await anyio.to_thread.run_sync(
\Desktop\Kohya\kohya_ss\venv\lib\site-packages\anyio_backends_asyncio.py", line 2177, in run_sync_in_worker_thread
return await future
\Desktop\Kohya\kohya_ss\venv\lib\site-packages\anyio_backends_asyncio.py", line 859, in run
result = context.run(func, args)
\Desktop\Kohya\kohya_ss\venv\lib\site-packages\gradio\utils.py", line 832, in wrapper
response = f(args, **kwargs)
\Desktop\Kohya\kohya_ss\kohya_gui\lora_gui.py", line 1171, in train_model
"network_module": network_module,
\Desktop\Kohya\kohya_ss\kohya_gui\lora_gui.py", line 1171, in train_model
"network_module": network_module,
UnboundLocalError: local variable 'network_module' referenced before assignment
GAny other settings I need to change before I hit train button?
Pare you using KohyaSS or SD scripts without GUI ? i cant find Checkbox for Flux in Kohya SD3-FLux branch. Only XL and SD 3 ones.
Mloaad the flux model fist, and make sure you're in the sd3 flux branch
V@Dr. Furkan Gözükara i forgot I could extract lora from ckpt 
V@Dr. Furkan Gözükara Traceback (most recent call last):
File "/opt/conda/bin/accelerate", line 8, in <module>
sys.exit(main())
File "/opt/conda/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 47, in main
args.func(args)
File "/opt/conda/lib/python3.10/site-packages/accelerate/commands/launch.py", line 1008, in launch_command
multi_gpu_launcher(args)
File "/opt/conda/lib/python3.10/site-packages/accelerate/commands/launch.py", line 666, in multi_gpu_launcher
distrib_run.run(args)
File "/opt/conda/lib/python3.10/site-packages/torch/distributed/run.py", line 797, in run
elastic_launch(
File "/opt/conda/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/opt/conda/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
/kaggle/working/kohya_ss/sd-scripts/sdxl_train.py FAILED
------------------------------------------------------------
Failures:
<NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
time : 2024-08-22_10:17:12
host : 979028f82d51
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 1222)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================
10:17:13-143688 INFO Training has ended. , what is wrong, i use P100
File "/opt/conda/bin/accelerate", line 8, in <module>
sys.exit(main())
File "/opt/conda/lib/python3.10/site-packages/accelerate/commands/accelerate_cli.py", line 47, in main
args.func(args)
File "/opt/conda/lib/python3.10/site-packages/accelerate/commands/launch.py", line 1008, in launch_command
multi_gpu_launcher(args)
File "/opt/conda/lib/python3.10/site-packages/accelerate/commands/launch.py", line 666, in multi_gpu_launcher
distrib_run.run(args)
File "/opt/conda/lib/python3.10/site-packages/torch/distributed/run.py", line 797, in run
elastic_launch(
File "/opt/conda/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/opt/conda/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
============================================================
/kaggle/working/kohya_ss/sd-scripts/sdxl_train.py FAILED
------------------------------------------------------------
Failures:
<NO_OTHER_FAILURES>
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
time : 2024-08-22_10:17:12
host : 979028f82d51
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 1222)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================
10:17:13-143688 INFO Training has ended. , what is wrong, i use P100
Vanother question i am training sdxl lora, directly, but why the progress is like this
Fdid you switch to flux branch and loaded as lora?
Fi use with gui
