Question: If I launch massedcompute with 2 a6000, can I run two trainings at the same time on that m
Question: If I launch massedcompute with 2 a6000, can I run two trainings at the same time on that machine? @Furkan GΓΆzΓΌkara SECourses
Xno train your 2048px images at 1024x1024 with bucket
but also train them at 512x and 768x
at least for me provides better results
but also train them at 512x and 768x
at least for me provides better results
οΏ½oh, never messed with those settings
Fexactly same way
Fjut dataset matters
Fi run 8 trainings on a single machine 
F8 gpus
οΏ½hm, but when starting the second isntance of kohya, should I use the same command in the terminal? or navigate to kohya ss folder in apps and run ./gui.sh, I went with option 1 while the first training already had started, hope I don't ruin anything
οΏ½you select the same folder for output for the 2nd training? diffusion models inside swarmui? or will that cause conflict
SThanks
Fas long as they are on different port
Fit will work
Falso just set different gpu id
Fin accelerator
οΏ½yes, I found that option, set gpu id 1 on the second training
οΏ½both are running now, I went with a different folder for the 2nd training, just diffusion_models2 to be sure they don't get mixed up
οΏ½Running smoothly 
Fye use different image folders
For 1 overwrites others cache
οΏ½yeah, luckily I thought so before I started the 2nd one
οΏ½Uploaded checkpoints and pics using the huggingface script, but having all those pics in the same directory as the models became very annoying, how to I delete all pics from the huggingface directory?
Fsadly i dont know how to delete all try llms
οΏ½ok, I don't really need hugging for pics anyways, it's really for models
οΏ½Getting error using cropper on the newest version of ultimate img batch crop/resize, it can't save the files...using massedcompute. Write Error
Fon windows i didnt have errors
οΏ½one of the files had format jiff, might that be the reason for the error?
Fpngs working?
οΏ½I used an earlier version of your program on my pc instead of massed, it completed the process, and just ignored the jiff file
Fit is jiff or jfif?
Fi dont know jiff format
οΏ½looks like the new version doesnt ignore it, but stops the whole thing
οΏ½ah yeah, jfif probably
Fplease verify extension for me
Fso i can try to fix
οΏ½ok sec
οΏ½it is jfif
οΏ½there was also a webp pic in the folder, but that format works fine
Fon windows jfif works
Fi wonder if your image is broken
Fthe error is yolo not supporting
F'pfm', 'mpo', 'jpg', 'jpeg', 'webp', 'tif', 'dng', 'png', 'bmp', 'heic', 'tiff'
οΏ½ok thank you Dr.
Fi am gonna make a fix
οΏ½I am missing the ae.safetensors for flux dev, is this the right one? https://huggingface.co/black-forest-labs/FLUX.1-schnell/blob/main/ae.safetensors says flux schnell, but the vae is the same, ye?
οΏ½tried use the download models again, but it starts downloading flux dev etc. all over again
οΏ½Got error on start of training, I have not gotten this error before. done 20 trainings and no problems starting:
"024-12-09 12:23:35 INFO Loading state dict from flux_utils.py:330
/home/Ubuntu/Downloads/t5xxl_fp16
.safetensors
Traceback (most recent call last):
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/library/utils.py", line 366, in load_safetensors
state_dict = load_file(path, device=device)
File "/home/Ubuntu/apps/kohya_ss/venv/lib/python3.10/site-packages/safetensors/torch.py", line 313, in load_file
with safe_open(filename, framework="pt", device=device) as f:
safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/flux_train.py", line 849, in <module>
train(args)
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/flux_train.py", line 222, in train
t5xxl = flux_utils.load_t5xxl(args.t5xxl, weight_dtype, "cpu", args.disable_mmap_load_safetensors)
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/library/flux_utils.py", line 331, in load_t5xxl
sd = load_safetensors(ckpt_path, device=str(device), disable_mmap=disable_mmap, dtype=dtype)
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/library/utils.py", line 368, in load_safetensors
state_dict = load_file(path) # prevent device invalid Error
File "/home/Ubuntu/apps/kohya_ss/venv/lib/python3.10/site-packages/safetensors/torch.py", line 313, in load_file
"
"024-12-09 12:23:35 INFO Loading state dict from flux_utils.py:330
/home/Ubuntu/Downloads/t5xxl_fp16
.safetensors
Traceback (most recent call last):
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/library/utils.py", line 366, in load_safetensors
state_dict = load_file(path, device=device)
File "/home/Ubuntu/apps/kohya_ss/venv/lib/python3.10/site-packages/safetensors/torch.py", line 313, in load_file
with safe_open(filename, framework="pt", device=device) as f:
safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/flux_train.py", line 849, in <module>
train(args)
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/flux_train.py", line 222, in train
t5xxl = flux_utils.load_t5xxl(args.t5xxl, weight_dtype, "cpu", args.disable_mmap_load_safetensors)
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/library/flux_utils.py", line 331, in load_t5xxl
sd = load_safetensors(ckpt_path, device=str(device), disable_mmap=disable_mmap, dtype=dtype)
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/library/utils.py", line 368, in load_safetensors
state_dict = load_file(path) # prevent device invalid Error
File "/home/Ubuntu/apps/kohya_ss/venv/lib/python3.10/site-packages/safetensors/torch.py", line 313, in load_file
"
οΏ½found it I think, the t5xxl was 9.0 GB, that means it hadn't completed the download...