Software Engineering Courses (SECourses)•13mo ago

Uploaded checkpoints and pics using the huggingface script, but having all those pics in the same di

Uploaded checkpoints and pics using the huggingface script, but having all those pics in the same directory as the models became very annoying, how to I delete all pics from the huggingface directory?

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭Uploaded checkpoints and pics using the huggingface script, but having all those...

Furkan Gözükara SECourses•12/8/24, 5:12 PM

sadly i dont know how to delete all try llms

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/8/24, 5:15 PM

ok, I don't really need hugging for pics anyways, it's really for models

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 10:08 AM

Getting error using cropper on the newest version of ultimate img batch crop/resize, it can't save the files...using massedcompute. Write Error

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭Getting error using cropper on the newest version of ultimate img batch crop/res...

Furkan Gözükara SECourses•12/9/24, 10:55 AM

let me test

Furkan Gözükara SECourses•12/9/24, 10:55 AM

on windows i didnt have errors

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 10:56 AM

one of the files had format jiff, might that be the reason for the error?

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭one of the files had format jiff, might that be the reason for the error?

Furkan Gözükara SECourses•12/9/24, 10:56 AM

yep

Furkan Gözükara SECourses•12/9/24, 10:56 AM

pngs working?

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 10:57 AM

I used an earlier version of your program on my pc instead of massed, it completed the process, and just ignored the jiff file

Furkan Gözükara SECourses•12/9/24, 10:57 AM

it is jiff or jfif?

Furkan Gözükara SECourses•12/9/24, 10:57 AM

i dont know jiff format

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 10:57 AM

looks like the new version doesnt ignore it, but stops the whole thing

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 10:57 AM

ah yeah, jfif probably

Furkan Gözükara SECourses•12/9/24, 10:58 AM

please verify extension for me

Furkan Gözükara SECourses•12/9/24, 10:58 AM

so i can try to fix

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 10:58 AM

ok sec

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 10:58 AM

it is jfif

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 10:59 AM

there was also a webp pic in the folder, but that format works fine

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭it is jfif

Furkan Gözükara SECourses•12/9/24, 11:07 AM

on windows jfif works

Furkan Gözükara SECourses•12/9/24, 11:07 AM

i wonder if your image is broken

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭ok sec

Furkan Gözükara SECourses•12/9/24, 11:31 AM

the error is yolo not supporting

Furkan Gözükara SECourses•12/9/24, 11:31 AM

'pfm', 'mpo', 'jpg', 'jpeg', 'webp', 'tif', 'dng', 'png', 'bmp', 'heic', 'tiff'

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 11:55 AM

ok thank you Dr.

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭ok thank you Dr.

Furkan Gözükara SECourses•12/9/24, 12:07 PM

i am gonna make a fix

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 12:10 PM

I am missing the ae.safetensors for flux dev, is this the right one? https://huggingface.co/black-forest-labs/FLUX.1-schnell/blob/main/ae.safetensors says flux schnell, but the vae is the same, ye?

ae.safetensors · black-forest-labs/FLUX.1-schnell at main

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 12:15 PM

tried use the download models again, but it starts downloading flux dev etc. all over again

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭I am missing the ae.safetensors for flux dev, is this the right one? https://hug...

Furkan Gözükara SECourses•12/9/24, 12:32 PM

yes

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 12:37 PM

Got error on start of training, I have not gotten this error before. done 20 trainings and no problems starting:
"024-12-09 12:23:35 INFO Loading state dict from flux_utils.py:330
/home/Ubuntu/Downloads/t5xxl_fp16
.safetensors
Traceback (most recent call last):
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/library/utils.py", line 366, in load_safetensors
state_dict = load_file(path, device=device)
File "/home/Ubuntu/apps/kohya_ss/venv/lib/python3.10/site-packages/safetensors/torch.py", line 313, in load_file
with safe_open(filename, framework="pt", device=device) as f:
safetensors_rust.SafetensorError: Error while deserializing header: MetadataIncompleteBuffer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/flux_train.py", line 849, in <module>
train(args)
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/flux_train.py", line 222, in train
t5xxl = flux_utils.load_t5xxl(args.t5xxl, weight_dtype, "cpu", args.disable_mmap_load_safetensors)
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/library/flux_utils.py", line 331, in load_t5xxl
sd = load_safetensors(ckpt_path, device=str(device), disable_mmap=disable_mmap, dtype=dtype)
File "/home/Ubuntu/apps/kohya_ss/sd-scripts/library/utils.py", line 368, in load_safetensors
state_dict = load_file(path) # prevent device invalid Error
File "/home/Ubuntu/apps/kohya_ss/venv/lib/python3.10/site-packages/safetensors/torch.py", line 313, in load_file
"

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 12:38 PM

found it I think, the t5xxl was 9.0 GB, that means it hadn't completed the download...

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭Got error on start of training, I have not gotten this error before. done 20 tra...

Furkan Gözükara SECourses•12/9/24, 12:40 PM

fixed batch pre processor

Furkan Gözükara SECourses•12/9/24, 12:40 PM

model download fgailed

Furkan Gözükara SECourses•12/9/24, 12:40 PM

yes you found accurately

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 12:41 PM

Thank you Dr. Great work as always

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 12:55 PM

Can I use the option RTX A6000 NVLink with 2gpus and run 2 trainings at once?

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭Can I use the option RTX A6000 NVLink with 2gpus and run 2 trainings at once?

Furkan Gözükara SECourses•12/9/24, 12:59 PM

yes for lora

Furkan Gözükara SECourses•12/9/24, 12:59 PM

for fine tune i didnt test probably OOM

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 1:00 PM

OOM? it's 48 GB each

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OOM? it's 48 GB each

Furkan Gözükara SECourses•12/9/24, 1:02 PM

yes

Furkan Gözükara SECourses•12/9/24, 1:02 PM

fine tuning need 80 gb gpus

Furkan Gözükara SECourses•12/9/24, 1:02 PM

when you do multi gpu

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 1:02 PM

I'm talking two separate trainings

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 1:02 PM

on same VM

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 1:02 PM

I already did that with 2 a6000 before, so that works, but I'm unsure what the NVlink means

�🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭I'm talking two separate trainings

Furkan Gözükara SECourses•12/9/24, 1:35 PM

seperate works

Furkan Gözükara SECourses•12/9/24, 1:35 PM

nvlink used when multiple gpu does a thing together

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 1:36 PM

CHat GPT O1-preview gave this answer: "Yes, you can run the two GPUs separately, treating each as its own device. Even though your RTX A6000 GPUs are connected via NVLink—which enables high-bandwidth peer-to-peer communication and memory access between them—they still present themselves as two distinct GPU devices to the system and to frameworks like PyTorch or TensorFlow"

Furkan Gözükara SECourses•12/9/24, 1:37 PM

yep

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 2:54 PM

If I have 1.jpg 1.txt 2.jpg 2.txt, the captions will automatically be read from the txt files in kohya, yes?

�

🍭🎀 𝒜𝓋𝒶 𝐹𝓇𝒾𝑔𝑔 🎀🍭OP•12/9/24, 7:51 PM

is it possible to convert an sdxl lora to a flux lora?

Uploaded checkpoints and pics using the huggingface script, but having all those pics in the same di

Similar Threads