Software Engineering Courses (SECourses)•3y ago

I'm considering fine-tuning Stable Diffusion with 80k clothing images, featuring 5k unique products.

I'm considering fine-tuning Stable Diffusion with 80k clothing images, featuring 5k unique products. I plan to assign a unique token for each apparel item. Is this approach suitable for training? I'm using the EveryDream2 trainer.

FFurkan Gözükara SECourses hello welcome. how much vram your gpu has

AMMAR |COTE|•12/19/23, 12:56 PM

My gpu have 8 gb vram

Motoko_Shi•12/19/23, 1:44 PM

I can't seem to clone this repo https://github.com/williamyang1991/Rerender_A_Video

GitHub

GitHub - williamyang1991/Rerender_A_Video: [SIGGRAPH Asia 2023] Rer...

[SIGGRAPH Asia 2023] Rerender A Video: Zero-Shot Text-Guided Video-to-Video Translation - GitHub - williamyang1991/Rerender_A_Video: [SIGGRAPH Asia 2023] Rerender A Video: Zero-Shot Text-Guided Vid...

Nnannigath I'm considering fine-tuning Stable Diffusion with 80k clothing images, featuring...

Furkan Gözükara SECourses•12/19/23, 1:51 PM

not much

Furkan Gözükara SECourses•12/19/23, 1:52 PM

how many unique apparel item you have?

AAMMAR |COTE|My gpu have 8 gb vram

Furkan Gözükara SECourses•12/19/23, 1:52 PM

8 gb is not good to do training

Furkan Gözükara SECourses•12/19/23, 1:52 PM

your best option is kaggle SDXL dreambooth for free

MMotoko_Shi I can't seem to clone this repo https://github.com/williamyang1991/Rerender_A_Vi...

Furkan Gözükara SECourses•12/19/23, 1:52 PM

ye solving their repo isntall hard

Furkan Gözükara SECourses•12/19/23, 1:52 PM

use my auto installer

Furkan Gözükara SECourses•12/19/23, 1:52 PM

it works

FFurkan Gözükara SECourses how many unique apparel item you have?

nannigathOP•12/19/23, 1:53 PM

I have 5000 for now. I wanted to experiment on this first.

nannigathOP•12/19/23, 1:53 PM

Have an L40 GPU with 48gb vram

Nnannigath I have 5000 for now. I wanted to experiment on this first.

Furkan Gözükara SECourses•12/19/23, 1:54 PM

how are you planning to find 5000 uniqu captions haha :d

nannigathOP•12/19/23, 1:55 PM

Planning to add new tokens to the clip tokenizer, found an option to do that.

Nnannigath Planning to add new tokens to the clip tokenizer, found an option to do that.

Furkan Gözükara SECourses•12/19/23, 1:55 PM

interesting where did you find it?

Furkan Gözükara SECourses•12/19/23, 1:55 PM

it literally tokenize everything

nannigathOP•12/19/23, 1:57 PM

pipe = StableDiffusionPipeline.from_pretrained(model_id, torch_dtype=torch.float16).to("cuda")
pipe.tokenizer.add_tokens(list_of_tokens))
pipe.text_encoder.resize_token_embeddings(len(pipe.tokenizer))

nannigathOP•12/19/23, 1:57 PM

like this

Furkan Gözükara SECourses•12/19/23, 1:59 PM

wow you gonna train with diffusers good luck

nannigathOP•12/19/23, 1:59 PM

since everydream2 uses diffusers i can simply put these lines in the pipeline

Furkan Gözükara SECourses•12/19/23, 1:59 PM

it is a very hard job to get it right

FFurkan Gözükara SECourses wow you gonna train with diffusers good luck 🙂

nannigathOP•12/19/23, 1:59 PM

i am using everydream2

Nnannigath i am using everydream2

Furkan Gözükara SECourses•12/19/23, 1:59 PM

i heard it makes more sense

Furkan Gözükara SECourses•12/19/23, 2:00 PM

i can also give you private consultation for different approach. but it wont be able to bring you certain product

Furkan Gözükara SECourses•12/19/23, 2:01 PM

it would be more general one

Furkan Gözükara SECourses•12/19/23, 2:01 PM

like type of clothing etc

FFurkan Gözükara SECourses i can also give you private consultation for different approach. but it wont be ...

nannigathOP•12/19/23, 2:01 PM

that would be helpful

nannigathOP•12/19/23, 2:02 PM

but is it worth trying this approach?

Nnannigath but is it worth trying this approach?

Furkan Gözükara SECourses•12/19/23, 2:03 PM

i dont think so

Furkan Gözükara SECourses•12/19/23, 2:03 PM

first of all assigning 5000 unique tokens

Furkan Gözükara SECourses•12/19/23, 2:03 PM

would take huge time

Furkan Gözükara SECourses•12/19/23, 2:03 PM

and you dont know if will work or not

Furkan Gözükara SECourses•12/19/23, 2:04 PM

also 80k images you really should use multi gpu

Furkan Gözükara SECourses•12/19/23, 2:04 PM

you can run on runpod really cheap

Furkan Gözükara SECourses•12/19/23, 2:04 PM

or you can wait days :d

FFurkan Gözükara SECourses also 80k images you really should use multi gpu

nannigathOP•12/19/23, 2:09 PM

yeah planning to use multi gpus. I have unique ids for each product so planning to put some random alphabets that is not in the clip vocabulary. I have tried this with 500 products and I got decent results. my concern will the model remember all the tokens when I scale it to 5000

Nnannigath yeah planning to use multi gpus. I have unique ids for each product so planning ...

Furkan Gözükara SECourses•12/19/23, 4:20 PM

with sufficient number of epochs i think should

FFurkan Gözükara SECourses i tried only batch size 1 and getting over 70% speed up

pat._.fenis•12/19/23, 4:41 PM

Still not seeing anywhere near a 70% increase.

25% increase, yes.
70%, no.

How are you equating this percentage ratio?

Ppat._.fenis Still not seeing anywhere near a 70% increase. 25% increase, yes. 70%, no. How...

Furkan Gözükara SECourses•12/19/23, 5:20 PM

what is your gpu

Furkan Gözükara SECourses•12/19/23, 5:28 PM

i am installing tensoRT right now on my new model

FFurkan Gözükara SECourses what is your gpu

pat._.fenis•12/19/23, 5:52 PM

4090

veldonlives•12/19/23, 6:14 PM

i have a ComfyUI workflow question. anybody have a good method of inpainting large images so that the sampler only works on the masked area?

example, i arrive at a large-ish image (3000x3000) by upscaling. I need to fix parts. if i use the Mask Editor and push it through the KSampler (VAEencode > Latent Noise Mask > KSampler) it seems it still works on the entire image rather than just the masked area. In A1111, I recall it just working on the masked area.

How would this be done in ComfyUI?

Ppat._.fenis 4090

Furkan Gözükara SECourses•12/19/23, 6:29 PM

here my difference

Furkan Gözükara SECourses•12/19/23, 6:29 PM