sure, id like to keep training without issues in future
sure, id like to keep training without issues in future
FThere is also one another thing
FUnet training
FI am still not sure how all works yet
FI am waiting 5060 hopefully for an upgrade
FI hope it be over 20 gb 
24 would be sweet
24 would be sweet
NI am training right now: Theres an issue with sanity prompt. I write "a photo of ---- by Tomer Hanuka. But, as soon as 640 steps, the style dissapears. And it gives me realistic photos instead. The model dont look overtraining, face are still not right. I have 32 sample images: portraits and some mid a full body..
FNvidia may also make 24 hb rtx 4060 that would be also a good upgrade
FCan you try half learning rate
NIt would be nice and more affordable for sure
FHalf learning rate can help a lot
F0.000001
NNum batches each epoch = 32
Num Epochs = 100
Batch Size Per Device = 2
Gradient Accumulation steps = 2
Total train batch size (w. parallel, distributed & accumulation) = 4
Text Encoder Epochs: 75
Total optimization steps = 3200
Total training steps = 6400
Resuming from checkpoint: False
First resume epoch: 0
First resume step: 0
Lora: False, Adam: True, Prec: bf16
Gradient Checkpointing: False
EMA: True
UNET: True
Freeze CLIP Normalization Layers: False
LR: 8e-06
Num Epochs = 100
Batch Size Per Device = 2
Gradient Accumulation steps = 2
Total train batch size (w. parallel, distributed & accumulation) = 4
Text Encoder Epochs: 75
Total optimization steps = 3200
Total training steps = 6400
Resuming from checkpoint: False
First resume epoch: 0
First resume step: 0
Lora: False, Adam: True, Prec: bf16
Gradient Checkpointing: False
EMA: True
UNET: True
Freeze CLIP Normalization Layers: False
LR: 8e-06
FYou are using 0.000008
Nok! i am going to try it
FTry half
FIf still try half again
FI saw your batch size
FAnd gradent
FIt is still experimental to increase learning rate
FSo trying half should help
NBatch Size
1
Gradient Accumulation Steps
1
and learning 0.000001?
1
Gradient Accumulation Steps
1
and learning 0.000001?
C
FYou can test this. Also you can test batch 2 gradient 2 learn 0.4
FBut I first suggest batch 1 gradient 1
FBecause I think implementation could be bugged
Nok
Chmm, @Furkan Gözükara SECourses can you install tokenizer without problems? I think I have to reinstall torch again
FDidn't test recently
Cwont work
FI can test in 30 min
Cdont worry, I got it!
C
C12 images
Cbecause we do not have a big dataset, we dont need files and all files use same 75 tokens
CHere are the class images being generated with the sampe prompt.
Theoretically, if I fine tune with proper weights
Theoretically, if I fine tune with proper weights
Cthe same prompt should in next generation have more red hair
Cand at least, not disturb the images
Cooooh
Fyou are really making experimetns
Fthat others didnt :d
CIs that bad?
C
CIm trying
Fvery low loss looks good
Fcurrently i am focused instruct-pix2pix
CWell I hope I keep you entertained 
Cactually
Cthose are REALLY low numbers
C
Cthats what normally people get