I didn't expect the 14b to be released too

11nfern0 I didn't expect the 14b to be released too

F

Furkan Gözükara SECourses•2/25/25, 2:21 PM

they released both amazing

F

Furkan Gözükara SECourses•2/25/25, 2:22 PM

making the gradio app installer

1

1nfern0OP•2/25/25, 2:31 PM

FFurkan Gözükara SECourses making the gradio app installer

1

1nfern0OP•2/25/25, 2:35 PM

I'd be interested in helping to test

11nfern0 Click to see attachment

F

Furkan Gözükara SECourses•2/25/25, 2:35 PM

trying to make installer

F

Furkan Gözükara SECourses•2/25/25, 2:46 PM

dmn it uses flash attention forcedly

F

Furkan Gözükara SECourses•2/25/25, 2:47 PM

i installed weird testing again

F

Furkan Gözükara SECourses•2/25/25, 2:58 PM

ok downloading models fingers crossed

F

Furkan Gözükara SECourses•2/25/25, 3:02 PM

generating video

F

Furkan Gözükara SECourses•2/25/25, 3:02 PM

1

1nfern0OP•2/25/25, 3:04 PM

looks promising

F

Furkan Gözükara SECourses•2/25/25, 3:08 PM

after prompt enhance it doesnt deload model lol

F

Furkan Gözükara SECourses•2/25/25, 3:09 PM

F

Furkan Gözükara SECourses•2/25/25, 3:23 PM

vae decoding taking forever

F

Furkan Gözükara SECourses•2/25/25, 3:35 PM

excellent

F

Furkan Gözükara SECourses•2/25/25, 3:35 PM

1

1nfern0OP•2/25/25, 4:01 PM

ohh damn, that's only the 1.3B model

11nfern0 ohh damn, that's only the 1.3B model

F

Furkan Gözükara SECourses•2/25/25, 4:06 PM

yes

F

Furkan Gözükara SECourses•2/25/25, 4:06 PM

downloading 14b model for image to video

F

Furkan Gözükara SECourses•2/25/25, 4:06 PM

but i dont know if will fit vram

F

Furkan Gözükara SECourses•2/25/25, 4:06 PM

probably not

F

Furkan Gözükara SECourses•2/25/25, 4:06 PM

and i can use 5090 yet

F

Furkan Gözükara SECourses•2/25/25, 4:07 PM

but i am gonna make installer for runpod and massed compute too

1

1nfern0OP•2/25/25, 4:07 PM

I think it's 50+Gb unquantised.

F

Furkan Gözükara SECourses•2/25/25, 4:07 PM

dmn probably wont fit

F

Furkan Gözükara SECourses•2/25/25, 4:08 PM

i dont see quant option on the repo

1

1nfern0OP•2/25/25, 4:08 PM

but that is FP32. FP16 should be managable

11nfern0 but that is FP32. FP16 should be managable

F

Furkan Gözükara SECourses•2/25/25, 4:10 PM

did you check their pipeline?

F

Furkan Gözükara SECourses•2/25/25, 4:10 PM

where it is cast :d

1

1nfern0OP•2/25/25, 4:10 PM

No, haven't had a chance to take a look, stuck in work meetings atm

1

1nfern0OP•2/25/25, 4:13 PM

I've just sold my 4090 too so will need to wait for torch to work with 5090

11nfern0 I've just sold my 4090 too so will need to wait for torch to work with 5090

F

Furkan Gözükara SECourses•2/25/25, 4:13 PM

i kept 3090 just for this casue

FFurkan Gözükara SECourses i kept 3090 just for this casue 😄

1

1nfern0OP•2/25/25, 4:14 PM

I've still got a 3090 but no space in the case. 5090 is taking too many slots

11nfern0 I've still got a 3090 but no space in the case. 5090 is taking too many slots �...

F

Furkan Gözükara SECourses•2/25/25, 4:15 PM

this is how i use

F

Furkan Gözükara SECourses•2/25/25, 4:17 PM

image to video failed 13b

11nfern0 I've still got a 3090 but no space in the case. 5090 is taking too many slots �...

F

Furkan Gözükara SECourses•2/25/25, 4:18 PM

my case

FFurkan Gözükara SECourses my case

1

1nfern0OP•2/25/25, 4:19 PM

reminds me of my mining days

11nfern0 but that is FP32. FP16 should be managable

C

computersheep•2/25/25, 4:27 PM

1xGPU FP32 vs FP16

FP32 (float) (Non Tensor)
    A100 SXM4 80G 19.49 TFLOPS
    A100 80G      19.50 TFLOPS
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    H100          51.22 TFLOPS 
    H100 NVL      60.00 TFLOPS
    H100 SXM5     66.91 TFLOPS 
    H200          66.91 TFLOPS 
    4090          82.58 TFLOPS 
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS


FP16 (float) (Non Tensor)
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    A100 80G      77.97 TFLOPS
    A100 SXM4 80G 77.97 TFLOPS
    4090          82.58 TFLOPS
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS
    H100          204.9 TFLOPS
    H100 NVL      248.3 TFLOPS
    H100 SXM5     267.6 TFLOPS
    H200          267.6 TFLOPS

1xGPU FP32 vs FP16

FP32 (float) (Non Tensor)
    A100 SXM4 80G 19.49 TFLOPS
    A100 80G      19.50 TFLOPS
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    H100          51.22 TFLOPS 
    H100 NVL      60.00 TFLOPS
    H100 SXM5     66.91 TFLOPS 
    H200          66.91 TFLOPS 
    4090          82.58 TFLOPS 
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS


FP16 (float) (Non Tensor)
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    A100 80G      77.97 TFLOPS
    A100 SXM4 80G 77.97 TFLOPS
    4090          82.58 TFLOPS
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS
    H100          204.9 TFLOPS
    H100 NVL      248.3 TFLOPS
    H100 SXM5     267.6 TFLOPS
    H200          267.6 TFLOPS

1xGPU FP32 vs FP16

FP32 (float) (Non Tensor)
    A100 SXM4 80G 19.49 TFLOPS
    A100 80G      19.50 TFLOPS
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    H100          51.22 TFLOPS 
    H100 NVL      60.00 TFLOPS
    H100 SXM5     66.91 TFLOPS 
    H200          66.91 TFLOPS 
    4090          82.58 TFLOPS 
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS


FP16 (float) (Non Tensor)
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    A100 80G      77.97 TFLOPS
    A100 SXM4 80G 77.97 TFLOPS
    4090          82.58 TFLOPS
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS
    H100          204.9 TFLOPS
    H100 NVL      248.3 TFLOPS
    H100 SXM5     267.6 TFLOPS
    H200          267.6 TFLOPS

1xGPU FP32 vs FP16

FP32 (float) (Non Tensor)
    A100 SXM4 80G 19.49 TFLOPS
    A100 80G      19.50 TFLOPS
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    H100          51.22 TFLOPS 
    H100 NVL      60.00 TFLOPS
    H100 SXM5     66.91 TFLOPS 
    H200          66.91 TFLOPS 
    4090          82.58 TFLOPS 
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS


FP16 (float) (Non Tensor)
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    A100 80G      77.97 TFLOPS
    A100 SXM4 80G 77.97 TFLOPS
    4090          82.58 TFLOPS
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS
    H100          204.9 TFLOPS
    H100 NVL      248.3 TFLOPS
    H100 SXM5     267.6 TFLOPS
    H200          267.6 TFLOPS

1

1nfern0OP•2/25/25, 4:29 PM

that 5090 performance

C

computersheep•2/25/25, 4:29 PM

Added L40

Nneo their pc is always booting and never ready to launch

C

computersheep•2/25/25, 4:45 PM

We had one server causing this to happen frequently over the last week, I suspect you ran into it. We stopped new rentals on it, and refunded a ton of VM rental attempts that went to it.

Nneo i think it is the updaters of the programmes that you got on desktop, cause some...

C

computersheep•2/25/25, 4:48 PM

We are working on some improvements around this. I know connecting to the VM before its fully ready can cause a problem, but if you continue to run into this and have error messages we can look at we'd appreciate if you send them them to us so we can be certain its something on our radar.

Nneo their pc is always booting and never ready to launch

C

computersheep•2/25/25, 4:50 PM

I know sometimes its takes a really long time to start and we are still working on speeding it up and making it more consistent. If a VM truly just doesn't start and you let us know we are pretty good about refunding them.

F

Furkan Gözükara SECourses•2/25/25, 4:54 PM

i just found a way to reduce 1.3B model for 6GB and 14B model for 10GB GPUs. i think it will be very reasonable time for 5 second videos working on it

Ccomputersheep ``` 1xGPU FP32 vs FP16 FP32 (float) (Non Tensor) A100 SXM4 80G 19.49 TFLOPS...

F

Furkan Gözükara SECourses•2/25/25, 4:54 PM

very nice table

FFurkan Gözükara SECourses very nice table

C

computersheep•2/25/25, 4:55 PM

expensive GPUs make a lot more sense at FP16

1

1nfern0OP•2/25/25, 5:00 PM

can't wait to start playing around with

FFurkan Gözükara SECourses i just found a way to reduce 1.3B model for 6GB and 14B model for 10GB GPUs. i t...