Software Engineering Courses (SECourses)•11mo ago

``` 1xGPU FP32 vs FP16 FP32 (float) (Non Tensor) A100 SXM4 80G 19.49 TFLOPS A100 80G 1

1xGPU FP32 vs FP16

FP32 (float) (Non Tensor)
    A100 SXM4 80G 19.49 TFLOPS
    A100 80G      19.50 TFLOPS
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    H100          51.22 TFLOPS 
    H100 NVL      60.00 TFLOPS
    H100 SXM5     66.91 TFLOPS 
    H200          66.91 TFLOPS 
    4090          82.58 TFLOPS 
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS


FP16 (float) (Non Tensor)
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    A100 80G      77.97 TFLOPS
    A100 SXM4 80G 77.97 TFLOPS
    4090          82.58 TFLOPS
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS
    H100          204.9 TFLOPS
    H100 NVL      248.3 TFLOPS
    H100 SXM5     267.6 TFLOPS
    H200          267.6 TFLOPS

1xGPU FP32 vs FP16

FP32 (float) (Non Tensor)
    A100 SXM4 80G 19.49 TFLOPS
    A100 80G      19.50 TFLOPS
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    H100          51.22 TFLOPS 
    H100 NVL      60.00 TFLOPS
    H100 SXM5     66.91 TFLOPS 
    H200          66.91 TFLOPS 
    4090          82.58 TFLOPS 
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS


FP16 (float) (Non Tensor)
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    A100 80G      77.97 TFLOPS
    A100 SXM4 80G 77.97 TFLOPS
    4090          82.58 TFLOPS
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS
    H100          204.9 TFLOPS
    H100 NVL      248.3 TFLOPS
    H100 SXM5     267.6 TFLOPS
    H200          267.6 TFLOPS

1xGPU FP32 vs FP16

FP32 (float) (Non Tensor)
    A100 SXM4 80G 19.49 TFLOPS
    A100 80G      19.50 TFLOPS
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    H100          51.22 TFLOPS 
    H100 NVL      60.00 TFLOPS
    H100 SXM5     66.91 TFLOPS 
    H200          66.91 TFLOPS 
    4090          82.58 TFLOPS 
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS


FP16 (float) (Non Tensor)
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    A100 80G      77.97 TFLOPS
    A100 SXM4 80G 77.97 TFLOPS
    4090          82.58 TFLOPS
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS
    H100          204.9 TFLOPS
    H100 NVL      248.3 TFLOPS
    H100 SXM5     267.6 TFLOPS
    H200          267.6 TFLOPS

1xGPU FP32 vs FP16

FP32 (float) (Non Tensor)
    A100 SXM4 80G 19.49 TFLOPS
    A100 80G      19.50 TFLOPS
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    H100          51.22 TFLOPS 
    H100 NVL      60.00 TFLOPS
    H100 SXM5     66.91 TFLOPS 
    H200          66.91 TFLOPS 
    4090          82.58 TFLOPS 
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS


FP16 (float) (Non Tensor)
    3090          35.58 TFLOPS
    A40           37.42 TFLOPS
    A6000         38.71 TFLOPS
    A100 80G      77.97 TFLOPS
    A100 SXM4 80G 77.97 TFLOPS
    4090          82.58 TFLOPS
    L40           90.52 TFLOPS
    A6000 Ada     91.06 TFLOPS
    L40S          91.61 TFLOPS 
    5090          104.8 TFLOPS
    H100          204.9 TFLOPS
    H100 NVL      248.3 TFLOPS
    H100 SXM5     267.6 TFLOPS
    H200          267.6 TFLOPS

1nfern0•2/25/25, 4:29 PM

that 5090 performance

computersheepOP•2/25/25, 4:29 PM

Added L40

Nneo their pc is always booting and never ready to launch

computersheepOP•2/25/25, 4:45 PM

We had one server causing this to happen frequently over the last week, I suspect you ran into it. We stopped new rentals on it, and refunded a ton of VM rental attempts that went to it.

Nneo i think it is the updaters of the programmes that you got on desktop, cause some...

computersheepOP•2/25/25, 4:48 PM

We are working on some improvements around this. I know connecting to the VM before its fully ready can cause a problem, but if you continue to run into this and have error messages we can look at we'd appreciate if you send them them to us so we can be certain its something on our radar.

Nneo their pc is always booting and never ready to launch

computersheepOP•2/25/25, 4:50 PM

I know sometimes its takes a really long time to start and we are still working on speeding it up and making it more consistent. If a VM truly just doesn't start and you let us know we are pretty good about refunding them.

Furkan Gözükara SECourses•2/25/25, 4:54 PM

i just found a way to reduce 1.3B model for 6GB and 14B model for 10GB GPUs. i think it will be very reasonable time for 5 second videos working on it