As you can see, the effect you are expecting is present for Float32: julia> rnd64 = rand(Float64, 1000); julia> rnd32 = rand(Float32, 1000); julia> rnd16 = rand(Float16, 1000); julia> @btime $rnd64.^2; 616.495 ns (1 allocation: 7.94 KiB) julia> @btime $rnd32.^2; 330.769 ns (1 allocation: 4.06 KiB) # faster!! julia> @btime $rnd16.^2; 2.067 μs (1 allocation: … Read more
TL;DR: 16-bit floats do exist and there are various software as well as hardware implementations There are currently 2 common standard 16-bit float formats: IEEE-754 binary16 and Google’s bfloat16. Since they’re standardized, obviously anyone who knows the spec can write an implementation. Some examples: https://github.com/ramenhut/half https://github.com/minhhn2910/cuda-half2 https://github.com/tianshilei1992/half_precision https://github.com/acgessler/half_float Or if you don’t want to use … Read more