Why is converting between string and float wrong?

You should use "R" format string:

https://msdn.microsoft.com/en-us/library/dwhawy9k(v=vs.110).aspx.

https://msdn.microsoft.com/en-us/library/dwhawy9k(v=vs.110).aspx#RFormatString

“R” or “r” Round-trip Result: A string that can round-trip to an
identical number. Supported by: Single, Double, and BigInteger.
Precision specifier: Ignored.

  float maxFloat = float.MaxValue;
  string s = maxFloat.ToString("R"); // <- "R"
  float result = float.Parse(s);

  bool mustEqual = (maxFloat == result);