1) How can I make sure the test works?
Use assertAlmostEqual, assertNotAlmostEqual.
From the official documentation:
assertAlmostEqual(first, second, places=7, msg=None, delta=None)
Test that first and second are approximately equal by computing the difference, rounding to the given number of decimal places (default 7), and comparing to zero.
2) Is it possible to do it without testing for an approximate value?
Esentially No.
The floating point issue can’t be bypassed, so you have either to “round” the result given by vec.normalize or accept an almost-equal result (each one of the two is an approximation).