As stated, if you want to get 4 as the result, you can do like this:
var nonintersect = array2.Except(array1);
If you want the real non-intersection (also both 1 and 4), then this should do the trick:
var nonintersect = array1.Except(array2).Union( array2.Except(array1));
This will not be the most performant solution, but for small lists it should work just fine.