How to check if $? is not equal to zero in unix shell scripting?

Put it in a variable first and then try to test it, as shown below

ret=$?
if [ $ret -ne 0 ]; then
        echo "In If"
else
        echo "In Else"
fi

This should help.

Edit: If the above is not working as expected then, there is a possibility that you are not using $? at right place. It must be the very next line after the command of which you need to catch the return status. Even if there is any other single command in between the target and you catching it’s return status, you’ll be retrieving the returns_status of this intermediate command and not the one you are expecting.