You should put a newline directly after the \:
sed '3i\
text to insert' file
This is actually the behaviour defined by the POSIX specification. The fact that GNU sed allows you to specify the text to be inserted on the same line is an extension.
If for some reason you need to use double quotes around the sed command, then you must escape the backslash at the end of the first line:
sed "3i\\
text to insert" file
This is because a double-quoted string is processed first by the shell, and \ followed by a newline is removed:
$ echo "abc\
def"
abcdef