Conditional UPDATE in MySQL

Unfortunately it’s not very clear what you want to get in the end, but here is how you could correctly use conditional SET in your UPDATE

UPDATE relation 
   SET name1 = CASE WHEN userid1 = 3 THEN 'jack' ELSE name1 END,
       name2 = CASE WHEN userid2 = 3 THEN 'jack' ELSE name2 END
WHERE (userid1 = 3 AND userid2 = 4) 
   OR (userid1 = 4 AND userid2 = 3);

Here is SQLFiddle demo.