You could use the following query:
SELECT table1.id
FROM table1
LEFT JOIN table2
ON table1.id IN (table2.user_one, table2.user_two)
WHERE table2.user_one IS NULL;
Although, depending on your indexes on table2
you may find that two joins performs better:
SELECT table1.id
FROM table1
LEFT JOIN table2 AS t1
ON table1.id = t1.user_one
LEFT JOIN table2 AS t2
ON table1.id = t2.user_two
WHERE t1.user_one IS NULL
AND t2.user_two IS NULL;