Workaround (thanks to this comment!)
COPY <tablename> FROM <filename> WITH CSV DELIMITER E'\t' QUOTE E'\b' NULL AS '';
So basically specifying a quote character that should never be in the text, but that’s pretty ugly.
I’d much prefer it if there was in fact a way to turn off quote processing altogether.