Basically correct. The important difference is that SET is an SQL command while the other two are psql meta-commands – indicated by the leading \.
SET is an SQL command to change run-time parameters. It is executed on the server and has nothing to do with psql per se.
\set is a psql meta-command:
Sets the psql variable name to value […]
Note: This command is unrelated to the SQL command
SET.
\pset is another psql meta-command:
This command sets options affecting the output of query result tables