You’re not missing anything. RestTemplate#exchange(..) is the appropriate method to use to set request headers.
Here’s an example (with POST, but just change that to GET and use the entity you want).
Here’s another example.
Note that with a GET, your request entity doesn’t have to contain anything (unless your API expects it, but that would go against the HTTP spec). It can be an empty String.