I eventually figured this out, so here’s how you can do it, at least if using React.
You need to define the variables in one file and export them.
// Variables.js
export const FONTSIZE_5 = '20px';
Then you need to import those variables into each component file.
// Button.js
import * as palette from './Variables.js';
Then you can use the variables in your styled-components like this:
const Button = styled.button`
font-size: ${palette.FONTSIZE_5};
`;