For single color foreground color
label.setForeground(Color.RED)
For multiple foreground colors in the same label:
(I would probably put two labels next to each other using a GridLayout
or something, but here goes…)
You could use html in your label text as follows:
frame.add(new JLabel("<html>Text color: <font color="red">red</font></html>"));
which produces: