I believe that it is mostly a matter of personal taste whether you should use RGB or HEX. Personally I use HEX, because it feels more convenient – it is easier to scan when reading the code and easier to copy from your preferred image editor. Also for colors like white (#fff), black (#000) it could save you a few bytes (not that it would make much difference though).
In the end it is really just two different representations of the same thing, and from a performance perspective I don’t think it matters. At least there are loads of other things you can do to increase performance, that will have a greater impact on performance.