I can’t speak to the elegance aspect, but the performance aspect here actually can make a huge difference.
It looks like, from a set of JavaScript testing, that your original method is actually the most efficient one, and contrary to the hypothesizing that the accepted answer linked to, non-CSS Sizzle selectors tend to be much less efficient than method selectors. There’s a reason for that. The $('.answer') can use the browser native getElementsByClass() without having to manually traverse the results. The :first selector complicates that. In this instance, using the sizzle selectors seems to slow the selection by a factor of between 4-5.
I’d argue that, with jQuery, performance should trump elegance, and all evidence (every single browser I’ve tested so far!) seems to indicate that OP’s inelegant solution is the fastest by a fair amount.
Here are the results of the browsers with the most test runs:
(The numbers are ‘operations per second’, so higher numbers are faster, lower numbers are slower.)