Which selector do I need to select an option by its text?

This could help:

$('#test').find('option[text="B"]').val();

Demo fiddle

This would give you the option with text B and not the ones which has text that contains B.

For recent versions of jQuery the above does not work. As commented by Quandary below, this is what works for jQuery 1.9.1:

$('#test option').filter(function () { return $(this).html() == "B"; }).val();

Updated fiddle