That is correct – Sizzle (jQuery’s selector engine) behaves the same way as CSS selectors. CSS and Sizzle selectors are evaluated right-to-left, and so #foo a will find all a nodes, then filter those by nodes that descend from #foo.
You improve this by ensuring that your leaf selectors have a high specificity, usually by giving them a class or ID.