How can I make a non-breaking space in reStructuredText?

You need the unicode directive, but it can only be used in substitutions. So you need to define a substitution like this:

.. |nbsp| unicode:: 0xA0 
   :trim:

and then use it like this:

xx |nbsp| xx

:trim: is there to get rid of those spaces around the substitution.