I solved this by using a placeholder in my Layout and then attaching my Fragment to it at runtime.
Like you, if I instantiated my Fragment within my xml layout then the contents would remain visible after replacing it with another Fragment at runtime.