By default, ViewPager recreates the fragments when you swipe the page. To prevent this, you can try one of three things:
1. In the onCreate() of your fragments, call setRetainInstance(true).
2. If the number of fragments is fixed & relatively small, then in your onCreate() add the following code:
mViewPager = (ViewPager)findViewById(R.id.pager);
mViewPager.setOffscreenPageLimit(limit); /* limit is a fixed integer*/
3. Use a FragmentPagerAdapter as part of your ViewPager.
If I remember correctly, the second option is more promising. But I urge you to try all three and see which of them work.