SwipeRefreshLayout with scrollView and Layout above

I found that if you replace your ScrollView with a android.support.v4.widget.NestedScrollView the scrolling behavior will work as you expect it to.