Unfortunately,
android:focusable="false"
android:focusableInTouchMode="false"
doesn’t work for ImageButton.
I finally found the solution here. In your layout xml for those items, add
android:descendantFocusability="blocksDescendants"
to the root view.
It works perfectly for a ListView that has ImageButtons. According to official reference, blocksDescendants means that the ViewGroup will block its descendants from receiving focus.