I’ve had a similiar problem. Not sure if that helps you, but you might try to install gems with rvmsudo gem install [gemname] instead of just doing gem install [gemname] or sudo gem install [gemname].
I try to explain this a bit because of the upvotes:
This basically is a path issue. If you set up gems via gem install, they mostly likely will be in another directory (e.g. /usr/local/) compared to gems set up via bundler (where you can override it with --path). Using sudo may also set them up into another directory since sudo starts a subshell which has a different environment then. This is why you have rvmsudo.
If you want to see the differences, compare $PATH, $GEM_HOME, $GEM_PATH and $BUNDLE_PATH when echoing directly, with sudo and with rvmsudo.