I think you need to have factory girl definition like that in Gemfile:
gem 'factory_girl_rails', :require => false
And then you just require it in your spec_helper.rb like that:
require 'factory_girl_rails'
This is the way I’m always using this gem. You don’t need to require it in other places than spec_helper.rb. Your current desired approach is just wrong.