How do I set up a kind of “belongs_to :through” association without a direct belongs_to?

Use delegate in the model class.

class Team < ActiveRecord::Base
  belongs_to :division
  has_many :players

  delegate :league, to: :division
end

Reference: http://api.rubyonrails.org/classes/Module.html#method-i-delegate