Rails model.valid? flushing custom errors and falsely returning true

In ActiveModel, valid? is defined as following:

def valid?(context = nil)
  current_context, self.validation_context = validation_context, context
  errors.clear
  run_validations!
ensure
  self.validation_context = current_context
end

So existing errors are cleared is expected. You have to put all your custom validations into some validate callbacks. Like this:

validate :check_status

def check_status
  errors.add(:status, "must be YES or NO") unless ['YES', 'NO'].include?(status)
end