You can use the addModelError
ModelState.AddModelError(key,message)
when you use that, it will invalidate the ModelState so isValid will return false.
Update
after seeing the comment to @Pieter’s answer
If you want to exclude an element from affecting the isValid() result, you can use the ModelState.Remove(field) method before calling isValid().