You can use GNU iconv:
$ iconv -f UTF-8 your_file -o /dev/null; echo $?
Or with older versions of iconv, such as on macOS:
$ iconv -f UTF-8 your_file > /dev/null; echo $?
The command will return 0 if the file could be converted successfully, and 1 if not. Additionally, it will print out the byte offset where the invalid byte sequence occurred.
Edit: The output encoding doesn’t have to be specified, it will be assumed to be UTF-8.