You can use the following to see the tags that exist locally but not in the specified remote:
git show-ref --tags | grep -v -F "$(git ls-remote --tags <remote name> | grep -v '\^{}' | cut -f 2)"
Note that git ls-remote shows both the annotated tag and the commit it points to with ^{}, so we need to remove the duplicates.
An alternative is to use the --dry-run/-n flags to git push:
git push --tags --dry-run
This will show what changes would have been pushed, but won’t actually make these changes.