The ‘File name too long’ is indeed an error coming from your system. It is not a restriction in Git. I don’t think there’s a way of testing the maximum branch name lengths of other systems, like GitHub’s, without experimenting – although I don’t see why you would want to know…
.
Source: the error mentioned is thrown on line 291 of branch.c. “File name too long” is the standard description for the ENAMETOOLONG error.