There is no such access-rights-mechanism implemented on GitHub repositories out of the box.
Using a submodule
Using a submodule does the trick, and is pretty much easy to set up. You can give the customers full access to the submodule repository, while you add it to your main project repository.
Note that the StackOverflow question “Using someone else’s repo as a Git Submodule on GitHub” and its answer are also worth reading.
This is in my opinion the most simple and straightforward way to do it.
Using a webhook
If you really don’t want to use submodules, you can still restrict access rights by creating a GitHub webhook that implements this mechanism. The Git SCM documentation explains how to do that part.