In general, all components in an application should be composed as late as possible, because that ensures maximum modularity, and that modules are as loosely coupled as possible.
In practice, this means that you should configure the container at the root of your application.
- In a desktop app, that would be in the Main method (or very close to it)
- In an ASP.NET (including MVC) application, that would be in Global.asax
- In WCF, that would be in a ServiceHostFactory
- etc.
The container is simply the engine that composes modules into a working application. In principle, you could write the code by hand (this is called Poor Man’s DI), but it is just so much easier to use a DI Container like Windsor.
Such a Composition Root will ideally be the only piece of code in the application’s root, making the application a so-called Humble Executable (a term from the excellent xUnit Test Patterns) that doesn’t need unit testing in itself.
Your tests should not need the container at all, as your objects and modules should be composable, and you can directly supply Test Doubles to them from the unit tests. It is best if you can design all of your modules to be container-agnostic.
Also specifically in Windsor you should encapsulate your component registration logic within installers (types implementing IWindsorInstaller) See the documentation for more details