I think the answers here are missing a vital point – that this behavior is configurable out of the box via release policies – check out the documentation on the castle project site here.
In many scenarios especially where your container exists for the lifetime of the hosting application, and where transient components really don’t need to be tracked (because you’re handling disposal in your calling code or component that’s been injected with the service) then you can just set the release policy to the NoTrackingReleasePolicy implementation and be done with it.
Prior to Castle v 1.0 I believe Component Burden will be implemented/introduced – which will help alleviate some of these issues as well around disposal of injected dependencies etc.
Edit:
Check out the following posts for more discussion of component burden.
The Component Burden – Davy Brions
Also component burden is implemented in the official 2.0 release of the Windsor Container.