As explained in this blog, the Loaded event is fired when ever a control is about to be rendered (i.e. added to the visual tree).
There are several controls that would cause your control to be loaded/unloaded multiple times. For example, the native WPF TabControl only renders the content of the selected tab. So when you select a new tab, the content of the previously selected tab is unloaded. If you click back to the previously selected tab, then it’s content is reloaded.
One work around is to use a Boolean to flag whether you have already initialized your control, as suggested by others. Alternatively, you may be able to use the Initialized event instead.