All the untracked files does not get impacted when you move in between different branches. As they belong to your filesystem and, GIT is unaware that to which branch these files belong. So when you commit those files, then GIT is aware of which files belong to which branch. And can remove or add files in working area based upon your branch.