Update: The recently released NuGet 2.0 also may require uninstalling an older version of NuGet first.
From the NuGet 2.0 Release Notes: (http://docs.nuget.org/docs/release-notes/nuget-2.0)
Known Installation Issue
If you are running VS 2010 SP1, you might run into an installation error when attempting to upgrade NuGet if you have an older version installed.
The workaround is to simply uninstall NuGet and then install it from the VS Extension Gallery. See http://support.microsoft.com/kb/2581019 for more information, or to go directly to the VS hotfix.
Note: If Visual Studio won’t allow you to uninstall the extension (the Uninstall button is disabled), then you likely need to restart Visual Studio using “Run as Administrator.”