I needed to install MSBuild separately, then after restarting the agent the variable was there.
(From MSDN article)
MSBuild is now installed as part of Visual Studio rather than as part of the .NET Framework. The current MSBuild version number is 12.0. If you want to install MSBuild separately, download the installation package from MSBuild Download.
edit: the answer link was broken. Here is the link for Microsoft Build Tools 2015 (most recently until today):
https://www.microsoft.com/en-us/download/details.aspx?id=48159