Using IIS I fixed it inside the Web.config-file, add the following inside <system.webServer>
:
<staticContent>
<remove fileExtension=".woff" />
<remove fileExtension=".woff2" />
<mimeMap fileExtension=".woff" mimeType="application/font-woff" />
<mimeMap fileExtension=".woff2" mimeType="application/font-woff" />
</staticContent>
Notice I remove the extensions first, in case they already exist.