This appears to happen when there are multiple network adaptors present for the host and the priority of said adaptor is misconfigured. Run the following to display a table of your network adaptors:
Get-NetIPInterface -AddressFamily IPv4 | Sort-Object -Property InterfaceMetric -Descending
You want your primary adaptor (in my case wifi) to have the lowest InterfaceMetric.
Set-NetIPInterface -InterfaceAlias 'Wi-Fi' -InterfaceMetric 1
My network adaptor table for reference:

See this github thread for more information:
https://github.com/docker/for-win/issues/2760#issuecomment-430889666