xunit Assert.ThrowsAsync() does not work properly?

You’re supposed to await the result (see xunit’s acceptance tests for examples and alternate forms).

[Fact]
async void Test1()
{
    await Assert.ThrowsAsync<ArgumentNullException>(() => MethodThatThrows());
}

In this specific degenerate case, you could just return the Task that Assert.ThrowsAsync yields without using await; but in that case, the key thing is to yield the Task back to the xUnit framework, i.e.

[Fact]
Task Test1() =>
    Assert.ThrowsAsync<ArgumentNullException>(MethodThatThrows);