How can I cancel Task.WhenAll?

Use TaskCompletionSource<T> to create a task for some asynchronous condition that does not already have an asynchronous API. Use CancellationToken.Register to hook the modern CancellationToken-based cancellation system into another cancellation system. Your solution just needs to combine these two.

I have a CancellationToken.AsTask() extension method in my AsyncEx library, but you can write your own as such:

<System.Runtime.CompilerServices.Extension> _
Public Shared Function AsTask(cancellationToken As CancellationToken) As Task
  Dim tcs = New TaskCompletionSource(Of Object)()
  cancellationToken.Register(Function() tcs.TrySetCanceled(), useSynchronizationContext := False)
  Return tcs.Task
End Function

Usage is as you expected:

Await Task.WhenAny(Task.WhenAll(taskCollection), cancellationToken.AsTask())