‘yield’ enumerations that don’t get ‘finished’ by caller – what happens

I am just left with a leaked connection.

No, you’re not.

When does the finally get invoked?

When the IEnumerator<T> is disposed, which First is going to do after getting the first item of the sequence (just like everyone should be doing when they use an IEnumerator<T>).

Now if someone wrote:

//note no `using` block on `iterator`
var iterator = Foo().GetEnumerator();
iterator.MoveNext();
var first = iterator.Current;
//note no disposal of iterator

then they would leak the resource, but there the bug is in the caller code, not the iterator block.