Is SqlConnection.Close() need inside a Using statement?

There is no need to Close (or Dispose) here; the using block will take care of that for you.

As stated from Microsoft Learn:

The following example creates a SqlConnection, opens it, [and] displays some of its properties. The connection is automatically closed at the end of the using block.

private static void OpenSqlConnection(string connectionString) 
{
    using (SqlConnection connection = new SqlConnection(connectionString))
    {
        connection.Open();
        Console.WriteLine("ServerVersion: {0}", connection.ServerVersion);
        Console.WriteLine("State: {0}", connection.State);
    } 
}