TcpListener is a convenient wrapper for TCP communications. This allows you to use TcpClient for accepted connections–although you can accept sockets instead of clients to use Socket instead of TcpClient. You can do the same thing with Socket; but you have to deal with some of the TCP-specifics (like SocketType.Stream, ProtocolType.Tcp). TCP is a stream-based protocol and TcpClient recognizes that by letting you do stream communications by providing a stream with TcpClient.GetStream(). Socket is at a higher different level and needs to support many different protocols like UDP that aren’t stream based.
TcpClient.GetStream returns a NetworkStream object that is suitable for SslStream; so, it should be much less work than using Socket directly. The documentation for SslStream details using TcpListener and TcpClient for SSL communications.