The session middleware implements generic session functionality with in-memory storage by default. It allows you to specify other storage formats, though.
The cookieSession middleware, on the other hand, implements cookie-backed storage (that is, the entire session is serialized to the cookie, rather than just a session key. It should really only be used when session data is going to stay relatively small.