Role of docker-in-docker (dind) service in gitlab ci

what is the difference between the docker:dind and the docker:latest images?

• docker:latest contains everything necessary to connect to a docker daemon, i.e., to run docker build, docker run and such. It also contains the docker daemon but it’s not started as its entrypoint.
• docker:dind builds on docker:latest and starts a docker daemon as its entrypoint.

So, their content is almost the same but through their entrypoints one is configured to connect to tcp://docker:2375 as a client while the other is meant to be used for a daemon.

why are both the service and the docker image needed […]?

You don’t need both. You can just use either of the two, start dockerd as a first step, and then run your docker build and docker run commands as usual like I did here; apparently this was the original approach in gitlab at some point. But I find it cleaner to just write services: docker:dind instead of having a before_script to setup dockerd. Also you don’t have to figure out how to start & install dockerd properly in your base image (if you are not using docker:latest.)

Declaring the service in your .gitlab-ci.yml also lets you swap out the docker-in-docker easily if you know that your runner is mounting its /var/run/docker.sock into your image. You can set the protected variable DOCKER_HOST to unix:///var/run/docker.sock to get faster builds. Others who don’t have access to such a runner can still fork your repository and fallback to the dind service without modifying your .gitlab-ci.yml.