Does this mean that the project has been set up to use only the packages and versions that were verified under LTS-3.8?
Exactly. (And if you ever need packages not included in the LTS 3.8 set you can specify them through the extra-deps section of stack.yaml. stack will grab them from Hackage and install them separately for your project, without affecting the LTS snapshot or your other projects.)
If I now want to start a new project and want to use the latest LTS version with the new project, how do I tell stack to do that by default?
Use e.g. stack new projectname --resolver=lts-3.11 to set the resolver for a new project. stack defaults to the most recent LTS snapshot you are already using, and so once you have a project with the latest LTS future new projects will follow suit.
What about if I want to “upgrade” an older project to use a new LTS version?
Change the resolver field in the stack.yaml file. The change will take effect the next time you build the project.