There are a few strategies which may work, some of which could be combined.
STRATEGY 1:
If you are using the Unix Makefiles generator with CMake, then simply invoking make from the build output dir that corresponds to the subdirectory you want to build should largely accomplish what you describe. Targets from other directories won’t get built unless they are needed by a target for the subdirectory you are building in (or a subdirectory below that). So while you do run CMake from the root directory, you still only build just the subdirectory(s) you want.
Pros: Easy, requires no changes to your CMakeLists.txt files.
Cons: Only works with Unix Makefiles generator, requires all parts of the source tree to be processed by CMake whether you want to build them or not.
STRATEGY 2:
As mentioned by @Emil in his comment to your question, you could simply just build the specific target(s) you want from the top level of your build output directory. This assumes you know the relevant targets you want to build though, or put another way, it requires you to have a good understanding of what targets are provided by which subdirectories.
Pros: Flexible, works with any CMake generator, requires no changes to your CMakeLists.txt files.
Cons: Requires some knowledge of what each subdirectory provides, requires all parts of the source tree to be processed by CMake whether you want to build them or not.
STRATEGY 3:
In the top level CMakeLists.txt file, you could make the add_subdirectory() call for each subdirectory depend on an option variable. This would allow you to turn inclusion of each subdirectory on or off individually, giving you precise control over what gets built. You could build everything, just one subdirectory or a set of subdirectories (more flexible than your requirements, but potentially useful).
Conditionally including a subdirectory with an option variable would typically be done something like this:
option(BUILD_SUBDIR1 "Enable building subdir1" ON)
option(BUILD_SUBDIR2 "Enable building subdir2" ON)
if(BUILD_SUBDIR1)
add_subdirectory(subdir1)
endif()
if(BUILD_SUBDIR2)
add_subdirectory(subdir2)
endif()
If some subdirectories depend on others, then the logic for whether to add a subdirectory or not would need to account for that (i.e. implement dependencies between options). It shouldn’t be hard to do if required.
Pros: Flexibility to build precisely whatever set of subdirectories you want in one build, CMake only has to process those subdirectories you are interested in (saves time for huge project trees and allows you skip problematic parts of the project tree) and works for any CMake generator.
Cons: You need separate build directories if you want to build different subdirectories individually. You can’t simply cd to a different part of your build output directory structure and run a build from there, for example. You still have just one source tree, but you have multiple build output trees. In practice, this may not be an issue depending on what you ultimately want to be able to do (I regularly have multiple build dirs for a given source tree, but that’s usually for Debug and Release builds rather than different sets of CMake options). Also requires an understanding of dependencies between subdirectories. Requires mild changes to top level CMakeLists.txt file.
Hopefully one of the above options or some combination thereof gives you some ideas for how to ultimately achieve what you are aiming for. I had a quick look at the top level CMakeLists.txt in the github project you linked to and it looks like it already makes use of options like in strategy 3, just not quite to turn on/off individual subdirectories.