chmod can actually do this itself; the X symbolic permission means “execute, if it makes sense” which generally means on directories but not files. So, you can use:
chmod -R u=rwX,go=rX /path/to/htdocs
The only potential problem is that if any of the plain files already have execute set, chmod assumes it’s intentional and keeps it. If this is a potential problem and you have the GNU version of chmod (i.e. you’re on Linux), you can get it to remove any stray execute permissions like this:
chmod -R a-x,u=rwX,go=rX /path/to/htdocs
Unfortunately, this trick doesn’t work with the bsd (/macOS) version of chmod (I’m not sure about other versions). This is because the bsd version applies the X permission based on “the original (unmodified) mode”, i.e. whether it had any execute bits before the a-x modification was done (see the man page).