Because 32 bits are able to represent numbers up to 232 − 1 = 4294967295 = 4 GiB − 1 and therefore address up to 232 individual bytes which would be 4 GiB then.
There are ways to circumvent that, though. For example using PAE even a 32-bit operating system can support more memory. Historically this has most commonly been used on servers, though. Also, the non-server Windows SKUs don’t support it. By now all that is moot, though, given that 64-bit CPUs, OSes and driver support are commonplace.