Update for 2020 – BinaryPrimitives should now be preferred over BitConverter. It provides endian-specific APIs, and is less allocatey.
byte[] bytes = BitConverter.GetBytes(i);
although note also that you might want to check BitConverter.IsLittleEndian to see which way around that is going to appear!
Note that if you are doing this repeatedly you might want to avoid all those short-lived array allocations by writing it yourself via either shift operations (>> / <<), or by using unsafe code. Shift operations also have the advantage that they aren’t affected by your platform’s endianness; you always get the bytes in the order you expect them.