Use dstack:
>>> np.dstack((A, B)).shape
(480, 640, 4)
This handles the cases where the arrays have different numbers of dimensions and stacks the arrays along the third axis.
Otherwise, to use append or concatenate, you’ll have to make B three dimensional yourself and specify the axis you want to join them on:
>>> np.append(A, np.atleast_3d(B), axis=2).shape
(480, 640, 4)