Add a CFBundleIconFiles key of type Array to your Info.plist. The array should contain 2 string items: the filenames of the two icons. The OS will then automatically choose the correct icon for each platform based on their pixel dimensions.
You can keep the CFBundleIconFile key and have it point to the icon file for the iPhone for 3.0/3.1 compatibility.
Update May 2010: Apple now has Technical Note explaining this in great detail: Technical Q&A QA1686 – App Icons on iPad and iPhone