Where is the correct place to store my application specific data?

Question 2:
I suggest using a subfolder in Environment.SpecialFolder.CommonAppData (maps to C:\ProgramData on Windows7 by default). This is a hidden folder.

Question 3:
Put those files into Environment.SpecialFolder.AppData(maps to C:\Users\[USERNAME]\AppData\Roaming by default, hidden folder), if you expect that the user does not intend to backup / modify those. Some games also put their save games into Environment.SpecialFolder.MyDocuments, probably because it is easier for users to find them there.

Example code:

var directory = Environment.GetFolderPath(Environment.SpecialFolder.AppData);


using (FileStream fs = File.Create(Path.Combine(directory, "myAppDirectory", "myFile.txt")))
{
    // write data               
}

For a complete list of special folders on Windows follow the link

SIDENOTES

• Users are allowed to move around those directories, so make sure you use the code provided above
• There is a bug in Windows 7 x64 regarding CommonAppData directory and the bug gets more severe in Windows 8 x64 CP. I’ve blogged about this: problems after moving CommonAppData directory on Windows 7 x64 and Windows 8 x64