How to delete file of certain extension?

You could try something like this:

DirectoryInfo di = new DirectoryInfo(@"C:\");
FileInfo[] files = di.GetFiles("*.msi")
                     .Where(p => p.Extension == ".msi").ToArray();
foreach (FileInfo file in files)
    try
    {
        file.Attributes = FileAttributes.Normal;
        File.Delete(file.FullName);
    }
    catch { }

Note that I first try to set attributes to “normal”, because File.Delete() fails if file is read-only…
Note the use of GetFiles(): see this link for details.

EDITED:
If you need to get more than one extension you could use this:

public List<FileInfo> GetFiles(string path, params string[] extensions)
{
    List<FileInfo> list = new List<FileInfo>();
    foreach (string ext in extensions)
        list.AddRange(new DirectoryInfo(path).GetFiles("*" + ext).Where(p =>
              p.Extension.Equals(ext,StringComparison.CurrentCultureIgnoreCase))
              .ToArray());
    return list;
}

so you can change part of my answer to

List<FileInfo> files = GetFiles(@"C:\", ".msi", ".zip");