You can ignore robots.txt and all the aspx pages in your routing.
routes.IgnoreRoute("{*allaspx}", new {allaspx=@".*\.aspx(/.*)?"});
routes.IgnoreRoute("{*robotstxt}", new {robotstxt=@"(.*/)?robots.txt(/.*)?"});
You might want to ignore the favicon too.
routes.IgnoreRoute("{*favicon}", new {favicon=@"(.*/)?favicon.ico(/.*)?"});
You can adjust the regular expression to exclude paths.
Haacked from the source.