The Vote of No Confidence lists several missteps and/or missing bits of functionality in the eyes of those who believe they know what features, and their implementations, are proper for ORM/Datamapper frameworks.
If none of those issues are a big deal to you, then I don’t see why you shouldn’t use it. I have yet to hear that it is a buggy mess that blows up left and right. All cautions against it are philosophical. I happen to agree with the vote of no confidence, but that doesn’t mean you should. If you happen to like the way EF works, then go for it. At the same time I’d advise you to at least read the vote of no confidence and try to get a rudimentary understanding of each of the issues in order to make an informed decision.
Outside of that issue and to the heart of your question – You need to keep an eye on the Sql that is being generated so you can make tweaks before a performance problem gets into production. Even if you are using procs on the backend, I’d still look for scenarios where you may be hitting the database too many times and then rework your mappings or fetching scenarios accordingly.