Performance can be profiled. Though in this case of so few items, you may need to run the code millions of times to actually get worthwhile differences.
I will say this: Queue<T> will expose your intent more explicitly, people know how a queue works.
A list being used like a queue is not as clear, especially if you have a lot of needless indexing and RemoveAt(magicNumber) code. Dequeue is a lot more consumable from a code maintenance point of view.
If this then gives you measurable performance issues, you can address it. Don’t address every potential performance issue upfront.