At least terminal vim stores the previous ten files into ~/.viminfo in the filemarks section. You can use '0, '1, '2, … '9 to jump among them.
(Probably only useful for '0 to get back to the last file you were editing, unless your memory is stronger than mine.)
You can also use the :browse oldfiles command to get a menu with numbers.