TL;DR
Go ahead: move your files and directories around. Just make sure you don’t make any edits to files in the same commit as your directory restructuring.
Why It Works
Git is a content tracker, not a file tracker. If you move/rename files, but make no other changes to the content of those files, then Git just rewrites the tree objects. The file blobs are not changed by directory operations; directory location information is stored separately in tree objects.
Actual rename detection is handled by diffing the blobs and trees, and looking for a configurable percentage of similarity between files. This means that Git doesn’t really store moves or renames directly; it computes them from differences between commits.
The upshot of all this is that your history is not tied to a particular filename or directory structure. This works really well for most use cases, but stands in contrast to systems like Bazaar that track renames as first-class operations.