Use scp with the -p option.
-p Preserves modification times, access times, and modes from the original file.
Example command copying a file from local to remote server:
scp -p /home/mylocaldata/test.txt remote.example.com:/home/remote_dir
Note that this will not preserve user and group only permission flags (rwx and such).