Setting smartindent on makes Vim behave like you describe for me, whereas with nosmartindent (which is what I tend to use) it behaves like you’d prefer it to.
Update: From the docs on smartindent:
When typing ‘#’ as the first character in a new line, the indent for
that line is removed, the ‘#’ is put in the first column. The indent
is restored for the next line. If you don’t want this, use this
mapping: “:inoremap # X^H#”, where ^H is entered with CTRL-V CTRL-H.
When using the “>>” command, lines starting with ‘#’ are not shifted
right.
That seems to be it.
Update: Probably no need to bother with the following… I’ll leave it here for the added informational value. 😉
If setting nosmartindent doesn’t help, perhaps you could use the :set command — with no parameters — to obtain the list of all settings in effect in your Vim session, then paste it somewhere (on Pastie perhaps). There’s a few other options which affect automatic indentation, as far as I remember.