In Erlang, 5 rem 3. gives 2, and -5 rem 3. gives -2. If I understand your question, you would want -5 rem 3. to give 1 instead, since -5 = -2 * 3 + 1.
Does this do what you want?
mod(X,Y) when X > 0 -> X rem Y;
mod(X,Y) when X < 0 -> Y + X rem Y;
mod(0,Y) -> 0.