I would use the following approach in the Dockerfile
RUN echo "Some line to add to a file" >> /etc/sysctl.conf
That should do the trick. If you wish to replace some characters or similar you can work this out with sed by using e.g. the following:
RUN sed -i "s|some-original-string|the-new-string |g" /etc/sysctl.conf
However, if your problem lies in simply getting the settings to “bite” this question might be of help.