You’ll need to reencode. Set x264’s keyint parameter to 5*fps and disable scenecut. If your fps is 24 for example :
ffmpeg -i <input> -vcodec libx264 -x264-params keyint=120:scenecut=0 -acodec copy out.mp4
This is obviously not optimal for quality but it’ll match your demand.
Edited to change no-scenecut to scenecut=0, as per sigh-boy suggestion.