You can use the vector-effect property set to non-scaling-stroke, see the docs. Another way is to use transform(ref).
That will work in browsers that support those parts from SVG Tiny 1.2, for example Opera 10. The fallback includes writing a small script to do the same, basically inverting the CTM and applying it on the elements that shouldn’t scale.
If you want sharper lines you can also disable antialiasing (shape-rendering=optimizeSpeed or shape-rendering=crispEdges) and/or play with the positioning.