Just want to add to the discussion, If you set:
"window.zoomLevel": "your number",
In my case I started with zero, you can use that as sort of a baseline:
"editor.fontSize": 14,
"editor.tabSize": 2,
"terminal.integrated.fontSize": 14,
"window.zoomLevel": 0,
"[Log]": {
"editor.fontSize": 14
},
And then tweaked until I was happy.
E.g. you want to increase the sidebar font size keeping editor font size unchanged: set "window.zoomLevel": 0.5 and "editor.fontSize": 18. Look at the result. Play with both values to get the result you like.