As of vscode 1.59 you can enable using the / separator even on Windows for the Copy Relative Path command.
Explorer: Copy Relative Path Separator set to /.
explorer.copyRelativePathSeparator
auto(default): Uses operating system specific path separation character/: Use slash as path separation character\\: Use backslash as path separation character
See https://github.com/microsoft/vscode/issues/56279

For the same ability to designate the path separator for the copy full path (tab context menu: Copy Path) command, there is an issue with a PR awaiting resolution: Allow to configure the which slash is used when copying a path.