Visual Studio Code Insiders is just a beta version. It includes the latest features and bug fixes, but new features may or may not be stable.
You can have both versions of the program installed, and you can open them simultaneously. You can also keep settings easily synced between the two with the Settings Sync extension.