If you care about SEO, one of the ways that angular.io was able to solve this problem (at least with Google anyway) is by using noindex meta tag “to indicate soft-404 status which will prevent crawlers from crawling the content of the page”. Apparently it can be added to the document via JavaScript.
Alternatively, using JavaScript, you can redirect to a page that will respond with an actual HTTP 404 status code. Google understands JavaScript redirects just fine. Your original /does-not-exist page, when redirected to /404-error?from=does-not-exist, will be associated with the 404 status code returned by the server. The URL structure does not matter, only the status code and the redirect are important here.
Your other options are SSR (Nuxt.js, Next.js, Angular Universal, etc) or pre-rendering (prerender.io, puppeteer, etc) which Google calls dynamic rendering where you respond to search bot requests with a pre-rendered version while human users get your normal client-side rendered app.