Basically, to upgrade a create-react-app project, all you need to do is update the react-scripts module to the latest version and update your app to be compatible with any breaking changes in react-scripts.
Run yarn upgrade --latest react-scripts, rebuild your app, and everything should more-or-less work, barring any breaking changes.