The usual approach is to add a hash to the script and other assets filenames depending on the time or content. So when before you had script.js now it will be script.[contenthash].js. The content hash will be different every time you change the content of the script.
Now when a user requests the index.html of your app it will reference the script including the individual content hash. If users previously loaded the script.abc123.js and now the index.html references a script.cba321.js the browser will know that it has not previously seen this file and load it. This way users have the current version of your scripts and other assets (js, css, images, …). For this to work however it is important that users always load the recent version of the index.html.
This approach is not react specific but an universal approach and best practice. It is however used and suggested by create-react-app (see: https://github.com/facebook/create-react-app).
Since it would be quite tedious to do this manually all the time there are a lot of scripts and strategies available to use module bundlers to achieve this goal. E.g. by using WebPack: https://webpack.js.org/guides/caching/
There are other approaches like setting caching response headers for script files, which however I can not recommend in this case.