Next.js now supports AMP. They provide multiple approaches (AMP-only & AMP-hybrid) to solve this problem. Since you can enable AMP on page level you can slowly migrate to a full AMP website.
Example
// pages/about.js
export const config = { amp: true }
export default function AboutPage(props) {
return <h3>My AMP About Page!</h3>
}
This way you can support both. React client rendering and pure AMP.
Docs Next.js AMP