I dug up the solution from here:
If you post to the /contact-form route (which is a known static HTML page), and not the /success route (which is handled by next.js), it should work.
I had to manually add a form.html that contains a static HTML version of the form so that the build could find it. Then I had to modify the ‘fetch’ path parameter.
It would be great if the Netlify Team could document this in the official Netlify with Next JS guide! 