General Question: When using create-react-app, and running npm run build, do I only need to upload the "build" folder as a manual deploy?

Yeah that’s all that you need to do. To fix the 404 caused by React, you need to do this (second redirect in that post):