Adding a netlify.toml file to the root and then change publish directory to build was the issue. Working just fine now!