I ended up fixing this on my end. Just posting a solution in case it might happen to someone else.
I ended up deleting node_modules and the yarn.lock to perform a fresh yarn install. After running that I started receiving weird issues in Jest about esModuleInterop. I then cleared my jest cache using yarn test --clearCache and re-ran the tests. After performing that my tests passed. I pushed the changes to my branch and netlify no longer complained about the loader.
Seems like my yarn.lock got messed up and rebuilding it seemed to fix it.
Thanks for everyone that looked at this.