Please push the commit removing .lfsconfig to the upstream repository. This will trigger a new build.
Please keep in mind that you might want to run the command below after deleting the .lfsconfig file:
git lfs push --all origin
This will sync the local repo’s LFS objects with the Git LFS service at the original Git host. It pushes any new binary blobs to their LFS service which may have been added after you moved the LFS service to Large Media.
Please let us know if there are any issues with the new build or deployed site.