Hi, @Coru89. This is a known issue and there is a long thread about it here:
In particular, this post should explain what is happening. We have an issue filed to improve this but I there is no official ETA of when this might change.
If you do want to remove the Large Media add-on you can still use Git LFS. Large Media is optional when using Git LFS, not required. If you do decide to remove it, please read the uninstalling Large Media support guide and then let us know when you are ready to proceed.
If there are other questions, we’ll be happy to answer.