I found the solution:
preview_path: '{{dirname}}'
Was the correct line, which I found via this thread.
Hope it helps other HUGO users in the future!
EDIT: I also had to add the specific directory manually, so in my case:
preview_path: 'blog/{{dirname}}'
I suppose the work-around here is to change each instance per collection to reflect which directory to correctly point to.