I just wanted to plus one this as it would be very useful for our team as well. In our case it is related to this other thread (Automatically deploy branches that match name pattern - #22 by knoxjeffrey) . We are running
[[plugins]]
package = "netlify-plugin-chromium"
but don’t need this plugin to run on our CMS build because we won’t be running tests. There are also a few other plugins that I can see this being useful for.