Best Practice for Using Brand To Split Tests?


We love to try out the Netlify’s brand split test feature. However, in practice, the tests can run anywhere between 1 - 4 weeks+, we are worried about holding up the development process. (since any bug fix, features etc needs to happen on all branches), and also, it looks like if we want to run more than 1 split tests at a time, the setup can be complicated.

I’m curious if anyone actively use the split test features on large team or scale and how they are overcoming these issues.

thx in advance!

Hi Jim,

No ability to run more than one split test at a time, so that’s an easy answer at least!

Re: having to “backport” fixes between branches while the test is running: yes, that is the price of doing business at present. When we have a long running test, it becomes the person who merges into master’s job to ALSO merge into the split-branch(es), since you’ll need that redeploy on all branches to update the logo or fix the typo.

If you were thinking of a different way to implement split tests, (besides allowing multiple which I do have an open feature request for and have added your name to), how might you change that setup? We’re already considering directory-specific splits (so you wouldn’t need to merge changes to /index.html if you’re only testing /products/*), but if you have some other dreams, it’s an opportune time to mention them since you have our ear :slight_smile:

thx for getting back to me. I haven’t given this a lot of thought; Folder level split would be nice. Or just simply URL level split would solve a lot of our testing needs - your URL rewrite/redirect feature is already super robust, (we can already do a lot of conditional redirect), but if we can also split the traffic on URL level and percentages, that would be super useful for us on testing as well.

thanks for that! I’ve captured it in a feature request so we can follow up in this thread if it comes to pass.