Hey y’all
So, Netlify’s split testing is perfect for code changes to provide behaviors such as feature flags and beta access (such as demonstrated by Phil Hawksworth), but when it comes to content changes, things start getting messy and quite limited…
Let’s say we want to allow editors of a headless CMS that powers a Netlify site to create variations of each page, individually. The homepage might have 3 variations, the about page 2, and so on. Currently, to pull this off without relying on client-side techniques - which, as we know, hurt performance, UX and SEO -, you’d have to automatically create repository branches for each combination of variations, and then hit the Netlify API to register the split tests. In the example above, here’s what we’d have:
- 3 variations for home (H1, H2, H3)
- times 2 variations for the about page (A1, A2)
- equals 6 total branches, 5 new (H1A1, H1A2, H2A1…)
- plus, there’s the whole issue of figuring out how to split between these variations: if H1 should represent 40% for the homepage, H1A1 + H1A2 should equal 40% of the traffic, but the distribution between them is dependent between A1 and A2. It’s easy to see how the complexity scales when we’re dealing with 3+ sets of pages.
- finally, orchestrating all of the API calls to the git provider and to Netlify would be a lot of work, and managing so many branches in the repository would be any developer’s nightmare.
Overall, relying on the split testing functionality for this use case doesn’t seem to be an option, unless I’m missing something, of course (any feedback appreciated!). My second thought was to take a look into the capabilities of redirects and rewrites. As powerful as those are (thank you for that, by the way!), there’s no way to run any logic that would allow us to attribute variations. So, here’s my suggestion for a redirects feature that would make Netlify that much more powerful for ambitious, performance-critical JAMstack websites:
# Variations for the homepage
/ 40% / 200!
/ 60% /home-2 200!
# Variations for about
/about 10% /about-2 200!
/about 60% /about-3 200!
/about 30% /about 200!
# As we already have key=id for denoting query strings,
# {number.decimal}% seems like a good option for variations
Let me know what you think of this suggestion This is based on my personal preferences and experience, without paying much attention to technical details, such as backward compatibility, so I’m 100% open for feedback and adjustments.
Netlify is an amazing product and this particular facet of a website is the last remaining puzzle piece for web projects at our agency, thank you for being awesome and keep up the good work