Recently we tried launching an A/B test using Netifly Split Testing but have run into some issues, we’re having difficulty solving. The test appears to be working in that we’re seeing sessions show up in our custom dimension in GA, the issue is that transactions don’t appear to be working. Currently for the last six days, it shows 149 transactions for the main branch and 1 for the test branch – even if the test branch is not as good, it seems highly unlikely that this would happen.
For additional context, our tech stack is Gatsby, Craft and Shopify (for the backend). We’re using Google Tag Manager extensively as well. My guess is that the GTM trigger for transactions is not registering the test branch but I’m really not sure. Can anyone help?
Hi @bartfish. We cannot troubleshoot what your Google Tag Manager behavior is as that is outside the scope of our support. We can and will troubleshoot what Netlify is doing though.
For example, regarding what Netlify is doing, I checked the time window since these two deploys occurred:
Since that time I do see a 50/50 traffic split for those two deploys. I can see the traffic split is working.
If in debugging this, you believe our service is sending an incorrect file, please let us know.
If you see us sending incorrect files, we need to be able to track the HTTP response with this issue. The simplest way to do this is to send us the x-nf-request-id header which we send with every HTTP response.
There more information about this header here:
If that header isn’t available for any reason, please send the information it replaces (or as many of these details as possible). Those details are:
the complete URL requested
the IP address for the system making the request
the IP address for the CDN node that responded
the day of the request
the time of the request
the timezone the time is in
If there are other questions of if there a response which is incorrect, please let us know.