Hi, at the moment I’m attempting to test two branches using a split test, and I’m always seeing an OUTDATED version of the production branch without being able to ever see the B version, despite seeing each respective branch’s updated versions when I deploy and publish individually with the split test disabled. This happened once before and resolved itself overnight somehow when I disabled it and re-enabled it the next day. I’m experiencing it again today so I’m raising the flag here.
Site: https://ecp-qa.netlify.app/
If I enable the split test, with prod branch “featureTest_develop” set to 100%, with “featureTest_deveop-b” set to 0%, I always see an outdated version of it from at least a few days ago.
This happened once before, and I disabled the split test and re-enabled it the next day, and then I got my updated branch.
My troubleshooting has included:
- clearing cache and deploying each updated version of the branch.
- swapping prod branch and other branch
- changing both branches to published, confirming the new versions are indeed the most recent versions.
- removing nf_ab manually from browser before enabling the split test
- manually setting nf_ab to either branch name (which has worked for the previous split test, no longer worked here)
At this point I’m at a loss of what to do besides give it time. Not sure if there’s something in the background that eventually enables split test to update to the most recent branch and adjust the nf_ab’s cookie value, but this is a blocker on immediately getting a test up and running.
UPDATE:
did nothing and tried to duplicate the steps above at the end of the work day yesterday, and it continued to not work.
Like I said, something happened overnight, and now when I enable the split test, it works as intended. There’s nothing in the documentation that says this is expected behavior.
Something’s either expiring or updating in that time. Not sure what.
Another update:
Without changing anything, we’re again seeing an old version of the prod branch with no a/b test functionality.