Cyclone3 Skin

A/B Testing

From Cyclone3 Wiki

Every regular user is attached randomly into an A/B group. The value is saved into the session variable $main::USRM{'session'}{'AB'} (A/B)

Contents

How to add A/B group into Google Analytics

Modify Google Analytics code in default.body file by adding a new custom var

<script type="text/javascript">
 var gaJsHost = ...
 document.write(unescape("...
</script>
<script type="text/javascript">
try {
 var pageTracker = _gat._getTracker("UA-??????-?");
 pageTracker._setCustomVar(1,"A/B Test","<$main::USRM{'session'}{'AB'}>",2); 
 pageTracker._trackPageview();
} catch(err) {}</script>

New type of google analytics invocation code:

<script type="text/javascript"> 
 var _gaq = _gaq || [];
 _gaq.push(['_setAccount', 'UA-??????-?']);
 _gaq.push(['_setCustomVar',1,"A/B Test","<$main::USRM{'session'}{'AB'}>",2]);
 _gaq.push(['_trackPageview']);
 (function() {
  var ga = document.createElement('script'); ...
  ga.src = ('https:' == .....
  var s = ....
 })();
</script>

To show A/B Test groups as segments in Google Analytics:

  • Advanced Segments
  • Create new custom segment
  • add rule Visitor -> Custom Variable (Value 1) matches exactly "A"
  • Name segment: A/B Testing (A)
  • Create Segment
  • repeat process for group B
  • To display click on Dashboard to Custom segments on right side and select the segments A and B.

Load custom CSS

<style type="text/css" media="screen">
 @import url(<$tom::H_css>/abtest_<$main::USRM{'session'}{'AB'}>.css);
</style>

abtest_A.css

.abtest_B {
 display: none;
}

abtest_B.css

.abtest_A {
 display: none;
}

Show new button only for group B and hide old button for group A

<button class="abtest_A" ...>
<button class="abtest_B" ...>

Customizing modules

if ($main::USRM{'session'} && $main::USRM{'session'}{'AB'} eq "B")
{
  # testing new features 
}
else
{
  # old feature
}

To-Do

  • Add to module param '-if'