A/B-TEST

Small change makes a big difference, new A/B-test shows

It’s well known that when writing texts for the web, we need the strongest chain of words possible to inspire the most engagement. This usually means cutting sentences down to make them crisp and to the point – something we know also applies to responsible gambling communication. In a new A/B-test, performed on La Française des jeux’ gambling site, this was confirmed once again.

When the player enters Playscan he or she is greeted with a scale of risk – ranging from low risk to high risk, a headline and a describing text about their risk level. The customised feedback is inspired by motivational interviewing, a communication method that is often used to treat gambling problems. The main purpose of the text is to make the player interested and to click the recommendation from Playscan.

Together with the responsible gambling team at La Française des jeux, Playscan ran an A/B-test on their gambling site to find what would happen if we reduced the number of words in the feedback to the player. The purpose of the test was to find out if that had an effect on the number clicks on the recommendation. Another goal of the test was to study if player’s tendencies to agree or disagree with their risk analysis could be affected by a different way of describing risk, with a new tone?

Testing a new shorter headline

In this test, 95% of the players got new headlines (A) with less text. A control group of 5% of the players got the old headlines (B). In the A-version we used the raw risk level information as the sole subject of the message: You are 2.5 times more likely than a low risk player to have an excessive gambling problem.

This new version was tested against the old (B) with longer and much more explanatory text.

Version A
Version A
Version B
Version B

 

Shorter feedback yields more clicks to recommendations and eventually higher rates of completed self-tests

When analysing the result of the test we found the click-rate of the (A)-group that received shorter feeback was 24 % higher. However, when looking at at-risk players only, we find that group clicks the recommendation at the same rate (~27 %).

In total we find that the (A)-group had completed at least one self-test at a higher rate (11 %) compared to the (B)-group (8.6 %).

Also, after a first self-test is completed, at-risk players get a new main recommendation, such as setting a limit or taking a break from gambling. We find that the (A)-group is a bit more likely to click (46 %) than the (B)-group (41 %).

Do players with shorter feedback agree more with their risk analysis?

We also studied if the at-risk players believe their risk assesment is accurate – which one might assume to be true, since we saw a higher click rate from these players. As it turns out, the (B)-group tends to agree significantly more (mean=4.76 out of 9) than with the (A)-group (mean=2.62 out of 9).

Players are asked to answer the question: How well does your risk assessment fit you?

These findings are interesting and make a good example of how quantitative analysis of data sometimes generate new questions. The overall test result indicates that shortening the feedback messages can increase the effect of RG-communication. It also raises further questions, that can be answered through using other methods, such as user testing and interviews.

 

 

 

 

 

 

 

 

Leave a Reply

Your email address will not be published. Required fields are marked *