The Many Faces of Multivariate Testing
Aug 14, 2014 6:02 PM PT
When Facebook and OKCupid recently admitted they had been tinkering with headlines and content to see how users reacted, many consumers were outraged -- but the business community just shrugged.
What the companies were doing was simply a form of multivariate, or A/B, testing, albeit in the case of OKCupid -- which changed content entered by users -- it was particularly aggressive. The practice itself has been around for years and is considered de rigueur among email marketers. However, it's only recently that companies have begun to realize the importance of such testing for their mobile apps.
This is not to say that mobile app developers have been throwing any old app at the proverbial wall to see what sticks. Indeed, the amount of data available for mobile apps is enormous and extraordinarily deep, all the way down to the device level.
Mobile Ads' Bright Future
One only has to look at mobile ad tech company Fiksu, which just reported annual revenue of US$100 million, largely fueled by demand for data-driven mobile ads.
"Fiksu's roots have always been in data and now, in our fourth year, the sheer mass of data we have accumulated delivers massive scale and unmatched targeting to businesses and agencies seeking to optimize their mobile advertising results," bragged Micah Adler, CEO of Fiksu. "The future is incredibly bright for data-driven mobile advertising."
Multivariate testing, though, is something else. Yes, there is plenty of industry knowledge accruing about app design and fonts and colors and so forth. But further testing is necessary to fine-tune these "rules" and customize them for a company's specific audience and its own goals. Behavioral research and targeting also fall under the multivariate umbrella.
For instance, only 17 percent of affluent consumers are likely to respond to a text message, especially from a high-end brand, the Luxury Institute has found.
Fortunately, a number of new offerings are coming to market to make it easier than ever to determine the best message and the best design in the mobile environment.
For example, Appboy's last month launched a multivariate testing tool for developers to test app messaging. The goal was to find out how various messages would affect such metrics as conversion rate and revenue generated.
Another recent example is SiteSpect, which in June rolled out an SDK that performs A/B testing and behavioral targeting for native iOS apps.
The Same Cycle for Wearables
It will be interesting to see how many companies apply these technologies -- in particular, the behavioral targeting -- to the next wave of mobile devices expected to swamp consumers: wearables.
In some cases -- namely, apps for wearable devices designed with a medical purpose in mind -- I suspect this will be more of a focus than with general mobile apps.
Consider the MediSafe project, one of the first mobile health apps based on Google's new Android Wear platform, which is designed to keep patients, such as diabetics, compliant with their pharmaceutical schedules.
Of course, any number of other apps -- not to mention nagging family members -- can accomplish the same thing. However, MediSafe built its app and interface by applying technological behavioral tools. It zeros in on a user's individual reason for forgetting to a pill or inject insulin, and then it creates a compliance program tailored to that situation.
MediSafe is the product of extensive multivariate testing, and although the result hasn't been, say, an increase in ad clicks or app downloads, it does boast its own impressive bona fides: MediSafe users have an average medication adherence rate of 86 percent, according to the company, compared to 50 percent of the general population, according to App Developer magazine.