Optimizing your company’s pricing strategy can be one of your most powerful growth levers, whether that means adjusting the prices of your offerings or changing how you communicate your pricing packages. However, creating poor user experiences around pricing can cost your company potential customers and erode trust, so it is essential to be careful when refining your strategy.
Common research methods such as surveys and focus groups can be valuable in understanding whether your prices are optimal, but as many growth experts will attest, using A/B testing can be a safe and effective method of identifying customers’ pricing preferences and driving revenue gains, provided that you navigate the unique complexity of pricing experiments with a clear understanding of how to maximize results.
In this article, we will provide guidance on how to thoughtfully design and analyze pricing tests to ensure you maintain a great user experience while driving monetization.
Tactic: Proactively identify experiment challenges before building
There are many ways you can experiment with pricing, but the choices you make about what type of experiment you run will likely define the level of technical investment and organizational alignment you’ll need to arrange. We generally categorize our pricing experiments into two buckets: “front-end” and “full-stack.”
Front-end pricing experiments test changes to the way you message or display prices, and are visual changes as opposed to actual price or plan adjustments. These experiments tend to be relatively easy to build from a technical perspective and require less upfront investment.
Full-stack experiments test changes to the prices and plans of your product. Because they aren’t simply visual changes but actual price and plan adjustments, they require significantly more technical investment and long-term planning. As part of your technical scoping, you’ll need to ensure that the changes you make to pricing are reflected not only on your website, but also in your product experience (e.g. if you’re selling a SaaS product and your product has a billing summary page), in your internal billing and accounting systems, and in your CRM, among other systems your company may need to keep in mind.
While full-stack experiments require a significantly higher level of investment than front-end experiments, they can also provide immense value to your business. Like any other experiment you’re planning, it is important to make an assessment of the revenue your experiment could yield if successful before shying away from making these commitments — big wins can justify big investments!
Considerations across different tax laws and currencies
One challenge of pricing experiments is that as you span different regions and countries, you become subject to different tax laws and currency exchange rates. These stipulations may affect the designs of your experiments and ultimately the success rate. For example, if you are running a test about “charm pricing” and change the price of a product from USD $20 to USD $19.99, this change coupled with the effects of the currency exchange rate may result in an odd price in other regions (e.g. AUD $27.43). These types of unintentional changes may conflict with your experimental hypotheses and render the results invalid. Therefore, when testing prices, it is important to keep in mind regional tax and currency differences (on top of cultural differences) that may affect your tests. Segmenting your audiences for your pricing experiments can help correct for this.
Be careful when charging different prices to customers in the same region
Charging people noticeably different prices for the same product, especially if they are in the same region, can result in a terrible customer experience. While testing something such as rounding $19.99 to $20 on webpages may be negligible, testing $15 versus $20 is risky. Imagine if people discovered they were being charged more for the same product!
In an effort to determine what people’s optimal willingness to pay is, some businesses may bundle price changes with different product packages, but note that this is not a true a/b test. The important thing to remember is that when it comes to pricing tests, there is a fine line between what is okay to test and what is not worth doing for the sake of maintaining customer loyalty.
Tactic: Design with your users’ end-to-end experience in mind
People can have strong reactions to pricing changes (both positive and negative), so it’s important to take into consideration the entire user experience when running pricing tests. Here are a couple of examples to consider:
Preserve visitors’ choices on your website
When your visitors have plan choices they can configure on your website, it is important to make sure that the pricing on all webpages reflect those configurations so as to a) not force visitors to frequently reset their preferences and b) not confuse visitors by showing them irrelevant or different prices.
For example, let’s say you want to run an experiment that allows users to toggle between monthly or annual billing plans on the homepage of your website. If users choose the monthly option on your homepage, it will be a better experience for them if the website is built in such a way that this choice is preserved across all pages they visit where pricing is displayed.
Maintain pricing consistency across channels
Pricing experiments are often a full-funnel experience — the changes you make in a front-end or full-stack experiment will have effects beyond your web experience (ex: ad campaigns, receipt emails, in-product billing summaries, subscription renewals). Consistency across your experiences whenever possible is imperative to maintaining trust among customers. A great experiment hypothesis can be undone by testing conditions that create confusion and distrust among users, and pricing is a major opportunity to gain or lose that trust. To get the most out of your pricing experiments and keep customers happy, make sure that people in each of your experiment variants are being offered the same prices across all channels before and after purchase.
Keep your Sales and Customer Support teams in the loop
Especially when it comes to pricing, it is essential to keep your Sales and Customer Support teams informed of experiments. Pricing experiments, while powerful levers for your business, can also cause unique reactions from users that require preparation. Sales and Customer Support teams can relay valuable qualitative feedback on the results of your experiment while managing questions and requests from customers.
For example, let’s say you’re running this front-end experiment where you’re changing the pricing display of your annual subscription to reflect the cost per month instead of the full annual cost (even though both plans when billed annually are the same price):
This experiment can be great at reducing sticker shock and driving revenue for your business while making customers happy (and these days it is actually a commonly used form of subscription price messaging), but some viewers might misinterpret your new messaging to say that this product is billed monthly at this price. They might even completely misread the messaging and think your product only costs $100 per year!
If your Sales and Customer Support teams are aware of this experiment, they’ll be much better equipped to handle calls where customers are looking to buy the $100 per year subscription that doesn’t actually exist. Also, if this confusion results in an increase in refund requests, your support team will be grateful for the opportunity to prepare in advance for this spike in support tickets, especially if your business has to manually process refunds. Last but not least, working closely with Sales and Customer Support creates a great feedback loop for experiments, which can help drive insights for future experiment iterations.
Tactic: Analyze beyond your standard top of funnel metrics
Analyze your full-funnel
When analyzing the results of your pricing experiment, it is especially important to make sure that you measure the impact beyond initial sign-ups. Experiments that change a user's perception of the price of your offerings can attract customers with different price sensitivities than before. This can impact their likeliness to refund their purchase, renew their contract, or expand their usage.
Let’s re-visit the front-end experiment we discussed earlier and assume we saw these results:
Seems like a winner, right? But what if we expand our analysis to include refunds and then we see this:
Because of enormous refund rates in the test variant, what appeared to be a clear win is now looking more like a potential loss.
In order to give yourself a more complete perspective of the impact an experiment has on revenue, make sure to track your experiment over a longer time horizon in order to detect any changes in upsell or renewal behavior and include those in your impact calculations.
Analyze performance across all of your offerings
In addition to analyzing the the impact of experiments on downstream metrics, it is also important to dig into how they may affect purchases across your billing plans (i.e., monthly vs. annual subscriptions) or other offerings. It is often the case that one of your product’s biggest competitors is in fact another one of your products. As such, when running an experiment that changes the perception of price on one of your offerings, it is important to measure whether there is an impact on the conversions of other products to ensure that the success you’re observing in one line of business isn’t cannibalizing others.
Cannibalization isn’t necessarily a negative result — if the combined revenue across your products is higher in your test variant than in your control, then that change might be worth keeping if your goal is to increase revenue. Even better, you can segment the results of your experiment to see whether there are audiences you can target with each of your offerings to help people find the right product or plan, maximizing revenue.
Running pricing experiments can be a powerful growth lever for your company, but like any other experiment, your results can only be as good as the preparation that goes into them. By proactively identifying the technical challenges of your experiments, designing with your end-to-end customer experience in mind, and rigorously analyzing your data throughout your funnel and across your offerings, you can run pricing experiments to drive sustainable growth while maintaining a great customer experience.