To Price Test or Not – We've Got Answers For You

For each of your products there is always a pricing sweet spot that maximises profit (not necessarily conversion rates). Unfortunately, without testing, it’s very difficult to determine how to optimally price your products to your advantage.

Is price testing the answer?

Split testing live with real customers carries risks.

Although it’s not illegal, customers who access your site from multiple devices or who clear cookies regularly may spot the inconsistencies in your pricing. It may also interfere with your paid search, shopping engine or email ads that contain prices.

To answer the question of whether you should price test or not, here are the opinions of some of the leading online thinkers on the subject:

Carlos del Rio, Director of Conversion Analysis & Digital Strategy with Unbounce says:

"The short answer is yes. But, there is a major caveat. Don’t do public testing. There are many places where it is illegal to arbitrarily vary your price, and any specific segmentation of that group can lead to other ethical/legal issues. Price testing should be handled like market research.

Choose a controlled group and run them through the test scenario and offer them all the same end compensation for their time."

Justin Rondeau, Editor & Evangelist of WhichTestWon.com says:

"I’ve never really liked the idea of price testing, and each day we see negative reactions to companies who have conducted price tests via different segmentations, e.g. device used, geographic location, etc.

Prices will undoubtedly impact conversions and unless you are selling luxury goods the best deal will likely win out. However, the second it gets out that you are profiling users based on their device used you will lose a lot of credibility with prospective customers.

In my opinion, marketers should keep price testing out of their A/B and MVT tests. There are just so many other ways to optimise a site that will produce solid lifts without sacrificing integrity."

Jon Powell, Senior Manager, Research and Strategy at MECLABS thinks:

"Yes. The real question is “when is the best time to test price?”

If you already have established pricing, then I would avoid testing price first. Price is a key component in the perceived value exchange.

Customers are weighing that number against how well they understand the value I am communicating to them. I should make sure I’m not dropping the ball on that before I start cutting into my company’s revenue potential (or taking the greater risk in trying to increase it).

I would first design a test sequence to determine if the presentation of the product(s) can be optimised to achieve an increase in my key performance indicator. I would also design a sequence of experiments to understand a channel’s impact on buyer behaviour here as well before concluding that prices need to change.

The best approach to test a price starts with creating price points based on their projected impact to a greater business metric/need, like total revenue, new customer acquisition, etc.

No matter which metric you choose, an optimal price point will be one that has a predictably positive financial impact on your business, either short term or long term.

The only situation you will want to test a price that doesn’t have some sort of positive effect on your business metrics is when you are simply testing to determine if you need to find a cheaper way to produce and/or make available your product.

Once you determine the metric you are trying to impact and why, you can use an approximation model to calculate suggested prices to test. If traffic allows, you will want to test a range of prices, not just cheaper ones.

Sometimes customers expect a higher price to completely buy-in to the perceived value of your product, and sometimes a higher price point will be responsible for bringing in an optimal amount of revenue, despite a reduction in orders.

In your price test do not forget to set up additional tracking and auditing so that what you see can be verified and audited. Often in a test, we make unintentional discoveries as a result of the additional data collected."

Chris Goward, Co-founder and CEO of WiderFunnel has said:

"Price testing can be very revealing for e-commerce. The traditional economic model that says lower prices always increase demand doesn’t hold up in some cases.

Retailers are familiar with the effect the 95 cent ending price has. This can communicate a discount and create more demand than other five cent price movements. Recent research also shows price size, boldness, colour, sound and cultural considerations can boost sales."

Can you really raise your prices and increase your sales? In some cases, yes!

To price test you have to make sure your testing tool is keeping a valid control and maintaining consistency across visitor sessions. You don’t want people in different sessions to see the price change.

For price testing you should also optimise for net contribution margin, rather than just conversions or revenue. Your goal needs to be to optimise total profit as lower sales with higher margin could be a bigger win than higher sales with lower margin.

Try testing price treatment as well as the price amount too. Do colour, size, cross-out comparison and even numbers make a difference? You should test that!"

Rich Page, author of "Website Optimization: An Hour A Day" and co-author of "Landing Page Optimization" says:

"I think price testing is fine, and can have a good influence on conversion rates. The issue is more whether the marketer actually has control over the price being charged to test it.

Don’t presume you know what will work best, test many different combinations in an MVT for price display (colours, size, location), and savings (monetary versus percentage savings) to find the best converting.

Also, don’t forget to continue your test prices on your product pages through the rest of your shopping cart and checkout, otherwise you will risk confusing (and annoying) your visitors."

Price tests are not perfect. Email subscribers are sometimes a law unto themselves and making pricing decisions based on existing customer responses, or conversion on products visitors was not necessarily in the market for is not good practice.

Paid search ad and landing page tests are better, but also have drawbacks. You can test various prices in search ads for the response, but it’s tougher to control price visibility once the visitor navigates away from the landing page and back to the product through your menus."

In his book "Landing Page Optimization: The Definitive Guide to Testing and Tuning for Conversions", Tim Ash describes one of the limitations of site-wide price testing:

"Most companies treat price as a discrete variable. If you try to test price as a discrete variable (e.g., you test three distinct prices - your current price, a specific lower price, and a specific higher price), you are only getting information about the exact prices you choose to test.

You will know which one of the tested prices is better but you will not know if any of them are at the best price for maximum profit. The only advantage to this approach is that it works with your landing page optimisation tools, and can be tested like all the other discrete variables.

If your only alternative is not to test price at all, then you should use spot testing - a little bit of something is better than a whole lot of nothing."

Pricing is a variable that can have a very short life. A price that “wins” in January when customers are reeling from the Christmas overspend may not be the optimal price in May, September or December.

Economic and competitive changes, the launch of a new generation of a product or the addition of more product substitutes to your portfolio can affect optimal pricing. A price testing strategy should factor this in, and regular re-testing should be carried out.

This may not be what you want to hear but basically there’s no definitive answer to whether you should price test or not.

There are arguments both for and against. If you do decide to price test, consider the strategies, controls and measurement tools you need to ensure you are not testing prematurely, testing too narrow a range of prices, and are optimising for profit, not conversion rate.