The Numbers Lie

Saturday, Sep 23, 2017| Tags: testing

Business to business marketers have adopted consumer marketing practices like data-driven decision making and funnel-based conversion rate optimization. However, we haven’t properly taken into account fundamental differences between business and consumer marketing in adapting these practices.

Misunderstanding the systems that generate business data, and how the systems that generates consumer data differ undermines the usefulness of our data. Making data-driven decisions and doing conversion rate optimization in these circumstances is dangerous.

The Consumer Funnel and CRO Problem

The data that we get in B-to-B marketing is divorced from important metrics like sales and revenue. When you are dealing with a nine-month sales cycle, increasing the number of leads or marketing qualified leads doesn’t result in a proportionate number of sales.

The AIDA (attention, interest, desire, action) model is the prototype funnel that has been built on and adapted to all sorts of marketing including the various flavours of B-to-B funnels.

The AIDA model works great when the action you want is a sale. The lag between the action and the sale is zero.

It also works well when used at the individual level on micro conversions. Helping new subscribers to your blog understand that they have a work challenge that can be fixed and that your product can fix their challenge is a useful abstraction when designing your blog and planning what content to send to subscribers.

If you are reaching 20 percent of the addressable market and you double that to 40 percent, then you have a reasonable expectation to double your sales.

If you improve a landing page that serves as a cog in your nurturing from a 20 percent to a 40 percent conversion rate, you are not going to double the number of sales that pass through that page.

When applied to conversion rate optimization for micro-conversions (one of many interactions on the path to a business sale) in a complex sale across an account, the funnel model falls apart.

  • The lag between action and sale is usually measured in months forcing us to test against leading indicators that aren’t well correlated with sales.
  • The relation between one person at a company understanding your solution (and most any other micro-conversion) and a sale is not measurable.
  • An account can still find a path to a sale even when an individual is blocked by a failed interaction (live chat, asking questions at a demo, having concerns over-ruled by a superior).

The most motivated prospects are more likely to overcome obstacles to consider your product.

I’m not saying that you should be proud of unnecessary obstacles and leave them in place, just that using conversion rate optimization to double the number of people who request a demo won’t double the number of sales because you’ve already captured the most motivated buyers.


Now it’s possible that you could measure your conversion rate over time and your sales (with a time lag to account for the length of the sales cycle) and model the effects of conversion rate changes to individual interactions on the path to a sale to predict how an improvement will impact revenue.

The minor problem is that the creative you use for each change could be more or less appealing to different demographics which would complicate predictions. But, I don’t see this to invalidate this approach.

The major problem is that with a long sales cycle, the time between interaction and sale is such that you’ll probably need to wait somewhere in the 6-12 months range to account for lag and then collect data while changing just that one landing page for at least 6 months to get the results you need.

In order for the test to be valid, you would need to hold your other variables stable over that time. That means no changes to your sales or marketing processes for at least a year.

The cost of not improving your sales and marketing for a year is not worth the tradeoff of knowing how improving one interaction affects your bottom line.

The Consumer Data Problem

Just like our funnels are derived from consumer models for how sales happen, our data practices are derived from consumer data practices.

Data-driven marketing shouldn’t mean the same thing when talking about B-to-B compared with B-to-C.

Data on a consumer funnel is close to the actual sale event. Data on a business funnel is quite removed from the sale event.

As a result, consumer marketing data often has a causal relationship with the sale and is closely correlated where it is not.

Business marketing data relies on leading indicators which don’t have a causal relation to sales.

You can use correlation data to understand where people are struggling with your marketing system, but trying to effect changes to revenue by optimizing leading indicators isn’t going to give the results that you want.

There are other approaches to measuring the interactions on the buying path, like multi-touch attribution, but none of them separate interactions that legitimately influence people and interactions that are speedbumps on the way to a sale. The “marketing influence” metric is measuring touches rather than influence.

Even if you understand influence, it doesn’t help with optimization since you still run into the problem that you can’t predict which improvements in micro-conversions will result in material downstream improvements in revenue.

The Role of Qualitative Data

Data in B-to-B marketing is a huge challenge but it’s a worthwhile challenge.

Having said that it’s time to rethink data where optimization is concerned.

Leading indicators and poorly correlated quantitative data is a poor substitute for the hard work of really understanding the needs of the people interacting with our marketing and how those needs intersect with their buying processes.

Quantitative data gives you information. You need good information to make good decisions.

Qualitative data gives you knowledge. Information without the context of knowledge is dangerous.

Combining qualitative data with quantitative gives us the context we need to give meaning to an improved conversion rate in the middle of your funnel.

Say, for example, that you’ve done qualitative research and know that the people who download your A/B Testing Best Practices Guide fall into the following categories:

  1. People who need guidance on what to start testing so they can feel better about their contribution to the company.
  2. Copywriters who are looking for different formulas to add to their repetoire.
  3. Veteran testers who need some ideas to keep their programs fresh.

Now you can optimize for each of these scenarios and see which one resonates the most with the audience. The qualitative data gives you the context to know that doubling the conversion rate on the page also represents twice as many people completing a satisfying interaction with your marketing.

And it also gives you ideas for new resources based on the actual challenges.

Our Solution

Qualitative data doesn’t just magically get you the information you need to make better decisions. You need to ask the right questions of the right people.

Caret Juice uses Jobs-to-be-Done (JTBD) interviews to understand the needs and motivations of the people interacting with your marketing and then understand the context in which those needs and motivations arise.

JTBD thinking is popular in product development circles because it forces interviewers to go beyond the superficial what-people think-they-want and understand why so that product owners can come up with novel solutions.

Understanding the needs and motivations points to avenues for optimizing the copy of landing pages, ads, and calls to action as well as changes to make to existing marketing assets in order to better meet those needs.

Using interviews to power your optimizations means that any increase in conversion rate represents a genuine improvement on the experience of interacting with your marketing rather than the phantom improvements of micro-conversions in complex sales that have little relation to revenue.

You will also discover unmet needs in your marketing helping you create valuable new assets.

Context helps you assemble assets into a persuasive system by understanding what typically comes before and typically what comes after an interaction. It also helps understand the relationships between the individuals interacting with your marketing and their coworkers and business processes that precipitate the interaction.

Surfacing the right next steps is a major challenge in a complex sale. By understanding the context of interactions your marketing does a better job of meeting the needs of people before the company is looking to buy and of monopolizing the research time of individuals actively involved in a buying process.

In practice, this will mean that you do a better job of adding next steps in thank you pages, at the end of whitepapers and other assets, and in automated emails. The result will be increased funnel velocity and happier leads.

The B-to-B marketing funnel is a useful abstraction when taking a strategic view of marketing. But it loses its usefulness when turning that strategic vision into action. Funnel metrics are great for understanding what is going on, but you can’t effectively optimize for them.

Looking at each interaction as a job to be done and trying to optimize for the success of the most important jobs on the path to a sale helps the people tasked with executing the strategy focus on genuine improvements to the buyer’s experience rather than important sounding metrics that get in the way of the hard job of understanding buyers’ needs and motivations and satisfying them.

Curious about how looking at interactions this way can help produce better interactions? Try our trial insights package and pay only if you think it worth it.

Get Updates

You're data will be protected in accordance with our privacy policy.

The Refresher ~ Analytics for Marketers

Marketer-friendly summary of the most important changes to browsers, privacy laws, and analytics delivered quarterly to your inbox.

Learn More

Let's talk about your data

I want to help you get more from your data now while setting up culture and processes that will help you scale your data analytics team in the future.

Schedule a Call