Tuesday, March 12, 2013

Focus On Causality, Skip Personas

There's a fundamental problem with personas: they are made up and not real. This means they are full of errors and prejudice. Using personas will distract and lead your team to make wrong decisions.


In science and engineering, there is the concept of signal-to-noise ratio. It's important because it helps analysts explore their data properly, ask the right questions, build accurate models and then begin interpreting that data. Learn more about this process (Obtain, Scrub, Explore, Model & Interpret) from here.

Within the context of Product, we use these models to interpret the intersection of customers (current and potential) and our product -  the who, what, where, why, how.

We also use these models to iterate on product / market fit and to predict where our customers will be in the future. The latter is very important because we need to be able to make sure our product stays ahead of the competition and continues to excite our customers.

An important fact about models comes from Box:

“all models are wrong, but some are useful”

Models are meant to aid in decision making; they should not be viewed as fact. This is why models should be as accurate as possible - slight miscalculations (noise) in your models will only compound upon their inherent fallibility.

Now, about Personas....

The problem with personas is that they encourage people to take that data, fill in the holes and even embellish them with made up data. In the context of signal-noise and data analysis, building personas would be adding noise to your data. This is one of the worst things to do, especially when scrubbing your data is the most important step and is about 80% of the process.                                                      

Another problem with making things up (building personas) is that they are prone to prejudices. When someone is creating a persona and feel it has holes, they will fill it their own prejudges, stereotypes and personal experience. Anyone who has run an A/B test and been surprised by the results will understand how wrong intuition can be.

Last, it's hard to create this persona without an overcast of confirmation bias. You're making this person up, asking this person your questions and then you answer these questions. It's been pointed out before that is a lot of 'you' in there and not a real customer.

What to look for

In the future, I'll explore some techniques and ideas on what to look for and how, but the short of it is that we should be studying trends (cluster analysis), relationships and ultimately causality, between those models and between other models. When your data tells you that people who eat fast food generally eat alone and that people who have long meals generally eat with others, you can start to build models around this behavior. The trick is then to how to follow the data, how to cross reference between those models and making educated guesses about what groups might like in the future.

20 years ago, when data was precious and scarce, personas made sense. Analysts needed to build models somehow and so they filled them with assumptions.  In today's data rich world, there is absolutely no need to make things up. The data is there, you just have to grab it.

update: Personas are not auditable.
Another problem with personas is that they are not auditable. They are not attached to real customers, so the validity and usefulness of each persona can be debated by anyone; even more sever, they cannot be defended.

update2: Personas are only a collection of attributes.
Ryan Singer makes this point in this interview from Inside Intercom. I'll paraphrase the point:
Attributes do not make someone use a product. People use a product because it's solving a problem they have. I'll take an example Ryan used and add to it.

We have two restaurants: a pizza joint and a 3 michelin star restaurant.
We have 2 customers: a billionaire and a lower income person.

Suppose you make personas for these two people, you decided on the car they drive, the clothes they have - all that other persona stuff. Now, these two personas both go to the pizza place and the fancy restaurant. Ultimately, the only thing that matters to these people is the experience they are expecting when they go to these two places. It doesn't matter if the billionaire, who normally eats every night at a fine dining restaurant, is there only because his 8 year old kid loves pizza and the lower income person is there eating pizza every night. The cause that brought them there is the same - they wanted a good piece of pizza.

The same goes if you flip the example: the lower income person is eating at the 3 michelin star restaurant only because of a very special occasion and the billionaire eats this way every night.  The cause that brought them there is the same - they both wanted the feeling of luxury food and service.

Suppose these two restaurants based their menus on personas, deciding what to serve based upon the car these people drove or the type of job...they'd get conflicting and noisy data. The attributes surrounding these people did not bring them to these restaurants, the causality did.