Know your bias before you build

Via Martha Adelaide Holton & Charles Madison Curry, Holton-Curry readers, 1914 · source

In the fable of the blind men and the elephant, blind men touch different parts of an elephant and argue over what it really looks like. Some touch the trunk, some touch the tail, and some touch the ears. Naturally, they all describe the elephant differently. They’re all correct, in their own limited way — but none of them really understands what an elephant actually is.

Businesses are like elephants in this story — they’re so complex that they’re hard to see all at once. So employees specialize into roles, each with a set of skills and values that help them get their job done. Marketers write ads, engineers write code, designers pick the colors, and so on.

These roles are convenient and necessary, especially to big organizations. Unfortunately that perspective can limit your thinking. Each role has its own built-in biases due to the nature of the role, and this can lead to arguing just like the blind men and the elephant.

In this article, I want to talk about the biases our roles impart on us when we make decisions about our products—specifically the irrational beliefs that a product isn’t successful because of a reason related to one’s job title. In many cases these biases occur because someone is good at their job (in the narrow sense), not despite it.¹

These are just my opinions from working in startups and talking with clients. If you think I’m wrong or missing something, let me know.

#Engineering

• Elegant Model Bias—the belief that a beautiful data model is inherently good, and will automatically create a better product. Sometimes engineers model systems in a vacuum and try to force that model onto the real world—but the developer’s design model and the user’s mental model are usually not a perfect fit. Good products fit users’ lives, and users’ lives have strange and funny shapes, making for less than beautiful models. If engineers design the perfectly abstractable, flexible paragon of clean code, they won’t design for the real world or for human beings.²
• Power Tool Bias—the belief that people are drawn to products that have more features. Engineers use highly generalized tools like programming languages to do their jobs, and sometimes they are tempted to design products with the same amount of flexibility. But tools like this have a huge learning curve and are extreme overkill for the vast majority of people. Most people report wanting products that are simple and easy to use, don’t cause problems, and don’t require thinking too much. Our job is to understand what users actually want and when they want it, then design something so obvious that it “just works”. Just because you can, doesn’t mean you should.
• Shipping Bias—the belief that putting code on a live server means a feature is successful. In reality, the moment code is shipped marks the beginning of your customers’ experience. Engineers can feel excitement or pressure to ship features, even when there’s evidence it will make the user experience worse. Be careful measuring your progress by closed tickets, merged branches, and shipped code—these are only means to an end, not the end goal.

#Designers and UX

• Design Perfection Bias—the belief that small design problems keep people from using products. In reality, the exact color of your buttons is unlikely to be the thing that makes your project a success. This can also be true for usability—if nobody wants a product in the first place, making it easier to use won’t help.³
• Research Purity Bias—the belief that all product decisions must be made with substantial evidence. It’s true that you should talk to users, but some UX people take this to the extreme. To them, product teams are like academic research labs that can only publish with sufficient evidence. In reality, businesses make decisions quickly and can easily fix most mistakes. (Also, UX people aren’t the only people talking to customers, and sometimes could stand to learn a lot from sales.)

#Product managers

• Grind Bias—the belief that if we’re over-worked, we must be doing our best. Product managers are supposed to be the voice of the market, but there are an infinite amount of other demands placed on PMs. Some product managers get lost in the hustle of trying to make everyone happy and forget to take time to build a great product. This can be exacerbated when PMs aren’t empowered to make decisions or goals from leadership are unclear, putting PMs in reaction mode. Work hard, but don’t let your nose get so close to the grindstone that you can’t see anything else.⁴
• Old Role Bias—the belief that making decisions the same way you did in your old role will still work when you’re a PM. Many PMs used to be engineers, designers, or marketers, which usually means they understand one aspect of product much better than the other aspects. In my experience, this can cause them to keep the biases they used to have from their previous role. For example, if a PM used to be a designer, they might still over-value visual design and postpone resolving technical debt.

#Marketing

• Armchair Maker Bias—the belief that making great products is as easy as making cool stuff.  Sometimes marketing, sales, and support people get frustrated because they perceive obvious flaws in the product. Great products look simple, and sometimes that leads marketers to mistakenly think they could fix everything if they were in charge. Steve Jobs said “innovation is saying ‘no’ to 1,000 things”.
• Feasibility Blindness—the belief that all products and features are equally easy to build and maintain at a high level of quality.  Product people talk about finding a product that is technically feasible, financially viable, and desirable to customers; people with this bias are basically forgetting the feasibility aspect. “People would really love to buy a flying car, why aren’t we working on that?”

#Sales

• The Whale Client Bias—the belief that if a customer with a lot of money wants a feature, we should build it. People in sales tend to be biased towards individual customers who can buy a lot, especially because that’s often how their commission incentives are set up. This leads to teams building one-off features that are good for a single customer when it would have been better to build something for 100 medium-sized customers. This bias makes sense for consulting shops doing work for hire, but it’s deadly to startups with plans to scale.

#Customer support

• Squeaky Wheel Bias—the belief that complaints to customer support are representative of customers’ experience of the product in general. Customer support people spend a lot of time solving problems on the phone, so they’re tempted to prioritize features based on what they hear the most complaints about. But customers don’t call for support when things are going well, or when the product is so unremarkable they fall out of the funnel before becoming customers at all.
• The Feature Hot Potato—the belief that if lots of customers request a feature, it’s a good idea to build that feature. When a customer wants a feature, they complain to customer support, who can toss the “hot potato” to product as a feature request. But many feature ideas that sound good at first are bad in practice — even ones requested by customers! They might overcomplicate the product for others, or are too prohibitive to build. The better approach is to pass on the root problem the customer is trying to solve and collaborate with product teams to address it in creative ways.

#Founders

• Original Vision Bias—the belief that the founder’s first conception of the company’s future is inevitable (see Manifest Destiny), and that deviation from that vision is giving up or a distraction. Since founders created the company, they tend to see it as their baby and get overly defensive. In reality, no founders’ vision survives first contact with customers, and the business that best serves customers will have differences from the founders’ original vision.⁵

#CEOs and Executives

• 10,000 ft view bias—the belief that products are best understood from the lens of strategy and management. For example, executives who look at market share and strategy without understanding the reality of the product. The devil is often in the details, and executives who are too busy for details make bad decisions.
• Growth Bias—the belief that increasing the company’s near-term revenue, profits, or valuation is automatically a good thing. CEOs have bosses : the board will fire the CEO if the company doesn’t make money. To keep their jobs, some CEOs try to grow at the expense of the long-term health of the organization. It takes a brave person to turn away quick profits in favor of the long-term success of the business.

#People managers

• Don’t Rock the Boat Bias—the belief that the least controversial opinion is the correct one.  People managers are responsible for making people feel happy and productive at work. This is critically important, but can sometimes clash with the shifting priorities of the top brass. I’ve seen managers resist shutting down a project for fear of disappointing their team, even when they believe it’s unviable and further time spent on it could jeopardize the company. These kinds of decisions can be tough for managers to explain to teams. But if you prioritize employees’ pet projects over your customers, you’ll fail.

#All employees

• Shortsighted Metric Bias—the belief that behavior that makes a key metric go up is always good. This is a more generalized version of the Growth Bias. Sometimes we are measured by our ability to move key metrics in the business, whether it be some release deadline, revenue, profit, net promoter score, hiring goals, or something else. Sometimes this drives us to make the wrong decision, like getting website visitors that don’t convert or hiring the wrong person for the job. Narrow or short-sighted goal-obsession can lead you away from serving users, the dilution of your brand, and incurring technical debt. (One way to avoid this is by pairing multiple measurements that offset each other.)
• “This Company Matters” Bias—the belief that it’s important to the world that your company exists, and that customers will support you when you make decisions that are good for the company but bad for them. In reality, your product might be useful, but the world doesn’t care who makes it. Even if your brand has a devout following, what customers love is your brand—not your office building or your org chart. Your company is important to employees because it gives them an identity and community, but customers generally don’t care. One way this manifests itself is the unfortunate tendency of companies to organize their product around the structure of the company rather than the other way around.

#What can we do about it?

• Bring a diversity of roles into product decisions to help counteract bias. This does not mean everyone gets to vote on an answer, it just means you should be talking to lots of different people before making a decision.
• Make it everybody’s responsibility to find a product that you can build and sell. Everyone in your company should be working toward the same goal: product/market fit. Hire people who think about products holistically, not people who perform their role with blinders on. Don’t over-specialize roles too early.
• Recognize your own bias. As a designer, I sometimes have to remind myself that pixel perfection and usability might not be the most important things for the project I’m working on. Give your teammates the benefit of the doubt when they raise concerns. You’re not going to get your way all the time, and that’s a good thing.

Thanks to Luke Persola for contributing to this article. Chris Schneider and Andrew Kirkegaard also added edits.