Designers aren’t looking for unintended consequences
Designers often don’t consider what consequences — good AND bad — can arise from their designs. The reason? Human psychology. But there are four things designers can do to try to avoid unintended consequences.
What are unintended consequences in design?
One example of unintended consequences is cyberbullying.
Today more than half of all teens have been harassed or bullied online — a staggering number. Parents have to worry about how their kids use their phones and whether they are abusing them or being abused. And the results can be deadly.
In 2016, David Molak took his own life in his San Antonio home. His family learned later that he had been cyberbullied – and it had gone on for a very long time. The final straw was a series of text messages – berating him for his physical appearance.
In the past — before the ubiquity of connected devices — teens may have been able to escape at home. Today, the reality is that there often is no escape.
When Computer Scientist Tim Berners Lee invented the Word Wide Web in 1990, he envisioned it as a way for scientists, governments and universities to share information with each other. It’s unlikely he considered that it would be used for cyberbullying, along with countless other ways the internet has been misused or caused harm.
It is an unintended consequence of a design that was intended to make the world a better place.
An example of unintended consequences
In November 2019, Instagram announced it was rolling out two new features in an attempt to combat bullying on its platform. When Instagram launched nine years earlier, it was intended to be an app that allowed people to easily share images with a wide audience. Its designers never dreamed it would be used for bullying users. But it has been.
Instagram makes it easy for people to set up anonymous profiles that can be used just for trolling. Teens report that some people create what’s called a “hate” page to target other teens and say hateful things.
In order to combat this issue, in 2019 Instagram created a new feature called “restrict,” which allows its users to silently identify their bullies. From there they can control what bullies can post on their profile and the comments people leave on their photos. The idea is that when bullies are ignored, they are more likely to stop the attack.
Unintended consequences are not dark patterns
Unintended consequences in design is not the same as dark UX patterns.
Dark UX patterns are intentionally deceptive. They are designed to mislead or trick users to make them do something they don’t want to do. They can be really subtle, too.
For example, if you’ve ever been tricked to click on an ad because it looked like a regular page of content — this is a dark pattern. Or when a service makes it REALLY hard to cancel a subscription. This is something I ran into recently when I couldn’t find the “Cancel” button on a website. I ended up having to go through a terrible ticketing system just to stop the company from charging me.
Dark patterns aside, I firmly believe that most designers have good intentions when they come up with a new idea that solves a problem.
Why are unintended consequences hard to anticipate?
Let’s take a step back and understand why we — as humans — aren’t able to anticipate when bad things happen. It’s rooted in human psychology.
Too often, we focus on what software developers refer to as “the happy path.” What is the happy path? This is the path a user takes that doesn’t have any errors to problems.
It’s kind of like little Red Riding Hood. The happy path would have taken her straight to grandma’s house. But she met the wolf on the road and he suggested she go off the happy path and pick flowers to take to grandma. We all know what happened to grandma when the wolf showed up!
All of us want to have solutions that keep us on the happy path and avoid the wolves in our lives. But that just doesn’t happen much of the time. The thing is, most people drift off the path or take shortcuts that cause big problems.
Designers need to plan for the many ways – good and bad – that people will use their designs. But they don’t.
How human psychology fits in
Optimism Bias
Here’s the reason — people tend to be overly optimistic about their chances of avoiding bad things. Psychologists call this Optimism Bias – also known as the Illusion of Invulnerability. It means that we think our own individual risk is less than other people’s risk. For example, choosing to drive home after having four or five drinks at a party. Or attending a large group event without a mask in the middle of a pandemic.
We think we’ll be fine. After all, bad things happen to other people.
It’s why when people are asked about their chances for going through negative events — like divorce or bankruptcy – they tend to underestimate the negative outcome. We rationalize that these things may be hard for other people, but we’ll come through them just fine. Ask someone who’s just gone through a divorce and they may have a different opinion.
Likewise, as designers, we’re more likely to underestimate the risk that our design could lead to negative outcomes.
Confirmation Bias
Then there is Confirmation Bias. Human beings tend to interpret new information as confirmation of our existing beliefs.
For example, let’s use cyberbullying as an example. If I believe social media is the cause of cyberbullying, I’m more like to pay attention to evidence that supports that belief. Even if most evidence showed that social media didn’t lead to cyberbullying, I wouldn’t believe it. My own confirmation bias shapes the way I interpret information.
The danger for designers – who so desperately want to help — is that we’ll look for evidence in our data to show that our design has a positive impact. And we won’t even realize we’re doing that. We will selectively interpret data without knowing we are doing it. We may connect dots that don’t actually connect.
GroupThink
And then there’s GroupThink. This is what happens when we’re working on a team and we want everything to be smooth sailing. We don’t want to rock the boat. The group wants to get along so much that we come to decisions without using critical evaluation. When this happens, we may fail to speak up, challenge questionable assumptions, confront issues or punch holes in the group consensus. In our desire to get along, we go along with faulty reasoning.
Have you ever been in a group situation where you made a bad business decision because you put the good feelings of the group ahead of asking tough questions? It’s happened to me more than once.
Here’s an example from more than 60 years ago.
In 1957, the Ford Motor Company released a car called the “Edsel” – the name of Henry Ford’s first son. The Edsel was supposed to be THE new car for middle class Americans. Ford was so confident that the company spent $2.5 billion in today’s money to bring it out.
Instead of listening to what car buyers wanted – company executives created the car they ASSUMED people wanted. But the car didn’t sell. It was an enormous flop and Ford discontinued it in just three years.
Groupthink is one of the issues that led to the Edsel’s failure. Historians who have studied the car’s launch point to how an executive vice president at Ford – Ernest Breech – persuaded the top management that the car would be a success based on the thriving economy and early market research. Some company leaders were concerned about how it would roll out and how much money they were spending, but Ford moved forward anyway.
In a designer’s world, if everyone else on a team isn’t worried about what MIGHT happen, designers may not think it worth bringing up their own concerns. They don’t want to be seen as negative and they do want to please the boss. So they say nothing and the team moves on.
Here’s what it comes down to – we are just not that great when evaluating future risk.
How to avoid unintended consequences
Be aware of our own biases
First – we need to be aware of our own biases.
Remember, our brains are already working against us in being overly optimistic about the future. And it’s not a bad thing. I’m certainly not suggesting that you become a Debbie Downer, but as UX designers, we need to understand how our brains work and what unconscious factors could influence our decisions.
Stock the mental cupboard
Secondly, we need to stock the mental cupboard.
This is advice that my first journalism teacher gave me, as part of her 10 Commandments for Reporters. I believe it makes sense for everyone.
She told us to read widely. Don’t just read one particular kind of book, but vary your reading. Read newspapers, magazines, and books on all kinds of topics. Understand what’s going on in the world. If you tend to read fiction, make sure you also read non-fiction. If you like mysteries, also try biographies. The wider your range of knowledge across diverse subjects, the more you bring to the table as a person – and a designer.
David Epstein writes in his book “Range” that people who learn a wide range of skills across disciplines have an edge on those who specialize in just one subject. It’s an advantage to have knowledge from multiple areas, because it allows us an opportunity to draw conclusions and find patterns we might not otherwise see.
In avoiding unintended consequences, knowledge of teen behavior and bullying might have helped the designers at Instagram head off issues with cyberbullying on the app. As designers we can draw on our own lives to understand what COULD happen.
Champion team diversity
Another area we need to consider is team diversity.
Reading widely can take us only so far. Our individual experiences are just a small slice of the overall human experience.
This is why designers should lean on a diverse team to help us understand a variety of experiences and viewpoints. Think about diversity of experiences, age, gender, ethnicity, geography, and ideology. What insights can people who don’t see things the same way we do bring to the table? Someone sixty and someone 22 will look at the world differently and have much to teach each other. An immigrant brings a perspective that someone whose family has been in America for 200 years does not have – _and vice versa. A Fox News fan and a CNN viewer represent equally important parts of the American viewpoint.
We must embrace diversity and pull together a team that is willing to think critically about design instead of focusing on the happy path.
Wear the “hat of malintent”
Next, we need to put on the hat of malintent.
Malintent means a person’s intention to hurt someone or do something wrong. This is likely the most uncomfortable part of our work. As designers, we will fight against thinking like a bad guy. It’s hard to face the reality that there are some people who will misuse our products and software.
The idea behind this is to discover the ways your design could be used or misused? What could possibly happen? How could it be used in the wrong way? Spend time thinking through potential outcomes 6 months, 1 year, 5 years, even 10 years into the future. That will stretch your thinking!
It’s important to make brainstorming a part of your process. And you need to think about both the positives and negatives of what could happen. Only by thinking about what could go wrong — and what could trigger it — can you possibly design to avoid the problem. You might decide to go a different path entirely. You also might be able to design controls that help you avoid the problem from happening altogether.
When you lack power and authority
At this point, you may be thinking, “that’s all well and good, but I have no power in my organization. Even if I do raise the red flag about an issue, no one will listen.”
In the first place, that may not be true. In some workplaces, with open-minded managers, you may be heard. Granted, that may be the exception, but it only takes one person with power to listen to help you make a difference.
It may feel daunting to speak up or challenge groupthink if you don’t feel like you have the power to make change in your organization — even if you foresee a real risk.
Maybe it’s best to talk to someone who does have power and can bring up your issue to the higher ups. The key is to know how to have this conversation.
How to have a “crucial conversation”
Prepare. Don’t wing it
First, you must prepare – not just wing it. It’s important that you are clear about your concerns. You might even want to role play with a colleague or friend to make sure you stay on track when you speak to your manager.
Initiate the conversation
Next, initiate the conversation.
Sit down with your supervisor and explain your concern. Remind her that you have a shared goal. You are both on the same team! You’re doing this because you have good intentions and you want the best for your users.
Listen to perspective
Then, take time to discuss.
Share the work that you did that led you to be concerned about future risks for your design. Tell the story so they can understand the issue, and then ask for their perspective.
Listen, and hear them out. You are not in charge of your supervisor’s response – so avoid becoming emotional or raising your voice.
Your goal is to arrive at a mutual understanding.
I can’t guarantee that your boss will immediately agree with everything you have to say, but by approaching the conversation this way, you can start the ball rolling toward change. Even if they don’t agree with your suggestions or concerns, they will likely respect your initiative in coming forward to speak to them. And you will have gotten great practice in speaking up and sharing your ideas.
I highly recommend reading the book, “Crucial Conversations” for more information on how to have tough conversations with people — even your boss.
Conclusion
I know that we can all look back in our careers and think about the design outcomes we didn’t intend to happen. We all have high hopes that we are designing solutions that improve society — whether it’s connecting scientists, improving families, or adding delight to a fun app.
But sometimes our designs don’t end up the way we think they will. They may be used or misused — that’s human nature.
What we can do as designers is to remember the things we’ve talked about today. We can take steps to look long and hard at the risks we may face. We aren’t condemned to create unintended consequences if we are deliberate in our attempts to avoid them.
I challenge you to commit to being aware of your own biases, read widely, embrace diversity, and — when appropriate — think like a bad guy.
I know we all want to share the good that our designs can do and how they can help people. But we must also do our best to avoid unintended consequences in our designs. And that responsibility doesn’t belong to other people. It belongs to all of us.
Cindy Brummer is a design leader who speaks widely on topics of design, strategy and leadership.
Get the slides
Check out the slides for this presentation. Download them here!
Learn more about booking her for speaking opportunities and presentations here.
