The right way to evaluate a UX agency is knowing what questions to ask (and what not to ask)
“I don’t want to make a mistake in this. It has to be right,” he told me as we started to wrap up our Zoom meeting.
I was presenting to the CEO of a B2B SaaS company who wanted to hire a UX agency. Initially, he had asked me to tell him about Standard Beagle’s experience in working with B2B companies like his.
His question was fair and not unexpected. We had developed a good rapport, and I wanted to make sure I understood why that was important to him. It turns out, he was worried about making a mistake.
Not wanting to hire the wrong team is the reason I hear most when companies ask about experience. Company leaders frequently evaluate multiple service providers rigorously because they don’t want to make a wrong move.
Services versus products
Buying professional services feels different from buying off-the-shelf products. According to the authors of “Why Clients Buy,” buyers see more at stake. Buying an office chair or software provides something tangible. You know what you’re getting before you pay.
Buying a service can feel risky. When buying a service, a company pays for talent and expertise. These are harder to see up front. Pricing can also vary widely, which makes it more difficult to compare and contrast design agencies. Hiring the wrong agency can put company initiatives at risk. At worst, the wrong decision can cause embarrassment, demotion, and even termination.
There’s a lot on the line.
Companies try to even the playing field and reduce risk in a number of ways. They open up RFPs (Requests for Proposal) in an attempt to force service providers — agencies like Standard Beagle — to look more similar. Or, more informally, they ask every agency the same set of questions and look for all evidence of past performance. Questions like:
- “What is your experience working on similar projects at other companies like mine?”
- “Have you ever worked with a company our size?”
- “How many years experience do you have?
- “Do you have case studies?”
Your evaluation questions may not uncover what you need to hire a UX agency
Hiring an agency for UX comes down to asking the right questions in the evaluation process. However, agencies offering user experience (UX) research and design vary so widely in their approach, training, and UX expertise, the standard evaluation questions don’t help companies make informed decisions. In fact, they may actually be the wrong questions to ask.
Here are the five questions we hear most at Standard Beagle and the questions you should ask instead so you can make a better decision in hiring an agency for UX projects.
What UX services do you offer?
Why are you providing UX services?
The term “UX” has become a buzzword. A lot of people have heard of UX and understand its benefits, but fewer understand its intricacies and how it’s done. And that’s OK. I don’t need to understand how a surgeon can operate on my eye and remove a cataract to know that cataract surgery is beneficial. Similarly, you don’t have to understand the ins and outs of UX to know it can work.
But asking what UX services are offered doesn’t give you insight into the expertise of the agency that offers them. It’s more important to understand why the agency offers UX services.
Do they have clients that ask for UX and need to meet demand? Do they have a passion for UX and have dedicated their life to this work? Both have merit.
Ask yourself: how important is it to hire a vendor that has a passion for UX? Does it matter if an agency that offers UX as an add-on because of market need?
There’s nothing wrong with offering a service to meet demand, but there are trade-offs. Agencies that have add-on UX services may not may not prioritize continual learning and growth. And that means they may not bring you the latest solutions or be up-to-date on the most efficient way to solve problems.
UX is evolving
It’s important to understand how UX is changing in order to keep up with new tools and methods. Agencies like Standard Beagle that focus on UX invest in continual learning so they can optimize client solutions. Standard Beagle designers devote a considerable amount of time both learning and teaching UX skills. And what we learn we bring to our client projects.
You should ask why, and then dive into how well they keep up with UX best practices.
What projects have you worked on?
What’s your UX process?
When prospective clients ask about past projects, their goal is usually to determine future success. The thought is that agencies that have successfully worked on similar projects in the past are more likely to have future success on the same kind of project.
This is a fair question. But when it comes to UX services, you can better understand future success by asking about process.
UX is a process. The outcome of the UX process creates a better experience for users and helps companies achieve their goals. And the beauty of UX is that it can be applied to many different kinds of problems. UX practitioners flex this process depending on each project’s needs.
Your company is unique. Just because an agency worked on a competitor or someone in your industry doesn’t mean the agency will be able to understand what makes you unique — unless they have a solid UX process. A skilled UX agency will be able to use their expertise to apply their process in a way that fits the uniqueness of your project.
It’s similar to a plumber’s toolbox. A plumber has a number of tools available to them, but not every tool is needed for every job. They wouldn’t use a sledgehammer to fix a leaky faucet. They are experts at choosing the right tool for the job.
The same is true for UX practitioners. Their toolbox — the list of UX methods — is huge. A UXer’s expertise comes from having a process and knowing which method is appropriate for the project.
Every skilled UX agency has a process
A UX process is far more than touching up a UI with color and fonts. When evaluating agencies.
- Make sure they have a process. If they don’t have a process, that’s a red flag that the agency doesn’t actually have UX expertise or understand how to apply it.
- Make sure the process has depth. UX is more than making a website “pretty.” Make sure the agency understands what UX is.
Are your designers and researchers in house?
How long have you been working with the designers and researchers who will work on my project?
Agencies work together in a variety of ways. It’s extremely common for agencies to work together or to augment core teams with contractors. These partner agencies and contractors don’t make an agency less capable or talented. But it depends on the nature of the relationship and who they partner with.
Just like you are looking for an outside service provider with the right expertise, you want to make sure that the agency you evaluate has fully vetted their UX talent. It’s common for a full-service agency not to have UX talent in house. But ask them:
- How well do they work with their UX team?
- How do they communicate?
- What is their process for exchanging information and deliverables?
At Standard Beagle, we’ve worked with several marketing agencies as their UX experts. We have also worked with other UX agencies to augment their teams. Every agency has a different process for how they communicate with us and integrate us into their teams. And we’ve augmented our own team with contractors when needed.
It’s not a negative to use outside talent
Collaboration is natural for agencies. What you need to evaluate is how well they work with partners. That will give you a better indication of how smooth the project will go.
How many years’ experience does the UX designer have?
What is the expertise of the designers and researchers who will work on my project?
Years of experience can be a misleading indication of skill, especially for newer designers and researchers. In my experience, I have met researchers and designers with many years experience that stagnated in their learning. These individuals learned what they wanted to learn and haven’t changed since. And it was evident in their work.
On the other hand, I have met designers with just a couple of years experience that have a strong growth mindset and have improved dramatically over a year. These designers demonstrate mid-level or even senior level expertise despite being new to UX.
Instead of looking at years of experience, look at expertise. Where does the agency demonstrate expertise? Do they focus on a few areas where they really shine? Or do they try to do it all? Do they have a fresh team that shows incredible skill?
At Standard Beagle, we have a mix of experience. While I have more than 15 years of UX experience, the rest of the design team currently has fewer years of experience in the field. However, as individuals, they are curious and embrace critique. They have focused on key areas of UX and enhanced their own skills over time. And in those years they have grown as designers and researchers. This makes my team top notch.
Find out more about the growth and interest of the UX team you evaluate
The more curious and growth-oriented the UX team is, the more likely they are to push themselves on your project and deliver strong outcomes.
Are the designers certified in UX?
Where and how did the UX designers train?
There is no standard training program for UX research and design. You don’t have to take a test for a professional license to say you do UX. Anyone can claim it, no matter what level of experience they have.
A better question is to understand where and how they trained.
First, you want to rule out the agencies that say they do UX without actually understanding what it means. Sometimes agencies with graphic designers will say they do UX because they design user interfaces. This doesn’t mean they understand the UX process. Conducting UX research with stakeholders and customers is key to designing a delightful user interface.
You can uncover fake UXer’s by asking about process as well as their training. Be forewarned: training comes in a lot of flavors — and they may all be good. Newer UX team members may have attended a bootcamp program — through General Assembly, universities, or Google — and then gained additional experience through freelance or corporate work.
Some may have gone through a UX degree program. More and more colleges and universities now offer both graduate and undergraduate programs in UX, human-computer interaction (HCI), digital design, and others.
And most senior designers, like myself, gained our experience through lots and lots of real-world practice, conferences, reading, and client work — what I like to call, the school of hard-knocks.
There’s no magic answer here.
You just want to understand how deeply they gained their knowledge. Rule out the fakes. Look for UX teams with a wide range of experience in UX research, UX thinking, wireframing, prototyping, testing, UI and interaction design. This will help you make sure you’re not just getting a graphic designer who decided that UX design looked good on a resume.
In conclusion
When you need to hire an agency for UX, it’s hard to make an evaluation if you don’t ask the right questions. Rather than rely on the typical evaluation methods, you can make an informed decision by asking more about process and expertise.
- Why are you providing UX services?
- What is your UX process?
- How long have you been working with the designers and researchers who will work on my project?
- What is the expertise of the designers and researchers that will work on my project?
- Where and how did the UX designers train?
These five questions will help you uncover the answers you need to make a better decision about which agency to choose for your UX project.
