Saying no to design challenges in the hiring process

Why our team decided to remove ours and how you can better prepare if you’re asked to do one.

A design challenge (sometimes called a design exercise) is a take-home or in-person exercise around a design problem. They’ll either be focused on one aspect of product design or cross several. Their purpose is to help design teams better understand how a candidate thinks about and works through problems.

After navigating several take-home design challenges earlier on in my career, I ended up becoming a massive proponent of in-person exercises like the one that was a cornerstone of the product design hiring process at Clio. We saw them as a great way for us to see how a candidate moves through the discovery and ideation process and gives candidates an opportunity to see how we work, all in about 90 minutes.

I (and the team) have since changed how we think about these exercises since we realized how unfair they can be to different types of candidates and the amount of undue stress they can create for candidates. I thought I’d share our insights and how we’re continuing to evolve our processes at Clio to hopefully change some minds on other teams who use design challenges as part of their recruitment process. I’ll also offer a few words of advice if you find yourself faced with one of these during your next interview.

Issues with design challenges

Before getting into describing why we removed ours from our process, let’s cover the various issues that affect design challenges in general:

1. The most obvious one I’ve come across is when the task is in the same problem space as the company giving the interview it can be seen as unethical (even if it’s not). Although the team may not think of it in this way, it can definitely come off as such because it looks like they’re asking candidates to do work for free (I’ve heard from a few designers of companies that have actually used these exercises to farm new ideas — not cool). I understand why this happens: it can be a challenge for teams working on hard problems that want to understand if candidates will be able to handle the types of problems they grapple with.
2. If the team uses the same challenge every time, it introduces a preference towards the ways the challenge has been solved previously or how those interviewing solved it when they were being recruited.
3. Design challenges are not representative of how designers or product teams actually work. For one, there’s just not enough time for the kind of rigour in research or solution exploration that we’d expect from a designer. Additionally, design challenges are often undertaken solo (the candidate by themselves) or collaboratively with 1–2 other designers, which means that we’re missing key collaborators in the design process (product management, engineering, subject matter experts, etc.)
4. Design challenges create a lot of unneeded stress by putting candidates into a very unfamiliar place surrounded by people who are judging them (I don’t know about you but I know I’d be stressed out!)
5. In-person exercises favour designers who are fast thinkers/talkers and penalize those who are more introspective thinkers. This can push teams toward becoming homogenized and stifle diversity in personality types, ways of working, etc.
6. Take-home exercises are unfair because candidates may decide to invest different amounts of time in them. Some candidates may only spend a couple of hours whereas others might invest 24 hours or more. On the team’s side, this discrepancy makes comparing the results of any two design challenges difficult.

The decision to remove design challenges from our hiring process

The challenge we used at Clio suffered from four of the above issues (bias from seeing the same challenge completed many times, not representative of how we actually work, undue stress, and favouring fast thinkers) but there was one more issue that was unique to our design challenge: inconsistency. Each time we ran it, it would invariably be different.

Our design challenge was a role-playing exercise, which meant that how it was administered would be entirely dependent on who was “playing” each role. My version of the “stakeholder” was different from the next person’s portrayal. Beyond simple mannerisms, I might answer a question in a different way which could send a candidate down an entirely different path.

In addition to the inconsistencies we introduced, the candidates themselves created even more. The purpose of the exercise was to understand a candidate’s design process so the exercise naturally needed to be candidate-led. The downside to this is that the workshop or interviewing techniques they decide to use could take the exercise in any direction.

These two combined to make assessing the exercise one of the most painful parts of my job. The interview team often found ourselves very unclear about how a candidate actually performed because we weren’t able to compare apples to apples across interviews.

We had to make a change.

How we addressed these issues

Last year we decided to do a general overall of our hiring process, and one of the first things we decided to do given all of the above was to nix the design challenge altogether. The issues with the exercise were so foundational to the format that we decided to go a different route in order to understand a candidate’s design process.

Instead of the design challenge, we decided to give more time and weighting to the case study portion of our interview process. A case study is a far better example of how a candidate actually works both by themselves and in the context of the broader team. It also gives candidates the opportunity to showcase themselves and their best work while showing off their presentation skills.

For specific attributes such as influence, high agency, and decision-making that may or may not come through in a case study, we adopted a Q&A-style interview with a consistent set of questions.

Lastly, we wanted to give designers the best possible chance to be successful in our hiring process, so we developed resources to help them that we share ahead of time. One example is this case study blog post which helps candidates understand what we look for and some of the common pitfalls we’ve seen so that they can better prepare and show us the amazing designers they are.

What to do if you’re asked to do a design challenge

If you’re looking for your next gig you’re probably thinking to yourself “This is all great Sean, but Clio’s not the only company out there. There are loads of teams that still use design challenges in their hiring process. How does this help me when I’m talking to them? What should I do if they ask me to do one?”

Unfortunately, there’s no easy answer to this. As my UX teacher in university often said, “it depends.”

On one hand, design challenges are unfair to candidates and can be unethical. If they go against your principles then you could say no and give your reasons why (feel free to quote this article). Perhaps they’d be open to an alternative.

On the other hand, that could also lose you the opportunity. If it’s for your dream role/team, then maybe it’s the price you’ll have to pay to make it through the door. It comes down to how principled you are about it and how badly you want that specific role.

If you do decide to move forward, here are a few tips that might help:

If they offer you a choice of in-person or take home, play to your strengths. If you’re someone who can think on your feet and work well in groups of unfamiliar people, then go for the in-person exercise; If you’re more introspective, ask for the take-home exercise.

If you’re doing an in-person exercise:

• Ask how you can best prepare for the design challenge. They may give you some insight, but often it’ll be vague (they don’t want to give away too much). Still, better to get as much intel as you can ahead of time and it shows your curiosity.
• Come in with a plan. There’s nothing worse than showing up and having to scramble for how to approach an exercise (it’s usually very obvious to the interviewers). Use your own design process with the tools and techniques you’re familiar with. Don’t Google a new technique you’ve never used and expect to be able to use it effectively for the first time in the exercise (I’ve seen this a number of times and it usually comes off as awkward at best).
• Be aware that you may have a curveball or two thrown at you (just like in real life). Teams want to see that you’re flexible and can react to the situation as it evolves.

If you’re doing a take-home exercise:

• Ask for specific expectations of what they want to see from you and the time they expect it should take. This will give you a frame of reference and allow you to make a bit of a work-back schedule so you spend your time effectively.
• Ask if you can get connected with one of their designers to get feedback from and/or collaborate with. This shows that you don’t work in a silo and will give you an opportunity to get to know someone on the team. It also shows how invested the team is in you (are they willing to spend one of their team’s time?)
• Ask to present your work. Yes, it’s more effort and takes more time, but it’s better than throwing your hard work over the wall and hoping that the team assessing interprets your work as intended. You’ll also be able to answer questions they have and defend your design decisions.

I hope this post helped change some minds about whether to continue using design challenges in your team’s hiring process. At the very least, I hope it gave you something to think about!

For those faced with one of these exercises, I hope these tips will help you better navigate your situation. And if you’ve recently completed a design challenge and have other suggestions for how to better prepare, reach out — I’d love to hear about them!