Testing usability for a complex feature redesign

Research and user feedback revealed that the match-and-rank feature in Torre was highly valued, but lacked visibility and wasn't easy to understand at first.


So I redesigned it, with a simpler, right-in-your-face interface to make decision-making easier and quicker for our two types of users. Talent seekers assessing applicants and job seekers evaluating potential jobs.

Join me as I walk you through the process of finding out the right visibility and design of a feature with so much (maybe too much) valuable information.

Company:

Torre

Role:

Product Designer

Team involved:

CEO, Product Manager, Tech Lead

Tools used:

Figma, UserBob, Notion, ChatGPT

About Torre and the match-and-rank feature

Torre’s match-and-rank feature is an AI-powered job-matching system driven by nine distinct AI models that evaluate more than twenty factors.


In simple words, it helps users decide whether an applicant or a job is a good fit for what they're looking for. That's why users see it as one of Torre's standout features compared to other platforms.


The match-and-rank interface appears in two main views of the platform:


  • For talent seekers: When reviewing a applicant's profile, it shows how well the applicant matches the role's requirements and how they rank among others.


  • For job seekers: When checking out a job opportunity, it highlights how the user's preferences and skill align with the job offer and its requirements.

Identifying an improvement opportunity

Through the customer service team and research done by colleagues in the design team, we gathered some key feedback:


  1. Retained talent seekers (those who came back and posted a second job) thought the match-and-rank feature was crucial when evaluating applicants. They rated it highly and saw it as a feature that set Torre apart from other ATS (Applicant Tracking Systems).


  1. New users, both talent seekers and job seekers, were often unaware of the match-and-rank feature because it wasn’t visible enough. In the applicant's profile and in the job post, there was too much information displayed before the match-and-rank.


  1. New users also mentioned that, once they did find it, the match-and-rank feature took some time to understand due to the amount of data points included.


So the big picture was that the match-and-rank feature was a key asset to retain users, especially for talent seekers, the group that could be monetized. We needed to make it more visible and intuitive.

As you can see in this example, in an applicant's profile there are five sections before you reach the match-and-rank section. Also, there are several factors to review, and you can't see the details of how the applicant matches in crucial factors such as skills or languages.

From insight to solution

Step 1: Brainstorming


With the PM (Product Manager), we had to define the information to be shown in the match-and-rank feature. The goal was to remove unnecessary data points, leaving only the information that would help users decide if the applicant or the job was a fit.


At the top of the interface we had the gauge with the match percentage and how the applicant or the job ranked amongst others. Then we agreed on showing the following factors for each type of view:


  • For talent seekers reviewing an applicant's profile (4): Skills required, Languages required, Compensation, Location


  • For job seekers reviewing a job post (7): Skills required, Languages required, Compensation, Location, Organization size, and Type of job.


This was still a lot of information, and the constraint was to show it as soon as the user open either of the views.

Step 2: Defining metrics


Before going into full design mode, we defined with the PM how we'd measure the impact of the effort.


Our hypothesis was that redesigning the match-and-rank feature would increase user engagement and help users make faster decisions. So, we aligned with the engineering team on tracking the following metrics:


  1. Engagement with the match-and-rank feature

    • Number of clicks on the match-and-rank section per visit per view (applicant's profile and job post)


  2. Time from application completed to applicant review

    • Number of days since an applicant applied until a talent seeker reviews them (matches or rules them out)


  1. Time from applicant acquisition to first job application

    • Number of days since a new user creates their profile until they apply to their first job

Step 3: Drafting designs and getting inspired by the CEO


I was ready to start drafting some early designs when my CEO surprised me with a hand-drawn sketch of the match-and-rank feature for the applicant's profile view. It was actually pretty good.


I liked his idea of where to show the interface. My issue was the amount of information he wanted to show in it. A lot more than we had originally considered:

  • The applicant's connections within the platform

  • The number of recommendations they had received

  • Their reputation weight (based on those recommendations)

  • A cultural fit summary between the applicant and the hiring team

  • The job opening's name

  • The applicant's stage in the recruitment process

  • A button to match or rule out the applicant

This was the CEO's idea, hand-drawn beautifully on paint.

So, I stole his idea of showing the match-and-rank as a sidebar. It allowed us to show crucial information immediately without covering other valuable information.


But I wasn't on board with showing all that information in such a tight space. I felt it would create too much cognitive load and work against our hypothesis of helping users make faster decisions.


With that in mind, I started some rough drafts to explore positioning, color balance, and most importantly, what information was actually relevant to show.

I managed to add most of the points requested, except the last three, and I had to defend that decision to both the PM and the CEO. My arguments were:


  1. This information was already shown in the bottom bar, it worked on desktop and mobile, and it didn't cover any valuable information.


  2. This information wasn't totally related to the match-and-rank feature, because it didn't affect how the match is calculated.


Luckily, I survived the review and received the sign-off to move forward with usability testing on my design proposal.

Step 4: Prototyping and testing usability


As I've mentioned throughout the story, the match-and-rank feature involves both types of users. Therefore I had to create guidelines with two different mindsets for the testers:


  1. Testers acting as talent seekers reviewing an applicant's profile


  2. Testers acting as job seekers reviewing a job opening



Like in any startup, I was against the clock to deliver results quickly. Since the designs for each type of user were similar, I decided to use a mobile prototype for one and a desktop prototype for the other. This allowed me to deliver results quicker, by working with two sets of testers, instead of four.


I set up my guidelines and database in Notion, I built my prototypes on Figma, and used UserBob to recruit random testers.

The 'Users acting as talent seekers reviewing an applicant's profile guideline

Try the desktop prototype used in the usability tests

Key takeaways and learnings

Finally, the data from the tests was collected, analyzed, and presented in a report, which highlighted the positive feedback received from both types of testers:


  • Most testers acting as talent seekers found the design efficient, with 8 out of 10 saying it made reviewing applicants easier and quicker.


  • All testers acting as job seekers (10 out of 10) mentioned that the design was easy to navigate. They appreciated how well-organized it was and how quickly it helped them determine if a job was a good fit.


  • The only issue identified was that some testers, in both groups, took extra time to expand the drawer. To address this, I added a small tab next to it to make it more noticeable (see design below 😬).


On the personal side, a couple of learnings stuck with me:


  • Trust my gut and defend my proposals. I pushed back on adding too much information and the usability test results backed that decision.


  • Work smart, not hard. By optimizing the guidelines and prototypes to work with two group of testers instead of four, I was able to deliver the usability report on a tight deadline.


This is the design that will be implemented soon at Torre. At the top, the match-and-rank drawer as a talent seeker reviewing an applicant. At the bottom as a job seeker reviewing a job opening.