The user at the center of the process - Core tenant of modern design philosophies and challenge to us and our ability to keep humans at the forefront. Although there are many methods to achieve this goal at every step, one of the most common and effective methodologies is usability testing. In the context of automotive digital products, we have a unique challenge: As designers, we often design for non-standard devices that we cannot easily prototype for, which means we cannot validate our concepts, or it will be costly.
What are the challenges of usability testing in the automotive context? Are there cheaper options?
At the core of usability testing is understanding your user on a deeper level with the added benefit of uncovering problems and discovering opportunities. It is the direct involvement of you as a facilitator with participants representing your current or future users. There are already plenty of informative pieces written on basic theory, and we would recommend starting with an NN group article, which walks you comfortably through the basics.
Usability testing can be expensive, take time and might throw you back to square one, so why do it? Besides making your UX designers very happy, we can start with basic ROI. Forrester found that, on average, every 1 dollar spent on UX yields around 100 dollars in return. Other statistics, like research by PWC, claim that only a single bad experience with a digital product can lead up to 32% of customers not to interact with the brand again. At the very least, eliminating bad UX will pay for itself and more.
As we mentioned at the start, the automotive industry is complicated. There are significant challenges that product teams and designers might go through in this context:
- Designing on different than final HW - Computer screen
- Inaccessible final HW - Long development cycle
- Expensive to build classic simulators
We have been facing these challenges through many projects, which lead us to develop ideas and techniques for diminishing negatives while gaining as much insight as possible.
- You are looking for problems, not solutions
- You are not introducing your biases and leading your users
- You are here to understand your user in the context of the feature, not pass the validation test
- Understand what usability testing can offer you.
- Especially if you are not testing on target HW and you are using mock-up situations
- Be able to weigh results against the situation.
- It will sound drastic but be ready to scrap your solution
- If you are not prepared to accept that you will introduce implicit biases into your testing and possibly pass a flawed product
- Of course, it is the weapon of last resort to start over, but we should keep in mind that it might happen and accept that as a learning experience and not a failure
- If testing multiple solutions for one feature, always test one thing at a time.
- For example, You are A/B testing two proposals for onboarding experience, and you decide to throw in two variants for the visual identity of your site - one for each.
- Now you explicitly affected your user’s behavior, and you will not be able to discern the cause of liking/disliking the solution - Is it the solution itself? Is it visual design?
- Are you interested in gouging interest in the feature? Sell fake doors.
- You do not need to design a feature to test for interest; create an entry point and monitor how many people will show interest.
- Your usability testing will be only good as your users are able to perform
- Not only screen your user’s properly so they truly represent your users but also brief them properly
- Users will get nervous when you watch them; make them relaxed - Hawthorne effect
- Users will not want to insult you, especially if there is a financial reward; make them comfortable speaking honestly
- Users will see “test” and assume they are being tested, not the product, making them realize there are no correct answers, just insights to be found
- This is ancient UX wisdom passed down but is not corroborated by any serious scientific research.
- There is no formula for how many users is the correct number. It will continually change depending on what you are testing (totally new product? Existing product?) and your circumstances (How much resources, money, and time do I have?)
- You can read more on this topic here.
We have been building over last year our own prototyping seating buck that is slowly becoming an integral part of the process for early validation and check of design concepts. It is mainly the DIY project we are building in our office that came from the innate need to see our designs in an environment resembling the final HW rather than our computer screen. Another requirement that we needed to quench was that we work with OEMs worldwide, and it can be challenging to meet in real life, so this solution is a quick and cheap way to connect on a more hands-on level.
No methodology or solution is perfect, so let's see what this is good for and what it is not.
- Quick to set up, easy to change to new screen/requirement.
- No coding required - runs off Figma/design software.
- Easy to use by any designer in the office.
- Cheap compared to classic simulators.
- You are limited to the screens you have available, which can change the experience.
- In the end, it is just a prototype machine that will be different from the final interior and experience.
This solution is not one for all but has its place in the process and can be effectively used as a cheap early validation tool that will save your design team time and headaches later.
We are on a mission to find everything about users who will use the products we design. Are you too? Let us show how we are pushing experience forward with an innovative and informed approach to user research.
Come chat with us and share what user challenges you are facing!
