Designers and entrepreneurs have been experimenting with live prototyping — putting unfinished product ideas in the context of real markets and real customer situations — for years, and now bigger businesses have begun to catch on. Many executives, eager to avoid over-investing in the wrong ideas, are intrigued by this approach, but they’re leery of putting unpolished products and services out in the market. Might we tarnish our brand? Will customers trust us less once they’ve experienced the rough edges of our prototype? Might we expose our strategy to competitors?
The concern can be valid, but by answering a few questions, and playing with a few variables, you can usually find a way to conduct market experiments that does not put your relationship with customers at risk.
A good first step is to get a sense of your customer’s sensitivity to change and to “rough edges” in your offer. We find the following approaches useful to help executives understand the degree of caution they need when running market experiments in public:
- Assess your brand history: Does your company have a record of experimenting in public? If so, how have consumers responded? Do your core customers value your inventiveness or your reliability? If you’re H&M, your customers expect you to be “out there.” If you’re Levi’s they might be fiercely loyal to your classics, and wonder why you’re shaking things up.
- Assess competitive benchmarks: Identify the quality “floor” of your offer. Is quality essential in your market, or simply a nice-to-have. For a car or legal services, this floor will be very high, but for an entertainment service, it might be low.
- Assess analogous industries: Looking at direct competitors can sometimes be misleading. Look across adjacent industries. If you’re in automotive, you might look at other highly regulated industries, like healthcare and finance, which manage to experiment considerably despite stringent regulatory environments.
- Prototype prototyping: Most simply, this means talk to you customers about prototyping. Instead of simply launching the prototype, bring your concepts or prototypes out and talk about them with customers. Get them to imagine what it would be like to come across these prototypes without knowing that they aren’t “real” products. Would they be excited about trying new things? You might have a group of customers who are willing to experience some rough edges of an unfinished product to feel part of building something new.
Armed with an understanding of your baseline customer expectations it’s time to plan your experiment. The most effective way to reduce your risk is typically to invest more to achieve an acceptable level of quality. But you have to be sure that increased spending will improve quality, and that you’re not simply wasting money. Consider investing more per customer, rather than investing in the operations that deliver quality. For example, in evaluating an automated digital legal service, you might have actual lawyers working behind the scene to deliver the service to test the market need first. You can invest in algorithms and automation later.
Another effective de-risking approach is to contain the experiment to specific moments of the customer experience. Focusing initially only on those moments that highlight the biggest business risks can decrease development requirements and customers’ interaction time with your prototype. For snack products, one of these key moments is when a customer chooses the product, so you might produce high-fidelity packaging so that the concept stands out against the shelf of competitors, but fill the packages with Legos (which sound and feel a lot like a crispy snack through the package). At this point, you’re testing the brand promise, not the customer’s willingness to pay or how the taste fulfills on the brand promise, so you don’t need to provide a full end-to-end experience.
You can also contain your experiment by calibrating exposure. Testing with large audiences might yield greater statistical significance, but nuance can get lost. Perhaps more importantly, testing new ideas with smaller audiences means that you can be bolder. Some companies like Facebook and AirBnB use limited release to allow any employee to release their product changes — large and small — to the public. These companies measure the impact of shipped features on customer behavior. Changes that have desirable outcomes are released to bigger audiences. This approach is easier for software companies, but manufacturing innovations like 3D printing have provided ways to launch physical products at limited scale.
If, after following these approaches, you feel that your market may still be hostile to the experiment or that the prototype still doesn’t deliver the value proposition to the customer, be transparent about that fact. Let participants know that they are engaging with a prototype that may not launch. One way of doing this is to create a sub-brand that is clearly about introducing new and unfinished ideas. For example, Google Labs is an arm of Google that focuses on creating revolutionary new products. They have been able to release Google Glass to the public before it’s ready for mass market use. We find that customers are generally enthusiastic about giving input if they know what they’re getting themselves into.
To protect against potential inconvenience to the customer, you could offer the option to switch back to the original service or product if they lose patience with the prototype. For example, if you’re developing software you might ask customers to opt in to product updates for the services that they would consider purchasing to get a sense of genuine product interest. This approach yields valuable information about product appeal, and you’ll gain access to a community to tap for feedback as your product evolves. Meanwhile, when customers opt-out, take it as an opportunity to understand why via a survey or even a live conversation.
While running prototypes can seem scary in the moment, the risks are much lower than launching a flawed product or service. Think about the above levers to reduce risk and avoid cutting corners in a way that is detrimental to your experiment. Markets move fast, and more of your competition will be experimenting to improve their market position. It’s better to test and build than be left with an outdated business.