What Users Want: High-Confidence Consumer Features at Ethos

Lani Oshima, Senior Product Manager

At Ethos, the people who land on our site in search of life insurance coverage come from wildly disparate backgrounds and knowledge levels. Some folks come to us extremely knowledgeable and ready to get same-day coverage. Others don’t yet know the basics of whole life vs. term life insurance and may be uncertain if they need a policy at all. The Consumer Growth team owns the challenge of connecting all of these users to the right information about life insurance, so they can complete their application with confidence and ease.

Building out a fantastic consumer experience that can support this fit-finding between user and policy is not straightforward. We have a robust user research program, but behavior in a research setting does not always translate to real-world results or to how a feature will perform at scale. We have a strong analytics team, but user analytics usually can’t tell us the “why” of how a feature is performing at scale. We are constantly A/B testing features, but testing takes time and our traffic sample size is finite. As a result, it can be all too easy for a team to spend weeks researching, designing, and developing a feature that fails to drive results in the wild.

Here are a few of the foundational principles that all teams can use to build high-confidence features for a direct-to-consumer audience:

1. Visibility. Testing is not the only source of knowledge about consumer behavior; a foundational step is for teams to have at least basic visibility into their user funnel. Reliable funnel visibility can help surface friction points and generate ideas for new features or hypotheses worth spending the time to develop and test.
2. Ask answerable questions. Not all tests are winners. But all tests should produce learnings — even a neutral or negative result helps the team understand where to focus resources next. For example, while scoping out a potential early-funnel email capture feature for an A/B test, we initially hypothesized that “users will want to save their answers so far at this point, so they can come back later to answer questions.” Although this hypothesis is trying to be user-focused, the reality is that an A/B test will not help us understand why users might be interacting or not with email capture. A better way to frame what we want to learn might be to ask, “Will asking for users’ emails early on increase lead generation without harming conversion?”. This rephrasing of the hypothesis allows us to set clear KPIs (leads and conversion) that will lead us to a clear test result. Even if this test turned out to be neutral, it helps us to at least understand that the new experience is not harmful, and it may be worth deploying even if it doesn’t directly lead to gains. If a team has the resources and time for user research, even better. This more qualitative approach can help fill in the why that may be less tangible from pure user analytics. User research teams will still need to ask answerable questions in order to deliver actionable insights.
3. There’s no wishcycling in user experience. “Wishcycling,” or wishful recycling, is the well-intentioned practice of putting an item in a recycling bin in the hopes that it will be recycled, in the absence of any supporting evidence. In developing user experiences, it is both tempting and common to add a button or a blurb of text in hopes that users will pay full attention and execute the action exactly as intended. Just as with recycling, this often doesn’t pan out as intended. This is another way of stating that consumer teams must remain outcome-focused; it is essential to clearly define and understand the user problem rather than declaring success upon shipping a solution. On our team, a solution idea might come from Marketing, Engineering, UX, Product, or Research teams — or a combination of all of the above — but true cross-functional problem-solving is best supported by a shared understanding and pursuit of the outcome.
4. MVP vs. RAT. “MVP,” or minimum viable product, is a popular and effective approach to testing a minimal version of a feature before investing design and dev time in a more fleshed-out version. But an important related concept is “RAT,” or riskiest assumption testing. For example, Ethos is exploring providing access to products beyond traditional life insurance, but actually providing these options in-platform would require a huge investment of time and resources. In order to put confidence ahead of investment, we’ve instead started with a “new products page” in our existing flow that helps quickly gauge interest via simple checkbox.
5. It worked! But why? Getting back positive test results is a wonderful feeling. Taking the time to understand or at least hypothesize why the good result came back can help generate extremely fruitful new tracks of feature development and testing.
6. Backlog, backlog, backlog. How to find the time in the day for all of these steps? Ultimately, there is no such thing as 100% confidence, and such an unattainable end is not even desirable for teams who want to move fast and generate learnings constantly. One fairly rote but effective process for maximizing speed of learnings is to think of the backlog cross-functionally; we often think of a Jira backlog of engineering tickets, but it’s important to scope out and prioritize areas of focus for the confidence-building functions upstream of development as well (research, analytics, design, etc). For example, if I am aiming to begin development on a new quiz that helps users understand what coverage they need in Q1, I likely need the research team to begin designing a research plan at least three sprints ahead.

Although we may all be citizens and users of the internet, finding the right user experience for the right audience at the right time is a constantly evolving pursuit that must always be done in the absence of full information. What’s so rewarding about life on the consumer team at Ethos is testing and developing solutions that help support better outcomes for families everywhere. If you’re excited about this work, we’d love to hear from you.

Lani Oshima, Senior Product Manager

Lani joined Ethos in March 2021 as the Senior Product Manager for Consumer Growth. When Lani isn’t optimizing the pre-application consumer experience, she enjoys hiking, surfing, and reading. Interested in joining Lani’s team? Learn more about our career opportunities here.