Connor earned his degrees in psychology and human resources management from Illinois State University, which dictated his career:
“I liked the combination because it both had the business side, which is where I knew I ultimately wanted to operate. But it also had the psychology side and I’ve always enjoyed understanding people more.”
He then worked for Deloitte as a management consulting analyst where he learned how changes were implemented in large companies:
“I found that change was implemented by having people do these large waterfall approaches. They’d build changes, get everybody to change, and then they’re going to freeze that change.”
At the same time, Connor was reading about behavioral science and an iterative approach to implementing change:
“It was more like agile product development where you build something small, test it, see what works, scale what works, and reiterate what doesn’t work.
I thought to myself: ‘Why aren’t we taking this approach more?’ because it seemed like this one would’ve been more effective.”
He then left Deloitte and took up his masters in behavioral science:
“It was there that I solidified my belief that good behavioral change is more rapid and involves experimentation. It just has to be done with an open mindset of how to create the actual movement in change.”
Connor also realized that the best way to do it was through technology:
“Technology is the way to take those micro-interventions (things that are driving behavioral change) and be able to scale them to large audiences. At that point, I said to myself that it’s time for me to go into tech.”
But the reality was different in the tech space:
“I soon found that tech is oversaturated with the belief that usage is the best metric. That if people are using something, it must be working. This is the biggest misconception in product development in our modern tech environment.”
And since then, Connor has been on a mission to change this misconception with his work and his new book.
The Impact Mindset and its four components
The Impact Mindset is a philosophy that ensures that products not only change behavior to satisfy user needs but also drive positive business outcomes.
This product development philosophy has four components.
User outcome connection
This core framework states that specific behaviors drive user outcomes that ultimately create business impact.
For example, you want to build a product that helps users create an emergency savings account. The product outcome is the emergency savings account.
“To get someone to save, you have to ensure that users increase their contributions to the savings account, decrease their withdrawals from that savings account, and others that will lead to higher and more sustained emergency savings.”
Feature impact analysis
To validate the existence of the framework, you use the feature impact analysis, a process of experimentation and testing.
This helps you check:
- If it’s true that changing a specific behavior changes that outcome; and
- If changing that user outcome actually changes the business impact
Insights hub
“It’s a place to store all of those definitions of features from the user outcome connection, and all of the data collected from the feature impact analysis.”
This may come in the form of a research repository, a Notion page, or whatever the company deems appropriate for their case.
“The important thing here is that it’s a centralized location that all teams can go and visit to create alignment around what the features are and how they’re contributing to the success of the product.”
Evidence-based decision-making culture
To work in this framework successfully, companies have to adopt an evidence-based decision-making culture:
“They have to be accepting of experimentation, which means sometimes you fail. They have to be willing to ask the questions: ‘Why do we believe this actually exists? What is the evidence behind the decisions we’re making?’”
Connor emphasizes that adopting the culture enables the other three components to be effectively deployed and create value for the company.
Focus on the most important features
In the real world, not all features are being used completely, regularly, and fully. Connor advises to choose the most important features and make sure they are working:
“For ride hailing apps like Uber and Lyft, a pretty standard feature is letting users have the ability to see where the car is as it’s making its way to your location. Email marketing tools have templates that help make campaign setup easier.
When you’re using this type of ideology, choose the most important features–the ones that build the environment that says we are the right product for this customer and make sure that those features are working. If they are working, it will organically drive more adoption of those features because people are going to recommend it to others.”
He also shares that a working feature will drive the other metrics, too:
“Ensuring that the most important features are being developed and deployed in a way that drives user outcomes, then usage, usability, and the other metrics you care about should also increase.”
But if a working feature is still not being utilized, Connor says that you now have to ask other questions:
“If it’s working but people aren’t using it, then that requires separate design related questions. How do we make this more enjoyable to use? How do we increase the ease of use? How do we increase the usability metrics to actually see people engaging with this feature?”
The five success metrics for products
Unlike the typical frameworks, Connor’s framework goes beyond that of usage and usability.
Usage
Usage answers the questions:
- Are people actually using the feature?
- Are they retaining the platform?
“I’m not trying to say usage should never be used, but I think it’s overused. It shouldn’t be the sole metric you look at.”
Usability
Usability answers questions like:
- Is the feature easy to use?
- Is it fun to use?
- Are users satisfied with using it?
“These are really important because you could build the best solution in the world, but if users aren’t using it, then it’s not going to have much impact. You need to build something that people actually enjoy interacting with.”
Behavior
This pertains to the specific behaviors involved in the user outcome connection framework.
For example, you build a fitness app that helps people lose weight. The change in behaviors that you want to see are:
- Increasing calories burned
- Decreasing calories consumed
“It’s about how to build features that will help with these changes in behaviors. Then you can measure the average increase in calories burned and decrease in calories consumed.”
User outcome metrics
“In the case of the fitness app, is the person’s weight actually changing? Are they feeling more confident in their body over time?”
Business metrics
In the fitness app example, the product and business teams could choose to focus on:
- Customer loyalty and retention
- Increase in organic word-of-mouth marketing
- Increased upsell opportunities – converting to a premium account or buying extra tokens
“These smaller business metrics would lead to the ultimate business metric of bringing in more revenue so you can make a profit.”
Approaches to measuring feature impact
“We have this problem of overfocusing on usage in the tech space and it’s not because we’re trying to be misled or we’re trying to ultimately choose the wrong metrics. It’s because we’re choosing what’s the easiest to gather.”
Iif you want to start collecting behavioral data, Connor suggests these approaches.
Simple approach
The simple approach starts with thinking like an ethnographer to find out what people are doing to change the outcome.
“You can recruit someone who’s within your target audience and ask them. If possible, you can follow them to observe what they’re doing to change their outcome.
For example, if someone’s trying to lose weight, what do they believe is going to work for them and how do they go about shedding the weight?”
Connor adds this will help you not only understand the behaviors, but will also give you ideas about the metrics you can build on your product:
“And even if you can’t find the exact metric, you can find a proxy metric so you can get as close to that specific behavior or outcome as possible with the data that you have.”
While this isn’t the easiest approach, he says that this is a much more effective approach for building a successful product.
Advanced approach
The advanced approach requires the use of technology, specifically generative AI.
Connor does warn that this isn’t meant to replace talking to actual customers, but it will give you feature ideas if you’re working with a low budget:
“If your budget doesn’t allow you to talk to actual users to do the proper testing and infrastructure developments to capture things like event data, you can go ask ChatGPT.
You can tell it: ‘Here’s the outcome I’m driving for. What are some of the ways that people change their behavior to drive this outcome?’”
The next step would be asking the AI to generate ideas for collecting the metrics:
“You can then ask it, ‘Now that I’ve defined these behaviors, what are all the ways I could collect metrics to see if people are actually doing this?’”
He adds that you should do additional research and validate the AI responses:
“This takes a lot of iterations and it requires validation to make sure that you didn’t get misled by these generative AI systems.”
Ask “Is it working?” during the early stages
If you don’t ask this question in the early stages, Connor shares that you might struggle and it might be too late:
“I’ve seen teams who look at their dashboards with everything turning red, asking themselves: ‘How did we get here? Everything was growing well but people are now churning. What happened and how do we stop this?’
That’s the worst time to ask, ‘Is our product working?’ because it’s too late to switch and build something that actually works. Your users have already switched to other solutions.”
That’s why he emphasizes that you should ask this early on in the prototype phase:
“If you ask this during the prototype phase when you’re still building something, you can ensure that your product is effective when you send it out into the market.
From there, you will also see success in adoption as long as people want to use it. You can also see success in retention because users will be hooked onto the platform as long as it’s doing what it’s supposed to do.”
Final advice
Do start small.
“You can be the catalyst of change. Start by asking your team, ‘Why do we think this product works?’ And if you don’t get a solid answer, you now have an opportunity to create some evidence that shows that the product does or does not work.”
Don’t overvalue the usage metric.
“Usage is an important metric. Think of it like a speedometer on a dashboard. It tells you how fast you’re driving but it’s not the only important metric. You also have to check the gas gauge, the RPMs, and all the other metrics that tell you the whole picture of the driving experience is working.”
Thanks for listening! If you found the episode useful, please spread the word on Twitter mentioning @userlist, or leave us a review on iTunes.
