产品背后:Duolingo 连胜机制 | Jackson Shuttleworth(留存团队 Group PM)
Behind the product: Duolingo streaks | Jackson Shuttleworth (Group PM, Retention Team)
Jackson Shuttleworth: … Duolingo is a $14 billion company, it’s hitting all-time highs too, it just keeps going up, but I think it doubled in value in the past six months. Streaks is the most impactful feature. We have, right now, over 9 million users with a year plus streak. If you look at the numbers, I think it’s been our biggest growth lever. What Duolingo really focuses on is, how do we help users build habits around language learning? Getting user come back the next day is the biggest problem to solve.
Introducing the Guest
Lenny Rachitsky: Let’s get into the motherload of learnings from the journey of streaks, talk about the key lessons, insights, and also wrong turns along the way.
Jackson Shuttleworth: I’d say test everything, we’ve run in the last four years over 600 experiments on the streaks, so every other day. We’ve actually set up really good infrastructure for copy testing. We used to say continue, our standard CTA is continue, and we changed that to commit to my goal, and it was a massive win.
Starting the Conversation
Lenny Rachitsky: There’s so much human psychology that you all learn through all these experiments of just how to motivate people, what motivates, what demotivates.
Jackson Shuttleworth: Say that you played a mobile game, or that you’ve done it for 3,000 days in a row, I don’t know, maybe that hits a little bit different than you’ve learned Spanish for 3,000 days in a row.
What is the Streak Feature
Lenny Rachitsky: Today, my guest is Jackson Shuttleworth. Jackson is a group product manager at Duolingo, leading the retention team. This is a different kind of episode that I’m experimenting with, where we spent the entire conversation focused on the journey and lessons of a single feature, in this case, Duolingo streaks. Duolingo is a $14 billion business, just over the past six months, they doubled in value, they’re hitting all time highs in usage and market cap, they’re also one of the very few successful and also the single biggest consumer app business in the world. And as you’ll hear from Jackson, the streaks feature is the single most impactful feature that most contributed to this growth and success. In other words, you could argue this one feature created billions of dollars of value, which to me means it is worth studying in depth.
In our conversation, Jackson shares the history of the streaks feature, all of the biggest wins and wrong turns they’ve taken along the way, what he and his team have learned about what works and doesn’t work with a streaks mechanic, and also how they set up their teams to operate in a way that allows them to run over 600 experiments on this product and continue to find big wins. I hope to do more episodes like this on features and products that you’d love to hear more about, so leave a comment either on YouTube or on Substack, and tell me which product or feature you’d love to see me cover. And if you enjoy this podcast, don’t forget to subscribe and follow it in your favorite podcasting app or YouTube, it’s the best way to avoid missing future episodes, and it helps the podcast tremendously. With that, I bring you Jackson Shuttleworth.
Jackson, thank you so much for being here and welcome to the podcast.
Jackson Shuttleworth: Yeah, thank you. Long time listener, first time caller.
The Impact of Streaks
Lenny Rachitsky: I appreciate it. So, this is going to be a really interesting conversation, I’ve never done an episode like this before, where we basically spent an hour, an hour and a half going into one feature of one product, but this is a very special feature, it’s a very special product. Have you ever spent an hour, an hour and a half just talking about this one feature with anyone?
Jackson Shuttleworth: Well, internally, so when we onboard new folks to the team, we’ll do, I actually just did this with somebody that joined the team recently, where we spent, yeah, I was like, hey, let’s spend an hour just talking about the streak. We got through an hour and I got through about 30% of what I wanted to share, there’s just so much, we’ll talk about this, but there’s so much that we’ve learned over the years, but never anything externally like this. I think we’ve shared bits and pieces of learnings, but this will be the motherload of learnings hopefully-
Origins of the Streak Feature
Lenny Rachitsky: Here we go.
Jackson Shuttleworth: … of how Duolingo built the streak.
Learning or Retention
Lenny Rachitsky:
I’m not surprised at all that over 10,000 companies use it today. Visit pendo.io/Lenny to create your free Pendo account today and start building better experiences across every corner of your product. P.S. Do you want to take your product-led know-how a step further? Check out Pendo’s lineup of free certification courses, led by talk product experts, and designed to help you grow in advance in your career. Learn more and experience the power of the Pendo platform today at pendo.io/Lenny.
Speaker 3 (00:05:05): Pendo.
Okay, so for people that don’t know anything about what Duolingo streaks are, can you just first give a brief explanation of what is Duolingo streaks? What is this feature all about?
Extreme Experiments Lowering Barriers
Jackson Shuttleworth: Well, presumably people know, Duolingo is a language learning app, the Duolingo streak tracks, currently anyway, how many days in a row you’ve done a lesson. So, you come to Duolingo, you do a lesson, your first lesson you’ll start a streak, and then every consecutive day that you come use the app, you’ll extend your streak. And I should actually put an asterisk around how many days in a row because we also have built flexibility into the feature. So, we have these things called streak freezes, it’s like insurance for your streaks. So, it’s a pretty simple feature, in theory, but over time we’ve layered on challenges, and goal setting, and rewards, and social features… A lot of our notifications are tied to the streak. So, pretty simple feature to understand, but it’s been a really rich feature for us to build on top of.
Deciding on Neutral Experiments
Lenny Rachitsky: Some people might be hearing this and be like, Duolingo streaks, what’s the big deal? There’s other people that are like, holy, I want to learn everything I can possibly learn about Duolingo streaks, so many companies are trying to copy what you’ve learned from this. Give people a sense of the impact that this one feature has had on Duolingo’s success and growth.
Focusing on the Zero to Seven Experience
Jackson Shuttleworth: And this is not just the subjective retention PM talking, I think this is our biggest feature, with the exception of the lessons. And I think it’s actually [inaudible 00:07:24] start that. Streaks are a great engagement hack. I’m of the opinion that any team, any app out there can introduce a streak, and if you figure it out, it probably works to retain users, but at the core, you have to have an app that people want to use, and people really like using Duolingo. It’s fun, it’s delightful, you learn something. And so, it allows us to layer an engagement mechanic on top of that the streak is really powerful. So, it ships a disgusting amount of DAUs, again, it is one of our golden geese. And again, what’s cool is that, you look at notifications, notifications for Duolingo is massive. Us sending better, whether it’s copy or timing, so many of our notifications work because they reference the streak, because users care about the streak.
And so, not only is it itself, us iterating on the streak a huge driver of DAUs, but it’s also something that enables other really high valuable features. I was looking up some stats before I came on, and it’s pretty crazy. So, we have right now over 9 million users with a year plus streak.
From Single to Optional Goals
Lenny Rachitsky: Wow.
Jackson Shuttleworth: So, 9 million of our users have used Duolingo every year, almost every day, for well over a year. Which is pretty… I don’t know, I like to imagine things in terms of like, okay, well, if you put all these people in a city, or in a place, where would it be? I’m like, ah, it’s like a very large city, 9 million people.
Methodology Behind the Experiments
Lenny Rachitsky: For a year, have a year long streak.
Copy Testing and Product Reviews
Jackson Shuttleworth: For a year.
Why Streaks Fit Duolingo
Lenny Rachitsky: That’s incredible. I was just looking at the stock of Duolingo, so Duolingo is a $14 billion company at the time we’re recording this. It’s hitting all-time highs too, it just keeps going up. I think it doubled in value in the past six months, something like that. And what I’m hearing from you is streaks is, other than just the core learning feature, which is just the product of Duolingo, this is the most feature in terms of growth, and in that retention specifically.
Jackson Shuttleworth: If you look at the numbers, I think pretty objectively has been our biggest growth lever for driving DAUs, and also say a lot of it’s just related to how we think about growth at Duolingo. And a lot of what we try to do is organic growth. We think about growth just as much as bringing new users onto the platform is not losing them. If you’re just bringing people onto the platform, then they churn, that’s not going to be sustainable. And so, as much as we can do to keep our users coming back and actually retaining as users, it’s going to give us a much easier base to continue growing DAUs off of.
Helping Users Understand Streaks
Lenny Rachitsky: Perfect. Okay. Talk about how this feature originally came to be. What was the original version of it? What was the original insight that led to [inaudible 00:10:05]-
Jackson Shuttleworth: Yeah, so the oldest streaks or as old as Duolingo itself. When we launched Duolingo, we launched with a streak… And I say we, I was just graduating undergrad when Duolingo launched. So, this is well before my time. But we launched with a streak feature. Initially how the feature worked was you’d come to Duolingo and you’d set a goal for yourself, and it was an XP based goal. So, Duolingo, a little bit of nomenclature, we have an experience point based system that drives a lot of our features in the app. And the way you’d set it is you’d say, based on what your language learning goal was, maybe you have a 10 experience point goal versus a 50 experience point goal. And so, extending your streak would be, hey, can you hit 50 experience points? If that’s what you set your goal to be.
And this worked well, I think this also speaks to how Duolingo initially grew. Luis launched it with a TEDTalk, we probably had a more tech-forward user base initially, and so this whole idea of an experience points based streak system made sense. But what that also meant is that you could have a user come and use the app, do multiple lessons a day, and maybe they just set too hard of a goal for themselves, and then lose their streak. Which, you don’t need to be an expert in streaks to understand that’s probably not good. The nice thing with how we initially set it up though is it really did connect with what your goal was. So, if you were serious, let’s track how good you are at being a serious language learner.
But I’d say one of the most impactful experiments we ran was about, this was actually as I was joining, or just after I joined we had run this experiment, was to move it from a XP based streak to just do one lesson a day, and you’d extend your streak. And the risk that you can sort of imagine is, well, then users kind of care less about it because it’s not connected with their goal. And we saw none of that. This was a huge driver of DAUs, just making it easier to extend your streak, but I think really importantly it’s still meaningful. The unit of use, and as you’re thinking about building a streak, I think it’s really important to think about what the unit of use of your app is. The unit of use for Duolingo is doing a lesson. And so, if what we care about is users coming back every day and doing a lesson, because it shows that they’re actually engaging with the app, then it doesn’t hurt us to make our streak focus on just do one unit versus multiple.
And so, that was probably the big [inaudible 00:12:25] change experiment that we ran at the time was moving from an XP-based streak to a one lesson streak. It’s also simple, and I think that’s one of the things to think about with streaks. It’s always easy as a PM to have a million goals or objectives for what you want your feature to be and potentially build a more complex feature, and a one lesson streak, it’s just easy for more users to understand.
How Experiments Are Measured
Lenny Rachitsky: Yeah, that’s exactly what I was going to say. And just for folks that aren’t super familiar with XP, it’s basically experience points and you get them from doing things, it’s like a [inaudible 00:12:53]-
Jackson Shuttleworth: Yeah, you do stuff on Duolingo, and then based on what you did and how well you did it, we give you experience points, this actually drives a number of our features in our apps. So, leaderboards is the big one, we have a leaderboard system, where you try over the course of the week, you battle with 29 other people, and you want to win, that’s all driven by XP. So, we do have other features in the app that really benefit from this XP system-
Winning but Unshipped Experiments
Lenny Rachitsky: Cool.
The Streak Flame and Cultural Differences
Jackson Shuttleworth: … the streak is just no longer one of them.
Calendar Design and Feature Communication
Lenny Rachitsky: Awesome. Okay, so I want to talk about the journey from that point to what it is today, but a quick tangent, I saw Luis tweet just this week, someone asked him, “How do you decide whether to optimize for learning or engagement?” And he’s like, “No question, everything we do is focused on engagement because you won’t learn anything if you’re not coming back to the app.”
Jackson Shuttleworth: As an engagement PM, that was the coolest thing he could have ever said. It’s, well, again, very much as subjectively I guess as an engagement PM, that’s how I’ve… I am sure the learning folks at Duolingo will cringe when they hear it, I see myself as a learning PM as much as an engagement PM, because the easiest way not to learn on Duolingo is not to come back the next day. And so, if we don’t make the app retentive, you will have no opportunity to engage with our learning features. Now, I do think that there is a long tail of learning, where, if you start to dumb down, and honestly this is something we wrestle with the streak as well, you start to dumb down the experience, and your users aren’t actually learning, they’re not going to care.
Streaks work best when they’re sitting on top of an app that users care about. But yeah, if you don’t come learn on Duolingo… If you don’t come back to Duolingo, then you’re not learning. We track a lot of this with, the work that we do, we make sure that as we’re making changes to the streak, we’re not hurting the learning experience and we don’t have a ton of interaction with it. So, we’re constantly thinking about this, but thank you Luis for saying that.
Flexibility and Streak Freezes
Lenny Rachitsky: It makes sense to me. Okay, so lets get into the motherload of learnings from the journey of streaks. So, the first version went from XP to one lesson, talk about the key lessons, insights, and also wrong turns along the way to what we see today.
Initial Flexibility of Streak Freezes
Jackson Shuttleworth: Duolingo has very much, has a strong test it philosophy, we’re willing to test a lot of different… Honestly, we’d much rather test it than debate it for days and days. So, we actually followed up this experiment with, and this is a little bit later, with hey, what if we make it even easier to extend your streak? And so, we actually tested, hey, if you do one exercise, just one exercise in a lesson will extend your streak. A lot of the insight was good, you look at the funnel, hey, there’s a lot of users who are starting but not finishing lessons, they’re not extending your streak, the loss aversion doesn’t kick in, they don’t come back… So, this followed that train of thought. What we realized when we ran this experiment is DAUs moved not one bit. And what we were doing by… And if we go back to unit of measure, we had dumbed down the unit of… Nobody thinks about, oh, I just want to come do one question on Duolingo, nobody thinks about that.
So, we had a less clear unit of measure that we were basing our streak around, and the users that we were capturing with our streak, you come, you do one or two questions on Duolingo, then you leave, were the least engaged users imaginable. And so, I think that’s something also to think about as you’re building your streak, is like what is the user that you’re solving for? So, not only what is the habit that you’re building for, but what is the level of commitment, and that was an example where we over indexed on a type of user who we honestly just weren’t going to keep, that was a very easy shutdown decision.
Lenny Rachitsky: That’s an awesome story, just to comment in that real quick. Just you went, let’s just go to the extreme and make it just like streaks, yeah, get everyone going streaks forever, and then… I love that it turned out and it’s not bringing users that you want, and it’s dumbing down experience. It makes me think of Farmville, where you have to go and harvest your crops every, whatever, hour, that lasts for a bit and then eventually people are like, what the hell am I doing with my life?
Streak Repair and Earning It Back
Jackson Shuttleworth: Yeah, exactly. We test every, I was looking at the numbers as well, we’ve run in the last four years, over 600 experiments on the streaks. So, every other day effectively-
Sacredness and Limits of Streaks
Lenny Rachitsky: 600, wow.
Jackson Shuttleworth: … we’re running an experiment. And they range from big, like this, changing how the mechanic works, to let’s swap a string with another string and see if that copy is better for users. So, we’re constantly testing on everything. I do think that, I’d be more careful running that experiment now, at some point your streak gets big enough that, again, I got 9 million users on the streak, I got to be really careful… Those are our best retaining users, you got to be careful. But in the early days of the streak, I’d say test everything. See what… Don’t get super caught up in it has to be like this, just test a bunch of stuff and see what speaks most to users. Because I think, again, you will constantly be surprised by the insights that you get from whether you… We shut down about half of our experiments, so half of our experiments lose, we still learned a ton by virtue of running them. So, super, super valuable.
Push Notifications and Reminder Timing
Lenny Rachitsky: That’s actually a good success rate, a lot of companies have only 20% of experiments be positive. What’s your policy on if it’s neutral, do you ship it, or do you kill it?
From Disclosed Behavior to Push Strategy
Jackson Shuttleworth: It really depends. If we’re adding something and it’s neutral, we tend to shut it down because it’s just more cognitive load, it’s something that we’re going to have to start building around, a new UI element that we have to figure out how it fits into our system. I’d say when we do ship a neutral experiment is, something that we have real conviction around, okay, yeah, maybe this was neutral but it’s going to give us a new platform to then build on top of, so that experience might be neutral, but now we can build these DAU positive experiments. My general take there is though, in that case, build that in as part of your V1 so that you make sure that at least your hypothesis around this roadmap has play should probably be the case, but in general, shutting down a neutral experiment so you don’t introduce more complexity to the app is the way we tend to go.
Lenny Rachitsky: Makes sense. All right, what else? What else have you learned along the journey?
Celebrating Perfect Scores
Jackson Shuttleworth: Well, maybe I can talk about a few different ways that we structure, a few different themes that I think we lean on. So, the first is focusing on the zero to seven day user experience. And I would say this is, if you look at whatever our breakdown of experiments are, we run definitely more than average number of experiments on getting users to go from a zero to seven day streak. And a lot of this is because we’ve looked at the data for our retention curves, and what we found is that once you get to seven days, loss aversion kicks in, and you retain. So, going from a one to a two-day streak, huge jump in retention, two to three day streak, slightly less but still huge and it’s up until day seven. Once you hit day seven, it flans out. And it’s not to say that if you have a 30-day streak, you’re way more attentive than day seven, but not in the order of magnitude that it is from day one through seven.
So, we do a ton of work to get users to that point where loss aversion kicks in, and then they don’t want to lead the app. One of the fun ones that we did, and it’s honestly as much about process as it is about the feature itself, was we have a feature called streak goal, and it is… Again, so much of this stuff seems so obvious in retrospect, but it was really novel at the time. We had this idea of like, hey, maybe we’ll just goal users to hitting a certain streak length. As you can imagine, this is pretty powerful user psychology, and we started with the simplest version of this. And this is how Duolingo does a lot of our testing. Rather than design the big complex feature for V1, just do the simplest encapsulation of what that feature can be, see if it has legs, and then just add to it iteratively over time, this is partially how we get to 600+ experiments on streaks, they’re not all big ones.
But we started… And it was funny, we actually took a learning from our monetization win teams. One of the strings that they had, the pieces of copy that they had, worked really, really well was, I think it was your 5.6 x more likely to finish the course if you subscribe to Plus. Now, it’s Super, our subscription. It was a really good hook that if you really cared, you’d sign up. And so, we had a similar thought, where it’s like, oh, let’s just tell you how much more likely you are to finish the course if you get a 30-day streak.
And so, we started with that, and I think it was like you’re seven times more likely to finish the course if you have a 30-day streak, and just that message when you started your streak, I telling you that, was awesome. Huge win. Indicating an outcome… And Duolingo doesn’t have, we have a gem economy but we don’t actually have, it’s all you learning. But being able to actually talk about it in terms of the outcomes that a user would think about, in this case, trying to finish the course, was a huge win. So, this is where we started and we’re like, ah, goal setting, all right, we should go much harder on this.
Lenny Rachitsky: Found a-
Haptic Feedback and Pausing Users
Jackson Shuttleworth: Yeah, we found our thing, and now let’s just beat the heck out of it. So, we followed that up with another experiment, where we tested different lengths. So, we test 14 days and 50 days, and we found that they were all good, but they appealed to different users. And so, we started to realize, all right, well, we probably need to be more thoughtful about who we give these different options to. And so, then we followed that experiment up with, all right, let’s start with 30 days, and then we’re going to let you opt out, we’ll say, no, I don’t think I can hit it. And our thinking was, and then if you say no, we’ll hit you with an easier goal. Because we just wanted to get you to commit to a goal. And this was a fascinating one because it was a good win to give users that easier goal, to try to capture them before they said no, but it was almost just as big a win to add that opt out button.
So, we tested separately, and I’m a huge fan of testing way too many arms for an experiment, just to be able to isolate your hypotheses, but we captured just what happens if we add an opt-out button, and adding an opt-out button… And you would think, as a PM, oh, now users might not engage with my feature, that’s a bad thing, but it was a huge win to let them do that. And the learning here was that this intentionality of saying, no, I want it… Previously it was just a continue button, but now it’s like, no, I want to hit 30 days, and having that be an intentional decision for them, yes or no, even though again, this had no impact, or no impact past this screen. Everything that I’m talking about now was just that screen that day, and then was all thrown out. So, that optionality was a huge insight, and so because of that, we built a goal setting feature where you could choose between different goals, giving users that optionality was likewise a huge win.
I’ll say one final learning on this, again, you talk about friction, and good and bad friction, we thought once we built a goal picker screen, where you could pick between different streak lengths, we were like, oh, well, let’s recommend that users do a harder goal, thinking that okay, well, a harder goal is going to be better retention, and we’ll the preselect harder goal for them. And based on all the learnings that I just shared with you, you probably imagine lost pretty significantly, we realized that yes, we could speed users through the screen more by virtue of picking a goal for them, but that act of selecting, I think it’s 30 days, I think it’s 14 days, was where we were getting so much of the engagement from this feature.
Designers of the Haptic Feedback
Lenny Rachitsky: There’s so much human psychology that you all learn through all these experiments of just how to motivate people, what motivates, what demotivates, it feels like you guys should write a book on human psychology and motivation.
Jackson Shuttleworth: I feel very much like a amateur armchair psychologist with everything that, at least as far as people who want to learn languages on their phone go, I really understand those folks.
Balancing Feature Complexity
Lenny Rachitsky: Right. So, one theme I’m hearing here so far is you guys are basically just mining for gold, just looking for a vein in some mine, and once you hit it you’re like, ooh, this worked, you just go crazy on just testing all kinds of things to see how far you can take that one little thread.
Jackson Shuttleworth: Yeah, and I think, I shared how this idea came partially from a monetization win that they had, I think there’s a lot of, because Duolingo runs… I don’t know what percentile we’re in, but it’s got to be a very high percentile of per capita experiments run for a company, based on company size. We’re just constantly learning so many things and there’s a really great cross-sharing of monetization say, hey, this thing worked for us, is that something you can use? So, I’d say it’s rare that we go into something where it’s just like, let’s just try something. Typically, we have some insight because of all of these experiments that we’ve run, that hey, if we do this, I don’t know, it’s one here, or it’s worked here, it’s driven this user engagement, if we massage that and try in this scenario, or a different screen, we come in at least with a strong hypothesis of this will work.
A lot of times we do look, and we’re like, a lot of the apps that we look at actually are games themselves, so it’s like, all right, you’re playing Royal Match, or there’s the new Pokemon trading card game that I’m spending way too much time on, you look at these games and see what they’re doing, and it’s really good fodder for what we can do. But a lot of times you’re at least going in with a strong hypothesis based on what you’ve seen work elsewhere.
Team Operations: Metric Centric Approach
Lenny Rachitsky: Got it. So, just to double down on that point, it’s not just random experiments, it’s here’s a hypothesis we’re fairly confident, or has a chance to be true, let’s try it. It’s not just let’s just try everything?
Jackson Shuttleworth: Yeah. And I think that how strongly you feel about the hypothesis directly ties to how hard that experiment is. With copy, for instance, we’ve actually set up really good infrastructure for copy testing. I’m of the opinion that companies should run, as long as you have the user base to do it, copy test constantly. The amount of copy tests that we’ve had that have won, and I don’t know, you just try things and you figure out what wins is definitely legion, and [inaudible 00:27:15] massive wins from little copy changes.
Process and Experiment Management
Lenny Rachitsky: Is there an example of that, of just the impact?
Resisting the Comprehensive First Version
Jackson Shuttleworth: Going back to that goal screen, we used to say Continue, our standard CTA is Continue, and we changed that to Commit To My Goal, and it was a massive win. And again, it was like, okay, users tapping on that, what are we asking them to do? Commit to the goal, what is that going to lead them to do? Commit, not churn. Just that little copy change, that one time, right there, led to huge wins. And copy changes are so cheap, it’s just you translate, for us, we have a lot of users all over the world, and a lot of UI languages, but just come up with a bunch of ideas, translate some strings… This is one where the feedback that you’ll typically get from Luis. So, all of our changes at Duolingo go through product review that are reviewed by Luis, so Luis reviews every single change that we propose, every experiment that we run.
Typically with copy, he’s just like, I don’t know, test it, there’s nothing better than be told by Luis, I don’t think this is going to win, but sure, if you want to. And a lot of times, to his credit, he’s right, and a lot of times our intuition was right, but it’s just so cheap to do it. I think when the lift is smaller, it’s great to have a hypothesis, but you don’t need to beat it up too much.
Summary: Core Metric Strategy
Lenny Rachitsky: So, on this thread, I didn’t realize Louis reviews everything you’re planning to change, and this may be the answer to the question I was going to ask, which is, one of the criticisms of running a product and company this way, of just experimenting constantly with all these micro improvements and changes, is it can lead to something, like a monstrosity of a product and experience that isn’t consistent and cohesive, and just that often happens. Is the solution to that having the founder basically review all the changes? Is there anything else y’all do to avoid it becoming Farmville or whatever, is a good example of that?
Jackson Shuttleworth: Yeah, our product review structure where we’ve got our head of product design, one of the lead, the product management leaders, and then Luis, NPR. And because they see everything, and they have a really high product bar, and so that helps. I think over time we just, as PMs, have to look at, okay, where is our feature headed? And so, we do this with the streak at least on a quarterly basis, to look at, okay, well, what have we learned, how has our streak developed, and how do we imagine this going in near future? It’s easy to do, and in some sort of awful local maxima, if you’re not constantly looking at your roadmap and you don’t have a clear strategy. For me it’s like, if you have a clear strategy where your feature is going, hopefully all of those A/B tests are not just done to get some cheap gain, they’re done with a long-term goal in mind.
I do think though, and we do this every now and then with the streak, eventually you just hit a local maxima, and you say, you talked about launching neutral experiments, this is a great example, where it’s like all right, cool, now we need to throw a bunch of this stuff out, based on all the learnings, can we reset this real estate, can we reset this UI, reset this feature, in such a way that is just as good as what we have now but is way more plain or simple, that we can, again, start to layer on? Those are really hard experiments to get to win obviously, because they are so optimized, but they’re really important to do, otherwise, yeah, you just end up with a kitchen sink of a feature.
Limits of the Streak Feature
Lenny Rachitsky: One other tidbit I just want to mention, an advantage you all have that other companies don’t have is people want to learn a language, and so getting pushed to come back to an app for something that they want to do, it’s not an advantage a lot of products have. So, anything you want to add there, of just like this is why Duolingo might be a little different from what you’re working on.
Jackson Shuttleworth: That is definitely a benefit, if I had an app… And that this is actually why I think a lot of mobile games do streaks differently, because to say that you played a mobile game, and as somebody who plays a lot of mobile games, that you’ve done it for 3000 days in a row, I don’t know, maybe that hits a little bit different than you’ve learned Spanish for 3000 days in a row. I think the comparison set is much larger though, than a lot of companies give themselves credit for, and I think that there’s… There’s a lot of ways that companies think about their… There are very few companies I imagine out there in the world saying, oh, we don’t do some degree of good for our users… Even if it’s like a game, it’s like, I don’t know, you’re giving somebody a moment away from the craziness of their lives.
And so, I do think though that is contingent on companies who are going to figure out if a streak works for them, to figure out how can you frame the streak in such a way that a user does feel good about it. And it’s easier for a Duolingo, but I think there’s creative ways to phrase this for users. The other thing maybe just on that that I’ll say is, the streak works really well for Duolingo because with language learning it’s really hard to see day-to-day progress in becoming more fluent. And fluency is not even the right word, it’s like becoming better at Spanish or whatever, it is a years long process for someone to get better at a language.
Duolingo makes it easier, but you still got to put in thousands of hours if you’re going to reach C1 or C2 fluency, and that is really hard to track on a day-to-day, and so the streak works really well for us because we might not be able to tell you, hey, you now know 0.01% more Spanish, but we can show you, hey, you’ve gotten your streak a little bit higher. And so, I think this works particularly well when you’re an app that is doing something that’s going to be sensed or felt over a longer term to help contextualize that progress in a way that makes more sense, or at least feels more tangible to a user.
Lightning Q&A Segment
Lenny Rachitsky: Great, that was a great context. Empowering to a lot of companies that aren’t necessarily doing language learning. Okay. So, it took us on a long tangent away from lessons and experiments you ran along the journey of iterating on the streak. So, a few things you’ve shared so far is just, it started with this XP idea, and then went to a lesson, then you iterated on ways to make it simpler, maybe harder, you added streak goals, where you commit to I’m going to hit a certain goal of streak. What else? What else have you found that has worked, didn’t work, lessons learned?
Jackson Shuttleworth: Again, sticking with this one to seven day streak, the idea of a streak, particularly probably to this audience, is obvious. Like, oh, it’s a streak, it just counts how many days. We’ve realized over time that a lot of users do not understand how a streak works, and it can be as small as, well, I don’t understand how streak freezes work, or I don’t… Like my mom the other day was talking to you about it, she’s like, “Oh, I didn’t use Duolingo and I come back and my streak’s still there.” So, there’s certain elements of the feature that we can do better at explaining, but even what a streak is, it’s tracking how many days that I’ve used the app, yeah, the more that we can make the feature easily comprehensible to users, the more retentive it is. And we’ve run a number of experiments to do this.
You asked about early easy copy changes that we made. Actually, this is my first win experiment when I joined Duolingo was, when you start a streak, we have little copy at the bottom of the screen that just, I don’t know, it’s like flavor copy, we use it to celebrate you, or give you context, and I ran some tests that just tried to in eight words explained what a streak was, that was it. And it was a massive win because it really dumbed down here is exactly how the streak worked, and it really helped users just understand what they needed to do. And I think this is something that’s like, you constantly got to remind yourself, particularly if you work in tech and you’re building cool tech features, but your user base is not a bunch of tech workers, to think about, all right, who is my audience? And for us it’s not just tech workers, it’s not people in America, it’s people all over the world of all ages of all cultures, and so making sure that your feature is even something as simple as the streak is understandable is critical.
Favorite Movies and TV Shows
Lenny Rachitsky: What was the actual copy, do you remember?
Jackson Shuttleworth: It was… I think it’s still on the app, it’s like, “Start a day to extend your streak, but miss a day and it resets,” something like that.
Great Products Discovered Recently
Lenny Rachitsky: That makes sense to me. Very clear.
Personal Life Motto
Jackson Shuttleworth: Yeah.
Favorite Team Traditions
Lenny Rachitsky: In eight words, I love that. Okay. And then, when you say massive win by the way, just to give people a reference point, what does that look like? What is a massive win in this scale?
Wrapping Up the Conversation
Jackson Shuttleworth: Well, and it’s funny, and this was four years ago, but I think it was in the order of magnitude of over 10,000 DAUs for us, and actually, maybe a small bit of context. So, Duolingo really cares about the metric CURR current user retention rate, and actually our first ever Duolingo post with Lenny was the newsletter that our former head of product wrote, Jorge.
Lenny Rachitsky: Still the single most popular newsletter post of all time in my newsletter, across 300 plus-
Jackson Shuttleworth: I would highly recommend that if you were interested in this, give it a read. To summarize, basically, what we found is that, if we wanted to drive DAUs, and Duolingo cares, our growth North Star is DAUs, the metric that is most effective, where a percentage change in that metric is most effective at driving DAUs is current user retention rate. And this is just users who are not new or resurrected, getting them to come back tomorrow. And so, most of the work that our teams do, our retention-based teams do is focused on CURR. And so, the retention team that I lead focused on CURR, it just so happens the streak is the best feature at driving CURR. And so, this experiment was the biggest CURR win that we had had, and it was like a top three CURR win, anyway for us. Just this little copy. And that’s why I say, test copy 1000 different ways, sometimes it’s not the big beautiful feature that’s going to drive the huge gains, sometimes it’s just something simple as a few words.
Lenny Rachitsky: I love this. Okay, so you said 10,000 DAUs, I think that references you guys measure incremental impact and absolute numbers of new daily users you’re going to drive attributed to that experiment, is what it sounds like.
Jackson Shuttleworth: Yeah, and we do both. We’ll also look at, a lot of times I’ll look at, for retention, day one versus day seven versus day 14, a lot of what I’m looking for is for us to have a better day 14 impact than better day one impact because it means that users are retaining better over time. This is particularly for users that would see a feature multiple times. I just like absolute DAU numbers because as long as you’re controlling for different biases, like a recency bias or a novelty bias, it’s a really easy way to just have an absolute comparison. You start to look at percentage changes and then it’s influenced by who your treatment, how many users saw the experiment, but at least an absolute number is easier, in my mind, to start comparing.. Again, there are pitfalls with it, but we find that that’s a pretty useful way.
Lenny Rachitsky: That lesson comes up a lot on this podcast, and that approach to experiment.
Jackson Shuttleworth: [inaudible 00:38:25].
Lenny Rachitsky: So, yeah, you’re in good company. Quick tangent, if there’s not an answer to this, no problem, coming back to the idea of just experimenting like crazy and not creating a product that nobody wants to use anymore, is there an example of an experiment that was positive that you all decided, no, we don’t actually think this is what we want in the product, they ended up not shipping?
Jackson Shuttleworth: Retention doesn’t only work on the streak, although you would think… Most of our work is the streak. We’ve touched a lot of different surfaces over the years and there was one experiment that we launched that, we talked about XP earlier, in the lesson, the only UI elements are a progress bar at the top, and then how many hearts you have. So, we keep it really simple, and this very much speaks to the design philosophy of Duolingo, which is simpler UI is better, and we decided, hey, let’s add XP in there. And so let’s show your XP ticking up as you’re going through a lesson, that’s going to make the user feel good, it’s going to show you what you’ve earned, you’re going to be less likely to quit, all of these good reasons to do it. And then you finish the lesson and then we’ll show you’ve collected all this XP.
And it won, the hypothesis was a good one. But we realized, and I remember having this conversation with Luis, is like, cool, this is our most important screen in the app, it is our lesson, it is where users learn, and the focus here is on learning. And now you’ve added this other thing up there that could be distracting for users. And I think the question, we talked about roadmaps and strategy here, the question that he had for me, and I didn’t have a good answer for at the time, was like, so what else are you going to do with this? What’s your iteration ideas? Where’s this going to go? Is this going to make the lesson experience more gamified? And what we realized is that, honestly, it was just an easy engagement win idea, but we had touched our most sacred space in the app to do that.
And so, that was a case where it’s like, yeah, it was a nice win, but we’d added that UI element, and at least at the time it was less clear what we would do with it and we realized that long term it was just going to get in the way, and we’d rather, for simplicity’s sake, pause that, shut it down, and keep the lesson to be a little bit of learning sanctuary it was. Now, it’s funny, nowadays I think we actually have enough XP based mechanics and fun things that we can do, that I think actually a lot of the beliefs about the in-lesson experience have changed, such as something like that could work, but at the time didn’t feel good to keep that around.
Lenny Rachitsky: That is an awesome example. Hopefully we have time to talk about how the team operates, where my mind goes is like, oh, but you have all these PMs and teams that want to show impact, and the performance reviews, and all that stuff, and you’re not shipping something, they’re like, oh look, we did a win. So, I want to chat about that later, but let’s keep going on things you’ve learned and things that didn’t work along the journey.
Jackson Shuttleworth: The other thing that I’ll call out with the streak, it’s like we have the… The image of the streak is this flame. And we have the streak flame, and it’s very much core to our iconography. It’s important to acknowledge that that’s a metaphor for a retention mechanic. The idea of keeping a flame lit. And again, I think we’ve established the flame is for a lot of users as sort of their understanding of a streak, which is great, but there’s a lot of people in different cultural context and different stages of life, where the idea of keeping a flame lit to show your commitment to something makes less sense. We did some UXR in India many years ago, and this was something that just did not resonate at all there, which was a really interesting learning. And that’s something it’s like, again, depending upon what your user base is, the more global UXR you can do to understand how users are actually understanding and experience your feature, the better, because you just, again, encounter insights like this.
And so, even our screen design, we used to have a flame, it was mostly this flame that would light up every day. But again, it was like an indication of a metaphor for a mechanic, and when we redesigned it, we did this, Kurt, one of our animators, did this awesome odometer animation where it’s like your number would tick every day… It looked good, but from a product perspective, what was cool is we actually focused the design on the screen to show your number going up, and then it would say seven day streak, eight day streak. And I think that as you’re thinking about designing around a streak, don’t get too caught up into what is this the beautiful story that you’re trying to tell, at the expense of it being a really comprehensible feature. And so, as you’re thinking about product design, making that product design a clear distillation of this is what we’re actually tracking, form should follow a function here, was a learning for us. And you’ll see that now in a lot of places where we’re showing streak, we’re really leading with the number, not necessarily the flame.
Lenny Rachitsky: That’s a theme that I’m hearing again and again is clarity, don’t obsess with making it too clever, and don’t ever think it, just clarity has a big impact.
Jackson Shuttleworth: Clarity also doesn’t have to come at the expense of delight. And this is something where you hit a milestone and Duo gets, it becomes, we call him a Phoenix Duo, and he becomes awesome, and lights on fire, and I think there are things that you can do to still make the experience really exciting and delightful and celebratory, and I would not lose that, but just don’t do it at the expense… And I think it’s also about figuring out, you can get away with doing more of this for users who are deeper into their streak experience than users who are starting, where it’s like your goal for the one-day streak user is just to make sure, do they understand how this feature works?
Even something… Again, just another random experiment. At the bottom of the streak screen we have a calendar, and over the years it just looks more and more calendar-like, and that is simply because we find that the more we make it look like a calendar, days on top, little circles, the check… The more we make it look like calendar, the more that people realize, hey, this is a daily mechanic. And so, think about the screen holistically, but every single thing that you’re doing on the screen, how can you use it to communicate what is the point of this feature? How does it work?
Lenny Rachitsky:
You can also access hundreds of pressure-tested templates for everything from roadmap strategy to final decision-making frameworks. See for yourself why companies like DoorDash, Figma, and Qualtrics run on Coda. Take advantage of this special limited-time offer just for startups, head over to coda.io/Lenny and sign up to get six free months of the team plan. That’s coda.io/Lenny to sign up and get six months of the team plan. Coda.io/Lenny.
I imagine one of the biggest wins was just giving people flexibility along the journey, like streak freezes and all these things, is that a big vein of opportunity discovery?
Jackson Shuttleworth: It is. Actually, I’m going to show you one of the most thoughtful gifts that anybody has ever given to me, this is our Duolingo Serenity, or Streak Serenity Prayer, my co-lead, Antonia-
Lenny Rachitsky: It’s like knitted, right?
Jackson Shuttleworth: … [inaudible 00:46:05] this for me. It’s amazing.
Lenny Rachitsky: Wow.
Jackson Shuttleworth: And so it says, “Luis, grant me the serenity to accept the flexibility I need, the courage to reach perfection when I can, and the wisdom to celebrate regardless.”
Lenny Rachitsky: Aw.
Jackson Shuttleworth: And that actually is kind of our strategy with the streaks.
Lenny Rachitsky: I love the show and tell by the way, that was great.
Jackson Shuttleworth: Yeah. Well, I guess for podcast listeners, we’ll have to get an image somewhere. This idea of flexibility versus perfection, and then regardless, celebration, is core to how we think about the streak. Because I think for the streak for us, it’s very much a bend not break. If you’re going to miss a day, I’d rather you come back, having missed that day, to an intact streak, but if you don’t have to miss a day, I’d much rather you don’t, I’d much rather you come back and use the app every day. So, that thing on flexibility though, that’s almost certainly been the biggest, from a mechanic perspective, the biggest DAU driver. One of the earliest experiments we ran was going from you used to only be able to have one streak freeze and then we let you have either two or three. So, we tested two different arms. It was, again, another huge DAU win.
This actually is funny, it was something that… And this is, again, a callback to that growth model post from Jorge. It actually was really bad for CURR because we were basically saying, hey, you can take a day off, and that’s okay, but it was really good for, this is going to be like Alphabet Soup, [inaudible 00:47:31], a weekly active user return rate. So, basically, users who had taken a day off, we were getting them to come back more at higher rates, and so it made up for our losses in CURR. But effectively, what this meant is that, why two streak freezes work better than one was, I don’t know, sometimes people just need a little bit more flexibility than one day. But again, the really interesting insight of this experiment was that three streak freezes was actually no better than two streak freezes. And there were two competing things here, and I think this is important if you’re going to build a streak to figure out what your flexibility mechanic is.
We were getting more users to return after longer times away to an intact streak. But if you start taking three days off from any habit, it’s just going to be less likely that you return even four days later. And so, we had these competing things where more users might be returning to a streak, but a lot of users were also just not coming back, we were training them to take more time off. So, that flexibility, what’s the right amount of… We’ve again, this is another area we’ve run hundreds of experiments on, what is the right amount of flexibility? And we are constantly surprised here. I still don’t have the answer for at every point in your streak journey how much flexibility you need. One thing that I can say with certainty though is, give more flexibility when a user is starting their streak. Again, one of our biggest streak freeze experiment wins… I feel like I’m constantly saying this, one of our biggest wins, but they all were really, really big.
One of our biggest streak freeze wins was when you start a new streak, we give you two streak freezes. And again, it’s so funny to think back, it’s like how are we not doing this to begin with? But at the time, the streak freeze was an overly gamified mechanic, you had to buy them with gems, that’s our in-app currency, because we wanted the whole idea of this to feel like it was really something you earned, that there was a little bit of pain to getting that streak freeze. And so, we tested though, what if we just give users when they start off their streak two streak freezes, and holy smokes to that win.
And it’s sort of obvious now, in retrospect, but if you have a one or a two or a three-day streak, it’s really easy just to let it die and restart, again, you need to get to seven days, what we’ve seen in the data, for it to really lock in. And so, giving users more flexibility so that it’s harder to lose their streak initially, and then conversely, and this is what we keep learning, eventually, once people get on long streaks, you don’t want to give them as much flexibility. Because there’s a lot of times where, yeah, users don’t… And I’m like this, I’ve got a 400-day streak. Note that that is a lot less time than I’ve been at Duolingo, I have lost and restarted streaks a lot of times at my time at Duolingo.
But you start getting on long streaks, and you really care about this feature, you really care about your streak. And most people, as long as you’re not backpacking through, I don’t know, the back country of Utah, you’ll be in a place where you can get service. And so, figuring out who is the user that actually could use Duolingo, and not conditioning them to start taking days off that they didn’t otherwise need to do is important to figure out where that line is for your feature.
Lenny Rachitsky: This is fascinating. You can also buy a streak, right? With money. That’s a feature, right?
Jackson Shuttleworth: Yeah. And it’s funny, this is also something that we [inaudible 00:50:54]-
Lenny Rachitsky: You can buy a freeze, sorry, not a streak.
Jackson Shuttleworth: Yeah. So, you can buy a streak freeze, and the way it works is you can buy gems, and then you can use those gems to buy a streak… And this is something we wrestle with. We’re actively working on an experiment, right now that’s having a small hit to revenue, but it’s a really nice win for retention, and I think it’s actually worth thinking about from day one, as you’re building a streak, do you see this more as a monetization feature, or do you see this more as a retention feature? What’s the role of monetization in this? What’s the role of retention? And I think for us it started out much more organically, and so we have a lot of monetization hooks, that again, is the retention PM, I would love to get rid of. But again, it’s sort of part of how the streak works right now. And so, we always have this tension of, hey, if we start to make it harder to buy streak freezes, then fewer people buy them, buy gems to buy them, and so there’s this more convoluted series of impacts that happen.
Lenny Rachitsky: Yeah, no, I love that people wanting to buy streak freeze is like the ultimate sign of how much streaks matter.
Jackson Shuttleworth: Yeah, streak freeze is the other big one that we’ve recently demonetized, or introduced a free option for, is getting back a lost streak. So, used to lose a streak, we had a feature, in the day, back in the day, called Streak Repair, we’d give you your streak back, you had to pay gems. But what we found that worked way better was a feature called Earn Back, and this is basically where you would have to do a certain number of lessons, as long as you came back within a window soon after losing your streak, do a few lessons and we just give you your streak back. And that was such a retention winner. And again, what we thought about was it feels like you’ve earned it so much more when you’ve done… You deserve to have your streak back, we haven’t cheapen the streak because you’ve done something.
And in this sense, this idea of cheapening the streak is something, like from a philosophical… Philosophy of the streak. From a philosophical level, we wrestle with all the time, of, cool, we’re giving out more streak freezes, at what point do we cross the line and users start to realize their streak means nothing? Now, everything that we’ve seen, users are totally cool with using streak freezes and still thinking about their streak is this meaningful thing, but my co-lead, Antonia, who made that awesome cross-stitch for me, she is the keeper for us of the sanctity of the streak. And a lot of times as we… And I think this is really important to have, as you’re thinking about building your streak. You can almost always get engagement wins, up to a certain point, by just cheapening the streak, making it easier to extend, letting users have more flexibility, but you kind of got to hold the line at some point.
And it’s not clear where that line is. And once you… You talk about one-way doors or two-way doors, there’s a point where you go too far and it’s a one-way door, and all of a sudden those users, those 9 million users on one-year streaks don’t care about their streak anymore. And that is, I don’t know, again, retention PM perspective, that’d be an extinction level event for us. I don’t want all of these users to stop caring about their streak. And so, to have somebody who is invested in the sanctity of this streak, and for us it’s Antonia and Luis, he’s very good about this, is really important, just so you make sure you don’t go too far.
Lenny Rachitsky: That’s an awesome insight. So, to protect… And push notifications I think are another example of this in general for companies, how much is too much? Because everyone is just like, let’s just send another push, it’s fine, just one more. And so, your solution to that is a person is like the keeper, and almost the gatekeeper, plus the founder, of how far is too far.
Jackson Shuttleworth: It’s good if you can have that. I think push notifications are also easier because there’s a lot of things you can do around, all right, we’ll send a budget cap for how many notifications we’ll send, you can-
Lenny Rachitsky: It’s like [inaudible 00:54:38] policy.
Jackson Shuttleworth: Yeah, policies. But I think with a lot of things, at least for the streak, it’s harder to create policies for in the same way, a lot of it has to be done based on feel, and so you just got to use your best judgment at times.
Lenny Rachitsky: Sweet. Okay. Any other, maybe one or two more lessons from this journey of what streaks has become today.
Jackson Shuttleworth: You mentioned notifications, and I’ve mentioned this a few times. One… It’s funny, you tend to think of, exactly as you say, you can just always send another notification, it’s going to be some win, and at some point it’ll be a bad experience, but it’s tough to see that. There’s actually a notification that we… So, we send two notifications related to your core streak each day, the first is a practice reminder, we send it, this is actually an interesting insight, 23 and a half hours after you practice the day before. Whoa, that is a lot.
Lenny Rachitsky: 23 and a half. Okay,
Jackson Shuttleworth: So, basically, if you practice at noon today, we’ll send it to you at 11:30 AM tomorrow, and we have done-
Lenny Rachitsky: And it’s, because it’s like assuming they are free in that time the day before, maybe they’ll be free at the same time?
Jackson Shuttleworth: And we actually moved, we used to let users set this practice reminder time, and our thinking was, cool, you’re going to say 7:00 PM, that’s when I really want to extend my streak each day, and then you know what? I say this to somebody with two kids, life gets in the way, life always gets in the way, and when you think you’re going to practice will change, your life will change, whatever. And what we realize is the best indicator of when you should practice was when did you practice the day before. We could almost certainly get more detailed, we have tried a bunch of ways to have much more complex logic, and what always wins is 23 and a half hours.
Lenny Rachitsky: That’s so interesting. Revealed behavior versus stated.
Jackson Shuttleworth: Yeah, exactly. So, again, we send this practice reminder 23 and a half hours later, the other thing that we’ll do though is we’ll send a, what we call a streak saver, and this is at 10:00 at night, if you have not extended your streak, we’ll send you a message saying, hey, it’s your last chance, this is it, if you don’t extend your streak. And you would think that, that’s kind of spammy, that’s kind of annoying, to get a notification from an app at 10:00 PM, but what we found is because people care about their streak, their streak is this good thing that they attach positive emotions, that they don’t really want to lose, that notification reminding them, hey, come back and… People see this by and large as a positive notification and not a negative notification.
Obviously, it serves our purpose as well of getting users to come back and not lose their streaks, but again, I think if you can think about your notification strategy related to what is the feature that it’s tied to, how do users perceive that? You can almost certainly, not get away with more, but you can be thoughtful about notification load, and when to send notifications, and again, for us, this late night message, that’s also highly impactful, super good, is actually something that could be perceived as spammy, but a lot of our users really do… Somebody who, it’s often late at night, and I work here, and I’m like, oh, forgot to do my… I was think about Duolingo all day, here it is, 11:15, and I still haven’t done it, that message is really powerful.
Lenny Rachitsky: Yeah. It has saved me many times, I totally know that message, and I love that it’s a late night message from an app, very rarely do you actually, are happy about that, and I love that this actually is a good example where-
Jackson Shuttleworth: It’s really funny, all of the stories that you hear about people extending their streak, if you look around a Duolingo party, where it’s like 11:30, 11:45, all the Duolingo employees, they’re are doing their lesson at the last minute. You always see these pictures of people in the club doing, or at a concert doing Duolingo, and yeah, because it’s like otherwise you’re going to use a streak freezer, or God forbid you will lose your streak.
Lenny Rachitsky: That’s so funny. Okay, anything else? And if there’s more than one more, definitely share, but any other really interesting lessons or wrong turns or insights?
Jackson Shuttleworth: I talked about streak freezes, and we’ve done a lot with streak freeze, but I think if you’re going to make flexibility a thing, it’s probably also useful thinking about how do you celebrate perfection. And so, we have a feature that we have, it is the simplest thing in the world, it’s called Perfect Streak. And it’s just, if you don’t use the streak freeze for a few days, we make your streak look gold, and we make your little progress bar on the calendar just look a little bit nicer. There’s no reward for doing it, you don’t get anything other than this nice little indication, and it is awesome. It is a simple feature, it is ultra not complex, and it is really powerful, not only for getting users to, as a bit of a reward, they’d be hey, get to seven days without using a streak freeze and your streak becomes perfect, but it’s also a really nice indication of users who aren’t using streak freezes.
Here is the thing that if you don’t use a streak freeze, which, again candidly, I would love for you never to use a streak freeze, if you don’t use a streak freeze, your streak will stay perfect. It’s funny, we actually just, we’re constantly responding to bug reports about the streak. It is… I swear to God, we have the best infrastructure around this feature because it is so important. We had an employee who lost her, I think a four-month perfect streak, and it was a big deal for her because she did her lesson, and she was crossing international dateline, there was a bunch of stuff going on that was like, it was just kind of weird in our backend. But people start to care about perfection as much as they do their streak, and for that person it was a big deal when they lost their perfect streak. And so, this is just an example of, look, if you’re going to go after flexibility, which is good, finding a way to pull users back into perfection is a really important counterweight to have.
Lenny Rachitsky: What I’m imagining is you guys need a Amazon style chatbot that just gives you the streak back, it’s just like, okay, here you go.
Jackson Shuttleworth: We have very much… So, we have, if people lose their streak, there’s ways to get in contact with us, but we’ve actually thought about that, where it’s like, okay, we should just build a self-service feature, and if we think that your excuse is good enough, whatever, we’ll just [inaudible 01:00:34].
Lenny Rachitsky: Yeah, yeah. [inaudible 01:00:35].
Jackson Shuttleworth: Because again, I’d much rather you be on a streak than have lost it, particularly [inaudible 01:00:41].
Lenny Rachitsky: Right. But it also can feel that easy. I love this, I also love this point about just the power of the animation and user experience having impact, that’s really interesting. Is that something you find often, just celebrating and making it feel really amazing without copy or any feature is just like, holy, you’re awesome?
Jackson Shuttleworth: This is another thing where it’s like when users care about the feature, using not only animation, haptics, sound effects, using… And it’s funny, we don’t have sound effects on the streak, this is probably something we’ll look at in the not too distant in future, but haptics are something we have done a lot of testing on-
Lenny Rachitsky: Like the phone vibrating in various ways.
Jackson Shuttleworth: Yeah, exactly, your phone, there being a really cool haptic pattern as you extend your streak, all of this stuff wins. And it’s cool because I think it wins… There’s a few reasons. One is it just makes you feel good, you get some cool moment in your streak, and we celebrate you, and we celebrate you in this visual way, and your phone’s buzzing, it just feels awesome. The other thing it does is it causes you to pause on that screen, and I think there’s this desire, as you think through a lot of, as PMs think through, oh, how can I get users through this funnel as painlessly as possible? I talked about [inaudible 01:01:52]. There’s a lot of times where I don’t, I want you to stop. I want you to stop and land on the screen.
You got to be careful not to do this for too many screens, but the one big ones, sometimes I just want you to pause there and enjoy the moment. If I can get you to enjoy the moment more, you’re going to care more about your streak, and you’re going to be coming back tomorrow. And so, animations that are cool, and that cause you to really soak it in, haptics that feel good, all of that comes together to make you really focus on that moment, all of that just gets users more connected to their streak. So, animation in the right times works well and it’s something we’ve had win quite a lot.
Lenny Rachitsky: Who designs the haptic stuff? Is there a haptic designer?
Jackson Shuttleworth: For the longest time it would be a product designer, or actually, it initially started as the engineer would be like, all right, would cobble together haptics, based on what they felt good, then it became a product design role, where they would use their best judgment. We actually just recently required an animation studio, Hobbs in Detroit, and now they are the sort of keepers of, they do a lot of motion design work, haptics very close to that, and so they do a lot of that. I do remember trying to hire for a while a haptics contractor, like haptics design, and it was the saddest hiring I’ve ever done, because it was just, I don’t know, it was such a specific… I don’t know. I just went through a lot of people who, it’s just a really tricky space of kind of sound effects, kind of motion design, sort of technical…
Lenny Rachitsky: Yeah, such a unique role.
Jackson Shuttleworth: Very unique specific skill set.
Lenny Rachitsky: Right. And there’s very few apps that really need this this deeply, so you’re almost creating this person.
Jackson Shuttleworth: Yeah.
Lenny Rachitsky: That is fascinating. That’s like a whole podcast on its own. By the way, I was going to say as you’re talking about this, I love that it’s a win to celebrate people that don’t lose their streak, you introduce this way to make it flexible, and that’s a big win, and then you go the opposite direction of, if you don’t use this feature, you also feel even better.
Jackson Shuttleworth: Well, and it’s funny, you talked about the danger of feature bloat, or we sort of talked about the danger of feature bloat, this is actually something I’m constantly thinking about with this. We have the streak, but then we also have the perfect streak, and we count how many weeks you’ve had a perfect streak. Well, all of a sudden we have two streak numbers that are competing with each other. It’s funny, we actually don’t introduce the concept of a perfect streak until after you’ve hit seven days, and some of this is just because the cognitive load of additional streak features. A lot of our cooler streak features, you got to get on a long enough streak. And not to say we haven’t tested it because we have, because we test everything, introducing these features earlier, but what we’ve found is that pretty universally they lose when we introduce too many things, too many concepts to users too early in the experience, it’s just hard for them to manage.
Lenny Rachitsky: Okay, sweet. I know that we can go down this track for hours and hours, there’s endless learnings about all the things you all have done along the journey. I want to shift to talking about how your team operates. So, there’s a lot of threads you touched on of just how a team can do this so well, shift 600 experiments, as you said, continue to find opportunity. What are some maybe lessons or advice you’d have for folks that are like, oh wow, I want to work more like this, from your team’s experience, how does your team to operate that folks can learn from?
Jackson Shuttleworth: Yeah, maybe just a little bit of context. Duolingo cares a lot about… So, most of our teams are metric-based teams. So, we do the most work with streak, but the metric, what we really care about at the end of the day is CURR, and DAUs, because we see that DAUs hit CURR. And so, when you can be really laser focused on, my goal each quarter is to make this metric go up, I think it’s much easier to make sure that you’re working on the highest ROI thing. I think when you think more about like, oh, I want to make this feature better, I think it’s easier to get lost in what better means, and how you think about better. And so, I do think that having a really strong degree of focus as a team on what is the metric that I’m caring about, and how is that directing my efforts is-
Lenny Rachitsky: Versus feature-oriented. So, basically your teams are structured around a metric/a goal/outcome versus we own this feature, or this product.
Jackson Shuttleworth: So, retention owns streak, I guess, but that’s only because we’ve seen streak drive CURR better than any feature. But we are not, we have this IAP hook with our streak freeze purchases, there are other teams that work on, that can and have worked on the streak, because it’s not ours to say, no, no, no, we do all the iterations here, we just know that it drives our metric better. In the same way that leaderboards, we have a team that focuses on how much time you spent, we want users to spend more time on Duolingo so they’re learning more. Leaderboards is the best vector for doing it, so that team does a lot of leaderboards work, but every now and then I have an idea that I think will be highly retentive, and I will go in, and I’ll pitch to them, and then we’ll do some change to the leaderboard to make it more retentive.
But I do think having that clear metric of we’re trying to drive CURR not we’re trying to just make this feature better, helps at least make sure, give the team clear marching orders, and that focus I think is really good for prioritizing backlog.
Lenny Rachitsky: Cool. This is a really important point, this is the same way Airbnb worked, when I worked there for a long time, is it’s, here’s a goal that we want your team to be responsible for, you can work on any product you need to hit this goal, as you said, often various products are most connected to what you’re doing, but what you’re describing is, even though a team’s kind of… I imagine you own it from a [inaudible 01:07:29] perspective, and you’re like are the shepherd of this part of the feature because it hits your goal, helps your goal most. But any other team can come in and be like, hey Jackson, we need to work on some streak stuff to help with learning, you’re like, go for it. Just a tangent there. Do they work really closely with your team if they want to do some work in the code? How does that work logistically?
Jackson Shuttleworth: Yeah. If you are, again, this is where I say there’s soft ownership, we’re not against teams doing things to the streak, but if we’re going to do something given we probably have multiple quarters worth of a roadmap around the streak… I say probably, we do. Multiple quarters of roadmap for what we can do to the streak, if other teams want to come and mess with it, okay, we got to just figure out how is that going to work with what our plans for the streak were, how do we make sure a lot of times when teams are coming in thinking, hey, let’s do this to the streak, they’re in context that we might have, and so there’s as much of a much simpler version of what we’re doing now, a bit of a knowledge sharing, of saying, all right, well, this is what we think about the streak, this is what we’ve seen work, hasn’t worked, how does that influence some of the hypothesis that you have. And so, I think getting that really… Making sure the juice is worth the squeeze.
Lenny Rachitsky: Good old-fashioned product management work right there.
Jackson Shuttleworth: Yeah.
Lenny Rachitsky: Cool. What else is interesting about how you all operate and how you all work to achieve this sort of success?
Jackson Shuttleworth: Again, my team lead runs, Antonia runs the most… A really process, if you’re going to run this experiments, you have to be really process-oriented and really thoughtful about which experiments am I going to run when, how is that going to set up the next one, there’s heavy Jira automation, I think sometimes the Atlassian suite makes my eyes bleed, but there’s a lot of times where that degree of process helps the team unblock engineers and make them move really fast. And so, making sure that you have really good process around how are you going to run so many experiments, it’s worth investing in.
Lenny Rachitsky: Can you follow that thread actually? Just when you say that, what does that look like? What are some elements of that process to make this work efficiently?
Jackson Shuttleworth: All the way down to, really detailed roadmaps around, all right, we’re running this experiment is based on the results of this experiment, or might hook into an element of this feature, how do we make sure that we’re lining up implementation on this so that as soon as this thing runs and we’re ready to go, we can start rolling out the next one. I hate features just sitting around and us not, again, continuing that thread. So, it’s not just thinking about what’s our engineering bandwidth, but also what’s the design bandwidth to make sure that we have the next iteration of this feature ready to go. We’re planning months out, as we think about these feature iterations, even small ones, feature iterations, because when you lose cycles, not pushing on a feature, it’s just sort of lost opportunity. And so, everything from being thoughtful about engineering roadmaps to design roadmaps to product roadmaps, all of that needs to come together in a system.
Lenny Rachitsky: So, essentially, mapping dependencies across function, and you’re saying in Jira you can do this.
Jackson Shuttleworth: You can do a lot of it in Jira, there is a non-zero amount of Google Docs that we have, that sometimes does things a little bit… I don’t know, sometimes it just looks a little bit nicer, it’s a little more flexible.
Lenny Rachitsky: Yeah. [inaudible 01:10:38].
Jackson Shuttleworth: But Jira is our, it is where the motherload of process is.
Lenny Rachitsky: Great. Okay. What else?
Jackson Shuttleworth: Another thing that I’ll just say is, we really resist the urge to do the big V1. And I think this is, I shared the streak goal example, where, a lot of times when we’re exploring something we will say, okay, well, that’s cool, how do we strip away a bunch of stuff and figure out what our core hypothesis is? And then, just ship that thing first as a V1. Because it’s easy, and I’ve found this time and time again, it’s easy to add things, features that make them win, I’ve worked in retention engagement long enough, I can add, I know enough things to pull, and bells to add, and whistles to make something win, but there’s a lot of times where it’s like cool, t [inaudible 01:11:27] because all the whistles you added, not because of what your core hypothesis was, and a lot of times if you can just really simplify what the feature is, it’s also much easier to ship, it’s easier to design.
You’re not designing for a whole system, you’re designing for something much simpler. And so, getting everybody to think that way, allows us to end up shipping faster, shipping simpler, designing faster, getting faster approval, getting insight, and then doing what I talked about with streak goal, being able to run iteration after iteration after iteration, add these things iteratively. And then not only by doing this are you able to move faster, but you get confidence at each step of the way, that, hey, my series of hypotheses is actually born out. Or if it’s not, cool, then we’re going to drop that part of the feature, and then just ship what actually matters.
Lenny Rachitsky: If I can try to summarize the broad lesson so far that I’m hearing, and maybe you would’ve shared this, but I’m just thinking, if I were to try to design a company to operate, that we all operate, you essentially map all the levers that drive the business. So, you have this mapping of all the metrics that drive up to growth and daily active users. CURR ended up being the biggest specific metric to drive growth long term, so there’s imagining a tree of all the opportunities you could work on, you found this is what is most connected to our growth. You basically just start mining, I don’t know if mining is the right metaphor, but just looking for things that move that specific metric.
You just look and poke and explore, and then once you find one, you just go real deep on trying a lot of different… You come up with a hypothesis, and a strategy of here’s how we think we can do this, and how we can move this, and then you just try a bunch of stuff. There’s also this element of the Arrested Development quote, “There’s always money in the banana stand,” comes to mind, where it’s just keep working on, see there’s more, there’s going to be more opportunity at this.
Jackson Shuttleworth: When I joined Duolingo, the PM that I took over for, Anton, who used to lead the retention team, I remember saying, “Dude, the streak, it just counts up, you guys have been testing on it for years, how much more work can we do on the streak?” And he was like, Jackson, you child… He didn’t say exactly this, but this is how I felt it. Like, Jackson, you child, we’re not even 30% of the way optimized. And four years later, I say that with such conviction, we are so far away from… We’ve made a ton of strides, but we are still so far away, and every quarter where we ship a ton of wins and improvements to the streak, it just continues to prove to me that there is so much more to be done.
So, I think your framing of it is… And I would say there’s a lot of thought that goes into, again, I talk about the strength of the hypothesis that you have to have as you start to build out larger feature strategy, I do think it’s really important to not just do a bunch of random stuff but do it with intent, with a goal in mind, otherwise, you do end up in these local maximas. But yeah, there’s still a bunch of stuff that we haven’t tried that I think we have high confidence in working out and so we’ll keep doing that.
Lenny Rachitsky: Are there any other, say, lasting lessons from this journey that if someone were to try to operate this way, build streaks into their product, anything you’d recommend?
Jackson Shuttleworth: Yeah, I really do think it starts with… Streaks are an engagement hack. You can make your app more retentive, I’m almost positive, almost every app out there can make it more retentive. It is loss aversion, that is, again, armchair psychologist Jackson, it’s just a thing that works on humans. But if your app is not something that users want to use every day, or whatever cadence you want your app to be, to work on, it’s going to be, you’re only going to get so much from that streak, and honestly, it’s probably going to distract you from what really should matter, which is making your app something that people want to use every day.
And so, if you start focusing on the streak but you haven’t made that an enjoyable experience, you’re just going to waste a lot of time, honestly. And so, I think making sure that you have your core loop of your app figured out, that it is giving value to users, it is something that they want to come back to every day, that really sets the stage for something to layer a streak on. So, resist the temptation, if you don’t think you’ve reached that point, to go too hard down the path of streak.
Lenny Rachitsky: That’s a really good point. Just like a streak is not going to solve your problems if people don’t actually care about the core value you’re providing.
Jackson Shuttleworth: No, and honestly, it’ll probably cause more problems if what you end up focusing on is how do I make the streak highly engaging, but your app is… You’re wasting time that could otherwise be better spent on solving more critical problems. So, that’s one learning, the other thing that I’ll say is, we met with one of our board members, Bing, Bing Gordon, a few weeks, or a few months ago rather, and he had this comment where he was just like, “The reason why users care about your streak so much is because you care about your streaks much.” You being Duolingo. The reason why users care about our streak so much is because Duolingo cares about the streaks so much. And we’re like, what do you mean? Well, he’s like, well, after every session you see a big streak screen, and it’s animated cooler than almost any other screen in the app, and then sometimes you see some other screens, and there’s all these other… You don’t let a user forget it, you talk about them in messages.
And so, I think it’s worth thinking about, look, if you’re going to build a streak, and then you’re going to [inaudible 01:17:06] it off into the corner of your app, where users aren’t going to see it, they’re probably not going to care about it as much, which might be fine, because there might be other levers that you think are more important to pull on, but there’s a reason why we focus as much on the streak as we do, and that’s because we want it to be top of mind for users. And that’s not by accident then that users start to care about it. And so, I think just as you’re thinking about building the streak, making sure that you’re giving it the visibility it deserves, if you want it to have the kind of impact that Duolingo has, it’s sort of an important hierarchy principle to think about as you design things.
Lenny Rachitsky: That’s such a good point. You look for cues to the app of tell me what I should pay attention to, what’s important? If you’re just like fire, explosions, you made a streak, oh, maybe I should pay attention to this feature. And then the push notifications obviously encourage you there too. Anything else along those lines?
Jackson Shuttleworth: Maybe final thing is, look, we ran so many tests on our Duolingo streak to figure out what worked. We have a philosophy at Duolingo, of test it first. We are a lot of times willing to test things. I really think that if you’re going to try to introduce a streak, or you want to improve on the streak you have, don’t get too caught up in the philosophy of everything, make sure your hypotheses feel like they’re good, but my recommendation is just try things. And this is, again, you said it earlier, it’s like this is as much human psychology as anything, and as soon as that becomes the case, you kind of just got to understand what users respond to. And the easiest way to do that is to stop spending time batting around ideas in a conference room and just try some stuff. So, huge recommendation to if you’re going to invest in a streak, try and figure out what works through testing with users rather than trying to get it perfect on the first try.
Lenny Rachitsky: Say someone’s listening and they’re like, should we do streak, is this worth doing? What’s your take on just the chances that a streak feature would be helpful to another consumer?
Jackson Shuttleworth: I am well known for saying in the company that I think every team, every app could benefit from a streak. Now, how you implement it is very different, and I think you got to, what is your user’s use case? If they’re going to come use, I don’t know, tax software… Okay, you know what? Now that I say this, tax software would be a hard one, but maybe it’s all about you need users to come back every day, during the tax season, or how many times… I don’t know. Now, that I say this out loud-
Lenny Rachitsky: Times you upload your [inaudible 01:19:37] forms.
Jackson Shuttleworth: Yeah, that is a hard use case. But the vast majority of companies I think have a good idea of like, all right, here is my ideal use case, I want users to come here three times a month, that would be ideal. Or four times a month. You can build a streak to work. Peloton has weekly streaks because the idea of doing a Peloton workout every single day was hard for this user during COVID. It was just like every now and then you get on the Peloton, that was great, but the idea of a weekly streak was something that I could keep up. And so, I think figure out what your usage pattern is, as a user, and then build your streak around it. But as long as you’re not a really, again, the tax example is probably a good counterfactual, but as long as you have some degree of frequency in your use, I think almost anything can have a streak.
Lenny Rachitsky: So, Duolingo, it’s, again, a $14 billion company, this feature, possibly the most contributing factor, other than the core product, to that level of success in market cap, and it’s hard to imagine another just feature of a product that has had this much impact on growth and revenue and building this sort of business. So, I love that we spent this much time on it, the motherload, the motherload of advice and insights. So, thank you again for putting-
Jackson Shuttleworth: Of course. Very fun.
Lenny Rachitsky: With that, we’ve reached our very exciting lightning round. Are you ready?
Jackson Shuttleworth: I’m ready.
Lenny Rachitsky: First question, what are two or three books you’ve recommended most to other people?
Jackson Shuttleworth: All right, I’ll start with A Guide to Midwest Conversation. So, I’m based in Kansas City, I’m a proud Midwesterner, and us Midwesterners talk in a certain way. I think you hear about Minnesota nice, but we tend not to say what we mean, and it is a very funny primer into what Midwesterners actually mean when they say what they say. So, highly recommend reading that.
Lenny Rachitsky: I like that you give that to people, just like, here’s what I might be telling you, which you may not read.
Jackson Shuttleworth: My wife is German, and I gave it to her so she could better understand.
Lenny Rachitsky: I see German being the opposite of that, okay.
Jackson Shuttleworth: Very different. Another book, this is a good one, Fate is the Hunter. This is a really cool book, it’s a memoir of one of the early commercial airline pilots, and it is wild to hear the stories about what flying was back in the day. I’m a former management consultant, I flew every week for almost six years, and I never once had to worry about, am I going to make it to the other end of this flight alive? That was not the case back then, and so some of the stories about what it used to be like to be a pilot on some of these planes, before modern aviation technology, is fascinating, and makes you really appreciate what we have.
Lenny Rachitsky: It feels good to read a book like that, being a software PM or engineer or whatever, how different that life is.
Jackson Shuttleworth: Hardware is hard.
Lenny Rachitsky: Hardware. Oh man, it’s not haptic design. Okay. Next, unless there’s any other books you’re going to share? No, okay, great. What’s a favorite recent movie or TV show you’ve really enjoyed?
Jackson Shuttleworth: So, I have two kids, I watch a lot of Bluey, it’s really good, I swear, it brings me no shortage of joy. But adult show or show not meant for four-year-olds that I have watched, I just finished the latest season of Emily in Paris, man, wonderful. I realize it’s not the highest brow of television, but just beautiful people in beautiful cities, solving problems that are not earth-shattering, sometimes it is nice to just tune out. Also, I’m learning French on Duolingo, slight plug for the app, I can understand a lot of the French that is being spoken, and there is no better joy than having invested as much time as I have in French, and actually being able to use it. So, huge fan of Emily.
Lenny Rachitsky: That is so funny, what a fun Venn diagram of interests. My mother-in loves Emily in Paris, I saw someone tweeting about what Visa is she on? How is she still in Paris?
Jackson Shuttleworth: Yeah. You best just not ask questions, there’s a lot of questions for this show that are better left unasked.
Lenny Rachitsky: [inaudible 01:23:40] Okay. Do you have a favorite product you recently discovered that you really like? Other than Duolingo.
Jackson Shuttleworth: Last week I went to Home Depot, and I bought a new ladder, and ladder innovations, you don’t think of often, but you can make one of the legs go a little bit further than the other leg. And as somebody like myself who has a house that is built on a slight slope, every time I go up on my ladder, I take my life in my hands, but with this ladder, I’m always even. I cleaned my gutters twice last week just because of how awesome this ladder has… How much this ladder has changed my life. So, ladder innovation, I don’t think it gets talked enough about and so I’m happy to give it the spotlight it deserves.
Lenny Rachitsky: I appreciate you doing that, and it’s the first ladder recommendation we’ve had on the podcast. Two more questions. Do you have a favorite life motto that you really find useful in work or in life they share with folks?
Jackson Shuttleworth: This probably will not be much of a surprise based on how I’ve talked about our willingness to test things, but you miss 100% of the shots that you don’t take. I’m a big fan of just trying things, even if your possibility of success is not 100, because you learn a lot along the way.
Lenny Rachitsky: Final question, do you have any fun traditions at Duolingo, amongst either the PM team or the company in general that might be delightful to share?
Jackson Shuttleworth: We have way too many traditions to count. I will share the weird tradition that we do at every retention standup, and this started during the pandemic, we obviously used to stand up in person, and then when we went remote, we did this thing where whoever’s the last person to go would count down 3, 2, 1, and then we’d all try to clap at the same time, which was kind of fun and dorky, but we fell in love with it, and four years later we’re still doing it. Recently we’ve added, we all say yee-haw in unison afterwards, I can’t tell you why. But trying to synchronize a clap via Zoom, and then all shouting yee-haw… I did this in a phone booth the other day and after I came out someone told me, “You know that those aren’t as soundproof as you think,” but when you get a good opportunity to give a yee-haw, you can’t pass up on it, so.
Lenny Rachitsky: I love these little things, they sound so minor, but they’re such important elements of team culture and tradition, and so important for PMs to find ways to just have fun and do something ridiculous.
Jackson Shuttleworth: I will say, it took a while to get people behind shouting yee-haw, but now that we have people doing it, you can’t take it away [inaudible 01:26:08], we all love it.
Lenny Rachitsky: Oh man, I called our all hands for a while, y’all hands, feel free to steal that.
Jackson Shuttleworth: You get it, you get it Lenny. Yeah.
Lenny Rachitsky: I get it. Jackson, this is incredible, I feel like people are going to listen to this with notebooks, and just like, okay, here’s a bunch of ideas we should try with whatever we’re working on, whether it’s streaks or not. Thank you so much for being here. Two final questions, where can folks find you online if they want to reach out, learn more, learn more about Duolingo? I know you’re hiring product managers, so share more there, and finally, how can listeners be useful to you?
Jackson Shuttleworth: Yes, you can find me on LinkedIn, that is where most of my online social media is, so Jackson Shuttleworth, and then how people can be useful to me, yes, as you said, we are hiring. We’re actually hiring for my team. Are you interested in thinking about streaks as much as we do? We might be the right home for you, so please, you can apply on our website. We’re also hiring from a number of other product management roles, and they’re all as thrilling as this work is.
And then, I’m always interested about how other companies have implemented streaks and what they’ve learned, and so what I’d say is if you’re a company who’s implemented a streak maybe in a different way than Duolingo has, or you found a whole ton of success and another vector, another element of the feature that we didn’t talk about today, I would love to know more. I used to catalog basically every streak I found out there, and as it’s become more of a popular feature, it’s just been hard to keep up on. So, if you have interesting streak insights to send my way, I would love to hear them.
Lenny Rachitsky: I love that, a collection of all the best ways of doing streaks. Jackson, just I want to say congrats to your team and you for having so much impact. This is like the dream of a lot of PMs and teams, is to see this much impact and continue to ship wins, and so congrats, nice work.
Jackson Shuttleworth: Thank you very much.
Lenny Rachitsky: Thanks for [inaudible 01:27:59]. And with that, Jackson, thanks so much for being here.
Jackson Shuttleworth: Yeah, thank you Lenny, this was a lot of fun.
Lenny Rachitsky: Same. Bye everyone.
Thank you so much for listening, if you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at Lenny’sPodcasts.com. See you in the next episode.
Glossary
| English | 中文 |
|---|---|
| Anton | Anton(Jackson 的前任产品经理,曾负责留存团队) |
| Antonia | Antonia(Jackson 的联合负责人) |
| bend not break | 宁弯不断 |
| Bing Gordon | Bing Gordon(多邻国董事会成员,游戏行业资深人士) |
| Bluey | 《布鲁伊》(Bluey,澳大利亚儿童动画) |
| C1/C2 | C1/C2(欧洲语言共同参考框架 CEFR 的语言能力等级) |
| copy testing | 文案测试 |
| CTA | 行动号召(CTA) |
| CURR (current user retention rate) | 当前用户留存率(CURR) |
| DAUs | 日活跃用户数(DAU) |
| Duolingo | 多邻国 |
| Earn Back | Earn Back(挣回来,多邻国通过完成课程找回连胜的功能) |
| Emily in Paris | 《艾米丽在巴黎》(Emily in Paris,Netflix 剧集) |
| extinction level event | 灭绝级别事件 |
| Farmville | Farmville(一款社交农场模拟游戏) |
| feature bloat | 功能膨胀 |
| Group PM | Group PM(集团产品经理) |
| haptics | 触觉反馈 |
| Hobbs | Hobbs(底特律动画工作室,被多邻国收购) |
| Home Depot | 家得宝(Home Depot,美国家居建材零售商) |
| IAP (in-app purchase) | 应用内购买(IAP) |
| Jorge | Jorge(多邻国员工,增长模型文章作者) |
| juice is worth the squeeze | 值得花这个精力(收益值得付出) |
| lightning round | 闪电问答 |
| local maxima | 局部最优 |
| loss aversion | 损失厌恶 |
| Luis | Luis(多邻国联合创始人兼 CEO Luis von Ahn) |
| one-way door / two-way door | 单向门/双向门(亚马逊决策框架中的概念) |
| Peloton | Peloton(家用健身器械与在线课程平台) |
| Perfect Streak | 完美连胜 |
| Phoenix Duo | 凤凰 Duo |
| PM | 产品经理(PM) |
| Pokémon | 宝可梦 |
| Retention Team | 留存团队 |
| Royal Match | Royal Match(一款消除类手游) |
| Serenity Prayer | 宁静祈祷文 |
| soft ownership | 软性所有权 |
| standup | 站会 |
| streak freeze | 连胜冻结 |
| Streak Repair | Streak Repair(连胜修复,多邻国早期付费功能) |
| streak saver | 连胜拯救(streak saver) |
| streaks | 连胜(连续打卡天数) |
| TEDTalk | TED 演讲 |
| UXR (user experience research) | 用户体验研究(UXR) |
| XP | 经验值(XP) |
Reformatted by reformat_english.py
产品背后:Duolingo 连胜机制 | Jackson Shuttleworth(留存团队 Group PM)
Transcript
Jackson Shuttleworth:……多邻国是一家市值140亿美元的公司,目前也在不断刷新历史新高,过去六个月市值就翻了一番。连胜是我们最具影响力的功能。目前我们有超过900万用户拥有一年以上的连胜。从数据来看,我认为它是我们最大的增长杠杆。多邻国真正关注的核心是:如何帮助用户围绕语言学习建立习惯?让用户第二天再回来使用,是我们需要解决的最大问题。
**Lenny Rachitsky:**让我们深入挖掘连胜功能开发历程中的宝贵经验,聊聊其中的关键教训、洞见,以及走过的弯路。
**Jackson Shuttleworth:**我的建议是测试一切。过去四年我们在连胜功能上运行了超过600次实验,基本上每隔一天就做一次实验。我们实际上搭建了非常好的文案测试(copy testing)基础设施。我们过去常用的行动号召(CTA)是”continue”(继续),后来我们把它改成了”commit to my goal”(坚持我的目标),这是一个巨大的胜利。
**Lenny Rachitsky:**通过所有这些实验,你们学到了大量关于人类心理学的知识——什么能激励人,什么会让人泄气。
**Jackson Shuttleworth:**比如,假设你玩了一款手机游戏,连续玩了3000天——也许这种感觉和”你连续学习了3000天西班牙语”是不太一样的。
嘉宾介绍
**Lenny Rachitsky:**今天的嘉宾是 Jackson Shuttleworth。Jackson 是多邻国的 Group PM(集团产品经理),领导留存团队。这是一期不同以往的节目,我在做一个新的尝试:我们将用整段对话集中讨论一个功能的发展历程和经验教训,这次讨论的就是多邻国的连胜功能。多邻国是一家市值140亿美元的企业,仅在过去六个月市值就翻了一倍,使用量和市值都创下了历史新高,也是全球极少数成功的、同时也是全球最大的消费级应用业务之一。正如你将从 Jackson 那里听到的,连胜功能是对这一增长和成功贡献最大的、最具影响力的功能。换句话说,你可以说这一个功能就创造了数十亿美元的价值,因此我认为它值得深入研究。
在我们的对话中,Jackson 分享了连胜功能的历史、他们一路上取得的最大成果和走过的弯路、他和团队关于连胜机制哪些有效哪些无效的经验教训,以及他们如何搭建团队以便在一个产品上运行超过600次实验并持续发现大的突破。我希望能做更多这样的节目,深入探讨你们想了解的功能和产品,所以请在 YouTube 或 Substack 上留言,告诉我你想看到我聊哪个产品或功能。如果你喜欢这档播客,别忘了在你最喜欢的播客应用或 YouTube 上订阅关注,这是避免错过未来节目的最佳方式,也对播客帮助极大。话不多说,有请 Jackson Shuttleworth。
对话开始
**Lenny Rachitsky:**Jackson,非常感谢你来做客,欢迎来到播客。
**Jackson Shuttleworth:**谢谢。老听众,新嘉宾。
**Lenny Rachitsky:**感谢支持。这会是一场非常有趣的对话,我以前从来没做过这样的节目——我们基本上会花一个小时甚至一个半小时深入讨论一个产品的一个功能,但这是一个非常特别的功能,也是一个非常特别的产品。你有没有跟谁花上一个小时、一个半小时只聊这一个功能?
**Jackson Shuttleworth:**内部的话有。当我们有新成员加入团队时,我们会做这样的介绍——实际上我最近刚跟一位新加入的同事做过——我说,嘿,我们花一个小时专门聊聊连胜吧。我们聊了一个小时,大概只讲了我想要分享内容的30%,要说的实在太多了。我们会聊到这一点,这些年来我们积累了大量的经验,但像这样对外分享还是第一次。我觉得我们之前零散地分享过一些经验教训,但这次希望能是一次集中全面的分享——关于多邻国是如何构建连胜功能的。
**Lenny Rachitsky:**我们应该把这一期命名为”多邻国连胜功能经验大汇总”。
(广告段落已跳过)
连胜功能是什么
**Lenny Rachitsky:**好的,对于那些完全不了解多邻国连胜的人来说,能不能先简要解释一下什么是多邻国连胜?这个功能是怎么回事?
**Jackson Shuttleworth:**想必大家知道,多邻国是一款语言学习应用。多邻国的连胜记录的是——至少目前是这样的——你连续多少天完成了一节课。你来到多邻国,做一节课,你的第一节课就会开启一个连胜,然后每连续一天使用应用,你的连胜就会延续。不过我其实需要在”连续多少天”这个说法旁边加个星号备注,因为我们也给这个功能加入了灵活性。我们有叫作连胜冻结(streak freeze)的东西,就像是为你的连胜买的保险。所以理论上这是一个很简单的功能,但随着时间推移,我们在上面叠加了挑战、目标设定、奖励、社交功能……我们的大量通知也与连胜绑定。所以这是一个理解起来很简单的功能,但对我们来说,它是一个非常适合持续丰富和扩展的功能。
连胜功能的影响力
**Lenny Rachitsky:**有些人听到这个可能会想,多邻国连胜,有什么了不起的?但也有另一些人会觉得,天哪,我想把关于多邻国连胜的一切都学过来,太多公司在试图复制你们从这个功能中学到的经验。能不能让大家感受一下,这一个功能对多邻国的成功和增长到底有多大的影响?
**Jackson Shuttleworth:**这不仅仅是我作为一个主观的留存 PM 在说话——我认为这是我们最大的功能,仅次于课程本身。连胜是一个很好的促活机制。我的观点是,任何团队、任何应用都可以引入连胜,如果你搞对了,它很可能对留存用户有效。但最根本的,你的应用必须是人们愿意使用的,而人们确实喜欢使用多邻国。它有趣、令人愉悦,你能学到东西。因此,在这样一个基础上叠加一个促活机制,连胜就变得非常强大。它贡献了令人难以置信的 DAU 量——再说一遍,它是我们的金鹅之一。而且很酷的一点是,你看看通知,通知对多邻国来说非常庞大。我们发送更好的通知——不管是文案还是时机——我们的很多通知之所以有效,是因为它们提到了连胜,因为用户在乎连胜。
所以,不仅是我们对连胜本身的持续迭代是 DAU 的巨大驱动力,它还赋能了其他非常有价值的功能。我来之前查了一些数据,真的很疯狂。我们目前有超过 900 万用户拥有一年以上的连胜。
**Lenny Rachitsky:**哇。
**Jackson Shuttleworth:**900 万用户在一年多的时间里,几乎每天都在使用多邻国。这个数字相当……我不知道,我喜欢用这种方式想象——好吧,如果你把所有这些人放在一个城市里,放在某个地方,那会是哪里?我想了想,这就相当于一座非常大的城市,900 万人。
**Lenny Rachitsky:**一年,保持了一年的连胜。
**Jackson Shuttleworth:**一年。
**Lenny Rachitsky:**太不可思议了。我刚才看了多邻国的股价,我们录制这期节目的时候,多邻国是一家市值 140 亿美元的公司。而且还在不断创下历史新高,一直在涨。我觉得过去六个月市值翻了一倍左右。而我从你这听到的是,除了核心的学习功能——也就是多邻国的产品本身之外,连胜是推动增长、尤其是留存方面最重要的功能。
**Jackson Shuttleworth:**如果你看数据的话,我认为客观地说,连胜一直是驱动 DAU 最大的增长杠杆。这也与我们多邻国思考增长的方式有很大关系。我们很多努力都集中在有机增长上。我们认为增长不仅仅是把新用户拉到平台上,同样重要的是不要失去他们。如果你只是把人拉到平台上,然后他们就流失了,那是不可持续的。所以,我们尽可能让用户持续回来、真正留存为用户,这会给我们一个更轻松的基础,在这个基础上继续增长 DAU。
连胜功能的起源
**Lenny Rachitsky:**好。来聊聊这个功能最初是怎么诞生的。最初的版本是什么样的?最初促使它诞生的洞察是什么?
**Jackson Shuttleworth:**连胜的历史和多邻国本身一样久远。我们推出多邻国的时候,就带着连胜功能一起上线了——我说”我们”,其实多邻国上线的时候我才刚本科毕业,所以这远在我加入之前。但我们确实在上线时就有了连胜功能。最初这个功能的运作方式是,你来到多邻国,给自己设定一个目标,而且是基于 XP 的目标。解释一下术语——多邻国有一套经验值系统,驱动着应用中的很多功能。你设定目标的方式是说,根据你的语言学习目标,也许你设定一个 10 经验值的目标,或者 50 经验值的目标。那么延续你的连胜就意味着——你能不能达到 50 经验值?如果你设的目标就是这个的话。
这套机制运作得不错,我觉得这也反映了多邻国最初的增长方式。Luis 通过一场 TED 演讲发布了多邻国,我们最初可能拥有一个更偏技术导向的用户群,所以这种基于经验值的连胜系统是说得通的。但这也意味着,一个用户可能来使用应用,一天做了好几节课,但仅仅因为他们给自己设了一个太高的目标,就失去了连胜。你不需要是连胜方面的专家也能理解,这大概不是什么好事。不过我们最初的设定有一个好处,就是它确实与你的目标相关联。如果你是认真的,那我们就来追踪你作为一个认真语言学习者有多自律。
但我认为我们做过的最有影响力的实验之一——大约是在我加入的时候,或者刚加入不久——就是把它从基于 XP 的连胜改成每天只要做一节课就能延续连胜。你可以想象到的风险是,用户可能会不那么在乎了,因为它不再与目标挂钩。但我们完全没有看到这种情况。这是一个巨大的 DAU 增长驱动力——仅仅是通过降低延续连胜的难度。但我觉得同样重要的是,它依然有意义。使用单位——当你在构建一个连胜功能时,我认为非常重要的一点是想清楚你的应用的使用单位是什么。多邻国的使用单位就是做一节课。所以如果我们关心的是用户每天回来做一节课,因为这表明他们真正在使用应用,那么让我们的连胜聚焦于”做一个单位”而不是”做多个”,对我们没有任何损失。
所以那可能就是我们当时做的最大的变革性实验——从基于 XP 的连胜转向一课一连胜。它也更简单了,而我觉得这是思考连胜时需要考虑的一点。作为 PM,总是很容易给你的功能设定一百万个目标或目的,然后 potentially 构建一个更复杂的功能。而”一课一连胜”,就是更容易让更多用户理解。
**Lenny Rachitsky:**是的,这正是我想说的。对不太熟悉 XP 的人来说,它基本上就是经验值,你通过做事来获得,就像——
**Jackson Shuttleworth:**对,你在多邻国上做事情,然后根据你做了什么以及做得怎么样,我们给你经验值。这实际上驱动了我们应用中的很多功能。排行榜是最大的一个,我们有一个排行榜系统,在一周的时间里你和 29 个人竞争,你想赢,这一切都是由 XP 驱动的。所以我们确实有其他功能真正受益于这套 XP 系统——
**Lenny Rachitsky:**了解。
Jackson Shuttleworth:——连胜只是不再是其中之一了。
学习还是促活?
**Lenny Rachitsky:**好。接下来我想聊聊从那个阶段到今天的演变过程,不过先插一个小话题。我看到 Luis 本周刚发了一条推文,有人问他”你怎么决定是优化学习效果还是用户参与度?“他的回答是,“毫无疑问,我们做的一切都聚焦于参与度,因为如果你不回到应用里,你什么都学不到。”
**Jackson Shuttleworth:**作为一个负责促活的产品经理(PM),这大概是他说过的最让我振奋的话了。再说一遍,这多少是作为一个促活 PM 的主观感受——我看待这个问题的方式是……多邻国的学习团队听到这话大概会皱眉头,但我认为自己既是学习 PM,也是促活 PM,因为在多邻国上不学习的最简单方式就是第二天不回来。所以,如果我们不能让应用留住用户,你根本没有机会让他们接触我们的学习功能。不过,我确实认为学习有一个长尾效应——如果你开始降低体验的深度,说实话这也是我们在连胜上一直在纠结的问题——你开始把体验做得越来越浅,用户实际上什么也没学到,他们就不会在乎了。
连胜发挥最大效果的前提,是它依托于一个用户真正在乎的应用。但确实,如果你不来多邻国学习……如果你不回到多邻国,那你就没有在学习。我们会大量追踪这些数据,确保在调整连胜机制时不会损害学习体验,而且我们并没有发现两者之间存在太多冲突。所以我们一直在思考这个问题,但也感谢 Luis 说了那句话。
**Lenny Rachitsky:**这个逻辑我能理解。好,接下来让我们进入正题——连胜机制一路走来积累的海量经验。第一个版本从经验值(XP)改为一节课,请聊聊其中的关键经验、洞察,以及一路上的弯路,最终才演变成我们今天看到的样子。
降低门槛的极限实验
**Jackson Shuttleworth:**多邻国有一个很强的”先测再说”的理念,我们愿意测试很多不同的方案。说实话,我们宁愿去测试,也不愿意讨论来讨论去好几天。所以在那次实验之后——时间稍微晚一些——我们接着测试了:如果把延长连胜的门槛再降低会怎样?我们测试了,如果你只做一道练习——一节课里只做一道题——就能延长连胜。很多洞察看起来是有道理的:你看漏斗数据,有很多用户开始了课程但没有完成,他们没有延长连胜,损失厌恶没有发挥作用,他们就不回来了。所以这个实验顺着这个思路走了下去。但我们跑完这个实验后发现,日活跃用户数(DAU)纹丝不动。我们做的事情是——回到度量单位来看——我们把度量标准又降低了。没有人会想”我就想来多邻国做一道题”,没有人会这么想。
所以我们用来构建连胜机制的度量单位变得更模糊了,而我们用连胜捕获的那些用户——进来做一两道题就离开——是你能想象到的参与度最低的用户。所以我认为,在构建自己的连胜机制时也需要思考的一点是:你到底在为哪类用户解决问题?不仅仅是你想培养什么样的习惯,还有你要求的承诺水平是什么。这就是一个反面案例——我们过度倾斜于一类用户,而这类用户说实话我们根本留不住。这是一个非常容易做出的下线决定。
**Lenny Rachitsky:**这个故事很精彩,我插一句评论。你们的思路是,干脆走到极端——让连胜变得超级简单,让所有人的连胜永远不断——然后……我很喜欢这个结果:它带来的不是你想要的用户,而且在拉低体验质量。这让我想到《Farmville》,你得每隔什么一小时就去收割庄稼,这能持续一阵子,但最终人们会想,我到底在干什么?
**Jackson Shuttleworth:**对,完全正确。我们什么都会测试。我之前也在看数据,过去四年里,我们在连胜上跑了超过 600 个实验。所以基本上每隔一天——
**Lenny Rachitsky:**600 个,哇。
Jackson Shuttleworth:——我们就在跑一个实验。实验的范围很广,从大的——比如改变机制本身如何运作——到小的——比如把一个字符串换成另一个,看看哪个文案测试对用户更好。所以我们在一切方面都在持续测试。我确实认为,现在我会更谨慎地跑那种实验——到某个阶段你的连胜用户基数已经足够大了,再说一次,我有 900 万连胜用户,我必须非常谨慎。那些是我们留存最好的用户,你必须小心。但在连胜的早期阶段,我建议什么都测。看看——不要执着于”它必须是这个样子”,就是大量测试不同的东西,看看什么最能打动用户。因为我认为,你会不断被实验洞察所惊讶——无论结果如何……我们大约有一半的实验被下线,也就是说一半的实验是失败的,但仅仅通过运行它们,我们就学到了大量东西。所以超级、超级有价值。
中性实验的决策
**Lenny Rachitsky:**这其实是一个相当不错的成功率了,很多公司只有 20% 的实验是正向的。你们的政策是什么?如果实验结果是中性的,是上线还是下线?
**Jackson Shuttleworth:**这要看情况。如果我们是在添加新东西,结果中性,我们倾向于下线,因为它只是增加了认知负担,而且后续我们还得围绕它来构建——一个新的 UI 元素,我们得想办法把它融入现有体系。我会说,我们确实会上线中性实验的情况是——我们对某个方向有真正的信念,比如,好吧,这次可能是中性的,但它会给我们一个新的平台,在此基础上继续构建,那个体验本身可能是中性的,但接下来我们可以在此基础上跑出对 DAU 正向的实验。不过我的总体建议是,在这种情况下,把它作为 V1 的一部分来构建,确保你至少验证了关于路线图的假设是成立的。但总的来说,下线中性实验以避免给应用引入更多复杂性,是我们更倾向的做法。
**Lenny Rachitsky:**有道理。好,还有什么?这一路走来还学到了什么?
聚焦零到七天用户体验
**Jackson Shuttleworth:**那我可以聊聊我们采用的几种不同的结构性思路,几个我们认为非常重要的主题。第一个就是聚焦零到七天用户体验。我想说的是,如果你看我们实验的分布,我们在让用户从零天连胜到七天连胜这件事上,跑的实验数量绝对高于平均水平。这很大程度上是因为我们分析了留存曲线的数据,发现一旦你到达七天,损失厌恶就会开始发挥作用,你就会留下来。从一天连胜到两天连胜,留存率有巨大的跃升;两天到三天,稍微小一点但仍然巨大——这个趋势一直持续到第七天。一旦你到了第七天,曲线就趋于平缓了。这并不是说 30 天连胜的用户比七天连胜的用户留存好不了多少,但两者的差距远没有从第一天到第七天之间的差距那么大。
所以我们做了大量工作,把用户推到损失厌恶开始发挥作用的那个临界点,然后他们就不想离开应用了。其中一个很有意思的功能——说实话它的过程和功能本身同样有意思——叫做”连胜目标”(streak goal)。再说一次,这些东西事后看都特别显而易见,但在当时确实是很新颖的。我们的想法是,嘿,也许我们可以直接给用户设定一个连胜长度目标。可以想象,这是一个非常强大的用户心理学工具。我们从最简单的版本开始。这也是多邻国做测试的典型方式:不要一上来就设计一个庞大复杂的功能作为 V1,而是先做这个功能最简化的封装,看看它有没有生命力,然后再逐步迭代添加。这也是我们为什么能在连胜上跑 600 多个实验的部分原因——它们并不都是大实验。
**Jackson Shuttleworth:**我们起步的时候……说起来挺有意思,我们其实是从变现团队那里借鉴了一个经验。他们有一条文案效果特别好——大意是,如果你订阅了 Plus(现在叫 Super,我们的订阅产品),你完成课程的可能性是 5.6 倍。这是一条非常有效的钩子——如果你真的很在意,你就会订阅。于是我们有了类似的想法:不如直接告诉用户,如果你达到 30 天连胜,完成课程的可能性会高多少。
我们从这里开始,当时的说法是”如果你有 30 天连胜,完成课程的可能性是七倍”。仅仅是在你开始连胜时展示这条信息,效果就非常棒——巨大的胜利。它指向的是一个结果……多邻国有宝石经济体系,但说到底你就是在学习。能够用用户自己关心的结果来表述——在这个案例中就是完成课程——是一个巨大的胜利。这就是我们的起点,然后我们想:啊,目标设定,好,我们应该在这方面投入更多精力。
从单一目标到可选目标
**Lenny Rachitsky:**找到了——
**Jackson Shuttleworth:**对,我们找到了我们的东西,接下来就全力把它做到极致。紧接着我们又做了一个实验,测试不同的目标长度。我们测了 14 天和 50 天,发现它们效果都不错,但吸引的是不同的用户群体。于是我们开始意识到,我们需要更审慎地考虑把不同的选项给到不同的用户。接下来我们又做了一个实验:先给你 30 天的目标,然后让你可以选择退出——我们会说”不,我觉得我做不到”。我们的思路是,如果你说做不到,我们就给你一个更简单的目标,因为我们只是想让你先对一个目标做出承诺。这个实验非常有意思,因为给用户一个更简单的目标确实带来了不错的提升,在他们彻底拒绝之前把他们留住。但几乎同样大的提升来自那个退出按钮本身。
我们单独做了对照测试——我个人非常赞成在实验中设非常多的分支,这样你就能把不同的假设隔离开来——我们单独测了”加一个退出按钮会怎样”。加一个退出按钮……作为产品经理(PM),你可能会想,哦,用户可能因此不参与我的功能了,这是坏事。但让用户能够这么做,反而是一个巨大的胜利。这里的发现是,这种”故意说我要”的意图性——以前只有一个”继续”按钮,但现在变成了”不,我要达到 30 天”,让用户做出一个有意识的决定——是或否——即使这个决定在离开这个屏幕之后没有任何后续影响。我目前谈到的所有东西,都只是那天那一个屏幕上的内容,之后就全部丢弃了。这种可选择性带来了巨大的洞察。正因如此,我们做了一个目标选择功能,让你可以在不同长度的连胜目标之间选择。给用户提供这种可选择性,同样是一个巨大的胜利。
最后再分享一个相关的经验。说到摩擦力——好的摩擦力和坏的摩擦力——当我们做出了目标选择屏幕、让你可以在不同连胜长度之间挑选时,我们想,不如推荐用户选一个更难的目标吧,认为更难的目标会带来更好的留存,所以我们替用户预选了更难的目标。基于我刚才分享的所有经验,你大概能猜到——结果大幅下降了。我们意识到,是的,替用户选好目标可以让他们更快地通过这个屏幕,但那个”我认为是 30 天”、“我认为是 14 天”的选择行为本身,才是这个功能中用户参与度的主要来源。
实验背后的方法论
**Lenny Rachitsky:**你们通过这些实验学到了大量关于人类心理学的知识——如何激励人、什么会起激励作用、什么会打击积极性。感觉你们应该写一本关于人类心理和动机的书。
**Jackson Shuttleworth:**我觉得自己完全就是一个业余的沙发心理学家。至少在”想在手机上学语言的人”这个群体上,我确实非常了解他们。
**Lenny Rachitsky:**没错。所以我目前听到的一个主题是,你们基本上就是在淘金——在矿井里找矿脉,一旦找到了就会说”哦,这个有效”,然后疯狂地在那一根线上做各种测试,看能把它推多远。
**Jackson Shuttleworth:**对。我想说的是,这个想法有一部分来自变现团队的成功经验。多邻国跑实验的数量……我不知道我们处于什么百分位,但按公司规模来算,人均实验量肯定是极高的。我们不断学到很多东西,而且团队之间有很好的经验交叉分享。变现团队会说”这个对我们有效,你能不能用?“所以我们很少是在完全没有头绪的情况下开始做某件事——通常我们都有一些洞察,因为我们跑过那么多实验:如果我们做这个,之前在某处奏效过,推动了这种用户参与度,那么我们把它调整后放到新场景或不同的屏幕上试试——我们至少会带着一个强有力的假设进入:这应该会有效。
很多时候我们也会去观察其他应用,而且我们看的很多应用本身就是游戏。比如你在玩 Royal Match,或者最近新出的 Pokémon 卡牌游戏——我在这上面花的时间太多了——你去看看这些游戏在做什么,对我们来说是非常好的素材来源。但通常你至少是带着一个基于别处成功经验的强假设进入的。
**Lenny Rachitsky:**明白了。所以我想确认这一点——不是随机做实验,而是”这里有一个我们相当有信心的假设,或者至少有机会成立,让我们试试”。不是”什么都试”?
**Jackson Shuttleworth:**对。而且我认为你对假设的信心程度直接决定了那个实验有多难做。以文案为例,我们实际上为文案测试搭建了非常好的基础设施。我的观点是,只要你有足够的用户量,公司就应该不断地做文案测试。我们做过的成功文案测试数不胜数,你试试,然后发现什么赢了——这类案例确实非常多,而且小小的文案改动带来巨大胜利的情况比比皆是。
**Lenny Rachitsky:**能举一个具体的例子来说明影响有多大吗?
**Jackson Shuttleworth:**回到那个目标屏幕,我们以前用的是”Continue”,我们标准的行动号召(CTA)就是 Continue。我们把它改成了”Commit To My Goal”(为我的目标做出承诺),效果是一个巨大的提升。仔细想想,用户点击那个按钮,我们在让他们做什么?为你的目标做出承诺——这会引导他们做什么?坚持,而不是流失。就是这么一个小小的文案改动,就那一次,就在那个位置,带来了巨大的收益。而且文案改动成本极低——你只需要翻译一下。对我们来说,用户遍布全球,支持很多界面语言,但核心就是想出一堆点子,翻译几条字符串……这也是那种你通常会从 Luis 那里得到反馈的场景。多邻国所有的改动都要经过产品评审,由 Luis 审阅。所以 Luis 会审阅我们提出的每一个改动、每一个实验。
文案测试与产品评审
**Jackson Shuttleworth:**通常关于文案,他的反应就是,我不知道,测一下呗。Luis 最好的一点就是他会说,“我觉得这个不会赢,但行吧,你想试就试。“而且公平地说,很多时候他是对的,也有很多次我们的直觉是对的,但做文案测试的成本实在太低了。我认为当预期提升较小时,有一个假设是好的,但没必要把它搞得太复杂。
**Lenny Rachitsky:**顺着这个话题说,我之前没意识到 Luis 会审阅你们计划的所有改动,这可能就是我要问的那个问题的答案。就是说,用这种方式来经营产品和公司——不断做各种微小的改进和变更实验——常常会被批评说,这可能导致产品变成一个怪物,体验不一致、不连贯,这种情况确实经常发生。解决方案是不是就是让创始人基本审阅所有变更?你们还有没有做其他事情来避免产品变成 Farmville 之类的?Farmville 就是一个很好的反面例子。
**Jackson Shuttleworth:**对。我们的产品评审架构里有产品设计负责人、产品管理负责人之一,还有 Luis。因为他们能看到所有改动,而且他们的产品标准非常高,所以这确实有帮助。我认为随着时间推移,我们作为产品经理(PM)需要审视的是:好,我们的功能走向何方?所以我们至少每季度会对连胜做一次这样的审视,看看我们学到了什么,连胜功能发展到了什么程度,以及我们设想近期会怎么走。如果你不持续审视路线图、没有清晰的战略,就很容易陷入某种糟糕的局部最优。对我来说,关键在于如果你对功能的走向有清晰的战略,那么希望所有的 A/B 测试不仅仅是为了获取一些廉价收益,而是着眼于长期目标来做的。
不过我确实认为——我们时不时也会对连胜这样做——最终你就是会触及局部最优。你提到过上线持平实验,这就是一个很好的例子:好,现在我们需要把一堆东西扔掉,基于所有的经验教训,我们能不能重置这块区域,重置这个 UI,重置这个功能,让它和现在一样好,但更加简洁清爽,这样我们可以再次开始往上叠加?显然,这类实验很难跑赢,因为已经优化得非常充分了,但它们非常重要,必须去做。否则,没错,你的功能最终就会变成一个大杂烩。
连胜机制为何适合多邻国
**Lenny Rachitsky:**我还想提一个细节,你们拥有的一个其他公司没有的优势是——人们想学语言。所以把用户推回一个他们本身就愿意使用的应用,这不是很多产品都能享有的优势。关于这一点你还有什么要补充的吗?比如说这就是为什么多邻国可能跟其他人做的产品有点不同。
**Jackson Shuttleworth:**这确实是一个优势。如果我自己做一个应用的话……实际上这也是为什么我觉得很多手游对连胜的处理方式不同。因为说你玩了一个手游——我自己也玩很多手游——你连续玩了 3000 天,我觉得跟你说你连续学了 3000 天西班牙语,感觉还是不太一样的。不过我认为,可以用来比较的范围比很多公司自己认为的要大得多。很多公司在思考自己的产品时……我想世界上应该很少有公司会说,我们对用户没有任何好处。哪怕是一个游戏,也是在给某人一个远离生活喧嚣的片刻。
所以我认为,对于想要弄清连胜是否适合自己的公司来说,关键在于你能否用一种让用户真的为此感到自豪的方式来呈现连胜。多邻国确实更容易做到,但我觉得在措辞上有很多有创意的方式可以让用户产生好感。另外一点我想说的是,连胜对多邻国之所以特别有效,是因为在语言学习中,你很难看到自己在变得流利方面的日进步。甚至”流利”这个词都不太准确,更像是在西班牙语或其他语言上变得更好——这是一个需要数年才能完成的过程。
多邻国让它变得更简单了,但如果你要达到 C1 或 C2 的流利程度,仍然需要投入数千小时。这很难在日常生活中追踪,所以连胜对我们来说效果很好,因为我们可能没法告诉你”嘿,你现在多掌握了 0.01% 的西班牙语”,但我们可以告诉你”嘿,你的连胜又高了一点”。所以我认为,当一个应用做的事情需要较长时间才能被感知到时,连胜机制就特别适合,因为它能以一种更合理、至少对用户来说更具体可感的方式来呈现进步。
让用户理解连胜机制
**Lenny Rachitsky:**很好,这个背景信息很有价值。对很多不是做语言学习的公司来说也是一种鼓舞。好的,我们之前偏离了主线,从你在迭代连胜过程中做的课程和实验这个话题岔开了一段。目前你分享的内容是:最开始是经验值(XP)的想法,然后变成课程,接着你迭代了让它更简单、也许更难的方式,你加入了连胜目标,用户可以承诺达到某个连胜天数。还有什么?还有哪些有效的、无效的经验和教训?
**Jackson Shuttleworth:**还是围绕一到七天的连胜来说。连胜这个概念——特别是对这档播客的听众来说——应该是显而易见的:哦,连胜,就是计算你用了多少天嘛。但随着时间推移我们发现,很多用户并不理解连胜是怎么运作的。小到”我不明白连胜冻结怎么用”,或者像我妈妈前几天跟我说,“哦,我没用多邻国,回来一看连胜还在”。所以这个功能中有一些元素我们可以解释得更好。甚至连”连胜”这个概念本身——它在追踪你使用应用的天数——我们能做得越容易被用户理解,留存效果就越好。我们为此做了不少实验。
你问到我们早期做过的简单文案改动。实际上,这还是我加入多邻国后第一个跑赢的实验。当你开始一个连胜时,屏幕底部有一小段文案,就是一种趣味性文字,我们用它来庆祝你或给你一些上下文。我跑了一些测试,就是试图用八个字来解释连胜是什么,仅此而已。结果是一个巨大的胜利,因为它确实把连胜的运作方式说得非常直白,帮助用户理解了他们需要做什么。我觉得这一点需要不断提醒自己——特别是如果你在科技行业工作、在构建各种酷炫的技术功能——你的用户群并不是一群科技从业者。你得想一想,我的受众是谁?对我们来说,不仅仅是科技从业者,不仅仅是美国人,而是全世界各个年龄、各种文化背景的人。所以确保你的功能——哪怕是像连胜这么简单的东西——能够被理解,是至关重要的。
**Lenny Rachitsky:**具体文案是什么,你还记得吗?
**Jackson Shuttleworth:**是……我觉得应用上现在还有,大概是”完成一天即可延续连胜,但缺席一天就会重置”之类的。
**Lenny Rachitsky:**这我能理解,非常清楚。
**Jackson Shuttleworth:**对。
**Lenny Rachitsky:**八个字,我喜欢。对了,你说”巨大的胜利”,给大家一个参照,具体是什么样的?在这个量级上,什么叫巨大的胜利?
**Jackson Shuttleworth:**嗯,说起来挺有意思的,这是四年前的事了,但大概是超过一万个日活跃用户数(DAU)的量级。其实可以稍微补充一点背景。多邻国非常看重一个指标叫 CURR,即当前用户留存率。实际上我们和 Lenny 合作的第一篇多邻国文章,是我们前产品负责人 Jorge 写的那篇 newsletter。
**Lenny Rachitsky:**至今仍然是我 newsletter 三百多篇中有史以来最受欢迎的一篇——
**Jackson Shuttleworth:**如果你对这个话题感兴趣,强烈推荐读一读。简单总结一下,我们的发现是:如果我们想要提升日活跃用户数(DAU)——多邻国的增长北极星指标就是日活跃用户数(DAU)——那么最有效的指标是当前用户留存率,该指标的一个百分比变化对日活跃用户数(DAU)的推动效果最强。所谓当前用户留存率,就是那些既不是新用户也不是回流用户的人,让他们明天还回来。所以我们大部分团队、尤其是留存团队所做的工作,都聚焦在 CURR 上。我带领的留存团队主要关注的就是 CURR,而连胜恰好是推动 CURR 最好的功能。所以这个实验是我们当时最大的 CURR 胜利,怎么也是前三名的 CURR 胜利。就是这么一小段文案。所以我才会说,用一千种不同的方式测试文案,有时候驱动巨大增长的不是什么华丽的大功能,有时候就是几个字这么简单的东西。
实验衡量方式
**Lenny Rachitsky:**我喜欢这个。你说一万个日活跃用户数(DAU),我想确认一下——你们衡量的是增量影响,也就是这个实验能带来的新增日活跃用户的绝对数量,听起来是这样对吧。
**Jackson Shuttleworth:**对,我们两种都看。很多时候对于留存,我会看第一天、第七天、第十四天的表现,我特别希望看到第十四天的效果比第一天更好,因为这意味着用户的长期留存更健康。尤其是对于那些会多次看到某个功能的用户。我个人偏好用日活跃用户数(DAU)的绝对数字,因为只要你控制好各种偏差——比如近期偏差或新奇偏差——这就是一个非常直观的绝对比较方式。你一旦去看百分比变化,就会受到你的实验组有多少用户看到等因素的影响,但绝对数字至少更容易做横向比较。当然它也有坑,但我们觉得这是一种相当实用的方式。
**Lenny Rachitsky:**这个经验在这个播客上反复出现过,这种实验方法论也是。
**Jackson Shuttleworth:**对。
胜利但不发布的实验
**Lenny Rachitsky:**所以你们并不孤单。快速插一个问题,如果答不上来也没关系。回到疯狂实验但不做出没人想用的产品这个话题——有没有一个实验结果是正向的,但你们决定,不,我们不认为这是产品里应该有的东西,最终没有发布?
**Jackson Shuttleworth:**留存团队不只做连胜,虽然你可能觉得……我们大部分工作确实是连胜。但这些年我们也接触过很多不同的界面。有一个实验,我们之前提到过经验值(XP),在课程中,唯一的 UI 元素就是顶部的进度条,以及你有多少颗心。所以我们保持得非常简洁。这非常能体现多邻国的设计哲学——UI 越简洁越好。然后我们决定,嘿,把经验值(XP)也加进去吧。让用户在上课的过程中看到经验值(XP)在跳动上升,这会让用户感觉很好,能看到自己赚了多少,不容易中途退出,种种好处。等你完成课程,我们再展示你总共收集了多少经验值(XP)。
它赢了,假设是对的。但我们意识到——我记得和 Luis 有过一次对话——就是,这是我们应用中最重要的屏幕,就是课程页面,是用户学习的地方,这里的核心是学习。而你在上面加了另一个东西,可能会让用户分心。他问了我一个问题,就是我们之前聊过的路线图和战略层面的问题——他对我的提问,当时我答不上来,就是:你接下来还要拿这个东西做什么?你的迭代思路是什么?它要往哪里走?它会让课程体验变得更游戏化吗?我们最终意识到,说实话,这只是一个轻松的互动提升方案,但我们为此动了应用中最神圣的空间。
所以那是一个这样的案例:是的,结果是不错的,但我们加了一个 UI 元素,而至少在当时,我们不太清楚拿它接下来做什么,我们意识到长期来看它只会碍事。为了简洁起见,我们宁愿暂停、关掉它,让课程页面保持它作为一个学习净地的状态。不过说起来也有意思,现在我觉得我们实际上有足够多的基于经验值(XP)的机制和有趣的玩法,我认为关于课程内体验的很多信念已经变了,类似那样的东西现在可能行得通了,但当时确实觉得不应该保留。
**Lenny Rachitsky:**这是一个非常好的例子。希望后面有时间聊聊团队是怎么运作的——我脑子里想到的是,你们有那么多产品经理(PM)和团队想要展示影响力,还有绩效考核之类的东西,结果你不发布,人家就没法说”看,我们赢了一个实验”。所以这个话题之后聊,但我们继续说说你一路上学到的东西和那些不成功的尝试。
连胜火焰与文化差异
**Jackson Shuttleworth:**关于连胜还有一点我想提。连胜的视觉形象是一团火焰。我们有这个连胜火焰,它是我们图标体系中的核心元素。需要承认的是,这是一个留存机制的隐喻——保持一团火焰不灭。同样,我觉得火焰对很多用户来说已经成为他们对连胜的理解方式,这很好,但对于不同文化背景、不同人生阶段的人来说,保持一团火焰不灭来表达对某件事的投入,这个概念并不那么直观。我们多年前在印度做过一些用户研究,发现这个意象在那里完全没有共鸣,这是一个非常有趣的发现。这再一次说明,取决于你的用户群是什么样的,你能做的全球化用户研究越多,就越能了解用户实际上如何理解和体验你的功能,因为你会不断发现这类洞察。
所以,就连我们的屏幕设计,以前是一团火焰,基本上就是每天会亮起来的火焰。但这同样只是一个机制的隐喻性指示,当我们重新设计时,我们做了这个——Kurt,我们的一位动画师,做了一个非常棒的里程表动画,你的数字每天会跳动一下……看起来很好看,但从产品角度来说,更酷的是我们实际上把屏幕设计的重心放在了展示你的数字往上走,然后它会显示”七天连胜”、“八天连胜”。我认为,当你在围绕连胜做设计时,不要过于沉迷于你要讲的那个美丽故事,以至于牺牲了功能本身的可理解性。所以,在做产品设计时,让设计清晰地提炼出”我们真正在追踪的是什么”——在这里,形式应该追随功能——这是我们的一个重要学习。现在你会看到在很多展示连胜的地方,我们真正以数字为主导,而不是那团火焰。
**Lenny Rachitsky:**我反复听到的一个主题就是清晰——不要沉迷于把它做得太巧妙,也不要想太多,清晰本身就能产生巨大影响。
**Jackson Shuttleworth:**清晰也不必以牺牲愉悦感为代价。这是我们的一个理念——当你达成一个里程碑时,Duo 会变成我们称之为”凤凰 Duo”的形象,他会变得非常酷,浑身起火。我认为你完全可以做一些事情,让体验仍然令人兴奋、愉悦、值得庆祝,我不会放弃这些,只是不要以牺牲核心为代价……而且我觉得这也关乎一个判断:对于已经深入连胜体验的用户,你可以在愉悦感上做更多;而对于刚开始的用户,你的目标只是确保他们理解这个功能是怎么运作的。
日历化设计与功能传达
哪怕是像这样的事……再举一个随机实验的例子。在连胜屏幕底部我们有一个日历,这些年来它看起来越来越像日历了,原因很简单:我们发现越让它看起来像日历——日期在上方、小圆圈、打勾——人们就越能意识到,这是一个每日机制。所以,请从整体上审视你的屏幕,你在屏幕上做的每一件事,都要问自己:这如何帮助传达这个功能的意义?它是怎么运作的?
灵活性与连胜冻结
**Lenny Rachitsky:**我想最大的成功之一就是给用户在旅途中提供灵活性,比如连胜冻结之类的,这是一条很大的机会线吗?
**Jackson Shuttleworth:**是的。实际上,我要给你看一份有人送给我的最用心的礼物——这是我们的多邻国宁静祈祷文,或者叫连胜宁静祈祷文。我的联合负责人 Antonia——
**Lenny Rachitsky:**这是针织的对吧?
Jackson Shuttleworth:……这个是给我的,太棒了。
**Lenny Rachitsky:**哇。
**Jackson Shuttleworth:**上面写着:“Luis,请赐我宁静,接受我所需的灵活性;赐我勇气,在能够达到完美时全力以赴;赐我智慧,无论如何都要庆祝。”
**Lenny Rachitsky:**啊——
**Jackson Shuttleworth:**这实际上差不多就是我们连胜的策略。
**Lenny Rachitsky:**我很喜欢你刚才的展示环节,太棒了。
**Jackson Shuttleworth:**嗯,对于播客听众,我们可能得在哪里放一张图片。这种”灵活性 vs 完美”的理念,以及”无论如何都要庆祝”,是我们思考连胜的核心。因为我觉得对我们来说,连胜的理念非常明确——宁弯不断。如果你要错过一天,我宁愿你回来时,即使错过了那天,连胜仍然完好无损;但如果你不必错过一天,我更希望你不要错过,我更希望你每天都回来使用应用。不过关于灵活性——从机制角度来说,这几乎可以确定是我们最大的日活跃用户数(DAU)驱动力。我们最早做的实验之一是:以前你只能有一个连胜冻结,后来我们允许你拥有两个或三个。我们测试了两个不同分组,同样又是一个巨大的日活跃用户数(DAU)提升。
这其实很有意思……这也再次呼应了 Jorge 那篇增长模型文章。这个实验实际上对当前用户留存率(CURR)很不好,因为我们基本在说,嘿,你可以休息一天,没关系;但它对周活跃用户回归率(weekly active user return rate)非常好。也就是说,休息了一天的用户,我们让他们以更高的比率回来,所以弥补了当前用户留存率(CURR)的损失。但实际效果是,为什么两个连胜冻结比一个效果好——我也不知道,有时候人们就是需要比一天更多的灵活性。但这个实验真正有趣的洞察是:三个连胜冻结实际上并不比两个好。这里有两种竞争性的力量在起作用,我觉得这很重要——如果你要做一个连胜功能,就得想清楚你的灵活性机制应该是什么样的。
我们确实能让更多用户在更长时间缺席后回到一个完好无损的连胜。但如果你开始连续三天不做任何习惯,你第四天回来的可能性就是会更低。所以我们面对的是两种竞争性的力量:一方面可能有更多用户回到了连胜中,但另一方面也有大量用户就是不回来了——我们实际上在训练他们缺席更长时间。所以灵活性,到底什么才是合适的程度……我们在这个领域又做了数百次实验——到底多少灵活性才是对的?而我们在这里不断感到惊讶。我至今仍然没有答案:在连胜旅程的每个阶段,你到底需要多少灵活性。但有一件事我可以确定地说:在用户刚开始连胜的时候,给予更多灵活性。再说一次,这是我们最大的连胜冻结实验成功之一……我感觉我一直在说”我们最大的成功”,但它们确实都非常、非常大。
连胜冻结的初始灵活性
**Jackson Shuttleworth:**我们最大的连胜冻结成功之一,就是当用户开始一段新连胜时,我们直接给他们两个连胜冻结。回想起来真的很有意思——我们当初怎么就没这么做呢?但当时,连胜冻结是一个过度游戏化的机制,你需要用宝石购买,那是我们的应用内货币,因为我们想让整个机制感觉起来是你真正挣到的东西,获得连胜冻结需要付出一点代价。后来我们测试了:如果用户刚开始连胜时,我们就直接给他们两个连胜冻结会怎样?结果效果惊人。
现在回看似乎很显然,但如果你只有一天、两天或三天的连胜,放弃重来真的很容易。你需要达到七天——从数据中我们看到——连胜才会真正锁定。所以,给予用户更多灵活性,让他们在初期更难失去连胜,这是必要的。反过来——这也是我们不断学到的——一旦人们进入了长连胜,你反而不想给他们那么多灵活性。因为很多时候,用户其实并不需要……我自己就是这样,我有 400 多天的连胜。注意,这比我在多邻国工作的时间短多了——在多邻国工作期间,我失去并重新开始过很多次连胜。
但一旦你开始进入长连胜,你就会真的在意这个功能,真的在意你的连胜。对大多数人来说,只要你不是在犹他州的偏远荒野背包旅行,你总能找到一个有网络的地方。所以,弄清楚哪些用户实际上能够使用多邻国,不去引导他们开始养成本来不必要的休息习惯——这一点很重要,你需要为你的功能找到那条界线。
**Lenny Rachitsky:**这太有意思了。你也可以花钱购买连胜对吧?那是一个付费功能,是吗?
**Jackson Shuttleworth:**是的。有意思的是,这也是我们一直在——
**Lenny Rachitsky:**你可以花钱买冻结,抱歉,不是连胜。
**Jackson Shuttleworth:**对。你可以购买连胜冻结,方式是你先购买宝石,然后用宝石购买连胜冻结……这也是我们一直在纠结的事情。我们现在正在做一个实验——对收入有一点影响,但对留存是一个很好的提升。我觉得这件事从第一天起就值得思考:当你构建连胜功能时,你把它看作一个变现功能,还是一个留存功能?变现的角色是什么?留存的角色是什么?对我们来说,它一开始更自然地生长出来,所以我们现在有很多变现钩子——作为留存产品经理(PM),我巴不得把它们都去掉。但这确实是连胜目前运作方式的一部分。所以我们始终存在这种张力:如果我们让购买连胜冻结变得更难,买的人就会更少,用来购买连胜冻结的宝石销量也会下降,接着就会产生一系列更复杂的连锁影响。
**Lenny Rachitsky:**是啊,人们愿意花钱买连胜冻结,这简直就是连胜有多重要的终极证明。
连胜修复与”挣回来”功能
**Jackson Shuttleworth:**对。我们最近去变现化的另一个大功能,或者说引入了免费选项的,就是找回失去的连胜。以前你失去连胜后,我们有一个功能,那时候叫 Streak Repair(连胜修复),我们会把连胜还给你,但你需要支付宝石。但后来我们发现效果更好的是一个叫 Earn Back 的功能——基本上,只要你在失去连胜后的一个窗口期内回来,完成一定数量的课程,我们就把连胜还给你。这个功能在留存方面取得了巨大的成功。我们的思考逻辑是:当你完成了这些课程,你会觉得自己真正挣回了连胜——你理应拿回你的连胜,我们没有让它变得廉价,因为你付出了努力。
从连胜哲学的角度来说,“让连胜变廉价”这个概念是我们一直在纠结的。我们不断增加连胜冻结的发放,到什么时候用户会开始意识到他们的连胜毫无意义?从我们看到的所有数据来看,用户完全可以接受使用连胜冻结,同时仍然认为自己的连胜是有意义的。但我的联合负责人 Antonia——就是给我做那个超棒十字绣的同事——她是我们团队中连胜神圣性的守护者。在我们不断推进的过程中,我认为这一点非常重要——当你构建自己的连胜功能时,你几乎总能通过让连胜变廉价来获得参与度的提升——让它更容易延续、给用户更多灵活性——但你在某个时刻必须守住底线。
连胜的神圣性与底线
而这个底线在哪里并不清楚。你谈到过单向门和双向门——当你做得太过,那就是一扇单向门,突然之间,那 900 万拥有一年连胜的用户不再在乎自己的连胜了。从留存产品经理(PM)的角度来说,那就是一个灭绝级别的事件。我不想让所有这些用户都不再在乎自己的连胜。所以,有一个真正在乎连胜神圣性的人——对我们来说是 Antonia 和 Luis,他在这方面也非常严格——这真的非常重要,确保你不会走得太远。
**Lenny Rachitsky:**这个洞察太棒了。所以,为了保护……推送通知我觉得也是公司普遍面临的一个类似问题——多少算太多?因为每个人都说,再发一条推送吧,没事,就一条。所以你的解决方案是,指定一个人作为守护者,差不多是看门人,再加上创始人,来判断什么是过度的。
**Jackson Shuttleworth:**如果你能做到这一点当然好。我觉得推送通知相对容易处理,因为你可以做很多事情——比如设定通知发送的预算上限,你可以——
**Lenny Rachitsky:**类似一种政策。
**Jackson Shuttleworth:**对,政策。但我觉得很多事情,至少对连胜来说,很难用同样的方式制定政策,很多决策必须靠感觉,有时候你只能凭借最好的判断。
推送通知与练习提醒的时机
**Lenny Rachitsky:**好。还有什么心得吗?关于连胜发展到今天这个过程,也许再分享一两个经验。
**Jackson Shuttleworth:**你提到了通知,我也提过几次。有意思的是,人们往往觉得——就像你说的——你总可以再发一条通知,总会有一些收益,到某个点才会变成糟糕的体验,但那一点很难看到。我们有一条通知……我们每天会发送两条与核心连胜相关的通知,第一条是练习提醒。我们发送这条通知的时机——这是一个很有意思的洞察——是你前一天练习后 23 个半小时。我们做了——
**Lenny Rachitsky:**23 个半小时。好的。
**Jackson Shuttleworth:**基本上,如果你今天中午练习了,我们明天上午 11:30 发给你。我们做过——
**Lenny Rachitsky:**这是因为假设用户在前一天那个时间段有空,也许他们在同一时间也有空?
从已披露行为到推送策略
**Jackson Shuttleworth:**我们实际上做过调整——我们以前让用户自己设置练习提醒时间,我们的想法是,你觉得”晚上 7 点,这就是我每天想要延续连胜的时间”,然后你知道吗?我作为一个有两个孩子的人说这话,生活总会横生枝节,生活总是会横生枝节。你认为自己会练习的时间会变,你的生活节奏会变,诸如此类。而我们意识到,你何时应该练习的最好指标,就是你前一天是什么时候练习的。我们几乎可以肯定能做得更细致,我们尝试过各种更复杂逻辑的方式,但最终胜出的永远是 23 个半小时。
**Lenny Rachitsky:**太有意思了。已披露的行为 vs 口头陈述的行为。
**Jackson Shuttleworth:**对,没错。所以我们就是在这 23 个半小时之后发送练习提醒。但我们还会做另一件事,就是发送我们所谓的”连胜拯救”提醒——晚上 10 点,如果你还没有延续你的连胜,我们会给你发一条消息说,嘿,这是你最后的机会,就是这样了,如果你不延续连胜的话。你可能会觉得这有点像垃圾推送,晚上 10 点收到一个 app 的通知挺烦人的。但我们发现,因为人们在乎自己的连胜,连胜对他们来说是一个好东西,他们赋予了它正面的情感,他们真的不想失去它——所以那条提醒他们”嘿,回来吧”的通知,大家大体上把它看作一条正向的通知,而不是负面的通知。
当然,这也服务于我们的目的——让用户回来,不丢掉连胜。但我还是认为,如果你能把推送策略和它所关联的功能结合起来思考,用户如何看待这个功能?那你几乎可以肯定——不是说可以肆无忌惮地发更多通知——而是你可以对推送负载、何时发送通知更加深思熟虑。对我们来说,这条深夜消息,影响力极大、效果极好,它实际上可能被视为垃圾推送的东西,但我们的很多用户真的……比如我自己,经常已经很晚了,我在这里工作,突然意识到,哦,忘了做我的……我一整天都在想多邻国,现在都 11:15 了,我还没做,那条消息真的很有力量。
**Lenny Rachitsky:**是啊,它救了我好多次。我完全知道那条消息,而且我很喜欢它是一个深夜来自 app 的消息——你很少会真正因为收到这种消息而高兴,我觉得这是一个很好的例子——
**Jackson Shuttleworth:**真的很好笑,你听到的所有关于人们延续连胜的故事——如果你环顾一个多邻国的派对,到了 11:30、11:45,所有的多邻国员工都在最后一分钟做他们的课程。你总能看到这些照片:人们在夜店里做,或者在演唱会上做多邻国,没错,因为否则你就得用连胜冻结,或者更糟——你会失去你的连胜。
**Lenny Rachitsky:**太搞笑了。好的,还有什么吗?如果不止一个的话,尽管分享——还有其他很有意思的经验教训、走错的弯路、或者洞见吗?
庆祝完美
**Jackson Shuttleworth:**我说过连胜冻结,我们在连胜冻结上做了很多工作。但我认为,如果你要把灵活性做成一个特性,那思考一下如何庆祝完美可能也很有用。所以我们有一个功能,它是世界上最简单的东西,叫做 Perfect Streak(完美连胜)。就是如果你连续几天不用连胜冻结,我们让你的连胜看起来变成金色,让你的日历上那个小小的进度条看起来更好看一点。做这件事没有任何奖励,你不会得到任何东西,只会得到这个好看的小标识——但它非常棒。它是一个简单的功能,极其不复杂,却非常强大。它不仅作为一种奖励激励用户——“嘿,坚持七天不用连胜冻结,你的连胜就变成完美连胜”——而且它也能很好地标识出哪些用户没有在使用连胜冻结。
事情是这样的,如果你不用连胜冻结——坦白说,我很希望你永远不用连胜冻结——如果你不用连胜冻结,你的连胜就会保持完美。说起来好笑,我们实际上一直在不断处理关于连胜的 bug 报告。我发誓,我们围绕这个功能拥有最好的基础设施,因为它太重要了。我们有一位员工失去了她——我想是四个月的完美连胜——对她来说是件大事,因为她做了课程,但她当时在穿越国际日期变更线,后端有一堆乱七八糟的情况,总之就是很诡异。但人们开始像在乎连胜本身一样在乎完美,对那位员工来说,失去完美连胜就是一件大事。所以这只是一个例子——如果你要追求灵活性(这是好事),找到一种方式把用户拉回到对完美的追求,是一个非常重要的平衡砝码。
**Lenny Rachitsky:**我能想象的是你们需要一个亚马逊风格的聊天机器人,直接把连胜还给你,就像,好的,给你。
**Jackson Shuttleworth:**我们确实很想……所以,如果人们失去了连胜,我们有途径联系我们,但我们确实想过这个问题——我们应该做一个自助功能,如果我们觉得你的理由够充分,管他呢,我们就直接……
**Lenny Rachitsky:**对对,直接恢复。
**Jackson Shuttleworth:**因为说到底,我宁愿你还在连胜中,也不要失去它,特别是……
**Lenny Rachitsky:**没错。但同时它也应该让人觉得就是这么简单。我很喜欢这个,我也很喜欢你说的这个观点——动画和用户体验本身就能产生影响,这真的很有意思。你经常发现这种情况吗?就是通过庆祝、让它感觉很棒,不需要文案或任何功能,就是让人觉得——哇,你太厉害了?
触觉反馈与让用户停下来
**Jackson Shuttleworth:**这也是另一件事——当用户在乎这个功能的时候,运用动画、触觉反馈、音效……好笑的是,我们的连胜功能目前没有音效,这可能是我们不久之后会考虑的事情——但触觉反馈是我们做了大量测试的领域——
**Lenny Rachitsky:**比如手机以不同方式振动。
**Jackson Shuttleworth:**对,没错,你的手机,在延续连胜时有一个很酷的触觉反馈模式,所有这些东西都有效。而且它有效的原因很酷,有几个方面。一是它让你感觉很好,你在连胜中获得了一个很棒的时刻,我们为你庆祝,用这种视觉的方式为你庆祝,你的手机在嗡嗡响,感觉就是很棒。另一件事是它让你在那个屏幕上停下来。我觉得有一种倾向——当你作为产品经理思考”我怎样才能让用户尽可能无痛地走完这个漏斗”时——我说过那个。有很多时候,我不想让你继续走,我想让你停下来,停在这个屏幕上。
**Jackson Shuttleworth:**你得注意不要在太多屏幕上这样做,但对于那些重要的时刻,有时候我就是想让你停下来,享受这个瞬间。如果我能让你更享受这个瞬间,你就会更在乎你的连胜,你就会明天再回来。所以,炫酷的动画能让你真正沉浸其中,舒服的触觉反馈,所有这些加在一起让你真正聚焦在那个时刻,所有这些就是让用户与他们的连胜建立更深的连接。所以在合适的时机使用动画效果很好,也是我们很多实验中经常赢的策略。
触觉反馈的设计者
**Lenny Rachitsky:**触觉反馈那些东西是谁设计的?有专门的触觉设计师吗?
**Jackson Shuttleworth:**很长一段时间里都是产品设计师来做,实际上最开始是工程师自己拼凑的,凭手感觉得什么好就用什么,后来变成了产品设计的职责,由设计师根据自己的判断来处理。我们最近刚刚收购了一家动画工作室——底特律的 Hobbs,现在他们基本上是这方面的负责人,他们做了大量动效设计工作,触觉反馈和那个非常接近,所以他们做了很多这类事情。我确实记得有段时间一直想招一个触觉设计方面的外包人员,就是做触觉设计,那是我做过最惨淡的一次招聘,因为就是……我也不知道,太专门了……我就是面试了很多人,它是一个介于音效、动效设计和技术之间的很棘手的领域……
**Lenny Rachitsky:**对,非常独特的岗位。
**Jackson Shuttleworth:**非常独特、非常专门的技能组合。
**Lenny Rachitsky:**没错。而且真正需要这么深入研究触觉的应用屈指可数,所以你几乎是在从零培养这个人。
**Jackson Shuttleworth:**是的。
功能复杂度的平衡
**Lenny Rachitsky:**这太有意思了,可以单独做一整期播客了。顺便说一下,你讲到这个的时候我想说,我很喜欢这个模式——庆祝不丢连胜的用户是一个成功点,你引入了让它变得灵活的方式,这本身就是一个大胜,然后你又反其道而行之,如果你不用这个功能,你反而感觉更好。
**Jackson Shuttleworth:**对,而且好笑的是,你提到了功能膨胀的危险,我们也或多或少讨论过功能膨胀的危险,这其实是我在连胜上一直在思考的问题。我们有连胜,但同时我们还有完美连胜,我们还计算你连续多少周保持了完美连胜。突然之间我们就有了两个连胜数字在互相竞争。好笑的是,我们实际上在你达到七天之前不会引入完美连胜的概念,一部分原因就是额外连胜功能带来的认知负担。我们很多更酷的连胜功能,你得先保持足够长的连胜才能解锁。不是说我们没有测试过,因为我们测试了——我们什么都测试——更早引入这些功能,但我们发现几乎无一例外,当我们在用户体验早期就引入太多东西、太多概念时,测试都输了,用户很难消化。
团队运作方式:以指标为中心
**Lenny Rachitsky:**好的,太棒了。我知道这个话题可以聊上好几个小时,你们一路上做的所有事情有太多可以分享的经验。我想转到你们团队的运作方式。你之前提到过很多关于一个团队怎样才能做得这么好的线索——跑 600 个实验,持续找到增长机会。对于那些说”哇,我也想这样工作”的人,你有什么经验或建议?从你们团队的经验来看,你们团队是怎么运作的,有哪些别人可以学习的地方?
**Jackson Shuttleworth:**好的,先给一点背景。多邻国非常看重……我们大多数团队都是基于指标组建的团队。我们做连胜的工作最多,但我们真正关心的指标归根结底是当前用户留存率(CURR)和日活跃用户数(DAU),因为我们看到日活跃用户数(DAU)受当前用户留存率(CURR)驱动。所以当你能非常聚焦于”我每个季度的目标是让这个指标上升”时,我觉得确保自己在做最高投资回报率的事情就容易得多。我觉得当你更多地在想”我想让这个功能更好”的时候,更容易迷失在”更好”到底意味着什么,以及你怎么定义”更好”。所以,我确实认为团队对一个指标有非常强的聚焦——我关心的是什么指标,以及它如何引导我的工作方向——
**Lenny Rachitsky:**而不是以功能为导向。所以基本上,你们的团队是围绕一个指标、一个目标、一个成果来组建的,而不是说”我们拥有这个功能”或”我们拥有这个产品”。
**Jackson Shuttleworth:**留存团队拥有连胜,可以这么说,但这只是因为我们看到连胜比任何其他功能都更好地驱动当前用户留存率(CURR)。但我们并不是——我们有内购钩子,连胜冻结的购买,其他团队也可以、也曾经在连胜上做过工作,因为连胜不是我们的”领地”——我们不会说,不不不,所有迭代都由我们来做,我们只是知道它驱动我们的指标最好。同样的道理,排行榜——我们有一个团队专注于你在上面花了多少时间,我们希望用户在多邻国上花更多时间,这样他们学得更多。排行榜是做这件事最好的载体,所以那个团队做了大量排行榜的工作,但偶尔我有一个我觉得非常有留存效果的想法,我就会去找他们提案,然后我们会对排行榜做一些改动来提升留存。
但我觉得有一个明确的指标——我们要驱动当前用户留存率(CURR),而不是我们只是想让这个功能变得更好——至少能确保给团队清晰的行动方向,这种聚焦对于排列待办事项的优先级我认为非常有帮助。
**Lenny Rachitsky:**很酷。这一点非常重要,Airbnb 也是这样运作的,我在那里工作过很长时间,方式就是——给你一个目标,你的团队对这个目标负责,你可以做任何你需要做的产品来达成这个目标,正如你所说,通常某些产品和你的工作关联最大,但你描述的是,即使一个团队某种程度上……我猜你从某种角度拥有这个功能,你是这个功能区域的守护者,因为它最有助于你的目标。但任何其他团队都可以过来说,嘿 Jackson,我们需要做一些连胜相关的事情来帮助学习,你会说,去做吧。稍微岔开一下——如果他们想在代码里做一些工作,他们会和你们团队紧密合作吗?具体流程是怎样的?
**Jackson Shuttleworth:**是的。如果你……再说一次,这就是我说的软性所有权,我们不反对其他团队对连胜做事情,但如果要做一些改动——考虑到我们很可能已经排了好几个季度的连胜路线图……我说”很可能”,我们确实有。好几个季度的路线图,关于我们可以对连胜做什么,如果其他团队想来动它,好吧,我们得弄清楚这怎么和我们原来的连胜计划协调,怎么确保——很多时候其他团队带着”嘿,让我们对连胜做这个”的想法过来,他们缺少我们可能有的上下文,所以很大程度上是一个简化版的信息共享,说,好吧,这是我们关于连胜的看法,什么是有效的,什么是没用的,这怎么影响你们的一些假设。所以,我觉得就是确保值得这么做,值得花这个精力。
**Lenny Rachitsky:**地道的产品管理工作。
**Jackson Shuttleworth:**是的。
流程与实验管理
**Lenny Rachitsky:**你们在运营方式和工作方法上,还有哪些有趣的做法促成了这种成功?
**Jackson Shuttleworth:**再说一次,我的团队负责人运营得……Antonia 负责运营最……一个非常强调流程的团队。如果你要跑这么多实验,就必须非常以流程为导向,非常深思熟虑地考虑什么时候跑什么实验,这个实验怎么为下一个实验做铺垫。我们有大量的 Jira 自动化,虽然有时候 Atlassian 套件让我眼睛流血,但很多时候这种程度的流程确实能帮团队为工程师扫除障碍,让他们跑得非常快。所以,确保你有非常好的流程来支撑跑这么多实验,这笔投资是值得的。
**Lenny Rachitsky:**你能沿着这条线展开说说吗?你说到这个的时候,具体是什么样的?这个流程中有哪些要素让它运转得如此高效?
**Jackson Shuttleworth:**细到非常详细的路线图——好,我们跑的这个实验是基于那个实验的结果,或者可能会接入某个功能的某个模块,我们怎么确保排好开发进度,让这个实验一跑完,我们马上就能开始推出下一个。我讨厌功能就那么搁着,然后我们不去继续推进那条线。所以不仅仅要考虑工程带宽,还要考虑设计带宽,确保我们准备好下一个迭代版本。我们会提前规划好几个月,考虑这些功能迭代——哪怕是很小的迭代。因为一旦你丢失了周期,没有持续推动一个功能,那就是白白流失的机会。所以,从工程路线图到设计路线图到产品路线图,所有这些都需要在一个体系里协同起来。
**Lenny Rachitsky:**所以本质上是跨职能地梳理依赖关系,你说的这些在 Jira 里就可以做到?
**Jackson Shuttleworth:**很多确实可以在 Jira 里做。我们也有不少 Google 文档,有时候会做一些……我不知道,有时候就是看起来好看一点,灵活一点。
**Lenny Rachitsky:**对。
**Jackson Shuttleworth:**但 Jira 是我们的核心,流程的大本营就在那里。
**Lenny Rachitsky:**好。还有什么?
抵制大而全的首版
**Jackson Shuttleworth:**另一个我想说的是,我们非常抵制做那种大而全的 V1 的冲动。我之前分享过连胜目标的例子——很多时候我们在探索一个方向时会说,好,这个想法不错,但我们怎么砍掉一堆东西,找到核心假设是什么?然后先把那个东西作为 V1 发出去。因为加东西让功能跑赢实验太容易了。我在留存和互动领域做了够久,我知道有哪些可以拉的杠杆,有哪些铃铛可以加上去让一个东西赢。但很多时候问题是——你赢了,但你不知道是因为那些你加的花哨玩意儿,还是因为你的核心假设本身。而且很多时候,如果你能把功能做得足够简洁,出货也更容易,设计也更容易。你不需要为整个系统做设计,只需要为一个简单得多的东西做设计。所以,让所有人都这样思考,能让我们出货更快,设计更简洁,审批更快,更快获得洞察。然后就像我之前说连胜目标那样,一个迭代接一个迭代地做,逐步加上这些东西。这样做不仅让你跑得更快,而且你在每一步都能建立信心——我的系列假设确实得到了验证。如果没有验证,那好,我们就砍掉那部分功能,只发真正重要的东西。
总结:围绕核心指标的打法
**Lenny Rachitsky:**如果让我试着总结一下到目前为止我听到的主要经验——也许你本来也会说这个——但我就在想,如果我要设计一家公司的运营方式,让大家都能这样运作的话,本质上就是先把驱动业务的所有杠杆都梳理出来。你画了一张所有指标如何推动增长和日活跃用户数的全景图。CURR 最终成为推动长期增长最大的具体指标。想象一棵所有可做机会的树,你发现这个指标和我们的增长关联最紧密。然后你就开始挖掘——不知道”挖掘”是不是正确的比喻——就是去寻找能影响那个具体指标的东西。你不断去看、去试、去探索,一旦找到一个,就深入去尝试各种不同的方向。你提出一个假设和一个策略,说我们认为可以这样做来影响这个指标,然后就大量尝试。这让我想到《发展受阻》里那句话——“香蕉摊里总有钱”,就是持续在这个方向上做,总是还有更多机会。
**Jackson Shuttleworth:**我刚加入多邻国的时候,我接手的那个产品经理 Anton——他之前负责留存团队——我记得我跟他说:“兄弟,连胜功能不就是往上计数吗,你们在这上面测了好几年了,还能做多少?“他说,Jackson,你这个毛头小子……他没原话这么说,但这是我的感受——Jackson,你这个毛头小子,我们连 30% 的优化都没做到。四年后的今天,我无比确信地说,我们离……我们已经取得了巨大进步,但我们离极限还远得很。而每个季度我们都能对连胜功能推出大量改进并跑赢实验,这不断向我证明还有太多事情可以做。所以我觉得你的概括是……我想补充一点,这里面确实需要很多思考——我之前说过假设的强度,当你开始构建更大的功能策略时,你必须要有强有力的假设。我认为真的不能只是随机做一堆事情,而是要有意图、有目标地去做,否则你确实会陷入局部最优。但没错,我们还有大量没尝试过的东西,而且我们对它们能跑通有很高的信心,所以我们会继续做下去。
连胜功能的适用边界
**Lenny Rachitsky:**在这段历程中,还有没有其他持久的经验教训?如果有人想以这种方式运营,在自己的产品中构建连胜功能,你有什么建议?
**Jackson Shuttleworth:**有的。我真的认为首先要明确的是——连胜是一个促活手段。你几乎可以让你的应用更有留存力,我几乎可以肯定,市面上几乎所有应用都可以通过连胜让留存变得更好。它本质上是损失厌恶——再说一次,这是”业余心理学家 Jackson”的说法——它就是对人有效的东西。但如果你的应用本身不是用户每天都想用的东西,或者不管你期望的使用频率是什么,连胜能给你的回报是有限的。坦白说,它甚至可能会分散你的注意力,让你忽视真正应该关注的事——让你的应用成为人们每天想用的东西。
所以,如果你开始把重心放在连胜上,但还没有把使用体验做得令人愉悦,坦白说,你只会浪费大量时间。因此我认为,首先要确保你应用的核心循环已经理清——它在为用户提供价值,是用户愿意每天回来的东西——这才能真正为叠加连胜功能打好基础。所以,如果你觉得自己还没到那一步,就克制住 temptation,不要过早地在连胜这条路上走得太深。
**Lenny Rachitsky:**这一点非常好。如果人们对你提供的核心价值根本不在意,连胜也解决不了你的问题。
**Jackson Shuttleworth:**是的,坦白说,如果你最终把精力花在”怎么让连胜更有吸引力”上,而你的应用本身……你是在浪费时间,而这些时间本可以更好地花在解决更关键的问题上。这是第一个教训。
另一件事是,我们几个月前和董事会成员 Bing——Bing Gordon——见面,他有一个评论,大意就是:“用户之所以这么在乎你的连胜,是因为你们自己这么在乎连胜。“这里的”你们”指的是多邻国。用户之所以这么在乎我们的连胜,是因为多邻国自己就非常在乎连胜。我们说,你什么意思?他说,你看,每次学完一课你都会看到一个巨大的连胜界面,而且它的动画比应用里几乎所有其他界面都更酷炫,然后有时候你还会看到其他界面,还有各种……你不会让用户忘记它,你在消息里也会提到连胜。
所以我觉得值得思考的是:如果你要构建连胜功能,然后把它塞到应用的一个角落里,用户根本看不到,那他们大概率不会太在意。这可能也没关系,因为你可能觉得其他杠杆更重要。但我们之所以如此重视连胜,是有原因的——我们希望它始终在用户脑海中处于最重要的位置。而用户开始在意它,并不是偶然的。所以,在考虑构建连胜的过程中,如果你希望它产生和多邻国一样的影响力,就要确保给它应有的曝光度——这是你在设计时需要考虑的一条重要的层级原则。
**Lenny Rachitsky:**这一点太好了。用户会从应用中寻找线索——告诉我应该关注什么,什么重要?如果你又是火焰又是爆炸特效,“你达成连胜了!“——哦,也许我该关注这个功能。再加上推送通知显然也在不断提醒你。这方面还有什么要补充的吗?
**Jackson Shuttleworth:**最后一点可能是:我们在多邻国的连胜上跑了无数测试来弄清楚什么有效。多邻国有一个理念——先测试再说。我们很多时候愿意去测试各种东西。我真的认为,如果你要引入连胜,或者想改进现有的连胜功能,不要过度纠结于各种理论上的东西。确保你的假设是合理的就行,但我的建议就是去尝试。这也是你之前说过的——这本质上是人类心理学,一旦涉及心理学,你就需要去了解用户到底对什么有反应。而最简单的方式就是,别在会议室里来回讨论想法了,直接去试。所以,如果你打算投入精力做连胜,我强烈建议通过与用户测试来找出什么有效,而不是试图一次就做到完美。
**Lenny Rachitsky:**假设有人在听,他们在想:我们应该做连胜吗?这值得做吗?对于连胜功能对其他消费类产品有帮助的概率,你怎么看?
**Jackson Shuttleworth:**我在公司里以这一点出名——我认为每个团队、每个应用都能从连胜中受益。当然,具体实现方式会非常不同。我觉得你需要考虑的是——你的用户使用场景是什么?如果他们来使用,比如说……报税软件……好吧,你知道么,我现在说到这里觉得报税软件确实是个难题,但也许可以围绕”税务季每天都需要用户回来”来设计,或者多少次……我不知道。当我说出声来——
**Lenny Rachitsky:**你上传税务表的次数。
**Jackson Shuttleworth:**对,那个使用场景确实很难。但绝大多数公司我觉得都有一个比较清晰的概念:好吧,这是我理想的使用场景,我希望用户每月来三次,那就是理想的。或者每月四次。你可以围绕这个构建连胜。Peloton 就有每周连胜,因为在新冠期间让用户每天做一次 Peloton 锻炼太难了。就是偶尔上一下 Peloton,挺好,但每周连胜是一个我能坚持的东西。所以我觉得,弄清楚你的使用模式是什么样的,然后围绕它来构建连胜。但只要你不是那种——再说一次,报税的例子可能是一个很好的反例——只要你有一定的使用频率,我觉得几乎任何产品都可以有连胜。
**Lenny Rachitsky:**多邻国是一家市值 140 亿美元的公司,而连胜功能,除了核心产品本身之外,可能是对这一市值成功贡献最大的单一因素。很难想象还有哪个单一产品功能对增长、收入和打造这样一家企业产生了如此大的影响。所以我很高兴我们花了这么多时间在这上面,这真是一份宝藏级的建议和洞察。所以再次感谢——
**Jackson Shuttleworth:**当然,非常有趣。
闪电问答环节
**Lenny Rachitsky:**接下来,我们进入非常令人期待的闪电问答环节。准备好了吗?
**Jackson Shuttleworth:**准备好了。
**Lenny Rachitsky:**第一个问题:你最常推荐给别人的两三本书是什么?
**Jackson Shuttleworth:**好,我先说一本——《A Guide to Midwest Conversation》。我在堪萨斯城,是一个自豪的中西部人,我们中西部人说话有特定的方式。你可能听说过”明尼苏达式友善”,但我们往往不会直说我们的意思。这本书是一份非常有趣的入门指南,告诉你中西部人说出那些话时到底是什么意思。强烈推荐读一读。
**Lenny Rachitsky:**我喜欢你把这本书送给别人,就好像在说”这是我可能会跟你说的话,你可能读不懂”。
**Jackson Shuttleworth:**我太太是德国人,我把这本书给她,好让她更好地理解我。
**Lenny Rachitsky:**我能想象德语恰恰是相反的风格。
**Jackson Shuttleworth:**非常不同。另一本书,这本很不错——《Fate is the Hunter》。这是一本很酷的书,是一位早期商业航空飞行员的回忆录,听听那时候飞行的故事简直疯狂。我以前做管理咨询,几乎每周都飞,飞了将近六年,我从来没有一次需要担心——我能不能活着到达目的地?那个年代可不是这样,所以在现代航空技术出现之前,驾驶那些飞机当飞行员是什么体验,有些故事非常精彩,也让你真正感激我们现在拥有的东西。
**Lenny Rachitsky:**读这样的书感觉很好——作为一个软件产品经理或工程师什么的,感受那种截然不同的生活。
**Jackson Shuttleworth:**硬件确实难做。
最喜欢的影视作品
**Lenny Rachitsky:**硬件。天哪,不是触觉反馈设计。好,下一个——除非你还有其他书想分享?没有,好的。最近有没有一部特别喜欢的电影或电视剧?
**Jackson Shuttleworth:**这么说吧,我有两个孩子,看了很多《布鲁伊》(Bluey),真的很好看,我发誓,它给我带来的快乐毫不吝啬。至于非儿童向的节目,我刚看完最新一季的《艾米丽在巴黎》(Emily in Paris),天哪,太棒了。我知道这算不上什么高雅电视,但就是漂亮的人待在漂亮的城市里,解决那些不至于天塌下来的问题,有时候就是需要这种放空。另外,我在多邻国学法语,顺便给自家 App 打个广告——我能听懂里面说的很多法语了。没有什么比投入大量时间学法语之后,居然真的能用上更让人开心的了。所以,我是《艾米丽在巴黎》的超级粉丝。
**Lenny Rachitsky:**太搞笑了,这个兴趣交集也太有趣了。我婆婆超爱《艾米丽在巴黎》,我看到有人在推特上问——她拿的是什么签证?她怎么还在巴黎?
**Jackson Shuttleworth:**对,最好别问,这剧有很多问题最好别深究。
最近发现的好产品
**Lenny Rachitsky:**好。最近有没有发现一个特别喜欢的产品?除了多邻国以外。
**Jackson Shuttleworth:**上周我去了趟家得宝(Home Depot),买了一把新梯子。梯子的创新,你平时不会经常想到,但现在的梯子可以把一条腿伸得比另一条腿长一点。我家房子建在一个小斜坡上,每次爬梯子都是在拿命冒险,但有了这把梯子,我永远都是水平的。上周我清扫了两次排水沟,纯粹就是因为这把梯子太棒了——这把梯子改变了我的生活。所以,梯子创新这件事,我觉得谈论得太少了,我很高兴给它应有的聚光灯。
**Lenny Rachitsky:**谢谢你分享这个,这也是我们播客历史上第一个梯子推荐。
人生座右铭
还有两个问题。你有没有特别喜欢的人生座右铭,在工作或生活中觉得特别有用的?
**Jackson Shuttleworth:**考虑到我谈到我们对测试新事物的态度,这可能不会让人太意外——你不投的球百分之百进不了。我非常推崇尝试,即使成功的概率不是百分之百,因为过程中你会学到很多东西。
团队传统
**Lenny Rachitsky:**最后一个问题,多邻国内部有没有什么有趣的传统,不管是 PM 团队还是整个公司的,可以跟大家分享的?
**Jackson Shuttleworth:**我们的传统多到数不清。我分享一个我们在留存团队每次站会上都会做的奇葩传统。这始于疫情期间——我们以前是面对面站会的,后来转远程后,我们搞了一个做法:轮到最后一个人发言时,他会倒数 3、2、1,然后大家尽量同时鼓掌。这有点呆但挺好玩,我们就爱上了这个环节,四年后的今天还在做。最近我们还加了一个环节,鼓掌之后大家一起喊”yee-haw”,我也说不清为什么。但通过 Zoom 同步鼓掌,然后一起大喊 yee-haw……前几天我在一个电话亭里做了这件事,出来后有人跟我说:“你知道那些电话亭没有你想象的那么隔音吧?“但遇到一个好的 yee-haw 时机,你不能放过。
**Lenny Rachitsky:**我喜欢这些小东西,听起来微不足道,但它们是团队文化和传统中如此重要的元素,对产品经理来说,找到有趣的方式、做一些荒唐的事情也特别重要。
**Jackson Shuttleworth:**说实话,让大家接受喊 yee-haw 花了一段时间,但现在大家都上手了,你再也拿不走这个传统了——我们都爱它。
**Lenny Rachitsky:**天哪,我之前主持全员大会的时候,管它叫”y’all hands”(全员/你们的大会),欢迎偷走这个。
**Jackson Shuttleworth:**你懂的,你懂的 Lenny。
**Lenny Rachitsky:**我懂。
结语
Jackson,太精彩了。我觉得大家会带着笔记本听这期节目,心想——好吧,这里有一堆我们可以拿来尝试的想法,不管我们做的是不是连胜功能。非常感谢你来。最后两个问题——大家想联系你、想了解更多关于多邻国的信息,去哪里找你?我知道你们在招产品经理,也顺便说说这个。最后,听众可以怎么帮到你?
**Jackson Shuttleworth:**好的,你可以在 LinkedIn 上找到我,那是我最主要的社交媒体平台,Jackson Shuttleworth。然后大家怎么帮到我——是的,如你所说,我们在招人。实际上就是在招我的团队。你对我们思考连胜这个课题感兴趣吗?说不定我们就是适合你的地方,请到我们官网上投递简历。我们还有其他很多产品经理岗位在招,每一份工作都和这些一样令人兴奋。
另外,我一直对其他公司如何实现连胜以及他们学到了什么很感兴趣。所以我想说的是,如果你所在的公司用不同于多邻国的方式实现了连胜,或者你在我们没有谈到的某个维度、某个功能要素上取得了巨大成功,我很想了解更多。我以前基本上会把市面上每一个连胜功能都记录下来,但随着这个功能越来越流行,确实跟不上了。所以,如果你有有趣的连胜洞察想分享给我,我非常乐意倾听。
**Lenny Rachitsky:**我喜欢这个想法——收集所有最佳连胜实践。Jackson,我只想对你的团队和你本人说声恭喜,产生了这么大的影响力。这是很多产品经理和团队的梦想——看到如此巨大的影响力,持续不断地交付成果。恭喜你,干得漂亮。
**Jackson Shuttleworth:**非常感谢。
**Lenny Rachitsky:**谢谢你。那么,Jackson,再次感谢你来。
**Jackson Shuttleworth:**谢谢你 Lenny,这真的很开心。
**Lenny Rachitsky:**我也是。大家再见。
非常感谢收听,如果你觉得这期节目有价值,可以在 Apple Podcasts、Spotify 或你喜欢的播客 App 上订阅。也请考虑给我们评分或留下评价,这真的能帮助其他听众找到这个播客。你可以在 Lenny’sPodcasts.com 找到所有往期节目或了解更多关于这个节目的信息。下期再见。
术语表
| 原文 | 中文 |
|---|---|
| Anton | Anton(Jackson 的前任产品经理,曾负责留存团队) |
| Antonia | Antonia(Jackson 的联合负责人) |
| bend not break | 宁弯不断 |
| Bing Gordon | Bing Gordon(多邻国董事会成员,游戏行业资深人士) |
| Bluey | 《布鲁伊》(Bluey,澳大利亚儿童动画) |
| C1/C2 | C1/C2(欧洲语言共同参考框架 CEFR 的语言能力等级) |
| copy testing | 文案测试 |
| CTA | 行动号召(CTA) |
| CURR (current user retention rate) | 当前用户留存率(CURR) |
| DAUs | 日活跃用户数(DAU) |
| Duolingo | 多邻国 |
| Earn Back | Earn Back(挣回来,多邻国通过完成课程找回连胜的功能) |
| Emily in Paris | 《艾米丽在巴黎》(Emily in Paris,Netflix 剧集) |
| extinction level event | 灭绝级别事件 |
| Farmville | Farmville(一款社交农场模拟游戏) |
| feature bloat | 功能膨胀 |
| Group PM | Group PM(集团产品经理) |
| haptics | 触觉反馈 |
| Hobbs | Hobbs(底特律动画工作室,被多邻国收购) |
| Home Depot | 家得宝(Home Depot,美国家居建材零售商) |
| IAP (in-app purchase) | 应用内购买(IAP) |
| Jorge | Jorge(多邻国员工,增长模型文章作者) |
| juice is worth the squeeze | 值得花这个精力(收益值得付出) |
| lightning round | 闪电问答 |
| local maxima | 局部最优 |
| loss aversion | 损失厌恶 |
| Luis | Luis(多邻国联合创始人兼 CEO Luis von Ahn) |
| one-way door / two-way door | 单向门/双向门(亚马逊决策框架中的概念) |
| Peloton | Peloton(家用健身器械与在线课程平台) |
| Perfect Streak | 完美连胜 |
| Phoenix Duo | 凤凰 Duo |
| PM | 产品经理(PM) |
| Pokémon | 宝可梦 |
| Retention Team | 留存团队 |
| Royal Match | Royal Match(一款消除类手游) |
| Serenity Prayer | 宁静祈祷文 |
| soft ownership | 软性所有权 |
| standup | 站会 |
| streak freeze | 连胜冻结 |
| Streak Repair | Streak Repair(连胜修复,多邻国早期付费功能) |
| streak saver | 连胜拯救(streak saver) |
| streaks | 连胜(连续打卡天数) |
| TEDTalk | TED 演讲 |
| UXR (user experience research) | 用户体验研究(UXR) |
| XP | 经验值(XP) |
此文档由 AI 分片翻译(translate_long_document)