职业发展框架、A/B 测试、入职建议、向工程师销售 | Laura Schaffer(Amplitude)
Career frameworks, A/B testing, onboarding tips, selling to engineers | Laura Schaffer (Amplitude)
Laura Schaffer: … Like the dead of the night. And by that, I mean 7:00 PM or something on. I’m pretty sure it was a Friday. We just asked for forgiveness and put these questions into the silent flow and ran as Navy test with a small group. And I’m fully expecting, “Okay, this is going to hurt our numbers, but maybe it won’t be so bad and I’m going to be prepared to advocate the power of this data that we’re getting.” And I was totally geared up thinking about written, started to write the framework for how I wanted to surface this. And we start to get the data for this thing. I’m not kidding, an improved conversion. There’s no personalization, nothing past it, just the questions. An improved conversion by like 5%, just improved signups. And it was one of those like, “What? Okay, what is going on here?”
New Roles & New Hire Rankings
Lenny: Welcome to Lenny’s Podcast where I interview world-class product leaders and growth experts to learn from their hard one experiences building and growing today’s most successful products. Today my guest is Laura Schaffer. The week we recorded this chat turned out to be Laura’s first week in a new gig as head of growth for Amplitude taking over for a previous legendary guest, Elena Verna. Prior to Amplitude, Laura was VP of product and growth at a company called Rapid. Before that, she spent over seven years at Twilio as Head of Growth and PM lead of the growth platform and experimentation platform at Twilio. In our conversation, we dig into Laura’s career growth framework and the importance of carving your own path versus waiting for one to be carved for you. We also get into a bunch of tactical and surprising advice around running experiments, making decisions on gut versus data, developing your growth strategy and how to sell your product to developers. Laura has a wealth of wisdom and I learned a lot from our conversation. With that, I bring you Laura Schaffer after a short word from our wonderful sponsors.
Laura, welcome to the podcast.
Laura Schaffer: Thanks, Lenny, it is so great to be here. Thanks for having me.
Career Development Frameworks
Lenny: It’s great to have you. So I asked Elena Elena, Elena, I’m not even sure how to pronounce her name, maybe. What is it?
Laura Schaffer: Elena. You got it.
Building Your Brand with Customer Insights
Lenny: Elena. Okay. Okay. I think I’ve said it wrong all the time. All this time. Okay, Elena. So I asked Elena Ferna, who’s a popular guest on this podcast who I should have on this podcast and you are the first person that immediately came to mind. And so I’m really excited that we’re doing this and that you agreed to be on.
Laura Schaffer: Well, she’s the best and I’m really happy that she referred me because I’m just stoked to be here. So thanks for listening to her guidance.
Proactively Carving Your Own Path
Lenny: Absolutely. And it’s a cool time to be chatting. You’re the newly minted head of growth at Amplitude and so congrats, first of all.
Amplifying the Impact of Customer Insights
Laura Schaffer: Thank you. Appreciate that. Yeah, this is my day two and a half here. So very [inaudible 00:05:04].
Proactively Creating Opportunities
Lenny: Wow, you’re a veteran.
How to Pitch to Management
Laura Schaffer: Yeah, right.
Experimentation and Growth
Lenny: I love it. Some companies, there’s a little percentage of that shows you how many people have joined before you and I wonder what that percentage already is [inaudible 00:05:12].
Laura Schaffer: We had that in Twilio and I got pretty, pretty high up there after a while. We had a stack rank and a spreadsheet. Yeah, but it is funny. So wherever that thing exists in Amplitude, I am right fresh there at the very bottom.
Bypassing Process with “Night Deployments”
Lenny: So what was the number you got to Twilio? Any, do you remember?
Which Specific Questions Were Added
Laura Schaffer: Yeah, no, I was very proud to crack the top 50. That was my claim the same because as people left, you move up. Right?
Lenny: Right. Yeah, it’s bittersweet.
”Pill in a Hotdog”: Understanding User Psychology
Laura Schaffer: Well, yeah, right. On one hand it’s like, “Ooh, very cool.” And one of the OGs on the other hand, it’s like, “Oh my gosh, this person’s [inaudible 00:05:47]. Bummer.” It’s a shift, but I’m excited about it for sure.
Lenny: So you have this new exciting role and I thought it’d be fun to start to chat about career growth and just how you think about career growth. I know you have a framework of how you think about your own career growth and clearly it’s worked out, so I’m curious to hear about it and see how it could be helpful to folks that are listening. So yeah, can you just tell us about how you think about career growth?
The “Horror Factor” of First Steps
Laura Schaffer: Career growth is definitely not a straight lineup, but there’s definitely some frameworks and methods that have worked really well for me. And I think to dive into it, it’s first good to just talk about the one that I most typically see people use to try to grow their career and why that can be a little problematic, which is that I see most people try to work really hard the job that they have within the role that they have at a company. Do whatever you can to grow there, show your manager all these things. I see people keep spreadsheets, it wins. So it can come up with performance reviews. Maybe you try to get better advocating for yourself, maybe try to get peers to notice or your manager’s peers. And that’s all good. It’s all stuff.
But the problem with it is that you’re limited to what your manager’s ability is to advocate for you, to promote you. And you’re also limited by the explicit trajectory of your role at that company and where the there’s room for that or not at the company. And then often that perception can sometimes be a little bit in contrast to what your perception is. And also other things that happen, your manager leaves and then you have to restart with someone else. So the method that I use tries to take that power back a little bit. And something that I learned really early on in my career, I was very lucky to learn by accident, was at a company called Bandwidth, which is my first real job. And Bandwidth is now a public company and they’ve done all kinds of crazy amazing things.
But I joined when it was just 50 people and I actually joined in sales and I was just hungry to make it succeed and grow and bright eyes and everything, first real job. But I realized after a few months of being in sales that I was often repeating the same thing over again, using the same thing to sell over and over again. And it’s like, gosh, this isn’t ideal for the customer because [inaudible 00:08:04] call me and ask these questions and get these answers and all this stuff. And it’s not ideal for the company because they’re paying commission on this every time. That’s not going to be efficient for our growth. And because we were small, I was able to catch our GM and I was just like, “Hey, I’ve noticed this pattern where I’m repeating things over and over again and they’re asking the same thing.
I think we should put that online. I think we should make that available so they can just see it and then buy it,” because we had an online checkout process. And I was expecting him to be like, “Oh, well, I know it’s important but for this [inaudible 00:08:46] another, we need to do it this way, and obviously you’ve thought all about it.” And thinking, “Oh, I’m going to come in this new person, he’s just going to help me understand what I’m missing here.” There’s a little bit of that that I was expecting. And he goes, “Wait a minute, tell me more about that. What do you mean?” And by the end of the conversation he was like, “Hey, why don’t you go do that? Why don’t you go build that experience? Why don’t you put that stuff in a self-serve flow?”
And we called it e-commerce manager and it was like got a growth before this growth, this is like 2010. And that moved me into a totally new position. And the main learning that I had from that was, which really took life at Twilio and absolutely worked for me there and I’m happy to talk about that too. But the core that learning was, your executive team and executive teams at companies are often very sharp, but the nature of their day-to-day just does not link them with customers.
And that means that over time, especially as a company grows, they often lose access to some of the best insights and in the heartbeat of the people who they’re providing value to in contrast to folks that are closer to the problem. And so that means that your superpower is in really pulling those insights in and bringing them to life, staying close to the customer. There’s not a single leader or executive that isn’t going to be stoked to hear about valuable customer insights that highlight problems they might not be seeing. And there’s a lot of those. So especially when they align to North Star metrics, those ones are the powerful ones. That was the way that I grew my career at Phil and I’m happy to share that journey too.
Complete Redesign vs. Iterative Optimization
Lenny: Yeah, it’d actually be cool to hear maybe another example of that. But I think an interesting thing that comes up for me here is sometimes you may have an awesome idea and it may not immediately happen. It may not be like, yes or let’s move on, that’s right, immediately. And I think it’s important to just recognize they’re not going to follow all your ideas, but they’re always looking for better ideas. And to your point, they may not have the information that will lead to an idea that you will have because you’re on the ground dealing with real problems, day-to-day. So I think it’s important to recognize you’re not going to always get your way and that’s normal.
Letting Failure Be Your Compass
Laura Schaffer: Yeah, totally. And it’s almost like building up your individual brand a little bit. And I think one of the most powerful and accessible ways is learning about your customers. There’s always those people at companies who’s like, “Oh, well, she just knows our customers or he just knows our customers, they just know our customers. They just know.” And it’s like, “Well, how?” “They just know. Let’s ask that person. Let’s get their feedback.” And those people often have a good amount of brand recognition of powers within the company and they’re often thought of when the company needs to do something new or different or if someone is hiring, maybe they’re thinking about that person for a cross team thing.
So it’s one of the ways that you can build that brand. And again, I think it’s a sweet spot because it’s something that is very valuable to everyone, all the way up to the most senior leaders, which we can talk about here in a minute. And so it’s going to be valuable for you in a valuable tool no matter where you’re at in your career. And that’s not always an immediate payoff, but it often does give you a trajectory outside of just your role and just your manager. It gives you something a little bit broader.
Lenny: So maybe a simple way of describing to mirror back what you’re saying is carve your own path. Don’t necessarily assume your managers will give you the path that makes most sense for you or even give you the biggest opportunity. Just propose, “Hey, I think this might be a better opportunity and I’d love to pursue it.” I’d love to hear the Twilio example if that’s generally-
Why We Fail So Often
Laura Schaffer: Yeah. So when I joined Twilio, there was no growth team at all, not even a breath of it. I joined in product marketing and I was leading our product marketing for our messaging lines, but I followed the same guy that I just mentioned. I made it my own personal policy to like, hey, I’m going to do my job. And I’m going to do well, I’m going to keep notes of things I’m doing well and all that stuff because it’s good, but I’m also going to get to know our customers. And I’m going to get to know our customers really well and I’m going to pay attention when I’m connecting with them, not just about the space I’m in, but just broadly what are some of the pain points and things they’re articulating that are relevant to the business and what we’re trying to get done.
And one of the things that came up was that users were struggling and folks were struggling to get started and use Twilio. And that contrasted so deeply to some of the things that our executive team was saying and had high conviction in our company had high conviction, which is that Twilio was so easy to use. In fact, it was top three things about Twilio that we were really trying to get out of their brand. Were so easy. Developers love us, they say we’re so easy. And there were tweets coming all the time, developers saying like, “Oh my gosh, they got started in a couple of minutes.” So there’s all these things that made that compound and made that conviction stick. But as I was talking to customers, I was hearing a very different story and it made sense as we were penetrating new markets, adding more products, we were adding complexity and we were pulling in folks who were a little bit less motivated and those things contributed to people saying this is difficult.
And so at the time, this wasn’t a 50 person in where I could just go to the floor and go to someone and be like, “Hey, there’s this thing I heard about, I think we should do something.” But there was another tactic that I could take, and I just started sharing a voice of the customer report. I started sharing my insights, started writing down and just sharing them. And it became with digest and eventually people were like, “Hey, can you share it with me? Can you share with me? Can you get on your list? Can share with me?” And this was in a few months of me joining, I was doing this. And then that turned into, “Hey, you should host a quarterly voice to the customer session or for all of product.” And this was a request that was coming from some of the senior leaders at the company.
And when our Jeff Lawson is our CEO at the time heard about, he started attending too. So now in the session I started pulling in other people’s insights too, because now they had a forum for this. I could do that and have people send that to me and I could compile it and all of these things. So then this established me as that person who knows about the customer even after short tenure. And then when came time to do annual planning that year, and I joined in 2014 at the end, so this is 2015, I pitched this idea, “Hey, we think that it’s easy. It is not. Here’s data that I have, the information that I have and I think that we need to start a growth team here and that needs to be a core focus.” And I was able to bring in a really critical partner to that and other folks who could support that because I had built up some of that trust.
So by the time I was making that pitch, I had someone on Andre Crow who was the seventh hire at Twilio and got to number three on that spreadsheet or whatever who was really close to the CEO being like, “Yeah, we desperately need this.” I’m seeing this. He led a website, he basically created the Twilio brand and he led all the website stuff and he is like, “Yeah, we definitely need this.” So not only did I have that little bit of trust from the executive team, but I also had folks who were just trusted on their own advocating and supporting this that I was doing. And so it was approved just almost very easily. I put stuff together for it, but it was the meeting before the meeting had already been done by this other thing. So it helped me create the growth engineering, growth product team at Twilio.
How to Improve Experiment Success Rates
Lenny: I love just how proactive your advice is here. There’s a lot of people that don’t do well and then just like, “Oh, I never had the opportunity or I kept got looked over all this time.” And I love that there’s all this just like, here’s things you can be doing to get in front of people to provide value, to just create opportunity for yourself. Any other advice along the lines of just like, here’s the things you could do for yourself versus waiting for someone to come and give you opportunity?
Laura Schaffer: Yeah, I think that’s the most easily actual because to do all of our jobs, we need to know customers, we need to know about customer insights, product, we need to know. But then also customer facing teams, social, those want to crack into product. Your insights are extremely valuable. You’re talking to customers every day more about their problems and their pain than a lot of other people do. And so that is by far and away to me the most powerful and accessible by anyone in any role in any space. But I’ll also say that that broader concept of just, hey, there’s things that, and things of value that you know that others can benefit from at your company and building your brand as someone that is supportive, smart, creative, able to solve problems, make sure that you’re sharing that. And so maybe you’re really freaking good at communicating with brevity.
I suck at that by the way. So more powered anyone that can do that, I’m actively working on it. So share that. Go to your general Slack channel or whatever and just be like, “Hey, just wrote some tips for how to do it, some ways that I am good at this.” And those kinds of things can really go a long way towards people starting to view you as an SME and not just the space that you’re in, but in broader areas. And that can always present open doors for you and other people are looking up to you and seeing you, someone who’s strong in ways outside of just the role that you’re in.
Where Do Good Ideas Come From?
Lenny: SME is a Subject Matter Expert, is that right?
Lowering Confidence to Run More Experiments
Laura Schaffer: Yes. Thank you for unpacking my acronyms. That’s another thing that I am actively working on.
Lenny: I got you. I’ll be on the lookout.
Cross-Validating Qualitative and Quantitative Data
Laura Schaffer: Yes.
Writing Plans and Life Stages
Lenny: Maybe one last question along these lines is do you have any advice for framing the proposal, framing an opportunity to your manager, higher ups that you see has worked best broadly?
Laura Schaffer: Yeah. Yeah. And one thing I want to say too is with this stuff, I don’t think that it necessarily does counter to what your manager’s doing. It’s more supporting them. I’ve done this stuff and then it’s helped my manager promote me. So it’s not necessarily, “Oh, we’ll do this if your manager’s failing you.” Or they are not the boarding you or they can’t support you. It’s more like do this because this is going to be an accelerator for yourself irrespective of your manager.
But then also it’ll be an accelerator for your manager in supporting you because one of the things that comes into play a lot when managers figure out promotions and doing those things is they’ll sit in a room, often calibrations and with a bunch of people, and it makes it a lot easier when those people have had some access or exposure or whatever to you in a positive light. So these things can all run with your manager and not against, but it’s just another way of you taking back the ability to build that momentum instead of relying on all of that going through one single other person.
Don’t Let Data Fit the Hypothesis
Lenny: And what I like about your second example is you just did it. You just started doing that tenant report for the company. It wasn’t like, “Hey, I have a proposal, here’s what I think you should do. Should we do it?”
Growth Team Timelines and Expectation Management
Laura Schaffer: Exactly.
Lenny: It’s just like, yeah, just do it.
The Low-Medium-High Planning Framework
Laura Schaffer: Yeah, ungate your knowledge I think is the buzzword that I’m hearing.
Non-Developer Insights & Quick Deploy’s Origins
Lenny: Mm. Never heard that.
Pitfalls of Sales-to-Product-Led Shifts
Laura Schaffer: I think that’s an Elena, see how many times we can bring her up. But you can do that within your own company. Everybody is skilled at things that they aren’t explicit to their role or their space. And I think that ungating that opens opportunities. And if you’re not sure, then go to my favorite go-to, which is talk to customers, get insights. Those are incredibly valuable. So rarely do people share those when they find them. So be the person that does that.
Lenny: Another area I want to chat about is experimentation and growth and data, which makes sense if strong perspectives is on being the new head of growth amplitude. So maybe we start with experimentation. You mentioned that there’s a really interesting surprising result in an experiment you ran at Twilio that maybe changed your perspective on experimentation and what you think might work and not work.
The Startup Analyst’s Dilemma
Laura Schaffer: A hundred percent. Yeah, I’m through a fortune of two mind-blowing experiments that really shifted the way that I think about growth. So one of them, one of my favorite ones happened very early on at Twilio. So after I created this growth team, one of the things that I saw as to me an issue was that under signup flow, we just asked people for a username and an e-mail, like a password, and that was it. And that’s actually relatively common at the time. This is a while ago now, everybody is [inaudible 00:21:16]. But we didn’t, and actually there was a lot of existing conviction around that. I was like, “Hey, we retarding developers. Developers, they just want to do, they just want to get their hands on things. Don’t put anything in their way, it’s going to be disastrous. We don’t want any shenanigans here with these folks, let’s just let them in the gates.”
But to me this was a really big assumption to make and a very costly one. It’s like, okay, if that’s the case, we’re not going to know anything about anyone. And we didn’t know who was signing up, we didn’t know what they wanted to do. And that hurt our ability to understand how people were performing from a quantitative perspective. We were a little bit lost with prioritization. There’s a number of implications here, but it’s obviously a very contentious space. So this is the very first thing that I did and the first experiment that I ran. I did some research to understand say, okay, what are the most important questions to answer? What would I do really, really need to know? And it was stuff like what language are you coding in, what’s your use case? What product you want to use?
And then there’s one around, are you developer at all or were you use something else? Because there is rumors that we’re having, not just developers sign up, which is this whole other interesting story. And I think of these questions that would also potentially be things that our developers signing up would understand why we’re asking that it would feel natural. But anyway, again, adding anything to the signup was very contentious, but I really just wanted to get a little bit of data on it. So I wanted to run a test. I didn’t have a team, I didn’t have an engineering team yet and none of that stuff had built out. It was just me, myself and I. But like I said, I had started to build a little bit of trust and [inaudible 00:22:50], Andre, who I mentioned earlier who, because he was early employee and he had access to everything, one of those people, and he also was supportive of this and had similar haunches.
And so like the dead of the night, and by that, I mean 7:00 PM or something, I’m pretty sure it was a Friday. We just asked for forgiveness and put these questions into the signup flow and ran as Navy test with a small group. And I’m fully expecting, “Okay, this is going to hurt our numbers, but maybe it won’t be so bad and I’m going to be prepared to advocate the power of this data that we’re getting.” I was totally thinking with written… Started to write the framework for how I wanted to surface this. And we start to get the data for this thing. I’m not kidding, an improved conversion. There’s no personalization, nothing past it, just the questions. It improved conversion by 5%, just improved signups. And it was one of those like, “What? Okay, what is going on here?” And I actually dug into it, and what I found from just talking to a few customers once through the flow, I’m just learning about [inaudible 00:24:00] about it.
It was actually for folks, it was comforting. When you think about it, when users are signing up for your product for the very first time, it’s new. This is new, that means it’s scary. They’re expecting it to be difficult. They’re anticipating that there’s going to be friction and challenges and that they’re not going to figure it out. Almost like looking for the bogeyman. And that’s the headspace. It’s often the headspace that any of us are in when we’re doing something new for the first time like ooh, this could be very challenging. And so by putting in these questions it’s like, what’s your language? It’s like, “Oh, I do. I code a JavaScript and I can select that.” Well, that’s something I’m uncomfortable with. That would make my journey easier. Like “Yeah, bingo. That’s my use case. Okay, I’m in the right place here.”
It was actually giving folks something comforting and challenging the notion that this was going to be difficult, just the questions because it was aligning to some of the things that they were organically thinking about, which is what if they don’t support my language? Or can I even do this use case I want to do? And so it was just a really interesting, the takeaway for me for this, the really interest takeaway was the psyche of the user is so, so critical. That’s just as important as understanding your product and the broader market you’re applying to and all those things. Just the psyche of users, new people doing things for the first time in your user flow, understanding that is powerful. And the simple catchy thing I say is that ultimately the learning here is, bad friction is bad, and good friction is good. There’s no such thing as it being simple. It’s just all friction is bad, which is what I had assumed going into this.
Favorite TV Shows
Lenny: I love that you were new to Twilio and you just year-load an experiment to production.
Laura Schaffer: Year-load. Yeah.
Favorite Interview Questions to Ask
Lenny: That’s a big move.
Everyday SaaS Tools We Use
Laura Schaffer: It ended up being very helpful for everyone. I shared the insights from it and all these things. I’ve shared the [inaudible 00:26:05].
Lenny: And [inaudible 00:26:05] conversion.
Small Changes in Product Development
Laura Schaffer: But for sure, use such processes with caution for sure. Yeah.
Tips for Using Amplitude
Lenny: I love it. It’s amazing.
Laura Schaffer: That the right way to do [inaudible 00:26:15] advocating for the engineers here is the right way to make any changes in production is through or with the approval of engineering, but it was the right move overall and definitely helped business, so yeah.
Final Thoughts and Outro
Lenny: Yeah, I love it. No, that’s great. I like that move. I think we need more of that probably. I want to dig into what you actually… So what is it you changed, you added how many questions and then what were the questions?
Laura Schaffer: There was a question around what language are you coding in? And then as an option to that, it was like, “Oh, I’m actually not coding, I’m not a developer.” So for us, it actually gave us two really, really interesting data points. One was how many developers versus people who are not coding or in our flow? And then what language of the coding, which was massively helpful not just for growth and onboarding, but our documentation team, dos team. Would that end up being a critical way for us to gauge trends over time and catch things before whatever reports would come out at the end of the year, what people are doing, you start to see it. And then also product. What product are you interested in using? That was very critical for knowing the basics of how to organize someone’s onboarding. Are you doing SMS? You’re doing voice? To 281 or whatever. And then use case in use cases, you’re doing a appointment reminders or are you doing a autoresponder or are you doing anonymous communications for a dating app or something. Right? So those were the very first questions.
Lenny: Wow, okay. So it was four dropdown questions and that increased conversion. I love these examples where friction and increased conversion… There’s so few of them. You hear about this could work and it’s rare. And so what did you take away? What’s the pattern you took from this? There’s the idea, it’s good friction, but is there something that you’re like, what is a sign of this is going to be good friction?
Laura Schaffer: There’s still alleviated a problem. They alleviated the problem they had where they’re coming in and worried that it was going to be difficult or that they weren’t going to be able to figure it out, they weren’t going to be able to get their footing. And I’d say that that’s not unique to Twilio. That’s something that I think users experience at any front door, at any company, any signup beginning the signup loads, it’s like, “Here we go, buckle up.” Especially when it’s in a work context and there might be extra pressure on you to succeed or for you to make as an accurate assessment. So I think that psyche of, “Okay, am I in the right place? Is it’s going to do what I need it to do? Can I figure it out? Am I capable?” These are extremely common things for people to feel when they’re signing up.
And so certainly, that I think can carry out to any place. I’d encourage absolutely everybody to be putting those experiences within their early onboarding, not just for you selfishly, so you can learn and segment them appropriately, but also so the user can feel more confident as they get going and like, “Hey, I’m in the right place. This is going to do what I needed to do.” But I think that the carry over there is just the psyche of the user and just being so aware that it’s not so cookie cutter as, “What is the problem my market experience is and what can my product do to help them?” There’s also this other thing in the room which is so important to people’s success, their ability to succeed with your products and your self-serve experiences, which is, what is the mentality and the psyche of the person at the various stages in your journey?
And if you’re not incorporating that or addressing that, you will absolutely miss things or things will fail and you’ll be very confused as to why. We had a great experiment that I’m happy to talk about where same concept, a totally different situation, which is later in onboarding. One of the things that we tried to do over time to make Twilio less complex was to offer steps like, [inaudible 00:30:08] onboarding, welcome step one, here’s what you want to build. Great, we all know that now, “Okay, step one, go do this thing. Step two, go to this thing. Three, this thing. Four, this thing. Five, bam, you’re live, congratulations, aha,” all these things. So we shipped that, got that out there and I was like, yeah, it was improved conversion. It wasn’t like that great. It’s like, man. We went from there being absolutely nothing, “Choose you’re adventure, figure it out, go figure it out. Good luck.” To this prescriptive thing. And it wasn’t converting [inaudible 00:30:37].
So to talk to some users and there wasn’t anything particularly obvious that was coming out as to what the issue was. It was like, “Oh yeah, Let’s go to step one.” And we did mock the people. “Okay, now I know, I do step two.” But there was one thing that I was hearing that was coming out that feels like something, and that was the telephone number, the telecom part. Developers when they were coming into Twilio, it was things that were familiar to them. APIs, the language they’re coding in, code samples, documentation, things like the bogeyman, the things that would psychologically trip them up, telecom, phone numbers. These things that just were completely out of the zone of anything that they’d ever worked with before, especially earlier on in Twilio’s journey.
But even now, right? Telecom’s very different beast for most developers. And guess what was step one? Get a phone number because that’s step one. Anytime that anyone’s trying to teach one to use Twilio one-on-one, always going to sit down, ask and be like, “Okay, here we’re going to go get a phone number and configure it.” And that’s what anyone every time will do. However, in a self-serve experience, when you don’t have that safe person sitting next to you being like, “Don’t worry, it’s going to be okay, I’m going to take you through this crazy telecom journey.” They’re on their own, but that’s psyche telling is them like, “Oh my god, telecom. Well, I can’t do that. That sounds scary. We’re getting a phone number configured. Whoa, I’m out of my debt.”
And so what did we do to test this out? Test out whether that was the issue? Actually, and it’s first we’re in the MVP. They kicked them out of the portal entirely and put them into a docs page where we could manufacture an experience where the first thing they saw was code and they’re in the docs safe place, the language that they’re coding in and then snuck in there.
It was like, “Oh, get a phone number, let’s go configure it.” Not as step one, not as the leading thing, but embedded. And the analogy I have for this is pilling a hot dog. So if anyone’s got a dog or an animal you have to feed a pill to, it’s like you can’t just feed the pill to the animal, it’s never going to happen. But if you shove it inside of a hot dog, which looks good and that’s exciting, then you can get them to consume it more easily. And so this was-
Lenny: Yeah. We do peanut butter, that’s [inaudible 00:32:55].
Laura Schaffer: Yeah, exactly, right? Yeah, hot hotdog, peanut butter, all that. You bury it. You embed the scary unpleasant thing. And so that’s what you said with the phone number stuff, that telecom stuff. And guess what? Even though we’re a good amount of the console and they’re going off and we had no easy return button, it converted better because we were addressing the big problem that was there at the time, which is their psyche. They were not ready to come in and immediately thrown into a phone number experience. That was letting the bogeyman and out to party and that’s not what was going to work. We needed to put that bogeyman pill and the hotdog.
And so then once that validated, then we can actually go through the business of putting that into the onboarding float correctly and then that could be even better. But so again, the psyche of your user is such a critical thing to be thinking about. And if something very logical isn’t converting well sometimes, it means that you’re battling against the psyche of a user and you want to take a step back and think about and learn about where someone is psychologically in your space.
Lenny: Feels like you had this experiment that was a complete redesign of the onboarding flow and that didn’t work. And then your second attempt was a different approach that’s like a full onboarding flow. And I’m curious, do you have a take on just when you run experiments? And it’s something we dealt with a lot at Airbnb in other places is like, do you just redesign the whole thing or is it better to iteratively work from where you’re today and just experiment piece by piece towards some future much better experience?
Laura Schaffer: Here’s what I would say to this is that from a high level, it’s always going to be better to be iterative. And the reason that it’s better is that roughly 80% of the times, ORs in the time are hypotheses and the things that we believe will be true [inaudible 00:34:42]. And this is amazing. There’s an amazing article out there I’m happy to share with you so you can put in the show notes.
Lenny: Yeah, absolutely.
Laura Schaffer: That really takes a scientific approach to proving that out. Companies like Netflix and Microsoft, there’s over and over again 80 plus percent. Some companies say 90% of things fail. And so the closer you get to something that you go bear your head in the sand or go into an attic and build something for six months and ship it, the more likely it is that you are going to ship the 80% wrong stuff.
Whereas the more iterative you are, the more likely that you’re going to catch it sooner. And failure doesn’t have to be a wall, it can be a compass, it can be the thing that leads you to the right thing. And so you always want to as best you can, get stuff in front of customers so that you can get that compass and get that compass activated, know where to go.
So that means doing ugly things. I tell my teams all the time, if it’s not embarrassing, you’ve gone too far. Got to be embarrassing. The first thing, that was embarrassing, kicking people out, onboarding, spend all this money and whatever to get them into your center flow, and then the first thing we do, get out of here. That’s nuts. But if it hadn’t validated, that would’ve been a very cheap but very valuable learning. Instead, it was a very powerful cheap learning in the other direction. Okay, now we know we can invest in it. We know that’s the right thing to do. So always better to be iterative so that you are letting failure work for you instead of having it be a trap that you fall into.
Lenny: I know that you just shared as per experiment, you’re probably wrong 80% of the time. In my experience, launching a halt redesign is as negative a hundred percent of the time. I’ve grown weary to avoid that as much as possible, which is like, you know that, you’re taught that as you go into growth and product, but you’re just like, “Nah, come on, let’s just make it awesome. Just redesign this whole thing.” Especially your designers is always like, “No, let’s start again. Let’s make it amazing.” But it always ends up being negative and you’re like, “Okay, well, it’s too late now we got to launch this thing, we don’t have time to start again.”
Laura Schaffer: Well, it’s funny, in the articles, and you’ll see it was written by somebody from Microsoft who built Implementation platform and did all these cool things, as he went into actually trying to apply a scientific method of figuring out how often people are wrong about their hypotheses and what they’re planning to do. He’s like, “I wonder if that applies to us here at Microsoft.” Even for him, that question of [inaudible 00:37:16]. And I think it’s challenging when there’s a lot of smart people in this space doing things and it’s very difficult to think, “Gosh, am I really wrong? 80 plus, 90% of the time?” But when you think about it, makes total sense because what has to happen for something to be successful? You have to understand the problem perfectly. You have to then understand who’s having the problem perfectly, the customer. At what time they’re having the problem.
Then you’ve got to put the right solution in front of them to solve that problem. Maybe you’ve got the problem, all that stuff, but your solution something off. Or maybe your solution is right, but maybe it’s just not presented, it communicated in the right way. You could have any one of those things off and it’s not going to succeed. It’s not going to have the metric impact you’re expecting it to have. So in that context, it’s almost like incredible. We do succeed 20 to 10% of the time given everything that has to line up. And so I think it’s one of those things where you really want to go into it embracing that, “Okay, this isn’t about how smart I am or how good my team is or any of that stuff. It’s just the logic of this is challenging to get it right and let’s embrace that and let’s lean into that knowledge and make it a part of our strategy,” instead of finding against it.
Lenny: Have you found anything that helps you increase those odds or is just, this is the way of the world and you probably can’t significantly increase the chances your experiment works out?
Laura Schaffer: So here’s the thing, I think there’s very little that we can do to make that space easier. All those things have to be figured out. And so I definitely think that everybody is going to be in a space where their original ideas, untested ideas are going to be around that hit rate. However, the way that you go about validating those can be totally different and you can be very fast about validating those ideas and that’s the key. And AB testing is one of the most expensive kinds of ways to validate an experiment. It often requires design and engineering and the PM or growth person or marketing person who’s crafting it. All these things are investments that take a lot of time even for simple thing. And then you have the time factor, how long’s the thing I have to run to have an impact?
So all of that is extremely expensive. And so I think the key is to just think through, “Okay, what are the things I can do to quickly validate what these ideas are that we [inaudible 00:39:59]?” And you can do that with painted doors, which is where you test rate the concept and the idea before it exists versus the actual experience. You can do mocks. If you’ve got a designer, create those mocks for that experience, put it in front of people, see how they engage with it. That can be so powerful.
You could invalidate tons of hypotheses at that state. The only things you want to get to that deep AB testing environment or ones that have been vetted along the way. And that way, you reduce your fail rate, because you’re failing faster by using other methods. So I think I more advocate for that side. Let’s fail fast by using those tools rather than figuring out a way that you can rise above where everyone else is operating and figure out ways to solve all that complex stuff better because that’s going to be challenging, but you can always get better at experimenting and validating things faster.
Lenny:
And unlike other AI applications, writers’ training happens securely on your data and your style and brand guidelines that you provide specific to your organization. The result is that you get consistent content in your brand voice at scale. Get AI that your people will love for a limited time. Listeners to Lenny’s podcast and get 20% off if they go to writer.com/lenny, that’s writer.com/lenny.
Where do you find the best ideas come from for driving meaningful lift? Is it gut instinct type and experience bucket or is it data telling you like, “Hey, or here’s a huge opportunity,” in your experience?
Laura Schaffer: I’m a very data-driven person. I self-describe and think of myself that way. In large part because of that, I feel you have to be constantly checking yourself and data is a really great way to do that. But I definitely think that I would be described as someone who’s going more by their gut when looking at date end results just because of the way that I approach it, which is I’m very comfortable and very common in using qualitative responses and things like that and supplement to quantitative data to make a decision and that puts less of a burden on the quantitative to really make an assessment of whether something was working or not.
One of the things I see, I think sometimes goes against what other folks do, although I’m seeing things shift a little, is that 95% confidence rate. My background in college, I was in a lab running experiments or really publishing two journal and stuff and we had to have that 95% confidence rate, had to because the things that were coming out of the lab and being published were influencing things like how we do education and how we understand how bias works and when it shows up and therefore how we can combat it.
Things were wrong. And sending a bunch of bologna, that can cause some significantly bad things like false positive, false negatives in that context can be very dangerous, for lack of a better word. And you think of other pharmaceuticals, the 95% confidence rate belongs in some companies and some industries because the risk of failure on the impact of a false success is very high.
But those of us converting users and trying to upsell folks, we are very fortunate to not have that level of burden on us and we can take advantage of that. And so there are definitely times where I will advocate for and I will push for and I will myself use lower confidence intervals and 95%, especially if that doubles amount of experiments that you can run in a year. End of the day, these are all methods that we use to try to validate the hypotheses that we have. And if you’re doing a 95% confidence in a role, you’re still accepting a 5%, some amount of false success, do that a little bit more, challenge you to do that a little bit more. And then run way more experience. If you look at the net of what your team is doing over the course of year, what you’re doing over the course of a year, you will be positive.
Lenny: Wow, that is a big idea. Idea of releasing the P-value confidence interval for experimentation and data teams. Everyone would be excited about this. Probably maybe not some data scientists on teams. Do you do that? How do you act? Is that how you operate on your teams? Just like we don’t need 95% competence?
Laura Schaffer: So I’ll say this, this is actually very critically important. You must have this game plan set before you run something. Failure mode that I see so many teams fall into is they’ll run the experiment or whatever it is and then they’ll make the data fit the hypothesis. Or sometimes they’ll go without a hypothesis and just be like, “This is going to do better things for our metrics,” but not a core reason as to why or what exactly are we testing here. And so this is another area we could absolutely fall into that trap. “Let’s [inaudible 00:45:36] on good 80. I think it’s good. That Laura person said it was cool. So I think that that’s fine.” That will always be a trap. So it needs to be very deliberately thought of in advance as a way of like, “Hey, here’s how we’re going to validate this.” And always, always, always, if you’re going to accept more risk of a false success or false positive, false negative, you want to then be really thinking about how you’re going to harden your validation of a hypothesis.
For example, let’s take that when we talked about with Twilio where we are kicking people out and we’re sending them to the pilling hotdog experiment, and we’re sending people to that experience to hide the phone number. Now in that case, let’s say that we were going to accept a lower confidence interval. I would very much want to see qualitative feedback to confirm that that hypothesis was true. I want to be looking at the qualitative data from the ones where people were thrown into the existing flow and one’s put into the dogs that one of them felt more confident and more like this was really easy to get through and they felt out of a territory and things like that. And I’d be wanting to hear from the ones who were in the other one, things like, “Oh, I got stuck on that [inaudible 00:46:48].” Like, “Figure this out, but it feels like it’s that amount of my depth.”
I would want to be looking for other things to corroborate the hard data that I’m seeing. And yes, it opens the door to whenever you open the door to more risk acceptance, you are going to have some false successes there. But all of these things together can overall make it more likely that you’re shipping more things that are going to positively influence the customer. And again, I can’t say it enough. It is a huge risk in and of itself to not ship as much as you possibly could in a year. That is a huge risk given that very high fail rate. So to those data scientists, and I’ve chatted with a few of my time, what I try to explain is that that article, that data that the 80%, that’s hard data about what a detriment it can be if you don’t run an enough experiments.
If you just run 10 in a year on there, maybe two around impact, two of a course of an entire year if you take that approach. So data scientists can understand, “Hey, if we do this, if we run this down, we can double or triple whatever it is, the number of experiments where we can run and overall net that’s going to result in more successes that will overall net us to a positive place.” You can still tell a data story to the data scientist about why you’re doing this. Again, this is why when you asked that question identify, as a very data-driven person. But I think some of the methods that I use can sound at the service level as more like, “Oh, I’m going by my gut.” But again, very data driven is just embracing the reality of some of the hard data that I don’t think we all embrace or are even aware of sometimes about that fail rate.
Lenny: This is awesome. This is a big idea. Have you written about this anywhere for folks that maybe want to try this approach at their company? And if not, you should.
Laura Schaffer: I appreciate that. It’s funny, it’s like all my general life to do is just start writing some of this down. I have three children, one of whom is five months old, and then I have two and four. And so sometimes I’ll start to write and then one of them will crawl across the keyboard. And by one of them, I mean all of multiple times. But eventually, yeah, I’ll be very happy to do that if folks would be interested. I’m always happy to do whatever I can to help folks, help empower folks with knowledge to do better because none of this is secret sauce really. It’s just learn from experience and it’s always better to learn from others’ experience than your own. It’s faster. So yeah, I would definitely [inaudible 00:49:35] is that, I think that’s the best that I can say, but eventually my kids will get older. I hear this and maybe I can do so.
Lenny: Hopefully. Cool. So maybe if you’re watching us on YouTube, leave a comment and if you want Laura to write in depth about this idea and spread it to your company. Okay, I want to talk about growth, but I have one last question just along the lines of experimentation. Is there any other just, I don’t know, big lessons or takeaways of running experiments that would be interesting to share?
Laura Schaffer: I think we got into this one a little bit, but I just really want to exclamation point, underline it, which is that notion of making the data wrap to fit a concept. I think a lot of teams feel and are under a lot of pressure to show progress and, “What did you do this month? Where did the metrics move?” And it can cause folks to feel like they have to do that, where it’s like, “Oh gosh, this experiment.” Everyone’s got the experience where you run an experiment and you’re like looking at the data, refresh, refresh, refresh, oh my gosh, and actually perform worse. Or it’s not the same and, “Gosh, we got everyone really excited about this thing that we all worked on really hard. Like, oh my god, what are we going to say in the QBR or the monthly report?” Whatever it is that the results come to light.
And to this, I’d say this, that it’s incredibly important for growth teams to educate out and for folks outside of growth and leading growth, especially to understand that the best way for a growth team to succeed, the only way really for them to succeed is to embrace the fact that they’re there to validate, to understand what the biggest opportunities are and to go after them.
And that is not something that can be done on a weekly timeline, sometimes even a monthly, depending on the space you’re in and what’s known and unknown. And so any growth team that’s beholden to short timeline wins and improvement is always going to be dangerous. That’s an environment that’s conducive to vanity Metric usage and massaging the data [inaudible 00:51:47]. And ones that are more successful are ones that are reporting over longer periods of time. Because I think growth team, given enough time to fail, enough time to learn the right thing to do is absolutely going to show success, real success. Not that, “Okay, we’re going to make this data fit.” But real moving the metrics success.
And so definitely educating out. If you find yourself in a position where you are beholden to that, share that 80% fail rate. Just math, statistics, data. You cannot be successful in an environment, but over time you can be. And so that’s one thing I definitely would draw on. I end up spending a decent amount of my pie chart at Twilio and then also at Rapid where I was after that and I’m sure I’ll spend some time at Amplitude as well. Just helping folks understand what is the healthiest ecosystem, most powerful ecosystem for a growth team to operate in. And time and expectations over time is a big part of that.
Lenny: When you say pie chart, it’s like the pie chart of your time like a big chunk of your time goes to this?
Laura Schaffer: Yeah.
Lenny: That’s awesome. I like that. I use white charts a lot as to describe that same idea. Just to be a little more concrete there, what is the timeframe you think is the minimum for a growth team to be thinking across?
Laura Schaffer: I think it’s good, especially for newer teams, but even teams in general. Commit to something that you can do over the course of a year and low, medium, high is always helpful in that space. A lot times-
Lenny: What do you mean by that, by low, medium, high?
Laura Schaffer: Low, medium high, more like, “Hey we’ve got a few bets that we have or few core hypotheses.” And if they take off that’s going to be our high bucket like, wow [inaudible 00:53:26]. We think these things could become lightening on a bottle here, but they could also be a bunch of [inaudible 00:53:31] missed. But until we run in, we’re not going to know. And if those bear out though, then yeah, that’s our high. And hey, we’ve got a few things that we think are safer. Maybe it was validated a bit in the previous year, what have you. And these looking really the metrics this amount.
So it’s helpful to give people though that construct, it deviates from it very hard deviates from this notion of like, here’s the single number that we’re going to hit. Just things that help people understand that space a little bit better and what to expect. And because of that it can be a little bit lumpy. There were some things that you released. Truly for the most number of years, can be easiest to talk about in this construct here, but there’s one thing that we did that generated tens of millions of dollars in the pipeline, was really, really powerful and took, sometimes navigate and validate. Other times we did that onboarding stuff that I was talking about catching those things. That could happen on a little bit of a faster clip but still took some time to validate and understand. But yeah, over the course of a year you should generally be able to commit to movement. But help people understand the methods there so that they’re not coming at you on a weekly basis being like, “And what did you do these past couple days?”
Lenny: Okay, I got to follow up on a couple of these things. What was that big change of Twilio that lead to tens of millions of dollars?
Laura Schaffer: This is part of the course that I teach at Reforge.
Lenny: Oh, amazing. You get to work for Reforge with [inaudible 00:54:57].
Laura Schaffer: I’m actually interested in retention, I think is my part. Right now, I’ll give you links so we can [inaudible 00:55:05] in the show notes. But yeah, the high level version, this was deeper into my journey at Twilio. This is fast-forward a few years, build up this team and some cool things going on. But I was really looking for what’s the next big thing for us to do? What could that be? And I noticed, remember that question very back of the day when I asked about the developer versus not developer folks [inaudible 00:55:34]?
Lenny: Mm. Yeah.
Laura Schaffer: We saw that little non-developer little dude ad was growing. We were actually the number of people in the ecosystem who were identifying themselves as not a developer were in the space.
But very interestingly they were, as we got more refined in our understanding of those folks, a lot of them wanted to build with Twilio. There was a hypothesis of like, “Oh well maybe they’re lost, maybe this want pricing, maybe they [inaudible 00:56:00] mistake.” And I was like, “Nope, they’re here to build, they won’t build.” And they struggled through the developer on onboarding and some of them would succeed and some would… But anyway, it was all about identifying what did they need to succeed. If we were made them successful, could it contribute to dollars? One of the core learnings I’d heard from sales at the time was, “Hey, it’s very challenging for us to get the folks when a developer’s not involved yet to go from zero to one to get something off the ground. But man, if we can get them to do that, if I can get them in $1 and spend, I can get them to five. If I get them to five, and get them to 50, like 10,000, then I can get them to hundred thousand.”
This whole long journey like, “Hey, Laura, if your team could just get them off the ground, man, we can do so much.” So yeah, the journey is all about, okay, what were the things that were missing in the experience we were offering and ultimately was they couldn’t write code from scratch. That was really difficult. And also we’re going to stand up a server. That was difficult. But we ended up iteratively experiencing a way to validate those hypotheses and what’s the right way to do this and yeah, that was great. It’s called quick deploy on code exchange. Anyone can go there and deploy an app without having to write code and get an aha moment there with Twilio.
Lenny: That is awesome. So basically it’s like a low code Twilio app?
Laura Schaffer: Yeah, it ended up being, we had a lot of pet names, nicknames for it. I think probably the one that most succinctly describes it as just, it ended up being a create your own demo experience which made you talk about the psyche of people. We talked about how developers telecom until can be intimidating. We’ll talk about the non-development, sometimes the buyers or the people who are instantly buying decisions, for them, it was not only was it telco, but it was the developer stuff was inaccessible but they still wanted to jump in and they wanted to have that experience. And so this was a way for us to give them momentum, give them comfortable. Geez, I can get this running my development team, I’d definitely do it. And so it was a very powerful moment where we could really address the psyche of those users, get them excited about Twilio, and then give sales the ability to give something powerful to those non-engineering buyers and folks they’re talking to.
Lenny: So genius, looking back seems like an obvious win. One of my readers suggested that I start a series of the story of a feature and walk through the discovery ideation development iteration and this feels like a really interesting example of that. But anyway. I got just a couple more questions. I know we’ve been going for an hour now. But I have questions, I don’t want to let you go just yet, and they’re on growth. So one question is just you worked at Twilio, which is very product led growth. You’re now Amplitude, which is more sales driven and I know you’re trying to go more product led. I know Elena talks a lot about this, how every company needs to have product led motions, otherwise they’re going to be disrupted by someone that comes product led. And I don’t know what hires, which blanket would they fall into?
Laura Schaffer: Between the AI like SLG and PLG. Yeah, for me they’re two sides of the same coin. Product growth and sales. It’s all to me, very thematically the same stuff. The difference is that with growth, you are selling with your product, and with sales you’re selling with person like one-to-one. And so companies need to be employing both of those forces to optimally convert their audience. We’re in a world where people are expecting both. They’re expecting to be sold by your product and sold at the enterprise level. And large companies buy by human beings. It’s going to listen to their specific needs and really break it off for them. And if you only have one, you’re going to miss stuff. So absolutely, I think you want those two forces together working well. And obviously there’s different stages, things work differently in different spaces, but I think when it comes to Amplitude, I think there’s a huge opportunity here.
I think the key is, and that the challenge for companies that have done the sales thing and are trying to crack into the PLG thing really comes down to how you fundamentally are approaching that space. And again, your users and where they’re at and the psyche of where they’re at, I think a lot of companies will say, “Well okay, hey, we’re going to do this PLG stuff. Let’s take that sales enterprise whatever offering that we have and let’s chop it up a bit and cut access here and cut out this feature here and we’re going to slap this plan out and we’re going to put a price on it and we’ll maybe have hours of debates over whether it’s like 10 99 or $104 or 75, and eventually someone will win that battle and slap it on and then [inaudible 01:00:54].” And anyway, the discussion of the focus is a lot around the product.
What are we going to do with this product? How are we going to crack it open and shift shift it in and then give it to these people, these users, these visitors? And what it’s missing, I think is, and a lot of times it’s easy to miss, is that when we’re doing PLG and we’re shifting from sales to PLG, we need to reset. We need to recognize that, again, this is sales, sales via the product. What does a good sales rep do when they’re engaging? They understand what the problem is of the person in the space they’re talking to. So we need the same thing here. What are the unique problems of people who are coming into our self-serve space? And I think when it comes to a company like Amplitude, a lot of the folks that will be looking to address via the PLG motion, there’s a number of things we want to achieve there, but one of the primary things is to tap into the SMV market market and really give them a really startups and give them a space to land and to grow.
And again, you have to think what are the challenges and unique problems that they have because we’re going to be using our product to settle them. We need to meet them where they’re at with the problems that they’ve got. And I think one of the things that I’ve observed from being in all these startups and advising some startups is I very rarely… I don’t think I’ve ever come across a startup where they have the right number of analysts for their needs. In fact, a lot of them don’t have any. And so what that means is that the CEO is being an analyst to create their dashboards for the board and the product manager is being an analyst to figure out what the heck’s going on and creating their boards for their product.
And that’s happening all over the place is that people are in their roles and they have to be an analysts too. And I think that that’s a problem that especially younger companies and early stage companies have. And so when they connect their psyche, what are they caring about? What are they thinking about when they’re sending it for product analytic product or something? They’re looking for something that’s going to help them feel reassured they’re going to be able to actually get to the bottom of the right metrics, create the reports that show things the right way. What’s the best way to show churn? There’s got to be a best way so many people are doing it. Guess what? Yes, there are some really good ways to do it and there are some really successful ways to set up dashboards for the board. People have done that too.
There’s a lot of that knowledge that exists in one of those frameworks that exist, benchmarking. Are these numbers even good? And so, one of the hypotheses that I have is that if we take that perspective and we understand that that is the problem, that there’s a number of things that we can do to really change the way that self-serve experience works to help convert people and show them how Amplitude can make them that powerful. But the thing that I think sticks across all companies, not just amplitude making the shift, is just that, that when you’re doing this, do not think this is a copy paste, but chop it for parts thing. Don’t start with your product when you’re building at your strategy, start with your customers, your users, your prospects, the people who are going to be coming into yourself or flow. Make sure you are understanding how their problems differ because they do from the people that you’re addressing at the sales led side. And then make sure that you’re orienting your experience of product around those people.
Lenny: It’s interesting that you almost have to start again as a product company, as a product because you may need to solve completely different problems that eventually lead to the same place. But it’s interesting what you’re saying that you may end up targeting analysts or PMs. I know Amplitude or has always focused on PMs, but-
Laura Schaffer: Yeah, it’s right. And there’s always nice thing about it is it’s in some ways, it does feel like you’re starting fresh because you do need to start with the customer again and what’s their problem. But in a lot of ways, you can carry over a lot of the same knowledge. At that point you know what’s working well. Amplitude for example, does have a ton of knowledge around what some of the best ways artists set up reports. There’s a lot of things that they have the momentum going, like where do you choose that momentum and how do you put that and curate that in front of users and make sure that they’re getting the right things. There’s a ton of momentum already there. It’s just a little bit about harnessing it and understanding like, yeah, where are the gaps because there are going to be gaps.
But anchoring in a customer problem as I think the way that you start any new product, any new thing that you’re releasing, should always think about the customer and the pain point. So no different than when you’re doing PLG for the first time or cracking into it. You need to be thinking again, starting again with the problem, the problems they have, the psyche that they have coming to your space so that you can build something that is going to effectively make them feel like, “Oh you can solve my problem, you get me.” And show them how your product’s going to do that.
Lenny: Final question, and this is around developers. You worked at Twilio, obviously Twilio sold developers. I think Rapid where you work right before Amplitude also sold developers. Selling to developers feels like such a hot space right now. There’s so many startups that are just building developer tools, such a huge market. Used to be not. Used to be like there’s not a market in developers. They’re not going to spend money, there’s not enough of them. And now it is a big popular spot. And so I’m curious, what have you learned about building a startup and a product that sells to developers? I imagine a lot of founders building search tools would be really curious.
Laura Schaffer: The first is that developers are just a very different audience from any others. I’ve seen so many people who have come in strong on growth really well or product really well with other audiences and like, “Oh, I’m going to take all those learnings, I’ll pivot into serving developers.” And as it being a very steep climb because developers are so different. And let me give you a couple just fun facts that make them really different. And some of these have some interesting stories. One is developers, almost two, one, do not look at your marketing website at all. They go straight to your signup flow. So what that means is all that beautiful context that you’re setting and the product aid pricing, all that stuff, very often they’re skipping all of it context free and going straight to your signup. And so anytime you make an assumption like, “Oh, well, they probably know this coming in to signup.” Or like, “Well, we don’t need to include, that’s on the marketing website.” None of that’s going to apply to this group of people.
They’re there. The analogy I have for this group is they’re the IKEA buyers who when IKEA package comes, they’re not opening up the instruction manual and reading in and then starting to go through, they’re in there tearing open the bags and starting to pull the pieces together and trying to build it. They’ll come up for context and steps and such when they get stuck if they’re motivated. So that’s one thing. And then another one is just the aversion to talking to sales. And I think hearing that, some they’re like, “Oh yeah, well, I hate sales too.” When I’m sent out and get bombarded by sales, that’s the worst. I totally get that. But developers are on this whole other level. There was a fang company sign up for Twilio, built a POC, launched to production, all this, and operated in that space for months without engaging once with sales.
I was trying to reach them and I ended up being the one that talked to them first because they reached out to support because there was something about their delivery that was off ,there missing a feature and they did not want to talk to sales. They ended up talking to me when I was in product marketing. And that was my first exposure of like, these people not want to talk to sales. And then there’s another one where a giant retail company where the engineering team signed up with their personal e-mail addresses so they wouldn’t get bombarded by sales. It was only later that we found out. Anyway. But the thing that’s most important, these are fun facts, but the thing that I would say is the most important, the thing to leave with listeners here is what makes them so different?
Why? What’s the deal here? And it stems from their charter and their responsibility. So if we put ourselves in developer’s shoes for a minute, a developer, if a developer is required to use your product, especially if they’re the primary user, the primary builder, it’s really important to recognize that they’re responsible for that. If your service goes down, that’s their responsibility. Not just for themselves but their team. If the pager wakes up someone because the service they bought from you goes down, that’s on them. If, oh, it turns out that doesn’t work with the systems that they said it was, well, that’s on them. Doesn’t integrate with the data the way that everyone wanted it to? That’s on them. Everyone lives to developer when it’s not working right and it cannot work right. In so many ways, that’s their failure, it can cost them their job, it could cost them the trust to their team.
It cost them their reputation. And that means that the stakes are very high for them every time that they’re adopting something new. So they can’t afford to take someone’s word for it. Especially a sales rep who might have some other motivations from their perspective, they can’t afford to trust your content or someone’s word. They must do it. They must prove it themselves. And so that’s why, for developers to be bought in, they need to do something, build something, a proof of concept at the very least, if not moving further than that. And so that means they’re going to be pretty darn deep in their self-serve experience with you before they’re ready to commit. And so if you are a company that is providing, that requires developers to build, you must invest in self-serve experiences in order to effectively convert your audience. And you should be thinking of them. Something akin your self-serve function and growth folks, someone akin to Salesforce because your developers are not going to accept sales coming in and trying to convert them at that stage.
Lenny: I love that you always come back to the psyche of the user and how in this case, developers like, here’s why they’re responsible for this thing. Salespeople are going to convince them this is going to work. And it’s not. That’s a really interesting tool and that’s a really cool takeaway.
Is there anything else that we didn’t cover before we get to our very exciting lightning round?
Laura Schaffer: Plenty. I think we covered it all, man.
Lenny: You got all my questions and more. So with that, welcome to the very exciting lightning round. I’ve got six questions for you. Are you ready?
Laura Schaffer: I am so ready.
Lenny: What are two or three books that you recommend most to other people?
Laura Schaffer: I’m a big believer in happiness. Not just being crunchy or we should all be happy, but also because it helps us do our best work and we’re more creative and all this. So one is Simple Path to Wealth by JL Collins. I don’t ignore the data, that money is something that often gets in the way of our happiness. I know so many smart people that just have not figured out the whole managing their finances thing. And this book will cover all of your basics. It’s very easy to read. He’s got an audiobook that he narrates himself. Simple Path to Wealth by JL Collins, he’s fantastic.
Lenny: What’s a recent movie… Oh wait, wait, there’s more?
Laura Schaffer: Oh, there’s one more.
Lenny: Oh, let’s do it. Let’s do it.
Laura Schaffer: [inaudible 01:12:18] Happiness, which is Atomic Habits by James Clear. If you ever want to change something about yourself or something’s not quite working for you, this guy will give you a framework to change it. Guaranteed.
Lenny: I really enjoyed that book. That guy’s killing it. He was on Tim Ferris, he had a great interview. Folks that don’t want to read it, they could listen to that. There’s a lot of cool tips there. Favorite recent movie or TV show?
Laura Schaffer: Unabashedly, the Great British Baking Show. I love that show. I love that show for all the reasons everyone loves that show. It’s heartwarming and makes you feel good and uplifts you. But also because it is a competitive show. They’re trying to be the best baker and they’re out there helping each other. They’re like a big family. Most reality competitive TV shows that I see, all of them are like cutthroat, they’re sabotaging. So I’m just endlessly fascinated also by the psychology of what’s happening here. I want somebody to do a research paper on it, get to the bottom of why they’re all helping each other. It’s wonderful though. Wonderful to watch.
Lenny: Interesting. I always comes back to psychology with you.
Laura Schaffer: I know. I know. I feel like I’m really, really sinking deep in there. And it’s true though. It’s very interesting to me, and I love that show.
Lenny: What’s a favorite interview question that you like to ask in interviews?
Laura Schaffer: I love asking about a ship or release that is not cherry-picked by the person you’re talking to. You can get it a lot of different ways. The thing is, everyone has a big success story. Everyone does. It really doesn’t actually tell you very much to ask someone like, “What’s a great thing you released?” Because everyone could tell that. Instead, take that away. What’s the most recent ship is a really easy one because recency, time. But there’s other things you can do to take that out. Just give them specific parameters for a ship that they’ve shared or whatever, and that will allow you to listen more and learn more about their frameworks versus the outcomes. Because if you’re picking a random ship, odds are it probably wasn’t fantastic. So they’re going to want to talk more about how they approach getting there and that’s what you want to know about to know if they’re going to succeed, what their frameworks to how they approach things.
Lenny: That is cool. I’ve never heard that one. That is a really clever idea. What are five SaaS products that you use in your day-to-day work? Can’t say Amplitude.
Laura Schaffer: I know, right? Still learning which ones we have here. But yeah, I’ll just share the ones that I like a lot that I’ve used elsewhere. So one is Hotjar. Hot Jar [inaudible 01:14:44] also works. Just anything that allows you to put some quick little thing in front of customers, get that qualitative feedback we talked about. It’s a critical, critical supplement to quantitative data to understand what’s really causing the change or not causing the change [inaudible 01:14:58]. So that’s important. I will say Amplitude is a fantastic tool that I have used and I would’ve said that if I hadn’t just joined Amplitude so I got to use it… I know. I got to use it for the first time at Amplitude and it was awesome. So again, like asterisk, because I’m like working there now, but I do actually like it. And Slack, it’s boring, everyone says Slack, but I just have to hand it to them.
It makes life so much easier and just nod their way. And then Builder, which I’ll also put in asterisk on that one, but I really want to serve this. A lot of people don’t know about it and it’s really helpful. I do advise them so I’m in their corner. But this is another one I also say would be a powerful one. I think a lot of team gets stuck. They’re relying on too much in their engineers to make changes. Again, we talk about rapid experimentation, getting these out, out, out. And Builder makes it really easy for folks to do that. Also, a headless CMS, you can drag and drop headless CMS so they do make it easy for non-engineers to make changes. So especially if you’re trying to figure out how to get around that 80% [inaudible 01:16:11] that I mentioned, this builder would be a good way.
And then yeah, if you want one more, I’ll give you Chat GPT, which is really boring and everyone’s saying that, but I think I’ll just say I don’t have any crazy things to say about it except that I do think we all need to figure out how we pull that in to [inaudible 01:16:28] put people who don’t do that are probably going to lose out or smart AI whatever bots. But that would be it for you, Lenny. But if you ask me in a few months after I’ve actually been an Amplitude for a bit, I’m sure I’d give you a different answer.
Lenny: That’s a good time to plug lennybot.com. And I wrote a newsletter or Dan Shipper who created the bot wrote a newsletter post about how he built this thing. And so you could go ask me questions using the content of my newsletter as answers. And it’s very cool. Lennybot.com or lennysbot.com.
Laura Schaffer: Amazing.
Lenny: There we go.
Laura Schaffer: I didn’t know about that. Well, there you go. I’ve changed my answer. It’s that.
Lenny: Yeah, there we go. That’s all I need.
Two more questions. What is something relatively minor you’ve changed in your product development process that has had a lot of impact on your team’s ability to execute?
Laura Schaffer: Yeah, the be embarrassed thing, like I mentioned earlier, be Embarrassed by the first iteration. If you are not embarrassed, you’ve gone too far. That really speeds up ships and helps people celebrate the unpolished as opposed to feel embarrassed about it. So just embracing that.
Lenny: Awesome. And final question. I know you just started Amplitude, but do you have a favorite pro-tip for how to use Amplitude or maybe Hidden Feature people may not know about?
Laura Schaffer: You tell me [inaudible 01:17:45], but I’ll say one thing that was super cool actually that someone put together on my team at Rapid, literally before I left, he put together a video of how powerful Amplitude could be when linked up and integrated with other things like in this case Hotjar and Segment. There was a Amplitude report that someone had created and there was something that was an anomaly happening there. Users were using something in a way we didn’t expect and Amplitude one of the reports surface it, but Al Kirsten, we want to know why is that happening. And so we could find out what the event is and then using by Segment, find out what that name was and look at Hotjar and actually go in and get screencasts of people doing that exact event.
And from that, we were able to form some really concrete hypotheses about what actually was causing it. And so obviously talking to customers is very powerful, but in this case, just that simple use of connecting and threading those technologies together could really get a good picture of that without needing to engage customers. So the tip would be how you can really amplify when you get an amplitude when you use it with.
Lenny: Amazing. Laura, we covered a lot of ground, career experimentation, growth, embarrassment, psychology. Thank you so much for being here. Two final questions, where can folks finding online if they want to reach out, learn more, maybe send you an appreciation or two? And two, how can listeners be useful to you?
Laura Schaffer: Yeah, find me on LinkedIn. I don’t post a lot. Yeah, I’ll blame my three children. Eventually, I promise that I will. But I’m pretty good about responding to messages, so definitely link with me there.
And then what listeners can do, I think I’m always happy to hear feedback, suggestion, all that, but I’ll just say I also know that it’s a little bit crazy out there right now, especially folks working in tech. So I’m also cognizant what I might be able to do to help all of you. I know there’s a few places I advise and rapids hiring. I know of a few folks that are hiring growth, strong growth people and product folks. So if you are interested in learning more about that, don’t hesitate to hit me up. I want to make sure that I help as many people as I can in that respect because it’s trying times and I’m sure you’ve heard it and read it, but if you’re laid off, this is not about you, it’s not your fault. It’s this crazy world we’re in. Things will get better. And I would be feel very lucky if I could help even one person land. So feel free to hit me up about that too.
Lenny: Awesome. And maybe if you share some links, we could include links to open roles in the show notes.
Laura Schaffer: Yes. I know there’s a few that don’t have JDs open yet. They’re that hot off the press, but I’m happy to surface a few things there for sure because I know that makes it easier for people to know.
Lenny: Awesome. We will do our best with the show notes then. Laura, thank you again for being here.
Laura Schaffer: Yeah, thanks so much for having me. This was awesome and so much fun.
Lenny: Bye everyone.
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcast, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.
Glossary
| English | 中文 |
|---|---|
| A/B test | A/B 测试 |
| aha moment | 顿悟时刻 |
| Amplitude | Amplitude(公司名,保留原文) |
| Bandwidth | Bandwidth(公司名,保留原文) |
| be embarrassed | ”感到尴尬”理念(第一版不完善到让自己不好意思,即快速迭代中接受粗糙初版的心态) |
| benchmarking | 基准对标(benchmarking) |
| Builder | Builder(产品名,保留原文) |
| calibration | 校准会议(绩效评估中多人对齐评审的环节) |
| churn | 流失率(churn) |
| Code Exchange | Code Exchange(Twilio 平台名,保留原文) |
| confidence interval | 置信区间 |
| conversion | 转化率 |
| Dan Shipper | Dan Shipper(人名,保留原文) |
| dashboard | 仪表盘(dashboard) |
| developer relations | 开发者关系 |
| e-commerce manager | 电商经理 |
| FAANG | FAANG(指 Facebook/Meta、Amazon、Apple、Netflix、Google 五大科技公司,保留原文) |
| false negative | 假阴性 |
| false positive | 假阳性 |
| false success | 假成功(指实验结果看起来成功但实际并非如此) |
| friction | 摩擦(用户体验中的阻力或额外步骤) |
| GM (General Manager) | 总经理 |
| Great British Baking Show | The Great British Baking Show(节目名,保留原文) |
| hard data | 硬数据(客观的定量数据) |
| headless CMS | 无头CMS(headless CMS) |
| Hotjar | Hotjar(产品名,保留原文) |
| low code | 低代码 |
| mocks | 模型(设计稿/原型) |
| North Star metrics | 北极星指标 |
| onboarding | 新手引导 |
| P-value | P 值 |
| painted doors | 画门(在功能开发前用假入口测试用户兴趣的验证方法) |
| performance reviews | 绩效评估 |
| pipeline | 销售管道 |
| PLG (Product-Led Growth) | 产品驱动增长(PLG) |
| PM (Product Manager) | 产品经理 |
| POC (Proof of Concept) | 概念验证(POC) |
| product marketing | 产品营销 |
| QBR (Quarterly Business Review) | 季度业务回顾(QBR) |
| Quick Deploy | Quick Deploy(Twilio 产品功能名,保留原文) |
| Rapid | Rapid(公司名,保留原文) |
| Reforge | Reforge(公司/平台名,保留原文) |
| retention | 留存 |
| Segment | Segment(产品名,保留原文) |
| self-serve flow | 自助流程 |
| ship | 上线/发布 |
| signup flow | 注册流程 |
| SLG (Sales-Led Growth) | 销售驱动增长(SLG) |
| SMB (Small and Medium Business) | 中小企业(SMB) |
| SME (Subject Matter Expert) | 领域专家 |
| Tim Ferriss | Tim Ferriss(人名,保留原文) |
| Twilio | Twilio(公司名,保留原文) |
| ungating your knowledge | 解锁你的知识(将个人知识开放共享) |
| vanity Metric | 虚荣指标 |
| voice of the customer | 客户之声 |
Reformatted by reformat_english.py
职业发展框架、A/B 测试、入职建议、向工程师销售 | Laura Schaffer(Amplitude)
翻译
Laura Schaffer:
……比如深夜。我说的深夜,其实就是晚上七点左右。我相当确定那天是周五。我们先做了再说,把这些问题塞进了静默流程里,对一个小组跑了一次 A/B 测试。我当时完全做好了心理准备——“好吧,这肯定会拖累我们的数据,但也许不会太糟,而且我要准备好为这套数据的价值据理力争。“我也已经摩拳擦掌,开始构思文档,着手写框架来规划怎样呈现这些数据。然后我们开始收到数据了。不开玩笑,转化率提升了——没有任何个性化,没有任何后续动作,仅仅是加了那些问题——注册转化率就提升了大约 5%。那一刻的感觉就是,“什么?好吧,这是怎么回事?”
**Lenny:**欢迎收听 Lenny’s Podcast,在这里我采访世界级的产品负责人和增长专家,从他们打造和增长当今最成功产品的宝贵经验中学习。今天的嘉宾是 Laura Schaffer。我们录制这次对话的那一周,恰好是 Laura 在新岗位的第一周——Amplitude 增长负责人,接替了之前做客本节目的传奇嘉宾 Elena Verna。在加入 Amplitude 之前,Laura 是一家名叫 Rapid 的公司的产品和增长副总裁。再之前,她在 Twilio 工作了七年多,担任增长负责人以及增长平台和实验平台的产品经理负责人。在这次对话中,我们深入探讨了 Laura 的职业发展框架,以及主动开辟自己的道路与等待别人为你开辟道路之间的区别。我们还聊到了很多关于跑实验的实用且出人意料的建议,凭直觉还是凭数据做决策,制定增长策略,以及如何向开发者推销你的产品。Laura 拥有丰富的智慧,我在这次对话中学到了很多。好了,简短感谢我们优秀的赞助商之后,为您带来 Laura Schaffer。
**Lenny:**Laura,欢迎来到播客。
**Laura Schaffer:**谢谢,Lenny,非常高兴能来这里。谢谢你邀请我。
**Lenny:**很高兴你能来。我之前问过 Elena——Elena,Elena,我甚至不太确定她名字怎么发音,也许——怎么念来着?
**Laura Schaffer:**Elena。你说对了。
**Lenny:**Elena。好吧。我想我一直都念错了。一直。好吧,Elena。所以我问了 Elena Verna——她是这个播客很受欢迎的往期嘉宾——我应该请谁来做客,而你第一个出现在她的脑海中。所以我非常兴奋我们能做这期节目,也很高兴你答应了。
**Laura Schaffer:**她是最棒的,我真的很开心她推荐了我,因为我特别激动能来。所以感谢你听了她的建议。
新角色、新人排名
**Lenny:**当然。而且现在聊天的时机也很巧。你是 Amplitude 新任增长负责人,所以首先恭喜你。
**Laura Schaffer:**谢谢,非常感谢。是的,我今天来这里才第二天半。所以一切都很新鲜。
**Lenny:**哇,老手了。
**Laura Schaffer:**是啊,没错。
**Lenny:**我喜欢这个。有些公司会显示一个小小的百分比,告诉你有多少人在你之前加入了公司,我很好奇在 Amplitude 那个百分比现在是多少。
**Laura Schaffer:**我们在 Twilio 就有这个,过了一段时间我排到了挺靠前的位置。我们有一个电子表格排行榜。不过确实挺有趣的。所以在 Amplitude,不管这个东西存在哪里,我都是最底部的新人。
**Lenny:**那你在 Twilio 排到多少号?还记得吗?
**Laura Schaffer:**记得,我非常自豪地冲进了前 50。那是我引以为傲的事,因为随着人员离开,你会往上移嘛。
**Lenny:**对。嗯,苦乐参半。
**Laura Schaffer:**是啊,没错。一方面觉得”哦,很酷”,成了 OG 之一;另一方面又想”天哪,这个人走了。真遗憾。“感觉是复杂的,但我对未来确实很兴奋。
职业发展框架
**Lenny:**你现在有了这个令人兴奋的新角色,我觉得可以从职业发展聊起——你怎么看待职业发展。我知道你有一个关于自身职业发展的框架,而且显然效果很好,所以我很好奇听你讲讲,看看对听众有什么帮助。那么,能跟我们聊聊你怎么思考职业发展吗?
**Laura Schaffer:**职业发展绝对不是一条直线上升的路,但确实有一些框架和方法对我来说非常有效。要深入展开的话,首先值得聊聊大多数人试图用来发展职业的最常见方式,以及为什么它可能有些问题——那就是,我看到大多数人都在自己现有的岗位上、在现有公司的角色里拼命努力工作,尽一切可能在那里成长,向经理展示各种成果。我见过有人维护电子表格来记录成绩,好在绩效评估时拿出来。也许你努力更好地为自己代言,也许你试图让同事或者经理的同事注意到你。这些都很好,都是应该做的事。
但这样做的问题在于,你能获得的提升,受限于你的经理为你争取和推介的能力。同时,你也被限定在所在公司该角色明确的发展路径上,以及公司内部是否存在相应的空间。而且,别人对你的认知有时可能与你的自我认知存在偏差。再加上其他变数,比如你的经理离职了,你就得跟新来的人从头开始。所以,我用的方法试图把一部分主动权夺回来。在我职业生涯早期,我非常幸运地偶然学到了这一点,那是在一家叫 Bandwidth 的公司,那是我第一份真正意义上的工作。Bandwidth 现在是一家上市公司,做了很多了不起的事情。
但我加入的时候公司只有50个人,而且我其实是做销售入职的。我满腔热情,渴望把事情做成、渴望成长,典型的初入职场的新人模样。但做了几个月销售后,我发现自己在不断重复同样的事情,用同一套话术一遍又一遍地推销。我就觉得,这对客户不太理想,因为他们每次打电话来问同样的问题、得到同样的回答。对公司来说也不理想,因为每次都要付佣金。这对我们的增长不会高效。而且因为公司小,我有机会直接找到我们的总经理,就跟他说:“我发现一个规律,我在反复重复同样的内容,客户也在问同样的东西。我觉得我们应该把这些内容放到网上,让客户自己就能看到然后直接购买。“因为我们当时已经有在线下单流程了。我本来以为他会说:“嗯,我知道这很重要,但鉴于某某原因我们得这样做,你肯定也考虑过了吧。“我心里想着,我一个新人来提建议,他应该会帮我指出我忽略了什么。但他却说的是:“等等,跟我说说这个,你是什么意思?“到谈话结束时,他说:“那你去做这件事吧,你去搭建那个体验,把这些东西做成自助流程。”
我们把这个叫做电商经理——这在”增长”这个概念流行之前就有了,大概是在2010年。这件事把我推到了一个全新的岗位上。我从中学到的最重要的东西——后来在 Twilio 得到了充分发挥,对我帮助极大,我也很乐意聊聊那段经历——核心教训是:你的高管团队,以及公司里的各级高管,往往非常敏锐,但他们日常工作的性质决定了他们与客户之间是脱节的。
这意味着,随着时间推移,尤其是公司规模扩大之后,他们往往无法接触到一些最前沿的洞察,无法感知他们所服务的人的心声——相比之下,那些离问题更近的人则不然。所以你的超能力就在于:把这些洞察提炼出来,让它们变为现实,始终贴近客户。没有一个领导者或高管会不愿意听到有价值的客户洞察,尤其是那些揭示他们可能忽略的问题的洞察。这类问题其实很多。当这些洞察与 North Star metrics(北极星指标)对齐时,它们就格外有力量。这就是我在 Twilio 走出来的职业成长路径,我也很愿意分享那段历程。
用客户洞察建立个人品牌
**Lenny:**我其实很想再听一个具体的例子。不过我觉得这里有一个很有意思的点——有时候你可能有一个很棒的想法,但它不会立刻被采纳。不会马上有人说”好的,我们来做吧”。我觉得需要认识到的是,他们不会采纳你的每一个想法,但他们始终在寻找更好的想法。而且正如你所说,他们可能恰恰缺乏那些能催生好想法的信息,因为你身处一线,每天处理的是真实的问题。所以要理解,你不会总是如愿以偿,这很正常。
**Laura Schaffer:**没错。这其实有点像在建立你个人的品牌。我认为最强大、也最容易上手的方式之一就是深入了解你的客户。每家公司都有这样的人,大家会说:“哦,她就是特别了解我们的客户”或者”他就是特别了解客户,他们就是懂。“然后别人问”怎么懂的?""就是懂。让我们去问问那个人,听听他的意见。“这些人往往在公司内部有很强的品牌认知度和影响力,当公司需要做新的事情或不同的事情时,他们往往会被想到;当有人在招聘时,也许也会考虑这个人去做跨团队的项目。
所以这是建立个人品牌的一种方式。而且我觉得它恰到好处,因为它对所有人都非常有价值,一直向上到最高层领导——这个我们待会儿可以聊。所以无论你处在职业生涯的哪个阶段,它都是一件有价值的工具。它不总是立竿见影的回报,但它确实能给你带来超越当前角色和当前经理的发展轨迹,给你一个更宽广的天地。
主动开辟自己的路径
**Lenny:**所以,简单概括你说的——也许可以说成是”开辟自己的道路”。不要理所当然地认为经理会给你规划出最适合你的路径,或者给你最大的机会。主动提出:“我觉得这可能是一个更好的机会,我想去试试。“我很想听听你在 Twilio 的经历——
**Laura Schaffer:**好。我加入 Twilio 的时候,根本没有增长团队,连一点影子都没有。我加入的是产品营销部门,负责我们消息产品线的产品营销,但我延续了我刚才说的那套方法。我给自己定了一个个人准则:首先,做好本职工作,而且要做得漂亮,记录下自己做得好的地方,这些都很有用。但同时,我还要去深入了解客户,真正地了解他们。在与客户交流时,我不只关注自己负责的领域,而是广泛地关注他们表达的各种痛点,以及那些与我们业务和目标相关的需求。
其中一个发现是,用户在使用 Twilio 入门和上手时遇到了困难。这与我们高管团队所说的、公司高度笃信的东西形成了鲜明对比——公司坚信 Twilio 非常易用。事实上,“易用”是 Twilio 想要传达的三大品牌信息之一。开发者爱我们,他们说我们太好用了。推特上不断有人发帖说:“天哪,我几分钟就上手了。“所有这些信息叠加在一起,让那种信念根深蒂固。但在我和客户交流时,我听到了一个截然不同的故事。其实这也说得通——随着我们渗透新市场、增加更多产品,我们也在增加复杂度,吸引的用户群体不再那么有动力,这些因素叠加在一起,就导致了人们觉得它变难了。
建立客户洞察的影响力
**Laura Schaffer:**当时 Twilio 还不是一家只有五十人的小公司,我没办法直接走到谁的工位前说:“嘿,我听到了一个情况,我觉得我们应该做点什么。“但我可以采取另一个策略——我开始分享客户之声报告。我开始把自己的洞察写下来,分享出去。慢慢地它变成了一份定期推送的摘要,最终有人主动来问:“嘿,能不能也分享给我?能不能也发我一份?能不能把我加到你的分发列表上?“这些事是我在入职后几个月内就开始做的。后来这演变成了:“你应该为整个产品团队主持一个季度性的客户之声分享会。“这个需求是公司的一些高级领导提出来的。
当时的 CEO Jeff Lawson 听说了这件事之后,也开始参加这个会议。所以在那个会上,我也开始把其他人的洞察整合进来,因为现在有了这样一个论坛,大家可以把自己发现的客户洞察发给我,我来汇总整理。这一切让我在短期内就确立了自己作为”最了解客户的那个人”的形象。到了那年做年度规划的时候——我是2014年底加入的,所以这是2015年——我提出了一个提案:“我们认为产品很简单,但实际上并不是。这是我的数据和信息,我认为我们需要在这里组建一个增长团队,这应该成为一个核心重点。“因为我已经建立起了一定的信任基础,所以我能够拉来一个非常关键的合作伙伴以及其他支持者。
到我正式做提案的时候,我已经有了 Andre Crow 的支持——他是 Twilio 的第七号员工,在公司里排名第三,跟 CEO 关系非常近——他也说:“是的,我们太需要这个了。“我看到了同样的问题。他负责网站,基本上打造了 Twilio 的品牌,主导了所有网站相关的工作,他说:“确实,我们一定要做这个。“所以我不但从高管团队那里积累了一些信任,还有那些本身就深受信赖的人在为我倡导和支持。因此提案几乎非常轻松地就通过了。我当然也准备了相关材料,但真正起决定作用的是”会前会”——那些前期的铺垫工作早就通过其他方式完成了。这件事帮助我在 Twilio 创建了增长工程和增长产品团队。
主动创造机会
**Lenny:**我很喜欢你给出的建议中那种主动性。有很多人发展不好,然后就抱怨说:“从来没有机会”或者”一直被忽视”。而你的建议非常务实——这些都是你可以去做的事情,去让自己被更多人看到,去提供价值,去为自己创造机会。沿着这个方向,你还有没有其他建议?就是那种”你可以为自己做的事”,而不是等待别人来给你机会?
**Laura Schaffer:**我觉得这是最容易落地的方法。因为我们所有人要做好自己的工作,都需要了解客户,需要了解客户洞察,需要了解产品。而对于那些面向客户的团队、社交媒体运营的人来说——如果你想转入产品岗,你的洞察是极其有价值的。你每天都在跟客户交谈,对他们的痛点和问题了解更多,比其他很多人都要深入。所以在我来看,这是最强大、也是最容易被任何角色、任何领域的人所使用的方法。但我也要说,这个更广义的理念就是——你掌握的知识中有一些是其他人可以受益的,你要把自己建立成这样一个形象:乐于助人、聪明、有创造力、能解决问题。确保你把这些分享出去。比如你特别擅长简洁有力地沟通。
顺便说一句,我不擅长这个。所以特别佩服能做到的人,我自己也在努力改进。所以去分享吧。去你的公共 Slack 频道或者什么地方发一条:“嘿,我刚写了一些关于如何简洁沟通的小技巧,分享一些我自己做得还不错的方法。“这类事情真的能产生长远的影响——让人们开始把你视为某个领域的专家(SME),不只是你本职工作的领域,而是更广泛的方向。这总能为你打开新的门路,其他人也会仰望你,认为你不仅仅在当前岗位上表现出色,在其他方面同样很强。
**Lenny:**SME 是 Subject Matter Expert,对吗?
**Laura Schaffer:**是的。谢谢你帮我解释缩写。这也是我正在努力改进的地方——少用缩写。
**Lenny:**明白了,我会帮你留意的。
**Laura Schaffer:**好的。
**Lenny:**也许在这个方向上最后一个问题——关于如何向你的经理、上级 framing 一个提案、一个机会,你有没有什么广泛适用的建议?怎样包装你的提案效果最好?
如何向管理层提案
**Laura Schaffer:**有的。我还想补充一点——做这些事情并不一定与你的经理在做的事情相冲突,更多是在支持他们。我做了这些事情之后,反而帮了我的经理更好地提拔我。所以不是说”只有当你的经理不称职时才这么做”,或者”他们不支持你、帮不了你时才这么做”。而是说,去做这些事,因为这会加速你自己的发展,不管你的经理怎么样。
同时,这也会帮助你的经理更好地支持你。因为经理在推动晋升和做这些决定时,经常要坐在一个房间里进行校准会议,面对一群人。当那些人对你在积极的方面有过接触、有所了解时,事情就容易多了。所以这些事情可以与你的经理并行推进,而不是对着干。这只是另一种方式,让你把构建势能的主动权拿回到自己手里,而不是完全依赖于一个单一的人。
**Lenny:**我喜欢你第二个例子的地方在于——你直接就去做了。你就开始做那份客户洞察报告,而不是先说:“嘿,我有个提案,我觉得应该这样做,我们要不要做?”
**Laura Schaffer:**没错。
**Lenny:**就是直接动手。
**Laura Schaffer:**是的。我想现在流行的一个说法是”ungate your knowledge”——把你的知识解锁出来。
**Lenny:**嗯,没听过这个说法。
**Laura Schaffer:**我觉得这是 Elena 说的——看看我们能提她多少次。但你在自己公司内部完全可以这样做。每个人都在某些方面有技能,而这些技能并不属于他们正式岗位的范畴。我认为把这些知识释放出来就能打开机会。如果你不确定该分享什么,那就用我最常用的方法——跟客户交谈,获取洞察。这些是非常有价值的。人们在获得客户洞察后很少会分享出来。所以,做那个愿意分享的人。
实验与增长
**Lenny:**接下来我想聊的另一个话题是实验、增长和数据。这也很合理,因为你现在是 Amplitude 的增长负责人,对这个领域有很强的观点。也许我们可以从实验开始。你之前提到过,在 Twilio 有一次实验得出了一个非常有趣、令人意外的结果,可能改变了你对实验的看法,以及你对什么会有效、什么不会有效的判断。
**Laura Schaffer:**完全同意。我确实经历了两次颠覆认知的实验,真正改变了我对增长的思考方式。其中一次,也是我最喜欢的实验之一,发生在 Twilio 很早期的时候。在我创建增长团队之后,我注意到注册流程存在一个问题——我们只要求用户填写用户名、邮箱和密码,仅此而已。这在当时其实相当普遍,毕竟那是很久以前的事了,大家都在这么做。但当时团队有一种根深蒂固的信念:我们在面向开发者。开发者只想直接上手,别在他们面前放任何障碍,否则后果会很严重。我们不想跟这帮人搞什么花招,直接放他们进来就好。
但在我看来,这是一个非常大胆且代价高昂的假设。如果真是这样的话,我们对用户将一无所知。我们不知道谁在注册,不知道他们想做什么。这严重影响了我们从定量角度理解用户行为的能力,在优先级排序上也有些迷失。这带来了一系列连锁反应,但这显然是一个非常有争议的领域。所以这就是我做的第一件事,也是我运行的第一个实验。我做了一些研究来确定:最重要的提问是什么?我们究竟需要了解什么?比如你用什么编程语言、你的使用场景是什么、你想用哪个产品?
还有一个问题是:你到底是不是开发者,还是别的什么角色?因为有传言说,注册的不只有开发者,这又是另一个有趣的故事了。而且我觉得,这些问题对于注册的开发者来说也是自然的,他们会理解我们为什么这样问。话说回来,在注册流程中添加任何内容都非常有争议,但我确实想拿一些数据来验证。我想跑一个测试。当时我没有团队,没有工程团队,什么基础设施都没搭建好,就我一个人。但正如我所说,我已经开始建立了一些信任,而且我之前提到的 Andre——他是早期员工,拥有所有系统的权限,也是那种什么都能做的人——他也支持这件事,有类似的直觉。
于是就在深夜——其实也就是晚上七点左右,我很确定是个周五——我们先做了再说,直接把这些加进了注册流程,对小范围用户跑了一个 A/B 测试。我满心以为这会损害我们的数据,但希望不会太严重,然后我准备好去论证我们获取的这些数据的价值。我甚至已经开始构思要如何呈现这些数据的框架了。结果数据开始出来,我没开玩笑——转化率提升了。没有任何个性化,问题之后也没有任何后续操作,仅仅是这些问题,就把注册转化率提升了 5%。我当时就觉得:“什么?这到底是怎么回事?“于是我深入调查,找了一些走完流程的客户聊了聊,慢慢理解了其中的原因。
原来,对用户来说,这些问题实际上是一种安慰。想想看,当用户第一次注册你的产品时,一切都是新的,而新就意味着令人不安。他们预期会很困难,预期会遇到摩擦和障碍,预期自己搞不定。就像在找潜在的麻烦一样——这就是用户的心理状态。我们每个人第一次做某件事的时候,往往都是这种心态:哎,这可能很有挑战性。所以当这些问题出现时——你用什么编程语言?“哦,我写 JavaScript,我可以选择它。“这是让我感到舒适的东西,会让我的使用过程更顺畅。“没错,这就是我的使用场景,好吧,我来对地方了。”
这些实际上给了用户一种安心感,挑战了”这会很难”的预期,仅仅是因为这些问题与他们内心自然在思考的东西对齐了——比如”他们会不会不支持我的语言?“或者”我想要的这个使用场景到底能不能实现?“对我来说,这次实验真正有趣的收获是——用户的心理至关重要。它的重要性不亚于理解你的产品、你所面向的更广阔的市场以及其他一切因素。理解新用户第一次在你的流程中做事时的心理状态,这种理解是强大的。我用一句简单的话来概括这次学习的核心:坏的摩擦是坏的,好的摩擦是好的。事情没那么简单——并不是所有摩擦都是坏的,而这正是我之前做这个实验时的假设。
绕过流程的”夜间部署”
**Lenny:**我太喜欢了——你刚到 Twilio,就直接把一个实验推上了生产环境。
**Laura Schaffer:**直接推上去的,没错。
**Lenny:**这招很大胆。
**Laura Schaffer:**最终这对所有人都很有帮助。我分享了从中获得的洞察和各种发现。当然,这种做法还是要谨慎使用,绝对要谨慎。
**Lenny:**而且转化率还提升了。
**Laura Schaffer:**是的,当然,务必谨慎使用这种流程。不过我也要为工程师们说句话——正确的方式来修改生产环境,还是应该通过工程团队的审批或者与他们协作完成。但回头看,这确实是正确的决定,也确实帮助了业务。
**Lenny:**是的,我很喜欢。这太棒了。我觉得我们可能需要更多这样的行动。我想深入了解一下你具体做了什么——你加了多少个问题,都是什么问题?
具体添加了哪些问题
**Laura Schaffer:**一个问题是你用什么编程语言。然后作为一个选项,可以选择”我其实不是在写代码,我不是开发者”。所以对我们来说,这实际上提供了两个非常有价值的数据点:一是我们的注册流程中有多少开发者 vs 非开发者;二是他们用什么编程语言。这不仅对增长和新手引导非常有帮助,对文档团队、开发者关系团队也是如此。后来这成了我们判断趋势变化的关键方式,能够在年底报告出来之前就捕捉到动向——人们在使用什么,你开始能看到趋势。还有一个问题是:你对哪个产品感兴趣?这对新手引导的基础安排至关重要——你是做短信、语音、还是 Twilio 的其他什么产品。最后是使用场景——你是做预约提醒,还是自动回复器,还是约会应用的匿名通信之类的。这些就是最初添加的几个问题。
**Lenny:**哇,好的。所以是四个下拉菜单式的问题,然后转化率提升了。我太喜欢这种摩擦增加却提升转化率的例子了——这样的案例真的很少。你会听到有人说这可行,但真正见到的并不多。那么你从中学到了什么?你总结出了什么模式?你提到了”好的摩擦”这个概念,但有没有什么迹象让你判断”这会是好的摩擦”?
“热狗藏药丸”:理解用户心理
**Laura Schaffer:**这些问题实际上解决了一个痛点。它们缓解了用户进来时的那种担忧——担心会很困难,担心自己搞不明白,担心找不到方向。而且我觉得这不只是 Twilio 独有的现象。我认为用户在任何公司的大门前、任何注册流程的起点,都会有这种体验——“好嘞,系好安全带。“尤其是在工作场景中,你身上可能还有额外的压力,要么必须成功,要么需要做出准确评估。所以那种心理——“我走对地方了吗?这东西能做我需要它做的事吗?我能搞明白吗?我有这个能力吗?“——这些都是人们在注册时极其常见的感受。
因此,这完全可以推广到任何地方。我绝对鼓励所有人都在早期新手引导中加入这类体验,不仅仅是为了你自己——让你能学习并做好用户分层——同时也让用户在开始使用时更有信心:“嘿,我走对地方了,这东西能做我需要它做的事。“但我认为这里的核心启示就是用户心理——要深刻意识到事情并非那么千篇一律,不是只有”我的目标市场遇到了什么问题,我的产品能怎么帮他们”这么简单。房间里还有另一个大象——它对用户能否成功使用你的产品和自助体验至关重要——那就是:在用户旅程的各个阶段,这个人的心态和心理状态是什么?
如果你没有把这个因素纳入考量、没有去应对它,你一定会遗漏东西,或者东西会失败,而你会对原因百思不得其解。我们有一个很好的实验,我很乐意分享一下,同样的理念,但完全不同的场景——发生在新手引导的后期阶段。我们一直以来为了降低 Twilio 的复杂度,尝试做的事情之一就是提供步骤式引导——欢迎来到新手引导,第一步,这是你要构建的东西。好的,现在我们知道了——“好,第一步,去做这件事。第二步,做这个。第三步,这个。第四步,这个。第五步,砰,你上线了,恭喜!“于是我们把它做出来上线了,我当时觉得,嗯,转化率是有提升。但并没有那么显著。就很让人沮丧。我们之前的状态是完全没有引导——“自选冒险,自己摸索去吧,祝你好运。“然后变成了这种规定好步骤的流程。结果转化并没有明显提升。
第一步的”恐怖因素”
**Laura Schaffer:**于是我们去跟一些用户聊,但没有发现什么特别明显的问题所在。他们会说,“嗯,好的,到第一步了。“我们模拟了用户操作,“好的,现在我知道了,我做第二步。“但有一个细节我一直在听到,感觉是个关键点——那就是电话号码,电信那部分。开发者来到 Twilio 时,接触到的是他们熟悉的东西:API、他们使用的编程语言、代码示例、文档。而那个”怪兽”——在心理上会绊倒他们的东西——是电信、电话号码。这些东西完全不在他们过去的工作范围内,尤其是在 Twilio 发展早期更是如此。
甚至现在也一样,对大多数开发者来说,电信是一个非常不同的领域。而第一步是什么呢?获取一个电话号码。因为那就是第一步。任何人教你使用 Twilio 入门的时候,都会坐下来跟你说:“好,我们去拿一个电话号码并配置它。“每个人每次都会这么做。然而,在自助流程中,你身边没有那个让你安心的人说着”别担心,没问题的,我带你走完这段疯狂的电信之旅”。他们只能靠自己,而他们的心理在告诉他们:“天哪,电信。这我做不了。听起来就吓人。还要获取并配置电话号码?哇,我完全超出能力范围了。”
那么我们怎么验证这个假设的呢?怎么测试这是不是问题所在?实际上,首先我们做了一个 MVP。我们把用户完全带出了控制台,放到了一个文档页面,在那里我们可以制造一种体验——他们首先看到的是代码,置身于文档这个安全空间里,使用着他们熟悉的编程语言,然后把电话号码那部分悄悄嵌进去。
就像是,“哦,获取一个电话号码,来配置一下吧。“不是作为第一步,不是作为引导的主打内容,而是嵌入其中。我对此有一个类比:热狗藏药丸。如果有人养过狗或任何需要喂药的动物,你知道不可能直接把药丸喂给它,那永远不会成功。但如果你把药丸塞进一截热狗里——热狗看起来不错,很诱人——那你就能让它更容易地吃下去。所以这个做法就是——
**Lenny:**对。我们用花生酱,一样的道理。
**Laura Schaffer:**没错,对吧?热狗、花生酱,都可以。你把它埋起来。你把那个吓人的、令人不快的东西嵌入其中。所以这就是我们处理电话号码、电信那部分的方式。你猜怎么着?尽管我们把他们踢出了控制台相当一部分,他们跳转出去了,而且我们当时还没有方便的返回按钮——转化率反而更好了。因为我们解决的是当时存在的核心问题——他们的心理。他们还没有准备好一进来就被直接扔进一个电话号码体验中。那等于把怪兽放出来开派对,这样做行不通。我们需要把这个怪兽药丸藏进热狗里。
一旦验证通过,我们就可以真正着手把它正确地整合到新手引导流程中,那样效果会更好。所以说,你用户的心理真的是一个极其关键的考量因素。如果某个在逻辑上很合理的东西转化效果不好,有时候意味着你正在对抗用户的心理。这时候你要退后一步,去思考、去了解在你的领域中,用户在心理层面处于什么状态。
全面重设计 vs. 迭代优化
**Lenny:**感觉你们做了一个完整的新手引导流程重设计,没成功。然后第二次尝试是另一种方式的全新引导流程。我很好奇,你对做实验有一个什么看法?这个问题我们在 Airbnb 和其他地方也经常遇到——你是直接把整个东西重新设计一遍,还是从现有的状态出发,逐步迭代、一块一块地实验,朝着一个更好的终极体验前进?
**Laura Schaffer:**对此我想说的是,从宏观角度来看,迭代永远是更好的选择。迭代更好的原因在于,大约 80% 的情况下——或者说很多时候——我们的假设以及我们相信会成立的东西,其实是错的。这一点非常值得注意。有一篇非常好的文章,我很乐意分享给你,你可以放到节目备注里。
**Lenny:**当然。
**Laura Schaffer:**那篇文章真正用科学的方法论证了这一点。像 Netflix、Microsoft 这样的公司,数据反复显示 80% 以上的实验是失败的。有些公司甚至说 90% 的实验会失败。所以,你越是埋头苦干、躲进阁楼里花六个月构建一个东西再发布,你就越有可能发布的是那 80% 错误的东西。
让失败成为指南针
**Laura Schaffer:**而你越是迭代,就越有可能更早发现问题。而且失败不一定要是一堵墙,它可以是指南针,可以是指引你找到正确方向的东西。所以,你总是要尽可能地让东西呈现在客户面前,这样你就能激活这个指南针,知道该往哪里走。
这意味着你要做一些难看的东西。我经常跟团队说,如果你不觉得尴尬,你就做得太多了。必须得尴尬才行。还记得我们之前那个很尴尬的第一次尝试吗——把人赶走那个。我们花了那么多钱、费了那么大劲把用户引导到核心流程里,结果第一件事就是让人家滚蛋。这太荒唐了。但如果那次验证失败了,那就会是一个非常廉价但非常有价值的学习。而实际上,我们在相反的方向上获得了一个非常有力、非常廉价的学习。好了,现在我们知道可以往里投入了,我们知道这是正确的方向。所以,迭代永远是更好的选择,让失败为你所用,而不是让它变成你掉进去的陷阱。
**Lenny:**我知道你刚说过,做实验大概有 80% 的概率是错的。根据我的经验,做一个全面改版的成功率基本是负的百分之百。我已经变得非常谨慎,尽可能避免这样做。虽然你在做增长和产品的时候都会被教导这一点,但你心里总觉得,“不,来吧,我们把它做得酷一点,直接把整个东西重新设计一遍。“尤其是你的设计师,他们总是说,“不,让我们推倒重来,做个惊艳的。“但结果总是负面的,然后你就想,“好吧,现在已经来不及了,我们得把这个东西上线,没时间重新来过了。“
为什么我们犯错的比例这么高
**Laura Schaffer:**说起来很有意思,在那篇文章里——你会看到它的作者来自 Microsoft,他搭建过实现平台(Implementation platform),做了很多很酷的事情。当他真正尝试用科学方法去验证人们的假设有多大概率是错误的时候,他也在想,“我想知道这个结论在我们 Microsoft 这里是否也适用。“即使对他来说,也存在同样的问题。我觉得在这个领域里有很多聪明人做各种事情,所以很难接受”天哪,我真的有 80%、90% 的时间是错的吗?“这种想法。但仔细想想,这完全说得通。因为要让一个东西成功,需要满足什么条件?你得完美地理解问题。然后你得完美地理解谁遇到了这个问题——也就是客户。以及他们在什么时间点遇到这个问题。
然后你还得把正确的解决方案放到他们面前来解决这个问题。也许你抓住了问题,这些都对了,但你的解决方案出了偏差。又或者你的解决方案是对的,但呈现方式、传达方式不对。这中间任何一个环节出了问题,就不会成功,就不会产生你预期的指标影响。所以考虑到这一点,我们能成功 10% 到 20% 简直就是不可思议了——毕竟有那么多东西需要同时对齐。我觉得关键在于你要坦然接受这一点——“好吧,这不是关于我有多聪明,或者我的团队有多强,而是这件事本身的逻辑就决定了很难一次做对。让我们接受这个事实,把这个认知融入我们的策略中”,而不是与之对抗。
如何提高实验成功率
**Lenny:**你有没有发现什么方法能提高这些概率?还是说这就是客观规律,你基本上没办法显著提高实验成功的概率?
**Laura Schaffer:**事情是这样的,我觉得我们能做的、让这件事变得更简单的事情非常少。所有那些环节都需要搞清楚。所以我确实认为,每个人最初的、未经测试的想法,命中率都会大致在这个水平。不过,你验证这些想法的方式可以完全不同,你可以非常快速地去验证,这才是关键。而 A/B 测试是验证实验最昂贵的方式之一。它通常需要设计师、工程师、产品经理或增长负责人或营销人员来共同参与。所有这些都是耗时很大的投入,即使对于简单的事情也是如此。然后还有时间因素——这个实验需要跑多久才能看到效果?
所有这些成本都非常高。所以我认为关键在于思考,“好吧,我可以用什么方法快速验证我们的想法?“你可以用”画门”(painted doors)的方式——也就是在功能不存在之前,先测试概念和想法,而不是完整的体验。你可以做模型。如果你有设计师,让他们为那个体验创建模型,放到用户面前,看看他们如何与它互动。这可以非常有威力。
在那个阶段你就可以否定大量的假设。只有那些经过层层筛选的想法,才值得进入深度的 A/B 测试环节。这样你就降低了失败率,因为你用其他方法更快地失败了。所以我更提倡这一条路径:用这些工具快速失败,而不是试图找到一种方法超越其他所有人的水平,去更好地解决那些复杂问题——那会非常困难。但你永远可以在更快地实验和验证方面做得更好。
好想法从哪里来
**Lenny:**你觉得驱动实质性提升的好想法主要来自哪里?是靠直觉和经验,还是数据会告诉你,“嘿,这里有一个巨大的机会”?根据你的经验?
**Laura Schaffer:**我是一个非常数据驱动的人。我这样自我定位,也这样看待自己。很大程度上正因如此,我觉得你必须不断地自我审视,而数据是做到这一点的绝佳方式。但我确实认为,考虑到我做判断的方式,我可能被认为是更依赖直觉的人——因为我非常习惯也很擅长在定量数据的基础上,结合定性反馈来做决策,这在一定程度上降低了对定量数据单独承担判断压力的依赖。
降低置信度,跑更多实验
**Laura Schaffer:**我看到有一件事,我觉得有时和其他人的做法相悖——虽然我也看到一些转变的迹象——那就是 95% 的置信度要求。我大学时在实验室做实验,真的在期刊上发论文之类的,我们必须达到 95% 的置信度,必须达到,因为从实验室产出的、发表的东西会影响教育怎么做,影响我们理解偏见如何运作、何时出现,进而影响我们如何对抗偏见。这些事情涉及对错。如果发出一堆垃圾,会造成很严重的后果——在那个语境下,假阳性、假阴性可以说非常危险。你想想制药行业也是这样,95% 的置信度在某些公司和某些行业是必须的,因为假成功的风险和代价极高。
但我们这些做用户转化、做追加销售的人,非常幸运,不需要承担那种程度的负担,我们可以利用这一点。所以在某些情况下,我一定会主张、一定会推动、自己也一定会使用低于 95% 的置信区间,尤其是当这能让你一年跑的实验数量翻倍的时候。说到底,这些方法都是用来验证我们的假设的工具。如果你在产品工作中用 95% 的置信度,你仍然在接受 5%——某种程度的假成功。那就再多接受一点,我鼓励你再多接受一点,然后跑多得多的实验。如果你看你团队一年下来的净成果,你自己一年下来的净成果,结果会是正面的。
**Lenny:**哇,这是一个很大的想法。放开 P 值的置信度要求,放开实验和数据团队的置信度门槛。大家都会很兴奋。可能团队里一些数据科学家不会兴奋。你真的这样做吗?你怎么操作?你的团队就是这么运作的吗?就是直接说我们不需要 95% 的置信度?
**Laura Schaffer:**我要说的是,这一点非常关键——你必须在做实验之前就把这套方案定好。我看到很多团队陷入的失败模式是,先跑实验,然后让数据去迎合假设。或者有时候根本没假设,就是说”这对我们的指标会有好处”,但没有一个核心原因说明为什么,也没有明确我们到底在测试什么。所以这也是一个我们可能掉进去的陷阱。“我们就做到 80% 吧,我觉得挺好,那个叫 Laura 的人说可以的,所以我觉得没问题。“这永远是个陷阱。所以必须事先非常审慎地想清楚:“好,我们打算这样来验证。“而且,如果你要接受更高的假成功或假阳性、假阴性的风险,你就要非常认真地思考如何加强对你假设的验证。
定性数据与定量数据互相印证
举个例子,回到我们之前说到的 Twilio 的那个案例——我们把用户踢出去,让他们进入那个实验,把他们导向那个体验来隐藏电话号码。在那种情况下,假设我们打算接受一个较低的置信区间,我会非常想看到定性反馈来确认假设是否成立。我会去看那些被放入现有流程的人和被放入新流程的人的定性数据,看哪一边的人觉得更有信心、感觉更容易通过、不会觉得不知所措之类的。我也想听另一边的人说类似”哦,我在那个地方卡住了”,“我搞不明白,感觉超出了我的能力范围”。
我会想寻找其他证据来印证我看到的硬数据。是的,这确实打开了一扇门——只要你打开门接受更多风险,就一定会有一些假成功。但所有这些方法加在一起,总的来说能让你更有可能发布更多对客户有正面影响的东西。而且我再怎么强调都不为过:一年里尽可能少发布,这本身就是一个巨大的风险。考虑到那么高的失败率,这是一个巨大的风险。所以对那些数据科学家——我这些年也跟不少聊过——我试着解释的是,那篇文章、那个 80% 的数据,那是硬数据,说明如果你实验跑得不够多,会有多大的损失。
如果你一年只跑 10 个实验,按照那个方式,一整年下来可能只有两个有影响。所以数据科学家可以理解:“嘿,如果我们这样做,如果我们降低门槛,我们可以把实验数量翻倍甚至三倍,无论具体多少,整体净效果会带来更多成功,让我们整体处在一个正面的位置。“你仍然可以用数据叙事向数据科学家解释你为什么这样做。这也正是为什么当你问那个问题时,我说自己是一个非常数据驱动的人。但我用的一些方法,表面上听起来可能更像是”哦,我在凭直觉”。但同样,非常数据驱动——只是拥抱了一些硬数据的现实,而那些现实我觉得我们并不都接受,有时甚至都不了解——比如那个失败率。
写作计划与人生阶段
**Lenny:**这太棒了,这是一个很大的想法。你在任何地方写过这个吗?给那些想在公司里尝试这个方法的人看看?如果还没写,你应该写。
**Laura Schaffer:**谢谢。说来好笑,我人生中一直想做的事就是把其中一些写下来。我有三个孩子,最小的五个月,另外两个分别是两岁和四岁。所以有时候我刚要开始写,就有一个爬到键盘上。说”一个”,其实是好多次。但总有一天,如果大家感兴趣,我很乐意写。我一直很乐意尽我所能帮助大家,用知识赋能大家做得更好,因为这些其实都不是什么秘方。无非是从经验中学习,而且从别人的经验中学总是比自己摸索更快。所以我当然愿意——我想这是我现在能说的最好的回答了——但总有一天我的孩子会长大。我听说会有那么一天的,到时候我就可以动笔了。
**Lenny:**但愿如此。好的,如果你在 YouTube 上看我们,留个评论吧,如果你想让 Laura 深入写写这个想法,把它传播到你的公司。
不要让数据迎合假设
**Lenny:**好的,我想聊聊增长的话题,但在那之前关于实验还有一个最后的问题。除了刚才说的,还有没有什么大的经验教训或者收获值得分享?
**Laura Schaffer:**我觉得我们之前稍微提到过这个,但我真的想给它加个感叹号、画个下划线——就是让数据去迎合某个概念的问题。我觉得很多团队感受到、也确实承受着很大的压力,要展示进展:“你这个月做了什么?指标动了没有?“这会让人觉得自己不得不这样做。就像”天哪,这个实验……”每个人都有这种经历——你跑了一个实验,盯着数据看,刷新、刷新、刷新,天哪,实际上表现更差了。或者没什么变化,“老天,我们把所有人都搞得好兴奋,大家这么拼命做了这个东西。现在可好,我们在季度业务回顾或月度报告里怎么说?“不管结果是什么。
增长团队需要的时间与期望管理
**Laura Schaffer:**对此我想说的是,增长团队向外做教育极其重要,增长团队之外的人,尤其是领导增长的人,需要理解一个道理:增长团队成功的最佳路径,实际上也是唯一真正能成功的路径,就是接受他们的定位——去验证、去理解最大的机会在哪里,然后去追求这些机会。
这并不是一个按周为单位就能完成的事情,有时候甚至不是按月,取决于你所在的领域以及哪些是已知的、哪些是未知的。所以任何被短期时间线和短期改进所束缚的增长团队,永远都会处于危险的境地。那种环境只会滋生虚荣指标的使用和对数据的粉饰。而那些更成功的团队,往往是以更长的周期来汇报成果的。因为我认为,增长团队只要被给予足够的时间去失败、足够的时间去学到正确的做法,就一定能展现出成功——真正的成功。不是那种”好吧,我们让数据去凑这个结论”,而是真正推动指标变化的成功。
所以一定要做好向外教育。如果你发现自己处于那种被短期指标束缚的处境,就分享那个 80% 的失败率。就是数学、统计学、数据。你不可能在那种环境下持续成功,但拉长时间来看是可以的。这绝对是我会反复强调的一点。在 Twilio 以及之后我去的 Rapid,我时间饼图中相当大的一块都花在这上面,我相信在 Amplitude 我也会花不少时间,就是帮助人们理解什么样的生态系统才是增长团队运作最健康、最强大的生态系统。而对时间的预期管理,是其中很大的一部分。
**Lenny:**你说”饼图”,是指你时间的饼图——就是你很大一部分时间花在这上面?
**Laura Schaffer:**对。
**Lenny:**太好了,我喜欢这个说法。我也经常用饼图来表达类似的想法。再具体一点的话,你觉得增长团队思考的最短时间周期应该是多少?
低-中-高规划框架
**Laura Schaffer:**我觉得很好的一点是,尤其是对于新团队,但其实对所有团队都适用——承诺一个可以在一年内完成的目标,然后用”低、中、高”的框架来规划,这在那个场景下非常有帮助。很多时候——
**Lenny:**你说的低、中、高是什么意思?
**Laura Schaffer:**低、中、高更像是这样的——“我们有几个赌注,或者说几个核心假设。“如果它们起飞了,那就是我们的”高”档——哇,了不起。我们认为这些东西可能会一飞冲天,但也可能全部落空。但不去跑实验你就不会知道。如果那些真的兑现了,那就是我们的”高”。而我们也有一些认为比较稳妥的东西,也许前一年已经做过一些验证,诸如此类。这些看起来能让指标提升这个幅度。
所以给人们这样一个框架是很有帮助的,它完全不同于那种”这是一个我们要达到的单一数字”的思路。就是帮助人们更好地理解这个空间和预期。也正因为如此,成果可能会有些起伏。有一些你发布的东西——说实话在大多数年份里,用这个框架来讨论是最容易的——但有一件事我们做了之后,在销售管道中产生了数千万美元的价值,非常非常强大,它需要时间去摸索和验证。另外有时候我们做的那些新手引导方面的工作,就是我之前说的捕捉那些问题的那些,那个节奏可以稍微快一些,但仍然需要时间去验证和理解。但总的来说,在一年的周期内,你通常能够承诺指标的移动。不过要帮助人们理解其中的方法论,这样他们就不会每周跑来找你说,“过去这几天你做了什么?”
**Lenny:**好的,我得追问几个点。Twilio 那个带来数千万美元的大变化是什么?
**Laura Schaffer:**这是我在 Reforge 教的课程的一部分。
**Lenny:**哦,太厉害了。你在 Reforge 授课。
**Laura Schaffer:**我教的其实是关于留存的部分,我想那是我的板块。回头我把链接给你,可以放在节目备注里。不过高层次来讲,这是我在 Twilio 历程中比较后期的阶段了。快进几年,团队建起来了,一些很酷的事情也在推进。但我当时真的在想,我们下一个大事是什么?那会是什么?然后我注意到了——还记得今天最开始我问的那个关于开发者与非开发者的问题吗?
**Lenny:**嗯,记得。
**Laura Schaffer:**我们看到那个非开发者的小群体在增长。在我们的生态中,自称为非开发者的人数正在上升。
非开发者用户的洞察与 Quick Deploy 的诞生
但非常有趣的是,随着我们对这些人了解得越来越深入,他们中的很多人是想用 Twilio 来构建东西的。当时有一个假设是,“哦,他们可能是迷路了,可能只是想看价格,可能是点错了。“而我的判断是,“不,他们是来构建的,他们想要构建。“他们在开发者新手引导中艰难前行,有些人成功了,有些人……总之,核心问题就是弄清楚他们需要什么才能成功。如果我们能让他们成功,这能转化为收入吗?当时从销售团队那里听到的一个核心认知是——“嘿,当开发者还没有介入的时候,让那些人从零到一、把东西跑起来,对我们来说非常困难。但天哪,如果我们能让他们迈出那一步,只要他们花到一美元,我就能让他们花到五美元。到了五,我就能到五十,到一万,然后到十万。”
整个漫长的旅程就是这样——“嘿,Laura,如果你的团队能帮他们把东西跑起来,我们能做太多事情了。“所以这段旅程就是,好,我们提供的体验中缺失了什么?最终发现就是他们没办法从零开始写代码。这太难了。而且还要搭一个服务器,那也很难。但我们最终通过迭代实验,一步步验证了这些假设,找到了正确的方式来做这件事。结果就是 Code Exchange 上的 Quick Deploy——任何人都可以去那里部署一个应用,不需要写代码,就能体验到 Twilio 的”顿悟时刻”。
**Lenny:**太棒了。所以基本上就是一个低代码的 Twilio 应用?
**Laura Schaffer:**对,最终它变成了——我们给它起了很多昵称、小名。我觉得最能简洁描述它的说法是,它最终变成了一个”创建你自己的 Demo”的体验。这就涉及到用户心理了。我们之前聊过,对开发者来说电信技术可能就挺吓人的。而对于非开发者——有时候他们是买家或者做采购决策的人——对他们来说,不光是电信技术难,开发者那一套东西根本就无法触及,但他们仍然想参与进来,想亲身体验。所以这就是一种方式,让我们给他们动力,让他们感到安心——“天哪,我能把这个跑起来给我的开发团队看,他们肯定也能搞定。“所以这是一个非常有力的时刻,我们真正触及了那些用户的心理,让他们对 Twilio 感到兴奋,然后也给了销售团队一个有力的工具,去跟那些非技术背景的买家和他们对话的人展示。
**Lenny:**太天才了,回头看来这似乎是一个显而易见的成功。我的一个读者建议我开一个”一个功能的故事”系列,走一遍发现、构思、开发、迭代的完整过程,这个案例感觉就是一个非常有趣的例子。不过先不说了。我还有几个问题。我知道我们已经聊了一个小时了。但我有问题,不想就这么放你走,都是关于增长的。一个问题是,你在 Twilio 工作过,那是一个非常产品驱动增长的公司。你现在在 Amplitude,更多是销售驱动的,我知道你们在努力往产品驱动的方向走。我知道 Elena 经常谈到这个,说每家公司都需要有产品驱动的模式,否则就会被产品驱动的竞争对手颠覆。我不确定那些招聘——它们属于哪一类?
**Laura Schaffer:**在 AI、SLG(销售驱动增长)和 PLG(产品驱动增长)之间做选择。是的,对我来说它们是一枚硬币的两面。产品增长和销售,在主题上对我来说是非常相通的东西。区别在于,产品增长是用你的产品来销售,而销售是用一对一的人来销售。所以公司需要同时运用这两种力量,才能最优地转化他们的受众。我们身处一个人们两者都期望的世界。他们期望被你的产品说服,也期望在企业层面被销售触达。大公司是由人来做采购决策的,会有人倾听他们的具体需求并为他们量身定制方案。如果你只有其中一个,就会有所遗漏。所以毫无疑问,我认为你需要这两种力量协同运作。当然,不同阶段、不同领域情况会有所不同,但我觉得就 Amplitude 而言,我认为这里有一个巨大的机会。
从销售驱动转向产品驱动的常见误区
我认为关键在于——对于那些已经做好了销售、现在想攻克 PLG 的公司来说,挑战归根结底在于你从根本上如何切入这个领域。还是那句话,要回到你的用户、他们所处的位置、他们的心理状态。很多公司会说:“好吧,我们要做 PLG 了。把我们的销售企业版产品拿来,把它切碎一点,这里砍掉访问权限,那里砍掉这个功能,然后我们给这个方案贴个标签、定个价,可能还要花好几个小时争论到底定 10 块 9 毛 9、104 块还是 75 块,最后有人赢了这场战役,贴上去就完事了。“总之,讨论的焦点很大程度上围绕着产品本身。
我们要把这个产品怎么样?我们怎么把它拆开、调整、然后交给这些人、这些用户、这些访客?但我觉得这里面缺失的——而且往往是很容易忽略的——是,当我们做 PLG、从销售转向 PLG 时,我们需要重新出发。我们需要认识到,这同样是销售,只是通过产品来销售。一个优秀的销售代表在跟客户接触时会做什么?他们会理解对方的问题和处境。所以我们在 PLG 中也需要同样的东西。进入我们自助流程的人,他们有什么独特的痛点?我觉得对于 Amplitude 这样的公司来说,我们想通过 PLG 模式触达的人群,有很多目标要实现,但其中一个核心目标就是切入 SMB(中小企业)市场和创业公司,给他们一个落地和成长的空间。
初创公司的分析师困境
同样,你得思考他们有什么挑战和独特的问题,因为我们要用产品来解决这些问题。我们需要在他们所处的位置、面对他们的问题去接住他们。根据我在这些初创公司工作以及为一些初创公司做顾问的经验,我观察到一件事——我几乎从未……我想我从来没遇到过一家初创公司拥有合适数量的分析师来满足他们的需求。事实上,很多初创公司根本没有分析师。这就意味着 CEO 在当分析师,为董事会制作仪表盘(dashboard);产品经理也在当分析师,搞清楚到底发生了什么,为自己的产品制作报表。
这种情况到处都是——人们在本职工作之外还得充当分析师。我认为这是年轻公司和早期阶段公司特有的问题。所以当他们——从心理上来说——在寻找产品分析工具的时候,他们在想什么?他们关心什么?他们在找的是一个能让他们安心的东西,确保自己能够真正搞清楚正确的指标,用正确的方式创建报表。展示流失率(churn)的最佳方式是什么?一定有最佳方式,那么多人都在做。你猜怎么着?是的,确实有一些非常好的方法来做这件事,也确实有一些非常成功的方式来为董事会搭建仪表盘。已经有人做过这些了。
这方面的知识大量存在,各种框架也存在,还有基准对标(benchmarking)——这些数字到底算不算好?所以,我的一个假设是,如果我们从这个视角出发,理解了这才是问题所在,那我们就有很多事情可以做,来真正改变自助体验的运作方式,帮助用户完成转化,向他们展示 Amplitude 如何能让他们变得如此强大。但我觉得对所有公司——不仅仅是 Amplitude——在做这种转型时都适用的一个核心要点是:做这件事的时候,不要认为这是一个”复制粘贴然后切碎零件”的事。在制定策略时,不要从你的产品出发,要从你的客户、你的用户、你的潜在客户出发——那些会进入你的自助流程的人。确保你理解他们的问题与你在销售端服务的人有何不同——它们确实不同。然后确保你的产品和体验是围绕这些人来设计的。
**Lenny:**有意思的是,你几乎需要作为一个产品公司重新开始,因为你们可能需要解决完全不同的问题,虽然最终殊途同归。不过你说到可能最终目标群体是分析师或产品经理,这很有意思。我知道 Amplitude 一直以来都是聚焦产品经理的,但是——
**Laura Schaffer:**是的,没错。这里面有一个好的方面是——在某些方面确实感觉像是重新开始,因为你确实需要从客户和他们的需求重新出发。但在很多方面,你可以迁移大量已有的知识。到那个时候你已经知道什么是行之有效的。以 Amplitude 为例,它在如何最好地搭建报表方面确实积累了大量的知识。很多东西已经有了很好的势头——你怎么利用那些势头,怎么把它们整合并呈现到用户面前,确保他们获得正确的东西。已经有大量的势能积累在那里了。关键就是如何驾驭它,并理解——是的,哪里有差距,因为肯定会有差距。
但始终以客户问题为锚点——我认为这与你做任何新产品、发布任何新东西时是一样的,都应该先想客户和痛点。所以,无论你是第一次做产品驱动增长(PLG)还是刚开始切入,都是一样的道理。你需要重新从问题出发——他们的问题是什么,他们进入你的领域时带着怎样的心态——这样你才能构建出真正让他们感到”哦,你能解决我的问题,你懂我”的东西,并向他们展示你的产品将如何做到这一点。
**Lenny:**最后一个问题,关于开发者。你曾在 Twilio 工作,Twilio 显然是面向开发者销售的。我想你在 Amplitude 之前工作的 Rapid 也是面向开发者的。面向开发者销售,现在感觉是一个非常火热的领域。有那么多初创公司专门做开发者工具,市场巨大。以前可不是这样。以前大家觉得开发者没有市场——他们不会花钱,人数也不够多。而现在这是一个非常热门的赛道。所以我很好奇,在面向开发者的创业和产品构建方面,你学到了什么?我想很多正在做搜索工具的创始人会非常感兴趣。
**Laura Schaffer:**第一点是,开发者与其他任何受众都非常不同。我见过很多在其他受众群体中增长做得很好、产品做得很好的人,心想”我要把这些经验全部带走,转向服务开发者”。结果发现这是一个非常陡峭的爬坡,因为开发者实在太不一样了。让我给你说几个有趣的事实,这些事实让他们真的很不一样。其中一些还伴随着有趣的故事。
第一,开发者几乎不会看你的营销网站。他们直接进入注册流程。这意味着什么?意味着你在网站上精心设置的所有上下文——产品介绍、定价信息等等——他们很多时候全部跳过,毫无上下文地直接进入注册。所以每当你做类似这样的假设:“哦,他们注册的时候应该已经知道这个了”,或者”这个不需要放进去,营销网站上有”——这些假设对这个群体统统不成立。
他们就是这样的人。我对这个群体的类比是:他们是那种宜家包裹寄到之后,不会先翻开说明书从头读到尾然后再动手的人——他们是直接撕开包装袋,把零件拉出来就开始试着组装。只有在卡住了的时候,如果他们有动力,才会回过头来找上下文和步骤。这是第一点。
另一点就是他们对与销售交谈的抗拒。我猜有些人听到这个会说:“哦对,我也讨厌销售。“当我被发出去、被销售轰炸的时候,那确实是最糟糕的体验。我完全理解。但开发者在这一点上完全是另一个级别。曾经有一家 FAANG 公司注册了 Twilio,搭了一个概念验证(POC),上线到生产环境,所有这些都完成了,并且在上面运行了好几个月——一次都没有跟销售接触过。
我当时试图联系他们,最后是我先跟他们说上话的——因为他们找的是客服支持,因为他们的交付有个问题,缺少某个功能,他们不想跟销售谈。他们最后是跟我聊的,当时我在做产品营销。那是我第一次认识到:这些人真的不想跟销售说话。还有一个例子,一家巨大的零售公司,他们的工程团队用个人邮箱注册——就是为了不被销售轰炸。这件事我们后来才知道。
不过最重要的——这些都只是有趣的事实——我想留给听众的最重要的东西是:到底是什么让他们如此不同?为什么?原因何在?这要追溯到他们的职责和责任。如果我们把自己放在开发者的鞋子里想一想:如果一个开发者被要求使用你的产品,尤其如果他们是主要用户、主要搭建者,有一点非常重要——他们对此负有责任。
如果你的服务宕了,那是他们的责任。不仅是对自己,还有对整个团队。如果因为从你这里买的服务出了问题导致 Pager 响了、把人叫起来——那是他们的锅。如果——哦,结果发现这东西跟他们说好的系统不兼容——那是他们的责任。跟所有人想要的数据集成不了?那是他们的责任。只要不工作,所有人都会找开发者;而它可能以各种方式不工作。这在很多方面意味着他们的失败——可能让他们丢掉工作,可能让他们失去团队的信任,也可能损害他们的声誉。
这意味着他们每次采纳新东西时,赌注都非常高。所以他们承担不起仅凭别人的话就做决定——尤其是一个可能有其他动机的销售代表的话,从他们的角度来看。他们不能相信你的内容或者某个人的承诺。他们必须亲自做。他们必须自己证明。这就是为什么,要让开发者真正认可,他们需要亲手做点什么、搭点什么——至少是一个概念验证(POC),甚至更进一步。
这意味着他们在你的自助流程中会走得相当深,才会准备好做出承诺。所以,如果你的公司提供的产品需要开发者来搭建,你必须在自助体验上投资,才能有效地转化你的受众。你应该把他们当作你的核心——你的自助功能和增长团队中的某个角色,应该等同于 Salesforce 的地位——因为你的开发者不会接受销售在那个时候介入来试图转化他们。
**Lenny:**我特别喜欢一点,你总是回到用户的心理——在这个例子中是开发者——比如,这是为什么他们要为此负责。销售代表会说服他们”这东西能行”,但事实并非如此。这是一个非常有洞察力的分析,也是一个非常棒的收获。我们还有什么没覆盖到的吗?在进入非常令人兴奋的闪电问答之前。
**Laura Schaffer:**足够多了。我觉得我们都聊完了。
**Lenny:**你回答了我所有的问题,甚至更多。那么——欢迎来到非常令人兴奋的闪电问答。我为你准备了六个问题。准备好了吗?
**Laura Schaffer:**我准备好了。
**Lenny:**你会最常向别人推荐的两三本书是什么?
**Laura Schaffer:**我是一个幸福感的坚定信奉者。不只是那种空洞的”我们都应该快乐”的说法,而是因为它确实能帮助我们做到最好的工作,让我们更有创造力,等等。所以第一本是 JL Collins 的 Simple Path to Wealth。我不会无视数据——金钱确实是经常阻碍我们获得幸福的东西。我认识很多非常聪明的人,就是没有搞定理财这件事。这本书会覆盖你所有的基础知识,非常容易读。他还有一本自己朗读的有声书。JL Collins 的 Simple Path to Wealth,非常棒。
**Lenny:**最近的一部电影——哦等等,等等,还有更多?
**Laura Schaffer:**哦,还有一本。
**Lenny:**来吧,说来听听。
**Laura Schaffer:**同样是关于幸福的——James Clear 的 Atomic Habits。如果你想改变自己的某个方面,或者有什么地方不太对劲,这个人会给你一个改变的框架。保证有效。
最爱的电视剧
**Lenny:**我特别喜欢那本书。他现在势头正猛,上过 Tim Ferriss 的节目,那期访谈很精彩。不想读书的朋友可以去听听那期播客,里面有很多实用的小技巧。那你最近最喜欢的电影或电视剧是什么?
**Laura Schaffer:**毫不犹豫地说——《The Great British Baking Show》。我爱那个节目,所有人喜欢它的理由我也都喜欢。它温暖人心,让你感觉良好、精神振奋。但同时它又是一个竞赛类节目——他们都在努力成为最棒的烘焙师,却又在赛场上互帮互助,像一家人一样。我看到的大多数真人竞赛类节目,全是尔虞我诈、互相拆台。所以我一直对这里面的心理机制深深着迷。我真希望有人就此写一篇研究论文,搞清楚他们为什么都在互相帮忙。不过这确实很棒,非常值得一看。
**Lenny:**有意思,你总是绕回心理学。
**Laura Schaffer:**我知道,我知道。我觉得自己真的在那方面陷得很深。不过确实如此,这对我来说非常有趣,我真的很喜欢那个节目。
面试中喜欢问的问题
**Lenny:**在面试中,你喜欢问什么问题?
**Laura Schaffer:**我喜欢让对方讲一个不是由他们自己挑选的上线或发布。你可以用很多种方式引导到这一点。问题是,每个人都有一个重大的成功故事,人人都有。所以你去问一个人”你发布过什么了不起的东西?“其实说明不了太多,因为谁都能讲出这么一个故事。所以要拿走这个选择权。一个简单的问法是”最近一次上线是什么”,因为最近的事受时间限制没法挑。当然还有其他方式可以拿走这个选择权——只要给他们设定具体的参数,比如某次他们分享过的发布,诸如此类。这样你就能更多地倾听、更多地了解他们的思维框架,而不是结果。因为你随机挑了一次发布,大概率它不会是什么惊天动地的成功。所以他们反而会更愿意谈自己是如何一步步走到那里的——而这正是你想了解的,你想知道他们的思维框架是什么,他们处理问题的方式能否带来成功。
**Lenny:**这个太酷了,我之前从没听过这个问法。真的是个很巧妙的点子。
日常使用的 SaaS 工具
那你日常工作中最常用的五个 SaaS 产品是什么?不许说 Amplitude。
**Laura Schaffer:**我知道,对吧?我还在了解我们这里有哪些工具呢。不过,我就分享几个我在别处用过、非常喜欢的。第一个是 Hotjar。Hotjar 也可以,任何能让你快速在客户面前放点东西、获取我们之前谈到的那种定性反馈的工具都行。它是定量数据不可或缺的关键补充,能帮你理解到底是什么导致了变化,或者什么没有导致变化。这非常重要。
我不得不说,Amplitude 是一款非常棒的工具,我之前就用过。如果我不是刚加入 Amplitude,我一定会提到它——我确实是到了 Amplitude 之后才第一次用它,体验非常棒。所以,加个星号,因为我现在在那儿工作,但我确实真心喜欢它。
还有 Slack,这个很无聊,每个人都说 Slack,但我不得不给它点赞。它让工作生活变得太方便了,向他们致敬。
然后是 Builder,这个我也得加个星号,但我真的很想推荐它。很多人不知道它,但它真的很有帮助。我是他们的顾问,所以站在他们那边。但我确实认为它是一款很有力量的工具。很多团队会卡住——他们过于依赖工程师来做变更。我们前面谈到过快速实验,要不断地、不断地、不断地把东西推出去。而 Builder 让这件事变得很容易。它也是一个无头CMS(headless CMS),支持拖拽操作,让非工程师也能做变更。所以,特别是当你想解决我提到的那 80% 工程瓶颈问题时,Builder 会是一个好方案。
如果你还想要一个,那我就给 ChatGPT 吧,这个也很无聊,所有人都在说。但我没有什么特别疯狂的观点,我只是觉得我们都需要搞清楚怎么把它融入工作流程。不这么做的人,大概率会吃亏。当然,不管是聪明的 AI 还是别的什么机器人都是一样。不过这就是我的回答了,Lenny。但如果你过几个月再来问我,等我在 Amplitude 真正待了一段时间之后,我肯定给你一个不一样的答案。
**Lenny:**正好借这个机会安利一下 lennybot.com。我写了一篇newsletter——或者说是创建了这个机器人的 Dan Shipper 写了一篇——讲他是怎么搭建这个东西的。你可以用它来问我问题,它会用我 newsletter 里的内容来回答。非常酷。lennybot.com 或者 lennysbot.com。
**Laura Schaffer:**太棒了。
**Lenny:**就是这样。
**Laura Schaffer:**我之前不知道这个。那好,我改答案了,就是它。
**Lenny:**对,这就够了,我只要这个。
产品开发流程中的小改变
还有两个问题。你在产品开发流程中做过什么相对较小的改变,但对团队的执行力产生了很大影响?
**Laura Schaffer:**就是之前提到的那个”感到尴尬”的理念——对第一次迭代感到尴尬。如果你不觉得尴尬,那就做得太多了。这真的能加速发布节奏,而且帮助人们去拥抱那些不完美的成果,而不是为之羞愧。就是这么简单地接纳它。
Amplitude 的使用技巧
**Lenny:**太棒了。最后一个问题。我知道你刚加入 Amplitude,但你有没有什么特别喜欢的 Amplitude 使用技巧,或者人们可能不知道的隐藏功能?
**Laura Schaffer:**这个你来告诉我吧。不过我可以说一件特别酷的事。在我离开 Rapid 之前,团队里有人做了一个视频,展示 Amplitude 和其他工具——比如 Hotjar 和 Segment——打通之后能有多强大。当时有人创建了一份 Amplitude 报告,报告里出现了一个异常:用户在以我们没想到的方式使用某个功能。Amplitude 的报告把它暴露出来了,但我们想知道为什么会出现这种情况。于是我们可以找到那个事件是什么,通过 Segment 查到对应用户的身份,再去 Hotjar 里查看,直接看到用户执行那个操作时的录屏。
由此,我们对背后的真正原因形成了一些非常具体的假设。当然,直接和客户对话是非常有价值的,但在这种情况下,仅仅是把这些技术串联打通,就能在不打扰客户的情况下获得一幅相当完整的画面。所以这个技巧就是:当你把 Amplitude 和其他工具结合起来用的时候,它能发挥的力量真的会被成倍放大。
尾声
**Lenny:**太精彩了。Laura,我们今天涵盖了很多内容——职业发展、实验、增长、“尴尬”、心理学。非常感谢你的到来。最后两个问题:一,大家如果想联系你、了解更多、或者给你发一两条感谢的话,在哪里可以找到你?二,听众怎样能帮到你?
**Laura Schaffer:**可以在 LinkedIn 上找到我。我不太发帖子,这个就怪我家三个孩子吧。但我保证最终会多发的。不过我回消息还是比较及时的,所以欢迎在那儿加我联系。
**Laura Schaffer:**至于听众能怎么帮到我,我始终很乐意收到反馈和建议。但我也想说,我知道眼下外面的环境确实有点艰难,尤其是科技行业的朋友们。所以我也在想着我能怎样帮助大家。我知道我担任顾问的几个地方正在招人,Rapid 也在招人。我也知道有几个团队在招优秀的增长和产品人才。如果你有兴趣了解更多,别犹豫,随时联系我。我希望能在这方面尽可能多地帮助大家,因为当下确实是不容易的时期。我相信你们也听过、看过这样的话,但如果你被裁了,这不是你的问题,不是你的错。是这个疯狂的大环境造成的。一切会好起来的。哪怕只能帮到一个人找到工作,我都会感到非常幸运。所以关于这件事也欢迎随时联系我。
**Lenny:**太好了。如果你方便分享一些链接的话,我们可以在节目简介里附上开放职位的链接。
**Laura Schaffer:**好的。我知道有几个职位还没有发布 JD,真的是新鲜出炉,但我很乐意整理一些信息放上去,因为我知道这样大家会更方便了解情况。
**Lenny:**太好了,我们会尽量在简介里做好整理。Laura,再次感谢你的到来。
**Laura Schaffer:**谢谢你邀请我。这次非常精彩,也很有趣。
**Lenny:**大家再见。
感谢大家的收听。如果你觉得这期节目有价值,可以在 Apple Podcast、Spotify 或你喜欢的播客应用上订阅。也请考虑给我们评分或留言,这真的能帮助更多听众发现这个节目。你可以在 lennyspodcast.com 找到所有往期节目或了解更多关于节目的信息。下期再见。
术语表
| 原文 | 中文 |
|---|---|
| A/B test | A/B 测试 |
| aha moment | 顿悟时刻 |
| Amplitude | Amplitude(公司名,保留原文) |
| Bandwidth | Bandwidth(公司名,保留原文) |
| be embarrassed | ”感到尴尬”理念(第一版不完善到让自己不好意思,即快速迭代中接受粗糙初版的心态) |
| benchmarking | 基准对标(benchmarking) |
| Builder | Builder(产品名,保留原文) |
| calibration | 校准会议(绩效评估中多人对齐评审的环节) |
| churn | 流失率(churn) |
| Code Exchange | Code Exchange(Twilio 平台名,保留原文) |
| confidence interval | 置信区间 |
| conversion | 转化率 |
| Dan Shipper | Dan Shipper(人名,保留原文) |
| dashboard | 仪表盘(dashboard) |
| developer relations | 开发者关系 |
| e-commerce manager | 电商经理 |
| FAANG | FAANG(指 Facebook/Meta、Amazon、Apple、Netflix、Google 五大科技公司,保留原文) |
| false negative | 假阴性 |
| false positive | 假阳性 |
| false success | 假成功(指实验结果看起来成功但实际并非如此) |
| friction | 摩擦(用户体验中的阻力或额外步骤) |
| GM (General Manager) | 总经理 |
| Great British Baking Show | The Great British Baking Show(节目名,保留原文) |
| hard data | 硬数据(客观的定量数据) |
| headless CMS | 无头CMS(headless CMS) |
| Hotjar | Hotjar(产品名,保留原文) |
| low code | 低代码 |
| mocks | 模型(设计稿/原型) |
| North Star metrics | 北极星指标 |
| onboarding | 新手引导 |
| P-value | P 值 |
| painted doors | 画门(在功能开发前用假入口测试用户兴趣的验证方法) |
| performance reviews | 绩效评估 |
| pipeline | 销售管道 |
| PLG (Product-Led Growth) | 产品驱动增长(PLG) |
| PM (Product Manager) | 产品经理 |
| POC (Proof of Concept) | 概念验证(POC) |
| product marketing | 产品营销 |
| QBR (Quarterly Business Review) | 季度业务回顾(QBR) |
| Quick Deploy | Quick Deploy(Twilio 产品功能名,保留原文) |
| Rapid | Rapid(公司名,保留原文) |
| Reforge | Reforge(公司/平台名,保留原文) |
| retention | 留存 |
| Segment | Segment(产品名,保留原文) |
| self-serve flow | 自助流程 |
| ship | 上线/发布 |
| signup flow | 注册流程 |
| SLG (Sales-Led Growth) | 销售驱动增长(SLG) |
| SMB (Small and Medium Business) | 中小企业(SMB) |
| SME (Subject Matter Expert) | 领域专家 |
| Tim Ferriss | Tim Ferriss(人名,保留原文) |
| Twilio | Twilio(公司名,保留原文) |
| ungating your knowledge | 解锁你的知识(将个人知识开放共享) |
| vanity Metric | 虚荣指标 |
| voice of the customer | 客户之声 |
此文档由 AI 分片翻译(translate_long_document)