从规模化 Uber 和 Opendoor 中学到的经验 | Brian Tolkin(Opendoor 产品负责人,前 Uber)
Lessons from scaling Uber and Opendoor | Brian Tolkin (Head of Product at Opendoor, ex-Uber)
Brian Tolkin’s Career Background
Lenny Rachitsky: You’ve worked at two businesses that have done incredibly well combining product in ops.
Brian Tolkin: Uber always has this mentality and Opendoor does two of the product operations, twin turbine jet plane where you can fly the plane on one engine for a little bit if you need to, but it’s operating most efficiently and effectively if both are working together.
How Ops Shapes Product Leaders
Lenny Rachitsky: What has having been in ops done to make you a better product leader?
Brian Tolkin: Gave a really deep understanding of how the business actually works. It’s a pretty good foundation for them going on to say, okay, what do we actually want to build in a more scalable technology way.
Common Product Misconceptions About Ops
Lenny Rachitsky: Something else I’ve heard that you’re very good at is staying very calm under pressure.
Brian Tolkin: I’ve slept on the floor in China before launching uberPOOL, and when you reflect the stress onto your teams, everybody tenses out. It counterintuitively doesn’t produce better outcomes.
The Manual Era of Surge Pricing
Lenny Rachitsky: Today my guest is Brian Tolkin. Brian is currently head of product and design at Opendoor. Before that, he spent nearly five years at Uber where he joined as employee 100. Before Uber had UberX or uberPOOL or any shared rides, he actually started on the ops team at Uber, moved into product, ended up leading product and launch of uberPOOL, and then taking it global. He also started the product operations function at Uber. Before that function was really even a thing, which I didn’t know until the chat that we had. In our conversation, Brian shares a ton of lessons about building products with a heavy operational component. Also, how to run great product reviews, how he implements the jobs to be done, framework and Opendoor’s successfully.
The story behind Zillow trying to compete with Opendoor failing and then partnering instead. Plus a ton of great stories from the early days of Uber and Opendoor, and so much more. If you enjoy this podcast, don’t forget to subscribe and follow it in your favorite podcasting app or YouTube. It’s the best way to avoid missing feature episodes and it helps the podcast tremendously. With that, I bring you Brian Tolkin. Brian, thank you so much for being here and welcome to the podcast.
Brian Tolkin: Thank you, appreciate it. Thanks for having me.
Early Uber Stories
Lenny Rachitsky: First of all, just a huge thank you to Kayvon Beykpour for connecting us, introducing us. He said all kinds of amazingly nice things about you. He also gave me some very hard questions to ask you. I hope you’ve come prepared.
Launching UberPOOL in China
Brian Tolkin: Terrific. Put me in my hot seat.
The Opendoor Story
Lenny Rachitsky: Okay, I want to spend a bunch of time talking about product and ops. You started your career in operations. At Uber you actually started on the ops team and you moved into product. You’ve also worked at both Uber and at Opendoor, which have both huge operational components. I think it’s really rare that people, one, see a company scale to the heights of Uber and Opendoor with such a heavy operational component that are still tech companies and also it’s really where someone starts an ops and then moves into product and ends up where you are, where your chief product officer, really successful company. So I have a bunch of questions here. Maybe the first is just what has having been in ops done to make you a better product leader? How does that change the way that you operate as a product leader?
How Product and Ops Collaborate
Brian Tolkin: Starting on the operations side gave a really deep understanding of how the business actually works. You are truly operating it day in and day out and the success of the city is in large part driven by the inputs that you are putting into it every single day on the ground and whether or not there was rain that weekend, which was a nice driver of metrics, but talking to customers every single day like one-on-one onboarding drivers responding to support tickets, there’s no centralized support team, there was no closer to the customer, right? And so I think that foundation actually for really understanding what moves the business and being super close to the customer actually is a pretty good foundation for them going on to say, okay, what do we actually want to build in a more scalable technology way?
The Evolution of Operations
Lenny Rachitsky:
Then Pendo integrates user feedback so that you can capture and analyze what people actually want. And the new thing in Pendo, session replays. A very cool way to visualize user sessions. I’m not surprised at all that over 10,000 companies use it today. Visit pendo.io/lenny to create your free Pendo account today and start building better experiences across every corner of your product. PS, you want to take your product led knowhow a step further? Check out Pendo’s lineup of free certification courses led by talk product experts and designed to help you grow and advance in your career. Learn more and experience the power of the Pendo platform today at pendo.io/lenny.
MUSIC: Pendo.
Operations and Automation
Lenny Rachitsky:
Try it for free at explo.co/lenny. That’s E-X-P-L-O/lenny. I’ve seen that a lot of companies, and this was definitely true to Airbnb, where the product team looks down a little bit on the ops team where they’re like, “oh, we’re doing things that are going to scale to millions of users. We’re doing these things that are going to apply to everyone.” There’s this ops team over there doing a few things that are going to not scale. They keep asking us for things to build for their one-off ideas. What do you think that product teams often maybe miss or don’t understand about the ops teams that would help them see them in a different light?
Brian Tolkin: Yeah, it’s a great question and I think Uber always had this mentality and OpenDoor does too of like a twin turbine jet plane where you can fly the plane on one engine for a little bit if you need to, but it’s operating most efficiently and effectively if both are working together, and I think that’s really true, right? The reality is operations teams, local teams, can iterate faster, can scale talking to customers really much more efficiently, have great qualitative insights, and so if it’s seen more as a harmony instead of a competition, I think that’s really, really helpful where it’s like, okay, how do we get the insights that are happening day in and day out in the field, on the ground, whatever that may be, and help us build better products because of that, right?
Like a PM sitting in San Francisco can’t be in Opendoor’s case 50 markets, walking houses every single day in Uber’s case, whatever, a thousand cities understanding the nuances of safety in South America and it’s just not possible, but what you can do is foster a really good relationship and a really good feedback loop of how people who do deeply understand those things can help give insights. Now is actually the birth of product operations was that insight as well.
The Product Review Process
Lenny Rachitsky: Can you say more on that?
Product Review Participants and Outputs
Brian Tolkin: Yeah, sure. So, sorry, I should probably define what product operations was. At Uber it was basically this notion that we had centralized, this was later in my career at Uber, but we had a centralized product team building stuff mostly in San Francisco, not strictly through the Ross, but at this point around the world, but mostly in San Francisco and then we had a very globally distributed operations team, and there was a bidirectional feedback loop that wasn’t super strong and that feedback loop was basically when the EPD teams in San Francisco built new features, how do we effectively put it in global markets and then how do we effectively get input from global markets to better build features. And so one solution to that problem, our solution at the time was to start up a new function called product operations who had accountability and reported into operations but physically sat with and operated much like a member of the product team to help solve that.
Jobs to Be Done at Opendoor
Lenny Rachitsky: Is that maybe the first time there’s a… Did you invent product operations as a function?
Keys to Implementing Frameworks Correctly
Brian Tolkin: I don’t think so because at the time, I believe Google had a function, I can’t remember what Google called it was something slightly different, but I met with a few folks who had been in similar type roles at Google and a couple other places, so I don’t take credit for certainly for inventing IT and other people have actually dabbled in this model at Uber before me there was just a formalization of it and our actual building out of the organization, et cetera.
Low Sample Size A/B Testing
Lenny Rachitsky: Did none of that. Sounds like you basically helped make it a thing. I know you’re being very modest, I think. Coming back to your point about decentralized operations teams, something I’ve read is that search pricing came out of one GM and a market just testing, emailing all the drivers, hey, we’re going to give you extra if you drive on Saturday night. Is that true?
Brian Tolkin: That would’ve been probably a little bit before my time, but that being said, one thing that is true is that surge pricing for actually quite sometimes all of 2012, certainly 2013 probably, I don’t know when we necessarily switched, was very much a human in the loop system or a very manual system where GMs in every city would control basically the parameters in which surge would operate and so much of the time that would need, for example, Monday through Friday, there would be no search, it couldn’t flip on, and then Friday nights and Saturday nights it would flip on from whatever you set, 7:00 PM to 3:00 AM and the cap was X, whatever the cap was. And then within those parameters the algorithm would optimize for what the price was. But yeah, GMs controlled whether it was on or off and what geographies were surging.
Intuition and Product Decisions
Lenny Rachitsky: Wow, I didn’t know that. Was that out of we believe we are better than the algorithms or we just don’t have time to make them amazing yet, so we’re just going to help them along?
Brian Tolkin: Yeah, I think it was probably a function of a bunch of stuff, one of which is like, hey, this is a fairly new concept and it’s powerful and dangerous and so let’s make sure we understand what’s happening. The second is this belief that local city teams know their cities best and so you might know that an event is happening, a baseball game gets out and it’s like, oh, I know that this baseball game’s going to get out at 10:00 PM so I’m going to set surge at 9:45 and the algorithm may not be able to pick that up. And then the third is the technical constraint of nowadays clearly it’s all automated, but it’s really hard to build a fully dynamic, always on geospatially aware pricing system and not just a little bit of time.
The Zillow Story
Lenny Rachitsky: That makes sense. I feel like you’re full of wild stories from your time at Uber. Is there one that comes to mind of just, I think you helped scale in China, uberPOOL?
Brian Tolkin: Yeah.
Stripe Atlas and AngelList Parallels
Lenny Rachitsky: Maybe that’s the one. I don’t know. Can you share wild story from early Uber days?
Brian Tolkin: Yeah, so in the early days of Uber, one fun story is obviously UberX is a random mainstream product but has a funny, silly name, UberX. This product in the early days was going to be all hybrid and had a bunch of different potential names. I was not personally driving this, this was someone else on the operations team, but they built the model for what this product could be and there’s no name for it yet, so it was going to be a placeholder. So what do you put in some placeholder? X. So UberX and then the company was moving quickly enough, the product got the green light, it launched, and here we are, I don’t know, 12 years later, 11 years later, whatever it is and UberX is the name that stuck.
Staying Calm Under Pressure
Lenny Rachitsky: That is hilarious. I love it. So it was a placeholder. It’s like many products start that way where they’re like, this is just the temporary name, and they’re like, okay, I guess everyone just knows it this way now we’re going to stick with it.
Brian Tolkin: Exactly. It’s too expensive to change and rebrand at this point.
Finding Core Truths Amidst Noise
Lenny Rachitsky: That’s an awesome story.
Brian Tolkin: One that is good about scaling Uber uberPOOL in China is we were launching Uber pool in China and this was going to be, China at the time was pretty big for Uber, but uberPOOL was not there yet, and so we were going to launch. And myself and a few other folks were in Chengdu China, which is the first Chinese market that we were launching uberPOOL in and we were going to be on the ground took to launch. We wanted to go live at, I believe it was 6:00 AM for rush hour on, I don’t remember the day, but whatever, Monday morning, and so we’re there over the weekend getting ready to set up and at the same time we were doing some data center testing. And so we flipped on all the testing infrastructure and thought it was going to work and nothing works, and the matching algorithm just isn’t working and we’re like, oh my god, now it’s whatever, 5:00 PM the day before we’re supposed to go live, 6:00 PM, 7:00 PM okay, let’s get on the phone with the US, try and figure out what’s going on.
I remember I slept about 30 minutes that night between 2:00 and 3:00 AM being like, okay, well, we have to go live at 6:00 AM I think there was some press around it. We were planning on going live and I think we got everything finally working at probably about five 30 or six in the morning and launched just in the nick of time. And I’ll never forget, we launched, it was great, we monitored, everything was good, and then we walked out for breakfast at 7:30 in the morning. Everyone sleep depressed, no one slept all night and we got these pancakes street food things and I have to imagine they were not that good, but in my mind it was the best meal I’ve ever had in my life.
The Particle News App
Lenny Rachitsky: It’s like a meal after a marathon or a big hike.
Brian Tolkin: Exactly. Yeah, exactly.
Personal Life Mantras
Lenny Rachitsky: Everything’s so delicious. This comes up a lot of just these moments that are so incredibly stressful and hard and leap deprived end up being the best memories and the best stories to tell and things you look back at fondly. It’s so weird how human nature is like that.
Finding Career Mentors
Brian Tolkin: Yeah, I mean another one more recent for Opendoor was when COVID hit, physically we buy and sell homes, and so we were physically going into people’s homes and suddenly March 2020 going into people’s homes was not something people were comfortable with. And you look at the real estate data coming out of China at the time and it looked like coming to a standstill, and so we actually turned off the core business and we stopped buying homes for a few months. hey, we can’t go in and we don’t know if anyone’s going to be buying any homes, and so what do we do? And we took those few months and then came out the other side and had virtualized the whole process. Then it was pretty stressful, right, because looking at a business that relies on going into people’s homes and suddenly you can’t do that anymore, what do you do? So, again, a fond memory to look back on a very stressful time in the moment where it feels very, very difficult.
The Uber Interview Story
Lenny Rachitsky: Just since you mentioned Opendoor, I think many people have heard of Opendoor. Maybe just give a quick explanation of what Opendoor does for people that aren’t exactly sure.
Brian Tolkin: So we’re a digital platform to buy and sell real estate. The core product today is a seller focused product where people can go online, enter some information about their home, and we’ll make an all cash offer to be able to sell simplicity and certainty. So the product really works for people who want something that is certain and simple and easy. I don’t know if you’ve ever sold a home, but it can be a very, very, very successful difficult process with showings and open houses and how to price it and will it sell and all of that stuff. And so we offer basically a way to skip the whole process.
Final Thoughts and Conclusion
Lenny Rachitsky: So you basically sell your house to Opendoor and it’s just like, cool, done, move on.
Brian Tolkin: You picked your closing date, you move out when you want. Yeah, there’s no hassle.
Lenny Rachitsky: Sounds amazing. I want that. Coming back to ops and product, just to close this thread again, you’ve worked at two businesses that have done incredibly well combining product and ops. Are there any just broad lessons you’ve taken away from how to make these two teams and functions work well together and to build a business that’s very ops heavy but also offer driven?
Brian Tolkin: The first one we touched on, which is a, there’s just got to be mutual respect. Both functions have their time and their place and their skillsets, and you just don’t build big businesses of this type without respecting the fact that that both need to exist. The second, particularly on the product and engineering side, is really understanding where and how the technology leverage comes from the business and then being really focused on making sure generally, especially in the earlier days, you are more limited on the technical resourcing side than you might be on the operational resourcing side. And so how do you be really focused on where to invest your time, effort, and energy technically, which is why most of the engineering effort for Uber was on the dispatching system and the pricing system. That’s just where the leverage was at the time, given the scarcity of resources.
And so I think the second one is being really intentional about where those techs are and then being really forthcoming and saying, hey, that means all these other places where yes, it can make things easier, more efficient, et cetera, et cetera. We are okay not investing in right now, and that needs to be an explicit decision and very transparent. But then the last bit I would say is a deep understanding that the real world has entropy and it’s hard and it’s messy for us at Opendoor, we go into homes, someone may not be home, scheduling may be off, at Uber the driver may cancel the radio GPS. All these things happen, right? Computers are deterministic, but humans aren’t, right? And so building products that have a little bit more flex or a little bit more fail safes in case those things happen becomes a little bit more of a paramount.
One other thing, the last thing I would say is I think that the companies evolve as well. So what I talked about at the beginning of Uber being very focused from an engineering and product side on the dispatching system and the pricing system, obviously over time not to evolve now it centralized all of these functions as the company got bigger and more mature and scale and optimization started to be more important and expansion and that Petri dish of trying new stuff and the tools got better and the tech got easier and there was more internal infrastructure. And so over time things can start one way and shift over time as the business needs.
Lenny Rachitsky: Let’s actually spend more time there. You keep saying things that make me want to dig deeper. So at Airbnb we went through the same thing where there was all these local ops teams driving supply, finding homes, bringing out the platform, and then there’s this tipping point where the product in organic growth or word of mouth ended up driving more and then orders of magnitude more. So there was no need for these folks to spend time doing these things. Can you just maybe share an example, either we Opendoor, when you talk about there’s a time and a place and a skillset for ops, how that evolved? What was the team doing initially and then what did they end up doing as things grew?
Brian Tolkin: Yeah, I mean, maybe a very easy good example to pick just one part of the Uber process in the early days is at small scale, actually back when it was Uber black drivers, every driver was individually onboarded in a 90 minute to a two-hour in-person in the office onboarding with deep setting of expectations. The next version of that… So that’s obviously very ops driven. The next version of that is a small classroom type setting of three or five or six drivers at a given time. Also, very ops driven. And then as we got into more mass market products like Uber Taxi or UberX, it was like, okay, maybe 20 or 30 at a time. So now it’s a little bit bigger classroom setting. And we said, okay, let’s make a video. Instead of giving verbally the same presentation, let’s just make an onboarding video and that was the next set of scale, but now suddenly we have a different problem, which is okay, you have to validate all of these credentials.
So most driver’s license who they are, all that stuff at one person, easy at three to four at a time, easy, 10 at a time, a little more challenging but fine. At 20 at a time, okay, you’re starting to run up onto it now you fast-forward six months and you’re doing a thousand a week or whatever, suddenly your system breaks and it’s like, okay, we have reached the point where operational system improvements is no longer viable. So, you say, okay, we have gone from the iteration stage to the scale stage and technology is uniquely good at scaling. So now we say, okay, instead of having a bunch of folks around the world taking pictures of driver’s licenses and validating and doing all that stuff, how do we integrate with some type of OCR technology or auto recognition of driver’s licenses that feeds to a system that knows what a driver’s license is or can do automatic validation and suddenly you’ve done two things.
One, you’ve scaled your system, and two, you’ve just created a ton of time for what at the time was probably dozens if not hundreds of people running these onboarding sessions all over the country, the world at the time to do other stuff. And so now you can level that up and say, okay, do we do more analytics? Do we do more figure out the next process that needs optimization or whatever the case may be in that virtuous cycle just continues.
Lenny Rachitsky: The way I like to think about this is do things that don’t scale and then scale the things that you’re doing. That’s the phrase I always come back to.
Brian Tolkin: Exactly.
Lenny Rachitsky: This reminds me of a hot take that previous podcast guest shared in a newsletter post Casey Winters. He talked about that operations is usually, and it’s a hot take, that operations is a sign of inefficiency and over time your job is to squeeze that away and make it product software as much as possible. Doesn’t mean you always get there. Thoughts?
Brian Tolkin: Yeah, I actually don’t. Fundamentally, it depends on what the operations is, but I don’t fundamentally disagree, but I think that the right lens to think about it is… And then those folks can move on to the next challenge. And so there’s always another hill to climb. And so I think that was one of the things at Uber and Opendoor where there’s this culture of on the ground experimentation that’s really helpful where we were just talking about driver onboarding may now be solved with technology so you have a few extra hours a day. How do we get better at optimizing the UberX system? How do you start tinkering with food delivery? How do you start thinking about higher capacity vehicles? How do you think about better feedback loop for those manual surge pricing toggles that we talked about? So I generally agree it just generally free across and solve more problems.
Lenny Rachitsky: It feels like a big part of this is making sure the operations teams understand there’s more opportunity, even if this ends up being automated, your job is not going to go away. We’re going to find something new to try and experiment and do things that don’t skip.
Brian Tolkin: Yep.
Lenny Rachitsky: Awesome. Okay. Going a completely different direction.
Brian Tolkin: Great.
Lenny Rachitsky: I hear you’re very good at product reviews.
Brian Tolkin: Okay.
Lenny Rachitsky: A few people told me this. I’m curious how you set up a product review and any things you’ve learned, any tips for how to run an effective product review?
Brian Tolkin: That’s very whoever mentioned that. But yes, big fan of doing them actually in particular to maybe bridge the conversations in companies that have ops driven cadences or start out very ops driven because the cadences can sometimes be different. And so the operational cadences that you might have something like a WVRA weekly business review may not be conducive to always picking your head up and saying like, Hey, where’s the product going on a slightly longer timeframe. And so I think product reviews in general for all companies are probably really helpful, but actually in particular for some of the product and operations led companies. In terms of things I’ve learned, I think being really intentional about what the goals are, I think it’s okay to say that there are two goals, a goal of accountability and inform to an audience.
But also most importantly, I think this is the primary goal is to help make the product better, to help the teams think through a problem and to have that, again, back to our earliest conversation, be a very intellectual conversation about the work and how to make the product better and not super scary. Product reviews hopefully are not feeling like firing squads. That’s a scary environment to be in and not necessarily one that’s conducive to how do we make the product better. Obviously sometimes the conversations have to get a little intense, but in general that’s what we’re shooting for is something that helps the team go back and think through how to make the product better.
Lenny Rachitsky: So the two goals you try to communicate for your product reviews, accountability slash informing people what’s happening, but also just like we are here to make the product better and setting that context. Yep. Is there anything you’d do specifically to make it not feel like a firing squad? Like you’re coming in here to be attacked and criticized? Do you set context at the beginning of the meeting? Is this just a part of the culture?
Brian Tolkin: Yeah, I think definitely part of the culture, but also I am a firm believer in general that the people closest to the problems also have the best context to solve that problem. And so as a more senior voice in the room, often the job is probing, asking questions, throwing out ideas in a way that says like, hey, this is an idea. This is not a mandate, this is a thought. And if there’s context missing that would inform the product direction, then providing that context in not a question asking sense, but hey, this is context that you might not be aware of. And so I think it’s all in how you show up as a leader and what that looks like in terms of probing and pushing the team on dimensions that they may not be thinking about. And then understanding that the team is bringing a perspective that you don’t have, which is they think about this problem 40, 50, 60 hours a week, and you might think about this problem three hours a week. So you bring them a breath, the team brings a depth in haven’t been there yet.
Lenny Rachitsky: I don’t know if you heard Dharmesh Shah’s episode or his thing on flash tags. Have you seen this?
Brian Tolkin: I have not, no.
Lenny Rachitsky: Okay. He has a whole system. So you talked about how as a leader you want people to not take everything you tell them as feedback as I need to do this. So he has a whole set of hashtags that communicate how important this is to him from hashtag FYI to suggestion, to a plea.
Brian Tolkin: Yes.
Lenny Rachitsky: A plea to you.
Brian Tolkin: This was actually explained to me. I don’t think I’ve seen the original source, so I’ll go back and watch it, but this was explained to me as I’m actually a big fan. I think that’s great.
Lenny Rachitsky: Yeah, just get everyone on the same page. Okay. Maybe one last question here. Who do you try to invite to product reviews? Do you have any frameworks and ways of thinking of who to invite, who not to invite?
Brian Tolkin: Yeah, good question. We, I would say have oscillated over time, but in general, big subscribers of the best conversations happen when they’re relatively small, so try and keep it under 10. Could be wide distribution of the document. The artifacts created are actually really powerful and they’re powerful for the whole team to understand. And secret power is they’re very powerful for new people who are onboarding to be like, here are the last 20 product reviews, you’ll get a pretty good idea of what’s going on. But generally the conversation itself try and be relatively tight. We try to keep it under 10.
Lenny Rachitsky: And these artifacts, you mean the recordings of the meeting that people can watch or?
Brian Tolkin: Yeah. Or just the document depends on what the company culture is, whether you want to record it or just have the document either way.
Lenny Rachitsky: And then is there some specific cadence you operate on? Is it like a weekly product review that people can sign up for? Does every team, how do you like to set this up the cadence?
Brian Tolkin: Yeah, obviously it scales are with the size of the company. For us right now, what’s working well is signup cadence. We have two slots a week that anyone can sign up for as their product area needs it. And then if there’s something that we would love to see that we haven’t seen in a bit, we do a little bit of all in telling to make sure that the work is generally cycling through on a quarterly basis.
Lenny Rachitsky:
Adjacent topic. I hear you’re a big fan of drops to be done, which is okay, so it’s a fun recurring topic on this podcast. We’ve had many people that love it, many people that hate it. I love seeing both sides of it. I love that you find it helpful and you implement it at Opendoor. I’d love to hear just how you actually apply it at Opendoor, what you’ve learned about how to apply jobs to be done effectively.
Brian Tolkin: Yeah. I think like all frameworks, the right answer is to pick your set of frameworks, have more tools in your toolbox, and then actually understand when and how to apply them. So we try to avoid being a hammer and everything’s a nail. We try to for course the framework if it’s not working. But I think what I really like about it is it forces you to put yourself in the customer’s shoes. I think in a slightly deeper way and be a little bit more empathetic when I think about building at Opendoor versus say building at Uber or when you are building at Airbnb is we are not… Most people at Opendoor, we don’t have homes to sell every week or every month, nor do we buy homes every week or every month. On average in the US is something people do once every seven years.
I’m sure the average at Opendoor is something similar, and so it’s a little bit harder to be a customer. I took Uber every day. You probably use Airbnb a number of times a year. And so in some sense for some of those companies, you can bill for yourself, you would intuit the job to be done because you’re just doing it for yourself. We don’t necessarily have that context. And so a framework that forces us to be really thoughtful and intentional about how a customer might perceive our product is really helpful. The other thing that I like about it is the canonical version of it encourages you to think about the context in which the user’s operating or the other things outside of your product that they might be going through.
And in our case, the home buying or selling journey often is a certainly multi-week, if not multi-month or multi quarter journey with a lot of complexity and a lot of conversations outside of our product. You may be talking to an agent, you may be talking to a friend, you may be driving around the city trying to find a house, and the framework is very flexible and encouraging of saying what is actually the job to be done of this user when they’re thinking about our product and what is the context in which they’re operating.
Lenny Rachitsky: I’d love to go one level deeper to talk about how you actually implement it. Do you have templates of, you have a startup project as a blank, I blank, blank, blank? How do you…
Brian Tolkin: I would say we’re medium rigorous on template standardization or adherence. So we do have a template, the standard product review template talks about jobs to be done and has a section for what is the problem statement and what are the jobs to be done.
Lenny Rachitsky: And this is a doc that when you’re coming to a product review, the person running it and coming is filling out this document?
Brian Tolkin: Correct, pre-filling it up, sorry, pre-filling it out. And again, I think we are not sticklers about always using that template, but I think the beauty of a template is yes, it sets expectations of what you expect, but it’s also just easier often for people to be able to work off something. And so yeah, it’s part of our product review template and then part of our planning process as well. Because we’ve used it for a while. I think there’s been an internalization of the culture where people will also just start commenting about it or writing about it and saying, hey, what is the job to be done here? Or what is the user trying to do? Which is maybe another colloquial phrase in it. And so, yeah, I think there’s a cultural seeping that has happened.
Lenny Rachitsky: From memory, just what is in this template. So what’s the phrasing that you try to use for setting up a problem?
Brian Tolkin: Yeah, yeah. I mean, the specific framing, I would have to go remind myself on the template itself, but generally it looks like context, problem, potential solution, risks, risks/premortal and measurement of success. And then we also try to bucket our product reviews by stage. So you could be in the ideation stage, which might look very different than at the very end of the process. Like, Hey, we’re getting ready to ship speak now forever hold your piece. Those two artifacts will also be very different.
Lenny Rachitsky: Okay. So it’s not as a blank the standard jobs to be done language, it’s not exactly how you implement it’s more just make sure we’re thinking of it what is the problem for the customer? What is the context of their problem?
Brian Tolkin: Correct. Yeah. Yeah, we’re not math living our template.
Lenny Rachitsky: Okay, awesome. Any other tips or lessons about just working well with this concept of jobs to be done? Maybe when you come into Opendoor and like, hey everyone, we’re going to be thinking this way. Is there anything there that would be useful to people if they’re trying to operate this way?
Brian Tolkin: Recognizing that correctly implementing a framework, any framework, but if you don’t in particular, we can talk about takes a little bit of time and getting used to and understanding. And so I don’t think you can just like, okay, we’re going to make the template and then that makes the content better. That just takes people’s content and they wedge it into the template. It’s actually the cultural internalization of like, hey, this might be phrased as the job to be done, but is this actually the job to be done? Let’s talk about why the customer might be in that situation or not be in that situation. Or I think the job to be done might actually be something else.
Or you might say, hey, the job to be done is maybe an early day version would be the job to be done is to get an offer from Opendoor. And it’s like, well kind of, but the broader job to be done might be price discovery for the customer. And so you can have a rich conversation where it’s like, well, one might be a little bit influenced by our business goals. I don’t think you just run around and people are like, yeah, I’m going to sell my house, but my goal is to get an offer from Opendoor. And so that’s like, okay, the template might be the same, but it’s actually the content that takes a little bit of cultural instantiation.
Lenny Rachitsky: Got it. And it sounds like people talk from what is the job to be done that feels like a core part of the way you think about it. What is the job to be done? Just that language alone feels really powerful. Is there a resource or a book that you point people to help your team learn about the jobs to be done work? Is there one thing you find useful?
Brian Tolkin: Not about jobs to be done. I do a lot more pointing people towards internal examples of where I saw other PMs maybe do as well or blogs and stuff. But your blog is a column when we pass around, not about jobs to done, but just about many topics.
Lenny Rachitsky: I’m flattered. Thank you.
Brian Tolkin: Yes.
Lenny Rachitsky: I really appreciate that. I was also thinking as you were talking, you’re friends with Kayvon and jobs to be done on Twitter was quite the journey for them, traumatic for a lot of people. I think it went very far to the extreme of the-
Brian Tolkin: Yeah. I think they’re more dogmatic about it.
Lenny Rachitsky: Very dramatic. And so I guess it’s a lesson here, maybe don’t take it that far.
Brian Tolkin: Yeah. And I think it probably, and I don’t know if Kayvon would agree with this, I imagine he would, the generalized version of you pick the right framework for the right job and if you say there’s one framework to rule them all and this is the only framework that works and then of course every problem into it, then we chop.
Lenny Rachitsky: The way I think about jobs to be done is exactly the way you’re describing it where it’s just think from the lens of the job to be done for our customers. So for my newsletter, what is the job to be done of my newsletter to help you become better at your job as a product person building product. And that actually ends up being really helpful. And it feels like that’s the way you guys think about it at opening.
Brian Tolkin: Yeah, absolutely. And you’re crushing it by the way.
Lenny Rachitsky: Thank you. So are you. You talked about, I’m going to go into another question to deflect your compliment. You mentioned that Uber, there’s a million transactions happening every second, it’s massive scale. Opendoor is completely different. You have very few very large transactions. Yeah. I’m curious how you do experiments, if you do experiments, do you do A/B tests? What have you learned about just how to think through low sample sizes plus A/B testing?
Brian Tolkin: Yeah, very hot topic of conversation. We do A/B test. It is obviously the gold standard. And so we do as much as we can. Of A/B testing there are parts of our funnel and flow that have more volume than others. So top of quality testing easier than down funnels, A/B testing purely product or tech features easier than A/B testing processes, operational processes. But you’re totally right. We are not doing hundreds of millions of transactions a year. And so experimentation can be more challenging. And so I think one way to think about it is A, acknowledge the problem, which just to say don’t, and we’ve made this mistake many, many times, but don’t just force yourself into A/B testing without running the power analysis and say like, hey, are we going to get results? What is the size that will detect and what is the runtime of that experiment?
And be honest, is that acceptable? A second lesson here, is there certain experiments that are important enough and it’s hard to triangulate signal in any other way that you may say a six-month runtime is an acceptable outcome and we are going to start it in June and we will be smarter for it for 2025 planning and we’re going to set it and forget it and we’re grateful we did, and that’s okay. But the only mistake here is thinking you’ll get an answer in a month when you won’t, and then pretending you do and then waking up a month later and being like, “Well, it was insignificant and this and that.” We could have known that. And then the third thing is, and experimentation is all about increasing your conviction in the problem or the solution. So the generalized version of the statement is, if there are parts of your funnel or flow that are low end and you can’t run a canonical A/B test, how might you otherwise increase your conviction in the solution that you’re building?
And there turns out there are a decent number of other ways to do that. The first best, most obvious is talk to more customers. But there are other statistical techniques that again, aren’t as rigorous or good, but may be possible. You may be able to use observational data, you review with diff and diff, you may be able to look at sister cities or twin cities. You may be able to segment by geo, you may be able to reduce your power and say, hey, we’re going to run at 80% confidence for all of our experiments instead of this traditional 95% because that’s a worthy trade-off. And if we’re wrong one more time out of 10, that’s okay.
You can do a long-term holdout to match your intuition. And so there’s a lot of other techniques to hone your intuition. There’s a lot of other techniques to build conviction and confidence. And so we try to be very creative on doing that. And then the last bit I would say is if you’re not going to get significance, if there’s no other techniques at your disposal, then sometimes you just got to trust your intuition and ship it. And if that’s what you believe, then that’s what you believed and you shouldn’t spend time trying to get false precision.
Lenny Rachitsky: I want to spend more time on that last point, but real quick, the power analysis you talked about, there’s people, don’t know, there’s calculators out there that you could just plug in, here’s how much traffic I’m getting, here’s how much of an impact difference I want to see, here’s how long it’ll take to find out.
Brian Tolkin: Yep, exactly. Totally. And some of the calculators are great where you can also plug in the traffic and your acceptable runtime and it will tell you the minimum detectable impact and then you can gut check your own intuition. So you can play around with that.
Lenny Rachitsky: Awesome. We’ll try to link to one of those in the show notes. So, on the intuition piece, is there anything more there? Just like how you think about when you run the product team, just how you recommend people leverage intuition versus not. Because some companies are like, we’re just going to trust the data. I don’t really trust your opinion. You don’t know this customer exactly. You talked about Opendoor, I’m not buying houses myself, so I don’t know how much I can trust my intuition. Just what’s your general advice to your product team of how to think about their intuition and when to rely on it versus not?
Brian Tolkin: So, at Opendoor, for example, I’d say on the relative spectrum, we’re quite data-driven. And then it’s when we come into this challenge where we say, okay, that is another technique or tool in the toolbox. I think the generalized version of that is customers, products, people can surprise you. And so this happens all the time for people who build products. I’m sure you’ve got great stories from Airbnb where you saw something, put it out there just was very-
Lenny Rachitsky: All the time.
Brian Tolkin: All the time. And so I think there’s definitely a humility to say if you can, if it’s relatively easy to test your assumptions or test your hypotheses. That is always better to gut check yourself. And that takes a little bit of humility to say that, but we’ve all been wrong plenty of times. But if that’s just not on the table, I think the reality is you can’t pretend it is. And sometimes you got to use taste and judgment and then you say, okay, what is my conviction level and do I have just medium, low or high conviction? And if I have anything low or medium conviction and it’s a decision of consequence, I should talk to more customers, check it with another person and see if their intuition matches. Something that gets me personally to the high bucket category.
And then I think the last part, which is some part of experimentation is if you just ship something because it’s your intuition or it’s where you want to see the product go, do you have a reasonable feedback loop to understand whether or not you are correct? So that could be customer support or ticket volume or feature adoption, whatever the case is, it may not be an output metric in the traditional A/B test, but some more rigorous system that says, hey, I had this hypothesis, we just shipped it for x, y, z constraint reason for red.
Lenny Rachitsky: I think that’s awesome advice. I agree with everything you’re saying. You mentioned this word humility, and it is a good segue to something I want to talk about, which is Zillow. One of the most interesting things that’s happened in your space is Zillow basically decided, hey, we’re just going to do what Opendoor is doing, they launched it, you’re basically frenemies for a while, and then they’re like, no, it’s not working. Now you partner and now you work with Zillow on this stuff. So are you able to share what went down there with the story of what happened, how it went, and where things are at now?
Brian Tolkin: Yeah, I mean we do partner with Zillow. Zillow’s been a fantastic partner for us and we’ve really enjoyed a working relationship with them. I think when you think about it, they have tremendous amount of reach and audience and all these online platforms have tremendous reach and audience. And we happen to have a fairly unique selling solution. And so there’s a nice, not to use a business school word, but there’s a nice synergy so to speak, between a high intended audience who’s doing a lot of browsing and searching and discovery and starting their process on one of these online platforms and what we offer, which is transaction services that allow people to actually move particularly on the seller side. And so there’s just a pretty nice symbiotic relationship there with the Zillow’s and the regimens of the world. And so both of those have been great partners for us.
Lenny Rachitsky: What do you think Zillow maybe underestimated or didn’t get about the space that made it harder than they anticipated? That seems obvious. Of course, let’s go down funnel, let’s just do it all. And they’re like, “Oh, shit, not working.” What do you think they didn’t get or what do you think they missed?
Brian Tolkin: I guess continuing on the humility point, I won’t necessarily pretend to be in their shoes, but I will say the business is challenging and it’s complex from a number of different dimensions. It’s not a traditional software only product, but you have to be really good at pricing. You have to be really good at product, you have to be really good at the operations. You have to be really disciplined at risk. You have to be really good in the capital markets. And so you have to put all of these functions together to build a vertically integrated product. And that’s the reality. And so that is something that’s been in Opendoor’s DNA from day one because we started with a vertically integrated product. And so we can’t deliver unless we have all of those things. Right? And so I think that’s something that continues to help us to this day, is that vertical integration requires all of those pieces coming together.
Lenny Rachitsky: That makes a lot of sense, and I think it’s a good reminder of there are adjacent markets and businesses that always feel like, “Oh, we could expand to that someday.” Such a big opportunity, this business could be so much bigger. And then you realize your business is completely not set up to operate this way. Zillow is very software driven, just I am not going to simplify what they do, but it’s a website, very software. And obviously as we talked about Opendoor, a huge operational component. And then as you said, the pricing piece and the debt stuff.
Brian Tolkin: Yeah. Yeah, totally. Yeah.
Lenny Rachitsky: Yeah. So I think it’s a really good reminder that just when you’re taking on something completely different, it may not fit into the way your company operates and partnering makes sense. Anything else there that’s interesting to share around the Zillow thing? I guess one is maybe it was just like, I imagine it was very stressful. Zillow’s getting into it. “Oh, shit what are we going to do? They got all the traffic.” Yeah, anything there?
Brian Tolkin: Yeah. I mean, it’s certainly stressful. I think in general we try to live by whether Zillow or anybody else being competition aware, but not necessarily competition focused. And the reality is vast, vast in our space. A vast majority of people still move the traditional way. And so this isn’t something where it’s like the size of the prize isn’t particularly large enough short or anything like that. The reality is it’s the largest asset classroom in the United States, and if we just stay super focused on, hey, who are the customers that we serve really well that we talk to every day, there’s a little bit of confidence that comes from being able to stay focused on that regardless of the competitive environment, again, because it’s not like the market is fully saturated.
This is the same thing back in the Uber days as well. It’s like transportation is almost infinitely large. And so, yes, it feels like there’s heated competition between Uber and Lester or whatever back in the day, but the reality is there’s plenty of trips that happens and people need to get around sitting in plenty of different ways. That’s neither Uber and more Lyft. And staying focused on how you can develop for your customer, I think is the best way to focus.
Lenny Rachitsky: There’s a podcast that will come out before this episode with Jeff Weinstein from Stripe who’s building Stripe Atlas. They had a similar experience with AngelList launched a direct competitor to Atlas, and then they realized Atlas is so much better. Forget it. We’re just going to send everyone to Atlas.
Brian Tolkin: Really?
Lenny Rachitsky: Yes. And I think it’s the same exact lesson that if you just stay focused on jobs to be done, let’s say, of what is the job to be done and do the best possible job, and knowing that the market is much bigger, that you’re not really competing with someone else, another company, it’s the default behavior in your case. It’s like people are just buying their house the old-fashioned way. That’s the actual competition.
Brian Tolkin: Exactly. Yep.
Lenny Rachitsky: Yeah. Okay. So, along these lines, something else I’ve heard that you’re very good at is staying very calm under pressure and staying very levelheaded when things are really crazy. This is something that a lot of people are not good at, especially leaders. They stress everyone out things. You go crazy, they don’t create a good vibe. And then two, something people want to get better. Leaders and non-leader are like any lessons, anything you’ve learned about just how to develop this skill?
Brian Tolkin: I think part of this may have been sharpened in the early days of Uber. Everything felt like a fire drill all the time. So the only way to operate, but I think you almost hit the nail on the head in the question, which is a little bit of an intellectual answer of when you reflect the stress onto your teams, everybody tenses up and tightens up. And so it counterintuitively doesn’t produce better outcomes. And so I think the other reality to remind ourselves, and these are a bunch of mantras that just are helpful in these moments, is you’re never as good as you think you are. You’re never as bad as you think you are. And so that more even keeled demeanor, I think allows you to have a clearer head when you’re operating under the pressure and to think more clearly.
I think one of the maybe least helpful answers, but unfortunately, the reality is you got to be in some stressful situations to also have the perspective that cycles past the things past. And that remaining calm is what matters. And so maybe the advice there is reflecting on one of these situations happen, exposing yourself to them, not running from them and then learning from them so that the next time it comes around you can say, Hey, I’ve been here before. I’ve slept on the floor in China before launching uberPOOL and thinking we’re going to miss a launch deadline. And what were the tools in my toolbox and my toolkit that showed me that in terms of getting it done or not, and what were the lessons?
Lenny Rachitsky: I love that. So part of it is just go through this experience many times and you’ll start to realize, okay, it’s not actually going to be as bad as people may think. You mentioned this toolkit instead of tools. Is there anything else there that you come back to that ends up being helpful? You mentioned this mantra of it’s never as bad as people think it is, it is never great as people think it is.
Brian Tolkin: Yeah, I mean, I think exposing yourself to other people’s stories or however you may learn is really, really helpful. So again, whether it’s your podcast or books or biographies or one of the podcasts that I love is Founders Podcast, which talks about historical famous entrepreneurs. And obviously these are elevating very famous people already, but there’s a lot to learn from a lot of these stories as well and understanding that the journey in pack is nonlinear, it never is for anybody. And so I think being able to expose yourself to other stories that even may if you don’t have those personal experiences and then understanding how others navigate.
Lenny Rachitsky: Got it. So just hearing of other people’s crazy experiences and building on this muscle of like, okay, they’ve gone through crazy stuff, things work out.
Brian Tolkin: Yeah, totally.
Lenny Rachitsky: We’ll make it through. Okay. I have this note here that I think either someone mentioned about you or you may have mentioned that product is finding the kernel of truth in a sea of ambiguity and signals. Does that mean anything to you?
Brian Tolkin: Yeah, absolutely. I mean, I think in most organizations and to do the job effectively, you’re going to get signals from everywhere and good ideas come from everywhere. It may be your CS team or CX team. It may be a customer directly. It may be a conversation you had. It may be a YouTube video you watched that sparked an idea. It may be feedback from an executive, it may be whatever. You went out and did a field visit. You are going to get a lot of inputs around what people think about your product, what people think you should do next. And I think that the core job is to understand what really matters, right? What is noise? What is a good idea, what is a suggestion and what is back to the jobs to be done for what is really going to move the customer forward?
And unfortunately that means saying no to maybe what sounds like some good ideas along the way, but if you can really figure out this is really what matters, that’s the core part of the job. It dovetails even back to our earlier conversation. In the early days of building tech and ops companies is where’s the tech leverage? Same question, where’s the kernel of what really matters that tech can uniquely solve? And let’s go do that and be comfortable with other fires maybe burning. That’s what really, really, really matters. It’s a hard discipline.
Lenny Rachitsky: I love that. If there’s not an example, that’s totally fine, but when you talk about this finding this kernel where tech could be highly leveraged, is there any example that comes to mind of that working out really well?
Brian Tolkin: I mean, I think back in the Uber days, I think it was like, hey, we’re not going to build sophisticated tooling infrastructure. We’re not going to build a centralized growth team. We’re not going to build any of that. Because if you think about the early Uber network from the simplest form, you’ve got a rider and a driver and you need to connect them, price the transaction and issue some receipts probably and collect payment. So it’s like, okay, do we do that really well? And until we do that really well, all the other stuff is noise. It’s immaterial how efficiently we answer support tickets. That’s not critical. And so now it’s super critical, but in the early days it’s not that critical. And even the customer acquisition costs may not be super critical, in this case it’s growing rapidly on the things. And so pouring fuel on the fire may not be super efficient there.
So I think that’s a very good generalized example. One other tip that maybe is helpful that I frankly constantly work on and try to get better at is all these ideas and feedback that comes from everywhere, make sure it’s written down for a number of reasons. One, you can then go reference it, but two, part of the job is making sure the people who present those ideas are heard and respected and know that it’s at least somewhere where it was considered. And then you can look at it all and say, okay, but what actually really, really, really, really matters here? And yeah, that’s another tip.
Lenny Rachitsky: When you say written down, is there tools you find really helpful here? Is it just put it in a big doc that we’re keeping? Is there anything you find to actually operationalize that?
Brian Tolkin: I’ve seen different companies do it differently, but wherever you tend to try and keep a backlog, whether that’s a Google sheet or your actual backlog in Jira, whatever you use, but at least it feels like, okay, the context was captured, and the idea is there.
Lenny Rachitsky: Awesome. Okay. I’m going to take us to a recurring segment on this podcast called Failure Corner.
Brian Tolkin: Okay.
Lenny Rachitsky: Is there a story you can share of a time you failed in your career had a big failure, and how that experience made you better?
Brian Tolkin: We can talk about the very early days of uberPOOL, and the first launch, if you will in San Francisco. So, carpooling product, multiple riders in a Mercedes car. And we had this idea that it would be effective for commuters, this was very, very early days. And so part of the launch was, okay, we’re going to beta it with just some popular commuting corridors with specific companies or maybe the marina to Google or whatever and try and match people according to what their companies and that’s how we’ll drive liquidity. And we very quickly realized that back to what the kernel of truth is here is liquidity is the only thing that matters. And there just wasn’t enough. There was never going to be enough to do this company based thing. That wasn’t the strategy that was going towards from us. And so the reason I don’t know if it’s a full failure is maybe this is true all failures, you learn from it, you pivot and you go on to the next thing.
And obviously we did that and then spent a lot of our time and effort trying to say, okay, what are the bounds of liquidity and driving liquidity that we can do to understand what the most important or what the limits of the product are. So as an example, we launched and maybe people in San Francisco. Remember this $5 anywhere in San Francisco, we work for promotion, which is obviously a great deal, obviously costs a lot of money, but the whole idea here is like, oh, okay, if liquidity is what really matters, if we were to juice that and really drive liquidity, how high can our metrics get? And then we can go chase more sustainable ways to do that. But it was a interesting fail case from launching and learning to say, hey, this initial strategy just isn’t going to work, we got to go. And any part of it was a hedging strategy where with a small audience and there’ll be a beta population, it’s like, well, this one you just got to go.
Lenny Rachitsky: I think a lesson there is also don’t overthink it, don’t try to get too cute. Just like we’re trying to make a perfect beta test versus realizing, okay, we just need a lot more people in it. Also, your $5 promotion made me think of the early promotions of the ice cream and the bunnies delivery and all that stuff.
Brian Tolkin: Yeah. That was by the way, a example of fully distributed, the benefit of having those early Petri dishes. Someone a local marketing manager like, Hey, this would be fun. Yeah, that would be really fun. The platform can support it. And those promotions were fantastic. And it started out, I can’t remember if the first one was ice cream or puppies, I think it was ice cream. But yeah, branched into all sorts of stuff. Boats, ice cream, puppies, kittens, I think, and all credit goes to local ideas of inspiration just being focused on trying to grow within.
Lenny Rachitsky: I love that we’ve circled back to the beginning of our conversation, product and ops working together, the benefits of both. Before we get to a very exciting lightning round, is there anything else that you wanted to share? Any last nuggets of wisdom that you think might be useful to people when they’re trying to build product companies teams?
Brian Tolkin: This was great. We covered quite a bit of ground. I think the only, I don’t know if this is a generalized wisdom, but something I’ve been thinking about as my career has progressed a little bit, especially building out proper organizations, especially as more tools come online, it’s very clear that there’s different types of PMs and we spent a lot of time talking about once we can operate in the physical and the digital or the product and operations worlds. But even within that, there’s more technical PMs who grew up in minute engineering discipline. There are people who came from ops and there are people who came from design and grew up in a more user experience background.
And one thing that I’ve been moved on as is build out the team is thinking similar to a product roadmap is it’s not really about is this person good or bad or whatever, it’s is this person’s skillset and context to the problem that is really needed. And so back to that conversation on, hey, where do we get tech leverage? It’s like, Hey, is this person who has this unique skillset as a PM for this problem type? I don’t know if that’s helpful, but it’s something I’ve been spending a lot of time thinking about, especially this view job posting niche view product manager. Or it’s actually like, well, how can we be a little bit more thoughtful about what the actual skillset needs off this type of team?
Lenny Rachitsky: Awesome. It’s like a person product fit.
Brian Tolkin: There you go.
Lenny Rachitsky: And I think it’s because a lot of companies hire generalists and they’re just like, we’ll hire someone smart, ambitious, and with experience and general experience and then we’ll put them on different things. So I think these are two different philosophies and it probably makes a lot of sense for an Opendoor with very unique type of business, with very specific skills that are necessary to be really good there. Okay, amazing. Brian, with this, we’ve reached our very exciting lightning round. Are you ready?
Brian Tolkin: Let’s do it. Can’t wait.
Lenny Rachitsky: Let’s do it. First question, what are two or three books that you’ve recommended most to other people?
Brian Tolkin: Shoe dog, Black Swan, Design of Everyday Things, and for a fun one, Shawn Theron.
Lenny Rachitsky: Amazing. Four books for the price of two to three. I love it.
Brian Tolkin: Apologies. I’ll stick to the rules.
Lenny Rachitsky: No, no, there’s no rules. There are no rules.
Brian Tolkin: There you go.
Lenny Rachitsky: Next question, do you have a favorite recent movie or TV show that you’ve really enjoyed?
Brian Tolkin: I like the sports docu ones on Netflix, so Full Swing, Drive to Survive, Break Points, tennis, golf, F1, all of them.
Lenny Rachitsky: And wasn’t there that Nike documentary recently with Ben Affleck?
Brian Tolkin: There is, which I have not seen. So if it’s good, I don’t know if that’s a recommendation or just an acknowledgement.
Lenny Rachitsky: It’s worth watching. If you like Shoe Dog, I feel like you’d enjoy it. It was entertaining Michael Jordan, things like that. Next question, do you have a favorite product that you have recently discovered that you really love?
Brian Tolkin: So we just got a puppy and we are about to have our first child. And so all of my purchases recently are puppies and children focused. My buddy gifted us the Phi collar for our dog, and so we’ve been really enjoying that. Another one as I’m getting busier for news and stuff is Particle, which is a great news aggravation tool, AI news tool.
Lenny Rachitsky: Cavan’s wife’s business. I am a huge fan, actually I think it just came out of beta and now it’s like a full app that anyone can download. I just actually installed it yesterday again and I love it. I get these pushes every few… I don’t know, it’s like a couple of times a day of just like, here’s what’s happening. Also, congratulations I should have said on your pending child.
Brian Tolkin: Thank you.
Lenny Rachitsky: Lucky for you. I have a newsletter post with all the products you should buy, it’s called New Parent Gift Guide for Product Managers.
Brian Tolkin: Love it. I will definitely probably buy all of them.
Lenny Rachitsky: If you don’t already have them all. And now everyone’s probably sending you their spreadsheets of all their favorite stuff there.
Brian Tolkin: Exactly.
Lenny Rachitsky: Okay, next question. Do you have a favorite life motto that you often come back to share with people either in work or life?
Brian Tolkin: Well, mostly just stay curious.
Lenny Rachitsky: Stay curious. I love it. Two more questions. Who has most influenced you in the course of your career?
Brian Tolkin: One of the people who inspired me very early on in my product journey. I’ve been fortunate to have a number of very good mentors and obviously we talked about earlier while I was founders of books or me a lot from other people’s journey. But one person who was personally important to me early in my product journey and very supportive of this guy named Jeff Holden, who was the chief product officer at Uber back in the day, and is like a young PM transferring into product really took me under his wing. And I think I’m forever grateful for that, for Jeff, for helping grow my career, but also try to pay it forward a little bit in terms of people who are going in the career. That was really meaningful for me.
Lenny Rachitsky: Last question. I hear that your interview at Uber was pretty wild. Can you tell that story?
Brian Tolkin: Yeah, I can. So, long story short, I was starting a company my senior spring before graduation and we had to go our separate ways. So I hadn’t done traditional recruiting ever. And my buddy called me up and was like, “Hey, we’re looking for smart, hardworking people at this Uber thing. Are you interested?” And quick side note, I had actually done some very early diligence work on these taxi apps back in 2011, looking at the time was Uber Cab and Cabilis and Taxi Manage can probably names nothing these days. And so I knew what Uber was and so I said, “Yeah, sure, I would love to.” And so I had the first round of interview went well, and they said, great. And at that stage is come on onsite, the full enchilada one works, and this was post-graduation and was helping out some companies but didn’t have a full-time job.
So I said, hey, I’m pretty flexible. How about next Tuesday? I said, great. So we scheduled it and then on Friday or Saturday or over the weekend I looked, I’m like, oh, Tuesday’s, July 4th. I scheduled my interview for July 4th. And so I called my buddy and I’m like, hey, I am so sorry. I don’t want to make people in on July 4th. Should I cancel? Should I reach out? Everyone’s accepted. Whatever you do, do not cancel your interview. Like, okay, I’ll be there on July 4th. And so I went in to the office on July 4th and there was a very small handful of people there. It was actually launching that day, was launching Uber’s second ever product type, which was Uber SUV. And I had this, I think it was probably five hour gauntlet interview on July 4th from noon to five and missed my July 4th barbecue. And it was quite the experience, but I think maybe set the stage for some of the early days chaos. I’m very glad I didn’t cancel the interview.
Lenny Rachitsky: And was Travis involved in that interview or is it just-
Brian Tolkin: Travis was involved in the interview. She was one of the… I think there were four or five people, and the two who were generally guiding my interview process and Travis and one other person starting that day. And part of the gauntlet interview was a simulation of the job, if you will. And so some of that was building some novels on the computers and some of it was writing potential emails to drivers and if she had the driver come in and you did chat, so I was in this room by myself typing away on the first part, which was building the model. And I hear him knock on the door and Travis comes in and he just sits down and says, “Hi, I’m Travis.” I don’t know who you are, but I’m Brian, and we have a 45-minute chat, or maybe it’s been about half hour, 45 minutes. And clearly I’m not producing the work that I’m supposed to of the interview.
I’m supposed to be building this model. I’m supposed to email it back to the person who sent it to me. Clearly I’ve done nothing chatting with the CEO. And I hear a knock on the door and the door opens and the person sees that I’m talking to Travis, “Oh, continue, continue” and it was very good. Also, pretty intense conversation with Travis that definitely set the expectations working there.
Lenny Rachitsky: And clearly worked out. And Travis was happy. Is what-
Brian Tolkin: I hope so.
Lenny Rachitsky: … I imagine.
Brian Tolkin: Yes.
Lenny Rachitsky: Amazing. Brian, thank you so much for being here. We went through everything that I was hoping to get through. Two final questions. Where can folks find you online, and is there anything you want them to check out that you might be up to? And how can listeners be useful to you?
Brian Tolkin: Super kind. They can find me online on Twitter, LinkedIn, both just Brian Tolkin, my name. In terms of being useful, if you have a home to sell, feel free to go on to Opendoor. More importantly, if you have feedback on the product, but would love to hear it. Otherwise, any feedback on what people liked or would love to learn more about from what we chatted about would be super great. So I’d love to hear from you.
Lenny Rachitsky: Brian, thank you so much for being here.
Brian Tolkin: Lenny, I really appreciate it. This was great. Bye everyone.
Lenny Rachitsky: Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.
Glossary
| English | 中文 |
|---|---|
| accountability | accountability(责任制/责任归属) |
| Casey Winters | Casey Winters |
| Cavan | Cavan |
| Dharmesh Shah | Dharmesh Shah |
| diff-in-diff | diff-in-diff(双重差分法) |
| EPD | EPD(Engineering, Product, Design,工程、产品、设计团队) |
| Fi | Fi(智能宠物项圈品牌) |
| GM | GM(General Manager,城市总经理) |
| holdout | holdout(保留组/对照组) |
| Jeff Holden | Jeff Holden(前 Uber 首席产品官) |
| jobs to be done | 待完成的任务(jobs to be done) |
| Kayvon | Kayvon(指 Kayvon Beykpour,前 Twitter 产品负责人) |
| on-site | onsite(现场面试) |
| onboarding | onboarding(入职/上线引导流程) |
| Opendoor | Opendoor(在线房屋交易平台) |
| Particle | Particle(AI 新闻聚合应用) |
| power analysis | power analysis(功效分析/统计功效分析) |
| product operations | 产品运营(product operations) |
| product review | 产品评审(product review) |
| surge pricing | surge pricing(动态溢价/ surge 定价) |
| Taxi Magic | Taxi Magic(早期打车应用) |
| Travis | Travis(指 Travis Kalanick,Uber 联合创始人兼前 CEO) |
| uberPOOL | uberPOOL |
Reformatted by reformat_english.py
从规模化 Uber 和 Opendoor 中学到的经验 | Brian Tolkin(Opendoor 产品负责人,前 Uber)
文字记录
Lenny Rachitsky: 你曾在两家将产品与运营结合得极为出色的企业工作过。
Brian Tolkin: Uber 始终有一种理念,Opendoor 也同样如此——产品和运营就像双涡轮喷气式飞机的两个引擎。如果需要的话,单靠一个引擎也能飞一阵子,但只有两个引擎协同运转,才能实现最高效的运行。
Lenny Rachitsky: 有过运营经历,对你成为更好的产品负责人有什么影响?
Brian Tolkin: 它让我对业务实际如何运转有了非常深入的理解。这为你接下来思考”我们到底想用更具扩展性的技术方式去构建什么”奠定了相当好的基础。
Lenny Rachitsky: 我还听说你非常擅长在压力下保持冷静。
Brian Tolkin: 当年在中国上线 uberPOOL 的时候,我睡过地板。当你把压力反射给团队时,所有人都会变得紧张,反而适得其反,并不会带来更好的结果。
Brian Tolkin 的职业背景
Lenny Rachitsky: 今天的嘉宾是 Brian Tolkin。Brian 目前是 Opendoor 的产品和设计负责人。在此之前,他在 Uber 工作了近五年,作为第 100 号员工加入。当时 Uber 还没有 UberX、uberPOOL 或任何拼车产品。他实际上是在 Uber 的运营团队起步的,后来转入产品部门,最终负责 uberPOOL 的产品和上线工作,并将其推向全球。他还在 Uber 创建了产品运营(product operations)职能,在这个角色真正成为一个行当之前就开始做了,这是我在我们之前的聊天中才知道的。在我们的对话中,Brian 分享了大量关于构建具有重度运营成分的产品的经验教训,还包括如何做好产品评审、他如何落地”待完成的任务”(jobs to be done)框架以及 Opendoor 的成功实践。Zillow 试图与 Opendoor 竞争、失败后转为合作背后的故事。还有 Uber 和 Opendoor 早期的大量精彩故事,以及更多内容。Brian,非常感谢你来做客,欢迎来到播客。
Brian Tolkin: 谢谢,感谢邀请,很高兴来到这里。
Lenny Rachitsky: 首先,非常感谢 Kayvon Beykpour 帮我们牵线搭识。他说了各种各样关于你的好话,还给了我一些非常刁钻的问题来问你。希望你做好了准备。
Brian Tolkin: 太好了,放马过来吧。
运营经历如何塑造产品领导者
Lenny Rachitsky: 好,我想花些时间聊聊产品和运营这个话题。你的职业生涯从运营起步,在 Uber 你确实是从运营团队开始的,后来才转入产品。你还在 Uber 和 Opendoor 两家公司都工作过,这两家公司都有巨大的运营成分。我觉得这很少见——一方面,见证一家公司像 Uber 和 Opendoor 这样,在仍然是科技公司的前提下,凭借如此重的运营成分规模化到这样的高度;另一方面,一个人从运营起步,转入产品,最终走到你今天这个位置,成为一家非常成功的公司的首席产品官。所以我有一堆问题。也许第一个问题就是:有过运营经历,对你成为更好的产品负责人有什么影响?这如何改变了你作为产品负责人的工作方式?
Brian Tolkin: 从运营起步,让我对业务实际如何运转有了非常深入的理解。你真正在做的是日复一日地运营它,城市的业绩在很大程度上取决于你每天在一线投入的要素——比如那个周末是否下雨,下雨对指标是个不错的驱动力。但更重要的是每天与客户一对一沟通、引导司机入驻、处理客服工单——当时没有集中的客服团队,没有谁比你离客户更近了。所以我认为,真正理解什么能推动业务、与客户保持极度贴近——这个基础实际上为你接下来思考”我们到底想用更具扩展性的技术方式去构建什么”提供了相当好的起点。
产品团队对运营团队的常见误解
Lenny Rachitsky: 我看到很多公司——在 Airbnb 肯定也是如此——产品团队多少有些看不上运营团队。他们会说:“我们在做的事情将扩展到数百万用户,我们在做的事情将适用于所有人。“而那边有个运营团队,在做一些没法规模化的东西,还不停地让我们为他们的一个个临时想法做东西。你觉得产品团队通常可能忽略了或没有理解运营团队的哪些方面,从而能帮助他们用不同的眼光看待运营团队?
Brian Tolkin: 这是个很好的问题。我觉得 Uber 一直有一种心态,OpenDoor 也是如此——就像一架双引擎喷气式飞机,如果需要的话,你可以靠一个引擎飞一阵子,但两个引擎协同运转时才能最高效、最有效地运行,我觉得这确实如此。现实情况是,运营团队、本地团队迭代更快,能更高效地大规模接触和沟通客户,拥有很好的定性洞察。所以如果把它看成一种协同而非竞争,我觉得是非常非常有帮助的——就是说,我们怎么把日复一日在一线、在现场发生的洞察获取过来,帮助我们因此做出更好的产品?
一个旧金山的 PM,在 OpenDoor 的情况下不可能同时出现在 50 个市场每天看房,在 Uber 的情况下也不可能跑遍一千个城市去了解南美安全问题的细微差别,这根本不可能。但你能够做的是建立一个很好的关系和一个很好的反馈闭环,让那些深度理解这些事情的人帮助提供洞察。而 product operations 的诞生,其实正是源于这个洞察。
Lenny Rachitsky: 能展开讲讲吗?
Brian Tolkin: 当然。抱歉,我应该先定义一下什么是 product operations。在 Uber,基本上是这样的概念:我们有一个集中的——这是我在 Uber 职业生涯后期的事了——我们有一个集中的产品团队,主要在旧金山做东西,不完全是那里,但当时基本上世界各地也有,不过主要还是在旧金山。然后我们有一个全球分布的运营团队,两者之间的双向反馈闭环并不是很强。这个反馈闭环基本上就是:当旧金山的 EPD 团队做了新功能,我们怎么有效地把它推向全球市场;以及我们怎么有效地从全球市场获取输入来更好地构建功能。针对这个问题,一个解决方案——我们当时的方案——就是启动一个叫做 product operations 的新职能,它有独立的 accountability,汇报给运营线,但物理上坐在产品团队中间,工作方式也和产品团队成员很像,来帮助解决这个问题。
Lenny Rachitsky: 这可能是第一次出现……你们是发明了 product operations 这个职能吗?
Brian Tolkin: 我不这么认为,因为当时 Google 好像也有一个类似的职能,我不记得 Google 叫什么了,名字略有不同,但我跟几个在 Google 和其他几个地方做过类似角色的人聊过。所以我不能说是我发明的,其他人其实在 Uber 也在我之前尝试过这个模式,只是我们做了正式化,并且真正搭建了这个组织等等。
Lenny Rachitsky: 你还是基本上帮它成为了一个真正的职能。我知道你很谦虚。回到你说的去中心化运营团队,我读到过一个说法,surge pricing 好像出自一个 GM(城市总经理)在一个市场做测试,给所有司机发邮件说,周六晚上出车的给额外奖励。这是真的吗?
Brian Tolkin: 那可能比我加入的时间稍早一点。不过话说回来,有一件事是真的——surge pricing 在很长一段时间里,至少整个 2012 年,可能 2013 年也在内,我不确定我们具体什么时候切换的——很大程度上是一个有人在环的系统,或者说一个非常人工的系统。每个城市的 GM 基本上会控制 surge 运行的参数。很多时候,比如周一到周五,surge 不会开,不会触发;然后周五晚上和周六晚上才会开,你设定比如晚上 7 点到凌晨 3 点,上限是 X,不管上限是多少。然后在这些参数范围内,算法会去优化具体价格。但确实是 GM 控制着开关以及哪些区域进入 surge 状态。
Surge Pricing 的手动时代
Lenny Rachitsky: 哇,这个我不知道。当时是出于”我们比算法更厉害”的信念,还是只是”我们还没时间把算法做好,所以先人工辅助一下”?
Brian Tolkin: 我觉得可能有多方面原因。一个是,这是一个相当新的概念,威力大但也有风险,所以我们先确保理解正在发生的事情。第二个是一种信念,即本地城市团队最了解自己的城市。比如你可能知道有一场活动要结束,一场棒球比赛,你知道比赛晚上 10 点结束,所以我 9:45 就把 surge 设好,算法可能捕捉不到这个。第三个是技术限制——现在显然全部自动化了,但要构建一个完全动态的、始终在线的、空间感知的定价系统,在当时确实很难,不是短时间能做出来的。
早期 Uber 的趣闻
Lenny Rachitsky: 有道理。我感觉你在 Uber 的经历里有各种疯狂的故事。你有没有想到什么——我记得你参与过在中国推广 uberPOOL?
Brian Tolkin: 对。
Lenny Rachitsky: 也许就是那个。我不确定。能分享一下早期 Uber 的疯狂故事吗?
Brian Tolkin: 好,早期 Uber 有一个有趣的故事。UberX 显然是一个很主流的产品,但名字挺逗的——UberX。这个产品最初打算全部采用混合动力车,有过很多不同的候选名字。这个不是我主导的,是运营团队的另一个人做的,他建了这个产品的模型,但还没有名字,所以需要一个占位符。占位符放什么呢?就放了个 X。所以就叫 UberX。然后公司推进速度很快,产品通过了审批,上线了。到现在,我不知道,十一二年过去了,UberX 这个名字就一直沿用下来了。
Lenny Rachitsky: 太搞笑了,我好喜欢。原来是个占位符。很多产品都是这样开始的,他们就说这只是临时名字,然后大家就这么叫开了,好吧,那就这样吧。
Brian Tolkin: 没错。现在要改的话,重新品牌的成本太高了。
Lenny Rachitsky: 这个故事太棒了。
在中国上线 uberPOOL
Brian Tolkin: 还有一个关于扩张的故事是关于在中国上线 uberPOOL 的。当时我们要在中国上线 uberPOOL,中国当时对 Uber 来说已经比较重要了,但 uberPOOL 还没进去,所以我们准备上线。我和其他几个人在成都,这是我们第一个要上线 uberPOOL 的中国市场,我们要在现场完成上线。我们计划在——我记得是早上 6 点——早高峰时段上线,具体哪天不记得了,总之是一个周一早上。所以我们周末就在那边做准备,同时也在做一些数据中心测试。然后我们开启了所有测试基础设施,以为会正常运行,结果什么都不 work,匹配算法就是不工作。我们就想,天哪,现在是上线前一天的下午 5 点、6 点、7 点了,赶紧跟美国那边打电话,看看怎么回事。
Brian Tolkin: 我记得那天晚上我只睡了大概 30 分钟,凌晨 2 点到 3 点之间吧,心里想着,好吧,早上 6 点就得上线了。我记得当时还有一些媒体关注。我们计划好的上线时间,我想大概是早上 5 点半或者 6 点终于把所有东西都搞定了,就在最后关头赶上了。我永远忘不了,上线了,一切顺利,我们监控了一会儿,一切正常,然后早上 7 点半我们出去吃早饭。每个人都筋疲力尽,一整夜都没睡,我们在街边买了那种煎饼类的食物,我现在觉得它们应该没有那么好吃,但在我记忆里那是这辈子吃过最好吃的一顿饭。
Lenny Rachitsky: 就像跑完马拉松或者一次大徒步之后的那个饭一样。
Brian Tolkin: 没错,对,就是这样。
Lenny Rachitsky: 什么都好吃。这种情况经常出现——那些极度紧张、极度艰难、严重缺觉的时刻,最后反而成了最好的记忆、最好讲的故事、回头看最珍惜的经历。人的本性就是这样,很奇怪。
Brian Tolkin: 是的,还有一个比较近的例子是 Opendoor 在 COVID 的时候。我们物理上要买卖房屋,所以我们需要实际走进人们的房子里,结果 2020 年 3 月突然间走进别人的家变成了一件人们不太能接受的事情。你看看当时中国出来的房地产数据,看起来整个市场都快停滞了,所以我们实际上关掉了核心业务,停了几个月买房。我们说,嘿,我们进不去,也不知道还有没有人会买房,那我们怎么办?我们利用那几个月的时间,然后从另一端走出来的时候,把整个流程都虚拟化了。那确实压力很大,因为看着一个依赖进入人们家中的业务,突然你不能再这么做了,怎么办?所以,又是一段日后回看很珍惜的记忆,但在当时那个非常、非常艰难的时刻确实非常困难。
关于 Opendoor
Lenny Rachitsky: 既然你提到了 Opendoor,我觉得很多人听说过 Opendoor。也许可以简单给不太确定的人解释一下 Opendoor 做什么。
Brian Tolkin: 我们是一个买卖房产的数字化平台。现在的核心产品是面向卖方的——人们可以上网,输入一些关于自己房屋的信息,我们会给出一个全现金报价,以简洁和确定性来出售。所以这个产品真正适合那些想要一个确定、简单、省心方案的人。我不知道你有没有卖过房子,但这可以是一个非常、非常、非常艰难的过程——要安排看房、开放日、怎么定价、能不能卖出去等等。所以我们基本上提供了一种跳过整个过程的方式。
Lenny Rachitsky: 所以基本上就是把房子卖给 Opendoor,然后就是,好,搞定了,继续生活。
Brian Tolkin: 你自己选交割日期,想什么时候搬就什么时候搬。对,没有那些麻烦事。
Lenny Rachitsky: 听起来太好了。我也想要。回到运营和产品的话题,把这个线索收一下——你在两家做得非常成功的企业都工作过,它们都很好地结合了产品和运营。关于如何让这两个团队和职能很好地协作,以及如何打造一个运营很重但同时也以产品驱动的业务,你有没有什么比较宏观的心得?
产品与运营如何协同
Brian Tolkin: 第一点我们之前提到过,就是必须有相互尊重。两个职能都有各自的位置和价值,都有各自的专业技能,你要打造这种类型的大公司,就必须尊重两者都需要存在这个事实。第二点,特别是在产品和工程侧,要真正理解业务中技术的杠杆在哪里、怎么发挥,然后非常聚焦地确保——特别是在早期——你在技术资源侧通常比运营资源侧更受限。所以你怎么非常聚焦地把时间、精力和能量投入到技术上去,这就是为什么 Uber 大部分的工程投入都在调度系统和定价系统上。因为当时在资源稀缺的情况下,杠杆就在那里。
所以我觉得第二点就是要非常有意识地认清这些技术杠杆在哪里,然后非常坦诚地说,嘿,这意味着所有其他那些——是的,可以让事情更简单、更高效等等的地方——我们现在不投入也是可以的。这必须是一个明确的、非常透明的决定。最后一点我想说的是,要深刻理解现实世界是有熵的,它很困难、很混乱。对我们 Opendoor 来说,我们要进到别人的房子里,可能没人在家,日程可能对不上;在 Uber,司机可能取消订单,GPS 信号可能不好。这些事情都会发生,对吧?计算机是确定性的,但人不是。所以构建那些有更多弹性、更多故障保护的产品,以防这些事情发生,就变得更加重要。
还有最后一件事我想说的是,我觉得公司也会进化。所以我刚才提到的 Uber 早期工程和产品侧非常聚焦于调度系统和定价系统,显然随着时间推移不会一直不变——随着公司变大、变成熟,规模和优化开始变得更加重要,公司把这些职能都集中化了,还有扩张的需求,以及那个尝试新东西的培养皿,工具也变得更好了,技术也变得更容易了,内部基础设施也更完善了。所以随着时间推移,事情可以以一种方式开始,然后随着业务需求的变化而转变。
运营角色的演变
Lenny Rachitsky: 我们在这方面多花点时间吧。你一直在说让我想深挖的东西。我们在 Airbnb 也经历过同样的事情——早期有很多本地运营团队在推动供给,找房源、拉上平台,然后到了一个拐点,产品和有机增长或口碑开始驱动越来越多的增长,然后是数量级的增长。所以就不再需要这些人花时间做这些事了。你能不能分享一个例子——不管是 Uber 还是 Opendoor——你提到的运营有它的时机、位置和技能组合,这个演变是怎么发生的?团队最初在做什么,后来随着增长又转去做什么了?
Brian Tolkin: 好的,一个很简单很好的例子,就拿 Uber 流程中的一个环节来说。在早期小规模的时候,其实回到 Uber Black 司机那个时期,每一个司机都是单独 onboarded 的,90 分钟到两小时的面对面、在办公室里的 onboard,深入地设定预期。下一个版本——那显然是非常运营驱动的。下一个版本是一个小型课堂式的设置,一次三五个或六个司机。同样非常运营驱动。然后当我们进入更大众化的产品,比如 Uber Taxi 或 UberX,就是一次 20 或 30 个人。所以变成了更大的课堂设置。然后我们说,好吧,我们来做一段视频吧。不用每次口头讲同样的内容,我们就做一个 onboard 视频,这是下一个规模化阶段,但突然我们有了一个不同的问题——你需要验证所有这些证件。
所以大多数驾驶证、身份确认之类的东西——一次一个人,很简单;一次三四个,也很简单;一次十个,稍有挑战但还行。一次二十个的时候,你开始有点吃力了。然后你快进六个月,一周要处理上千个,突然你的系统就崩了——好吧,我们到了运营系统改进已经不可行的那个点。于是你说,好吧,我们已经从迭代阶段进入了规模化阶段,而技术恰恰擅长规模化。所以我们说,好吧,与其让全世界一堆人拍照驾驶证、做验证这些事,不如我们接入某种 OCR 技术或驾驶证自动识别,对接一个知道驾驶证长什么样的系统,或者能做自动验证的系统——突然之间你完成了两件事。
一是你把系统规模化,二是你一下子释放了大量时间——当时可能有几十甚至上百人全国各地、全世界在跑这些 onboarding 场次的——让他们去做别的事情。然后你可以把这再提升一个层次:好,我们是不是做更多分析?是不是去找下一个需要优化的流程?不管是什么情况,这个良性循环就持续下去了。
Lenny Rachitsky: 我喜欢这样理解——先做不规模化的的事,然后再把你在做的事规模化。这是我经常回想起来的那句话。
Brian Tolkin: 完全正确。
运营与自动化的关系
Lenny Rachitsky: 这让我想起之前播客嘉宾 Casey Winters 在一篇通讯文章里分享的一个犀利观点。他谈到——这确实是个比较有争议的说法——运营通常是一种低效的信号,随着时间的推移,你的工作就是把它挤压掉,尽可能地用产品软件来替代。并不意味着你总是能做到。你怎么看?
Brian Tolkin: 对,我其实不完全同意。根本上取决于运营是什么类型的,我并不是从根本上反对这个说法,但我认为正确的思考角度是——然后这些人可以转到下一个挑战上去。总会有下一座山要爬。所以我觉得这是 Uber 和 Opendoor 的一个特点,就是有一种在一线做实验的文化,非常有帮助。我们刚才说到司机 onboarding 现在可以用技术解决了,所以你每天多出来几个小时。那我们怎么更好地优化 UberX 的系统?怎么开始尝试外卖配送?怎么开始想大容量车辆的事?怎么去改进我们之前提到的那些手动的 surge pricing 开关的反馈机制?所以我总体上同意,只是它不断推进,去解决更多问题。
Lenny Rachitsky: 我觉得这里面很大一部分是要确保运营团队明白还有更多机会——即使这件事最终被自动化了,你的工作也不会消失。我们会找到新的东西去尝试、去实验、去做那些不规模化的事。
Brian Tolkin: 对。
产品评审(Product Review)
Lenny Rachitsky: 好,换个完全不同的话题。
Brian Tolkin: 好的。
Lenny Rachitsky: 我听说你在产品评审方面非常厉害。
Brian Tolkin: 好吧。
Lenny Rachitsky: 有几个人跟我说的。我很好奇你是怎么设置一场产品评审的,有什么心得和技巧,怎么跑一场高效的产品评审?
Brian Tolkin: 谁跟你说的我很感谢。不过是的,我确实是做产品评审的坚定拥护者。特别是——也许可以把它放在那些以运营为驱动节奏的公司或者最初就是运营驱动的公司的语境下来看——因为节奏有时候不一样。运营节奏可能是类似 WVRA(Weekly Business Review,周业务评审)这样的东西,不一定总能让你抬起头来说,嘿,产品在稍微长一点的周期里往哪走。所以我觉得产品评审对所有的公司大概都很有用,但实际上尤其对那些产品和运营驱动的公司特别有帮助。说到我学到的东西,我觉得要对目标非常明确。我觉得可以有两个目标——accountability 和向某个受众同步信息。
但最重要的是——我认为这是首要目标——是帮产品变得更好,帮团队把一个问题想清楚,让这场对话——还是回到我们最早的讨论——成为一场关于工作和如何让产品更好的有深度的对话,而不是让人害怕的。产品评审最好不要让人感觉像是在挨审。那种氛围很可怕,对”我们怎么把产品做得更好”这个目标并不有利。当然有时候讨论确实会有点激烈,但总体来说我们追求的是帮助团队回去重新思考如何把产品做得更好。
Lenny Rachitsky: 所以你在产品评审中想传达的两个目标是——accountability 和让大家了解进展,另外就是”我们在这里是为了把产品做得更好”,并且把这个前提设定好。对。你有没有什么具体做法让它不像是审讯——就是那种你进来就要被攻击、被批评的感觉?你会在会议开始时设定基调吗?还是说这就是文化的一部分?
Brian Tolkin: 对,我觉得确实是文化的一部分。但同时我也一贯坚信,离问题最近的人也拥有解决那个问题的最佳上下文。所以作为一个房间里更资深的声音,通常的工作方式是去试探、提问、抛出想法——以一种”嘿,这是一个想法,这不是一个指令,这是一个思考”的方式。如果有缺失的上下文可以影响产品方向,那就提供那个上下文——不是以一种反问的方式,而是说”嘿,这是你们可能不知道的上下文”。所以我觉得这完全取决于你作为领导者如何出现,以及那看起来是什么样子——在团队可能没有想到的维度上去试探和推动。然后要理解团队带来的是你没有的视角——他们一周在这个问题上花四五十、六十个小时,你可能一周只花三个小时。你带来的是广度,团队带来的是深度。这种组合还没被充分发掘。
Lenny Rachitsky: 你有没有听过 Dharmesh Shah 那期节目,或者他关于 flash tags 的那套东西?你见过吗?
Brian Tolkin: 没有见过,没有。
Lenny Rachitsky: 好的,他有一整套体系。你刚才说到作为领导者你希望人们不要把你的每一条反馈都当作”我必须这么做”。所以他有一整套 hashtag 系统,用来传达这件事对他有多重要——从 hashtag FYI,到 suggestion,到 plea。
Brian Tolkin: 对。
Lenny Rachitsky: 一个恳求。
Brian Tolkin: 这个其实有人给我解释过。我觉得我可能没看过原始出处,我会回去看看,但这个理念别人给我解释过,我确实很喜欢。我觉得很棒。
Lenny Rachitsky: 对,就是让所有人都在同一频道上。好,也许这最后一个问题——你会邀请谁来参加产品评审?你有没有什么框架或者思路来决定该请谁、不该请谁?
产品评审的参与者与产出物
Brian Tolkin: 好问题。应该说我们在这方面的做法随着时间有所摇摆,但总体而言,我们坚信最好的对话发生在参与者相对较少的时候,所以尽量控制在 10 人以内。文档的分发范围可以很广。产出的文档本身非常有价值,它们能帮助整个团队理解全局。一个隐秘的优势是,这些文档对新入职的人特别有帮助——给他们看最近 20 次产品评审的文档,他们就能对当前的状况有个很好的了解。但对话本身,我们尽量保持精简,控制在 10 人以内。
Lenny Rachitsky: 你说的这些产出物,是指会议录像让大家观看吗?
Brian Tolkin: 对,或者就是文档,取决于公司的文化,你是想录制会议还是只保留文档,两种方式都可以。
Lenny Rachitsky: 那你们的节奏是怎样的?是每周固定一次产品评审,大家可以报名参加吗?还是每个团队不一样?你怎么设置这个节奏?
Brian Tolkin: 这显然会随着公司规模而变化。对我们目前来说,效果不错的是一个报名制节奏。我们每周设两个时间段,任何团队都可以根据自己的产品需求来报名。然后如果有些我们想看但有一阵子没看到的工作内容,我们会稍微主动点名,确保各项工作大致以季度为周期轮流过一遍。
(广告段落已跳过)
待完成的任务(jobs to be done)在 Opendoor 的实践
Lenny Rachitsky: 接下来换个话题。我听说你是待完成的任务(jobs to be done)的忠实粉丝。这个话题在这个播客上挺有意思的,来过的嘉宾里有人很喜欢,也有人不太感冒。两边的观点我都爱听。我知道你觉得它很有用,而且在 Opendoor 也有实际落地。我很想听听你是怎么在 Opendoor 应用它的,以及在实践中有什么心得。
Brian Tolkin: 我觉得和所有框架一样,正确的做法是挑选一组适合自己的框架,在工具箱里多备几样工具,然后真正理解何时以及如何应用它们。所以我们尽量避免手里拿个锤子看什么都像钉子。如果框架不奏效,我们也会灵活调整。但我特别喜欢待完成的任务(jobs to be done)的地方在于,它迫使你站在客户的立场去思考,而且比一般的做法更深一层,让你更有同理心。当我想到在 Opendoor 做产品,对比在 Uber 做产品,或者在 Airbnb 做产品时——有一个很大的不同:我们大多数人并不是每周或每月都有房子要卖,也不是每周或每月都在买房。在美国,平均来说人们大概每七年才会做一次这件事。我估计在 Opendoor 的平均频率也差不多,所以想要以客户身份亲身体验是比较困难的。我在 Uber 时每天都打 Uber,你可能一年也会用 Airbnb 好几次。所以在某些公司里,你可以为自己做产品,凭直觉就能感受到待完成的任务(jobs to be done),因为你自己就是用户。我们不一定有这种切身感受,所以一个能迫使我们真正认真、有意识地去思考客户如何看待我们产品的框架,对我们来说非常有帮助。另一个我喜欢它的地方是,经典版本会鼓励你去思考用户所处的环境——也就是你的产品之外,他们可能还在经历什么。在我们的场景下,买房或卖房的过程通常至少要持续几周,甚至几个月、几个季度,中间充满复杂性,而且有大量发生在我们产品之外的互动。你可能在与中介沟通,可能与朋友商量,可能开车在城市里到处看房。这个框架非常灵活,它鼓励你去思考:这个用户在考虑我们的产品时,真正待完成的任务(jobs to be done)是什么,他们所处的环境又是怎样的。
Lenny Rachitsky: 我想再深入一层,聊聊你们具体是怎么落地的。你们有模板吗?比如一个新项目开始时,有”作为____,我想要____,以便____“这样的格式?你们是怎么操作的?
Brian Tolkin: 我觉得我们在模板的标准化和执行上算中等严格。我们确实有一个模板,标准的产品评审(product review)模板里会提到待完成的任务(jobs to be done),其中有一个部分专门写问题陈述和待完成的任务(jobs to be done)。
Lenny Rachitsky: 这是在来做产品评审(product review)之前,负责推进的人需要事先填好的文档?
Brian Tolkin: 对,事先填好。另外,我觉得我们并不是死板地要求每次都用那个模板,但模板的好处在于,它一方面设定了期望——你需要准备什么,另一方面,对大家来说,有一个现成的东西照着写通常更方便。所以它是我们产品评审(product review)模板的一部分,也是我们规划流程的一部分。因为我们用了挺久了,我觉得它已经在文化中内化了——大家会自然而然地在评论或文档中提起它,比如说”这里的待完成的任务(jobs to be done)是什么?“或者”用户想要做什么?“后者可能算是一个更通俗的说法。所以确实有一种文化渗透的过程。
Lenny Rachitsky: 凭记忆说说,这个模板里具体包含什么?描述问题的时候,你们用的措辞是什么样的?
Brian Tolkin: 具体的措辞我得回去看模板才能准确说,但大致的结构是:背景、问题、潜在方案、风险——包括风险和预 Mortem 分析,以及成功的衡量标准。另外我们还会按阶段来区分产品评审(product review)。你可能处于构思阶段,那文档看起来会和流程末尾的阶段非常不同——比如”我们马上要发布了,有意见现在提,否则以后别说了”。这两个阶段的产出物也会很不一样。
Lenny Rachitsky: 好的,所以你们并没有严格使用经典的待完成的任务(jobs to be done)的标准语句格式,更多是确保大家在用这个思维方式思考——客户的问题是什么?他们的问题所处的背景是什么?
Brian Tolkin: 对,是的。我们没有在模板上做教条式的限定。
Lenny Rachitsky: 明白了,很棒。关于运用待完成的任务(jobs to be done)这个概念,你还有什么其他建议或心得吗?比如说你刚加入 Opendoor 时跟大家说”我们以后要用这种方式思考”,中间有什么经验是其他人想要采用这种方式时可以借鉴的?
正确实施框架的关键
Brian Tolkin: 要认识到,正确实施一个框架——任何框架都是如此——是需要时间和练习的,需要去理解和适应。所以我不认为你可以直接说,好吧,我们做一个模板,然后内容自然就变好了。那样做只是把大家已有的内容硬塞进模板里。真正需要的是文化上的内化——比如说,这个表述看起来像是待完成的任务(jobs to be done),但它真的就是待完成的任务吗?我们来讨论一下为什么客户会处于或不处于那种情境。或者我觉得真正的待完成的任务可能是别的什么。
举个例子,早期的版本可能是——待完成的任务是从 Opendoor 获得一个报价。嗯,某种程度上可以这么说,但更广泛的待完成的任务可能是客户的价格发现。所以你可以展开深入的讨论——其中一个可能受到我们商业目标的影响。我不认为有人会到处说”我要卖房子,但我的目标是获得一个 Opendoor 的报价”。所以,模板可能是一样的,但真正需要文化层面落实的是内容本身。
Lenny Rachitsky: 明白了。听起来”待完成的任务是什么”这种提问方式是你们思考方式的核心部分。“待完成的任务是什么?“单是这句话本身就非常有力量。你有没有推荐的资源或书,用来帮助团队学习待完成的任务相关的方法论?有没有你觉得特别有用的?
Brian Tolkin: 关于待完成的任务没有什么特别推荐的。我更多是指向内部案例,比如我看到其他 PM 做得好的例子,或者博客之类的。不过你的博客是我们经常传阅的,虽然不是关于待完成的任务,而是涉及很多话题。
Lenny Rachitsky: 过奖了,谢谢。
Brian Tolkin: 确实。
Lenny Rachitsky: 非常感谢。你刚才说的时候我还在想,你和 Kayvon 是朋友,而待完成的任务这个概念在 Twitter 上的发展历程相当曲折,对很多人来说简直是噩梦。我觉得它走向了一个非常极端的方向——
Brian Tolkin: 对,我觉得他们更教条一些。
Lenny Rachitsky: 非常极端。所以这也许是一个教训——不要走那么远。
Brian Tolkin: 是的。而且我觉得——我不知道 Kayvon 是否同意这个说法,我猜他会同意——更普遍化地讲就是,你要为正确的问题选择正确的框架。如果你说有一个框架可以统治一切,这是唯一有效的框架,然后把所有问题都硬塞进去,那当然就会出问题。
Lenny Rachitsky: 我对待完成的任务的理解方式恰恰就是你们所描述的——就是从客户待完成的任务的视角去思考。比如对我的 newsletter 来说,待完成的任务就是帮助作为产品人员的你在做产品的过程中变得更好。这实际上非常有帮助。感觉你们在 Opendoor 也是这样思考的。
Brian Tolkin: 完全同意。顺便说一下,你做得非常出色。
低样本量下的实验与 A/B 测试
Lenny Rachitsky: 谢谢,你也是。你提到——我要赶紧转到下一个问题来回避你的夸奖——你提到 Uber 每秒有上百万笔交易,规模巨大。而 Opendoor 完全不同,你们的交易数量很少但金额很大。我很好奇你们怎么做实验,你们是否做实验,是否做 A/B 测试?关于低样本量加 A/B 测试这个组合,你有什么心得?
Brian Tolkin: 这是一个非常热门的话题。我们确实做 A/B 测试,它显然是黄金标准,所以我们尽可能地去做。在 A/B 测试方面,我们漏斗的不同环节流量大小不同——漏斗顶部的测试比底部容易,纯产品或技术功能的 A/B 测试比运营流程的 A/B 测试容易。但你说得完全对,我们一年并不是在做几亿笔交易,所以实验确实更具挑战性。
我觉得一种思考方式是:第一,承认这个问题——不要犯我们犯过很多次的错误——不要不做 power analysis 就硬上 A/B 测试。要先想清楚,我们能拿到结果吗?能检测到的效果量是多少?这个实验需要运行多长时间?然后诚实地问自己,这可以接受吗?
第二个经验是:有些实验足够重要,又很难通过其他方式来捕捉信号,那你说一个六个月的运行周期是可以接受的结果,我们六月启动,到 2025 年规划时我们会因此更明智——设定好了就放着不管,而且事后很庆幸我们这么做了,这完全可以。但唯一的错误是,以为自己一个月就能得到答案——实际上不可能——然后假装得到了,一个月后醒来发现”结果不显著”等等。这本就是可以预见到的。
第三点是,实验归根结底是为了提升你对问题或解决方案的信心。所以这句话的更一般化版本是:如果你的漏斗或流程的某些部分流量较低,无法运行标准的 A/B 测试,那你还能用什么其他方式来增强对正在构建的解决方案的信心呢?
实际上有不少其他方法。首先最明显的是——跟更多客户交谈。但还有一些其他统计方法,虽然没那么严格或精确,但也是可行的。你可以使用观察数据,做 diff-in-diff 分析,可以对比姐妹城市或双胞胎城市,可以按地域细分,也可以降低 power 要求——比如我们所有实验都在 80% 的置信水平上运行,而不是传统的 95%,因为这是一个值得做的权衡。如果我们多犯一次错,那是可以接受的。
你还可以做一个长期 holdout 组来验证你的直觉。所以有很多其他方法来磨炼你的直觉,建立信念和信心。我们在这方面尽量保持创造性。
最后一点是:如果你确实无法获得统计显著性,也没有其他方法可用,那有时候你就得相信自己的直觉,直接上线。如果你相信这样做,那就是你相信的,你不应该花时间去追求虚假的精确度。
Lenny Rachitsky: 我想再多聊聊最后那一点,不过先快速说一下你提到的 power analysis——有些人可能不知道,有现成的计算器可以拿来用,你只需输入:我的流量是多少,我想看到的差异幅度是多少,它会告诉你需要多长时间才能得出结论。
Brian Tolkin: 对,完全正确。有些计算器很好用,你还可以输入流量和你能接受的运行时长,它会告诉你最小可检测效果,然后你可以用自己的直觉去校验。你可以来回调整这些参数。
直觉与产品决策
Lenny Rachitsky: 太好了,我们争取在节目说明里附上一个计算器的链接。回到刚才说的直觉这一点,你还有什么补充吗?比如说你管理产品团队的时候,你建议大家在什么时候依靠直觉、什么时候不依靠?因为有些公司的态度是:我们只信数据,我不太信任你的个人意见,你又不是这个客户。你刚才提到 Opendoor,我自己又不买房,所以我也不知道自己的直觉能信多少。你对产品团队在直觉这件事上有什么通用建议?
Brian Tolkin: 拿 Opendoor 来说,我觉得在相对光谱上我们算是相当数据驱动的。然后我们会遇到一种挑战,就是——好吧,直觉也是工具箱里的一种技术手段。我觉得这个问题的通用版本是:客户、产品、人都会让你意外。做产品的人对这一点太熟悉了。我相信你在 Airbnb 肯定也有很多这样的经历——你看到某个东西,做出来投放出去,效果就是非常——
Lenny Rachitsky: 一直都有。
Brian Tolkin: 一直都有。所以我觉得确实需要一种谦逊的态度:如果你的假设或判断相对容易测试,那最好还是去验证一下,这永远比自己拍脑袋强。这需要一点谦逊才能说出来,但我们每个人都被打脸过很多次。但如果测试确实不现实,那你就不能假装它现实。有时候你就得依靠品味和判断力,然后问自己:我的信心水平是什么——是低、中还是高?如果信心只有低或中等,而这又是一个有影响的决策,我就应该去跟更多客户聊,跟其他人核实,看看他们的直觉是否一致,这样能帮我把信心提升到高水平。
还有一点我认为也很重要:实验的一部分在于,如果你纯粹凭直觉上线了某个东西,或者因为这是你希望产品走的方向——那你有没有一个合理的反馈闭环来检验自己到底对不对?这个反馈闭环可以是客户支持、工单量、功能采纳率等等。它可能不是传统 A/B 测试里的输出指标,但它是一个更严谨的系统,告诉你:嘿,我有这个假设,由于 x、y、z 的限制,我们直接上线了。
Zillow 的故事
Lenny Rachitsky: 这些建议非常好,我完全赞同你说的每一点。你提到”谦逊”这个词,正好可以引出我想聊的另一个话题——Zillow。你们这个领域最有趣的事情之一就是,Zillow 基本上决定:嘿,我们就去做 Opendoor 正在做的事。他们上线了,你们基本上是亦敌亦友了一段时间,然后他们说:不行,这搞不通。现在你们成了合作伙伴,一起在这些事情上合作。你能分享一下当时到底发生了什么吗?事情的经过怎样,现在又是什么状态?
Brian Tolkin: 可以。我们确实和 Zillow 有合作关系,Zillow 一直是我们的优秀合作伙伴,我们很享受与他们的合作关系。你想一下的话,他们拥有巨大的触达和用户群体,所有这些线上平台都有巨大的触达和用户群体。而我们恰好拥有一个相当独特的卖房解决方案。所以两者之间有一种——不想用商学院的词来形容——但确实有一种协同效应:一方是大量有明确意向的用户,在这些线上平台上浏览、搜索、发现、开始他们的流程;另一方是我们提供的交易服务,让用户真正完成搬迁,尤其是在卖房端。所以和 Zillow、Redfin 这样的平台之间,存在一种相当不错的共生关系。这两家都是我们很好的合作伙伴。
Lenny Rachitsky: 你觉得 Zillow 可能低估了什么,或者对这个行业有什么没搞明白的地方,导致事情比他们预想的更难?看起来理所当然嘛——我们往漏斗下游走,直接全部自己干。然后他们发现:“哦,糟了,搞不通。“你觉得他们没搞明白什么?错过了什么?
Brian Tolkin: 接着刚才谦逊的话题说吧,我不会假装站在他们的立场上去分析,但我可以说这个生意很有挑战性,从多个维度来看都很复杂。这不是一个纯软件产品——你必须在定价上非常出色,在产品上非常出色,在运营上非常出色。你必须在风险管理上非常自律,在资本市场方面非常擅长。你必须把所有这些职能整合在一起,才能构建一个垂直整合的产品。这就是现实。而这从第一天起就刻在了 Opendoor 的 DNA 里,因为我们最初做的就是垂直整合产品。除非这些环节都到位,否则我们无法交付。所以我认为这种垂直整合的要求一直帮助我们走到今天——它要求所有这些环节协同运转。
Lenny Rachitsky: 这很有道理,我觉得这也是一个很好的提醒:总有一些相邻市场看起来像是”哦,我们哪天也可以扩展到那个领域”——那么大的机会,业务可以做得更大。然后你发现你的公司根本不是按那种方式运作的。Zillow 是非常以软件驱动的——我不想把他们做的事说得太简单——但它本质上是一个网站,非常偏软件。而 Opendoor 正如我们讨论的,有巨大的运营成分,还有你说的定价和债务的部分。
Brian Tolkin: 对,完全同意。
Lenny Rachitsky: 所以我觉得这是一个很好的提醒:当你涉足一个完全不同的领域时,它可能根本不适合你公司的运作方式,这时候合作就是合理的。关于 Zillow 这件事还有什么有趣的可以分享吗?我想象当时应该压力很大——Zillow 进来了,“糟了,我们怎么办?他们有所有的流量。“有什么可以讲的吗?
Brian Tolkin: 压力肯定是有的。但总体来说,我们一直努力遵循一个原则——不管是 Zillow 还是其他竞争对手——保持竞争意识,但不以竞争对手为中心。事实是,在我们这个领域,绝大多数人仍然通过传统方式搬迁。所以这不存在市场不够大、天花板太低之类的问题。现实是,这是美国最大的资产类别。如果我们始终专注于——嘿,我们服务的那些客户是谁,我们每天和他们对话,服务得很好——这种专注会带来一种信心,不管竞争环境如何变化。原因很简单,因为市场远远没有饱和。
这跟当年在 Uber 的情况一样。交通出行几乎是无限大的市场。是的,当时感觉 Uber 和 Lyft 之间的竞争很激烈,但现实是有大量的出行需求,人们需要用各种方式出行,其中很多既不是 Uber 也不是 Lyft。专注于如何为你的客户做好产品开发,我认为是最好的聚焦方式。
Stripe Atlas 与 AngelList 的类似经历
Lenny Rachitsky: 在这期节目之前会播出另一期播客,嘉宾是来自 Stripe 负责 Stripe Atlas 的 Jeff Weinstein。他们有过类似的经历——AngelList 推出了一个直接竞争 Atlas 的产品,后来他们意识到 Atlas 做得好太多了,于是放弃了自己的产品,直接把所有人都推荐到 Atlas。
Brian Tolkin: 真的?
Lenny Rachitsky: 真的。我觉得这是完全相同的教训——只要你始终专注于待完成的任务(jobs to be done),搞清楚要完成的任务是什么,然后做到最好,同时清楚市场远比你想象的大,你并不是在和另一家公司竞争。在你的情况下,竞争对手其实是人们的默认行为,就是人们用老一套的方式买房,那才是真正的竞争对手。
Brian Tolkin: 完全正确。
压力下的冷静
Lenny Rachitsky: 好。说到这里,我还听说过你非常擅长的一点,就是在压力下保持冷静,在情况一团糟的时候依然保持清醒。这是很多人——尤其是领导者——做不好的事情。他们搞得所有人都很紧张,整个氛围都不好。这是很多人想提升的能力。不管是领导者还是非领导者,你在这方面有什么经验教训或心得吗?
Brian Tolkin: 我认为这部分可能是在 Uber 早期磨练出来的。那时候一切都是十万火急的状态。所以唯一的运作方式——但我觉得你问题里其实已经点到了关键,就是一个有点理性的认知:当你把压力传递给团队,所有人都会紧张起来、束手束脚。这反而适得其反,不能带来更好的结果。所以我还需要提醒自己的另一个现实是——这些都是在这种时刻很有用的信条——你永远不会像你以为的那么好,也不会像你以为的那么糟。所以保持更加平稳的心态,我认为能让你在压力下头脑更清醒,思考更清晰。
我觉得一个可能最没什么帮助、但不幸的是事实的答案是:你必须亲身经历一些压力场景,才能获得那种”一切都会过去”的视角。保持冷静才是最重要的。所以也许建议就是,当这些情况发生时,去面对它们、暴露在其中,不要逃避,然后从中学习,这样下次再遇到的时候你就能说,嘿,我经历过这种情况。我曾经在中国为了上线 uberPOOL 睡在地板上,以为会错过发布截止日期。当时我工具箱里有什么工具帮助我完成了任务(或者没有完成),经验教训又是什么?
Lenny Rachitsky: 很喜欢这个说法。所以部分原因就是多次经历这些事情,然后你开始意识到,好吧,实际情况不会像人们想象的那么糟。你刚才提到了工具箱,还有什么是你会反复回来用、确实有帮助的东西吗?你提到了那个信条——事情永远不会像人们想的那么糟,也不会像人们想的那么好。
Brian Tolkin: 是的,我觉得接触其他人的故事——不管你通过什么方式学习——真的非常有帮助。无论是你的播客,还是书籍、传记,或者我很喜欢的一个播客叫 Founders Podcast,专门讲述历史上著名企业家的故事。当然这些本身已经是把非常出名的人高亮出来了,但很多故事确实有很多值得学习的地方,让你理解创业之路本质上是非线性的,对任何人来说都是如此。所以我觉得,即使你自己没有那些亲身经历,通过接触他人的故事,了解别人是如何应对的,也会很有帮助。
Lenny Rachitsky: 明白了。就是听听别人经历的疯狂事情,锻炼这种心理肌肉——好吧,他们经历了那么疯狂的事,最后还是闯过来了。
Brian Tolkin: 没错。
在模糊与噪音中找到核心真相
Lenny Rachitsky: 我们会挺过去的。好。我这里有一条笔记,好像是有人提到你,也可能是你自己说的——做产品就是在一片模糊和信号的海中找到那个核心真相。这个说法对你来说有意义吗?
Brian Tolkin: 当然有意义。我觉得在大多数组织中,要有效地做好这份工作,你会从四面八方收到信号,好主意可能来自任何地方。可能是你的客服团队,可能是客户直接反馈,可能是你的一次对话,可能是你看的一个 YouTube 视频激发了灵感,可能是来自高管的反馈,也可能是你出去做了一次实地走访。你会收到大量关于人们对你产品的看法、人们认为你下一步应该做什么的输入。而我认为核心工作就是搞清楚什么才是真正重要的——什么是噪音?什么是好主意?什么是建议?回到待完成的任务(jobs to be done),什么才能真正推动客户前进?
而不幸的是,这意味着你要对一些听起来不错的主意说不。但如果你能真正弄清楚什么才是最重要的,那就是这份工作的核心。这其实和我们之前聊的话题也是相通的。在早期建设技术和运营结合的公司时,关键问题是:技术的杠杆在哪里?同样的问题——什么是真正重要的、技术可以独特解决的核心?然后就去做那件事,对其他可能在燃烧的火保持坦然接受的态度。这才是真正、真正、真正重要的事情。这是一个很难的修炼。
Lenny Rachitsky: 说得太好了。如果没有合适的例子也没关系,但当你谈到找到技术可以发挥高杠杆的核心真相时,有没有什么让你印象深刻的成功案例?
Brian Tolkin: 回想 Uber 早期,我觉得就是——我们不建复杂的工具基础设施,不建中央增长团队,这些都不做。因为如果你从最简单的角度来想早期的 Uber 网络,就是有乘客和司机,你需要把他们匹配起来、给交易定价、发收据,大概还需要收款。所以就是——我们把这些最基本的事情做好了吗?在这些做好之前,其他所有的东西都是噪音。客服工单处理得有多高效,这根本不重要。现在当然很重要,但在早期没那么关键。甚至客户获取成本可能也不是最关键的,因为在这种情况下增长本身就很迅猛,继续火上浇油未必需要那么追求效率。
所以我觉得这是一个很好的通用案例。还有一个小建议可能有用,坦白说我自己也在不断练习和改进——就是这些来自四面八方的想法和反馈,一定要写下来,原因有几个。第一,你可以回头查阅。第二,确保提出这些想法的人感到被倾听、被尊重,知道这些想法至少被考虑过,这也是工作的一部分。然后你可以统观全局,说,好吧,那到底什么才是真正、真正、真正、真正重要的?是的,这是另一个建议。
Lenny Rachitsky: 你说写下来的时候,有没有觉得特别好用的工具?是放在一个大文档里统一记录?还是有什么方法能真正把这个流程落地?
Brian Tolkin: 我见过不同公司做法不一,但无论你用什么东西来维护需求池——不管是 Google 表格,还是 Jira 里真正的 backlog——至少要让人觉得,好吧,上下文被记录下来了,想法也在那里了。
Lenny Rachitsky: 太好了。好,我们进入这档播客的固定环节——“失败角”(Failure Corner)。
Brian Tolkin: 好。
Lenny Rachitsky: 你能不能分享一个你职业生涯中失败的经历——一次重大失败——以及这段经历如何让你变得更好?
Brian Tolkin: 我们可以聊聊 uberPOOL 最初期,也就是在旧金山的第一次上线。这是一款拼车产品,一辆奔驰车里坐多位乘客。我们当时有一个想法,认为这对通勤人群会很有效——那是很早期的事情。上线方案的一部分是,我们打算先在一些热门通勤线路上做 beta 测试,跟特定公司合作,比如从 Marina 到 Google 之类的,尝试按照公司来匹配乘客,以此来驱动流动性。但我们很快意识到——回到这里最核心的真相——流动性是唯一重要的事情。而流动性根本不够,按公司划分来做这件事永远不可能有足够的量。这不是通往成功的策略。所以我觉得这算不算一次彻底的失败也不确定——也许所有失败都是这样:你从中学到东西,你转向,然后继续做下一件事。
显然我们确实这么做了,然后把大量时间和精力花在研究流动性的边界上,搞清楚我们能驱动多少流动性,从而理解产品最重要的指标是什么、极限在哪里。举个例子,我们上线时——也许旧金山的人还记得——旧金山范围内 5 美元任意行,这是个大力推广,显然非常划算,显然也烧了很多钱,但整个思路就是,好吧,如果流动性才是真正关键的东西,那如果我们猛推一下、真正把流动性拉起来,我们的指标能到多高?然后我们再去追求更可持续的方式来实现同样的效果。但这是一个很有意思的失败案例——从上线到学习,发现初始策略根本行不通,必须转向。任何保留策略——比如用小范围的 beta 用户群——行不通,这次你只能全面铺开。
Lenny Rachitsky: 我觉得这里的一个教训也是不要想太多,别试图玩得太精巧。就像我们在做一个完美的 beta 测试,而没有意识到——我们只是需要更多的人参与进来。另外,你提到 5 美元推广,让我想起了早期的冰淇淋配送、小兔子配送那些活动。
Brian Tolkin: 对,顺便说一下,那就是完全分布式运作的例子,也是拥有那些早期”培养皿”市场的好处。某个本地的营销经理说,嘿,这个挺好玩的。是啊,确实挺好玩的。平台能支持。那些推广活动效果非常好。最开始我不记得第一个是冰淇淋还是小狗,好像是冰淇淋。但后来延伸出了各种各样的事情。船、冰淇淋、小狗、小猫,我觉得都有。功劳全归本地的创意灵感,他们就是专注于在自己的市场里推动增长。
Lenny Rachitsky: 我很喜欢我们又回到了对话最开始的话题——产品和运营协同工作,两者结合的好处。在进入非常令人期待的快问快答环节之前,你还有什么想分享的吗?有没有什么最后的智慧 nugget,觉得对那些正在打造产品、公司、团队的人可能有用的?
Brian Tolkin: 这次对话很棒,我们聊了很多内容。我觉得唯一——我不知道这算不算通用智慧,但随着我的职业发展,尤其是搭建正式组织的过程中,我一直在思考一件事,尤其是在越来越多工具出现之后——很明显,PM 有不同的类型,我们花了很多时间讨论能够在物理和数字、产品和运营两个世界之间切换的 PM。但即使在这个范围内,也有更偏技术型的 PM,他们从工程 discipline 成长起来;有从运营出来的人;也有从设计出来、在用户体验背景下成长起来的人。
有一件事我在搭建团队时越来越深有感触——跟做产品路线图类似——这不是说这个人好还是不好,而是这个人的技能组合和经验背景是不是当前问题真正需要的。回到之前关于技术杠杆的讨论,就是,这个人作为 PM 所具备的独特技能组合是否匹配这类问题?我不知道这是否有帮助,但这确实是我花了很多时间在思考的东西,尤其是现在那种”发个 JD、招个 PM”的方式。实际上应该是,我们能不能更深入地思考一下这类团队真正需要的技能组合是什么?
Lenny Rachitsky: 没错,就像是”人岗匹配”(person-product fit)。
Brian Tolkin: 就是这个。
Lenny Rachitsky: 我觉得这背后是因为很多公司招的是通才,他们就想,我们招一个聪明、有野心、有通用经验的人,然后把TA放到不同的事情上。所以我认为这是两种不同的理念,而对于 Opendoor 这样业务非常独特的公司,需要非常特定的技能才能在那里做得很好,这种方式可能很合理。好,太棒了。Brian,到这里,我们进入了非常令人期待的快问快答环节。准备好了吗?
Brian Tolkin: 来吧,迫不及待。
Lenny Rachitsky: 开始。第一个问题:你推荐最多的两三本书是什么?
Brian Tolkin: Shoe Dog、Black Swan、Design of Everyday Things,再来一本轻松的,Shawn Theron。
Lenny Rachitsky: 太棒了,问两三本给了四本。我喜欢。
Brian Tolkin: 抱歉,我遵守规则。
Lenny Rachitsky: 不不不,没有规则,这里没有规则。
Brian Tolkin: 好的。
Lenny Rachitsky: 下一个问题:你最近有没有特别喜欢的电影或电视剧?
Brian Tolkin: 我喜欢 Netflix 上的体育纪录片,比如 Full Swing、Drive to Survive、Break Point,网球、高尔夫、F1,全都看。
Lenny Rachitsky: 最近不是还有一部 Ben Affleck 拍的 Nike 纪录片吗?
Brian Tolkin: 有,但我还没看。所以如果好看的话——我不知道这算推荐还是仅仅是确认它的存在。
Lenny Rachitsky: 值得看。如果你喜欢 Shoe Dog 的话,我觉得你会喜欢的。挺有娱乐性的,Michael Jordan 那些内容。下一个问题:你最近有没有发现一个特别喜欢的、让你爱不释手的产品?
Brian Tolkin: 我们刚养了一只小狗,而且第一个孩子也快出生了。所以我最近买的东西全都围绕小狗和孩子。我一个朋友送了我们 Fi 项圈给狗用,我们非常喜欢。另一个是随着我越来越忙,获取新闻用的 Particle,一个很棒的新闻聚合工具,AI 新闻工具。
Particle 新闻应用
Lenny Rachitsky: 那是 Cavan 的妻子的项目。我其实特别喜欢,我觉得它刚从 beta 出来,现在是一个所有人都可以下载的完整应用了。我昨天刚重新装上,非常喜欢。它会每隔几小时推送一下,我也不知道,大概一天几次,就告诉你发生了什么。另外,恭喜你即将迎来孩子,我刚才应该说这个的。
Brian Tolkin: 谢谢。
Lenny Rachitsky: 你运气好。我有一篇 newsletter 文章列了所有你该买的产品,叫”产品经理新手父母礼物指南”。
Brian Tolkin: 太好了。我大概率会把它们全买了。
Lenny Rachitsky: 如果你还没全买的话。而且现在估计所有人都给你发他们最喜欢的东西的表格了。
Brian Tolkin: 没错。
人生座右铭
Lenny Rachitsky: 好,下一个问题。你有没有一个特别喜欢的人生座右铭,经常在工作或生活中拿出来跟别人分享的?
Brian Tolkin: 基本上就是:保持好奇心。
Lenny Rachitsky: 保持好奇心。我喜欢。还有两个问题。在你的职业生涯中,谁对你影响最大?
职业导师
Brian Tolkin: 在我产品之路的早期,有一个给了我很多启发的人。我很幸运,一路上有不少非常好的导师,当然我们之前也聊过,我从很多其他人的经历中学到了很多。但有一个人在我早期的产品旅程中对我个人非常重要,也非常支持我,这个人叫 Jeff Holden,他当年是 Uber 的首席产品官。作为一个转到产品岗位的年轻 PM,他真的非常照顾我、带我成长。我对此永远感激,感谢 Jeff 帮助我发展职业生涯,同时我也在努力把这份恩情传递下去,帮助那些正在走这条路的人。这对我的意义非常大。
Uber 面试故事
Lenny Rachitsky: 最后一个问题。我听说你在 Uber 的面试相当疯狂。能讲讲这个故事吗?
Brian Tolkin: 好,可以说。长话短说,我大四春季、毕业前正在创业,后来团队散了。所以我从来没有做过传统招聘。我一个朋友打电话给我说:“嘿,我们在 Uber 这个项目上在找聪明、勤奋的人,你有兴趣吗?” 顺便说一下,我在 2011 年其实对这些打车应用做过一些早期调研,当时研究的是 Uber Cab、Cabily 和 Taxi Magic 这些,现在大概没人听说过这些名字了。所以我知道 Uber 是什么,就说:“好啊,我愿意去试试。”
第一轮面试很顺利,他们说好,下一步就是来 onsite,全套面试流程。那时我已经毕业了,在帮一些公司做事,但没有全职工作。我说,我时间比较灵活,下周二怎么样?他们说好。我们就约了。然后周五还是周六,过周末的时候我看了一下,周二,7 月 4 号。我把面试约在了 7 月 4 号。我打电话给我朋友说,嘿,非常抱歉,我不想让别人 7 月 4 号还来加班。我要不要取消?要不要联系一下?他说,所有人都接受了,你千万别取消你的面试。好吧,那 7 月 4 号我去。我就 7 月 4 号去了办公室,那里只有很少几个人。那天其实是在上线 Uber 有史以来第二个产品类型——Uber SUV。我经历了一场大概五个小时的连环面试,从中午到下午五点,错过了我的 7 月 4 号烧烤派对。那确实是相当难忘的经历,但也许也预示了早期那些混乱的日子。我很庆幸没有取消那次面试。
Lenny Rachitsky: Travis 有参与那场面试吗,还是说——
Brian Tolkin: Travis 参与了面试。他是其中之一……我记得当时有四五个人,引导我面试流程的主要是 Travis 和另一个人,那天开始上班的。连环面试中有一部分是模拟实际工作。其中一部分是在电脑上建模型,另一部分是给司机写邮件,还有一个司机进来跟你聊天。我一个人在房间里做第一部分,也就是建模型,然后听到有人敲门,Travis 进来就坐下了,说:“嗨,我是 Travis。“我说我不知道你是谁,但我叫 Brian。然后我们聊了大概四十五分钟,也许是半小时、四十五分钟。很明显我该做的面试任务一点也没做。我应该在建模,然后把结果发回去给发给我的人。显然我什么都没做,一直在跟 CEO 聊天。然后又听到敲门声,门开了,那个人看到我在跟 Travis 说话,“哦,继续继续”,就关上门走了。非常有意思。跟 Travis 的那次对话也相当激烈,确实让我对在那儿工作有了心理预期。
Lenny Rachitsky: 显然结果是好的。Travis 也很高兴。我猜——
Brian Tolkin: 希望如此。
Lenny Rachitsky: 我想象的。
Brian Tolkin: 是的。
结语
Lenny Rachitsky: 太精彩了。Brian,非常感谢你来。我们聊完了所有我想聊的内容。最后两个问题。大家在网上哪里可以找到你?有没有什么你正在做的事情想让大家了解的?听众怎样才能帮到你?
Brian Tolkin: 太客气了。大家可以在 Twitter 和 LinkedIn 上找到我,都是 Brian Tolkin,就是我的名字。至于帮忙的话,如果你有房子要卖,可以去 Opendoor 试试。更重要的是,如果你对产品有反馈,我非常想听到。另外,关于我们今天聊的内容,大家喜欢什么、想了解更多什么,也非常欢迎告诉我。所以期待大家的消息。
Lenny Rachitsky: Brian,非常感谢你来。
Brian Tolkin: Lenny,真的很感谢。这次很棒。大家再见。
Lenny Rachitsky: 非常感谢收听。如果你觉得这期节目有价值,可以在 Apple Podcasts、Spotify 或你喜欢的播客应用上订阅。也请考虑给我们评分或留下评论,这对其他听众发现这个播客真的很有帮助。你可以在 lennyspodcast.com 找到所有往期节目或了解更多关于这个节目的信息。下期再见。
术语表
| 原文 | 中文 |
|---|---|
| accountability | accountability(责任制/责任归属) |
| Casey Winters | Casey Winters |
| Cavan | Cavan |
| Dharmesh Shah | Dharmesh Shah |
| diff-in-diff | diff-in-diff(双重差分法) |
| EPD | EPD(Engineering, Product, Design,工程、产品、设计团队) |
| Fi | Fi(智能宠物项圈品牌) |
| GM | GM(General Manager,城市总经理) |
| holdout | holdout(保留组/对照组) |
| Jeff Holden | Jeff Holden(前 Uber 首席产品官) |
| jobs to be done | 待完成的任务(jobs to be done) |
| Kayvon | Kayvon(指 Kayvon Beykpour,前 Twitter 产品负责人) |
| on-site | onsite(现场面试) |
| onboarding | onboarding(入职/上线引导流程) |
| Opendoor | Opendoor(在线房屋交易平台) |
| Particle | Particle(AI 新闻聚合应用) |
| power analysis | power analysis(功效分析/统计功效分析) |
| product operations | 产品运营(product operations) |
| product review | 产品评审(product review) |
| surge pricing | surge pricing(动态溢价/ surge 定价) |
| Taxi Magic | Taxi Magic(早期打车应用) |
| Travis | Travis(指 Travis Kalanick,Uber 联合创始人兼前 CEO) |
| uberPOOL | uberPOOL |
此文档由 AI 分片翻译(translate_long_document)