打造世界级的数据组织 | Jessica Lachs(DoorDash 分析与数据科学副总裁)
Building a world-class data org | Jessica Lachs (VP of Analytics and Data Science at DoorDash)
Introducing the Guest
Lenny Rachitsky: So you’ve built one of the largest and most respected data teams in all of tech.
Jessica Lachs: For me, analytics is a business impact driving function and not purely a service function, not just answering the why, but answering the, “What do we do now that we know this?”
Building a Data Team
Lenny Rachitsky: One of your colleagues told me that you are incredibly good at defining metrics.
Centralized vs Embedded Team Structures
Jessica Lachs: Retention is a terrible thing to goal on. It’s almost impossible to drive in a meaningful way in a short term. Ultimately, you want to find a short-term metric you can measure that drives a long-term output.
Lenny Rachitsky: You mentioned the early team. I felt extreme ownership.
Compromise Between Embedded and Centralized
Jessica Lachs: Yes, you are a data scientist, but your goal is to figure out what’s happening. And if that means that you’re going to pick up the phone and call customers, then that is what you’re going to do to roll up your sleeves.
Lenny Rachitsky: Today my guest is Jessica Lachs. Jessica is Vice President of Analytics and Data Science at DoorDash, which has built one of the biggest and most impactful data teams in tech. She’s been at DoorDash for over 10 years and was the first GM at DoorDash responsible for launching new markets. Previously, Jessica founded GiftSimple, a social gifting startup and began her career in investment banking at Lehman Brothers.
In our conversation, we go deep on how to build and scale your data org, including why a centralized org model is so effective. What to look for when hiring data people, how to pick the right metrics for teams to align incentives and drive the right sorts of outcomes. Examples of how the data team at DoorDash has helped the business make better decisions, a bunch of great stories about the early days of DoorDash and a ton more. If you enjoy this podcast, don’t forget to subscribe and follow it in your favorite podcasting app or YouTube. It’s the best way to avoid missing feature episodes and helps the podcast tremendously.
With that, I bring you Jessica Lachs. Jessica, thank you so much for being here and welcome to the podcast.
Business Partners, Not a Service Team
Jessica Lachs: Thank you so much for having me. I’m very excited to be here.
Four Advantages of Centralization
Lenny Rachitsky: So you’ve built one of the largest and most respected data teams in all of tech. I’ve heard from a number of people that look to you for advice when they’re trying to build and scale their data teams. And then DoorDash in particular is an incredibly complex business. There’s three or maybe even four sites to the marketplace. There’s this operational element. From the outside, it just feels extremely complicated and wild. I imagine from the inside it’s even more wild. Let’s talk about some of the things you’ve learned about building and scaling the team. You have a fairly contrarian perspective on how to structure data teams. This was referenced when we had Elizabeth Stone on the podcast too. She approaches data the same way. So I’d love to hear just your take on how to structure data teams within companies.
Most product managers will tell you, “Paperwork sucks.” Anvil’s document SDK helps teams get to market fast, incorporate your brand’s style, and give you back time to focus on your company’s core differentiated features. For your users paperwork often starts with an AI-powered web form styled and embedded in your application. From there, you can route data to your backend systems and to the correct fields in your PDFs via API. Complete the process with a white labeled e-signature. The best part about Anvil is the level of customization their SDK provides. Non-technical folks love Anvil’s drag-and-drop builder and developers love their flexible APIs and easy-to-understand documentation.
Build documents software fast with Anvil, that’s useanvil.com/Lenny to learn more or start a free trial. That’s useanvil.com/Lenny.
Jessica Lachs: There’s two main things that I think are important when you’re structuring a team. The first is I believe that analytics should have a seat at the table just like engineering and product and the business folks, the operators. For me, analytics is a business impact driving function and not purely a service function. I think there are analytics teams at other companies where they are answering people’s questions, maybe even through Jira tickets, we’re building dashboards. That was never really of interest to me. That wasn’t the team that I wanted to build.
For me, it’s about finding opportunities, about having a point of view on the decisions that we should make, not just answering the why but answering the so what. “So what do we do now that we know this?” And so that’s definitely one thing as far as my point of view on building a data team. I think the second thing which may be a little more contrarian is I think there are people out there who think that analytics should be embedded into business units. I strongly disagree. I believe a central model, a center of excellence is superior and I’m happy to talk about why, but that’s something that I feel quite strongly about. We’ve tried it or I shouldn’t… well, we’ve experimented in the past with the alternative, so putting it into a business unit and it’s just much more problematic and I think the value you get from a central model is far greater than some of the things that you might lose.
The A-Team Legacy
Lenny Rachitsky: Yeah, let’s definitely talk about it. And just to make sure people understand, when you say central versus embedded, is that in terms of reporting lines, in terms of their goals?
Jessica Lachs: It’s a great question. So mostly it’s in terms of reporting lines because I think on the goal side, that is something where we have the same goals that our partner teams have, and I think that that’s actually an important part of a successful central model. So when I say central model, it just means that for marketing analytics, marketing analytics is part of the broader analytics team. It does not sit and report in through marketing. Just to clarify.
Balancing Exploratory Work and Requests
Lenny Rachitsky: Got it. So the reporting functions at some companies, there’s the head of marketing or some partners to the head of marketing where the data, say, analyst or biz ops people or data scientists would report potentially to them and that’s it. And they’re not as connected to the core, to the rest of the data team, the rest of the analytics team versus-
Jessica Lachs: Exactly. Yeah.
Deep Dive into Recommendation Channels
Lenny Rachitsky: Yeah.
Jessica Lachs: So you’d have a bunch of smaller, of course, data teams that sit embedded within the functions. And I understand why business leaders like that. You’re embedded within the function, so you’re a part of the team. That ownership, that camaraderie that comes with that, I think you can solve for that. But I do understand that that is a benefit. I think the other benefit of course is the business leaders control the roadmaps so they get to dictate the work. They know that they have help and resources in that area when they need them. So that certainty, that control, I totally understand the value there, but I think that those are two things that you can solve for if you know that those are the biggest issues with a central team. So for us, we have a central analytics team, but we are divided up into pods that map perfectly with how product engineering, operations marketing are structured as well.
And so our team de facto has these folks embedded with our partner teams, even though the reporting structure is up through a central org through me. And that helps the team to feel like they are one team, both in terms of the analytics team feeling like it’s one team, but also to use the marketing example, the marketing folks are one team and because the analytics shares the same goals as the marketing leaders, your incentives are aligned to work on the most important things and your success is their success and vice versa. So I think that that’s been really a happy medium, but still preserves all the benefits of a central org. And there are a lot of them.
Learning to Say No
Lenny Rachitsky: I want to hear about them, but I think something that some people may think when you say essential org is like a silo data team that sits there and they’re like a service org a little bit within the company. It’s like, “Hey, I need some data help.” And you try to convince that, “Hey, I need some help on this thing.” And that’s not what you’re saying.
Jessica Lachs: Oh, no. No, no, no. That job seems terrible. I don’t want that job. No, to the earlier point, we have a seat at the table. We are business partners, we are thought partners with our product counterparts, with our engineering counterparts, with our ops counterparts, and we again, share the same goals and have the same initiatives that they do. And it’s just our job to come at it from a data-driven place. We bring to the table insights on things that we’ve noticed, deep dives that we do to understand the problems that we’re trying to solve better. If we need to grow, what are the most efficient ways to grow? What are the trade-offs that we have to make? Where are their pockets of opportunity? That is what I expect my team to be able to bring to that table, the proverbial table, that we want to see that. And in order to earn their spot, that’s the deal. We get the seat at the table and we need to earn it by bringing opportunities that we all can go and go after.
Key Traits Valued in Hiring
Lenny Rachitsky: Awesome. So in a sense, it is embedded. They’re embedded in cross-functional teams across the org, but they report up to essential org to you essentially in the end?
Jessica Lachs: Yeah.
The Case Interview Method
Lenny Rachitsky: Cool. What are some of the benefits of this approach?
Jessica Lachs: Oh, there’s so many. Okay, so the first thing is a consistent and high talent bar. I think this is something I saw when we would have some pockets of analytics folks embedded is having a consistent bar for talent in terms of what we’re looking for, what are the technical skills, what are the soft skills? And being able to evaluate candidates with that same bar, using our same rubric. You just get more consistent and higher talent in my opinion. I think that’s number one. Number two is actually growth opportunities. So if you’re siloed, you may be the most senior data person within… I keep picking on marketing. But you might be the most senior data scientist within marketing. Where do you go from there? I think when you have the central org, you’re able to see if there are growth opportunities in other areas within the company.
And so that really helps folks to stay engaged because they can look at new problems if the problems they’ve been working on for several years are getting maybe boring and they want something new, there’s an opportunity, move from marketing over to merchant analytics. And then I think similarly, if there isn’t a promotion or room to grow, if you want to be a people manager and there just isn’t a people management role within your functional area, well, you’ve got 10 other ones to look at and maybe there is that opportunity. So I think it helps with the growth opportunities for the team, which helps to retain talent. So that’s a second thing. The third thing is just consistency of methodologies and metrics. So you don’t have sales that was as defined by one team and sales as defined by another team. You just have sales and everybody is using the same metrics, the same methodologies, and you’re able to improve your methodologies with input from more people.
And rather than recreating the wheel, building the same churn prediction model on six different teams. You can instead build one and have the input of six different teams. I think that’s definitely another benefit. Also helps you just scale because you start to see the same problems across teams and so you’re like, “Ooh, this is an issue that we need to get ahead of. This is something we need to automate,” or, “This is something that we need to improve upon,” or, “a problem that is going to grow as our business, as our teams scales.” So I think it helps you see around corners a little bit more.
And then just lastly, there’s a team culture brand. I think that’s really important, not just externally for recruiting top talent, but the team is really proud to be members of the analytics team. We have a unique culture of learning, of sharing. You have someone you can go to talk about your challenges. You have someone who can peer review your work. I think just having that team culture that we have is really important. And it’s a lot harder to get when you have the individual silos, particularly in an earlier stage when it’s a smaller team, you just don’t have as many people around. Everybody wants to have friends at work and we’re creating an environment where they can find like-minded data nerds.
Valuing Non-Traditional Backgrounds
Lenny Rachitsky: It makes me think about Airbnb’s first data team. I don’t know if you know Riley Newman well, but he built Airbnb’s first data team and it was actually an analytics team. They called themselves the ‘A-Team’ on the point of culture, and that always felt a lot of fun and they loved being part of that team.
WeDash Project and Ownership
Jessica Lachs: Yeah. We have the same, but now I feel a lot less special for coming up with that name.
The Art of Choosing Metrics
Lenny Rachitsky: Oh, you called it A-Team also?
Common Currency: Unified Metrics System
Jessica Lachs: Yeah, we got the A-Team, yeah.
Lenny Rachitsky: And then I think they moved away from it when there was a push. Now we’re data scientists, we’re not analytics or analysts. And that was like, I don’t know, 10 year ago, like [inaudible 00:15:08] data science. We’re data scientists.
Translating Metrics Across the Business
Jessica Lachs: We’ll always be the A-Team.
Lenny Rachitsky: There’s so many threads I want to follow here, one that’s a tangent, but something that I think a lot of people struggle with is you talked about how you want your data team, your analytics team to be proactive, to find opportunities, to give you ideas, to help you figure out what to build, not just answer questions. At the same time, there are many questions that teams need to get answered. Do you have any advice for just how to set up a team where they both find time to explore, dig, show opportunities and come up with big ideas and also, “Hey, we just need to figure out the funnel conversion on this thing,” or “Hey, what do you think? What’s happening in China right now?” Thoughts there?
Simplifying Metrics: Merchant Health Example
Jessica Lachs: Yeah, such a good question. I think it’s something that never gets easier. You have to be very intentional to carve out time for exploratory work for deep dives because as you mentioned, there are always more questions and more work to be done than hours in the day. And so I think being intentional about it and setting goals for your team around finding these insights through self-directed work is an important mechanism for holding ourselves accountable to that goal because it tends to be the first thing that goes when you get a lot of inbounds, you’re like, “All right, well, let’s deep dive on something that I don’t know if it’s really something. It could be high ROI, it could be low ROI, I don’t know.” So the expected value is lower than this known thing that I can deliver and make someone happy.
So I think to prevent that time from just slipping away, you really have to be intentional. We would do hackathons for our team to carve out days to just go and look into these really interesting things and find opportunities. And I think we have the support of our business partners because so many great insights have come from these deep dives and it really has been some of the work that drives future roadmaps. So they’re always really great at allowing us to have this time and actually encourage us often to have this time for some self-directed work, to go find the next big opportunity.
Team Focus and Metric Rotation
Lenny Rachitsky: If there’s no answer that comes to mind, that’s totally cool. But is there an example of one of these insights that someone on the data team came up with that led to something big for DoorDash that you’re able to share?
Jessica Lachs: So one interesting example was from a hackathon we did a couple of years ago where we were looking at referral as a channel for consumer acquisition. And when you compare that channel to others, it was below average in terms of the engagement you’d see from consumers who came through that channel and the payback period. And rather than just lowering spend on referrals and moving right along, we really wanted to understand what was happening. And so during the hackathon, we did a deep dive into referral. We actually tried referring each other. We tried committing referral fraud, creating new accounts to get around rules. And we uncovered a lot of fraudulent behavior through this deep dive. We ordered so many cupcakes to the office. I remember using referral credits because you had to place an order to be able to get the referral bonus. So we would create the account, place the orders, and we just kept ordering cupcakes.
And what we noticed was that referral as a channel was a bit misleading when you would look at the average in terms of payback and that it was really a bimodal distribution and you had one group of really great consumers who were referring other really great consumers, and the payback on those consumers was really strong. In fact, if that’s all you saw, you would spend a lot more on that channel.
And then what was happening was you had this other group of consumers that were not as good people who were posting referral codes online and getting people who were just in it to get free discounts and credits. And we had at that point in time, pretty lax fraud rules. And we didn’t have caps on these things. All of which came about from this deep dive where we found that this group of consumers was really a drag on the efficiency of this marketing channel. And so I think that’s an example of a few things that we like to do at DoorDash. One being these deep dives and taking the time to really understand the problem and then ultimately make a bunch of recommendations for what we should do, including better fraud checks, caps on referrals, et cetera, et cetera. But also how the average can be incredibly misleading. And so looking at distributions and trying to break down what you’re seeing to find ways that you can optimize in ways that you can gain in efficiencies.
Proxy Metrics vs Long-Term Outcomes
Lenny Rachitsky: That’s an awesome story, great memory to come up with that one. So this is a really good example of a way to carve out time for the data team to think long-term, think look for opportunities, find big ideas. So the hackathon is one idea. Imagine many data people are struggling often to push back on asks that are just like, “h, we need to know. We just need this one thing. Here’s a question, just answer this one question part.” Do you have any advice to data to get better at pushing back? Sounds like a bit of cultural like, “We have time, we need to work on these bigger things.” But just any advice for data leaders or data ICs to find time for these sorts of things?
Focusing on Edge Cases and Failures
Jessica Lachs: Yeah, saying no to someone is never fun. I think as a self-proclaimed people-pleaser, you don’t want to say no, especially when it’s something you can do and you know that you can very easily with maybe an hour’s work, make someone happy. I think it’s really important to establish a culture and for leadership to really establish the rules of working and that operating model so that some of the junior folks aren’t forced to always have to say no. And I think one of the ways we do that is through our goaling. So because our goals are the same as our business partners, we’re able to pretty easily say, “Hey, we’ve got a limited amount of time. These are our goals. What are the most important things that we are going to work on this week or this month in order for both of us to hit our goals?”
And so when something comes up to be able to say, “Hey, this data poll that you want me to do, is this more important than these other three things that I was going to be working on? Yes or no?” And I think sometimes people don’t necessarily realize the trade-offs, and when you make them apparent and you put them front and center, they realize that, “Oh, actually, you know what? That asset’s not important. That can wait.” So I think that that’s definitely something I would recommend, which is always share the trade-offs. Don’t suffer in silence with, “How am I going to do all four of these things?” Bring it up and say, “Hey, this is what I was planning to do. If you want me to do this extra new thing, then one of these other things is going to have to drop.
I personally don’t think that your ask is more important than these three things, but maybe there’s new information, maybe there’s context I don’t have, so let’s talk about it.” Rather than just being like, “No, I won’t do that.” That’s not a great approach either. I think having the conversation and constantly reevaluating your prioritization to make sure you’re working on the most important things or your team is working on the most important things is really good hygiene to have with your business partner. So some teams do that through a weekly standup like, “Here’s what we’re going to do this week. Do we like this prioritization? Do we not?” Some folks do it less formally than that. I think you got to figure out what works for you. But to the earlier point, it’s a conversation with your engineering partner, your product partner, your ops partner, you’re all on the same team, you’re all trying to achieve the same goals and you’re all incentivized to have your analytics team working on the most impactful things.
Why Focus on Extreme Negativity
Lenny Rachitsky: This advice is great for any role basically. And if I were to summarize it to a couple words, it’s just prioritize and communicate what your priorities are and then align on the trade-offs of shifting your priorities.
Jessica Lachs: Every once in a while you just throw one over and say, “You know what? This is quick. I’ll do it.” At least I do. I think sometimes just knock it out, build some goodwill. I think that that’s also important. But usually it’s not something you can do in five minutes and in that case it’s that ruthless prioritization for sure.
Blind Spots That Data Misses
Lenny Rachitsky: And then there’s also the side that you talked about of just show that you can provide value doing these things that are longer term, like prove your worth. “Hey, look at all these opportunities I found for our team over time, I should keep spending time on these other areas,” versus the on fire stuff.
Jessica Lachs: Exactly.
Managing a Global Data Organization
Lenny Rachitsky: When you’re hiring people for your team, I’m curious what you look for and you think is incredibly important that maybe other people aren’t prioritizing as much. What do you focus on when you’re hiring?
Jessica Lachs: Yeah. So everybody needs to have a certain set of technical skills. I think that’s a non-starter. We have a technical bar, we do a technical screen. So I think that’s table stakes. There’s some really unique characteristics that I’ve noticed when I look at some of the top talent that I’ve had on the team or have on the team. I think the first thing is just curiosity. You can’t teach curiosity, or at least I haven’t found a way to do it. If somebody else knows how, please let me know. Somebody who is just self-motivated to pull on the threads when they find them. So they don’t just answer a question. They’re like, “Hmm, this thing seems a little odd. I’m going to dig in and look. Even though I could say I’m done, I answered the question, I did the thing I was going to do.” The person that has that curiosity, something seems off, something doesn’t really make sense and goes and proactively looks into what that is. That is just so valuable. So I really look for that curiosity and that self-motivation to do it without being told.
Applications of AI Tools
Lenny Rachitsky: How do you test for that? How do you do that in an interview and get a sense of if they’re good at that?
An Unconventional Approach to Talent
Jessica Lachs: One way you can do it through the questions you ask is have something that is not quite right within the case that you’re presenting and see if people notice first and foremost. And even if they don’t, if you point it out like, “Where do they go with that?” I think that that’s something that you can test for. I think you can also ask for examples that for these folks typically will highlight this, they’ll talk about, “I noticed this thing, and so we decided to investigate.” So I think that there are ways that you can get that signal through the interview process, but it’s really hard. I think testing for hard skills is a lot easier than testing for soft skills. And I think in some of the questions we ask, we’ll ask a question with the idea that we’re assessing something separate than what the question is necessarily asking. And I think that this is one example of where that really works.
Lightning Q&A Round
Lenny Rachitsky: You said that you give them a case. What does that look like? What is the actual approach to how you do this interview?
Korean Sunscreen Recommendation
Jessica Lachs: Our interview process has in the early stages a coding exercise. So we do our technical screen and a shortened version of a business case. So real world problem solving. Typically, it’s something actually from DoorDash history, like a real problem that we had to see how people can problem solve on the fly. I think that that’s an important skill to be able to have, which is, how do you take a problem, break it down, talk through it. A little bit like some of those consulting cases that you hear about, but something that’s really rooted in real problems. And I think you can learn a lot from those types of cases where, yes, you get to see how people handle ambiguity and structured problem solving, but ultimately most people get something wrong. They make an assumption that’s wrong because well, I would hope that the interviewer knows the business better than the interviewee.
And seeing how people react to being told they’re wrong is a really important signal in my opinion. Seeing how people respond, how they’re able to take new information and pivot, how they’re able to make a decision. So that’s another thing that I like to see in cases where, hey, you may not know the real right decision. You might say, “Hey, I could see it going one way, I could see it going the other way.” But I always push people to say, “If you had to make a call right now, what would it be?” So are people able to have a point of view without full information because that’s life. Sometimes you have to just pick a direction and make a decision even though you don’t have perfect information. So I like to see some of these softer skills and how they manifest throughout a case interview, even if it’s not specifically what I’m asking with the literal problem we’re solving in the case.
Lenny Rachitsky: Along these lines, but in a different direction. You don’t actually have a deep data science data background before you got into this stuff. I know you had some art background, you had an art portfolio back in school, and I think a lot of people wouldn’t imagine that for someone being head of analytics for a company like DoorDash. I don’t exactly know the question, but I guess is there anything there that you think would be interesting for people to know or hear?
Personal Life Motto
Jessica Lachs: Yeah, it’s funny. I joke that I have a job I’d never be hired for because I don’t have a traditional data science background. And I know that Elizabeth Stone on her podcast with you talked a lot about her non-traditional background for a CTO. So hey, maybe there’s something to it. But I became a data scientist out of necessity. I completely self-taught in terms of SQL and Python and I did it because there was a need at DoorDash for someone to help figure out what the right goals were, how we set those goals, how we were performing different markets early in the DoorDash story, so 10 years ago at this point. And I think I just gravitated towards that type of work and Tony recognized that superpower in me even though I don’t have that formal training. So yeah, I’m a bit of an artist for fun, but I guess a data scientist in practice or for career.
But I think that that non-traditional background has been a great thing because I’m able to hire people who have the technical skills that I don’t have, the folks with PhDs in statistics and the data scientists, machine learning and otherwise. I am able to hire those folks and yet keep them really focused on driving business impact because my background was on the finance side, and so I’ve always been a pragmatist. And for me, the purpose of our team is to drive business impact. And so the mix between the technical skills of the smarter people that I’ve hired, the smarter than myself, and my grounding in driving business impact has been a really great partnership.
Lenny Rachitsky: That’s quite an inspiring story for someone that is just starting out and doesn’t necessarily have a lot of experience in data, but also just generally. I think this is a really cool example. You could be successful in a field that you don’t have a ton of background in. I’m curious what you think it was in you that allowed you to succeed in this and get to where you are today. What do you think you did right or what is some habits or ways of thinking that you think helped you achieve that?
Biggest Career Influencers
Jessica Lachs: First off, I have imposter syndrome like everybody else. So it’s not like I have this crazy sense of confidence of like, “Oh, I can do anything.” I definitely have the same doubts that others have. I think part of it was probably not even realizing what I was doing. When you’re at a startup and things are moving quickly and you see a problem, and I’ve always liked solving problems, so I was like, “All right, how do I solve this problem?” It was like, “Oh, well, I need access to the data. I don’t have access to the data. All right, I’ll ask an engineer to get me the data. Well, this isn’t going to scale. I can’t always bother an engineer, so how do I figure out how to get the data myself? Well, let’s learn Python.” So I think it happened organically and I don’t think I realized at the time what I was even doing.
And then I think if you think about things from first principles about what you need right now in front of you to unblock yourself or solve a problem, and you just focus on that instead of thinking about a global org that you’re trying to build. I think that that helps. So for me, it was always about solving the problem in front of me the best way I could. And if that meant I needed to hire an engineer to report into me through the finance org, then that was what we were going to do and nobody was going to tell me I couldn’t do it. So I think it’s a belief in yourself, and ultimately it’s just my desire to solve problems and figure out what has to get done is, I think, ultimately how it came about.
Lenny Rachitsky: I love that so much. There’s so many elements there that I think a lot of people can learn from. I feel like there’s also this underlying current of you’re just motivated for this to work. You wanted DoorDash to succeed, and you’re just like, “I will do what I need to do to make this happen. I need to solve these problems. I’m not going to overthink. Do I have the skills necessarily to do these things [inaudible 00:33:48]?”
Realizing DoorDash Would Succeed
Jessica Lachs: Yeah, I think I’m competitive. I think that’s a trait that you find in a lot of early DoorDash folks and current DoorDash folks, to be honest, just wanting to win and being willing to do whatever you need to win. So roll up your sleeves, do something that’s not your job. I think back to early days of taking out the garbage on Saturday nights because it needed to get done. I think that that was something that is ingrained in our culture from Tony Xu, from our founder and CEO, and I think that really resonated with me, and I feel like I’ve always operated that way as well. And I think that that helped me in my career to be able to do what I’ve done without really thinking about it too much.
Final Outro and Goodbye
Lenny Rachitsky: Are there any other memories or stories of the early days of DoorDash that would be fun to share? Something that sticks with you of like, “Wow, I can’t believe that’s what it was like?”
Jessica Lachs: Oh man, there’s so many, including so many mistakes that we’ve made. But I think something that really stands out to me is before I moved to the analytics area, I was actually a GM. I was the first GM at DoorDash and I was in Boston in 2014 launching the city of Boston when nobody knew who we were. And we would wake up early in the morning, 5 A.M. and we would go out, it was the winter of 2014. We’d go out and we’d hand out promo codes consumers outside of the [inaudible 00:35:30] in Boston, and these promo cards would be attached to kind bars so people would take them. And the whole team, it was a small team, there were four of us, but the whole team would go out in the morning to do this. And I think back to our sales guy, shout out to Joey G. So Joe Graccio is our sales guy in Boston-
Final Outro and Goodbye
Lenny Rachitsky: [inaudible 00:35:47] Joey G.
Jessica Lachs: And he was gold on signing merchants on the platform.
That was how he was gold. His compensation was tied to that. And yet in the morning when we would go out, he was with us handing out promo codes because he was part of the team because he wanted to win. We wanted to grow the business. And I think that that is just a great example of the culture that Tony and the early employees and Stanley and Andy, other co-founders really instilled in all of us early in those days. So I think that that ownership, that extreme ownership of the outcome is definitely one of the things.
I think the other is just being very customer first. And I say customer, I mean consumers, dashers and merchants as all being our customers. And the first time I ever went to the office headquarters in Palo Alto, which at the time was in an animal hospital. The first time I went there, there was a huge site outage and the whole company, it was like 20 people at the time, the whole company jumped online to do customer support, to answer the phones, to make sure that folks were getting refunds for orders that weren’t going through, make sure the orders that were out there were getting delivered, just dropped everything and hopped on to do support.
And I was brand new, didn’t really know how to use the tools, and so it was like, “How can I be useful?” And so back in those days, we used to order dinner to the office using DoorDash. And so in order to preserve about three dashers who would’ve had to deliver food to us, I was like, “I’m going to go out, go out dashing, go get everyone pizza so that we could feed the masses doing credits and refunds and do what we had to to make sure that we were serving our customers well.” And I think that night was one of the largest refunds as a percent of our bank account that we had ever given out. And I think Tony, there were two examples that he’s talked about where we just gave a lot of money back to customers because it was the right thing to do because our service failed and we wanted to do right by them.
So I think that those are two stories that stick out in my mind and really highlight culturally what makes DoorDash unique and what I think has been a really important part of our success.
Lenny Rachitsky: It reminds me of the story that Tony and all the early employees, and I imagine you did this just like, “We’re dashers,” it’s like a rotation where you dash for a while. Is that part of the culture?
Jessica Lachs: Yeah, so we have a program, a WeDash program, and Keith Yandell, who’s our chief business officer, did your podcast last year and he talked about this. But four times a year all the employees go out and go dashing or do customer support, and it’s part of our culture that I love. I actually go pair dashing, so I go together with one of my colleagues. We’ve done it for years now, and it’s a fun thing that we do together four times a year. Actually, usually more than that. And it’s important because you get to use the product, you build empathy with all the audiences. I think all of us order DoorDash a lot, so we’ve built empathy with consumers. But being able to go and understand what it’s like to go out dashing and when you’re in the restaurant going and talking with merchants and seeing the experience from their point of view, I think it’s just incredibly important. And of course we find a lot of bugs like, “Hmm, this doesn’t work the way it should, let me report this.” So I think it’s also just great for catching bugs in the product.
Lenny Rachitsky:
The next era of companies deserves more than an inflexible one-size-fits-all CRM. Join modal, replicate 11 labs and more, and scale your startup to the next level. Head to at attio.com/Lenny and you’ll get 15% off your first year. That’s attio.com/Lenny.
I want to come back to a thread, something you mentioned where you and a lot of the early team had felt extreme ownership over the company and that’s why a lot of this stuff happened. For people, every founder, every product team, they’re going to like, “Yes, we need that. Let’s make sure everyone on the team feels extreme ownership.” Is there anything that you think that the early team did to create that or is it hiring, just pick people that will have that feeling already, or is cultural?
Jessica Lachs: I think it’s both. It’s definitely cultural. I think it comes from the top and I think that Tony exhibits this extreme ownership and looks for it in others. So I think that helps. But I think even today I expect of my team that same extreme ownership over the outcomes. And so I’m more interested in our team figuring out how to solve a problem than the box that someone fits in like, “I’m a data scientist and so I only do these things.” Right? It’s like, “No. I mean, yes, you are a data scientist, but your goal is to figure out what’s happening, and if that means that you’re going to pick up the phone and call customers, then that is what you’re going to do.” And I think that expecting that and setting that as the norm for the team, this ownership of the outcome is something that we continue to do at DoorDash and instill in everyone whether you were early or just joined last month.
Lenny Rachitsky: Is there an example of that that comes to mind of someone practicing extreme ownership, like a data scientist calling someone or something along those lines?
Jessica Lachs: Yeah, so I actually had a meeting yesterday morning with the team that’s working on some of our affordability initiatives and we had shipped something that we expected to work, and it didn’t. And instead of, “You can dig into the data,” to understand the segments of consumers that you would expect it to work with and those that it wouldn’t, of course we did that. But ultimately it was like, “I don’t know why.” And that’s where qualitative research is superior to quantitative research, it’s asking for the context, to actually talking to people to figure out what was the motivation, what worked, what didn’t for them. And so the team, data scientists included, just sat and made phone calls. And so they were talking about what they found from those phone calls and that’s going to inform future decisions. And I think rather than saying, “Well, that’s what the qualitative research team is supposed to do,” it’s like, “No, no, no, that is what our team, anyone’s team is supposed to do because that’s what’s needed to unblock us from this next test that we want to run because we need to know what we are testing.”
So I think that it happens every day. I think I really love when I see team members go outside the traditional bounds of what a data science role might be and do some product management work, do some engineering work. I think that that’s part of what keeps the job interesting. I think it’s part of what makes our team special is that that is not only allowed, it’s encouraged, and probably also a reason why we’ve had folks who’ve gone from my team to the product org and to the ops org and to the finance org is because they get to do and experience parts of that job and get a good sense for what that’s like and then realize it’s something that they love. So I think it’s definitely something we encourage at DoorDash.
Lenny Rachitsky: I love that. I want to move in a slightly different direction. One of your colleagues told me that you are incredibly good at defining metrics, which is so important to get right for a business, especially when it’s complex at DoorDash. And I hear you’re especially good at finding the right metric to drive the right incentive, especially when the business is really messy and things like that. So I’m just curious what you’ve learned about how to pick good metrics and align incentives well.
Jessica Lachs: I’ve learned a lot of things about metrics, mostly from bad metrics. I actually think you learn a lot from picking the wrong metric. Ultimately, you want to find a short-term metric you can measure that drives a long-term output. So people always talk about, “Oh, we want to drive an improvement in retention.” Retention is a terrible thing to goal on because it’s almost impossible to drive in a meaningful way in the short term, and yet you want to be able to experiment and iterate quickly. So what are the things that drive retention? What are the inputs? So I think it’s really important to find the right inputs, and then through experimentation test whether or not those short-term inputs are driving the long-term output that you’re looking for. I think that’s one thing. I think keeping things simple is another thing I’ve learned over the years, maybe it’s data scientists, but they tend to love these composite metrics with a coefficient.
“We’re going to wait this input at X and this input at X+2.” And then you end up with a metric that nobody really understands that doesn’t actually mean anything. And you’re like, “I don’t know if a 0.1 increase is it a lot? Is it good? Is it bad?” So they’re just hard to work with.
And so I always encourage folks, just pick something simple, even if it’s not perfect and your composite would be more perfect. If people understand it, if they have an intuition around it, if it’s something that people can talk about across the company, it’s going to be a much better metric in terms of driving real outcomes than your made up composite score that nobody understands. So I think keeping things simple is also really important. And then I think the last thing I’ll say is it’s important to understand how metrics across the company equate to one another.
And so we spend a lot of time quantifying things in terms of a common currency. So for example, if I were to lower price by a dollar, what would I get in terms of, we’ll say, volume? Well, what if I lowered delivery times by a minute? What do I get for that in terms of volume? And so now you can make trade-offs between maybe your marketing team and your logistics team because you have this common currency that everyone can talk about. And so we’ve done that. We’ve tried to quantify all of the levers of our business, price, selection, quality in common terms, so that if we have, say, a dollar to spend, we know what we get depending on where we put it, over what timeframe. I think that that helps us make decisions more quickly because we know what our options are. We know we have our inventory of things that we can do, short-term, long-term, and what we get for it. So it definitely helps us to make decisions more quickly and hopefully better decisions.
Lenny Rachitsky: These are so awesome. I definitely want to follow up on some of this. This is so good. So maybe on this last one, which we did at Airbnb also like, how does everything translate into Knight’s book and booking? Every decision we make, what is the actual Knight’s book impact? And so I imagine in your case, I don’t know if you want to talk about these things. I imagine it’s transactions, or purchases, or GMB or something like that, so I’m guessing is the final metric. I don’t know. Is that something you talk about or you don’t talk about that?
Jessica Lachs: We measure things in terms of GOV, so Gross Order Value, and also volume.
Lenny Rachitsky: Got it. [inaudible 00:48:34]. Okay, so basically every other metric that people are gold on as much as you can translate, there’s a model that translates that into Gross Order Value and volume? Awesome. So when a team is saying, “Hey, we’re going to change the onboarding flow and impact conversion here.” I don’t know. I guess what are some examples of other metrics on teams that potentially translate into GOV and volume just to make it even more real?
Jessica Lachs: Yeah. So everything from the example that you started with, which is an improvement in the login flow, how many more consumers are getting onto the app and ultimately placing orders. And so you can translate that to, of course, orders and GOV. But then something as interesting as selling a Thai restaurant in Sacramento, we’re able to say, “What do we think that that gets us in terms of GOV from the consumer by selling that Thai restaurant?” So it’s every area of the business, it’s mobilizing more dashers on the road. What does that do to our quality metrics in terms of delivery times? How does that translate? So because of that, we’re able to figure out if we want to spend the dollar or spend the time, the team’s time, on improving conversion or spending more money in marketing or onboarding more dashers or signing more restaurants or adding more grocery stores. So we are able to look across the whole business and figure out what is the right mix of actions to take to achieve our goals.
Lenny Rachitsky: I could see as you talk about this why this is so important in a marketplace, especially a multi-sided marketplace where there’s always trade-off decisions between supply investment and demand growth and dasher growth. I don’t even know, my brain would explode trying to think about all these things, so I get exactly why this is so important to business. Okay. And then in terms of the simple recommendation, I think when people hear like, “Yeah, keep it simple,” they’re like, “Yeah, yeah, we’re going to keep it simple.” What are some things that point to, “This is not simple,” that tell you like, “No, this is way too complicated. You should try to simplify this metric even though it’s not ideal. It’s not the perfect metric, but it needs to be simpler.”
Jessica Lachs: Yeah. So we had a score for merchant health, which we tried experimenting with, which was a combination of factors that we had found would lead to a merchant being on the platform and getting an order. So we wanted to make sure that the merchant had active hours on the platform and had images and had a full menu that was accurate and robust. A number of different inputs. And we created a composite that weighted all of these different inputs. And then we were like, “What is our merchant health score?” And you were like, “It’s 0.35. It’s not 35%. So what is that, that 0.35? I don’t know what it is.” So instead of that, we said, “What are the most important factors in order… First, let’s measure how many of the new merchants are getting an order within their first, say, seven days on the platform.
And then let’s look at how many of our merchants are doing these things we know are important. So these inputs. So let’s goal our team on getting merchant photo coverage up. Let’s goal the team on making sure that we have open hours, accurate hours.” So yes, someone might say it’s simpler to have a composite metric, but it was so hard to understand what it was and how to move it that it became meaningless. And ultimately moving to something that was simpler to understand, even if it meant having three metrics instead of one, it ultimately was better for the team because folks knew what they were trying to move. And so yeah, maybe we missed number four, five and six on the list of things, but you got one through three and that’s 95% of it anyway. So once we get success with that 95, then let’s talk about figuring out the other 5%.
Lenny Rachitsky: It’s so funny because this is exactly what we went through at Airbnb, we had, we call that a healthy host. I led the host quality team for a while and we came up with this healthy host metric that was six factors of a host, like the cancellation rate, the review rate, their response rate and things like that. And then we’re just like, “Cool, let’s move this, let make more hosts healthy.” And then you end up like, “Okay, which one do we focus on?,” And, “Oh, what about all these others?” And we ended up basically focusing on one at a time. And so let’s just make that the goal for now and then rotate through the different biggest [inaudible 00:53:28] opportunities to move. [inaudible 00:53:30].
Jessica Lachs: Exactly. I think in hindsight for the example you give, which of those six things are actually the most important? And if you’re able to then quantify which one matters most, you work on that one first and you materially move that one and then you work on the next one. You want to move them all. But being able to prioritize and know what you’re going to get for a 20% improvement in, say, your cancellation rate, that’s where analytics I think can add a lot of value. Because yes, ultimately you’ll get to all of them, but the way you do that and the time can have a meaningful impact on your growth. If you can target the most problematic things first and solve those, you get more bang for your buck and that compounds over time. And so doing the things that matter first and most quickly is a competitive advantage in my opinion.
Lenny Rachitsky: The other thing we found along those same lines is rotating between different metrics is so not efficient because you get good at, “We’re going to move this metric.” And your team’s like, “Cool, we totally understand this lever,” like cancellation rate. We become really smart at cancellation rate and then three months later, you need to switch to response rate and they have to learn a whole new paradigm of how to think about it. And it’s just super inefficient. So we found basically, just keep a team on the metric until there’s no more opportunities and give another team one of these other metrics.
Jessica Lachs: Yeah.
Lenny Rachitsky: So many lessons. Okay. And the first thing you said on how to pick a good metric about this idea of short-term metrics that have long-term impact. How did you phrase that again?
Jessica Lachs: Yeah, so we find proxy metrics for long-term outcomes.
Lenny Rachitsky: Awesome. And it’s similar to the simple metric, and it all comes down to, again, just like the metric should be something probably, you can move, you can understand, that’s close enough to this ideal, perfect metric, but isn’t necessarily the entire ideal. Okay, awesome. Anything else along these lines of just picking metrics, working with metrics that you’ve learned that would be worth [inaudible 00:55:28]?
Jessica Lachs: With metrics, we are often looking at the average, and I think we talked about this a little bit earlier, but making sure that you’re looking at the edge cases and your fail states is also really important. And so we often will set goals actually and create metrics around those edge cases. So like the disaster deliveries, the ones that go terribly wrong. So we have this concept of Never Delivered, which is orders that are never delivered. We’re really great at naming things at DoorDash, and they’re very rare. And so if you were just looking at the average effect or the average consumer experience, it would never come up. If you were just measuring quality based on average values of delivery times and lateness [inaudible 00:56:18], these wouldn’t show up because they are so rare, but they’re terrible. They’re terrible experiences for consumers. They lead to churn.
They’re incredibly expensive because you’re refunding an order or repurchasing food and having to send another dasher to deliver that repurchased food. So they’re very expensive, they’re costly from a consumer experience standpoint. And I think if you’re not looking for these fail states, they are often missed. So I think when you’re picking metrics, yes, you want to improve engagement and you want to improve conversion, and there’s a lot of things that are averages overall that you want to move, but it’s so important to find these edge cases in these fail states and actually set concrete goals around eliminating them because it can be really powerful.
Lenny Rachitsky: So the tip here is actually make that a goal like, never deliver at some team, just keep cutting that down?
Jessica Lachs: Exactly. So we have part of our quality analytics team and we have product engineering and ops on it as well. Their goal is to eradicate Never Delivered. And in order to do that, you have to understand why they happen. Sometimes it’s human error, sometimes it’s fraud. And then figure out ways that you can prevent them, that you can fix them while it’s happening and ultimately just get rid of them from the system. And you’re never going to completely get rid of them, but you can make a meaningful impact to make them even more rare than a fraction of a percent.
Lenny Rachitsky: Yeah. And I feel like people may be hearing this and like, “Of course, why would you not focus on terrible work experiences?” But I think in most companies, they look at the big numbers, they look at the averages as you said like, “Oh, it almost never happens. Why do we even spend any time on this?” And your point is, you should actually spend time on these really terrible experiences, even if it’s a tiny portion of your business. I guess maybe share why that’s important. Is it just because that has trickle-down effects on the brand?
Jessica Lachs: Yeah, I think it’s a couple of things. So just because something doesn’t happen frequently doesn’t mean that it’s not important. So the Never Delivered example is a great one in that this is leading directly to churn and it’s also costing a lot of money far more than its frequency would suggest. And I think the fact of the matter is is when you have things that cause churn, you’re losing all of that consumer’s subsequent orders, and that is not necessarily observed. You’re just seeing one bad experience, you’re not seeing all of the lost orders because they’re lost. And so I think that sometimes this is an area where the data doesn’t show you the full picture. And being able to quantify the impact on engagement, on profitability, will make it stand out as something that really, that you would maybe miss if you weren’t really looking for it.
And then I think the other thing is with something like login errors, sometimes you don’t see it in the data because people can’t even get into the data. If you’re not able to log in, you’re not making any purchases, you’re not ordering, and so you may not see it in the data that you’re looking at. And so that’s also something that I think is important for data folks to think about, which is what data don’t we have? What data might we be missing? Where might there be opportunities and things that we actually need to identify and fix that we may not see? Because in this case, with login failures, they’re not able to log in. They’re not in the denominator, and so we’re missing out on them from the data set entirely.
Lenny Rachitsky: Just a couple of more questions. There’s one that I skipped that I’m just going to come back to. It’s completely out of nowhere, but I think it might be interesting is about a global data org. So you run a global data org, you have data scientists and analysts and biz ops people all over the world, not just the US. I’m curious just how is it different managing data people in different countries versus just the US? What’s a big difference?
Jessica Lachs: Everyone always asks about the differences. What I am surprised by is how similar things are, how similar people are, the data scientists themselves, but also consumers and dashers and couriers, as we call them at Volt. There’s a lot more similarities than differences. I do think that when you built a business in the US and then you introduce new countries, having different currencies and different languages adds complexity that you weren’t necessarily familiar with. I think similarly in EU countries versus non-EU countries in Europe, there’s different regulation. So that adds a fun layer of complexity. So I do think that it adds complexity to the problem set, but ultimately so many of the problems are the same. It feels a little bit like going into a test having seen the answer key. And so for me, there are problems we’ve encountered at Volt through Volt analytics where I’m like, “Oh, we’ve had a similar problem.
I have an instinct for what the answer might be. Let’s still test because there could be differences cultural or otherwise, but I feel like I know where we’re going to end.” And then sometimes there are problems where it’s new for one reason or another, and it’s exciting because you’re like, “All right, let’s see if things are different here.” Let’s see what ideas might work in a Volt country that don’t work in a DoorDash country and vice versa. So I think I tend to focus more on what’s the same, and then I’m pleasantly surprised when I find things that are different because that keeps you on your toes and keeps things interesting.
Lenny Rachitsky: I’m going to take us to AI Corner. This is a segment we have in the podcast where I try to understand how people are using AI in their day-to-day and in their business. I’m curious if you’ve found some really interesting way of using AI ideally in… you can go in either one of these directions, and how you or your team work day-to-day using AI tools to make you more efficient, or integrating AI into your product, making DoorDash better.
Jessica Lachs: Yeah, I think that there are opportunities in both. I think one of the things I’m really excited about is actually the former. So in helping to make the team more productive, we do something called Office Hours at DoorDash, the analytics team. And it’s something that we started eight years ago, and it was a way to provide support for teams that at the time we just didn’t have the bandwidth to support. So we would go, in the early days, we’d go sit in a room and we’d say, “Come on in and we’ll help you with anything you need help with. We’ll help teach you SQL. We’ll help look at some of your work. We’ll be a thought partner. You could just come learn what we’re working on.” Whatever it was. We would do two hours every week of Office Hours at different times to be friendly to different time zones.
And I think one of the things I’m excited about is being able to really empower some of the folks that are still coming to Office Hours for one thing or another to be able to use AI to help edit queries on their own for example, to be able to say, “Here’s a query. I want to make this. Please adjust this to our grocery business so that I can see the GOV for grocery.” And so working to build these tools that will help not just our team in terms of time saving, and also to be honest, folks are going to use it on our team, but really to be able to empower non-technical users to be able to do things on their own and not have to take up bandwidth for the analytics team.
Lenny Rachitsky: So essentially it’s a chatbot that anyone in the company can talk to you to get advice on how to write SQL queries, query data and things like that?
Jessica Lachs: Yeah.
Lenny Rachitsky: Is there a clever name for this chatbot per chance?
Jessica Lachs: So it’s not clever. It’s called Ask Data AI, and that’s named for our internal Slack channel that used to be the open Q&A for people to ask data. So it’s not at all clever.
Lenny Rachitsky: But it’s clear.
Jessica Lachs: But again, it goes with the theme of very, very specific naming conventions that we have at DoorDash; Never Delivered and Ask Data AI.
Lenny Rachitsky: I love it. Just clarity above all else. That’s something I’ve learned from an editor that I work with. Jess, is there anything else that you want to share or leave listeners with? For folks that are trying to build their data teams, make their data teams more efficient, is there any final wisdom nugget you’d want to share?
Jessica Lachs: I think the only thing that I want to reiterate is that you don’t necessarily need a formal training in whatever it is you’re building. And I think that also goes towards the folks that you hire onto the team. And so I mentioned earlier that we’ve had a lot of folks go to product or go to ops from the team. What I didn’t mention is how many folks we’ve actually had join the analytics team from partner teams. So whether that was from engineering or from our ops team or marketing or finance, we are a net importer of talent as opposed to a net exporter of talent. And I think that that’s because my own experience coming over from operations, from being a GM and making that transition into analytics, I find that I’m drawn to other folks who want to make a similar transition.
Now again, you have to have the technical skills, and most of these folks have acquired these skills on the job, whatever job they are doing at DoorDash before they transition to the analytics team, or they had maybe some formal training in school. But I love seeing the folks that make that transition and actually want to join the analytics team, even if they’re not a career or data scientist. I think it creates a really unique environment where you have folks on the team from different backgrounds with different expertise who can teach each other things. So I can teach you how to build a discounted cash flow model in Excel, and I can learn how to make kick-ass slides from someone who has a background in consulting. And I can learn about common gotchas in statistics from someone who comes to us with a Master’s or a PhD in statistics, and we’ve got our econometrics folks and we’ve got our economists. We just have a group of people with different backgrounds who can all teach each other how to be better. And we’re not all carbon copies of each other.
Lenny Rachitsky: What I’m hearing is you try to optimize almost for a lot of different complementary skills and very different backgrounds almost.
Jessica Lachs: Exactly. And also people who have experience at different size companies. I think I love folks from startups who have that hustle and grit, but I also love folks who’ve seen what scale looks like and can help us see around corners as far as what problems we will encounter as the business is growing. And I think it is not just about a diversity of skill and a diversity of background, it’s also diversity of prior company and stage. That can be really a unique way to think about structuring your team so that you get the best of both worlds.
Lenny Rachitsky: Amazing. Well, just when you thought we were done, we reached our very exciting lightning round. Are you ready?
Jessica Lachs: I am. Let’s do it.
Lenny Rachitsky: Let’s do it. Okay. First question, what are two or three books that you’ve recommended most to other people?
Jessica Lachs: I tend to read fiction, particularly historical fiction, and I love spy novels. So I think my brain is always in problem-solving mode even when reading. A recent book that I read that I enjoyed was The Rose Code by Kate Quinn, and it’s about women code breakers in World War II, and I really enjoyed that. But rather than recommending a book… I guess I did just recommend a book, but rather than recommending another book, I am going to recommend the Libby app and supporting your local public library because I love the library and I love Libby, so I’ll give that as my other recommendation.
Lenny Rachitsky: Beautiful. Very on brand with sharing economy, company stuff. Libby. Cool. Okay, next question. Favorite recent movie or TV show?
Jessica Lachs: Yeah, another one. I don’t actually watch a lot of TV, definitely don’t watch a lot of movies. In fact, haven’t seen some of the movie greats. I get yelled at a lot by my friend. “I can’t believe you haven’t seen that.” I tend to re-watch things, so series from the past, over and over again. I think it’s just like how I shut my brain off. So I’ve recently re-watched The West Wing, which is one of my favorite shows of all time, probably for the 50th time.
Lenny Rachitsky: Oh my God.
Jessica Lachs: And Alias, which was a Jennifer Garner series from the early 2000s. Also, Spy. So I’m noticing a theme. I think I really love the spy genre. But yeah, I’ve watched those. They’re both great, but not at all current.
Lenny Rachitsky: Perfect. Perfectly acceptable. Do you have a favorite product that you recently discovered that you really love?
Jessica Lachs: This is a bit of a curveball. So Korean sunscreens. So I burn really easily, so I have to wear sunscreen and I love Korean sunscreens. I was introduced to them by a friend of mine, and they’re just far superior to what we have in the US. So I highly recommend people give Korean sunscreens a try, particularly there’s a Beauty of Joseon on branded sunscreen. It’s just amazing and is delightful to wear, which is important when you have to wear it every day.
Lenny Rachitsky: I’ve been trying to wear more sunscreen as I age, and so this is a really good tip. Was that a brand you recommended?
Jessica Lachs: Yeah. So Beauty of Joseon is the brand.
Lenny Rachitsky: Beauty of Joseon.
Jessica Lachs: There’s another brand, Isntree, which also has a great sunscreen. But I’ll be honest, almost every Korean sunscreen I’ve tried is great.
Lenny Rachitsky: Okay. I’m Googling this as soon as we get off. Do you have a favorite life motto that you often come back to and share and or share with family and friends even more [inaudible 01:11:39]?
Jessica Lachs: I do. So there’s a John Steinbeck quote, which I’m not big on quotes, but I like this one, which is that, “It’s a common experience that a problem difficult at night is resolved in the morning after the committee of sleep has worked on it.” I find that that’s something I really live by. First off, I love sleep and I try to get as much of it as possible. But the other thing is that if I’m stuck on a problem or if I am writing a response to something like a tense issue or an emotional issue, often I find that if I put down my thoughts, go to sleep, check it in the morning, I end up with a better outcome. So all of a sudden you have a new perspective and clarity on a problem you were stuck on, or you realize that you weren’t clear in the way you were communicating your thoughts because you were emotional about something and you’re able to put together a much better response to an e-mail or to whatever problem you’re handling. So sleep can solve lots of problems.
Lenny Rachitsky: I love sleep as well. I’m always telling my wife, “Let’s go to sleep.” Like, “Okay, I’ll be there soon.” I love that advice. Okay, two more questions. Who’s influenced you most in your career? Is there someone that comes to mind?
Jessica Lachs: So I think two answers, a multi-part answer. So I think first my career has been in male-dominated industries and I’ve worked with just some incredible women who’ve really influenced me. When I was a banker, there were two senior bankers, Vanessa Roberts and Gina Tarone at Lehman Brothers where I worked. And they were just so incredible. They were just so good at their jobs and I found that really inspiring.
And then at DoorDash, Tia Sherringham, who is our GC, and Liz Jarvis-Shean, who leads comms, are just dominant in their fields. And I think that that’s really empowering and have been big influences on me to just see strong, powerful women kicking ass and that helps me believe that I can do the same. So that’s one answer.
And then the other answer, sort of cliche, but my parents. My mom was a statistician at the UN before she got married, and she actually chose to stay home and raise three children, so I’m the youngest. And when I was in, I think it was elementary school, decided to go back to school, switch careers and become a nurse. And so the fact that she embarked on this completely new career in her forties after 15 years as a stay-at-home mom and my father supported this. I think that that was really, really influential and was probably the first time I saw that you can do whatever you put your mind to, no matter your age, no matter your circumstances. So that was really influential and I don’t think I’ve ever told her that. So hi, Mom.
Lenny Rachitsky: Hi Mom. Thank you, mom.
Jessica Lachs: Yeah, I think that was influential for my career. Definitely.
Lenny Rachitsky: That’s a beautiful answer. Fun fact, I worked with Liz at Airbnb. Your person you just mentioned in the comms team.
Jessica Lachs: [inaudible 01:14:57]. She is great.
Lenny Rachitsky: She’s amazing. Final question. So when you joined DoorDash, imagine it wasn’t obvious that it was going to work. I imagine it was still like, “This was a crazy idea. Maybe it’ll work, maybe not.” Is there a moment you recall where you’re like, “I think this is going to be a big success? I think this is actually going to work out?”
Jessica Lachs: To be honest, I went into DoorDash because I wanted to learn for the experience. I thought it was interesting, problems with interesting people. I never thought too much about whether it would work. I of course wanted it to work and was very competitive and wanted to win. I think there’s two moments that stand out. One was when the third party market share data showed that we had become the number one player after, I think we started at number four or five. And I think that that was really exciting to see the trajectory and to see us gain in category share. That was exciting. I probably didn’t see it until months after it had happened because we don’t spend a ton of time focusing on it, but I do remember somebody wanted to include the graph in some presentation, some sales material, and we’re like, “Oh, we’re number one. That’s incredible. We used to be number five.” So I’d say that that was one.
The other one that stands out was, the first talk I gave in a lot of these startup talks in the early days in Boston, and I’d asked the audience, “How many of you have used DoorDash?” And there’d be like three people who would raise their hand. And then it was a few years ago, maybe 2018, 2019, and I was giving a talk and I asked the audience, “How many of you have used DoorDash?” And almost everyone’s hands went up. And that was actually pretty memorable for me because in my mind, we were still the small startup that no one had heard of where I had to over enunciate the D’s in DoorDash. So people didn’t think I worked for Jordash, the nineties’ denim company. And so that was pretty meaningful to me when just so many people had used the product or were consumers of DoorDash. It was pretty exciting. And I still get excited. I saw DoorDash mentioned in a book recently that I was reading. It was like, “We’re in the book.” So those little things when you become part of the cultural lingo that I think are really, really special.
Lenny Rachitsky: Well, I’m a very happy customer of DoorDash. I’ve never had a Never Deliver. It’s always there, sometimes a little late. Usually it’s perfect. Thank you for everything you do. Go team DoorDash.
Two final questions. Where can folks find you online if they want to follow stuff that you do? I know you’ve been doing more writing on LinkedIn and things like that, so just help people understand where to find you and how can listeners be useful to you?
Jessica Lachs: Yeah, so as you mentioned, to find me LinkedIn, I don’t have a huge social presence, but I am on LinkedIn and I am currently writing a series of blog posts about my experience building a global analytics org at DoorDash. Some of the lessons I’ve learned over the last 10 years. So definitely check those out.
And as far as your second question of how listeners can be useful to me, I guess read the post on LinkedIn and I’d love to hear what people think, whether you agree with my point of view or not. That being said, be nice. I want honest feedback, but I want kindness as well. So yeah, just engage with the content and let me know what y’all think. I think I do have a broader ask, which is just to encourage folks listening to TruthSeek, something I take seriously at DoorDash. It’s a company value. But there’s a lot of misinformation out there and it’s often up to us as individuals to figure out what’s fact and what’s fiction. So I have a plea for folks to do your best, to search for the truth and speak the truth, and I think we’ll all be better off for it. And of course, use DoorDash.
Lenny Rachitsky: Of course.
Jessica Lachs: Yes, there are three things that listeners can do.
Lenny Rachitsky: You’re at DoorDash.com. That was awesome. I love that last point as well in addition, to use DoorDash. Jessica, thank you so much for being here.
Jessica Lachs: Thank you for having me. It was a lot of fun.
Lenny Rachitsky: Same for me. Bye, everyone.
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcast, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lennyspodcast.com. See you in the next episode.
Glossary
| English | 中文 |
|---|---|
| affordability initiatives | 可负担性项目 |
| AI Corner | AI Corner(AI 问答环节) |
| Alias | 《Alias》 |
| Andy | Andy |
| Ask Data AI | Ask Data AI |
| Beauty of Joseon | Beauty of Joseon |
| case interview | 案例面试 |
| center of excellence | 卓越中心 |
| central model | 集中模式 |
| churn prediction model | 流失预测模型 |
| common currency | 共同货币 |
| composite metric | 复合指标 |
| couriers | couriers(配送员) |
| dashboard | 仪表盘 |
| dashers | 骑手(dashers) |
| data nerd | 数据极客 |
| data org | 数据组织 |
| discounted cash flow model | 折现现金流模型 |
| Elizabeth Stone | Elizabeth Stone |
| embedded | 嵌入式 |
| extreme ownership | 主人翁意识 |
| first principles | 第一性原理 |
| GC | GC(首席法务官) |
| Gina Tarone | Gina Tarone |
| GM | GM(城市总经理) |
| GOV (Gross Order Value) | GOV(总订单价值) |
| healthy host | 健康房东 |
| imposter syndrome | 冒名顶替综合征 |
| Isntree | Isntree |
| Jira tickets | Jira 工单 |
| Joe Graccio | Joe Graccio |
| Joey G | Joey G |
| John Steinbeck | 约翰·斯坦贝克 |
| Jordash | Jordash |
| Keith Yandell | Keith Yandell |
| Lehman Brothers | 雷曼兄弟 |
| Lenny Rachitsky | Lenny Rachitsky |
| Libby | Libby |
| Liz Jarvis-Shean | Liz Jarvis-Shean |
| merchant health | 商家健康度 |
| Never Delivered | Never Delivered(从未送达) |
| Office Hours | Office Hours(办公时间) |
| people manager | 人员管理者 |
| pod | 小组(pod) |
| Retention | 留存率 |
| soft skills | 软技能 |
| standup | 站会 |
| Stanley | Stanley |
| table stakes | 基本门槛 |
| The Rose Code | 《The Rose Code》 |
| The West Wing | 《白宫风云》(The West Wing) |
| Tia Sherringham | Tia Sherringham |
| Tony | Tony |
| TruthSeek | TruthSeek(追求真相) |
| Vanessa Roberts | Vanessa Roberts |
| Volt | Volt |
| VP of Analytics and Data Science | 分析与数据科学副总裁 |
| WeDash | WeDash |
Reformatted by reformat_english.py
打造世界级的数据组织 | Jessica Lachs(DoorDash 分析与数据科学副总裁)
文字记录
Lenny Rachitsky: 你打造了科技界规模最大、最受尊敬的数据团队之一。
Jessica Lachs: 对我来说,分析是一个驱动业务影响力的职能,而非纯粹的服务职能。不只是回答”为什么”,而是回答”既然知道了这些,我们接下来该怎么做?”
Lenny Rachitsky: 你的一位同事告诉我,你在定义指标方面极其出色。
Jessica Lachs: 留存率(Retention)是一个糟糕的目标指标。在短期内几乎不可能以有意义的方式去推动它。最终,你需要找到一个可测量的短期指标,用它来驱动长期结果。
Lenny Rachitsky: 你提到了早期团队。我感受到了极强的主人翁意识(extreme ownership)。
Jessica Lachs: 是的,你是数据科学家,但你的目标是弄清楚到底发生了什么。如果这意味着你要拿起电话给客户打电话,那就去做,挽起袖子干。
嘉宾介绍
Lenny Rachitsky: 今天的嘉宾是 Jessica Lachs。Jessica 是 DoorDash 的分析与数据科学副总裁,在那里她打造了科技界规模最大、影响力最强的数据团队之一。她在 DoorDash 已经工作了超过十年,是 DoorDash 的首位总经理,负责开拓新市场。在此之前,Jessica 创办了 GiftSimple——一家社交礼品创业公司,并在雷曼兄弟(Lehman Brothers)开启了她的职业生涯,从事投资银行业务。
在这次对话中,我们深入探讨了如何打造和扩展数据组织,包括为什么集中式组织模式如此有效、招聘数据人才时应该看重什么、如何为团队选择正确的指标以对齐激励并驱动正确的成果,以及 DoorDash 的数据团队如何帮助业务做出更好决策的实际案例,还有大量关于 DoorDash 早期的精彩故事。如果你喜欢这档播客,别忘了在你喜欢的播客应用或 YouTube 上订阅和关注,这是避免错过后续节目的最佳方式,也对播客帮助极大。
那么,让我请出 Jessica Lachs。Jessica,非常感谢你来做客,欢迎来到播客。
Jessica Lachs: 非常感谢你的邀请。我很高兴来到这里。
如何构建数据团队
Lenny Rachitsky: 你打造了科技界规模最大、最受尊敬的数据团队之一。我听不少人说过,他们在打造和扩展自己的数据团队时会向你寻求建议。DoorDash 本身也是一个极其复杂的业务——市场有三方甚至四方参与,还有运营层面的要素。从外面看就觉得极其复杂和疯狂,我猜想从内部看可能更加疯狂。我们来聊聊你在打造和扩展团队方面学到的一些东西吧。你在数据团队的结构搭建上有一个相当反主流的视角。之前 Elizabeth Stone 上播客的时候也提到了这一点。她对待数据的方式和你一样。所以我很想听听你对公司内部数据团队应该如何搭建的看法。
集中式 vs 嵌入式:数据分析团队的架构选择
Jessica Lachs: 在搭建团队时,我认为有两件主要的事情很重要。第一,我认为分析应该像工程、产品和业务方——运营人员一样,在决策桌上占有一席之地。对我来说,分析是一个驱动业务影响力的职能,而非纯粹的服务职能。我知道其他一些公司的分析团队是通过 Jira 工单来回答人们的问题,或者搭建仪表盘(dashboard)。这从来不是我感兴趣的方向,也不是我想打造的团队。
对我而言,分析的核心在于发现机会,在于对我们应该做的决策有自己的观点,不只是回答”为什么”,而是回答”所以呢”——“既然知道了这些,我们接下来该怎么做?“这是我对建设数据团队的一个核心理念。
第二点可能更有些反主流:我知道有些人认为分析应该嵌入到各个业务单元中去。我强烈反对。我认为集中模式(central model),即卓越中心(center of excellence)的模式更优。我很乐意谈谈原因,但这是我非常坚定的一点。我们过去尝试过——或者说我们试验过另一种方式,即把分析放入业务单元,结果问题要多得多。而且我认为集中模式带来的价值远远超过你可能因此失去的一些东西。
Lenny Rachitsky: 好,我们一定要聊聊这个。不过先确保大家理解,当你说集中式与嵌入式时,指的是汇报线的归属,还是目标的设定?
Jessica Lachs: 这个问题问得好。主要是指汇报线的归属。因为就目标而言,我们的目标和合作伙伴团队的目标是一致的,而且我认为这其实是一个成功的集中模式的重要组成部分。所以当我说集中模式时,意思就是——比如营销分析,营销分析是整个分析团队的一部分,它并不隶属于营销部门内部并向营销汇报。澄清一下。
Lenny Rachitsky: 明白了。也就是说,在某些公司,营销负责人或者营销负责人的搭档手下会有数据分析师、业务运营人员或数据科学家直接向他们汇报,就这样。他们与核心团队、与数据分析团队的其他部分联系不那么紧密,而——
Jessica Lachs: 完全正确,是的。
Lenny Rachitsky: 嗯。
嵌入式小组与集中式组织的折中
Jessica Lachs: 所以你会有一批较小的数据团队,嵌入到各个职能部门中。我理解为什么业务负责人喜欢这种模式。你嵌入在职能部门里,你就是团队的一部分,那种归属感和团队凝聚力随之而来,我认为这些是可以解决的。但我确实理解这是一个好处。另一个好处当然是业务负责人掌控路线图,由他们来决定工作内容。他们知道需要时有帮助和资源可用。那种确定性和掌控感,我完全理解其价值。但我认为,如果你知道这些是集中式团队最大的问题,这两点都是可以解决的。所以我们有一个集中的分析团队,但我们被分成了若干小组(pod),与产品、工程、运营、营销的组织结构一一对应。
因此,我们的团队事实上已经嵌入到了合作伙伴团队中,尽管汇报关系是通过集中式组织向上到我这里。这让团队感觉他们是一个整体——既让分析团队觉得自己是一个团队,也以营销为例,让营销人员觉得自己是一个团队。因为分析团队与营销负责人共享相同的目标,大家的激励是一致的,都会去攻克最重要的问题,你的成功就是他们的成功,反之亦然。所以我认为这真的是一个很好的折中方案,同时仍然保留了集中式组织的所有好处。而好处确实很多。
Lenny Rachitsky: 我很想听听这些好处,不过我觉得有些人听到你说集中式组织时,可能会联想到一个孤岛式的数据团队,坐在那里,有点像公司内部的服务部门——“嘿,我需要一些数据方面的帮助”,然后你得去说服他们:“嘿,我这有个事需要帮忙。“但这不是你说的那种模式。
不是服务部门,而是业务伙伴
Jessica Lachs: 哦不,完全不是。那个工作听起来太糟糕了,我可不想做。不,回到前面说的,我们在桌前有一席之地。我们是业务伙伴,是思维伙伴——与产品方、工程方、运营方都是如此。我们同样共享他们的目标,推进相同的倡议,只是我们的职责是从数据驱动的角度切入。我们带到桌面上的是我们观察到的洞察、所做的深入分析,帮助更好地理解要解决的问题。如果需要增长,最高效的增长方式是什么?需要做哪些权衡?机会的切入点在哪里?这就是我期望我的团队能够带来的东西——所谓的桌面上——我们要看到这些。而为了赢得这个席位,这就是交换条件:我们在桌前有一席之地,但需要通过提供大家可以一起追逐的机会来赢得它。
Lenny Rachitsky: 太棒了。所以某种意义上,确实是嵌入式的——他们嵌入到跨职能团队中,遍布整个组织,但最终向你汇报,向集中式组织汇报?
Jessica Lachs: 对。
集中模式的四大优势
Lenny Rachitsky: 很好。这种做法有哪些好处?
Jessica Lachs: 哦,太多了。好的,第一点是统一且高水平的人才标准。我以前看到一些分散嵌入的分析人员时就有这个感受——在人才方面拥有统一的标准,包括我们看重什么、技术技能要求是什么、软技能要求是什么,并且能够用同样的标准、同样的评分表来评估候选人,在我看来你会得到更一致、更高质量的人才,这是第一点。第二点是成长机会。如果你被隔离在某个部门里,你可能是……我又拿营销举例了。但你可能是营销团队里最资深的数据科学家,然后你往哪里发展?当有一个集中式组织时,你就能看到公司其他领域是否有成长机会。
这确实帮助人们保持投入感,因为他们可以接触新的问题——如果手头做了好几年的问题开始变得无聊,想要新鲜感,就有机会从营销转到商户分析。同样,如果没有晋升或成长空间,比如你想做人员管理者但职能领域内没有这样的岗位,你还有十个其他选择可以看,也许机会就在那里。所以我认为这有助于为团队提供成长机会,从而帮助留住人才,这是第二点。第三点是方法论和指标的一致性。这样就不会出现一个团队定义的销售指标和另一个团队定义的销售指标各不相同的情况。你只有一种销售定义,每个人都使用相同的指标、相同的方法论,而且能够汇集更多人的输入来改进方法论。
与其在六个不同团队重复造轮子、分别构建相同的流失预测模型,不如构建一个,汇集六个不同团队的输入。这绝对是又一个好处。同时也帮助你更好地扩展规模,因为你开始在不同团队间看到相同的问题,于是你会说:“哦,这是一个我们需要提前应对的问题,需要自动化的东西,“或者”这是一个我们需要改进的地方,“又或者”一个随着业务和团队扩展而会增长的问题。“所以我认为这帮助你看得更远一些。
最后一点是团队文化品牌。我认为这非常重要,不仅仅是对外招募顶尖人才,团队成员本身也很自豪能成为分析团队的一员。我们有独特的学习和分享文化。你有可以倾诉挑战的对象,有人可以同行评审你的工作。拥有这样的团队文化非常重要,而当你有各自孤立的团队时,这要难得多——特别是在早期阶段团队较小的时候,身边没有那么多人。每个人都想在工作中有朋友,我们正在创造一个环境,让他们找到志同道合的数据极客。
A-Team 的传承
Lenny Rachitsky: 这让我想到 Airbnb 的第一个数据团队。不知道你是否熟悉 Riley Newman,他组建了 Airbnb 的第一个数据团队,实际上就是一个分析团队。他们自称”A-Team”,从文化的角度来说,那种感觉很有趣,他们很享受成为那个团队的一员。
Jessica Lachs: 对,我们也是一样的。但我现在觉得自己没那么特别了,因为我们起了同样的名字。
Lenny Rachitsky: 哦,你们也叫 A-Team?
Jessica Lachs: 对,我们就是 A-Team。
Lenny Rachitsky: 后来我记得他们放弃了这个名字,当时有一股风潮——“我们是数据科学家,不是分析师。“那大概是十年前的事了,数据科学热的时候,“我们是数据科学家。”
Jessica Lachs: 我们永远是 A-Team。
平衡探索性工作与响应需求
Lenny Rachitsky: 这里有很多条线索我想展开。其中一个是有点偏题的话题,但我觉得很多人都在为此苦恼——你谈到希望你的数据团队、分析团队能够主动出击,发现机会,提出想法,帮助你们决定该构建什么,而不仅仅是回答问题。但与此同时,团队也有许多问题需要得到回答。对于如何搭建一个既能腾出时间去探索、深挖、展示机会、提出大想法,又能回答”嘿,我们得搞清楚这个漏斗的转化率”或者”嘿,你觉得中国那边现在是什么情况?“这类问题的团队,你有什么建议吗?
Jessica Lachs: 这个问题问得太好了。我觉得这件事永远不会变得更容易。你必须非常刻意地为探索性工作、深度分析腾出时间。因为正如你所说,问题和待做的工作永远比一天中的小时数要多。所以我认为,要刻意安排这件事,并围绕通过自主工作来发现洞察为团队设定目标,这是一个重要的机制,让我们对此目标保持负责。因为当你收到大量需求时,这类工作往往最先被挤掉——你会想:“好吧,让我去深挖一个我不确定是不是真有价值的东西,它可能是高回报,也可能是低回报,我不知道。“所以它的期望值低于那个我能明确交付、让别人满意的已知任务。
所以我认为,为了防止这段时间就这样悄然流失,你必须非常刻意地去安排。我们会为团队举办黑客马拉松,专门腾出几天时间让大家去研究那些真正有趣的事情、发现机会。而且我觉得我们得到了业务合作伙伴的支持,因为如此多很棒的洞察都来自这些深度分析,这些工作真正驱动了未来的路线图。所以他们一直很乐意让我们拥有这段时间,实际上还经常鼓励我们留出时间做一些自主工作,去发现下一个大的机会。
推荐渠道的深度分析案例
Lenny Rachitsky: 如果一时想不到也没关系。但你能否举一个例子,说明数据团队的某个人通过这种方式发现了某个洞察,最终为 DoorDash 带来了重大成果,而且是你可以分享的?
Jessica Lachs: 一个很有意思的例子来自我们几年前做的一次黑客马拉松。当时我们在研究推荐(referral)作为用户获取渠道的效果。当你把这个渠道和其他渠道做比较时,从通过该渠道进来的用户参与度以及回收周期来看,它是低于平均水平的。但我们没有简单地削减推荐渠道的投入就了事,而是想真正搞清楚到底发生了什么。所以在黑客马拉松期间,我们对推荐渠道做了一次深度分析。我们真的互相推荐,尝试进行推荐欺诈,创建新账户来绕过规则。通过这次深度分析,我们揭露了大量欺诈行为。我们往办公室点了超多纸杯蛋糕——我记得是用了推荐优惠券,因为你要下一单才能拿到推荐奖励。所以我们创建账户、下单,然后就一直不停地点纸杯蛋糕。
我们注意到,当你看推荐渠道的平均回收周期时,这个指标其实有点误导性。它实际上是一个双峰分布——有一群非常好的用户,他们推荐了其他同样优质的用户,这些用户的回收周期非常强劲。事实上,如果你只看到这部分数据,你会在这个渠道上投入更多。
而另一边发生的情况是,还有另一群质量不那么好的用户,他们把推荐码发到网上,吸引来的都是冲着免费折扣和优惠券而来的人。而我们在当时,欺诈规则相当宽松,也没有设置上限。所有这些问题都是通过这次深度分析发现的——我们发现这群用户确实拖累了这个营销渠道的效率。所以我觉得这个例子体现了我们在 DoorDash 喜欢做的几件事:一是做深度分析,花时间真正理解问题,然后最终给出一系列关于我们应该怎么做的建议,包括更好的欺诈检测、推荐上限等等。同时也体现了平均值可能会产生多大的误导,所以要去观察分布,尝试把所见的数据拆解开来,找到可以优化的方式,找到可以提升效率的空间。
学会说”不”
Lenny Rachitsky: 这个故事太精彩了,能想起这个例子真好。这是一个非常好的例子,展示了如何为数据团队腾出时间来思考长期、寻找机会、发现大想法。黑客马拉松是一个方法。我想很多数据从业者在面对”哦,我们需要知道这个,我们只需要这一个东西,这里有一个问题,回答一下就好”这类需求时,往往很难拒绝。对于数据人员更好地学会拒绝,你有什么建议吗?听起来这有点文化层面的因素——“我们需要时间来做这些更大的事情。“但不管是数据管理者还是数据个人贡献者,对于如何为这类工作争取时间,你有什么建议吗?
Jessica Lachs: 对别人说”不”从来都不是一件容易的事。作为一个自称的讨好型人格,你不想说”不”,尤其当那是你能做到的事情,而且你知道也许只需要一个小时的工作就能让别人开心。我认为建立一种文化、由领导层真正确立工作规则和运营模式非常重要,这样一些初级同事就不会总是被迫去当那个说”不”的人。我觉得我们做到这一点的方式之一就是通过目标设定。因为我们的目标和业务合作伙伴是一致的,所以我们能够比较容易地说:“嘿,我们的时间有限。这些是我们的目标。为了让我们双方都能达成目标,这周或这个月我们要做的最重要的事情是什么?”
所以当新的事情冒出来时,你可以说:“嘿,你要我做的这个数据查询,比我原计划要做的另外三件事更重要吗?是还是不是?“我觉得有时候大家并不一定意识到存在取舍,当你把这些取舍摆出来、放在显眼的位置时,他们就会意识到:“哦,其实你知道吗?那个东西没那么重要,可以等等。“所以这绝对是我会推荐的做法——永远把取舍亮出来。不要默默忍受”我到底怎么才能把这四件事都做完”的煎熬。把它摆出来说:“嘿,这是我原计划要做的事情。如果你要我做这个额外的新任务,那这些事情中就得有一项被搁置。”
Jessica Lachs: 我个人并不认为你的需求比这三件事更重要,但也许有新的信息,也许有我不了解的背景,所以我们来讨论一下。“而不是直接说”不,我不做。“这也不是什么好方法。我觉得与业务合作伙伴保持这种对话、不断重新评估优先级,确保你或你的团队正在做最重要的事情,这真的是非常好的工作习惯。有些团队通过每周站会来做这件事,比如”这是这周我们要做的事情。这个优先级排序我们满意吗?不满意?“有些团队做得没那么正式。我觉得你要找到适合自己的方式。但回到之前的观点,这是与你的工程合作伙伴、产品合作伙伴、运营合作伙伴之间的对话,你们都在同一个团队里,都在努力实现同样的目标,你们的激励机制都是让分析团队专注于最有影响力的事情。
Lenny Rachitsky: 这个建议基本上适用于任何角色。如果让我用几个词来概括,就是确定优先级、沟通你的优先级,然后在调整优先级的取舍上达成一致。
Jessica Lachs: 偶尔你也可以通融一下,说”你知道吗?这个很快,我来做吧。“至少我是这样的。我觉得有时候就把事情做了,积累一些好感,这也很重要。但通常来说能在五分钟内搞定的事情很少,那种情况下就确实需要果断的优先级取舍了。
Lenny Rachitsky: 然后还有你提到的另一面,就是要展示你在做这些长期事情时能提供的价值,证明自己的价值——“嘿,看看我为团队发现的这些机会,我应该继续花时间在这些其他领域”,而不是只做救火的事情。
Jessica Lachs: 没错。
招聘中看重的特质
Lenny Rachitsky: 当你为团队招人的时候,我很好奇你看重什么,以及你认为有哪些非常重要的东西可能是其他人没有足够重视的。你在招聘时关注什么?
Jessica Lachs: 是的。每个人都必须具备一定的技术技能,我觉得这是基本门槛。我们有技术标准,会做技术筛选。所以我觉得这是最基本的。在我观察团队中一些顶尖人才时,我注意到他们有一些非常独特的特质。第一个就是好奇心。好奇心是教不会的,至少我还没找到方法。如果有人知道怎么做,请告诉我。就是那种自我驱动、在发现线索时会主动去深挖的人。他们不只是回答一个问题就完了,他们会想:“嗯,这个东西看起来有点奇怪,我要深入看看。即使我已经可以说我完成了,我回答了问题,我做了该做的事情。“那种拥有好奇心、觉得有什么不对劲、有什么不太合理就会主动去调查到底是什么的人,真的非常有价值。所以我非常看重这种好奇心和这种不需要别人告诉就去做的自我驱动力。
Lenny Rachitsky: 你怎么测试这一点?面试中你怎么判断他们是否擅长这个?
Jessica Lachs: 一种方式是通过你提出的问题——在你呈现的案例中设置一些不太对劲的地方,看看人们首先能不能注意到。即使他们没注意到,你指出来之后,看他们会怎么处理。我觉得这是可以测试的。你也可以让人们举例,这类人通常会主动提到这些,他们会说”我注意到了这个东西,所以我们决定去调查。“所以我觉得通过面试过程是可以获得这个信号的,但这确实很难。测试硬技能比测试软技能容易得多。我们有些问题,提问时的意图实际上是在评估与问题表面不直接相关的东西。我觉得这就是一个很好的例子。
案例面试的方法
Lenny Rachitsky: 你说你给他们一个案例。具体是什么样的?你做这种面试的实际方法是什么?
Jessica Lachs: 我们的面试流程在早期阶段有一个编程练习。所以我们做技术筛选和一个简化版的商业案例,也就是真实世界的问题解决。通常这是 DoorDash 历史上真实发生的事情,一个我们实际遇到过的问题,看看人们如何即兴应对。我觉得这是一项很重要的能力——如何面对一个问题、拆解它、阐述思路。有点像你听说的那些咨询案例面试,但基于真实问题。我觉得你可以从这类案例中学到很多——是的,你可以看到人们如何处理模糊性和结构化的问题解决,但最终大多数人都会有弄错的地方。他们会做出错误的假设,因为——嗯,我希望面试官比面试者更了解这个业务。看到人们被告知错了之后的反应,我觉得这是一个非常重要的信号。看他们如何回应,如何接受新信息并调整方向,如何做出决策。所以这是我在案例面试中喜欢看到的另一面——你可能不知道真正正确的决定是什么。你可能会说:“我觉得可能是这样,也可能是那样。“但我总是会推动人们:“如果你现在必须做一个判断,你会怎么选?“看人们能否在信息不完整的情况下持有自己的观点,因为这就是现实生活。有时候你就是得选一个方向、做一个决定,即使你没有完美的信息。所以我喜欢在案例面试中观察这些软技能是如何体现的,即使那不是我在案例中具体提出的那个问题本身。
非传统背景
Lenny Rachitsky: 顺着这个话题,但换个方向。你在进入这个领域之前,实际上并没有深厚的数据科学或数据背景。我知道你有艺术背景,上学时还有艺术作品集,我想很多人不会想到 DoorDash 这样的公司的分析负责人会有这样的背景。我也不确定具体想问什么,但我猜这方面有没有什么是你觉得大家可能会感兴趣知道的?
Jessica Lachs: 是的,这挺有趣的。我开玩笑说我做了一份永远不会被录用的工作,因为我没有传统的数据科学背景。我知道 Elizabeth Stone 在你的播客中谈到了很多她作为 CTO 的非传统背景。所以嘿,也许这有点道理。但我成为数据科学家是出于必要。SQL 和 Python 完全是自学的,我之所以去学是因为 DoorDash 有这个需求——需要有人来搞清楚正确的目标是什么、如何设定这些目标、不同市场的表现如何,那是在 DoorDash 故事的早期,也就是十年前。我觉得我就是自然而然地被这类工作吸引,而 Tony 在我身上看到了这个超能力,尽管我没有受过正式的训练。所以我算是一个爱好上的艺术家,但职业上或者说实践中,是一个数据科学家。
Jessica Lachs: 但我觉得这种非传统背景其实是一件好事,因为我能招到拥有我所不具备的技术能力的人——那些统计学博士、数据科学家、机器学习等领域的专家。我能够招到这些人,同时让他们真正专注于推动业务影响,因为我的背景在财务方面,所以我一向是一个务实主义者。对我来说,我们团队的使命就是推动业务影响。所以我招来的那些比我更聪明的人的技术能力,加上我在推动业务影响方面的根基,形成了一种非常好的合作关系。
Lenny Rachitsky: 对于一个刚起步、在数据领域没有太多经验的人来说,这是一个很鼓舞人的故事,而且不仅仅是数据领域。我觉得这是一个很棒的例子——你可以在一个没有深厚背景的领域取得成功。我很好奇,你觉得你身上是什么让你能够在这个领域取得成功、走到今天的位置?你觉得你做对了什么,或者说有哪些习惯或思维方式帮助你走到了这一步?
Jessica Lachs: 首先,我和所有人一样有冒名顶替综合征。所以我并没有那种”我什么都能做”的疯狂自信。我确实和其他人一样有自我怀疑。我觉得部分原因可能是我当时甚至没有意识到自己在做什么。当你在一家创业公司,事情进展很快,你看到一个问题,而我一直喜欢解决问题,所以我就想,“好,我怎么解决这个问题?“然后就是,“哦,我需要拿到数据。我没有数据权限。好吧,我去找个工程师帮我拉数据。但这样不可持续,我不能总去打扰工程师,那我怎么自己获取数据呢?好,来学 Python 吧。“所以我觉得这一切是自然而然发生的,我当时甚至没意识到自己在做什么。
如果你从第一性原理出发思考,为了解除自己的阻碍或解决眼前的问题,你现在需要什么,然后把注意力集中在那上面,而不是去想你要搭建一个什么样的全球性组织——我觉得这很有帮助。对我来说,一直是用最好的方式去解决面前的问题。如果那意味着我需要在财务组织下招聘一个工程师向我汇报,那我们就这么做,没有人能告诉我不能这样做。所以我觉得这是一种对自己的信念,而归根结底,就是我解决问题的欲望,搞清楚需要完成什么——我觉得这就是最终的原因。
Lenny Rachitsky: 我太喜欢这段话了。里面有很多很多人可以学到的东西。我感觉到这背后还有一条暗线,就是你发自内心地希望这件事能成。你希望 DoorDash 成功,所以你就想,“为了做成这件事,我需要做什么就做什么。我需要解决这些问题。我不会去过度思考我有没有这些技能——”
Jessica Lachs: 对,我觉得我很好胜。我觉得这是你在很多早期 DoorDash 人身上都能看到的特质,说实话现在的 DoorDash 人身上也有,就是想赢,愿意为赢而做任何需要做的事。卷起袖子,做一些不是你本职工作的事情。我回想早期的时候,周六晚上去倒垃圾,因为那需要有人去做。我觉得这是从 Tony Xu、我们的创始人兼 CEO 那里深深植根在我们文化中的东西,我觉得这和我产生了很强的共鸣,我觉得我一直以来也是这样做事的。我认为这在我职业生涯中帮助了我,让我能够在没有太多思虑的情况下做到今天这些。
Lenny Rachitsky: 还有什么关于 DoorDash 早期的回忆或故事可以分享的吗?有没有什么让你印象深刻,觉得”天哪,真不敢相信那时候是这样的”?
Jessica Lachs: 天哪,太多了,包括我们犯过的很多错误。但我觉得让我印象特别深刻的一件事是,在我转到分析领域之前,我其实是一个 GM(城市总经理)。我是 DoorDash 的第一个 GM,2014 年我在波士顿负责启动波士顿这座城市,那时候根本没人知道我们是谁。我们会早上五点起床出去,那是 2014 年的冬天。我们出去给波士顿地铁站外面的消费者发优惠码,这些优惠卡片会绑在 KIND 坚果棒上,这样人们才会拿。整个团队——当时是个小团队,就我们四个人——早上会一起出去做这件事。我回想起我们的销售哥们, shout out to Joey G。Joe Graccio 是我们在波士顿的销售——
Lenny Rachitsky: 传奇的 Joey G。
Jessica Lachs: 他在签约商家入驻平台方面简直是金牌。
这就是他的强项,他的薪酬也是和这个挂钩的。但早上我们出去的时候,他跟我们一起发优惠码,因为他是团队的一份子,因为他想赢。我们想做大业务。我觉得这就是 Tony 和早期员工,还有 Stanley 和 Andy——其他联创——在那些早期日子里在我们所有人身上植入的那种文化的一个绝佳例子。所以我觉得那种主人翁意识,那种对结果的主人翁意识,绝对是其中之一。
另外一个就是真正以客户为先。我说的客户,是指消费者、骑手(dashers)和商家,他们都是我们的客户。我第一次去帕洛阿尔托的总部办公室——那时候总部还在一家动物医院里。我第一次去的那天,出现了大规模宕机,整个公司——当时大概 20 个人——所有人都上线做客户支持,接电话,确保那些没成功的订单得到退款,确保已经发出的订单能送到,放下手头一切去做客服。
我当时是新人,还不太会用那些工具,所以就想,“我能帮上什么忙?“那时候我们会用 DoorDash 给办公室订晚餐。为了不让大约三个骑手专门给我们送餐,我就说,“我出去跑一单,给大家拿披萨回来,这样就能让正在处理退款和积分、努力为客户服务的人有饭吃。“我记得那天晚上我们给出了占银行账户百分比最高的退款之一。我记得 Tony 谈到过两个例子,就是我们给客户退了很多钱,因为那是正确的事,因为我们的服务出了问题,我们想对他们负责。
所以我觉得这两个故事在我脑海中特别突出,真正体现了 DoorDash 在文化上与众不同之处,也是我认为我们成功中非常重要的一部分。
Lenny Rachitsky: 这让我想起 Tony 和所有早期员工的故事,我猜你也参与了吧,就像”我们去当骑手”——大家轮换去做一段时间骑手。这是文化的一部分吗?
WeDash 项目与主人翁意识
Jessica Lachs: 对,我们有一个项目,叫 WeDash 项目。我们的首席商务官 Keith Yandell 去年上过你的播客,也谈到过这个。每年四次,所有员工都会出去跑单或做客户支持。这是我们的文化,我个人非常喜欢。我其实是结对跑单,就是和一个同事一起出去。我们已经这样做很多年了,每年四次,变成了一件我们一起做的趣事。实际上通常还不止四次。这很重要,因为你可以亲身体验产品,与所有用户群体建立共情。我想我们所有人都会经常用 DoorDash 下单,所以对消费者已经有了共情。但能走出去,了解出去跑单是什么体验,在餐厅里和商户交谈,从他们的视角看整个体验,我觉得这极其重要。当然,我们也会发现很多 bug,比如”嗯,这个功能不是按预期工作的,我去报告一下。“所以我觉得这对发现产品中的 bug 也非常有帮助。
Lenny Rachitsky: 我想回到之前提到的一条线索——你说过你和很多早期团队成员都对公司有着强烈的主人翁意识,所以很多事情才会发生。对于每一个创始人、每一个产品团队来说,他们都会想,“对,我们需要这个。让我们确保团队里每个人都有这种主人翁意识。“你认为早期团队做了什么来营造这种氛围吗?是招聘时就挑选本身就有这种感觉的人,还是文化层面的塑造?
Jessica Lachs: 我觉得两者兼有。文化肯定是有的。我觉得这种意识是自上而下的,Tony 本身就展现了这种主人翁意识,并且在他人身上寻找这种品质。所以我认为这有帮助。但我觉得即使在今天,我对团队也有同样的期望——对结果承担主人翁意识。所以我更关心的是团队能想办法解决问题,而不是某个人被框在什么角色里,比如”我是数据科学家,所以我只做这些事情。“对吧?而是,“不,没错,你是数据科学家,但你的目标是搞清楚发生了什么,如果这意味着你要拿起电话打给客户,那这就是你要做的事。“我认为设定这种期望,把它作为团队的常态——对结果的主人翁意识——是我们一直在 DoorDash 坚持做的事情,无论你是早期员工还是上个月刚加入的。
Lenny Rachitsky: 有没有什么具体的例子?比如有人践行主人翁意识,一个数据科学家打电话之类的?
Jessica Lachs: 有,我昨天早上刚和一个团队开了会,他们在做我们的一些可负担性相关项目。我们上线了一个预期会有效的功能,结果没有。当然,团队做了数据分析,去了解哪些消费者群体预期会有效、哪些不会。但最终发现,“我们还是不知道为什么。“这时候定性研究就优于定量研究了——去询问上下文,真正和人交谈,了解他们的动机是什么、什么有效、什么不行。所以这个团队,包括数据科学家,就坐下来打电话。他们在会上分享了从这些电话中发现的洞见,这些将指导后续的决策。我觉得不是去说”那是定性研究团队该做的事”,而是”不不不,这就是我们团队——任何团队——该做的事,因为这是推进我们下一个测试所必需的,我们需要知道到底在测什么。”
所以我觉得这种事每天都在发生。我真的很高兴看到团队成员跳出数据科学角色的传统边界,去做一些产品管理的工作、做一些工程的工作。我觉得这也是让工作保持有趣的原因之一。我觉得这也是我们团队特别的地方——这不仅是被允许的,而且是受到鼓励的。大概这也是为什么我们有团队成员从我的团队转到了产品团队、运营团队、财务团队——因为他们有机会接触并体验那些工作的内容,对那些岗位有了真切的了解,然后发现自己真正热爱那方面的工作。所以我觉得这绝对是我们在 DoorDash 鼓励的。
指标选择的艺术
Lenny Rachitsky: 我很喜欢这个。我想换个稍微不同的方向。你的一个同事跟我说,你在定义指标方面非常出色,这对一个企业来说至关重要,尤其是像 DoorDash 这样复杂的业务。我听说你特别擅长在业务非常混乱的情况下,找到正确的指标来驱动正确的激励。所以我很好奇,关于如何选择好的指标和有效对齐激励,你学到了什么?
Jessica Lachs: 关于指标我学到了很多东西,主要是从错误的指标中学的。我确实认为从选错指标中学到很多。归根结底,你想找到一个可以短期测量的指标,它能驱动一个长期产出。所以大家总说,“哦,我们想提升留存率。“留存率是一个很糟糕的目标指标,因为在短期内几乎不可能以有意义的方式驱动它,而你又希望能够快速实验和迭代。那么驱动留存率的是什么?输入变量是什么?所以我觉得找到正确的输入变量非常重要,然后通过实验测试这些短期输入是否真的在驱动你所期望的长期产出。这是一点。
另一点我多年来学到的,就是保持简单。也许这是数据科学家的通病,他们总喜欢那种带系数的复合指标。“我们要把这个输入加权为 X,那个输入加权为 X+2。“然后你得到一个没人真正理解的指标,实际上没有任何意义。你会想,“0.1 的增长是多是少?是好事还是坏事?“这种指标就是很难用。
所以我总是鼓励大家,就选一个简单的指标,哪怕它不完美,哪怕你的复合指标会更完美。如果大家能理解它、对它有直觉、能全公司一起讨论,那它在驱动真实结果方面会比你那个没人能看懂的自造复合分数好得多。所以保持简单也非常重要。最后一点我想说的是,理解公司各部门的指标之间如何相互关联也很重要。
共同货币:统一指标体系
Jessica Lachs: 因此我们花了大量时间,用一种共同货币来量化各种事物。比如,如果我降价一美元,能换来多少——比如说——单量?那如果我把配送时间缩短一分钟,又能换来多少单量?这样一来,你就可以在不同的团队之间做权衡取舍,比如市场团队和物流团队之间,因为你们有了一个所有人都能讨论的共同货币。
我们就是这么做的。我们尝试用统一的度量来量化业务中的所有杠杆——价格、品类选择、质量——这样如果我们有一美元要花,我们就知道根据投向哪里、在什么时间范围内,能得到什么回报。我认为这帮助我们更快地做出决策,因为我们清楚自己的选项是什么。我们知道自己手头有哪些可做的事——短期的、长期的——以及每件事能带来什么。所以这确实帮助我们更快地做决策,也希望是更好的决策。
Lenny Rachitsky: 这些太棒了。我真的很想就其中一些继续深入聊,太好了。也许就最后这一点——我们在 Airbnb 也做过类似的事情——每一件事如何转化为间夜量和预订量?我们做的每个决策,实际的间夜量影响是什么?我猜在你的情况中——我不知道你是否方便谈这些——我猜是交易量,或者购买量,或者 GMV 之类的,所以我猜最终的指标是那个。我不知道。这是你可以谈的吗,还是不方便谈?
Jessica Lachs: 我们用 GOV(总订单价值,Gross Order Value)以及单量来衡量。
Lenny Rachitsky: 明白了。[听不清]。好的,所以基本上每个团队被考核的其他指标,都尽可能转化为 GOV 和单量?有一个模型来做这种转换?太棒了。所以当一个团队说”嘿,我们要改注册流程,影响转化率”——我不知道——你能不能举一些其他团队指标的例子,看看它们是怎么转化为 GOV 和单量的,让它更具体一些?
全业务视角的指标转化
Jessica Lachs: 可以。从你刚才提到的例子开始——改善登录流程,能让更多消费者进入 app 并最终下单。这当然可以转化为订单和 GOV。但更有意思的例子,比如在萨克拉门托上了一家泰国餐厅,我们也能说出来:“这家泰国餐厅上线后,从消费者端能为我们带来多少 GOV?“所以这覆盖了业务的方方面面——让更多骑手上路,这对我们的配送时间等质量指标有什么影响?怎么转化?正因为如此,我们才能判断:我们是想花这一美元、或者花团队的时间去改善转化率,还是投入更多营销费用,或者引入更多骑手,或者签约更多餐厅,或者增加更多杂货店。我们能够纵览整个业务,找出实现目标的正确行动组合。
Lenny Rachitsky: 听你讲这些,我能理解为什么这在平台型业务中如此重要,尤其是一个多边平台,在供给投入、需求增长、骑手增长之间总是要做权衡取舍。我都不敢想,想这些事情脑子会炸掉,所以我完全理解为什么这对业务如此关键。
好的。然后关于”保持简单”这个建议,我觉得当人们听到”对,保持简单”,他们都会说”对对,我们会保持简单的”。但有哪些迹象表明”这不简单”——能让你看出来”不行,这太复杂了,你应该试着简化这个指标,哪怕它不完美、不是最理想的指标,但它需要更简单”?
简化指标的实践:商家健康度
Jessica Lachs: 对。我们曾经有一个商家健康度评分,我们试过用这个指标。它是多个因素的综合——我们发现了哪些因素能让一个商家在平台上活跃并获得订单。我们希望确保商家在平台上有活跃的营业时间,有图片,有完整准确的菜单,等等,很多不同的输入变量。我们创建了一个复合指标,给所有这些输入加权。然后我们问:“我们的商家健康度评分是多少?“答案是 0.35。不是 35%,就是 0.35。那这个 0.35 是什么意思?没人知道。
所以我们换了个思路。我们说:“最重要的因素是什么?首先,让我们测量有多少新商家在入驻平台后的——比如说——前七天内获得了第一个订单。然后看看我们的商家中有多少在做那些我们知道重要的事情,也就是这些输入变量。那我们就用提升商家图片覆盖率来考核团队,用确保营业时间准确来考核团队。”
是的,有人可能会说有一个复合指标更简单,但它太难理解了,也不知道怎么去推动它,结果就变得毫无意义。最终,转向更简单易懂的东西——哪怕这意味着用三个指标代替一个——对团队来说反而更好,因为大家知道自己要推动的是什么。也许我们错过了第四、第五、第六重要的事情,但前三件你做了,那就是 95% 了。等我们在那 95% 上取得成功之后,再来讨论剩下那 5%。
Lenny Rachitsky: 太有意思了,因为我们在 Airbnb 经历了完全一样的事情。我们管它叫”健康房东”。我之前负责过房东质量团队一段时间,我们搞出了这个健康房东指标,包含房东的六个因素——取消率、评价率、回复率之类的。然后我们就说:“好,让我们推动这个指标,让更多房东变得健康。“结果你就会发现:“好吧,我们重点推哪个?“然后,“那其他那些怎么办?“最终我们基本上就是一次聚焦一个。那就先把这个作为目标,然后在不同最大的机会之间轮换推进。
Jessica Lachs: 完全正确。我觉得事后来看,在你举的这个例子中,那六个因素里哪些实际上最重要?如果你能量化哪个因素影响最大,你就先做那个,实实在在地推动那个指标,然后再做下一个。你当然想把所有指标都推动起来。但能够排出优先级,知道取消率提升 20% 能换来什么——这就是分析能发挥巨大价值的地方。因为是的,最终你都会做到所有这些,但你做事的顺序和时间安排会对增长产生重大影响。如果你能优先解决最关键的问题,你的投入产出比更高,而且会随着时间复合增长。所以先做最重要的事情、最快地做——在我看来这就是竞争优势。
团队专注与指标轮换
Lenny Rachitsky: 在同样这条线上,我们还发现,在不同指标之间轮换是非常低效的。因为你的团队会变得很擅长:“好,我们要推动这个指标。“你的团队会说:“太好了,我们完全理解这个杠杆了。“比如取消率,我们变成了取消率方面的专家。然后三个月后,你需要转向回复率,他们又得学习一套全新的思维范式。效率太低了。所以我们发现,基本上就让一个团队守着一个指标,直到没有更多空间了,然后给另一个团队分配其他的指标。
Jessica Lachs: 对。
代理指标与长期结果
Lenny Rachitsky: 太多经验了。好的。你一开始提到如何选择好指标,讲了那些具有长期影响的短期指标这个思路。你当时怎么说的来着?
Jessica Lachs: 对,我们会为长期结果寻找代理指标。
Lenny Rachitsky: 很好。这和”简单指标”的思路类似,归根结底还是——指标应该是你能推动的、你能理解的,足够接近那个理想的完美指标,但不一定就是那个完整的理想指标。好的,太棒了。关于选择指标、使用指标,还有什么其他经验值得分享的吗?
关注边缘案例与失败状态
Jessica Lachs: 在指标方面,我们经常看平均值,我觉得我们之前也稍微谈到过这一点,但确保你同时关注边缘案例和失败状态也非常重要。所以我们实际上经常围绕那些边缘案例设定目标、创建指标。比如灾难性的配送——那些出了严重问题的订单。我们有一个概念叫 Never Delivered(从未送达),就是那些从未被送达的订单。DoorDash 在命名方面真的很擅长,而且这种情况非常罕见。所以如果你只看平均效果或消费者的平均体验,它根本不会出现。如果你只根据配送时间和延迟的平均值来衡量质量,这些也不会显现出来,因为它们太稀少了,但它们是灾难性的。对消费者来说是极其糟糕的体验,会导致流失。
它们代价极高,因为你要退款整个订单,或者重新购买食物,还得再派一个骑手去配送那份重新购买的食物。所以成本很高,从消费者体验的角度来说代价也很大。我觉得如果你不去关注这些失败状态,它们往往会被忽略。所以在选择指标的时候,是的,你想提升参与度,想提升转化率,有很多基于整体平均值的指标你想去推动,但找到这些边缘案例和失败状态,并围绕消除它们设定具体目标,这非常重要,因为效果会非常显著。
Lenny Rachitsky: 所以这里的建议是,真的把它变成一个目标——比如某个团队专门负责”从未送达”,持续降低这个数字?
Jessica Lachs: 没错。我们有一个质量分析团队,里面也有产品工程师和运营人员。他们的目标就是消灭 Never Delivered。要做到这一点,你得先弄清楚它们为什么会发生。有时候是人为失误,有时候是欺诈。然后想办法预防,在发生时进行补救,最终从系统中消除它们。你永远不可能完全消除,但你可以做出显著改善,让它们变得比一个百分点的零头还要稀少。
为什么关注极端负面体验
Lenny Rachitsky: 是的。我觉得有人听到这个可能会想,“当然啦,为什么不去关注那些糟糕的工作体验呢?“但我认为在大多数公司,他们看的是大数字,看的是平均值,就像你说的,“哦,这几乎不会发生,我们为什么要在上面花时间?“而你的观点是,你确实应该在这些真正糟糕的体验上花时间,即使它只占你业务的一小部分。也许可以分享一下为什么这很重要?是因为它对品牌有连锁影响吗?
Jessica Lachs: 对,我觉得有几个方面。首先,某件事不常发生并不意味着它不重要。Never Delivered 就是一个很好的例子——它直接导致流失,而且花费远超其发生频率所暗示的代价。我认为事实是,当有些事情导致流失时,你失去的是那个消费者后续所有的订单,而这些并不一定能被观察到。你只是看到了一次糟糕的体验,你看不到那些因为流失而失去的订单,因为它们已经消失了。所以我觉得有时候数据并不能展现全貌。如果你能够量化对参与度、对盈利能力的影响,它就会凸显出来,成为你如果不特意寻找就可能会忽略的东西。
数据看不到的盲区
然后我觉得另一件事是,像登录错误这种情况,有时候你在数据中看不到,因为人们根本无法进入系统。如果你无法登录,你就不会产生任何购买,不会下单,所以你可能在你查看的数据中根本看不到它。所以我觉得这也是数据从业者需要思考的问题:我们没有哪些数据?我们可能缺失了哪些数据?哪些地方可能存在我们实际上需要识别和修复但我们看不到的机会和问题?因为在登录失败这个例子中,用户无法登录,他们不在分母中,所以从数据集中我们完全遗漏了他们。
管理全球数据组织
Lenny Rachitsky: 再问几个问题。有一个我之前跳过的问题,现在想回过头来聊聊。话题转换比较突然,但我觉得可能会很有趣——关于全球数据组织。你运营着一个全球数据组织,你的数据科学家、分析师和业务运营人员遍布世界各地,不只是在美國。我很好奇,管理不同国家的数据人员和管理美国的数据人员有什么不同?最大的区别是什么?
Jessica Lachs: 大家总是问有什么不同。让我感到惊讶的反而是相似之处——人有多么相似,数据科学家本身相似,消费者、骑手,以及我们在 Volt 所称的 couriers(配送员)也很相似。相似之处远多于差异。不过我确实认为,当你在美国建立了业务,然后引入新的国家时,不同的货币和不同的语言会带来你之前不一定熟悉的复杂性。类似地,在欧洲,EU 国家与非 EU 国家之间有不同的监管要求,这又增加了一层有趣的复杂性。所以我认为它确实增加了问题集的复杂性,但最终很多问题是一样的。感觉有点像看了答案钥匙之后再去考试。所以对我来说,有时候在 Volt 通过 Volt 分析遇到问题时,我会觉得,“哦,我们遇到过类似的问题。我有一种直觉知道答案可能是什么。但我们还是要测试,因为可能存在文化或其他方面的差异,不过我感觉我知道最终会走到哪里。“然后有时候确实会出现因为某种原因而全新的问题,这就令人兴奋了,因为你可以说,“好,让我们看看这里是不是不同。“看看在 Volt 所在国有效的方案在 DoorDash 所在国是否无效,反之亦然。所以我倾向于更多地关注相同之处,然后当我发现不同之处时会感到惊喜,因为那能让你保持警觉,让事情保持趣味性。
AI 工具的应用
Lenny Rachitsky: 接下来我们进入 AI Corner(AI 问答环节)。这是我们播客中的一个环节,我试图了解人们在日常工作和业务中如何使用 AI。我很好奇你是否发现了一些非常有趣的 AI 使用方式——你可以从这两个方向来谈:你和你的团队在日常工作中如何使用 AI 工具来提高效率,或者如何将 AI 整合到产品中,让 DoorDash 变得更好。
Jessica Lachs: 是的,我认为两个方向都有机会。不过让我真正兴奋的其实是前者——帮助团队提高生产力。DoorDash 的分析团队有一项叫做 Office Hours(办公时间)的活动,这是我们八年前就开始做的,最初是为了给那些我们当时没有带宽去支持的团队提供帮助。早期我们会坐在一个房间里,说”欢迎过来,我们帮你解决任何需要帮助的问题。我们可以教你 SQL,帮你看看你的工作成果,做你的思考伙伴,你也可以来看看我们在做什么。“什么都可以。我们每周会在不同时段做两个小时的 Office Hours,方便不同时区的人参加。
让我感到兴奋的是,我们可以真正赋能那些仍然因为各种原因来参加 Office Hours 的同事,让他们能够用 AI 自己编辑查询语句。比如可以说:“这是一个查询,我想把它改成适用于我们的生鲜杂货业务,这样就能看到生鲜杂货的 GOV(总订单价值)。“所以我们在构建这些工具,不仅仅是帮我们自己的团队节省时间——说实话,我们自己团队的人也会用——而是真正赋能非技术用户,让他们能自己完成一些事情,不必占用分析团队的带宽。
Lenny Rachitsky: 所以本质上它是一个聊天机器人,公司里任何人都可以向它请教如何编写 SQL 查询、查询数据之类的问题?
Jessica Lachs: 是的。
Lenny Rachitsky: 这个聊天机器人有没有一个巧妙的名字?
Jessica Lachs: 其实并不巧妙。它叫 Ask Data AI,这个名字来源于我们内部原来的 Slack 频道,那个频道之前就是供大家提问数据相关问题的开放问答区。所以一点也不巧妙。
Lenny Rachitsky: 但很清楚。
Jessica Lachs: 不过这也符合 DoorDash 的命名传统——非常、非常具体直白,比如 Never Delivered(从未送达)和 Ask Data AI。
Lenny Rachitsky: 我很喜欢。清晰高于一切。这是我从一个与我合作的编辑那里学到的。Jess,你还有什么想分享的,或者想留给听众的吗?对于那些正在搭建数据团队、想让数据团队更高效的人,你有什么最后的智慧锦囊想要分享吗?
不拘一格的团队人才观
Jessica Lachs: 我唯一想重申的是,你并不一定需要在所构建的领域受过正式训练。我认为这也适用于你招聘到团队里的人。我之前提到过我们团队有很多人转去做产品或运营,但我没提到的是,实际上有很多人是从合作团队加入分析团队的——不管是来自工程团队、运营团队,还是市场或财务团队。我们是人才的净输入方,而不是净输出方。我觉得这和我自己的经历有关——我从运营转过来,从做 GM(城市总经理)转型到分析领域,所以我天然会被那些想做出类似转型的人所吸引。
当然,你需要具备技术能力。这些人中的大多数是在 DoorDash 岗位上逐步习得这些技能的——无论他们在转型到分析团队之前做的是什么工作——或者他们在学校时可能接受过一些正式训练。但我很喜欢看到那些主动转型、想加入分析团队的人,即使他们不是职业数据科学家。我认为这创造了一个非常独特的环境——团队里的人背景不同、专长不同,可以互相学习。我可以教你如何在 Excel 里搭建折现现金流模型,我可以从有咨询背景的人那里学到如何做出精彩的演示文稿,我可以从拥有统计学硕士或博士学位的同事那里了解统计学中常见的陷阱,我们还有计量经济学专家和经济学家。我们就是一群背景各异的人,可以互相学习,让彼此变得更好。我们不是彼此的复制品。
Lenny Rachitsky: 我听下来,你几乎是在优化团队中互补技能的多样性和截然不同的背景。
Jessica Lachs: 没错。还有就是在不同规模公司有经验的人。我喜欢来自初创公司的人,他们有那种拼劲和韧性,但我也喜欢见过规模化是什么样子的人,他们能帮我们预判业务增长过程中会遇到什么问题。所以不仅仅是技能的多样性和背景的多样性,还包括此前所在公司和阶段的多样性。这可以成为一种非常独特的团队构建思路,让你兼得两者的优势。
闪电问答环节
Lenny Rachitsky: 太棒了。就在你以为我们要结束的时候,我们迎来了非常精彩的闪电问答环节。准备好了吗?
Jessica Lachs: 准备好了,来吧。
Lenny Rachitsky: 来吧。好,第一个问题:你最常向别人推荐的两三本书是什么?
Jessica Lachs: 我倾向于读小说,尤其是历史小说,而且我很爱间谍小说。我觉得我的大脑即使在阅读时也总是处于解决问题模式。我最近读了一本很喜欢的书,是 Kate Quinn 的《The Rose Code》,讲的是二战时期的女性密码破译员,我非常喜欢。但与其再推荐一本书……虽然我刚才确实推荐了一本,但我想推荐的是 Libby 这个应用,以及支持你当地的公共图书馆,因为我爱图书馆,也爱 Libby,所以我就把这个作为我的另一个推荐。
Lenny Rachitsky: 太好了。很符合共享经济、公司的风格。Libby。好的,下一个问题。最喜欢的近期电影或电视剧?
Jessica Lachs: 又是一个我答不好的问题。我其实不太看电视,电影更不怎么看了。事实上,一些经典大片我都没看过。我朋友经常为此冲我喊:“你竟然没看过那部!“我倾向于反复重看以前的作品,一遍又一遍。我觉得这就是我让大脑关机的方式。我最近又重看了《白宫风云》(The West Wing),这是我有史以来最喜欢的剧之一,大概已经是第五十遍了吧。
Lenny Rachitsky: 天哪。
Jessica Lachs: 还有《Alias》,那是 Jennifer Garner 2000 年代初的一部剧集。也是间谍题材。所以我发现了一个规律——我真的很爱间谍类型。不过这两部都很棒,但完全不算是新剧。
Lenny Rachitsky: 完全可以。完全可以接受。你最近有没有发现一个特别喜欢的产品?
Jessica Lachs: 这个回答有点出乎意料。韩国防晒霜。我很容易晒伤,所以必须涂防晒霜,而我非常喜欢韩国防晒霜。是一个朋友介绍给我的,它们确实远远优于我们在美国能买到的产品。所以我强烈建议大家试试韩国防晒霜,特别推荐一个品牌叫 Beauty of Joseon 的防晒霜,非常棒,用起来也很舒适——这一点很重要,毕竟你每天都得涂。
Lenny Rachitsky: 随着年龄增长我也在尝试多涂防晒霜,所以这个建议很好。你刚才推荐的是什么品牌?
Jessica Lachs: 是的,品牌叫 Beauty of Joseon。
Lenny Rachitsky: Beauty of Joseon。
韩国防晒霜推荐
Jessica Lachs: 还有一个品牌叫 Isntree,防晒霜也很不错。不过说实话,我试过的韩国防晒霜几乎都很好用。
Lenny Rachitsky: 好的,我一结束就去搜索。你有没有最喜欢的人生座右铭,经常回到它、或者会分享给家人朋友的那种?
人生座右铭
Jessica Lachs: 有的。有一句 John Steinbeck 的话——我其实不太喜欢引用名言,但这句我很喜欢——他说:“一个常见的经验是,夜晚难以解决的问题,在睡眠委员会处理之后,到了早晨就迎刃而解了。“我发现这确实是我生活中一直践行的原则。首先,我热爱睡眠,尽量多睡。另一方面,如果我在某个问题上卡住了,或者正在写一份回应,比如一个紧张或情绪化的问题,通常我发现如果先把想法写下来,去睡一觉,第二天早上再检查,结果会好很多。突然之间,你对之前卡住的问题有了新的视角和清晰度,或者你意识到自己表达想法时不够清晰,因为当时情绪上头了,然后你就能组织出一个更好的回复——不管是邮件还是其他任何你在处理的问题。所以睡眠可以解决很多问题。
Lenny Rachitsky: 我也很爱睡觉。我总是跟我妻子说”我们去睡觉吧”,她说”好的,我马上来”。我很喜欢这个建议。好的,还有两个问题。谁对你的职业生涯影响最大?有没有一个人浮现在脑海里?
职业生涯中的影响者
Jessica Lachs: 我觉得有两个答案,一个多部分的答案。首先,我的职业生涯一直在男性主导的行业中,我跟一些非常优秀的女性共事过,她们对我影响很大。当我是银行家的时候,在我工作的雷曼兄弟,有两位资深银行家——Vanessa Roberts 和 Gina Tarone。她们非常出色,工作能力极强,我觉得那真的很鼓舞人心。
然后在 DoorDash,我们的 GC Tia Sherringham,以及负责传播的 Liz Jarvis-Shean,在各自领域都是顶尖的。我觉得这非常 empowering,对我影响很大——看到强大、有力的女性大放异彩,让我相信自己也能做到同样的事情。这是一个答案。
另一个答案有点老套,但就是我的父母。我妈妈在结婚之前是联合国的统计学家,后来她选择留在家里抚养三个孩子,我是最小的。我记得大概是在我上小学的时候,她决定重返学校、转行成为一名护士。她在做了 15 年全职妈妈之后,四十多岁时踏上了一条全新的职业道路,而我父亲也全力支持。我觉得这对我影响非常大,可能是第一次让我看到,无论你的年龄、无论你的处境,只要下定决心就可以做到任何事情。所以这对我影响很深,我想我从来没有跟她说过这些。所以,嗨,妈妈。
Lenny Rachitsky: 嗨,妈妈。谢谢你,妈妈。
Jessica Lachs: 是的,我觉得这对我的职业生涯影响很大。绝对是。
Lenny Rachitsky: 这个回答真美好。说个趣事,我跟 Liz 在 Airbnb 共事过。就是你刚才提到的负责传播的那位。
Jessica Lachs: 她非常棒。
Lenny Rachitsky: 她确实很厉害。最后一个问题。你加入 DoorDash 的时候,我想当时并不能确定它会成功。我猜当时还是那种”这是个疯狂的想法,也许行得通,也许不行”的状态。你有没有回忆起某个时刻,心想”我觉得这会是一个巨大的成功?我觉得这真的能成?“
意识到 DoorDash 会成功的时刻
Jessica Lachs: 说实话,我加入 DoorDash 是因为我想学习,为了这段经历。我觉得它很有趣——有趣的问题,有趣的人。我从来没有太多地去想它是否会成功。我当然希望它成功,我也非常好胜、想赢。我觉得有两个时刻比较突出。一个是当第三方市场份额数据显示我们已经成为第一名的时候——我想我们最初是第四或第五名。看到这个增长轨迹和品类份额的提升,真的非常激动人心。不过我可能是在它发生好几个月之后才看到的,因为我们不会花太多时间关注这些数据。但我确实记得有人想在某个演示、某个销售材料中放入那张图表,然后我们发现,“哦,我们是第一名了。太不可思议了。我们以前是第五名。“这是其中一个。
另一个让我印象深刻的时刻是,早年在波士顿的很多创业分享会上,我第一次做演讲时问观众:“你们中有多少人用过 DoorDash?“大概只有三个人举手。然后几年后,大概是 2018、2019 年,我在做一次演讲,问观众:“你们中有多少人用过 DoorDash?“几乎所有人都举手了。那一刻对我来说相当难忘,因为在我心中,我们仍然是那个没人听说过的小创业公司,我还需要故意把 DoorDash 里的 D 音发得很清晰,以免人们以为我在 Jordash 工作——就是那家 90 年代的牛仔品牌。所以当那么多人用过我们的产品、成为 DoorDash 的消费者时,这对我来说非常有意义。我现在还是会为此激动。最近我在读一本书的时候看到里面提到了 DoorDash,就像”我们上书了!“那些你成为文化用语一部分的时刻,我觉得真的、真的很特别。
结语
Lenny Rachitsky: 我是 DoorDash 非常满意的客户。我从来没有遇到过 Never Delivered。东西总在,有时候会晚一点,但通常都很完美。感谢你所做的一切。DoorDash 团队加油。
最后两个问题。如果有人想关注你做的事情,在网上哪里可以找到你?我知道你最近在 LinkedIn 上写了更多内容,请告诉大家去哪里找到你,以及听众怎样才能帮到你?
Jessica Lachs: 好的,正如你所说,可以在 LinkedIn 上找到我。我没有很大的社交媒体存在感,但我在 LinkedIn 上,目前正在写一系列博文,分享我在 DoorDash 建设全球分析组织的经验,以及过去 10 年学到的一些教训。欢迎大家去看看。
至于你的第二个问题,听众怎样才能帮到我,我想说请去 LinkedIn 上读那些文章,我很想听听大家的想法,无论你是否同意我的观点。不过请友善一些。我想要诚实的反馈,但也希望保持善意。所以,去和内容互动吧,让我知道你们的想法。我确实还有一个更广泛的请求,就是鼓励听众们追求真相(TruthSeek)——这是我在 DoorDash 非常认真对待的事情,也是公司的一个价值观。现在外面有大量错误信息,而辨别事实与虚构往往是我们每个人自己的责任。所以我恳请大家尽最大努力去寻找真相、说出真相,我认为我们都会因此变得更好。当然,还有使用 DoorDash。
Lenny Rachitsky: 当然。
Jessica Lachs: 是的,听众可以做这三件事。
Lenny Rachitsky: 你的网址是 DoorDash.com。这次对话太棒了。除了使用 DoorDash 之外,我也很喜欢你最后那一点。Jessica,非常感谢你来参加节目。
Jessica Lachs: 谢谢你的邀请。非常有趣。
Lenny Rachitsky: 我也是。大家再见。
结语
Lenny Rachitsky: 非常感谢你的收听。如果你觉得这期节目有价值,可以在 Apple Podcast、Spotify 或你最喜欢的播客应用上订阅本节目。也请考虑给我们评分或留下评价,这真的能帮助更多听众发现这个播客。你可以在 lennyspodcast.com 找到所有往期节目或了解更多关于本节目的信息。下期再见。
术语表
| 原文 | 中文 |
|---|---|
| affordability initiatives | 可负担性项目 |
| AI Corner | AI Corner(AI 问答环节) |
| Alias | 《Alias》 |
| Andy | Andy |
| Ask Data AI | Ask Data AI |
| Beauty of Joseon | Beauty of Joseon |
| case interview | 案例面试 |
| center of excellence | 卓越中心 |
| central model | 集中模式 |
| churn prediction model | 流失预测模型 |
| common currency | 共同货币 |
| composite metric | 复合指标 |
| couriers | couriers(配送员) |
| dashboard | 仪表盘 |
| dashers | 骑手(dashers) |
| data nerd | 数据极客 |
| data org | 数据组织 |
| discounted cash flow model | 折现现金流模型 |
| Elizabeth Stone | Elizabeth Stone |
| embedded | 嵌入式 |
| extreme ownership | 主人翁意识 |
| first principles | 第一性原理 |
| GC | GC(首席法务官) |
| Gina Tarone | Gina Tarone |
| GM | GM(城市总经理) |
| GOV (Gross Order Value) | GOV(总订单价值) |
| healthy host | 健康房东 |
| imposter syndrome | 冒名顶替综合征 |
| Isntree | Isntree |
| Jira tickets | Jira 工单 |
| Joe Graccio | Joe Graccio |
| Joey G | Joey G |
| John Steinbeck | 约翰·斯坦贝克 |
| Jordash | Jordash |
| Keith Yandell | Keith Yandell |
| Lehman Brothers | 雷曼兄弟 |
| Lenny Rachitsky | Lenny Rachitsky |
| Libby | Libby |
| Liz Jarvis-Shean | Liz Jarvis-Shean |
| merchant health | 商家健康度 |
| Never Delivered | Never Delivered(从未送达) |
| Office Hours | Office Hours(办公时间) |
| people manager | 人员管理者 |
| pod | 小组(pod) |
| Retention | 留存率 |
| soft skills | 软技能 |
| standup | 站会 |
| Stanley | Stanley |
| table stakes | 基本门槛 |
| The Rose Code | 《The Rose Code》 |
| The West Wing | 《白宫风云》(The West Wing) |
| Tia Sherringham | Tia Sherringham |
| Tony | Tony |
| TruthSeek | TruthSeek(追求真相) |
| Vanessa Roberts | Vanessa Roberts |
| Volt | Volt |
| VP of Analytics and Data Science | 分析与数据科学副总裁 |
| WeDash | WeDash |
此文档由 AI 分片翻译(translate_long_document)