揭秘训练每一个前沿 AI 模型的专家网络 | Garrett Lord
Inside the expert network training every frontier AI model | Garrett Lord
What Is Data Annotation
Garrett Lord: There will never be a time like this. I’ve never seen anything like it. I doubt I’ll ever feel anything like this in business again where there’s unlimited demand. How do you make sure that three months from now, six months, you have no regrets? Get on the plane to go talk to a customer, make the late night push, check the data six times over again.
Inside the Annotation Process
Lenny Rachitsky: Your company creates new data to continue advancing the intelligence of models. This is a business that you built on top of a business you’ve already had.
Garrett Lord: We’re the largest expert network in the world. We have this massive strategic advantage, which is like no customer acquisition costs. The only moat in human data is access to an audience.
Trajectory Data and Human Cognition
Lenny Rachitsky: You guys come in after the model’s trained to tweak the weights based on additional data that you you’ve created.
Multimodal Data and Scoring Criteria
Garrett Lord: The models have gotten so good that the generalists are no longer needed. What they really need is experts.
Market Validation and Model Builders’ Needs
Lenny Rachitsky: There’s this tension between all these students training models to become smarter, and then there’s that they will have harder time potentially finding jobs.
Garrett Lord: That’s not what we’re hearing from our employers, this is just enabling human beings to be even more productive. You used to put a Google Search on a skill on your resume because you grew up with Google. Being AI native, young people are at a huge advantage.
Types of Post-Training
Lenny Rachitsky: Today my guest is Garrett Lord. Garrett is the co-founder and CEO of Handshake, which is one of the most interesting and incredible AI success stories that you probably haven’t heard of. Handshake has been around for over 10 years, they’re essentially LinkedIn for college students, it’s a place for students to connect with companies to find a job. They are the platform of choice for every single Fortune 500 company. Over 1,500 colleges, over 20 million students and alumni, and over 1 million companies use them to hire graduates. At the start of this year, Garrett and his team realized that their huge proprietary network of students, including tens of thousands of PhDs and master’s students, is extremely valuable to AI labs to help them create and label high quality training data. So, they launched a new business from zero to one in January. Four months later, they hit 50 million ARR. They’re now on pace to blow past 100 million ARR within just 12 months. They’ll exceed the revenue that they’re making with their decade old business in under two years.
This is a truly incredible and rare story, and one that I think a lot of teams can learn from because AI is creating a lot of opportunity but also a lot of potential disruption, and this is an amazing story where the company basically disrupted themselves. This episode is packed with insights, including a primer on what the heck are people actually doing when they’re labeling and creating data to train models? A huge thank you to Garrett for making time for this, his wife just had a baby this week. He’s also in the middle of scaling this insane new business. So thank you, Garrett. If you enjoy this podcast, don’t forget to subscribe and follow it in your favorite podcasting app or YouTube.
Also, if you become an annual subscriber of my newsletter, you get a year free of a bunch of incredible products, including Lovable, Replit, Bolt, n8n, Linear, Superhuman, Descript, Wispr Flow, Gamma, Perplexity, Warp, Granola, Magic Patterns, Raycast, ChatPRD, and Mobbin. Check it out at lenny’snewsletter.com and click bundle. With that, I bring you Garrett Lord.
Garrett, thank you so much for being here. Welcome to the podcast.
The Human Role in AI Training
Garrett Lord: Yeah, thanks for having me. A long-time subscriber.
The Tension Between AI and Employment
Lenny Rachitsky: I appreciate that. Okay, so before we get into the insane trajectory that your data labeling business is on, which is just an amazing story that I think a lot of founders and product teams that are trying to navigate this AI disruption that’s happening will have a lot to learn from. I want to first help people understand what the hell data labeling actually is. Just like, what are people actually doing? Why is this so valuable? Some of the most, I don’t know, fastest-growing companies in the world today, including you guys are just, this is what you do. Clearly there’s something really important here. I sort of understand it, probably not really. I think a lot of listeners feel the same way. So let me just ask you this, what is data labeling actually? What are people actually doing? And then, just why is this so valuable to frontier AI labs?
Garrett Lord: Yeah. So, I think it’s helpful to take a step back of what does training a model look like? So, there’s really two primary functions. There’s a pre-training and a post-training process in training a model, and for a long time these AI providers, or LLMs, or Frontier Labs we’re focused on basically sucking up more and more information on the pre-training side of the house. And that’s basically the entire corpus of written human knowledge. So, that’s not just written, but every YouTube video, every book, basically the pursuit of sucking up everything that was on the internet, and that was the pre-training side. And there was a lot of gains from pre-training, like models continue to get better. And about 18 months ago, 24 months ago, we started to really see an asymptoting of gains coming from, because they had essentially sucked up all of the knowledge on the internet. And so, labs really shifted towards most of the gains now coming from the post-training side of the house.
And what post-training is, is it’s augmenting and improving the data they have across every discipline or capability area that they care about. So take coding, or mathematics, or law or finance, they are focused on collecting high quality data that really improves the state of our capabilities, their models, and you can see a lot of these popular benchmarks on what are called model parts. When Llama IV is released, you’ll see the benchmarks across various domains, and each one of the research teams inside of the labs have different use cases. Basically they’re running experiments, almost think like the scientific process. They have a hypothesis around how to improve the model. They’re trying to collect small pieces of data to see if that hypothesis works out. If that hypothesis is proving true, then they expand the overall collection of the data in that advert. And it could look like reinforcement learning environments, it could look like trajectories, it could be audio and multimodal, it can be text-based like prompt-response pairs.
It can also be reinforcement learning with human feedback, which is like preference ranking data. And so, that’s the state of the art of models. And most of the gains that are happening from models right now are coming from the post-training side of the house. And there’s just an incredible amount of demand to stay at the absolute frontier of where models are going.
Advantages of the AI-Native Generation
Lenny Rachitsky: So training, pre-training is feeding it, say the entire internet. Here’s like all the data that the humans have ever created, figure out knowledge and facts, and how to reason and all these things. Post-training, is it correct to say there’s essentially two buckets of things to do? There’s reinforcement learning, human feedback RLHF, and then there’s kind of this bucket of fine-tuning?
Garrett Lord: I mean, yes and no because take for example trajectories, or you want to be able to do, people use flight search or an accounting end-to-end process, or you want to be able to conduct biological experiments, you need actual trajectory data. There’s still very much, a lot of the labs are still, they have points of view on what data collect. It’s evolving very quickly. But I think reinforcement learning is really preference ranking, like which question do you like more, question A or question B? SFT data is a prompt and a response, and obviously the labs are very focused on these thinking or reasoning models. So, in order to improve a reasoning model you need to actually have the step-by-step instructions, of which when you interact with a lot of these frontier models they struggle in very advanced domains. And so, I think there’s a variety of data that they’re working with to improve capabilities in their models.
Cutting-Edge Insights on Data Annotation
Lenny Rachitsky: What I’m hearing is there’s other ways to post-train. Which of these are you guys focused on? Where do you help models most of these three-ish buckets?
Garrett Lord: Our real unique proposition as a business is the fact that we have an engaged audience. We have 18 million professionals across, we have 500,000 PhDs, we have 3 million master students, we’re a global platform. And so, depending on what you’re looking for across any area, academic knowledge, what is the definition of a PhD? How do you get your PhD? You defend your thesis. Defending your thesis means, generally speaking, you have proven that you have extended the world’s knowledge in a particular domain. And so, the ability to hyper-target this audience into chemistry, math, physics, biology, coding and really touch parts of human knowledge that have never before made it to the internet is really where we excel. And I would say that when you talk about the labeling market, something to make it more abstract is like it used to be generalists’ work.
A lot of the market before the model started to get better was leveraging talented international lower cost labor to do basic generalist tasks. But really what’s happened is the models have gotten so good that the generalists are no longer needed. What they really need is experts, experts across every area that the models are focused on. And really, you could think about these model builders as they’re focused on the most economically valuable capability areas in the economy. And so that, generally speaking, right now is focused on advanced STEM domains, advanced science and math domains, and then the derivative functions of accounting, law, medicine, finance, where they want to make the models more capable. And then the work that we’re doing, I think to come full circle to your question, we’re doing work across so many domains. I mean, we have millions of bachelor students that are being used for work in audio, work in customizing a model depending on the voice and tone, where you are geographically in the country, what do women versus men prefer? All the way to the most advanced PhD STEM domains out there.
From Career Platform to AI Data Business
Lenny Rachitsky: Okay. So, is it fair to say essentially all the data that is available has been trained on, and your company creates new data, new knowledge to continue advancing the intelligence of models?
Garrett Lord: Yep. And I also say we help point out where the models are weak. So, in order to break a model, it’s pretty tough for the average person to break a model and get an incorrect response. But if you are a PhD in physics, you can go in multiple subdomains of physics and prove where the model’s actually breaking, either breaking in its reasoning steps or it’s where it’s broken in its ground truth right answer, or we start throwing tools in there or needing to follow some step-by-step process. And I wouldn’t say it’s easy for them, but the average person cannot break the models and that’s where we really come in.
Timing and Unfair Advantages
Lenny Rachitsky: So, essentially it’s just catching mistakes that the model has made. Okay. So, what are these people actually doing? I know there’s all kinds of different types, you described all the ways that data’s generated, what kind of data is useful? So, maybe just the most common examples. Let’s say a PhD person is sitting there doing stuff, what are they actually doing?
Garrett Lord: A great example is a public paper called GPQA. So, for the engineers out there that want to read about it, essentially the crux of the paper is you break the model, you provide a ground truth, the right answer to the question, you provide the step-by-step a reasoning steps. So, you might imagine because models are non-deterministic, the model can get the answer right once, but it might not get the answer right three out of five times. So, you actually prove where the model’s failing. You actually break down into where is it failing? Maybe it can get, it knows the question, it can get the right answer, but the actual steps to get there are wrong and they really focus on the steps to get there. Say there’s 10 steps in a math problem, step 6 through 10 is wrong. So, how do you fix the actual steps?
And what are they doing? So they’re going in, we’re really focused on calling this branding the experience and treating people like experts. PhD students expect to be treated different than a lower cost international labor with a different work expectation. And so, these PhDs come into a community, we have a instructional design team and an assessments team that’s going through and basically iteratively helping them understand how to use the tools that we built, and how to interact with the latest models. Then they go in and start actually creating data. And that process is, on our side the model builders, they want to know that the data we’re producing is high quality. Som we have our own research team, our own post-training team.
I hired a gentleman from Meta that went along on the post-training over there, and I-
How to Source Data Annotations
Lenny Rachitsky: Hope you paid him well.
User Acquisition and Lifetime Value
Garrett Lord: Yeah. So, war for AI talent is very expensive, but super, super privileged and proud to be working with him. And so, each unit of data, we have to build it an environment for them to actually create the data. Then we have to understand in a unit level we’re trying to approximate the actual gain from that piece of data and whether it can improve in a particular capability area. And then, we’re also focused on evolving the use cases to also follow what the model builders want, which is they want more more real world tool use and trajectory based data as well.
Lenny Rachitsky: Okay. There’s so much here, and we could go infinitely down here but I think that this is really interesting because just like people hear so much about all of this and they barely understand what the hell it actually is. So, this is for me really interesting. I think it’s going to help a lot of people. So essentially a PhD, say a biologist, biology PhD is just their job is find flaws in what, say ChatGPT is producing, and then come up with here’s the correct answer. And that is used to fine tune the mode, here’s something you are doing incorrectly, here’s the correct answer and that improves the model.
From Observation to Action in Data
Garrett Lord: Yep.
Lenny Rachitsky: Is that a simple way to think about it? And please correct anything I’m saying that is incorrect because I don’t want people to misunderstand it.
Building New Ventures Inside Existing Businesses
Garrett Lord: Yeah. I mean, a great example, let’s take a non-verifiable domain like education. So there’s a PhD student, Rachel on the network, she got her PhD from the University of Miami, spent two decades as a teacher teaching students in the eighth grade. And she was an adjunct professor at a local community college in the field of education. And so, she is interacting with the state-of-the-art models in educational design. So, actually trying to understand what is the best way to teach people, and how do you spot incorrect issues in a model in the way that they’re training people, and help the models understand the forefront of educational design with the hands-on experience of being an eighth grade teacher for 10 plus years and having a PhD in education? So, that’s an example of you can have that all the way down to a verifiable engineering problem that you’re seeing the latest models fail on.
Yeah, I think that gives you the gamut. You also have, we talk about professional domains like these reinforcement learning environments, there’s a bunch of papers out there that basically speak to people narrating over their step-by-step tool use. So, as they go to solve a problem from start to finish, interact with multiple different service areas, interact with multiple different tools, they’re like, there’s papers that talk about this, talking over what they’re doing, actually following and screen recording where their mouse is going, how they’re problem solving. When they run into a roadblock, what do they do? So, they really want to understand how humans think.
Lenny Rachitsky: You mentioned this term trajectory. Can you just explain what that actually means? Because it feels like you’ve mentioned that a few times and that feels important to all this.
The Rhythm of Independent Operations
Garrett Lord: Yeah. A trajectory is basically just like the entire environment that is collecting what you’re doing. So it’s your screen, it’s your mouse-
Trust and Market Vision
Lenny Rachitsky: Oh, I see. Oh, wow.
Data Exhaustion and Model Bottlenecks
Garrett Lord: Yeah.
Advice for Young Entrepreneurs
Lenny Rachitsky: Including this voiceover, okay. And then, this might be too technical, but what is the output of all this work? This, say teacher, is it just like a JSON file, an XML file, like a text file?
Garrett Lord: Yeah, it can be managed JSON data.
Rapid Fire Q&A
Lenny Rachitsky: JSON data? Okay.
SNOO and Parenting (Continued)
Garrett Lord: And then, we also have multimodal work like audio, like classifying music and understanding … We’re engaging thousands, or not thousands, probably hundreds of top music students at the music schools in the country who are improving models of understanding of music. And you also have the thing called, which we haven’t talked about here, a rubrics, and rubric models are, you can put a model in as a judge. What is a good educational design, or what’s a good MRI result? In some of these domains, you actually don’t have a guaranteed correct right answer. And so, models can sit in the middle as a judge and actually understand what is … Think back on your school days. How do you get A on your 5,000 word paper? Well, there’s a great introductory statement and there’s scientific proof. So, you can build a rubric that allows a model to sit in the middle and actually auto-evaluate responses. We’re seeing a lot of rubrics work as well.
Guiding Life Principles
Lenny Rachitsky: And you would think, why would you trust this one teacher’s opinion that this is the right way to do it? But what’s cool is the market speaks for itself. If these models are being used more and more, and people love them and value them, I imagine steps in between to verify this is good and other people think this is a good idea. It feels like the market dynamics will tell you if the data you’re providing is correct at what people want. Is there something more there?
Garrett Lord: I didn’t get a PhD in AI, or math, or physics, and I haven’t trained myself, we have frontier models, but there is a lot to each unit of data whether it’s improving. There’s a ton of science and research out right now around how do you make sure that the data that you’re producing is improving the model? And it’s very hard for model builders to understand. They really care about, to zoom out, they care about three things. They care about quality first and foremost. You have to have high quality data. And if you imagine you’re training a model, like teaching a student and you’re giving it the wrong data, it’s extremely challenging to overcome that. So, quality is first and foremost. And then, the other huge problems you have is volume. How do you generate thousands of pieces of data in the most advanced domains of chemistry, and mathematics, and physics, and how do you ensure that it’s high quality?
Well for us, say in physics, we just reach out to students at Stanford, and Berkeley, and MIT, and they’re at the top GPA at the best physics schools in the country. And so, our ability to get to scale or volumes of data, to produce very high quality data, is something they care deeply about. And then, the other thing I would say model builders care about is speed, because they have all these hypotheses and they’re constantly testing different pipelines. And so, you might have three or four bets going at once, and then as soon as one is actually showing a gain, imagine you’re a researcher or you’re assigned to the processes, once you’re running a gain then you’re trying to grow that pipeline and grow that piece of data that’s actually improving it, and you’re maybe ditching two or three other projects you had that weren’t showing improvement.
So, your ability to quickly turn around for them in a period of days, and then get to high volumes of data that are high quality is the number one thing they care about. And so, there’s quite a bit of technology we built on our side to assess each unit of data. We have our own post-training teams, we’re renting our own GPUs, and we’re trying to make sure that we can sit directly with these researchers and help share what we’re seeing with the data that we’re creating and how it could improve their model, how they could best train with it. So, hopefully that helps.
The Princeton Pool Story
Lenny Rachitsky: Going back to the types of post-training, just because I think this might be helpful, at least for me the mental model of there’s pre-training, there’s post-training, within post-training there’s reinforcement learning, human feedback, there’s this concept of fine-tuning. There’s also evals and stuff like-
Contact Info and Hiring
Garrett Lord: There’s SFT, yeah.
Closing Thoughts and Outro
Lenny Rachitsky: SFT, which is supervised fine-tuning? Is that-
Garrett Lord: Yeah.
Lenny Rachitsky: Yeah. So, the stuff you’ve been describing, would you mostly describe that as supervised fine-tuning?
Garrett Lord: Yes, and we’re doing all of the above. We don’t do the auto eval, we produce rubrics which are used auto evals. But yeah.
Lenny Rachitsky: Okay, awesome. So essentially there’s a model, it’s trained on all this amazing data. You guys come in after the model’s trained to tweak the weights based on additional data that you create. What’s interesting is that this is a scalable system. I want to talk about just the supply of amazing people that you have producing this, but it’s amazing that humans can do this. You would think it needs to be this infinitely scalable thing, but humans sitting there creating data is working and improving the intelligence of models significantly.
Garrett Lord: Oh, yeah. I mean, I think maybe a funny joke is all the MBAs think this is all just going to go away. And I think for as long as models are improving, humans will be needed in this process. And when you talk to the lead scientists and researchers at these labs, it’s like the data types will evolve and what they’re trying to capture and collect, but there will be humans needed in the space for the next decade until we reach full ASI. So yeah, I mean, you think about in a lot of them will struggle to do basic trajectories right now. So, right now people are very focused on academic domains, and I think they’ll continue to be focused on academic domains, but there will also be far, far more demand for professional domains as well across basically every trajectory or step-by-step problem that a knowledge worker solves in the workplace, it’s the pursuit of these labs to make sure that they’re trying to collect the data to help add as much value in that process for humans as possible.
Lenny Rachitsky: So, let me ask you about this. There’s this tension, I imagine, people might feel between all these students training models to become smarter, and smarter, and smarter, and then there’s that they will have harder time potentially finding jobs if models are so smart that people at entry level aren’t being hired as much. How do you think about just that tension? Do you think this is a real problem or not, or do you think this goes-
Garrett Lord: I’m probably in the camp of like GDP growth over universal basic income. I very much believe that this is going to improve and accelerate every human’s ability to create an impact in the economy in the world, and that we’re hearing from, there’s like a million companies that use Handshake. 100% of the Fortune 500 uses Handshake, so we basically power the vast majority of how young people find jobs, and a lot of people are hyperbolic at saying that all young people won’t have jobs, and that’s not what we’re hearing from our employers. What we’re hearing is pick social media marketing, before you needed somebody that could do Photoshop, and take pictures, and create the videos. Then you needed somebody that understood marketing analytics platforms to track your posting on different social media forms. It’s like now one person, one young, talented, AI native, Iron Man suit enabled young person can get on and they can build their own videos, produce their own creative assets, post across multiple social media platforms, run all of their own analytics. They don’t need a data science degree to be able to do that.
Or take an intern in our company, he had his first PR up I think the afternoon he started. You were a PM, you realize how challenging that would’ve been historically to get your dev environment set up and figure out where to add value. You just took a bug and squashed it. And so, I’m really a believer this is just enabling human beings to be even more productive and create more impact. And yeah, of course, hundreds of millions of jobs, the jobs will evolve. People will become displaced, they’ll have to upscale and rescale, and I think Handshake has a huge role to play in helping knowledge workers evolve.
Lenny Rachitsky: This has come up a couple of times this point that I think is really good, that younger people coming out of school are actually going to be much more likely to be successful because they’re growing up with these tools, and are much more native to all these advanced tools and so they just come in as beasts just doing so much more.
Garrett Lord: Well, do you remember when, this a little bit predates me, but you used to put Google search on as a skill on your resume. You were person, you were good at Googling, because you grew up with Google. It’s like I think being AI native and having your Iron Man suit on, and understanding how to leverage these tools is like young people are at a huge advantage.
Lenny Rachitsky: Yeah. And especially if they’re involved in training these models, I imagine there’s some other cool advantage there.
Garrett Lord: Yeah. Well I mean, just to hit on that, what we’re hearing from our thousands of fellows is they’re in the classroom, they’re actually producing research. We’re talking about PhDs at the top institutions in the country. They can make 100, 150, 150 an hour breaking the latest models, and what we’re hearing from our fellows is they’re bringing a lot of those insights into the classroom to help them be more effective at teaching. More importantly, they’re starting to learn how to leverage these tools to actually advance their area of research. So, they believe that these tools can help them advance their area of research by helping them be more effective with their time. And so, it is quite cool to get paid to learn a skill.
Lenny Rachitsky: Before I get to the story of how this all emerged, because that is an incredible story, is there anything else about this whole field of labeling, of reinforcement learning that you think people just don’t fully understand or you think that is really important? There’s just so much happening. Like I said, some of the fastest growing companies in the world are in the space, Scale was just acquired for 30, sort of acquired for $30 billion. Just what else is there, if there’s anything, that you think people need to understand?
Garrett Lord: Generally speaking, anytime that you’re interacting with a model and you’re asking it to do really advanced things, and it’s not performing your expectations, like somewhere there’s probably an expert that is the top mind in that domain working directly for the best researchers in the world at the Frontier Labs trying to understand and go through the scientific iteration process of how to make that better. And that the assumption there is that they already have the entirety of human knowledge that’s written and recorded. And so, for as long as there are problems in solving any problem with AI, any human problem, there will need to be humans in the loop helping advance that. And models don’t generalize. I mean, obviously the field will advance a lot and the type of data they’ll collect will evolve a lot, but it’s pretty exciting at the frontier.
Lenny Rachitsky: Kevin Wheel was on the podcast, the CBO at OpenAI, and he made this point that really stuck with me that the model of today is the worst model you will ever use.
Garrett Lord: I love that line.
Lenny Rachitsky: Will only get better, just boggles the mind, and now we know why things are getting better because all the work you guys are doing. Just one quick question on this whole scale thing, I guess they were, I don’t know, the main company doing this, now they’re swallowed up and Alex is running superintelligence in Meta. Are they still a big player in this labeling space or are they out of it and that’s a big opportunity?
Garrett Lord: Yeah. I mean, kudos to the whole Scale team, a lot of respect for what they built, just many great companies operating the space. I think to the core of your question, I think if you viewed your research team and your model building team, and the experiments they’re running to be really the cornerstone of how you’re improving, you probably wouldn’t want the latest research of what you’re trying to work on being invested in by a peer. I mean, that’s just generally what we hear in this space. And so, we have seen an incredible search and demand, and are I think extraordinarily well positioned. We like to say the only moat in human data is access to an audience. Basically, there are many, many small players in this space, some midsize players in the space, and they’re basically running TikTok ads, running Instagram ads, paying money for Google Search display ads, YouTube ads, and they will be like, “Can you get me 200 physics PhDs?”
What do they do? They only can do one thing. They have 100 recruiters on staff, they all get on LinkedIn, they all send messages, they spend a couple million bucks on performance advertising campaigns. Somebody’s scrolling their Instagram feed that’s a physics PhD of which you can’t target them that well and they like see, “Come train a model.” It’s like, “I’ve never heard of this brand before.” The huge advantage that we’ve had and why we’ve resonated so fast in the marketplace is we built a decade of trust with 18 million people, and they trust us, and we built a ton of brand affinity, and they use Handshake, and they have an active profile, and we have a ton of information around their academic performance and what they’ve done in school. And so, we’re able to really target people really effectively, and get to scale and volume of high quality data faster than anyone else. And I think that competitive advantage of access to an audience is really resonating in the marketplace.
Lenny Rachitsky:
If you want to try it out, get started at Claude.ai/Lenny, and using this link you get an incredible 50% off your first three months of the pro plan. That’s Claude.ai/Lenny.
Okay, this is an awesome segue to where I wanted to go, which is just how this business emerged. This is a business that you built on top of a business you’ve already had. From what I understand, you were at like $150 million in revenue, you’ve been at this for a long time. You found this opportunity, and now that looking back it’s like obviously this is an amazing idea, labs need data, you guys have the supply of incredible experts. What an opportunity. Talk about just how you first realized this was something that you could be doing, and should be doing, and then how you started to execute down this path.
Garrett Lord: Yeah, I think it’s been a pretty natural extension from helping people jumpstart, restart or start their career. Monetizing your skills and this new employment ecosystem is going to look very different in the future, and to zoom into how we discovered it, it’s like because we have such a large access to this audience, and as the world shifted from generalists to experts, we’re the largest expert network in the world. We have more PhDs, 500,000 of them use Handshake than any other platform. We have three million master students who are in school or alumni. And so, we started to see all what I would call middleman companies reaching out to us saying, “Can we recruit your PhDs and master’s students?” And like any great marketplace we started sending them to these different platforms, and started to really realize that from hearing from our users that the experience was really frustrating.
Training was very transactional, it was very amorphous how you could get paid. There was immense amount of drop-off in the process to actual project like completion on these other platforms. So, we started to think the company was making tens of millions of dollars from helping these other platforms, and we started to realize what really kicked it off was hearing also from the Frontier Labs, they started to reach out to us and started to go direct and trying to almost cut out the middleman. And we started to realize, well, we could really serve our fellows, our PhDs, our experts, we could treat them. We just believe there will need to be a platform, an experts first platform in the pursuit of ASI and advancing AI, and there will need to be a place that everyone in the world could go to, to monetize their skills and their knowledge as these labs are focused on improving in all these multidisciplinary. And yeah, we entered the business in, really I started doing it over Christmas and New Year’s. That’s when I started flying around.
My family thought it was a little wild that I was on planes trying to chase different leaders, but we built an incredible team of people that came from the human data world, and really started building out our platform in January, and then started really monetizing the relationships about five months ago. Fast-forward to today, we’re working with seven of the Frontier Labs, basically every lab that’s doing work and building the best large language models, and the team has exploded and revenue’s exploded, and it’s been really a incredible ride running back new company inside of a company for the second time over again.
Lenny Rachitsky: And just to share some numbers, tell me if this is correct or if you’re sharing these, but I heard that you hit 50 million in revenue just four months into this? Today we’re at eight months in and you’re on track to hit $100 million in revenue in the first year.
Garrett Lord: I think we’ll blow through that number, but yeah.
Lenny Rachitsky: Okay. Incredible. And I didn’t even know there were seven Frontier Labs, that’s-
Garrett Lord: Zero to 50 is pretty good in four months, I think.
Lenny Rachitsky: Think zero to 50 million in four months, that’s something. It’s like the bar has been shifting constantly. A year ago that’d be legendary. Now it’s like, all right, well another one of these. 50 million in four months, no big deal. It’s truly insane. Just to zoom out one second, for people that don’t know a ton about Handshake, the original business, what was that? What was actually this network that you had, that you sat on top of?
Garrett Lord: Yeah, that network does about 200 million. This will do about [inaudible 00:37:21]
Lenny Rachitsky: 200 million.
Garrett Lord: Yeah.
Lenny Rachitsky: Okay.
Garrett Lord: So, we have 600-ish super passionate teammates that work on the core business, which I would separate those. These aren’t two businesses, I think it’s one business, but what is that business? If you’re a young person in America that’s graduated in the last five, six, seven, eight years, you probably have Handshake on your phone. You definitely know what Handshake is. It’s a verb with young people in America, it’s a verb with people that are in college in their PhD or master’s program, and it is, I call it an unconnected graph, meaning you don’t need to … LinkedIn is very focused on who you know and what your experience is. The first question on LinkedIn is what’s your job? And a lot of young people start off, they’ve never had a job before. They don’t have 500 connections to add to their graph.
Whereas on Handshake, you start off trying to discover, and explore, and figure out how to navigate through school and figure out, “Oh, I’m an engineer. Maybe I want to be a PM, maybe I want to work at a startup, maybe I want to go to a larger company.” What are the pros and cons you want to learn from your peers and young alumni? And so, Handshake’s this I call a very social platform with groups, and messaging, and profiles, and short-form video and feed, all focused on your interests and helping really build your confidence in your early career to find your first job, your second job, and to manage 18 to 30, I would say.
Lenny Rachitsky: And how long has that business been around?
Garrett Lord: It’s been around 10 years.
Lenny Rachitsky: 10 years, okay. So it’s just again, it just feels like such a holy shit, you guys are in the right place in the right time with the right network that is extremely valuable now. What an interesting story. I feel like it’s just another interesting example of you’ve been doing something for a long time and then all of a sudden AI is just, opens up a whole new way of leveraging something that you have been doing for a long time. It makes me think a little better about Bolt, and StackBlitz, which was building for seven years this browser based OS where you could run an OS in the browser. And they’re like, “I don’t know, no one needs this. What are we doing?” And then, all of a sudden AI and they’re like, “Oh, what if we build AI apps in the browser and just generate products for you with AI?” And now it’s, I don’t know, one of the fastest growing companies in the world.
Garrett Lord: Yeah.
Lenny Rachitsky: So interesting. And so, I think this is just an interesting time for our people to think about what have we done that may give us a new opportunity to build something huge based on this unfair advantage that we have?
Garrett Lord: I think also as your company grows in size and headcount, and maturity, it’s also hard to incubate something new inside of a business. It’s hard in so many ways. The way that you build zero to one, and find product market fit, and scale a team very quickly and is very different than the way that you run a more mature business that has been around for 10 years with hundreds, and hundreds, and hundreds of people. So, I’ve really had a ton of fun and found a ton of passion in running it back again for the second time inside the business. And then yeah, we have this massive strategic advantage, which is no cost or acquisition costs, and we have much higher conversion rates and retention than any of the other platforms by a large margin because we have such consumer affinity.
Lenny Rachitsky: There’s actually two threads here I want to follow, I’m going to follow the second one first, this idea of where this data labeling work can come from. This isn’t a really clear, simple, understandable one, which is just experts sitting there creating data. Another one that I know a lot of other companies in this space use Scale, I know especially with just like low-cost labor internationally. Are there other methods for doing this that isn’t one of those two? How are other companies doing this?
Garrett Lord: I think if you care about building a really high quality business, and having good gross margin and high quality growth, the ecosystem here is, one of the leading players, they have 200 recruiters. It’s unsustainable. There are like 200 people on LinkedIn sending individual messages to acquire these people, because there’s no brand, there’s no trust. They’re spending tens of millions of dollars a month on performance advertising, Google Ads-
Lenny Rachitsky: To find experts and to find folks.
Garrett Lord: Find experts.
Lenny Rachitsky: And it’s experts mostly at this point.
Garrett Lord: Yeah. And then they put them onto an experience that is treating them like they’re drawing boundary boxes around stop signs in the Philippines. The frontier tax accountants don’t want to be treated like low cost international labor, and I don’t think anyone enjoys that process. And so, the ability to build a experience that’s rooted in community, that’s rooted in high quality training. If you’re getting your PhD at MIT, chances are you’re just not being taught well enough on how to use the tools.
It’s not you can’t break the models, it’s just like the other platforms, they’re spending thousands of hours to acquire an individual user and they’re put right into a project with no training. So, we just started from day one at building this expert … We believe there’d be a deep network effect here that’s very connected to our core business of starting, jumpstarting or restarting your career. And you come in, you build a profile, you see the community, there’s groups and a feed of here’s how people are learning. You come into actual individual cohort with peers that look like you and have your similar background. You’re being taught on how to interact, and there’s a trial and error, and we have an instructional design piece so you can’t do it. Then you’re put on the projects where building … There’s certain swim lanes where we’re actually pre-building data and selling that data to all the labs.
So, we can do this thing where we produce one unit of data ourselves. We pay for it, almost like a movie production. We pay for a unit of data, and then we make sure it’s very high quality. We run our own post-training on it, and then we produce a bunch of specifications of the data, and we actually sell that individual package of data to many different labs. And so, you get put on a project like that. Once you’re doing a really, really good job on our projects, oftentimes then we’ll put you on customer projects where they only want the best of the best people in machine learning. And then they go from our projects to their projects. And so, there’s a huge customer acquisition. You love going deep on your podcast, so just to talk about it, it’s like you really have a couple of things that matter.
You have a cost to customer acquisition, your CAC, and then you have your LTV, like the lifetime value of a user. And an LTV is calculated pretty simply in this business. It is based on the retention of a person, and how many projects they can participate in. So, if you treat people really well, you train them really well, well, A, we have no customer acquisition costs because we partner with 1,600 universities, power 92% of the top 500 schools in the country. We power almost every institution and community college in the country. We have no customer acquisition cost to acquire the people. We have a ton of brand and trust with them built up, so they convert at really, really high rates. And then, if you treat them really well, because what they expect from us, they know Handshake, their school buys Handshake, we care about treating these people well but the universities would not tolerate our partnership with these fellows unless we treated them well.
So, you put them into this process where our LTVs and repeat engagement rate and retention rate on different projects is really high. And so, these structural advantages are quite significant when you contrast a leading provider that has 200 individual contributing recruiters, and are spending tens of millions of dollars a month on performance marketing. So, that’s I think why we’ve seen so much success.
Lenny Rachitsky: That’s extremely interesting. And it feels like, as you said, there used to be a big focus on generalists, which is people anywhere in the world for low-cost can do the work, like draw bounding boxes around things. And essentially the market has shifted from low-cost generalists to experts. And a lot of these companies like Scale, we’re optimizing for general work model training data, and you guys are set up to be extremely good at expert based data. And so, you’re in the right place at the right time, at the right supply. What a business.
Garrett Lord: Yeah.
Lenny Rachitsky: Nice work.
Garrett Lord: I would say it’s not been easy building business two inside of business one, but-
Lenny Rachitsky: Yeah. So, let me follow that thread. That’s where I wanted to go. What was just that like? So, you started noticing that model companies were coming to your people, that people were having hard times with some of these other companies in this space and you’re like, “Oh, maybe we should be doing this sort of thing”? How did that just initial inception start, and how did you start to explore that idea and to see if it was a real thing?
Garrett Lord: Tactically we were working with many of the middleman companies doing work. We started to see the demand, as I talked about earlier. We started to see direct outreach from the Frontier Labs reaching out to us, trying to cut out the middleman in their pursuit of getting higher-quality data. When we started to put together the dots on we could build a way better experience for our fellows, we could serve them directly to the labs and build a direct customer relationship with the labs, and basically cut out the middleman. And provide a better experience to the labs, provide a better experience to our fellows and provide a better experience to our million companies in the network.
And you might think about just upskilling and reskilling, what’s going to happen there. So, we walked into this space. We started in really December, exploring and learning more about it, like on expert calls and hammering down. I hired three expert firms, AlphaSights and GLG, and started doing a bunch of calls with the latest researchers, because we had resources. One of the cool things about being a larger company is our core business is $200 million ARR, so it’s like we had resources to be able to accelerate the learning curve here. And then, we started working with arguably the number one lab about five months ago.
Lenny Rachitsky: I wonder who that is.
Garrett Lord: Yeah.
Lenny Rachitsky: Yeah, wonder who it is.
Garrett Lord: [inaudible 00:47:39] different answers working with the number one lab, and have just now we’re working with Devin on the Frontier Labs and the number one thing we’re trying to do is just focus on scaling up. And we’ve gone from four or five people working on this to 75 plus people working on it. I think we had 12 people start last Monday. It’s like we are so bottlenecked on just meeting this opportunity, because in this market there’s essentially unlimited demand. If you can produce high quality volumes of data, you most likely will be able to sell whatever you produce. And so on our side, it’s like we’re really focused on making sure that we pick the right longer term strategy, making sure that we don’t grow too fast as to erode the trust that we’ve built up with these Frontier labs. Yeah, but it’s been fun.
Lenny Rachitsky: You said it’s also been really hard to start those business within an existing business. What’s been hard? What’s been hardest? You touched on a couple of these elements already, but what else?
Garrett Lord: I think I just followed a lot more of my intuition around this, doing this. The story of Handshake was we had to sign up 1,600 universities, so I had to learn how to be the best … We are the fastest growing higher education company in history. So, we signed up six 1,600 schools. Then we had to build an employer business, where we had to figure out how to sell the 100% … All these Fortune 500 companies use it and 70% of it pay for it, so I had to learn about upmarket sales to Goldman Sachs, and General Motors, and Google and the biggest companies in the world, which is totally different than selling universities. And then we had to learn how to build an incredible student social network. What does the best feed look like? What does group messaging look like? So, I felt a little bit of familiarity in those zero to ones.
Oftentimes marketplaces are like many zero to ones. Sometimes I dream that we just, I actually don’t dream, but I make a joke that I just wish we were a cybersecurity company and we had one buyer and just one product, and it was just like we had to … In a marketplace, you have to serve three different sides, you know from your time at Airbnb. And so, one of my learnings in spinning up these three different businesses in starting Handshake was I was pretty hands-on. So, everyone reported directly to me. I really said in a lot of meetings, “I’m not trying to be the boss, I’m just trying to get another smart guy in the room.” We’ve hired an incredible team of people that have spent a lot of time in the space and have been big leaders at a lot of the human data companies in the space.
And so, everyone saw very clearly the structural advantages that we had, and a lot of the focus was on making sure that we could deliver high-quality data to one customer before we expand to anyone else. You had to say no to a lot of things. And then, you also had a lot of people in the core part of the business that, rightfully so, there’s just checks and balances that there’s a lot of people that try to get involved. Everyone wants to say, not everyone, this is a stretch, but it’s easy to say no. It’s easy to be like, “I can’t prioritize that this week or this month. I have an existing set of priorities.” So essentially, with the exception of a few things, everyone just came straight into this new org that I built, everyone did not have any responsibilities in the existing part of the business. It was extremely clear who was the directly responsible individual across each area of the new co. And now we’ve got deeper coupling and integration points across the rest of the business, but we sat in a separate part of the office.
Everyone’s in the office five days a week, a lot of weekends. There’s a totally different expectation in hiring talent too, where it’s like, “Hey, this is a 24/7 job. This is an early-stage company.” The compensation was also different too, and based on hurdles in this new business so people felt owners creating the new co. And yeah, it’s still extremely nimble, very, very flat. Just because you run one function doesn’t mean you’re the directly responsible individual on a project. We pick the best person who’s most capable of driving an initiative forward, regardless of the function to be the DRI. We’re a lot more metrics-oriented. When I built Handshake, we resisted this operating cadence for a long time, this weekly, monthly, quarterly operating cadence. With Handshake AI, we’ve been way more focused on operating with data, and metrics, and rigor from an early stage. There’s a gentleman named Sahil on our team who’s been doing an incredible job with that. Shout out Sahil, shout out young, shout out Paco. Yeah.
Lenny Rachitsky: Okay, this is incredible. So, a few elements of what allowed this to succeed within a decade-old company. And by the way, so you’re at 200 million a year in revenue with the traditional business. You’re going to, as you said, blow past 100 million in the first year of this new business. So, it’s wild that in the first couple years, if things continue to go this way, you’ll exceed the run rate of a business that took you 10 years to build. Incredible. To make this successful, a few of the things I noted as you were talking, one is clearly you were just in founder mode. You’re the lead of this new business. You weren’t delegating it to someone, “Hey, go start this thing.” You dedicated people, “Here, we’re going to pick people. You have nothing else going on, this is your new job. You’re going to work on this stuff.”
You worked in different part of the office. There’s a metrics-based cadence. It’s just like, let’s stay really diligent about here’s how it’s going, here’s where we’re going, here’s our track, here’s our KPIs, things like that. Anything else there that you felt really important to making this work? Because a lot of companies are going to try to do this, I imagine, and so I’m curious what else you found important to make this work.
Garrett Lord: Yeah. I mean, I just really believe in separate and everything. Separate engineering team, separate design team, separate accounts and operations team, separate finance team. Early on, everything was separate. People only had one job and one job only, and that was making Handshake AI successful. We had a couple integration points more, and I had an incredible executive team and a core part of business, and now there’s becoming more and more involvement. But our executives that have built Handshake for a long time ran the core business, and I focused 80 plus percent of my time and attention on just this. And we hired an incredible engineering leader like Avery, who … We have a lot of entrepreneurs, people that have started companies inside the company. Or pardon me, people that started companies before. That was huge. A lot of familiarity with hiring talent that have only worked at early stage companies so [inaudible 00:54:44] that feels super comfortable with ambiguity.
We were also way more upfront around this is going to be chaotic. Just owning that narrative in front of all hands at the core company, owning it directly with the team. We have a separate all hands, we have separate onboarding, we have a separate recruiting team. I had some connection points, but mostly separate. And I think that was absolutely critical. We took some of the top people, I mean, we have great people in the core business, we took some great people from the core businesses and basically said, “Sorry, I know you love your old team. I know you love what you’re doing. Will you join us in Handshake AI?” And they completely forego their historical responsibilities and came over. That became really critical with engineering when things started to scale and topple, and we’re growing so quickly we took some of our top senior engineers, who were very entrepreneurial, and principal engineers, staff of engineers, parachute them in. It’s been awesome to ask some of the most talented people in the core business like, “Hey, do you want to come over here and do this?”
And sometimes they say no. They’re like, “I don’t want to work most of the weekends.” The number of 2:00 AM, 3:00 AM nights we done in this business, I mean, it’s quite regular. People sometimes don’t want to commit to that, but we’ve been up front, like here are the expectations for this team. It’s an insane pace. If you want to be a part of one of the fastest growing businesses in Silicon Valley, you can join it. The ownership too has also been huge, like owning this outcome, and we have this motto to leave nothing to chance. For a while there we drew the number of days in the year on the whiteboard and it was like, there will never be a time like this. I’ve never seen anything like it, I doubt I’ll ever feel anything like this in business again where there’s unlimited demand and it’s just our ability to execute against it.
And so, we had this motto like leave nothing to chance. How do you make sure that three months are not six months? You have no regrets. Get on the plane to go talk to a customer, make the late night push, check the data six times over again, ship the extra feature that helps. And really, a huge celebratory culture too. It’s very flat so there really isn’t this principle of … There’s so many people putting up points, directly calling out the people that are putting up points and creating a really fun environment around impact I think has been awesome.
Lenny Rachitsky: The leave nothing to chance piece I imagine speaks partly to the value of trust in what you’re doing. You win if they can trust that your data’s awesome, and great, and consistent, and I could see why that ends up being such an important part of what you’re building. And just listening to you describe this, I understand it’s obviously a massive opportunity, obviously a massive advantage you guys have, and just the stress that comes with that burden also imagine is very high of just like we can’t screw this up.
Garrett Lord: No. Yes, Handshake should be a … Business does billions dollars revenue as a public company, we should be able to continue to … I mean, and it also helps our core business. The longer term opportunity that we see is it’s connecting, it’s building the best job mashing marketplace on the internet. It’s probably one of the largest problems in the world like labor supply mashing. It’s where people spend most of their time and energy, just hours of their life they spend it at work. The process of searching for a job, applying to a job is going to be completely reinvented with AI. We’ve been leading the charge there. An AI interviewer that’s collecting skills and actually asking about your experiences, doing work simulation experiences that help employers find the best candidates. I mean, I don’t know the last time you’ve done this, but the hiring manager process, reviewing 200 resumes, are you kidding me?
I’m going to sit there and review 200 resumes? Not a chance five years from now. Students manually making cover … Not a chance. So, there will need to be a marketplace that wins in connecting supply and demand, and talent with opportunity, and we think and get psyched about the opportunity for impact here. That’s my story, I went to community college, I paid my way through school. I went to a no name school in Upper Peninsula of Michigan. I worked at Palantir as an intern, it totally changed my life, and I started Handshake because I wanted to make it easier for anyone regardless of who you knew, what your parents did, what school you went to, to find a great opportunity. And I think AI, totally step function improvement in matching. And I think that our human data business is really serving as the foundation for improving matching.
A lot of things that we’re doing in the human data business are being integrated to our core business. I think that’s going to improve outcomes for employers, save them in the aggregate like billions of dollars over time. And I think it makes the experience way better for students. So, it’s just like we have to meet the moment. We still have the stamina, and the excitement, and the passion internally in our core and in the new business to go charge after this. And that’s a lot of the message we’ve been sharing internally is it’s time to amp it up. This is a once in a lifetime opportunity to be positioned as well, and we are going to make the moment as a team.
Lenny Rachitsky: It really is. This very much feels like a once in a lifetime opportunity. Let me ask a few other questions along these lines that are something I’ve been thinking about, something that a lot of people think about, just while I have you, there’s always this question of will we run out of data? Will model stop advancing? Are we going to hit some plateau and there’s not actually going to be some AGI moment, SGI moment? So first of all, do you think we’ll run out of data? There’s a point at which we just can’t produce more knowledge and data to feed these models? And along those lines, what do you think is the biggest bottleneck to advancing models faster and further?
Garrett Lord: Yeah, I mean, it’s just the type of data we’re going to need is going to evolve. It’s going to be CAD files, it’s going to be scientific tool use data as they try to automate scientific discoveries and drug discovery. It’s going to be esoteric operating systems that exist on scientific tools. So, I love this trajectory and stitching together step-by-step instruction following. The type of data we’re going to need is going to evolve a lot. And we haven’t even talked about multimodal, and video, and text and audio. There’s a huge demand for audio data right now. So, the type of data’s going to evolve.
Lenny Rachitsky: Yeah, I use voice mode all the time. That’s on my default ChatGPT experience, just talking to-
Garrett Lord: It’s amazing. It’s amazing. I just had a baby on, or my wife had a baby on Sunday, and voice mode’s been incredible. I mean, every night, every two hours it’s like I have more questions. Voice mode’s been huge. So, shot out voice mode. Yeah, so the type of data is going to collect a lot, or change a lot. I think synthetic data has a role to play and in verifiable domains, but what we consistently hear from companies it’s like synthetic data is not going to dominate. There’s billions, and billions, and billions of dollars of value to extract as a company over the next decade and following the frontier of AI development.
Lenny Rachitsky: Let me first say just huge kudos to you for just having a kid, your wife just having a kid a few days ago, and building this business that is growing bananas and doing this podcast conversation. I really appreciate you.
Garrett Lord: Thank you.
Lenny Rachitsky: Of course. Is there anything else that we haven’t covered that you think might be helpful for folks to hear, or a part of your story that you think might be helpful for folks to learn from, or something you may want to just double down on that we’ve talked about before we get to a very exciting lightning round?
Garrett Lord: I mean, the thing I always love talking, I’m really passionate about people starting companies and helping them do so. I just think in this moment right now with AI, for young entrepreneurs that listen, that read this podcast, because I’ve been a reader since 2020. We looked.
Lenny Rachitsky: Yeah, we did check. That’s incredible.
Garrett Lord: Yeah, been a long-term reader. I’m just so curious and love sucking up-
Lenny Rachitsky: Appreciate it.
Garrett Lord: … your interviews. But it’s like you just focus on doing something of meaning that really helps people. And I think with AI, there’s going to be so many opportunities to improve the way people learn. I’m just really passionate about trying to make Handshake a platform that is not only an incredible business, but is also something that really helps solve a societal problem that matters. And yeah, that’s be my one shout out here. If anyone wants advice on how to do that or wants to reach out, I’m happy to chat.
Lenny Rachitsky: Okay, so this is an offer to share advice on starting companies within AI. Is that the offer here? Just so folks-
Garrett Lord: Yeah, that’d be great.
Lenny Rachitsky: Okay. I don’t know how much time you’ll have for the hundreds of thousands of people coming your way, but I appreciate the offer. That’s very cool. Anything else before we get to a very exciting lightning round?
Garrett Lord: No.
Lenny Rachitsky: Well, with that Garrett, we reached our very exciting lightning round. We’ve got five questions for you. Are you ready?
Garrett Lord: Ready.
Lenny Rachitsky: What are two or three books that you find yourself recommending most to other people?
Garrett Lord: I’m a sucker for Peter Thiel’s Zero to One. I read it when I started the company, and watched Peter Thiel’s startup school class at Stanford he taught back in the days where there wasn’t everything written on the internet about how to start companies, and just think he was the coolest. Love Shoe Dog. I think it’s the epitome of starting a company. Hard Things About Hard Things obviously, but these are all quite common books.
Lenny Rachitsky: But also classics. Ben Horowitz is coming on the podcast, talk about Hard Things About Hard Things.
Garrett Lord: Super cool.
Lenny Rachitsky: The Hard Thing About Hard Things. Yeah. Okay, have you seen a recent movie or TV show you really enjoy it? I imagine you don’t have much time for this, but-
Garrett Lord: I’m going to get blasted for this, but I did start Game of Thrones with my wife, and I cannot-
Lenny Rachitsky: For the first time.
Garrett Lord: Yeah.
Lenny Rachitsky: Okay, cool.
Garrett Lord: So, I got a lot of catching up to do.
Lenny Rachitsky: Why would you get … No, this is great. It’s like people that have watched it-
Garrett Lord: I’ve loved it so far.
Lenny Rachitsky: You’ve loved it so far? Okay. It’s quite gruesome, that’s the only downside of that show. Don’t watch it before you go to bed, I don’t know how many gruesome scenes you’ve seen already. Do you have a favorite product you’ve recently discovered that you really love?
Garrett Lord: The SNOO. The baby automated SNOO has really helped us a lot. So love the, shout-out SNOO team.
Lenny Rachitsky: Amazing. I had a SNOO as well. We never actually turned it on, we just ended up using it as a basinet the whole time.
Garrett Lord: Yeah, most of the time it’s not turned on, but a couple of cries it’s been turned on, it’s been very helpful.
Lenny Rachitsky: Do you have favorite life motto that you find yourself coming back to, sharing with other people?
Garrett Lord: I love that leave nothing to chance, leave it all out on the field. Grew up in a really hardworking family, and dad worked really hard to provide, make it happen for us and it’s like just give it your all. Leave nothing a chance.
Lenny Rachitsky: Okay. So the last question, I was researching you in prep for this podcast and there’s a story that I love about your hustle early on is when you were going from campus to campus pitching schools to join Handshake, and there’s a story where you had to shower in the Princeton’s pool to save money because you just didn’t have a place to stay. Is there something there? Is there a story there you could share?
Garrett Lord: Yeah, so it was a tough one. I mean, I almost got arrested at Princeton, because I mean, I guess for entrepreneurs that are traveling around all the time, we were sleeping out of our car. We had this Ford focus, we would put 20, 30,000 miles on it, sleep in the back of like McDonald’s parking lots because they’re well lit and had good wifi back in the day. And instead of staying in a hotel, a way to freshen up ahead of your meeting is every university has a pool and the pool’s almost always, it is always open. We never had a situation where it’s always open for people to swim in the morning, like fitness. Faculty, students. And every pool, what do they have? They have a shower.
So, you could go to any pool at any university in the country, and you can get a free shower and freshen up. So, the Princeton campus security did not appreciate me showering as a non-student, but I think it meaningfully helped us because the Princeton campus security called the career service center director we were selling to, being like, “Who’s Garrett Lord? Is he really here to pitch you software for your career center?” And it made the start of the meeting with the career center really stimulating and exciting, because they were like, “You showered in our pool, you drove here?” “Yeah, we drove here from Michigan.” And so, I think that showed a level of commitment that was exciting for them.
Lenny Rachitsky: Fast-forward to all these founders now starting to use this growth lever of getting in trouble with the campus police to get better meetings with the school leaders. Incredible. Garrett, this is such an insane, amazing, inspiring story, just like what you’re building and the opportunity here, and just how it’s fast, it’s going, and all the advantages you have. If I was an investor in Handshake, I’d be like, “All right, 10 years, it’s going great.” And now it’s like, “Whoa, holy shit. Where did this come from?” Incredible. And it’s just also really meaningful. So, I’m really happy that you made time for this in spite of the madness you are in right now. Two final questions, where can folks find you if they want to maybe reach out or maybe if you’re hiring, let us know. And then, how can listeners be useful to you?
Garrett Lord: I mean, sign up for Handshake. If you want to message me on there, it’s the easiest way to reach me. Just find me at garrettlord@Handshake, and you find me on Twitter. Love X, huge X guy. You can email me at garrett@joinhandshake.com and double R, double T. And how can you be helpful? We are trying to hire so many people. We have offices in New York and in San Francisco, and London and Berlin. If you have friends that are maybe passionate about this, you let them know, or you’re interested in the learning more, please reach out. We’d love to talk to you. Hiring is like the number one problem we have right now to meet the demand. So, if you’re talented and interested in learning more about Handshake, if you want to work on our consumer product, if you want to work on our employer products, cool PLG issues or the state-of-the-art consumer social experience, like reach out, or you want to work on the AI business we’d love to talk to you.
Lenny Rachitsky: To make it even more clear for folks, what roles are you most hiring for? Is it every role? Is it engineering?
Garrett Lord: Engineers.
Lenny Rachitsky: Engineering, all right. If you’re an engineer and want to join one of the fastest growing AI companies in the world right now, here we go. We’ll link to your careers page in the show notes.
Garrett Lord: Thank you.
Lenny Rachitsky: Yeah, of course. Garrett, thank you so much for being here. This was incredible.
Garrett Lord: Of course.
Lenny Rachitsky: Bye everyone.
Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at Lennyspodcast.com. See you in the next episode.
Glossary
| English | 中文 |
|---|---|
| ARR | ARR(年度经常性收入) |
| ASI | ASI(通用超级智能) |
| auto eval | 自动 eval |
| data labeling | 数据标注 |
| DRI | DRI(直接负责人) |
| expert network | 专家网络 |
| frontier AI model | 前沿 AI 模型 |
| gross margin | 毛利率 |
| instructional design | 教学设计 |
| KPI | KPI(关键绩效指标) |
| lightning round | 快问快答 |
| LLM | 大语言模型 |
| matching market | 匹配市场 |
| moat | 护城河 |
| multimodal | 多模态 |
| network effect | 网络效应 |
| post-training | 后训练 |
| pre-training | 预训练 |
| principal engineer | principal engineer(首席工程师) |
| product market fit | 产品市场契合点 |
| reinforcement learning | 强化学习 |
| rubric | rubric(评分标准) |
| SFT | SFT(监督微调) |
| staff engineer | staff engineer(资深工程师) |
| step function improvement | 阶跃式提升 |
| supervised fine-tuning | 监督微调 |
| swim lanes | 泳道 |
| synthetic data | 合成数据 |
| trajectory | 轨迹 |
| universal basic income | 全民基本收入 |
| verifiable domains | 可验证领域 |
Reformatted by reformat_english.py
揭秘训练每一个前沿 AI 模型的专家网络 | Garrett Lord
文字记录
Garrett Lord: 再也不会有这样的时刻了。我从未见过这样的景象,我怀疑在商业生涯中再也不会有类似的感受——需求是无限的。你怎么确保三个月后、六个月后自己不会后悔?登上飞机去和客户面谈,深夜加班冲刺,把数据反反复复核查六遍。
Lenny Rachitsky: 你的公司通过创造新数据来持续推进模型的智能水平。这是你在已有业务之上又建起来的一门新生意。
Garrett Lord: 我们是全球最大的专家网络。我们拥有巨大的战略优势——几乎没有客户获取成本。人类数据领域唯一的护城河就是对受众的触达。
Lenny Rachitsky: 你们是在模型训练完成之后介入,根据你们创造的额外数据来调整权重。
Garrett Lord: 模型已经强大到不再需要通才了。他们真正需要的是专家。
Lenny Rachitsky: 一方面是所有这些学生在训练模型变得更聪明,另一方面他们自己可能反而更难找到工作,这两者之间存在一种张力。
Garrett Lord: 我们从雇主那里听到的并不是这样,这只是让人类能够更加高效。过去你会在简历上写”熟练使用 Google 搜索”,因为你是伴随 Google 长大的。作为 AI 原生代,年轻人拥有巨大的优势。
Lenny Rachitsky: 今天的嘉宾是 Garrett Lord。Garrett 是 Handshake 的联合创始人兼 CEO,Handshake 是你可能还没听说过的最有趣、最了不起的 AI 成功故事之一。Handshake 已经运营了十多年,本质上就是大学生的 LinkedIn,是学生与企业对接找工作的平台。它是每一家财富 500 强公司的首选平台。超过 1,500 所高校、超过 2,000 万学生和校友、以及超过 100 万家企业使用他们来招聘毕业生。今年年初,Garrett 和他的团队意识到,他们庞大的、独有的学生网络——其中包括数以万计的博士和硕士——对 AI 实验室来说极其有价值,可以帮助他们创建和标注高质量的训练数据。于是,他们在今年一月从零到一启动了一项新业务。四个月后,ARR 达到了 5,000 万美元。目前他们正朝着 12 个月内突破 1 亿美元 ARR 的速度前进。不到两年,这项新业务的收入就将超过他们经营了十年的老业务。
这是一个真正令人难以置信且罕见的故事,也是我认为很多团队可以从中学习的——AI 正在创造大量机会,但也带来大量潜在的颠覆,而这是一个公司基本上自我颠覆的精彩案例。这期节目干货满满,包括一个入门科普:人们标注和创建数据来训练模型到底是在做什么?非常感谢 Garrett 抽出时间来做这期节目,他的妻子本周刚生了宝宝,同时他还在扩展这项疯狂的新业务。谢谢 Garrett。如果你喜欢这个播客,别忘了在你最喜欢的播客应用或 YouTube 上订阅关注。
Lenny Rachitsky: Garrett,非常感谢你来做客。欢迎来到播客。
Garrett Lord: 谢谢邀请。我是长期订阅者。
Lenny Rachitsky: 感谢支持。好,在我们深入讨论你的数据标注业务那疯狂的轨迹之前——这确实是一个精彩的故事,我认为很多创始人和产品团队在面对这波 AI 颠覆时都能从中学到很多——我想先帮大家理解一下,数据标注到底是怎么回事?人们到底在做什么?为什么这东西这么有价值?当今世界上增长最快的一些公司,包括你们,做的就是这件事。显然这里面有非常重要的东西。我大概算了解一点,可能也不太了解。我想很多听众也有同感。那我就直接问了,数据标注到底是什么?人们到底在做什么?还有,为什么这对前沿 AI 实验室如此有价值?
什么是数据标注
Garrett Lord: 好的。我觉得可以先退一步,看看训练一个模型到底是什么样的。基本上有两个主要环节:预训练(pre-training)和后训练(post-training)。很长一段时间以来,这些 AI 提供商、大语言模型(LLM)或前沿实验室主要专注于在预训练端不断吸入越来越多的信息。那基本上就是整个人类文字知识语料库——不仅是文字,还包括每一段 YouTube 视频、每一本书,基本上就是在追求把互联网上的一切都吸收进来,这就是预训练的部分。预训练带来了很多收益,模型持续变得更好。大约 18 到 24 个月前,我们开始看到来自预训练的收益明显趋于平缓,因为他们基本上已经把互联网上的所有知识都吸收完了。于是,实验室的重心转向了后训练端——现在大部分的收益都来自那里。
Garrett Lord: 后训练做的事情,就是对他们关心的每个学科或能力领域的数据进行增强和改进。比如编程、数学、法律或金融,他们专注于收集能真正提升模型能力的高质量数据。你可以看到很多流行的基准测试,就是所谓的模型榜单。当 Llama IV 发布时,你会看到各个领域的基准成绩,实验室内部的每个研究团队都有不同的用例。基本上他们在做实验,你可以想象成类似科学方法的过程。他们对如何改进模型提出假设,然后尝试收集小规模的数据来验证这个假设是否成立。如果假设被验证为真,他们就会在该方向上扩大数据收集的规模。具体形式可能是强化学习环境,可能是轨迹数据,也可能是音频和多模态数据,也可以是基于文本的提示-响应对。
也可以是基于人类反馈的强化学习,即偏好排序数据。这就是当前模型训练的前沿。现在模型的大部分提升都来自后训练环节。而要保持模型始终处于绝对前沿,需求量是极其庞大的。
Lenny Rachitsky: 所以,预训练就是把整个互联网喂进去——这是人类创造的所有数据,让模型自己学会知识、事实、推理等等。后训练的话,能不能说基本上分两大类?一类是基于人类反馈的强化学习 RLHF,另一类是微调?
Garrett Lord: 算对也不完全对。比如拿轨迹数据来说,或者你想让模型完成航班搜索,或者端到端的会计流程,或者进行生物实验,这些都需要实际的轨迹数据。目前很多实验室对于应该收集什么数据仍然有自己的判断,而且这个领域变化非常快。但我觉得强化学习主要就是偏好排序——你更喜欢哪个回答,A 还是 B?SFT 数据则是一个提示和一个响应。显然,各实验室现在非常关注这些思维或推理模型。要提升推理模型的能力,就需要真正提供逐步的推理步骤。而当你与这些前沿模型交互时,它们在非常高级的领域中仍然会遇到困难。所以,我认为他们在使用多种类型的数据来提升模型的各种能力。
Lenny Rachitsky: 我听到的是,后训练还有其他方式。在这些分类里,你们主要聚焦哪个?这三个左右的类别中,你们在哪些方面对模型的帮助最大?
Garrett Lord: 我们作为企业真正的独特价值在于我们拥有一个高度活跃的专业受众群体。我们有 1800 万专业人士,其中包括 50 万博士,300 万硕士,是一个全球性平台。所以,无论你在任何领域需要什么——学术知识也好——博士的定义是什么?怎么拿到博士学位?你要答辩你的论文。答辩意味着,一般来说,你已经证明了自己在某个特定领域拓展了人类的知识边界。所以我们能够精准地将这些受众定位到化学、数学、物理、生物、编程等各个方向,真正触及那些从未出现在互联网上的人类知识领域,这正是我们的优势所在。我想说的是,谈到数据标注市场,为了让它更直观——过去这种工作主要是通才做的。
在模型变得更好之前,市场上大量依靠的是有才华的、低成本的国际劳动力来完成基础的通用任务。但实际上发生的是,模型已经变得足够好,通才不再被需要了。他们真正需要的是专家——模型所关注的每个领域的专家。你可以把这些模型构建者理解为,他们聚焦于经济中最具经济价值的能力领域。目前来说,主要集中在高级 STEM 领域、高级科学和数学领域,以及由此衍生的会计、法律、医学、金融等领域,他们希望在这些方面让模型变得更强。至于我们正在做的工作,我想回到你的问题来回答——我们跨越了非常多的领域。我们有数百万本科生被用于音频方面的工作,用于根据语音和语调、你所在的地理位置、女性和男性各自的偏好来定制模型。一直到最前沿的博士级 STEM 领域。
标注工作的实际内容
Lenny Rachitsky: 好。那是不是可以说,基本上所有可获得的数据都已经被训练过了,而你们公司创造新的数据、新的知识,来持续提升模型的智能水平?
Garrett Lord: 对。我还会补充说,我们也帮助指出模型薄弱的地方。要让一个模型出错,对普通人来说其实挺难的,普通人很难让模型给出错误回答。但如果你是一个物理学博士,你可以进入物理学的多个子领域,证明模型到底在哪里出了问题——要么是推理步骤出错,要么是标准答案本身有误,或者当我们引入工具或需要遵循某些步骤流程时出错。我不会说这对他们来说很容易,但普通人确实无法让模型出错,这正是我们发挥作用的地方。
Lenny Rachitsky: 所以本质上就是发现模型犯的错误。好的,那这些人到底在做什么?我知道有各种不同的类型,你也描述了数据生成的各种方式和什么样的数据有用。那能不能举几个最常见的例子?假设一个博士坐在那里做标注,他们实际上在做什么?
Garrett Lord: 一个很好的例子是一篇公开发表的论文,叫 GPQA。感兴趣的工程师可以去读一下。这篇论文的核心思路是:你先让模型出错,然后提供一个标准答案——即问题的正确答案,再提供逐步的推理步骤。你可以想象,因为模型是非确定性的,模型可能偶尔一次答对了,但五次里不能稳定答对三次。所以你要真正证明模型在哪里失败了,并具体拆解它在哪里失败。也许它知道问题是什么,也能得出正确答案,但得出答案的步骤是错的,而他们真正关注的是到达答案的那些步骤。比如一道数学题有十个步骤,第六到第十步是错的。那怎么修复这些步骤呢?
他们具体在做什么呢?他们进入平台——我们非常注重打造这种体验,把这些人当作专家来对待。博士学生对被对待方式的期望,与低成本国际劳动力是不同的,工作期望也不一样。这些博士进入一个社区,我们有教学设计团队和评估团队,会反复帮助他们了解如何使用我们构建的工具,以及如何与最新的模型交互。然后他们就开始实际创建数据了。而这个过程——从我们这边来看,模型构建者需要确认我们产出的数据是高质量的。所以我们有自己的研究团队,自己的后训练团队。
我从 Meta 招了一位负责过后训练的同事,然后我——
Lenny Rachitsky: 希望你给他开出了不错的薪酬。
Garrett Lord: 当然。AI 人才的争夺战确实代价不菲,但我非常、非常荣幸能与他共事。那么,对于每一条数据,我们都要为他们搭建一个实际创建数据的工作环境。然后我们需要在单元级别上评估那条数据带来的实际收益,以及它能否在某个特定能力方向上带来提升。此外,我们还专注于不断演进使用场景,紧跟模型构建者的需求——他们需要越来越多的是真实世界中的工具使用和基于轨迹(trajectory)的数据。
Lenny Rachitsky: 好的。这里面的内容太多了,我们可以无限聊下去。但我觉得这真的很有意思,因为人们总是在听到这些概念,却几乎不了解它到底是怎么回事。所以对我来说这非常有趣,我觉得对很多人都会有帮助。所以本质上来说,一位博士,比如一位生物学博士,他们的工作就是找出 ChatGPT 产出的内容中的缺陷,然后给出正确答案。这些被用来微调模型——这里是你做得不对的地方,这是正确答案——从而改进模型。
Garrett Lord: 没错。
Lenny Rachitsky: 这样理解是否简单准确?请纠正我说错的任何地方,我不想让人产生误解。
Garrett Lord: 嗯,一个很好的例子,我们拿一个不可验证的领域来说,比如教育学。我们的网络上有位博士学生 Rachel,她在迈阿密大学获得了博士学位,当了二十年的老师,教八年级学生。她还在当地一所社区学院担任教育学兼职教授。所以,她正在与最先进的模型进行教育设计方面的交互——实际上是在尝试理解教导人的最佳方式,以及如何发现模型在培训方式上的错误,并帮助模型理解教育设计的前沿——这是建立在她十多年教八年级的一线经验和教育学博士学位之上的。这就是一个例子。从这种不可验证的领域,一直到可验证的工程问题——你可以看到最新模型在哪些问题上失败了。
轨迹数据与人类思维研究
还有一个方面,我们谈到专业领域,比如那些强化学习(reinforcement learning)环境。目前有不少论文,基本就是让人在完成逐步工具操作时进行旁白解说。比如他们从头到尾解决一个问题,与多个不同的服务区域交互,使用多种不同的工具。有论文专门研究这个——一边操作一边讲述自己在做什么,实际跟踪并录屏,记录鼠标的移动轨迹,展示他们是如何解决问题的。当他们遇到障碍时会怎么做?模型构建者真正想理解的是人类的思维方式。
Lenny Rachitsky: 你提到了”轨迹”这个词。能解释一下它到底是什么意思吗?感觉你已经提了好几次了,而且这个概念对这一切似乎很重要。
Garrett Lord: 轨迹基本上就是收集你正在做的一切的完整环境。所以它包括你的屏幕、你的鼠标——
Lenny Rachitsky: 噢,我明白了。哇。
Garrett Lord: 对。
Lenny Rachitsky: 包括旁白解说,好的。另外,这可能太技术了,但所有这些工作的最终产出是什么?比如那位老师,产出的是 JSON 文件、XML 文件,还是文本文件?
Garrett Lord: 是结构化的 JSON 数据。
Lenny Rachitsky: JSON 数据?好的。
多模态数据与评分标准
Garrett Lord: 然后我们还有多模态(multimodal)方面的工作,比如音频——对音乐进行分类和理解。我们正在与全国顶尖音乐院校里数百名最优秀的音乐专业学生合作,改进模型对音乐的理解。还有一个东西我们还没谈到,叫做 rubric(评分标准)。Rubric 模型就是让模型充当评判者。什么是好的教育设计?什么是好的 MRI 结果?在某些领域,你其实没有一个确定无疑的正确答案。所以模型可以坐在中间充当评判者,实际去评估什么是对的。回想一下你的学生时代,你怎么才能在五千字的论文里拿到 A?需要有出色的开篇陈述和科学论据。所以你可以构建一套 rubric,让模型在中间自动评估回答。我们也看到了大量关于 rubric 的工作。
市场验证与模型构建者的核心需求
Lenny Rachitsky: 你可能会想,凭什么相信这一个老师的意见,认为这样做就是对的?但很酷的一点是,市场自己会说话。如果这些模型被越来越多地使用,人们喜欢它们、认可它们,那么我猜中间一定有验证步骤来确认这些做法是好的、其他人也认为这是个好主意。市场机制本身就会告诉你,你提供的数据是否正确、是否是人们想要的。这里面还有更多的说法吗?
Garrett Lord: 我自己没有 AI、数学或物理的博士学位,也没有亲自训练过模型。我们有前沿 AI 模型,但每一条数据是否真的带来了改进,其中涉及大量的学问。目前有很多关于如何确保你产出的数据确实在改进模型的科学研究。这对模型构建者来说也是非常难判断的。退一步看,他们真正关心的有三件事。首要的是质量。你必须拥有高质量的数据。你可以想象,训练模型就像教学生,如果你给了它错误的数据,后续要纠正极其困难。所以质量是第一位的。其次是另一个巨大的难题——数量。你怎么在最前沿的化学、数学、物理领域生成数千条数据,同时确保质量足够高?
拿物理来说,我们直接联系斯坦福、伯克利、MIT 的学生,他们是全美最顶尖物理院系中 GPA 最高的。所以我们能够快速扩展数据规模,产出高质量数据,这是他们非常看重的。第三点是速度,因为模型构建者有大量假设,在不断测试不同的 pipeline。他们可能同时押注三四个方向,一旦其中一个开始显示出收益——想象你是研究员或负责这个流程——你就会开始扩展那条 pipeline,扩大那类确实带来改进的数据的生产,同时可能砍掉另外两三个没有显示提升的项目。
所以,能够在几天内快速为他们周转,并提供大量高质量数据,是模型构建者最关心的事情。为此我们在自己的端上构建了相当多的技术来评估每一条数据。我们有自己的后训练团队,租用我们自己的 GPU,力求能够直接与这些研究员坐在一起,分享我们在创建数据过程中观察到的信息,以及这些数据如何能改进他们的模型、他们最好如何利用这些数据来训练。希望这些能帮助你理解。
后训练的类型
Lenny Rachitsky: 回到后训练的类型,我觉得这也许会有帮助——至少对我建立心智模型来说:有预训练,有后训练,后训练内部有强化学习、人类反馈,还有微调的概念。还有 evals 之类的——
Garrett Lord: 还有 SFT,对。
Lenny Rachitsky: SFT,就是监督微调(supervised fine-tuning)?
Garrett Lord: 对。
Lenny Rachitsky: 好。那你刚才描述的那些工作,主要算是监督微调吗?
Garrett Lord: 是的,而且我们上述所有类型都在做。我们不搞自动 eval(auto eval),我们产出 rubric,这些 rubric 会被用于自动 eval。但没错,其他的都在做。
Lenny Rachitsky: 好的,太棒了。所以本质上就是:有一个模型,在所有这些了不起的数据上训练好了,你们在模型训练完之后介入,根据你们创建的额外数据来调整权重。有趣的是,这是一个可扩展的系统。我想谈谈你们拥有的那些产出这些数据的优秀人才的供给问题,但更令人惊叹的是,人类居然能做到这件事。你可能以为这需要某种无限可扩展的东西,但人类坐在那里创建数据,确实在起作用,并且在显著提升模型的智能。
人类在 AI 训练中的角色
Garrett Lord: 哦,是的。我觉得可能有个有趣的笑话:所有 MBA 都觉得这一切都会消失。而我认为,只要模型还在进步,这个过程中就需要人类的参与。当你和这些实验室的首席科学家和研究员交谈时,你会发现数据类型会不断演进,他们试图捕获和收集的内容也会变化,但在未来十年内,在达到完整的 ASI 之前,这个领域都需要人类的参与。你可以想想,目前很多模型连基本的 trajectory 都难以正确完成。所以现在人们非常关注学术领域,我认为他们会继续关注学术领域,但对专业领域的需求也会远远大得多——基本上,知识工作者在工作场所解决的每一个 trajectory 或分步问题,这些实验室都在追求收集数据,以便在这个过程中为人类创造尽可能多的价值。
AI 与就业的张力
Lenny Rachitsky: 那我来问你一个问题。我觉得人们可能会感受到一种张力:一方面,这些学生在训练模型变得越来越聪明;另一方面,如果模型聪明到入门级岗位不再大量招聘,他们自己可能更难找到工作。你怎么看待这种张力?你觉得这是一个真实的问题吗,还是说——
Garrett Lord: 我可能更倾向于”GDP 增长”而不是”全民基本收入”那个阵营。我非常相信这将提升和加速每个人在经济和世界中创造影响力的能力。我们听到的是——大约有一百万家公司在使用 Handshake,百分之百的财富 500 强都在用 Handshake,所以我们基本上支撑了绝大多数年轻人找工作的渠道。很多人夸张地说所有年轻人都会找不到工作,而我们从雇主那里听到的并非如此。我们听到的是:拿社交媒体营销来说,以前你需要一个会用 Photoshop 的人,要拍照,要做视频。然后你需要一个懂营销分析平台的人来追踪你在不同社交媒体上的发帖情况。而现在,一个年轻的、有才华的、AI 原生的、穿上钢铁侠战衣的年轻人就可以上手了——他们可以自己制作视频,自己产出创意素材,在多个社交媒体平台上发布,运行自己的所有分析。他们不需要一个数据科学的学位就能做到这些。
拿我们公司的一个实习生来说,我记得他第一天下午就提交了他的第一个 PR。你做过 PM,你知道历史上要搭好开发环境、搞清楚在哪里创造价值有多难。他直接拿了一个 bug 就修掉了。所以,我真的相信这只是让人类变得更加高效、创造更大影响力。当然,数以亿计的工作岗位,工作会演变。人们会被替代,他们需要提升和重新学习技能,我认为 Handshake 在帮助知识工作者完成这种转型方面可以发挥重大作用。
AI 原生代的优势
Lenny Rachitsky: 这一点已经出现了好几次,我觉得非常好——刚从学校出来的年轻人实际上更有可能取得成功,因为他们是伴随这些工具长大的,对所有这些先进工具更加原生态,所以他们一进来就像猛兽一样,能做的事情多得多。
Garrett Lord: 你还记得吗——这稍微早于我的时代——以前你会把 Google 搜索作为一项技能写在简历上。你是一个善于用 Google 的人,因为你是伴着 Google 长大的。我觉得 AI 原生、穿上钢铁侠战衣、懂得如何利用这些工具,就像年轻人拥有巨大优势一样。
Lenny Rachitsky: 对。而且如果他们参与了训练这些模型,我猜还有其他很酷的优势。
Garrett Lord: 是的。关于这一点,我们从数千名 fellow 那里听到的是:他们在课堂上的同时,实际上也在产出研究成果。我们说的是全国顶尖机构的博士。他们可以在自己的专业领域赚取每小时 100、150、200 美元。这很棒。当助教每小时赚 25 美元,或者你可以每小时赚 150 美元来破解最新的模型。我们从 fellow 那里听到的是,他们把很多这样的洞察带回课堂,帮助自己更有效地教学。更重要的是,他们开始学习如何利用这些工具来真正推进自己的研究领域。他们相信这些工具可以帮助他们更高效地利用时间,从而推进自己的研究。所以,靠学习一项技能还能拿钱,确实很酷。
数据标注领域的前沿认知
Lenny Rachitsky: 在我聊这一切是怎么诞生的之前——那确实是一个很精彩的故事——关于数据标注、强化学习这整个领域,还有什么你觉得人们没有充分理解的,或者你认为非常重要的东西?现在发生的事情太多了。就像我说的,全球增长最快的一些公司都在这个领域,Scale 刚刚被收购——某种程度上被以 300 亿美元收购。还有什么你觉得人们需要理解的吗?
Garrett Lord: 总体来说,任何时候你在与一个模型交互,让它做一些非常高级的事情,而它没有达到你的预期——那么在某处,很可能有一位该领域的顶尖专家,正在直接为前沿实验室最优秀的研究员工作,试图理解和经历科学迭代的过程来让它变得更好。这里的假设是,他们已经拥有了人类所有已书写和记录的知识。所以,只要用 AI 解决任何问题——任何人类问题——还存在困难,就需要人类参与其中来帮助推进。而且模型不会泛化。我的意思是,显然这个领域会有很大进步,他们收集的数据类型也会发生很大变化,但在前沿上,这一切相当令人兴奋。
Lenny Rachitsky: Kevin Wheel 上过我的播客,他是 OpenAI 的 CBO(首席商务官),他有一个观点让我印象深刻:你今天使用的模型,将是你用过的最差的模型。
Garrett Lord: 我特别喜欢这句话。
Lenny Rachitsky: 只会越来越好,想想就令人难以置信,现在我们知道了事情为什么会变得更好,因为你们所有人都在做这些工作。关于 Scale 这件事还有一个简单的问题,他们大概算是这个领域的主要公司,现在他们被收购了,Alex 去了 Meta 负责超级智能部门。他们在数据标注领域还是一个重要参与者吗,还是说他们已经退出,这是一个巨大的机会?
Garrett Lord: 是的。我的意思是,向整个 Scale 团队致敬,非常尊重他们所建立的一切,这个领域有很多优秀的公司在运营。我觉得你问题的核心是,如果你把你的研究团队和模型构建团队,以及他们正在运行的实验,视为你改进模型的基石,那你大概不会希望你正在研究的最新研究成果被一个同行投资。我的意思是,这基本上就是我们在行业中听到的普遍声音。所以,我们看到了令人难以置信的需求激增,而且我认为我们的定位极为有利。我们喜欢说,人类数据领域唯一的护城河就是对受众的触达。基本上,这个领域有很多很多小玩家,一些中等规模的玩家,他们基本上就是在投放 TikTok 广告、Instagram 广告,花钱买 Google 搜索展示广告、YouTube 广告,然后他们会说,“你能帮我找 200 个物理学博士吗?”
他们能做什么?他们只能做一件事。他们有 100 个招聘人员在岗,全都上 LinkedIn,全都发消息,花几百万美元做效果广告投放。某个人在刷 Instagram 动态,他是个物理学博士——而你其实没法精准定位到他们——然后他们看到”来训练模型吧”。他们的反应是,“我从来没听过这个品牌。“我们拥有的巨大优势,以及为什么我们在市场上如此迅速地引起共鸣,是我们用十年时间与 1800 万人建立了信任,他们信任我们,我们建立了很强的品牌亲和力,他们使用 Handshake,他们有活跃的个人资料,我们拥有大量关于他们学业表现和在校经历的信息。所以我们能够非常精准地触达目标人群,以比任何人都更快的速度实现高质量数据的规模和体量。我认为这种触达受众的竞争优势在市场上确实产生了强烈的共鸣。
从职业平台到 AI 数据业务的诞生
Lenny Rachitsky: 这正好引向了我接下来想聊的话题,就是这个业务是如何诞生的。这是一个建立在你已有业务之上的新业务。据我了解,当时你们已经有大约 1.5 亿美元的收入,你们在这个领域已经做了很长时间。你发现了这个机会,现在回过头来看,这显然是一个绝妙的主意——实验室需要数据,而你们拥有大量顶尖专家的供给。多么好的机会。谈谈你是如何首先意识到这是你可以做、也应该做的事情,然后又是如何开始沿着这条路执行的。
Garrett Lord: 是的,我觉得这是从帮助人们启动、重启或开始他们的职业生涯这个核心使命出发的一次非常自然的延伸。在这个新的就业生态系统中,用你的技能变现,未来会呈现出非常不同的面貌。具体说到我们是怎么发现这个机会的,因为我们拥有对这个受众群体如此庞大的触达,而且随着世界从通才转向专才,我们成为了全球最大的专家网络。我们拥有比任何其他平台都多的博士——其中 50 万博士在使用 Handshake。我们有 300 万硕士在校生和校友。所以我们开始看到大量中间商公司来找我们说,“我们能不能在你的平台上招募博士和硕士生?“就像任何一个优秀的交易平台一样,我们开始把这些用户导流到各个平台上去,然后从用户的反馈中开始意识到,他们的体验非常糟糕。
训练过程非常事务化,你怎么获得报酬也非常不透明。从开始到实际完成项目,中间有大量的流失。所以我们开始思考——公司当时从帮助这些其他平台中赚取了数千万美元——而真正推动我们行动的是,我们也听到了前沿实验室的声音,他们开始直接联系我门,试图直接合作,几乎想要跳过中间商。我们开始意识到,我们可以真正服务好我们的用户、我们的博士、我们的专家,可以善待他们。我们坚信,在追求 ASI 和推进 AI 的过程中,必然会需要一个平台,一个以专家为先的平台,也会需要一个全世界所有人都可以去的地方,在这些实验室专注于改善所有这些多学科能力的时候,去将自己的技能和知识变现。是的,我们进入这个业务——实际上我是在圣诞节和新年期间开始做的。那是我开始四处飞的时候。
我的家人觉得我有点疯狂,居然在飞机上追着各种负责人跑,但我们组建了一支令人难以置信的团队,成员来自人类数据领域,然后真正开始在一月份搭建我们的平台,大约五个月前开始真正将这些合作关系变现。快进到今天,我们正在与七家前沿实验室合作,基本上是每一家在做工作、构建最好的大语言模型的实验室,团队规模暴涨,收入暴涨,这真的是一段不可思议的旅程——在公司内部重新运营一家新公司,相当于第二次创业。
Lenny Rachitsky: 分享一些数字——告诉我这些是否准确,或者你是否愿意公开——但我听说你们仅用四个月就达到了 5000 万美元的收入?如今已经八个月了,你们有望在第一年达到 1 亿美元的收入。
Garrett Lord: 我觉得我们会远超这个数字,但确实如此。
Lenny Rachitsky: 好的。太厉害了。我甚至不知道有七家前沿实验室——
Garrett Lord: 四个月从零到 5000 万,确实还不错。
Lenny Rachitsky: 从零到 5000 万四个月,这确实了不起。门槛一直在不断变化。一年前这会是传奇级的成就。现在就像,好吧,又一个这样的。四个月 5000 万,没什么大不了的。真的太疯狂了。稍微拉远一点,对于那些不太了解 Handshake 原始业务的人,那是什么?你当时坐拥的那个网络到底是什么?
Garrett Lord: 那个网络大概做到两亿。这个大概会做到——
Lenny Rachitsky: 两亿。
Garrett Lord: 对。
Lenny Rachitsky: 好的。
Garrett Lord: 我们大约有 600 位非常投入的团队成员在核心业务上工作,我想把他们区分开。这不是两个业务,我认为是一个业务,但这个业务是什么?如果你是美国的年轻人,在过去五、六、七、八年里毕业的,你手机上大概率有 Handshake。你一定知道 Handshake 是什么。它在美国年轻人中已经成了一个动词,在大学读博士或硕士的人中也成了一个动词。我把它叫做一个无需连接的图,意思是 LinkedIn 非常关注你认识谁、你的经历是什么。LinkedIn 上第一个问题是你的工作是什么?而很多年轻人起步时,从来没有过工作。他们没有 500 个人脉可以加到自己的关系图里。
而在 Handshake 上,你从探索和发现开始,弄清楚如何在学校中找到方向,“哦,我是工程师。也许我想做 PM,也许我想去创业公司,也许我想去大公司。” 你想从同龄人和年轻校友那里了解利弊是什么。所以 Handshake 是一个我称之为非常社交化的平台,有群组、消息、个人资料、短视频和动态流,全部围绕你的兴趣,真正帮助你在职业生涯早期建立信心,找到第一份工作、第二份工作,管理 18 到 30 岁这个阶段。
Lenny Rachitsky: 这个业务做了多久了?
Garrett Lord: 已经十年了。
时机与不公平优势
Lenny Rachitsky: 十年,好的。所以这感觉就是——真他妈的,你们在正确的时间、正确的地点,拥有这个现在极其有价值的网络。多么有趣的故事。我觉得这只是又一个有趣的例子——你做了一件事情很长时间,然后突然间 AI 就打开了一种全新的方式,来利用你长期积累的东西。这让我想到 Bolt 和 StackBlitz,他们花了七年时间构建一个基于浏览器的操作系统,可以在浏览器里运行一个 OS。他们就像,“我不知道,没人需要这个。我们在干什么?” 然后突然间 AI 来了,他们说,“哦,如果我们用 AI 在浏览器里构建应用,直接用 AI 为你生成产品会怎样?” 现在它大概是世界上增长最快的公司之一。
Garrett Lord: 是的。
Lenny Rachitsky: 太有意思了。所以我觉得这对大家来说是一个值得思考的时刻——我们做过的事情中,有没有什么能给我们一个新的机会,基于这种不公平优势去构建一些巨大的东西?
Garrett Lord: 我还觉得,随着你的公司规模、人数和成熟度的增长,在业务内部孵化新东西也很难。在很多方面都很难。从零到一构建、找到产品市场契合点、快速扩张团队的方式,和运营一家已经存在十年、有几百几百人的成熟业务的方式是非常不同的。所以,我非常享受并在业务内部第二次重新来过中找到了大量热情。而且我们有这个巨大的战略优势——零获客成本,而且我们的转化率和留存率比其他任何平台都高出一大截,因为我们有这么强的用户认同感。
数据标注的获取方式
Lenny Rachitsky: 这里其实有两条线索我想追问,我先追第二条——数据标注的工作从哪来的问题。有一种非常清晰、简单易懂的方式,就是专家坐在那里创建数据。另一个我知道这个领域很多其他公司使用 Scale,特别是用国际低成本劳动力。除了这两种之外还有没有其他方法?其他公司是怎么做的?
Garrett Lord: 我觉得如果你在乎构建一个真正高质量的业务,有好的毛利率和高质量的增长,这个生态里的一家领先玩家有 200 个招聘人员。这是不可持续的。有 200 个人在 LinkedIn 上逐个发消息来获取这些人,因为没有品牌,没有信任。他们每个月在效果广告上花费数千万美元,Google Ads——
Lenny Rachitsky: 为了找专家、找人。
Garrett Lord: 找专家。
Lenny Rachitsky: 而且这个时候主要是专家了。
Garrett Lord: 对。然后他们把这些专家放到一种体验里,就像让他们在菲律宾给停车标志画边界框一样。顶尖的税务会计师不想被当作低成本国际劳动力来对待,我觉得没人喜欢那种体验。所以,能够构建一个根植于社区、根植于高质量培训的体验——如果你在 MIT 读博士,大概率你在如何使用这些工具方面就是没被教够。
不是你无法突破模型,而是其他平台花了数千小时获取一个用户,然后直接把他们扔进一个项目里,没有任何培训。所以我们从第一天起就构建这个专家……我们相信这里会有一个深度网络效应,与我们帮助人们启动或重启职业生涯的核心业务紧密相连。你进来,建一个个人资料,看到社区,有群组和动态流,展示大家是如何学习的。你进入一个实际的个人小组,组员和你相似、背景类似。你被教导如何交互,有试错的过程,我们有教学设计环节确保你能胜任。然后你被放到项目上——我们正在构建……有某些泳道,我们实际上在预生产数据并把数据卖给所有实验室。
所以我们可以做这样一件事:我们自己生产一个单位的数据,我们付费,几乎像电影制作一样。我们为一个单位的数据付费,然后确保它质量非常高。我们用自己做后训练,然后生成一堆数据规格说明,实际上把那个数据包卖给很多不同的实验室。所以你被放到那样的项目上。一旦你在我们的项目上做得非常非常好,通常我们就会把你放到客户项目上——他们只想要机器学习领域最顶尖的人才。然后他们从我们的项目转到他们的项目。所以这里有巨大的用户获取优势。你喜欢在播客里深入探讨,所以就说一下——真正重要的其实就是几件事。
用户获取与生命周期价值
Garrett Lord: 你有客户获取成本,即 CAC,然后还有 LTV,即用户的生命周期价值。在这门生意里,LTV 的计算相当简单,基于一个人的留存率,以及他能参与多少项目。所以,如果你把人对待得非常好,培训得非常好——首先,我们没有客户获取成本,因为我们与 1600 所大学建立了合作,覆盖了全美排名前 500 的学校中的 92%。我们几乎覆盖了全国所有的大学和社区学院。我们获取这些人才几乎没有客户获取成本。我们与他们之间建立了大量的品牌和信任积累,所以他们的转化率非常非常高。然后,如果你把他们对待得很好——因为他们对我们的期望,他们了解 Handshake,他们的学校购买的是 Handshake,我们关心善待这些人,但大学不会容忍我们的合作关系,除非我们善待了这些人。
所以,你把他们放进这个流程里,我们的 LTV、重复参与率和在不同项目上的留存率都非常高。因此,当你对比一下——一个领先的供应商有 200 个独立的个人招聘者,每月在效果营销上花费数千万美元——这些结构性优势就显得非常显著了。我想这就是我们取得这么大成功的原因。
Lenny Rachitsky: 这极其有趣。而且感觉就像你说的,过去人们的关注点很大程度上在通才上——世界上任何地方的低成本劳动力都能做这些工作,比如给东西画边界框。而本质上市场已经从低成本通才转向了专家。像 Scale 这样的公司,很多都在为通用工作模型的训练数据做优化,而你们则被定位为在基于专家的数据上做到极致。所以,你们在对的时间出现在了对的地点,拥有对的供给。多么好的一个生意。
Garrett Lord: 是的。
Lenny Rachitsky: 干得漂亮。
Garrett Lord: 我想说的是,在第一个业务里面构建第二个业务并不容易,但是——
Lenny Rachitsky: 是的。那让我顺着这条线聊下去。这也是我想聊的方向。具体是什么情况?所以,你开始注意到模型公司在找你们的人,发现这些人在这个领域的其他公司那里遇到了困难,然后你想,“哦,也许我们应该做这种事情”?最初的想法是怎么产生的,你又是怎么开始探索这个想法、判断它是否真的可行的?
从观察到行动:进入数据业务的起点
Garrett Lord: 具体来说,我们当时正在与许多中间商公司合作。我们开始看到需求,正如我之前谈到的。我们开始看到前沿实验室直接联系我们,试图绕过中间商,以获取更高质量的数据。当我们开始把这些线索串联起来——我们可以为我们的专家打造一个更好的体验,可以直接把他们对接给实验室,与实验室建立直接的客户关系,基本上就是砍掉中间商。同时为实验室提供更好的体验,为我们的专家提供更好的体验,也为我们网络中的一百万家公司提供更好的体验。
你可能也在思考技能提升和再培训,那个领域会发生什么。所以我们进入了这个领域。我们大概在十二月真正开始探索和深入了解,做专家电话访谈,深入钻研。我聘请了三家专家公司——AlphaSights 和 GLG,开始与最新的研究人员做大量电话访谈,因为我们有资源。作为一家较大公司的一个好处是,我们的核心业务已达 2 亿美元 ARR,所以我们有资源来加速这里的学习曲线。然后,大约五个月前,我们开始与可以说是排名第一的实验室合作。
Lenny Rachitsky: 我想知道那是谁。
Garrett Lord: 是啊。
Lenny Rachitsky: 是啊,想知道是谁。
Garrett Lord: 我们与排名第一的实验室合作了不同的项目,现在我们正在与前沿实验室合作,我们目前最关注的就是扩大规模。我们从四五个做这件事的人增长到了 75 人以上。我记得上周一就有 12 个人入职。我们在这个机会面前严重受限——因为在这个市场上,需求基本上是无限的。如果你能生产高质量的大量数据,你大概率能卖掉你生产的所有东西。所以对我们来说,我们真正关注的是确保选择正确的长期战略,确保不要增长太快以至于侵蚀了我们在这些前沿实验室中建立的信任。不过确实很有趣。
Lenny Rachitsky: 你说在现有业务中启动新业务也非常困难。最难的是什么?最难的部分是什么?你已经提到了其中一些因素,还有什么?
在现有业务中构建新业务的经验
Garrett Lord: 我觉得在这次经历中,我更多地追随了自己的直觉。Handshake 的故事是我们必须签约 1600 所大学,所以我必须学会成为最好的……我们是历史上增长最快的高等教育公司。所以我们签约了 1600 所学校。然后我们必须构建一个雇主业务,我们必须学会如何销售——所有这些财富 500 强公司都在用,其中 70% 是付费的——所以我必须学习向上销售,面向高盛、通用汽车、谷歌和世界上最大的公司,这与向大学销售完全不同。然后我们还必须学会构建一个出色的学生社交网络。最好的信息流应该是什么样的?群组消息应该是什么样的?所以我对这种从零到一的过程有一种熟悉感。
通常来说,市场平台就像很多个从零到一。有时候我开玩笑——实际上我不做梦,但我开玩笑说我真希望我们是一家网络安全公司,只有一个买家、一个产品,只需要……在市场平台里,你必须服务三个不同的边,你在 Airbnb 的时候应该深有体会。所以在 Handshake 启动这三个不同业务的过程中,我的一个心得是我非常亲力亲为。每个人都直接向我汇报。我在很多会议上真的说的是,“我不是要当老板,我只是想多一个聪明人在房间里。“我们招聘了一支令人难以置信的团队,他们在这个领域花了大量时间,曾是这个领域很多人力数据公司的重要领导者。
所以每个人都非常清楚地看到了我们拥有的结构性优势,而很多精力集中在确保我们能在扩展到其他客户之前,先把高质量的数据交付给一个客户。你必须对很多东西说不。然后,核心业务中也有很多人——这理所当然——存在着制衡,有很多人想参与进来。每个人都想说——不是每个人,这有点夸张——但说”不”很容易。很容易说,“我这周或这个月无法优先处理这件事,我有一组现有的优先事项。“所以基本上,除了少数几件事之外,每个人都直接进入了这个我建立的新组织,每个人都不再承担现有业务部分的责任。在新公司的每个领域里,谁是直接负责人非常清晰。现在我们与业务其他部分有了更深的耦合和整合点,但最初我们坐在办公室里一个独立的区域。
独立运营的节奏
Garrett Lord: 每个人每周五天在办公室,很多周末也在。在招聘人才方面也有完全不同的期望,就是”嘿,这是一份 24/7 的工作。这是一家早期公司。“薪酬也不一样,是基于新业务的阶段性目标来设定的,所以人们觉得自己是这家新公司的创造者、所有者。而且,现在依然非常敏捷,非常非常扁平。仅仅因为你负责某个职能,并不意味着你就是某个项目的直接负责人。我们会选择最有能力推动某个项目前进的人来担任 DRI,而不论他属于哪个职能。我们也更加以指标为导向。当初做 Handshake 的时候,我们很长一段时间都抗拒这种运营节奏——每周、每月、每季度的运营节奏。而 Handshake AI 从早期开始就更加注重以数据、指标和严谨性来运营。我们团队里有一位叫 Sahil 的同事在这方面做得非常出色。向 Sahil 致敬,向 Young 致敬,向 Paco 致敬。
Lenny Rachitsky: 好的,这太了不起了。所以,一家十年历史的公司内部能够成功做出这件事,关键要素是什么?顺便说一下,你的传统业务年收入已经达到两亿美元。而正如你所说,这个新业务第一年就要突破一亿美元。也就是说,如果继续按这个趋势发展,新业务头几年的收入就将超过你花了十年打造的业务的规模。太不可思议了。要让这一切成功,根据你刚才说的,我注意到了几个关键点:第一,很明显你就是以创始人模式在运作。你是这个新业务的领导者。你没有把它委托给别人,说”嘿,你去启动这个东西。“你配备了专职人员,“来,我们挑选一些人。你现在没有其他事情了,这是你的新工作。你就做这件事。“大家在办公室的不同区域工作。有一套基于指标的运营节奏——就是,让我们保持高度的严谨,关注进展如何、方向在哪里、我们的轨迹是什么、KPI 是什么之类的。你觉得还有什么对做成这件事特别重要的?因为我想很多公司都会尝试这么做,所以我很好奇你还发现了哪些关键因素。
Garrett Lord: 是的。我的意思是,我真的非常相信”分离”二字。独立的工程团队,独立的设计团队,独立的账户和运营团队,独立的财务团队。早期一切都分开。每个人只有一个工作,而且只有一个工作,那就是让 Handshake AI 成功。我们有几个更多的整合点,而且我有一支非常出色的核心业务高管团队在运营着核心业务,现在两者的联动越来越多。但那些长期打造 Handshake 的高管们负责运营核心业务,而我把 80% 以上的时间和精力专注在这件事上。我们招募了一位非常出色的工程负责人,比如 Avery,他……我们有很多创业者,那些之前创过业的人。或者说,抱歉,是之前自己创过业的人。这一点非常重要。我们在招聘人才方面有很多经验,专门招那些只在早期公司工作过的人,所以他们非常习惯模糊性和不确定性。
我们也在一开始就更加坦率地说明——这会很混乱。在全公司的大会上直接讲清楚这个叙事,跟团队直接沟通。我们有单独的大会,有单独的入职流程,有独立的招聘团队。我有一些连接点,但基本上是独立的。我认为这绝对至关重要。我们从核心业务中调走了一些顶尖人才——我们核心业务有很多优秀的人——我们对他们说:“抱歉,我知道你热爱你原来的团队,我知道你喜欢你在做的事情。你愿意加入 Handshake AI 吗?“他们完全放下了过去的职责,过来了。当业务开始快速扩张、快要撑不住的时候,这一点变得尤为关键——我们的增长如此之快,所以调来了一些最顶尖的高级工程师,他们非常有创业精神,还有一些 principal engineer 和 staff engineer,让他们空降进来。能够问核心业务中最有才华的人”嘿,你想过来做这件事吗?“这感觉太棒了。
有时候他们会说不。他们会说:“我不想大部分周末都在工作。“我们这个业务里凌晨两三点还在干活是家常便饭。确实很频繁。有些人不想做出这样的承诺,但我们从一开始就说得很清楚——这个团队的期望就是这样。节奏疯狂。如果你想成为硅谷增长最快的业务之一的一部分,你可以加入。主人翁意识也非常重要——对这个结果负责。我们有这样一个座右铭:不留任何遗憾。有一段时间我们在白板上画出一年的天数,就是那种——这样的时机再也不会有了。我从未见过这样的景象,我怀疑在商业中再也不会有这种感受了——需求是无限的,就看我们能不能抓住它去执行。
所以我们有这样一个座右铭:不留任何遗憾。怎么确保三个月不会变成六个月?不留遗憾。坐飞机去跟客户当面谈,做深夜的代码推送,把数据反复检查六遍,多上线一个有帮助的功能。而且,我们还有一种非常浓厚的庆祝文化。组织非常扁平,所以没有什么层级观念……有太多人在做出贡献,直接点名表扬那些正在立功的人,围绕影响力营造一个非常有趣的环境,我觉得这一点非常棒。
信任与市场愿景
Lenny Rachitsky: “不留任何遗憾”这一点,我想部分说明了信任的价值。只有当他们能信任你的数据是出色的、优质的、一致的,你才能赢。我能理解为什么这成为你们所构建的东西中如此重要的一部分。听你描述这些,我理解这显然是一个巨大的机会,你们显然拥有巨大的优势,但同时这种机会带来的压力也一定非常高——就是那种我们不能搞砸的感觉。
Garrett Lord: 没错。是的,Handshake 应该是一家……作为一家上市公司做到数十亿美元收入的企业,我们应该能够继续……我的意思是,这也在帮助我们的核心业务。我们看到的更长期的机会是——它是连接、是构建互联网上最好的招聘匹配市场。这可能是世界上最大的问题之一——劳动力供需匹配。人们把大部分时间和精力都花在工作上,人生中大量的时间都花在了工作上。找工作、投简历的过程将会被 AI 彻底重塑。我们一直在引领这一进程。一个 AI 面试官,收集技能信息,真正询问你的经历,做工作模拟体验,帮助雇主找到最佳候选人。我的意思是,我不知道你上一次做这件事是什么时候,但招聘经理审阅两百份简历的过程,你在开玩笑吗?
Garrett Lord: 我还会坐在那里审阅两百份简历?五年后不可能了。学生手动写求职信……不可能了。所以,必然会有一个招聘匹配市场胜出,把供给和需求、人才和机会连接起来。我们对这里能产生的影响力感到非常兴奋。这就是我的故事——我上的是社区大学,自己打工挣学费。我上的是密歇根上半岛一所不知名的学校。我在 Palantir 做实习生,那段经历彻底改变了我的人生。我创立 Handshake,是因为我想让任何人——不管你认识谁、你父母做什么、你上什么学校——都能更容易地找到好的机会。我认为 AI 在匹配上是一个阶跃式的提升。而我认为我们的人类数据业务,真正在为改善匹配奠定基础。
人类数据业务中我们正在做的很多事情,正在被整合到核心业务中。我认为这会改善雇主的招聘结果,长远来看能为他们节省数十亿美元。我也认为这会大大改善学生的体验。所以,我们必须抓住这个时刻。我们在核心业务和新业务中,仍然保持着体力、热情和激情去冲锋。这也是我们在内部不断传达的信息——是时候加把劲了。这是一生一次的机会,我们的位置非常好,我们作为一个团队,一定要抓住这个时刻。
Lenny Rachitsky: 确实如此。这真的非常像一生一次的机会。让我沿着这个方向再问几个问题,这也是我一直在思考的、很多人也在思考的问题——既然你在这里,趁这个机会聊聊。大家一直在问:我们会用完数据吗?模型会停止进步吗?我们会不会撞上某个平台期,实际上根本不会出现什么 AGI 时刻、ASI 时刻?所以首先,你认为我们会用完数据吗?会不会有某个时间点,我们就是无法再产生更多的知识和数据来喂这些模型了?与此相关,你认为推进模型更快、更远的最大瓶颈是什么?
数据枯竭与模型进步的瓶颈
Garrett Lord: 嗯,我的看法是,我们所需的数据类型会不断演变。它会是 CAD 文件,会是科学工具使用数据——当模型试图自动化科学发现和药物发现时。它会是在科学仪器上运行的各种冷门操作系统。所以我非常看好这条轨迹——把逐步指令遵循的能力拼接起来。我们所需的数据类型会发生很大的变化。而且我们甚至还没谈到多模态,以及视频、文本和音频。现在对音频数据的需求非常巨大。所以,数据的类型会不断演变。
Lenny Rachitsky: 对,我一直在用语音模式。那是我默认的 ChatGPT 体验,就是直接跟它对话——
Garrett Lord: 太棒了,真的太棒了。我刚刚——我太太周日刚生了宝宝,语音模式帮了大忙。我是说,每天晚上,每隔两小时,我就有更多的问题。语音模式真的太重要了。向语音模式致敬。嗯,所以数据的类型会发生很大的变化、很大的演变。我认为合成数据会有一席之地,在可验证的领域中尤其如此。但我们从各家公司的反馈中一致听到的是——合成数据不会占主导地位。在接下来的十年里,作为一家公司,紧跟前沿 AI 模型的发展轨迹,有数十亿、数十亿、数十亿美元的价值可以被提取出来。
Lenny Rachitsky: 首先,我要向你致敬——你太太几天前刚生了孩子,你还在经营这家疯狂增长的公司,还来做这期播客。真的非常感谢你。
Garrett Lord: 谢谢。
给年轻创业者的建议
Lenny Rachitsky: 当然。还有什么我们没有聊到的、你觉得可能对大家有帮助的内容?或者你故事中的某个部分,你觉得可能值得大家学习?或者有什么之前聊过的你想再强调一下的?在我们进入非常精彩的快问快答环节之前。
Garrett Lord: 我最想聊的、一直很有热情的话题,是鼓励人们创办公司并帮助他们做到这一点。我觉得在当下这个 AI 的时刻,对于年轻的创业者们——那些收听、阅读这期播客的人——因为我从 2020 年起就是读者了。我们查过的。
Lenny Rachitsky: 对,我们确实确认过了,太厉害了。
Garrett Lord: 对,长期的读者。我就是特别好奇,喜欢汲取——
Lenny Rachitsky: 感谢你。
Garrett Lord: ——你的访谈内容。但我想说的是,你只需要专注做一些有意义、真正能帮助人的事情。我认为在 AI 领域,会有非常多的机会去改善人们学习的方式。我一直很有热情要把 Handshake 打造成一个平台,不仅是一个出色的企业,也是一个真正能帮助解决重要的社会问题的平台。嗯,这就是我想在这里说的。如果有人想要关于如何做到这一点的建议,或者想联系我,我很乐意交流。
Lenny Rachitsky: 好的,所以这是一个关于在 AI 领域创业的建议分享的邀请,对吗?只是确认一下,让大家知道——
Garrett Lord: 对,那就太好了。
Lenny Rachitsky: 好的。我不知道你哪来时间应对成千上万找你的人,但我感谢你的这份慷慨。非常酷。快问快答之前还有什么要补充的吗?
Garrett Lord: 没有了。
快问快答
Lenny Rachitsky: 好的,那么 Garrett,我们到了非常精彩的快问快答环节。有五个问题。准备好了吗?
Garrett Lord: 准备好了。
Lenny Rachitsky: 你最常推荐给别人的两三本书是什么?
Garrett Lord: 我特别喜欢 Peter Thiel 的《从零到一》。我在创业时读了这本书,还看了 Peter Thiel 早年 在斯坦福教的创业课——那时候互联网上关于如何创业的内容远没有现在这么多。我就是觉得他太酷了。还很喜欢《鞋狗》(Shoe Dog),我觉得它就是创业的典范。当然还有《创业维艰》(The Hard Thing About Hard Things),不过这些书都挺常见的。
Lenny Rachitsky: 但也都是经典。Ben Horowitz 要来上播客了,聊《创业维艰》。
Garrett Lord: 太酷了。
Lenny Rachitsky: 对,《创业维艰》。好,你最近看过什么特别喜欢的电影或电视剧吗?我猜你可能没什么时间看这些——
Garrett Lord: 说出来可能会被喷,但我确实刚开始和我太太一起看《权力的游戏》——
Lenny Rachitsky: 第一次看。
Garrett Lord: 对。
Lenny Rachitsky: 好的,不错。
Garrett Lord: 所以我有好多要补的。
Lenny Rachitsky: 为什么会被喷……不,这很好。那些看过的人——
Garrett Lord: 我到目前为止非常喜欢的。
Lenny Rachitsky: 你到现在都很喜欢?好的。那剧挺血腥的,这是唯一的缺点。别在睡前看,不知道你已经看了多少血腥场景了。你最近发现了一款特别喜欢的产品吗?
Garrett Lord: SNOO。那个自动婴儿床 SNOO 真的帮了我们大忙。很喜欢,向 SNOO 团队致敬。
Lenny Rachitsky: 太棒了。我家也有一个 SNOO。不过我们从来没开过机,从头到尾就当个普通摇篮用了。
SNOO 与育儿(续)
Garrett Lord: 是的,大部分时间确实没开过,但有几次哭得厉害就打开了,确实帮了大忙。
人生信条
Lenny Rachitsky: 你有没有特别喜欢的座右铭,经常挂在嘴边或分享给别人的?
Garrett Lord: 我很喜欢那句话——不留遗憾,全力以赴。我在一个勤劳的家庭长大,爸爸为了养家糊口、让我们有好的生活,工作非常辛苦。就是那种拼尽全力的精神,不留任何侥幸。
普林斯顿泳池的故事
Lenny Rachitsky: 好的。最后一个问题,我在为这期播客做功课时研究了你的经历,有一个我特别喜欢的关于你早期拼搏的故事——你当时从一个校园跑到另一个校园,去推销让学校加入 Handshake。有个故事说你在普林斯顿的泳池里洗澡,就为了省钱,因为你根本没有地方住。这事儿是真的吗?能跟大家分享一下吗?
Garrett Lord: 是的,那段日子确实很艰难。我差点在普林斯顿被抓起来。对于我们这些到处跑的创业者来说,我们当时就睡在车里。我们有一辆福特福克斯,跑了大概两三万英里,晚上就停在麦当劳的停车场睡觉,因为那里灯亮,而且那时候 Wi-Fi 信号也好。与其花钱住酒店,想在会议前收拾一下自己的办法就是——每所大学都有泳池,而且泳池几乎总是开放的。我们从来没遇到过不开放的情况,它总是在早上对教职工、学生开放供人游泳健身。而每个泳池都有什么?有淋浴。
所以你可以去这个国家任何一所大学的泳池,免费洗个澡,让自己焕然一新。不过普林斯顿的校园保安对我这个非本校学生来洗澡这件事不太高兴。但我觉得这件事实际上反而帮了我们,因为普林斯顿的校园保安给我们正在推销的职业服务中心主任打了电话,问:“Garrett Lord 是谁?他真的是来给你们职业中心推销软件的吗?“这让跟职业中心的会议一开始就变得非常有活力和令人兴奋,因为对方会说:“你在我们泳池洗了澡?你开车过来的?” “对,我们从密歇根开过来的。” 所以我觉得这展现了一种承诺和投入的程度,让他们感到振奋。
Lenny Rachitsky: 可以想象,现在所有创始人都要开始用这个增长杠杆了——跟校园警察惹点麻烦,好跟学校领导约到更好的会面。太不可思议了。Garrett,这是一个疯狂的、了不起的、令人振奋的故事,就像你正在做的事情和这里的机会一样,以及它发展的速度和所有的优势。如果我是 Handshake 的投资人,我可能会说,“好吧,十年了,发展得不错。” 而现在就像,“哇,天哪。这是从哪冒出来的?” 太了不起了。而且这件事情本身也非常有意义。所以我很高兴你在目前这么忙碌的情况下还抽出了时间。最后两个问题:大家在哪里可以找到你,如果想联系你或者你们在招人的话?以及,听众怎样才能帮到你?
联系方式与招聘
Garrett Lord: 注册 Handshake 吧。如果你想给我发消息,那是最简单的方式。在 Handshake 上找 garrettlord 就行,你也可以在 Twitter 上找到我。我很喜欢 X,是 X 的重度用户。你也可以发邮件到 garrett@joinhandshake.com,注意两个 R,两个 T。至于怎么帮忙——我们现在正在大量招人。我们在纽约、旧金山、伦敦和柏林都有办公室。如果你有朋友对这个方向有热情,请告诉他们;或者你自己想了解更多,也请联系我们。我们很乐意聊聊。招聘是我们目前为满足需求需要解决的头号问题。所以如果你有才华,对 Handshake 感兴趣,想做我们的消费者产品,想做我们的企业端产品,解决有趣的 PLG 问题,或者打造最前沿的消费者社交体验——请联系我们。或者你想做我们的 AI 业务,我们也非常愿意跟你谈。
Lenny Rachitsky: 为了让大家更清楚,你们最需要招什么岗位?是所有岗位都招?还是主要是工程师?
Garrett Lord: 工程师。
Lenny Rachitsky: 工程师,好的。如果你是工程师,想加入当今全球增长最快的 AI 公司之一,机会来了。我们会在节目说明里附上招聘页面的链接。
Garrett Lord: 谢谢。
Lenny Rachitsky: 当然。Garrett,非常感谢你来做客。太精彩了。
Garrett Lord: 当然。
Lenny Rachitsky: 大家再见。
尾声
Lenny Rachitsky: 非常感谢大家的收听。如果你觉得这期节目有价值,可以在 Apple Podcasts、Spotify 或你喜欢的播客应用上订阅。也请考虑给我们评分或写评论,这真的能帮助更多听众发现这个播客。你可以在 Lennyspodcast.com 找到所有往期节目或了解更多关于本节目的信息。下期再见。
术语表
| 原文 | 中文 |
|---|---|
| ARR | ARR(年度经常性收入) |
| ASI | ASI(通用超级智能) |
| auto eval | 自动 eval |
| data labeling | 数据标注 |
| DRI | DRI(直接负责人) |
| expert network | 专家网络 |
| frontier AI model | 前沿 AI 模型 |
| gross margin | 毛利率 |
| instructional design | 教学设计 |
| KPI | KPI(关键绩效指标) |
| lightning round | 快问快答 |
| LLM | 大语言模型 |
| matching market | 匹配市场 |
| moat | 护城河 |
| multimodal | 多模态 |
| network effect | 网络效应 |
| post-training | 后训练 |
| pre-training | 预训练 |
| principal engineer | principal engineer(首席工程师) |
| product market fit | 产品市场契合点 |
| reinforcement learning | 强化学习 |
| rubric | rubric(评分标准) |
| SFT | SFT(监督微调) |
| staff engineer | staff engineer(资深工程师) |
| step function improvement | 阶跃式提升 |
| supervised fine-tuning | 监督微调 |
| swim lanes | 泳道 |
| synthetic data | 合成数据 |
| trajectory | 轨迹 |
| universal basic income | 全民基本收入 |
| verifiable domains | 可验证领域 |
此文档由 AI 分片翻译(translate_long_document)