走进 Devin:今年将为公司编写 50% 代码的 AI 工程师 | Scott Wu
Inside Devin: The AI engineer thats set to write 50% of its company’s code this year | Scott Wu
Using Devin Daily
Scott Wu: Our whole team is only like 15 engineers a year. We use a ton of Devin when we’re building Devin. Most folks on the team are definitely working with up to five Devins at once, and so Devin merges like several hundred pull requests into production in the Devin code bases every month.
Lenny Rachitsky: What percentage of your PRs are Devin versus humans right now?
The Unique AI Revolution
Scott Wu: It’s in the neighborhood of a quarter or so.
Future of the Engineer Role
Lenny Rachitsky: Where do you think this will be at the end of the year?
Scott Wu: Honestly, we expect it to be a decent bit more than half.
Introducing Today’s Guest
Lenny Rachitsky: You guys are so ahead of how companies work with AI engineers.
Scott Wu: AI is going to be the biggest technology shift of our lives, so most of the big tech revolutions that we’ve had over the last 50 years, like personal computer, and the internet, and the mobile phone, they all had this big hardware component that was a big part of the distribution. Folks who were building for those industries kind of saw their market grow and grow and grow basically steadily year over year as the number of people with mobile phones increased, right, as the number of people connected to the internet increase. One of the things which is already I’d say different in AI, is just how explosive the technology can be. There’s no weight on hardware distribution. It means that the space is just growing so exponentially.
The Interview Begins
Lenny Rachitsky: How is the act of being an engineer and building changing?
Origins of Devin
Scott Wu: I think there’s going to be way more programmers and way more engineers a few years from now. Pretty quickly. The form factor of what it means to be a programmer obviously is going to change, but at the end of the day, of course the discipline is all about just being able to tell your computer what’s do. And so in that lens, I really think that programming is only going to become more and more important as AI gets more powerful.
Devin’s Origin Story
Lenny Rachitsky: Today my guest is Scott Wu. Scott is the co-founder and CEO of Cognition, which makes a product called Devin, the world’s first autonomous AI software engineer. Unlike other AI tools that I’ve highlighted on this podcast, Devin is designed to act like an actual remote engineer that you chat with like you would with any other human engineer through Slack or through its dedicated website. When Devin launched about a year ago, it was very much a junior engineer. Over the past year, they’ve made a lot of progress and Devin is now being used by tons of companies in production. We chatted about how their engineering team of 15 uses Devins to build Devin, including how every engineer uses about five Devins each to help them code and move faster. How a quarter of their pull requests today are committed by Devins and that they expect this to be over 50% by the end of the year.
We also talk about how Scott imagines software engineering is going to look in the future and how the role of an engineer changes from a coder to an architect. We also get into the eight pivots that they went through before landing on this path, why Scott believes AI tools like this will lead to more engineer hiring versus less. Also where the name Devin comes from and so much more. This episode is going to blow your mind. I highly recommend you listen to it if you’re at all interested about where engineering, product building, and AI is going. A huge thank you to Claire Voue for suggesting a bunch of great questions for this conversation.
What makes Enterpret special is its ability to build and update customer specific AI models that provide the most granular and accurate insights into your business. Connect customer insights to revenue and operational data in your CRM or data warehouse to map the business impact of each customer need and prioritize confidently and empower your entire team to easily take action on use cases like win-loss analysis, critical bug detection, and identifying drivers of churn with Enterpret’s AI assistant Wisdom. Looking to automate your feedback loops and prioritize your roadmap with confidence like Notion, Canva, and Linear visit E-N-T-E-R-P-R-E-T.com/lenny to connect with the team and to get two free months when you sign up for an annual plan. This is a limited time offer. That’s enterpret.com/lenny. Many of you are building AI products, which is why I’m very excited to chat with Brandon Foo, Founder and CEO of Paragon. Hey Brandon.
Brandon Foo: Hey Lenny. Thanks for having me.
From Text Completion to Agents
Lenny Rachitsky: So integrations have become a big deal for AI products. Why is that?
Brandon Foo: Integrations are mission-critical for AI for two reasons. First, AI products need contacts from their customer’s business data such as Google Drive files, Slack messages or CRM records. Second, for AI products to automate work on behalf of users, AI agents need to be able to take action across these different third-party tools.
Lenny Rachitsky: So where does Paragon fit into all this?
Brandon Foo: Well, these integrations are a pain to build and that’s why Paragon provides an embedded platform that enables engineers to ship these product integrations in just days instead of months across every use case from RAG data ingestion to agentic actions.
Lenny Rachitsky: And I know from firsthand experience that maintenance is even harder than just building it for the first time.
Brandon Foo: Exactly. We believe product teams should focus engineering efforts and competitive advantages, not integrations. That’s why companies like You.com, AI21 and hundreds of others use Paragon to accelerate their integration strategy.
Lenny Rachitsky: If you want to avoid wasting months of engineering on integrations that your customers need, check out paragon@useparagon.com/lenny. Scott, thank you so much for being here and welcome to the podcast.
From Hacker House to Product
Scott Wu: Thanks so much for having me. Excited to be on.
Why Devin Is a Person
Lenny Rachitsky: I’m really excited to have you here because you are building and you’ve been building something that is very different from what a lot of other AI companies have been doing for a long time, although they are starting to converge to where you guys are now. We’re going to talk about that and it’s also just such a unique point in the history of AI and just the journey of AI. And so it’s really cool to be chatting right now. And I feel like we’re going to chat again in a few years and be like, wow, we were so right about so much and so wrong about so much.
Changes in Software Engineering
Scott Wu: Yeah.
PR Ratios and Future Outlook
Lenny Rachitsky: And so I’m excited to have you here. Let’s start with talking about Devin, giving people an understanding of just what the heck Devin is, is the main product that you guys build. What is the simplest way to understand what is Devin?
From Bricklayers to Architects
Scott Wu: Absolutely. And so Devin is a fully autonomous software engineer that is going to work on tasks end to end, and so there are a lot of great tools for all parts of the stack of the AI code workflow. What Devin does is it is a full asynchronous workflow, and so you can tag Devin on an issue in Slack, you’re talking about an issue and you tag Devin, you can tag Devin in Linear, you can have Devin and Devin will make pull requests in your GitHub, and so it’s very much built to work with engineering teams as your junior engineer.
The Jevons Paradox
Lenny Rachitsky: Amazing, okay. So I remember when you guys launched this, there was this big pitch of this is your new AI engineer and it was really good at a lot of stuff. It wasn’t great at other things. It’s been a year now about since you guys launched, is that right?
Scott Wu: Yeah, yeah.
How We Use Devin Daily
Lenny Rachitsky: What’s the best way to think about the level of seniority that engineer had back in the day when you guys launched and then the level of seniority of engineer today if that’s, I don’t know, measure of how to think about Devin?
Scott Wu: Yeah, and it’s crazy to think about by the way, because a year ago when we did the initial launch, I mean people didn’t really believe that an agent was possible. Right. And it was, I mean, it was a very different time. So like start of 2024, things with model capabilities were definitely quite a bit earlier on, reasoning especially was quite a bit earlier on. And yeah, I mean, in the time since then it’s obviously developed a lot. I think in terms of practical skills, there’s some comparisons we make. Sometimes we kind of say, well, when we got started it was kind of like a high school CS student and then as time went on, it became more of a college intern and now it’s like a junior engineer. But I would say though that those are more rough guidelines because I really like the phrase jagged intelligence for example, because obviously there are certain things that it is much better at than a human. There are certain things that it’s much worse at than a human.
And I think over the last year we’ve learned a lot especially about not just coding agents but agents in general just really building out how all of us should be working and interacting with agents as part of our flow. And so a lot of the things that we built, I mean, there was no Slack, there was no GitHub integration, there was no Linear, there was no interactive planning phase working back and forth. There was no way to touch up Devin’s code. And so a lot of the features that we’ve built on the product side since then have really been about basically yeah, figuring out how to make working with Devin and handing off tasks to Devin as smooth of an experience as possible.
More Devins Than Engineers
Lenny Rachitsky: That’s so interesting. So a lot of the work has gone not into how do we just make Devin the best possible engineer, but it’s how to work with this new type of entity that we haven’t ever worked with.
Scott Wu: I think it’s a 50/50 of both. I think the capabilities obviously have improved a ton and we’ve seen these get better and get measurably better. But I think the other side of it is everything to do with yeah, really the product interface and the tools and so on. And I think today folks generally know how to use chatbots and to with chatbots, right, and that’s an interface that people are familiar with and obviously with agents it’s still like a real curve, I think to learn how to use them and how to get the most out of them. And so it’s really exciting to see a lot of others starting to build and do a lot more in the agent space as well. But I think this is the kind of thing that we’re all really figuring out together as a space.
Takeoff Moments and Async Workflows
Lenny Rachitsky: What can you share about just the scale of Devin at this point, whatever you’re comfortable sharing, and then just where do you think the level of Devin’s coding abilities will be in a year?
Scott Wu: So we work with companies of all stages and sizes. On the smallest end, it goes to startups of just one or two people who are using Devin to build out a lot of their kind of initial prototype or initial product all the way up to big public companies, Fortune 100 companies or public banks or things like that who are using Devin across their engineering teams. In general, we’ve seen a huge range of the use cases there. And obviously the kinds of engineering work that you’re doing at a one or two person startup, they’re very different from the kind of work that you’re doing at a public bank.
But throughout it’s all been basically yeah, being that junior buddy of yours that makes you go faster and really multiplies you, I would say. I think it can multiply you as an engineer obviously by just letting you work with your own team of Devins instead of having to be kind of fully synchronous on a single task. And then it’s also kind of multiplying your team and multiplying your team’s knowledge base because Devin really accumulates a lot of the knowledge from working with every member of your team and is able to bring that into each new session.
The Live Demo Segment
Lenny Rachitsky: Awesome. We’re going to show people how it actually works later in the podcast. We’re going to do a few live demos, but let’s actually go to the beginning of the journey. What’s just the origin story of Devin? How did this all begin?
Live Demo of Devin
Scott Wu: The founding team, I mean most of us have known each other for years and years and years actually. And for almost everyone, this is our first time working together, but we’ve known each other a long time. And we all actually had our own kind of journeys in AI for the last decade or so. And so for myself, I ran a company before this called Lunchclub, which was an AI for a professional networking product and I ran that for about five years. And one of my co-founders, Steven was one of the first engineers at a company called Scale AI, which has obviously grown a lot and done very well. My other co-founder Walden, was an early engineer at a company called Cursor, which has also obviously grown a lot and done really well. And our whole team was kind of like that. Many of us knew each other from competitive programming and math competitions, but we had stayed very closely in touch in the decade since then and we’ve all kind of all had our own journeys.
And so we had one person who was running teams at Neuro, we had one person who was at Waymo, someone who had their own YC tools startup for machine learning, and we were really excited to build something together. And this was around late 2023, so about a year and a half ago at this point. And yeah, when we got started, I mean I think there were a couple of things that we felt really strongly about and one was that reinforcement learning was really working and was going to be the next big paradigm shift in capabilities. Back then it was the initial ChatGPT launch in 2022, and those models were, to first order were what we would call imitation learning in AI, right, which is basically you have the model read all the texts that you can find on the internet and then train it to talk like somebody on the internet would talk. Right. And there are kind of obviously a lot more details on top of that, but that’s kind of the first order pass of what was really done.
And it was amazing. Right. I mean, it passed the churn test, it was able to respond and to have encyclopedic knowledge about a lot of things. And I think this new paradigm which we’ve gotten into over this last year or year and a half is really high compute RL, which is a very different paradigm, right, which is basically the ability to go and do work on task and put something together and then be evaluated on whether that was correct or incorrect and use that knowledge to decide what to do and to learn from that. Right. And so we felt very strongly that that was going to happen. I think for us, code was the natural thing to work on for a couple reasons. One, because we’re all programmer nerds ourselves, and so teaching AI to code is about as cool as it gets for us, but also because code has this whole automated feedback loop, right, where you can run the code and that is the kind of automated feedback that really feeds into the RL, which makes these models so great at coding.
And then the other thing that we felt very strongly about was that the product experience was going to shift from what I’ll call text completion to agents basically. Right. And to first order, I would kind of say there’s been a lot of great experiences in text completion. It’s been used for marketing, it’s been used for customer support, it’s been used for education and in Coda obviously as well. The GitHub copilot was kind of really the dominant product of that initial wave. Right.
But I think that the big shift that we really felt we would see is moving from kind of this text to text model to an actual autonomous system that can make decisions, that can interact with the real world, that can take in feedback, that can iterate and take multiple steps to solve problems. And now we call that agents, but that was what we were really excited about at the time. So it was always coding, it was always agents. And in some ways that kind of feels like it should have been cleared from the start. But even with that, I feel like we’ve pivoted eight times or something within coding agents over the last year and a half, so.
Lenny Rachitsky: I just noticed recently all the AI, top AI companies sort of, not all, but many of them, the product that is winning is different, has a different name from the company, which is not typical, Cursors, Anysphere, Bolt, StackBlitz, you guys are Cognition Labs, like V0 is Vercel. And it just tells me these all emerge later in the company’s journey and they tried a bunch of stuff and like, oh wow, this thing worked and it’s so interesting that it’s so common amongst these top AI companies.
Devin Wiki and Codebase Understanding
Scott Wu: Yeah, and there’s even, I mean OpenAI, ChatGPT, Anthropic and Claude and Google. Yeah, it’s funny. Yeah, yeah, I agree. So when we got started, it wasn’t even really a company. I mean, it was more like a project or a hackathon almost. We got a bunch, we booked an Airbnb basically for a couple of weeks. This was around Thanksgiving time and just got a bunch of people together who were just excited to hack on some projects and build something cool. And it’s funny, actually the first thing that we were building for actually was more solving these more contest programming problems and using an agentic loop to really do better on that. And so obviously if you run your code on the test cases, you can evaluate, there’s a lot of agentic work that you can do there to try and do better, and that we spent some time on that initially. And then we’ve kind of gone from, I mean the story of the whole company for us in some sense has been going from hacker house to hacker house.
So after that we had another hacker house and that’s where kind of some of the initial ideas for Devin came and really building a software engineering agent and not just a coding agent and having to interact with a lot of these tools, but even then there were so many iterations. And even the idea of talking to Devin for example was like, it was something that we had to come up with. Right. Initially, it was just like you hand off a task and then it works and then it shows you this whole finished code. Right. And now obviously it’s like you can jump in at any time, you can get feedback on the plan, you guys can scope out the task together when you’re working with Devin. And a lot of these things we had to develop obviously, and certainly we’ve learned a lot about the use cases, the form factor. We’ve made a lot of big improvements and step function improvements on the capabilities and Devin’s ability to use tools and to bug and make decisions.
And so it’s yeah, it’s been a fun journey. I mean, I think I would say that the grounding question for us really, which is one that we think about all the time, is really just what is the future of software engineering and how should we be working with AI to write code? Because I think at the end of the day, of course that’s what underlines all of the product decisions that we make, so.
Codebase Scale and Architecture Understanding
Lenny Rachitsky: I like that you’re asking the juicy question I wanted to get to. Before I ask it just for the history books, when did you guys start kind of hacking around and when did Devin launch? How long was that period?
What Devin Excels At
Scott Wu: Yeah, so we started in November of 2023, which was then yeah, just like hackathon mode. We officially made it into a company around the start of 2024, and then our initial launch was in March. And so it was like nonstop, I mean it’s been nonstop for the entire last 17 months, but it’s getting to the launch and then obviously working with enterprises and developing the product a lot more, building it and getting it to work for a lot of practical use cases and then making it fully available self-serve in December of last year. And now we’ve rolled out 2.0 obviously just a few weeks ago. And so it’s been a very busy time for us.
Debates in Designing Devin
Lenny Rachitsky: Understatement of the century. Let me ask this question because you touched on it a bit, this whole idea of Devin as a person and this idea of creating a personality for Devin. It’s unlike any other, I believe, AI app. No one else has a name and you don’t think of it as a person. What made you guys decide to go that approach and just how do you design it to work well that way?
Scott Wu: I would say it’s a decision we’re pretty proud of, I would say. I mean, I think there’s a lot of different product experiences out there, and I think the thing that really makes Devin unique in what it does is that you can really hand off and more and more what we’ve seen honestly is that I think a lot of kind of explaining the Devin experience to folks is really just explain it as, yeah, this is your junior buddy. And that goes for a lot of the parts of the flow where in the onboarding for example, initially I would say we’ve definitely had a lot of users come in and just kind of see the blank screen and not really know, or they’d ask, “Hey, I’m going to do this whole big re-architecture, the whole code base.”
And basically what we’ve learned over time is to basically get folks to think more like whoa, whoa, whoa, let’s work on getting the repository set up first. Let’s make sure we hand Devin a couple one pointer tasks so it can get familiar with the code base. Let’s get it the thing. If Devin needs to be able to test the code or run the linter or CI or things like that. Obviously we want to make sure Devin’s got its own virtual machine set up to be able to do that.
And similarly, I think the usage pattern, I think often it wasn’t clear and obviously you can sit and just kind of watch Devin do it action by action and work that way, but we found that the best workflow really as a team building a lot of stuff was to work with multiple Devins and to run them asynchronously and to kick them off and to only jump in basically as you needed to provide feedback or steer the plan or anything like that. And so in many ways, I think Devin as a name really is our attempt to kind of capture the soul of that as a product where it really is treating it like a bit more of an autonomous entity that you can hand off tasks, that you can work with, that you should be teaching and learning with over time.
Competitive Landscape and Long-Term Positioning
Lenny Rachitsky: I want to come back to an area you started us down and then I took us away from which is impact on software engineering and how software engineering is going to change. So there’s kind of two parts to this. Just like when people are using Devin today, say this year, how is the act of being an engineer and building changing for those companies? What does that look like?
Scott Wu: By the way, we’re all software engineers ourselves. It’s like I’m a programmer by training and still a programmer at heart certainly. And I think the way that we’ve always thought about it is there’s layers of abstraction and there’s tools, and one way I would say it at a high level is kind of I think of AI in general as, yeah, I mean computers are obviously getting more and more intelligence and are able to do more and more and it’s possible there may come a day where computers truly do everything that we do and humans are not responsible for any of it.
I don’t expect that to come particularly soon, but I guess what I would say is until that point, for as long as we’re still part of the equation, one of the most important things to do obviously is for us as humans is to instruct our computers on what we want and what we want to build and what we want to do. Right. And software engineering is, we think of it today obviously as Python and C++ and JavaScript and all these things, but at the end of the day, of course the discipline is all about just being to tell your computer what to do. And so in that lens, I really think that programming is, if anything, is only going to become more and more important as AI gets more powerful. And I think the thing that’s really exciting for us is yeah, it’s really seeing that kind of iterative transformation. And so you ask what things look like today, and I would say, yeah, it really is like having a junior buddy or really a team of junior buddies that you can work with. Right.
And so every engineer on our team, we use a ton of Devin when we’re building Devin, and so Devin merges several hundred pull requests into production in the Devin code bases every month, which is, I mean, our whole team is only 15 engineers, and so it’s a pretty sizable fraction of all the code that we write. And the way that we use it is basically, yeah, everyone’s got their whole team of Devins. If you’re going to be looking through various issues, if you’re going through feature requests, if you’re going through bugs, if you’re going through new paradigms that you want to build, then it is naturally the case that there’s a lot of handoff points where you just say, “Hey, at Devin, here’s what’s going on. Can you please take a pass at this?” Right.
And sometimes Devin will be able to do the task 100% autonomously and just makes the PR and then you merge the PR and that’s great. Sometimes you want to be able to jump in for the 10 or 20% that really needs your help. Maybe there’s a few details with how exactly you want to scope it or how you’re architecting this feature, or maybe you want to go and test the front end at the end yourself to make sure it looks exactly the way that you want and give your one or two lines of feedback after that. Right. But a lot of it is really yeah, is kind of like yeah, learning to work with Devin to be able to just do more in parallel and build more.
Moats and Defensibility
Lenny Rachitsky: What percentage of your PRs are Devin versus humans right now?
Scott Wu: Yeah, I’d have to look, but it’s in the neighborhood of a quarter or so of all of our-
Technical Architecture and Models
Lenny Rachitsky: Wow.
Scott Wu: Yeah. Yeah.
Explosive Growth in AI Technology
Lenny Rachitsky: And then what was it like six months ago?
Scott Wu: Oh yeah, it’s grown a ton for, I mean, we’ve seen it grown exponentially internally ourselves as well. And so it’s kind of an interesting one where again, it’s always both the capabilities and the product interface. And so I think the intelligence has increased a lot. But the other thing of course is that we’ve spent a lot of time in figuring out how to build and to really kind of build for an interface where you can get Devin’s value on tasks where Devin is able to do the 80 or 90%. And so Devin is obviously not perfect and it’ll make mistakes and so on. And a lot of the question is basically, yeah, how do you scope out your initial task with Devin and then just kind of set Devin off and have it go and do the things that you want do? How do you come in at the end and review and give feedback? How do you make sure Devin learns over time? How are you able to kind of just check in as needed and course correct if you want to?
Enterprise Adoption of Devin
Lenny Rachitsky: Okay, so today about quarter of your PRs are Devins. Where do you think this will be at the end of the year? What would you guess?
Scott Wu: I think by the end of this year, we expect it to be more than half. And I mean, as time goes on, one of the things that we’ve seen is just you’re able to do more and more and more work asynchronously. Right. And you’re able to hand off more and more. I think the soul of programming, the soul of software engineering has really been about through all the areas, not just now, but even when it was assembly, right, and even when it was Pascal and even when it was punch cards or whatever, I think the soul of it has really been basically just about defining the problem that you’re facing and really thinking through exactly…
… about defining the problem that you’re facing and really thinking through exactly what is the solution that you want to build. Thinking through the architecture, thinking through the details, and really mapping out in your mind exactly what you want to build basically and what you want to have your computer do. I think that’s what makes software engineering really great and I think that’s the funnest part of software engineering.
I think, at the same time, that’s probably in the neighborhood of 10% of the average software engineer’s time, right? Because 90% of the time is you’ve got this Kubernetes error, then you’ve got to debug, and you have to see what went wrong and the system crash, or you left some port open and this is messing up, or there’s a bug report that you have to take care of, or you’ve got to migrate your code, or you got it upgraded to a new version or things like that. A lot more implementation.
One of the ways that we’ve kind of thought about Devin in building Devin is really allowing engineers to go from bricklayer to architect, so to speak. A lot of it is just getting to the point where you can do the high-level directing and you can basically specify things exactly how you want. I think it’s very much about still having the human in control and having the human able to do the full specification, but just multiplying the magnitude of what you can do and what you can build in one day or one hour or however long.
Future of Managing AI Engineers
Lenny Rachitsky: In the future, say someone is trying to get into software engineering, thinking about becoming an engineer, first of all, do you think people should… Classic question everyone’s getting these days. Should you still learn to code?
Bricklayers, Architects, and Managers
Scott Wu: Yeah.
Lenny Rachitsky: I’d love your perspective there. And then two, for people that are engineers today, what skills do you think will be more and more important and then less important in this discussion of moving from bricklayer to architect?
Counterintuitive Aspects of Startups
Scott Wu: Yeah, for sure. I love this question. First of all, the question of whether you should still learn to code, my answer would be absolutely yes. I think, to a large extent, when you take computer science classes and when you learn these fundamentals, sure you’re learning a little bit about how a particular language is, syntax works or something like that. But honestly, most of what you’re learning really is about the ability to logically break down problems for number one. And two, I would say is just the model of a computer and a lot of these decisions and a lot of the abstractions that we’ve built over time.
What is a database and how should you think about a database? What is a garbage collection system and how do those work and all of these different pieces? The reason I think that’s important is because it’s the same with a lot of these other… Arguably we’ve already gone through these phases in programming and I think this next one is going to be somewhat faster and somewhat bigger, but in many ways a similar flavor, which is when you work with Python today, obviously, a lot of things are already abstracted away from you.
In some sense, someone from 50 years ago might already call Python. You just get to explain in English what you want and now the computer does it for you. That’s great and I think it’s really powerful. It’s opened it up. I mean, we have far more programmers, obviously, than we ever have before because of that, but I would say certainly as you’re building your skills as an engineer, it really helps a lot to understand the abstractions and to be able to peel the layers beneath. Folks will use assembly for example if they’re really performance optimizing a piece of code, but also in order to build good systems and to understand these things, you certainly want to understand these abstractions of how does networking work.
What is TCP/IP like exactly or what happens with this Python code when it gets interpreted or all of these details. Similarly, I think we will get to a state where, with no experience at all, you’re going to be able to build some pretty cool stuff and to do some pretty amazing work just by explaining what it is that you want. But I think that, for quite some time, you really want to be able to think precisely about the details, to peel back the abstractions, to be very precise about what it is that you want to build and how.
Optimistic Outlook on AI
Lenny Rachitsky: And then for skills that you think are more and more valuable for engineers, where should engineers today be leaning more and more into versus like, “Forget this. I don’t need to think about this anymore”?
Scott Wu: For sure. I think architect… I mean, we already have a term for architect and engineering and I think it is directionally the right term. It’s, I think, one thing to just do a routine implementation and write boilerplate code and things like that. I would say that, in many ways, AI coding has already made us much faster at that, but I think a lot of the core questions of understanding very complex systems and working in the context of the whole company and thinking about the product that you’re building or the work that you’re doing and understanding, “Okay, what are the problems that we want to solve? How do we want to solve those problems? What is exactly the solution that we want to build? What are all of these key decisions and trade-offs that we’re going to be making?”
Basically, I think folks who are able to do that really, really well are just going to be able to leverage themselves more and more. If anything, I think there’s going to be way more programmers and way more engineers a few years from now than there are today. I think pretty quickly the form factor of what it means to be a programmer, obviously, is going to change. And in some sense, it already has, but I think there’s just going to be so much more for us to build. Folks talk about Jevons Paradox all the time. I mean, software is truly the shining example of Jevons Paradox, where we have always managed as a society to find more and more things that we want to build software for and build more code for. I really think there’s a lot more out there to do.
The Quick Fire Q&A
Lenny Rachitsky: For people that don’t know Jevons Paradox, can you briefly explain it?
My Favorite Products
Scott Wu: Absolutely. Yeah. Jevons Paradox just says that as the price of something goes down, it can still be the case that the total spend on it actually goes up. You can think about this with money, you can think about this with time or resources, but the direct version here is, I think, as it becomes easier and easier to program and as programming becomes more and more effective, I think we’re going to have a lot more programmers. I think in a kind of zero-sum view, you might say, “Well, we’re going to be 10 times faster at software engineering and it means that we’re going to need 10 times fewer software engineers.” But I think in practice, what really is going to happen is actually we’re going to build even more than 10 times as much code. And because all of the work that we do is so capped, obviously, on our ability to actually build and execute and iterate, we’re going to have so many great ideas out there, we’re going to have so many great products out there. People are going to build a lot more personalized experiences, for example, and there’s going to be a lot to do.
Lenny Rachitsky: Going back to the way you guys use Devin, you said that every engineer has this fleet of Devins. How many Devins per engineer do you find most people are working with these days at your company?
My Life Motto
Scott Wu: Yeah. It’s very asynchronous. Obviously, you can kick them up and start them up and shut them down basically as you see fit, but most folks on the team are often working with up to five Devins at once, I would say. It’s a nice flow where you think through, “All right, what are the five things that we want to get done today?” You have Devin one do number one, you have Devin two do number two, Devin three… For what it’s worth, I think it’s taken us some time to adjust to it and get to the point where it’s really intuitive for us, but I think it’s definitely a different experience where you’re handing off most things asynchronously.
The goal for each of your tasks is to be there for the parts that really need your expertise, either you really, really need to define exactly what it is that you’re solving for and what you’re building or maybe some of the more complex parts where you want to steer Devin towards, particularly what kinds of changes you want to make. “I want the class to be set up this way and we should go and change all the downstream references to this as well or whatever.” But basically having Devin do the bulk of the work asynchronously with you.
Origin of the Name Devin
Lenny Rachitsky: And then how many engineers do you guys have roughly?
Scott Wu: Yeah, our engineering team today is about 15 people.
Lenny Rachitsky: 15. 15.
Scott Wu: 15, yeah.
Lenny Rachitsky: Holy moly. Okay. And then each one has five-ish Devins.
Scott Wu: Yeah.
Lenny Rachitsky: So there’s five times the number of Devins as engineers. What I love about this is just a glimpse into where the future is going. You guys are so ahead of how companies work with AI engineers. Seeing how you operate is going to be, as a sense, essentially how most companies will end up operating.
Scott Wu: Yeah. For what it’s worth, we’ve already seen this shift, I would say, ourselves where… In terms of the team, obviously, folks don’t spend that much of their time just writing out boilerplate or just doing pure implementation of features. People get to spend much more of their time focused on really just thinking about the core questions of, “How do we make Devin better? What is the right interface for Devin? What is the right flow or the right set of features that’s really going to make this as great of an experience as possible?” That’s how we like things.
Lenny Rachitsky: When is the point you reach, where there’s takeoff of this being the by… Your Devin starts moving so much further ahead of everyone else. Once you have enough Devins doing all these things, they’re just like wherever and you’re 10 years, 20 years, 30 years, 100 years ahead.
Scott Wu: Honestly, as a community, I think all of us as engineers around the world, I think, are going to have to think about this and build for this and adapt to these new technologies. But what I would say is, I think more and more… Especially as capabilities get better, but certainly even in studies say today, I think more and more things are going to shift towards this asynchronous flow. And one of the reasons I would say for that is in the real world, you’re just capped by real world constraints. I think that one way to put it is kind of like… Don’t take these numbers exactly, but it’s kind of like the first order math of it is, of course, being able to write files or to complete this function or complete this line or things like that.
It helps a ton. It’s a really great experience. There’s a lot of parts of building software that obviously are almost not that at all, right? If you have a bug that you’re trying to fix and so you spin up the local server, you click around on your own product on the front end and try to reproduce the bug yourself. Once you have the error, you take a look at Datadog and you see what happened and you try to find other errors in the logs. You look at those files and you see what went wrong, you make some edits, maybe you go and rerun the whole process again now that you make sure your change looks right. That’s a lot of what it means to be a software engineer. These are processes that take real time. I think we’re going to shift more and more towards this agentic workflow, because that’s in some ways the way to really get to that 200%, 500%, 1000% gains that we’ll be getting to with software engineering over the next few years.
Lenny Rachitsky: Enough talk. Let’s show people what the heck this actually looks like. You’ve got a couple demos prepped that show a few use cases that you found helpful. You’re going to pull up your screen and then we’ll kick it off and then we’ll talk as it’s happening.
Scott Wu: Yeah. The whole process obviously of working with Devin is working asynchronously. I thought it’d be cool for us to actually just watch Devin a little bit in action and then we can go through some other examples of work that Devin’s done or things that Devin does for us, even on our team for example. But then we can check back in asynchronously with our Devin after.
Lenny Rachitsky: Let’s do it.
Scott Wu: I’ll share this real quick. The key thing that I would just emphasize here is a lot of it obviously is really just about thinking about as a software engineer or as engineers ourselves or engineering teams, PMs, and so on. What are the things that we would want to build that we would want to hand off? We have Devin set up with our own Devin code base, for example, and so I’ll go ahead and kick off with Devin for that. I’ll just say, “Devin, I’m on with my friend, Lenny.”
Lenny Rachitsky: Hi, Devin.
Scott Wu: “Can you modify Devin web app two?” Let’s feature your newsletter as part of the Devin website.
Lenny Rachitsky: Let’s do it. On the real Devin website.
Scott Wu: “Feature Lenny’s site.”
Lenny Rachitsky: Yeah, lose all your features.
Scott Wu: We’re going to kick this off. As you can see, Devin gets started instantly and goes ahead and responds. Again, you can work with this asynchronously, you can work with it synchronously as well. For this, we’ll just kind of go in a little bit and see exactly what’s going on. But as you can see here, Devin’s going through files and taking a look through a lot of stuff. We can follow here basically as we need to and see what makes sense. You can see Devin’s already called out a few particular pieces where there’s the sidebar, which we have implemented on the front end, and there’s pieces there. We’re going to have a new component and that component’s going to link to Lenny’s website. That all sounds good. Devin’s asking us any questions if there’s anything that we have here. Same story here where you can let Devin make its own decisions and hand off, or you can go ahead and give some more thoughts. Should the button open in a new tab or within an application? I’ll say let’s open it in a new tab.
Lenny Rachitsky: And you could answer these at any point. Is it waiting for the answer?
Scott Wu: You answer these at any point, you can hand off, hand back off.
Lenny Rachitsky: It’s not going to be like, “Just god damn it. I just wrote it this way. Why didn’t you tell me earlier?”
Scott Wu: That’s right. One of the big pieces with Devin is Devin will always be enthusiastic, will always be ready to put in the hours.
Lenny Rachitsky: Thanks for changing scope. Thanks, Scott.
Scott Wu: We’ll give Devin a chance to work and it’s going to go through these files and it’ll make a pull request for us and we’ll see and go from there. But I thought it’d be fun to show some other examples of Devin in action as well. One of the examples actually this morning, which I just used Devin for, is I asked Devin to help me brush my own facts up for this podcast. Obviously, a huge fan of the podcast and the newsletter. I asked Devin, “Hey, Devin. I’m to be on the podcast. Could you please research everything you can about him and make a nice website quiz for me so that I can make sure I know my facts?” This was just this morning, I asked Devin to do this and I’ll just show what Devin did, it looks like. Went to Wikipedia first. Unfortunately, it’s not a page in Wikipedia, which is… Lenny, we will work on that.
Lenny Rachitsky: I’m not a big enough deal yet.
Scott Wu: They did you dirty. I mean, we need a page list. And so then it went and found it on Spotify.
Lenny Rachitsky: So you’re watching what it’s researching live?
Scott Wu: Yeah, yeah, yeah. This was this morning, obviously.
Lenny Rachitsky: This is a playback of what Devin did. This is part of Devin you could just watch what it did.
Scott Wu: Yeah, yeah. Especially when you’re building engineering products or something like that, you can see each of the steps that Devin was doing, or if Devin tested the code locally, obviously you want to be able to go and look and see what Devin was cooking around with and testing or things like that. It found the newsletter. It’s going and looking at this and it’s going and reading all of this. And then it says, “Okay, let’s get started with putting the code together.” It says, “Hey, I’ve researched.” It’s going through and writing all of this, putting the app together. It plays its own quiz itself. Actually, we should just play this quiz actually. Let’s see how much I know.
Lenny Rachitsky: [inaudible 00:39:05]
Scott Wu: What is the name?
Lenny Rachitsky: What is the name of the podcast? Lenny’s Podcast. For people not watching, so to say, approximately how many subscribers? A million. Very good.
Scott Wu: Yeah. Yeah.
Lenny Rachitsky: What are three main topics Lenny is focused on? Oh, product, growth, and career. Very good. It’s a good quiz [inaudible 00:39:25] of people.
Scott Wu: What does [inaudible 00:39:26] besides podcasting?
Lenny Rachitsky: What does Lenny do besides podcasting?
Scott Wu: I’d say writing, investing, and advising. How often does Lenny-
Lenny Rachitsky: [inaudible 00:39:35]
Scott Wu: Once a week, right?
Lenny Rachitsky: Once a week.
Scott Wu: Yeah. We can go through all these and do all these. I took this quiz, by the way, obviously, to make sure that I was well-prepped, but this is kind of one of the more fun examples, obviously, of [inaudible 00:39:46]
Lenny Rachitsky: Scott, how many subscribers do I have at my newsletter?
Scott Wu: Over a million, actually.
Lenny Rachitsky: [inaudible 00:39:52]
Scott Wu: And then one last one I’ll show and then maybe we can come back to our initial run after. Like I was saying, a lot of this is really built to work with all of the existing code workflows out there. For example, we were doing some exploration with the DeepSeek repository on GitHub and we imported it into Devin and we got our own fork of it set up in Devin. A couple of things I just wanted to show here. One is Devin sets up its whole wiki with all of its internal understanding. When Devin indexes the code base, obviously, building a representation of the code base and learning it and improving it over time is one of the big things that Devin does. Funnily enough, we found that naturally humans really are interested to understand this code base representation as well.
Devin Wiki is something that we built here and you can take a look at all these different pieces and see each of these different things. Here are the FP8 operations, here’s an SG [inaudible 00:40:46] integration. There’s diagrams of how the different layers are built and put together. There’s deployment operations. There’s a lot of details about the architecture as well. You can ask questions about it as well. For example, you can say, “How does DeepSeek handle multi-token prediction designed for spec deck?” It’ll go through and it’s able to search through the entire code base and give you an informed answer based off of that.
We use this a lot. It helps when you’re scoping out a task for Devin and doing an initial prompt. It also helps, obviously just in a vacuum, you often have questions about your code base that are really nice.
Lenny Rachitsky:
With Attio, AI isn’t just a feature. It’s the foundation. You can do things like instantly prospect and route leads with research agents, get real-time insights from AI using customer conversations, and build powerful AI automations for your most complex workflows. Industry leaders like Flatfile, Replicate, and Modal are already experiencing what’s next for CRM. Go to A-T-T-I-O.com/lenny to get 15% off your first year. That’s A-T-T-I-O.com/lenny.
Something that I’ve learned as I’ve been talking to more and more AI building companies and apps is there’s a big difference in how large of a code base they could integrate into. That’s a big deal for companies that are existing versus startups, people that have large existing code base. How should people think about what kind of code base Devin can plug into?
Scott Wu: Yeah. We go all the way to the biggest code base as possible. One way I’d kind of put it is how… The way that we as engineers would think about a large code base is certainly when you’re making changes or when you’re thinking about a particular task. You’re not bringing in every single line of the code base at once. You have a high level of traction that you’re able to think about and look into and then you’re obviously able to zoom in and get to higher resolution on each of these different things. Devin works in much the same way, where the first thing it’ll do is it’s going to figure out the high level architecture of what’s going on here and what this is built for and so on. But within each of the components, it’s obviously also going to be able to zoom in and give some more detail about each of these. Here’s FP8 to be float 16 and how exactly a lot of that is set up. Here’s each of the different parts of the code base. Similarly, we built this to be scalable.
Lenny Rachitsky: It’s essentially coming back to the engineer as architect. Now, it’s helping you understand the architecture, kind of circling back to that.
Scott Wu: Yeah. Yeah, exactly. One of the fun use cases that we’ve seen actually with folks is they’ll often actually get Devin’s help to onboard new engineers on the team. When you’re new and you’re joining, there’s obviously a lot of questions that you have about the code base or about how things are set up. It also sometimes can be a little bit awkward to ask your mentor or your manager the questions and if you’re worried that they’re going to be really dumb questions. It’s nice to just be able to ask Devin and to go through Devin’s wiki and to understand these internal representations.
Lenny Rachitsky: I think that’s really interesting, because it comes back to your point that Devin is not just a junior engineer. It’s what you call a jagged engineer.
Scott Wu: A jagged intelligence.
Lenny Rachitsky: A jagged intelligence, where it’s almost like a staff engineer at understanding the code base. Usually, you have to ask an engineer that’s been there a long time, ” What does this do? Where is this thing? How does this work?” It feels like Devin’s very good at that.
Scott Wu: Yeah. Yeah. Obviously, the retrieval and processing a lot of code and a lot of tokens at once is something that language models are really great at. Basically being able to get those gains in the places that you need them is really great. Yeah. Sweet, cool.
Lenny Rachitsky: All right. You got a couple more use?
Scott Wu: Yeah. One last thought I’ll show is just… We just rolled this out last week actually, but it’s a full Devin automation setup with Linear. If you have tasks that you’re doing on the DeepSeek repository, for example, and it’s all set up, all you have to do is you just add the Devin label and Devin will come through and it’ll give you this. It’s going to give you its thoughts on what the tasks looks like and you can take a look at each of the particular files that you see, or it’ll point out snippets that it thinks are important. From there, if you feel good about what was built or the conclusions that we came to, then you can just start off the Devin session that will go and actually do that work.
Lenny Rachitsky: That is insane. That sounds like such a simple idea, but essentially what you’re saying is there are tasks in Linear that are fixes and features and now Devin just goes off and can just do them for you.
Scott Wu: Yeah. Definitely it’s a hands-on process. You certainly want to be involved when Devin is scoping out the task or giving you its thoughts. The nice thing, too, by the way, is Devin will give you its confidence level. Here’s how likely I am to really understand this piece or that piece or whatever, but it helps make things a lot faster. To your point, a lot of product managers, for example, obviously love to be able to use Devin in Linear to understand the code base better or things like that. Claire Vo, for example, from LaunchDarkly is a big Devin user and she loves basically going and scoping out tasks or asking data questions or asking, “Hey, what’s going on? Or is this merged into production yet? Or is this a feature flag right now? Or what percent of people are getting this or that feature?” It’s a clean way basically to make that intelligence much more accessible.
Lenny Rachitsky: I love, just with the integration with Linear, that you can still keep it really simple. You add a little ticket like, “Hey, this link to this home page, do this,” and Devin will be really good at understanding what you mean and then show you, “Here’s what I’m thinking.” Is this right?
Scott Wu: Yeah. Yeah, cool. Okay. Yeah, Devin did finish working. It seems like there’s something going on with the CI and it’s debugging that right now, but it went ahead and put up the initial first pass pull request and we can take a look.
Lenny Rachitsky: Let’s do it.
Scott Wu: This is the Devin website, obviously, in this custom deploy and we have Lenny’s newsletter right here.
Lenny Rachitsky: Let’s ship this to production. We won’t be so confused.
Scott Wu: Yeah.
Lenny Rachitsky: That’s amazing. Okay. Show it again real quick. Just added it to the home page of Devin.
Scott Wu: Yeah. Devin, obviously, has access to our Devin code base. It does a lot here and so it’s super familiar with all the pieces here.
Lenny Rachitsky: Beautiful.
Scott Wu: Yeah, I like how that looks. We’ve got Devin search, we got Devin [inaudible 00:47:33], and we’ve got Lenny’s newsletter.
Lenny Rachitsky: [inaudible 00:47:34]. You link to my site. We’ll get some PageRank going.
Scott Wu: Yeah, yeah, yeah.
Lenny Rachitsky: Okay. Is that a good example? Oh, there it is. What a beautiful website for my newsletter. Is that just a good example of the kind of thing Devin is very good at like, “Here’s a very specific thing to change on the website”? How does your people think about what Devin is very good at and maybe where it starts to fall apart?
Scott Wu: The way that we often describe it is, I think, Devin is best when it is working on tasks that are well-defined. One way to put it, you might…
On tasks that are well-defined. One way to put it is, you want to be giving Devin tasks, not problems. And a lot of these things like what you just saw, which was kind of like a quick front-end feature request or a bug fix or adding testing and documentation or things like that.
One of the things that makes a loop really nice obviously is a quick way to iterate and test. And so with something like this, obviously super easy for us, for example, to just go pull up the preview and see that the link worked. Obviously it would be easy for Devin to do as well. Devin will often go and log in to Devin and start a Devin session and make sure when it’s working on our code base, which is kind of hilarious. But yeah, you generally want something that is kind of easy to verify and easy to test is the main thing. And you can work on bigger projects or bigger asks as well, obviously. But in that case you should certainly expect to need to steer Devin more to make sure it’s going the right direction.
Lenny Rachitsky: It’s interesting because that’s very similar to the way people talk about synthetic data and reinforcement learning, creating data that’s very easy. There’s a very definitive answer, yes and no.
Scott Wu: Yeah.
Lenny Rachitsky: It’s very clear.
Scott Wu: Yeah.
Lenny Rachitsky: Okay. Let me ask you this question. What’s something that you guys debated a lot as you were designing and building Devin?
Scott Wu: I’ll give a couple that comes to mind. One I would say is a question of, I’ll call it how opinionated we should be. We had the workflows that we used to Devin for, which was very much as you can see for basically integrating to our Slack and GitHub making pull requests for us in our repos, responding to issue reports or things like that. And naturally we’ve had certainly a lot of other different things that have come up that folks have tried. I mean, we have folks who order their DoorDash with Devin, for example. Even we have folks, certainly a lot of people who are building cool websites from scratch or working on things like that.
Yeah, I mean it is been an interesting trade off for us where I think the way that I would describe it is in our product, certainly the large bulk of the features that we build are for this kind of making pull requests and engineering teams use case. But I think basically our kind of general stance with all the others is obviously if folks want to use Devin for that, that’s great and we want to just make sure that they’re fully aware about the limitations and about where things can get caught up.
It is funny with AI and especially because I would say one of, I would say the most common pieces of advice out there I would say is focus on a really niche cohort. Do things that don’t scale, make one use case that’s really great and then you grow from there. And I think that’s great advice across the board. But yeah, it’s kind of interesting because I think with generative AI, you naturally see this where a lot of product experiences can turn out to be more general. And so it’s an interesting trade-off for us. This is something that we still always go back and forth on and how much do we want to do more to support all the other kind of use cases out there to handle other things that folks might want to do with Devin.
I think another one that comes to mind is how much Devin should be, let’s say a single comprehensive project experience versus a suite of tools. And as you can see here, we have Devin search, we have Devin Wiki, we have the linear ticket scooping, and certainly these tools interact with each other, but I think as time has gone on, we’ve seen it more and more as really building this suite of tools. And I think the core agent experience and the core kind of agent that will go off and build each of these build things for you, for example, is always of course going to be that’s Devin, and that is the core piece. I think that will always be what’s really special about our work. But I think that all of the other features out there, there is a complex suite of work that’s required for real-world software sharing and engineering is just messy at the end of the day.
And so I think there are a lot of different flows and a lot of different use cases that makes sense. And an obvious thing to point out is you could ask the same questions to Devin search as you could to Devin and Devin will go through and it’ll do the same thing. It’ll go through and look through the files and give you an answer and stuff.
But with that said, on the one hand, I think on the capability side, there’s certainly a lot you can do to really optimize things for very specifically question answer about this repository and that made sense to really build into a specific kind feature.
And then on the other side, I would say we found that users actually really, really like having this access of control. Sometimes you have a task that you’re thinking about, but you actually don’t want Devin to get started on the task just yet. You want to ask Devin and understand what parts of the code base might be relevant. And so you want to be very direct about saying, “This is just an ask and I just want to see the snippets of the code base that relate,” or, “I just want to look at the Wiki and understand the existing representation.”
And so it’s on both the capabilities and on the UX side, we’ve found that that’s kind of what’s naturally made sense over time.
Lenny Rachitsky: Well, let’s talk about the landscape then of just other companies in the space, which is something a lot of people are always thinking about. There’s all these different approaches. You guys are going full on AI engineer, there’s obviously ID companies. There’s also just models being built that are really good at engineering. Everyone’s kind of starting to build agents now. You guys are ahead on this in a lot of ways. OpenAI just recently said they’re going to build a software engineering agent. Anthropics got something there. Cursor and Windsurf have their only agents and Replit. Thoughts on just where you guys fit in the landscape and then how do you think you win long term. How do you think about that?
Scott Wu: Yeah, and for what it’s worth, I think all of these are incredible teams. I think really smart and really forward-thinking folks who are building a lot of great products out there. And I think there’s a lot to do honestly over the next few years with the advent of AGI or whatever you want to call it. I think one of the quotes that I love is in 2017, if you asked if we had AGI, the answer is no. And in 2025, if you ask if we have AGI, the answer is, “Well, you have to define AGI. And it depends on your subject.” And I think it does get to the point of, I mean there is a lot of really amazing stuff happening. I think that it’s easy to underrate, I would say just how big of a shift it is that we’re seeing where I think there are a lot of great products out there, for example, over the last 10 years, 20 years, 30 years, that have made each of these different niches of the life cycle of building a product a little bit easier, for example, right?
There’s great products out there for instant response, there’s great products out there for logging, there’s great products out there for billing, there’s all of these different tools. And the obvious thing is what we’re seeing with AI is all of these spaces are going to be moving multiple times faster and it’s going to be like an order of magnitude shift, if anything.
And so I think from our perspective, we’ve obviously had a very specific lens that we’ve bet on this whole time, and that is autonomous coding agents. And there’s a lot of problems to solve there, to be honest, right? There’s still a ton to do on the core capabilities, certainly, and we see cases all the time where it’s like, wow, why did Devin make that decision? That seems, no human engineer would’ve ever done that. There’s all sorts of spots where, with the product interface, there’s obviously a lot to think about.
And I think it’s by the way, not just a single thing that we’re working towards, but something that will change with every edition of capabilities. I kind of think of it as there’s 20 generations of agent product, agent coding experiences to come. I think the one that we’ll get to over the course of several years is probably something where you don’t even look at the code at all, right? And you’re actually just looking at your own product and you’re just able to look and specify and say, “Hey, this button should be a little bit rounder. Let’s do that. And by the way, let’s add a new tab here and maybe we should save this information. Let’s start up a database table and let’s index it on X, Y, and Z columns.” And you’re just basically working with your products in real time and having your agent build out those things for you. Obviously there’s going to be a lot of generations in between here and there, but I think the product experience itself is going to change every single time.
And then obviously there, there’s all of the practicality of just getting it out there in the world. And so folks obviously need to learn how to use the new technology. There’s a lot to do to deploy into all of the messiness of real world software. There’s a lot of COBOL out there still. There’s a lot of FORTRAN out there still. There’s lots of kind of abstractions and details that folks have done.
And so I think from our perspective, we have since the beginning have been laser-focused on agentic coding, and that is the one thing that we’ve really believed in. It’s the one thing that we’ve designed for and that goes all the way to even the revenue model with ACUs and having the usage-based setup. It does into obviously all the product experiences of thinking how, okay, where do you want to talk to Devin? You want to be able to talk to Devin in Slack, you want to be able to spin this up from your issue. You want to be able to all of these things and then of course the capabilities.
And so I don’t think there’s any one easy answer. I think it’s obviously a combination of things, but this has been the space that we’ve lived in and spent all of our time in for the last year and a half, and it’s going to be that way for the next five or 10 years too.
Lenny Rachitsky: Along these lines, a big question everyone always has in AI’s moats and defensibility, it’s a question I’ve been asking every founder that comes on. How do you just think about how to build a moat in this space when it’s so much easier to build and so much is built on these models that are themselves advancing so quickly?
Scott Wu: I’d give one slight tweak on that, which is I think it’s often less about moats and more about stickiness. And what I mean by that is moats are in some sense, typically what folks mean by moats is something that means that a competitor couldn’t even enter the market. And I agree that at a high level, a lot of different folks at different layers of the AI spectrum, the foundation labs or the application layer or so on, I don’t think there’s any kind of hard barrier that would prevent others from entering. I think what does exist is stickiness, which I would kind of define as once you have a product experience that you really like, are you excited to keep using that experience or is there an effect where it is just as easy from now on to just switch on to a new one and learn a new one and so on.
And I think from that perspective, I think there’s a few things that are really great about coding agents in particular. One I would say is there is a lot of just inherent stickiness and learning and buildup over time, which is that as you use Devin and as your whole team uses Devin, it’s the same thing with an engineer. If you’re joining on day one versus you’ve been at the company for five years, you wrote half the code yourself, you’ve touched every file you’ve built every single piece, you know all the engineers. And so similarly, it’s like Devin will really learn and build its representation of your code base and of your stack and of your process over time and will be able to do a lot more with that.
And then the other piece of it, which I think is really exciting I’d say is there really is a lot to do of what I would call a multiplayer aspect of code, which if you think about it, is how a lot of things get done in the real world, certainly. And so it’s one thing to have your own experience, which you use yourself as just an engineer, but for example, ourselves, we see this all the time where some engineers are working with Devin and teaching Devin things and as I mentioned, folks will have Devin on board their new engineers and kind of convey that knowledge to them.
Or similarly, it’s like I’ll start a session with Devin in Slack and I’ll say, “Hey, it’d be cool if we could do this thing.” And some other engineer will chime in and say, “Oh, by the way, the reason we did it initially was X and Y.” And so Devin, just make sure when you do this change that you still support that workflow. And Devin will say, “Okay, sounds great.” Or Devin will make a PR, I’ll be working with Devin, we’ll make pull requests and GitHub and somebody else will be reviewing that PR or give some comments and Devin will work on that too. You’ll be in linear.
So all these kind of spaces, it really does just kind of set up for an experience basically where Devin can just grow in the value that it can provide for your whole work over time. And so I think from that perspective, if anything, we want there to be a lot of innovation and a lot of new products and so on. I don’t think that the goal is to try to lock other people out of building. There’s a lot of stuff to build, and I think there’s going to be a lot of different experiences. I think from our perspective, what we think about is more like how can we make Devin more and more and more useful as you’re using it more?
Lenny Rachitsky: It’s very similar. We had Michael from Cursor, the CF Cursor on the podcast, and he had a similar point of just he thinks moats are just kind of like consumer, like Google is the way, he thinks it’s like Google where people can easily switch. You just have to stay the best, and that’s the answer. And it feels like you’re adding to that of just like, but also if you can create some stickiness where it is very hard to leave because it’s so good at what it’s doing and it’s built knowledge and integrated to your workflows and builds on that on that stickiness.
Scott Wu: Yeah, and I think one of the things that’s nice about our space too is, software engineering for better for worse has a very clear tie to value. And what it means is, I guess one way to put it is there is always kind of a clear next level, at least for the next while. I think there could be some point where you’re just like, “All right, just build the entirety of YouTube for me.” And Devin just does the whole, it’s like there’s probably been a hundred million hours of human engineering time building YouTube, building the algorithm, building all the infrastructure, all of the, everything, every little detail. And maybe there’s some time where Devin just does that out of the box. That’s obviously going to be a long time from now.
I think on the interim, on every level in between, obviously it makes a difference the quality of software engineering. And I think one of the cool things with developers obviously is developers are really willing to learn new experiences and to put in effort if it means that they’re able to have a higher and higher quality experience.
Lenny Rachitsky: Awesome. I’m going to spend a little time on the tech that enables Devin. Without divulging trade secrets, just what allowed you to make Devin so good? Was there an unlock with a certain model? Some folks have shared three points on a 3.5 was a huge unlock for a lot of their products. Just what’s kind of the key to the way you’ve architected or built Devin that makes it work so well?
Scott Wu: We obviously, we’ve been betting on agents for a long time. I think that agents were doable and workable a lot earlier than most folks might’ve thought. But certainly I think as the community has really rallied around it, I mean you see the impacts of that in the pre-training, you see the impacts of that in a lot of the work that’s done with these models. I actually don’t think there’s been any, from our perspective, I don’t think there’s been any single step function based model shift or anything that has been kind of like a night and day difference in Devin, but I certainly think that the curve of every point on the chart, I mean there’s a new model that comes out every week now has obviously made a big difference in terms of what we’ve been able to do. And then obviously on top of that, we work with the research teams at all these foundation labs to do a lot of our work on top.
And so I think that my hot take here at which I would give is, I think in terms of base intelligence, we’re honestly basically already there. And I think a lot of what we see actually and what we spend our time on is less so, obviously, we don’t our own models or things like that. It’s less so increasing the base IQ of a model, for example, and more about teaching it all of the idiosyncrasies of real-world engineering and thinking about here’s how you use Datadog and do this, and here’s how you might diagnose this error and here are the different things that you could run into and here’s how you handle each of those. And when you’re ready, here’s how you make GitHub PR.
And this is true in engineering. It’s true in every other space as well. I mean, there’s so much detail and idiosyncrasy to the work that we all do obviously day to day. And a lot of it is kind of like teaching the model to mirror the complexity of the real world, I would say, rather than getting it to some higher fundamental level of problem solving, which I think the foundation labs are doing a really great job.
Lenny Rachitsky: There’s something you shared when we were chatting before we started recording around the growth of previous transformative technologies were very hardware oriented and there was a limiting factor to their growth and AI is not that. Can you just share that insight?
Scott Wu: For a number of reasons? I think AI is going to be the biggest technology shift of our lives, but I think one thing, which is what we were just talking about before this, which is most of the big tech revolutions that we’ve had over the last 50 years, I mean, I’m thinking about personal computer and the internet and the mobile phone and stuff, they all had this big hardware component that was a big part of the distribution. And so you had the internet, and initially it was just these universities that were talking with one another, but obviously over time we got the whole world plugged into the internet and it took years and years and years. Same thing was true with mobile phones. Same thing was true with PC.
And the thing that’s interesting about that in particular, which is I would say we’re already seeing the effects of that, is in these hardware distribution machines, obviously there’s a lot that depends on real time. And so folks who were building for those industries saw their market grow and grow and grow basically steadily year over year as the number of people with mobile phones increased, as the number of people connected to the internet increased. And many of those businesses, it’s still crazy to think, but many of those businesses got started right in the beginning. I mean, Apple and Microsoft were started right around the same time. And same is true for a lot of the great internet businesses or wherever, but certainly it was something that touched whole world with time or a large fraction of the whole world. And it had a really massive impact, but it took place over several years because of the time that it took. And I think one of the things which is already I’d say different in AI, is just how explosive the technology can be. Once AI could, and I think we’re firmly past the inflection point in AI code where it’s, as an engineer, if you’re not using AI at all to write code, I mean you’re falling behind honestly. And it is a technology that everyone should have and should be using, and there’s no kind of weight on hardware distribution that is causing that. It means that the space is just growing so exponentially, basically.
Lenny Rachitsky: Michael Ballin has this interesting point that cliches are cliches because they’re so true and that’s why they’re cliches. I heard that a million times and I think it’s like people hear this like, “Yeah, yeah, I know,” but it’s actually insane what is happening. That’s why you’re here to help us through this transition.
Scott Wu: Yeah, no, I mean, it is a fun time and I think there will be real investment and real work that it takes. But I think from the perspective of us as engineers, for example, I think it just means it’s so important to stay in the loop with everything that’s happening. And as we’re seeing it’s not only because of your learning and your ability to work these technologies, but it’s also about basically teaching the AI what there is to know about your code base in order to make it really effective at building with you and doing more of the things that you would want it to do.
Lenny Rachitsky: So along those lines, for people listening that they’re like, “Hey, we should be using Devin at our company,” what are things you’ve found to be helpful in helping an engineer at a company get adoption and be able to use Devin either culturally or logistically?
Scott Wu: So a pattern which we often see with folks is there will be a few folks at the team who are really excited and want to try out the new thing, and they want to put in the investment and are really excited to get it going. And they’ll go through all the setup. They’ll give Devin the repos, they’ll teach Devin how to run the lint and the CI and all of those details. And they’ll start off by giving it those initial tasks and help Devin build a foothold basically. And as time goes on, eventually folks will see, “Wow, Devin’s writing all these PRs, Devin’s doing this [inaudible 01:08:14].
Lenny Rachitsky: Who’s this Devin person that just joined the company’s just knocking out PRs?
Scott Wu: Yeah. And they’ll see that, and then naturally they’ll get on and they’ll get an account. And one of the cool things of course is by the time they join, Devin already knows a good amount of detail about the repositories that they’re already working in and they’re working with that. And so one of the really cool things which we often see is that the early adopters themselves can really pave the way I think, for everyone else on the team.
But yeah, I think the main thing I would just kind of call out is it really does take, it’s a very different product experience, and I think for what it’s worth, I think there’s still a lot more that we can do to make it as intuitive and as clear as possible to folks like how to use Devin and what the right steps are and how to really maximize value out of Devin. But I think that, yeah, it’s the kind of thing where if you put in the investment and understand exactly what it takes to get Devin to be successful, we’ve found ourselves that as time has gone on, we just use Devin more and more and more with every next update.
Lenny Rachitsky: So let me follow that thread. There is a question I ask every AI app building founder, which is, if you could sit next to every new user of Devin and whisper something in their ear to help them be successful with Devin, one or two tips, what would those tips be?
Scott Wu: I think the biggest thing I would say is it really is just treat Devin like your new junior engineer. And I think that’s the biggest thing. I think folks come in and they see the blank page and they think of all sorts of various things that they want to try out. They think of lots, where I think typically the flow that we see that works best is obviously you can try demos and you can do things, but a lot of it is just like, “Yeah, let’s figure out what tickets we want to get done today or this week and let’s have Devin get started on those and let’s start with the easier ones and then work with Devin and understand what things Devin needs to get set up to be able to test its own code and do this well. And then let’s scale up over time.” And then obviously as you work with your engineer, you understand better how to communicate with them or what are the right tasks or projects to bring them in on. But I think that really is the one-liner for us.
Lenny Rachitsky: Okay. There’s a question I’d be meeting task. I just want to get back to this because it’s something I think a lot about with Devins. Everyone’s going to have five Devins, let’s say 10 Devins. Everyone’s kind of turning into basically an engine manager with a bunch of junior engineers, which isn’t necessarily the best job in the world because it’s just a bunch of, at least you don’t have to do performance reviews and one-on-ones, but it’s sitting around, checking a lot of PRs all day. There’s a sense of you become an architect, which is kind of what every engineer wants to become eventually, right? They’re all, “I just want to think about the architecture. I don’t want to code all these stupid, fix bugs.”
So I get that there’s a good side to that, but just I imagine you’re thinking a lot about this, just like how do you make life pleasant and fun and enjoyable as basically an engine manager of say, 500 Devins in the future?
Scott Wu: Yeah, I can just imagine the performance, “Devin, you’ve done a really great job on your task, but I really would like you to be more proactive in the team meetings.” So what I’d say-
Lenny Rachitsky: [inaudible 01:11:29].
Scott Wu: It’s funny actually because something that in terms of the wording that we thought a lot about as well is just, we’ve used the term manager of Devins in the past, which of course I think is a big part of it. But I think that the only thing I would point out here is I think that the bricklayer versus architect is closer to the experience than being a manager. Because I think a lot of the difficulty of management or the reason that people shy away from it is more because of a lot of the various.
… [inaudible 01:12:00] because of a lot of the various, let’s say… There’s all of the context, and the ownership, and the responsibility and stuff, and then there’s also all of the emotional aspects of it. Where, I think working with Devin is a little more like just being, more as having an interface to hand off tasks and build tasks. And so, the parallel that I would draw is when we invented Python, obviously… It’s like, in many ways the description and the outlining of tasks, obviously it was a different paradigm, but I think certainly it was nowhere near what folks typically think of as management bureaucracy today.
And I think that with Devin, a lot of it is just like, it’s more like finding the right levels of abstraction that you could work with Devin on, and just finding the workflows that work really well, and the obvious thing to say here is, it’s like you can always have Devin take a first pass at things. And so you have Devin take the first pass, if it’s great, you merge it right away, if it needs some touch up, you can obviously give that feedback, for example, but a lot of it is like, it’s more about, basically making Devin part of your flow than it losing control, which I think is the main thing that folks are scared of with management.
Lenny Rachitsky: Are you thinking about a manager Devin? Like a Devin that manages other Devins?
Scott Wu: Yeah, yeah. So, for what it’s worth, Devin can start other Devins through the API, right? And so, we’ve seen this happen quite a bit of times, where naturally if you have some big tasks that you want to do, Devin will do this all the time, it’ll chunk up and then [inaudible 01:13:53] into smaller Devins. And so, it’s the kind of thing that you need to give Devin the credentials to be able to do that, it’s not currently something that is default enabled, but I can certainly imagine as time goes on that there’s more and more of that
Lenny Rachitsky: Devins all the way down.
Scott Wu: Yeah, yeah. I think the thing that’s kind interesting too is, with humans, the way I almost say it, in technical terms, it’s like there’s this coupling of a context and a thread, and what I mean by that is basically, each human can only operate a single threaded on the work that they do, and they have their set of context, and then there’s other humans obviously who can do other stuff at the same time, but they have their own context. With agents, one of the cool things is you can have an agent that’s doing multiple lines of exploration at once, but is sharing all of the context of everything that they find, and so I think that this is very early, and I think we’ll see this, but folks obviously love to talk about basically systems of agents communicating with one another, and I think that there will be a lot of new paradigms to build for, once we get there.
Lenny Rachitsky: And it’s so interesting what you said about the decision between having one Devin and only one Devin do all the things, and you just tell them things and they fire off jobs versus you have five Devins, and they’re each doing individual things, it’s such an interesting decision to make.
Scott Wu: Yeah, yeah, yeah, for sure.
Lenny Rachitsky: Okay, two more questions. Maybe the most counterintuitive thing you’ve learned so far building Devin, that maybe goes against startup wisdom, common startup wisdom?
Scott Wu: Something I’ve thought about a lot lately as we’ve built this is, this is not my first company, actually for a lot of us, it’s not our first company, I think of our 26 or 27 people total on the team, I think 18 of us have started our own company before this. And yeah, one of the things I think about is, there’s actually your point about cliches I think really spoke to me as well, which is, there’s the really common things which you hear all the time in startups, where you’re like, you got to move fast, or you got to hire great people, it’s like, okay, well, obviously you do, I wasn’t planning on not hiring great people, I wasn’t planning on going slow. And similarly it’s like, yeah, you really got to build something that people want. And there’s these three to five things which are always repeated, and they’re always the common wisdom in startups.
And I definitely had this idea as a founder, when I was starting initially, that, all right, so those are the three to five basic things, but as you get really deep into it, you spend a lot of years into it, you learn all of the thousands of other things that you have to learn to build a company. And I think to some extent that’s, of course, true, and there’s lots of little details that you’ll get into with all these different things, including team building, and product, and strategy, and engineering decision-making, and fundraising, and sales, and every other component. But I also realized that as time has gone on, more and more, I felt like building companies well sometimes just comes down to doing those three to five things just even more than you could possibly expect.
And so, with us, it’s like… And everyone says we go fast, but it’s like, yeah, we had a hackathon in November, we had another hackathon in December, we started the company officially in January, we got the prototypes out to initial users in February, we did a launch in March, we got our first customers in April… It’s just like, basically truly pushing the pace in every spot where we possibly could has really made a difference for us. And similarly, it’s like, yeah, everyone always says you should hire great people, but I think that the truth within that truth is basically you should fight to all ends basically, to get the folks that you really want to bring in. And one of my favorite stories to share is we had a candidate who came and interviewed, he was a junior at MIT, so he was very, very young, and we gave him an interview, and he did way better than almost any of the full-time candidates that we had ever talked to. And so, we said, hey, what do you think about taking some time off of school and working with us and building out Devin?
We really think you’re just going to be able to come in and just have a ton of impact already from day one. And he thought about it for a while, and he came back and he said, you know what? I’m down, I want to do it, but my parents really want me to graduate from school, and I’m just not sure there’s a way to make it work. And so, we talked him more and just understood the situation, and then we flew to North Carolina, went straight from the airport to his parents’ house, had dinner with him and his parents, we talked a lot, a really nice Gujarati family, we gave them some gifts and just talked to them about it, and try to understand what would it take, and what would we need to make work. And they just said, it sounds like a great opportunity, but we really want our son to be able to graduate.
And we talked that through and we figured out a setup basically, where he could work for us essentially full time, but then come in for his required classes, and do what he needed to do to get the diploma basically, but no more than that. And we talked that through, and then we got to a point where everyone was happy with that, and then went straight back to the airport and flew right back, basically. And that was the first and only time that I’ve ever been in North Carolina, and it was just a great trip. And it is the kind of thing where it’s like, hiring great people is one thing, but truly just never giving up, and really giving it everything that you can to make it work for people who really makes sense to be on the team. And he’s been with us on the team for over a year now, and he’s been an incredible, incredible engineer, and we wouldn’t be here without him.
And similarly, we had someone else who was again, really, really talented candidate, did amazingly well, very young, and had a lot of great offers at a lot of other companies, and we were talking to them about, he wanted to start his own company someday as well, and we were talking to him about certainly a lot of the obvious things, which are having him meet our investors, or get to do work with customers, or see a lot of these other components so that when the time came that he would have all the experience he needed to start his own company. But one of the other things that was big is he was talking with a lot of great companies already, he didn’t want to burn any bridges, and so we actually worked with him and basically hand wrote all of his rejection responses to each of the other companies, and worked with him on it to say, here’s how you should say it in a way that’s going to come off as that you really did appreciate the time with them, and that obviously you want to remain close with them and stay in touch. And it was the kind of thing obviously where it’s like, look, obviously our job is to make sure that he’s happy enough that he doesn’t want to leave at any time in the near future, but I think it’s the kind of thing where the way that you put together a really, really great team is by fighting for what’s right for them too.
Lenny Rachitsky: Wow, those are incredible stories. And it makes so real these, as you say, cliches, hire the best people, this is what it sounds like to hire the best people. This is what it takes.
Scott Wu: Yeah, no, and I was just saying, a lot of things we’ve fought very hard to just reimagine things from the ground up because a lot of it really is just thinking about where do we think the technology is going over the next five, 10 years, and what is the place that we want to have in that future?
Lenny Rachitsky: I wonder if people are going to be fighting for the best Devin someday. They’re just going to [inaudible 01:21:19] Devins.
Scott Wu: Yeah.
Lenny Rachitsky: They’re [inaudible 01:21:22] so smart.
Scott Wu: I’ll give you overtime pay, benefits, free healthcare, and everything, and then the Devin’s like…
Lenny Rachitsky: Three Devins like Magic: The Gathering cards.
Scott Wu: Yeah.
Lenny Rachitsky: And then just going back to your three to five things. So, essentially this is incredible advice, essentially, it’s like you always hear hire the best people, move fast, build things people want.
Scott Wu: Yeah, build something people want, stay as close as possible to your customers, and then I think the other thing is just always think about where things are going, not where they are today. I feel like those are the five things, which is… Especially in AI, with things moving so fast, and there’s so much great talent, I feel like a lot of these are even more true, where it’s like it’s not just thinking about where things are going to be in 10 years, it’s like thinking about what’s going to happen next week. And obviously, things are moving very quickly, and it is very hard to predict, but you really have to be very rigorous with yourself, I’d say, about thinking through those things and evaluating all of the decisions that you make in that lens.
Lenny Rachitsky: And staying focused is the big takeaway to me here, is it ends up feeling like there’s 1000 things you should do, but it’s always these five things.
Scott Wu: Yeah.
Lenny Rachitsky: Scott, we covered a lot of ground. We went through every question I had, which is great, is there anything else that you want to share? Anything else you want to leave your listeners with, maybe a final nugget or something really you want to double down on that we said before we let you go?
Scott Wu: The biggest thing that comes to mind for me is there’s a lot of different perceptions about AI, right? I think basically every emotion under the sun right now. There’s a lot of fear, for example, there’s also a lot of skepticism, and we’re very skeptical types as well, and we always wanted to try it ourselves to really see it and believe it. And I think the main thing that comes to mind for me, is I’m honestly really optimistic about what we’re building here with AI, not just with code and with Devin, but the whole space, and everything that’s getting done. And I think one of the cool things that is really actively happening is just the ability for everyone to multiply themselves, and that’s how we’ve always thought about it, it’s how we’ve thought about what we’re building. And I think there is a lot more to do out there in the world, I’m not too worried about us running out of things to do, and from that lens, I think that the thing that we’ve always been most excited about is, how can we all do more?
Lenny Rachitsky: I hear you Scott. Well, with that optimism, we’ve reached our very exciting lightning round. Are you ready?
Scott Wu: Yeah, let’s do it.
Lenny Rachitsky: Okay, here we go. First question, what are two or three books that you find yourself recommending most to other people?
Scott Wu: In terms of nonfiction, I think for folks in startups, I think one of the things that I’ve really enjoyed is just learning and understanding the history of Silicon Valley. And it’s all of these things that we think about, somebody invented them. It’s one of the great realizations I feel like, is somebody invented the idea of a seed route, somebody invented the idea of venture capital, somebody invented the idea of product-market fit, and all of these different principles that we talk about. And so, for that, there’s a book called The Power Law by Sebastian Mallaby, which I really like. And basically, it’s just a tour of many of the great businesses and the great products that have been built over the last 60, 70 years in Silicon Valley, which I really love. I think in terms of fiction, I actually have always really liked The Great Gatsby by F. Scott Fitzgerald, it’s one of my personal favorites as a fiction book.
Lenny Rachitsky: Do you have a favorite recent movie or TV show that you’ve really enjoyed?
Scott Wu: I have to admit, I have not watched… I can’t think of a single movie or TV show that I have watched in the last while, so I’m sure, I’m looking forward to watching a lot of great ones post-AGI.
Lenny Rachitsky: That’s got to be in the trailer, that’s great, I like that. And that just shows how hard you’re working, just how much shit is going on, and how fast everything’s moving. Do you have a favorite product you’ve recently discovered that you really love? Could be an app, could be something physical, could be a toothbrush.
Scott Wu: One I would say is I got an Aura frame recently, it’s just like a frame that shows photos, and you can show a new photo every day, or every hour, or every 15 minutes, or whatever you like. I’ve actually, I’ve really enjoyed it a lot, I think it’s a nice way to basically just have a picture frame memories that come up. And then, the other thing I would say as a general purpose thing, it’s not particularly new, but I would say, I think AirPods are extremely well-built and well-designed. And I realize now that it’s like, I basically use them for all… I’m taking calls on a walk and I’m using AirPods, obviously I’m doing work at my computer, at my desk, I’m plugged into AirPods, and it works quite well, honestly, for a lot of different situations, and they’re very comfortable, they’re very consistent.
Lenny Rachitsky: I’m going to double down on the Aura frame, I also, I got one of these for my mom and my mother-in-law, and they’re so great for just sharing photos of your kids with your family. And people have, they’ve heard of digital picture frames, but the Aura just does it really well, and it’s really easy to add photos, and they’re just really nice looking.
Scott Wu: You can imagine not that long from now, we’ll have the Aura Frame except Studio Ghiblifies every photo that you have in it, and then it’s… Yeah.
Lenny Rachitsky: Or just imagines things you’ve done that are really cool. Look at my sweet life. Yeah. Cool. And it’s Aura, it’s A-U-R-A, I believe is how you spell it. Folks who want to check it out, we’ll link to it. Not affiliated. Okay, two more questions. Do you have a favorite life motto that you often come back to and find useful in work or in life?
Scott Wu: Yeah, something I’ve thought about a lot is a lot of the proverbs out there are actually contradictions, right? It’s like birds of a feather, and then you also have opposites attract, and you have all… And it’s kind of funny, because you feel that both of them are true, and often they both are true, and a lot of it is about understanding why. And one of those that I feel like, especially in the world of startups that I think about all the time is, I think it is very important to be focused and driven, and to really maximize your potential, and then at the same time, it’s also very important to not let your own personal emotion get tied up in your success or failure, and I think especially with startups, because there’s always ups and downs, honestly, even in the most successful companies ever, it’s just like, it’s a rocky road, there’s a lot that happens, and a lot that goes down.
And I think one of the things which I’ve thought a lot about is that somehow you really want to do your best, and put everything you can into it, and do everything you can to… Basically, you want to put it all out on the field. But at the same time, you want to be okay with both wins and losses, and you want to be able to move on, and go into the next one each time. And something, yeah, it’s funny, but what I’ve found personally is that obviously it’s really important for your own emotional state and mental state to be able to do that, and we’ve had lots of mistakes, and I’ve had a lot.
I had my first company, which obviously, which was cool, but there are a lot of tricky spots there, and then over the course of Cognition, it feels like it’s been already eight years compressed into one year, and it’s still going at that pace. But somehow it also actually makes you more successful, I think, too. It’s like, you are just more able to give it your best, and to do the things that will lead to success if you’re not tying it up in your own personal worth.
Lenny Rachitsky: That is so interesting, I just had a podcast recording recently where with an executive coach, Jerry Kelowna, that I think will come out before this, might be after this, that’s one of his big pieces of advice, and it’s a very Buddhist approach of just [inaudible 01:29:33] and attaching to a certain outcome.
Scott Wu: Yeah.
Lenny Rachitsky: Okay. Final question, I’m curious if there’s a story here, but we could keep it short, is there a story behind Devin as the name, or is there another contender for Devin being the [inaudible 01:29:48]?
Scott Wu: Devin was the name from pretty early on, we were interested, we were working on coding agents from the beginning, and my co-founders are Steven and Walden, for example, and we had this idea, all right, let’s get started, and let’s try to expand the box as much as we can. So, have everyone think out of the box and do their own thing, and let’s have everyone do their own thing first for a bit, and then we’ll consolidate and take everything that we’ve learned. And so, Walden made a virtual developer version of him, which was called DevWalden, and then Steven made one of him, which is called DevSteven, and we had all these… And then we were kind of combining it all into one thing, and we’re like, okay, you know? It’s Devin. And that was the thing. And so Devin was, yeah, Devin stuck for us quite early on, I would say.
One thing which we did have a big decision on though actually is what the image of Devin would be. And so, as folks know, there’s the hexagons and then people have seen this more recently, but there’s actually also an otter, a little otter with a laptop in its lap, and that is Devin as well. And we had this debate over what to go with, and what not to go with and stuff, and it’s been a while now, but somehow we still have both the hexagons and the otter.
Lenny Rachitsky: You skipped over where the Devin, did you have just have, it just came to you?
Scott Wu: Oh, so Devin is, it’s a dev. Yeah.
Lenny Rachitsky: [inaudible 01:31:10].
Scott Wu: And so, it was kind of like, when we were consolidating all the names, it just seemed clear then that this would be the universal dev that we all liked to work with.
Lenny Rachitsky: Wow.
Scott Wu: Yeah.
Lenny Rachitsky: Incredible. Scott, this was so much fun. Oh my God, I learned a ton, which is always a really good sign. Two final questions, where can folks find you/Devin/anything else you want to point them to, and how can listeners be useful to you?
Scott Wu: Awesome. Yeah, no, we’re at App.devin.ai, and you can find us as well on Twitter or a lot of other social media. We’d obviously love to hear any feedback you have about the Devin product, there’s so much to figure out, and I think the, like I said, I think we’re all still 20 steps away from really the future of software engineering, and so it really means a lot to hear what folks think about the product as they’re trying it out. And so, please let us know anytime if there’s things that we can do to make it better.
Lenny Rachitsky: Scott, thank you so much for being here.
Scott Wu: Thank you so much for having me. I had a great time.
Lenny Rachitsky: Me too. Bye everyone.
Thank you so much for listening, if you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app. Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at LennysPodcast.com. See you in the next episode.
Glossary
| English | 中文 |
|---|---|
| ACU | ACU(Autonomous Coding Unit,Devin 的计费单位,保留原文) |
| agentic workflow | agentic 工作流(自主代理式工作流) |
| assembly | 汇编语言 |
| Aura frame | Aura 相框 |
| base intelligence | 基础智能 |
| boilerplate code | 样板代码 |
| CI | CI(Continuous Integration,持续集成,保留原文) |
| Claire Vo | Claire Vo(人名,保留原文) |
| COBOL | COBOL(编程语言名,保留原文) |
| Cursor | Cursor(公司名,保留原文) |
| Datadog | Datadog(监控服务平台名,保留原文) |
| DeepSeek | DeepSeek(AI 公司名,保留原文) |
| defensibility | 可防御性 |
| Devin | Devin(Cognition 公司开发的 AI 软件工程师产品名,保留原文) |
| DoorDash | DoorDash(美国外卖配送平台名,保留原文) |
| feature flag | feature flag(功能开关/特性标志,软件开发中的功能发布控制机制,保留原文) |
| fork | fork(软件开发中的分支副本,保留原文) |
| FORTRAN | FORTRAN(编程语言名,保留原文) |
| GitHub Copilot | GitHub Copilot(产品名,保留原文) |
| idiosyncrasy | 特异性 |
| imitation learning | 模仿学习 |
| jagged intelligence | 锯齿状智能 |
| Jerry Kelowna | Jerry Kelowna |
| Jevons Paradox | 杰文斯悖论(经济学概念,首次出现保留原文并附中文译法) |
| LaunchDarkly | LaunchDarkly(功能管理平台公司名,保留原文) |
| Linear | Linear(项目管理工具产品名,保留原文) |
| lint | lint(代码静态检查工具,保留原文) |
| Lunchclub | Lunchclub(公司名,保留原文) |
| Michael Ballin | Michael Ballin(人名,保留原文) |
| moat | 护城河(商业竞争中的防御优势) |
| multiplayer aspect | 多人协作维度 |
| Neuro | Neuro(公司名,保留原文) |
| opinionated | opinionated(产品设计术语,指产品对使用方式有强立场/偏好,保留原文) |
| PageRank | PageRank(Google 网页排名算法,保留原文) |
| pivoted | pivoted(创业术语,指战略方向转型,保留原文) |
| product-market fit | 产品市场契合度 |
| pull request | pull request(软件开发中的合并请求,保留原文) |
| reinforcement learning | 强化学习 |
| Replit | Replit(在线编程平台名,保留原文) |
| Scale AI | Scale AI(公司名,保留原文) |
| seed route | 种子轮融资(口语中实为 seed round,原文转写为 route) |
| Slack | Slack(团队协作工具产品名,保留原文) |
| speculative decoding | speculative decoding(推测解码,一种大模型推理加速技术,首次出现保留原文) |
| Staff 级工程师 | Staff 级工程师(Staff Engineer,高级别工程师职级) |
| step function | 阶跃式变化 |
| Steven | Steven(人名,保留原文) |
| stickiness | 粘性 |
| Studio Ghiblifies | 吉卜力化(Studio Ghiblifies) |
| TCP/IP | TCP/IP(网络协议名,保留原文) |
| ticket | 工单(软件开发中的任务/工单) |
| Walden | Walden(人名,保留原文) |
| Waymo | Waymo(公司名,保留原文) |
| wiki | wiki(知识库/维基,保留原文) |
| Windsurf | Windsurf(AI 编程工具产品名,保留原文) |
| YC | YC(Y Combinator 缩写,保留原文) |
Reformatted by reformat_english.py
本文深入对话Cognition创始人Scott Wu,揭示全球首个自主AI工程师Devin的演进历程与核心洞见。目前,Devin已承担公司约四分之一的代码提交,预期年底该比例将过半。Scott指出,AI革命摆脱了硬件分发的掣肘,其爆发力与增长速度将是指数级的。
面对AI对软件工程的冲击,他提出了独到见解:工程师的角色正从“写代码的人”转变为“架构师”,而AI的强大不仅不会削减人类工程师的需求,反而会使编程这一学科变得愈发重要。文章不仅呈现了Devin从实习生到初级工程师的成长轨迹,更深入探讨了人机协作的全新范式,为理解AI时代的工程未来提供了务实而前瞻的参考。
走进 Devin:今年将为公司编写 50% 代码的 AI 工程师 | Scott Wu
逐字稿
Devin 的日常使用
Scott Wu: 我们整个团队一年前只有大约 15 名工程师。我们在开发 Devin 的过程中大量使用 Devin。团队里大多数人基本上同时在跟多达五个 Devin 协作,所以 Devin 每个月会往 Devin 的代码库里合并好几百个 pull request。
Lenny Rachitsky: 目前你们的 PR 中,Devin 和人类各占多少比例?
Scott Wu: 大约四分之一左右。
Lenny Rachitsky: 你觉得到年底会到什么程度?
Scott Wu: 说实话,我们预期会超过一半不少。
Lenny Rachitsky: 你们在让公司与 AI 工程师协作方面,走在了前面太多了。
AI 革命的独特之处
Scott Wu: AI 将是我们有生之年最大的技术变革。过去 50 年我们经历的大多数重大技术革命——个人电脑、互联网、智能手机——都有一个很大的硬件组成部分,这也是分发的重要环节。那些为这些行业做开发的人,基本上随着手机用户数量增长、互联网用户数量增长,会看到自己的市场年复一年地稳步扩大。而我认为 AI 已经展现出一个不同之处,就在于这项技术的爆发力。它不受硬件分发的掣肘,这意味着整个领域的增长是指数级的。
工程师角色的未来
Lenny Rachitsky: 当工程师、做开发这件事本身正在发生怎样的变化?
Scott Wu: 我认为几年后程序员和工程师会多得多,而且会很快。当然,作为一名”程序员”这个概念的外延显然会发生变化,但归根结底,这个学科的核心就是告诉计算机该做什么。从这个角度来看,我确实认为随着 AI 变得越来越强大,编程只会变得越来越重要。
本期嘉宾介绍
Lenny Rachitsky: 今天我的嘉宾是 Scott Wu。Scott 是 Cognition 的联合创始人兼 CEO,这家公司打造了一款名为 Devin 的产品——世界上首个自主 AI 软件工程师。与我之前在这个播客中介绍过的其他 AI 工具不同,Devin 被设计成像一个真正的远程工程师——你可以像和任何其他人类工程师一样,通过 Slack 或它的专属网站与它交流。Devin 大约一年前刚发布时,还完全是个初级工程师。过去一年里他们取得了很大的进展,Devin 现在已经被大量公司在生产环境中使用。我们聊了他们的 15 人工程团队如何使用 Devin 来构建 Devin——包括每个工程师如何同时使用大约五个 Devin 来帮助他们编码和加速推进。目前他们的 pull request 中有四分之一是由 Devin 提交的,而他们预期到年底这个比例会超过 50%。
我们还聊到了 Scott 如何想象软件工程在未来的面貌,以及工程师的角色如何从”写代码的人”转变为”架构师”。我们也谈到了他们在找到这条路径之前经历的八次方向调整,为什么 Scott 认为像这样的 AI 工具会导致更多的工程师招聘而非更少,还有 Devin 这个名字的由来,以及更多内容。这一期绝对会让你大开眼界。如果你对工程、产品构建和 AI 的未来走向有一丝兴趣,我强烈推荐你听一听。非常感谢 Claire Voue 为这次对话提出了许多好问题。如果你喜欢这个播客,别忘了在你喜欢的播客应用或 YouTube 上订阅和关注。
正式访谈开始
Lenny Rachitsky: Scott,非常感谢你的到来,欢迎来到播客。
Scott Wu: 非常感谢邀请,很高兴来到这里。
Lenny Rachitsky: 我非常高兴能邀请你来,因为你在构建的东西——而且已经构建了一段时间——跟很多其他 AI 公司长期以来所做的非常不同,虽然他们现在开始向你所在的方向靠拢。我们会聊到这一点。而且,当前这个时间点在 AI 的历史进程中和 AI 的发展旅途中都是如此独特,所以现在能聊一聊真的很棒。我觉得几年后我们再聊的时候会说:哇,我们当时有些判断太对了,有些判断又错得太离谱了。
Scott Wu: 是的。
Lenny Rachitsky: 所以我很高兴你在这里。让我们从 Devin 开始聊起,让大家了解 Devin 到底是个什么东西——这是你们开发的主要产品。理解 Devin 最简单的方式是什么?Devin 到底是什么?
Devin 的起源
Scott Wu: 没错。Devin 是一个完全自主的软件工程师,能够端到端地完成任务。现在 AI 编码工作流的各个环节都有很多优秀的工具,而 Devin 做的是提供一个完整的异步工作流——你可以在 Slack 里的一个 issue 上 @Devin,你在讨论某个问题时 @Devin;你也可以在 Linear 里 @Devin;然后 Devin 会在你的 GitHub 上提交 pull request。它的定位非常明确,就是作为工程团队的一员,扮演你团队中初级工程师的角色。
Lenny Rachitsky: 太棒了。我记得你们发布的时候,核心宣传语就是”这是你的新 AI 工程师”,而且它在很多方面表现得确实很好,但也有一些方面不太行。你们发布到现在大概一年了,对吧?
Scott Wu: 对,对。
Lenny Rachitsky: 如果用工程师资历来衡量的话,发布时的 Devin 相当于什么级别的工程师,现在的 Devin 又相当于什么级别——这种衡量方式好吗?还是说你有更好的方式来描述?
Scott Wu: 说实话回想起来挺不可思议的,因为一年前我们做最初发布的时候,大家其实并不真的相信 agent 是可能的。那个时候和现在确实很不一样。2024 年初,模型能力方面明显还要更早期一些,尤其是推理能力差得比较多。之后的发展当然是有目共睹的。就实际技能而言,我们有时会做一些类比——刚起步的时候,大概像是一个高中计算机学生;后来逐渐变成了大学实习生;现在已经像一个初级工程师了。不过我想说这些都只是粗略的参照,因为我非常喜欢”锯齿状智能”(jagged intelligence)这个说法——显然它在某些方面比人类强得多,在另一些方面又比人类差得多。
过去一年我们学到了很多,尤其是关于不仅仅是编码 agent,而是 agent 这个整体——大家到底应该如何在工作中使用 agent、与 agent 协作。我们早期构建的产品没有 Slack 集成,没有 GitHub 集成,没有 Linear 集成,没有来回交互的规划阶段,也没有办法对 Devin 的代码进行调整。所以之后在产品层面做的很多功能,本质上都是在解决同一个问题——如何让与 Devin 的协作和任务交接变得尽可能顺畅。
Lenny Rachitsky: 这太有意思了。所以大量工作其实不只是”怎么让 Devin 成为最好的工程师”,而是”怎么与这种我们从未合作过的新型实体一起工作”。
Scott Wu: 我觉得两方面各占一半。能力方面显然有了巨大提升,我们看到它在持续变好,而且是可衡量地变好。但另一面确实是产品界面、工具等等。现在大家基本都知道怎么使用聊天机器人,这是一个大家熟悉的交互方式。但 agent 的使用仍然有一个相当陡峭的学习曲线——怎么用它们、怎么发挥它们最大的价值。所以看到越来越多其他人开始在 agent 领域构建和投入,真的很令人兴奋。但我觉得这是我们整个行业正在一起摸索的事情。
Lenny Rachitsky: 关于 Devin 目前的规模,你方便分享什么就分享什么,然后你觉得一年后 Devin 的编码能力会达到什么水平?
Scott Wu: 我们的合作方覆盖各种阶段和规模的公司。最小的是只有一两个人的初创公司,用 Devin 来搭建他们最初的 prototype 或初始产品;最大的是大型上市公司、财富100强企业、上市银行等等,在他们的工程团队中使用 Devin。使用场景的范围非常广。显然,一两个人的初创公司所做的工程工作和上市银行所做的工程工作差别很大。
但贯穿其中的核心定位就是——做你那个初级搭档,让你走得更快,真正地放大你的能力。作为工程师,它让你可以拥有自己的 Devin 团队来并行工作,而不需要事事都完全同步地守在一个任务上。它也在放大你的团队和团队的知识库,因为 Devin 会从与团队每个成员的合作中积累大量知识,并能把知识带入每一次新的会话中。
Devin 的起源故事
Lenny Rachitsky: 太好了。稍后我们会在播客中给大家展示 Devin 实际运行的效果,会做几个现场 demo。不过让我们先回到故事的起点——Devin 的起源故事是什么?这一切是怎么开始的?
Scott Wu: 创始团队的成员大多彼此认识了很多很多年。不过对几乎所有人来说,这是我们第一次一起工作,但我们确实认识很久了。而且我们每个人在过去十年左右都各自有着自己在 AI 领域的经历。就我自己而言,之前我经营了一家叫 Lunchclub 的公司,是一个用 AI 做职业社交的产品,我经营了大约五年。我的一位联合创始人 Steven 是 Scale AI 最早的工程师之一,Scale AI 显然发展得很大、做得很好。我的另一位联合创始人 Walden 是 Cursor 的早期工程师,Cursor 同样发展得很大、做得很好。我们整个团队基本是类似的背景——我们很多人是通过竞赛编程和数学竞赛认识的,但之后十年一直保持着非常密切的联系,各自走了自己的路。
团队中有在 Neuro 带团队的人,有在 Waymo 做过的人,有自己做过面向机器学习的 YC 工具类初创公司的人。我们非常兴奋能一起构建点什么。那是 2023 年底,距今大约一年半。刚开始的时候,有几件事我们非常确信。其中一件是:强化学习(reinforcement learning)确实在起作用,而且将成为能力提升的下一个重大范式转换。当时是 2022 年 ChatGPT 的初次发布,那些模型用 AI 的术语来说,一阶近似就是所谓的”模仿学习”(imitation learning)——基本上就是让模型读遍互联网上能找到的所有文本,然后训练它像互联网上的人那样说话。当然在这之上还有很多细节,但本质上的第一阶近似就是这样。
Scott Wu: 它确实令人惊叹。它通过了图灵测试的门槛,能够做出回应,拥有关于许多事物的百科全书式知识。而我认为过去一年或一年半以来我们所进入的这种新范式,本质上是高算力强化学习,这是一种非常不同的范式——基本上就是让模型去执行任务、组装成果,然后对其正确与否进行评估,再利用这些认知来决定下一步行动并从中学习。我们对此非常笃定。对我们来说,选择代码领域是自然而然的事,原因有几个。一是因为我们自己就是程序员极客,教 AI 写代码对我们来说简直是最酷的事情了。同时,代码天然拥有一个完整的自动反馈闭环——你可以运行代码,而这种自动化反馈恰好是强化学习所依赖的,也正是这些模型在编码上如此出色的原因。
从文本补全到智能体
另一件我们非常确信的事情是,产品体验将从所谓的文本补全转向智能体。一阶近似来说,文本补全已经催生了许多优秀的产品体验——被用于营销、客户支持、教育,当然也包括代码领域。GitHub Copilot 基本上就是那波浪潮中最具代表性的产品。
但我认为我们真正预感到的巨大转变,是从这种文本到文本的模式,转向一个真正能够做出决策、与真实世界交互、接收反馈、迭代并多步解决问题的自主系统。现在我们把它叫做智能体,但当时这就是我们最兴奋的事情。所以从一开始就是编码,从一开始就是智能体。从某种程度上说,这似乎应该是显而易见的。但即便如此,我觉得过去一年半里我们在编码智能体这个方向上至少 pivoted 了八次左右。
Lenny Rachitsky: 我最近注意到,那些顶级的 AI 公司——不是所有,但很多——赢的产品名字跟公司名字不一样,这不常见。Cursor 和 Anysphere,Bolt 和 StackBlitz,你们是 Cognition Labs,V0 和 Vercel。这说明这些产品都是在公司发展后期才涌现出来的,他们尝试了很多东西,然后——哇,这个东西成功了。这在这些顶级 AI 公司中如此普遍,非常有趣。
从黑客屋到产品
Scott Wu: 对,甚至还有 OpenAI 和 ChatGPT、Anthropic 和 Claude、Google 也是。确实很有意思,我同意。我们刚开始的时候,甚至算不上一家公司,更像是做一个项目,或者几乎是一场黑客松。我们订了一间 Airbnb,租了两周左右。那时候正好是感恩节前后,我们就把一群只想一起做项目、做点酷东西的人聚在了一起。说来有趣,我们最初其实在做的方向是解竞赛编程题,用智能体循环来提升表现。你可以在测试用例上运行代码来做评估,有很多智能体层面的工作可以做来提升成绩,我们最初花了一些时间在这上面。然后我们就这样一路走过来了——从某种意义上说,整个公司的故事就是从一个黑客屋搬到另一个黑客屋。
之后我们又搞了另一个黑客屋,也就是在那个阶段,Devin 的一些最初想法开始浮现——真正构建一个软件工程智能体,而不仅仅是编码智能体,需要与大量工具交互。但即便在那个时候,迭代仍然非常多。比如让用户和 Devin 对话这个想法,也是我们后来才摸索出来的。最初就是直接把任务丢过去,然后它做完,展示给你完整的代码。现在显然不同了——你可以随时介入,可以对计划提出反馈,你可以和 Devin 一起界定任务范围。很多东西是我们逐步开发出来的,当然我们也从用户的使用场景、产品形态中学到了很多。我们在 Devin 的能力、工具使用、调试和决策方面做了很多大幅度的跃升式改进。
所以这确实是一段有趣的旅程。我想说,对我们来说最根本的问题——我们一直在思考的问题——其实就是:软件工程的未来是什么?我们应该如何与 AI 协作来写代码?因为归根结底,这个问题的答案是我们所有产品决策的根基。
Lenny Rachitsky: 我喜欢你主动提到了那个我最想聊的核心问题。不过在问之前,为了留下历史记录——你们大概是什么时候开始一起做这些事情的?Devin 是什么时候发布的?中间隔了多久?
Scott Wu: 我们是 2023 年 11 月开始的,当时就是黑客松模式。正式注册为公司大约在 2024 年初,然后我们的初次发布是在 3 月。整个过程一刻不停——事实上过去 17 个月一直没有停下来过——先是冲刺到发布,然后是与企业客户合作、大幅开发产品、让它适配更多实际使用场景,接着在去年 12 月全面开放自助服务。几周前我们又推出了 2.0。这段时间确实非常忙碌。
为什么 Devin 是一个”人”
Lenny Rachitsky: 这大概是本世纪最轻描淡写的说法了。让我问一个问题,因为你刚才提到了一些——关于把 Devin 当作一个人的理念,为 Devin 塑造一种人格。这与其他任何 AI 应用都不同,没有别的产品有自己的名字,你不会把它当作一个人。你们是怎么决定走这条路的?又是如何设计才能让它运作良好的?
Scott Wu: 我会说这是一个让我们相当自豪的决定。市面上的产品体验很多种,而 Devin 真正独特的地方在于你可以真正把任务交出去。而且越来越多地我们发现,向人们解释 Devin 的体验,其实最简单的方式就是说——这就是你的初级搭档。这一点贯穿了整个使用流程的许多环节。比如在引导阶段,最初确实有很多用户进来看到空白屏幕不太知道该做什么,或者一上来就说”我要让 Devin 做一次整个代码库的大重构”。
而我们在实践中逐步学到的,就是引导用户换一种思路——等等等等,先把仓库设置好,先给 Devin 几个简单的单点任务让它熟悉代码库,确保 Devin 拥有所需的条件。如果 Devin 需要测试代码、运行 linter、跑 CI 这类事情,我们显然要确保 Devin 有自己的虚拟机环境来完成这些操作。
软件工程的变化
Scott Wu: 类似地,我觉得使用模式在当时也不是很清晰。显然你可以坐在那里一步步地看 Devin 的操作,按那种方式工作,但我们发现作为一支要大量产出的团队,最佳的工作流其实是同时与多个 Devin 协作,异步运行它们,把它们派出去,然后只在需要提供反馈、调整方向或类似的时候才介入。所以在很多方面,Devin 这个名字确实是我们在产品层面试图捕捉的那种灵魂——把它当作一个更接近自主实体的存在,你可以把任务交给它,你可以和它协作,你应该在长期中不断地教它、和它一起学习。
Lenny Rachitsky: 我想回到你之前提到的、后来被我岔开的话题——对软件工程的影响,以及软件工程将会如何变化。这个问题大致分两部分。就说当下,人们在使用 Devin 的时候,比如今年,做工程师、做开发的体验在这些公司里发生了怎样的变化?具体是什么样的?
Scott Wu: 顺便说一下,我们自己全都是软件工程师。我本人是学编程出身的,骨子里也依然是个程序员。我们一直以来的思考方式是:存在不同的抽象层和工具,从一个高层次来说,我会这样概括——我认为 AI 总体而言就是,计算机显然变得越来越智能,能做的事情越来越多,也许有一天计算机会真正胜任我们所做的一切,而人类不再需要承担任何责任。
我并不认为那一天会很快到来,但我想说的是,在那天到来之前,只要我们还是方程式中的一部分,最重要的事情之一显然就是——作为人类,去指挥我们的计算机,告诉它们我们想要什么、我们想构建什么、我们想做什么。软件工程,我们今天理所当然地把它理解为 Python、C++、JavaScript 等等这些,但归根结底,这门学科的本质就是能够告诉计算机该做什么。从这个视角看,我确实认为编程只会随着 AI 变得更加强大而变得越来越重要。而让我们真正兴奋的是亲眼见证这种渐进式的变革。你问到今天的情况是怎样的,我会说——确实就像是有一个初级搭档,或者其实是一队初级搭档,可以和你一起工作。
我们团队里每位工程师,在构建 Devin 的过程中都会大量使用 Devin。Devin 每个月会往生产环境的代码库合并数百个 PR——而我们整个团队只有 15 名工程师,所以这在我们所有代码中占了相当大的比例。我们的使用方式基本上就是——每个人都有自己的一整队 Devin。如果你要梳理各种 issue,如果你要处理功能请求、bug,或者你想构建新的范式,自然就会出现很多交接节点——你直接说,“嘿 Devin,情况是这样的,你能先处理一下这个吗?”
有时候 Devin 能够百分之百自主完成任务,直接提交 PR,然后你合并就行了,这就很棒。有时候你需要在那最后的百分之十或二十介入——也许有几个细节涉及你具体想要的范围界定,或者这个功能的架构方式,又或者你想最后自己去前端测试一下,确保效果完全符合预期,然后给出一两条反馈。但总体来说,核心就是学会与 Devin 协作,从而能够并行地做更多事情,构建更多东西。
Lenny Rachitsky: 目前你们的 PR 中 Devin 和人类各自占比多少?
Scott Wu: 我得查一下具体数字,但大概在四分之一左右——
Lenny Rachitsky: 哇。
Scott Wu: 是的。
Lenny Rachitsky: 那六个月前是多少?
Scott Wu: 哦,增长了很多。我们自己内部也看到了指数级的增长。这个挺有意思的,因为它始终是能力和产品界面两方面的共同作用。我认为智能水平提升了很多,但另一方面,我们当然也花了很多时间去弄清楚如何构建一个真正合适的界面——让你在 Devin 能完成百分之八十或九十的任务上获得 Devin 的价值。Devin 显然还不完美,它会犯错,诸如此类。所以核心问题就是——你如何与 Devin 界定初始任务,然后把 Devin 放出去让它去做你想做的事?你如何在最后介入进行审查和反馈?你如何确保 Devin 随着时间推移不断学习?你如何在需要时随时查看并进行纠偏?
PR 占比与未来展望
Lenny Rachitsky: 好的,目前大约四分之一的 PR 来自 Devin。你觉得到年底这个比例会是多少?你估计呢?
Scott Wu: 我预计到今年年底会超过一半。随着时间推移,我们看到的一个趋势就是——你能够异步完成的工作越来越多,你能交接出去的也越来越多。我认为编程的灵魂,软件工程的灵魂,一直以来都是——无论是在哪个阶段,不仅仅是现在,哪怕是汇编语言的时代,哪怕是 Pascal 的时代,哪怕是打孔卡片的时代——我认为它的灵魂始终是:定义你面临的问题,认真思考……
……定义你面临的问题,认真思考你到底想要构建什么样的解决方案。思考架构,思考细节,在脑海中清晰地描绘出你到底想要构建什么、你想要让你的计算机做什么。我认为这就是软件工程真正了不起的地方,也是软件工程最有趣的部分。
同时,我觉得这大概只占普通软件工程师百分之十左右的时间。因为百分之九十的时间是——你遇到了一个 Kubernetes 错误,然后你得调试,看看哪里出了问题、系统崩溃了;或者你留了某个端口没关,导致出了问题;或者有个 bug 报告需要处理;或者你得迁移代码;或者你得升级到新版本之类的。大量时间花在具体实现上。
从砌砖工到建筑师
在构建 Devin 的过程中,我们的一种思考方式就是让工程师从砌砖工变成建筑师。很大程度上就是达到这样一种状态——你可以做高层的指挥,你可以精确地按你想要的方式定义规格。我认为关键仍然在于让人类保持掌控,让人类能够完成完整的规格定义,只是极大地放大你在一天、一小时或任何时间段内能够完成的事情的量级。
Lenny Rachitsky: 展望未来,假设有人想进入软件工程领域,考虑成为一名工程师——首先,你觉得人们应不应该……这是现在大家都在问的经典问题:还应该学编程吗?
Scott Wu: 对。
Lenny Rachitsky: 我很想听听你的看法。第二个问题,对于已经是工程师的人来说,在你刚才说的”从砌砖工到建筑师”这个转变中,你觉得哪些技能会越来越重要,哪些会越来越不重要?
Scott Wu: 当然,我很喜欢这个问题。首先是还应不应该学编程,我的回答是绝对应该。我觉得,在很大程度上,当你上计算机科学课程、学习这些基础知识的时候,当然你确实学到了一些特定语言的语法之类的东西。但说实话,你真正学到的大部分内容,第一是逻辑性地拆解问题的能力。第二,我想说的是计算机的模型,以及我们长期以来构建的大量决策和抽象。
什么是数据库,应该怎么理解数据库?什么是垃圾回收系统,它是怎么运作的?所有这些不同的组成部分?我认为这之所以重要,是因为它和很多其他情况一样……可以说我们在编程领域已经经历过好几个这样的阶段,我认为下一个阶段会在某种程度上更快、规模更大,但在很多方面味道是相似的——当你今天使用 Python 的时候,显然很多东西已经被抽象掉了。
从某种意义上说,50 年前的人可能已经会觉得 Python 就是你用英语解释你想要什么,然后计算机就帮你做到了。这很好,我觉得它非常强大,也让编程变得更加普及。显然,我们现在拥有的程序员比以往任何时候都多。但我想说的是,在你作为工程师积累技能的过程中,理解这些抽象、能够剥开底层,确实会有很大的帮助。比如人们在做极端性能优化时会使用汇编语言,但同样地,为了构建好的系统、理解这些东西,你肯定需要理解这些抽象——网络是怎么运作的。
TCP/IP 具体是什么样的,或者这段 Python 代码被解释时发生了什么,所有这些细节。类似地,我认为我们会达到这样一种状态:即使完全没有任何经验,你也能仅仅通过解释你想要什么,就构建出相当酷的东西、做出相当出色的工作。但我认为,在相当长的一段时间内,你仍然需要能够精确地思考细节,能够剥开抽象层,对你想要构建什么以及怎么构建做到非常精确。
Lenny Rachitsky: 那么在你看来,哪些技能对工程师来说越来越有价值?今天的工程师应该越来越倾向于哪些方向,又有哪些可以说”算了,这个不用再考虑了”?
Scott Wu: 当然。我觉得架构师……我的意思是,工程领域本来就有”架构师”这个词,我认为方向上它就是正确的词。仅仅做例行的实现、写样板代码之类的事情是一回事。我想说的是,在很多方面,AI 编程已经让我们在这些事情上快了很多。但我觉得很多核心问题在于理解非常复杂的系统,在整个公司的上下文中工作,思考你正在构建的产品或正在做的工作,理解”好,我们想解决什么问题?我们想怎么解决这些问题?我们到底想构建什么样的解决方案?我们要做的所有关键决策和权衡是什么?”
基本上,我认为能够把这些事情做得非常好的人,将能够越来越多地放大自己的能力。如果说什么的话,我觉得几年后程序员的数量和工程师的数量会比今天多得多。我认为很快,“做一个程序员”意味着什么的外在形态,显然会发生变化。从某种意义上说,它已经变了。但我觉得我们需要构建的东西会多得多。大家一直在谈论杰文斯悖论(Jevons Paradox)。我的意思是,软件真的是杰文斯悖论最闪耀的范例——作为一个社会,我们总是能找到越来越多想要用软件去构建的东西、想要为之编写代码的东西。我真的觉得还有很多很多事情等着我们去做。
杰文斯悖论
Lenny Rachitsky: 对于不了解杰文斯悖论的人,你能简单解释一下吗?
Scott Wu: 当然。杰文斯悖论说的就是,当某样东西的价格下降时,花在它上面的总支出反而可能上升。你可以用钱来理解,也可以用时间或资源来理解。但这里直接相关的版本是,我认为随着编程变得越来越容易、编程变得越来越高效,我们会有更多的程序员。在一种零和视角下,你可能会说:“好吧,我们的软件工程效率会提高十倍,那就意味着我们需要的软件工程师会少十倍。“但我认为实际上会发生的是,我们构建的代码量会超过十倍。因为我们所做的工作显然受限于我们实际构建、执行和迭代的能力。我们会有那么多好的想法,那么多好的产品。比如人们会构建更多个性化的体验,会有很多事情可做。
Devin 的日常使用方式
Lenny Rachitsky: 回到你们使用 Devin 的方式,你说每个工程师都有一支 Devin 团队。你们公司大多数人现在一般是同时跟多少个 Devin 一起工作?
Scott Wu: 对,它是非常异步的。显然,你可以随时启动和关闭它们,基本按照你觉得合适的方式来。但我会说,团队里大多数人经常同时操作多达五个 Devin。这是一个很好的工作流——你想一想,“好,今天我们要完成哪五件事?“让第一个 Devin 做第一件,第二个 Devin 做第二件,第三个 Devin……话说回来,我觉得我们花了一些时间来适应它,让它变得真正直觉化,但我认为这确实是一种不同的体验——你把大部分事情都异步地交接出去。
你每个任务的目标是,只在你专业能力真正需要的环节介入——要么是你真的需要精确地定义你要解决的问题和要构建的东西,要么是一些更复杂的部分,你需要引导 Devin 按照特定的方向去修改,比如”我想让这个类按照这种方式设置,然后把所有下游的引用也改掉”之类的。但基本上就是让 Devin 异步地完成大部分工作。
Lenny Rachitsky: 那你们大概有多少工程师?
Scott Wu: 我们的工程团队目前大约是 15 个人。
Lenny Rachitsky: 15 个。15 个。
Scott Wu: 15 个,对。
Lenny Rachitsky: 天哪。好,然后每个人有差不多五个 Devin。
Scott Wu: 对。
Devin 数量远超工程师
Lenny Rachitsky: 所以 Devin 的数量是工程师的五倍。我很喜欢这一点,它让我们瞥见了未来的方向。你们在 AI 工程师的使用方式上遥遥领先于其他公司。观察你们的运作方式,从某种意义上说,基本上就是大多数公司最终的运作方式。
Scott Wu: 是的。可以说,我们自己已经感受到了这种转变……就团队而言,大家显然不再花那么多时间去写样板代码或单纯做功能实现。大家可以把更多时间集中在思考核心问题上——“我们怎样让 Devin 变得更好?Devin 的正确界面是什么?怎样的流程或功能组合才能真正带来最好的体验?“这就是我们想要的状态。
起飞时刻与异步工作流
Lenny Rachitsky: 什么时候会到达那个拐点——Devin 远远超越其他所有人的起飞时刻?当你有足够多的 Devin 做所有这些事情的时候,它们无处不在,你们就领先了十年、二十年、三十年、一百年。
Scott Wu: 说实话,作为一个社区,我认为全世界所有的工程师都需要思考这件事,为此做好准备,并适应这些新技术。但我想说的是,越来越多的事情——尤其随着能力不断提升,即使就目前而言——会转向这种异步的工作流程。其中一个原因是,在真实世界中,你会受到现实条件的制约。一种说法是……不要对这些数字太较真,但从一阶近似来看,当然是能够写文件、补全函数、补全代码行这些事情。
这些确实帮助很大,体验也很好。但构建软件有很多环节完全不是这样的,对吧?比如你要修一个 bug,于是启动本地服务器,在前端点击操作自己的产品,尝试复现这个 bug。拿到错误之后,你去 Datadog 查看发生了什么,在日志里寻找其他错误。查看相关文件,看看哪里出了问题,做一些修改,确认改动看起来没问题后再重新跑一遍整个流程。这才是软件工程师很大一部分工作内容。这些流程都需要真实的时间。我认为我们会越来越多地转向这种 agentic 工作流,因为在某些方面,这才是真正实现 200%、500%、1000% 效率提升的途径,而这正是未来几年软件工程领域即将迎来的。
演示环节
Lenny Rachitsky: 说够了,给大家看看这东西到底长什么样吧。你准备了几段 demo,展示了一些你觉得很有用的场景。现在你要共享屏幕,我们边看边聊。
Scott Wu: 好。与 Devin 协作的整个过程本身就是异步的。我觉得让咱们实际看看 Devin 的操作过程会很酷,然后我们可以再看一些 Devin 做过的其他工作示例,比如在我们团队中 Devin 为我们做的事情。之后我们可以再异步地回去查看 Devin 的进度。
Lenny Rachitsky: 开始吧。
Scott Wu: 我快速分享一下。我想强调的关键点是,这很大程度上其实就是以软件工程师、工程师团队、产品经理等角色的视角去思考——有哪些东西是我们要构建的、想要交接出去的?比如我们给 Devin 配置了 Devin 自己的代码库,我就先用这个来启动一个任务。我直接说,“Devin,我正在和朋友 Lenny 做直播。”
Lenny Rachitsky: Hi, Devin.
Scott Wu: “你能修改一下 Devin web app two 吗?“把我们播客的链接作为一个功能加到 Devin 网站上。
Lenny Rachitsky: 好啊。加到真正的 Devin 网站上。
Scott Wu: “把 Lenny 的网站展示出来。”
Lenny Rachitsky: 好,把你的功能都撤掉吧。
Scott Wu: 我们来启动这个任务。如你所见,Devin 立刻就开始工作了,并给出了回复。再说一次,你可以异步处理,也可以同步处理。这次我们稍微跟进一下,看看它到底在做什么。可以看到,Devin 正在浏览文件,查看大量内容。我们可以随时跟随查看,看看情况。你可以看到 Devin 已经指出了几个具体的地方——比如侧边栏,我们在前端已经实现了的部分,还有其他一些组件。我们要新建一个组件,这个组件会链接到 Lenny 的网站。听起来都没问题。Devin 在问我们有没有什么补充要求。同样,你可以让 Devin 自己做决定然后交接,也可以给它更多想法。比如按钮是在新标签页打开还是在应用内打开?我就说在新标签页打开吧。
Lenny Rachitsky: 这些你随时都可以回答吗?它在等你的回复吗?
Scott Wu: 你随时可以回答,可以交接,也可以再交接回去。
Lenny Rachitsky: 它不会说”拜托,我刚写好的,你怎么不早说?“这样吧?
Scott Wu: 没错。Devin 的一个重要特点就是——Devin 永远充满热情,永远愿意投入时间。
Lenny Rachitsky: “谢谢你在变更需求范围,谢谢你,Scott。”
Scott Wu: 我们给 Devin 一点时间工作,它会过一遍这些文件,然后为我们创建一个 pull request,到时候我们再看结果。不过我觉得展示一些其他 Devin 的实例也会很有意思。今天早上我就用 Devin 做了一件事——我让 Devin 帮我复习资料,为这次播客做准备。我一直是这个播客和 newsletter 的粉丝。我跟 Devin 说,“嘿,Devin,我要上播客了。能不能尽可能研究一下他的资料,然后给我做一个好看的网页问答,确保我把该知道的都记住了?“就是今天早上,我让 Devin 做的,来看看它做了什么。它先去了维基百科。不幸的是,维基百科上没有专门的页面……
Lenny Rachitsky: 我还不够出名。
Scott Wu: 维基百科亏待你了。我们得想办法弄一个页面。然后它去了 Spotify 上找到了相关信息。
Lenny Rachitsky: 所以你在实时看它的研究过程?
Scott Wu: 对对对,当然这是今天早上的了。
Lenny Rachitsky: 这是 Devin 之前所做操作的回放。这是 Devin 的一个功能——你可以直接观看它之前做了什么。
Devin 的实际演示
Scott Wu: 对对对,尤其是你在做工程产品之类的东西时,你可以看到 Devin 执行的每一个步骤,比如 Devin 是否在本地测试了代码,你显然希望能进去看看 Devin 在鼓捣什么、测试了什么。它找到了那个 newsletter,然后去查看这些内容,把所有这些都读了一遍。然后它说,“好,开始组装代码吧。” 它说,“我已经研究完了。” 然后就开始一行行地写,把应用搭起来。它还自己给自己出了个测验。实际上我们真的来玩一下这个测验吧,看看我知道多少。
Lenny Rachitsky: ……
Scott Wu: 叫什么名字?
Lenny Rachitsky: 这个播客叫什么名字?Lenny’s Podcast。对于没在看视频的听众来说,大概有多少订阅者?一百万。非常好。
Scott Wu: 对,对。
Lenny Rachitsky: Lenny 主要关注哪三个话题?哦,产品、增长和职业发展。非常好。这个测验做得不错……
Scott Wu: Lenny 除了做播客还做什么?
Lenny Rachitsky: Lenny 除了做播客还做什么?
Scott Wu: 我觉得是写作、投资和顾问。Lenny 多久发一次——
Lenny Rachitsky: ……
Scott Wu: 一周一次,对吧?
Lenny Rachitsky: 一周一次。
Scott Wu: 对。我们可以把这些都过一遍,全都做一遍。顺便说一句,我当然自己先做过这个测验了,确保准备充分。不过这显然是比较有趣的例子之一了……
Lenny Rachitsky: Scott,我的 newsletter 有多少订阅者?
Scott Wu: 超过一百万了,实际上。
Lenny Rachitsky: ……
Devin Wiki 与代码库理解
Scott Wu: 然后我再展示最后一个,之后我们可以回到之前那个初始运行。就像我说的,这一切的设计初衷是与现有的各种代码工作流配合使用。比如,我们之前在 GitHub 上对 DeepSeek 仓库做了一些探索,把它导入到了 Devin 里,然后在 Devin 里搭建了我们自己的 fork。我想展示几个东西。一是 Devin 会搭建自己完整的 wiki,记录它对代码库的全部内部理解。当 Devin 对代码库建立索引时,显然,构建代码库的表征、学习它并随着时间不断改进,这是 Devin 做的重要事情之一。有趣的是,我们发现人类其实也很自然地想去了解这个代码库表征。
Devin Wiki 就是我们为此搭建的功能,你可以查看所有这些不同的部分,看到每一个细节。这里是 FP8 运算,这里是 SG 集成。还有不同层如何搭建和组合在一起的示意图,部署操作,以及很多关于架构的细节。你也可以对它提问。比如你可以说,“DeepSeek 是如何处理为 speculative decoding 设计的多 token 预测的?” 它会在整个代码库中搜索,并基于此给出一个有理有据的回答。
我们经常使用这个功能。在你为 Devin 规划任务、撰写初始提示词时,它很有帮助。当然,即使不依赖 Devin,你对自己的代码库也经常会有各种疑问,这时它也很好用。
代码库规模与架构理解
Lenny Rachitsky: 我在和越来越多 AI 构建公司和应用交流的过程中学到的一点是,它们能集成多大规模的代码库,差异非常大。这对已有公司 versus 初创公司来说是很重要的一件事——那些拥有大量现有代码库的公司。人们应该怎么理解 Devin 能接入什么规模的代码库?
Scott Wu: 我们的目标是支持尽可能大的代码库。可以这样来想——我们作为工程师在面对一个大型代码库时的思维方式是,当你在做修改或者思考某个具体任务时,你并不是把代码库的每一行都同时加载进来。你会有一个高层次的认知框架,然后可以深入查看,对每个部分获得更高分辨率的理解。Devin 的工作方式非常类似。它做的第一件事就是弄清楚整体架构——这里是怎么回事,它是为什么而构建的,等等。然后在每个组件内部,它显然也能进一步深入,给出更多细节。比如这里是将 FP8 转为 float16 的具体实现方式,这里是代码库各个不同的部分。同样,我们在设计上也考虑了可扩展性。
Lenny Rachitsky: 这本质上又回到了”工程师即架构师”的概念。现在它在帮你理解架构,算是绕回之前的话题了。
Scott Wu: 对对对,没错。我们实际看到的一个有趣用例是,很多人会借助 Devin 来帮助团队新工程师入职。当你刚加入一个团队时,对代码库或项目设置显然会有很多问题。有时候向你的导师或经理提问也有点尴尬,特别是如果你担心问题太蠢的时候。能直接问 Devin,翻看 Devin 的 wiki,了解这些内部表征,就很方便。
Lenny Rachitsky: 我觉得这很有意思,因为它又回到了你之前的观点——Devin 不仅仅是一个初级工程师。你称之为锯齿状工程师。
Scott Wu: 锯齿状智能。
Lenny Rachitsky: 锯齿状智能,在理解代码库方面几乎像一个 Staff 级工程师。通常你得找一个在那待了很久的工程师问,“这个是干什么的?那个东西在哪?这个怎么运作的?” 感觉 Devin 在这方面非常擅长。
Scott Wu: 对对。显然,检索和处理大量代码、一次性处理大量 token,这是语言模型非常擅长的事情。基本上能在需要的地方获得这些能力提升,是非常棒的。好,好了。
Lenny Rachitsky: 好,你还有几个用例要展示?
Scott Wu: 对,最后一个我想展示的是……这个其实我们上周刚上线,是 Devin 与 Linear 的完整自动化集成。比如你在 DeepSeek 仓库上有一些任务要做,一切都配置好了,你只需要给任务加上 Devin 标签,Devin 就会过来处理,给你这样的结果——它会给出对任务的看法,你可以查看它提到的每个具体文件,或者它会指出它认为重要的代码片段。看完之后,如果你对它构建的内容或得出的结论满意,就可以直接启动 Devin 会话,让它去实际完成那项工作。
Lenny Rachitsky: 这太疯狂了。听起来是个特别简单的想法,但本质上你是说,Linear 里有各种修复和功能的任务,现在 Devin 直接就能替你搞定。
Scott Wu: 对,但这绝对是一个需要亲力亲为的过程。当 Devin 在界定任务范围或给你它的想法时,你肯定需要参与。顺便说一个很好的点,Devin 会给你它的置信度——“我对这部分的理解有多大概率是准确的”之类的——这确实能大幅加快进度。你说得对,很多产品经理就特别喜欢在 Linear 里用 Devin 来更好地理解代码库之类的。比如 LaunchDarkly 的 Claire Vo 就是一个重度 Devin 用户,她特别喜欢用它来界定任务、问数据问题,或者问”现在进展如何?这个合并到生产环境了吗?这个现在是通过 feature flag 控制的吗?有多少用户在用这个功能?“这基本上是一种很干净的方式,让这些智能能力变得更加可及。
Lenny Rachitsky: 我很喜欢的是,即使与 Linear 集成了,你还是可以保持特别简单——加一个小工单,写上”嘿,这个链接到这个主页,做这个”,Devin 就能很好地理解你的意思,然后展示给你看”我是这样想的”,对吗?
Scott Wu: 对对。好了,Devin 确实完成了工作。看起来 CI 那边有点问题,它正在调试,但它已经提交了第一版 pull request,我们可以看一下。
Lenny Rachitsky: 来看看。
Scott Wu: 这是 Devin 的网站,显然是在这个自定义部署里,然后我们把 Lenny’s Newsletter 放在了这里。
Lenny Rachitsky: 让我们把它发布到生产环境吧。不用再纠结了。
Scott Wu: 哈哈。
Lenny Rachitsky: 太厉害了。好,再快速演示一遍。就是把 Lenny’s Newsletter 加到了 Devin 的主页上。
Scott Wu: 对。Devin 显然有我们 Devin 代码库的访问权限,它在这里做了很多工作,所以对所有组件都非常熟悉。
Lenny Rachitsky: 漂亮。
Scott Wu: 嗯,我觉得看起来不错。这里有 Devin 搜索,有 Devin 的其他功能,还有 Lenny’s Newsletter。
Lenny Rachitsky: 你还链接到了我的网站。可以刷点 PageRank 了。
Scott Wu: 哈哈,对对。
Lenny Rachitsky: 好。这个例子不错吧?哦,出来了。我这 Newsletter 的网站真漂亮。这个是不是一个很好的例子,说明 Devin 特别擅长的那种事情——“网站上有一个非常具体的改动”?你们内部是怎么看待 Devin 擅长什么、以及它开始力不从心的地方的?
Devin 擅长什么
Scott Wu: 我们经常这样描述:Devin 在处理定义明确的任务时表现最好。一种说法是,你应该给 Devin 的是任务,而不是问题。刚才你看到的那些就属于这类——一个快速的前端功能需求、一个 bug 修复、添加测试和文档之类的。
闭环之所以好用,显然是因为有一个快速迭代和测试的途径。像这种场景,我们非常容易就能拉起预览页面,确认链接能正常工作。Devin 做到这一点当然也不在话下。Devin 甚至经常自己登录 Devin,然后启动一个 Devin 会话来确保在我们自己的代码库上工作正常,这其实挺搞笑的。但总的来说,核心就是——你应该给它那些容易验证、容易测试的东西。当然你也可以让它处理更大的项目或更大的需求,但在那种情况下,你肯定需要更多地引导 Devin,确保它在正确的方向上。
Lenny Rachitsky: 这很有意思,因为这和人们讨论合成数据和强化学习的方式非常相似——创建那些答案很明确的数据,就是是或否,非常清晰。
Scott Wu: 对。
Lenny Rachitsky: 非常清楚。
Scott Wu: 嗯。
设计 Devin 时的争论
Lenny Rachitsky: 好,我来问你一个问题。在设计和构建 Devin 的过程中,有什么是你们争论特别多的?
Scott Wu: 我说几个想到的。一个是关于我们应该有多强的主见性(opinionated)的问题。我们自己的工作流是将 Devin 集成到 Slack 和 GitHub,在我们的仓库里提交 pull request、响应 issue 报告之类的。自然而然地,用户也尝试了很多其他不同的用法。比如有人用 Devin 点 DoorDash 外卖,也有很多人从零开始搭建酷炫的网站,做类似的事情。
这对我们来说一直是一个有趣的权衡。我会这样描述:在我们的产品中,绝大部分功能确实是为提交 pull request、服务工程团队这个用例打造的。但对于其他所有用法,我们的基本态度是——如果用户想用 Devin 做那些事情,那很好,我们只是要确保他们充分了解局限性,了解哪些地方可能会卡住。
这其实挺有趣的,尤其是在 AI 领域。我会说最常见的一条创业建议就是:聚焦一个非常垂直的用户群,做不规模化的的事情,把一个用例做到极致,然后从这里扩展。我觉得这个建议在大多数情况下都很好。但在生成式 AI 这里,你会自然地看到很多产品体验天然就是更通用的。所以这确实是一个有趣的权衡,我们至今仍在反复讨论——要在多大程度上更多地支持所有那些其他用例,去处理用户可能想用 Devin 做的各种其他事情。
Scott Wu: 另一个想到的问题是,Devin 应该是一个综合性的项目体验,还是一套工具组合。正如你在这里看到的,我们有 Devin Search、Devin Wiki、Linear 工单自动归集,这些工具之间当然会互相配合,但随着时间推移,我们越来越把它看作是在构建一套工具组合。核心的 agent 体验——那个会自己去完成各种构建任务的核心 agent——当然始终是 Devin,这是我们产品最核心的部分,也将永远是我们工作中真正特别的地方。但除此之外的所有功能背后,是现实世界软件开发所要求的一整套复杂工作——工程说到底就是一件 messy 的事。
所以我认为有很多不同的工作流和用例都是合理的。一个显而易见的例子是,你可以向 Devin Search 提问和向 Devin 提问完全一样的问题,Devin 也会走同样的流程——逐个查看文件,然后给你一个回答。
但话虽如此,一方面在能力层面,确实有很多优化空间,可以针对”关于这个代码仓库的问答”这类场景做深度优化,这做成一个专门的功能是合理的。
另一方面,我们发现用户其实非常、非常喜欢拥有这种控制权。有时候你脑子里在想一个任务,但你其实还不想让 Devin 马上开始做。你想先问问 Devin,了解代码库中哪些部分可能相关。你希望能很明确地说,“这只是一个查询,我只想看到代码库中相关的代码片段”,或者”我只是想看一下 wiki,了解已有的文档”。
所以在能力和用户体验两个方面,我们都发现这种模式是随时间推移自然形成的最佳实践。
竞争格局与长期定位
Lenny Rachitsky: 那我们来聊聊这个领域中其他公司的格局吧,这是很多人一直在思考的问题。现在有各种不同的路径。你们是全力押注 AI 工程师,显然也有 IDE 公司,还有那些在工程能力上非常强的基础模型。现在大家都开始做 agent 了。你们在很多方面是走在前面的。OpenAI 最近刚说要做一个软件工程 agent,Anthropic 也有自己的方案。Cursor 和 Windsurf 有各自的 agent,还有 Replit。你们如何看待自己在格局中的位置,以及长期如何取胜?你怎么思考这些问题?
Scott Wu: 我想说,这些都是非常出色的团队,非常聪明、非常有前瞻性的人,在做很多很棒的产品。老实说,未来几年随着 AGI——或者你想叫它什么——的到来,有太多事情要做。我很喜欢的一句话是:2017 年你问我们有没有 AGI,答案是没有。2025 年你问我们有没有 AGI,答案是”那取决于你怎么定义 AGI,取决于你的领域”。我觉得这确实说明了一个问题——现在确实有非常多令人惊叹的事情在发生。我认为人们很容易低估我们正在经历的这场变革的规模。过去十年、二十年、三十年里,有很多很棒的产品,把构建产品的生命周期中每一个细分环节都变得稍微轻松了一些。
有做即时响应的出色产品,有做日志的出色产品,有做计费的出色产品,有各种不同的工具。而显而易见的是,我们在 AI 上看到的是,所有这些领域都将加速数倍,甚至可能是一个数量级的跃升。
所以从我们的角度来看,我们显然一直有一个非常明确的押注方向,那就是自主编程 agent。老实说这里面有大量问题要解决——核心能力上当然还有很多要做的,我们经常看到这样的情况:天哪,Devin 为什么会做出那个决定?这似乎没有任何人类工程师会那么做。产品界面方面也显然有很多需要思考的地方。
而且我要说的是,这不是我们在朝某一个单一终点努力,而是随着每一轮能力升级都会发生变化。我把它看作未来还有二十代 agent 产品、agent 编程体验要来。我认为几年后我们可能会到达这样一个阶段——你甚至根本不看代码了。你只是看着你自己的产品,然后直接指指点点说,“这个按钮应该再圆一点,来改一下。对了,在这里加一个新 tab,也许我们应该把这些信息存下来,建一个数据库表,按 X、Y、Z 列建索引。“你基本上就是实时地跟你的产品交互,让 agent 帮你把这些东西构建出来。当然,从现在到那个阶段之间还有很多代要做,但产品体验本身每次都会发生变化。
然后还有将所有这些实际推向世界的工作。用户需要学会使用新技术。要部署到现实世界软件的各种混乱环境中,有太多事情要做。世界上还有大量 COBOL 代码在跑,还有大量 FORTRAN 代码在跑,还有各种人们构建的抽象层和细节。
所以从我们的角度来看,我们从一开始就死死盯住 agentic 编程这一件事,这是我们真正相信的方向,也是我们为之设计的方向。这甚至延伸到了商业模式——ACU 和按用量计费的模式。当然也延伸到了所有产品体验的考量:你想在哪里跟 Devin 对话?你希望在 Slack 里跟 Devin 对话,你希望能从 issue 里直接启动 Devin,你希望能做所有这些事情,当然还有核心能力本身。
所以我认为没有一个简单的答案。这显然是多方面的结合,但这是我们一直深耕的领域,过去一年半我们所有的时间都花在了这里,未来五到十年也会如此。
护城河与可防御性
Lenny Rachitsky: 顺着这个思路,AI 领域一个大家都关心的大问题就是护城河和可防御性。这也是我每位来做播客的创始人都会问的问题。当构建变得如此容易,而且这么多东西都建立在那些本身就在飞速进步的模型之上,你怎么思考在这个领域建立护城河?
Scott Wu: 我对此会做一点微调,我认为与其说护城河,不如说更重要的是粘性。我的意思是,从某种意义上说,通常人们所说的护城河,是指一种能阻止竞争对手进入市场的壁垒。我同意,从高层来看,AI 频谱不同层级的很多不同参与者——无论是基础实验室还是应用层等等——我认为不存在任何硬性壁垒能阻止其他人进入。但我认为确实存在的是粘性,我把它定义为:一旦你有了一个你真正喜欢的产品体验,你是否愿意持续使用这个体验,还是说从现在开始切换到一个新产品、学习一个新产品同样容易。
我认为从这个角度来看,编程代理有几个特别好的地方。第一点是,它天然具有大量的粘性,以及随着时间推移的学习和积累。也就是说,随着你使用 Devin,随着你整个团队使用 Devin,这和工程师是一样的道理——你第一天入职和你在公司待了五年、自己写了一半的代码、碰过每一个文件、搭建了每一个组件、认识所有工程师,是完全不同的。类似地,Devin 会真正地学习并逐步构建起对你的代码库、你的技术栈、你的流程的理解,并在此基础上能做到的事情会多得多。
另一件我认为非常令人兴奋的事情是,代码领域确实存在一个我称之为多人协作的维度,如果你想一下,现实世界中很多事情都是这样完成的。所以,拥有你自己作为工程师使用的个人体验是一回事,但以我们自己为例,我们时时刻刻都在看到这种情况——一些工程师在跟 Devin 协作、教 Devin 一些东西,正如我提到的,人们会让 Devin 参与新工程师的入职培训,把知识传达给他们。
或者类似地,我会在 Slack 里跟 Devin 开始一个会话,说”嘿,如果我们能做这件事就太好了”。然后另一个工程师会插话说,“哦对了,当初我们这样做的原因是 X 和 Y。所以 Devin,你做这个改动的时候确保仍然支持那个工作流。” Devin 会说”好的,没问题”。或者 Devin 会提交一个 PR,我在跟 Devin 协作,我们在 GitHub 上创建 pull request,然后其他人会来 review 那个 PR 或者给一些评论,Devin 也会处理这些。你会在 Linear 里操作。
所以所有这些空间,确实为一种体验奠定了基础——基本上 Devin 随着时间推移能在你整个工作中提供越来越大的价值。因此我认为从这个角度来看,如果非要说什么的话,我们希望有大量的创新和大量的新产品出现。我认为目标不是试图阻止其他人来构建东西。有太多东西可以构建了,我认为会有很多不同的体验。从我们的角度来看,我们思考的更多是如何让 Devin 在你越用越多的过程中变得越来越好用、越来越有价值。
Lenny Rachitsky: 非常相似。我们请 Cursor 的 Michael 来做播客的时候,他也有类似的观点——他认为护城河更像是消费品的逻辑,就像 Google 那样,人们可以轻松切换。你只需要保持最好的,这就是答案。感觉你在这上面又加了一层,就是如果你能创造某种粘性,让人们很难离开,因为它做的事情太好了,已经积累了知识,深度整合到了你的工作流中,并在此基础上不断增强这种粘性。
Scott Wu: 对,我觉得我们这个领域还有一个很好的地方,就是软件工程,无论好坏,都与价值有非常清晰的关联。这意味着,一种说法是,至少在接下来一段时间里,始终有一个明确的下一个水平。我觉得可能会有那么一个时刻,你就像,“好吧,直接给我把整个 YouTube 搭建出来。“然后 Devin 就把整个事情做完了——要知道 YouTube 背后大概有一亿小时的人类工程时间,构建算法、构建所有基础设施,每一个细节。也许未来某个时候 Devin 能开箱即用地做到这些。那显然还有很长的路要走。
在此之间的每一个层面上,软件工程的质量显然是有差异的。我觉得对于开发者来说,一件很酷的事情是,如果意味着能获得越来越高质量的体验,开发者非常愿意学习新的工具、投入精力。
技术架构与模型
Lenny Rachitsky: 太好了。我想花点时间聊聊支撑 Devin 的技术。在不泄露商业机密的前提下,是什么让你们能把 Devin 做得这么好?是不是某个模型带来了突破?有些人分享过,3.5 上的三个点对他们很多产品来说是巨大的突破。你们架构或构建 Devin 的方式中,让它运作得如此出色的关键是什么?
Scott Wu: 显然,我们在 agent 上押注了很久。我认为 agent 比大多数人想象的要更早可行、更早能运作起来。但当然,随着整个社区真正地汇聚到这个方向上——你可以看到这在预训练中的影响,可以看到在这些模型所做的大量工作中的影响。从我们的角度来看,我并不认为有任何一次单一的、基于模型的阶跃式变化,或者任何一种对 Devin 来说是天壤之别的东西。但我确实认为曲线上每一个点——现在每周都有新模型出来——显然对我们能做到的事情产生了很大的影响。在此基础上,我们与所有基础实验室的研究团队合作,在它们之上做了大量工作。
所以我的一个热乎观点是:在基础智能方面,我们实际上基本上已经到位了。我认为我们看到的大量情况、以及我们花时间做的事情,不是提升模型的基础 IQ——毕竟我们不训练自己的模型——而更多是教它现实世界工程中所有的特异性和细节,比如:这里是你在 Datadog 里怎么做这件事的,这里是你可能怎么诊断这个错误的,这里是你可能遇到的各种情况,这里是每种情况怎么处理的,当你准备好了,这里是你怎么创建 GitHub PR 的。
这在工程领域是如此,在其他每个领域也是如此。我们日常所做的工作中显然有大量的细节和特异性。很多工作更像是让模型去映射现实世界的复杂性,而不是让它达到某种更高的基础问题解决能力——后者我认为基础实验室做得非常好。
Lenny Rachitsky: 在我们开始录音之前聊天的时候,你分享过一个观点——之前那些变革性技术的增长非常依赖硬件,存在限制其增长的瓶颈,而 AI 不是这样。你能分享一下这个洞见吗?
Scott Wu: 出于很多原因?我认为 AI 将是我们一生中最大的技术变革。但有一点,也就是我们刚才在录音前聊到的:过去 50 年我们经历的大多数重大技术革命,我是说个人电脑、互联网、移动电话等等,它们都有一个庞大的硬件组成部分,这是普及过程中很大的一部分。所以有了互联网,一开始只是这些大学在彼此通信,但显然随着时间的推移,我们让整个世界都连上了互联网,这花了无数年的时间。移动电话也是如此,个人电脑也是如此。
这其中特别有趣的一点,我想说我们已经看到了它的影响,在这些硬件普及机制中,显然有很多是依赖于时间的。因此,为那些行业打造产品的人,看着他们的市场随着拥有手机的人数增加、连上互联网的人数增加,基本上年复一年地稳定增长。许多这类企业,现在想起来依然觉得不可思议,但它们中的很多都是在最开始就成立了。我是说,苹果和微软几乎是在同一时期成立的。许多伟大的互联网企业也是如此,但毫无疑问,随着时间的推移,它触及了整个世界或世界的大部分地区。它产生了巨大的影响,但因为需要时间,这个过程持续了好几年。
AI 技术的爆发式增长
而我认为 AI 已经显现出的一个不同之处,就在于这项技术可以多么具有爆发力。一旦 AI 能够做到,我想我们在 AI 编程领域已经坚定地越过了拐点,作为一名工程师,如果你完全不用 AI 来写代码,说实话你正在落后。这是一项每个人都应该拥有并使用的技术,而且没有任何硬件普及上的负担在阻碍这一点。这意味着这个领域基本上正在以指数级增长。
Lenny Rachitsky: Michael Ballin 有个有趣的观点:陈词滥调之所以是陈词滥调,是因为它们太真实了,所以才成了陈词滥调。这话我听过一百万次了,我觉得人们听到这个时会想,“是啊是啊,我知道”,但正在发生的事情实际上太疯狂了。这就是为什么你在这里帮助我们度过这个转型期。
Scott Wu: 是的,我是说,这确实是个有趣的时期,我认为这需要真正的投入和真正的工作。但我认为,从我们作为工程师的角度来看,例如,这意味着跟上正在发生的一切是如此重要。正如我们所见,这不仅关乎你的学习能力和运用这些技术的能力,还基本上关乎教 AI 了解你的代码库,以便让它在与你共同构建时真正高效,做更多你希望它做的事情。
Devin 的企业落地与推行
Lenny Rachitsky: 沿着这个思路,对于听众中那些想“嘿,我们应该在公司里使用 Devin”的人,你发现哪些东西有助于帮助公司的工程师接受并能够使用 Devin,无论是在文化上还是实际操作上?
Scott Wu: 我们经常在大家身上看到的一种模式是,团队里会有几个人非常兴奋,想尝试新事物,他们愿意投入精力,并且非常激动能把它运转起来。他们会完成所有的设置。他们会把代码仓库给 Devin,教 Devin 如何运行 lint 和 CI 以及所有这些细节。他们会从给它那些初始任务开始,基本上帮助 Devin 建立立足点。随着时间的推移,最终大家会看到,“哇,Devin 写了所有这些 pull request,Devin 做了这个[听不清]……”
Lenny Rachitsky: “这个叫 Devin 的新员工是谁,刚进公司就这么疯狂地提交 pull request?”
Scott Wu: 是的。他们会看到这些,然后自然而然地就会加入并获取账号。当然,很酷的一点是,当他们加入时,Devin 已经知道了很多关于他们正在工作的代码仓库的细节,并且正在处理这些内容。所以我们经常看到的一件非常酷的事情是,早期采用者真的可以为团队里的其他所有人铺平道路。
但我想我要指出的主要一点是,这确实需要投入,这是一种非常不同的产品体验,而且我想不管怎样,我们仍然有很多事情可以做,让它对人们来说尽可能直观和清晰,比如如何使用 Devin,正确的步骤是什么,以及如何真正从 Devin 中最大化价值。但我认为,如果你投入精力并准确理解让 Devin 取得成功需要什么,我们发现随着时间的推移,随着每一次更新,我们只会越来越多地使用 Devin。
Lenny Rachitsky: 那让我顺着这个话题问下去。有一个问题我会问每一位构建 AI 应用的创始人,那就是,如果你能坐在每一位 Devin 新用户旁边,在他们耳边悄悄说些什么来帮助他们成功使用 Devin,一两句提示,你会说什么?
Scott Wu: 我想说的最重要的一点是,真的就把 Devin 当作你的新初级工程师。我认为这是最重要的事情。我认为人们进来后看到空白页面,会想到各种他们想尝试的东西。他们想得很多,而我们认为通常最有效的流程显然是你可以尝试演示,你可以做各种事情,但很大程度上就像这样:“好吧,让我们搞清楚今天或这周我们想完成哪些工单,然后让 Devin 从这些工单开始,我们从更简单的任务开始,然后与 Devin 合作,了解 Devin 需要设置什么东西才能测试自己的代码并把它做好。然后我们再随着时间推移逐步扩大规模。”显然,随着你和你的工程师一起工作,你会更好地理解如何与他们沟通,或者让他们参与哪些合适的任务或项目。但我认为这真的是我们的一句话总结。
管理众多 AI 工程师的未来
Lenny Rachitsky: 好的。有个问题我一直想问。我只想回到这一点,因为这是我对 Devin 思考很多的事情。每个人都会有五个 Devin,假设是十个 Devin。每个人基本上都变成了一个带着一堆初级工程师的工程经理,这未必是世界上最好的工作,因为这只是……至少你不用做绩效评估和一对一面谈,但就是整天坐在那里审查大量的 pull request。从某种意义上说,你变成了一个架构师,而这多少是每个工程师最终都想成为的,对吧?他们都在想,“我只想思考架构,我不想写这些愚蠢的代码、修这些 bug。”
所以我明白这有好的一面,但我猜你在这方面也想了很多,就是比如未来当你基本上成为 500 个 Devin 的工程经理时,如何让生活变得愉快、有趣和享受?
Scott Wu: 是的,我都能想象那个绩效面谈,“Devin,你的任务完成得非常出色,但我真的希望你在团队会议上更主动一些。”所以我想说的是——
Lenny Rachitsky: [听不清]
砖匠、建筑师与管理者
Scott Wu: 其实挺有趣的,因为我们在措辞上也有很多思考,我们过去用过“Devin 的管理者”这个词,当然我认为这是其中很重要的一部分。但我认为我在这里唯一想指出的是,砖匠与建筑师的对比比做管理者更接近真实体验。因为我认为管理的很多困难,或者人们回避它的原因,更多是因为许多不同的……比方说……包含了各种上下文、所有权、责任之类的东西,然后还有所有的情感层面。而在与 Devin 合作时,我觉得更像是仅仅作为一个……更多是拥有一个接口来交接任务和构建任务。所以,我想打的比方是,当我们发明 Python 时,显然……在很多方面,对任务的描述和概述,显然那是一个不同的范式,但我认为它绝对远不及人们今天通常认为的管理官僚主义。
Scott Wu: 我认为对于 Devin,很多时候就像是,更像是找到合适的抽象层级来与 Devin 协作,以及找到真正有效的工作流,这里显而易见的一点是,你总是可以让 Devin 先做第一版。所以你让 Devin 做第一版,如果很好,你立刻合并它,如果需要一些润色,你显然可以给出反馈,例如,但很大程度上,它更多是关于基本上让 Devin 成为你的工作流的一部分,而不是失去控制,我认为失去控制是人们对管理感到恐惧的主要原因。
Lenny Rachitsky: 你有考虑过一个管理者 Devin 吗?比如一个管理其他 Devin 的 Devin?
Scott Wu: 是的,是的。所以,顺便说一下,Devin 可以通过 API 启动其他 Devin,对吧?我们已经见过这种情况发生过很多次,自然而然地,如果你有一些想要完成的大任务,Devin 会一直这么做,它会分块然后……变成更小的 Devin。所以,这是一种你需要给 Devin 凭据才能做到的事情,目前它不是默认启用的,但我完全可以想象随着时间的推移,这样的情况会越来越多。
Lenny Rachitsky: 全都是 Devin 一路到底。
Scott Wu: 是的,是的。我认为还有一点很有趣,对于人类,我几乎会用技术术语来说,就像是存在上下文和线程的耦合,我的意思基本上是,每个人只能在他们所做的工作上单线程操作,他们有自己的一套上下文,然后显然有其他人可以同时做其他事情,但他们有自己的上下文。对于代理,很酷的一点是你可以让一个代理同时进行多条探索线,但共享它们发现的所有上下文,所以我认为这还处于非常早期的阶段,我想我们会看到这一点,但人们显然很喜欢谈论代理之间相互通信的系统,我认为一旦我们达到那个阶段,将会有很多新的范式需要去构建。
Lenny Rachitsky: 你刚才说的关于让一个 Devin 而且只有一个 Devin 做所有事情,你只需告诉它们事情然后它们就执行任务,对比你有五个 Devin,它们各自做各自的事情,这个决定真的非常有趣,这是一个非常有趣的决定。
Scott Wu: 是的,是的,当然。
Lenny Rachitsky: 好的,还有两个问题。也许是在构建 Devin 至今你学到的最反直觉的事情,可能是违背了创业常识、普遍的创业智慧?
创业中的反直觉
Scott Wu: 在我们构建这个产品的过程中,我最近思考了很多的一件事是,这不是我的第一家公司,实际上对我们很多人来说,这不是第一家公司,我想我们团队总共有 26 或 27 人,我想我们中有 18 人在此之前创办过自己的公司。是的,我思考的一件事是,你关于陈词滥调的观点真的让我产生了共鸣,也就是,你在创业中总是听到那些非常常见的事情,比如,你必须快速发展,或者你必须雇用优秀的人才,就像,好吧,显然你会这么做,我没打算不雇用优秀的人才,我也没打算走得很慢。类似地,就像,是的,你确实必须构建人们想要的东西。总是被重复的就有这么三到五件事,它们总是创业中的常识。
Scott Wu: 作为创始人,当我最初起步时,我确实有这样的想法,好吧,那三到五件事是基础,但当你真正深入其中,花了多年时间,你会学到建立公司必须学会的成千上万的其他事情。我认为在某种程度上,这当然是正确的,在所有这些不同的事情中会有很多小细节,包括团队建设、产品、战略、工程决策、融资、销售以及所有其他组成部分。但我也意识到,随着时间的推移,我越来越觉得,把公司建设好有时仅仅归结为把那三到五件事做到甚至超出你所能想象的程度。
Scott Wu: 所以,对我们来说,就像……每个人都说我们走得快,但事实是,我们在 11 月办了一场黑客松,12 月又办了一场,1 月正式创立公司,2 月向初始用户提供原型,3 月发布产品,4 月获得第一批客户……就像,基本上在每一个我们能推动的地方真正地加快节奏,这对我们产生了真正的影响。类似地,就像,是的,每个人总是说你应该雇用优秀的人才,但我认为那个真理背后的真相基本上是你应该不惜一切代价去争取,来得到你真正想招进来的人。我最喜欢分享的故事之一是,我们有一个候选人来面试,他是 MIT 的大三学生,所以他非常非常年轻,我们给他做了面试,他比我们聊过的几乎任何全职候选人都要好得多。所以我们就说,嘿,你觉得休学一段时间来和我们一起工作、构建 Devin 怎么样?
Scott Wu: 我们真的认为你一进来就能从第一天开始产生巨大的影响。他想了一阵子,然后回来对我们说,你知道吗?我愿意,我想做,但我父母真的很希望我能从学校毕业,我只是不确定有没有办法行得通。所以,我们跟他多聊了聊,了解了情况,然后我们飞到了北卡罗来纳州,从机场直接去他父母家,和他还有他父母一起吃了晚餐,我们聊了很多,他们是一个非常好的 Gujarati 家庭,我们送了一些礼物,就跟他们谈了这件事,试图了解需要什么条件,我们需要怎么做才能成。而他们只是说,听起来是个很好的机会,但我们真的希望我们的儿子能够毕业。
Scott Wu: 我们就此深入沟通,基本上想出了一个安排,他可以为我们全职工作,但需要去上必修课,完成拿到文凭所需做的事情,但仅此而已。我们把这个谈妥了,最终达到了大家都很满意的状态,然后我直接回到机场飞了回来。那是我第一次也是唯一一次去北卡罗来纳州,这是一次非常棒的旅行。这就是那种事情,雇用优秀的人才是一回事,但真正不放弃,为了那些真正适合加入团队的人倾尽全力去促成,又是另一回事。他加入我们团队已经一年多了,一直是一位不可思议的出色工程师,如果没有他,我们不会走到今天。
Scott Wu: 类似地,我们还有另一位非常非常有才华的候选人,表现得极其出色,也非常年轻,并且拿到了很多其他公司很棒的录用通知。我们和他聊过,他希望有一天自己也能创业,我们和他探讨了许多显而易见的事情,比如让他见我们的投资人,或者让他参与客户工作,或者让他了解许多其他环节,这样当时机成熟时,他就会拥有创业所需的所有经验。但另一件很重要的事情是,他已经在和很多优秀的公司交谈,他不想过河拆桥,所以我们实际上和他一起,亲手写了给其他每一家公司的拒绝回复,并和他一起打磨,告诉他你应该这样说,才能表现出你真的很感激与他们共度的时间,而且显然你希望与他们保持亲近和联系。显然,这就像是,我们的工作当然是确保他足够开心,以至于在不久的将来任何时候都不想离开,但我认为,组建一支真正伟大的团队的方式,也是为对他们有利的事情而战。
Lenny Rachitsky: 哇,这些都是令人难以置信的故事。这让你说的那些陈词滥调变得如此真实,雇用最优秀的人,这就是雇用最优秀的人听起来样子。这就是必须付出的代价。
Scott Wu: 是的,我想说的还有,我们在很多事情上都非常努力地从头重新构想,因为其中很大一部分确实只是在思考,我们认为未来 5 到 10 年技术会走向何方,以及在那个未来我们想占据什么位置?
对 AI 的乐观展望
Lenny Rachitsky: 我想知道将来人们会不会去争夺最优秀的 Devin。他们会直接[听不清] Devins。
Scott Wu: 是的。
Lenny Rachitsky: 他们[听不清]太聪明了。
Scott Wu: 我会给你加班费、福利、免费医保等等一切,然后 Devin 就像……
Lenny Rachitsky: 三个 Devin,就像《万智牌》卡牌一样。
Scott Wu: 是的。
Lenny Rachitsky: 然后再回到你说的那三到五件事上。基本上,这是一个令人难以置信的建议,就像你总是听到的那样,雇用最优秀的人,快速行动,构建人们想要的东西。
Scott Wu: 是的,构建人们想要的东西,尽可能贴近你的客户,然后我认为另一件事就是永远思考事情的发展方向,而不是它们今天的样子。我觉得这就是那五件事……特别是在 AI 领域,事情发展得如此之快,又有那么多优秀的人才,我觉得这些道理更加真实,这不仅仅是思考 10 年后事情会怎样,而是思考下周会发生什么。显然,事情发展得非常快,而且很难预测,但我得说,你真的必须对自己非常严格,去彻底思考这些事情,并在这个视角下评估你做出的所有决定。
Lenny Rachitsky: 保持专注是我在这里得到的最大收获,最终你会觉得有 1000 件事你应该做,但其实总是这五件事。
Scott Wu: 是的。
Lenny Rachitsky: Scott,我们涵盖了很多内容。我们问过了我所有的疑问,这很棒,你还有什么想分享的吗?还有什么想留给听众的,也许是一个最后的金句,或者你真正想重申的我们在节目结束前说过的话?
Scott Wu: 我想到的最重要的一点是,人们对 AI 有很多不同的看法,对吧?我觉得现在基本上涵盖了所有的情绪。比如有很多恐惧,也有很多怀疑,我们自己也是非常多疑的人,我们总是想亲自尝试,真正看到并相信它。我想到的主要事情是,说实话,我对我们正在用 AI 构建的东西非常乐观,不仅仅是在代码和 Devin 方面,而是整个领域,以及正在完成的所有工作。我认为真正正在发生的一件很酷的事情就是每个人都有能力成倍地复制自己,这也是我们一直以来的思考方式,是我们思考我们正在构建的东西的方式。我认为世界上还有很多事情要做,我不太担心我们会无事可做,从这个角度来看,我认为我们一直以来最兴奋的事情就是,我们怎样才能做得更多?
Lenny Rachitsky: 我听懂你的意思了,Scott。好了,带着这份乐观,我们进入了非常激动人心的快问快答环节。你准备好了吗?
Scott Wu: 好的,开始吧。
快问快答环节
Lenny Rachitsky: 好的,开始吧。第一个问题,有两三本你会发现自己最常向别人推荐的书吗?
Scott Wu: 在非虚构类方面,我认为对于初创公司的人来说,我真正喜欢的一件事就是学习和了解硅谷的历史。我们思考的所有这些东西,都是有人发明了它们。这是我觉得最伟大的认知之一,有人发明了种子轮融资的概念,有人发明了风险投资的概念,有人发明了产品市场契合度(product-market fit)的概念,以及我们谈论的所有这些不同的原则。因此,关于这一点,有一本 Sebastian Mallaby 写的叫《风险投资史》(The Power Law)的书,我非常喜欢。基本上,它就是对过去 60、70 年硅谷建立的许多伟大企业和伟大产品的巡礼,我真的非常喜欢。在虚构类方面,其实我一直很喜欢 F. Scott Fitzgerald 的《了不起的盖茨比》(The Great Gatsby),这是我个人最喜欢的小说之一。
Lenny Rachitsky: 你最近有特别喜欢的电影或电视剧吗?
Scott Wu: 我得承认,我没看过……我想不起最近一段时间我看过哪一部电影或电视剧,所以我确信,我期待在实现 AGI 之后去看很多好作品。
Lenny Rachitsky: 这句话一定要放进预告片里,太棒了,我喜欢。这也说明你工作有多努力,事情有多繁杂,一切发展得有多快。你最近有没有发现并真正喜欢的最爱产品?可以是应用,可以是实体物品,可以是一把牙刷。
最爱的产品
Scott Wu: 我想说的一个是,我最近买了一个 Aura 相框,它就像一个显示照片的相框,你可以每天、每小时或者每15分钟展示一张新照片,或者任何你喜欢的频率。我其实非常喜欢它,我觉得这是一种让回忆像照片一样浮现出来的好方式。另外我想说的另一个更通用的东西,它不是特别新,但我觉得 AirPods 的做工和设计都极其出色。我现在意识到,我基本上在所有……散步时接电话,我用 AirPods;显然,我在电脑前、办公桌前工作,我也插着 AirPods。说实话,它在很多不同场景下都表现得很好,而且非常舒适,非常稳定。
Lenny Rachitsky: 我要强烈附议 Aura 相框,我也给我妈妈和岳母各买了一个,用来和家人分享你孩子的照片简直太棒了。人们听说过数码相框,但 Aura 做得特别好,添加照片非常容易,而且外观非常精美。
Scott Wu: 你可以想象在不久的将来,我们会有一个 Aura 相框,只不过它会把你里面的每一张照片都吉卜力化(Studio Ghiblifies),然后它就……是的。
Lenny Rachitsky: 或者直接想象出你做过的非常酷的事情。看看我多美好的生活。是的。酷。它的名字是 Aura,拼写是 A-U-R-A。想要了解的朋友,我们会放上链接。非利益相关。好的,还有两个问题。你有没有最喜欢的、经常回想并在工作或生活中觉得有用的座右铭?
人生座右铭
Scott Wu: 是的,我思考了很多的一点是,现有的很多格言实际上是相互矛盾的,对吧?比如物以类聚,然后也有异性相吸,还有很多……这有点好笑,因为你会觉得它们都是对的,而且往往它们确实都是对的,很大程度上在于理解为什么。其中一条我特别有感触的,尤其是在我一直在思考的创业领域,我认为保持专注和内驱力、真正发挥你的最大潜能非常重要,同时,不让个人的情绪与你的成功或失败捆绑在一起也非常重要。我认为尤其是在创业中,因为总会有起起落落,老实说,即使是有史以来最成功的公司,也是一条坎坷的道路,会发生很多事情,也会走很多下坡路。
我思考很多的一件事是,无论如何你真的想做到最好,把你能做到的一切都投入进去,尽你所能去……基本上,你想在场上倾尽全力。但同时,你也想坦然面对胜负,能够继续前进,每次都进入下一个阶段。这很有趣,但我个人发现,显然这对你的情绪状态和心理状态来说非常重要,我们犯过很多错,我也犯过很多错。我创办的第一家公司,显然,它很酷,但那里也有很多棘手之处,然后在 Cognition 的过程中,感觉就像是把八年的时间压缩成了一年,而且现在还在以那种节奏继续。但不知怎么的,这实际上也会让你更成功,我觉得。就像,如果你不把它与你的个人价值捆绑在一起,你反而更能倾尽全力,去做那些会带来成功的事情。
Lenny Rachitsky: 这太有趣了,我最近刚录制了一期播客,嘉宾是一位高管教练 Jerry Kelowna,我想那期会在本期之前播出,也可能在本期之后,那是他的一条重要建议,而且这是一种非常佛学的方式,就是[听不清]以及执着于某个特定结果。
Scott Wu: 是的。
Devin 名字的由来
Lenny Rachitsky: 好的。最后一个问题,我很好奇这里有没有什么故事,但我们可以简短点,Devin 这个名字背后有什么故事吗,或者有没有其他候选名字差点成为 [听不清]?
Scott Wu: Devin 这个名字很早就有了,我们很感兴趣,我们一开始就在做编程智能体,比如我的联合创始人是 Steven 和 Walden,我们有这个想法,好吧,我们开始吧,让我们尽可能地扩展边界。所以,让大家跳出框框思考,做自己的事情,让大家先各自做一段时间,然后我们再整合,吸取我们学到的所有东西。所以,Walden 做了一个他自己的虚拟开发者版本,叫 DevWalden,然后 Steven 也做了一个他的版本,叫 DevSteven,我们有了所有这些……然后我们把它全部合并成一个东西,我们就说,好吧,知道吗?就是 Devin 了。就是这么来的。所以 Devin,是的,Devin 很早就被我们确定下来了,我想说。
不过我们确实在一个问题上做了很大的决定,那就是 Devin 的形象是什么。所以,正如大家所知,有六边形,然后人们最近也看到了这个,但实际上还有一只水獭,一只膝盖上放着笔记本电脑的小水獭,那也是 Devin。我们曾经争论过用什么,不用什么等等,现在已经过了一段时间了,但不知怎么的,我们仍然保留了六边形和水獭。
Lenny Rachitsky: 你跳过了 Devin 的来源,你们就是突然想到的吗?
Scott Wu: 噢,所以 Devin 就是,它是一个 dev。是的。
Lenny Rachitsky: [听不清]
Scott Wu: 所以,就像,当我们整合所有名字的时候,当时就很清楚了,这会是我们都喜欢与之合作的通用 dev。
Lenny Rachitsky: 哇哦。
Scott Wu: 是的。
Lenny Rachitsky: 太不可思议了。Scott,这次聊天太开心了。我的天,我学到了超多,这总是一个非常好的迹象。最后两个问题,大家可以在哪里找到你、Devin 或其他你想推荐的东西?听众怎样才能帮到你?
Scott Wu: 太棒了。是的,我们在 App.devin.ai,你也可以在 Twitter 或很多其他社交媒体上找到我们。显然,我们非常希望听到你对 Devin 产品的任何反馈,还有太多东西需要弄清楚,而且我认为,就像我说的,我认为我们距离软件工程的真正未来还有 20 步之遥,所以在大家试用产品时,听到大家的想法对我们来说意义重大。所以,如果有什么我们可以做得更好的地方,请随时让我们知道。
Lenny Rachitsky: Scott,非常感谢你来到这里。
Scott Wu: 非常感谢你的邀请。我度过了一段美好的时光。
Lenny Rachitsky: 我也是。大家再见。
非常感谢你的收听,如果你觉得这期节目有价值,你可以在 Apple Podcasts、Spotify 或你最喜欢的播客应用上订阅本节目。另外,请考虑给我们评分或留下评论,因为这真的能帮助其他听众找到这个播客。你可以在 LennysPodcast.com 找到所有往期节目或了解更多关于本节目的信息。下期见。
术语表
| 原文 | 中文 |
|---|---|
| ACU | ACU(Autonomous Coding Unit,Devin 的计费单位,保留原文) |
| agentic workflow | agentic 工作流(自主代理式工作流) |
| assembly | 汇编语言 |
| Aura frame | Aura 相框 |
| base intelligence | 基础智能 |
| boilerplate code | 样板代码 |
| CI | CI(Continuous Integration,持续集成,保留原文) |
| Claire Vo | Claire Vo(人名,保留原文) |
| COBOL | COBOL(编程语言名,保留原文) |
| Cursor | Cursor(公司名,保留原文) |
| Datadog | Datadog(监控服务平台名,保留原文) |
| DeepSeek | DeepSeek(AI 公司名,保留原文) |
| defensibility | 可防御性 |
| Devin | Devin(Cognition 公司开发的 AI 软件工程师产品名,保留原文) |
| DoorDash | DoorDash(美国外卖配送平台名,保留原文) |
| feature flag | feature flag(功能开关/特性标志,软件开发中的功能发布控制机制,保留原文) |
| fork | fork(软件开发中的分支副本,保留原文) |
| FORTRAN | FORTRAN(编程语言名,保留原文) |
| GitHub Copilot | GitHub Copilot(产品名,保留原文) |
| idiosyncrasy | 特异性 |
| imitation learning | 模仿学习 |
| jagged intelligence | 锯齿状智能 |
| Jerry Kelowna | Jerry Kelowna |
| Jevons Paradox | 杰文斯悖论(经济学概念,首次出现保留原文并附中文译法) |
| LaunchDarkly | LaunchDarkly(功能管理平台公司名,保留原文) |
| Linear | Linear(项目管理工具产品名,保留原文) |
| lint | lint(代码静态检查工具,保留原文) |
| Lunchclub | Lunchclub(公司名,保留原文) |
| Michael Ballin | Michael Ballin(人名,保留原文) |
| moat | 护城河(商业竞争中的防御优势) |
| multiplayer aspect | 多人协作维度 |
| Neuro | Neuro(公司名,保留原文) |
| opinionated | opinionated(产品设计术语,指产品对使用方式有强立场/偏好,保留原文) |
| PageRank | PageRank(Google 网页排名算法,保留原文) |
| pivoted | pivoted(创业术语,指战略方向转型,保留原文) |
| product-market fit | 产品市场契合度 |
| pull request | pull request(软件开发中的合并请求,保留原文) |
| reinforcement learning | 强化学习 |
| Replit | Replit(在线编程平台名,保留原文) |
| Scale AI | Scale AI(公司名,保留原文) |
| seed route | 种子轮融资(口语中实为 seed round,原文转写为 route) |
| Slack | Slack(团队协作工具产品名,保留原文) |
| speculative decoding | speculative decoding(推测解码,一种大模型推理加速技术,首次出现保留原文) |
| Staff 级工程师 | Staff 级工程师(Staff Engineer,高级别工程师职级) |
| step function | 阶跃式变化 |
| Steven | Steven(人名,保留原文) |
| stickiness | 粘性 |
| Studio Ghiblifies | 吉卜力化(Studio Ghiblifies) |
| TCP/IP | TCP/IP(网络协议名,保留原文) |
| ticket | 工单(软件开发中的任务/工单) |
| Walden | Walden(人名,保留原文) |
| Waymo | Waymo(公司名,保留原文) |
| wiki | wiki(知识库/维基,保留原文) |
| Windsurf | Windsurf(AI 编程工具产品名,保留原文) |
| YC | YC(Y Combinator 缩写,保留原文) |
此文档由 AI 分片翻译(translate_long_document)