“All your Coding Jobs are Belong to us, Devin probably…

Cognition Labs dropped one of the biggest updates of 2024 with their First AI Software Engineer, Devin! You’ll notice that we use the word ‘Coding’ instead of ‘Engineer’ even though Devin is touted to be the first AI Engineer. And that’s primarily because Engineers are actually problem solvers that use any and all tools to solve a problem. But more importantly Engineers need to solve human problems that require humans to interact with each other to figure out the problem in the first place. And in our experience there’s still a gap between a perceived problem (that happens to be the task for a lot of engineers) and the real problem (which is what they eventually have to figure out.)

So with that out of the way, let’s dive in to what Devin can actually do – and he can do quite a bit!

A real world AI Software Engineer

Devin basically operates much like an engineer would – has its own command line, code editor. It also debugs its own software. On the SWE-Bench benchmark, which evaluates an AI on their ability to accurately solve GitHub issues in the real world, Devin blows everyone else out of the water with an accuracy of 13.86% – which is ~3x the next best LLM, Claude 2. (We need to verify if Claude 3 is better)

Devin can actually learn

Devin is really good at figuring out new stuff. As demonstrated by the video below.

The fact that it can follow written instructions, go through documentation and figure stuff out is actually what makes us doubt our intro paragraph. But we’re going to stick with it for now.

Devin can contribute to mature repositories

When working in a live environment and trying to fix large repos, one can be expected to spend quite a few sleepless nights debugging and figuring things out. Not anymore. Thanks to Devin

Devin can train its own AI models

This one is a little bit like if I told you my pet dog had its own pet fish. And while that’s definitely possible, it’s still a little odd. We’ll just let you take a look at the video to get what we’re saying.

Devin is already stealing your jobs

This one is our favourite. Devin is already doing live projects on Upwork. What better way to introduce Devin to the world than let him loose on gig workers.

We are certainly bullish about what this means for the future of work. We genuinely believe that Devin won’t replace engineers, rather enhance and partner them to enable them to do more. However, we are convinced that grunt-working coders are rather in hot soup. Or maybe not. In either case, Cognition will have to be careful with how they proceed as releasing Devin into the real world is likely to have far-reaching consequences.

Bonus: Scott-Wu proving why he’s the boss!

Cognition Labs has already raised a USD 21mn Series-A round. Things are only just getting started for them. Even their job postings are lit🔥as witnessed below:

Our team is small and talent-dense. Among our founding team, we have world-class competitive programmers, former founders, and leaders from companies at the cutting edge of AI including Cursor, Scale AI, Lunchclub, Modal, Google DeepMind, Waymo, and Nuro.

https://jobs.ashbyhq.com/cognition/4841bda0-057a-4471-801f-70309c3c02d5

We’re already big fans and will be tracking this very closely.

PS: All video credits to Cognition Labs