Making Code Explain Itself – Observability Through AI

Transcript

Shane Hastie: Good day, folks. This is Shane Hastie for the InfoQ Engineering Culture Podcast. Today I'm sitting down with Dr. Elizabeth Lawler. Elizabeth is the founder and CEO of AppMap and is passionate about observability. Elizabeth, welcome. Thanks for taking the time to talk to us today.

Elizabeth Lawler: Thank you so much for having me on the podcast.

Shane Hastie: The way I like to start is, who's Elizabeth?

Introductions [01:15]

Elizabeth Lawler: Well, I am a programmer and data scientist by training, and I am a serial entrepreneur who's been working in building developer tools for the last, whoa golly ,now over a decade. First in the cybersecurity space and now in the observability space. And I love building tools for developers that solve white space problems that they face on a day-to-day basis.

Shane Hastie: Problems that developers face on a day-to-day basis, what are they?

Problems that developers face on a day-to-day basis [01:43]

Elizabeth Lawler: Well, a lot of time, I'll say my last company solved a very tactical problem, which was that we needed to get secrets out of plain text and source control and API keys into cloud environments and to secure DevOps pipelines. And that was a very concrete problem. I need to get keys out of here, vault them, deliver them, scale them to a million secrets delivered a minute across the cloud infrastructure. But this AppMap one is a little harder problem, a little bigger problem space, but I think it's really interesting.

The challenge of communicating design principles [02:10]

We started with the premise that one of the most challenging things we do as engineering teams is to try and communicate to one another issues of design and design principles, which is that we've all sat in a room with whiteboard, or sat on a Zoom and tried to explain code to one another. And we kind of sat back and said, what if the code could just explain itself? Wouldn't that be nice? Because it's the only thing that really knows how it's built and how it works when it runs. And so that led us into working in the observability space to try and figure out ways to surface more context for developers as they were writing code and more context for development teams as they were trying to communicate software design and software behavior to one another using really the most declarative interpreter of how the software works, which is the software itself.

Shane Hastie: But we've now got these magical AI tools that we can take that code base, throw it into an LLM and say, "Tell me what it does?"

Elizabeth Lawler: Yeah, I've been a huge fan of AI for several years now. I predicted in 2021, but I was probably a couple of years early that software would be writing and editing code sooner maybe than it actually came to mark. But it's exploded and I think it's a truly transformational tool. Generative AI definitely improves developer happiness and takes some of the tedium out of writing code. But at the same time, I think there are some potential limitations for people who've worked with generative AI as a coding assistant have experienced.

And if you think about it, we're all kind of working off of the same dataset, right? It's the generalized knowledge that LLMs have created around software and software design. And so when it gives recommendations, those recommendations are accepted around 25 to 30% of the time. But to really solve some of the problems which are most vexing, right, you need much more personalized information. You need deeper insights and general AI tools haven't quite met the mark in that regard. And I think that we're starting now to see the edges of what the real productivity gains that can be had for generative AI tools.

Shane Hastie: You were talking before we started about studies and so forth, about half an hour a day. That doesn't seem very much.

Generative AI is only saving around ½ hour per day of coding activity [04:16]

Elizabeth Lawler: Well, so yeah, the real studies out say that generative AI tools can improve developer productivity about 10%, or about half an hour per day. So it helps you code about 50% faster, but developers really only spend about an hour to 80/84 minutes a day coding. The majority of software development time is actually spent in problem-solving. So that's debugging, test case analysis, trying to get broken test cases to work, communicating, documenting, asking for help, doing pull request reviews. So really there's a whole swath of developer enablement that can't be addressed through generalized AI models, right? You can't take the general Internet's knowledge and help accelerate those types of activities. What you need is something that's much more personalized, much more attuned to the actual information, data and activity that a developer is working on in that context, but it's much more context typically than can be provided to the model. And then furthermore, when you think about what AI coding assistance that could have additional context, say from about what the developer is working on, how the software runs, etc. Then you can start to get into some new insights and new models.

Shane Hastie: And what would that look like?

Adding context to the AI tooling [05:27]

Elizabeth Lawler: Well, really I think that if you think about what developers are spending a lot of their time on, so think about the life cycle of a ticket, right? Your ticket comes in, I have to onboard to the problem. I need to understand where in the code base that new feature or that bug I'm going to have to make code changes. If you look at the way that AIs work today, they're very much structured across looking at code within a particular file, or repo, or code area because the amount of information that you can send up to the generalized AI is limited by its token limit. And so what we're looking to do now is to try and figure out ways of augmenting that so that there's more context, there's more data associated with that prompt going out to the AI, or coming back and then providing much more tailored experiences.

It's really interesting because if you think about it, every single developer is working on their own version of the code, right? There isn't one common version of the code. Every branch that someone's got locally on their laptop is their own interactive thought process of development. And it has its own context. It has its own localized behavior that's different than the behavior of the last release, which is different than the recommendations from the generalized AI. And so when you think about how one can try to optimize for enhancing the creativity within that creative moment, you really need to create a much more personalized, tailored experience to help that developer get to the next level of productivity.

Shane Hastie: As a developer, what does this mean to me?

Developer happiness increases as time spent coding increases [06:56]

Elizabeth Lawler: Well, I think really the question is if you look at the good days bad days paper from Microsoft where they talked about how developer metrics of happiness increase with more coding time, being able to keep people in coding and inflow and less going out for information to APMs or other sorts of information sources in order to be able to do that coding work. I think that's really what the goal is. And although coding assistants have gotten us an extra half an hour of productivity, I think bringing observability left, shifting some of that other context left can help us get even more time, potentially up to 50% more time out of the problem-solving realm and into the code flow realm. Right now, you're just creating as opposed to searching for information, searching for context, searching for ways of figuring out how to problem solve, and now you're in that creative flow of building. And I think that's really the goal that all these tools strive to achieve.

Shane Hastie: Moving us into that flow state of being able to focus on the code.

Elizabeth Lawler: Totally. And I think we have the opportunity when you think about how developers can potentially use these tools to not only educate them about how systems work. It's interesting, I learned even by using AI tools now, like ChatGPT and other things about issues of frameworks that I didn't really know about, right? I can ask a question and it's always got a deeper knowledge base and a more instantaneous ability to provide me with that kind of research and insight that I would be able to get on my own. So in many ways, that's really taken a lot of the pain out of trying to find answers, but at the same time, it doesn't necessarily help me do the implementation step. And I think that's really where code completion and code block suggestion versus the way we think about design pattern, the way we think about architecture, the way we think about structure, and working within an existing set of architectural considerations and constraints. That's really where the rub is right now in generative AI.

Shane Hastie: And stepping away from generative AI, it's one of the things that are helping us, but what are the other big developer experience challenges and how do we address that?

Developer experience challenges [09:02]

Elizabeth Lawler: Well, I think that one of the movements that's been a fit for several years now is Shift Left bringing more information or more responsibility onto development teams in order to be able to make sure they're taking consideration for performance, security, maintainability issues of scalability. But I think that's also created a lot of burden, and certainly, there's been a lot of system and accounts sprawl associated with that. So when we think about trying to improve developer experience, we also need to think about how to create the right kinds of guardrails versus overwhelm, right? As you're trying to deliver a new feature function. Yes, you're trying to do that with the best possible design and implementation as possible, but you can't also constantly inundated with bells and alarms and things like that over things that may, or may not be problems that may, or may not impact these issues downstream.

And so I think that one of the things we need to think about is how we're going to create some harmony around that. I know certainly alert fatigue is something that people get a lot of when it comes to things like static code analysis for security issues. You'll get alerts over things that perhaps were only going to be implemented in the testing environment and will be dropped when the application moves to production. And those kinds of things can be frustrating, and then you end up losing, I think the hearts and minds of some of the people that you're trying to help educate along the way.

Shane Hastie: If we can switch topics somewhat, quite a lot, your background, technologist through to founder and CEO along that way, I'm sure that you've learned a few lessons.

Elizabeth Lawler: Oh yeah, sure.

Shane Hastie: A fair number of our audience will be relatively new to the leadership roles, technical influences, stepping into leadership roles. What advice would you have for them?

Creating a supportive educational environment [10:47]

Elizabeth Lawler: Oh, I think that's a great question. One of the things I think that served me well doing that function first, rising up to leading teams and then ultimately starting my own group, my own firm, is that there's a lot of education that goes along with that role. Your job is to bring along others, both to encourage them to stand tall and to take chances technically, but also to put their most creative potentially vulnerable ideas out there at the same time. And so I think that creating a supportive educational environment is one of the best ways to get the most out of young professionals who are trying to grow. And one of the things that I love doing is working with people who are just out of coding school, who are students, and who are taking those first steps along that journey so they can see what a strong, intellectually challenging, but also supportive environment can be like.

Shane Hastie: So what does that environment look like?

Elizabeth Lawler: I think it's a high amount of communication. We definitely tend to skew toward written communication because I think it gives people time to vet their own ideas before implementing them, or before taking that next step. But generally speaking, I think one of the things that we encourage is experimentation, right?

Shane Hastie: I want to say that we encourage experimentation, we talk about making things safe to fail, learn from your mistakes. And yet the culture I see in many organizations is almost contrary to that.

Elizabeth Lawler: Yes, well, that's why they keep me in the startups, as opposed to letting me go work in larger organizations because fundamentally, I believe in creativity. And I think that when developers are at their best is when they're most in that creative flow, and you get the most innovative ideas. Now, that being said, you have to work within certain architectural or technical constraints. Not everybody can pick whatever language they want off of the truck, but I do think people should be encouraged to design things in creative ways. And so yeah, obviously I'm probably biased because I prefer to work in startups which have a lot more liberal coding environments. That being said, I did spend 11 years working in the government, which had the opposite kind of development environment, which was highly structured, very regulated, very well documented before you could implement anything. It was perhaps, it's like the opposite experience.

Shane Hastie: As a leader, how do I shift from one to the other, how do I nudge the culture of my organization

Enabling a creative culture [13:11]

Elizabeth Lawler: More toward more creativity? Oh, I wish I had the answer to that question. I think I probably go and start a competitive consulting firm to McKinsey. But I think it really comes down to what small teams are able to do, and the less bureaucracy and hierarchy you can put into those decision-making groups, I think the better. But yeah, I think ultimately it's a pendulum and you see different periods of time. I think there were periods of time where people were much more encouraged to take chances and move back to some bright things. And then there are periods of time when people are encouraged to be more conservative. If you look at the current mode, there's a strong sense of do more with less, right? We're looking for productivity advancements, but without having to make investments in more people. If you look at the layoffs of the last couple of years, there hasn't been a real rehiring amongst those companies that had 5, 10, 20% layoffs. There's been a contraction, but the market still has incredible demand for developer talent. But I think the pressure for productivity is higher probably than it has ever been before.

Shane Hastie: What's the implication of that for us?

The most valuable contribution of developers is in their creativity [14:14]

Elizabeth Lawler: Well, I mean, I think that's driving forward faster some of these augmenting technologies. What is the most valuable contribution that you can get from a developer? I might argue that I think it is that creativity and that ability to create new things rather than maintain, or debug or factor existing things. So you see those types of applications of some of these technologies coming to before, right? Refactoring is actually an area where generative AI works very well. So I think some of the tedium potentially might go away and some of the maintenance and stasis might go away. And then you'll see things like more of the creative becoming the domain of the developer and then looking for guidance from AI assistants on how they can implement that faster.

Shane Hastie: How do I teach that creative thinking?

Elizabeth Lawler: That's a great question. I've been reading that Rick Rubin book, I don't know if you happen to be reading it, The Creative Act, it's by a producer of music, but he democratizes the notion of creativity to all walks of life. And he talks about how if you're working on something for which you might be the audience, or you might have empathy for the audience, you first can start to have develop a creative sense and a sense of direction and a sense of what sounds right in his parlance. And I think if you can start to give people the confidence for having such a point of view about the way that they're building their software, about the way that they're writing their code, about the way they're architecting things, I think that can be a very strong motivator and contributor to both confidence and creativity.

Many products I've worked on have had contrarian approaches to building them. My current product started life as a fully local, in your code editor on-prem code editor extension, not a SAAS, which was already like an architectural. I was already going against the norm with my architectural choices in that regard. And so I think that there's an opportunity, if you can give people confidence to kind of take those sort of chances, then you get to see the benefits of those design choices down the line.

Shane Hastie: The blue sky question, where is our industry heading?

Where is our industry heading? [16:21]

Elizabeth Lawler: That is the blue sky question, right, which is that some will have, you think that all the developers will be out of jobs and that AIs will be taking over. But I don't think that's necessarily the case. I think what we'll find is actually that we'll have built better designers as a result of some of the new technologies that are coming online. Because if you think about how some of the newest generative AI coding assistants work, it really requires you to be a better editor, a better critic, a better critical thinker about quality as opposed to somebody who's worried about syntactical correctness.

And I think that there will be a new generation of developers who will never have known life without that, and they will come in thinking already at a higher level than I did when I wrote my first line of code, which is like, can I get this JCL to work? Can I get this COBOL program to run? That was the level I was operating when I first became a programmer, and programmers now have the benefit of much more abstraction, and then we'll have basically suggestions that will help guide their thinking and they'll be able to make those critical decisions. So I think in some ways it's going to make for better architects, we'll all be architects as opposed to having sort of this alternative hierarchy of the functions that we provide in the design and development of software.

Shane Hastie: Elizabeth, a lot of really interesting topics and points there. Thank you for meandering with me. If people want to continue the conversation, where do they find you?

Elizabeth Lawler: I'm on Twitter @ElizabethLawler, and certainly, you can find me through AppMap.

Shane Hastie: Thank you so much.

Elizabeth Lawler: Thank you.

Mentioned

• AppMap
• Good Days Bad Days Paper
• Book: The Creative Act
• Elizabeth Lawler on Twitter/X