Tips and Techniques for Improving Retrospectives

Transcript

Shane Hastie: Good day folks. This is Shane Hastie for the InfoQ Engineering Culture podcast. Today I have the privilege of sitting down with Esther Derby and David Horowitz. Sadly, Diana Larsen is not able to join us today, but that does give me an excuse to get Diana on a later episode. So we're going to be talking about Esther, David and Diana's new book, the second edition of Agile Retrospectives. But first of all, Esther, David, welcome. Thank you for taking the time to talk to us today.

Esther Derby: Thank you.

David Horowitz: Thank you so much. Pleasure to be here.

Shane Hastie: Now we obviously know each other, you've both been guests on this podcast before, but I suspect there's some of our audience who haven't come across you before. So I'll ask you if you wouldn't mind, just give us the two minutes. And David, let's start with you. Who's David?

Introductions [01:04]

David Horowitz: So I'm David Horowitz. I am the CEO and co-founder of a company called Retrium. And I landed on starting this company about 10 years ago after working at a big international organization and I couldn't for the life of me figure out how to run a good distributed retrospective and thought, hey, someone has to solve this problem. And so here we are. So that's a little bit about me. And besides the professional life, I have four little kids running around at home. So quite a busy guy.

Shane Hastie: Welcome. Esther.

Esther Derby: Well, I started my professional career as a programmer and I have done a lot of things over the years around software development, although I haven't touched code in quite a few years. I really focus on working with leaders and managers in organizations to manage more effectively, more systemically and more humanely.

Shane Hastie: Excellent, thank you. And again, welcome to you both. Esther, when was the first Agile Retrospective's book published? It's a while back.

What’s changed since the original Agile Retrospectives book was published? [02:37]

Esther Derby: I think it was 2006. I think I have a copy here on my desk. I could confirm it. Well, it's somewhere in the stack here. I think it was 2006.

Shane Hastie: 2006. So that's a fair while ago. I'm going to guess that a fair amount has changed, but also a fair amount probably hasn't because we are talking about people stuff, aren't we?

Esther Derby: Well, there's a lot that's still the same. But among other things, we had this pandemic and everybody learned that you could work remotely. And so that whole thing has changed, which David alluded to that people started doing remote retrospective. So that's one thing that has changed. I think another important thing that has changed is when we wrote the book, facilitation was not regarded as a skill that was generally applicable to work. It was viewed as very specialized, there are those people over there who are facilitators, but I think in some part, due to this book, and also probably to Jean Tabaka's Collaboration Explained, facilitation is now a much more valued skill. And I think the general level of facilitation skills is higher in the agile world than when the original book came out.

I personally started my journey of learning facilitation skills facilitating technical reviews, Weinberg style technical reviews in the '80s. And I just found that applying those skills, everything worked better. So I think those are just two things that changed. What else stands out for you, David?

David Horowitz: Well, another thing that's changed is the general adoption of agile as a way of doing work that it's become almost commonplace now and expected sometimes in name only, but some version of agile to be adopted by the workers that you're going to be working with at your new job. Whereas back when the first edition was published, that may have been somewhat new to people still. And the byproduct of that is when you look at the state of agile surveys that come out, retrospectives are always near the top of the list in terms of the most commonly used agile techniques. So standups are almost always number one. And I believe retrospectives recently have been number two or number three on that list. So high 80%, maybe 90% of teams according to the survey are running retrospectives now, which is I'm sure much higher than it was in 2006 when the book was first written.

And as a result of that, there's a wide range of retrospective styles out there, some of which are more effective than others. And the book, I hope, is going to help those who are struggling to run effective ones effectively to level up their skills a bit and learn some tips and tricks from people who've been doing it for quite some time.

Shane Hastie: What are some of the key themes that are in the new book?

The importance of having empirical data to ground your retrospectives on [05:26]

Esther Derby: Well, one of the things that I thought was really important to add was there's much more emphasis on having data to ground your retrospective. So it's not just mad, sad, glad, it's depending on what the problem is, you may actually need hard data. So you may need subjective data or objective data, but you need to be conscious about thinking about what are we working on? What are we focusing on here and what data will enable us to think about that and make sense about that and understand the issue in a way that's actually grounded rather than just opinion. I think that's one of the biggest issues I see in retrospectives is people don't have data, they're working on opinion. So there's a lot more information on how to use data and retrospectives.

Shane Hastie: So if we can dig into that one, what are some of the important data points that people should be looking for when they come to a retrospective and where do they gather them? Where do they find them?

Esther Derby: Well, that depends entirely on what they're focusing on. So data always follows focus. So if you're trying to understand a pattern of defects, for some reason all these defects have been getting into production, escaping to the customers, sitting around and talking about your opinions about that is speculation. So then in a case like that, you might want data about numbers of defects, classes of defects, how many people are affected by the defects, what are the effects? And some of that stuff may be in various tracking systems, you may have to do a little tracking on your own or digging on your own. That's a case where you might want to gather the data before the retrospective.

If the focus of the retrospective is how are we doing with our technical practices? That might be data you can gather in the retrospective because it's subjective, right, as well. How do we feel we're doing about it? Do we feel like we've got it cranked up to 10 or do we think we're only going halfway? Those are opinions. And then it's the discussion about those subjective perceptions that become important for the conversation. Why do you think we're at a 10? Why do you think we're at a five? What are our different perceptions about this or our different definitions about what the practice is? So really, data follows focus. It really depends.

David Horowitz: One of the focus areas that we mention in the book as it regards to data is using customer data actually as data in your retrospective. It's not necessarily just engineering technical practices that you can look at. Let's say for example, you have been working for the past month on some new feature, some new release, and you got it out the door, yay, we're celebrating and we're starting to get customer feedback. And this could come in the form of support requests. We could come in the form of NPS scores or some sort of numerical rating by your customers on how they feel that the release is going. But then using that information in your retrospective helps to complete that feedback loop. So it's not just how do we perform better inside the team, it's also is what we're doing as a team actually having an impact on the customers that we are targeting.

Shane Hastie: If I think about the five-step structure of the retrospective, gather data is really, it's key there. In my experience, it doesn't happen a lot.

Techniques for gathering data [08:48]

David Horowitz: And one of the downsides of that, Shane, is the lack of a shared mental model on the team when it comes to trying to decide what to do. People skip to that right away. And if you don't have a shared mental model before you decide what to do, and even before you decide what to do, you have to generate insights per the model that we're talking about, but all of that relies on having a shared mental model across the team. So a simple example of that, we include an activity called timeline in the book, which I believe was in the first edition as well. But the idea timeline is just on the x-axis, you have time and you ask people to put little sticky notes up of just what happened in the last two weeks, four weeks, whatever the period of time that you're looking back at, and you just put them up on the board. And you'd be surprised, at least in my experience, how many times people go, "Huh, I forgot about that one."

I mean, I don't remember what I had for dinner last night, let alone what happened two weeks ago or a month ago. And so something as simple as building a shared mental model of just the facts. And then you can add in emotions, how are you feeling during certain times of the sprint overlaid on top of just what happened, portion of it. But this sort of thing can really help ground the team in a shared sense of what occurred so that you can dive deeper into the issues together as a team.

Esther Derby: I had a really interesting experience several years ago. I was with a group of scrum masters talking about retrospectives, and I got to the point about talking about data and I was struck that it was hard for them to come up with any ideas about, well, what kind of data would we want other than do differently or stop doing? Which by the way are conclusions or opinions. And that incident made it really clear to me that people need a little, or at least some people need a little more guidance about how to think about what data to bring into your retrospective. Because as David points out, if you don't have a shared pool of information, everybody is just thinking about their thin slice of data. They may not have the whole picture or they think about their own opinion and then you're not grounded in reality.

David Horowitz: Or they could have been sick for two days and actually missed some things. I mean, it can be as-

Esther Derby: Absolutely.

David Horowitz: Right?

Esther Derby: Or I mean you don't even have to be sick. Even if you're working in the same team room, there's stuff going on at the other end of the room that you might not be aware of or you step out or when you're remote, your communication paths may be fractured. So yes, data.

David Horowitz: Bring up the remote part of it too. I think that it's almost a given if you work on a remote team, that everyone will have a different perspective on what happened because everyone had a different thing happen to them. If I'm working from my home computer, the internet might've gone out or I had disruptions because of the dog or the kids or everyone has a different context. It's not quite as shared. The physical space isn't shared. And so there are different pieces of information that you could bring to the retrospective that others quite literally did not experience and therefore need to be shared to have a shared mental model before digging in further.

Shane Hastie: Let's dig into that, remote and hybrid in particular. You are right, we've gone through a radical transformation. The studies are saying people don't want to go back to five days a week in office. We are enjoying the flexibility, and then we're also struggling with collaboration, teamwork, and the mix that is hybrid is also challenging. So what are some of the new ideas or key elements that retrospectives need to adapt to or can help us with in this remote and hybrid environment?

The challenges with retrospectives are similar irrespective of working in-person or remote [12:24]

David Horowitz: I can start by just saying that, and you might not like this answer, Shane, but I'm going to give it anyway, most of the same problems that teams who are in-person experience with retrospectives are identical to the ones that remote and hybrid teams experience. And I'll throw a number on the wall, let's say 80% of them, right? Maybe it's a lack of follow-through that will be the same whether you're in person or remote, and there's some twists here and there, but generally, broadly speaking, it's the same. Maybe it's we're not actually following the five phases and we're just doing what went well, what didn't go well. Guess what? That's going to be the same whether you're in person or remote. So broadly speaking, we share challenges, which is both a blessing and a curse. It's a blessing because, well, we wrote a whole book about how to solve some of these challenges, whether you're in-person or you're remote.

It's also a curse because they happen to be somewhat difficult to overcome, at least historically, that's what we've experienced. So if you are going from remote to in-person, my message to you is don't expect it just to magically get easier and all of a sudden start working.

With that said, there are specific challenges that remote teams face that in-person ones do not. I mentioned already the lack of shared context because we are physically in different spaces. Another one that's very low level here, but really tends to matter is when you're in person, there's a physical arrangement of people, let's say around a circle, you've set up chairs and everyone's looking at each other, and I know that Joe is to my left and Alice is to my right, just as an example. When you're online, at least in my experience, most video chat software rearranges the tiles of people and the order of the tiles on my screen is not necessarily the order that Esther has on hers or Shane, that you have on yours. And it makes it very hard to refer to "The person on my left said," or "the sticky note on my right." The lack of shared context there is a real challenge to overcome.

So finding a tool that helps you overcome some of those additional hurdles is really important. It's one of the reasons why we built Retrium, but there's other tools out there of course that can do that for you as well.

Esther Derby: Yes, back in 2006 when we wrote this book, the tools for remote collaboration were very, very different than they are now. We didn't have Zoom, we had, I think Skype was around, so you could use Skype, but the online whiteboards are very different now than they were back then. We have tools like Retrium that there are several of them that provide varying levels of support for retrospectives. So in some ways doing a remote retrospective, it's much more possible to have a robust remote retrospective now than it was when the book first came out. Just the tools are much better. I think the most challenging situations are the hybrid retrospectives, and that's always been the case. The challenge is to get the people who are in the room talking to the people who aren't in the room. Someone said, "Get the roomies talking to the Zoomies," which I thought was kind of cute.

And to make sure that all of the activities that you use can be accessed by everyone in the retrospective. We have lots of stories from people who have been out in the world practicing retrospectives in the book. There's a story in the book that one of our colleagues told us about a retrospective she attended, she was not facilitating, she was attending it where the well-intentioned facilitator had designed this activity for the group to do and said, "Okay, those of you on the phone, you can't participate in this, so we'll just go ahead and do this activity." Which is like, oh, that's very sad.

So I think that's a big challenge is to make sure that everybody can participate in the activities, which may mean that even though you have some people who are in the room, there are some activities where everybody's going to have to get out their computer. And we used to say, don't have your computer in your retrospective, just be with the people in the room. But if you've got a hybrid situation, there are just some things you're going to have to do using your computer so that everybody can participate in a relatively equal amount. It's never going to be perfect. It's never perfect in person either but it's a relatively equal participation.

Shane Hastie: What does a great retrospective feel like, look like?

What makes a great retrospective? [16:40]

David Horowitz: That's a great question. I suppose for me, there would be three components to it. So the first one is that there is full engagement in the room when it happens. There's just a sense of aliveness in the room. And I've been in my fair share of retrospectives where the opposite is true and you can just walk into that room and you feel it. You know that there's something, it's just people are not engaged. And it's not something that that's particularly enjoyable, nor is it constructive when people aren't fully participating in the retro. So I'd say full aliveness, full engagement is one component of it. Another one has to do with full psychological safety because even when people are engaged, if they aren't speaking what's actually on their mind, then what's the purpose of the conversation? So a team that trusts each other, that has psychological safety to share what might otherwise be unshareable, to stir the pot a little and know there won't be consequences, that that's a constructive part of the conversation is part of a successful retrospective.

And then the third component to me would be what happens after the retrospective is over. So even if we had high engagement and we had full psychological safety and a constructive conversation, I'm looking for a retro that something changes after the retro is over, that we do something. It doesn't mean that it's going to work. I think that's an important distinction. We might pick something that actually doesn't fix the problem, but that reduces the potential solution set by one and we've learned something, and that in and of itself has value. But we change something, we do something to the system to introduce something new. And if you have those three things, so full engagement, high psychological safety, and we change something after the retro is over, I'd call that a big win.

Esther Derby: Yes, I really like the way you put that. Those characteristics can show up very, very differently depending on the team and their relationships. I know some teams who it just looks like they're having fun and joking all day, but yet they're super effective and they're doing fabulous work. And I know other teams that are very quiet, and I know teams that like to argue, I know teams that don't. And so all of their retrospectives are going to look different. But if they're engaged and they have psychological safety and they accomplish something, they use their agency, then that's success. So I really like the way you put it, David.

Retrospectives need to result in something changing, but it doesn’t have to be a big change [18:58]

David Horowitz: Yes, I think the changing something doesn't have to be changing something in the traditional sense either, right? Most people think of we need to come up with one action item at the end of the retro to call it a success. And I actually don't think that's true. Certainly there are retrospectives that will lead to an action item, and that's wonderful, but there's also retrospectives that lead to an aha moment, just some learning on behalf of the team. We discovered something we weren't aware of about ourselves, about the system, about the environment that we're in. And simply by gaining that aha moment, collectively as a team, we've changed something. Now we go into the next iteration with a new understanding. So in many cases, good retrospectives, the ones that feel really good at the end actually don't have action items. And far be it for me to encourage people not to have action items. That's not what I'm trying to say. But an understanding that just that aha can be good enough sometimes also.

Esther Derby: I think that's one thing that we emphasized a lot in the first edition, was having an action to take, one or two, don't overwhelm yourself. And I think that's something that we approach differently in the second edition because you may have a learning goal, you may do an experiment, you may have a significant insight that you just kind of need to sit with and it's going to sink in and it's going to affect people's individual actions, but it's not necessarily a team action item.

And yes, you can have good old action items. You might have an influence plan. You might discover that there's something that the problem is affecting the team, but it is not of the team. And then you have to go figure out, well, who does have some ability to shift this particular thing and what do they care about, and how can we state our issue in a way that they can connect with, that they care about so they'll want to take some action to help us? So an influence plan might be what comes out.

It might be just we're going to respond to this particular stimuli differently. And then we talk a lot more about action items and influence plans in this edition than we did in the first one.

Shane Hastie: One of the most common anti-patterns that I've seen in retrospectives is the retrospective that goes nowhere. That it's Groundhog day. We're bringing up exactly the same issues. Most often issues that the team feels they have no agency over, no control over, we are complaining about management haven't given us the support we need in whatever space, and maybe it's cathartic, but we get nothing out of it. How do we shift from that?

Avoiding retrospective Groundhog Day [21:35]

David Horowitz: So this happens all the time, as you're alluding to, Shane, that it feels good to complain, right? It feels good. It doesn't necessarily mean that it actually feels good in the long run, but in the moment, it can feel good for teams to complain about the context that they're in, especially in my experience at least, in larger organizations. Oh, it's just where we work. And the reality is that there often is at least something that you can do, even if it's somewhat small, to change your reality. So there's usually some amount of responsibility that you can take for your portion of creating whatever it is that you're experiencing. And that doesn't mean that you're responsible for the system as a whole, but you can, as Esther said a minute ago, at a minimum, change your response to what's occurring around you. So that at least is a good place to start, is to separate the stimulus from the response and to realize you actually have control over how you're responding as a team.

The second thing you can do is to come up with that influence campaign. Who is it that actually needs to hear about this and what do they care about? And how are we going to structure our message and who should be talking to them to try to change whatever it is that we're trying to change? There's a whole section in the book actually that talks about what we call retrospective radiators, which are a type of information radiator. Information radiators, if you don't know about them, are ways of communicating through big visualizations out from your team to your context, your organization. Burn down charts are a great example of that. When they're not stuck in your platform, in your wiki somewhere, when they're actually radiating, they're a good example of that. But retrospective radiators are a specific instance of an information radiator where at the end of a retro, the team might record down what it chooses to share transparently with the rest of the organization.

Now, by default, the retro should be private to the team because we want to protect the psychological safety of the individuals in the room. But when the retro is over, you might ask yourself, what do we need to share with the rest of the company or with another team or with a coach or with a stakeholder to get them to understand where we're coming from here? And if you come up with one of those things, the simplest action you can take is to make a retrospective radiator by sharing it publicly on some big visualization so that people are aware of what's going on. And if every team in your company started doing that, imagine what information you could glean simply by walking down the hall and seeing these radiators hanging from team rooms, from different offices. So those are a couple of things you can do. One is change your own response. One is trying to find someone to influence. And another is to radiate out information to the rest of the organization.

Esther Derby: There's an activity we do sometimes called circle and soup. It has concentric circles. And the inner one is what we can control, what we can influence 'cause we know who it is, and then there's the soup. What's the flavor of soup today, and we just have to deal with it. So sometimes I use that sort of activity to help people get beyond just looking at what other people are doing.

Shane Hastie: So going into the book, what are some of the really interesting stories that you're bringing to light and the messages from this?

Stories of retrospectives [24:44]

Esther Derby: One of the stories that stands out to me is about a dilemma that people often face, particularly on smaller teams when they're facilitating their own retrospectives. This comes from George Dinwiddie. He had a situation where he was facilitating, but he also was involved in the situation. And he handled that by being very, very explicit that now I'm in my facilitator role and now I'm not. And I think he even moved to a different part of the room so it was very evident, not just in his words, but from where he was placed that he was participating, and when he was done participating, he would go back to his facilitation role. And I think that's a situation that people come across a lot. So that's one story that I think is useful and important story from someone who's out there doing retrospectives frequently.

David Horowitz: There's another one about asynchronous retrospectives by Kirsten Clacey at Automatic, I believe, where Automatic is a relatively large company specifically for a remote company, and they have people who are located around the world. And so the question was how do you run not just a remote retrospective, but a retrospective with individuals across many, many different time zones? Of course, if you tried to do it synchronously, somebody will be inconvenienced, probably a large portion of the team will be inconvenienced. And so there's a story in there about Kirsten who tried to run the retrospective asynchronously and how she still used the five phases of the retrospective that we present just in a time-bound, asynchronous manner so people could contribute at a time that made sense for them.

Esther Derby: Yes, I think the common thread in the stories is there are people sharing how they dealt with situations that come up frequently. There's another story I can't off the top of my head remember who told us that story about going into retrospective and something had come up. And so whatever they had decided to focus on no longer made sense because there was this other very urgent concern that people were going to be distracted by if they didn't deal with it. So this person shifted their old plan for the retrospective to what they were facing in the moment. That requires some skill and some guts, but it's sometimes what you have to do. And so there are all these beautiful stories from situations that people had to face and respond to congruently and quickly. And there's a lot of stories about how people adapted particular activities to suit a particular situation.

Shane Hastie: So a rich thread of stories through the book. Wrapping up, what would be the key message that you would want listeners to this podcast to take away from our conversation?

Good retrospectives require thoughtful preparation and facilitation skills [27:32]

David Horowitz: Well, the first thing that comes to mind for me is that retrospectives are not a check the box activity that you can just show up to and hope that they go well and assume that they'll provide the benefit that everyone claims that they will. They actually require thought and practice, and they require skill. And the more that you learn about them and the more that you're able to practice them, the better off you'll be and the better off you'll be serving your team. It's why we wrote the book. One example of that, by the way, in the book that I should add, there's a whole chapter that we added that I just love, I think it's wonderful, which has to do with common scenarios that teams tend to face. I believe there are 10 of them in the book. Things like when a new team forms or when there's an outage or an issue that the team has experienced in the production environment, or when the team is celebrating a big success. These are common scenarios that teams face. And how we might design a retrospective in response to those scenarios popping up.

And what we talk about in the book is not copy, paste this into your team, although you could do that. It's more this is how we're thinking about the design of the retro, and we're hoping that it's how we're thinking part is what people will learn from so that they can design their own retrospectives in response to their own contexts that they experience. So the message I want people to take away from this podcast is if the retrospective isn't serving you, try to do more to understand how to shape them to your context so that they have the best odds of success.

Esther Derby: Yes, I would add to that that people's minds all work a little differently, but we all have kind of a common set of mental processes we go through when we're thinking clearly. And a good retrospective helps the team go through those mental processes together so that you don't have one person's thinking of solutions while another person's still thinking of data. So you can go through those steps together because that's where you tend to get profound insights. And it's also where you tend to have a higher level of agreement when you get to the end of the retrospective.

Shane Hastie: Some advice, some useful tools. If people want to continue the conversation, where do they find you?

David Horowitz: Well, I'm on LinkedIn. You can search for my name, David Horowitz.

Esther Derby: You can also find me on LinkedIn.

Shane Hastie: Thank you so much.

David Horowitz: Thank you, Shane. Pleasure being here.

Esther Derby: Thank you so much, Shane.

Mentioned:

• David Horowitz on LinkedIn
• Esther Derby on LinkedIn
• Book: Agile Retrospectives, Second Edition