The Agile Manifesto that was released in the early 2000s was a massive step in the right direction for helping to eliminate or streamline complicated, impractical, or otherwise inefficient software development processes. The Agile Manifesto included four values and twelve principles designed to provide an alternative to the traditional waterfall approach to application development that was slowing down the development and deployment of new software solutions.
But the early 2000s were a long time ago, and a number of very influential and important world events have happened in the twenty years between when the Agile Manifesto was drafted and today.
One of the most impactful and recent world events – the COVID-19 pandemic – changed how software development teams worked. The pandemic – and subsequent “stay at home” orders – forced applications development teams apart by forcing them to work from home alone, instead of collaboratively in offices.
This created a problem for Agile devotees. One of the Manifesto’s twelve principles is, “The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.”
This is just one of many reasons why some Agile devotees have introduced Agile 2 – a more modern take on the Agile approach to application development.
In a recent episode of the Continuous X Podcast series, hosts Mike Fitzurka and Rick Stewart speak with Cliff Berg, one of the coauthors of Agile 2: The Next Iteration of Agile, about Agile software development, and why a new approach to Agile was needed.
Transcript: ContinuousX Podcast (Season 2, Episode 9) on Agile 2: The Next Iteration of Agile
Rick Stewart: Hello and welcome to another episode of the ContinuousX podcast, where we “Solve for X in the SDLC Equation.” I’m very pleased to announce that Mike and I have a special guest today, Cliff Berg, co-author of the book Agile 2: The Next Iteration of Agile.
Cliff Berg: Thank you, Rick. It’s a pleasure to be here.
Rick Stewart: So, we’ll get right into the question, the Socratic question, as you as you mentioned in the book; What is Agile 2? And why did you and your fellow Agilists decide it was time for a change?
Cliff Berg: Yeah. Well, it’s an interesting story, but the pandemic provided the spark. Because we realized that one of the core messages of the Agile community was face-to-face conversation and Pandemic forced us all to separate. And so, it looked like a good time, because we felt that that message was a little bit too blunt. It wasn’t nuanced enough. Face-to-face is important, but written communication is important too. They’re both important.
And it really goes back to… I like to look back at the 90s when… In 2001, when the Agile Manifesto came out, I was 20 years into my career. And I remember during the 80s being in a lot of projects that were very agile; from complete automated testing, list of features that we worked down, servant leadership, constant collaboration.
But then, in the 90s, we saw an effort to systematize software development. We had these methodologies come out, and people tried to turn it into a cookbook process. And we saw procurement approaches where procurement officers would define project structures as these phased things. And programmers knew that that couldn’t work, but no one asked them. Because there were methodologies coming out from PMI and other places that define this phased approach. We saw a lot of big initiatives that tried to do it that way, and it didn’t really work.
And so, when the Agile Manifesto came out in 2000… well actually, Extreme Programming became popular a little bit before that. And it was a breath of fresh air, even though Extreme Programming I felt was too extreme, because extremes usually don’t work except in extreme situations. Usually, you need balance, you need flexibility, you need judgment. Not everybody does things a certain way, some extreme way. So, I didn’t like the one size fits all, Extreme Programming; everybody should pair, everybody should use TDD. They’re good ideas, but they work for some people, not for others.
But then the Agile Manifesto came out and it was the opposite of that. A lot of people don’t realize the Agile Manifesto is actually the opposite of Extreme Programming, because in every phrase, it talks about judgment and balance. We value these things, we value both these things, but these more than these. But we value them both. It depends, how much to do one more than the other. It depends. It was all about balance and judgment, not about extremes.
But what happened then, is the Agile movement came to be dominated by the extreme narratives, that became more and more extreme. And the core messages and the Agile Manifesto are dumbed down. To really understand the Agile Manifesto, you have to be pretty experienced and knowledgeable. It got applied in more and more extreme ways, and it didn’t work that well. We saw a lot of organizations struggle and fail. I watched for 20 years as people struggled.
Then the DevOps movement began as a separate movement. A separate movement that branched off from Agile. Jez Humble, seminal author of Continuous Delivery, was an Agilist, considers himself as an Agilist. But today, people who are in the DevOps community generally know very little about Agile. They know just some buzzwords. And people in the Agile community, Agile coaches, usually know almost nothing about DevOps. That’s a huge dysfunction.
So, there was a lot that was wrong. And we realized that we didn’t want to throw the baby out with the bathwater. We thought the core ideas had kernels of truth in them and wanted to save that. So Agile 2, is an attempt to embrace more recent, more intelligent, more thoughtful and nuanced, judgment-oriented narratives around what actual agility looks like.
So, books like Turn the Ship Around! by David Marquet…he was a sub commander, and his whole book is about leadership style. And how he taught his crew to think out loud. And how he watched, but he empowered them to achieve objectives. But he paid close attention. And Nicole Forsgren’s book Accelerate has a whole chapter on leadership.
Rick Stewart: Got that one too.
Cliff Berg: Yeah, it’s an excellent book and its data based. It’s not someone’s bizarre idea. It’s based on data and research. So, Agile 2 is an attempt to be smarter about these things. And to pull on actual research: Behavioral Psychology, Organizational Psychology, Leadership Theory of Cognitive Science, Operations Research. And those all meld together with ideas like Flow and Lean and Theory of Constraints and so on.
So, Agile 2 is an attempt to knit those things together into a narrative that’s open. We say very clearly, this is not like a Bible. This is just a set of ideas to keep in mind when you’re organizing things. And read everything else. There’s the whole library, world of books on leadership. Read everything. So, we wanted to create a smarter, newer narrative around what agility looks like in complex organizations.
Michael Fitzurka: Fantastic. Yeah, I actually was a part of one of those XP projects back in the 90s. Long time ago. 90s. Yeah. And it was even… I actually grew to like the paired programming, but we quickly were like, let’s do this for really hard things only. We tried everything. But then we were immediately, already going to softening some of that. And the test driven… We went to the extreme of saying, you cannot write a single line of code, unless it’s tested. And so, we were testing… We’re writing lines of code to test getters and setters. I mean, this is this is pointless. You know what, so we, again, had to, we had to. But it did get us at least out of some, I think, very bad things that have happened before it.
Not everything was Waterfall before it and that’s not a… I hate that comparison because it’s a false comparison. There are many other things between Agile and Waterfall. And so, Agile has always been a good thing to me. Except, there seem to be a lot of people that were very unwilling to modify it. XP was… They were more willing to modify it, even though it was more extreme. Agile seemed to be becoming more extreme in that sense that; No, no, no, this is the way, this is the only way.
Rick Stewart: One of the things that I’ve seen is, from a business perspective, you have to mold that in there. And when people see the Extreme Programming format. I am paying two people for solving one problem. That, from a labor perspective, that gets business people… They don’t understand the value of that. And you have to articulate it in a way that they can grasp it and convince people that that’s a better model.
Michael Fitzurka: It was more about training, though. It was more about hands-on training. You would pair people with knowledge and without knowledge. That doesn’t mean they were both always Senior/Junior.
If this was database related… I’m really good with databases. I’m not so good with databases. Great, you two pair up and do that. And it was more of an emphasis on training, than wasting two people on a single project or a single task.
Cliff Berg: It’s very… pairing is very effective for helping someone to get experience from, benefit from, someone else’s knowledge about something. It’s not a silver bullet. Some knowledge is transferable, some is not.
Imagine if you’re a doctor, you’re not going to pair with someone with no medical training, and then they become a doctor after a few months. That doesn’t work like that. If you’re an engineer, you’re not going to make someone else into an engineer because being an engineer requires a lot of mathematics and theory and there’s a lot of layers of analytical content that you have to build understandings of in your mind in terms of the knowledge pyramid.
Michael Fitzurka: True. But I’d rather learn engineering from an engineer or learn doctoring from a doctor. So, I mean, there is that. But not every engineer is a good teacher.
Rick Stewart: Yeah, that’s true.
Michael Fitzurka: So, when you got to pairing and there were people who were just like type-type-type, you’re not sharing anything with me, you’re just taking the keyboard. So, it was a risky thing, but it can be good, is my point. I still thought of it good in some situations. Yeah, but not every single product. Not
Rick Stewart: As a project manager, you look at that and say, well, okay, if you’re typing-typing-typing away, I might as well put the junior guy on something more simple. Because now I can parallel my efforts, as opposed to not realize the value of the teaching, to your point. So, it can be… anything can be done poorly, is the point.
Cliff Berg: Because there’s a short term versus long term value. The short-term value is put the most productive people on the things they’re most productive at. If it’s possible to train other people, now you have more productive people. And so, your speed of getting things done in the future increases. There’s a trade off of right now versus a month from now. Which is a judgment. It could be that what you’re trying to do right now really needs to get done right away.
So, what if there’s a crisis, Apollo 13, where there was an emergency and they had to figure out how to get those people home. No, you put the friggin’ smartest people you have on it and give them whatever they need. Everyone else get out of the way. But normally you want to balance short term and longer term, and developing people is really important. That’s actually a major thrust of Agile 2, is the whole professional development aspect.
Today we are worried about retention and having enough people because a lot of companies… I don’t know if this really translates to the government or not, but a lot of commercial companies today are more worried about market share than they are about efficiency or cost. Because they want to keep their market share. They want to grow their market share. Profit will follow eventually, but market share is far more important because it’s your future profit. And you can’t have good market share, if you don’t have agility; if you can’t get new features out quickly to stay ahead of your competitors. And in order to get features out quickly, you have to have enough people who have the skills.
And so, there are three strategies, retention, recruiting and training. And you really have to approach all three, holistically. You can’t just… it used to be 10 years ago; people would just recruit. Oh, we need some people, recruit them. But today, there’s scarcity. If you train people, some will leave, yeah, but at least if you train enough, you’ll have the people you need and you’ll retain the market share, which is really what your goal is. So even if you lose a bunch of people you train, you capture market share.
Rick Stewart: I liken it to a sports metaphor with a baseball team that brings up some of their younger rookies, in order to give them experience in the big leagues, because they have to experience that. They can’t learn it from a book. They can’t.
And they’re not seeing this type of competition that they see in the majors. So, you have to bring them up and you got to be risk taking. And let that person sink or swim in order to gain that experience because you’re short sighted if you’re always using experienced players.
Cliff Berg: Yeah, we need to see development of people as strategic, not as a cost.
Michael Fitzurka: I think that also touches on your Agile 2 points of leadership. That you can’t just put a methodology in place and then let the next task drive it. You’ve got to have someone being more judicious.
You have to have someone leading it, balancing requirements, who makes that judgment call that this is all important, I need everyone on it, or this is a good training opportunity, or this is just a regular task. It shouldn’t just be up on a Kanban board, and I guess I’m next. Now serving ticket number four.
Cliff Berg: Exactly. The Agile community is culturally… The Agile community has a culture. And culturally it’s a little bit anarchistic. I hope this isn’t inappropriate… it’s a little bit communistic.
Rick Stewart: Public Sector!
Michael Fitzurka: It’s an Anarcho-Communist… commune, I guess.
Cliff Berg: Yeah, I know. If you really do Scrum the way it’s prescribed, it’s like a Soviet collective farm. No one owns… collective code ownership and everyone’s the same and everything. There’s a good idea there, but not if it’s practiced in the extreme. Collective decision making is good. But not if all decisions are collective, all the time. Then you’ll never get anything done.
There needs to be leadership. Good leaders don’t always use their authority. They generate discussion. The transformational leadership model… one of the elements of it that people often forget, is intellectual stimulation, asking hard questions, Socratic inquiry. How’s this gonna work? How do we know this is gonna work? What happens if? Are we going to get that capability done in the timeframe where we really need it? How is this all going to fit together? Let’s talk that through. Good leaders generate that discussion. But they also make sure it happens timely, and make sure decisions get made as they’re needed, not drag out. So, if you don’t have a collective consensus at some point, someone has to say, okay, well, let’s go this way and just try it. But I’m going to decide, we’re going to try it this way and see what happens. And if it doesn’t work, I own it, but it’s just a test. They aren’t gonna be put on the vehicle, it’s just a test.
SpaceX has a rule of thumb, 51%. If they’re 51% certain that something will work… So, it will probably work with a little bit of, 51%, confidence. They figure, good enough to try. They put a vehicle on the test stand. No people on it. They put a vehicle on the test stand. They see what happens. They put the heat shield in the oven, and they see what happens… with the torch and whatever it is. They try it and they see what happens. So, you need people, you need leaders who keep things moving. Peter Drucker’s person of action. If you just rely on a team to develop consensus, it’s too slow. It’s too slow.
Rick Stewart: And you don’t want the other one, being so autocratic, like your analogy with the Soviet farm system. That didn’t work out too well. And you have a dictator providing “guidance” and leadership, and people starve.
Cliff Berg: Yeah, the toxic autocratic leader, that just says do this and do that and I understand it all, but none of you need to. That’s dispiriting. It fails to utilize people’s brains. The people are afraid to criticize. Which we see happening in the geopolitical sphere right now.
Good leadership is open. It generates discussion where people can voice ideas and they’re not criticized. Oh, that’s a stupid idea. Or your idea didn’t win so you lose standing now. It needs to be a dialectic or intellectual based collaboration, but then someone needs to decide on what we’re going to try.
And they need to own it and not blame the team if it doesn’t work out. They just need to view it. Well, we tried it. It didn’t work out. We learned from that. And now, we’ll have another discussion, decide what the next step is.
Rick Stewart: Well, Cliff, I think you knocked that question out of the park. And it’s usually longer than we usually have, but it was so fascinating. So, I’d like to thank the listeners for this session. Stay tuned for more of Cliff, where we talk, where he mentioned DevOps. We’re going to talk about how the Agile 2 movement and the DevOps practices meld, where they differ, and how they joined together to make organizations more efficient.
