When the Agile Manifesto and the Agile approach to development were introduced, they were intended to provide a new, more efficient approach to application development that expedited processes and resulted in better products. However, much like with any new technology, law, or approach, Agile has had unintended consequences.
Just like the sinking of a ship that results in a new habitat for sea creatures as an “artificial reef,” many of the provisions and principles in Agile can result in unforeseen benefits or even challenges. Many of these result from assumptions in the framework of how processes run, but some are a result of omissions or changes in technology.
In the latest episode of the ContinuousX Podcast, hosts Rick Stewart and Mike Fitzurka continue their discussion with Cliff Berg, one of the coauthors of “Agile 2: The Next Iteration of Agile,” about the unintended consequences of Agile.
Click the “PLAY” button below to watch their conversation, or scroll down the page to read the transcript.
Transcript: ContinuousX Podcast (Season 2, Episode 11) on Agile’s Unintended Consequences
Rick Stewart: Welcome to our ContinuousX Podcast where we “Solve for X in the SDLC Equation.” We’re in the middle of a conversation with Cliff Berg, co-author of Agile 2: The Next Iteration of Agile already in progress.
I think you hit the nail on the head. There’re so many bits of truth that you went through there from testing, which Mike and I are extremely vocal about how testing…but in order to set up a proper test, you need the proper holistic view of your environment. You’re not just testing, like you said, your code. You’re testing its interaction with other codes. Is your workload resilient? Can it scale? Is it available? All of those things. Can it circuit break? All these things you have to test. And most contractors are not challenged to do that. They’re challenged to get the workload out there and make it functional. Not: Can it scale? Can it perform?
And I think the other part, where you said about partnership; the paradigm shift from commercial markets to the public sector is the commercial markets, as you’ve mentioned in an earlier episode, they’re based on market share. If they don’t perform, they lose market share, they go out of business. Not from a government perspective, they’re not going out of business. You’re just wasting taxpayers’ money, potentially. But the onus of having a workload, not only scale, but evolve and mature and learn, and you’re learning along the way to make it better. It’s not part of the cultural mindset within the federal government.
Now, I met Mark Schwartz before. He was speaking at a conference downtown and they did a dinner before, and I sat next to him. And everything you said about him…he’s first of all, he’s very, he questions, he puts things out in questions. He does not talk to you as being an expert. He asked the community: What do you feel about DevOps? What do you think is failing? And what do you think in the public sector needs work? And that stimulated such conversation at the dinner table. He was quite fascinating to be around.
Cliff Berg: He also used to skip levels all the time.
Rick Stewart: And I bet ya’ that that didn’t go over well in the public sector.
Cliff Berg: Well, he came from Silicon Valley. He had gone to Wharton, so he had a business background. And he was involved with a startup, immigration related startup, and then came to Immigration Service. And so, he…I don’t know if I should share all this, but when I started, he basically…
Rick Stewart: We have editing capabilities.
Cliff Berg: He basically said, talk to who you want to talk to. Don’t let anyone get in your way. Talk to who you want to talk to. And that was very empowering. Yeah.
Rick Stewart: Difficult in the DoD because of the ranking and the pecking order. But possible in other federal agencies. But it’s unusual.
Cliff Berg: Yeah, you have to…the Agile community talks about gemba walking. In the clinical community, in the medical field, there’s a movement toward what they call rounding, where senior managers have practices, go on rounds with physicians. The CEO of a hospital will go on rounds to experience firsthand, because what you hear from the people on the ground is usually very different from what the reports say that you got.
Michael Fitzurka: I agree. And that’s been a problem I’ve seen with every Agile. It’s not that they’re not well meaning but, XP was like, you’ve got to talk to the customer. We had so many customer meetings. I’ve met every single user of the system. In an Agile project, I met the project owner/product owner, and who was supposed to have represented all of that, but you missed that firsthand knowledge. You’ve missed to see the frustration on their face.
Not everything can be conveyed by a customer owner. When you actually interact with the customers, you learn so much more. And as a developer, it’s not that… Yes, some developers you don’t want to put in front of other people…but there are personable developers out there. We exist. And you can put them in front of people, and talk to a customer, and beneficial things will happen.
Cliff Berg: Yeah, it’s really essential. And this is true, whether we’re talking about engineering or software development. One problem with some of these Agile frameworks is they assume this one directional flow of ideas.
But if you if you put engineers or software developers in the setting where the users are, they actually learn how those products will be used. They become a fountain of ideas. Because they know the technology. If they understand and can really internalize what we’re trying to do with this thing, suddenly, they have all kinds of ideas. You can shut them up.
Rick Stewart: They can break down problems more generally than the general public. Sorry, Mike. Go ahead.
Michael Fitzurka: I was just… that was the true tragedy of the Agile Manifesto is that it was designed to help developers and yet it turned them into task takers. Your developer people are extremely smart. You hire them to be that and then, here, build a login page. It’s, okay, I’ll do that. But where you really want them is resolving problems, not just taking orders.
Rick Stewart: Yeah, yep.
Cliff Berg: Yeah, and also the Agile movement had a lot of unintended consequences. That was one of them. The manifesto mentions both the user and the customer several times kind of interchangeably. But these frameworks like Scrum and so on aren’t really true to that. And also, the frameworks… And plus, the manifesto didn’t really mention data, and data is even more important than code. It mentions software, but it doesn’t mention data. And today, it’s all about data. So, we need to bring data to the forefront.
And, also, it unintentionally axed data architecture and product design. And product design is, maybe not so much in the government sector, although maybe, depends on what you mean by product design. What you’re talking about is mission effectiveness. So, that’s a specialty, and its domain specific. People who understand the mission need to conceptualize; what is this thing going to do, how’s it going to work. And you need a team of people to focus deeply on that and to be trying iterations. And to just basically expect that you have some guru, the product owner, just feeding features to a team.
That’s not how you do it. You need product design, or system/Kan ops design. And so, there are ways of doing that in an agile way. But the Agile community unintentionally, because existing approaches aren’t agile, they were too slow, that got chopped out. Instead of figuring out how to embed it and how to make it fluid, and how to make it flow with development in step, it just got chopped out.
And data architecture got chopped out. In the early 2000s, it was customary for most large organizations to have data architects who maintained an organization-wide information model. And the practice of time was, if the development team wanted to create new software, they had to give all the requirements upfront to the data architects, and then wait a month. They’re architects would craft a logical model and then they’d create a physical model. Two months later they say, here you go. And you better give me the right requirements because it’s gonna take another two months to change.
Michael Fitzurka: Right, right.
Cliff Berg: And that’s a non-starter. So, it got chopped out. But the result is today, companies have data lakes that they can’t use, because there’s a bunch of junk in it and they can’t make heads or tails of it. But what we need is, we need data architects embedded in the teams, who are maintaining data models, domain models, as you go. And tentatively, okay, I’m probably going to change it, but this looks like it will work for you. And I’m documenting it so that machine learning people later will understand it. And so go with that. And the next day; Oh, I think we should really change this a little bit. Because it’s more consistent with this other data over here. Okay, we’ll change the code, no big deal. Have it be fluid and collaborative. And that way it doesn’t slow it down. No one’s waiting. You can’t have people waiting for another group.
Rick Stewart: Well, Cliff, I think you’ve whet people’s appetite. We’re gonna end this episode right now. Because what you’re embarking upon is a question that I want to deal with on our next episode, which is, how government treats labor categories, skill sets, etc. And I think you’re setting the stage nicely on that.
So, I’d like to thank our listeners and our viewers. Thank you, Cliff. And stay tuned for our next episode where we’re gonna let Cliff just rock on.
Cliff Berg: Thank You.
