I propose that the motivating concept for all managerial advancements since the emergence of computers can be primitively summed up as, “Hey, you know that thing that messed up the project last time? Let’s do that earlier in the process.” Shocking, right?
Today this is more eloquently known as, “shifting left,” with “left” implying earlier because we read from left to right. And as it turns out, left is “early” but maybe not often enough, so the guiding principle has evolved into, “let’s do that continuously.” This is an oversimplification to be sure, but allow me to illustrate this “ContinuousX” concept through the ages.
I am not old enough to have started programming during the earliest days of computers, so I envision them with an air of romanticism, when computers were, “real computers,” with punch cards and bright primary-colored buttons. Software development must have been like crossing the oceans during the 18th century. Every ship was unique and every tool on board was custom built and tweaked to the idiosyncrasies of the ship, her captain and crew.
“How long will it take us to build your application?”
“I don’t know, we’ve never done it before. Let’s find out!”
Swashbuckling tales of punch card adventures aside, the process clearly must have been exciting, but impossible to manage, with success difficult to achieve let alone replicate.
Enter the Waterfall management model, an attempt to apply engineering principles to software development. The waterfall process breaks any project into distinct stages, with each stage completed and documented before flowing down to the next stage. The model is as follows:
- Gather requirements
- Conduct analysis
- Design, code
- Test
- Operate
It appeals to the engineering mindset because, conceptually, it’s understandable, methodical and repeatable. And it’s still practiced in many places today.
But what, “messes you up,” at the end of a waterfall project? Typically, something is discovered later in the process that requires going back to earlier stages. It could be a missed requirement found during coding, or it could be testing that uncovers a problem with the design. Initially, to shift these problems left (or would that be “up” the waterfall), more effort was spent in earlier stages and more documentation was created.
Problem solved? Not quite. With large scale systems, the time it takes to go through this entire process can be years. Years between when a customer says, “this is what I think I want,” to then seeing a working application that may or may not match what they said. Years where the underlying business problem can fundamentally change. And years where the previous steps have taken so long that the last stages of testing and operating get sacrificed due to deadlines.
Enter the Agile Manifesto, which again primitively states let’s do testing and get a working product to the customer earlier. Various flavors of Agile management practices arise based on the manifesto, although some are just waterfall in disguise.
Generally, the customer is involved earlier, or at least by proxy or through an advocate. Development is broken down into smaller chunks, first as prototypes built over iterations, and then as even smaller incremental tasks on a Kanban board. Test-driven development (TDD) is introduced which leads to acceptance test-driven development (ATDD) and behavior-driven development (BDD), all aimed at bringing testing to the forefront. In this model, test cases are part requirement, part code, and part documentation. Analysis is minimized as a “paralysis” of the project’s velocity toward completion. And the overall design is said to be emergent from the changes that occur.
In my view, this Agile revolution is akin to the industrial revolution. Instead of building each software product as a unique engineering effort, software development is now modeled as a mass production, manufacturing problem. But what spoils Agile projects? Operations wasn’t invited to the table!
Operations as a separate unit has diametrically opposed objectives compared to development. Operations desires stability and predictability, while development is now built for innovation and frequent change.
Without including operations, launching changes into production typically goes poorly. Since deployments used to be infrequent and different each time, the process is largely manual and can’t keep pace with the new release schedule. Also, cracks get revealed under the stress of production workloads, where missed requirements and technical debt issues result in customer-facing system outages. And these new features are not built for monitoring and care, so operations must compensate on the fly.
And thus, enters the DevOps culture shift, where the guiding principle evolves to continuous improvement and where automation is vital. Again, oversimplifying here, but a DevOps culture persuades against waiting until the end to consider how we will operate and monitor our software changes and advocates doing it continuously.
If it is difficult to move the code to production because it’s the first time all of the changes have been made together in the same environment, a continuous integration (CI) process can ferret out the problems earlier. Likewise, an automated continuous delivery (CD) process for updating environments can be implemented to continuously prove the deployment works, instead of waiting until a production launch night.
So, what messes up a DevOps project? Who was not invited to sit at the table? Security, and thus enter DevSecOps.
I think that establishes the pattern, and if it holds, where do we go from here? Who’s not invited? Is it legal counsel? Should this site become the GovDevSecLegOpsHub? Is it Authority to Operate (ATO) or Change Control Boards (CCB)?
Maybe it is a re-emphasis on craftsmanship that was diminished in the name of Agile velocity. If so, will we rename this site to the GovDevSecLegAtoCcbCraftOpsHub? And if everything is shifted left, is there a right anymore? Because if everything is a priority, then nothing is a priority.
So, I propose the concept of “ContinuousX,” where X is everything that you are trying to solve, and make sure that everyone is invited to the table.
