So far, 2020 has been a challenging year. As more of us work remotely from home, it’s more important than ever that the code we produce is as high quality as possible. Static analysis, along with automation, is a perfect point of focus for software developers and security and functional safety professionals.
We all know that static code analysis is something we should be doing. Some of us know that we should be doing it because it will help us. Others might feel like static analysis is being forced on them and that it’s a waste of time, tedious, or it creates work with little benefit. While I would like to say those people are wrong, unfortunately, they are probably right!
Biting Off More Than You Can Chew
If static analysis is slow, noisy (false positives), painful to use, creating work, or not providing value, it’s usually because you bit off more than you can chew. In this post, I want to concentrate on a mistake that many first-time adopters of static analysis make — trying to do too much too early.
Overall, getting static analysis workflow and configuration correct from the beginning is critical to success — less pain, more gain. The first step is to make sure that you have the right tool for your organization.
Start Small with Static Analysis
Once you select the static code analysis tool, you need to configure it. Which checkers do you want to run? I would start with a simple guideline: It’s better to start with a small set of checkers, even only one checker if necessary, and make sure that you and your team are honing your static analysis skills and workflow while getting demonstrable benefit from the effort. The common alternative is turning on a hundred or more checkers that produce large reports that you follow up with sporadically.
Static analysis tools can produce a lot of noise until they are set up correctly. When I say “biting off more than you can chew,” the noise (sometimes called false positives) overwhelms the important warnings. Early adoption sometimes starts as an academic pursuit with the intent of early investigation to get a feel for how many errors are in the codebase. This method doesn’t consider the developer workflow and their day to day use of tools. Nor does it fit the pragmatic need to get software written, tested, and out the door.
The best approach is to start small with the aim to constantly and incrementally improve the quality of the code. Eliminate your most troublesome critical code problems, and then expand to look for other issues that are still important but perhaps not quite as dangerous. Let’s take a look at how that works in a practical sense.
Tackle the Big Problems First
We sometimes see organizations try to assemble a configuration list of checkers by consensus, based on a large list of all possible checkers that a static analysis tool provides. Although plausible, if you consider that Parasoft tools have over a thousand checkers, the approach is flawed because it focuses on what the tool can do, and not on addressing the real issues that your application faces based on customer support, expected environment, and so on.
Teams can spend a lot of time chasing down violations from checkers that aren’t providing a lot of value, even if the error type seems important in the abstract. It’s essential to connect static analysis to the real problems the team is solving or expecting to face. Narrowing down the list of checkers is important but might still produce too many violations for a team or project that is just getting started. You need to look at the size of the “bite” you are taking.
Limit the “Bite” of Static Analysis
A great way to minimize the size of the bite you’re taking (it isn’t just the number of checkers) is to evaluate what code you’re running it against. Most projects these days consist of existing, legacy, or inherited code bases. If these are large enough, it’s not practical to run static analysis on this code and dedicate resources to fix issues. You must limit the “bite” in terms of the amount of code analyzed.
A common approach is to limit action on reported warnings to newly developed or recently modified code. For example, the team lead decides that as the next sprint, all new code is going to be analyzed with a starting set of prioritized checkers. The first set of analysis reports are used to gauge the amount of work needed. If this is too much for one sprint, the analysis can be further limited. On the other hand, if this approach isn’t producing enough warnings, or quality goals are being missed, increase the scope.
All this to say: Take a small bite, a high-value bite. Fix it, accomplish something, build it into your daily process and into your culture. Slowly add to it. Set a few more checkers, and eventually, you will get to your aspirational goal of all the checkers wanted at the beginning.
Watch and Grow the Value of Static Analysis Over Time
Starting small is more than controlling a manageable workflow. It also helps the developers, who are new to the technique and new to the tool, quickly see the value. By giving them critical findings first, they see bugs that are important to fix and understand how testing or code reviews might have missed them. As you ramp up the analysis, developers are always getting the most actionable results. The aim is to never get to where you’re giving developers low-value warnings. It’s a waste of their time and tools, and you should leverage your processes to filter these out.
Avoid Triage
It’s not uncommon for triage to come up when discussing static code analysis results. Triage is only ever used in real life when a process is overwhelmed. Software teams that take too big of a bite run static analysis tools on a large code base with default checkers enabled and inevitably end up with lots of warnings. They then try to go through them and say, which ones do I need to fix? Which ones are most important? This is tedious, overly subjective, error-prone, and mostly unnecessary.
Tools should triage for you. If they don’t, you probably got the wrong tool, the wrong configuration, or the wrong approach. A modern static analysis tool must be able to filter warnings based on severity, priority, and inherent risk. Don’t let the tool itself force you to take too big of a bite.
Successful adoption of static analysis needs baby steps before making strides in quality, safety, and security improvement. Limiting how much your team “bites off” by limiting the scope of analysis both in terms of checkers and the amount of code — everyone can see the value. Avoiding “too big of a bite” avoids many of the common complaints about static analysis tools. Continuous and incremental improvements help everyone learn how to work with static analysis and get better value for the investment.
Click HERE to read the Parasoft whitepaper on choosing the right static analysis tool.
