by Anthony Maiello, President Southwest Region (USA) -Tier4 Advisors

Looking back in my early experience of writing business and gaming software for fun on my TRS-80 and Commodore 64, I’ve observed through my career methods of software testing I used in the late 70’s and early 80’s were also used in the “real world”! I quickly learned in the commercial arena, businesses and customers cared a lot about “time to market” and “cost.” To be successful, we took a “command and control” approach that focused on process and tools to force teams to test for quality and go faster.

Six Sigma, Lean, CMM, CMMi, and other methodologies made their way into the arena of the software life cycle. User expectations around software quality increased over time, the processes, metrics, and “command and control” approaches and methods became more and more complex, higher cost, and less effective.

What seemed to work for software development were methods around “iterative development.” Agile started making its way into the formality of the software lifecycle. These types of methodologies helped organizations focus on customer needs and business priorities and less on process driving people which seemed to work very well. The most important part of testing software wasn’t the process or the tools. It was the talent and knowledge of the people creating the software (the developers, testers, architects). Process and tools did play an important part, but the relentless iterative approach to the software lifecycle by talented technical engineering teams was the major key to success and remains the key to success today. It’s the 80/20 rule – software quality requires 80% smart and informed people, and 20% process/tools. Pure Six Sigma isn’t the best approach for software as it’s focus is mostly on improving the repetitive process (i.e., manufacturing) and not improving the people/talent side of things with software that changes from one iteration to the next. Delivering continuous feedback to teams helps focus on customers and business priorities.

The success of your systems delivered to your customers and your business has always been measured in terms of “time, quality, cost, and features enabled.” But how do you measure time, quality, cost, and features enabled? What exactly are the best metrics? “Metrics drives behavior.” Identify the right metrics to avoid poor outcomes for your business and customers.

Over the last decade, the fantastic advances in the software lifecycle have helped companies advance in software quality and hence, deliver very successful systems that customer enjoy and appreciate. Methodologies like Agile have helped transform development practices to focus more on enabling the team (people) to deliver quickly and on time, and have laser focus on the highest priorities for the business and customers. Instead of trying to improve only the “process” like many past methodologies, Agile helps improve the team first, thus supporting the 80/20 rule I noted earlier. More and more, we see successful engineering teams using a few key metrics that help their organizations move forward in an informed manner by enabling continuous feedback back to the team. With outcomes that improve the process as well as the talent and knowledge of the teams. What metrics should be measured? Metrics have to provide feedback to the teams and not used to drive a command and control environment.

Let’s start with the following key metrics:

  • measuring the number of stories (use cases) delivered successfully to customers,
  • the time it took to deliver,
  • production incidents/recurrence,
  • and customer feedback

These are critical metrics to track that provide continuous feedback and knowledge to the team assuring great quality and successful systems to customers. Doing this in an iterative “sprint” fashion is also critical for success and together drives the behavior of your company to deliver success to your customers and business.

CIOs, CTOs, or software managers who are still working with methodologies that stress improvements to process to drive and measure your teams, will not help you stand the test of time at your company.The successful technology leader makes some changes and puts the right metrics in place that focuses on delivering value to customers. So don’t focus only on improving process and tools that you think will drive the behavior of teams. Focus on enabling teams to evaluate, modify, and improve the process that enables the highest priorities for the business and your customers to set the foundation for a learning culture enabling teams, as well as process, to get better and better over time.

Tier4 Blog Edition 36: April 18, 2018