In this episode, Jeffrey shares how to measure a software team.

Situation 

Many software team lead architects don't implement management practices that are standard in other parts of the business. Whether it be OKRs (Objective, Key Results), EOS, Scaling Up's Scoreboard, or Kaplan's Balanced Scorecard, business measurement has long been a staple of ensuring that a part of a business was functioning well. But executives overseeing software teams often don't have a tool for measuring the effectiveness of a team or an entire software department.

Mission

Anyone overseeing a software group of any kind needs a way to measure the effectiveness of that group. Let's zoom down to a single software team and look at what must be measured at a minimum for a single software team. Once the measures are identified, the team can then report them weekly to the appropriate layer of management. And just like every other department, if the measures are aligned with business objectives, then the reports can be relied on to know that the objectives are on track to be accomplished.

Execution

The tool you need in order to measure a software team is a good, old-fashioned scorecard. It's not high-tech. Every business methodology in the last 3 decades has employed some format of the scorecard for the purposes of measurements that are tracked over time and given thresholds of acceptable values. We'll go over the Clear Measure Way scorecard template and how to use it.

Mental Model

From cash flow forecasts to sales pipelines and order shipping, most businesses are used to tracking numbers weekly. Some numbers are tracked monthly, but in software, weekly is better aligned with the normal flow of a software team. You can obtain the Clear Measure Way scorecard template for free from the Clear Measure website. It's a Microsoft Excell worksheet. The first tab is the scorecard itself. The next tab is instructions for how to use the scorecard. It comes prepopulated with the minimum suggested measures for a software team. As you become more comfortable with it, you'll undoubtedly add more measures to it. The researched DORA metrics are part of our minimum, so you'll find those on the scorecard.

At the top of the scorecard template, you'll find a link to a tutorial article that explains how to use the scorecard and how the Excel template is put together.

Each week, you'll have the team populate the numbers in the column that represents the current week. Over time, you'll probably choose to hide the rows in the past so that you can glance at the current week and probably the trailing 12 weeks, thereby getting a good glance as a rolling quarter of performance.

Team Alignment

The measures on the scorecard are divided by the pillars of the Clear Measure Way but are preceded by a Team Alignment section. We suggest that the software team's scorecard include the top-level business measures that are managed by the executive overseeing the team. Without tracking the impact the software team is making in the business, it's easy to become misaligned with business objectives.

If you don't already have these quarterly targets, I'd invite you to use the free Team Alignment Template, also provided by the Clear Measure Way. We have plenty of information about how to align a software team to become effective. Once it's clear what the team is trying to accomplish, add those few measures to the scorecard. If the measures have an acceptable threshold, add that into column F. This will cause the auto highlighting to work, coloring green for numbers within the thresholds and red for numbers outside the threshold.

Establishing quality

The first pillar we suggest you measure is quality. This should be the first priority of any software team. Without it, a team cannot be effective. Without consistently high quality, the team will constantly be circling back to diagnose, analyze, and fix defects. This tends to accumulate, and teams without quality end up having little time left over to actually work on new features or valuable changes.

We recommend a few essentials when it comes to measuring quality. - Defects Caught - Defects Escaped - Defects Repaired - Mean Time to Resolve

Ultimately, you want zero defects to escape into production. But you also want to track the defects caught before production. Think about it, every time to move a card to the left on your work tracking board, that signifies a problem that has to go backward in your process to be corrected. That's a defect. Track it.

Achieving stability

The stability pillar looks at what is happening with software running in a production environment, serving customers. Two of the DORA metrics live here as well as a couple of others. Our goal is to empower our team to deploy changes frequently to production and at any time during the week, all without business disruptions. Additionally, we want to know that the software runs in a way that supports the users, again, without business disruption. Software spends most of its useful life in a state of slow changes but running day to day yielding its return on the investment made in it. The slow changes are mostly changes required so that they can be properly maintained. The measures we recommend for this pillar at a minimum are: - Number of deployments - Major production issues - Minor production issues - MTTR, mean time to recovery

Regardless of the service desk system you use to track production issues, there are always more statuses than you need, so choose which statuses represent a business disruption and which ones do not. There will always be production issues from time to time. The key is to never have a business disruption due to it.

Increasing speed (productivity)

Our last category on the scorecard is for the Speed pillar of the Clear Measure Way. This is where we track the productivity of the team. The throughput of new features and valuable software changes. It is appropriately last because quality and stability must take priority over it if we have any hope of speed that is acceptable to the business.

These measures are very simple and follow the DORA model as well. - Items Delivered - Work in Process (WIP)

Because we are tracking a new value every week, we know the cycle time by comparing the number of items in process and the items delivered. Kanban research has some good findings for thresholds of WIP that tend to work. My favorite is to start with a value equal to the number of members of the software team. This allows each team member to be working on one item at a time. Then, as you are confident, you can increase this threshold as you verify that the items delivered each week are increasing and not decreasing.

When it comes to measuring the throughput or speed of your software team, those two are typically sufficient. But as you go along, your team may want to measure additional items if you find them valuable.

Mechanics of measurement

One often cited reason for software teams not reporting up to executives with a scorecard is the administrative time it takes to compile the numbers, put together the report and answer questions that invariably come back down. But any department in any business could give those same reasons.

Forty years ago, Fred Brooks wrote an essay in his Mythical Man-Month book called "The Surgical Team". In this essay, he lays out a framework for the ideal software team structure. Effective large teams end up with a network made up of many of these team units. Large teams typically aren't effective without subdivision into a structure similar to Mr. Brooks's structure. This is probably where the notion of "Feature Teams" came from, but I digress. In this essay, Mr. Brooks discusses a team secretary. This role is responsible for all the recordkeeping, logistics, and outside communication for the team, much like what is necessary for a surgical team in an operating room. Surgeons need to stay focused on the patient, so there is a need for someone to enable them to do just that.

Each team should have a non-engineer responsible for administrative excellence for the team. Without this role, we frequently see teams that underperform not because of a lack of engineering prowess, but because of completely non-technical administrative misses. In short, managing a scorecard is an administrative task, so it should be done by someone strong in that area, even if an engineer is asked for a particular number.

Conclusion

To conclude, every effective software team needs a scorecard. The scorecard is the basis for a periodic report to a company's executive team. It answers so many questions, such as "how much can my team deliver". Without a scorecard, all we know is how many hours the team works. It doesn't do a company much good to have a team that works 100-hour weeks if the production system is brittle and new changes take months to implement. A scorecard tells us how our team is doing now, and it tracks our progress as we implement the principles and practices in the Clear Measure Way on our journey to a fully effective and high-performing software team.

Thanks to Clear Measure for sponsoring this sample and episode of Programming with Palermo.

This program is syndicated on many channels. To send a question or comment to the show, email [email protected]. We’d love to hear from you.

To use the private and confidential Chaplain service, use the following
Gentleman: 512-619-6950
Lady: 512-923-8178