The fallacy of maturity assessments

Maturity assessments are a kind of qualitative audit during which the current “maturity” of an organization is compared to a maturity reference model and ranked accordingly to its score.

As explained in the wikipedia article about maturity model (https://en.wikipedia.org/wiki/Maturity_model), the implementation is either top-down or bottom-up, but from my experience it is mostly top-down. The desired maturity score is set by the corporate top management in its desire to bring the organization to a minimum level of maturity about… Lean, Supply Chain practices, project management, digital… you name it.

The maturity assessment is usually quite simple: a questionnaire guides the assessment, each maturity level being characterized by a set of requirements. It is close to an audit.

The outcome of such an assessment is usually a graphic summary displaying the maturity profile or a radar chart, comments about the weak points / poor scores and maybe some recommendation for improvement.

The gap between the current maturity and the desired maturity state is to then to be closed by an action plan or by following a prescribed roadmap.

What’s wrong with maturity models/assessments?

1 – The fallacy of maturity assessments

A maturity assessment would be ok if it would be considered for what it is: a maturity assessment. But the one-dimensional assessment is too often used as a two-dimensional tool by assuming that the level of operational performance is positively correlated to maturity.

In other words: the better the maturity, the better the operational performance.

Indeed, such a correlation can frequently be found, but correlation isn’t causation, which means that there is no mechanical nor systematic link between the maturity and performance level.

Even so the high level of maturity matches a high level of performance and vice-versa, there is no guarantee that performance will raise if maturity is raised.

Furthermore, studies have shown that there are exceptions and organizations with low maturity perform better than some high maturity ones. You may be interested reading my post How lean are you part 2, about Awareness / performance matrix about this subject.

Therefore the belief in the positive correlation between maturity and performance that makes it a kind of law is flawed or is nothing more than wishful thinking.

Many organizations boast about their high maturity, the number of kaizen events, number of workshops, number of colored belts, the number of training sessions or worker’s suggestions but there is nothing impressive to be noticed on the gemba.

Now I can understand why most managers and improvement champions like the sole maturity assessment:

• it is much easier to do
• the assessment items can be common to very different units with different activities
• the general roadmap and global target are easy to set
• maturity objectives are qualitative

On the other hand:

• measuring overall performance that can be compared can be more tricky, especially in an organization with several different core businesses
• it is annoying to admit that all efforts to raise maturity are not paying-off in terms of performance and painful to explain why

2 – The one-fits all maturity targets

Another problem with maturity assessment is that some corporations dictate a minimum maturity level regardless to local realities.

That’s how some subsidiaries doing well with regards to performance get bad maturity scores because they do not apply SMED (Single Minute Exchange of Die, an approach to reduce the changeover duration). The point is these subsidiaries have more or less continuous production processes with huge batch sizes that barely change. Why would they go for SMED when they don’t need it? The same case can be told with one-piece flow or heijunka (load levelling) enacted as a must do.

Others are scoring poor because they didn’t Value Stream Map (VSM) their processes. The fact is that those units had no problems a VSM could help to solve. The example list can go on and probably, dear reader, you have faced such situations yourself (leave your testimony in the comments..!)

3 – Doing it to be compliant, not because it makes sense

This third point is a corollary to the previous one. Because the objectives have been set at higher level and in order to be compliant, most unit manager will pay lip service to the dictated targets, get the scores good enough and be left alone once the assessment is done.

The local staff recognizes the nonsense of the demanded score, yet goes for the least effort and instead of fighting against the extra unnecessary work, chose to display what top management wants.

This the typical “tell me how you’re measured, I tell you how you behave” syndrome inducing counterproductive behaviors or practices.

While top management will be pleased with the scores enforcing its flawed belief, the local units managers did not embrace at all the practices, tools or methods prescribed. They only camouflaged the reality.

Wrapping up

Maturity assessment are not a bad thing per se, but their practicality and simplicity are often misused to assess more than just maturity (or awareness). This is most often misleading because of the false underlying assumptions and promoting wrong behaviors and practices.

---

PS: You may be interested to read Michel Baudin’s comments on his own blog about this post: http://michelbaudin.com/2017/08/22/the-fallacy-of-maturity-assessments-chris-hohmann/

---

Follow @HOHMANN_Chris