The fallacy of maturity assessments

Maturity assessments are a kind of qualitative audit during which the current “maturity” of an organization is compared to a maturity reference model and ranked accordingly to its score.

As explained in the wikipedia article about maturity model (https://en.wikipedia.org/wiki/Maturity_model), the implementation is either top-down or bottom-up, but from my experience it is mostly top-down. The desired maturity score is set by the corporate top management in its desire to bring the organization to a minimum level of maturity about… Lean, Supply Chain practices, project management, digital… you name it.

The maturity assessment is usually quite simple: a questionnaire guides the assessment, each maturity level being characterized by a set of requirements. It is close to an audit.

The outcome of such an assessment is usually a graphic summary displaying the maturity profile or a radar chart, comments about the weak points / poor scores and maybe some recommendation for improvement.

The gap between the current maturity and the desired maturity state is to then to be closed by an action plan or by following a prescribed roadmap.

What’s wrong with maturity models/assessments?

1 – The fallacy of maturity assessments

A maturity assessment would be ok if it would be considered for what it is: a maturity assessment. But the one-dimensional assessment is too often used as a two-dimensional tool by assuming that the level of operational performance is positively correlated to maturity.

In other words: the better the maturity, the better the operational performance.

Indeed, such a correlation can frequently be found, but correlation isn’t causation, which means that there is no mechanical nor systematic link between the maturity and performance level.

Even so the high level of maturity matches a high level of performance and vice-versa, there is no guarantee that performance will raise if maturity is raised.

Furthermore, studies have shown that there are exceptions and organizations with low maturity perform better than some high maturity ones. You may be interested reading my post How lean are you part 2, about Awareness / performance matrix about this subject.

Therefore the belief in the positive correlation between maturity and performance that makes it a kind of law is flawed or is nothing more than wishful thinking.

Many organizations boast about their high maturity, the number of kaizen events, number of workshops, number of colored belts, the number of training sessions or worker’s suggestions but there is nothing impressive to be noticed on the gemba.

Now I can understand why most managers and improvement champions like the sole maturity assessment:

  • it is much easier to do
  • the assessment items can be common to very different units with different activities
  • the general roadmap and global target are easy to set
  • maturity objectives are qualitative

On the other hand:

  • measuring overall performance that can be compared can be more tricky, especially in an organization with several different core businesses
  • it is annoying to admit that all efforts to raise maturity are not paying-off in terms of performance and painful to explain why

2 – The one-fits all maturity targets

Another problem with maturity assessment is that some corporations dictate a minimum maturity level regardless to local realities.

That’s how some subsidiaries doing well with regards to performance get bad maturity scores because they do not apply SMED (Single Minute Exchange of Die, an approach to reduce the changeover duration). The point is these subsidiaries have more or less continuous production processes with huge batch sizes that barely change. Why would they go for SMED when they don’t need it? The same case can be told with one-piece flow or heijunka (load levelling) enacted as a must do.

Others are scoring poor because they didn’t Value Stream Map (VSM) their processes. The fact is that those units had no problems a VSM could help to solve. The example list can go on and probably, dear reader, you have faced such situations yourself (leave your testimony in the comments..!)

3 – Doing it to be compliant, not because it makes sense

This third point is a corollary to the previous one. Because the objectives have been set at higher level and in order to be compliant, most unit manager will pay lip service to the dictated targets, get the scores good enough and be left alone once the assessment is done.

The local staff recognizes the nonsense of the demanded score, yet goes for the least effort and instead of fighting against the extra unnecessary work, chose to display what top management wants.

This the typical “tell me how you’re measured, I tell you how you behave” syndrome inducing counterproductive behaviors or practices.

While top management will be pleased with the scores enforcing its flawed belief, the local units managers did not embrace at all the practices, tools or methods prescribed. They only camouflaged the reality.

Wrapping up

Maturity assessment are not a bad thing per se, but their practicality and simplicity are often misused to assess more than just maturity (or awareness). This is most often misleading because of the false underlying assumptions and promoting wrong behaviors and practices.


PS: You may be interested to read Michel Baudin’s comments on his own blog about this post: http://michelbaudin.com/2017/08/22/the-fallacy-of-maturity-assessments-chris-hohmann/


View Christian HOHMANN's profile on LinkedIn

Advertisements

Is Lean dead?

Is Lean dead? is the provocative title of a podcast hosted by Mark Graban with guest Karen Martin. The question, the podcast description says, is “easy to discuss, but hard to answer”.

The reason Karen proposed to discuss this question is because of hearing and reading about “what’s next”, “is it time for something new?” and what seems to be a waiting for a “post-Lean” world.

There is an invitation to share thoughts about this, but instead of leaving a comment on Mark’s podcast-related post, I add my two cents here. I hope Mark and Karen won’t mind this piggybacking.

Some takeaways from the podcast

From 4:35 Karen frames the topic and from 6:05 she shares her thoughts, especially two reasons for the “what’s next?” question (rephrased by me):

  1. people don’t get great results from Lean and quit too early with the conclusion Lean doesn’t work
  2. the attention span, especially in business, is (very) short

By 19:45, Karen reminds the listeners that Lean is made of layers of quite “meaty” subjects and is made for constant learners. Yet Lean teaching programs cannot (?) do not go very deep into Lean beyond chosen tools. So it’s up to everyone to go for a never-ending learning journey. At 43:00 listeners get the wrap-up.

My thoughts on this

Ironically, the introduction of new methods and tools was once mocked as “the flavor of the month” with the preconceived belief that it won’t last. Now that Lean has demonstrated a longer lifespan than other management ways, it seems to be precisely too lasting in a time where  fashions come and go very quickly.

The methodologies life cycle

However long Lean is around now, it follows the same life cycle curve than others, made of a slow takeoff as long as long as no organization publicized extraordinary achievements with this approach or no book draw wider attention to it. Once the word spread, the methodology gets hype and many organizations and consultants go for it. After the hype spike, there is a loss of interest and a final plateau. In this later stage the methodology does not totally disappear but does not get the attention it once had.

In that regard, the methodologies life cycles look very much like Gartner’s hype cycle for technologies.

I started my career in the midst of Total Quality Management (TQM) hype, in the mid-1980s. Who aged less than 40 knows about TQM? It is still around in some form, like in the various ISO standards, but it does not get the excitement of the all the problem solving tools deployment TQM once had.

Similarly what happened to Total Productive Maintenance (TPM)? Parts of it is embedded in Lean and some companies keep TPM alive, but I barely hear anything about it anymore.

My friend and mentor Bill Dettmer witnessed the same phenomenon with Theory of Constraints. Not totally gone now, but barely known and not likely to get its fame back again. Six Sigma is said to be dead or is reduced to SIPOCs and DMAIC.

So maybe time has come for Lean to lose attention of the mass and remain a thing for true believers?

Quick wins and newcomers

The impatience about the post-Lean next thing can also come from the younger staff that did not experience the first attempts with Lean, when the organization was so inefficient that almost any structured tool deployment and kaizen events demonstrated significant quick wins. After a while and continuous sustained efforts, the remaining pockets of gains are few and hard/long to address. Newcomers experience Lean from hearsay or don’t notice anything about Lean because they are amidst of a more or less Lean environment. It’s just part of the scenery and nothing to get excited about.

Furthermore, many people have been repeatedly exposed to Lean methods and tools, have been involved in Lean workshops, kaizen events, sketched Value Stream Maps and identified wastes, sorted out, cleaned up and rearranged stuff 5S style. They share the feeling of being Lean, of being done with Lean.

The startup praise of failing fast and pivoting

My last thought about the possible fading of Lean is the growing influence of the startup movement and the praise of failing fast and pivoting. It keeps surprising me that failure can be praised, even so I understand the value of learning from failures. One Undesirable side Effect though may be the spreading of the belief that anything that does not work quickly is a failure – ok, we learned something – and it’s time to move on with something else.

Pivoting is getting away from an original idea that does not prove good fast enough and go for something else that can be 90 or 180 degrees from the original intent. What can make sense in a startup venture, stopping the experiment before the scarce resources are burnt up, may not be the suitable option where long commitment to cultural change and constant learning is required.

With allusion to Kahneman’s work, I think that for business there is a fast way and a slow one, and Lean is definitely a slow one.

View Christian HOHMANN's profile on LinkedIn

Jargon doesn’t make an expert

In a blog post I read the warning about candidates exaggerating their insight by using lot of jargon. It was about Lean Management. The author stated that when recruiting, mastering enough Lean vocabulary is important in order to catch candidates exaggerating their insight by using jargon. Any talented Lean manager can explain the same concepts without Lean management specific language, the author wrote, but inexperienced or unskillful interviewees may lean (pun intended?) on “concept-dropping.

Even so I agree with everything above, the heavy use of peculiar lingo is not specific to Lean and Lean “experts” are not even the worst.

>Lisez-moi en français

I remember a recent (July 2017) conference in which a speaker delivered a pretty convincing presentation about a somewhat uncommon approach we are familiar of. One of my colleagues, intrigued, went to see the speaker and asked him a question on a specific aspect only a true experienced expert could answer. This very question reminded the speaker of an important call he had to make and he vanished. He was indeed only “concept-dropping”.

Nothing really new. Molière, our most famous (French) playwright and actor (1622 – 1673) used to ridicule the physicians of his time in several of his works. Those experts were depicted as pompous and disputing in fantasy latin about this or that just to impress their audience or others fellow “experts” with fake erudition, while their patients usually were bleeding away.

In French slang, a “faisan” (pheasant) is a crook, a good-looking but stupid pretender. I used to hear fake experts being called “faisans”. Nice feathers, but that’s barely all.

View Christian HOHMANN's profile on LinkedIn

Did you already SWOT yourself?

SWOT stands for Strengths, Weaknesses, Opportunities, and Threats and SWOT analysis is often used to assess a project, a venture or reflect on the organization’s relative forces compared to competition and business.

A SWOT analysis can be done on one’s self in order to get clarity about one’s Strengths, Weaknesses, Opportunities, and Threats, before a job interview or when facing an important decision.

“SWOTing” oneself can also be useful just to get clarity on one’s status, so to say. Being aware of these four dimensions helps to get clarity about thyself and to take decisions with more than just gut feeling. Let’s start with Strengths and Weaknesses

Strengths and Weaknesses are very self-centered. It’s all about individual traits and how they compare to others. Of course, strengths and weaknesses are relative to the circumstances and the self-assessment should be done with a specific “use case” in mind.

Strengths

List your distinctive strengths, what you are really good at, what makes you different from colleagues and other people, what makes you stand out of the crowd, and would be a real advantage over similar profiles. Be honest.

Strengths must be specific and difficult or long to acquire for your “competitors”.

Weaknesses

Being clear about own weaknesses sets the boundaries about what you can do and what not. Awareness about your own limitations and weaknesses will probably prevent you to try something out of reach or likely to fail.

Weaknesses may be disqualifiers when applying for a job or when looking for a promotion.

It is also an indication of what to improve – if possible – and about what your competition is potentially better.

Now to the external factors. Circumstances and social and professional environment are changing constantly, providing new Opportunities but also exposing to Threats.

Opportunities

Circumstances and environment at large may provide personal development or new professional opportunities. Clarity about one’s strengths and weaknesses helps to decide to seize an opportunity or to get prepared for it. Sometimes the gap is too big and thanks again to clearly knowing one’s limits, the right decision can be made.

Threats

Everything is going VUCA, an acronym for Volatile or unstable, temporary, Uncertain, Complex and Ambiguous. Threats are somehow the flip side of the Opportunities: what can be a real chance for one can be disastrous for another.

Threats can come in a form of a new “competitor” or a technology that trump your skills or make you as a contributor… unnecessary.

Threat can be the obsolescence of your knowledge, the decline of some abilities in time…

Threats can come in so many ways that it is wise to question them about being plausible and their probability. Take into account only the most plausible and likely ones.

Once clear about the exposure to risks, figure out how to mitigate or bypass them.


Related: When facing a choice, get clarity with the change matrix

View Christian HOHMANN's profile on LinkedIn

When facing a choice, get clarity with the change matrix

The change matrix is primarily a tool to explain why people seem to resist change, but it can be used to make a decision when the appeal of the proposed change is facing some doubts about losing more than gaining.

Doing the exercise of filling the matrix should help getting clarity about the plusses and minuses of the change, and base the decision on some rational weighing.

In order to understand the matrix and the associated metaphors, I recommend watching the video.


When facing a choice with significant impact on current and future situation

An envisioned or proposed major change in life can be scary. Who never faced the dilemma of daring a change and face the uncertainty or keep everything as is for the sake of some “safety”?

The safety here can be nothing more than an illusion, but the familiarity of the current situation gives some impression to remain in control. In the current situation, everything seems predictable and known while a change will modify many things, adding a lot of unknown and uncertainty.

Furthermore the popular saying states that every improvement is a change but every change isn’t an improvement, adding to the fear of giving up something good for worse.

Relying only on gut feeling may not be the best way to make the decision unless one trusts his or her intuitions. The change matrix can bring some clarity when the exercise is done honestly.

Pot of gold and mermaid

Write down all promoted benefits as well as those the intuition suggests. What makes the change desirable and that CANNOT be gotten or achieved in current situation?

Switch to the mermaid and ask yourself what would make you ignore the pot of gold, something of great value ONLY provided in current situation.

The capital letters stand for extreme wording, a technique useful for identifying false assumptions. If it sounds weird or not true, the assumption is probably false or overstated.

Crutches and alligators

Assess the risks of change figured by the crutches. What can possibly go wrong with the change that WILL end up with SIGNIFICANT damage?

On the other hand what CATASTROPHE WILL happen by keeping the status quo?

Looking at the matrix

It is time to look at all quadrants and check in which direction the matrix points. Hopefully a clear indication is shown, either favoring change or recommending to stay put.

The last time I applied the matrix to a personal important choice I was surprised how clear the best choice appeared.

It was consistent with my intuition but was more elaborate, thus added much clarity to the best choice. The result could have been opposite and could have put a rationalized end to a fantasy. The clarity and the list of pro and cons gives great confidence about the decision to make. I really recommend to give it a try.

Possible biases

When facing a desirable change, one may overestimate the size of the pot of gold as well as the threat of alligators while underestimating the risks (the crutches) and the sex-appeal of the mermaid.

In plain English this means overestimate the gain or benefits of the change as well as the potential danger of not changing, thus making the change desirable. This looks much like fulfilling a self prophecy.

In order to complete the demonstration or reinforce the desirability of the change, the risks associated with the change are minimized or ignored and the advantages of the status quo downplayed.

Conversely, when facing a less desirable change and even more in case of an undesirable change, the person may evaluate the quadrants in an opposite manner: overestimating the number of crutches and the sex-appeal of the mermaid while underestimating the value in the pot of gold as well as playing down the threat of alligators.

Again the translation in plain English: to justify the rejection of a proposed change the risks of the change are magnified and the advantages of the status quo highlighted while the benefits of the change are questioned and the threat of not changing minimized or even denied.

I  order to avoid this pitfall, it is meaningful to share all (emphasize “all”) the elements of the choice in the most neutral manner to a person of confidence or (someone selected as) a coach. A new external point of view may question the rationale and propose a new perspective.


View Christian HOHMANN's profile on LinkedIn

Reader question: Goal Tree vs. Current Reality Tree

Here is a reader’s question: I have difficulty seeing the difference between the Goal Tree and the  Current Reality Tree (CRT). With these two trees we assess the process. What are the main differences between the two?

The Goal Tree and Current Reality Tree (CRT) have nothing in common. They are not even meant to care about processes but about the system as a whole. Neither the Goal Tree nor the CRT are process maps.

>Lisez-moi en français

A Goal Tree lists all Necessary Conditions to achieve a Goal, which is not yet achieved, so it is about the future.

The CRT describes why the Goal is not yet achieved in the current state. It starts with identified Undesirable Effects (undesirable for the system as a whole) and drills down to the few critical root causes.

A Goal Tree is built from top-to-bottom with necessity logic while the Current Reality Tree (CRT) is built from top-to-bottom using sufficiency logic. This building top-to-bottom is maybe the sole commonality between the two.

To learn more about the differences between necessity and sufficiency logic, check out my post: Goal Tree Chronicles – Enablers vs.triggers

The name Current Reality Tree is somewhat misleading because the CRT is limited to the description of the negative outcomes. It does not describe all the Current Reality. This is saving a lot of unnecessary analysis as well as a warning to not mess with what is currently producing Desired Effects!

What could have caused some confusion to my reader is the fact that a Goal Tree is a benchmark against which to measure the gaps in current reality.

When doing this I use a 3-color code to indicate each Necessary Conditions status. I assess the current condition of the system with the Goal Tree as benchmark. The first autumnal-colored tree should be kept as is as a snapshot of the situation at the beginning. Distinct trees are used later to monitor the progress of ‘greening’ the tree, i.e. closing the gaps to achieve the Goal.

I hope this helps to understand the differences between a Goal Tree and a Current Reality Tree.

View Christian HOHMANN's profile on LinkedIn

Scrutinizing and improving a Current Reality Tree (video tutorial)

In this video, I scrutinize and suggest improvements on a Current Reality Tree (CRT) found on the Internet. A logically sound CRT is key to convince audience about the robustness of the analysis and the reality of the causes to the trouble. If there is room for doubt or the logical has flaws, chances are that the audience will not buy-in, especially those having some “skin in the game”…


View Christian HOHMANN's profile on LinkedIn