In my teaching, consulting, and general discussion with students and practitioners, I have noticed that many people are eager to learn the mechanics of different tools, e.g. how to choose a specific type of process control chart or how to determine the right parameters for cumulative sum (CUSUM), so they can get the job done.  But few ask the question: “what makes the tool effective in the real world?”

In the case of a control chart, a crucial condition that makes the control chart effective is process standardization. 

Continuous Improvement (CI) professionals know that standardized work is a fundamental principle of the Toyota Production System (TPS) or Lean.  Standardization minimizes process variation, which enables greater sensitivity in detecting special cause variation by the control chart.

Many people don’t realize that even if a control chart shows no special cause variation, it does not mean that the process is in statistical control.  In many cases, such as processes that lack standardization, there are simply too many uncontrolled variables present and they become part of the process.  But these variables are not inherent to the process and inflate the common cause variation.

The accompanying figure shows a hypothetical example.  The top chart shows a stable process, except that points 51 to 55 have a positive deviation of 20.  The individuals chart (or I-chart) detects the change.  The bottom chart is the same process with the same positive deviation for points 51 to 55, but some random deviations (or noise) are added. The control limits are more spread out, and the special cause variation is not detected.

Additional noise reduces the ability of a control chart to detect change.

In my observation of real processes, many contain both special cause variation and the additional noise illustrated above.  So naturally, the CI professionals tend to focus on reducing the special cause to bring the process back in control. However, with the persisting noise the process never reaches its true state of control.

The additional noise can come from many sources.  A major source is lack of standardization. 

In a regular production environment, operating procedures have room for interpretation and thus can lead to process variation.  In my experience in R&D and manufacturing, many people honestly believe that they follow the same procedure each time.  But upon careful investigation, deviations are common.

Those familiar with gage repeatability & reproducibility (R&R) studies appreciate the potential for human errors or deviations. Using a well-established measurement procedure, the same operator can still have varying results measuring the same items (i.e. repeatability error).  Different operators likely introduce additional variability (i.e. reproducibility error).  In a less standardized process, there are many more opportunities for deviation.

The effectiveness of standardization to reduce noise is limited by our understanding of the design space and critical process variables.  Because many processes are not well studied and designed using Quality by Design (QbD) principles, some residual noise will likely remain after standardization.

In summary, if you want to improve change detection, make sure that you identify the sources of the extra noise in the process and operationally control them.

Over 15 years ago, I received my LSS Black Belt (BB) training sponsored by my employer.  It was three weeks of classroom training delivered over three months by external consultants.  It kick-started my Continuous Improvement (CI) journey.  Since then, I have delivered LSS training as an internal trainer or external consultant to many large global organizations.  I also helped organizations in their LSS deployment, led many CI projects, and coached Green Belt (GB) and BB leaders in their projects.

Despite my own positive experience with LSS training, what I have learned over the years is that in most situations the traditional weeks long LSS training is ineffective in driving CI. 

If measured by the number of people trained or certified or the number of methods and tools covered, such training programs are very effective and easily justified for the investment.  

But if we start to measure the improvement of business outcomes, the desired problem-solving skills and behavior of the trained employees, and the positive impact on the CI culture and mindset of the organization, the training is very often ineffective.  Some troubling signs are

• It took 12 months or more to complete the first GB project.
• The GB could not recall some basic topics only a few weeks after the training.
• BB candidates have to create flash cards to prepare for their certification exams.
• GBs or BBs are no longer engaged in CI after obtaining their certifications.
• Certified BBs fail to exhibit or apply knowledge of some fundamental concepts, such as process stability, in their daily work.
• The trained employees do not perform or behave differently from those untrained in the CI methodology

I can see two main factors contributing to this poor outcome.

First, the training program only teaches the general methods and tools and does not improve skills.

Previously, I discussed training and coaching considerations in LSS deployment in The First Six Sigma Project and recommended customized training in Making Employee Training Effective.

Most LSS training programs developed by universities, professional organizations, and commercial vendors are designed for efficiency and profitability. The generic programs do not connect the content to the client organization’s problems and operational reality.  Few external trainers have the subject matter or industry knowledge to tailor the training to each client’s need.  Even if they are able to customize, few clients are willing to pay the substantial premium.

Corporate internal programs are not much better in terms of sufficiently relevant materials that relate to each employee’s job.  Employees do not start learning real problem-solving skills until they encounter problems in their projects, by which time they already forgot most of what was taught in the training. 

Second, the organization overly relies on training to improve business performance.

Two common fallacies can lead to this “improvement training trap.”

1. Employees have to be trained in the methods and tools or they won’t be able to learn themselves.
2. Once the employees are formally trained, they will solve all the problems on their own.

Can classroom training help accelerate learning? Absolutely.  Is it necessary or sufficient to develop the skills, mindset, and behavior for CI?  No.

These programs train methods and tools, whereas what the organizations really need is leadership development and behavior modification.  

Management has to understand that employees’ knowledge in CI methodologies is only a small but essential driver in business improvement.  When employees are not engaged in effective CI activities, it is not necessarily due to lack of knowledge – something else is likely limiting.  The root cause is rarely lack of training, and the solution is not more or even better training.  

It is management’s job to critically analyze all aspects of the organization, e.g. processes, structure, policies, resources, people, and culture, to identify the barriers to CI.  When they do, they will likely find out that LSS training is not the solution to their problem.

“The Project Economy is one in which people have the skills and capabilities they need to turn ideas into reality.” — The Project Management Institute (PMI)

When people fail to turn ideas into reality or when businesses fail to turn strategies into results, a common root cause is that people and organizations lack the right skills and capabilities.

The people include project managers (regardless of their formal titles), team members (or contributors), and project sponsors (management).  Almost everyone is involved in a project in organizations.  The project management (PM) maturity of an organization depends on the skills and capabilities of all people, not just the project managers.

What are the skills and capabilities required for project success?

The PMI Talent Triangle® outlines three skill sets:

• Technical Project Management
• Leadership
• Strategic and Business Management

It is the combination of these skills possessed by people throughout the organization that is required to realize the idea or strategy. In other words, we need adequate skill levels in all nine cells in the role vs. skills matrix (pictured above).

As a project manager, line manager, or consultant in the industry in the past two decades, my observation is that most organizations have low PM maturity, and PM skill development is focused in technical project management for the project managers, i.e. only one of the nine cells.

While the traditional PM skills (such as scope and time management) are critical, they are insufficient because of the increasing complexity, ambiguity, and uncertainty in the problems we try to solve using projects.  Empowering project teams is impossible if project managers and members do not understand the business priority and strategy to make the right decisions.  Unless they demonstrate their business acumen and ability to think strategically, project managers will not be fully empowered.  In my previous blog “Project Managers are Managers,” I offered some suggestions to new project managers to help them think strategically and manage stakeholders effectively.

I was very pleased that the PMI developed the Talent Triangle to help close the skill gap in project managers.  Furthermore, I’d say that we have to assess and develop PM skills for everyone in the organization, for two reasons. 

First, we prepare future project managers for their roles.  People don’t become a competent project manager overnight – it takes years of learning and practice before and after they are given the project manager role.  Even if not in a PM role, each person is leading projects of different sizes and complexity and can benefit from the skills.

Secondly, to ensure project success, project sponsors and members should have the PM skills to perform their roles effectively.  Otherwise, the project managers have to spend much time doing technical project management, unable to focus on the big picture – the business and strategic value.  When project sponsors do not know how to manage projects at the strategical level, their management practices can lead to project problems or failures that even the best project managers cannot prevent.  I touched on some of the practices in my blog “Projects on Schedule.”

How are you assessing and developing PM skills in your organization?

Understandably, many CI projects are chartered because there is a need to improve business performance.  But if the opportunity or path to improvement is not clear, the project has a high risk of failure¹.  Even if the goal was somehow achieved, it likely took far more time and effort than necessary, as evidenced by many 6-12 month long Green Belt (GB) projects. 

While CI professionals are often trained to not assume a solution or even a root cause in Lean Six Sigma projects, the approach to the problem should be well defined for the specific problem.  DMAIC or similar one-size-fits-all frameworks are too generic to be helpful.  Because most project leaders do not have enough experience to identify the right opportunity for a CI project and follow a proven path to improvement, it is essential that the organization implements an effective system to differentiate opportunities into categories suited for distinct approaches.  For example, the categories can include

• Routine improvement by the operators
• Kaizen events
• Lean Six Sigma or DMAIC projects  
• Technical projects that require Subject Matter Experts (SMEs)
• Management

The system will vary by the organization.  In a manufacturing or transactional environment where CI methodologies are applied, I recommend process management as a basic component of the system.  Specifically, process management should include, but is not limited to

• Process mapping to understand how things are being done
• Standardization to implement the best knowledge currently available
• Standard Operating Procedures (SOPs) up to date
• Employees trained and qualified to perform the jobs
• All preventative maintenance followed
• Measurement system analysis to ensure reliable data
• Statistical Process Control (SPC) to monitor and stabilize processes

These foundational activities are prerequisites for any process to perform at its optimal level that is achievable by design.  If these activities are not consistently followed, even an initially high-performing process will deteriorate. 

Any organization striving to improve their processes should start by incorporating these activities into the responsibilities of various roles.  Following these activities will regularly uncover many improvement opportunities, most of which can be accomplished by those who are closest to the process. If needed, Quality and CI professionals can train and coach others proper methods and tools. 

Ideally, process management should be implemented before initiating CI projects in the area.  Routine improvement as a result of process management eliminates countless potential root causes for poor process performance, reducing the need for project-based improvement effort.  Any CI projects, if needed, will have a clearer focus, less encumbered by confounding factors.

When organizations fail to build process management in their operations, CI projects are often initiated as a reaction to emergent problems, which are likely due to years of neglect.  They hope, with the aid of some magic methodology and heroic efforts, that the projects alone will solve the problems.  What they encounter, however, are numerous causes that compound the problem, making a “perfect” situation for inexperienced project leaders in a low CI maturity organization.

Thus projects are a tool of continuous improvement.  They are not a substitute for it.²

1. See my blog Six Sigma Project Management for suggestions to reduce project risks.

2. I borrowed a statement by Peter Drucker when he discussed merger & acquisition — “Thus financial transactions are a tool of business policy.  They are not a substitute for it.” 

Conceptually, process capability is simple.  If a process makes products that meet the customer requirements all the time (i.e. 100%), it has a high process capability.  If the process does it only 80% of the time, it is not very capable.

For quality attributes measured as continuous or variable data, many organizations use Process Capability Index (Cpk) or Process Performance Index (Ppk) as the metric for evaluation.  In my consulting work, I often observe confusion and mistakes applying the concept and associated tools, even by Quality and CI professionals.  For example,

• Mix-up of Cpk and Ppk
• Unclear whether or when process stability is a prerequisite
• Using the wrong data (sampling) or calculation
• Misinterpretation of process capability results
• Difficulty evaluating processes with non-normal data, discrete data, or binary outcomes

The root cause of this gap between this simple concept and its effective application in the real world, in my opinion, is lack of fundamental understanding of statistics by the practitioners.

Statistics

First, a process capability metric, such as Cpk, is a statistic (which is, by definition, simply a function of data).  The function is typically given as a mathematical formula.  For example, mean (or the arithmetic average) is a statistic and is the sum of all values divided by the number of values in the data set.   

The confusion between Cpk and Ppk often comes from their apparently identical formulas, with the only difference being the standard deviation used.  Cpk uses the within-subgroup variation, whereas Ppk uses the overall variation in the data.  Which index should one use in each situation?

It is important to understand that any function of data can be a statistic – whether it has any useful meaning is a different thing.  The formula itself of a statistic does not produce the meaning.  Plugging whatever existing data into a formula rarely gives the answer we want. 

To derive useful meaning from a statistic, we must first define our question or purpose and state assumptions and constraints.  Then we can identify the best statistic, gather suitable data, calculate and interpret the result. 

Enumerative and Analytic Studies

Enumerative and analytic studies¹ have two distinct purposes. 

• An enumerative (or descriptive) study is aimed to estimate some quantity in the population of interest, for example, how many defective parts are in this particular lot of product? 
• An analytic (or comparative) study tries to understand the underlying cause-system or process that generates the result, for example, why does the process generate so many defective parts?

If the goal is to decide if a particular lot of product should be accepted or rejected based on the number of defective parts, then it is appropriate to conduct an enumerative study, e.g. estimating the proportion of defectives based on inspection of a sample from the lot.  A relevant consideration is sample size vs. economic cost – more precise estimates require larger samples and therefore cost more.  In fact, a 100% inspection will give us a definite answer.  In this case, we are not concerned with why there are so many defectives, just how many.

If the goal is to determine if a process is able to produce a new lot of product at a specified quality level, it is an analytic problem because we first have to understand why (i.e. under what conditions) it produces more or fewer defectives.  Methods used in enumerative studies are inadequate to answer this question even if we measured all parts produced so far.  In contrast, control charts are a powerful analytic method that uses carefully designed samples (rational subgroups) over time to isolate the sources of variation in the process, i.e. understanding the underlying causes of the process outcome.  This understanding allows us to determine if the process is capable or needs improvement.

Cpk versus Ppk

If our goal is to understand the performance of the process in a specific period (i.e. an enumerative study), we are only concerned with the products already made, not the inherent, potential capability of the process to produce quality products in the future.  In this case, demonstration of process stability (by using control charts) is not required, and Ppk using a standard deviation that represents the overall variability from the period is appropriate.  

If our goal is to plan for production, which involves estimating product quality in future lots, the process capability analysis is an analytic study.  Because we cannot predict what a process will produce with confidence if it is not stable, demonstration of process stability is required before estimating process capability. 

If the process is stable, there is no difference between within-subgroup variation (which is used for Cpk) and overall variation (which is used for Ppk), except estimation errors. Therefore, Cpk and Ppk are equivalent.

If the process is not stable, the overall standard deviation is greater than the within-subgroup variation — Ppk is less than Cpk as expected.  However, Ppk is not a reliable measure of future performance because an unstable process is unpredictable.  If (a big IF) the subgroup is properly designed, the within-subgroup variation is stable and Cpk can be interpreted as the potential process capability if all special causes are eliminated.  In practice, the subgroup is often not designed or specified thoughtfully, making interpreting Cpk difficult.

In summary, process capability analysis requires good understanding of statistical concepts and clearly defined goals.  Interested practitioners should peruse many books and articles on this topic.  I hope the brief discussion here helps clarify some concepts. 

1. Deming included a chapter “Distinction between Enumerative and Analytic Studies” in his book Some Theory of Sampling (1950).

What makes a goal achievable?  In my work as a Continuous Improvement (CI) coach and consultant, I have seen some common practices setting a numerical goal using, for example

1. A target set by management, e.g. a productivity standard for the site
2. Customer requirements, e.g. a minimum process capability
3. Some benchmark value from a similar process
4. A number with sufficient business benefit, e.g. 10% improvement

At the first glance, these methods seem reasonable.  In practice, they are problematic for two reasons.

First, the goals are based on what is desirable, not sound understanding of the opportunity using data.  How do we know if a desirable goal is achievable?   In many organizations, a numerical goal is “set in stone” when the project starts; failing to achieve the goal can have potential career repercussions.  While management tends to aim for aggressive targets, the project leaders are more concerned with the risk of failing to achieve them.  They prefer a more “realistic” target that can be met or even exceeded and negotiate with the sponsors to make the desirable target a “stretch” goal.  In the end, no one knows what the real improvement opportunity is.

Secondly, the practices create a mindset and behavior inconducive to the CI culture.  I have seen too many organizations’ Lean, Six Sigma, or other CI initiatives focus only on training and project execution.  They fail to build CI into their daily decisions, operations, and organization’s culture.  Quality improvement cannot be accomplished by projects alone – numerous incremental improvement opportunities exist in routine activities outside any project.  Projects, by their nature, are of a limited duration and are merely one mechanism or component of continuous improvement. Most improvement does not require a project.  Depending on projects to improve a process is a misunderstanding of CI, reinforces reactive (firefighting) behavior, and sends a wrong message to the organization that improvement is achieved through projects, and even worse, by specialists.

Creating a project with only a desired target leads to high uncertainty in project scope, resources, and timelines – a lot of waste. 

To be effective, a CI project should have a specific opportunity identified based on systematic analysis of the process.  Furthermore, the opportunity is realized through a project only if it requires additional and/or specialized resources; otherwise, the improvement should be carried out within routine activities by the responsible people in collaboration. 

What kind of systematic analysis should we perform to identify the opportunities?

One powerful analysis is related to process stability.  It requires our understanding of the nature and sources of variation in a process or system.  In a stable process, there is only common cause variation – its performance is predictable.  If a process is not stable, there exists special cause variation — its performance is not predictable.  Depending on process stability, the opportunity for improvement and the approach are distinct. 

The first question I ask about the goal of any improvement project is “Is the current performance unexpected?”  In other words, is the process performing as predicted?  No project should start without answering this question satisfactorily in terms of process stability.  Most often the answer is something like “We don’t really know but we want something better.”  If you don’t know where you are, how do you get to where you want to be?  This is a typical symptom of a project driven by the desirability rather than a specific opportunity based on analysis.  If the process stability was examined, most likely the first step toward improvement would be to understand and reduce process variation, which does not need a project.

For people familiar with Deming’s 14 Points for Management, I have said nothing new.  I merely touched point 11 “Eliminate management by numbers, numerical goals.”  His original words¹ are illustrative.

“If you have a stable system, then there is no use to specify a goal.  You will get whatever the system will deliver.  A goal beyond the capability of the system will not be reached.”

“If you have not a stable system, then there is again no point in setting a goal.  There is no way to know what the system will produce: it has no capability.”

A goal statement that sounds SMART does not make a project smart.  A project devoid of true improvement opportunity achieves nothing but waste.  But if we follow the path shown by Deming, opportunities abound and improvement continues. 

1. Deming, W. Edwards. Out of the Crisis : Quality, Productivity, and Competitive Position. Cambridge, Mass.: Massachusetts Institute of Technology, Center for Advanced Engineering Study, 1986.

Following the DMAIC steps, the project team can think rigorously and approach the problem systematically.  Books and training materials include applicable tools for each phase and checklists for tollgate reviews. Organizations often have DMAIC templates that define mandatory and optional deliverables for each phase.  All of these are supposed to help the GBs and BBs to determine the right questions to ask and the right tools to apply along the DMAIC process.

In reality, the templates are not as helpful.  I observe many project leaders either confused with what to do in each DMAIC phase or doing the wrong things.  For example,

• Project teams include a tool or analysis simply because it’s a “required” phase deliverable, even if it doesn’t improve the process or our knowledge. 
• The project leaders are more concerned with presenting visually impressive slides to the management than understanding the process. They re-create a SIPOC or Fishbone diagram on a slide from the flipchart or white board when a snapshot is perfectly legible.
• Project teams go to a great length to document the current state electronically (e.g. in Visio) as a single process (which is futile), rather than spending the time “Go Gemba” to understand the variation.
• The project continues even after the evidence and analysis show that the project baseline or business case is no longer valid.  Instead of using the tollgate to stop or re-scope the project, the team shows various tools and analyses to justify the value of going forward.  They are afraid that terminating the project will reflect negatively on them.
• The project team is sent back to complete a deliverable at the tollgate because it is not satisfactory to the management even when the deliverable is not critical to the next step in the project.  As a result, teams always overprepare for the tollgates in fear of imperfect deliverables.
• Instead of seeing an inadequate measurement system as an opportunity re-scope the project to address it, the team is asked to demonstrate an adequate measurement system before closing the Measure phase.  They are stuck in Measure to perform Improve activities.

Why are these happening?

I discussed in my earlier blogs about some related challenges in “Starting Lean Six Sigma” and “The First Six Sigma Project.”  By understanding how Lean Six Sigma fits in the organization’s objectives, strategy, and capabilities, the leaders can choose the right deployment approach for the organization.  By selecting the right candidates and projects and by providing the right training/coaching to both sponsors and GBs/BBs, the leaders can avoid many common mistakes when the organization is in the low continuous improvement (CI) maturity state.

While the experience of the project leaders is a factor, I attribute the main cause of many Lean Six Sigma deployment issues to the organization, not the individual GBs or BBs.

Beyond the initial stage of the deployment, the organization’s chance to achieve and sustain a CI culture and high return on investment depends on its leaders.  Many Lean Six Sigma challenges simply reflect the existing organizational and leadership issues. Using the DMAIC methodology as a “plug & play” solution by the leaders only exacerbates the underlying problems.

DMAIC templates and tollgate reviews can help guide newly trained GBs and BBs as they practice scientific problem solving.  But when they become prescriptive requirements and project performance criteria dictated by management, it discourages dialogue and organizational learning, which are basic elements in a CI culture.  Judging project progress against a fixed set of DMAIC phase deliverables without understanding the applicability and true contribution in each case only causes confusion and fear.  It reinforces the “fear of failure” mindset in many organizations. 

The DMAIC stages are not linear, but iterative within the project, e.g. if a solution in Improve is insufficient to solve the problem, the team can go back to Analyze.  A DMAIC project should not be run like a “waterfall” project, but an Agile project with rapid learning cycles. With reasonable justification, the team should be allowed to decide to pass the tollgate and continue to the next phase.  Empowering the teams is risky and comes at a cost, but they should be given the opportunities to learn from their mistakes (if it’s not too costly).  Competent coaching will minimize the risk.

Compounded by the fear, poor training, and lack of experience, project efforts are often driven by management expectations at tollgate reviews.  A polished presentation with a complete set of phase deliverables beautifully illustrated with tables and graphs shows team’s accomplishments and satisfies untrained reviewers.  But it often fails at facilitating critical analysis and deep understanding required to address root causes – it sends the wrong message to the organization that the new CI methodology is all about presentation not substance.

If any of the examples sounds familiar or if you are concerned with building a CI culture and capability, one area for improvement might be in your DMAIC stage-gate process. 

Despite rapid advancement in life science and technology, much is still unknown about disease biology and variation in individual human responses to the pathogens and therapeutic interventions.  Our ability to understand the mechanisms and create effective solutions is still limited relative to the wide, complex problems.  Research and development as well as the manufacturing of vaccines, medicines, and diagnostic tests take much more time and resources than most people realize.  One of the reasons is that these products must meet very high standards of safety and quality, and therefore, require very rigorous development and testing.  Another reason is that the biological processes take time, e.g. time for patients to respond to the treatments, time for cells to grow in production, etc.

While we have limited ability to accelerate the natural processes in biology or control their outcomes, there are significant opportunities to reduce unnecessary failures, defects, delays, and waste in general that we can control.  As a scientist and quality professional, I have worked in R&D, manufacturing, quality, and business improvement in life sciences for over two decades.  No one wants to generate failures, defects, delays, or any type of waste.  Nevertheless, waste occurs and impedes the development and delivery of life science products, impacting the life and well-being of people.  

Waste stifles the innovative potential of life science and technology.  Waste must be reduced.

The first step to reducing waste is to see it. 

Lean practitioners are familiar with the traditional 7 or 8 types of waste that are common in all organizations across different industries.  However, many types of waste, especially in R&D, are not as visible as defects in manufacturing.  They are hidden in plain sight because they appear to produce desired results.  Even negative results are often explained away as expected biological or natural variation.  It is only when we look closely and compare them to the alternatives, do we realize they are full of waste.

Here are a few examples.

1. Design and perform experiments that only marginally improve our existing knowledge or decisions.   Even if successful, the outcome merely confirms what we already knew — we could have made the same decision without it.  The cost benefit analysis should clearly define the incremental knowledge sought before committing time and resources.  

2. Fast-track product development without a thorough characterization of the design space or without proper process and method validation, resulting in high costs, rework, and poor quality downstream in development and/or manufacturing. Quality by Design (QbD) would have been much more effective over the long term.

3. Conduct poorly designed experiments that unknowingly include high variation (noise), leading to failure to detect the change or difference (signal).  Poorly executed experiments can also create noise and lead to similar failures.  

4. Include an unnecessarily large number of runs or replicates when a properly designed experiment can get the same results at a fraction of the time and cost.  Statistically designed experiments can also reduce the likelihood of inconclusive results due to lack of power (i.e. too few replicates). 

5. Use manual procedures to perform tasks when technology is available to automate the job with much less time, cost, and errors.  It is not uncommon to see highly educated, expensive resources perform routine, manual tasks in the laboratories and on the computer.  A few lines of code can turn hours of manual data analysis into instant results.

6. Acquire and/or build complicated solutions when a simple and robust solution exists.  Ambitious scientists/engineers tend to chase cutting-edge solutions without investigating simpler, cheaper, readily available solutions.  Dedicated equipment and sophisticated algorithms cost much more time and resources but may not perform significantly better.

The above are only some examples of hidden waste at the operational level.  Bigger waste can happen at a strategic level (e.g. developing the wrong product and solving the wrong problem) and at an organizational level (e.g. misaligned objectives and broken processes); they must be addressed by the senior executives of the organization.  But everyone can improve productivity by learning to see the waste in what we do every day.

I hope the COVID-19 pandemic heightens the awareness of the value of life sciences and the need for higher productivity.  I am proud to work in this industry, but also feel strongly our duty to continually reduce waste – the opportunity cost is simply too high.

Life science products save not only lives but also our livelihood. 

A true Six Sigma project attempts to address a new or long-lasting problem that no one knows the real cause or has a clear solution for.   If the cause or solution is known, it is not a Six Sigma project – Just do it.  This uncertainty obviously makes some people less willing to initiate a Six Sigma project and/or can lead to unsuccessful projects.  In many ways, a Six Sigma project is similar to a high-risk R&D project.

How can we manage Six Sigma projects more effectively?

Assuming that the project is the right one for the organization and receives adequate resources and support, consider the following to reduce project delays and pitfalls.  (If the assumptions are not true, see my earlier post “The First Six Sigma Project” for discussion on some common Six Sigma deployment issues first.)

Train Project Management (PM) Skills

Many newly trained Black Belts (BBs) and Green Belts (GBs) lack sufficient project management skills.  Few received formal PM training, and their previous jobs did not require them to lead cross-functional teams.  A minimum of 2 days of PM fundamentals should be provided as a part of Six Sigma training or a separate program.  If the total training budget or days are limited, some more advanced or less frequently used Six Sigma contents (such as statistical tools) should be removed to accommodate the PM need. 

Having the basic PM knowledge is necessary for project success.  Particularly important, the BB/GB should be clear of their role as a project manager relative to the others in the organization.  The PM skills and experience will benefit the organization beyond the Six Sigma projects.  (See my earlier post “Project Managers are Managers” for suggestions for new project managers.) 

Apply Multi-generational Project Planning

Many project issues are a result of an overly large scope.  A Six Sigma project is already high risk without trying to solve too many problems at the same time.  Both the sponsors and BB/GBs tend to be overambitious and include multiple related metrics in the goal, which leads to diluted efforts and project delays.  If the project lasts more than 5-6 months, it is likely the original business case, assumptions, or metrics will no longer be true before they complete the solution due to external circumstances.  Often projects get cancelled before the benefits are achieved.

Instead, it is better to follow multi-generational project planning and break the goal into a series of smaller ones.  For example, two six-month projects sequentially are better than one 12-month project using the same resources.  Ideally, we follow the Pareto principle to achieve 80% of the goal in the first project and then the remaining in the second one.  In many cases, the second project becomes unnecessary because the business environment has changed by the time we finish the first.  This approach is similar to the Lean and Agile principles used in product development to manage uncertainty.

Use DMAIC Tollgates Properly

Most Six Sigma projects follow the DMAIC methodology, that has a tollgate for each of the Define, Measure, Analyze, Improve, and Control phases.  Many organizations have a list of required and recommended deliverables for each phase and check them at the tollgate review.  Unfortunately, many sponsors and even coaches do not understand why and when a deliverable is required for a particular phase; their insistence on completing the deliverable before the tollgate can cause confusion and project delays. 

Too often organizations make the mistake of using a tollgate to evaluate if the BB/GB has done a good job following the DMAIC methodology.  The primary purpose of a tollgate should be to help the sponsor make the right and timely decisions, such as stopping the project or providing resources.  The tollgates should not be the only times when such decisions are made; many inexperienced project managers make the mistake of delaying decisions until the tollgates.  Organizations can avoid such mistakes by setting the right expectations upfront for the tollgates and decision process for all projects.

In summary, to manage the inherent risks in Six Sigma projects, the sponsor and the BB/GB have to be proactive and methodical in planning and execution.   DMAIC should be rigorous, not rigid.   

This is one reason that Measurement System Analysis (MSA) is included in all Six Sigma training.  Because Six Sigma is a data-driven business improvement methodology, data is used in every step of the problem-solving process, commonly following the Define-Measure-Analyze-Improve-Control (or DMAIC) framework.  The goal of MSA is to ensure that the measurement system is adequate for the intended purpose.   For example, a typical MSA evaluates the accuracy and precision of the data. 

In science and engineering, much more comprehensive and rigorous studies of a measurement system are performed for specific purposes.  For example, the US Food and Drug Administration (FDA) publishes a guidance document: Analytical Procedures and Methods Validation for Drugs and Biologics, which states

“Data must be available to establish that the analytical procedures used in testing meet proper standards of accuracy, sensitivity, specificity, and reproducibility and are suitable for their intended purpose.”

While the basic principles and methods have been available for decades, most organizations lack the expertise to apply them properly.  In spite of good intentions to improve data quality, many make the mistake of sending newly trained Six Sigma Green Belts (GB’s) or Black Belts (BB’s) to conduct MSA and fix measurement system problems.  The typical Six Sigma training material in MSA (even at the BB level) is severely insufficient if the trainees are not already proficient in science, statistical methods, and business management.  Most GB’s and BB’s are ill-prepared to address data quality issues.

Here are just a few examples of improper use of MSA associated with Six Sigma projects.

• Starting Six Sigma projects to improve operational metrics (such as cycle time and productivity) without a general assessment of the associated measurement systems.  If the business metrics are used routinely in decision making by the management, it should not be a GB’s job to question the quality of these data in their projects.  It is management’s responsibility to ensure the data are collected and analyzed properly before trying to improve any metric.
• A GB is expected to conduct an MSA on a data source before a business reason or goal is specified.  Is it the accuracy or precision that is of most concern and why? How accurate or precise do we want to be?  MSA is not a check-box exercise and consumes organization’s time and money.  The key question is “is the data or measurement system good enough for the specific purpose or question?”
• Asking a GB to conduct an MSA in the Measure phase and expecting him/her to fix any inadequacy as a part of a Six Sigma project.  In most cases, changing the measurement system is a project by itself.  It is out of scope of the Six Sigma project.  Unless the system is so poor that it invalidates the project, the GB should pass the result from the MSA to someone responsible for the system and move on with his/her project.
• A BB tries to conduct a Gage Repeatability & Reproducibility (R&R) study on production data when a full analytical method validation is required.  A typical Gage R&R only includes a few operators to study measurement variation, whereas in many processes there are far more sources of variation in the system, which requires a much more comprehensive study.  This happens when the BB lacks domain expertise and advanced training in statistical methods.

To avoid such common mistakes, organizations should consider the following simple steps.

1. Identify critical data and assign their respective owners
2. Understand how the data are used, by whom, and for what purpose
3. Decide the approach to validate the measurement systems and identify gaps
4. Develop and execute plans to improve the systems
5. Use data to drive continuous improvement, e.g. using Six Sigma projects

Data brings us opportunities.  Is your organization ready?