The Elusive Goal of Quality in Complex Medical Systems
By Wes Chapman, Charles Hutchinson Ph.D., & Don Bialek MD • October, 2011
This is the first of four articles regarding quality issues in the delivery of clinical healthcare. This article looks at the popular methods for quality improvement in healthcare, and their applications and limitations. Specifically, it looks at those techniques that are founded in the concept of continuous improvement, including: 1) Lean, 2) Six Sigma, 3) Medical Checklists, and 4) ISO 9001. The second article will examine the alignment/quality issues and system design considerations regarding high volume and high RVU procedures. The third article will propose recommendations on system design for optimized protocol-based care in delivery systems involving small and large hospital systems. The fourth article looks at the development and utilization of process and outcome metrics, and how metrics can help in the development and continuous improvement of medical care protocols.
Quality means many different things in medical systems, and the superimposition of standard quality tools (Lean and Six Sigma) has only complicated the matter further. Medical systems are complex systems, not easily improved using Lean and Six Sigma tool sets, particularly at the macro level. With results that range from middling improvement to failure, healthcare managers are faced with a choice: maintain programs that have not met the majority of their goals, or start the long search for—and testing of—other options. The first question to ask, however, is whether any single system can, in of itself, provide the quality improvements everyone is seeking. Is the final solution a hybrid? If so, what are its parts?
Quality in Medical Delivery – A Story in Three Parts
Much has recently been made about the application of modern quality systems and tools into clinical care delivery environments. First, it is worth taking a minute to look at a fundamental question: What is Quality? There are three fundamental schools of thought on this question.
The first is “old school quality,” which is Quality is the degree to which a product or service meets the specifications for it.
This is the basis of the definition used by the ASQ (American Society for Quality
), and is quite limited and probably outdated, yet is the basis for most quality metrics in healthcare today. The Centers for Medicare & Medicaid Services
(CMS) has further diluted this definition by fundamentally dodging altogether the question of defining specifications for the outcome of the service, relying instead on “process metrics” as a surrogate for real specifications.
Second is the modern Japanese concept of quality stated by Noriaki Kano
et al, which is Quality is the degree to which a product or service meets or exceeds customer expectations.
This is a real step forward, and accounts for the intellectual underpinnings of the supremacy of Japanese manufacturing in a variety of industries. The problem with this definition in modern healthcare is that patients have no idea what the rational expectations
for service outcomes are. Nobody wants to get sick, old, or die, and we have developed the ethos that modern healthcare can fix all of that. It cannot – not at any price.
“…each of these pieces of ‘scrap’ is a failure to treat a patient properly…”
The third school of thought integrates the system that produces a product or service with the value to the customer: Quality is the ability to deliver, through a consistent and efficient system, a product or service that meets or exceeds a customer’s rational value expectations. This definition captures both the necessity of price considerations and the operational characteristics of the system that produces it. This is a critical concept for healthcare. In industrial systems, it is possible (although not desirable) to operate with a very high scrap rate and utilize only product that meets specifications. In healthcare, each of these pieces of “scrap” is a failure to treat a patient properly, resulting in waste, pain, injury, and even death. CMS’ aforementioned focus on process and system function makes a lot more sense in this light.
All modern quality systems share the common goals of waste reduction and continuous improvement. The traditional view of inspection-based quality is anathema to modern quality systems. In modern parlance, quality means doing the job right the first time, in the most efficient manner possible.
Healthcare as a Non-Linear,
In light of our preference for the third definition of quality in healthcare applications, it is worth taking a minute to consider the “consistent and efficient system
” portion of the definition. Consider two large and technically advanced facilities: 1) A modern automobile factory and 2) A modern medical hospital. The engineers designing the process flow for the automobile factory can specify the exact production process for the factory, and the linearity of production is guaranteed. All inputs can be tightly controlled, and the output is virtually identical. Any failure in any part of the system is immediately felt in all other parts (the line stops), and process variation
is immediately reflected in the product and can be easily traced back to the point of failure.
A modern hospital has very few of these characteristics. Incoming patients have a huge range of reasons for entering the system, ranging from routine checkups to organ transplants. Nevertheless, both the patient in for the checkup and in for the transplant use the same hospital delivery systems (e.g., radiology and laboratory services). Both also have the same expectations for quality, even though the near-term impact of any process failure is clearly greater for the transplant patient. Hospitals are very complex environments from a process perspective, and it is worth taking a look at such environments relative to their unique operational characteristics.
Characteristics of Complex Systems
Stated simply, complexity can be defined as a situation where an “increasing number of independent variables are interacting in interdependent and unpredictable ways” (Ilachinski 2001). Traffic is a good example of complexity, as is the weather, the stock market, and the United Nations. So are healthcare organization and delivery.
Healthcare in the US is clearly a complex system – if you can call it a system at all. It is important to note that healthcare is complex at both the macro level (organization) and at the micro level (delivery). Macro system elements are the fundamental organizations and “agents” acting to control and organize activities at the micro/delivery level, including patients, physicians, hospitals (and other venues), vendors of products and services to the system, regulators, and payors.
We will focus on the issues of defining and implementing quality systems at the micro level of care delivery.
Improving systemic quality in healthcare delivery is not straightforward. This is primarily due to the fact that the system is composed of non-linear processes. Linear production systems (production lines of all types) are governed by rigidly defined operating rules and specification limits. In such tightly controlled environments, quality is reasonably easy to implement, measure, and monitor due to the inherent predictability of linear processes. Healthcare is not linear; it is a complex adaptive system in which the actions of the agents in the system are interconnected in such a way that the actions of one agent change the context for others. The characteristics of such complex systems make it impossible for a single quality concept to be capable of sustaining quality on its own. These are:
· The number of agents in the system is very large.
· Interactions between agents are rich; i.e., any agent in the system is affected by and affects several other systems.
· The interactions are non-linear, which means that small causes can have large results.
· Any interaction can feed back onto itself directly or after a number of intervening stages, and such feedback also varies in quality.
· Such systems are open, and it may be difficult or impossible to define system boundaries.
· Complex systems operate under conditions that are far from equilibrium; there has to be a constant flow of “energy” to maintain the organization of the system.
· All complex systems have a history: they evolve and their past is co-responsible for their present behavior.
· Elements in the system are ignorant of the behavior of the system as a whole, responding only to what is available to them locally.
These challenges make it absolutely imperative for quality-improvement measures to be supported by a formalized structure, so that the complexity of the system does not undermine the efforts being taken to sustain quality.
An Example – Opaque Feedback Loops
As an example, I was recently talking with a friend, who described a quality improvement effort at his hospital. They had determined that quality was to be defined as “meeting or exceeding customer expectations,” and further determined that the waiting time in the ED was “excessive,” in excess of 60 minutes.
Leaving aside the determination of the goal, they set about to fix the excess time in the ED by hiring more nurses and providing incentives and penalties around the sub-60-minute goal. Despite their best efforts, waiting times only improved slightly. Then they became stuck, and the staff became increasingly frustrated. Worse, other areas of hospital performance were deteriorating markedly, even as the ED improved only a little.
A more detailed “root cause analysis” showed that the problem was not really in the ED at all, but in radiology and diagnostic testing. The small gains that had been made in the ED came from personal appeals to radiology from the large crew of ED nurses, and this had lead to significant process flow disruption, which spilled into other areas of the facility – a classic complex, non-linear production problem with opaque feedback loops.
Quality vs. Accreditation – Not the Same
Thing at All
For the purpose of this discussion, we will not address the role that the Joint Commission (TJC), Det Norske Veritas (DNV), and other accreditation bodies play in process quality in healthcare delivery. Accreditation is treated as an episodic event, distinct from the process improvement tools that are the basic application for modern quality improvement. While it is important to note that integrated quality and accreditation is coming closer to reality, the two are generally treated differently at the application level.
Quality Focused on “Never Events”
Historically, healthcare providers have attempted to use quality-improvement techniques and tools to reduce medical errors and help ensure patient safety – with a particular focus on “never events,” such as surgically removing the wrong organ or administering the wrong medication. It is critical to note that these are absolutely desirable process improvement outcomes, but the problems being addressed are (hopefully) very infrequent and disproportionately guarded against by redundant systems. Again, while perfectly laudable goals, these are not the focus of our inquiry here.
Quality Focused on Process Control
Our discussion focuses on the application of modern quality concepts in clinical-care delivery systems, particularly how the techniques and tools derived from these concepts have been used to address process variation and eliminate waste. Eliminating waste and rework increases patient-handling capacity and flow, which decreases wait times and potentially harmful delays in care. The result is a safer, more efficient, and more cost-effective system that better satisfies patients and healthcare workers. Some of the techniques relevant to the healthcare field are:
· Six Sigma
· ISO 9001 standards
These quality techniques act in their own characteristic ways to improve quality, and it is important to accept that in linear environments (systems they were designed for) they do a very good job of it. It is also important to note that the ways through which these techniques improve processes are completely dependent on the historical purpose for which they were devised in the first place. None of these techniques can independently succeed in improving the quality of healthcare delivery because, as we explained earlier, healthcare is a complex, non-linear system, fundamentally different from the linear processes from which the underlying quality concepts were derived. The implementation of quality techniques is necessary but not sufficient for sustaining customer satisfaction and eliminating waste in the healthcare system.
Quality Techniques – A Sampler
Lean operates on the principle of creating value for the customer by doing away with activities in the process that the customer would not be willing to pay for. The origins of this principle lie in quality improvement efforts that were taken in the Japanese manufacturing industry, exemplified by the Toyota Production System (TPS).
The complexity of healthcare delivery systems significantly blunts the benefits of Lean; focus on the sub-process level has actually shown the greatest value for Lean techniques.
Lean has gained great popularity in healthcare due to: 1) the concepts of value stream mapping (i.e., process mapping), 2) 5S applications (sort, straighten, scrub, standardize, sustain), and 3) waste identification and elimination. Lean tools are fairly easy to understand and implement, and are based on the intuitive identification and elimination of waste. Front-line employees, whose inputs are extremely valuable in Lean undertakings, do not need to invest time and energy to learn new skills and can be easily engaged. The focus is on creating value immediately, and the results are instantaneous. Through Kaizen, Lean also aims also institute continuous improvement of the optimized processes. Lean does a good job in identifying waste and co-dependencies at the individual process level.
· 5S (sort, straighten, scrub, standardize, sustain) and visual management in stock rooms
· Identify and map processes, with a focus on the identification and elimination of waste
· Lean has proven very effective at waste elimination at the “micro-process” level
Lean emphasizes the identification and elimination of waste at the process level. It relies on visual cues, and is not a project-based system. It is short of particular tools for implementation, and needs to be a “bottom up” process with broad stakeholder involvement. Its effectiveness is diminished by the blurred lines of responsibility and occasionally hierarchical (even dictatorial) reporting structures in healthcare. Lean methods interface poorly with the idiosyncratic variability associated with “the art of medicine.” The impact of these practices is also difficult to quantify, since Lean does not specify any data gathering or measurement methodologies.
The origins of Six Sigma processes began at Bell Labs in the 1920s, when Walter Shewhart invented the control chart, to measure and monitor the manufacture of electrical components. Post the pioneering work of Deming, Taguchi, et al, the idea of Six Sigma was born, wherein processes that operate at “Six Sigma quality” limit defect levels to below 3.4 defects per million opportunities. Six Sigma is entirely focused on the elimination of variation in outcomes and product. Six sigma uses very specific tools and techniques based on the DMAIC process (define, measure, analyze, improve, control). Lean provides a total system approach but is short on details, organizational structures, and analytic tools for diagnosis. Six Sigma, on the other hand, offers fewer standard solutions but provides an organizational infrastructure and a general analytic framework for problem solving.
Six Sigma and the DMAIC process have proven to be useful in healthcare in limited applications where system linearity can be assured and the impact of random exogenous variables can be minimized. As an example, Six Sigma has been used successfully in laboratory quality systems.
Six Sigma details a process-based approach to improving quality by reducing variation in output. It propagates an evidence-based decision making approach by specifying tools and techniques to measure and monitor processes for changes in variation, i.e. quality. Through quantifying financial return on quality improvement projects, Six Sigma helps realize the impact of such projects on an organization’s bottom line. Six Sigma utilizes an elegant mixture of data, control charts, and statistical analysis. It is the perfect tool for the identification and elimination of variation in controlled production environments.
Six Sigma is most frequently used in manufacturing processes, both linear and non-adaptive; for example, the manufacture of ICs and medical devices. Six Sigma is also used in detailed, linear, controlled environments in healthcare such as labs. A number of care providers have reported significantly improved Lean process improvements by using Six Sigma tools for more complex problems. In any event, the smaller the inherent system complexity, the more useful Six Sigma tools become.
Six Sigma fundamentally depends on linear systems for its function. If variation can enter a complex system randomly and episodically, Six Sigma is a sub-optimal analytical tool. The DMAIC process begins with “define,” and complex systems are notoriously difficult to define. Six Sigma is inherently limited for applications in healthcare delivery systems.
Checklists are tools that help reduce failure rates in non-linear or complex processes by specifying a list of activities that need to be performed for the effective execution of the process. Checklists have been used extensively for process control in the military and in the aviation industry, because human interaction, which is a major part of processes in these fields, introduces extensive non-linearity. The origins of the usage of checklists in the industrial sector lie in the failed flight of Boeing’s B-17 prototype. The ensuing investigation identified the cause to be “pilot error,” and its aftermath led to the conclusion that the B-17 was “too much airplane for one man to fly.”
Checklists are a universally understood method of process control. A streamlined checklist is a necessary part of process control in a non-linear process that can feedback onto itself; they can confirm that essential functions were carried out and also provide instructions to quickly respond to known variances in the process.
Checklists are used typically in processes that require human initiation or intervention; for example, aviation, healthcare, and the military. The recent publication of The Checklist Manifesto, by Dr. Atul Gawande highlighted the extreme process control and the consistent and measurable outcome improvements that checklist use has accomplished in healthcare across a widely varied set of healthcare providers around the globe. It is important to note that checklists are really just process control documentation, which enforces best practice protocols. Documented process control is a fundamental element of ISO 9001, as discussed more fully below.
Using checklists necessitates defining roles and responsibilities, controlling versions of checklists, and ensuring that required personnel have been trained accordingly. Checklists are a fundamental element of simple process control, as well as system recovery in complex problems, but people resist checklists, as they seem to represent unnecessary bureaucratic oversight.
ISO 9001 Systems
ISO 9001 specifies the basic requirements for a quality management system (QMS) that an organization must fulfill to demonstrate its ability to consistently provide products (which include services) that enhance customer satisfaction and meet applicable statutory and regulatory requirements. The history of the ISO 9000 family of standards can be traced to 1959, when the Department of Defense published the MIL-Q-9858 standard, to control all supplies and services under contract in order to assure compliance to contractual obligations through an effective and economical quality program.
Adhering to ISO 9001 standards enables organizations to maintain a footprint of quality that demonstrates their commitment toward customer satisfaction and continuous improvement. This also leads to better marketability, increased employee morale, and increased productivity.
Diverse organizations have successfully implemented ISO 9001 quality systems, since the standards only prescribe guidelines to maintain quality and do not dictate the way internal operations should be performed. ISO 9001 has been widely used to ensure uniform process and product output by multinational corporations in diverse geographical environments. It has recently been utilized by DNV as a credible accreditation alternative to TJC. It has been very widely adopted in rapidly expanding industrial societies, with notable success. China has grown into the world’s largest industrial exporter, and has over ten times the number of ISO certified entities as the US.
A standalone ISO 9001 system promotes only process control that uses a robust documentation structure; it is possible to have limited tools for process improvement while adhering to ISO 9001 standards. The standards are only a support for quality-improvement initiatives that should be of strategic importance to the organization. ISO can be cumbersome and bureaucratic in nature, and needs to be used with practical judgment, being implemented only in those organizations and operations that can benefit from documented process control.
The Role of Experts
“If all you have is a hammer, everything looks like a nail”
– Bernard Baruch
The last 15 years have seen an explosion of certified “Quality Experts,” largely through the widespread adoption of Lean and Six Sigma practices in corporate America, with Jack Welch’s pervasive use of Six Sigma in GE during his tenure being the best example. These are relatively short (one week to a few months) programs, designed to foster a common language and culture at the sponsoring firms. The result has been to flood the market with experts equipped with hammers, looking for nails.
Unfortunately, most modern quality tools were not designed for complex systems in general, and healthcare applications in particular. The operational silos, complex feedback loops, complex governance, need to respect never events, and related problems in healthcare limit the applicability of quality tools there compared to other operational environments.
Movement to a Hybrid – Healthcare
As we have seen, healthcare systems are complex in nature. The natures of the problems that arise in complex systems are also complex – there are no simple problems in complex systems. The solution could well be inherently simple, but a “naïve” interpretation of it will most definitely lead to an ineffective resolution. This is exactly why the simple application of the above-mentioned quality concepts leads to results that fall below expectations. In some cases, it could even be impossible to judge if the quality projects yielded any benefit at all. More on this below.
Lean provides a great baseline tool kit for identifying and defining targets for quality projects in healthcare applications. In particular, process and value stream mapping are valuable at the managerial level, and the 5Ss are great tools at the hands-on level. Lean has limited project management tools, and benefits can easily prove ephemeral without good process definition and control tools.
Six Sigma provides a rigorous analytical framework and a clear and consistent data management and analysis capability for implementation of quality projects. The reality is that any Lean or Six Sigma healthcare project must be evaluated to look for opaque/unanticipated changes to operations outside of the project scope itself. In any complex system, it is always possible to “improve” one process to the detriment of the function of the entire system. Any Six Sigma quality project in a healthcare organization that is set up without a flexible and robust data-gathering system is being set up to fail. Data is required to: 1) Determine need and establish a baseline, 2) Determine efficacy of the project as implemented, and 3) Determine the ongoing efficacy of the project. Key performance indicators (KPIs) that can gauge the success of the project appropriately must be identified, and data points must be gathered to continuously monitor these KPIs.
“In God we trust; all others must bring data.”
– W. Edwards Deming
ISO 9001 and robust Checklist Systems are designed to provide documented process control. ISO 9001 provides an integrated and time-proven method for the control of complex systems and processes. An ISO 9001 system will always include: 1) A definition and description of the processes being controlled; 2) Clear definition of standard operating procedures (SOPs) and work instructions; 3) Clear description of the process and operational pathways; 4) Clear definition and description of organizational structure, training requirements, and related HR matters; 5) Clear definition and description of relationships with suppliers, vendors, and customers; 6) Clear description and definition of how compliance will be measured, and clear corrective and preventative action plans for when they are not; and 7) The documents to control it all. It sounds like (and is) a lot of work, but it is the only comprehensive method for the control of complex systems.
Checklists can be their own tool set, or fit pretty handily into both Six Sigma and ISO applications. Checklists have been used in multiple environments to 1) Ensure compliance in routine operations and to help eliminate “never events”, and 2) To provide operational guidance in time-critical emergent situations.
Problems in complex systems do not have simple solutions. A single quality concept cannot act as the panacea for improving quality in a healthcare organization. The time consuming, but inherently simple, solution to this problem is an ISO 9001-based quality management system to control the operation and evaluate the success of quality efforts in healthcare. ISO is a perfect overall tool to evaluate the quality efforts of an institution, and has the added benefit of de-mystifying the evaluation of quality improvement efforts.
In any event, quality system design and implementation in healthcare requires practical hands-on judgment. No single system exists that can deal with the complexities of healthcare – there are a lot of great quality tools and systems, but using them effectively requires judgment and a fair amount of patience.
Process control plays a bigger part in sustaining quality in healthcare systems than the quality concepts themselves. This is simply due to the fact that the uniqueness of healthcare systems – their complex and recursive nature – can be contained only by process control. The execution of quality-improvement measures could well be rendered meaningless without robust process control.
As a final point, it is worth noting that if a quality system cannot provide real-time data and generate a quantifiable ROI, then it is not a quality system at all. Quality systems are expensive to design and implement, and difficult to maintain. Any effective effort in quality systems can demonstrate variances from plan and historical levels of activities. The effectiveness of every project should be traceable. All too often, quality groups operate as final arbiters, answering to nobody and held accountable for nothing. In these cases, operations drift into anecdotal evaluations, and nothing can progress.
Quality of outcome – both surgical success and patient satisfaction – can be accurately predicted by a single factor: volume. Yet the RVU structure currently standard in U.S. healthcare drives low volume hospitals (such as critical access hospitals) to perform surgeries that are best referred to central high volume facilities. Furthermore, there is good evidence that patient satisfaction depends, in part, on care close to home, and that primary care provider (PCP) involvement is critical to protocol compliance and quality outcomes. How do we change the system to provide the best outcomes without crimping revenue flow at the critical access hospitals?
2. WHO – World Health Statistics 2009
4. The Centers for Medicare & Medicaid Services (CMS) also has developed a set of quality measures through a standardized framework that uses defined business processes and decision criteria.
10. The challenge of complexity in health care – Plsek and Greenhalgh (2001)
11. Health Systems Financing: The path to universal coverage, WHO’s World Health Report 2000
13. Brent C. James and Lucy A. Savitz – How Intermountain Trimmed Health Care Costs Through Robust Quality Improvement Efforts, 2011
14. Primary Care — Will It Survive? Thomas Bodenheimer, M.D.; 2006
15. Health, United States, 2010 – US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics
16. 1997 Population Profile of the United States – US Department of Commerce, Economics and Statistics Administration
A Lean process for controlling production output quality: sort, straighten, scrub standardize, sustain.
Acute Care Hospital; a medical center or surgical practice hospital capable of providing specialty procedures. Usually they are relatively large, high-volume facilities.
American Medical Association.
American Society for Quality.
Bundled Payment Initiative
A program initiated by the CMS in August 2011 to allow groups of providers to share in the provision of services and fees related to specified medical procedures. Allows small primary care facilities (such as CAHs) to form shared service/payment arrangements with larger acute care hospitals, bypassing the self-referral and anti-kickback restrictions of the Stark Laws.
Critical Access Hospital; a small, regional hospital/clinic limited to 25 beds or less and designated as critical to care in a geographically isolated community.
The Centers for Medicare & Medicaid Services.
Current Procedural Terminology Codes, maintained by the American Medical Association, describe medical, surgical, and diagnostic services. They are designed to communicate uniform information about medical services and procedures.
A process for improving production systems: define, measure, analyze, improve, control.
Det Norske Veritas, an independent foundation that classifies and certifies healthcare organizations internationally using ISO 9001.
Emergency Department in a hospital or clinic.
International Organization for Standardization; creates standards that business must meet in order to satisfy customer and statutory requirements.
Specifies basic quality requirements that a company must fulfill to consistently provide products and services that enhance customer satisfaction and meet statutory regulatory requirements. The ISO 9000 series specifically deals with quality management systems.
Japanese for “improvement”, or “change for the better”; refers to philosophy or practices that focus upon continuous improvement of processes. Applied to any business or workplace, it refers to activities that continually improve all functions, and involves all employees.
Key Performance Indicator; a performance measurement that provides data critical to evaluating the effectiveness or efficiency of a process. KPIs will vary from process to process and organization to organization.
A quality concept and system that creates value for the customer by doing away with activities in the process that the customer would not be willing to pay for. Lean emphasizes the identification and elimination of waste at the process level.
Macro Level (Processes or Quality)
Macro system elements are the fundamental organizations and “agents” acting to control and organize activities in a system.
Micro Level (Processes or Quality)
The detail level at which goods or services are actually produced; e.g., healthcare delivery.
An event or outcome that should never occur in a properly controlled process; for example, administering the wrong medication.
Quality Management System; a generic term.
Return on Investment.
Specialty Society Relative Value Scale Update Committee of the AMA that defines and updates Relative Value Units
Relative Value Unit; a comparable service measure used by hospitals to permit comparison of the amounts of resources required to perform various services within a single department or between departments. It is determined by assigning weight to such factors as personnel time, level of skill, and sophistication of equipment required to render patient services.
A quality concept and system focused on the elimination of variation in outcomes and product through the DMAIC process (define, measure, analyze, improve, control).
The Joint Commission, an independent, not-for-profit organization that accredits and certifies healthcare organizations and programs in the United States.
Toyota Production System; the exemplar of a Lean implementation.