About Andrea’s Blog

Welcome to my periodic musings on the state of American education, business, journalism, the consumer experience, women and food—how it is grown, cooked and eaten. A systems thinker since I wrote my first book, The Man Who Discovered Quality by W. Edwards Deming, most of my ideas and writing are informed by a systems view of the world.

Posted in Uncategorized | 6 Comments

New CREDO Study, New Credibility Problems: from New Orleans to Boston

Last month, Stanford University’s Center for Research on Education Outcomes (CREDO) released a new study on urban charter schools, which purports to show, for the first time, that charters outperform city public schools, at least on standardized-test scores. If true, the study’s findings are a potential bombshell since, thus far, studies have shown no meaningful difference between charter and public schools.

The new study, Urban Charter School Study Report on 41 Regions claims to show that “urban charter schools in the aggregate provide significantly higher levels of annual growth in both math and reading compared to their TPS [public school] peers.”

The years I’ve spent researching schools on the ground in charter-heavy districts like New Orleans and New York City made me skeptical of such an outcome. But because I am not an expert in research methodology, I decided to hire a respected statistician, Kaiser Fung, author of Numbersense and an adjunct professor of statistics at New York University who has no connection to the education-reform movement (and thus no axe to grind), to help me analyze the CREDO study.

After combing through the study and its accompanying technical document, and after exchanging a series of emails with Macke Raymond, Director of CREDO, we found significant problems with the CREDO study. The problems go well beyond technical quibbles and suggest that any generalizations drawn from the study about the quality of traditional public schools relative to charter schools would be a big mistake. In particular, the study does a poor job of explaining the basis on which it includes or excludes charter- and public-school students; an email exchange with Raymond clarified the study’s methodology, but also revealed that it introduced, in many cases, an anti-public-school bias. And, in at least one case—the findings on New Orleans, the first all-charter district in the country—Raymond admits that CREDO violated its own methodology, a fact not disclosed in either the study or its accompanying technical documents.

Let’s begin with a brief description of the study itself. The study analyses data from 22 states, covering just over a million charter-school students, during the 2006/2007-to-2011/2012 school years. It seeks to measure charter-school performance in 41 urban areas against students who attend “feeder” public schools in the same urban areas. The study includes about 80 percent of the charter students in the areas under study (20 percent are excluded from the study because CREDO could not find any matching public-school students.)

The study also relies on a controversial methodology that the researchers used in past CREDO studies and that has been critiqued here and here and here. What’s important to know about the methodology is that it purports to compare each charter student in the study to a “virtual twin,” a composite of as many as seven public-school kids who attend “feeder” schools and who “match” the charter students on both demographics and test scores; the virtual twin is literally an averaged kid. The demographic criteria for creating each virtual twin includes: grade level, ethnicity, gender, Title 1 eligibility, special-education and English-language-learner status.

In this post, I will not revisit the problems referenced above with CREDO’s virtual-twin methodology; rather I will focus on three major problems with this study:

First, the study excludes public schools that do NOT send students to charters, thus introducing a bias against the best urban public schools, especially small public schools that may send few, if any, students to charters. The study implies that the “virtual twins” are drawn from the general population of traditional public schools—specifically that a school is considered to be a feeder if even a single student transferred during the study period. This is not the case. In our email exchange, Raymond explained that to qualify as a “feeder school” a public school must send at least five students to charter schools, a detail not revealed in the study. The study never explains that it uses this stricter, five-student-minimum criteria that public schools must meet to be included in the study. (Nor does the study explain why it didn’t look at all “neighboring” public schools with comparable/charter-like demographics—whether they send kids to charters or not.)

To test my theory, I contacted two of the better Title 1 middle schools in New York City whose demographics I knew would mirror those of local charter schools to see if they meet the criteria that would qualify them as charter “feeder schools.” Global Technology Preparatory, in East Harlem, where most kids are black or Latino, estimates that it has sent only three of its graduating eight graders to charter high schools over the last three years. West Side Collaborative, a ten-year-old school with similar demographics, hasn’t sent a single transfer-student or graduate to a charter school, according to Jeanne Rotunda, the recently retired founding principal. Both schools received an “A” and a “B” on the last two graded report cards from the New York City Department of Education, and are given high marks for quality from parents, students and teachers. Yet, although both GTP and West Side have charter-like demographics and are in an area rich with charter schools, they would not count as feeders and, therefore, their students wouldn’t be included among the virtual twins in the CREDO study.

This also raises several further questions: The study’s geographical filtering mechanism for determining which schools qualify as “feeders” isn’t disclosed, except for some qualitative description in the Technical Appendix. It would have been much more straight forward to rely on simple geographic or district demarcations. New York City, for example, neatly divides its schools into clearly defined neighborhoods, such as East Harlem North and East Harlem South etc., as well as distinct educational districts.

Global Tech and West Side Collaborative also highlight the ways in which CREDO’s matching criteria miss critical differences between public- and charter-school demographics. Urban public school students are often poorer, more likely to attend schools with large number of kids with special needs and English language learners than their charter-school counterparts. They are also likely to have parents who are less engaged, for a variety of reasons, than those in charter schools, which target the most engaged families via everything from lotteries to requiring that parents attend a set number of open houses before they can even enter lotteries. These distinctions are not addressed by the CREDO study.

Second, in the case of New Orleans, the study compares charter students to virtual twins who go to school, not in New Orleans, but anywhere in Louisiana—a clear violation of the study’s feeder-school criteria, and one that isn’t disclosed in the study. In 2007, the first year of the study, 56 percent of New Orleans students were enrolled in charter schools; by 2012, the last year of the study, over 80 percent were enrolled in charter schools. Since each virtual twin is a composite of an average of five public-school student test scores, it seemed logical that there were not enough public-school students in New Orleans to meet the methodological requirements of the CREDO study. (See chart below) When I asked Raymond about this, she wrote: “In Nola, we use similar schools that operate in similar communities in Louisiana to provide our matches.”

CREDO screenshotofspreadsheetRaymond also claims that New Orleans is the only city where she bent her own rules on drawing virtual twins from “feeder schools.” Yet, a similar problem is likely to impact at least two other cities—Washington, D.C. and Detroit, where 40 percent and 47 percent, respectively, of students attend charter schools. Assuming that only 80 percent of the charter students are matched in each case (the other 20 percent are dropped from the study because of insufficient public-school matches), then there are at most 1.4 and 1.9 public-school students, respectively, as potential matches for each charter student, far less than the 5-student average needed for virtual twins. However, the 1.4 and 1.9 public-student estimate assumes every public student is a match for a charter school student. We already know this to be false. Some public school students don’t attend feeder schools. There will be other reasons why public-school students drop out (e.g. CREDO’s methodology deletes public school students who transfer to a charter or who are demographically dissimilar to charter-school kids.)

The reverse problem—too many potential public school student matches per charter student—plagues the study’s findings in cities like Boston where a very small percentage of students attend charter schools. See below.

Problem Three: Subjective decisions on which charter schools and public schools to include or exclude introduce a number of additional anti-public-school biases.

The study includes both selective and non-selective charter schools, but eliminates an undisclosed number of demographically similar public schools as per above, again introducing the potential for anti-public school bias. Among charter schools, the study eliminates only those that operate in “secure” settings, such as detention centers, as well as “charters that are permitted to use entrance exams (these occur only in two of the regions, a total of 4 schools, I believe),” writes Raymond in an email. “Other than these exceptions we include all TESTED charter students.” Again, this detail is not available in the study or its attendant technical documents.

While relatively high-quality schools like Global Tech and West Side Collaborative do not match the study’s “feeder” criteria, selective charter schools are included in the study. Take the two-tier charter school system of New Orleans, where about a dozen of the city’s charter schools are part of the Orleans Parish School Board, which includes many of the city’s most successful schools, most of which are selective. These schools do not typically give entrance tests that would disqualify them from the CREDO study. But they do require applicants to submit, among other things: test scores, school grades, and attendance records. Even kindergarten applicants are required to submit a record of their work: In the case of the Lake Forest school, these include “1 current student artwork sample, a self-portrait drawn by said student, and one student handwriting sample. By way of partial explanation, Raymond writes: “The object is to create controls that mirror the range of charters, not the range of TPS [public schools.] We do not presume that the peers are a complete mirror of the entire TPS [public school] population.”

Says Fung: “This is exactly the reason why observers shouldn’t interpret the finding as representative of urban public schools.”

However, when a public-school student transfers to a charter, the entire record of that student is deleted from the virtual-twin control group. At the same time, she is eligible to be included in the study as a charter student. “This is doubly bad,” says Fung.

Here’s why: The public-school student who is transferring to a charter is presumably a good student. By deleting the record of this student during the time she was in public school, the study drags down the performance of the public-school matches during that period. Simultaneously, because this student can now be considered a charter student in future periods, and can be counted as part of the charter student population in future periods, she contributes to the performance of the charter students in the study.
Conversely, if a student drops out of a charter school, he is eliminated from the study. When I asked Raymond why this didn’t artificially improve the scores of charter-schools because drop-outs are likely to be among the weakest students, she answered: “Since we have very little test data on high schools, we actually don’t see if students drop out.”

Says Fung: Her answer amounts to “because we don’t have data, we don’t know if there is bias, and because we don’t know, we assume there isn’t”. The reality is the students are dropping out whether or not CREDO sees them. This is known as “survivorship bias”. All CREDO gets to see in the data are the “survivors”, students who have not dropped out of the system.

Imagine a clinical trial comparing cancer drug A and cancer drug B. You measure the increase in survival time of patients in each arm. Now, suppose each time someone dies, you drop them from the study. The problem is that the reason for the dropout is related to the outcome being measured (i.e. their survival time is so low that they died before the end of the study).

Similarly, if the students who have dropped out of charter are those who perform worse, then for sure, by letting these students drop out of the study, they have a bias in the selection of the charter student population.

Thus, the CREDO study appears to include the “cream” of the charter schools—the selective schools—while excluding the best public schools even among those that serve students who are demographically similar to those of nearby charter schools. At the same time, Raymond doesn’t acknowledge that creaming takes place among even non-selective charter schools in cities like New Orleans. Writes Raymond, in response to my question about creaming: “It is not widely acknowledged that there is cream skimming in Nola. With 90 percent of the students attending charter school it seems infeasible the cream skimming would occur.”

This despite a recent study, by the Educational Research Alliance at Tulane Univ., in which principals of New Orleans charter schools admit that they respond to market pressures—in particular, competition for scarce funds that come to schools with the highest test scores—by engaging in “actively selecting or excluding particular types of students.”  Such clandestine selectivity may, indeed, contribute to large and under-counted dropout rates in New Orleans.

In addition, while CREDO says it uses an average of five public-school students, and a maximum of seven, to form a virtual twin, it never explains what proportion of public-school students in each region are included in the study, a problem that can, again, introduce bias.

In the cities with small charter-school populations, such as Boston, where only 13 percent of students attend charter schools, there are too many potential public-school student matches, requiring CREDO to make additional judgments, which again are not explained in the methodology, on which public-school kids to include. Here’s why: Since only 13 percent of Boston students attend charter schools, there are a lot more than 5 possible virtual twins (see chart); thus, CREDO needs to use an additional screening mechanism. If five matches are randomly selected from among all public school students in Boston, then large schools will predominate, introducing a large-school bias. In the most extreme cases—i.e. cities with tiny charter-school populations, such as El Paso, where less than 5 percent of the student population attends charter schools, there are many, many eligible matches per charter student.(see Kaiser’s chart)

Further, while the study matches the test scores of, on average, five public-school students to each charter student, not all the scores are exact matches. Yet, the study never discloses either the percentage of “inexact” matches nor whether these inexact matches are on the high, or the low side. If the majority of “inexact” matches are below those of their charter twins, it would not be surprising that, by the end of the study, the public-school virtual twins would still be testing lower than their charter counterparts. (The converse, of course, is also true. But we don’t know, because the study doesn’t say!)

Says Fung: “There is no clear statement from CREDO as to how they select matches under this scenario, and they do not control for school variability.”

Fung found numerous other problems, some of them technical, which I will not elaborate here. However, the examples above are more than enough to cast serious doubt on the study’s conclusions. And this, even without further challenging two key assumptions behind the study: A) That standardized-test scores are an adequate measure of school quality and B) That creaming in charter schools does not exist.

Finally, readers of the Credo report are likely to think the public-school matches are representative of public-school students in general (which Raymond herself said was not the intention), and to think that somehow a finding here, even if one were to accept CREDO’s methodology, can be used to advocate expanding charter schools.

Unfortunately, the myriad problems with this study have not stopped many charter advocates and even some respected journalists from blindly accepting the study’s findings.

Posted in Education | Tagged , , , , , , , , , , , , , , , , , , , , , , , | Leave a comment

Round Two In the Bay State’s Battle Over the Common Core

One of the big mysteries of the education-reform movement is why Massachusetts, the gold-standard of American education, jettisoned its highly successful education standards for the untested Common Core State Standards. One reason was a much-needed, post-recession cash infusion via Race to the Top.

The Bay State’s first bid for RTTT funds failed—the commonwealth came in a miserable 13th—because it had not adopted the Common Core. “There’s a lot of disappointment and anger in Massachusetts that our outstanding track record in education reform was not recognized,” said Paul Reville, the state’s education secretary at the time.

Massachusetts finally won $250 million in RTTT funding, in 2010, after agreeing to adopt the Common Core. That “win” would usher in a series of changes to the state’s highly regarded, two-decade old education system.

Massachusetts’s poor showing in its first RTTT sweepstakes—and the Federal government’s apparent willingness to tie funding to the adoption of the Common Core, even for the country’s most advanced education system—troubled many experts on both sides of the ed-reform divide. Beginning in 2005, Massachusetts kids had the highest performing test scores in every subject tested by the National Assessment of Education Progress, which is known as the Nation’s Report Card. It has also outscored kids around the world on a range of international tests (More on Massachusetts’ test scores below.)

Now the Massachusetts reforms are once again under assault by Common-Core enthusiasts. Strangely, many of those attacking the reforms are its erstwhile defenders. In February, the Massachusetts Business Alliance for Education, a leading advocacy group for the 1993 Massachusetts Education Reform Act, issued the first of several reports that found, or are expected to find, the Bay State standards and an accompanying high-stakes test, the Massachusetts Comprehensive Assessment System or MCAS, wanting when compared to the still-untested “Common-Core aligned” PARCC tests (PARCC stands for Partnership for Assessment of Readiness for College and Careers.)

“The current MCAS high school tests do not identify students who are college- and career-ready, and they do not contain the right content to measure college- and career-readiness,” concludes the MBAE study.

By contrast, the MBAE cautiously endorses the PARCC test: “As we are preparing this report in early 2015, the PARCC tests hold the promise of being a good indicator of college- and career-readiness.” (Emphasis added.)

In response, researchers from the Pioneer Institute, a market-oriented Massachusetts think thank, argue that money, once again, is playing an outsized role in the latest anti-MCAS research. The turncoats, according to Pioneer, include MBAE, which was cofounded by the aforementioned Paul Reville, as well as the Thomas B. Fordham Institute and Achieve Inc., both national Common-Core advocates. What these organizations all have in common is that they have receive funding– lots of it—from the Bill and Melinda Gates Foundation, which also invested over $200 million in developing the Common Core.

The most recent Massachusetts skirmish over the Common Core is no coincidence. This year, Massachusetts elementary and middle schools had the choice of taking the PARCC test or the MCAS. In the fall, Massachusetts will make a final decision about whether to ditch the MCAS entirely in favor of PARCC, at a time when half the states that initially agreed to adopt the Common-Core aligned test have since backed out.

In their OpEd, Charles Chieppo and Jamie Gass, detail the tangled web of relationships that tie the critics of the Massachusetts reforms to the Gates foundation, the PARCC tests and the Common Core. The OpEd is particularly scathing about the role of the MBAE:

The Mass. Business Alliance study’s credibility was further compromised by the fact that its author is an adviser to PARCC. An earlier report from the Alliance — written by the senior education adviser to the giant testing company Pearson, which is near the top of a long list of entities that stand to gain from the switch to Common Core — was so bereft of intellectual integrity that it lifted an entire purported “case study” from The Boston Globe without attribution.

However, the winner of the “conflict-of-interest derby,” according to Chieppo and Gass, is Teach Plus, a Boston-based national education-reform organization, which published a pro-PARCC report, “Massachusetts Teachers Examine PARCC“, in March:

The group recently released a study in which 23 of its fellows conclude that the commonwealth should ditch MCAS for PARCC. Teach Plus has received over $17 million from the Gates Foundation, including stipends for each of those 23 fellows.

While Pioneer Institute says it has neither solicited nor received funding from the Gates foundation, it has been approached by the Gates foundation. “The more noise we made the more they seemed interested in ‘working with’ us,” Gass wrote me in an email.

Like major philanthropists before him, Gates’s foundation has done a great deal of good around the world—for example, in joining the fight against the Ebola outbreak in Africa. However, the Gates foundation’s work in education reform illustrates the danger of allowing a single individual (or foundation)—no matter how well intentioned—to have too much influence on public policy. Gates brings his own data-driven world view to education—one that values STEM subjects over literature, history and the arts. There is no countervailing force with comparably deep pockets to argue that children would be better off, say, producing Shakespeare plays and studying violin instead of focusing on high-stakes tests.

By controlling the purse strings and the megaphone, the Gates foundation engineered “the swiftest and most remarkable shifts in education policy in U.S. history,” according to a Washington Post analysis.

That story is now replaying itself in Massachusetts; the only difference is that in Massachusetts, there’s a real danger that by jettisoning the Bay State’s reforms, the Common Core could turn a widely recognized success into a failure. Like a bio-engineered super plant nurtured by Gates and bred for the arid conditions of ed-reform soil–and to be bug-resistant and uniform—the Common Core is threatening to crowd out the healthiest varietals.

Among the local ed-reform crop, none is sturdier than the Massachusetts standard and the MCAS. Beginning in 2005, the scores of kids in Massachusetts surpassed those of every other state, in every subject, at every grade tested on the NAEP. The state also outscored kids from around the world on international tests. Based on the latest PISA (Program for International Student Assessment) results, if Massachusetts were a country, it would rank ninth in the world in math proficiency, tied with Japan; fourth in reading, tied with Hong Kong. Moreover, Massachusetts eighth graders ranked second only to Singapore in science competency, according to the TIMSS (Trends in International Mathematics and Social Studies) tests.

MBAE essentially argues that the “proficiency” cut-off score on the MCAS is set too low. In Massachusetts, other arguments for adopting the Common Core range from a 21st century penchant for all things new and shiny to an educational vision of “national citizenship”—as one Common Core advocate put it: “If we’re going to develop this country and thrive as a democracy” national standards will help lift states like Mississippi, which have the lowest education performance in the nation.

MCAS advocates counter that low-performing states will exert “downward pressure on PARCC’s rigor” eventually dumbing down both the test and the standards.

Indeed, in Massachusetts, even the Common Core’s staunchest advocates couch their support in conditional terms. “[I]t it is not possible to know how much” of the “promise” of the PARCC tests “will be fulfilled,” concedes the MBAE report. Reville meanwhile says: “Our hope is that they’re a more sophisticated generation of tests, they’ll be aligned with the Common Core and will generate information more quickly and in a more useful fashion than we’ve been able to do in the past.” (Emphasis added.)

Long-time Massachusetts educators disagree. Sue Szachowicz the recently retired principal of Brockton High, the largest high school in Massachusetts, says she “cringed” when she saw the MBAE report, which she says compared “apples” and “oranges.” Says Szachowicz: “MCAS was never ever” intended to test college and career readiness. “It’s a 10th grade test designed to be a check in, to see if our kids’ reading and math proficiency is where we need it to be.”

“I’m a strong supporter of MBAE,” adds Szachowicz who led a one of the Bay State’s most heralded and enduring turnaround efforts in response to the Massachusetts Reforms. But the report, she says, “is wrong,” noting that studies have found a strong correlation between MCAS performance and college success.

Tom Birmingham, a key architect of the Massachusetts reforms who recently joined the Pioneer Institute, is now also weighing in with his own series of OpEds, arguing that the Common Core, and thus PARCC, is inferior to Massachusetts standards. Most recently he wrote: “A distinguished presidential panel found that Algebra I is the key to advanced math study and recommended that students study it in 8th grade, as currently occurs under Massachusetts’ state standards. But Common Core would delay Algebra I until early in 10th grade, preventing students from reaching high levels of math in high school… Common Core ends with ‘Algebra II lite,’ which is insufficient for students aiming for college majors in science, technology, engineering or math.”

Advocates of the Common-Core and PARCC also miss a key ingredient that made the Massachusetts standards and the MCAS a success: They were the result of a dynamic democratic process, that solicited input from a wide range of constituencies. “We developed traveling groups of educators to go around the state and find out what should be included in the curriculum frameworks,” recalls Sandra Stotsky, who then served as senior associate commissioner of the state’s Department of Elementary and Secondary Education. By the time the accountability piece of the reforms kicked in 2001—the MCAS results would determine whether kids received a high school diploma kicked—there was widespread acceptance of the process.

And after each MCAS testing period, the state released the MCAS questions and results, allowing teachers and administrators to pour over the tested material. Recalls Szachowicz: “We used the information, to see what areas we were strong on and where we were weak. It was a very open process.”

The MCAS continued to be updated and improved each year for more than a decade.

PARCC, by contrast, is a locked box, entirely controlled by Pearson, the testing giant that is developing the PARCC tests. It isn’t designed to be improved by educators over time, nor to help educators use the test to improve what or how they teach.

For now, at least in Massachusetts, the war over the Common Core will continue for at least a few months. Fordham Institute is expected to produce a study this summer examining the MCAS’s alignment to the Common Core; if its earlier support for the PARCC test is any indication, it too is likely to find against MCAS.

In Massachusetts, a final decision will be made by Mitchell Chester, the current education commissioner. Chester, it must be noted, also chairs PARCC’s governing board.

Posted in Education | Tagged , , , , , , , , , , , , , , , , , , , , | 11 Comments

Letter to Hillary: Beware Education Advice From Hedge-Funds

Last week, The New York Times, published a front page story about the pressure Hillary Rodham Clinton is under to declare herself on a host of controversial education-reform initiatives, including the Common Core State Standards and charter schools, which are championed by the business community, especially hedge fund executives.

This open letter offers some suggestions about the business advice worth heeding, and the kind that’s not.

Dear Secretary Clinton,

There are a great number of lessons that educators and policymakers can learn from business and industry, but they are rarely the ones touted in mainstream education reform circles.

Let me begin by suggesting that the advice of hedge-funders, the high-rollers in what have become “aggressive, highly-leveraged, speculative” casinos—this from Forbes magazine, hardly a tool of the left—is the worst kind you could follow.

The industry’s influence can most recently be seen in New York’s latest regressive teacher-evaluation scheme, which was championed by Gov. Cuomo, the chief water carrier for hedge funds and their war on public schools.

A far better source of wisdom would come from business people and educators who have the proven ability to nurture organization over time—especially those who pursue an all-hands-on-deck, systems-oriented approach to long-term improvement. By arguing a good-school (charters), bad-school (public schools)—or good teacher, bad teacher—duality, many education reformers ignore the extraordinary complexity of the American education landscape, as well as a key challenge for all organizations, including schools: They should all strive for continuous improvement.

Many in the business world have learned this lesson. You could look at folks like Ursula Burns, the CEO of Xerox and a decades-long veteran of a company that has built on a hard-won reputation for quality and customer service; Don Petersen, who rescued the Ford Motor Co. when it was being beaten by the Japanese in the 1980s; and the curious case of Alcoa, which declared that safety was more important than profits. All are turnaround stories in tough competitive industries.

The story of Alcoa is particularly instructive. In 1987, a new CEO, Paul O’Neill, promoted a strategy that defied most quick-change solutions. He didn’t fired anyone. Nor did he focus on improving profits. Instead, O’Neill pushed both management and rank-and-file workers to find ways to improve safety in dangerous aluminum smelting plants. O’Neill’s approach was so inclusive and created such a collaborative culture that the suggestions of hourly workers not only improved Alcoa’s safety record dramatically, it also empowered ordinary employees to propose process changes that would save the company tens-of-millions of dollars throughout its operations. “By taking care of those nonfinancial indicators,” O’Neill said at the time, “I had a really strong feeling that the financial result would take care of itself.”

Alcoa’s focus on collaborative problem solving is a hallmark of the the process-oriented systems thinking that was popularized by W. Edwards Deming, the American statistician and management expert whom I’ve written about extensively, including here. It’s also at the core of Japanese Lessons Study, which has its roots in Deming’s ideas, and is just now sweeping classrooms across the U.S.

A key requirement of credible continuous-improvement systems is the need to create collaborative cultures that are free of fear. That realization has helped to define some of the greatest (and most unsung!) public education success stories, in both red and blue states—including among the poorest hard-scrabble schools and school districts. Like Alcoa, they have focused first on the process—improving school wide literacy, say—not the bottom line, i.e. test scores.

Even Arnie Duncan has discovered the benefits of teacher-led improvement efforts. The U.S. Secretary of Education recently launched a new initiative, Teach to Lead, which “seeks to catalyze fundamental changes in the culture of schools and the culture of teaching so that teachers play a more central role in transforming teaching and learning, and in the development of policies that affect their work.”  Some of the best charter schools, according to a new book A Smarter Charter, Richard Kahlenberg and Halley Potter also are “breaking the mold and providing explicit means for teachers to collaborate and participate in school decisions.”

“[W]e’re at a tipping point where we’re realizing that we cannot have those outside education, with no education background, with no education experience” lead education policy, argues Megan M. Allen, director of programs in teacher leadership at Mount Holyoke College, in South Hadley, Mass.

But, like industry’s doomed quality circles, a 1970s and 1980s fad that withered inside the authoritarian regimes of failing companies, well-meaning teacher- leader/cooperation/collaboration efforts are up against myriad contradictory (and punitive) reform initiatives that alternately seek to shame, scapegoat or incentivize teachers. A 2013 government study, “States Implementing Teacher and Principal Evaluation Systems Despite Challenges,” found numerous problems with how the latest round of teacher evaluations are working on the ground and cited the challenge states were having in “prioritizing evaluation reform amid multiple educational initiatives.”

So, when The New York Times runs a front page story suggesting that you should heed the education prescriptions of hedge fund managers, that’s a sure sign the education establishment is going in the wrong direction.

It’s time to look at schools and entire schools districts that have, over the course of years, developed strong teacher-led school communities that have successfully improved education for their kids—including the poorest kids. They long predate Arnie Duncan’s latest epiphany—some by decades. Yet you probably haven’t heard of them because they are not the stuff that today’s quick-change ed-reformers promote. (My new book will explore these quiet revolutions.)

Here is a preview of the most important lessons those examples can teach:

Search out examples that have stood the test of time: You could begin by looking at the two-decade-old so-called Massachusetts miracle, which is no miracle at all. Since 1993, when Massachusetts passed its landmark education reform legislation, the commonwealth has served as the gold-standard of American education.

“Beginning in 2005, the commonwealth’s students have scored first in the nation in every subject at every grade tested on the National Assessment of Educational Progress, known as the Nation’s Report Card” and outscored kids from around the world on international tests, notes Tom Birmingham, who co-authored the education law. But the Massachusetts reforms were no overnight success; they were the result of years-long hard-work and collaboration at almost every level—among teachers and administrators, Republicans and Democrats, policy makers and local business leader. Now Birmingham warns that former Gov. Deval Patrick’s decision to embrace the common core and PARRC tests, in order to win a one-time Race-to-the-Top grant, threatens Massachusetts’s progress.

Process matters more than the Common Core standards: Outside Massachusetts, the biggest problem with the common core is not the standards themselves, but the way they have been implemented. In some states, like New York, kids were subjected to poorly designed ostensibly common-core aligned tests long before curriculum materials were available and before teachers had received any training in the new standards, spurring a widespread backlash among everyone from educators to the general public.

And unlike in Massachusetts, which developed its standards and state tests over the course of years, improving them with input from educators and key stakeholders at all levels, the common core was developed behind closed doors. It remains a locked system that protects the profits of Pearson, the testing company that is administering the common-core tests, over the long-term improvement that greater transparency would provide.

Why not call on states or regions to bring together top local educators, from K-12 teachers to university experts, along with parents and community leaders to work on improving education standards. And, like Massachusetts, do it for all subjects. That way science and social studies won’t become orphans, neglected because they are not tested. (Ironically, the Patrick administration pulled the plug on U.S. history testing requirements in 2010 citing budget issues even while embracing the common core, which has added new costs.)

The process won’t be quick and it will be messy. But if you select the right people to lead the process and keep it local and collaborative, you could come up with a set of best-practices and standards that states might adopt voluntarily—not because they’ve been offered carrot-and-stick incentives, a la Race to the Top–but because you’ve built a robust framework that makes sense to key stakeholders.

The one thing that won’t work is top-down mandates no matter how smart they are. I recently visited a fast-growing red-state district that has pursued a continuous improvement strategy for two decades; the chief architect of the strategy told me that she has assiduously avoided district mandates, preferring to lead by example and seed experiments among the most forward-looking schools.

Experience matters and teachers, especially veterans, are important. Perhaps the most destructive education-reform shibboleth is the anti-teacher—often deeply misogynistic impulse—of many ed-reformers. While the industrial-era work rules of many teachers’ contracts need to change—even Diane Ravitch favors restructuring seniority rules—the states that promote dumbing down teacher standards and replacing experienced teachers with well-meaning, but virtually untrained recent college graduates pose the biggest threat to improving education.

The best examples of long-term improvement, whether in Massachusetts or the aforementioned red-state district, are driven by schools with lots of veteran teachers and low teacher turnover. Curiously, aside from the fact that it didn’t have a union, the red-state district looked and felt much like its best Massachusetts counterparts. One reason is that both have worked to foster a culture that is free of “fear”–in the red-state district this included winning exemptions from state-mandated test-based teacher evaluations.

To be sure, there are bad teachers just as there are bad investment bankers, bad lawyers and bad CEOs. How many is the subject of great dispute. While Gov. Cuomo and many of his hedge fund backers are convinced that the best strategy for improving education is to weed out bad teachers, even Paul Vallas, the controversial former superintendent of both Bridgeport, CT. and New Orleans’ so-called Recovery School District, says: “The vast majority” of teachers “are excellent when provided with the curriculum, instructional models and supports” they need.

All this raises the important question: Does it make sense to build an entire education-reform apparatus —as we have done for over a decade—on weeding out and punishing bad teachers, instead of supporting good ones or potentially good ones?

Lavish attention—and funding—on education; test sparingly: Since the Great Recession, fair-funding levels nationwide have decreased. That is, the number of states with historically “progressive” funding formulas that provided extra money for high-poverty districts are tightening their purse strings. These states include both Massachusetts and New Jersey, both of which had narrowed—though not eliminated—the achievement gap by redistributing funds to poor districts.

One way to help restore progressive funding would be to cut back on the panoply of low-quality high-stakes tests; annual testing is both expensive and wasteful. Instead, develop high-quality tests and administer them during benchmark years—say 3rd, 8th grade and 10th grade. Refocus resources on helping educators analyze the data they have from both tests and classroom assignments to help improve instruction and learning.

Here’s another idea. Rafe Esquith, who has been called the best teacher in America, suggests buying kids violins instead of computers. Actually, Esquith, whose teaching methods and fifth grade Shakespeare productions have won him both worldwide acclaim and lots of private funding, buys his kids both violins and computers. But his point is this: art and music—from violin lessons to Shakespeare productions to creating hook rugs—are all about teaching kids the importance of process, patience and stick-to-it-ness, as well as giving them an outlet for their creativity. While some of the country’s most heralded education-reformers, including KIPP, have borrowed the superficial trappings of Esquith’s class—the slogans and college banners—they missed the arts, which are at the heart of his lessons and a key to helping students improve their own learning.

Charters—look behind the curtain: The highest performing charter schools in the country are in Boston, at least in part because there are so few of them and charter-authorizations are meted out with great care. Also, the kids in Boston charter schools (like the ones in Harlem) are economically much better off, less likely to be English Language Learners than their public-school counterparts. Over a decade into the charter movement, there is still no evidence that charter schools, as a whole, perform better than public schools with comparable student populations.

Even the grand experiment in New Orleans is more mirage than miracle. A new study by the pro-charter Education Research Alliance for New Orleans confirms what many critics have long charged–that high-performing charter schools engage in illicit “student selection” practices, otherwise known as “creaming”; these practices include counseling out students who are considered a “poor fit” for the school or holding invitation-only events to advertise openings.

Thus, any charter-school strategy has to address the central dilemma of the charter-school movement: What’s the tipping point at which fostering charter schools undermines traditional public schools and the neediest children they serve?

As Mathew Di Carlo recently wrote, it’s time to go back to Albert Shanker’s original idea of charter schools, that is: to create varied “policies and practices within districts, and thus expanding schools’ ability to try different things, test their impact (hopefully on a variety of different outcomes), and inform the design of all schools, regardless of their governance structures.”

Finally, beware the modern-day Moloch, with its insatiable appetite for test scores and other simplistic, and misleading, data points.

The mainstream education-reform movement is at an inflection point. There is more and more dissatisfaction over test mania and its impact on kids. There is deep concern over the corrosive high-stakes teacher evaluations. There is distrust—if not of the common core itself—then of the top-down way it’s being imposed in many states, as well as the opaque testing regimes that are being imposed in its name. And, there is deep suspicion over the enormous potential profits generated by these tests and evaluation systems, as well as education surveillance systems.

This may be the best time in years to promote a new more collaborative, process-oriented approach to education—the kind that has worked at companies like Aloca, in states like Massachusetts, and even in some red-state districts. It is a community and woman-friendly, teacher-focused approach that has demonstrated great promise to improve education. You have a unique opportunity both to turn the page on zero-sum education policies like Race to the Top and NCLB and to challenge all stakeholders—teachers, families, local business people and unions –in supporting a professional, improvement-oriented culture for schools.

Posted in Business, Education, Quality Management, Women | Tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , , | 4 Comments

Adventures in Cuba with My Journalism Students

Ludwig photo

Last month, I led 11 Baruch College journalism students to Cuba as part of a class on covering emerging entrepreneurship in that island nation. The lessons learned for both me and my students were profound.

The purpose of the trip was to report on the recent economic changes instituted under President Raul Castro, especially in the new small-business sector of so-called cuentapropistas, which now number close to 500,000 Cubans, triple the number in 2010. We arrived at the most opportune time—just weeks after President Obama and President Castro announced their historic détente, which will open Cuba to more American goods and visitors and established formal relations between the countries. These changes are likely to have a profound effect on the Cuban economy. Whether they will spread to the political arena is less clear.

My students learned about the emerging entrepreneurial sector and the vagaries of being a so-called cuentapropista, both through Cuba Emprende, a private non-profit that helps train entrepreneurs in Cuba, and from small family-run businesses. These businesses ranged from restaurants to companies like Nostalgiacar, which operates a car service and refurbishes vintage automobiles, to a party planning company. We discovered Cuban ingenuity, honed during the extreme austerity of the post-Soviet so-called Special Period, at Casa Vera, the guest house for foreign students where we stayed and where neighbors have banded together to form an informal cooperative, and at La Finca Marta, an experimental sustainable farm that supplies many of Havana’s top paladares, or private restaurants. We found paquete, the must-have gray-market digital package of entertainment—from soap operas to video games to local advertisements—that Cuban’s share among each other via USB and other computer storage devices. The excellent articles produced by my students can be seen here

Nostalgiacar photo                                                  Reporting at Nostalgiacar

The lessons I learned as a teacher were also noteworthy. After teaching journalism at Baruch for over 15 years, this was the first time I had tried anything of this kind: a project tightly honed around one theme that was at once highly focused, collaborative, practical and intensive. We would be in Cuba for a week and the entire course lasted just three weeks in which we worked together every day. Teaching journalism is in many ways a skills-based discipline; students must learn how to report, write, use a variety of databases with the aim of understanding the news and its context and producing publishable articles about it. But, when teaching students how to cover a specific beat—such as the environment, education or entrepreneurship—content also becomes key.

This was the first course I had taught that was built around a trip, a project in group reporting in which we would share our sources, our notes, our observations; the experience turned out to be extremely valuable. In the coming week, I would learn a great deal about the power of both mutual respect and high expectations and the pedagogical benefits of teamwork and intense collaboration.

Preparation for the trip was daunting. I began planning in January of 2014, a process that lasted a year and involved hundreds of emails and phone calls, as well as navigating the thickets of both Cuban government regulations and a large city university bureaucracy. While I had unwaivering support from my department and administration, and the help of many able Cuban colleagues, I found myself wishing, more than once, that I had never begun the process. But by the fall, I had been able to nail down a rich itinerary filled with everything from field trips to top speakers, including leading academics from the Univ. of Havana to the country’s best filmmakers; an affordable and safe place to stay; and travel arrangements.

This being Baruch College, one of the most diverse colleges in the country, I corralled a highly diverse group of students; most were African-American or Latino. Virtually none of my students was able to afford the cost of the trip. So the preparations included the complicated calculus of figuring out who could get funding through their existing scholarship and honors programs, as well as collecting the funds from various programs, and nailing down funding for the few students who did not receive academic scholarships.

During the first of several planning sessions with prospective students, one young men asked whether I would impose a curfew during our trip—a question that sparked a new worry: What would it be like chaperoning a dozen twenty-somethings in Havana?? In a country where neither our cell phones nor credit cards would work, and where Alan Gross, a U.S. citizen, had spent five years in a Cuba jail, was I going to wake up one morning only to discover that a student had disappeared? Would I get stuck in Cuba for weeks if one of them lost her passport?

Gradually, as I began working with the students, those fears evaporated. Our trip was part of a three-week mid-January intensive course that began with one-week of research and lectures by experts, and culminated in a week of intensive writing. Even before the course began, I explained to the students that in addition to the trip’s pedagogical purpose, they would be serving as ambassadors for Baruch College; any major hiccups, I explained, would jeopardize future trips. I also shared with them the wonderful opportunities ahead of us, as well as the pitfalls of traveling to a foreign country with which we had strained diplomatic relations.

By the time the first week of planning and lectures began, I was beginning to enjoy the process. I had lined up a roster of U.S. experts to speak about Cuba, which helped crystallize the goals of our trip. My students appeared in class on time and fully prepared, having read most of the readings I had given them. They peppered our speakers with excellent questions.

We set off on January 10. When I got to LaGuardia for the first leg of our journey to Havana, via Miami, I discovered that two of our students had headed to JFK by mistake. Yet, strangely, I was feeling quite relaxed. It helped that I had invited Gisele Regatao, my wonderful former graduate student and now the culture editor at WNYC to accompany me as an adjunct professor on the trip.

We arrived in Havana on a Sunday afternoon. And, after an early dinner at Casa Vera, our charming guest house, we headed to the Ludwig Foundation of Cuba, an autonomous cultural organization funded by foreign philanthropists, where our students heard a lecture and slide show about Cuban society and culture. Before my departure, my colleague Ted Henken who has traveled to New Orleans with his students on a number of occasions, suggested what sounded like an exhausting solution to my fears about chaperoning 20-somethings in a city known for salsa, nightlife and partying: Plan something every night, he advised, and give students the choice of going with you—and having their transportation taken care of—or figuring things out on their own. (Ted’s excellent book Enterpreneurial Cuba also served as an important resource for our research.)

Heeding Ted’s sage advice, I had lined up an evening out. And at about 8 p.m. we headed to El Sauce, an outdoor music venue; the size of football field, El Sauce was popular among locals and jammed with music-and-dance lovers of all ages. As Gisele and I elbowed our way to the bar, we wondered how we would find our students, let alone corral them at 10:30 p.m. when, supposedly, we were going to be picked up. I needn’t have worried: Within an hour my kids, as I began to think of them, had congregated together; some were dancing with locals, others were chatting with each other, foot-tapping and swaying to the music. In good journalistic form, a few were taking photos and shooting video.

I went to bed that first night feeling exhausted and excited about the week ahead.

There was a bar, not far from Casa Vera, and a few hotels with internet access within a one-half mile-or-so that the students would occasionally wander off to in the evenings after what was invariable a 10-to-12 hour day of lectures and reporting. But it became clear to me and Gisele that our students had fully absorbed my admonishments about staying sober in a strange city. Equally important, though I had laid down no rules about curfews or anything else, I had made clear my expectations. My students, in turn, rose to the occasion–at all times showing respect for our local hosts and the efforts I had made in planning the trip.

In the mornings, as we boarded our buses, and, throughout the day, as we alighted at various venues—Gisele and I were invariably head counting. By the second day, the kids had started teasing that instead of counting sheep, I would be counting students in my dreams.

I had certainly begun counting on my students. We interviewed entrepreneurs and experts on Cuban economy and culture, and visited the farm and the Cuban Food Industry Research Institute; the students took notes, asked questions and videotaped. We debriefed every night, and I encouraged them to start honing the ideas that they were interested in developing; two or three students, I explained, could work on a story together—especially as some students were were writing print articles and others producing multimedia projects. I encouraged the two or three Spanish-speakers to team up with non-Spanish speakers. Again and again, the students rose to the occasion, sharing their language skills, their technology and their notes.

farm photoAt Finca Marta, a sustainable farm run by agronomist Fernando Funes-Monzote

Another valuable lesson emerged during the course of the week. While the students were writing about disparate businesses, these cuentapropistas shared many common challenges and experiences. As the students and Gisele and I asked questions, the best students began to hone their own questions. Hearing one entrepreneur talk about the difficulty of acquiring credit, prompted questions to other entrepreneurs about how they funded their businesses. Meanwhile, the weaker students who had been silent for the first few days, joined the fray, learning how to ask good questions from their classmates.

The subject of grades never came up. In planning the course, I had decided to give an exam based on the first-week’s readings and lectures. The decision was based, in part, on the need to fulfill the “rigor” requirements of some of the honors programs that were helping to fund individual students, as well as my own fear that the kids wouldn’t complete their readings without an assessment.

I needn’t have worried. When, just before our departure, I asked the students if they wanted their exam grades before we left, they all said: NO! They didn’t want grades interfering with their learning or the fun they were having. When they returned, I felt obligated, by the system, to give them each a grade on the first drafts of their stories; again, I think it was clear to both students and me that the grades were next to meaningless. At the end of the semester, most students had earned an A or A- –most certainly not the usual outcome in my classes. One of the lowest grades went to a student who had, because of work, missed the exam and not made it up; but she had tackled a particularly complicated final project, and I told her that if she got it into publishable shape, I’d bump up her grade. She did, but never asked about her grade.

There are limits to what you can extrapolate from this experience. Our group, while typical in some ways of the Baruch student profile, differed in important respects. It included a large number of honors college and scholarship students. And the need to show up for multiple meetings and complete endless paperwork in advance of both the course and the trip required a high degree of commitment on the part of the students. Then too, I had invested a year of planning—not realistic for a typical college course.

One interesting anomaly: Of 11 students, only two were men. Though I have no way of verifying this, I was told by the young men who did join us that the requirement that students make a $1,000 deposit—a sum that I told the students I could probably wave in cases of high need—dissuaded many of the young men who had initially expressed interest.

I came away from my trip persuaded that an “authentic” experience combined with a high degree of collaboration tightly tied to the aims of the course would be a great model for future courses. Whether that structure will help replicate the close-knit bonds and trust that the students developed both amongst each other and with me in Cuba, is an open question.

Posted in Business, Education, Uncategorized, Women | Tagged , , , , , , , , , , , , | 1 Comment

Diane Ravitch on the Uses and Abuses of Data in Education Reform

Ravitch-Diane-Jack-Miller-a-bit-lighter

I’m delighted to announce that education historian Diane Ravitch will be joining me and Errol Louis, director of the Urban Reporting Program at the CUNY Graduate School of Journalism,  next week to discuss the crisis in education reform. Dr Ravitch, research professor of education at New York University, is one of the nation’s leading scholars warning about the uses and abuses of testing, privatization, charter schools and other education reform strategies.

The conversation will take place at:

CUNY Graduate School of Journalism
219 W. 40th Street, New York, N.Y. 10018

Wednesday, December 3
10 a.m. to 11:30 a.m.

Space is limited. To attend, please register at this link:

 

Posted in Education | Tagged , , , , , , , , , , | Leave a comment

Lessons for Education Reformers from W. Edwards Deming, America’s Leading Management Thinker

W. Edwards Deming united scientific and humanistic appoachs to management

W. Edwards Deming united scientific and humanistic approaches to management

When I returned from speaking at the annual conference of the Deming Institute in Los Angeles last month, the education sites were abuzz about a new Time magazine cover trumpeting “Bad Apples”, the latest example of what has become a new national sport–knee-jerk teacher bashing.

It was a sad reminder of how much our quick-fix, here-today-gone-tomorrow society has forgotten about what our leading institutions learned, less than four decades ago, about the best approach to improving quality—whether at companies, schools or other institutions. These were hard fought lessons learned during a period of deep economic malaise—during the late 1970s and early 1980s—from the man who may have been the most important, and most misunderstood, management thinker of the 20th century.

As I pondered the Time magazine cover and the national narrative of education failure, which scapegoats classroom teachers as the principle culprit for all that ails American education, I couldn’t help but think of W. Edwards Deming and how much has been forgotten since he “rescued” American manufacturing 35 years ago. (Deming’s influence on Japan and the U.S. was the subject of my first book The Man Who Discovered Quality.)

Mainstream education reformers want schools and educators to learn from business. The problem is that they are pushing the wrong business lessons. As the leaders of America’s major corporations learned from Deming, meaningful long-term improvement cannot come from top-down punitive solutions.

Deming’s breakthrough was in combining an understanding of how science–in particular statistical theory–can be used to achieve meaningful systems improvement with what had heretofore been seen as touchy-feely, unscientific approaches to empowerment. He used statistical theory to demonstrate how employees, properly trained, can serve as a key resource for identifying organizational problems, solutions and opportunities for improvement. Importantly, he believed that employees can only meet this challenge if they work in an atmosphere that is collaborative and free of fear.

He also believed that because only management can control the larger systemic factors that impact quality—everything from purchasing to hiring to organizational culture—the onus for creating the underlying conditions for quality improvement rests with management. Thus, Deming’s mantra, thundered again and again in his basso profundo: The responsibility for quality rests with senior management.

Deming’s approach to organizational improvement transformed entire industries in post-war Japan and, later, in the U.S. In the years leading up to his death, in 1993, he began turning his interest to education. He believed that the same principles he advocated for companies—systems thinking, collaborative improvement, understanding statistical variation, creating organizational cultures free of fear and conducive to creative problem-solving—could also transform schools.

Simply put, Deming would be appalled by much of what passes for education reform today. In this blog post, I will describe how the lessons Deming taught American industry might apply to education. His ideas are already being applied by a handful of schools and districts that are explicitly adopting his management ideas (more on this in a future post.) Others, see here and here, have used systems thinking to achieve remarkable improvements, and have created conditions where Deming’s ideas would flourish and, likely, produce even greater gains.

In Japanese schools, Deming’s work is associated with the wildly popular “lesson study,” a collaborative, continuous-improvement approach to lesson planning by small groups of teachers, which is just being discovered by American educators.

Deming’s work has important implications for education: First, it is based on management (everyone from principals to education bureaucrats) recognizing its responsibility for creating a climate conducive to meaningful improvement, including building trust and collaboration, and providing the necessary training; this involves hard work, Deming admonished, not quick-fix gimmicks, incentives or threats.

Second, for many teacher advocates, it means dropping the defensive—education-is-good-enough—posture and embracing a mindset of continuous improvement; it also may mean adopting union contracts that mirror the professional practices of many teachers and are based on more flexible work rules. (Though not the unsustainable sweat-shop hours that are common at many charters.)

Third, by ending the finger-pointing and building a more collaborative approach to improvement, schools and districts could create cultures that are far more rewarding and productive for both children and educators.

But first, a quick primer on Deming’s influence on American management. The time was 1979. U.S. industry was being beaten by foreign competition. Chrysler would be the subject of its first (but not last) government bailout; the Ford Motor Co. was about to lose $1 billion for that fiscal year, and at least as much in 1980; and GM’s profits were expected to plunge by a breathtaking $2.5 billion. Meanwhile, Japanese automakers were gaining market share; Toyota would soon surpass GM as the world’s largest car company.

Then, as now, the convenient scapegoat were rank-and-file employees—in Detroit’s case, the hourly workers whose high wages and ostensibly poor work ethic were initially blamed for the automakers’ problems. Only as Japanese wage rates reached parity with the U.S. and Japanese automakers began hiring American workers for their U.S. plants did some Detroit auto executives begin rethinking that narrative of blue collar failure.

Deming was already an octogenarian when he got his first, storied invitation to help revive the sputtering fortunes of the Detroit auto makers. The catalyst was a documentary, If Japan Can, Why Can’t We, which introduced Deming’s methods and his influence on Japan, to an American audience, providing a much needed wake-up call for Detroit. Deming had helped Japanese companies rebuild following World War II, becoming a Japanese icon; the much-heralded Toyota production system grew out of a years-long dialogue between Deming and the automaker.

Responding to an urgent appeal from Donald E. Petersen, then the new president of Ford, Deming flew to Detroit and, during the next few years, proceeded to rip the lid off of the prevailing assumptions about the quality problems of U.S. companies. It is a testament to how desperate the auto executives were that they grudgingly embraced Deming’s message, despite the fact that he lay the lion’s share of the blame for quality problems on senior management instead of labor.

Petersen alone among the big-three CEOs embraced Deming’s message without reservation. At GM, executives in charge of Cadillac and Pontiac, fearful of how senior management would react, brought Deming in via the back door. In the coming years, his ideas about a bottoms-up systems-oriented approach to management helped transform both automakers and became associated with, among others, the revival of Cadillac and the creation of the highly successful Ford Taurus and Sable automobiles.

Yet, Deming was a quintessential outsider—both in his pedigree and his outlook. Raised in Wyoming, Deming was trained as an engineer and physicist, but became a pioneer in statistical sampling methods. At the peak of his popularity, he worked out of the basement of his modest home in Washington, D.C. Driven by a messianic belief in his ideas, Deming never sought to build a business or a fortune. And so he would never be as wholeheartedly embraced as his contemporaries, especially Peter Drucker, who never challenged the fundamental status quo and, thus, became the darling of 20th century CEOs.

At a time when American industry was becoming ever more siloed and finance focused, Deming advocated a collaborative, systems-focused, process-obsessed approach to management. While he was often derided as a mere statistician, Deming made a crucial breakthrough by linking the scientific explanation for how systems work (in particular, how to understand and manage the statistical variation that erodes the quality of all processes) and the humanistic (an intuitive feel for the organization as a social system and a collaborative, democratic vision of management.)

Both strains—the scientific and humanistic—could be traced to a single deceptively simple, and profoundly elegant, understanding of how all processes work. Every process is subject to some level of variation that is likely to diminish quality. Variation is the enemy of quality; yet it is as ubiquitous as gravity.

What makes variation a particular nuisance is that it comes in two distinct guises: “Special causes” of variation, which are the result of special circumstances or a temporary glitch in the system are opportunistic and, by definition, unpredictable; thus they can wreak havoc with a process and give management no basis on which to predict the quality level of a organization’s products or services. They can, however, be identified and eliminated by workers who have been properly trained to analyze the process.

A simple education example: On a recent New York State English test, students were required to answer questions based on a color map. Most schools, however, lacked color copiers and gave their students the test in black and white, which made the distinctions on the map difficult to read. As a result, many students couldn’t answer the map-based questions correctly. Teachers would have been best equipped to both identify and solve the problem—perhaps by finding a nearby Kinkos or arranging a temporary copier rental—assuming they were given the authority to do so.

“Common Causes” of variation, by contrast, are more difficult to isolate because they are inherent in the system. As such, they are, by definition predictable. While common causes can never be fully eliminated, they allow the process to function with a predictable level of variation. Thus an organization that has only common causes to contend with in its process will produce products of a level of quality that is predetermined by the capabilities of the system.

For example, at a school that doesn’t invest in new text books and chooses to recycle outdated ones, the decision is likely to diminish student performance. This decline will be predictable. While teachers might be able to identify the problem, only senior management (the principal) has the power to change the purchasing policy. Because they are systemic, common-cause variation holds the greatest opportunity for long-term improvement.

The key to reducing variation and improving quality, Deming believed, was to train employees who work with the system every day and who know it best to distinguish between special- and common-cause variation and to empower them to develop creative improvements. Ordinary employees—not senior management, or hired experts—are in the best position to see the cause-and-effect relationships in each process. The challenge for management is to tap into that knowledge on a consistent basis and to make that knowledge actionable. To do so, management must also shake up the hierarchy (if not eliminate it entirely), drive fear out of the workplace, and foster the intrinsic motivation of its employees.

Statistical theory also led directly to Deming’s most controversial ideas on pay incentives. Deming built on Abraham Maslow’s ideas about intrinsic motivation and the hierarchy of needs: He believed that if you create the conditions that allow people to do their best, most people will rise to the occasion. (The obverse is also true: A climate of fear and insecurity is the surest way to kill intrinsic motivation.)

Again, Deming invoked the power of statistical theory: If management is doing its job correctly in terms of hiring, developing employees and keeping the system stable, most people will do their best. Of course, there will always be fluctuations—human beings, after all, aren’t automatons. Deming understood that an employee with a sick child, a toothache or some other “special cause” problem may not function at peak performance all the time. However, in a well-designed system, most employees will perform around a mean.

There will also be outliers who perform above or below the mean—though well-run organizations will have the fewest outliers because they’re hiring and training practices will guarantee a consistent level of performance. The work of high performers, Deming believed, should be studied; their work can serve as a model for improving the system.

Low performers, Deming believed, represent a failure of management to perform one of its key functions. Deming believed that hiring represents a moral and contractual obligation. Once hired, it is management’s responsibility to help every employee succeed whether via training or relocation. While it might occasionally be necessary to fire a poor performer, Deming believed this option should be a last resort.

Of course, Deming’s focus on the role of leadership and organizational culture presents special challenges for schools working under strict union contracts. In New York City, the best public schools are led by strong principals who consciously built collaborative cultures, in many cases because they were able to select their teachers and establish common expectations early on.

Transforming a troubled school can be harder—but not impossible; at Brockton High in Massachusetts, strong leadership and a core group of teachers who were willing to work together made remarkable change possible. But for many schools, building a collaborative culture will be challenging; it will require strong leadership and some flexibility for school principals to move the people they want into key positions.

In their new book A Smarter Charter, Richard Kahlenberg and Halley Potter highlight charter schools that are “breaking the mold and providing explicit means for teachers to collaborate and participate in school decisions.” Many of these charters, like Amber Charter in New York City, are unionized and have exceptionally high teacher-satisfaction ratings and an 89 percent retention rate. Some, including Amber, have specially tailored union contracts that are designed to balance teacher protections with flexibility.

One strategy that almost certainly does not promote collaboration is individualized incentive pay, which according to Deming, “nourishes short-term performance, annihilates long-term planning, builds fear, demolishes teamwork, nourishes rivalry and politics.”

Deming’s opposition to merit pay made him so unpopular among business leaders that many shunned him, a key reason he is not better known today. Even at Ford, Petersen’s immediate successors dismantled Deming’s legacy; but, when the company once again fell on hard times, it was revived by a new CEO, Allan Mulally .

Indeed, contrary to popular belief, and the magical thinking of business people who love individualized incentives, there is no evidence that pay is a driver of long-term  performance improvement in industry. Decades ago, Frederick Herzberg, whose 1968 treatise against incentive pay, “One More Time: How Do You Motivate Employees?”, was the most-requested Harvard Business School article for decades, explained why money doesn’t motivate in the long term. Money, he argued, is a “hygiene factor”: Not enough of it causes distress, but money alone has little to do with job satisfaction or performance. Not surprisingly, companies are perpetually dissatisfied with their incentive systems, which leads to constant tinkering and more business for compensation consultants.

In education, the research on this is clear. A 2010 Vanderbilt University study shows that the performance of teachers who were offered a bonus of up to $15,000 was no better than that of teachers who were offered no incentive. And a survey of 40,000 teachers funded by Scholastic and the Bill and Melinda Gates Foundation found that only one-quarter of teachers felt that performance pay was likely to have a strong impact on student achievement; instead, what the teachers valued the most, according to the study, was “supportive leadership, family involvement in education, access to high quality curriculum and student resources, and time for collaboration with colleagues.”

Just a few years after Deming died, a hot new industry—software design—demonstrated another break in the connection between pay and performance. I have long wondered what Deming would have thought had he lived to see the open-source software movement in which thousands of volunteers produced higher-quality software than the private sector. Think Mozilla’s Firefox Web browser vs. Microsoft’s Explorer. The no-hierarchy, all-meritocracy culture of open source violates every assumption about the link between pay and performance: In open source, many software developers collaborate without any monetary compensation at all. Although some designers eventually find a way to monetize their contributions, few experts dispute that the principle incentive for open-source developers is reputational—the opportunity to do their best work.

If pay for performance is such a problem, why hasn’t this become more apparent in practice? At some highly competitive organizations, such as securities firms or companies like GE, pay incentives foster a culture of competitiveness that is considered important to the company’s organizational DNA. Then, too, incentive pay seems to work beautifully during good times when budgets are fat and there is enough money available to give most employees a “merit raise” or bonus. The problem with incentive-based pay is exposed during bad times, such as the Great Recession, when it becomes a zero-sum game that produces more losers than winners.

Indeed, the biggest problem with incentive pay is that it is inevitably seen as unfair. Evaluation systems linked to single metrics, like test scores, are easily gamed. More-nuanced approaches that include multiple measures, such as graduation and attendance rates, are often seen as too subjective. (While group incentives are more successful, they are not as popular.)

Consider what is happening in Louisiana, where education reform has focused on value-added measurement of teachers and privatization. With the help of Race-to-the-Top funding, which promotes incentive pay tied to student performance on test scores despite a continuing vigorous critique of such methods, the State of Louisiana instituted a bonus system in 2012/2013 school year.

The bonus plans were opaque, widely seen as unfair and based on constantly shifting criteria. As Mercedes Schneider, an English teacher, education blogger and recent bonus recipient explains, the incentive scheme is a crap-shoot, and one that unfairly disadvantages some teachers relative to others. (Schneider donated her $427.76 bonus to a friend raising an autistic child.)

To understand the Louisiana bonus system, it helps to know that by state law,  50 percent of teacher evaluations are based on student-learning measures. Louisiana began using highly controversial value-added measurements (VAM) to evaluate some teachers in 2012-13, but put the plan on hold in 2013-14.

The new bonus system is also based, in part, on student-learning measures. During 2013-14, the first year the bonuses were given, Schneider’s district administered a pretest at the start of the semester, which was intended to serve as a benchmark for the teacher-evaluation measure; but the bonuses were ultimately based solely on end-of-semester test results that were decoupled from the pretest.

This school year, 2014-15, the pretests were scrapped entirely and teacher performance was pegged to performance on end-of-semester tests with arbitrarily determined cut-off scores. This year, too, some teachers of non-tested subjects are being judged on core-subject tests they had nothing to do with.

Then consider the bizarre results of a bonus system at Pierre Capdau, a New Orleans charter school that is part of the New Beginnings network, where a handful of teachers got gargantuan bonuses. The highest award, for $43,000—not a typo—went to a fourth grade teacher who increased her student’s test scores by 88 percent. Meanwhile, the kindergarten teacher who had the highest test score improvement at the school—165 percent on the so-called DIBELS test, which is administered to grades K-2—got a fraction of that amount, $4,086, because kindergarten scores aren’t factored into state evaluations.

But things got even stranger. The stated purpose of the bonuses at Capdau, a failing school that got an F on its most recent state report card (see appendix here,) was to promote teacher retention.  In New Orleans, a virtually all-charter district where large numbers of inexperienced teachers are not only overworked and underpaid, but have little training–a five-week Teach-for-America summer course for new college grads is typical—both morale and retention are real problems.

So, Capdau’s policy was to deliver the bonuses only if the recipients signed up to teach again the following fall. Yet, at least one teacher who was told he would get a bonus, was later informed that there had been a miscommunication and there would be no bonus after all. And the kindergarten teacher with the highest test score improvement was suddenly fired “without cause.”

These Louisiana’s examples reveal the pitfalls of a system that relies on individualized incentives to motivate employees to do their jobs. Deming’s work teaches us that schools, like all organizations, are interconnected systems that require close collaboration among stakeholders to improve. Yet, what possible incentive do teachers have to cooperate with each other if bonus systems are set up to benefit the very few? And imagine what $772,000, the total bonus pool allocated to New Beginnings’ three elementary schools, could have done in terms of hiring subject- matter experts, coaches etc.?

But don’t ask the Capdau principal who developed the bonus scheme—she’s already left for another job. (Indeed, the most rational minds in the charter community recognize that the key to retention isn’t bonuses, but better working conditions and training.)

Meanwhile, Capdau’s fired bonus recipient, Ashleigh Pelafigue, who immediately found a job in another parish, had this to say: “It was the best thing they ever did for me…I am flourishing and becoming even better in a supportive, appreciative and engaging environment that is well on its way to becoming an A school and leading the way to our parish’s continued success.”

If bonus plans, as a way to retain valued employees, tend to backfire, then ed-reform fantasies of forcing out ostensibly lackluster employees and replacing them with a cadre of superstars are not much better. Joel Klein, in his new book Lessons of Hope, proudly invokes the mantra of Jack Welch, the former GE CEO who encouraged New York City’s former schools chancellor to “hire slow and fire fast.” At GE, Welch was known as Neutron Jack for his penchant for firing employees. (More on Klein’s book in a future post.)

But even without a union contract, schools can’t realistically fire (and hire) their way to better results, according to a report by the conservative (and pro-charter) American Enterprise Institute. The total number of college graduates from Barron’s “highly competitive” or “most competitive” institutions in the United States is approximately 141,956 annually, according to AEI. If fully 10 percent entered into teaching for a two year period before moving onto other careers, it would provide 27,655 such educators annually, only 6 percent of the (438,914) teachers at work in the nation’s largest school districts.

Simply put, schools have no choice but to work with the teachers they’ve got. Let’s concede that teachers—like other professionals—can and should continuously improve their craft; and that some teachers should probably never have gone into the profession in the first place. (It’s also true that teacher preparation programs need to be improved. See Arthur Levine’s devastating critique of teacher education.)

Yet, in most school districts the percentage of teachers who are poor performers is quite low. Grover J. Whitehurst, director of the Brown Center on Education Policy at the Brookings Institution, recently estimated that a 5 percent ineffectiveness rate would be typical. For the sake of argument, double or even quadruple that number for the most ineffective districts. Yet, the education-reform movement is largely focused on weeding out a relatively few “bad apples,” rather than on finding ways too help the vast majority—85 to 95 percent of the work force—who are at-least-competent improve their practice.

Today’s education-reform consensus is a reflection of the ideology and outlook of the business people and philanthropists who fund the movement and who bring to it the same top-down, blame-the-rank-and-file mindset of the auto executives of the late 1970s and 1980s. Today’s quick-fix authoritarian strategies—from testing regimes to the failed $1 billion iPad gamble in Los Angeles—mirror the war footing of Detroit when GM’s CEO Roger Smith wasted billions of dollars on robotics as a way to solve the “people problem” in Detroit.

Then as now, the more authorities seek to blame rank-and-file employees, the worse things get. Fear and loathing led, in Detroit, to look-alike cars, poor quality, lost market share. In schools it has resulted in endless testing and test-prep, a narrowing of curriculum, and, no doubt, a deterioration in meaningful education—the kind that is difficult to test.

Then as now, the cognoscenti have sought to quantify all outcomes by looking at a narrowly defined bottom line. For automakers this meant focusing on short-term profits—even if that meant eroding quality and, eventually, market-share. In schools, they look at an ever growing array of test scores—most recently in New York State kids were subjected to at least three different standardized tests per year (one State test and two city tests designed for the sole purpose of evaluating teachers.) One problem is that because of budget constraints, the more you test, the lower the quality of the tests and the less meaningful they become. And given that the tests are cloaked in secrecy, they are stripped of their only pedagogical purpose—the ability to help teachers analyze what their kids know so they can improve.

Deming had little to say about labor unions. But he had famously excellent relationships with rank-and-file employees. Having said that, he would object to work-rules that make it difficult to foster collaboration and problem-solving. He knew that when quality improvement became a team project in the auto industry, it was  associated with a much more motivated workforce, as well as more flexible work rules. Hourly workers who had been written off by the automakers rose to the challenge when given an opportunity to make meaningful contributions to quality. See here.

The lessons for education are clear: Quality improvement must begin with senior management (principals and education bureaucrats) establishing the conditions for collaboration and iterative problem-solving. It requires flexibility and professionalism from both teachers and education leaders. Finally, a climate of fear and finger-pointing will do nothing to improve schools; indeed, it is likely to set back the effort for years to come.

Posted in Business, Education, Quality Management | Tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , | 13 Comments

A Watchdog Reflects on the Failures of Former Superintendent Deasy and Other Grown Ups in the Los Angeles Public Schools

Stuart-Magruder-e1401999511809

During a recent visit to Los Angeles, I sat down with Stuart Magruder, a local architect and controversial watchdog of the Los Angeles Unified School District, to talk about public education in L.A., the iPad debacle, and the recent resignation of Superintendent John Deasy.

Reflecting on Deasy’s tenure, as well as the role of the local teachers’ union in another recent technology disaster, Magruder declared a pox on both their houses.

At the crux of the mess in Los Angeles, are “adults who don’t know how to play together,” explained Magruder, over lunch at a downtown Los Angeles eatery.

Magruder was on the front lines of a key, and contentious, Deasy initiative–an effort to put an Apple iPad in the hands of every teacher and child in the Los Angeles public schools. The strategy, which was devised despite the objections of many educators who believe the iPad is not “the right” device for schools, came to exemplify the top-down decision-making and lack of transparency that would, eventually, derail Deasy’s tenure in L.A.

There were also questions about the $1.3 billion cost of the iPad strategy and its funding, which is where Magruder comes in.  Deasy planned to pay for the iPads with school construction bonds. And Magruder, who serves as a member of the School Construction Bond Citizens’ Oversight Committee,  which was established to offer community stakeholders oversight of bond expenditures, didn’t think construction bonds should be used to pay for the iPads.

While others also have raised questions about the iPad strategy, Magruder was the most outspoken. For his efforts, he was briefly ousted by the school board from his committee seat.

Given the LAUSD’s grave fiscal problems—and the deteriorating condition of its schools–Magruder says he gave voice to local concerns about the wisdom of diverting scarce resources from school repair to purchasing technology devices that would last just three to five years. The LAUSD master plan calls for $40 billion to keep the schools up-to-date. Facilities maintenance will cost an additional $12.9 billion.

Another problem was what Magruder calls Deasy’s “technological determinism.” Magruder, who describes himself as a tech-savvy Luddite, says he was aghast to hear Deasy “denigrate” Shakespeare during a bond oversight committee meeting. Deasy suggested that preparing students for the realities of today’s world and teaching them, say, to read a newspaper is more relevant than reading Hamlet, Magruder recalls.

Magruder is convinced that Deasy saw iPads as a way to solve the “teacher problem”—an all-too-familiar refrain of ed-reformers. The plan was for Pearson, the education technology and text book giant, to load the iPad’s with curriculum materials and lessons that, Magruder says, “were aimed at making teacher’s “less pro-active and engaged” in the lesson-planning process.

Deasy was also responding to pressure from federal and state officials to “roll out a technology program” that would support the Common Core State Standards and related online tests, according to The Los Angeles Times.

Yet, pedagogically, the iPad is flawed. It’s essentially a “closed” device designed to make you a “passive consumer,” argues Magruder who uses a range of computer technology in his architectural practice.  Magruder also questions whether kids in grades K-5 need any technology at all, noting that parents of young children struggle to negotiate basic rules around their use of devices such as cell phone. And, he points out, there is little research on what if any benefits technology holds for K-12 learning.

Maybe, says Magruder wrily, that’s why the late Steve Jobs had a no-iPad policy for his own kids. “They haven’t used it,” Steve Jobs once told a reporter when he was asked about how his kids like the iPad. “We limit how much technology our kids use at home.”

Other tech moguls also embrace a tech-free education for their own children. Execs from Google, Yahoo, Ebay and HP send their children to the Waldorf School which is famous for banishing all electronic technology from its classrooms. Waldorf, which has campuses around the country, promotes an approach to education that emphasizes hands-on experiential learning and physical activity designed to promote creating thinking, focus and collaborative problem solving.

As an architect, Magruder has a bias for hands-on work. Give kids a computer they can take apart. Teach them coding, which Magruder says should be a “core class” for every LAUSD student. Offer robotics classes.

LAUSD is now pursuing a new technology pilot project that allows each school to select its own technological solutions. An investigation is also underway into the fairness of the iPad bidding process, and close ties, and possible conflicts of interest, among Deasy, district officials and both Apple and Pearson, which was to supply the curriculum for the iPads. John Rogers, a UCLA education professor told the LA Times: “We view this moment as an opportunity to establish the sort of reflective and inclusive policy process that would have been helpful to have at the start…The rush and lack of meaningful public dialogue did not serve the district well.”

Deasy’s downfall, according to Magruder, was not the iPad fiasco, but a more recent debacle involving a new electronic student information system. The system was part of a response to a law suit, and subsequent consent decree, which found that the rights of special education students were being violated because the LAUSD routinely lost track of their records, which describe each students needs.

But the new information system, known as MiSiS, overloaded the LAUSD’s servers so that weeks after the start of school, kids still didn’t have workable schedules and many couldn’t attend class. (The head of the district’s technology division, Ron Chandler, abruptly resigned yesterday, the second official  to leave in the wake of the technology crisis.)

Magruder recently instructed his own son, a high school student at a district magnet school (he also has a daughter in middle school), to camp out in front of the school counselor’s office until the problem was resolved.

This brings us to Magruder’s scathing indictment of the teachers’ union. He thinks it’s no accident that in the midst of the information-systems crisis, this fall, his son’s counselor left the school every day at 3 p.m. sharp, even though his son’s school day didn’t end until 4. “The teacher’s union is a joke—a stone wall to progress,” says Magruder whose wife is a union representative for the California State University, Dominguez Hills.

“You’ve got a crisis, and you’re a counselor, and you don’t log some extra time?” says Magruder. “I first get angry, then depressed.”

Magruder is convinced that the union deliberately chose not to do the extra work needed to help resolve the problem. “When adults decide to use kids as a pawns,” he says. “That’s unacceptable.”

Maybe so. But if you want kids–or grown ups–to play nicely together, it helps for one of them to be a leader. For all of Deasy’s zeal, his hostility toward the union–on full display during his star turn as a prosecution witness in the Vergara v. California trial–undermined any hope of building the kind of collaboration necessary for long-term improvement of the district. (The Vergara ruling, which is being appealed, marked a victory for those who wish to overturn the state’s tenure rules and teacher protections.)

“You take something that needs a scalpel and careful instrumentation and instead you take out the sledgehammer,” said Steve Zimmer, a member of the LAUSD school board who supported many of Deasy’s efforts, but criticized his handling of the Vergara case. “Deasy wasn’t careful enough to avoid the perception that he enjoyed using the sledgehammer.”

In Los Angeles, Deasy has been lauded for increasing graduation rates and test scores. But he failed as a leader, and admitted as much shortly after his resignation. While defending his tenure,  Deasy said: “I wish I could have found a better balance between my feeling of urgency in my observation of overwhelming peril and poverty for kids and the ability to have built a more unified will to move quickly to do that.”

As Magruder spoke of Deasy defeat and the union’s intransigence, I was struck by an irony: My principle purpose in traveling to Los Angeles was to attend the annual conference of the Deming Institute, which was founded in order to continue to work of W. Edwards Deming, the management guru whose ideas about systems thinking and collaborative improvement–informed by statistical theory–helped turn around struggling American industries in the 1980s.

The unraveling in Los Angeles is just the latest example of education reformers who have yet to absorb the most valuable management lessons of the last half century–achieving lasting institutional change and improvement involves teamwork, collaboration among all the constituencies in an organization, and systems thinking. None of which have been on display in Los Angeles.

More on Deming in a future blog post.

Posted in Education, Quality Management | Tagged , , , , , , , , , , , , , , , , , , , , | 5 Comments