About Andrea’s Blog

Welcome to my periodic musings on the state of American education, business, journalism, the consumer experience, women and food—how it is grown, cooked and eaten. A systems thinker since I wrote my first book, The Man Who Discovered Quality by W. Edwards Deming, most of my ideas and writing are informed by a systems view of the world.

Posted in Uncategorized | 6 Comments

Diane Ravitch on the Uses and Abuses of Data in Education Reform

Ravitch-Diane-Jack-Miller-a-bit-lighter

I’m delighted to announce that education historian Diane Ravitch will be joining me and Errol Louis, director of the Urban Reporting Program at the CUNY Graduate School of Journalism,  next week to discuss the crisis in education reform. Dr Ravitch, research professor of education at New York University, is one of the nation’s leading scholars warning about the uses and abuses of testing, privatization, charter schools and other education reform strategies.

The conversation will take place at:

CUNY Graduate School of Journalism
219 W. 40th Street, New York, N.Y. 10018

Wednesday, December 3
10 a.m. to 11:30 a.m.

Space is limited. To attend, please register at this link:

 

Posted in Education | Tagged , , , , , , , , , , | Leave a comment

Lessons for Education Reformers from W. Edwards Deming, America’s Leading Management Thinker

W. Edwards Deming united scientific and humanistic appoachs to management

W. Edwards Deming united scientific and humanistic approaches to management

When I returned from speaking at the annual conference of the Deming Institute in Los Angeles last month, the education sites were abuzz about a new Time magazine cover trumpeting “Bad Apples”, the latest example of what has become a new national sport–knee-jerk teacher bashing.

It was a sad reminder of how much our quick-fix, here-today-gone-tomorrow society has forgotten about what our leading institutions learned, less than four decades ago, about the best approach to improving quality—whether at companies, schools or other institutions. These were hard fought lessons learned during a period of deep economic malaise—during the late 1970s and early 1980s—from the man who may have been the most important, and most misunderstood, management thinker of the 20th century.

As I pondered the Time magazine cover and the national narrative of education failure, which scapegoats classroom teachers as the principle culprit for all that ails American education, I couldn’t help but think of W. Edwards Deming and how much has been forgotten since he “rescued” American manufacturing 35 years ago. (Deming’s influence on Japan and the U.S. was the subject of my first book The Man Who Discovered Quality.)

Mainstream education reformers want schools and educators to learn from business. The problem is that they are pushing the wrong business lessons. As the leaders of America’s major corporations learned from Deming, meaningful long-term improvement cannot come from top-down punitive solutions.

Deming’s breakthrough was in combining an understanding of how science–in particular statistical theory–can be used to achieve meaningful systems improvement with what had heretofore been seen as touchy-feely, unscientific approaches to empowerment. He used statistical theory to demonstrate how employees, properly trained, can serve as a key resource for identifying organizational problems, solutions and opportunities for improvement. Importantly, he believed that employees can only meet this challenge if they work in an atmosphere that is collaborative and free of fear.

He also believed that because only management can control the larger systemic factors that impact quality—everything from purchasing to hiring to organizational culture—the onus for creating the underlying conditions for quality improvement rests with management. Thus, Deming’s mantra, thundered again and again in his basso profundo: The responsibility for quality rests with senior management.

Deming’s approach to organizational improvement transformed entire industries in post-war Japan and, later, in the U.S. In the years leading up to his death, in 1993, he began turning his interest to education. He believed that the same principles he advocated for companies—systems thinking, collaborative improvement, understanding statistical variation, creating organizational cultures free of fear and conducive to creative problem-solving—could also transform schools.

Simply put, Deming would be appalled by much of what passes for education reform today. In this blog post, I will describe how the lessons Deming taught American industry might apply to education. His ideas are already being applied by a handful of schools and districts that are explicitly adopting his management ideas (more on this in a future post.) Others, see here and here, have used systems thinking to achieve remarkable improvements, and have created conditions where Deming’s ideas would flourish and, likely, produce even greater gains.

In Japanese schools, Deming’s work is associated with the wildly popular “lesson study,” a collaborative, continuous-improvement approach to lesson planning by small groups of teachers, which is just being discovered by American educators.

Deming’s work has important implications for education: First, it is based on management (everyone from principals to education bureaucrats) recognizing its responsibility for creating a climate conducive to meaningful improvement, including building trust and collaboration, and providing the necessary training; this involves hard work, Deming admonished, not quick-fix gimmicks, incentives or threats.

Second, for many teacher advocates, it means dropping the defensive—education-is-good-enough—posture and embracing a mindset of continuous improvement; it also may mean adopting union contracts that mirror the professional practices of many teachers and are based on more flexible work rules. (Though not the unsustainable sweat-shop hours that are common at many charters.)

Third, by ending the finger-pointing and building a more collaborative approach to improvement, schools and districts could create cultures that are far more rewarding and productive for both children and educators.

But first, a quick primer on Deming’s influence on American management. The time was 1979. U.S. industry was being beaten by foreign competition. Chrysler would be the subject of its first (but not last) government bailout; the Ford Motor Co. was about to lose $1 billion for that fiscal year, and at least as much in 1980; and GM’s profits were expected to plunge by a breathtaking $2.5 billion. Meanwhile, Japanese automakers were gaining market share; Toyota would soon surpass GM as the world’s largest car company.

Then, as now, the convenient scapegoat were rank-and-file employees—in Detroit’s case, the hourly workers whose high wages and ostensibly poor work ethic were initially blamed for the automakers’ problems. Only as Japanese wage rates reached parity with the U.S. and Japanese automakers began hiring American workers for their U.S. plants did some Detroit auto executives begin rethinking that narrative of blue collar failure.

Deming was already an octogenarian when he got his first, storied invitation to help revive the sputtering fortunes of the Detroit auto makers. The catalyst was a documentary, If Japan Can, Why Can’t We, which introduced Deming’s methods and his influence on Japan, to an American audience, providing a much needed wake-up call for Detroit. Deming had helped Japanese companies rebuild following World War II, becoming a Japanese icon; the much-heralded Toyota production system grew out of a years-long dialogue between Deming and the automaker.

Responding to an urgent appeal from Donald E. Petersen, then the new president of Ford, Deming flew to Detroit and, during the next few years, proceeded to rip the lid off of the prevailing assumptions about the quality problems of U.S. companies. It is a testament to how desperate the auto executives were that they grudgingly embraced Deming’s message, despite the fact that he lay the lion’s share of the blame for quality problems on senior management instead of labor.

Petersen alone among the big-three CEOs embraced Deming’s message without reservation. At GM, executives in charge of Cadillac and Pontiac, fearful of how senior management would react, brought Deming in via the back door. In the coming years, his ideas about a bottoms-up systems-oriented approach to management helped transform both automakers and became associated with, among others, the revival of Cadillac and the creation of the highly successful Ford Taurus and Sable automobiles.

Yet, Deming was a quintessential outsider—both in his pedigree and his outlook. Raised in Wyoming, Deming was trained as an engineer and physicist, but became a pioneer in statistical sampling methods. At the peak of his popularity, he worked out of the basement of his modest home in Washington, D.C. Driven by a messianic belief in his ideas, Deming never sought to build a business or a fortune. And so he would never be as wholeheartedly embraced as his contemporaries, especially Peter Drucker, who never challenged the fundamental status quo and, thus, became the darling of 20th century CEOs.

At a time when American industry was becoming ever more siloed and finance focused, Deming advocated a collaborative, systems-focused, process-obsessed approach to management. While he was often derided as a mere statistician, Deming made a crucial breakthrough by linking the scientific explanation for how systems work (in particular, how to understand and manage the statistical variation that erodes the quality of all processes) and the humanistic (an intuitive feel for the organization as a social system and a collaborative, democratic vision of management.)

Both strains—the scientific and humanistic—could be traced to a single deceptively simple, and profoundly elegant, understanding of how all processes work. Every process is subject to some level of variation that is likely to diminish quality. Variation is the enemy of quality; yet it is as ubiquitous as gravity.

What makes variation a particular nuisance is that it comes in two distinct guises: “Special causes” of variation, which are the result of special circumstances or a temporary glitch in the system are opportunistic and, by definition, unpredictable; thus they can wreak havoc with a process and give management no basis on which to predict the quality level of a organization’s products or services. They can, however, be identified and eliminated by workers who have been properly trained to analyze the process.

A simple education example: On a recent New York State English test, students were required to answer questions based on a color map. Most schools, however, lacked color copiers and gave their students the test in black and white, which made the distinctions on the map difficult to read. As a result, many students couldn’t answer the map-based questions correctly. Teachers would have been best equipped to both identify and solve the problem—perhaps by finding a nearby Kinkos or arranging a temporary copier rental—assuming they were given the authority to do so.

“Common Causes” of variation, by contrast, are more difficult to isolate because they are inherent in the system. As such, they are, by definition predictable. While common causes can never be fully eliminated, they allow the process to function with a predictable level of variation. Thus an organization that has only common causes to contend with in its process will produce products of a level of quality that is predetermined by the capabilities of the system.

For example, at a school that doesn’t invest in new text books and chooses to recycle outdated ones, the decision is likely to diminish student performance. This decline will be predictable. While teachers might be able to identify the problem, only senior management (the principal) has the power to change the purchasing policy. Because they are systemic, common-cause variation holds the greatest opportunity for long-term improvement.

The key to reducing variation and improving quality, Deming believed, was to train employees who work with the system every day and who know it best to distinguish between special- and common-cause variation and to empower them to develop creative improvements. Ordinary employees—not senior management, or hired experts—are in the best position to see the cause-and-effect relationships in each process. The challenge for management is to tap into that knowledge on a consistent basis and to make that knowledge actionable. To do so, management must also shake up the hierarchy (if not eliminate it entirely), drive fear out of the workplace, and foster the intrinsic motivation of its employees.

Statistical theory also led directly to Deming’s most controversial ideas on pay incentives. Deming built on Abraham Maslow’s ideas about intrinsic motivation and the hierarchy of needs: He believed that if you create the conditions that allow people to do their best, most people will rise to the occasion. (The obverse is also true: A climate of fear and insecurity is the surest way to kill intrinsic motivation.)

Again, Deming invoked the power of statistical theory: If management is doing its job correctly in terms of hiring, developing employees and keeping the system stable, most people will do their best. Of course, there will always be fluctuations—human beings, after all, aren’t automatons. Deming understood that an employee with a sick child, a toothache or some other “special cause” problem may not function at peak performance all the time. However, in a well-designed system, most employees will perform around a mean.

There will also be outliers who perform above or below the mean—though well-run organizations will have the fewest outliers because they’re hiring and training practices will guarantee a consistent level of performance. The work of high performers, Deming believed, should be studied; their work can serve as a model for improving the system.

Low performers, Deming believed, represent a failure of management to perform one of its key functions. Deming believed that hiring represents a moral and contractual obligation. Once hired, it is management’s responsibility to help every employee succeed whether via training or relocation. While it might occasionally be necessary to fire a poor performer, Deming believed this option should be a last resort.

Of course, Deming’s focus on the role of leadership and organizational culture presents special challenges for schools working under strict union contracts. In New York City, the best public schools are led by strong principals who consciously built collaborative cultures, in many cases because they were able to select their teachers and establish common expectations early on.

Transforming a troubled school can be harder—but not impossible; at Brockton High in Massachusetts, strong leadership and a core group of teachers who were willing to work together made remarkable change possible. But for many schools, building a collaborative culture will be challenging; it will require strong leadership and some flexibility for school principals to move the people they want into key positions.

In their new book A Smarter Charter, Richard Kahlenberg and Halley Potter highlight charter schools that are “breaking the mold and providing explicit means for teachers to collaborate and participate in school decisions.” Many of these charters, like Amber Charter in New York City, are unionized and have exceptionally high teacher-satisfaction ratings and an 89 percent retention rate. Some, including Amber, have specially tailored union contracts that are designed to balance teacher protections with flexibility.

One strategy that almost certainly does not promote collaboration is individualized incentive pay, which according to Deming, “nourishes short-term performance, annihilates long-term planning, builds fear, demolishes teamwork, nourishes rivalry and politics.”

Deming’s opposition to merit pay made him so unpopular among business leaders that many shunned him, a key reason he is not better known today. Even at Ford, Petersen’s immediate successors dismantled Deming’s legacy; but, when the company once again fell on hard times, it was revived by a new CEO, Allan Mulally .

Indeed, contrary to popular belief, and the magical thinking of business people who love individualized incentives, there is no evidence that pay is a driver of long-term  performance improvement in industry. Decades ago, Frederick Herzberg, whose 1968 treatise against incentive pay, “One More Time: How Do You Motivate Employees?”, was the most-requested Harvard Business School article for decades, explained why money doesn’t motivate in the long term. Money, he argued, is a “hygiene factor”: Not enough of it causes distress, but money alone has little to do with job satisfaction or performance. Not surprisingly, companies are perpetually dissatisfied with their incentive systems, which leads to constant tinkering and more business for compensation consultants.

In education, the research on this is clear. A 2010 Vanderbilt University study shows that the performance of teachers who were offered a bonus of up to $15,000 was no better than that of teachers who were offered no incentive. And a survey of 40,000 teachers funded by Scholastic and the Bill and Melinda Gates Foundation found that only one-quarter of teachers felt that performance pay was likely to have a strong impact on student achievement; instead, what the teachers valued the most, according to the study, was “supportive leadership, family involvement in education, access to high quality curriculum and student resources, and time for collaboration with colleagues.”

Just a few years after Deming died, a hot new industry—software design—demonstrated another break in the connection between pay and performance. I have long wondered what Deming would have thought had he lived to see the open-source software movement in which thousands of volunteers produced higher-quality software than the private sector. Think Mozilla’s Firefox Web browser vs. Microsoft’s Explorer. The no-hierarchy, all-meritocracy culture of open source violates every assumption about the link between pay and performance: In open source, many software developers collaborate without any monetary compensation at all. Although some designers eventually find a way to monetize their contributions, few experts dispute that the principle incentive for open-source developers is reputational—the opportunity to do their best work.

If pay for performance is such a problem, why hasn’t this become more apparent in practice? At some highly competitive organizations, such as securities firms or companies like GE, pay incentives foster a culture of competitiveness that is considered important to the company’s organizational DNA. Then, too, incentive pay seems to work beautifully during good times when budgets are fat and there is enough money available to give most employees a “merit raise” or bonus. The problem with incentive-based pay is exposed during bad times, such as the Great Recession, when it becomes a zero-sum game that produces more losers than winners.

Indeed, the biggest problem with incentive pay is that it is inevitably seen as unfair. Evaluation systems linked to single metrics, like test scores, are easily gamed. More-nuanced approaches that include multiple measures, such as graduation and attendance rates, are often seen as too subjective. (While group incentives are more successful, they are not as popular.)

Consider what is happening in Louisiana, where education reform has focused on value-added measurement of teachers and privatization. With the help of Race-to-the-Top funding, which promotes incentive pay tied to student performance on test scores despite a continuing vigorous critique of such methods, the State of Louisiana instituted a bonus system in 2012/2013 school year.

The bonus plans were opaque, widely seen as unfair and based on constantly shifting criteria. As Mercedes Schneider, an English teacher, education blogger and recent bonus recipient explains, the incentive scheme is a crap-shoot, and one that unfairly disadvantages some teachers relative to others. (Schneider donated her $427.76 bonus to a friend raising an autistic child.)

To understand the Louisiana bonus system, it helps to know that by state law,  50 percent of teacher evaluations are based on student-learning measures. Louisiana began using highly controversial value-added measurements (VAM) to evaluate some teachers in 2012-13, but put the plan on hold in 2013-14.

The new bonus system is also based, in part, on student-learning measures. During 2013-14, the first year the bonuses were given, Schneider’s district administered a pretest at the start of the semester, which was intended to serve as a benchmark for the teacher-evaluation measure; but the bonuses were ultimately based solely on end-of-semester test results that were decoupled from the pretest.

This school year, 2014-15, the pretests were scrapped entirely and teacher performance was pegged to performance on end-of-semester tests with arbitrarily determined cut-off scores. This year, too, some teachers of non-tested subjects are being judged on core-subject tests they had nothing to do with.

Then consider the bizarre results of a bonus system at Pierre Capdau, a New Orleans charter school that is part of the New Beginnings network, where a handful of teachers got gargantuan bonuses. The highest award, for $43,000—not a typo—went to a fourth grade teacher who increased her student’s test scores by 88 percent. Meanwhile, the kindergarten teacher who had the highest test score improvement at the school—165 percent on the so-called DIBELS test, which is administered to grades K-2—got a fraction of that amount, $4,086, because kindergarten scores aren’t factored into state evaluations.

But things got even stranger. The stated purpose of the bonuses at Capdau, a failing school that got an F on its most recent state report card (see appendix here,) was to promote teacher retention.  In New Orleans, a virtually all-charter district where large numbers of inexperienced teachers are not only overworked and underpaid, but have little training–a five-week Teach-for-America summer course for new college grads is typical—both morale and retention are real problems.

So, Capdau’s policy was to deliver the bonuses only if the recipients signed up to teach again the following fall. Yet, at least one teacher who was told he would get a bonus, was later informed that there had been a miscommunication and there would be no bonus after all. And the kindergarten teacher with the highest test score improvement was suddenly fired “without cause.”

These Louisiana’s examples reveal the pitfalls of a system that relies on individualized incentives to motivate employees to do their jobs. Deming’s work teaches us that schools, like all organizations, are interconnected systems that require close collaboration among stakeholders to improve. Yet, what possible incentive do teachers have to cooperate with each other if bonus systems are set up to benefit the very few? And imagine what $772,000, the total bonus pool allocated to New Beginnings’ three elementary schools, could have done in terms of hiring subject- matter experts, coaches etc.?

But don’t ask the Capdau principal who developed the bonus scheme—she’s already left for another job. (Indeed, the most rational minds in the charter community recognize that the key to retention isn’t bonuses, but better working conditions and training.)

Meanwhile, Capdau’s fired bonus recipient, Ashleigh Pelafigue, who immediately found a job in another parish, had this to say: “It was the best thing they ever did for me…I am flourishing and becoming even better in a supportive, appreciative and engaging environment that is well on its way to becoming an A school and leading the way to our parish’s continued success.”

If bonus plans, as a way to retain valued employees, tend to backfire, then ed-reform fantasies of forcing out ostensibly lackluster employees and replacing them with a cadre of superstars are not much better. Joel Klein, in his new book Lessons of Hope, proudly invokes the mantra of Jack Welch, the former GE CEO who encouraged New York City’s former schools chancellor to “hire slow and fire fast.” At GE, Welch was known as Neutron Jack for his penchant for firing employees. (More on Klein’s book in a future post.)

But even without a union contract, schools can’t realistically fire (and hire) their way to better results, according to a report by the conservative (and pro-charter) American Enterprise Institute. The total number of college graduates from Barron’s “highly competitive” or “most competitive” institutions in the United States is approximately 141,956 annually, according to AEI. If fully 10 percent entered into teaching for a two year period before moving onto other careers, it would provide 27,655 such educators annually, only 6 percent of the (438,914) teachers at work in the nation’s largest school districts.

Simply put, schools have no choice but to work with the teachers they’ve got. Let’s concede that teachers—like other professionals—can and should continuously improve their craft; and that some teachers should probably never have gone into the profession in the first place. (It’s also true that teacher preparation programs need to be improved. See Arthur Levine’s devastating critique of teacher education.)

Yet, in most school districts the percentage of teachers who are poor performers is quite low. Grover J. Whitehurst, director of the Brown Center on Education Policy at the Brookings Institution, recently estimated that a 5 percent ineffectiveness rate would be typical. For the sake of argument, double or even quadruple that number for the most ineffective districts. Yet, the education-reform movement is largely focused on weeding out a relatively few “bad apples,” rather than on finding ways too help the vast majority—85 to 95 percent of the work force—who are at-least-competent improve their practice.

Today’s education-reform consensus is a reflection of the ideology and outlook of the business people and philanthropists who fund the movement and who bring to it the same top-down, blame-the-rank-and-file mindset of the auto executives of the late 1970s and 1980s. Today’s quick-fix authoritarian strategies—from testing regimes to the failed $1 billion iPad gamble in Los Angeles—mirror the war footing of Detroit when GM’s CEO Roger Smith wasted billions of dollars on robotics as a way to solve the “people problem” in Detroit.

Then as now, the more authorities seek to blame rank-and-file employees, the worse things get. Fear and loathing led, in Detroit, to look-alike cars, poor quality, lost market share. In schools it has resulted in endless testing and test-prep, a narrowing of curriculum, and, no doubt, a deterioration in meaningful education—the kind that is difficult to test.

Then as now, the cognoscenti have sought to quantify all outcomes by looking at a narrowly defined bottom line. For automakers this meant focusing on short-term profits—even if that meant eroding quality and, eventually, market-share. In schools, they look at an ever growing array of test scores—most recently in New York State kids were subjected to at least three different standardized tests per year (one State test and two city tests designed for the sole purpose of evaluating teachers.) One problem is that because of budget constraints, the more you test, the lower the quality of the tests and the less meaningful they become. And given that the tests are cloaked in secrecy, they are stripped of their only pedagogical purpose—the ability to help teachers analyze what their kids know so they can improve.

Deming had little to say about labor unions. But he had famously excellent relationships with rank-and-file employees. Having said that, he would object to work-rules that make it difficult to foster collaboration and problem-solving. He knew that when quality improvement became a team project in the auto industry, it was  associated with a much more motivated workforce, as well as more flexible work rules. Hourly workers who had been written off by the automakers rose to the challenge when given an opportunity to make meaningful contributions to quality. See here.

The lessons for education are clear: Quality improvement must begin with senior management (principals and education bureaucrats) establishing the conditions for collaboration and iterative problem-solving. It requires flexibility and professionalism from both teachers and education leaders. Finally, a climate of fear and finger-pointing will do nothing to improve schools; indeed, it is likely to set back the effort for years to come.

Posted in Business, Education, Quality Management | Tagged , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , | 11 Comments

A Watchdog Reflects on the Failures of Former Superintendent Deasy and Other Grown Ups in the Los Angeles Public Schools

Stuart-Magruder-e1401999511809

During a recent visit to Los Angeles, I sat down with Stuart Magruder, a local architect and controversial watchdog of the Los Angeles Unified School District, to talk about public education in L.A., the iPad debacle, and the recent resignation of Superintendent John Deasy.

Reflecting on Deasy’s tenure, as well as the role of the local teachers’ union in another recent technology disaster, Magruder declared a pox on both their houses.

At the crux of the mess in Los Angeles, are “adults who don’t know how to play together,” explained Magruder, over lunch at a downtown Los Angeles eatery.

Magruder was on the front lines of a key, and contentious, Deasy initiative–an effort to put an Apple iPad in the hands of every teacher and child in the Los Angeles public schools. The strategy, which was devised despite the objections of many educators who believe the iPad is not “the right” device for schools, came to exemplify the top-down decision-making and lack of transparency that would, eventually, derail Deasy’s tenure in L.A.

There were also questions about the $1.3 billion cost of the iPad strategy and its funding, which is where Magruder comes in.  Deasy planned to pay for the iPads with school construction bonds. And Magruder, who serves as a member of the School Construction Bond Citizens’ Oversight Committee,  which was established to offer community stakeholders oversight of bond expenditures, didn’t think construction bonds should be used to pay for the iPads.

While others also have raised questions about the iPad strategy, Magruder was the most outspoken. For his efforts, he was briefly ousted by the school board from his committee seat.

Given the LAUSD’s grave fiscal problems—and the deteriorating condition of its schools–Magruder says he gave voice to local concerns about the wisdom of diverting scarce resources from school repair to purchasing technology devices that would last just three to five years. The LAUSD master plan calls for $40 billion to keep the schools up-to-date. Facilities maintenance will cost an additional $12.9 billion.

Another problem was what Magruder calls Deasy’s “technological determinism.” Magruder, who describes himself as a tech-savvy Luddite, says he was aghast to hear Deasy “denigrate” Shakespeare during a bond oversight committee meeting. Deasy suggested that preparing students for the realities of today’s world and teaching them, say, to read a newspaper is more relevant than reading Hamlet, Magruder recalls.

Magruder is convinced that Deasy saw iPads as a way to solve the “teacher problem”—an all-too-familiar refrain of ed-reformers. The plan was for Pearson, the education technology and text book giant, to load the iPad’s with curriculum materials and lessons that, Magruder says, “were aimed at making teacher’s “less pro-active and engaged” in the lesson-planning process.

Deasy was also responding to pressure from federal and state officials to “roll out a technology program” that would support the Common Core State Standards and related online tests, according to The Los Angeles Times.

Yet, pedagogically, the iPad is flawed. It’s essentially a “closed” device designed to make you a “passive consumer,” argues Magruder who uses a range of computer technology in his architectural practice.  Magruder also questions whether kids in grades K-5 need any technology at all, noting that parents of young children struggle to negotiate basic rules around their use of devices such as cell phone. And, he points out, there is little research on what if any benefits technology holds for K-12 learning.

Maybe, says Magruder wrily, that’s why the late Steve Jobs had a no-iPad policy for his own kids. “They haven’t used it,” Steve Jobs once told a reporter when he was asked about how his kids like the iPad. “We limit how much technology our kids use at home.”

Other tech moguls also embrace a tech-free education for their own children. Execs from Google, Yahoo, Ebay and HP send their children to the Waldorf School which is famous for banishing all electronic technology from its classrooms. Waldorf, which has campuses around the country, promotes an approach to education that emphasizes hands-on experiential learning and physical activity designed to promote creating thinking, focus and collaborative problem solving.

As an architect, Magruder has a bias for hands-on work. Give kids a computer they can take apart. Teach them coding, which Magruder says should be a “core class” for every LAUSD student. Offer robotics classes.

LAUSD is now pursuing a new technology pilot project that allows each school to select its own technological solutions. An investigation is also underway into the fairness of the iPad bidding process, and close ties, and possible conflicts of interest, among Deasy, district officials and both Apple and Pearson, which was to supply the curriculum for the iPads. John Rogers, a UCLA education professor told the LA Times: “We view this moment as an opportunity to establish the sort of reflective and inclusive policy process that would have been helpful to have at the start…The rush and lack of meaningful public dialogue did not serve the district well.”

Deasy’s downfall, according to Magruder, was not the iPad fiasco, but a more recent debacle involving a new electronic student information system. The system was part of a response to a law suit, and subsequent consent decree, which found that the rights of special education students were being violated because the LAUSD routinely lost track of their records, which describe each students needs.

But the new information system, known as MiSiS, overloaded the LAUSD’s servers so that weeks after the start of school, kids still didn’t have workable schedules and many couldn’t attend class. (The head of the district’s technology division, Ron Chandler, abruptly resigned yesterday, the second official  to leave in the wake of the technology crisis.)

Magruder recently instructed his own son, a high school student at a district magnet school (he also has a daughter in middle school), to camp out in front of the school counselor’s office until the problem was resolved.

This brings us to Magruder’s scathing indictment of the teachers’ union. He thinks it’s no accident that in the midst of the information-systems crisis, this fall, his son’s counselor left the school every day at 3 p.m. sharp, even though his son’s school day didn’t end until 4. “The teacher’s union is a joke—a stone wall to progress,” says Magruder whose wife is a union representative for the California State University, Dominguez Hills.

“You’ve got a crisis, and you’re a counselor, and you don’t log some extra time?” says Magruder. “I first get angry, then depressed.”

Magruder is convinced that the union deliberately chose not to do the extra work needed to help resolve the problem. “When adults decide to use kids as a pawns,” he says. “That’s unacceptable.”

Maybe so. But if you want kids–or grown ups–to play nicely together, it helps for one of them to be a leader. For all of Deasy’s zeal, his hostility toward the union–on full display during his star turn as a prosecution witness in the Vergara v. California trial–undermined any hope of building the kind of collaboration necessary for long-term improvement of the district. (The Vergara ruling, which is being appealed, marked a victory for those who wish to overturn the state’s tenure rules and teacher protections.)

“You take something that needs a scalpel and careful instrumentation and instead you take out the sledgehammer,” said Steve Zimmer, a member of the LAUSD school board who supported many of Deasy’s efforts, but criticized his handling of the Vergara case. “Deasy wasn’t careful enough to avoid the perception that he enjoyed using the sledgehammer.”

In Los Angeles, Deasy has been lauded for increasing graduation rates and test scores. But he failed as a leader, and admitted as much shortly after his resignation. While defending his tenure,  Deasy said: “I wish I could have found a better balance between my feeling of urgency in my observation of overwhelming peril and poverty for kids and the ability to have built a more unified will to move quickly to do that.”

As Magruder spoke of Deasy defeat and the union’s intransigence, I was struck by an irony: My principle purpose in traveling to Los Angeles was to attend the annual conference of the Deming Institute, which was founded in order to continue to work of W. Edwards Deming, the management guru whose ideas about systems thinking and collaborative improvement–informed by statistical theory–helped turn around struggling American industries in the 1980s.

The unraveling in Los Angeles is just the latest example of education reformers who have yet to absorb the most valuable management lessons of the last half century–achieving lasting institutional change and improvement involves teamwork, collaboration among all the constituencies in an organization, and systems thinking. None of which have been on display in Los Angeles.

More on Deming in a future blog post.

Posted in Education, Quality Management | Tagged , , , , , , , , , , , , , , , , , , , , | 5 Comments

Building A Better Teacher: Some Hard Lessons of Ed Reform

Building a Better Teacher

I picked up Elizabeth Green’s new book, Building a Better Teacher, with great anticipation. By the time I finished reading the nicely written, highly detailed descriptions of some of the latest efforts to improve teaching, I was alternatively gratified, intrigued and more-than-a-little frustrated.

Let’s start with why I was gratified: Her book argues that good teaching can and must-be taught. This would seem to be mere common sense. But what Green calls the “black box” of teaching has been long neglected not only by university based schools of education, but also by education-policy makers.

Green’s narrative seeks to debunk the notion that the secret to improving U.S. education is to place a superstar teacher in every classroom. She argues, instead, what many education reformers would consider heresy: That improving education is about teaching teachers, including ordinary ones, how to improve. Being a good teacher, Green painstakingly shows us, is extraordinarily difficult. And teaching someone to become a good teacher is even more difficult. Writes Green: “By misunderstanding how teaching works, we misunderstand what it will take to make it better—ensuring that, far too often, teaching doesn’t work at all.”

Green argues that by making accountability (via test scores) and autonomy (the notion that teachers are professionals who should be treated accordingly) the two dominant theories of teacher improvement, policymakers and pundits “have left us with no real plan. Autonomy lets teachers succeed or fail on their own terms with little guidance. Accountability tells them only whether they have succeeded, not what to do to improve. Instead of helping, both prescriptions preserve a long-standing culture of abandonment.”

Students of both management and management’s influence on education also will be intrigued to learn that much as the Japanese bested U.S. manufacturers 40 years ago by neglecting home-grown quality management practices, the Japanese education system is built on long-neglected American ideas. Moreover, the methods used to improve education in Japan are strikingly similar to the collaborative, iterative practices they use in industry. Without saying so explicitly, Green reminds us that it is possible for educators to learn from business. But they need to learn the right lessons!

Green’s subject is hugely important. But to make her argument she must navigate the minefield of the highly politicized education-reform movement. And, at times, she seems to go to extraordinary lengths to avoid offending education reformers who have made accountability—not improvement of either teachers or pedagogical methods—the centerpiece of both private education-reform efforts and the nation’s education policy.

In a recent phone interview, Green noted that the education-reform “insiders,” especially those she calls the “entrepreneurs” are much more invested in teacher training and development—and less focused on accountability—than “outsiders” realize. As I will explain later, even if this is true, it is a problematic argument given the adverse impacts the accountability movement has had on American education and its close connection to the education “entrepreneurs” she writes about.

Green, it should be noted, is CEO of Chalkbeat, which describes itself as a “nonprofit news organization covering educational change.” Chalkbeat funders include, among others, philanthropies at the forefront of the privatization and accountability movement—the Bill and Melinda Gates Foundation, the Walton Family Foundation and Seachange Capital. Fair disclosure: Green and I both worked at U.S. News and World Report, though our tenures didn’t overlap and we have never met.

Green begins her narrative with Spartan Village, a lab school affiliated with Michigan State University, and the efforts of Magdalene Lampert and Deborah Ball, two extraordinary educators who are trying to unlock the mystery of how kids learn and to develop better teaching methods and teachers. (See Aaron Pallas’s concise explanation of the three types of knowledge teachers need in order to help students learn. )

Efforts to scale the model, painstakingly developed at Spartan Village and nurtured by MSU, were overtaken by the accountability movement, which promised a quick fix for the ills of the American public education system. In the process, the Spartan Village ideas were sidelined even as a very similar pedagogy was developing in Japan.

Thus, students of American business will see history repeating itself—this time to the detriment of kids. Green recounts how the most innovative American approaches to teacher training were exported. Japan, for example, took inspiration from three key thinkers, all of them American: John Dewey, the philosopher; George Polya a Stanford Univ. mathematician; and the National Council of Teachers of Mathematics, which was inspired by Magdalene Lampert and written in part by Deborah Ball of Spartan Village.

Green’s chapter on the lessons Japan learned from the U.S. marked a jarring déjà vu for me. As researchers on Japan’s education reforms toured the island nation, they kept hearing the name of W. Edwards Deming, the Iowa-born statistician and quality expert who had taught the Japanese what we have since come to associate with Japanese quality management. (My own first book, The Man Who Discovered Quality, became a best-seller in the Midwest and Japan, was about Deming.) Writes Green:

“Like Deming’s work, the NCTM standards had a more loyal following in Japan than in the country that birthed them. Not only had the Japanese discovered the American math standards…They’d taken a population of earnest but ordinary teachers and produced a country full of Magdalene Lamperts.”

And: “The Americans produced wonderful intellectual work on what teaching could look like, but they had failed to implement any of it.”

The reasons U.S. education reformers failed to adopt their own best teachings recall the experience of U.S. industry, which came to be clobbered by the Japanese in the 1970s and 1980s after they embraced quality ideas long neglected by American companies. American education-reformers established goals and standards (management-by-objective in biz-school parlance) and tests (accountability) but they didn’t develop the systems and tools for helping teachers achieve those goals. By contrast, Green tells us that Japanese educators pursued a continuous improvement philosophy called jugyokenkyu or “lesson study” that was the educational counterpart to Japanese industry’s kaizen, which is all about developing the training, mindset and processes for the continuous improvement culture that, for years, made Toyota the world’s leading auto manufacturer.

This is how Green describes jugyokenkyu: “(A) bucket of practices that Japanese teachers use to hone their craft, from observing each other at work to discussing the lesson afterward to studying curriculum materials with colleagues.” Japanese teachers were good, not because they had been born that way. Rather, studying how to teach was part of their every-day job, putting their work under a microscope and working collaboratively with colleagues to constantly improve their practice.

“The truth is, with teaching 10 percent of it is the technology or the idea or the innovation,” Green quotes James Stigler, a UCLA professor who has devoted his career to studying and improving classroom teaching. “Ninety percent of it is figuring out how to actually make it work to achieve our goals for students. American ideas might have taken the Japanese 10 percent of the way there, but Japanese jugyokenkyu had done the rest.”

The parallels to American industry and its failure to adopt the homegrown ideas that would transform Japan are striking. Ignored at home, Deming’s theories on quality improvement laid the foundation for Japanese kaizen, and soon companies like Toyota were beating U.S. competitors with superior quality products and services. By the 1980s, U.S. companies had rediscovered Deming, and a few successfully adapted his ideas. But many others who sought to replicate Toyota’s production techniques failed because they saw only the most visible manifestations of kaizen—the statistical tools that were used to analyze process quality or the “andon” cords that allowed production workers to stop the line when they sensed a problem.

U.S. companies either misunderstood, or rejected, the underlying philosophy that informed kaizen, especially its repudiation of the command-and-control methods at the heart of American management. Command-and-control has been a cornerstone of American business culture for over a century, ever since Frederick Winslow Taylor’s scientific management experiments sought to deskill work and workers.

By contrast, Deming argued that long-term improvement could only be achieved by enlisting the knowledge and creativity of every employee in an organization. Deming’s work was built on two interwoven ideas both of which were informed by statistical theory. First, a system cannot be improved unless it is in control, i.e. stable and predictable (Predictable doesn’t mean perfect; it just means that you can anticipate, say, a 10 percent defect rate.) Second, once a system is in control, it is only those closest to a given process—in the case of a classroom, the teachers and the students—who are in the best position to identify opportunities for improvement, assuming they have been given the tools and training to do so.

However, because only senior management can control key factors needed for systemic change—everything from organizational culture to the way supplies and technology are purchased—the responsibility for improvement ultimately rests with senior management. The biggest problem with American quality wasn’t the worker or the union, Deming invariably intoned in his basso profundo, “The problem is management!”

Deming

Quotation-W-Edwards-Deming-fear-Meetville-Quotes-5513

Via kaizen, Japan created an organizational culture that made improvement a priority and that both welcomed employee ideas and acted on them. Employees receive special training and learn to use statistical tools in combination with their specialized knowledge of a given process—as well as their intuition—to develop creative improvement ideas. Throughout there is constant collaboration, process improvement and iterative learning.

So, how can American’s reclaim lesson study and similar grass-roots improvement efforts? Don’t look to traditional university based schools of education, says Green, because these have “marginalized” the practice-aspect of teaching. Green notes that when Lampert, one of her heroines, went to the Harvard Graduate School of Education, only one class in the course catalogue included the word “teaching.”

What’s needed, Green writes, is a better infrastructure for teacher education. One solution might be a return to the residency type lab schools, like Spartan Village or the old normal-school model. A good idea although, as she noted during our phone conversation, residency programs are expensive and the work they do is time-consuming, and both U.S. culture and policymaking favor quick-fixes.

Less credible is Green’s assertion that the charter movement has taken up an approach akin to “lesson study” and that it might help provide part of the much-needed infrastructure for teacher improvement. She devotes a surprisingly large chunk of her book to Doug Lemov, the managing director of Uncommon Schools, a charter chain, and author of best-selling books that have become the gospel of the no-excuses behavioralist approach embraced by most charter schools. This approach holds that what disadvantaged kids need more than anything is strict discipline, even if it sometimes verges on being “punishing, even cruel.” Lemov also holds an MBA from Harvard where he focused on studying accountability systems and once served as chief of accountability at the State University of New York’s Charter Schools Institute.

Green tells us that Lemov eventually embraced Japanese lesson study. “Imagining the educational equivalent of an efficient and responsive assembly line, Doug and his colleagues did not quite reinvent lesson study. But they came close, holding standing meetings on the minute details of the homework system and devising schedules to enable teachers to make regular visits to each others classrooms.”

Skeptics alert: the assembly line is an artifact of command-and-control Taylorism, NOT of employee-driven continuous improvement efforts.

Green acknowledges that Lemov 2.0 has “learned some of the most important teaching lessons the hard way, and he’d done so, he knew all too well, on the backs of some children. (Indeed, he not only rejected many of his own early practices; he rejected the ‘no excuses’ label altogether.)”

Whatever Lemov has learned, my experience in New Orleans, the chief laboratory of education reform, where Lemov’s books grace every entrepreneur’s book shelf, suggests that the no-excuses model is still used as much to help inexperienced teachers manage a classroom as it is to help kids learn. It is in New Orleans that education reformers fired unionized mostly African-American teachers (illegally, according to a recent court ruling) and replaced them with an army of inexperienced college graduates who had just five weeks of training from Teach for America. According to the latest published figures, 42 percent of the teachers in the city’s non-selective schools have less than three years of experience, 22 percent have less than one year of experience. **

Long-touted as a miracle of educational entrepreneurship, New Orleans, where virtually 100 percent of public schools have been converted to charters, has fallen short. The typical charter school in New Orleans, “is not sustainable for the adults, not fun for kids,” says Anthony Recasner, one of the few African-American charter-school pioneers in New Orleans.

Although Green suggests that the best charter schools are focused on teacher improvement and training, schools in places like New Orleans are caught in the accountability trap that is now the centerpiece of American education policy. New Orleans schools, for example, are graded largely based on standardized test scores. They not only compete for scarce funding from venture philanthropists based on those scores, but risk being closed down if they do not keep their test scores and school grades up. The result has been curricula focused on test-prep and a system that offers few incentives for schools to serve the neediest kids while penalizing the ones that do.

The accountability movement itself owes much to Eric Hanushek, an economist and another key figure in Green’s book. Hanushek popularized value-added measurement of teachers—the idea that you can use test scores to measure the contribution every teacher makes to a child’s learning. Among those taken with Hanushek’s research, Green tells us, is Bill Gates who once said: “If the entire U.S., for two years, had top quartile teachers, the entire difference between us and Asia would go away. Within four years we would be blowing everyone in the world away…All you need are those top quartile teachers.”

By 2009, accountability in the form of government-mandated outcomes-linked teacher evaluations had been enshrined into law via the Obama administration’s Race to the Top competition, a zero-sum game in which some schools win extra money if they promise to meet certain Federal requirements—chief among them punitive teacher evaluations linked to student growth on test scores.

The problems with this outcomes-oriented approach are manifold. For one thing, driven by federal mandates and venture philanthropists who equate education quality with high “value-added” test scores, states put in place complicated testing regimes before they had developed training and curriculum materials for teachers. In other words, they set kids and teachers up to fail. Some schools and districts responded by cheating.
Even without cheating, a new DOE study shows the evaluation systems haven’t been working well.

While Green’s narrative suggests that accountability at the expense of better teacher training and lesson study is a mistake, she is reluctant to criticize either Hanushek or the privatization movement, which was built on the up-or-out accountability movement. Of Hanushek she writes:

“He had not studied education’s vast middle. ‘The black box of the production process,’ he callied it. That is, classroom teaching and learning. He looked at teachers’ effects, but not at their work—at teachers, but not at teaching.

“Hanushek made the observation as an aside, but the decision to overlook teaching’s ‘black box’ would prove just as influential as his ‘value-added’ innovation.”

Of course, it’s possible to pursue both continuous improvement and a well-thought out system of accountability (though a thoughtful accountability system would be very different from the standardized testing regimes currently imposed by many states.) Green suggests that organizations like KIPP have gotten the balance between accountability and teacher training right. She even invokes one of KIPP’s superstar math teachers, Joe Negron, who returned to the classroom after serving as the founding principal of KIPP Infinity in New York City, to underscore the idea that even the best teachers need—and want—more training. Green writes of Negron: “Every night he stayed up late reworking his lesson plans from scratch. What he needed was guidance. Help. A coach.”

But charter schools like KIPP Infinity are an exception—something that Green doesn’t acknowledge in her book. (KIPP Infinity also avoids hiring first-year teachers and has experimented with shorter hours to retain teachers.) In our phone conversation, Green also argued that TFA exemplifies the commitment of educational entrepreneurs to teacher training, noting TFA’s newly announced plans to provide a year of up-front training for some of its new recruits. But by defending even the most “disruptive” entrepreneurs, and downplaying the role that organizations like TFA have played in undervaluing career teachers and teaching experience, Green risks undermining her own argument.

It’s noteworthy that Green makes no mention whatsoever of another high-profile teacher-training effort founded and embraced by the education “entrepreneurs,” the Relay Graduate School of Education, a non-university based graduate teaching program. Not only was it established by three leading no-excuses charter-school chains, presumably to solve the teaching “infrastructure” problem Green is writing about, but as she noted in a 2011 article about Relay, the school itself is built on an accountability model that makes “proven student learning gains a requirement of receiving a Master’s degree.” To graduate, students must prove that their students can make “at least a year’s worth of academic progress. Carol Burris, a respected New York State principal wrote this critique of Relay in 2012:

Both Relay and TFA’s new yearlong teacher-training pilot probably grow out of the recognition that there simply aren’t enough superstar ivy leaguers to fill openings in large urban school districts. They may also represent tacit acknowledgement that the promise of charter schools far outshining public schools has yet to materialize. Steven F. Wilson, in a paper for the conservative American Enterprise Institute, writes that Matt Candler, former vice president of school development for KIPP, addressing an audience of charter enthusiasts at a fall 2007 conference at the American Enterprise Institute (AEI) in Washington, figured that at most, 200 schools nationwide are truly excellent.” Two hundred. Out of what was then just over 4,000 charter schools. “Some of the movement’s most prominent leaders have begun to voice their own concerns—even their exasperation—with the indifferent results of many, if not most, charter schools,” writes Wilson.

Green might have made her argument more strongly if she had focused on a broader range of schools that have—despite the vicissitudes of Federal and state mandates—succeeded in pursuing that illusive goal of educational “coherence” and real long-term improvement. Readers of this blog will be familiar with Brockton High, the largest, once-failing high school in Massachusetts, which used an iterative approach to improving literacy—a strategy it has continued to refine over the last 15 years—to achieve an extraordinary turnaround. As in Japanese “lesson study” or kaizen, the effort was led by teachers, many of whom have taught at the school for over 20 years, including during the years when Brockton was failing. What changed at Brockton was not the teachers, but a strategic focus on teacher-led improvement.

By contrast, Green’s principle narrative example, Spartan Village, did not fare as well. The school’s principal had been able to make accommodations with the unions to build in more time for lesson-study-like teacher meetings. (Indeed, at no point does Green identify the union as a serious structural impediment to the work Spartan Village was doing or to lesson study.) But it was the outside forces that threatened the experiment’s sustainability. “Each time a new superintendent arrived,” Green explains, the principal had to “defend the Spartan Village exceptions. Every time budgets few tight, the school board always seemed to turn to Spartan Village. Did the training school across the tracks really need to exist?”

Green correctly identifies the need for a better infrastructure for more effective teacher training and teacher-led continuous improvement. But, first, the education establishment will have to abandon the fight-to-the-death now playing itself out between charters and publics. And it will need to develop public policies that, instead of imposing ever more top-down mandates, foster grassroots improvement efforts; education “entrepreneurs,” policy makers will learn, can be found in traditional public schools as well as charters. The future of American education depends on encouraging grassroots improvement everywhere.
** In a feat of pro-charter legerdemain, the Cowen Institute’s most recent 2013 study masks the high number of inexperienced teachers at the charter schools that teach the neediest children. The 2012 and 2013 reports use exactly the same data published by the State of Louisiana for the 2010-2011 school year. Yet, the 2013 report, conflates the data for the schools run by all three authorizers—the very large Recovery School District, which educates the vast majority of kids, including the neediest; the much smaller Orleans Parish School Board, which educates the most affluent; and the tiny Board of Elementary and Secondary Education, which operates only a handful of schools—to show that just 38 percent of its teachers have less than three years of experience. By contrast, the 2012 report shows that 42 percent of the teachers at non-selective RSD schools had less than three years of experience compared to just 28 percent at the selective OPSB schools. At BESE, which operated just five schools in the 2010-2011 school year, 60 percent of teachers had less than three years experience.

Posted in Business, Education, Quality Management | Tagged , , , , , , , , , , , , , , , , , , , , , , , , , | 6 Comments

Unwrapping New York State’s latest common core tests

Once again I am in possession of a bit of educational contraband.

For the second year in a row, I have received a copy of the New York State English Language Arts tests for grades 6 to 8, which were administered in April. (Though, this year, my set appeared incomplete as it contained only books one and two for each grade–not the three books that were included last year and that I was told were given this year. So my analysis here is confined to only two booklets for each grade.)

Anyone who has followed the controversy around the introduction of the New York State’s “common-core aligned” tests, knows that there has been a growing backlash–and not necessarily against the common core itself. Rather, a great many educators object to the quality and the quantity of tests–in addition to six days of “common core” testing, New York kids are now finishing the Measurements of Student Learning (MOSL) tests, the sole purpose of which is to evaluate teachers, as well as field tests for next year’s “common core” tests. In the fall, students as young as kindergarteners endured base-line testing for the MOSL.

Most importantly, educators are outraged by the secrecy in which the tests are cloaked. Under its $32 million contract with Pearson, the publisher and educational-testing giant, the state is barred from making the Pearson-designed tests public;  New York educators are under a gag order prohibiting them from revealing anything about the test.

As of this writing, it is not clear to me why this gag order is not in violation of New York State’s Truth-in-Testing law, which requires disclosure of test questions. If anyone knows the answer, please write in!

Back to the tests themselves. In April, 557 principals across New York State  wrote an open letter to parents outlining their concerns about the latest round of testing.

A similar letter-writing followed the first round of common-core testing last spring.

Elizabeth Phillips, the principal of Public School 321 in Park Slope, Brooklyn, and one of the signatories of this year’s letter, followed up with this critique of the tests in an April New York Times OpEd. It read in part:

“I want to be clear: We were not protesting testing; we were not protesting the Common Core standards. We were protesting the fact that we had just witnessed children being asked to answer questions that had little bearing on their reading ability and yet had huge stakes for students, teachers, principals and schools. (Among other things, test scores help determine teacher and principal evaluations, and in New York City they also have an impact on middle and high school admissions to some schools.) We were protesting the fact that it is our word against the state’s, since we cannot reveal the content of the passages or the questions that were asked.”

Bill Heller, an expert on common core-aligned assessments who defends the common-core standards themselves, agrees that secrecy is a big problem. “It’s unconscionable that they no longer release the tests,” says Heller, a senior education consultant at Teaching Matter who argues that tests need to be linked to formative assessments. “When testing is something that is outside of the arc of student improvement, then it becomes an obstacle to learning and teaching and does more damage then good.”

Formative assessments provide a way to gather feedback “that can be used by the instructor and the students to guide improvements in the ongoing teaching and learning context,” according to Carnegie Mellon’s Eberly Center for Teaching Excellence and Educational Innovation. “The goal of formative assessment is to gather feedback.”

In other words, while today’s gotcha accountability regime seems as focused on evaluating teachers and highlighting the flaws in public education as on examining student learning, it almost completely ignores the potential for low-stakes assessments to actually improve teaching and learning.

Then there is the problem of Campbell’s Law, which essentially argues that  the higher the stakes for a measurement, the less accurate it is likely to be. Donald Campbell was an American social psychologist and noted researcher who did pioneering work on social-science methodology.

Also, secrecy prevents the most knowledgeable educators–classroom teachers–from critiquing test questions so that they too can be improved. It is the input from teachers and a continuous iterative improvement process that, according to leading educators in Massachusetts, made that state’s highly regarded curriculum and MCAS test the best in the nation. This is how Massachusetts described that process in 2009, just before the state decided to jettison both curriculum and MCAS in favor of the Common Core State Standards and related new tests:

“The MCAS test development process–from the selection of the learning standards that are included in each test to the development of test items (questions) to the production of test booklets–is designed to ensure that test results are valid and reliable. Items undergo extensive review and field testing before they become common items.

“The item development cycle, from the beginning of item development to the inclusion of an item as a common item on an MCAS test, generally takes two to three years.”

In New York State where the common core-aligned tests were administered for the first time last spring, well before any “common core” curriculum was made available for teachers, the process is not just secretive, but also haphazard. The most fundamental problem with last year’s tests was that they were administered before teachers had any meaningful training in the new standards, and before students had much exposure to them; schools, kids and educators, in short, were being set up for failure.

New York State is part of PARCC, a consortium that is developing a common core assessment. But it is one of the few states that rushed to implement a common-core based test before PARCC itself is ready to roll out its own assessment in the 2014/2015 school year.

A close reading of this year’s tests reveals that while the 2014 ELA exams appear to be somewhat shorter than last year’s tests, which were criticized for being too long, both years’ tests are dominated  by a focus on non-fiction texts with little content that speaks to the urban experience. Some curriculum materials were available this school year, but schools reported receiving them late–as of December, 2013, many hard copies were still missing.

Among the three grades, this year’s eighth grade test is heavily weighted to science-related readings (5 of 9 texts,) including one about the iceberg that sank the Titanic. The remaining texts include one nonfiction text–an OpEd about the high cost of rescuing hiker–one poem and two fiction texts.

The seventh grade tests are the most evenly distributed among genres with three science-lated texts; three fiction, including an excerpt from Louisa May Alcott’s Jo’s Boys; and two texts relating to history or current events–one of them a first-person travelogue about Tibet.

The sixth grade test is dominated by non-fiction. It includes three science-related and five other non-fiction texts with one poem serving as the sole example of a fiction piece.

The sixth grade nonfiction topics include an excerpt from a memoir about how a sewing machine and electrification changed the life of one American woman. Among the clothes she sewed, the article explains, were matching outfits for a granddaughter and her Barbie doll.

The reference to the Barbie doll, identified in the text as a trademark of the Mattel Corp., is among a few references to products and corporations that have drawn charges that Pearson is engaging in “product placement.” There is no evidence that Pearson has accepted or solicited funding for product placements; but given the high stakes and big-bucks at stake with these tests, educators are right to be wary.

Based on the first two booklets of the middle school tests, they appear to be shorter than last year’s test. For example, the first two booklets of the seventh grade test weighed in at 45 pages and contained nine passages of one-to-two pages each; three short-answer and one longer essay question; and 49 multiple-choice questions.

By contrast, last year’s seventh grade test included the following in three booklets: At 72 pages, it included 14 passages, the vast majority of which are one-to-two pages in length. There were also eight short-answer questions that require writing about one long paragraph each, as well as two essay questions, and over 100 multiple-choice questions.

On the assumption that greater knowledge of the tests, and greater debate about the tests, can only serve to improve them, here’s a recap of the items in Books 1 and 2 of the ELA tests for Grades 6, 7 and 8 along with a few sample questions.

The sample questions I’ve chosen to include here represent a range of different types of questions, including a few that I found excessively complicated. As one educator said to me: If you, with a master’s degree from a respected institution, need to think about a question, it’s probably too much to ask of a sixth grader.

Grade 6, Book One:

“Ring of Horses” by Cindy Seiffert–non-fiction article about carousels

SAMPLE question: Which phrase in the last paragraph of the article best supports the author’s claim that preserving carousels has value?

A “nearly disappeared” (line 44)

B “began to realize” (line 44 and 45)

C “keeping the music” (line 45)

D “back in service” (line 48)

(The questions and lines in parenthesis refer to the paragraph below. Every fifth line is numbered in the test; thus I have repeated the line breaks as they appear on the test so you can count up or down from #45 to find the correct line numbers; thus this paragraph represents 5 lines, from 44 to 48:

These beautiful machines had nearly disappeared when, in the

1970s,  people began to [end of line 44]

45      realize the importance of keeping the magic of the carousel alive

for  future generations.

Enthusiasts formed the National Carousel Association and the

American Carousel Society [end of line 46]

to raise money, restore, and preserve the wooden carousels.

Thanks to their efforts, today [end of line 47]

about 150 antique carousels are back in service [end of line 48]

“Terra-Cotta Soldiers” by Corinne Bobb-Somers–non-fiction…about the necropolis built by an ancient Chinese ruler

“Pick-Up Your Shovel; Grow A Better City” by Ron Finely–non-fiction…on urban gardening

“The Pit Ponies” by Leslie Norris–poem

SAMPLE question: Why are the ponies scared in line 3?

A They are unable to see where they are headed.

B They are unfamiliar with the world above ground.

C They are uncomfortable being around other ponies.

D They are unsure why they are no longer in the mine.

These questions refer to the first stanza of the poem:

They come like ghosts of horses, shyly,

To this summer field, this fresh green,

Which scares them.

“The Sea Turtle’s Built-In Compass” by Sudipta Bardhan-non-fiction… Sea Turtles and how their unique GPS systems help them navigate

“Planes Saving Cranes” by Peg Lopata–non-fiction. On saving the endangered whooping crane…

SAMPLE Question: Read Lines 45 through 47

However, long-term survival could not be assured with only one flock of birds. another flock was introduced in Florida, but this group never learned to migrate.

What role do these sentences play in the article?

A They show why teaching the birds to migrate is necessary,

B They explain the limited vision of early conservation programs.

C They show how diminishing habitats affect the number of birds.

D They explain why the migration program has been so successful.

Grade 6, Book 2

“Stitches in Time” by Philip Gulley–non-fiction….A woman and her sewing machine

“Windblown” by Britt Norlander–non-fiction…on the increasing frequency of dust storms and their global impact

“The National Anthem” by Patricia Ryon Quiri–non-fiction…the anthem in context

SAMPLE Question: How does the article change the reader’s understanding of the first stanza of “The Star-Spangled Banner”? Provide specific examples of the relationship between the article and the first stanza. Use details from both the first stanza of “The Star-Spangled Banner” and the article to support your response.

In your response, be sure to

–explain how the article changes the understanding of the first stanza of “The Star-Spangled Banner”

–provide specific examples of the relationship between the article and the first stanza

–use details from both the first stanza of “The Star-Spangled Banner” and the article to support your sponse

(NOTE: Quiri’s article provides the historical context for the anthem, including the War of 1812, the burning of Washington, D.C. and the attack on Fort McHenry.

A number of inner-city teachers singled out this question as one with which their students would have difficulty. While acknowledging that kids had probably heard the anthem at some point, perhaps during the broadcast of a ball game, they were completely unfamiliar with the text and its complex 19th century language.)

Grade 7, Book 1

“Coyotes on the Move”–non-fiction…why coyotes are proliferating and moving into urban areas

SAMPLE Question: Read Line 57.

Do not tolerate coyotes that enter your yard.

Which definition best fits “tolerate” as it is used in this sentence?

A encourage by giving food to

B allow the continued presence of

C challenge the aggressive behavior of

D frighten with sudden movements toward

(Note: That sentence is immediately followed by: “Scare them away by yelling, waving your arms, or banging on pots and pans.” Two lines earlier the article says: “DO NOT feed coyotes. Keep all pet food and water inside at night, and secure your garbage cans.)

“The Girl Who Threw Butterflies” by Mick Cochrane–fiction

“Asteroids, Meteoroids, Comets” by Kenenth C. David–non-fiction

Excerpt from “The Car” by Gary Paulsen–fiction

“Jo’s Boys” by Louisa May Alcott–fiction

SAMPLE Question: Read Lines 48 and 49.

“But it’s so much nicer to sweep about in crowns and velvet trains than to wear every-day clothes, and just be myself, though its so easy.”

Which line best expresses how Miss Cameron would respond to Josie’s statement?

A “If you feel this, I can give you no better advice than to go on loving and studying our great master…” (Lines 12 and 13)

B “…a single talent makes a very imperfect character.” (Lines 29 and 30)

C “Now and then genius carries all before it, but not often.” (Lines 33and 34)

D “…mere beauty and rich costumes do not make an actress…” (Line 40)

“On the Roof of the World” by Benjamin Koch–nonfiction, travelogue about Tibet

Grade 7, Book 2

“Telling Plastic to ‘Bag It’” by Patricia Smith with reporting by

WilliamYardley of The New York Times

SAMPLE Question: In lines 52 through 58 of the article, the author explains events surrounding a citywide vote. Based on this information, readers can infer that

A Seattle voters were familiar with high fees

B Economic concerns overrode concerns for the environment

C Seattle city officials knew what the people really wanted

D leaders in the plastics industry were helpful in explaining a complex issue

“Vinnie Ream” by Phillip Hoose—non-fiction

“Our Expedition” by Shaun Tan—non-fiction

Grade 8, Book 1

“What Do Flies Think About?” From Discoveries & Ideas Magazine

SAMPLE Question: the author compares flies to fighter pilots in lines 10 and 11 to show that flies are

A Complicated

B forceful

C skillful

D mysterious

(Note: these are the two sentences that fall wholly or partially within lines 10 and 11– “For a long time, scientists believed the flies turn around in flight much like a fighter pilot performs loops. This would require them to first ‘visualize’ a mental rotation–in other words, to plan the loop beforehand.”)

“Edgar, The Falconer’s Son” by Laura Amy Schlitz—poem

“A Bigfoot by Any Other Name” by Kelly Milner Halls, Rick Spears, and Roxanne Young—non-fiction

SAMPLE Question: How do lines 1 through 3 help to develop a key concept of the article?

A It emphasizes that many different creatures have been confused with Bigfoot

B It illustrates that people all over the world have believed Bigfoot exists

C It shows that Bigfoot has travelled all over the world.

D It confirms that Bigfoot has remained unidentified.

Lines 1 through 3: Bigfoot. Sasquatch. Yeti. Yeren. Yowie. The names and the details may differ from place to place, but from North America to China to New Zealand, one thing is certain: Something is out there.

“Paying a Stiff Tab to Rescue Hikers” by Karin Klein—non-fiction

“Cleaning Up” by Mark David Whitehead–fiction

“Soccer…For Robots?” By Marcia Abidon Lust—non-fiction

Grade 8, Book 2

“Cowgirl Morning” by Bryn Fleming–fiction“A Planet Without Apes?” By John C. Mitani–non-fiction

“The Calf That Sank the Ship” by Sonja M. Oetzel—non-fiction

SAMPLE Question: The author of this article provides a unique perspective on the “greatest sea disaster of all time.” explain how the unique perspective contributes to the understanding of the events surrounding the disaster. Use details from the article to support your response.

 

In your response be sure to

–explain the unique perspective in the article

–explain how this unique perspective contributes to the understanding of the events surrounding the disaster

–use details from the article to support your response

Posted in Education | Tagged , , , , , , , , , , , , , | 8 Comments

A Demographic Divide In Harlem: The Neediest Kids Go to Public Schools, Not Charters

Last month I published an OpEd in The New York Times, “Charter School Refugees,” which asked: “Is there a point at which fostering charter schools undermines traditional public schools and the children they serve?”

The OpEd looked at Harlem, where nearly a quarter of students are enrolled in charter schools, and the sizable demographic disparities between the students who attend  public schools and charter schools in that neighborhood. I argued that while “high-quality charters can be very effective at improving test scores and graduation rates…they often serve fewer poorer students and children with special needs.”

The OpEd focused on the reasons why “public open-enrollment elementary and middle schools have double, and several have triple, the proportion of special-needs kids of nearby charter schools.”

With the help of my research assistant Emma Kazaryan, I have now compiled the 2012/2013 data, published by the New York City Department of Education,** on each elementary and middle school in East Harlem into easy-to-read charts. (The city’s school map divides the neighborhood into North and South. So, we have done the same, showing the percentages of kids in poverty, with special needs and English language learners with separate charters for East Harlem North and East Harlem South.)

The data shows that the demographic disparities cut across the board. East Harlem public schools not only have disproportionate numbers of special needs kids compared to nearby charter schools…

EastHarlemNorthIEP%

EastHarlemSouthIEP%

…the kids in public schools are much poorer than those in neighboring charter schools:

EastHarlemSouthEconomicNeedIndexEastHarlemNorthEconomicNeedIndexAnd East Harlem public schools have much higher percentages of English language learners than do their charter-school counterparts:

EastHarlemSouthELL%EastHarlemNorthELL%

Most of the traditional public schools with relatively low levels of poverty and special needs, including Tag Young Scholars and Manhattan East School for Arts and Academics in East Harlem South, are selective schools. Meanwhile, the disproportionate levels of poverty and special needs, as well as the high percentage of English language learners, at most East Harlem public schools are undoubtedly influenced by the landscape in neighboring Central Harlem where more than half the schools are charters.

My Oped was written in response to new New York State legislation, spearheaded by Gov. Cuomo, that virtually guarantees charter schools in New York City access to space, either in already crowded public school buildings or in rented spaces largely paid for by the city. I concluded: “If charter schools are allowed to push out existing public schools, they should, at the very least, be subject to the same accountability measures for enrollment, attrition and disciplinary procedures, to ensure that the neediest students are being treated fairly…We should not allow policy makers to enshrine a two-tier system in which the neediest children are left behind.”

 

** To see the Progress Report data for each school, click on the area of the map you are interested in–for example, upper Manhattan. Then click on the neighborhood. Then click on the school and select “statistics.”

 

Posted in Education | Tagged , , , , , , , , , , , , , , , , | 8 Comments

Charter School Refugees–A New York Times OpEd

Late last month, the New York State Legislature struck a deal ensuring that charter schools in New York City would have access to space, either in already crowded public school buildings or in rented spaces largely paid for by the city. Over the next few years, charters are expected to serve an increasing proportion of city students. Which brings up the question: Is there a point at which fostering charter schools undermines traditional public schools and the children they serve?

In this OpEd for the New York Times, I explore the experience of Harlem, where nearly a quarter of students are enrolled in charter schools and where here is a marked disparity between the special-needs populations in charter and traditional public schools, according to the city education department’s annual progress reports. In East Harlem, data for the 2012-13 school year shows that most of the public open-enrollment elementary and middle schools have double, and several have triple, the proportion of special-needs kids of nearby charter schools. The charter schools also often serve fewer poorer students.

My OpEd argues that if charter schools are allowed to push out existing public schools, they should, at the very least, be subject to the same accountability measures for enrollment, attrition and disciplinary procedures, to ensure that the neediest students are being treated fairly. We should not allow policy makers to enshrine a two-tier system in which the neediest children are left behind.

Posted in Uncategorized | 1 Comment