Five years ago, I found myself drafted onto a New York State Department of Education committee charged with revamping the English Language Arts standards. As a journalism professor at Baruch College/CUNY, I had the non-fiction expertise that were seen as so important to developing the new standards. Although I had no experience in public schools, I was on the receiving end of the abilities and deficits of kids graduating from the city’s high schools.
As a journalist who had spent much of my career writing about business and management issues, I also had become intrigued with the corporate education-reform movement; I couldn’t resist a chance to participate, even in a small way, in the sausage making of public education policy.
The work of our committee would take close to two years to complete, with multiple trips to Albany, at a cost that must have run to several hundred-thousand dollars—if not a few million. We debated how to balance literature and non-fiction, how to accommodate the needs of immigrant kids who are struggling to learn English and how to account for the radically different cultural experiences of the students who would be taking the tests–kids in remote rural areas, in suburbs, in New York City.
The committee’s work was eventually jettisoned when New York State decided to adopt the Common Core State Standards.
So, when an educator recently handed me an illicit copy of the controversial “common-core”-based New York State standardized tests, I was eager to see what the fuss was about.
The tests, which were developed by Pearson and administered to students in April, were so poorly received by both educators and parents, that veteran New York City principals mounted a grassroots campaign in opposition to what they say are “unfair” tests that take an“intolerable toll” on children. Now as New York City gets ready to develop even more tests in every conceivable subject—in response to the new teacher-evaluation system imposed this weekend by New York State, which calls for 20 percent of a teacher’s ratings in all subjects to be based on student test scores–this is an opportune time to reflect on the considerable flaws, and costs, of these most recent standardized tests.
I received full copies of each of the sixth, seventh and eighth grade tests (minus one eighth-grade booklet.) Grades 3 to 5 were also tested, though I did not see copies of these tests.
New York State is part of PARCC, a consortium that is developing a common core assessment. But it is one of the few states that have rushed to implement a common-core based test before PARCC itself is ready to roll out its own assessment in the 2014/2015 school year.
The most fundamental problem with the tests is that they were administered before teachers have had any meaningful training in the new standards, and before students have had much exposure to them; schools, kids and educators, in short, are being set up for failure.
A close reading of the tests given to grades 6 to 8 raised many concerns regarding their contents. I was especially surprised to see not only how heavily the tests were skewed to non-fiction, but also the nature of those non-fiction readings, which were dominated by scienc(y) writings, with very few readings that drew on civics, American history or the urban experience.
Others have commented on the grueling length of the tests. Diane Ravitch who saw a bootlegged copy of the 5th grade tests noted: “Based on test questions I had reviewed for seven years when I was a member of the NAEP board, it seemed to me that the test was pitched at an eighth grade level. The passages were very long, about twice as long as a typical passage on NAEP for eighth grade. The questions involved interpretation, inference, and required re-reading of the passage for each question.”
Each of the tests for grades 6, 7 and 8 are completed in 90-minute segments over the course of three days. The seventh grade test, for example, is about 72 pages long (there are a few blank pages added for essay questions.) It includes 14 passages, the vast majority of which are one-to-two pages in length. There were also eight short-answer questions that require writing about one long paragraph each, as well as two essay questions. Then there were the endless multiple choice questions—over 100 of them, far more than the number on earlier test, according to education experts. (More on this later.)
Taken together, the 6 to 8 grade tests are weighted two-to-one in favor of non-fiction, far more than even the common core standards require for these grade levels. The common core calls for a 45/55 fiction-to-non-fiction ratio in the eighth grade. David Coleman and Susan Pimentel, lead authors of the common core, argue that this will not work against the teaching of literature because “the bulk” of the responsibility for nonfiction reading “will be carried by non-ELA disciplines” such as science and social studies. “Said plainly, stories, drama, poetry, and other literature account for the majority of reading that students will do…”
Even if you leave aside the small detail that only ELA teachers will be judged, VAMed and, perhaps, fired for poor performance based on the assessment, there is very little fiction or poetry in the NYS test.
As I read through the tests, I also was reminded of Jackie Pryce-Harvey, one of the few New York City public school teachers who participated on the New York State ELA standards committee. A 50-something year old Jamaican-born educator, Jackie had spent most of her teaching career in the Bronx and Harlem. Once or twice a day, during those long sessions in Albany, Jackie, who was often the only African-American in the room, would explain why this or that idea wouldn’t work for inner city kids of color, or why certain readings—ones about happy families going hiking were among her favorites– do not make sense to many of her students. Jackie is now the assistant principal at Global Technology Preparatory, a middle school she helped found in Harlem. During test prep sessions at the school, she invariably tells her kids, only half joking: “You have to remember, white people are strange: They think hiking and camping are fun.”
About the same time, I was also reading Unafraid of the Dark, a wonderful memoir by Rosemary Bray, a reverend with the Fourth Universalist Society in New York City, who as a poor, African-American scholarship student attended Francis W. Parker, the elite private high school in Chicago that is my alma mater. In one passage Bray recalls being invited to a friend’s house after school. Upon arriving at the address, a four-story building, she became confused because she could not find a panel listing the tenants and apartment numbers. Eventually, her friend saw her outside and let her in; the friend lived in a single-family home, something Bray had not even considered a possibility.
You don’t have to be “culturally sensitive” to notice that the content of the New York State tests seem designed for non-urban kids. Only one of 14 passages in the seventh grade test, “The Harlem Renaissance,” by Lisa Beringer McKissak, deals with an explicitly urban subject. By contrast, one of the few fiction texts in the eighth grade test is a short-story, “An Uncomfortable Bed,” by Guy de Maupassant, which takes place in a French chateau. The rest—fiction and non-fiction alike—read like they had been culled from Field and Stream or Rock and Ice.
There are passages about long hikes, horse-back riding, travel via air-balloon and ship. There are stories for young naturalists—on meteorites, aquatic nurseries, octopuses and jelly fish. There are also readings about adventures in remote parts of the world, such as the hunt for gold in the Klondike region of Canada.
But there is little that speaks to the experience of kids who live in New York City and other urban areas—nothing about museum-going or hip-hop culture, riding subways or buses, living in apartment buildings. Of course, city kids should be exposed to readings about nature and the outdoors, just as rural kids should be expected to read and understand texts about the urban experience.
Interestingly, one of the test’s only passages on American history is the essay “Eight Hours and Nothing Less,” by Samuel Gompers. I wonder whether Pearson recognized the irony of including a text by one of the founding fathers of the American labor movement at a time when the education reform movement, including new teachers evaluations, has become almost synonymous with union busting.
As a journalist who has spent my professional life writing non-fiction, I confess that I am puzzled by the non-fiction mania reflected in the common core and in New York State’s test. I believe fervently that every high-school student should be able to read, and understand, The New York Times. For young people, the newspaper is not just a window on the world, it is also a way to learn about civics. (That is why my infamous news quizzes invariably include questions about the Supreme Court, important legislation and the legislative process, areas about which many of my students know little.)
Yet, this (over)emphasis on non-fiction strikes me as excessively utilitarian and, in the end, counterproductive. Here it is important to remember that the common core assessments are meant to test ELA, not history or civics or science. Moreover, I’ve found that kids who have read Fitzgerald, Twain or Doctorow, are the ones who are most likely to read, and understand, non-fiction texts like The New York Times.Yet, the emphasis on non-fiction in both the standards—and the assessments—already has put pressure on ELA teachers to deemphasize literature; in other words, less Fitzgerald, Twain and Doctorow…
Then, too, there is the question of how the new assessments are scored. Bill Heller, an expert in data and assessment at Teaching Matters, writes about the importance of having a reliable norming process. “Norming is the process of calibrating the use of a single set of scoring criteria among multiple scorers,” Heller writes. “If norming is successful, a particular piece of work should receive the same score regardless of who is scoring it.”
Heller describes how a good norming process works here.
The problem is that New York State assessments have not typically developed a common norming process, in part, perhaps, becuase doing so is expensive. Heller writes: “Different sections of the state have different norming procedures, which means the state as a whole has none. I’ve talked with many New York City teachers who have scored the exam, and they report that there was very little effort to norm. Different scorers had wildly different standards for interpreting the rubric, and even the same scorer could become more lenient as the days went on. The final scores, then, were as much of a function of geography, timing, and luck as they were of student performance. How can we possibly make use of this data to reliably identify student learning problems, let alone make high-stakes decisions about school, teacher, or student performance?”
In New York City, the April test will be scored by New York City teachers who are pulled out of their classrooms for that purpose (one of the little discussed costs of high-stakes testing.) Presumably the same is true in other parts of the state. “Based on past experience with the New York State tests, I fear there will not be a rigorous state-wide norming process in place for this year’s exam,” says Heller.
It is hard to know anything for sure, though, as everything about the test is cloaked in secrecy. Each test comes with a bold warning against reproducing or transmitting the test in any form. As Adam Urbanski, president of the Rochester Teachers Association puts it: “If these tests are never made public there is absolutely no way to hold the state accountable.”
It is easy to point the finger at Pearson, which has a five-year, $32 million contract to create tests for the state. The costs of the tests, though, are just the tip of the iceberg. There is also the cost of scoring them–especially if that process is done well. Then there are the opportunity costs–the time spent on test-taking instead of learning, the costs to schools that have to hire substitutes so their teachers can score tests.
This brings us back to all those multiple choice questions on the latest New York State assessments. One of the factors that is supposed to set the Common Core apart is that they are aimed at developing “authentic” learning experiences that help connect what kids are learning to the real world. For example, at West Side Collaborative, teachers developed an interactive assessment based on the Pacific Trash Vortex, a Texas-sized mass of garbage in the North Pacific; drawing on a range of documents that students need to study, including news stories, lab reports and maps of ocean currents, students are asked to propose solutions to the garbage problem.
On Sunday, Shael Polakow-Suransky, chief academic officer of the city’s Department of Education, told The New York Times, that he wants new tests that will challenge students to think critically and creatively. “We don’t want to just invent hundreds of new bubble tests in order to satisfy these requirements for teacher evaluation,” said Polakow-Suransky.
The problem is this: Developing better tests that assess “authentic” work, creativity and critical thinking will be complicated and expensive. Multiple choice questions, like the ones in such abundance on the latest Pearson test, provide a minimalist view of what children understand and do little to foster critical thinking skills; but they are cheap. As Polakow-Suransky once told me, when it comes to tests “you get what you pay for.”
As more subject tests and common core assessments are rolled out in the coming years, the costs of developing and administering those tests will soar. Without substantial increases in funding, it is unlikely that testing organizations like Pearson and PARCC will be able to resolve the tension between the demand for “authentic” learning experiences and the constant push for for ever more high-stakes standardized tests.