Some Thoughts about MCAS and NCLB

This past week, there was an interesting discussion at the Southborough School Committee meeting about the MCAS scores from this past spring, including the fact that we are no longer able to use these scores effectively to analyze our students’ performance because the state is no longer releasing most of the test questions, thus making the tests much less useful for school improvement efforts. There are many other things about these scores as well, though, that I’m not sure everyone realizes, and I want to mention some of the things that I see, for others to think about and respond to. For the most part, educators’ voices have not often been heard in the conversation about standardized testing – the groups discussing, initiating, and designing the testing systems have included very few K-12 educators, and comments from educators are sometimes dismissed by the media as defensive and self-serving, so some of the questions and criticisms of the tests are not well known. To me, there are many aspects of both MCAS and NCLB (the federal law, titled “No Child Left Behind”) that are a bit like the story, “The Emperor’s New Clothes,” and I have kept waiting for the media to dig behind the press releases and official statements to understand the real effects of these programs. Unfortunately, that hasn’t happened yet, so I thought that I would share some things that I wonder about as I look at education policy these days.

•• In last year’s NAEP testing (NAEP stands for “National Assessment of Educational Progress” and is a test that has been widely regarded as a valid national assessment since 1990 (http://nationsreportcard.gov/)), Massachusetts 8^th graders led the nation in mathematics achievement, and Massachusetts 4^th graders tied with two other states for first place. (See the October 14^th, 2009, press release from the Massachusetts Department of Elementary and Secondary Education – www.doe.mass.edu.) But in the same year, on the MCAS, 80% of Massachusetts middle schools were regarded as failing to make “adequate yearly progress” for at least one subgroup under NCLB (see data on the DESE website).

•• In 2007, Massachusetts results on the TIMSS (Trends in International Mathematics and Science Study) testing were separated out as if Massachusetts were its own nation. The results, according to the Massachusetts Department of Education (http://www.doe.mass.edu/news/news.asp?id=4457), were that:

“According to the results of the 2007 Trends in International Mathematics and Science Study (TIMSS), Massachusetts 4th graders ranked second worldwide in science achievement and tied for third in mathematics; the state's 8th graders tied for first in science and ranked sixth in mathematics.”

What’s interesting to me about this is that this is the Massachusetts average score for 4^th graders and for 8^th graders – when you look at the MCAS results, Southborough and other similarly situated communities score much higher than the state averages, so our students must in fact be scoring with the best in the world. Yet MCAS and NCLB and the media slant on those results would have us believe that our students are not achieving as well as they should.

•• A recent Boston Globe article reported on the 2009 NAEP results with a concerned headline indicating that NAEP results have not increased significantly since the beginning of “education reform” efforts, and implying that the NAEP testing results should have shown an increase. Aside from the possibility that adding testing may not be the best method to improve our children’s education, a question I have about this is the following: how do we “know” that students can in fact achieve at higher levels than they are currently reaching? If, for example, we compare this year’s 4^th graders to last year’s 4^th graders, which is what we are doing, and we expect that the scores should be continually rising, the assumption behind that expectation is that 4^th graders are capable of achieving much more than they are currently achieving and that schools are doing a lousy job. But how do we know that and how much do we think they can achieve? 4^th graders are generally nine years old. Probably everyone would agree that we would not be able to achieve a goal of having most nine-year-olds understand calculus, and probably everyone would also agree that most nine-year-olds should know their basic math facts. But where in between these two extremes is the appropriately challenging standard that is reasonable to expect of nine-year-olds? And, when our nine-year-olds lead the nation in math achievement and are 6^th in the world among nations, how is it that we are deciding that developmentally they can do more, particularly in our highest-achieving communities?

•• NCLB defines “adequate yearly progress” as increasing test scores in equal increments, from a 2002 baseline, until 2014, in which year all students are to be proficient or advanced. Everyone who really thinks about that regards it as an impossible goal – we have many students with severe disabilities for whom proficient or advanced scores on a paper-and-pencil standardized test are simply beyond their capabilities – yet politicians, government bureaucrats, and the media continue to speak as if this is possible, the law applies sanctions to schools that fall behind, and more and more schools are regarded as being in need of improvement. On the one hand, this doesn’t seem like a serious problem, if everyone realizes that it’s an impossible goal (and many legislators will admit this to you in private) – BUT – the negativity about schools and educators that their public discourse reflects is damaging to public trust in schools, to parents’ trust in educators, and ultimately to educators’ ability to do their best work with kids.

•• NCLB also defines “adequate yearly progress” as meaning that each subgroup – including low-income students, different ethnic groups, “limited English proficient” students, and students with disabilities – makes sufficient progress so that each group will achieve proficiency by 2014. This standard is one thing as applied to low-income students and members of defined ethnic groups, but completely another thing as applied to English language learners and students with disabilities. “Limited English proficient” students, by definition, are students who are not yet proficient in English. As soon as students become proficient in English and no longer need special services, they are removed from the group, and new students, who perhaps have just come to this country and don’t yet speak English, are added to the group. So what the law requires is continuous progress, to a goal of complete proficiency by 2014, in a group whose members move out of the group and are no longer counted in it as soon as they make progress and which is continuously adding new members as new students who do not yet speak English move in. The subgroup of students with disabilities is similar in some ways. This group is made up of students with disabilities who are not able to “make effective progress” independently because of their disabilities. As previously noted, some of them may simply not be able to achieve proficiency as defined by NCLB because of their disabilities. Others do make progress – BUT – once they make enough progress that they can “make effective progress” independently without services, they are removed from the group. So the group, as defined, is students who are not able to independently “make effective progress,” yet NCLB requires continuous progress toward complete proficiency for this group. Most schools that do not make AYP fail to make AYP because of the performance of one or both of these subgroups, which by definition are students whose progress is lower than others.

•• With respect to the MCAS itself, there has often been much concern expressed about student performance in grades 3-8, in particular criticism of student achievement in middle schools, along with praise for the higher achievement of 10^th grade students across the state. When I look at this in more detail, however, one of the things I wonder about is the threshold scores set by the state for the various levels of performance. In 2009, on the MCAS math test, the 7^th grade “cut scores” were 93% for “advanced,” 74% for “proficient,” and 48% for “needs improvement,” while the 10^th grade “cut scores” were much lower, at 67% for “advanced,” 45% for “proficient,” and 27% for “needs improvement.” (The threshold scores for grades 3, 4, 5, 6, and 8 are similar to the 7^th grade.) I understand that the tests are different, and there are complicated explanations of how they calculate the threshold scores, but I do wonder about this discrepancy. . .

I have recently finished reading an excellent book entitled Catching Up or Leading the Way: American Education in the Age of Globalization, by Yong Zhao, a professor at Michigan State University, analyzing and comparing the American education system and the Chinese education system, and explaining the reasons that China is working to make its education system and philosophy more like the American system was in the past, before the recent push toward standardized testing. Some of the points he makes, unfortunately little noted in the American media, are the following:

> In a 2008 study by the National Science Foundation, U.S. students scored highest among nations on “science literacy,” or understanding scientific concepts (as opposed to standardized test-taking). Zhao notes, “Chinese students may test well in science, but their science literacy remains low.” (Zhao, p. 84)

> Education “that is oriented solely to preparing students to achieve high scores on tests can be harmful to both individuals and the nation it is supposed to serve. Professors Sharon Nichols and David Berliner (2007) documented the potential damage of high-stakes testing to public education in the United States in their book Collateral Damage: How High-Stakes Testing Corrupts America’s Public Schools. The damage. . . in China is much broader and has been proven because the whole education system is test oriented and all that matters is to prepare test takers. . .” (Zhao, p. 85) [Note: the Nichols and Berliner book is also an excellent analysis of the damage being done by the current standardized testing efforts in the United States.]

> Zhao also documents rates of student suicide, obesity and other health issues in China, as well as the “creativity gap” resulting from the standardized testing system. (Zhao, pp.85-95)

Zhao continues with an analysis of the skills and knowledge that our children will need for the future, concluding that:

“. . . schools should offer a comprehensive, balanced curriculum that includes opportunities for students to explore and develop both R-directed thinking and L-directed thinking; to learn math, science, technology, history, economics, geography, government, reading, literature, music, foreign languages, and art; to develop global awareness and appreciation for differences; to develop understanding of and ability to interact within the digital virtual world; and to develop a healthy body and mind.” (Zhao, p. 155)

The following are excerpts from his concluding chapter, entitled “Catching Up or Keeping the Lead: The Future of American Education”:

“I have used the previous chapters to show how the current reform efforts [in the U.S.] are the result of a history of flawed reasoning based on incomplete information, driven by unfounded fear, and influenced by politics. . .”

He explains how China is working to move away from a test-driven system and expand its definition of success, and then continues: “How we define success for students, teachers, and schools determines how students, teachers, and schools are evaluated, and it directs their energy, efforts, and resources to what are considered indicators of success. This effect has already been seen in the U.S. reform efforts of the past several years. As a result of No Child Left Behind, many schools have narrowed their curriculum to focus only on what is tested – math and reading – and teachers have been pushed to teach to the test. . . expanding the definition of success should be one of the first changes we make in our efforts to ensure a bright future for our children. . .”

“Expanding the definition of success first means that we need to elevate the status of other subjects, abilities, skills, and talents to the same level as math and reading. . . Expanding the definition of success also means changing how we measure success. Many of the valuable skills, knowledge, abilities, attitudes, and perspectives one needs and schools cultivate do not have widely accepted standardized tests. Some of them may never be easily measured through standardized tests. Thus we need to adopt a broad range of indicators to assess student learning, including student products, teacher observations, classroom performances, and some psychological measures of student motivation, creativity, and perspectives that have not typically been part of mainstream educational assessment. . .”

“Expanding the definition of success of schools should not be limited to only outcome measures, because many factors affect student learning. . . It is simply misleading and wrong to use students’ performance on tests to judge the degree of a school’s success. A more just and useful way to judge the quality of schools is to assess the quality of input and hold schools accountable for providing the best educational environment for all students. . .”

In my view, we need to value and encourage our students’ abilities and skills in all areas, and not to regard or make a student feel like a failure if, for example, he has difficulty in reading but is a brilliant trumpet player (of course we should help him with his reading, but if we eliminate music to focus more on high achievement in tested subjects, we do this child serious harm), and we need to be careful about allowing the current mania for standardization and measurement to destroy the public education system that has resulted in our country leading the world in innovation. We also need to understand and appreciate the excellent job that most of our schools are doing with our students, and not undermine educators’ efforts and energy through the constant negativity and criticism in the media. I am still hopeful that journalists and politicians will eventually begin to investigate, look deeper into the many questions surrounding NCLB, and realize that not only is the emperor wearing no clothes, but that the standardized testing movement has the potential and is beginning to seriously damage our public schools.