Wednesday, April 29, 2009

More Musings about NAEP

Guest post by Ze'ev Wurman

Yesterday’s release of the long-term NAEP results has been already written about here, here, and here. For 9-year-olds and 13-year-olds the news is good. Significant increases since 1999, and particularly since 2004, in both reading in math. Most gratifying is the jump in reading achievement for 13- and 17-year-olds -- age groups that have been resistant to change for almost three decades.

Another piece of good news is the fact that much of the growth, in both reading and math, was concentrated in the lower achieving students, at the 50th percentile and below. The ethnic achievement gap was either reduced in the early years or did not change because all groups kept advancing—a rising tide lifts all boats. And some of the reduction has been nothing but spectacular, as noted by Sandy Kress in his comments on Bill McKenzie's Dallas Morning News blog post. So far so good. Enhanced accountability -- whether through the standards movement of the 1990s or through NCLB in the last decade -- is finally paying off.

Much journalistic coverage has concentrated on the fact that our 17-year-olds are still essentially stuck in place, barely returning to their achievement level during the 1970s. Other coverage has pointed out that despite students taking more impressive-sounding courses, perhaps the reality is grade inflation or course-name inflation. But there is a larger question we should ask ourselves. Why should we even expect NAEP results to rise much? After all, if our education level was rather high in early 1970s when NAEP started, why should it get much better? For the 17-year-olds, there was an achievement dip in the late 1970s and early 1980s, so clearly we should be able to reclaim the 1971-73 level. But we have achieved this level in the 2008 results. Why should we expect to do notably better in the future?

One could answer that since we are doing better with our 9- and 13-year-olds, we can also expect more of 17- year-olds. Yet one could argue that the increases in the earlier grades have come not because they know more in absolute terms, but simply because now we teach material earlier, material that 30 years ago was delayed to later grades. Think of it—we push algebra to earlier grades, we expose kids earlier to a variety of cultures and facts through TV and the Internet, so they broaden their vocabulary earlier—it may be only natural that their skills look better at a fixed age. But the question still is how easily we can educate our HS graduates to a significantly higher level than in the past. Could it be that the immovability of achievement for out 17-year-olds represents some cultural constant or barrier that is hard to break, and not simply the failure of high school? It would be interesting to compare how other nations have fared with their 17-year-olds over time, but the data is hard to come by. For example, the last year of high school was sampled by the 1995 TIMSS, and it found that the best countries achieved only about 50 points above the international average, while the best countries' elementary and middle school students achieved 100-130 points over the international average. That would be consistent with the existence of some 'natural' constant that is hard to break. Makes me wonder.

[Ze'ev Wurman was a senior policy adviser in the U.S. Department of Education's Office of Planning, Evaluation, and Policy Development, 2007-2009. He also serves on the panel that reviews mathematics test-items for the California standards-based tests, and he was a member of the California State mathematics curriculum-framework committee.]


  1. I am trying to find some consistency between this article and the fact, both you and Mr. Evers are founding members of Palo Alto's HOLD and DOE analysts. Especially, since the DOE is mostly responsible for labeling Everyday Math as 'exemplary?' Along with a dozen or so reform math programs - including UCMP (Isaacs), Connected Math (Lappen), Core Plus (Hirsch) and Math in Context (Romberg). Especially, when one considers SRI's role in this enterprise. It seems this will require considerable explanation when one explores how the US public school system deteriorated over a space of about twenty years. We have high school drop out rates currently near 40%. Future progress reports look very grim.

  2. That rating of Everyday Math as 'exemplary' occurred a decade ago, during Bill Clinton era, long before either Bill Evers or I set a foot in the U.S. Department of Education. In fact, that outrageous labeling elicited a strong reaction from over 200 mathematicians and physicists on the pages of the Washington Post, condemning that designation. See for more details.

  3. Yes, Richard Riley was then Sec of Education and he started the eighth grade voluntary testing by the NAEP. But then why hasn't the DOE removed the 'exemplary' status - I'm sure you are both aware that it affects how the NSF-EHR evaluates curriculum and 'who' evaluates it. Who is preventing the funding for a 'modernized' curriculum? The Governor's Roundtable? I mean seriously, I try not to mention Singapore, but it is the best curriculum written in English and its available, except schools are prevented from buying it. Singapore provides standards, assessment, and a textbook. And best of all, parents, teachers, and students are satisfied. The Singapore MOE is held in high respect.

    Call me skeptical, but does school reform success depends on 'exemplary' curriculum failing students?

  4. The Singapore series was adopted by the California Dept of Education last year. Schools are not prevented from buying or using it in California, at least not by the Calif Dept of Education. Is someone else stopping them?

    Regarding the "exemplary" label for Everyday Math and other atrocities, it isn't as if people haven't asked Dept of Education to remove that label.

  5. I thought you were correct, so I looked it up and I did not find Singapore. In addition, I thought it was only adopted for grades 1 - 8.

    November, 2007 40 programs selected out of 54 programs submitted for review. This was off the Sonoma County website. The list included Everyday Math which I thought was dismissed when California adopted new content standards in 1997. And only last month, I learned LAUSD kept with the old standards and one can easily see now what a terrible mistake that was.

  6. Is Keys School (Palo Alto) still using Singapore? I made an inquiry at my district and was told that Singapore was not available. But then I inquired at San Diego City School (Learning Choices Academy) and they said yes that it was available for students. I have not verified it - but it sounds like this has something to do with school district adoptions? In addition, if LAUSD (600,000 students) for over 20 years has failed to show any widespread school improvement then you would think that school reform just might be a failure.

    The laws in some states regarding textbook selection seem extremely vague - Texas adopted all 162 submitted elementary math textbooks - school board members there have two choices - conforming and non-conforming. The only textbook that was not rejected was 3rd grade Everyday math and McGraw sued. They cited a Dallas study (Malone, 2001) that showed inner city students did better on the TEKS using Everyday.

    I researched the DISD claims and found considerable dissent even amongst board members.

    "Submitted by IS, Jan 21, 2008 15:06

    I am a member of our local school board and after adoption Everyday Math our scores plummeted from many years of near 100% passing. Biggest mistake I have made as a board member. I began my objections mid year of the first year after adopting it. The spiral learning that is purported only leads to a vortex of confusion. I believe that our district owes an applogy to our students for allowing this confusing and defecient program to infest our school.

    Read the other comments. Everyone that is educated in the sciences deplores this program. It should also be known and spread about that the program originated at the University of Chicago Department of Education not in the Math Department. A short time after the program was developed the Board of Regents dissolved the Department of Education at the University of Chicago. In oter words they FIRED everyone associoated with this program. It is and was too much of a cash cow for the university to get rid of.

    Although they say they do most of the text book folks don't give a flying flip about the kids. IT'S ALL ABOUT THE MONEY."

    This was a response to an editorial by Jesse Arnett in which he writes - " DISD has improved the math test scores of 3rd graders from 58% passing in 2000 to 71% passing in 2007. This is a 22% increase in passing students."

    This is the same wag:
    "Jan 14, 2008 19:10

    JG'a response is typical when a disgruntled parent is faced with the facts of success. They always resort to name-calling or add a personal story of failure. Nevertheless, facts do not lie, and statistics help tell the story. The results were outstanding.

    And BTW... Dallas was using old style textbooks with the same teaching techniques used in the 60's before Everyday Math was adopted."

    And I should credit Carson, NYC Hold in correctly identifying the poster JA.

    "To previous poster Jesse Arnett

    In regard to your association with Everyday Math

    It appears that one Melissa Arnett is a TX representative for the publisher of Everyday Math !"


    I'm confident that Everyday Math doesn't count, so why is it still being tossed around school districts like an unwanted child.

  7. Great subject. I have been playing around with the idea of the comment structure recently.
    I think now Internet user are increasing day by day so company are preferring online marketing now.
    I think that Marketing Online is still the best choice, cause we have high ROI´s with less investments, especially with SEO.
    internsive language

  8. I think its cool that you blog about this. Thanks!

  9. Excellent post! Thanks a lot for sharing.