More Musings about NAEP

Guest post by Ze'ev Wurman

Yesterday’s release of the long-term NAEP results has been already written about here, here, and here. For 9-year-olds and 13-year-olds the news is good. Significant increases since 1999, and particularly since 2004, in both reading in math. Most gratifying is the jump in reading achievement for 13- and 17-year-olds -- age groups that have been resistant to change for almost three decades.

Another piece of good news is the fact that much of the growth, in both reading and math, was concentrated in the lower achieving students, at the 50^th percentile and below. The ethnic achievement gap was either reduced in the early years or did not change because all groups kept advancing—a rising tide lifts all boats. And some of the reduction has been nothing but spectacular, as noted by Sandy Kress in his comments on Bill McKenzie's Dallas Morning News blog post. So far so good. Enhanced accountability -- whether through the standards movement of the 1990s or through NCLB in the last decade -- is finally paying off.

Much journalistic coverage has concentrated on the fact that our 17-year-olds are still essentially stuck in place, barely returning to their achievement level during the 1970s. Other coverage has pointed out that despite students taking more impressive-sounding courses, perhaps the reality is grade inflation or course-name inflation. But there is a larger question we should ask ourselves. Why should we even expect NAEP results to rise much? After all, if our education level was rather high in early 1970s when NAEP started, why should it get much better? For the 17-year-olds, there was an achievement dip in the late 1970s and early 1980s, so clearly we should be able to reclaim the 1971-73 level. But we have achieved this level in the 2008 results. Why should we expect to do notably better in the future?

One could answer that since we are doing better with our 9- and 13-year-olds, we can also expect more of 17- year-olds. Yet one could argue that the increases in the earlier grades have come not because they know more in absolute terms, but simply because now we teach material earlier, material that 30 years ago was delayed to later grades. Think of it—we push algebra to earlier grades, we expose kids earlier to a variety of cultures and facts through TV and the Internet, so they broaden their vocabulary earlier—it may be only natural that their skills look better at a fixed age. But the question still is how easily we can educate our HS graduates to a significantly higher level than in the past. Could it be that the immovability of achievement for out 17-year-olds represents some cultural constant or barrier that is hard to break, and not simply the failure of high school? It would be interesting to compare how other nations have fared with their 17-year-olds over time, but the data is hard to come by. For example, the last year of high school was sampled by the 1995 TIMSS, and it found that the best countries achieved only about 50 points above the international average, while the best countries' elementary and middle school students achieved 100-130 points over the international average. That would be consistent with the existence of some 'natural' constant that is hard to break. Makes me wonder.

[Ze'ev Wurman was a senior policy adviser in the U.S. Department of Education's Office of Planning, Evaluation, and Policy Development, 2007-2009. He also serves on the panel that reviews mathematics test-items for the California standards-based tests, and he was a member of the California State mathematics curriculum-framework committee.]

NY Times Panel Members Far Apart on Best Strategies for K-12 Progress

The U.S. Department of Education today (Apr. 28) released the 2008 results for the National Assessment of Educational Progress (NAEP) Long-Term Trend Reading and Mathematics Assessments. After the release, the New York Times asked a panel of individuals interested in education policy to suggest the best strategies for progress.

The Times picked people who they thought would have different views. The editors got what they hoped for.

Writer Sandra Tsing Loh contends that the presence of middle-class white and Asian children would lead low-performing students to succeed. Hence she calls for centrally-designed class and racial intermingling, if possible through magnet schools.

Howard "Multiple Intelligences" Gardner (Harvard psychologist) suggests "benign neglect" toward inter-group comparisons. Instead he proposes concentrating on minimum academic competency for every student.

NYU education professor Marcelo Suarez-Orozco says that we should "throw out" all high-stakes tests and instead "nurture interpersonal sensibilities." In particular, he promotes the wolly concept of 21st-century skills:

[W]e’ll need intellectually curious and cognitively flexible workers comfortable with ambiguity, able to synthesize knowledge within and across disciplines and work collaboratively in diverse groups.

I (Bill Evers) was one of the panelists. In my commentary, I said: that incentives matter for administrators, teachers, and students, and that solid academic content matters. I also point out that pluralism in delivery of K-12 education (charter schools, opportunity scholarships, virtual schools, and the like) is now a permanent part of the educational landscape.

"One of the Most Important Education Books of Our Time"

Christopher F. Chabris has a book review in today's (April 27) Wall Street Journal of a truly important book. The book is Why Don't Students Like School?, by University of Virginia psychologist Dan Willingham.

I have long been an admirer of Willingham's effort to make empirical educational psychology accessible to teachers. He writes a regular column ("Ask the Cognitive Scientist") in the American Educator, the national magazine of the American Federation of Teachers. The fact that the AFT presents Willingham's understandable, but nonetheless intellectually challenging articles on a regular basis is a great credit to that organization and has undoubtedly helped thousands of teachers.

Willingham's brand-new book, Why Don't Students Like School?, is one of a handful of books about which one can honestly say: This is one of the most important education books of our time.

Reviewer Christopher Chabris accurately summarizes Willingham's approach:

[Willingham] poses nine questions that a teacher might want to ask a cognitive scientist -- beginning with the question in the title -- and then answers each, citing empirical studies and suggesting ways for teachers to improve their practice accordingly.

Striking the Right Balance When Teaching Abstract Concepts

Let's consider the question that Willingham sets forth in the title: Why don't students like school? Here's Chabris's summary of the answer:

[W]hat school requires students to do -- think abstractly -- is in fact not something our brains are designed to be good at or to enjoy. When we confront a task that requires us to exert mental effort, it is critical that the task be just difficult enough to hold our interest but not so difficult that we give up in frustration.

When this balance is struck, it is actually pleasurable to focus the mind for long periods of time. For an example, just watch a person beavering away at a crossword or playing chess in a noisy public park.

But schoolwork and classroom time rarely keep students' minds in this state of "flow" for long. The result is boredom and displeasure. The challenge, for the teacher, is to design lessons and exercises that will maximize interest and attention....

Drill & Practice

One of the constant refrains of today's proponents of Progressive education is that we should do away with "drill and kill." Willingham tackles this issue by showing how drill-and-practice enables learners to commit information to long-term memory, thus freeing their short-term memory to deal with novel problems. As Chabris puts it:

[R]esearch shows that practice not only makes a skill perfect but also makes it permanent, automatic and transferable to new situations, enabling more complex work that relies on the basics.

Critical Thinking Skills

Anyone who has been around schools in the last few decades has heard the Progressive mantra that we need to give priority to a stand-alone skill of "critical thinking." It is part of the repertoire of "21st-century skills" that President Obama wants to impose on our classrooms. But Willingham punctures this fashionable ballon. As Chabris writes:

[S]tudents cannot apply generic "critical thinking skills" (another voguish concept) to new material unless they first understand that material. And they cannot understand it without the requisite background knowledge. The same is true of [reading comprehension]: Trying to use "reading strategies" -- like searching for the main idea in a passage -- will be futile if you don't know enough facts to fill in what the author has left unsaid.

Multiple Learning Styles

My favorite article of Willingham's in the American Educator has always been his 2005 piece "Do Visual, Auditory, and Kinesthetic Learners Need Visual, Auditory, and Kinesthetic Instruction?"

Willingham also has a video on the topic on YouTube, entitled "Learning Styles Don't Exist." (He replies to his critics on this topic here.)

Chabris summarizes what Willingham says in the new book:

The trendy notion that each person has a unique learning style comes under an especially withering assault. "How should I adjust my teaching for different types of learners?" asks Mr. Willingham's hypothetical teacher. The disillusioning reply: "No one has found consistent evidence supporting a theory describing such a difference. . . . Children are more alike than different in terms of how they think and learn."

Willingham doesn't just asset that learning styles don't exist. He carefully explains the experimental evidence against these supposed learning mechanisms.

Content is King

Dan Willingham is a proponent of content-rich instruction. He argues for this approach because scientific research demonstrates that it is the best one.

Mr. Willingham...is not in favor of merely making learning "fun" or "creative." He advocates teaching old-fashioned content as the best path to improving a student's reading comprehension and critical thinking.

This is a book aimed at teachers, but it will be of interest to all who seek better k-12 education policy. Willingham is one of the rare academic authors who can take a technical finding in cognitive psychology, explain its importance, and show how to apply it in the classroom -- and present this classroom application in way that charms and fascinates the reader.

Must We Give Up High School Exit Exams?

Education school researchers at U.C. Davis and Stanford assert that California's high school exit exam is keeping girls and non-whites from graduating from high school.

The study speculates that girls and non-whites have been failing the exit exam in high numbers because of a "stereotype threat." "Stereotype threat," as suggested by psychologist Claude Steele, is a situation in which blacks, women, etc. are put off their pace by awareness that others expect them to fail.

The Los Angeles Times story on the study paraphrases Sean F. Reardon, one of the study's co-authors:

Reardon said there was no other apparent reason [than stereotype threat] why girls and non-whites fail the exam more often than [those] white boys who are their equals in other, lower-stress academic assessments.

Here's what the executive summary of the study says:

Among students in the lowest quartile of achievement, the CAHSEE [exit exam] requirement has no effect on the graduation rate of white students, but a large negative effect on graduation rates of black, Hispanic, and Asian students.

On average, among students in the bottom quartile of achievement, graduation rates were 19 percentage points lower among Black students, 15 points lower among Hispanic students, and 17 points lower among Asian students who were subject to the CAHSEE requirement than among similar students not subject to the requirement.

Likewise, graduation rates were 19 percentage points lower among female students, but only 12 points lower among male students who were subject to the CAHSEE requirement than among similar students not subject to the requirement.

I wonder about several things:

• In theory, the stereotype threat should not be lowering scores for Asians.
• It should not be lowering scores for boys in the lowest quartile almost as much as for girls.
  
• Why is the stereotype threat working only on the lowest-quartile students? In theory, it should be lowering scores for blacks and girls in all quartiles on this high-stakes test.

There would seem to be many difficulties for stereotype threat as the explanation for the results.

(The authors anticipate these objections and attempt, not very persuasively, to respond to them in their paper.)

Without getting into my skepticism about some aspects of the stereotype-threat effect, let's assume that it's true or can sometimes be true. The need then is to accustom blacks and women to competition and challenges, and the potential would seem to be there for greater success when people have instead high, demanding expectations about blacks and women.

The solution cannot be to cushion blacks and women from all challenges (like passing the not-very-tough high school exit exam). [This is a point that has been made about the stereotype threat by John McWhorter both in interviews and in his book Losing the Race.] Life is full of challenges.

Getting rid of the high school exit exam cannot be the solution. The solution has to be preparing low-performing students to pass the exam and telling them that their teachers, parents, ministers, and other community leaders expect them to succeed and will accept no excuses.

California State Supt. Jack O'Connell and others put the exit exam in place in order to have a meaningful high school diploma. It will help the students in life if they reach at least the foundational levels of knowledge that the exit exam measures. The Los Angeles Times describes those foundational levels:

The exit exam, which students can take multiple times beginning in their sophomore year, includes math and English tests, with the math aligned to eighth-grade standards and English to 10th-grade standards.

Abolishing the test and handing out meaningless diplomas to uneducated students will not really help them.

High achievement on tests stems from diligent effort and effective study habits. These need to be persevered in over time, so that knowledge, skills, and understanding can accumulate and be stored in long-term memory.

Blaming all problems on a stereotype threat and then giving up all high-stakes tests (as Reardon and others have suggested) will solve nothing.

[Thanks to Ze'ev Wurman for his advice and counsel on this post.]

UPDATE (4/23): A better place to look for advice is the 2008 report of the Public Policy Institute of California (PPIC) called "Predicting Success, Preventing Failure: An Investigation of the California High School Exit Exam," by Andrew Zau and Julian Betts. This report says that after-school reading programs and the use of peer coaches in professional development -- both instituted in San Diego during the Bersin-Alvarado years -- could help students pass the exam. It also says that the evidence suggests that intensive tutoring focused on academic content is likely to be more effective that less-intensive tutoring focused on test-taking strategies.

The Zau-Betts report also proposes spotting students early who are at risk of failing the exit exam. Then schools should provide them with academic tutoring in elementary and middle school -- rather than waiting until high school when making up the academic deficit is much more difficult.

There is room for improvement on the particulars of what San Diego did, but the Zau-Betts report points out an evidence-based path to better exit exam results: early detection, supplemental reading instruction, and intensive tutoring on academic content.

California Schools Superintendent Wants to Water Down Academic Standards in Name of "21st-Century Skills"

California State Schools Superintendent Jack O'Connell spoke to the annual EdSource Forum in Irvine today (April 17).

O'Connell, who holds a nonpartisan office, began his speech with political partisanship:

President Obama won a mandate for change that has placed him in a position to cause a massive shift in the way our government operates and in the manner in which it serves the needs of its citizens....

In just the first few months of this Administration, I can easily and confidently say that we have seen a dramatic shift in the willingness of this White House to be a partner to states — this is a welcome difference from the previous Administration....

There was more, but you get the general idea.

O'Connell then went on to identify "four key areas" that the Obama administration wants states to concentrate on:

• Adopting internationally benchmarked standards and assessments...;
• Building high-quality data systems that track students' academic careers, making it possible to tell which teachers, programs and schools are effective;
• Recruiting more high-quality educators to underperforming schools, as well as to subjects like math and science; and
• Supporting effective strategies to turn around underperforming schools.

Superintendent O'Connell proceeded to voice views that reflect the standards-and-accountability consensus on most of these matters.

I should point out that he was particularly careful to note that any "creative approaches" and "creative solutions" on teacher pay would be done only "in partnership with all the professionals involved in teaching and learning."

A flag of caution almost went up when the Superintendent said that we need to test California students more "comprehensively" than we do now. Often such phrasing is coded language for statewide portfolios or project-based testing. But then the Superintendent went on to promise that any revised testing would be as "valid" (a technical term in testing) as the current California standards-based tests and the high-school exit exam (of which the Superintendent is rightfully the proud author).

But there was one place where Superintendent O'Connell set off alarm bells: His desire to revamp and "re-up" California's academic-content standards.

The Superintendent acknowledged that this is "an area in which California has served as an established leader for some time now":

I am proud, as all of you should be, that California purposely set the bar high for what our children should know and be able to do, and we have never wavered from that commitment.

But instead of maintaining these internationally-benchmarked high standards, he wants to "rededicate and reinvigorate" them. "Evolving them is healthy," the Superintendent says.

Superintendent O'Connell wants to revamp the California academic standards because, he says, they are “a mile wide, but only an inch deep.” But, with all due respect -- as I have stated in the Los Angeles Times -- I have to disagree. Indeed, the California standards have been judged among the best in the country by the Fordham Foundation and the American Federation of Teachers. California already has official Key Standards in mathematics, for example, to identify teaching priorities and the topics that should be covered in depth.

The Superintendent also wants to revamp the California standards:

...in order to fully engage both students and teachers in the learning process in a way that sees both parties benefit and helps to better prepare students for success in the economy of the 21st century.

Translation from education jargon: He wants to water down California's existing high standards in the name of the wolly concept of "21st-century skills," that is, communicating with each other, working in groups, media literacy, and so forth. He wants to subtract from classroom time spent on solid subject-matter content to teach these supposed stand-alone skills

Unless advocates of solid academics speak up, California will -- in the name of 21st-century skills -- follow Massachusetts down the primrose path of diluted standards.

Inadequate Journalism: Will Money Buy Happiness in Arizona?

The Apr. 8 issue of Education Week has a story about Flores v. State of Arizona, a lawsuit that claims that school programs for English-language learners (ELLs) in the border town of Nogales receive inadequate funding from the state of Arizona. This is one of a series of school-finance lawsuits around the country seeking, in the name of "adequate funding," to have the courts determine school budgets.

Much of the long and detailed article is filled with quotations from advocates of higher spending saying that “adequate” (that is, massively higher) spending would fix the schooling of English-language learners.

But there are some details that might escape notice if they are not highlighted:

--Despite relying largely on local tax funds rather than state money (requiring more state spending is what the suit is about), four of the Nogales district’s six elementary schools ranked in the top 10 in the state for ELL performance.

--The child on whose behalf the suit was filed is now a student at the University of Arizona, having taken AP and IB courses in high school – so much for having had a blighted K-12 experience.

--An unforeseen consequence is that the lawsuit has forced Nogales and school districts around the state to alter the way they teach English-learners, including separating them for four hours each day from other students. Nogales school superintendent Shawn A. McCollough says that this court-mandated separation program constitutes "segregation" rather than "inclusion" for ELLs.

At Nogales High School, ELLs are grumbling about the required four-hour block, and some teachers oppose it as well. Many ELLs who had been taking only regular classes are being required to take four hours of English each day.

….In a class of 21 ELLs at an advanced level of proficiency in English, at least seven said they had been attending Nogales schools since kindergarten.

Maximiliano Bonorand, 16, a 10th grader who was born in Tucson but has spent his whole school career in Nogales, said he was in regular classes last school year and doesn’t think he needed the four-hour English block this school year. "I speak and write English better than Spanish," he said.

The reporter Mary Ann Zehr is on target in saying that if the suit succeeds before the U.S. Supreme Court, members of the plainiffs’ bar in other states will sue under civil-rights law in federal court, calling for states to provide “adequate funding” for ELL education.

Now let’s turn to the professional-journalism side of the article, focusing on this important issue of “adequate funding.” Let’s see whom the reporter quotes.

On the more-spending side, quotations from:

• The plaintiffs (mother and child);
• Roger L. Rice, the executive director of Multicultural Education, Training, and Advocacy Inc., a Somerville, Mass.-based advocacy group for ELLs;
• A U.S. circuit court judge;
• the school improvement director for the Nogales district; and
• two ELL teachers in the Nogales district (the number of ELL teachers has already increased three-fold in the district because of the court-mandated four-hour block).

Also listed as supporting more spending are:

• Arizona Center for Law in the Public Interest,
  
• a federal district judge,
  
• Arizona Attorney General Terry Goddard (identified as a Democrat),
  
• Gov. Janet Napolitano (identified as a Democrat), and
  
• the Obama administration (which has filed an amicus brief).
  

On the side of opposing court-ordered spending, quotations from:

• Nogales school superintendent Shawn A. McCollough;
• State superintendent Tom Horne; and
• The attorneys’ brief for the state (the article points out that the defense’s legal team includes Kenneth Starr)

Also listed as opposed:

• Gov. Janice K. Brewer (identified as a Republican) and
  
• the Republican leadership of the state legislature.
  

Here we have biased journalistic judgment. The reporter has lined up nonprofits, grassroots teachers-parents-and-children, and Democrats (including President Obama) on one side. She then has lined up local and state management (defended by Kenneth Starr of Monica Lewinsky fame) and Republicans on the other.

The problem with this morality play is that it leaves out any grassroots teachers, parents, and children who oppose more spending -- indeed, any taxpayer activists who are informed about Arizona education -- and any nonprofits on the other side (like the Goldwater Institute -- see its analysis here and here).

The Education Week article also fails to look at the evidence on whether spending alone boosts student achievement. Yet there was an amicus brief by stellar school-finance specialists which argued that attention in Arizona should focus on outcomes for students rather than money per se, as well as another much less impressive brief of experts for the plaintiffs.

Inadequate journalism on adequate spending.

UPDATE (4/15/09) Mark Walsh on the School Law Blog has an Apr. 13 preview on this case that covers the dueling amicus briefs from school-finance specialists.

This Is Not an Item from The Gadfly's April Fool Issue

The Mar. 30 Guardian in Great Britain reports that Birmingham City University is offering a £4,000 one-year master's degree in Social Media.* The degree program will study Twitter and other social networking websites. Students will get the degree for learning such things as how to start a blog.

An organizer of one the Social Media courses, Jon Hickman, says:

It's very relevant and very scholarly. It's a new course, but its importance is unquestionable.

One student says the course is

a complete waste of university resources....Virtually all of the content of this course is so basic it can be self-taught.

The item ("Never Too Soon") in the Apr. 1 issue of The Gadfly about 21st Century Skills was made up. But this isn't.

* £4,000 = about $6,000

HT to the Core Knowledge Blog

Confining Virtual Schooling (in Fla.)

Bill Tucker had a Mar. 31 post on The Quick & the Ed about efforts to limit virtual schooling in Florida. Recently, the Center for Digital Education ranked Florida as having the #1 online-learning program in the nation, and other states are, as Tucker puts it, "trying to match Florida's current success."

Yet, Tucker points out that a bill in the Florida legislature would block students students from taking additional credits (beyond the annual norm) through the state-operated Florida Virtual School:

Want to extend the school day virtually? Nope. Fail algebra I your freshman year and want to take an extra course online to catch up to the college track? No dice. Want to graduate early (and save the state money)? Not gonna happen.

Excited about school and want to take a high school course while in middle school? Way too ambitious. Stuck because you need to pass a course and there is no summer school option? Sit in the same class again next year.

Tucker adds that the bill would also restrict the courses that the Virtual School could teach to those in the core curriculum:

Forget...students options to take AP Art History, Computer Science, or any number of other courses. Elective options are limited to what your bricks and mortar school can offer.

Today (Apr. 1), an anonymous commenter on Tucker's post says that a "huge chunk" of the Florida Virtual School's current courses are outside the core curriculum and points to what Florida Tax Watch said about the Virtual School in its 2007 evaluation. The commenter summarizes the tax group's evaluation as follows:

[The Virtual School] saves $1000 per student vs. traditional schools, [has] higher AP scores, higher test scores in math and reading -- the list goes on and on.

Virtual schooling is an innovation with tremendous transformative potential. (See the forthcoming book Liberating Learning by Terry Moe and John Chubb.) But realizing that potential in public schools may be delayed by this sort of hobbling.

UPDATE (4/3/09) The Apr. 2 Tampa Tribune had a detailed article on this story.

You Can't Escape If You Don't Know Soon Enough

Chad Aldeman has a post on The Quick & the Ed today (Apr. 1) about an Obama administration retreat on options for children to escape failing schools. As Aldeman puts it:

Secretary of Education [Arne Duncan's] letter rolls back a regulation that could have helped provide parents of children enrolled in unsuccessful schools the option of choosing a better one.

Aldeman explains the retreat by asking us to use our imaginations to consider a parent in the following situations:

1. [I]f you're a parent, imagine going through the school registration process in the fall, buying supplies for your child, and believing that your child will begin attending school X. Then, on the first day of school, your child brings home a letter that says her school failed to make adequate yearly progress last spring, and she now has the option to transfer to another school.
2. Instead, imagine you, as a parent, were notified at least two weeks in advance of the new school year.
   

The second scenario -- adequate notice -- is what the Bush administration's regulations called for. With adequate notice, you as a parent would be in a much better situation:

You would have time to consider your options, visit new schools (maybe even new teachers), and plan transportation. You might be altogether more interested in exercising your right to choose [a better school for your child].

The Obama administration's retreat traps parents (and their ill-served children) in scenario #1. This retreat also exacerbates another important problem: late reporting of test results.

[Secretary Arne Duncan's letter] makes it less urgent for states to turn around test results promptly, which has implications beyond just an under-used school choice provision. Late data results also penalize schools labeled in need of improvement, because it gives them little time to implement a real school improvement plan.

There you have it: less school improvement and less opportunity to escape failing schools.