Questions about US results in PISA

What do international tests really show about U.S. student performance?: ”Although the U.S. sample included disadvantaged students in appropriate propor- tion to their actual representation in the U.S. 15-year-old population, the U.S. sample included a disproportionate number of disadvantaged students who were enrolled in schools with unusually large concentrations of such students. Because, after controlling for student social class status, students from families with low social class status will perform more poorly in schools with large concentrations of such students, this sampling flaw probably reduced the reported average score of students in the bottom social class groups (perhaps Groups 1-3).”

(Hittat via Pasi Sahlberg.)

I am not a statistician even if I consider myself qualified to discuss a lot of statistician issues. This however I do not understand completely. Martin Carnoy, Stanford Graduate School of Education and EPI
and Richard Rothstein, EPI argues that the conclusions based on US results in international tests like PISA and TIMSS are oversimplified, frequently exaggerated and misleading (anyone thinking the same here regarding Swedish results?).

However, Carnoy and Rothstein have made a detailed analysis of the PISA 2009 database and found that the US results are much better than reported if one would consider the ”sampling error” described in the quote above. Question is however if this argument is valid – I thought PISA is supposed to present a result fot the whole educational system in a country regardless of which schools the students attend. In Carnoy/Rothsteins perspective it is reasonable to take the fact that ”students from families with low social class status will perform more poorly in schools with large concentrations of such students” as a reason to exclude them from the sample or at least not count their results. To me that seems a bit strange, it is like as if we would exclude the results for students att schools in poor performing schools in certain areas with the same argument. Although this is something I would like comments on.

Technorati Tags:
Pasi Sahlberg, PISA, Martin Carnoy, Richard Rothstein, TIMSS

What do international tests really show about U.S. student performance?: ”Although the U.S. sample included disadvantaged students in appropriate propor- tion to their actual representation in the U.S. 15-year-old population,...