Finding Patterns in International Assessments

The results now being reported from a study of 45 countries describe complex patterns of science education practices. The Third International Mathematics and Science Study (TIMSS) ([wwwcsteep.bc.edu/TIMSS][1]) for the first time connects the intended curriculum in texts and curriculum guides, the instructed curriculum that teachers report implementing, and the achieved curriculum as measured by student performance. The wide variations in the intended and instructed science curriculum in countries participating in TIMSS influenced the design of the international assessment and now guide interpretation of the results. Each country proposed and critiqued potential questions. For earth science, some countries identified questions about fjords and glaciers as central to instruction while others rejected them as esoteric. Many countries objected to testing on topics their students would study only after they took the test. Scoring written responses to questions such as, “Carbon dioxide is the active material in some fire extinguishers; how does carbon dioxide extinguish a fire?” required distinguishing memorized information from thoughtful connections between ideas. Studies like TIMSS underscore the complex influences on science learning but also suggest patterns that lead to success. Variables such as annual hours of science instruction, curriculum standards, class size, amount of homework assigned, reported time spent doing homework, hours spent viewing television, and frequency of classroom tests cannot, by themselves, account for the results across countries. In Korea, where students performed best on the international assessment, eighth-grade teachers report the largest class sizes and students report the fewest classroom tests. At the same time, many TIMSS countries can serve as case studies for addressing complex questions such as: Do students learn more from deep coverage of a few topics or broad coverage of many topics? Or, what mix of topics, course activities, classroom organization, and assignments makes science learning more efficient? For example, the pattern of covering a few topics in depth can help explain why student performance in the top-scoring countries meeting international guidelines diverges between the fourth and eighth grades. Korea performs best at fourth grade, followed by the United States, Japan, and the Czech Republic, whose students perform similarly. At eighth grade, Korea again performs best, Japan and the Czech Republic remain high in student performance, but the United States falls to just above the middle. Compared to the U.S. curriculum, the intended curricula in Japan and the Czech Republic cover fewer topics. For example, experts estimate that between the fourth and eighth grades, Japan and the Czech Republic add about 16 topics to their curricula, while the United States adds about 27 topics. In addition, textbooks for the population of 13-year-olds tested by TIMSS have more than 70 topics in the United States, while the mean for TIMSS counties is about 27. How can students who study fewer topics achieve more? In Japan and the Czech Republic, about half of the TIMSS teachers report meeting with other teachers at least weekly to plan instruction and discuss pedagogy. Fewer than one-third of U.S. teachers report meeting this often. On a recent visit to Japan, I observed teachers there considering how to revise lessons so that students made better connections to previous lessons and to everyday experience. The pattern of teaching topics in depth and regularly testing improvements resonates with results from a decade-long research project I direct ( ). We find that students who make links between science and everyday experience become lifelong learners. Thus, glaciers and fjords can help students familiar with them make sense of geology, weather systems, thermal phenomena, and the like. Adaptations of desert plants may focus the understanding of students familiar with deserts. We cannot possibly teach students all the science we want them to learn, but perhaps we can design courses so that students revisit and strengthen their science ideas. I believe that we should form partnerships of experts in science, mathematics, pedagogy, classroom teaching, and assessment to regularly redesign instruction so it builds on what students know, connects to future science problems, and takes advantage of patterns from studies like TIMSS.