The Effectiveness of a Primer to Help People Understand Risk

Context Educational materials aimed at improving people’s ability to understand information about risk are scarce. Contribution In 2 trials, adults with high or low socioeconomic status (SES) were randomly assigned to receive a primer about understanding risk or a general health booklet. In both SES groups, adults receiving the primer more often passed a medical data interpretation test than did those receiving the general health booklet. They also expressed greater interest in medical statistics but not greater confidence in interpreting statistics, and most rated the primer helpful or very helpful. Cautions The authors did not examine whether improved data interpretation skills affected decision-making. The Editors People face a bewildering array of medical decisions (for example, should I be tested for BRCA1? Should I be screened for prostate cancer? Should I have lumpectomy and radiation to treat my breast cancer?). If people are to make informed decisions, they need to understand risk: What is my chance of staying healthy if I undergo this intervention? What is my chance of staying healthy if I forgo it? What is the chance of harm? Understanding risk data entails a set of skills. First, people must be able to work with probabilities and changes in probabilities, the typical measures used to communicate risk information. Next, they need to have a framework on which to organize data. In essence, this means having a sense of what additional information is needed to give meaning to a statement about probability (conversely, how to know when such additional information is lacking). For example, putting a particular risk in context (your chance of breast cancer is 1 in 8) entails knowing what outcome is being considered (diagnosis vs. death), being clear about the time period (5 years vs. lifetime), and having some perspective on the magnitude of the probability (How does the risk for a particular condition compare with other important health risks?). Third, people need to be able to make some basic assessment about the quality of the evidence to know whether they can believe the numbers that they are given. Anecdotal experience and a growing body of literature document the trouble many people have in making sense of risk information (14), which is now ubiquitous in health messages. This is not surprising: Much more effort goes into disseminating health information (for example, drug advertisements, media reports, and decision-making aids) than into preparing the target audiences to understand it. To address this gap, we developed a primer to help people develop the basic skills needed to make sense of the medical risk data that surround them. In this paper, we report on 2 randomized trials that tested the primer. We sought to learn how people would respond to the primer: Would they read it? Would they value this knowledge? Most important, would the primer improve patients’ abilities to interpret medical data? Methods Design Overview To test the effect of the primer on how well people understand risk, we conducted 2 randomized trials in distinct populations: people with high and low socioeconomic status (SES). Figure 1 shows an overview of the study design. The Committee for the Protection of Human Subjects at Dartmouth College approved this project, and the survey cover letter was considered to be informed consent (signed consent was waived). Figure 1. Overview of 2 randomized trials in distinct populations. SES = socioeconomic status; VA = Veterans Affairs. Setting and Participants We calculated our sample size under the most conservative conditionswhen the pooled proportion of passing scores was 50%. We asserted that a 20-percentage point absolute difference in the proportion passing the test in the primer group versus the control group would be clinically important. In calculating the sample size requirements, we therefore assumed that the proportion passing would be 50% in the control group and 70% in the primer group. When a power of 0.8 and a 2-sided P value of 0.05 were used, 100 patients were required for each study group. Assuming that 10% of participants would be lost to follow-up, we planned to enroll 110 patients per group in each trial. High Socioeconomic Status Trial To study the effect of the primer in a highly educated and affluent group, we recruited alumni from Dartmouth’s Community Medical School. This 9-lecture series on various health and medical topics is taught by Dartmouth faculty and guests; is held annually in Hanover, New Hampshire, and Manchester, New Hampshire; and has a $25 registration fee (5). For this study, the program organizers mailed recruitment letters to 1138 alumni. A total of 334 people who responded to the letter were eligible (that is, they met the age criterion of 35 to 79 years, spoke English, and had attended the Community Medical School in the past) and were subsequently randomly assigned. Ninety-six percent (n= 322) returned a completed survey (completion rates were not significantly different between the primer and control groups [95% vs. 98%, respectively]) (P= 0.192). Low Socioeconomic Status Trial To study the effect of the primer in a sample with lower income and less formal education, we recruited veterans and their families at the White River Junction Veterans Affairs (VA) Medical Center, White River Junction, Vermont, by posting study advertisements in waiting areas of the outpatient clinic. A total of 221 people who responded to the advertisement were eligible (that is, they met the age criteria of 35 to 79 years, spoke English, and were a veteran or the family member of a veteran who was enrolled in a VA clinic) and were subsequently randomly assigned. Ninety percent (n= 200) returned completed surveys (completion rates were significantly lower in the primer group than in the control group [85% vs. 96%, respectively]) (P= 0.005). Randomization and Interventions The letter and the advertisement asked people to participate in a research study to learn how to better give people health information. We did not mention our interest in enhancing quantitative skills. People who responded to the recruitment letter (high SES group) or to the advertisement (low SES group) were first interviewed to confirm eligibility. We only accepted 1 participant per household. Within each trial, we assigned participants on an individual basis to receive either the primer or control booklet. A list of random numbers (created by using a random-number generator) was given to a research assistant and was used to determine assignments (randomization was not stratified or blocked within each trial). The research assistant had access to participants’ characteristics. The investigators did not have access to the assignment list, and the deidentified code was only revealed after recruitment and completion of data collection. The appropriate booklet and a survey were mailed or given in person to participants. The survey included the major outcomes that will be described in the next section. To make the survey seem relevant to the control group, it also included additional questions about facts presented in the health booklet (we did not analyze responses to these questions). All participants were asked to read the booklet and return the completed survey within 2 weeks using the self-addressed stamped envelope included with the survey. Reminder letters were sent to nonresponders. Participants who returned surveys were given their choice of a $25 gift certificate to a local bakery, restaurant, bookstore, or large retail store. We recruited participants from October 2004 through August 2005. Primer Group The participants in the primer group received the booklet entitled Know Your Chances: Understanding Health Statistics. The goal of the primer is to teach people how to understand risk messages and health statistics. The first part of the primer teaches people how to understand disease risk by using the example of colon cancer. The second part focuses on how to understand the benefits and harms of interventions by using a Zocor direct-to-consumer advertisement for secondary heart disease prevention. Figure 2 shows the summary of main concepts that were taught, which is excerpted from the booklet. The contents of the primer and many of the examples have been developed and revised over years of teaching and through focus groups with people across a diverse socioeconomic spectrum. Figure 2. Key concepts taught in the primer, excerpted from the final pages. Because many people are intimidated by numbers and statistics, we worked hard to make the primer inviting and nonthreatening by liberal use of cartoons and figures; by working through examples, separating the most technical material into optional learn more boxes; and by providing readers with quizzes (with answers) to assess their mastery of the material as they read the primer. Figure 3 shows sample pages from the primer. Most of the primer is written at the eighth-grade or lower reading level (6) and is a color document approximately 80 pages in length. Figure 3. Excerpts of primer. Control Group The control group received a 70-page booklet entitled The Pocket Guide for Good Health for Adults, which is published by the U.S. Department of Health and Human Services Agency for Health Care Research and Quality (7). We chose this booklet because the length is similar to that of the primer, it is written at a similar reading level (eighth grade or lower) (6), and it contains general information about risk and reducing risk by following recommended prevention or screening activities. However, it does not include training on how to interpret quantitative information. Measurement and Outcomes All outcome measures were assessed in the survey. Primary Outcome The primary outcome was a measure of participants’ abilities to interpret medical statistics. To measure this outcome, we developed the 18-item data interpretation test during the same time as