Levels of Evidence of Effectiveness Pyramid
| The Central CAPT web site has moved to be part of a new National CAPT web site. In 10 seconds you will be automatically redirected to http://captus.samhsa.gov/central/central.cfm -- Use this form if you have any questions. |
Level 5 Multiple Site Replication Studies
Prevention programs, principles and policies designed to directly affect youth perceptions of drug and alcohol use, age of first ATOD use, frequency of ATOD use and abuse, or related risk or protective factors that have been successfully replicated in several settings, preferably across multiple target populations with consideration for age, gender, race/ethnicity, and geographic context. These processes or programs must be evaluated using scientific methods that include a pre and post test to show positive results. Program replication requires high levels of fidelity within the context of the new settings. These results must be published in more than one scientific, peer reviewed, academic journal. (Examples: Botvin et al., 1995; Pentz, et al., 1989)
<Return to Top>
Level 4 Meta-analyses, Expert Review, and Peer Consensus
A number of methods have been used to synthesize prevention research and evaluation reports and scientific publications. This process may include a meta-analysis, consensus, and expert review panels. These techniques may be part of a meta-analysis whereby various program evaluations (Ennet et al., 1994) and/or components of programs (Tobler & Stratton, 1997) that are analyzed for program effectiveness. Another way to determine the effectiveness of various prevention strategies is often done by convening professional prevention organizations or groups of prevention "experts" who review and rate programs, principles, and policies for effectiveness. Both of these strategies typically use refereed publications, dissertations, evaluation reports, and source documents. (Examples: U.S Department of Health & Human Services, 1996; Drug Strategies, 1997; Tobler & Stratton, 1997)
<Return to Top>
Level 3 Single Trial Effectiveness
Programs, principles and policies designed to directly affect youth perceptions of drug and alcohol use, age of first ATOD use, frequency of ATOD use and abuse, or related risk or protective factors that have been reported in a single population or in only one setting. These programs, principles, or policies use scientific methods that must include pre and post test with either a comparison or control group to assess impact and must show some overall positive results. The results must be published in at least one scientific, peer reviewed, academic journal. (Examples: Perry et al., 1997)
<Return to Top>
Level 2 Program Evaluations and Source Documents
Programs, principles and policies designed to directly affect youth perceptions of drug and alcohol use, age of first ATOD use, frequency of ATOD use and abuse, or related risk or protective factors for which positive outcomes have been documented in written form (e.g. conference or workshop report, internal report, published non-academic article or newsletter, etc.). These prevention approaches should be evaluated using methods such as pre/post designs, qualitative analyses, cohort evaluation, or some type of comparison of participants outcome norms to local or state averages. Qualitative data reporting client satisfaction, program implementation, participants' perceptions of program benefits may have also been used to determine and document program effects. These programs may be published in refereed or non-refereed publications or professional publications.
<Return to Top>
Level 1 Testimonials, Newspaper Reports, or Non-refereed Publications
Reports programs, principles and policies designed to directly affect youth perceptions of drug and alcohol use, age of first ATOD use, frequency of ATOD use and abuse, or related risk or protective factors for which there is only anecdotal evidence of positive results in the form of participant testimonials, quotes, or media coverage. This anecdotal evidence is frequently reported in non-refereed publications, newspapers, professional newsletters or commercial advertising with little or no systematic evaluation or empirical support. These types of initiatives are not considered research-based.
<Return to Top>
Research-based Prevention: A Pyramid for Effectiveness
Developed by Peter Mulhall, Ph.D & Carol Hays, Ph.D.
Center for Prevention Research and Development
Institute of Government and Public Affairs
University of Illinois
References
Botvin, G.J., Schnike, S.P., Epstein, J.A., Diaz, T., & Botvin, E. (1995). Effectiveness of culturally focused and generic skills training approaches to alcohol and drug abuse prevention among minority adolescents: Two-year follow up results. Psychology of Addicitive Behaviors, 9(3), 183-194.
Botvin, G.J., Baker, E., Dusenbery, L., Tortu, S., & Botvin, E. M. (1990). Prevention adolescent drug use through a multi modal cognitive-behavioral approach. Journal of Counseling and Clinical Psychology, 58, 437-446.
Drug Strategies (1997). Making the Grade: A Guide to School Drug Prevention Programs. Drug Strategies: Washington, D.C.
Ennet, S.T., Tobler, N.S., Ringwalt, C.L., & Flewlling, R.L. (1994). How effective is Drug Abuse Resistance Education? A meta-analysis of Project DARE outcome evaluation. American Journal of Public Health, 84, 1394-1401.
Pentz, M.A., MacKinnon, D.P., Flay, B.R., Hansen, W.B., Anderson Johnson, C., & Dwyer, J.H. (1989). Primary prevention of chronic diseases in adolescence: Effects of the Midwestern Prevention Project on tobacco use. American Journal of Epidemiology, 130(4), 713-724.
Williams, C.L., Veblen-Mortenson, S. et al. (1996). Project Northland: Outcomes of a communitywide alcohol use prevention program during early adolescence. American Journal of Public Health, 86, 956-965.
Perry, C.L., & Grant, M.G. (1998). Comparing peer-led to teacher-led youth alcohol education in four countries. Alcohol Health and Research World, 12(4), 322-326.
Tobler, N., and Stratton, H. (1997). Effectiveness of school-based drug prevention programs: A meta-analysis of the research. Journal of Primary Prevention, 18, 71-128.
U.S. Department of Health and Human Services. (1996). Alcohol and other drug prevention: The National Structured Evaluation. Third Report to Congress. Department of Health and Human Services: Washington D.C.
