Factors That Influence the Effectiveness of Primary Prevention of Substance Use: A Review of Reviews

authors:

avatar Maral Mardaneh Jobehdar ORCID 1 , avatar Emran Razaghi ORCID 2 , 1 , * , avatar Ali Akbar Haghdoost ORCID 3 , avatar Alireza Baleshzar 4 , avatar Kaveh Khoshnood ORCID 5 , avatar Mohammad Reza Ghasemzadeh 6 , avatar Seyed Abbas Motevalian ORCID 7

Department of Neuroscience and Addiction Studies. School of Advanced Technologies in Medicine, Tehran University of Medical Sciences, Tehran, Iran
Department of Psychiatry, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
Health Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran
Department of Psychometrics, Allameh Tabatabaei University, Tehran, Iran
Yale School of Public Health, New Haven, Connecticut
Department of Psychiatry, AJA University of Medical Sciences, Tehran, Iran
Iran University of Medical Sciences, School of Public Health, Iran University of Medical Sciences, Tehran, Iran

how to cite: Mardaneh Jobehdar M, Razaghi E, Haghdoost A A, Baleshzar A, Khoshnood K, et al. Factors That Influence the Effectiveness of Primary Prevention of Substance Use: A Review of Reviews. Iran J Psychiatry Behav Sci. 2021;15(4):e116288. https://doi.org/10.5812/ijpbs.116288.

Abstract

Context:

The fact that substance use may result in dependence and severe harm to the user and the community has led to serious and extensive endeavors to design and implement preventive interventions. Prevention programs, however, have been shown to have different effects, and that should not be taken for granted. Systematic reviews and meta-analyses in the field have tried to reveal the effects of different types of substance use prevention programs and to identify the influencing factors.

Objectives:

The purpose of the current review was to more accurately identify factors associated with the effectiveness of interventions or reversal of desired effects.

Evidence Acquisition:

We systematically reviewed systematic review and meta-analyses studies within the period of 2009 - 2019 that reported the effectiveness of substance use prevention programs. After the systematic screening of 1586 articles, 43 articles were finally reviewed.

Results:

In preparation for extracting our data, we designed a structural model for analyzing the results of research on the effectiveness of substance use prevention programs. In general, universal types of prevention programs were reported as more effective than the two other types of selective and indicated programs. Regarding the context of programs, the school setting was more effective than others. In the case of interventions performed in the family setting, selective programs were more effective. Interventions provided by specialists were more effective than those provided by laypeople and / or peers. Interventions provided by more than one person and from more than one discipline showed higher effectiveness.

Conclusions:

Based on our structural design, we feel that further studies are needed to extend our knowledge regarding the effectiveness of substance use prevention programs.

1. Context

According to the 2019 United Nations report, 5.5 percent of those aged 16 - 64 used some form of drugs in the previous year worldwide, revealing a 30 percent increase from 2009 (1). It is estimated that four out of ten substance users are younger than 25 years, with 16 percent being aged below 15 (2). The use of drugs is higher in the youngsters compared to the old, and it has been shown that substance use peaks in those aged 18 - 25 (2). Ages 12 - 14 and 15 - 17 are the critical period for the first time use (3, 4). The younger the user, the higher possibility of developing substance dependence compared to starting substance use in adulthood (5-8). Substance use in early life has direct harmful effects on the individual, the family, and the community. Being exposed to alcohol and other substances results in problems with physical growth, mental development, and problems related to psychological factors (9-13). Having in mind that 44 percent of the world population is below 24 and the fact that substance use comprises a significant fraction of disability adjusted life years (DALYs) worldwide, prevention of substance use becomes of great importance from a public health view (14, 15) and increases the importance of its consideration (16, 17). Aside from the public health point of view, economic development is also strongly correlated to a healthy younger generation (18-22). Therefore, keeping communities safe, especially in preventing teenagers and young adults from substance use, has become an essential social objective (23). Implementation of effective prevention programs is, therefore, a crucial step in safeguarding individuals and communities (24, 25). In the quest for the identification of effective substance use prevention, the following approaches have been at the focus of scholars’ attention (24, 26-28).

1.1. The Public Health Model

The priorities of this model include keeping the target group abstinent, decreasing the incidence of substance use, and avoiding dependence in cases that substances are being used (28-30). This approach identifies three categories of intervention.

1.1.1. Universal

In this category, the whole population is targeted, and no exclusive or specific subgroups related to substance use or else are defined (24, 27, 30).

1.1.2. Selective

This category focuses on higher risk groups with individual, social, and psychological risk factors (28-30).

1.1.3. Indicated

Its target groups include those who have already used substances and are at the highest risk for progression to severe stages of substance use (28-30).

1.2. Risk and Protective Approaches

The focus of this model is on controlling the risk factors and promoting the protective factors. As people exposed to risk conditions do not necessarily start to use substances or become dependent, both risk and protective factors can be different from one person to another (24, 27, 31). Furthermore, risk and protective factors depend on demographic characteristics such as age, gender, race, and culture (27). Intervention programs based on this approach focus on the improvement of self-efficacy, decision-making, communication skills, life skills, resistance against peer pressure, resilience, and awareness about substance use and its consequences (24, 25, 27, 30).

(1) Risk factors depend on psychological, social, biological, environmental, and behavioral elements that have the potential of being the source of health problems and, therefore, an underlying cause for substance use (18, 24, 27).

(2) Protective factors include all conditions that reduce or prevent the possibility of using substances (27, 32).

1.3. Communicable Disease Model

This model consists of a triangular arrangement of elements, ie, (a) the host, (b) the agent, and (c) the environment. According to this model, the objective of prevention programs is to block the effects of the agent on the host, either directly or through modifications in the environment. Examples of this approach include keeping schools free of substances, smoke-free schools, and taxation on alcohol sales (24, 27, 30).

1.4. Context-based Setting of Interventions

The focus of this approach is on the setting where the intervention is provided and the context of this setting that might influence the intervention and its effects (27, 29).

1.4.1. Family Based Programs

Targeting the family as a whole and focused on improving the parent-child relations and parenting skills (27, 29).

1.4.2. School Based Programs

These types of programs are implemented on a school platform with a focus on awareness-raising, social skills training, and enhanced students’ resilience against environmental substance-using impulses (27, 29).

1.4.3. Community Based Programs

In this type, community members participate in the program and contribute to the program design and its implementation (27, 29).

1.4.4. Technology Based Programs

A subtype that primarily takes advantage of emerging (digital) technologies (33-36).

To achieve their substance use prevention objectives, all different types of prevention categories described above, rely on specific theories that explain the roots of substance use initiation (24, 37). Those theoretical explanations, therefore, are the source of defining the design of each prevention program. The following are the four major theoretical models:

(1) Information- and fear-based theories. According to this theoretical model, the lack of essential information about substances and their negative consequences is the underlying reason for starting substances. Therefore, by providing information about negative aspects of substance use or by changing beliefs that substances are beneficial to a negative attitude, individuals will be scared of trying substances. The tactic of this model is to increase knowledge about the negative consequences of behavior and to revise the attitudes of the target group about the impact of substance use. The ultimate goal of this method is an interrelated modification and promotion of substance-related knowledge, attitude, and behaviors (KAB) (27, 37, 38).

(2) Social influence and learning theory. According to this theory, observational learning and the function of social influence are key elements to one's behavior, which will be a reflection of role models. Positive and negative reinforcements are, therefore, the key to behavior modification. Modified behavior and acceptable social norms, at the social level, and resistance against social influences for using substances, at the individual level, are regarded as the impact of this approach (37, 39).

(3) Environmental theory. The focus of this approach is on social-environmental variables that are of concern as risk factors for substance use (27, 40). According to this model, the social gap may lead to reduced social bonding and, hence, results in substance use in the disadvantaged members of the community.

(4) Social bonding theory (24, 27). This theory is about social bonding with the family, peers, school, and other social groups, which are of particular importance. It is believed that weak family bindings end up in stress, helplessness to confront stresses, and rebelliousness. The resulting distress, isolation, suicidal tendencies, and depression can, therefore, be a reason for using a substance (24, 30).

With the above models in mind, a less touched upon the aspect of substance use prevention is the role of individuals or institutions who are providing them. For example, when prevention programs are provided by peers, they might have different outcomes compared to programs provided by teachers or parents. While peers can be effective role models for abstaining from substances, they might lack motivation or skills, and therefore, result in unwanted role-modeling effects (41, 42). Therefore, it appears that in addition to the extent of the problem, and the exquisiteness of performance of a prevention intervention, factors attributing to theoretical approaches to substance use and the design of the interventions play a role in the effectiveness of substance use prevention programs. The existing systematic reviews have mainly focused on the effectiveness of different types of preventive programs.

2. Objectives

In order to move to a more accurate level, the objectives of this review of systematic reviews and meta-analyses were (1) identification of factors that make an intervention to be effective, (2) identification of ineffective factors in drug prevention interventions, and (3) identification of factors that can reverse the desired effects of prevention programs.

3. Evidence Acquisition

Following a review design (43), systematic review and meta-analysis studies on the effectiveness of substance use preventive interventions published in English peer-reviewed journals were systematically searched over PubMed and Cochrane databases from 2009 to 2019 (Figure 1) (Table 1). Based on prevention settings, we, further excluded articles focusing on the workplace and the health sector. Quality appraisal was performed using the AMSTAR 2 (a critical appraisal tool for systematic reviews that include randomised or nonrandomised studies of healthcare interventions, or both) (44), which consists of 16 items. The quality of included articles ranged from medium to good. According to the objectives of our study, we extracted and categorized the following data from our search results:

(1) Intervention type. Universal, selective, and indicated.

(2) Target groups by age. Middle childhood (5 - 10), early adolescence, adolescence (11 - 18), and adulthood (19 and above) (25).

(3) Program providers. Specialists, teachers, peers, mentors, family, and parents.

(4) Duration of intervention and follow-up. Short-term and long-term.

(5) The theoretical model of intervention.

Table 1.

Search Strategy

Database and StrategySearch Items
PubMed
Mesh termsSmoking / prevention and control, Marijuana smoking / prevention and control, Tobacco use disorder / prevention and control, substance-related disorders / prevention and control, Binge drinking / prevention and control, Alcohol
KeywordsAddict*, use*, abuse*, misuse, consumption*, intoxicate*, substance*, drug*, alcohol*, marijuana, hashish, stimulant*, tobacco*, cigarette*, prevent*, "primary prevention", "universal prevention", " selective prevention", "indicated prevention", " prevention intervention", "prevention program"
LimitedSystematic-review, meta-analysis, review, review of reviews, umbrella-review, an overview of reviews
Year2009 - 2019
Cochrane
KeywordsAddiction, use, abuse, misuse, consumption, intoxicate, substance, drug, alcohol, marijuana, hashish, stimulant, tobacco, cigarette, prevent, prevention, intervention "primary prevention", "universal prevention", "selective prevention", "indicated prevention”, "prevention intervention", "prevention program"
LimitedSystematic-review, meta-analysis, review, review of reviews, umbrella-review, an overview of reviews
Year2009 - 2019

4. Results

In the systematic search of databases, of a total of 1586 articles, 1563 were found in PubMed and 23 in Cochrane. After duplicate deletion, 1025 articles remained. Then, titles of articles were screened separately by two of the authors, leaving 362 articles, and were reduced to 234 after abstract review. A full-text review was performed by the same two authors, resulting in 95 articles being eligible to enter the quality evaluation stage. After quality appraisal of articles, 43 were included in the present study (Figure 1). According to the setting of the interventions, the articles were divided into the following four categories. As some interventions were performed in more than one setting, we counted the articles according to settings separately, meaning that one article might have been counted more than once according to all settings it could fulfill. As we were not focusing on the type of substances as the factors influencing the result of interventions, for a clarification purpose, we differentiated between types of substances that prevention programs had focused on under the category of settings.

(1) School-based primary preventive interventions to prevent the use of tobacco, cannabis, alcohol, substance, or drugs (n = 26).

(2) Family-based primary preventive interventions to prevent the use of tobacco, cannabis, alcohol, substance, or drugs (n = 13).

(3) Community-based primary preventive interventions to prevent the use of tobacco, cannabis, alcohol, substance, or drugs (n = 3).

(4) Technology-based primary preventive interventions to prevent the use of tobacco, cannabis, alcohol, substance, or drugs (n = 6).

With a focus on different categories of interest that might have a potential role on the effectiveness of prevention interventions — as explained above — and under each setting category, we further extracted and identified the findings of each article according to the types of interventions, theoretical models, duration of intervention and follow-up, and program providers based on target group age, as effective or ineffective. Therefore, a typical table was designed to present the findings for each setting.

4.1. The Results for School-based Prevention Interventions

The results obtained in this setting were extracted from 26 articles (Appendices 1 - 4) (25, 42, 45-68). The results of school-based prevention interventions for tobacco, alcohol, cannabis, and substance and drugs are available in Appendices 1 – 4 (see Supplementary File).

4.2. The Results for Family-based Prevention Interventions

The results obtained in this setting are extracted from 13 articles (Appendices 5 - 8) (25, 41, 47, 50, 51, 60, 66, 69-74). The results of family-based prevention interventions for tobacco, alcohol, cannabis, and substance and drugs are available in Appendices 5 - 8 (see Supplementary File).

4.3. The Results for Community-based Prevention Interventions

The results obtained in this setting were extracted from one systematic review article for tobacco, as well as from two overview of reviews articles for substance (Appendix 9) (25, 41, 71). The results of community -based prevention interventions for tobacco are available in Appendix 9 (see Supplementary File). There were no data available from 2009 to 2019 on community-based prevention interventions for alcohol, cannabis, and drugs.

4.4. The Results for Technology-based Prevention Interventions

The data obtained in this setting were gathered from six studies (Tables 2 and 3) (25, 35, 36, 67, 75, 76). There were no data available from 2009 to 2019 on technology-based preventive interventions for Tobacco and Cannabis.

Table 2.

Technology-based Preventive Interventions for Alcohol

EffectivenessMiddle Childhood (5 - 10)Early Adolescence, Adolescence (11 - 18)Adulthood (19 and Above)
Type of intervention
EffectiveNo dataNo dataSelective
IneffectiveNo dataNo dataNo data
Theoretical model
EffectiveNo dataNo dataComputer-delivered interventions (CDIs); Feedback plus moderation skills in Alcohol consumption short-term risky alcohol consumption; Multi-dose computerized assessment and feedback; Text massage
IneffectiveNo dataNormative feedback and relapse/prevention/single-focus therapeutic strategy, mostly/personalized normative feedback and combined treatment approaches (based on motivational interviewing, personalized normative feedback/cognitive-behavioral therapy and/or behavioral self-control, and change principles)Normative feedback and relapse prevention/single-focus therapeutic strategy, personalized normative feedback and combined treatment approaches (based on motivational interviewing, personalized normative feedback, cognitive-behavioral therapy and/or behavioral self-control and change principles)
Duration of intervention and follow-up
EffectiveNo dataStudent group: Minimal effect at < 5 weeksMulti-session interventions more effective than one-time interventions/single-session computerized task delivered via the Internet, intranet, or CD-ROM / DVD lasting a median of 20 minutes
IneffectiveNo dataIn mixed population: No significant effects were maintained after 12 months; Alcohol consumption long-termAdult (Non-Student) Populations: No significant difference found at 12 months
Program provider
EffectiveNo dataComputer feedbackComputer-based interventions are more successful in reducing alcohol-related problems ain short-term (≤ 5 weeks), when including human interaction vs. using the computer alone; Computer feedback
IneffectiveNo dataNo dataNo data
Table 3.

Technology-based Preventive Interventions for Substance or Drugs

EffectivenessMiddle Childhood (5 - 10)Early Adolescence, Adolescence (11 - 18)Adulthood (19 and Above)
Type of intervention
EffectiveNo dataUni-modal programs (universal and targeted)No data
IneffectiveNo dataNo dataNo data
Theoretical model
EffectiveNo dataNo dataNo data
IneffectiveNo dataNo dataNo data
Duration of intervention and follow-up
EffectiveNo dataPeriod of 6 months or longerNo data
IneffectiveNo dataNo dataNo data
Program provider
EffectiveNo dataNo dataNo data
IneffectiveNo dataNo dataNo data

5. Discussion

This study is a review of systematic reviews and meta-analyses on the prevention of substance use. We categorized prevention interventions according to the setting in which they were performed and according to the age of their target groups. We then reviewed potential factors that could be identified as having an association with the effectiveness of the programs.

5.1. Types of Interventions

Compared to other types of interventions, universal interventions were more likely to be effective, irrespective of setting, age, and substance type. In specific instances such as school-based interventions, uni-modal targets programs have been reported to be effective for cannabis use. In one school-based review (61) more than 70 percent of the studies were universal, about 18 percent were selective, and 9 percent were indicated. Forty-three (out of 30) articles of the reviews were on "universal studies", two were on "selective studies", another two were on "indicated studies", and five were collective studies of all tree intervention types, the rest of the studies were reviews on reviews. An explanation for the wider practice of universal prevention programs might be their less costliness and coverage of larger populations. One would also think that universal programs are easier to perform due to the less complex nature of a universal program compared to other types that need to further be tailored for specific groups. Therefore, the higher reported effectiveness of this type of prevention needs to be adjusted by their frequency of practice.

Selective intervention programs have been reported to be effective in the family and in technology-based settings. While no data is available for selective interventions in early age groups, it has been reported to be effective in adulthood (ages 19 and above). Although universal interventions are reported to be relatively effective in white target groups and only for alcohol and tobacco, it appears that selective and indicated interventions are effective in the reduction of cannabis use (77). Furthermore, selective and indicated interventions have resulted in the more frequent use of social services by families, increased knowledge of parents about substances, and reduced aggressive behaviors (77). A majority of the studies have reported that universal interventions to be less effective in the family setting (73). Perhaps this difference is a result of the variety of target groups in universal interventions compared to selective and indicated interventions. Where selective programs concentrate not only on parents and could involve both the family and children, universal interventions mostly focus on the parents alone.

Selective studies, which generally focus on family relationships (FR) and positive parenting (PP), have been investigated by meta-analyses. Moreover, the higher incidence of effective selective and indicated prevention programs in family settings can also be attributed to the fact that universal programs reflect a higher level of heterogeneity and thus measuring their effectiveness is more difficult by meta-analysis studies, which in turn translates into underreported (73, 74).

When provided to those older than 19 years, selective interventions that follow a technology-based model are more effective. Such interventions, however, have commonly been performed to reduce or regulate the use of alcohol in universities and did not target total abstinence. In other words, brief intervention and short-term motivational interviews are mostly intended to reduce harm, rather than primary prevention (67). This finding leads to the impression that perhaps limited or non-effectiveness of primary prevention programs for alcohol is related to the legalized status of the substance. For comparison, we found that while the availability of cannabis in countries where it is legalized has not been shown to result in increased use, universal interventions have also been shown to be non-effective when compared to a control group (78). Therefore, it can be argued that universal, but not selective interventions, are not effective against legalized substances. In the case of smoking, however, that is also legalized, universal primary prevention has shown to be effective. Perhaps the difference between cigarette smoking, as a legal substance, and alcohol and cannabis is about their attitudinal difference. While it is widely accepted that smoking has severe negative consequences, in the case of alcohol and cannabis, there is a common belief that they have at least some benefits.

5.2. Theoretical Models

Although limited or no reports are available on interventions designed based on a social model for other substances, they are effective when used for tobacco prevention. More specifically, social learning-based interventions designed for school settings for those aged 11 to 18 years and those designed for community settings with the target group of older than 11 are shown to be effective. One, therefore, could say that interventions based on the social learning model, regardless of the age of the target group, are effective. Although the social learning model has been criticized for providing the role modeling of substance use behavior, rather than abstaining, our review showed that when social learning programs include negative consequences of substances, the desired effect of not initiating substances outweighs the unwanted learning of the behavior of using substances. Furthermore, self-efficacy and self-belief in one's abilities can also have positive effects on substance use prevention, despite being exposed to parents and peers who use substances.

It has been shown that health education-based interventions provided to those aged 11 to 18 years old can be effective in school settings. Likewise, interventions based on social competence are also shown to be effective in the age group of 11 - 18, when performed in a school context. However, neither of the two models have shown to be effective in the age group of 5 - 10 as long as they are performed exclusively. Yet, when health education was combined with social-competence or social-influence programs in a school context the objective of primary prevention is achieved. Interventions based on health education are not limited to the prevention of alcohol and substances and have effectively been used for the prevention of violence, high-risk sexual behaviors, and delinquent behaviors either combined with substance use prevention or else.

The commonly observed effectiveness of health education models in school settings is a reminder of the above-mentioned observation that universal models have generally been shown to be effective when performed in a school setting. The effectiveness of universal models in school settings might be attributed to different factors including (a) the presumption and receptive preparedness of students for being exposed to new ideas in schools; (b) interaction with peers for shaping permanent social and behavioral characteristics is at its highest capacity during school age; (c) the school and its programs are considered as a template for social norms and values by students; (d) the students are prepared to face and accept new rules and regulations; (e) prescreening of school staff for appropriate behaviors and background provides a more acceptable social milieu of role models who already follow the contents and objectives of prevention programs; and (f) compared to selective and indicated substance use interventions that commonly result in stigmatization of the target group, the comprehensive and buffering nature of the school prevents the students from such consequence.

Interestingly, universal models seem to be less effective or ineffective in the age group of 5 - 10, even in a school setting. The observation that health education programs are not effective in the age group of 5 - 10 reveals the fact that if the effect of health education is by the means of fear of death, this won't be the case for the age group that has not yet formulated a general concept of death. Another explanation is focused on the bright side of health and promoted health conditions that result in better fulfillment of life. In addition, it is also an advanced concept that is difficult to understand for 5 - 10 years old children. Moreover, it has been observed that in the age group of 5 - 10, interventions based on the social influence that are performed either in a school context or a community setting cannot be effective alone and only show effectiveness when they are combined with other models such as health education or social competence. The reason that the solo model is not effective in students aged 5 - 10 years could again be a result of an age-related psychological development.

Compared to selective programs, universal programs are less complicated to perform. Therefore, in family settings, one would expect a more frequent practice of universal programs. This was not, however, shown in systematic reviews included in this study. By definition, when the family is not problematic, substance use prevention would become a universal category intervention. Interestingly, it appears that parents are less motivated to participate in universal prevention programs (74). This lack of interest of parents from non-problematic families might reflect their concern that by participation in substance use prevention programs and by being exposed to new information about substances, their children might become more vulnerable to using substances. Universal interventions intended to prevent alcohol consumption are an exception to this pattern. It appears that as alcohol use is normal behavior and parents would logically anticipate the potential for alcohol consumption to become problematic for their children in the future, they actively participate in such programs, which translates into the effectiveness of the program (79).

The information model and problem-solving have been less effective when exclusively performed in a family setting. However, interventions focusing on family relations and bonding have been reported to be effective when performed in family settings; a condition that fits under the category of selective interventions. Therefore, one may conclude that although universal interventions are more or less effective in different intervention settings when performed in the family setting, they appear to be less effective (73). Aside from universal programs, selective programs focused on problem-solving were also shown to be less effective. It appears that, much further to the skills training in those programs, problem-solving is influenced by a wide range of social and cultural factors from the background. Perhaps the lack of coverage of high heterogeneity of such social and cultural factors in problematic families has played a role in selective programs being less effective (74).

3.3. Duration of Interventions

The school setting seems to be an attractive context to prevention researchers. Therefore, a wide range of prevention programs with different durations have been performed in that setting. However, we couldn't identify any association between duration of intervention and effectiveness of programs reported in systematic reviews; hence, keeping the field open to future exquisite duration-based effectiveness studies. Interestingly, some systematic reviews found an inverse relationship between the duration of programs and their effectiveness. However, this pattern was not examined by meta-analyses. Therefore, the validity of such descriptions by systematic reviews needs further examination. Discussing the duration of interventions and follow-up of their effects, the literature is silent on optimal dose-duration of prevention programs. There is also a lack of reference to the expected durability of effects of an intervention, with an unproven implicit assumption that repeated boosters will continuously keep the effects alive. Perhaps based on age group and other related factors, a person needs different types of substance use preventions in different settings throughout one's phases of life. The observation that in those older than 18 it is the indicated prevention program type with a higher focus on harm reduction with more effectiveness is a sign of the age-related transformation of prevention needs. In general, the factor of duration in substance use prevention, so far, seems to suffer from adequate substance.

3.4. Providers of Interventions

Interventions were more effective in most contexts only when they provided by a specialist or a trained person who was an external educator to the intervention setting. In school and family settings, prevention interventions were shown to be more effective when provided directly by a specialist or by specialists accompanied by usual providers such as teachers. Interventions with a combined approach of family-based and computer-based were more effective than unparalleled approaches. Alcohol prevention programs for those older than 18 were effective either when performed on a long-term basis and provided in person by a specialist or when they were short-term but computer-based. When interventions were provided on a co-provider basis such as combinations of specialist-teacher, peer-teacher, teacher-parents, and specialist-parents, they were more effective than when provided by single providers. A reverse effect is reported (ie, increased substance use) for interventions focused on high-risk youth that are provided solely by peers. The observation on providers of prevention programs shows that prevention programs require higher knowledge and skills and may not be left to untrained lay providers. In addition to expert skills, the observation that a co-provider model increases the effectiveness of the program leads to an impression that the presence of a co-provider increases the internal consistency and structured format of the intervention. The fact that computer-based programs showed significant effectiveness, even in interventions with shorter durations, might reflect the role of internal consistency and structural orderliness of interventions, which is a key feature of computer-based programs (80, 81).

Our study revealed that in some important areas, such as universal interventions in family settings and in those aged more than 18, community-based interventions intended to prevent alcohol and cannabis, and technology-based intervention designed to prevent tobacco and cannabis, there is a scarcity of evidence, particularly during the past decade. The school setting is the most common context used for the implementation of substance use prevention programs with effective outcomes. However, the school setting is now in a transitional phase of replacing much in-person training with online programs. It is, therefore, important to think and study alternatives to this effective context for the future. Another suffering of the substance use prevention literature, as reflected in most systematic reviews, is the high heterogeneity of the studies. Therefore, most of such reviews did not provide a meta-analysis with common denominators on the effectiveness of interventions. We, therefore, suggest the need for further classification of preventive studies such as the template we used in our study.

Evaluating the effectiveness of substance use prevention programs is difficult because their evaluation should be according to the time frame that participants face the risk of substance use or the settings where substance use is at higher risk. Therefore, many short-term effectiveness evaluations of substance use prevention interventions are far from strong evidence. Even with this shortcoming, effectiveness studies are not yet standardized. We have proposed a template that can be used to increase comparability and homogeneity of effectiveness studies.

Acknowledgements

References

  • 1.

    United Nations Office on Drugs and Crime. World Drug Report. Vienna, Austria: United Nations Office on Drugs and Crime; 2019.

  • 2.

    United Nations Office on Drugs and Crime. World Drug Report. Vienna, Austria: United Nations Office on Drugs and Crime; 2018.

  • 3.

    Strashny A. Age of substance use initiation among treatment admissions aged 18 to 30. The CBHSQ Report. Rockville (MD): Substance Abuse and Mental Health Services Administration (US); 2014.

  • 4.

    Bakirhan A, Sahiner SY, Sahiner V, Çeti̇n Ş, Balcı Şengül MC, Kuş S, et al. 9th International Congress on Psychopharmacology & 5th International Symposium on Child and Adolescent Psychopharmacology. Psychiatr Clin Psychopharmacol. 2017;27(sup1):47-84. https://doi.org/10.1080/24750573.2017.1308707.

  • 5.

    King KM, Chassin L. A prospective study of the effects of age of initiation of alcohol and drug use on young adult substance dependence. J Stud Alcohol Drugs. 2007;68(2):256-65. [PubMed ID: 17286344]. https://doi.org/10.15288/jsad.2007.68.256.

  • 6.

    Dawson DA, Goldstein RB, Chou SP, Ruan WJ, Grant BF. Age at first drink and the first incidence of adult-onset DSM-IV alcohol use disorders. Alcohol Clin Exp Res. 2008;32(12):2149-60. [PubMed ID: 18828796]. [PubMed Central ID: PMC2760820]. https://doi.org/10.1111/j.1530-0277.2008.00806.x.

  • 7.

    Chen CY, Storr CL, Anthony JC. Early-onset drug use and risk for drug dependence problems. Addict Behav. 2009;34(3):319-22. [PubMed ID: 19022584]. [PubMed Central ID: PMC2677076]. https://doi.org/10.1016/j.addbeh.2008.10.021.

  • 8.

    Lopez-Quintero C, Perez de los Cobos J, Hasin DS, Okuda M, Wang S, Grant BF, et al. Probability and predictors of transition from first use to dependence on nicotine, alcohol, cannabis, and cocaine: Results of the National Epidemiologic Survey on Alcohol and Related Conditions (NESARC). Drug Alcohol Depend. 2011;115(1-2):120-30. [PubMed ID: 21145178]. [PubMed Central ID: PMC3069146]. https://doi.org/10.1016/j.drugalcdep.2010.11.004.

  • 9.

    Friedman AS, Terras A, Zhu W, McCallum J. Depression, negative self-image, and suicidal attempts as effects of substance use and substance dependence. J Addict Dis. 2004;23(4):55-71. [PubMed ID: 15339714]. https://doi.org/10.1300/J069v23n04_05.

  • 10.

    National Institute on drug Abuse. Handout Physical and Psychological Effects of Substance Use. National Institute on drug Abuse; 2004.

  • 11.

    Sussman S, Skara S, Ames SL. Substance abuse among adolescents. Subst Use Misuse. 2008;43(12-13):1802-28. [PubMed ID: 19016166]. https://doi.org/10.1080/10826080802297302.

  • 12.

    Anglin DM, Corcoran CM, Brown AS, Chen H, Lighty Q, Brook JS, et al. Early cannabis use and schizotypal personality disorder symptoms from adolescence to middle adulthood. Schizophr Res. 2012;137(1-3):45-9. [PubMed ID: 22325079]. [PubMed Central ID: PMC3591468]. https://doi.org/10.1016/j.schres.2012.01.019.

  • 13.

    National Institutes of Health. Health consquence of drug misuse, Mental health effect: Center for Behavioral Health Statics and Quality, National Survey on Drug use and Health Center for Behavioral Health Statics and Quality; 2015. 2020. Available from: https://www.drugabuse.gov/publications/health-consequences-drug-misuse/mental-health-effects.

  • 14.

    Degenhardt L, Charlson F, Ferrari A, Santomauro D, Erskine H, Mantilla-Herrara A, et al. The global burden of disease attributable to alcohol and drug use in 195 countries and territories, 1990–2016: a systematic analysis for the Global Burden of Disease Study 2016. The Lancet Psychiat. 2018;5(12):987-1012. https://doi.org/10.1016/s2215-0366(18)30337-7.

  • 15.

    Rehm J, Shield KD. Global Burden of Disease and the Impact of Mental and Addictive Disorders. Curr Psychiatry Rep. 2019;21(2):10. [PubMed ID: 30729322]. https://doi.org/10.1007/s11920-019-0997-0.

  • 16.

    Nimh Multisite HIV/STD Prevention Trial for African American Couples Group. Methodological overview of an African American couple-based HIV/STD prevention trial. J Acquir Immune Defic Syndr. 2008;49 Suppl 1:S3-14. [PubMed ID: 18724188]. [PubMed Central ID: PMC2910525]. https://doi.org/10.1097/QAI.0b013e3181842570.

  • 17.

    United Nations. World Population Prospects 2019. United Nations: Department of Economic and Social Affairs Population Dynamics; 2019.

  • 18.

    Gore FM, Bloem PJ, Patton GC, Ferguson J, Joseph V, Coffey C, et al. Global burden of disease in young people aged 10–24 years: a systematic analysis. The Lancet. 2011;377(9783):2093-102. https://doi.org/10.1016/s0140-6736(11)60512-6.

  • 19.

    Sawyer SM, Afifi RA, Bearinger LH, Blakemore S, Dick B, Ezeh AC, et al. Adolescence: a foundation for future health. The Lancet. 2012;379(9826):1630-40. https://doi.org/10.1016/s0140-6736(12)60072-5.

  • 20.

    Dick B, Ferguson BJ. Health for the world's adolescents: a second chance in the second decade. J Adolesc Health. 2015;56(1):3-6. [PubMed ID: 25530601]. https://doi.org/10.1016/j.jadohealth.2014.10.260.

  • 21.

    Erskine HE, Moffitt TE, Copeland WE, Costello EJ, Ferrari AJ, Patton G, et al. A heavy burden on young minds: the global burden of mental and substance use disorders in children and youth. Psychol Med. 2015;45(7):1551-63. [PubMed ID: 25534496]. [PubMed Central ID: PMC5922255]. https://doi.org/10.1017/S0033291714002888.

  • 22.

    Degenhardt L, Stockings E, Patton G, Hall WD, Lynskey M. The increasing global health priority of substance use in young people. The Lancet Psychiatry. 2016;3(3):251-64. https://doi.org/10.1016/s2215-0366(15)00508-8.

  • 23.

    Mack AH. Adolescent Substance Use: America's #1 Public Health Problem. Yearbook of Psychiatry and Applied Mental Health. 2012;2012(0):39-40. https://doi.org/10.1016/j.ypsy.2011.09.030.

  • 24.

    Leukefeld CG, Bukoski WJ. Drug abuse prevention intervention research: methodological issues: US Department of Health and Human Services. Public Health Ser. 1991. https://doi.org/10.1037/e496282006-001.

  • 25.

    United Nation Office on Drug and Crime. International Standards on Drug Use Prevention; Vienna. UNODC. 2015.

  • 26.

    Springer F, Phillips JL. The IOM model: A tool for prevention planning and implementation. Prev Tact. 2006;8(13):1-8.

  • 27.

    Sloboda Z, Bukoski WJ. Handbook of Drug Abuse Prevention. Springer; 2007. https://doi.org/10.1007/0-387-35408-5.

  • 28.

    Springer JF, Phillips J. The Institute of Medicine Framework and its implication for the advancement of prevention policy, programs and practice. California: Community Prevention Initiative; 2007.

  • 29.

    Bosworth K. Prevention science in school settings: Complex relationships and processes. Springer; 2015. https://doi.org/10.1007/978-1-4939-3155-2.

  • 30.

    Scheier LM. Handbook of adolescent drug use prevention: Research, intervention strategies, and practice. Am Psychol Assoc; 2015. https://doi.org/10.1037/14550-000.

  • 31.

    Hawkins JD, Catalano RF, Miller JY. Risk and protective factors for alcohol and other drug problems in adolescence and early adulthood: implications for substance abuse prevention. Psychol Bull. 1992;112(1):64-105. [PubMed ID: 1529040]. https://doi.org/10.1037/0033-2909.112.1.64.

  • 32.

    Hawkins JD, Catalano RF, Arthur MW. Promoting science-based prevention in communities. Addict Behav. 2002;27(6):951-76.

  • 33.

    Bewick BM, Trusler K, Barkham M, Hill AJ, Cahill J, Mulhern B. The effectiveness of web-based interventions designed to decrease alcohol consumption: A systematic review. Prev Med. 2008;47(1):17-26. [PubMed ID: 18302970]. https://doi.org/10.1016/j.ypmed.2008.01.005.

  • 34.

    Canale N, Vieno A, Santinello M, Chieco F, Andriolo S. The efficacy of computerized alcohol intervention tailored to drinking motives among college students: a quasi-experimental pilot study. Am J Drug Alcohol Abuse. 2015;41(2):183-7. [PubMed ID: 25700006]. https://doi.org/10.3109/00952990.2014.991022.

  • 35.

    Gulliver A, Farrer L, Chan JK, Tait RJ, Bennett K, Calear AL, et al. Technology-based interventions for tobacco and other drug use in university and college students: a systematic review and meta-analysis. Addict Sci Clin Pract. 2015;10:5. [PubMed ID: 25928221]. [PubMed Central ID: PMC4422468]. https://doi.org/10.1186/s13722-015-0027-4.

  • 36.

    Mason M, Ola B, Zaharakis N, Zhang J. Text messaging interventions for adolescent and young adult substance use: a meta-analysis. Prev Sci. 2015;16(2):181-8. [PubMed ID: 24930386]. https://doi.org/10.1007/s11121-014-0498-7.

  • 37.

    West R. EMCDDA Insights: Models of addiction. European Monitoring Centre for Drugs and Drug Addiction; 2013.

  • 38.

    Sussman S, Ames SL. Drug Abuse: Concepts, prevention, and cessation. Cambridge University Press; 2008. https://doi.org/10.1017/cbo9780511500039.

  • 39.

    Skara S, Sussman S. A review of 25 long-term adolescent tobacco and other drug use prevention program evaluations. Prev Med. 2003;37(5):451-74. https://doi.org/10.1016/s0091-7435(03)00166-x.

  • 40.

    Leukefeld CG, Bukoski WJ. Prevention Evaluation Research Methods: Findings and Consensus. Drug Abuse Prev Inter Res. 1991:264-71. https://doi.org/10.1037/e496282006-016.

  • 41.

    Carson KV, Brinn MP, Labiszewski NA, Esterman AJ, Chang AB, Smith BJ. Community interventions for preventing smoking in young people. Cochrane Database Syst Rev. 2011;(7). CD001291. [PubMed ID: 21735383]. https://doi.org/10.1002/14651858.CD001291.pub2.

  • 42.

    Georgie JM, Sean H, Deborah MC, Matthew H, Rona C. Peer-led interventions to prevent tobacco, alcohol and/or drug use among young people aged 11-21 years: a systematic review and meta-analysis. Addiction. 2016;111(3):391-407. [PubMed ID: 26518976]. [PubMed Central ID: PMC4833174]. https://doi.org/10.1111/add.13224.

  • 43.

    Aromataris E, Fernandez R, Godfrey CM, Holly C, Khalil H, Tungpunkom P. Summarizing systematic reviews: methodological development, conduct and reporting of an umbrella review approach. Int J Evid Based Healthc. 2015;13(3):132-40. [PubMed ID: 26360830]. https://doi.org/10.1097/XEB.0000000000000055.

  • 44.

    Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. [PubMed ID: 28935701]. https://doi.org/10.1136/bmj.j4008.

  • 45.

    Pan W, Bai H. A multivariate approach to a meta-analytic review of the effectiveness of the D.A.R.E. program. Int J Environ Res Public Health. 2009;6(1):267-77. [PubMed ID: 19440283]. [PubMed Central ID: PMC2672328]. https://doi.org/10.3390/ijerph6010267.

  • 46.

    Porath-Waller AJ, Beasley E, Beirness DJ. A meta-analytic review of school-based prevention for cannabis use. Health Educ Behav. 2010;37(5):709-23. [PubMed ID: 20522782]. https://doi.org/10.1177/1090198110361315.

  • 47.

    Foxcroft DR, Tsertsvadze A. Universal family-based prevention programs for alcohol misuse in young people. Cochrane Database Syst Rev. 2011;(9). CD009308. [PubMed ID: 21901733]. https://doi.org/10.1002/14651858.CD009308.

  • 48.

    Foxcroft DR, Tsertsvadze A. Universal school-based prevention programs for alcohol misuse in young people. Cochrane Database Syst Rev. 2011;(5). CD009113. [PubMed ID: 21563171]. https://doi.org/10.1002/14651858.CD009113.

  • 49.

    Foxcroft DR, Tsertsvadze A. Universal multi-component prevention programs for alcohol misuse in young people. Cochrane Database Syst Rev. 2011;(9). CD009307. [PubMed ID: 21901732]. https://doi.org/10.1002/14651858.CD009307.

  • 50.

    Korczak D, Steinhauser G, Dietl M. Prevention of alcohol misuse among children, youths and young adults. GMS Health Technol Assess. 2011;7.

  • 51.

    Broning S, Kumpfer K, Kruse K, Sack PM, Schaunig-Busch I, Ruths S, et al. Selective prevention programs for children from substance-affected families: a comprehensive systematic review. Subst Abuse Treat Prev Policy. 2012;7:23. [PubMed ID: 22691221]. [PubMed Central ID: PMC3490747]. https://doi.org/10.1186/1747-597X-7-23.

  • 52.

    Norberg MM, Kezelman S, Lim-Howe N. Primary prevention of cannabis use: a systematic review of randomized controlled trials. PLoS One. 2013;8(1). e53187. [PubMed ID: 23326396]. [PubMed Central ID: PMC3543459]. https://doi.org/10.1371/journal.pone.0053187.

  • 53.

    Thomas RE, McLellan J, Perera R. School-based programmes for preventing smoking. Cochrane Database Syst Rev. 2013;(4). CD001293. [PubMed ID: 23633306]. [PubMed Central ID: PMC7028068]. https://doi.org/10.1002/14651858.CD001293.pub3.

  • 54.

    Faggiano F, Minozzi S, Versino E, Buscemi D. Universal school-based prevention for illicit drug use. Cochrane Database Syst Rev. 2014;(12). CD003020. [PubMed ID: 25435250]. [PubMed Central ID: PMC6483627]. https://doi.org/10.1002/14651858.CD003020.pub3.

  • 55.

    Jones LM, Mitchell KJ, Walsh WA. A Systematic Review of Effective Youth Prevention Education: Implications for Internet Safety Education. CCRC. 2014.

  • 56.

    Strom HK, Adolfsen F, Fossum S, Kaiser S, Martinussen M. Effectiveness of school-based preventive interventions on adolescent alcohol use: a meta-analysis of randomized controlled trials. Subst Abuse Treat Prev Policy. 2014;9(1):48. [PubMed ID: 25495012]. [PubMed Central ID: PMC4274678]. https://doi.org/10.1186/1747-597X-9-48.

  • 57.

    Espada JP, Gonzálvez MT, Orgilés M, Lloret D, Guillén-Riquelme A. Meta-analysis of the effectiveness of school substance abuse prevention programs in Spain. Psicothema. 2015;27(1):5-12. https://doi.org/10.7334/psicothema2014.106.

  • 58.

    Foxcroft DR, Moreira MT, Almeida Santimano NM, Smith LA, Foxcroft DR. Social norms information for alcohol misuse in university and college students. Cochrane Database Sys Rev. 2015. https://doi.org/10.1002/14651858.CD006748.pub3.

  • 59.

    Hennessy EA, Tanner-Smith EE. Effectiveness of brief school-based interventions for adolescents: a meta-analysis of alcohol use prevention programs. Prev Sci. 2015;16(3):463-74. [PubMed ID: 25294110]. [PubMed Central ID: PMC4636019]. https://doi.org/10.1007/s11121-014-0512-0.

  • 60.

    Thomas RE, Baker PR, Thomas BC, Lorenzetti DL. Family-based programmes for preventing smoking by children and adolescents. Cochrane Database Syst Rev. 2015;(2). CD004493. [PubMed ID: 25720328]. [PubMed Central ID: PMC6486099]. https://doi.org/10.1002/14651858.CD004493.pub3.

  • 61.

    Shackleton N, Jamal F, Viner RM, Dickson K, Patton G, Bonell C. School-Based Interventions Going Beyond Health Education to Promote Adolescent Health: Systematic Review of Reviews. J Adolesc Health. 2016;58(4):382-96. [PubMed ID: 27013271]. https://doi.org/10.1016/j.jadohealth.2015.12.017.

  • 62.

    De Cock B, Bekkering T, Hannes K. Evaluating the potential negative effects of school-based prevention programs aiming to reduce alcohol and drug misuse in adolescents: A systematic review of research articles prior to 2013. KU Leuven. 2017.

  • 63.

    Hefler M, Liberato SC, Thomas DP. Incentives for preventing smoking in children and adolescents. Cochrane Database Syst Rev. 2017;6. CD008645. [PubMed ID: 28585288]. [PubMed Central ID: PMC6481498]. https://doi.org/10.1002/14651858.CD008645.pub3.

  • 64.

    Hodder RK, Freund M, Wolfenden L, Bowman J, Nepal S, Dray J, et al. Systematic review of universal school-based 'resilience' interventions targeting adolescent tobacco, alcohol or illicit substance use: A meta-analysis. Prev Med. 2017;100:248-68. [PubMed ID: 28390835]. https://doi.org/10.1016/j.ypmed.2017.04.003.

  • 65.

    Lize SE, Iachini AL, Tang W, Tucker J, Seay KD, Clone S, et al. A Meta-analysis of the Effectiveness of Interactive Middle School Cannabis Prevention Programs. Prev Sci. 2017;18(1):50-60. [PubMed ID: 27785662]. [PubMed Central ID: PMC5680036]. https://doi.org/10.1007/s11121-016-0723-7.

  • 66.

    Newton NC, Champion KE, Slade T, Chapman C, Stapinski L, Koning I, et al. A systematic review of combined student- and parent-based programs to prevent alcohol and other drug use among adolescents. Drug Alcohol Rev. 2017;36(3):337-51. [PubMed ID: 28334456]. https://doi.org/10.1111/dar.12497.

  • 67.

    Boumparis N, Loheide-Niesmann L, Blankers M, Ebert DD, Korf D, Schaub MP, et al. Short- and long-term effects of digital prevention and treatment interventions for cannabis use reduction: A systematic review and meta-analysis. Drug Alcohol Depend. 2019;200:82-94. [PubMed ID: 31112834]. https://doi.org/10.1016/j.drugalcdep.2019.03.016.

  • 68.

    Ladis BA, Macgowan M, Thomlison B, Fava NM, Huang H, Trucco EM, et al. Parent-Focused Preventive Interventions for Youth Substance Use and Problem Behaviors: A Systematic Review. Res Soc Work Pract. 2018;29(4):420-42. https://doi.org/10.1177/1049731517753686.

  • 69.

    Thomas RE, Baker PR, Lorenzetti D, Thomas RE. Family-based programmes for preventing smoking by children and adolescents. Cochrane Database Sys Rev. 2007. https://doi.org/10.1002/14651858.CD004493.pub2.

  • 70.

    Thomas RE, Lorenzetti DL, Spragins W. Systematic review of mentoring to prevent or reduce alcohol and drug use by adolescents. Acad Pediatr. 2013;13(4):292-9. [PubMed ID: 23830016]. https://doi.org/10.1016/j.acap.2013.03.007.

  • 71.

    Emmers E, Bekkering GE, Hannes K. Prevention of alcohol and drug misuse in adolescents: An overview of systematic reviews. Nord Stud Alcohol Drugs. 2015;32(2):183-98. https://doi.org/10.1515/nsad-2015-0019.

  • 72.

    Allen ML, Garcia-Huidobro D, Porta C, Curran D, Patel R, Miller J, et al. Effective Parenting Interventions to Reduce Youth Substance Use: A Systematic Review. Pediatrics. 2016;138(2). [PubMed ID: 27443357]. [PubMed Central ID: PMC4960727]. https://doi.org/10.1542/peds.2015-4425.

  • 73.

    Van Ryzin MJ, Roseth CJ, Fosco GM, Lee YK, Chen IC. A component-centered meta-analysis of family-based prevention programs for adolescent substance use. Clin Psychol Rev. 2016;45:72-80. [PubMed ID: 27064553]. [PubMed Central ID: PMC4856569]. https://doi.org/10.1016/j.cpr.2016.03.007.

  • 74.

    de Vicente MV, Brage LB, Socías MDCO, Fernández JAA. Meta-analysis of family-based selective prevention programs for drug consumption in adolescence. Psicothema. 2017;29(3):299-305. https://doi.org/10.7334/psicothema2016.275.

  • 75.

    Smedslund G, Wollscheid S, Fang L, Nilsen W, Steiro A, Larun L. Effects of early, computerized brief interventions on risky alcohol use and risky cannabis use among young people. Campbell Sys Rev. 2017;13(1):1-192. https://doi.org/10.4073/csr.2017.6.

  • 76.

    Sundstrom C, Blankers M, Khadjesari Z. Computer-Based Interventions for Problematic Alcohol Use: A Review of Systematic Reviews. Int J Behav Med. 2017;24(5):646-58. [PubMed ID: 27757844]. [PubMed Central ID: PMC5608865]. https://doi.org/10.1007/s12529-016-9601-8.

  • 77.

    Griffin KW, Botvin GJ. Evidence-based interventions for preventing substance use disorders in adolescents. Child Adolesc Psychiatr Clin N Am. 2010;19(3):505-26. [PubMed ID: 20682218]. [PubMed Central ID: PMC2916744]. https://doi.org/10.1016/j.chc.2010.03.005.

  • 78.

    Zuckermann AME, Gohari MR, de Groh M, Jiang Y, Leatherdale ST. The role of school characteristics in pre-legalization cannabis use change among Canadian youth: implications for policy and harm reduction. Health Educ Res. 2020;35(4):297-305. [PubMed ID: 32623462]. https://doi.org/10.1093/her/cyaa018.

  • 79.

    Negreiros J, Brage LB, de Vicente MV, Carmo R, Da Gama J. A Systematic review of participation in prevention family programs. Pedagog Soc. 2019;(34):6.

  • 80.

    Nation M, Crusto C, Wandersman A, Kumpfer KL, Seybolt D, Morrissey-Kane E, et al. What works in prevention. Principles of effective prevention programs. Am Psychol. 2003;58(6-7):449-56. [PubMed ID: 12971191]. https://doi.org/10.1037/0003-066x.58.6-7.449.

  • 81.

    Amesty E, Páez D. Using project-based learning with Venezuelan teachers to enhance teacher attitudes about school-based drug abuse prevention: A control-group comparison study. Psychol Sch. 2018;55(8):969-81. https://doi.org/10.1002/pits.22159.