The Genetics of Opioid Addiction Risk Evaluation Tool (GREAT) for Treatment Response in Methadone Patients

1. Background

Opioid agonist treatment (OAT) is used to relieve physical withdrawal symptoms and reduce illicit opioid use in patients with opioid-dependence (1). Although its reported effectiveness varies across studies, methadone maintenance treatment is the most widely used form of OAT (2). Methadone’s reported effectiveness for reducing illicit opioid use ranges between 20% and 70% (3-5). This large variation may be in part due to the inconsistent definitions and measurement of response to treatment. While some studies evaluate response to MMT by measuring the number of days of illicit heroin or opioid use in the last month (6, 7), others choose to report the percentage of patients using illicit opioids in the last 30 (8) or 7 days (9). Measurement of substance use is also subject to additional variation with some studies relying on self-report (8-10), and others using various tools to assess the severity of substance use behavior (11).

Addiction severity measurement tools capture the behavioral and social context of substance use, providing a more complete picture of how participants are functioning across different life spheres. While addiction is characterized as a medical disorder that can be treated by health care practitioners within healthcare setting, it often has a direct relationship to a patient’s social environment. A combination of factors including social stability, absence of illicit opioid use, reduced risk taking behavior (such as criminality), as well as physical and psychological health impact response to treatment are important risk factors to capture with measurement scales (12, 13).

The addiction severity index (ASI) (13) and maudsley addiction profile (MAP) (12) are the most common tools used to assess the impact of addiction. The ASI was created in 1980 for physicians and health care practitioners in the field of addiction medicine to evaluate treatment outcomes and clinical profiles for addiction patients (13). The tool assesses a wide range of outcomes for different domains of substance abuse (i.e. physical, social and psychological domains), but was designed to be administered by a trained interviewer and lacks concurrent reliability when self-administered (13). The MAP was developed in 1998 for the UK addiction patient population (12). The purpose of this tool, alike with the ASI, is to evaluate treatment outcomes within global domains such as substance use, physical and psychological health, personal/social functioning as well as health risk behavior (12). Contrary to the ASI, the MAP can be self-administered and is available online at no cost.

Both the MAP and ASI appear as feasible alternatives to the strict assessment of urine toxicology results, however both tools suffer from their generic quality, where they fall short in capturing important treatment outcomes and risk factors specific to methadone patients such as pain management, social environment, access to illicit prescription substances, retention in treatment, methadone dose, onset age of opioid abuse, and whether opioids were originally obtained from physicians prescriptions. In addition, the MAP and ASI evaluate treatment response over a short time frame, assessing patients within a 30-day context. While this time-frame may be used to prevent recall bias from impacting research findings, this time-frame may be too conservative to evaluate treatment outcomes for MMT patients since opioid addiction is a chronic disorder with the average MMT duration being 2 years (1). While the MAP and ASI are generally applied prior to and at regular intervals during and after treatment, in an effort to explore the changes in substance use disorder patients across multiple domains (13), a validated tool to assess patient’s addiction severity and progression through methadone treatment, accounting for factors influencing health outcomes specific to a methadone population is both valuable and non-existent.

The purpose of this study is to develop a tool that assesses addiction severity and response to methadone treatment across multiple physical and psychological domains as well as to identify high-risk patients for continued opioid abuse. We propose the addition of new relevant items for opioid-dependent patients as well as an amendment to the original MAP items that best predict substance use outcomes in the MMT patient population. We have chosen to apply these modifications to the MAP in light of its availability at no cost and feasible administration design.

2. Objectives

Our aim is to provide an accessible tool that can be applied easily with minimal costs to researchers and clinicians. The addition of items that assess the presence and severity of chronic pain, retention in treatment, methadone dose, severity of comorbid psychiatric symptoms, onset age of opioid abuse, whether opioids were originally obtained from physicians prescriptions, and the accessibility to illicit substances through family, friends, or acquaintances will serve to more adequately capture a patient’s response to MMT.

3. Patients and Methods

To develop a new instrument that assesses addiction severity among MMT patients we modified the MAP. Modifications to the MAP which included the addition of new items were made using: (1) an analysis of MAP scoring among opioid dependent patients from the Genetics of Opioid Addiction (GENOA) research collaborative, (14) consultation with experts in the field of addiction medicine (item-generation and assessment of the instrument’s face validity), and (2) a comprehensive literature review. To assess the psychometric properties of the new tool we preformed a generalizability (G) study. We lastly completed the criterion validation of the new addiction severity instrument using the GENOA sample of opioid dependent patients.

3.1. Genetics of Opioid Addiction (GENOA) Research Collaboration Methods

The GENOA research collaborative is a research initiative partnership between the Ontario addiction teatment center (OATC) and the population genomics program at McMaster University. GENOA is a prospective cohort investigation approved by the Hamilton integrated research ethics board. The GENOA research collaborative aims to evaluate the genetic determinants of methadone response. Participants were recruited from 10 clinical sites throughout southern Ontario, where they were then followed for 12-months. GENOA pilot study and methods have been described previously (15).

Information on medical history, methadone dose, duration on MMT, number of MMT treatments, and original source of opioid use was collected during the initial baseline assessment. The baseline assessment includes the M.I.N.I. International Neuropsychiatric Interview version 6.0 (16), which has been validated in the assessment of psychopathology. The baseline assessment also included instruments such as the MAP and brief pain inventory (BPI) to capture the severity and amount of interference pain has on a patient’s daily activities. Weekly urine drug screens were performed throughout the study period as part of routine clinical care at the OATC recruitment sites using the Innovacon, Inc One Step Drug Screen methods.

3.2. Item Generation

The important subdomains included in this tool were identified through collaborative discussions between psychiatrists, addiction medicine physicians, counselors experienced in MMT, methodologists and epidemiologists. These brainstorming sessions resulted in the identification of four subdomains: substance use, physical and psychological health, health-risk behavior as well as personal and social functioning. Using patient’s MAP scores from the GENOA investigation we determined items that most highly correspond with continued opioid use, as measured through urine toxicology screening. We also generated new items through an extensive literature review in addition to consultation with addiction medicine experts on the characteristics of “high-risk” methadone patients. Once we identified all relevant MAP/clinical/demographic items, we constructed our tool and distributed this tool to physicians working at the Ontario addiction treatment centers (OATC) in an effort to determine the face validity and acceptability of the instrument.

Exploratory factor analysis is often employed to identify domains during questionnaire development. Factor analysis is able to recognize variability among observed correlated variables. Within this study we modified an existing tool, thus utilizing domains previously identified by factor analysis and expert opinion. The extent of our modifications consisted largely of question addition and amendment. We felt it methodologically questionable to perform an exploratory factor analysis when utilizing questions from previously established domains.

3.3. Statistical Item Generation Results

We performed univariate analyses of patients’ responses to MAP items and opioid urine toxicology screening results. This allowed us to identify the most highly correlated MAP items for continued opioid abuse. The MAP scores among GENOA participants are summarized in Appendix B. Each MAP item was evaluated against participants’ opioid urine results using univariable linear regression. The results of the univariate analyses showed the number of days of illicit heroin, benzodiazepine, cocaine, crack and cannabis use as MAP items with high predictive ability for continued opioid abuse (P < 0.05, Appendix B). The results of the univariate analysis showed a higher severity of psychological symptoms are also associated with continued opioid abuse (P < 0.05, Appendix B). When determining the effect of risk-taking behavior on response to MMT, the number of days of injecting drug use in addition to a variable comprising the number of days of injecting drug use multiplied by the number of times per day correlate strongly with continued opioid abuse (P < 0.05, Appendix B). When exploring subdomains such as relationships (social/family), items calculating the frequency of conflict with friends proved significantly correlated with continued opioid abuse (P < 0.05, Appendix B). We tested additional clinical and demographic variables in an effort to generate additional items important for predicting response to MMT. Please refer to Appendix C for a summary of these results.

3.4. Expert Item Generation Results

Important developments arose when we consulted opioid addiction experts on the risk factors they encounter in their practice. The addiction medicine physicians suggested that we include an item that captures the environment patients live in, particularly whether the patient’s partner or close friends are still abusing substances, a factor that the most patients report upon relapse. The physicians also suggested we include items that capture work experience beyond the traditional “paid,” employment, with the greater significance being that we capture a patient’s responsibilities beyond paid work. This prompted us to also include “non-paid,” or “volunteer work,” as employment surrogates.

3.5. Literature Review: Item Generation Results

After carefully reviewing the literature, we identified additional items not captured by the MAP that may prove useful for predicting response to MMT. These items include: age of onset of opioid abuse, methadone dose, and duration on MMT. While not scored as part of the questionnaire, these items serve as important variables that should be captured when applying GREAT tool in a clinical setting.

Studies have demonstrated that longer duration of opioid abuse prior to and/or during treatment is associated with worsening health implications for at-risk populations. The risk of mortality from overdose, violence, infectious disease, and alcohol related causes are substantially higher among opioid addiction patients with long history of drug use (17, 18). One study also noted that patients who continue to use opioids into their 30’s face an unlikely chance of eventual opioid cessation (18). These results suggest that consideration of the age of onset of opioid abuse could prove helpful in the assessment of MMT patient response and prognosis.

Methadone dose is commonly discussed in the literature as a determinant for response to treatment. Although higher doses of methadone are associated with a higher risk of serious side effects, such higher doses are also associated with a positive treatment response (19).

Lastly, duration on OAT or “retention” in treatment is highly correlated with better health outcomes in opioid dependent patients (11, 20-23). The longer patients continued in a treatment program, maintain contact with a health care team and provide evidence of maintaining their will to remain abstinent, the better their treatment outcomes. Patients who leave treatment early or are just beginning MMT are vulnerable to relapse during their initial months often because of variable dosing, breakthrough withdrawal symptoms, and their vulnerable mental state (1).

3.6. Scale Generation

The MAP addresses questions through direct recall over a 30-day timeframe, where patients are asked to recall “how many times they injected drugs,” or “how many times they used heroin,” over a 30 day period. Directly asking patients to recall the number of times they have injected drugs or used a substance is subject to bias, for people’s ability to recall specific past events is often inconsistent (24). Memory recall is especially problematic for chronic conditions resulting in multiple recurrent events (25). Autobiographical memory is not organized by events such the amount of drinking or exercise one participates in, but rather by a hierarchy with prolonged periods at the top (i.e. during my first job I felt sad) (24). Thus, recall of the specific number of times people engage in a particular behavior should be regarded with doubt (24). To overcome such limitations we modified our assessment tool to include responses on a seven point Likert scale. Please refer to Appendix A to view the completed GREAT tool. The responses are ordinal and include seven categorical options. All questions were revised into a 7 point Likert scale format and piloted among addiction medicine experts. Further revisions addressed the language of questions, changing items such as “non-prescription,” to “not prescribed to you.” The experts helped to identify specific response scale descriptions that may limit the scale’s ability to capture variation between subjects. For example, the original response scale for the question: “In the last 3 months how often have you smoked cigarettes,” included categories such as (sometimes: 2 - 5 cigarettes per day, regularly 6 - 10 cigarettes per day, frequently: 1 - 11 pack per day). Addiction experts suggested that the categories would have limited the scale to capture variance between subjects since the majority (GENOA sample: 87%) of patients smoke more than a pack of cigarettes per day. Thus, our inclusion of so many categories below a pack of cigarettes (n = 20 cigarettes) would drive the majority of the participants to select 1 category (all of the time: > 1 pack per day).

The scale has been designed so that patients who score higher on the GREAT will be considered at a higher risk for continued opioid abuse and poorer MMT treatment response. All items have been categorized in an ordinal fashion with 7 being the “high risk” response. For example, we determined the amount of substance abuse by asking, “In the last 3 months, how often have you consumed opioids that were not prescribed to you?” Responses were scaled from 1 to 7, where 1 represented “not at all, 0 times,” and 7 represented “all of the time, every day.” For questions about educational achievement, education represented a surrogate component of socioeconomic status (SES). Lower SES is often equated with increased risk for poor outcomes, therefore the response category labeled 1 represented high educational achievement such as graduate university training, and 7 represented no formal educational training (elementary school or lower). It is suggested that all items on the tool be formatted similarly, reflecting the same scale categories (24).

3.7. Participant Testing

The tool was administered to opioid addiction patients from the GENOA study described above. A random sample of 21 GENOA participants from 4 clinical sites were administered the GREAT tool on 2 occasions with a 5-day interval between tests. The GREAT is intended to be a reliable and efficient self-administered tool for the assessment of a patient’s propensity for continued opioid abuse.

The 5-day interval was selected in an effort to ensure participants would not be influenced by their original answers while also maintaining the same level of addiction severity they had during the original test. Empirical data suggest that a re-test interval of 2 - 14 days is most appropriate for ensuring: 1) the attribute being measured has not significantly altered and 2) a minimal effect change on the magnitude of the reliability coefficient (24).

The criterion validation performed to assess the validity of the substance use domain against patients’ urine toxicology screens was performed on a sample of 117 MMT patients from the GENOA sample. This sample includes the 21 original participants used to test the reliability properties of the tool, using only results from their first test administration.

3.8. Statistical Analyses: Scale Modification, Reliability and Validity

To assess reliability we conducted a generalizability (G) study. G theory allows us to calculate reliability coefficients that acknowledge multiple sources of the errors of variance (24). Error is not only introduced when considering the between subject error, error can also be introduced with each test administration, and G theory allows for the integration of all possible sources of error into a single analysis of variance, and the individual variance components are computed (24). We performed a G study using the G-string software supplied by GENOVA (26). The reliability coefficients constructed from the G-study allowed us to determine the extent we can generalize 1) an individual’s scores across occasions (test-retest reliability), 2) items in one subscale to items in another subscale (internal consistency across subscales), 3) items within subscales (internal consistency within subscales) and 4) and individuals scores across subscales and test-administrations (average reliability).

When determining to what extent we can generalize across items, subscales and test administrations (average reliability), we set all facets of generalization to be random, and used the absolute error co-efficient in an effort to allow us to presume that each item, subscale or test administration is one of a random sample of all possible items, subscales or test-administrations (27). Using the relative error coefficients (Eρ²) would have restricted our ability to generalize results to just the items and parameters we have designed for this study (27).

We performed generalizability analyses for the overall test with all subscales included, providing reliability estimates for internal consistency and test-retest within each subscale. The design of our tool features crossed and nested facets. Our facets include time, subscale and item. Item is nested in subscale, however time and subscales are crossed facets. Our facets of generalization altered between reliability measurements (i.e. internal consistency, test-retest). Our facet of differentiation is the MMT patient, where we are measuring each patient’s addiction severity and level of response to methadone treatment.

There are currently no ways to objectively validate participant responses in the personal/social functioning or health risk behavior domains. These domains were created with the purpose of measuring high-risk behavior (e.g. sharing drug consumption paraphernalia), and social determinants of health such as intimate partner conflict, and criminal behavior. Items within these domains capture the behavioral and social context of participants, providing a more complete picture of how participants are functioning across different life aspects. This information is important to capture, however we are at a loss in identifying an objective method for criterion validation. Thus, we were limited to only performing a criterion validation on the GREAT substance use domain scores.

We performed a criterion validation using participant’s scores from the substance abuse domain of the GREAT using urine toxicology screening as the gold standard for assessing continued opioid abuse. As part of the GENOA protocol participants have provided up to date opioid urine screens, providing access to an objective measure of continued opioid abuse within the 3-month timeframe assessed in the GREAT. The urine data were used to determine whether or not participants had experienced an illicit opioid use in the three months preceding the questionnaire administration. We then used the participants’ 3-month opioid use history as the dependent variable in a logistic regression model. We used the participants’ GREAT scores from the substance abuse domain as the independent variable to determine the direction and strength of association between the GREAT substance domain responses (all answers summed to a maximum of 42 and minimum of 7) and 3-month history of opioid use. This model adjusted for age, sex, duration on MMT (years), and age of onset of opioid abuse, and methadone dose (mg/day). We also re-performed this analysis using three-month history of poly-substance use (THC, benzodiazepine, or cocaine) as the dependent variable. All statistical analysis were performed in STATA Version 13 (28).

4. Results

The GREAT resulted in a 23-item questionnaire, which lead to a score ranging from 23 to 161. The GREAT was administered on 2 occasions (5 day intervals) to 21 MMT patients in the GENOA study. Table 1 summarizes the participants’ demographic characteristics. Participants took an average of 5 minutes to complete the GREAT.

Please refer to Appendix A for the finalized GREAT administered to MMT patients.

4.1. Item Response

Results showed 100% completion for item response among the 117 participants completing the GREAT. We presented histograms of the distribution of participant responses across the 7-category Likert scale in Appendix D. The histograms suggest there is minimal variation in participant response for Substance Use Behavior Subscale items 2 - 4, as well as health risk behavior subscale items 1 - 4. Participants are selecting the extreme categories on these items, with the majority of responses collecting in Likert scale category 1 or 7, suggesting minimal end aversion bias. However, the majority of participants’ GREAT responses are varied along the Likert scale (Appendix D).

4.2. Generalizability Theory

We determined the internal consistency, test-retest, and overall reliability for the GREAT and the individual subscales using G Theory analysis (G-Study). Table 2 summarizes the results of our G-Studies.

Our results showed excellent test-retest reliability for the GREAT (0.95) and its subscales (all ≥ 0.94). Internal consistency across subscales was very low for the GREAT (0.2), indicating we cannot generalize well across subscales, an important finding which also confirmed the existence of subdomains. Internal consistency (ability to generalize well across items) was fair when assessed in subscales (substance use: 0.33, health risk: 0.55, health: 0.54, and personal/social functioning: 0.32). Among participants completing the test-retest generalizability study (n = 21), demographic characteristics did not vary from the larger sample (n = 96) used for the criterion validation.

4.3. Criterion Validation

Due to the high performance of the test-retest reliability, we summed the participant’s total score on the substance use domain from the first administration of the GREAT (all items numbered 1 - 7 were added to create a score) and used this summed score as an independent variable in a logistic regression analysis using the patients’ 3 month history of illicit opioid use as our dependent variable. Additional participants (n = 96) were recruited from the GENOA sample to complete one administration of the GREAT for the criterion validation, with a total sample of 117 MMT patients. The regression model adjusted for other important covariates including: age, sex, duration on MMT (years), onset age of opioid abuse, and methadone dose (mg/day).

Results from the regression analysis showed the GREAT substance use domain score is significantly correlated with illicit opioid use (Odds Ratio [OR]: 1.16, 95% Confidence Interval 1.05, 1.29; P = 0.003), indicating that increasing scores on the substance use domain is associated with illicit opioid use. Results from this analysis are summarized in Table 3. Increasing substance abuse domain score was also highly associated with poly-substance use as detected by urine toxicology (OR: 1.20, 95% Confidence Interval 1.04, 1.37; P = 0.011) (Table 3).

5. Discussion

This study provides a new tool to assess addiction severity and treatment response for opioid-dependent patients on MMT. Results from the application of this tool within a sample of 117 MMT patients show the GREAT tool to have strong measurement properties such as high test-retest reliability (0.95), as well as modest internal consistency within subdomains (substance use: 0.33, health risk: 0.55, health: 0.54, and personal/social functioning: 0.32). The tool also performed well in the criterion validation, where GREAT substance use domain scores modestly predicted continued opioid abuse among MMT patients.

The G-study is the central component of the tool’s reliability analysis. Performing the G-Study allowed us to determine whether we can draw accurate inferences about MMT patient’s treatment response across test administrations, subscales, and items from their individual scores on the GREAT. The reliability coefficients constructed from our G-Study suggest we can generalize from an individual’s scores across test administrations with a higher degree of certainty, where we found excellent test-retest reliability of the GREAT (consistently 0.94 or higher). The G-study also showed we had fair internal consistency within separate subdomains (Table 2), with g-coefficients ranging from 0.3 to 0.54. We found the Personal and Social Functioning subscale to have the poorest internal consistency (G-coefficient: 0.33). This subscale may have suffered from pooling very different questions together. This subdomain included items assessing criminal activity, employment, conflict in relationships, and family history of addiction or mental illness. While these items are important to understanding personal/social functioning, participants’ answers indicate we may actually be measuring attributes other than personal and social functioning with this collection of items.

The results of the G-Study suggest the existence of subdomains. In a sort of “quasi-confirmatory factor analysis,” we showed that the overall GREAT has poor internal consistency across subdomains, meaning we are limited in our ability to generalize from one subscale to another (G-coefficient: 0.20). This finding confirms our selection of the health, personal/social functioning, substance use, and health-risk behavior domains, suggesting we are selecting different attributes. If our internal-consistency across subscales was high (example: G-coefficient 0.85), this would indicate we are able to generalize well across subdomains, or in other words we are not measuring very different aspects of addiction severity across domains and therefore there is no need for the multiple domains.

The predictive criterion validation suggests that participants’ score on the GREAT substance use domain is highly correlated with participants’ three month history of opioid and poly-substance use. This confirms the reliability of self-reported drug use among patients treated with methadone. This suggests the GREAT substance use domain could serve as a proxy measure for substance abuse during time-sensitive occasions or for instances when urine toxicology reports may prove expensive or inaccessible at point of care. We chose not to evaluate the global GREAT score against urine toxicology screening because the additional domains (health-risk behavior, health, and personal/social functioning) were not intended to be predictive of opioid abuse. We maintain the purpose of this tool was to provide a complete picture of how participants are functioning across different life spheres by including domains that measure high-risk behavior (e.g. sharing drug consumption paraphernalia), as well as a patient’s social functioning.

In comparison to original MAP, the GREAT has shorter completion time and higher test-retest reliability. Overall the MAP was found to have a high response rate, with only 23 item non-responses across 16 participants and an average completion time of 11.7 minutes (SD = 3.8) (12). After 3 days of initial testing, the MAP was readministered to patients, where they found test-retest reliability for all substances was high (0.88 for clients reporting use), however there was variability for the ICC among different substance user groups (12). Variability in MAP test-retest reliability between interviewer groups was high (0.84 average) (12). The internal reliability of the anxiety and depression scales were good (alpha = 0.88 and 0.86 respectively), while the health scale internal reliability was satisfactory (alpha = 0.79) (12). Both the depression and health scale were not sustained for the modified GREAT, inhibiting our ability to comment on both these sections. Consistent with the primary field investigation undertaken by the original authors to assess the reliability and validity of the MAP, other studies confirm the original study report showing the internal and test-retest reliabilities of the MAP are satisfactory and that the instrument is adequate for health service evaluation in addition to other appropriate research purposes (29). In contrast with the MAP, the GREAT reviews patient’s responses over a 3-month time frame, allowing for a broader scope to capture treatment response. Due to the chronic remitting-relapsing nature of opioid dependence it was necessary to evaluate patient’s behavior over a longer time frame than 30 days. In comparison to the fragmented scoring structure of the MAP, the GREAT also provides a unified global score, an important feature that will improve its utility as a measurement scale for research purposes.

This study is limited by the small sample of participants available to partake in the G-Study, where only 21 participants underwent multiple testing administrations to confirm the reliability properties of the tool. An increased sample size would have improved our confidence in the estimates generated from the G-Study.

To be included in the regression analysis participants require complete data for all variables selected the regression model. At any point participants have missing data for a single variable they will be dropped from the regression analysis. While 117 patients were recruited for inclusion into the study, we were only able to perform analyses on 107 of these patients due to 10 patients lacking data on variables selected for inclusion to the model. Systematic bias posed by missing data is a potential limitation of the majority of studies. Within this study there remains less than 10% missing data, which is within the commonly accepted threshold by which studies with < 10% missing data are unlikely to be significantly confounded (30).

In addition, the sample used to create this tool may be an important factor limiting the results of this study. We generated a new tool using GENOA participants’ responses to the MAP, and as such this new tool is impacted by the generalizability of the GENOA sample. For instance, the use of amphetamines was not reported among the GENOA sample. Since it is known the preference and impact of amphetamines is greater in certain areas of the U.S. (31) the GREAT may not perform well in capturing concurrent substance use among such populations. Nevertheless, the tool can be modified to include additional illicit substances not currently included (example: amphetamines, LSD).

5.1. Conclusions

Assessment of addiction severity and MMT response is a pertinent topic to clinicians and researchers. A modified tool to assess a patient’s progression through methadone treatment in conjunction with their addiction severity serve to identify high-risk patients for relapse, as well as look beyond the results of a urine test, by including the relevant physical and psycho-social domains affecting patients with opioid use disorders. The future directions of this study will be to develop a risk score to quantify the risk for relapse from the GREAT. The GREAT will serve as a useful adjunct to regular opioid testing, allowing physicians to comprehensively assess the functioning of patients across different domains.