Development and Psychometric Properties of the Justified Death Attitude Scale

authors:

avatar Pouya Zandian ORCID 1 , * , avatar Mojtaba Habibi 2 , 3 , avatar Seyed Vahid Shariat ORCID 4 , avatar Nasrin Tayeri ORCID 5

Cognitive Psychologist, Clinical Psychologist, Tehran, Iran
Department of Addiction Studies, Tehran Institute of Psychiatry, School of Behavioral Sciences and Mental Health, Iran University of Medical Sciences, Tehran, Iran
Family Research Institute, Shahid Beheshti University, G. C., Tehran, Iran
Mental Health Research Center, Tehran Institute of Psychiatry- School of Behavioral Sciences and Mental Health, Iran University of Medical Sciences, Tehran, Iran
Cognitive Psychologist, Tehran, Iran

how to cite: Zandian P, Habibi M, Shariat S V, Tayeri N. Development and Psychometric Properties of the Justified Death Attitude Scale. Iran J Psychiatry Behav Sci. 2016;10(4):e3217. https://doi.org/10.17795/ijpbs-3217.

Abstract

Background:

Both capital punishment and euthanasia are highly disputed practices that are considered justified by their proponents. Emotional component of attitude has been ignored in the scales developed to assess attitude on capital punishment and euthanasia.

Objectives:

This development study aimed to develop an assessment tool to evaluate emotional attitude (ignored by previous scales) toward capital punishment and euthanasia that are considered justified by their proponents in 2015. The scale, as defined by the authors, measures individuals’ attitude revealing their sense of self or non-self, although lawis in no condition to tackle the exception either crime or euthanasia. After assessing face validity with two psychologists, a clinical psychologist and a psychometric expert at the school of behavioral sciences and mental health, a justified death attitude scale (JDAS) was performed on 571 students of national universities of Tehran to evaluate construct validity of the scale. Data were analyzed using the exploratory factor analysis, chi-square, and multivariate analysis of variance.

Materials and Methods:

The justified death attitude scale included 59 items; other than the first 4 warm-up questions, each item included a scenario in which a decision should be made by the subject on life of the personage. Scenarios are related to rape, adultery, murder, drug trafficking, and active and passive euthanasia. The aforementioned scenarios and items are developed based on social interviews.

Results:

The justified death attitude scale showed an acceptable reliability and included eight factors that could explain 80.56 variance of the scale. A comparison with warm-up items and main items indicated that poll did not assess euthanasia attitude (P > 0 .05), but it could assess death penalty attitude (P < 0.05).

Conclusions:

The justified death attitude scale has acceptable validity and reliability in emotional attitude assessment of capital punishment and euthanasia.

1. Background

Both death penalty and euthanasia are thorny issues whose the point of contact is justification for self-components. Also, these issues provoke conflict attitudes of the subjects, and evaluation of their attitudes takes easy. It is therefore not important as to whether one is endangered by either criminal activities, and execution, or euthanasia; the endangered person’s emotion about the issue is the crucial point. As a result, an emotional-cognitive and multivariable scale, called the justified death attitude scale (JDAS), is required for assessing the aforementioned issues. The scale, as defined by the authors, measures individuals’ attitude revealing their sense of self or non-self, albeit law is in no condition to tackle the exception either crime or euthanasia. So, participants could express their attitudes about death penalty and euthanasia according to self-factors, which are on the JDAS. Meanwhile, attitude and self-factors intertwine with each other because the development of them is paralleled in mind (1).

As some attitude scales such as the euthanasia attitude scale (EAS), attitude euthanasia scale (ATE), and death qualified (DQ) do neither assess attitudes very well, due to a lack of emotional evaluation, nor can simultaneously study euthanasia and death penalty, an attitude assessment scale is required to systematically evaluate both execution and euthanasia attitudes according to the most important element of attitude, which is emotion so that participants can firmly stick through conflict choices to evaluate their attitude meticulously (2-4).

Conclusively, we meant to develop a new scale to evaluate attitude of the subjects toward death penalty and euthanasia because the former is against human right and under the normal circumstance you could not kill others intentionally even if he has done a heinous crime although others suggest that we should consider the crime not only in victims’ point of view but also in safety and religious point of views. Besides, euthanasia, a very critical issue in medical ethic, is an empathy with a patient having a terminal painful malignant disease; however, it is against religious training. Likewise, euthanasia suggests hopeless ideas among patients. All of us have these ambivalence attitudes about the aforementioned issues and if they are combined with main self-components, we will evaluate the justified death attitude, which is part of self. Since, decision about others’ lives needs comparison between self and non-self traits (5). We aimed to assess the validity and reliability of the scale. Also, it was assumed that warm-up questions of two scales correlate with each other in the cognitive section but not in the main part of the scale that is emotional-cognitive.

2. Materials and Methods

2.1. Procedure

2.1.1. Item Generation

Authors as a master student of clinical psychology, a graduate of cognitive psychology, a health psychologist, and a psychiatrist generate the JDAS, having six scales of rape, adultery, murder, drug trafficking, and malignant conscious and unconscious cancer patients. From a total of 59 questions, 44 questions are for legal and sixteen are for medical scales. The aforementioned scenarios and items are developed based on social interviews.

Firstly, 80 subjects, in parks and libraries, were interviewed with an unstructured interview system to understand what their points of views were about justified death attitudes, which an interviewer had explained. Meanwhile, we wanted to know what their prejudice, and how we could make a conflict situation, so that we evaluated their justified death attitudes because men indicated their attitudes in conflict situations. Interviewees could continue the interview with scenarios well, and they indicated their attitudes when self-factors was in questions. Questions tried to evaluate hot cognition of the subjects by changing point of views from first person to third person, and also by changing nationality, religion, and age of the scenario character. Moreover, each legal scale contains 11 questions but each medical scale had eight questions.

The scale starts with warm-up questions, which are polls, in order for individuals to get ready for the main part. The questions were Do you agree death penalty is a suitable punishment for criminals who have committed a heinous crime? And if yes, what kind of death penalty, painful or painless, will you choose? Next, each scenario should be read by the participants before answering the main questions. To avoid gender bias, the scale was generated in a gender-free format. So, we selected participants with a quota sampling method in this development research.

2.2. Participants

Five hundred and seventy-one single individuals were participated in this study and grouped according to their gender (women or men, the mean age = 23, age range: 18 - 32 years (academic major) art, medicine, engineering, humanities, psychology, experimental sciences, and arithmetic), and mental health (hassle or ordinary). That is, the inclusion criteria to participate in the study are to fill in the general health questionnaire-12 (GHQ-12) and JDAS. Likewise, subjects should be students and between the ages of 18 - 32 years old. All of the participants were selected from national universities of Tehran. In fact, the aforementioned group was suitable because Tehran, the capital of Iran, is a multiethnic city and many top universities of Iran are there. So, if a student passes the entrance examination, he will immigrate to the city in order to study. Besides, one-fifth of Iranian students are all in Tehran. Indeed, the study was conducted in four stages of assessing comprehensibility (30 persons), pilot reliability (60 participants), construct reliability (419 individuals), and ideological differentiation of the justified death attitude (62 subjects). Firstly, 30 participants evaluated the comprehensibility of the scale. Sixty persons were tested likewise to assess the reliability of the scale in the pilot stage. Next, 419 participants were selected from the top 10 state universities of Iran, namely Tehran, Shahid Beheshti, Medical Sciences of Shahid Beheshti, Sharif Technology, Kharazmi, Tehran University of Medical Sciences, Modares, Tehran Polytechnic, Science and Technology, and Allameh Tabatabai, for initial development of the scale. Entering these universities is quite demanding for students as they need to pass the national educational testing (NET) to enter the universities. Furthermore, 62 seminarians, assumed as highly religious, were selected for comparison with students. All of the participants took a gift for their participation. Finally, 15 answer sheets were cancelled due to incomplete responses. So, the first data analysis, which was for evaluation of validity and reliability, was based on 466 students in Table 1.

Two hundred and thirty-three (50%) participants were females and the remainder were males. Due to the inadequate number of participants, both those studying art and humanities were combined with those studying psychology. Participants studying arithmetic were also combined with those studying engineering. All of the participants were categorized by using the GHQ-12, measuring current mental health, as hassle or ordinary individuals (6, 7).

2.3. Measures

2.3.1. The Justified Death Attitude Scale

A 59-question emotional-cognitive attitude assessment scale of 2 legal and medical scales and 6 subscales, 4 of which are for criminal activities, namely rape (11 questions), adultery (11 questions), murder (10 questions), and drug traffic (11 questions), and two subscales for conscious (eight questions) and unconscious patients (eight questions) suffering from terminally painful and malignant cancers. Questions were written for five situations: nationality and race (10 questions), age (11 questions), religiosity (20 questions), first person perspective (six questions), and third person perspective (12 questions), in order to create multitragic circumstances and participants had to answer the multichoice questions with freedom, short-term sentences, long-term sentences, painless execution, and painful execution as choices. In addition, there were medical subscales containing multichoice questions with active euthanasia, passive euthanasia, and cancer treatments as choices.To evaluate the comprehensibility of the scale items, 30 individuals read the questions and answered so as to determine whether they were comprehensible. After rectifying the problematic items, 30 participants responded to the beta version of the JDAS in order to assess its reliability in the pilot stage. Since the reliabilities of third (Euthanasia is deliberately killing a terminally ill person in order to stop him/her suffering. Generally, do you agree with euthanasia?), fourth (Do you agree with euthanasia for conscious patients?), and fifth (If you agree with euthanasia, what kind of euthanasia, active or passive, is more appropriate?) questions were not very high, they were removed from the main scale, but were retained as warm-up questions to avoid surprising participants.

Then, the reliability of the JDAS was re-assessed by 30 other individuals to maintain a higher standard (α > 0 .8). Next, 481 participants responded to questions according to the following direction: The following questions are about justified death attitudes. Please, answer the questions according to scenarios and conditions you would confront. Finally, 466 participants’ responses were admitted. Conclusively, it is the first version of the scale that will re-evaluate on more heterogeneous and larger sample.

2.4. Statistical Strategy

Less than five percent of data were missing from the dataset in this study. Thus, list- wise deletion with no imputation of data was used in the present analyses. The assumption of normality was checked and skewness was evident in the first and second order subscales of the JDAS. As the sample was large, no transformation was performed (8).

3. Results

3.1. Factor Structure

The survey contained 64 items, concerning emotional-cognitive attitudes, related to justified death. All items were analyzed, but preliminary results indicated that four items, listed below and not used later, did not fit well in any of the scales due to lack of correlation with the total scales, nor could contribute to the alpha coefficients.

A varimax rotation, a principle component method, was conducted as an exploratory factor analysis for the JDAS. The sample size was sufficient to get the minimum five-to-one ratio, and might be an erroneous or specific sample (9, 10). Results of 64 items of the JDAS indicated that the scale was factorable. The Kaiser-Meyer-Olkin measure of sampling adequacy was 0.93 indicating a marvelous level of intercorrelation among the items (11). Similarly, the Bartlett’s test of sphericity showed that there was a significantly good correlation, approximately χ2 (1770) = 38354.67, P < .001, between the items to perform factor analysis.

Underlying factors were identified via Eigen values and scree test. Six out of seven factors had Eigen values that explained for 77.54% of the variance (10). However, the scree test showed that six factors had a larger proportion of variance apportioned to them than that of the remaining factors (12). The initial factor analysis was run with principle components, common factor extraction methods, oblique, and orthogonal rotations. The solution was similar throughout all methods, indicating that the factors were stable (13). Principle axis factoring, a method of common factor analysis, was chosen to extract the solution as it relied only on common variance. Therefore, it decreased likelihood of error and bias, and increased replicability in comparison with principle component models (14).

Extraction communalities of number 34 whose extraction communalities were less than or equal 0.3 were dropped in order to ensure that scale items of the remaining variance were related to the JDAS construct (15). Examination of a six-factor solution pointed towards scales with apparent substantive meaning. Thus, items whose loading of unique factors were less than 0.4 were deleted when the main matrix structure showed that total loadings were low (10).

The factor analysis results obtained through a 59-item scale were loaded on six factors presented in Table 2. Respectively, the first, second, and fourth factors were rape, adultery, drug trafficking comprising 11 items, and the third, fifth, and sixth factors were murder, euthanasia for conscious cancer patients, euthanasia for unconscious cancer patients, comprised of 10, 8, and 8 items.

Table 1.

Demographic Characteristics (N = 466)

CharacteristicsData
Gender
Male233
Female233
Major
Art2
Physician57
Engineering104
Psychology25
Humantarity115
Experimental Sciences62
Arithmetic39
Seminaries62
Religion
Islam
Shie383
Soonie83
Mental Health
Normal305
Abnormal161
Range of Age18 - 32
Table 2.

Parameter Estimates from Exploratory Factor Analysis of the Six-Factor Oblique Model (N = 466)

NumbersRapingAdulteryMurderingDrug TraffickingEuthanasia for Conscious patientsEuthanasia for Unconscious PatientsTotal VarianceInitialExtraction
50.900.890.86
60.900.930.89
70.890.900.88
80.910.930.90
90.900.920.89
100.910.920.88
110.900.910.85
120.660.630.49
130.790.710.67
140.760.750.63
150.690.620.55
160.910.930.91
170.920.950.93
180.920.960.94
190.920.950.93
200.920.940.93
210.900.930.89
220.880.900.86
230.820.820.76
240.970.850.83
250.830.840.79
260.730.660.65
270.920.910.88
280.900.920.86
290.930.920.89
300.920.930.89
310.910.910.88
320.880.880.83
330.880.870.83
350.820.750.74
360.760.790.67
370.680.600.51
380.870.830.81
390.890.880.85
400.880.850.82
410.910.910.90
420.910.910.88
430.870.880.84
440.880.870.83
450.740.700.71
460.550.540.53
470.770.770.71
480.680.600.61
490.930.920.91
500.940.960.93
510.930.960.91
520.910.920.88
530.670.680.70
540.850.830.82
550.770.700.70
560.720.690.63
570.940.970.93
580.940.980.94
590.950.980.93
600.890.850.85
610.840.810.79
620.780.770.76
630.740.770.76
640.770.710.75
Variance15.7514.4413.6313.4410.6210.5978.47InitialExtraction
Reliability0.970.980.960.960.960.9610.87
Means of Inter- Items- Correlation0.750.840.690.690.760.7610.89
Means of Items’ Means3.812.723.343.411.891.7510.88
11111111886010.90
Table 3.

Parameter Estimates from Exploratory Factor Analysis of the Two-Second Order Factor Oblique Model (N = 466)

ScenariosLegalMedicalInitialExtraction
Adultery0.781,0000.61
Rape0.751,0000.57
Murder0.691,0000.39
Drug traffic0.621,0000.47
Cancer0.851,0000.73
Unconscious Patient0.851,0000.72
Variance33.8724.64
Reliability0.670.63
Means of inter-item correlation0.340.46
Means of items’ means37.5317.42
Number item42

The total variance was explained by six factors, calculated from the sums of squared loadings in the structure matrix. Explanation of the total variance was for scenarios of rape, adultery, murder, drug trafficking, terminally ill conscious and unconscious patients, which were respectively 15.75, 14.44, 13.63, 13.44, 10.62, and 10.59 (see Table 2). As the total variance (80.56 or 78.47) explained by oblique rotation was overestimated due to a moderate correlation between factors, it should be carefully interpreted (15).

The second order exploratory factor analysis, which was conducted on the first six order factors, namely rape, adultery, murder, drug trafficking, and conscious and unconscious malignant cancer patients, indicated that they are categorized according to two factors, legal (33.87) and medical (24.64) (Table 3).

3.2. Reliability

Both total sample of internal consistency coefficients and mean inter-item correlation for the JDAS, whose alpha coefficient for the six first-order scale was 0.96 to 0.98 (Table 2) and for the two second-order scale was 0.63 to 0.67 (Table 3). The range item analysis which had been conducted with corrected item-total correlation for drug trafficking and adultery were 0.56 to 0.96 and average corrected item-total correlation for rape, adultery, murder, drug trafficking, and terminally ill conscious, and unconscious patients were 0.85, 0.91, 0.81, 0.81, 0.61, and 0.61, respectively. The inter-item correlation ranged from 0.69 (murder) to 0.84 (adultery), and the obtained results suggested that the scales had adequate internal consistency.

3.3. A Comparison with Warm-up Items

The warm-up questions only evaluate both execution and euthanasia ideas, which are part of the JDAS and we want to know how much death penalty and euthanasia ideas can predict the justified death attitude and what is the relationship between these two ideas with each other when eight elements of self are omitted.

Relationship between the initial question of general idea, for death penalty, and the third general idea, for euthanasia, was significant (χ2 (1) = 4.93, P < 0.05).

To evaluate the influence of the first general idea question, which is about agreement and disagreement on execution for the six factors of the JDAS, a multivariate analysis of variance (MANOVA) was performed. The result of MANOVA showed a significant effect [F (6, 396) = 16.28, P < 0.001, partial Eta squared = 0.20]. Subsequent tests of between-subject effects showed that individuals agreeing on execution scored significantly higher on rape [F (1,401) = 70.61, P < 0.01 partial Eta squared = 0.15], adultery [F (1, 401) = 31.87, P < 0.01 partial Eta squared = 0.074]), murder [F (1, 401) = 36.685, P < 0.01 partial Eta squared = 0.08], drug trafficking [F (1, 40) = 20.51, P < 0.01 partial Eta squared = 0.05 ], and lower but no significant on euthanasia for conscious patients [F (1, 40) = 2.37, P > 0 .05], and euthanasia for unconscious patients [F (1, 40) = 1.31, P > 0.05].

For the third question, the result of MANOVA indicated significant effects of agreement or disagreement on euthanasia on two factors of the JDAS [F (6, 392) = 34.66, P < 0.001]. Subsequent tests of between-subject effects illustrated that the persons agreeing on euthanasia scored significantly higher on euthanasia for conscious patients [F (1, 397) = 187.62, P < 0.01 partial Eta squared = 0.32], and euthanasia for unconscious patients [F (1, 397) = 62.85, P < 0.01 partial Eta squared = 0.14].

4. Discussion

The current study examined psychometric properties of a newly developed scale on the justified death attitude. It was assumed that the scale estimated an emotional-cognitive, and not a cognitive attitude by providing different scenarios and asking subjects to decide which one option was “fair” for the situation (16). The factor analysis showed six factors for the JDAS.

The analysis of the 67-question scale indicated that the four initial cognitive questions should be deleted due to lack of reliability, and the other three initial cognitive questions cannot be categorized in one of the eight extracted factors of the scale. Thus, it was decided to retain the four cognitive questions as warm-up questions to avoid surprising participants.

The JDAS has two second-order scales, legal and medical, and six first-order scales, rape, adultery, murder, drug trafficking, and a conscious patient in extreme pain and a terminally ill unconscious patient. In other words, the JDAS is a valid scale because of the correlation between the four initial legal scenarios and the two last medical scenarios. In addition, the first question correlates with the four legal scenarios, and the third one with the medical scenarios. Questions were all placed in the scale based on the individual and in group, and empathized out group and were precisely categorized in their appropriate scales and subscales except for the 37th question that was removed. Indeed, the first and second-order scales, having moderate reliability, were loaded on six factors, and the validity of the first-order scale of the JDAS was supported. Thus, except for the ten-item third scenario, murder, other legal subscales contained eleven items and two remaining medical subscales were eight-item subscales.

Moreover, there was a significant relationship between the first and third warm-up questions assessing general ideas, which indicated that when participants’ opinions are assessed, they are not in an emotional state. So, a significant correlation between the death penalty and euthanasia appeared initially but nowhere else the correlation has been found in the main part of the scale when individuals emotionally realized the crimes.

Besides, there was a significant relation between the second (If you agree with the death penalty, which one is more suitable, painless or painful) and fourth (If you agree with euthanasia, which one is more suitable, active or passive) warm-up questions, determining kinds of punishment and euthanasia pertaining to legal or medical scenarios, respectively (17-20).

Finally, the comparison with warm-up items and main items indicated that the stronger attitudes could be evaluated with poll, but weaker attitude could not be evaluated easily. For instance, participants have a very strong pessimistic attitude towards rape, and have a higher correlation with the initial warm-up questions, but euthanasia make ambivalence attitude among the subjects and could not be evaluated with poll because it is weaker. So, if you want to make a scale for attitude, you should consider emotional questions in order to evaluate precisely because stronger attitudes are in behavior and it combines with emotion (21, 22).

4.1. Limitations and Suggestion

This study faced several limitations like a nonprobability sampling technique. Likewise, we do not consider other religions, ethnicity, and generations to develop the scale. Next, sampling was in Tehran, and other cultures did not consider precisely. Another, neuropsychological tools like dot probe, emotional stroop, and approach-avoidance tasks could help us in order to develop the scale well because they can evaluate attitudes liminal and subliminal. Finally, a better inventory such as MCMI instead of GHQ-12 could improve the study, and we could compare normal and abnormal attitudes, empathy, and schemas.

Acknowledgements

References

  • 1.

    Aghababaei N. Attitudes towards euthanasia in Iran: the role of altruism. J Med Ethics. 2014;40(3):173-6.

  • 2.

    Aghababaei N, Farahani H, Hatami J. Euthanasia attitude; A comparison of two scales. J Med Ethics Hist Med. 2011;4:9. [PubMed ID: 23908751].

  • 3.

    Blackmore S. Consciousness: an introduction. Oxford University Press; 2011.

  • 4.

    DeMarree KG, Petty RE, Briñol P. Self and attitude strength parallels: Focus on accessibility. Soc Personal Psychol Compass. 2007;1(1):441-68.

  • 5.

    Gaertner L, Sedikides C, Luke M, O'Mara EM, Iuzzini J, Jackson LE, et al. A motivational hierarchy within: Primacy of the individual self, relational self, or collective self? J Exp Soc Psychol. 2012;48(997):1013.

  • 6.

    Rahmati Najarkolaei F, Raiisi F, Rahnama P, Gholami Fesharaki M, Zamani O, Jafari MR, et al. Factor structure of the Iranian version of 12-item general health questionnaire. Iran Red Crescent Med J. 2014;16(9):11794. [PubMed ID: 25593708]. https://doi.org/10.5812/ircmj.11794.

  • 7.

    Thienpont L, Verhofstadt M, Van Loon T, Distelmans W, Audenaert K, De Deyn PP. Euthanasia requests, procedures and outcomes for 100 Belgian patients suffering from psychiatric disorders: a retrospective, descriptive study. BMJ Open. 2015;5(7):7454. [PubMed ID: 26216150]. https://doi.org/10.1136/bmjopen-2014-007454.

  • 8.

    Tabachnick BG, Fidell LS. Using Multivariate Statistics. 6 ed. Pearson; 2007.

  • 9.

    Ellwanger J. Processing of own versus other names differentially affects startle reflex modification. Int J Psychophysiol. 2006;62(2):233-7. [PubMed ID: 16678292]. https://doi.org/10.1016/j.ijpsycho.2006.03.004.

  • 10.

    Hair JF, Anderson RE, Black WC. Multivariate data analysis. Prentic Hall; 1998.

  • 11.

    Kaiser HF. An index of factorial simplicity. Psychometrika. 1974;39:31-6.

  • 12.

    Catell RB. Handbook of multivariate experimental psychology. Rand McNally; 1966.

  • 13.

    Garsuch RL. Factor analysis. 2 ed. Lawrence Erlbaum Associates; 1983.

  • 14.

    Snook SC, Gorsuch RL. Component analysis versus common factor-analysis - A monte- carlo study. Psychol. Bull. 1989;106(1):148-54.

  • 15.

    Tabachnick BG, Fidell LS. Using multivariate statistics. Pearson; 2001.

  • 16.

    Burki TK. Trends in euthanasia and assisted suicide. Lancet Oncol. 2015;16(9):433. [PubMed ID: 26299603]. https://doi.org/10.1016/S1470-2045(15)00242-9.

  • 17.

    Reeve C, Rogelberg SG, Spitzmüller C, DiGiacomo N. The caring‐killing paradox: Euthanasia‐related strain among animal‐shelter workers1. J Applied Social Psychol. 2005;35(1):119-43.

  • 18.

    Smets T, Cohen J, Bilsen J, Van Wesemael Y, Rurup ML, Deliens L. Attitudes and experiences of Belgian physicians regarding euthanasia practice and the euthanasia law. J Pain Symptom Manage. 2011;41(3):580-93. [PubMed ID: 21145197]. https://doi.org/10.1016/j.jpainsymman.2010.05.015.

  • 19.

    Van Hooff AJ. Ancient euthanasia: 'good death' and the doctor in the graeco-Roman world. Soc Sci Med. 2004;58(5):975-85. [PubMed ID: 14732610].

  • 20.

    van Tol DG, Rietjens JA, van der Heide A. Empathy and the application of the 'unbearable suffering' criterion in Dutch euthanasia practice. Health Policy. 2012;105(2-3):296-302. [PubMed ID: 22417862]. https://doi.org/10.1016/j.healthpol.2012.01.014.

  • 21.

    Wiers RW, Rinck M, Dictus M, van den Wildenberg E. Relatively strong automatic appetitive action-tendencies in male carriers of the OPRM1 G-allele. Genes Brain Behav. 2009;8(1):101-6. [PubMed ID: 19016889]. https://doi.org/10.1111/j.1601-183X.2008.00454.x.

  • 22.

    Chen M, Bargh JA. Consequences of Automatic Evaluation: Immediate Behavioral Predispositions to Approach or Avoid the Stimulus. Per Soc Psychol Bull. 1999;25(2):215-24.