User Acceptance of Picture Archiving and Communication System in the Emergency Department

1. Background

Utilizing modern technologies to improve the quality and efficiency of healthcare system has increasingly enhanced (1). Among these types of technologies, picture archiving and communication system (PACS) can be mentioned, which has allowed medical images to be transmitted, stored, retrieved, and displayed in different locations of a hospital or health system (2-5). It can be stated that with the advancement of digital imaging devices (such as magnetic resonance imaging (MRI), computed tomography (CT) scan, and ultrasound) during the past two decades, the advent of tools and technologies has improved and facilitated their application. Since the early 1980s till the present time, PACS has replaced the traditional imagery paper-based and film-based printing with an accelerated process (6). In recent years, setting up PACS in the Middle East countries has been significant. Among them, countries such as Saudi Arabia, United Arab Emirates, and Turkey are placed in a higher rank than other countries. It is predicted that the number of hospitals using this technology from 984 in 2010 will reach to 1680 by 2014 (nearly twice). In addition, the Middle East market for PACS and radiology information system (RIS) till 2014 is estimated to be 140 billion dollars (7, 8).

PACS has led to abundant well-established benefits such as improving operational efficiency, and productivity of the medical image system, facilitating accessibility of images anytime and anywhere, reducing waiting time for imagery retrieval and turn-around times of clinical reports, and more effective application of radiology equipment (9, 10). Many of the mentioned benefits will significantly influence an area such as emergency medicine. Emergency medicine processes highly depend on rapid diagnosis; thus, using PACS in the emergency department will eventually result in improved efficiency and patient care. Previous studies support and emphasize the benefits of using PACS in the emergency department (11-13).

In spite of the abundant benefits of employing PACS, there are some challenges in implementing and using this technology like necessity to make changes in the work flow, cost, and users’ resistance to accept the technology. The present study deals with investigating one of the above-mentioned challenges, i.e. user acceptance of PACS. Denial of a technology is a big barrier that causes the best and most expensive information technologies to fail. Therefore, addressing various aspects of this issue on a technology such as PACS is an essential task (9, 14-16).

Iran is also among the countries that have widely applied this technology in the health system and consider its development in the future. Few and limited studies have been carried out on the user acceptance of PACS. According to our search, only six studies have quantitatively investigated the acceptance issue of PACS of which only one belongs to the Middle East region (5, 10, 17-20). In Iran, such a study has not been performed so far. Significance of this issue, the influence of local features on the technology acceptance, and the critical role of the emergency department in the health system implies the necessity to perform such studies, especially in developing countries (such as Iran). Wise use of resources in order to adapt to the growing advances of modern technologies is an important factor in these countries. We chose three general educational hospitals as our target group. All of these hospitals have been using hospital information system (HIS)/PACS technology for at least two years. HIS/PACS integration was complete in three hospitals. All of these hospitals apply the similar brand of PACS.

2. Objectives

This study primarily aims to evaluate PACS acceptance level in the emergency department based on technology acceptance model (TAM) model. Efficiency of this model in assessing IT acceptance in the health field has been proved by numerous studies, and it has been previously used in more than 20 scientific studies in various health related ITs (21). This assessment has been performed in three big educational hospitals in Tehran in order to assess the user acceptance of PACS in the emergency department, compare its acceptance in three different hospitals, and investigate the effect of socio-demographic factors on the user acceptance.

3. Materials and Methods

In this study, a variant of TAM model has been used in order to measure the acceptance level of PACS. The original TAM model, which is rooted in the theory of reasoned action (TRA), describes how a technology is accepted and applied by users. In this model developed by Davis, two parameters of 1- “the degree to which a person believes that using a particular system (perceived usefulness (PU)” and the degree to perceived ease of use (PEU) would enhance his or her job performance and 2- “the degree to which a person believes that using a particular system would be free from effort” have been specified as the primary variables. PEU has a casual effect on PU and each of these variables would influence the component of user’s attitude towards use, and eventually, the acceptance issue is defined to be influenced by three factors (Figure 1) (14, 22-24).

The technology acceptance model (TAM) of Davis

Figure 1.

The technology acceptance model (TAM) of Davis

In order to describe the acceptance issue more precisely, the model was developed in different ways and more variables have been introduced to it. According to the study conducted by Lee et al. (25), more than twenty variables including education level, computer application proficiency, socio-demographic variable, system management and support, and user’s interaction in different versions of TAM have been used. Subsequent to the investigations performed in this regard, including unified theory of acceptance and use of technology (UTAUT), Holden and Karsh’s study (21), and TAM2, the model used in the present study was selected (Figure 2). The difference between this model (similar to the model in Aldosari’s study (17)) and the TAM original model is the introduction of the change variable, and also, eliminating the effect of PEU on PU (differentiated it from UTAUT model) and omitting the attitude toward use as a variable (distinguished it from TAM2 model).

Model framework used in this study

Figure 2.

Model framework used in this study

Data of this research were collected from the emergency department of three general educational hospitals. Baqiyatallah hospital includes 56 beds located in the emergency department with nearly 95000 admission ratio per year. Shohada Tajrish includes 36 emergency beds with about 50000 admission ratio per year. Imam Hossein Hospital consists of 68 emergency beds with about 140000 admission ratio per year. The data were collected using a questionnaire as an instrument. This questionnaire had been previously used in several studies. It comprises five sections (17). The first part consists of personal information including age, gender, education, occupation, and rate of using PACS. The second part involves six questions related to PU. The third part consists of four questions regarding PEU. The fourth section includes four questions about the changes resulted from use of PACS. The fifth part contains ten questions related to acceptance. Five part Likert scale was used for each question. Point zero means completely disagree and point five means completely agree. All questions are listed in the next section. The study surveyed all health workers in three emergency departments of three educational hospitals selected in a random process.

Linguistic validation of the questionnaire was achieved by translation in Persian and back translation in English. Two experts of medical informatics validated the content of the questionnaire. Twenty participants filled the questionnaire with the time interval of two weeks and intra-class correlation coefficient (ICC) was calculated to be 0.9. Cronbach’s Alpha was calculated to be 0.83.

3.1. Statistical Analysis

All data were analyzed using SPSS ver. 18 (SPSS Inc. Released 2009. PASW Statistics for Windows, Version 18.0. Chicago: SPSS Inc.) and STATA ver. 12 (StataCorp. 2011. Stata Statistical Software: Release 12. College Station, TX: StataCorp LP.). Descriptive statistics were used describing mean and frequency for study variables. Student t-test was conducted comparing means in two groups and analysis of variance (ANOVA) was used to compare means in more than two groups. Chi square test was used to analyze qualitative variables.

Stepwise multiple regression model was used to predict factors influencing acceptance score as the dependent variable. All variables correlating with acceptance score in bivariate analysis at significance level of 0.1 were entered in the model. Variables that were significant in the model at the level of 0.05 remained in the model.

3.2. Ethical Statement

All questionnaires were fully anonymized to make sure that the project is fully reported. Participants were free to take part in the study.

4. Results

Characteristics of the participants are shown in Table 1. A total of 128 individuals returned the questionnaire (response rate < 40%).

Mean age of participants was 32.9 years (SD: 6.08). Nurses had the largest participation in the study in all three hospitals (57%). Most of the participants had a bachelor educational degree (54%). Participants had the experience of working with PACS of two months to 3 years.

The three groups were significantly different in gender and use of PACS. Female workers were significantly lower in Baqiyatallah hospital (Chi square: 12.23, P value: 0.002). Rate of using PACS was higher in Baqiyatallah hospital compared to two other hospitals (Chi square: 12.04, P value: 0.017). Three groups were comparable in age, job and education (Table 1).

Table 2 summarizes the questions related to 4 factors including PU, PEU, change and acceptance. More than 95% of participants had the experience of working with PACS more than one year.

Regarding PU and PEU, most of the respondents recognized PACS as a useful and simple tool in their daily practice (Mean (SD) of PU: 4.25 (5), Mean (SD) of PEU: 4.20 (52)).

According to change construct, 81% of participants reported that PACS makes their job much easier. “Making their work happier” had the lowest role between change related questions (Mean (SD): 3.98 (0.73)).

The behavioral was the variable measuring acceptance. According to this variable, mean of behavioral was 4.01 with the standard deviation of 0.35 suggesting a high level of acceptance with PACS system in three hospitals. Compared to PU, PEU and change, more items in behavioral construct acquired scores less than 4 (6/10). The question asking about cooperation with PACS personnel had the lowest rating Mean (SD): 3.65 (8).

Considering three hospitals, men had more acceptance than women, but this difference was not statistically significant (T: 0.28, P value: 0.80).

Among the demographic variables investigated here, no statistically significant difference was observed between different subgroups of age and gender for mean score of PU and PEU and change.

Regarding their educational degree, participants of this study were classified into four groups of people with a bachelor’s degree, master’s degree, MD, PhD or specialist. Participants with the specialty degree got a higher acceptance score than the three other groups (Mean (SD): 4.17 (0.20)). Running one-way ANOVA and TUKEY test as post-hoc test, this difference was statistically significant (F: 3.33, P value: 0.02) (Table 3). Yet, this difference on PU, PEU, and change is not significant. Moreover, according to their occupation, participants were divided into four groups of physicians, nurses, residents and other professions. These four groups were not statistically different in their scores for acceptance, PU, and change. Regarding use of PACS, subjects were classified into three groups including those who always used PACS, those who sometimes used it, and those who rarely used it. These groups did not show any significant difference in their scores for PU, acceptance, PEU and change.

Stepwise multiple regression model was used to predict the significant variables influencing acceptance as the dependent variable. In our model, all three variables of PU, PEU and change had a significant effect in prediction of acceptance. The most influencing factor was change with the beta of 0.21 (P value < 0.001). The second factor was PU with beta of 0.18 (P value < 0.001) and the third factor was PEU with beta of 0.17 (P value < 0.001). Adjusted R square for the model was 0.39 (Table 4).

5. Discussion

Studies that have investigated the acceptance of PACS system are small in number. Among the Middle East countries, only one study has been carried out investigating the acceptance level of PACS in the radiology department of a hospital in Saudi Arabia (17). Pare and Trudel investigated the challenges of PACS actualization in hospitals. In their study, they pointed to the significance of human factors among other factors (6). Duyck et al. have examined the acceptance of PACS quantitatively in the radiology department of a hospital in Belgium. In their study, user acceptance of PACS was examined before and after its initiation in the hospital (quantitative investigation) (18). In a study carried out by Pynoo et al. (20), two factors influencing physicians’ acceptance of PACS and the possibility of changing these factors by enhancing their experience in PACS utilization were considered. Finally, a research performed by White et al. focused on the challenges of emergency department in employing digital radiology instead of the traditional paper and film-based system for printing images. Among the findings of this research, the importance of human factors in creating this change were mentioned (12). The model used in the present study to evaluate PACS acceptance is in accordance with the model applied in the existing studies. Use of TAM model is one of the routine methods in assessing the acceptance of information technologies. In Holden’s review, this method is generally capable to evaluate 30 to 40 percent of the changes and factors affecting the acceptance level (21). In the current study, the variance of this model is in congruence with Holden’s review.

All three factors of PU, PEU, and change have significant effect on the acceptance. The factor that had the highest impact was change, which includes factors affecting practice situation. The amount of this effect was 21 percent according to the obtained model. Findings of this study are in accordance with Aldosari’s research (17). The TAM model was also used in the mentioned study, and it was concluded that all three factors of PU, PEU, and change influenced the acceptance, and the model obtained in that study also predicted 41 percent of acceptance change, which corresponds to the value gained in the present research.

In a study conducted by Aldosari (17), PU had the highest effect on predicting the acceptance changes and in the present study, change had the greatest coefficient. To account for this difference, it could be mentioned that the present research was performed in the emergency departments of those referral and busy hospitals of Tehran. In these emergency departments, the change that has been made by PACS system in the performance of physicians and nurses due to patients’ crowd is mostly regarding the participants’ perception about the system itself. On the other hand, it is expected that according to the existing infrastructures in the developing countries, PACS performance may have more dysfunctions compared to other countries where the studies have been carried out; hence, PU and PEU factors may differ.

According to the study performed in Imam Khomeini hospital of Urmia on PACS problems in Iran, 4.38 percent of the administrative practitioners of the hospital and 6.21 percent of the staff were not aware of PACS benefits (26). Since the questionnaire was distributed among all subjects, this can be considered as one of the reasons for low effect of PU on PACS.

In a study performed by Bayat et al. in Iran, technical infrastructures have been introduced as the greatest barrier in utilizing health technologies in Iran (27). A closer look at the PU factors, makes it evident that facilitating the work and making it efficient as well as enhancing the quality of patient care and control over the system require a system with appropriate infrastructure than change factors. Hence, this factor differs with the study carried out by Aldosari (17).

On the other hand, the study performed by Aldosari (17) was performed in the radiology department. In the emergency department, due to the higher significance of diagnosis speed and efficiency of the health technology, it is expected that a change made by technology has more effect on the acceptance level.

In the study carried out by Duyck et al. (19), another model has been used to investigate the acceptance, but in that model, PU and PEU have been among the factors affecting the acceptance. The applied model has also predicted 33 percent of acceptance, which is approximately in accordance with the current model (19).

Among the other results of this study are the effects of age, gender, occupation, and educational degree on the four factors of acceptance, PU, PEU, and change. In this study, different age and gender groups did not have a significant effect on the investigated factors. Studies conducted by Duyck et al. (18) and Aldosari (17) also confirm this finding. In a review by Ward et al. (28), gender and age did not have a significant effect on the participants’ attitude towards health-related IT. In a study carried out by Duyck et al. (18), no significant difference of acceptance scores was observed among different occupations which accords with our results, although in the mentioned study, various medical specialty groups have not been compared with each other. In our study, the group with specialty degree had a higher mean acceptance score compared with the other three groups which is also statistically significant. Regarding the reasons for these results, it can be noted that this group has the highest application of PACS and on the other hand, working with the technology is easier for them and they do not just rely on the instructional courses to work with PACS (29).

Comparing the three hospitals with each other, Baqiyatallah hospital has a higher mean acceptance than the other two hospitals. This hospital has more robust infrastructures to set up a PACS system than the other two hospitals. It is worth mentioning that in our results, the use of PACS was also higher in Baqiyatallah hospital. Further studies are required in this area in order to justify this result.

Among the strengths of this study, it can be noted that according to our research, this study is the first one that has examined the acceptance in the emergency department. Based on studies performed by Ward et al. (28) and Kukafka et al. (30), when employing a technology, success of that technology can vary from section to section. On one hand, the emergency department is one of the departments that has the highest level of PACS system use in a hospital (12) and performing this study in the emergency department will largely contribute to the literature on PACS acceptance. On the other hand, this study has been carried out in the emergency department of three educational hospitals in Tehran that are among the most crowded and most referral emergency departments. Although emergency specialists are young people with more acceptance of new technologies, they are the main target of PACS in hospitals. According to the study conducted by Holden et al. (21), a key point in increasing the use of IT is to enhance the acceptance, and high acceptance in these emergency departments can be evidence for an overall increase in using PACS in Iran.

On the other hand, today, the model used in this study in order to assess the acceptance is the gold standard model (21, 31, 32). Despite being simple, this model constitutes 10 percent of the literature on the evaluation of acceptance (21).

According to the limitations of this study, it can be mentioned that using the resulted model, less than half of the acceptance changes can be predicated. In spite of being in accordance with other studies in this area, this means that there are unknown and uninvestigated variables that must be considered in future researches. Another limitation of this study is the low response rate and low participation of residents. This is a potential source of bias that residents with more positive attitude have more participation in our study.

In conclusion, PACS is highly accepted in all three hospitals especially among the specialists. PU, PEU, and change are factors influencing PACS acceptance. Our study can be used as an evidence of PACS acceptance in emergency wards. Further studies are needed in order to discover other factors influencing PACS acceptance or to compare the acceptance levels in different wards.