Abstract
Background:
Regularly monitoring of educational activities is of undisputed importance in residency training programs. Various educational and training activities can be recorded in a logbook, including giving a lecture, participating in training courses, morning reports, providing particular procedures and skills.Objectives:
The current study aimed to compare a newly launched web-based logbook to a paper-based logbook to identify potential strengths and weaknesses.Methods:
Emergency medicine residents of postgraduate year 1 (PGY1) and 2 (PGY2) and faculty members participated in the present study. Electronically collected data from February-April 2014 were compared to those of paper logbooks for the same period in the previous year. Furthermore, to evaluate the perspectives of both students and faculty members on the web-based logbook, a survey was implemented.Results:
In total 47 residents recorded their activities in the web-based logbook. For the same period in the previous year, logbooks of 40 residents were available. The frequency of work shifts in paper and web-based logbook was 668 and 1020, respectively (P = 0.4). However, a significant difference was observed concerning the work shifts recorded in the logbook and those confirmed by a faculty member [416 (62.3%) in the paper logbook and 945 (92.7%) in the web-based logbook, P value = 0.05]. Also, the frequency of unconfirmed procedures, review of medical cases, morning reports, rotations, and direct observation of procedural skills were significantly lower in the web-based logbook (P value = 0.0001). Based on the results of the survey, 55.55% of faculty members and 66.66% of residents reported the effectiveness of the web-based logbook.Conclusions:
This study demonstrated the potential of web-based logbooks to improve recording activities of residents and assisting faculty members to precisely monitor their activities.Keywords
Residency Emergency Medicine Education Assessment Resident Monitoring
1. Background
Regularly monitoring of educational activities is of undisputed importance in residency training programs (1). Various educational and training activities can be recorded in a logbook, including giving a lecture, participating in training courses, morning reports, providing particular procedures and skills (2). Unified logbooks help trainees and attendants to identify learning objectives and review development and self-assessment results. Attendants can also monitor the progress of their trainees and provide necessary feedback. Faculty members should independently evaluate their residents in the clinical competency evaluations, hence providing an annual report concerning the students' overall performance would be useful for a fair evaluation of their activities.
On a bigger scale, logbooks can guide residency directors, education authorities, and accreditation councils in monitoring the quality of medical education. Currently, the use of paper logbooks for medical students is considered out of date due to observed inconsistencies, which reflects the benefits of using computers to supervise educational activities and clinical practice of medical students (3, 4). More specifically speaking, graduates of emergency residency programs must have different skills and knowledge obtained in different settings and places. Therefore, creating standard electronic logbooks for emergency medicine residents and recording their activities is necessary to set up an organized program.
2. Objectives
The current study aimed to compare a newly launched web-based logbook to a paper-based logbook to identify potential strengths and weaknesses.
3. Methods
This was a cross-sectional study conducted in the emergency department of a major university-affiliated teaching hospital in Tehran, Iran. A web-based logbook was developed by the faculty members of the Health Informatics Department of the Tehran University of Medical Sciences to replace the old paper logbook. Emergency medicine residents of postgraduate year 1 (PGY1) and 2 (PGY2) and faculty members participated in the present study. Electronically collected data from February-April 2014 were compared to those of paper logbooks for the same period in the previous year (February-April 2013). Furthermore, to evaluate the perspectives of both students and faculty members concerning the web-based logbook, a survey was implemented. The survey consisted of seven subscales, which were scored on a seven-point Likert scale, ranging from one (strongly agree) to four (disagree).
An email-based survey was carried out amongst PGY1, PGY2, and faculty members who were using the newly developed web-based logbook. Those who were not logged in to the web-based logbook were excluded. Since this was a pilot study, the sample size was determined by the rule of thumb. We assumed that 40 participants would provide 80 percent power to detect differences between the web-based and paper logbooks. The survey was anonymous, and researchers were blind to respondents' identities. The study is approved by the Ethics Committee of the Tehran University of Medical Sciences. Data were analyzed using SPSS version 17 (SPSS Inc., Chicago, IL, USA). The chi-square test was used to compare the two groups. Statistical significance was considered when P value < 0.05.
4. Results
Eighty-seven emergency medicine residents were enrolled in the present study, that 47 of them recorded their activities in the web-based logbook. For the same period in the previous year, logbooks of 40 residents were available. The frequency of work shifts in paper and web-based logbooks was 668 and 1020, respectively (P = 0.4). However, a significant difference was observed concerning the work shifts recorded in the logbook and those confirmed by a faculty member [416 (62.3%) in the paper logbook and 945 (92.7%) in the web-based logbook, P value = 0.05]. Concerning the specific procedures, there was a significant difference between the two groups [4180 (92.7%) for the paper logbook and 6825 (98.5%) for the web-based logbook, P value = 0.0001]. Also, the frequency of unconfirmed review of medical cases, morning reports, rotations, and direct observation of procedures (DOPS) was significantly lower among those who were using the web-based logbook (P value = 0.0001) (Table 1).
Comparison of the Participants' Data Separated by the Group
| Item | Paper Logbook, No. (%) | Web Based Logbook, No. (%) | P Value |
|---|---|---|---|
| Shift | |||
| Confirmed | 416 (62.3) | 945 (92.7) | 0.05 |
| Unconfirmed | 252 (38.1) | 75 (7.3) | 0.05 |
| Total | 668 (100) | 1020 (100) | 0.4 |
| Procedure | |||
| Confirmed | 4180 (92.7) | 6825 (98.5) | 0.001 |
| Unconfirmed | 332 (7.3) | 104 (1.5) | 0.05 |
| Total | 4512 (100) | 6929 (100) | 0.02 |
| Review of medical cases | |||
| Confirmed | 60 (68.1) | 132 (100) | 0.4 |
| Unconfirmed | 28 (31.8) | 0 | 0.001 |
| Total | 88 (100) | 132 (100) | 0.7 |
| Morning report | |||
| Confirmed | 36 (50) | 122 (100) | 0.2 |
| Unconfirmed | 36 (50) | 0 | 0.001 |
| Total | 72 (100) | 122 (100) | 0.4 |
| Presentation | |||
| Confirmed | 28 (100) | 36(100) | 1 |
| Unconfirmed | 0 | 0 | 1 |
| Total | 28(100) | 36(100) | 1 |
| Rotation | |||
| Confirmed | 32 (66.6) | 57 (100) | 1 |
| Unconfirmed | 16 (33.3) | 0 | 0.001 |
| Total | 48 (100) | 57(100) | 1 |
| DOPS | |||
| Confirmed | 64 (76.2) | 113 (100) | 0.4 |
| Unconfirmed | 20 (23.8) | 0 | 0.001 |
| Total | 84 (100) | 113 (100) | 1 |
In total 18 faculty members (including 14 males and 4 females) and 60 emergency medicine residents (34 males, 26 females) filled the questionnaires. About half of the faculty members surveyed (55.55%) and over half of the residents (66.66%) agreed with the effectiveness of the web-based logbook. Also, half of the faculty members and 35% of residents agreed with having the opportunity to provide further feedback on the web-based system. Moreover, 66.66% of faculty members were "strongly agree" that evaluations are accurate. About 53% of residents also reported the same idea. Also, 61.66% of faculty members and 66.66% of residents were strongly agreed that web-based logbook is economical. Most of the faculty members (55.55%) and residents (70%) were strongly agreed that electronic registration and verification of activities are always available. Moreover, 61.11% of faculty members and 60% of residents were strongly agreed that web-based logbooks are more secure (Table 2).
Items of the Survey
| Number | Item |
|---|---|
| 1 | I prefer a web-based logbook over the paper logbook |
| 2 | Web-based logbook enables me to provide more feedback |
| 3 | Assessment of a web-based logbook is accurate |
| 4 | A web-based logbook is economic |
| 5 | Web-based logbook enables me to access to the activities at any time that I want |
| 6 | Web-based logbook provides more data security |
| 7 | I have a favorable viewpoint regarding the web-based logbook |
5. Discussion
The Web-based logbook allows students to record all educational activities and procedures during their training courses. Also, it allows faculty members to continuously monitor the performance of their students (5, 6). On the other hand, it encourages the residents to assess themselves by reviewing their activities and receiving feedback from faculty members (7). In the present study, the frequency of registered and confirmed shifts and procedures in the web-based logbooks was significantly higher than in paper logbooks. Also, we found that most faculty members and residents have a positive viewpoint about the web-based system. Recently web-based logbooks have turned into the core component of education systems and are using as a monitoring or evaluation tool for recording the performance of students (8, 9). It also enables the program directors to identify areas of incomplete training and to make appropriate corrections. The web-based system improves the supervision system and encourages cooperation between educators and residents which leads to improved methods of training. Some studies have indicated that logbooks could improve communications between residents and faculty members because each resident would have the opportunity to receive timely feedback from his/her mentor (10). Although web-based logbook can outperform the old paper-based logbooks, there are still problems that should be addressed. For instance, some studies have reported inaccuracies in the logbook database due to the possibility of inserting free texts rather than choosing predefined options in the logbook web page. They argued that the bugs should be fixed (10). Also, they emphasized that data entry in the web-based logbook should be simple and easy. Furthermore, logbooks should be periodically evaluated by the admins to ensure that they are up to date and user-friendly.
Another drawback of a web-based logbook is that it may cause anxiety among residents, particularly if they do not record their activities in due time. Also, patients' outcomes should be registered in the logbook, mainly because of its important role in self-learning (10). Meanwhile, it should be noted that all students do not have access to a laptop or smartphone, particularly in resource-limited developing countries (5). Considering the findings of the present study, the electronic recording of students' performance would be useful to identify the strengths and weaknesses of training programs as well as evaluating their activities. Nevertheless, electronic systems need periodic maintenance and updates to be used as an evaluation tool for grading the residents during their training.
In conclusion, this study demonstrated that web-based logbooks can positively influence the number of recorded activities in the residency program. Moreover, such systems improve the accuracy of evaluations and are more time-saving than paper-based logbooks. It is crucial to evaluate the effect of web-based systems on the quality of residency programs in the future.
References
-
1.
Lie D, Trial J, Schaff P, Wallace R, Elliott D. "Being the best we can be": medical students' reflections on physician responsibility in the social media era. Acad Med. 2013;88(2):240-5. [PubMed ID: 23269290]. https://doi.org/10.1097/ACM.0b013e31827bf5fc.
-
2.
Strang TI. Anaesthetic log books. Anaesthesia. 2007;48(1):69-74. https://doi.org/10.1111/j.1365-2044.1993.tb06799.x.
-
3.
Kannan J, Kurup V. Blended learning in anesthesia education: current state and future model. Curr Opin Anaesthesiol. 2012;25(6):692-8. [PubMed ID: 23147669]. https://doi.org/10.1097/ACO.0b013e32835a1c2a.
-
4.
Barbieri A, Giuliani E, Lazzerotti S. Log-book implementation in anaesthesiology residency training: a retrospective analysis. Med Teach. 2012;34(6):513. [PubMed ID: 22489977]. https://doi.org/10.3109/0142159X.2012.675100.
-
5.
Jameson SS, Lamb A, Wallace WA, Sher JL, Marx C, Reed MR. Trauma experience in the UK and Ireland: analysis of orthopaedic training using the FHI eLogbook. Injury. 2008;39(8):844-52. [PubMed ID: 18632101]. https://doi.org/10.1016/j.injury.2008.03.030.
-
6.
Achuthan R, Grover K, MacFie J. A critical evaluation of the electronic surgical logbook. BMC Med Educ. 2006;6:15. [PubMed ID: 16509987]. [PubMed Central ID: PMC1413530]. https://doi.org/10.1186/1472-6920-6-15.
-
7.
Khorashadizadeh F, Alavinia SM. Students' perception about logbooks: advantages, limitation and recommendation - a qualitative study. J Pak Med Assoc. 2012;62(11):1184-6. [PubMed ID: 23866407].
-
8.
McGrath BP. Integration of overseas-trained doctors into the Australian medical workforce. Med J Aust. 2004;181(11-12):640-2. [PubMed ID: 15588194].
-
9.
Witt A, Iglesias S, Ashbury T. Evaluation of Canadian family practice anesthesia training programs: can the Resident Logbook help? Can J Anaesth. 2012;59(10):968-73. [PubMed ID: 22836577]. https://doi.org/10.1007/s12630-012-9759-8.
-
10.
Watters DA, Green AJ, van Rij A. Requirements for trainee logbooks. ANZ J Surg. 2006;76(3):181-4. [PubMed ID: 16626362]. https://doi.org/10.1111/j.1445-2197.2006.03663.x.