Developing a Web-Based Learning Management System (LMS) for Master’s Thesis Process in the Shahid Beheshti University of Medical Sciences

1. Background

The process of developing a thesis is often a big step toward graduation for the students in higher education. They often experienced new challenges in education. They have to make new understandings and meaning. Specific goals for developing a thesis include learning how to research and developing a scientific attitude, acquiring literacy skills, and promoting critical thinking. These characteristics make the thesis a valuable source of knowledge creation for those interested in the subject area (1).

As writing a thesis is time-consuming and is completely different from usual courses with the main interaction of students and teacher in class time, a mechanism to facilitate and organize students and supervisor interaction is very helpful. The initial step started with critical thinking and many influencing factors, such as access to information resources and supervisors’ support. Lack of proper management of the thesis process causes many risks and problems. For example, students cannot choose a topic effective for society, or they may not be able to provide useful data if they do not choose a valuable subject. Proper management of the thesis process must include all steps. This requires proper communication and a link between the student, the supervisor, and the necessary information resources system (2, 3).

In practice, a large number of students, the low ratio of the supervisor to students, and the cumulative increase of students in previous semesters have challenged the process of proper communication, integrated supervision, and accountability (1). On the other hand, the tasks of university administration are limited to registration and course management. Also, the management of the supervisor’s time needs a new business model.

Today, the industry and university provide an appropriate communication system support (4-7). The Web-based software learning management system (LMS) provides applications used for planning, implementing, and evaluating the learning processes, and also for producing the content and monitoring collaborators and educators. It provides interactive features, such as disciplinary discussions, video conferencing, and discussion forums (8, 9). LMS helps higher education institutions to offer distance courses and discussion forums and makes the faculty-student and student-student interactions through peer feedback, faculty feedback, and communication (8-10). Using this software is growing exponentially over time because it is fair and makes the users able to access the needed information (11).

Hansson et al. introduced using ICT for thesis support and indicated the importance of collaboration among the components of a thesis for improving its quality (10).

Hansson et al. implemented collaborative supervision for doctoral theses (3). They applied several main factors, such as project management, discussion forum, resource websites, reflective journals, private correspondence files, exercises, and course material databases.

Another research in South Africa, in the Department of Management at the Durban University of Technology, implemented a web-based (WebCT) system for research management among the postgraduate students from 2005 to 2006 (3).

The SciPro (Supporting Scientific Processes in Thesis Writing) system, which was developed at Stockholm University, is an online ICT support system for administration and supervision of the bachelor, master, and PhD thesis. It was designed to support thesis writing and fulfilling all stakeholders’ needs. The SciPro system was continuously developed, and as a result, the SciPro version 3 provides a wide range of facilities for the thesis initiation step, such as the idea bank. The SciPro system, which is specifically used today all over the world to do the thesis, caused an improvement in the quality and quantity of theses. It evaluates the performance of the thesis process from many important aspects, such as an increase in the rate of thesis completion, an increase in student autonomy, a shorter time spent by the supervisor regarding the beginners' questions, and an improvement in providing feedback and thesis quality (3, 4, 6, 11). However, there is no evidence in this regard in Iran.

Scientifically, research is a scientific activity that is consciously, systematically, and creatively formulated and developed. Also, education scholarship not only considered the quality and quantity of education but also it regards educational activities to apply knowledge in solving educational problems. However, these educational activities are provided to experts to critique and evaluate (12, 13).

Over the past years, doing a thesis has been one of the issues in the Research Department of the School of Management and Medical Education of Shahid Beheshti University with more than 400 virtual students. After obtaining sufficient evidence, thesis support for the new students has been conducted formally through the LMS. LMS is a user-friendly system, and the virtual students are familiar with it because they take their courses using this system.

2. Objectives

The aim of this study was to develop the web-based LMS for master’s thesis support and assess its quality in comparison with common alternative methods, such as face-to-face meetings, email, and social networks from the viewpoint of the supervisors and faculty.

3. Methods

This study is a translational research design focusing on the adaptation of educational technology. The study was carried out in the school of Management and Medical Education of Shahid Beheshti University of Medical Sciences. The study Population were 67, 65, 51 students of virtual master community-based education in cohorts registered in 2017, 2018, and 2019, respectively.

The study had three stages: Preparation, Implementation, and Assessment. In the preparation stage we designed a course in the Moodle based learning management system (LMS) of the school to fit to supervision requirements of thesis supervision. In the implementation stage carried out by students and supervisors were briefed about the specifications and functions of the platform that facilitated the supervision and used it for supervisory interaction during thesis development. In the evaluation stage, the program was evaluated based on quality criteria (Figure 1).

The process of the three phases of the study

Figure 1.

The process of the three phases of the study

Three stages of the study were described in detail as follow:

Preparation stage before the implementation thesis management using LMS, the following technical support requirements were considered in the website:

- Facilitating the communication between the faculty and the student during the thesis process, such as having the system available 24 h a day for questioning or submitting files and faculty feedback.

- Facilitating access to the resources needed by the student because the supervisor has several students at the same time.

- Writing discussions between the supervisor and the student during the whole process to review the progress of the work.

- Facilitating the submission of multiple student files, keeping track of their history, and facilitating faculty feedback to each file, and making any necessary corrections.

- Facilitating handling a large number of students who share a single supervisor.

- Securing the information so that all documents and files are visible only to the students and their supervisors.

- Documenting the student activity from the beginning to the end of the process so that the faculty can evaluate the student's work and also the accuracy of the student's data and work process can be better tracked.

- Possibility of evaluating the activity of the faculty in guiding and answering students and checking his / her activity level.

3.1. Implementation

Pairs of student- supervisor were assigned to a course on the LMS of the school at the beginning of the thesis development process. All communication and interaction were managed through this e-environment. The supervisor was able to give written feedback on every report the students prepared and to communicate his/her comments. All questions were asked, and the answers were recorded, which facilitates students reviewing the supervisor's comments and answers as many times as needed. The supervisor was able to activate reminder for important dates, and the student was informed and reminded of agreed upon time frames. This helps the student to better organize their due activity to meet the specific deadlines. The supervisor also had a comprehensive image of the whole process of the thesis development for each student with the exact progression of the project recorded. Students at risk of prolongation of the thesis process were readily identified by the supervisor. This provides the possibility of taking action in due time and prevented unnecessary delays. The students had more protected access to their supervisor. They had all activities and reports recorded. They monitored their progress based on the agreed upon schedule. They were able to use the LMS as a project manager since the communication of reports, feedbacks comments time tables, and even meeting times al were recorded in their supervisory course in the LMS.

3.2. Evaluation

Evaluation was conducted in four steps: Literature review design, or make a checklist criteria, focus group discussion, cite visits.

Step 1: A literature review for identifying the criteria of applying LMS in higher education and thesis management was conducted using non -systematic review in the literature by the keywords, such as learning management system, distance learning, medical education, thesis support, collaborative learning, interactive learning, and theses.

Step 2: making a checklist of criteria. In this step, 22 criteria were extracted, including privacy, accessibility, simultaneous access to announcements, etc. for all students, the unification of methods, providing key content and guidance to everyone at the same time, satisfaction, sympathy, interaction, cooperative learning, supervision by the faculty, Justice in responding, monitoring and supervision by the head of the department, motivating, increasing quantity, quality enhancement, time-saving, loading multiple forms, timely notification, planning, identifying problematic students, competition, and identifying common weaknesses of the core knowledge (Table 1). Only one criterion, i.e., identifying problematic students, was specific according to the faculty’s view.

Step 3: Focus group discussion. Content validity of the extracted criteria was assessed by focus group method and a panel of 10 experts considering medical education, health policy, medical ethics, and community medicine in two sessions, and a checklist of the criteria was developed. The criteria were scored subjectively from zero as the lowest value and 10 as the highest value. A score of less than four was considered less desirable, a score of more than 7 was Highly desirable, and any number in between was considered moderate. For measuring reliability, the test-retest method was used. The criteria checklist was provided to 15 students with an interval of 14 days. Then, the Pearson correlation coefficient were calculated (R = 68%; P-value = 0.044).

Step 4: cite visit. This step was done to assess the quantity of the interactions between professors and students and completed thesis.

The data were analyzed by Excel and SPSS (v22) software and descriptive indices, such as frequency, mean, and standard deviation (SD). The results were compared using analysis of variance (ANOVA) and post-hoc tests.

4. Result

The 22 quality criteria were extracted by literature review and validated (Tables 1, 2 and 3). Descriptive result of the 67 students of the first cohort and their supervisors after 2 years of follow up was presented in Table 1. The result showed the scores of all criteria in applying the LMS system in comparison to other alternative methods were at the highest desirable level (mean score > 7). Providing key content and guidance had the highest score.

Mean scores of perspective the social network and email mostly were at the moderate level, and meeting mostly was less desirable (mean score < 4).

Table 2 indicates the mean and SD of the quality criteria from the supervisors’ perspective. According to the results, the scores of all criteria after using LMS in comparison with other alternative methods were at the desirable level. Also, the mean scores of providing key content and guidance and monitoring and supervision were mostly desirable. The social network was found at a moderate level, and the mean scores of meeting and email were less desirable (mean score < 4).

The results of ANOVA provided in Table 3 indicated that the mean difference of most criteria was statistically significant at a 95% confidence interval (CI). However, some criteria, such as supervision by the faculty, identifying common weaknesses of the core knowledge, and justice in responding to information, were not statistically significant at a 95% CI from the students’ perspective (Table 3).

The results of the ANOVA regarding the use of alternative methods from the supervisors’ view are also indicated in provided in Table 3. The mean difference between the scores of criteria, such as motivation, quality enhancement, interaction, and accessibility was not statistically significant in 95%CI. The result post-hoc comparisons was also significant for all comparisons and the result of the F index was statistically significant for studied criteria.

Table 4 showed the viewpoints of the supervisors and students. The result indicated the scores of the applying LMS system in comparison from the viewpoints of the supervisors and students were the highest scores (95% CI: 8.76 ± 0.83) and (95% CI: 8.71 ± 0.64) respectively (Table 4).

The number of feedbacks, except for problematic students, varied from at least 6 to 64 times, which was impossible or extremely difficult for a virtual student to travel to Tehran city during the follow-up period. We found that most of the criteria, such as interactions, feedback, updates, and documentation economical. Also, after the implementation of the program, 51 out of 67 students completed their theses during the follow-up period.

5. Discussion

The results of this study indicated that implementing LMS for thesis support was almost twice more successful in achieving the quality criteria than other alternative methods from the perspective of both students and supervisors. The results of ANOVA indicated that the mean difference of the scores of all criteria showed a statistically significant increase after using LMS compared with alternative methods.

Consistent with our study, Byungura (2015) at Rwanda University emphasized the implementation of an E-learning management system for thesis support. They found that there are several students and a limited number of tutors, which diminished the quality of theses as the supervisors did not have enough time to interact with students. This study concluded that for the implementation of the system, the status of the students should be examined, and also the requirements are needed to be considered (4).

Hansson et al. were the first to use the ICT system for thesis support and indicated that a good collaboration and high-quality thesis (7, 10). Hansson et al. implemented LMS for a doctoral dissertation and reported an improvement in thesis quality. They reported many main requirements for the system implementation, such as project management, discussion forum, exercises, reflective journal, resource websites, private correspondence files, and course material databases (3).

Hansson et al. conducted a study in order to facilitate thesis processes considering how to accommodate a huge number of students with the low number of supervisors and resources. They used the ICT system and developed the idea bank to initiate a good start. Data were collected by observations, interviews, focus group discussions, and log analysis during 1.5 years. They indicated an improvement in the quality of the thesis process, saving time, and adding value (3, 6, 10).

A study by Larsson revealed that 30% of the students of the Department of Computer and Systems Sciences at Stockholm University, admitted for the Master's thesis, did not finish their thesis during 2001 - 2006 because of a large number of students (up to 300,000 students in 2002) that increased in next years, and a limited number of supervisors and resources. They conducted a partial pilot study regarding the implementation of the ICT system in 2011 and found positive results both regarding quantity and quality (14).

The results of the study by Nouri et al. indicated that the implementation of the IT-system (SciPro) for 300 students and 70 supervisors was usable for the campus student and distance student. They also presented many features and functionalities of the SciPro System (15).

Peiris et al. conducted a case study on ICT for thesis support. They indicated that ICT is highly supportive for thesis management (16). Larsson et al. assessed factors affecting the thesis process from 2010 to 2014. They assessed the changes over time, dropout rates, and the time for completion after the registration of a student. Their provided many recommendations regarding the thesis process and making immediate changes for its management (17).

In the present study, it was also indicated that after the implementation of the program, 51 out of 67 students completed their theses during the follow-up period. In line with this result, Nouri et al. studied the bachelor thesis dropout rate. They assessed 2436 theses of the Bachelor's students. Their results indicated that the supervisors’ experience and capability play the main role in determining the achievement of the thesis (18).

A study by Washington demonstrated the experiences of the use of LMSs in comparison with traditional methods, such as face-to-face courses. He reported that it is possible to better understand the educational potential of the LMS to enhance traditional face-to-face courses (19).

Dana et al. in the E-Learning Center of Tarbiat Modares University indicated dissatisfaction with the quality of the courses from the student’s point of view (20). Abhari et al. in the Shiraz University assessed 200 postgraduate students' points of view on a thesis tele-supervision system and reported a low accessibility score (21). The LMS is a crucial web-based platform that provides software (web) application used for planning, implementing, evaluating learning processes, producing content, and presenting and monitoring collaborators and educators (8, 9).

A study by Zuriyati and Kadir on the Malaysian higher education system showed the effectiveness, efficiency, and student satisfaction at a high level after using LMS (22). Another study by Hiltz discussed the importance of measuring collaborative learning using distance learning tools compared with traditional methods (12). Also, cost savings due to fair access and resource sharing have improved the efficiency and use of advanced tools over time. In these advanced versions, special attention has been paid to the ways of enhancing the interactive, integrated, and flexible learning interactions. In 2005, Hammond, in his study, referred to the creation of good discussion forums and very good interaction between students and faculty and students together (7). In another study, the authors emphasized not having the time and place limitations to access the forums and participating in discussions (8). In this study, they indicated an opportunity for those who are not able to attend the classroom to pursue further education and considered this feature as efficient use of time in today's world (8). Persico et al. highlighted the feature of computer-supported collaborative learning (CSCL) (23), and Abel referred to the feature of documentation and the ability to retrieve information and the content of web-based interactions and feedback (24).

One of the main reasons for this success was the availability of infrastructures in the school and the familiarization of students and faculty with the distance learning system. The existence of infrastructure and the use of its capacities made it a cost-effective way from different viewpoints. Due to a large number of students and adding students in each academic year, and limited recourse, this approach, with integrated monitoring and supervision system, has provided qualitative and quantitative performance management to students and faculty, and also can improve the quantity and quality of theses.

LMS, such as the Scientific Process System (SciPro) is not a specialized thesis process management system, but most of its features that are defined to perform in the mentioned system were equated and applied using the built-in features of the LMS. These include synchronous and asynchronous communication, private messages, forum, scoring various activities, checking, uploading, and storing student personal files, as well as the availability of files and resources (22). It should be noted that the ability to add required features tailored to the specialized thesis monitoring systems is not out of the reach of an open-source system.

Given that these studies are not limited to virtual students, and given the ease of learning and the high capacity of this system, all faculty and students can use it even in general courses (25, 26). Studies on the use of social networks for doing thesis by Aghaee (2) and Hansson (3-6) also highlighted the use of social networks to facilitate the thesis process, and this leads to the solve problems.

One of the weaknesses of this study was the lack of assessing the simultaneous application of LMS and social networks in order to better thesis management. Social networks are widely used in education today. One of the strengths of our study was the comparison of social networks compared with LMS. Given the academic benefits of the LMS (27, 28) and its use in higher education, it is superior to social networks for the following reasons:

- The e-learning system is an open-source system. Due to the modular structure of the system, it is possible to add, modify, or improve the features required as custom plugins in the system. The ability to define specific roles with varying permissions to different contexts enables the division of work at different levels (student, faculty, and administrator). Storing users' data on a secure server (Linux operating system) is done with high security.

- Activities, like forums, messages, online classes, chat, etc. through the courses, allow for structured interaction between students and the faculty. It should be noted that to access existing courses and activities only the browser is required and no special software needs to be installed. Course content can be accessed, both online and offline, via the Moodle mobile application.

- It is possible to monitor the individual and group activity of users in the system and to evaluate their performance in different activities in different ways and the results can be presented in different formats.

5.1. Conclusion

The findings showed that the implementation of this ICT-based program caused an improvement in the supervision process and made it possible for supervisors to provide the necessary knowledge to a large number of students in a faster and easier way than traditional methods using the capacities available in the system while saving time, facilities, and reduced the workload. Also, in order to improve the quality and social need responsiveness. It is s suggested to make wide-ranging facilities, such as access multidimensional information resources and developing idea bank in the system.