Identifying the Weaknesses of Formative Assessment in the e-Learning Management System

1. Background

With the development of e-learning at universities and higher education institutions, the issue of assessment has become particularly significant in the quality of the teaching and learning process in higher education because assessment is an integral part of any educational system (1). Without assessment, e-learning cannot be a desirable learning environment, nor can its quality be enhanced (2). Assessment is a key element of the learning process, aiming to determine the extent to which educational goals are achieved. E-learning assessment can be either formative, summative, or combined (3). Formative assessment can promote learning and give quality assurance in regard to distance and electronic learning (4). It is a process used by teachers and students to identify and respond to learners’ learning and enhance it throughout the learning process (5). Summative assessment is used both at the end and during the course in the form of a midterm or final exam. It aims to determine the level of student learning in critical behaviors, and not the ones taught to students (6). However, what has drawn the researchers’ attention is that the assessment method, particularly the formative assessment of learners in the Iranian e-learning system, has numerous weaknesses. Farhangi et al. (7) showed that preparing and presenting assessment reports in e-learning centers was one of the problems in this field. Also, Abbasi Kasani et al. (8) demonstrate that there are setbacks in the field of assessment in e-learning environments, such as lack of specific standards for assessment and uncertainty of assessment accuracy.

Therefore, the e-learning centers of Iran suffer from many weaknesses in formative assessment, and they cannot perform well, unless they identify these weaknesses and attempt to overcome them. In fact, e-learning centers in Iran, by identifying their weaknesses in the field of formative assessment and trying to eliminate the weaknesses, can perform the assessment well and ensure the improvement of the quality of learners’ learning. The e-LMS of Shahid Beheshti University as one of the LMS in the country suffering from a number of setbacks.

2. Objectives

The purpose of this study was to identify the weaknesses of formative assessment in the e-learning management system of Iran (case study: Shahid Beheshti University).

3. Methods

This study was applied in terms of objective and qualitative in terms of data collection. The study population was the students of the e-learning Center of Shahid Beheshti University, 15 of whom were selected as the research sample using the purposive sampling method. In purposeful sampling, the researcher continues to achieve theoretical saturation, and in this study, the researcher reached theoretical saturation with the 15th person. Moreover, the research instrument comprised semi-structured interviews. In the first part of the interview, an introduction (purpose, request, and definition of the terms, and conceptual framework) was presented, and in the second part of the interview, questions were raised regarding the weaknesses of formative assessment in the learning management system of Shahid Beheshti University. The validity and reliability of the research instrument were examined through confirmation by the professors and experts in the field, and self-review methods of researchers.

To analyze the findings, the audio recordings were first transcribed in Microsoft Word as a text document. After the interviews, the themes were coded using content analysis or thematic analysis, and for ease of coding and categorization, all speech cues from the interviews were embedded in Maxqda 10. The data were also analyzed using this software. In this regard, the concepts were first extracted, then categorized, and finally the categories were defined.

4. Results

This section analyzes the findings of interviews with the students of the e-Learning Center of Shahid Beheshti University. Table 1 shows what the interviewees mentioned about the assessment weaknesses in the LMS of Shahid Beheshti University.

Table 1 presents the basic concepts and main categories extracted. Accordingly, the initial 54 concepts were embedded into 11 main categories. The following section describes the categories obtained from the students’ interviews.

5. Discussion

5.1. Theme 1: Weakness in Infrastructure Technology Resources

Infrastructural and technological resources are highly important for the effective administration of formative assessment, but the e-LMS of Shahid Beheshti University has serious weaknesses in this respect. These resources are the foundation of e-learning. Miller and Rector (9) argue that most organizations encounter countless issues in establishing communication in e-learning, such as low Internet speed, bandwidth limitations, and telephone and Internet outages. Shafiei Sarvestani et al. (10) also consider the low Internet speed and sudden disconnections as e-learning weaknesses.

Consistent with the findings of this study, the study of Seraji et al. (11) showed that the lack of equipment and hardware and software facilities was one of the weaknesses of e-learning, which has a direct impact on the function of this type of educational system.

5.2. Theme 2: Not Using Other Formative Assessment Tools

There is a need for accurate knowledge of assessment tools to evaluate learning in the sense that every teaching situation requires a specific assessment tool of its own. e-learning employs different tools for evaluating learners, some of which may be similar to the tools used in traditional classrooms (12). As the interviewees have stated, formative assessment in the e-LMS of Shahid Beheshti University is limited to few tools. They believed that a set of assessment tools was not used throughout the teaching process. Only were few tools employed which could not measure the student learning alone

The findings of the present study are substantiated by the results of Beyrami and Rahimirad (13), according to which there are weaknesses in assessment tools in e-learning environments, such as the use of traditional assessment methods, the incompatibility of assessment methods with the nature of virtual courses, and failure to use a variety of assessment methods.

5.3. Theme 3: Weakness in Giving and Receiving Feedback

Feedback is one of the underlying features of formative assessment. It is a process for interpreting learners’ current status, needs, future potentials, and the way to fulfil those potentials. Feedback can have positive effects on learners’ learning and participation; however, interviewees held that the feedback provided by e-learning was not adequate, immediate, and well-considered. It should be noted that immediate feedback is usually more effective than delayed feedback (14) because immediate feedback can quickly lead to the identification of learners’ mistakes, so the problem can be solved before incorrect information is consolidated in learners (15). Feedback may have a positive or negative effect on learners’ motivation and self-confidence, and that such emotions affect their participation in the learning process. As a result, it must be continuous and powerful enough to encourage the learners. The findings of Kearns (16), which are consistent with the present study, referred to weaknesses such as lack of proper and timely feedback and delays in feedback.

5.4. Theme 4: Inability to Authenticate

One of the challenges of e-learning is the inability to verify the identity of individuals attending classes or assessment. Lack of sufficient tools to track user behavior is one of the most important problems in the LMS (17). Only is information about the login and logout time of students recorded in traditional methods. Users can log in to the LMS, then leave the system and come back in a few minutes near the end of the class without anyone noticing. Fenu et al.’s research (18) referred to the problem of verifying learners’ identities in electronic environments, which is consistent with the results of this research.

5.5. Theme 5: Weakness in Class Presentations

Class presentations, if used well as a formative assessment tool, can play a significant role in improving learning as well as identifying strengths and weaknesses. During the learning process, professors sometimes ask learners to give presentations to evaluate their learning process. These presentations can be in the form of posters, PowerPoint, oral, etc. The PowerPoint presentation as a form of assessment has empirical validity because it provides learning opportunities through feedback and peer assessment (19). Also, if class presentations are based on a regular sequence and the discussions are organized and clear, the students can concentrate on the topics and understand them more effectively (20). The professors and even presenters can ask questions before their presentations to remind the learners of the relevant topics in the previous sessions and introduce their main discussions. However, in the e-LMS of Shahid Beheshti University, the situation is different as the class presentations have been somewhat neglected.

5.6. Theme 6: Weakness in Exercises and Projects

Exercises and projects can be a reference for learning evaluation if they are purposeful and based on student tendency and interest Assessment through projects and exercises can have several advantages. For example, project-based assessments are dynamic, flexible, and expansive. These projects can have a significant impact on education due to their applicability, and creativity, and innovation. Unlike the formal and substantive limitations of the summative assessment including essay, multiple-choice, or practical questions, project-based assessment has a high degree of freedom in terms of what form they can take and enhances higher levels of cognitive development such as complex and innovative problem-solving (21). Nonetheless, the interviewees reported that classroom exercises as well as projects and articles used by the professors have their own problems, which can affect the learning process.

5.7. Theme 7: Weakness in Online Tests

Should the conditions of conducting a variety of online tests be well provided, one can obtain very useful information on learners’ learning status and provide prompt feedback in addition to identifying their strengths and weaknesses in a timely fashion. Most LMS are capable of creating and processing online tests. These tests can also be used for self-assessment, which allows the user to track their progress and review their acquired knowledge (12).

5.8. Theme 8: Weakness in Discussions

In e-learning environments, learning and participation of learners can somehow be evaluated through presenting questions to learners and observing how they discuss them. This theme shows that the e-LMS of Shahid Beheshti University faces various problems which have rendered the formative assessment tool not useful.

Discussion tools in the LMS are applicable to almost the whole process of formative assessment. Instructors can use both chat and discussion rooms in different ways, such as discussing films, lectures, student essays, etc. These tools are employed to determine learners’ knowledge and skills since learners cannot argue without knowledge and must demonstrate skills in using knowledge to reason in comments or responses (22). At the same time, discussions can bridge the gaps in learners’ knowledge and show areas for further learning. In discussions, learners can also receive feedback from their peers.

5.9. Theme 9: Inattention to Formative Assessment

This theme shows that no special attention has been paid to formative assessments in the e-LMS of Shahid Beheshti University. Instead, more emphasis is placed on summative assessments, which merely seek to measure what is learned, not improve learning.

The interviewees believed that despite the important role of formative assessment in reforming and improving learning, the professors do not pay any attention to formative assessment. In fact, they are more interested in teaching educational content that assessing during instruction and so place more emphasis on the final exam. In fact, if assessment serves as a tool in the learning process, it should be chosen in a way that it can measure a variety of learner behavior, allow the learner to make adjustments to the lesson plan, and improve learning. The findings of Mousavi et al. (23) also in accordanve with results found in the present study.

5.10. Theme 10: Restrictions on Uploading Assignments and Activities

As for class assignments, the students should always submit files in a variety of formats to their classmates or professors. According to the interviewees, the e-LMS of Shahid Beheshti University has faced some limitations in this respect because it is not easy to upload files.

According to the interviewees, activities and assignments are not easily uploaded in the e-LMS of Shahid Beheshti University due to the possible technical errors, file format, and even file size.

5.11. Theme 11: Observance of the Ratio of Teachers to Students

According to the interviewees, due to the virtual nature of classes, the number of students in these classes is usually too high and the required proportion is not observed in this regard.

Some believe that the participation of a large number of students is a benefit in e-learning classes. However, they do not realize that this will make learning assessment very difficult. The interviewees stated that the number of students in each class was very high, so it was impossible for the professors to pay attention to all students. As a result, that there was virtually no opportunity for proper feedback and identification of all students’ strengths and weaknesses. Adherence to the teacher-student ratio allows the professor and students to have more contact with each other, enabling students to acquire more information (24) and to be provided with ample continuous feedback (25). When the number of students increases, professors often do not have the opportunity to support each student (26).

5.12. Conclusions

Overall, the results showed that assessment and especially formative assessment in LMS tem include 11 main weaknesses according to the students of this center. The students at the center believed that these weaknesses have a great impact on their learning quality and prevent them from achieving the main goal of formative assessment, which is learning improvement. In formative assessment, both learners and professors need to recognize their weaknesses and strengths, but in the wake of the aforementioned weaknesses, they cannot be expected to gain a comprehensive understanding of the quality of learning. As such, they can perform well, only if they recognize these weaknesses and try to ameliorate them. In fact, e-learning centers should be able to rigorously identify the challenges and weaknesses in the field of formative assessment and provide operational solutions for them, so they can fulfil a better function in assessment and guarantee improvement in the quality of learning.