1. Background
With the development of e-learning at universities and higher education institutions, the issue of assessment has become particularly significant in the quality of the teaching and learning process in higher education because assessment is an integral part of any educational system (1). Without assessment, e-learning cannot be a desirable learning environment, nor can its quality be enhanced (2). Assessment is a key element of the learning process, aiming to determine the extent to which educational goals are achieved. E-learning assessment can be either formative, summative, or combined (3). Formative assessment can promote learning and give quality assurance in regard to distance and electronic learning (4). It is a process used by teachers and students to identify and respond to learners’ learning and enhance it throughout the learning process (5). Summative assessment is used both at the end and during the course in the form of a midterm or final exam. It aims to determine the level of student learning in critical behaviors, and not the ones taught to students (6). However, what has drawn the researchers’ attention is that the assessment method, particularly the formative assessment of learners in the Iranian e-learning system, has numerous weaknesses. Farhangi et al. (7) showed that preparing and presenting assessment reports in e-learning centers was one of the problems in this field. Also, Abbasi Kasani et al. (8) demonstrate that there are setbacks in the field of assessment in e-learning environments, such as lack of specific standards for assessment and uncertainty of assessment accuracy.
Therefore, the e-learning centers of Iran suffer from many weaknesses in formative assessment, and they cannot perform well, unless they identify these weaknesses and attempt to overcome them. In fact, e-learning centers in Iran, by identifying their weaknesses in the field of formative assessment and trying to eliminate the weaknesses, can perform the assessment well and ensure the improvement of the quality of learners’ learning. The e-LMS of Shahid Beheshti University as one of the LMS in the country suffering from a number of setbacks.
2. Objectives
The purpose of this study was to identify the weaknesses of formative assessment in the e-learning management system of Iran (case study: Shahid Beheshti University).
3. Methods
This study was applied in terms of objective and qualitative in terms of data collection. The study population was the students of the e-learning Center of Shahid Beheshti University, 15 of whom were selected as the research sample using the purposive sampling method. In purposeful sampling, the researcher continues to achieve theoretical saturation, and in this study, the researcher reached theoretical saturation with the 15th person. Moreover, the research instrument comprised semi-structured interviews. In the first part of the interview, an introduction (purpose, request, and definition of the terms, and conceptual framework) was presented, and in the second part of the interview, questions were raised regarding the weaknesses of formative assessment in the learning management system of Shahid Beheshti University. The validity and reliability of the research instrument were examined through confirmation by the professors and experts in the field, and self-review methods of researchers.
To analyze the findings, the audio recordings were first transcribed in Microsoft Word as a text document. After the interviews, the themes were coded using content analysis or thematic analysis, and for ease of coding and categorization, all speech cues from the interviews were embedded in Maxqda 10. The data were also analyzed using this software. In this regard, the concepts were first extracted, then categorized, and finally the categories were defined.
4. Results
This section analyzes the findings of interviews with the students of the e-Learning Center of Shahid Beheshti University. Table 1 shows what the interviewees mentioned about the assessment weaknesses in the LMS of Shahid Beheshti University.
Table 1.Categorization of Concepts Emerged From Students’ Perceptions of Formative Assessment Weaknesses in E-LMS of Shahid Beheshti University
| Concepts | Categories | Quote |
|---|---|---|
| Internet disconnection | Weakness in technology infrastructure resources | “Another weakness of our classes in general, and especially during class discussions, presentations, and the like was our disconnection with the class. Because the Internet was disconnected and we were being dropped out of class, thus we were considered as absent. Code 2” |
| Limited software platform | ||
| Limited hardware platform | ||
| Low Internet speed | ||
| Impossibility of using smart board simultaneously | ||
| Defects of systems and equipment | ||
| Volume decrease | ||
| Assessment limited to few tools | Not using other formative assessment tools | “During these three semesters, our grades were composed of class presentations, end-of-semester exams, and practical work. Code 4” |
| Not using peer reviews and assessment | ||
| Failure to provide prompt and timely feedback | Weakness in giving and receiving feedback | “Not having enough time on the part of the professors to review and provide feedback was itself a reason why the professors are unable to give feedback and tell the students where they have problems so that they make an effort to fix them.” |
| Professors’ feedback being time-consuming | ||
| Wrong feedback from professors | ||
| Insufficient opportunity for professors to provide feedback | ||
| Unclear class attendance or assessment | Inability to authenticate | “Another problem is that the students ditch the class. I mean, they turn on the system and then leave. But there the professor sees s/he is logged-in. Code 6” |
| Miscellaneous class presentations and using them as the criterion for summative assessment | Weakness in class presentations | “Due to the small number of sessions, the number of presentations in a session is usually two or more, and this may not give the professor any chance to evaluate the student’s performance and provide accurate feedback. Code 6”; “Because the presentations had a lot of content being read by the presenter, usually the professors would not listen very much and very few students would listen. Code 9” |
| Long and intensive class presentations | ||
| Plagiarized class presentations | ||
| Impossibility of knowing student learning with the help of class presentations | ||
| Little interaction in class presentations | ||
| Students reading from class presentations | ||
| Lack of attention by professors and students to class presentations | ||
| Difficult access to class presentations | ||
| Time limit for class presentations | ||
| Inappropriate and irrelevant class presentations | ||
| Disorganized class presentation schedules | ||
| Subject discrepancies in class presentations | ||
| Too many class presentations per session | ||
| Time-consuming class exercises | Weakness in exercises and projects | “Exercises and projects have disadvantages in the learning management system, in the sense that some of these projects are not interesting for the students, and any material, that is contrary to the ideas, beliefs, and interests of the student does not amount to anything, despite trying. Code 5” |
| Students’ lack of interest in some class projects | ||
| Failure to convince students to do projects | ||
| Little attention to the purposefulness of class exercises | ||
| Limited online test time | Weakness in online tests | “We did not have an online test during this period and even if there were, it could not have been very accurate because we have access to our materials and books and we can easily use them to answer the questions. Code 7” |
| Failure to measure higher levels of learning in online tests | ||
| No online tests | ||
| Possibility of cheating in online tests | ||
| Failure to provide a complete report of the assessment results | ||
| Low validity and reliability of online tests | ||
| Lack of management of class discussions | Weakness in discussions | “Student discussions in online classrooms are not as good as they should be due to lack of time, not all students are constantly involved in discussions. Lack of continuous student participation impedes the awareness of their learning process. Code 4” |
| Lack of participation of students in class discussions | ||
| Lack of time for class discussions | ||
| Partial knowledge of students during discussions | ||
| Failure to use class discussions for assessment by professors | ||
| Failure to review class discussions by professors | ||
| Limiting discussions to the classroom | ||
| Non-educational and non-academic discussions | ||
| Impossibility of using different media simultaneously during discussions | ||
| Time-consuming class discussions | ||
| Little attention by professors to formative assessment | Inattention to formative assessment | “There is no proper formative assessment and the student is not aware of his/her academic status during the semester and will only be notified at the end of the semester and after the exam. Code 8” |
| Emphasis on summative assessment | ||
| Impossibility of uploading assignments again by students | Restrictions on uploading assignments and activities | “We could upload the file in the classroom in the first semester, but in the later semesters system errors would occur and we would have to coordinate with the technical support before that. Code 12” |
| Impossibility of uploading large files | ||
| Impossibility of uploading files in different formats | ||
| Error in uploading assignments | ||
| Too many students in a single class | Observance of the ratio of teachers to students | “Due to the large number of students there is less opportunity for exchange of ideas and discussion. Code 4” |
Table 1 presents the basic concepts and main categories extracted. Accordingly, the initial 54 concepts were embedded into 11 main categories. The following section describes the categories obtained from the students’ interviews.
5. Discussion
5.1. Theme 1: Weakness in Infrastructure Technology Resources
Infrastructural and technological resources are highly important for the effective administration of formative assessment, but the e-LMS of Shahid Beheshti University has serious weaknesses in this respect. These resources are the foundation of e-learning. Miller and Rector (9) argue that most organizations encounter countless issues in establishing communication in e-learning, such as low Internet speed, bandwidth limitations, and telephone and Internet outages. Shafiei Sarvestani et al. (10) also consider the low Internet speed and sudden disconnections as e-learning weaknesses.
Consistent with the findings of this study, the study of Seraji et al. (11) showed that the lack of equipment and hardware and software facilities was one of the weaknesses of e-learning, which has a direct impact on the function of this type of educational system.
5.2. Theme 2: Not Using Other Formative Assessment Tools
There is a need for accurate knowledge of assessment tools to evaluate learning in the sense that every teaching situation requires a specific assessment tool of its own. e-learning employs different tools for evaluating learners, some of which may be similar to the tools used in traditional classrooms (12). As the interviewees have stated, formative assessment in the e-LMS of Shahid Beheshti University is limited to few tools. They believed that a set of assessment tools was not used throughout the teaching process. Only were few tools employed which could not measure the student learning alone
The findings of the present study are substantiated by the results of Beyrami and Rahimirad (13), according to which there are weaknesses in assessment tools in e-learning environments, such as the use of traditional assessment methods, the incompatibility of assessment methods with the nature of virtual courses, and failure to use a variety of assessment methods.
5.3. Theme 3: Weakness in Giving and Receiving Feedback
Feedback is one of the underlying features of formative assessment. It is a process for interpreting learners’ current status, needs, future potentials, and the way to fulfil those potentials. Feedback can have positive effects on learners’ learning and participation; however, interviewees held that the feedback provided by e-learning was not adequate, immediate, and well-considered. It should be noted that immediate feedback is usually more effective than delayed feedback (14) because immediate feedback can quickly lead to the identification of learners’ mistakes, so the problem can be solved before incorrect information is consolidated in learners (15). Feedback may have a positive or negative effect on learners’ motivation and self-confidence, and that such emotions affect their participation in the learning process. As a result, it must be continuous and powerful enough to encourage the learners. The findings of Kearns (16), which are consistent with the present study, referred to weaknesses such as lack of proper and timely feedback and delays in feedback.
5.4. Theme 4: Inability to Authenticate
One of the challenges of e-learning is the inability to verify the identity of individuals attending classes or assessment. Lack of sufficient tools to track user behavior is one of the most important problems in the LMS (17). Only is information about the login and logout time of students recorded in traditional methods. Users can log in to the LMS, then leave the system and come back in a few minutes near the end of the class without anyone noticing. Fenu et al.’s research (18) referred to the problem of verifying learners’ identities in electronic environments, which is consistent with the results of this research.
5.5. Theme 5: Weakness in Class Presentations
Class presentations, if used well as a formative assessment tool, can play a significant role in improving learning as well as identifying strengths and weaknesses. During the learning process, professors sometimes ask learners to give presentations to evaluate their learning process. These presentations can be in the form of posters, PowerPoint, oral, etc. The PowerPoint presentation as a form of assessment has empirical validity because it provides learning opportunities through feedback and peer assessment (19). Also, if class presentations are based on a regular sequence and the discussions are organized and clear, the students can concentrate on the topics and understand them more effectively (20). The professors and even presenters can ask questions before their presentations to remind the learners of the relevant topics in the previous sessions and introduce their main discussions. However, in the e-LMS of Shahid Beheshti University, the situation is different as the class presentations have been somewhat neglected.
5.6. Theme 6: Weakness in Exercises and Projects
Exercises and projects can be a reference for learning evaluation if they are purposeful and based on student tendency and interest Assessment through projects and exercises can have several advantages. For example, project-based assessments are dynamic, flexible, and expansive. These projects can have a significant impact on education due to their applicability, and creativity, and innovation. Unlike the formal and substantive limitations of the summative assessment including essay, multiple-choice, or practical questions, project-based assessment has a high degree of freedom in terms of what form they can take and enhances higher levels of cognitive development such as complex and innovative problem-solving (21). Nonetheless, the interviewees reported that classroom exercises as well as projects and articles used by the professors have their own problems, which can affect the learning process.
5.7. Theme 7: Weakness in Online Tests
Should the conditions of conducting a variety of online tests be well provided, one can obtain very useful information on learners’ learning status and provide prompt feedback in addition to identifying their strengths and weaknesses in a timely fashion. Most LMS are capable of creating and processing online tests. These tests can also be used for self-assessment, which allows the user to track their progress and review their acquired knowledge (12).
5.8. Theme 8: Weakness in Discussions
In e-learning environments, learning and participation of learners can somehow be evaluated through presenting questions to learners and observing how they discuss them. This theme shows that the e-LMS of Shahid Beheshti University faces various problems which have rendered the formative assessment tool not useful.
Discussion tools in the LMS are applicable to almost the whole process of formative assessment. Instructors can use both chat and discussion rooms in different ways, such as discussing films, lectures, student essays, etc. These tools are employed to determine learners’ knowledge and skills since learners cannot argue without knowledge and must demonstrate skills in using knowledge to reason in comments or responses (22). At the same time, discussions can bridge the gaps in learners’ knowledge and show areas for further learning. In discussions, learners can also receive feedback from their peers.
5.9. Theme 9: Inattention to Formative Assessment
This theme shows that no special attention has been paid to formative assessments in the e-LMS of Shahid Beheshti University. Instead, more emphasis is placed on summative assessments, which merely seek to measure what is learned, not improve learning.
The interviewees believed that despite the important role of formative assessment in reforming and improving learning, the professors do not pay any attention to formative assessment. In fact, they are more interested in teaching educational content that assessing during instruction and so place more emphasis on the final exam. In fact, if assessment serves as a tool in the learning process, it should be chosen in a way that it can measure a variety of learner behavior, allow the learner to make adjustments to the lesson plan, and improve learning. The findings of Mousavi et al. (23) also in accordanve with results found in the present study.
5.10. Theme 10: Restrictions on Uploading Assignments and Activities
As for class assignments, the students should always submit files in a variety of formats to their classmates or professors. According to the interviewees, the e-LMS of Shahid Beheshti University has faced some limitations in this respect because it is not easy to upload files.
According to the interviewees, activities and assignments are not easily uploaded in the e-LMS of Shahid Beheshti University due to the possible technical errors, file format, and even file size.
5.11. Theme 11: Observance of the Ratio of Teachers to Students
According to the interviewees, due to the virtual nature of classes, the number of students in these classes is usually too high and the required proportion is not observed in this regard.
Some believe that the participation of a large number of students is a benefit in e-learning classes. However, they do not realize that this will make learning assessment very difficult. The interviewees stated that the number of students in each class was very high, so it was impossible for the professors to pay attention to all students. As a result, that there was virtually no opportunity for proper feedback and identification of all students’ strengths and weaknesses. Adherence to the teacher-student ratio allows the professor and students to have more contact with each other, enabling students to acquire more information (24) and to be provided with ample continuous feedback (25). When the number of students increases, professors often do not have the opportunity to support each student (26).
5.12. Conclusions
Overall, the results showed that assessment and especially formative assessment in LMS tem include 11 main weaknesses according to the students of this center. The students at the center believed that these weaknesses have a great impact on their learning quality and prevent them from achieving the main goal of formative assessment, which is learning improvement. In formative assessment, both learners and professors need to recognize their weaknesses and strengths, but in the wake of the aforementioned weaknesses, they cannot be expected to gain a comprehensive understanding of the quality of learning. As such, they can perform well, only if they recognize these weaknesses and try to ameliorate them. In fact, e-learning centers should be able to rigorously identify the challenges and weaknesses in the field of formative assessment and provide operational solutions for them, so they can fulfil a better function in assessment and guarantee improvement in the quality of learning.
