1. Background
Project management is one of the most critical skills in today's world. Given the increasing complexities of technology and globalization, its importance has grown exponentially. This skill plays a pivotal role in achieving organizational goals and directly impacts cost reduction, efficiency enhancement, and the quality of project outputs (1). Despite this, graduates' lack of practical skills remains a major obstacle to project success in numerous countries, such as Iran (2). This shortfall often leads to project delays, cost overruns, and subpar outcomes, underscoring the importance of robust, hands-on education (3).
Studies have consistently shown that conventional education, with its strong emphasis on theoretical concepts, falls short of equipping graduates with the necessary skills for the workplace (4). For instance, Abante found that modular training, characterized by its flexibility and practical focus, is more effective in empowering learners (5). Likewise, in Iran, research by Mehravar Giglou and Zahed Babelan has underscored the absence of managerial abilities among university graduates, associating this shortfall with reduced productivity in national projects (6). Modular courses offer a flexible and learner-centered approach to education. By breaking down the curriculum into smaller, self-contained modules, learners can progress at their own pace and focus on specific areas of interest. This modular structure promotes active learning and enables the integration of theoretical knowledge with practical skills. As highlighted by Bacomo et al., modular courses can significantly enhance student engagement and motivation (7).
Modular courses often incorporate various assessment methods, including formative and summative assessments, to ensure optimal learning outcomes. These assessments provide valuable feedback to learners and instructors, allowing for timely adjustments to the learning process. By aligning assessment strategies with learning objectives, modular courses can effectively measure student achievement and identify areas for improvement. As Ambayon and Millenes emphasizes, a well-designed assessment system is crucial for the success of modular classes (8).
However, a significant challenge in the development and effectiveness of modular courses lies in the absence of comprehensive, scientifically grounded frameworks for their design and evaluation. Many existing studies have focused on isolated course design or implementation aspects, failing to provide a holistic perspective for examining these processes. The design of a practical educational course requires careful integration of various stages, from needs assessment and objective setting to content delivery and impact evaluation. Without a clear framework, inconsistencies may arise between learning objectives, content, and evaluation methods (9).
This research addresses the gap in comprehensive frameworks for designing and evaluating modular project management courses by developing, implementing, and evaluating such a course for university graduates. Utilizing instructional models like ASSURE for content development and Kirkpatrick's model for evaluation, this study investigates the effectiveness of these courses in enhancing participants' managerial capabilities. Beyond addressing educational challenges, this research can improve organizational performance, reduce project failure rates, and enhance national projects' efficiency. Additionally, the findings can serve as a model for designing and implementing similar courses in other professional fields. Ultimately, this research seeks to answer whether modular project management courses can significantly improve the practical skills of graduates.
2. Objectives
The results of this study will provide valuable guidance to instructional designers, project managers, and policymakers to enhance the quality of education and the professional competencies of graduates through this approach.
3. Methods
This study falls under the descriptive-analytical category and is classified as applied research. It aims to address practical challenges in project management education by designing, implementing, and evaluating a modular course to enhance participants' professional skills. The methodology follows the ASSURE instructional design model, presented in three main sections: Design, implementation, and evaluation.
3.1. Course Design (Design)
3.1.1. Learner Analysis (ASSURE Step 1: Analyze Learners)
The target audience for this course included three main groups:
(1) University faculty members: Especially those engaged in management disciplines or research projects.
(2) Administrative staff: Responsible for project planning and execution within organizations.
(3) Graduate students: Eager to develop project management skills for career advancement.
3.1.2. Defining Educational Objectives (ASSURE Step 2: State Objectives)
(1) Empower practical project management participants by integrating theoretical knowledge and hands-on skills.
The objectives were formulated following the SMART principles, ensuring they were specific, measurable, achievable, relevant, and time-bound.
3.1.3. Selecting Methods, Media, and Materials (ASSURE Step 3: Select Methods, Media, and Materials)
The instructional methods included:
(1) Interactive lectures for delivering foundational knowledge.
(2) Simulations and case studies for hands-on practice.
(3) Group workshops to encourage teamwork and problem-solving.
Teaching tools like interactive videos, project management software, and instructional guides were employed to support the course delivery.
3.1.4. Course Topics and Structure
The modular course covered 35 comprehensive topics related to project management. The course was conducted over 15 sessions, each lasting 2 hours, for a total of 30 hours. It was delivered in a hybrid format, combining online and in-person sessions.
3.2. Course Implementation (Implementation)
3.2.1. Utilizing Resources (ASSURE Step 4: Utilize Media and Materials)
Course materials included:
(1) Educational videos.
(2) Online platforms were used for content delivery, while in-person sessions focused on discussions and collaborative activities.
3.2.2. Encouraging Learner Participation (ASSURE Step 5: Require Learner Participation)
Active participation was encouraged through:
(1) Practical assignments and group projects.
(2) Interactive workshops and peer discussions to foster engagement.
3.3. Course Evaluation (ASSURE Step 6: Evaluation)
3.3.1. Evaluation Framework
The evaluation utilized Kirkpatrick's levels 1 and 2:
(1) Level 1: To assess participants' satisfaction, we utilized semi-structured interviews with a purposively sampled group of participants. The interviews were arranged in advance and coordinated with participants to create a comfortable environment for discussion. With the interviewees' consent, the sessions were recorded and transcribed immediately afterward to ensure accuracy. The data analysis process commenced with transcribing the recorded interviews, followed by a conventional content analysis approach. This involved multiple readings of the transcripts to identify initial themes and insights related to participant experiences. A coding process was then implemented, where text segments were labeled with codes representing specific ideas or concepts. These codes were subsequently grouped into broader categories, facilitating a structured understanding of participant feedback.
(2) Level 2: Pre- and post-tests evaluated knowledge acquisition. In this regard, we implemented a pre-test and post-test design to assess knowledge acquisition among participants. An exam blueprint was developed before the assessments to outline key content areas and skills participants were expected to master. This blueprint ensured the exam aligned with course objectives and effectively measured intended learning outcomes. Participants completed standardized tests before and after the course to assess their knowledge and skill development in project management. Statistical analyses were performed on the pre- and post-test results using SPSS software to evaluate the significance of learning outcomes between these two assessments.
Figure 1 shows the course methodology, which comprises the course design, implementation, and evaluation.
Figure 1.
Course methodology
To ensure ethical conduct in our study, we followed established guidelines for research involving human participants. We obtained approval from the Ethics Committee of Shahid Beheshti University of Medical Sciences (IR.SBMU.SME.REC.1403.026). We ensured participants received clear, straightforward information about the study's goals, procedures, and potential risks. They were informed that their participation was voluntary and that they could withdraw at any time without negative consequences. To protect their privacy, we anonymized and stored all personal data securely. Participants were also assured that their identities would remain confidential in any reports or publications resulting from the research.
3.4. Data Analysis Method
3.4.1. Qualitative Data Analysis
The qualitative data from semi-structured interviews was analyzed using content analysis, which involved identifying, coding, and categorizing themes within the data to gain insights into participants' experiences and perceptions.
3.4.2. Quantitative Data Analysis
SPSS 26 was used in this part. Quantitative data, including pre- and post-course test scores, survey responses, and performance checklists, were analyzed using descriptive statistics to summarize the data. Inferential statistics like t-tests and ANOVA were used to compare scores and identify significant differences. Additionally, correlation analysis was used to explore relationships between variables. All statistical tests were conducted at a 95% confidence level.
4. Results
This study aimed to evaluate the effectiveness of modular courses in project management education using Kirkpatrick's model, explicitly focusing on levels 1 and 2. level 1, which assesses participant satisfaction, was evaluated through structured interviews, while level 2, which measures knowledge acquisition and learning outcomes, was evaluated using pre- and post-tests.
4.1. Demographic Characteristics
A total of 150 participants completed the modular course on project management. The demographic data is summarized in Table 1.
Table 1.Demographic Characteristics of Participants
| Demographic Feature and Category | No. (%) |
|---|---|
| Gender | |
| Male | 90 (60) |
| Female | 60 (40) |
| Age (y) | |
| Under 25 | 22 (15) |
| 25 - 35 | 75 (50) |
| 35 - 45 | 38 (25) |
| Over 45 | 15 (10) |
| Education level | |
| Bachelor's degree | 60 (40) |
| Master's degree | 75 (50) |
| Doctoral degree | 15 (10) |
| Work experience (y) | |
| Less than 1 | 15 (10) |
| 1 - 3 | 45 (30) |
| 3 - 5 | 60 (40) |
| More than 5 | 30 (20) |
4.2. Kirkpatrick's Level 1: Evaluating Participant Satisfaction
To assess level 1 of Kirkpatrick's model, interviews were conducted with a subset of participants (n = 15), comprising six males and nine females. The interviews aimed to gauge participant satisfaction with the course content and delivery. The results indicated that 85% of interviewees expressed satisfaction with their learning experience. The interviews were analyzed using conventional content analysis, which allowed for identifying common themes and insights regarding participant experiences. Several key themes emerged from the interviews, highlighting the strengths and areas for improvement in the modular course structure.
Participants shared their thoughts on the course, with many expressing appreciation for the practical applications of the material. Participant No. 1 noted:
"The hands-on projects helped me understand how to apply what I learned in real-world scenarios."
Another participant emphasized the quality of instruction, stating:
"The instructors were knowledgeable and approachable, which greatly impacted my learning experience."
In addition to positive feedback, some participants provided constructive criticism. For instance, Participant No. 8 remarked:
"While I enjoyed the course overall, I felt some modules could benefit from more interactive elements."
This feedback suggests that while participants were largely satisfied, there is room for enhancement in engagement strategies. Furthermore, several participants highlighted the collaborative aspects of the course. Participant No. 10 stated:
"Working with peers on group assignments was invaluable; it allowed us to learn from each other's perspectives."
The meaning units, codes, subcategories, and categories extracted from interviews are shown in Table 2.
Table 2.Meaning Units, Codes, Subcategories, and Categories Extracted from Interviews
| Category and Subcategory | Code | Meaning Unit |
|---|---|---|
| Strengths of the course | ||
| Practical applications | Hands-on projects | The hands-on projects helped me understand how to apply what I learned in real-world scenarios. |
| Quality of instruction | Knowledgeable instructors | The instructors were knowledgeable and approachable, which significantly impacted my learning experience. |
| Collaborative learning | Value of collaboration | Working with peers on group assignments was invaluable; it allowed us to learn from each other's perspectives. |
| Networking opportunities | Appreciation for networking opportunities | |
| Areas for improvement | ||
| Need for more interactivity | Interactivity | While I enjoyed the course, I felt some modules could benefit from more interactive elements. |
This sentiment was echoed by others who appreciated the networking opportunities afforded by group activities. The qualitative data gathered from these interviews provided rich insights into participant experiences. The combination of positive feedback and constructive suggestions will inform future iterations of the modular course, ensuring continuous improvement in project management education. The high satisfaction rates observed underscore the course structure's effectiveness while pointing to areas where enhancements can be made for an even more engaging learning experience.
4.3. Kirkpatrick's Level 2: Assessing Learning
For the level 2 assessment, a pre-test and post-test design was implemented to assess learning. Statistical analyses were performed to evaluate the significance of learning outcomes between these two assessments. The results demonstrated a positive and statistically significant increase in knowledge among all participants by the end of the course (Table 3).
Table 3.Pre-test and Post-test Results
| Assessment | N | Mean ± SD | t-Value | P-Value |
|---|---|---|---|---|
| Pre-test | 150 | 65.4 ± 10.2 | 12.34 | < 0.001 |
| Post-test | 150 | 82.1 ± 8.5 |
This table presents a significant increase in knowledge acquisition among participants from the pre-test to the post-test. The results highlight the effectiveness of the modular course in enhancing participants' understanding of project management concepts.
5. Discussion
The evaluation of educational programs, particularly in the context of project management training, is crucial for understanding their effectiveness and areas for improvement. This study employed Kirkpatrick's model, focusing on levels 1 and 2, to assess participant satisfaction and knowledge acquisition. The findings indicate a positive reception of the modular courses and a significant increase in knowledge among participants.
The assessment of participant satisfaction, as outlined in level 1 of Kirkpatrick's model, is a critical component in evaluating the effectiveness of educational programs. In this study, 85% of participants expressed satisfaction with the modular project management course, indicating a strong positive reception. This satisfaction level is consistent with previous research findings emphasizing learner engagement and satisfaction in educational outcomes.
Despite the positive feedback, some participants provided constructive criticism regarding the course structure. For example, one participant suggested that "some modules could benefit from more interactive elements." This feedback indicates an opportunity for course designers to enhance engagement strategies further. More interactive elements could cater to diverse learning styles and preferences, potentially increasing overall satisfaction. Moreover, while high satisfaction rates are encouraging, they must be contextualized within broader educational goals. Cabual noted that satisfaction does not always equate to effective learning or skill application in professional settings (12). Therefore, continuous evaluation and adaptation of course content and delivery methods are essential to ensure participant satisfaction translates into meaningful learning outcomes (13).
The second level of Kirkpatrick's model focuses on knowledge acquisition, which was assessed through a pre-test and post-test design in this study. The results demonstrated a positive and statistically significant increase in knowledge among participants by the end of the course, highlighting the effectiveness of modular training in enhancing understanding of project management concepts. Knowledge acquisition is a fundamental goal of any educational program. The significant improvement observed from pre-test to post-test scores indicates that participants retained information and could apply it effectively. This finding aligns with research by Talevski et al., which asserts that structured training programs can substantially improve knowledge retention and application (14). The statistical analysis revealed a mean score increase from the pre-test to post-test assessments, underscoring the effectiveness of the modular course design. Such results are consistent with studies like those conducted by Yuliansyah and Ayu, which emphasize that practical training must impart knowledge and facilitate its application in real-world contexts (15). While the increase in knowledge acquisition is promising, it raises questions about how this knowledge translates into practical skills and behavioral changes in professional settings. Lowell and Moore highlight that barriers often prevent individuals from applying newly acquired skills effectively within their organizations. Therefore, it is crucial for future iterations of the modular course to incorporate elements that facilitate real-world application (16). For instance, integrating case studies or simulations that mimic real project management challenges can help bridge the gap between theory and practice. Additionally, providing opportunities for participants to apply their skills in actual projects or internships could further enhance knowledge retention and application (17).
While providing valuable insights into the effectiveness of modular project management courses, this study has several limitations that should be acknowledged. One limitation is the short-term assessment. The pre-test and post-test design primarily evaluate immediate knowledge acquisition without measuring long-term retention or practical application in real-world settings.
Another limitation is the study's limited scope, which focuses on levels 1 and 2 of Kirkpatrick's model, neglecting levels 3 (behavior) and 4 (results), which are essential for a comprehensive evaluation of training effectiveness. Additionally, the study was conducted within a specific educational context, which may limit the applicability of the findings to other settings or regions with different organizational cultures or industry standards.
5.1. Suggestions
Future studies should explore several avenues related to knowledge acquisition:
(1) Longitudinal studies: Investigating long-term retention of knowledge acquired through modular courses would provide insights into the sustainability of learning outcomes over time.
(2) Behavioral assessment: Assessing changes in behavior post-training through follow-up interviews or performance evaluations can offer a deeper understanding of how knowledge is applied in professional settings.
(3) Comparative analysis: Conducting comparative studies between different instructional methods (e.g., online versus face-to-face) could help identify best practices for enhancing knowledge acquisition.
(4) Integration with workplace training: Researching how modular courses can be integrated with organizational training programs may yield insights into creating a seamless transition from education to practical application.
5.2. Conclusions
In summary, while this study contributes valuable insights into the effectiveness of modular project management courses through Kirkpatrick's model, it is essential to recognize its limitations. Addressing these limitations in future research will enhance the understanding of training effectiveness and inform best practices for designing educational programs that meet the needs of diverse learners in various professional contexts. By acknowledging these constraints, researchers can better interpret results and develop strategies for continuous improvement in project management education.
5.3. Highlights
Improved graduate readiness: The modular course equips graduates with essential project management skills, making them more competitive in the job market.
Enhanced learning experience: Interactive and practical elements of the course foster deeper understanding and retention of knowledge.
Framework for future courses: The study provides a model for developing similar educational programs across various fields, ensuring alignment with industry needs.
5.4. Lay description
This study focused on enhancing project management skills among medical science graduates through a modular course. Many graduates struggle with practical skills needed for effective project management, which can lead to failures in real-world projects. The course combined theoretical knowledge with hands-on experiences, using interactive lectures and simulations. After the course, participants reported high satisfaction and demonstrated significant knowledge gains. This approach not only prepares graduates for their careers but also aims to improve project outcomes in healthcare settings.
