In-Training Assessments of University Residents at Rawalpindi Medical University Pakistan

Objectives: To deeply analyze in-training assessment results of university residents and to compare the mean scores of the major training programs.

Subjects & Methods: A cross-sectional analytical study was done among 56 university residents who were enrolled in MS / MD training programs of Rawalpindi Medical University & Allied Hospitals. This was a record-based study with comprehensive analysis of in-training assessment-1st year results pertaining to all training programs. Assessments were taken during February 2024. Data analysis was done by using Microsoft Excel 2016. Descriptive statistics were applied. Mean scores of trainees employed in MD Medicine & Allied, MS Surgery & Allied, MS Obstetrics & Gynecology and MD Pediatrics were compared by independent sample t-test. P <0.05 was taken as significant.

Results: Of the 56 trainees undergoing in-training assessment-1st year, most (19) were enrolled in Medicine & Allied training programs followed by 16 from Surgery & Allied programs, 9 form Obstetrics & Gynecology and 6 from Pediatrics. Trainees assessed in Ophthalmology, Radiology and Otorhinolaryngology were 3,2 and 1 respectively. Mean theory and OSCE scores of MD Medicine & Allied trainees were 63.6 ± 9 and 67.3 ± 8.2 respectively. mean theory and OSCE scores of MS Surgery & Allied trainees were 70.01 ± 8.9 and 60.74 ± 3.8 respectively. The mean OSCE score of MS Obstetrics & Gynecology trainees was relatively higher 72.8 ± 6.3 than that of theory. Both assessment scores of MD Pediatrics trainees were almost equal. Mean theory score of MS Surgery & Allied was comparatively greater than those of MD Medicine & Allied (P = 0.04) while OSCE score in MD Medicine & Allied was significantly higher (P=0.006). OSCE score of MS Obstetrics & Gynecology trainees was also significantly higher (P=0.0001) than those of MS Surgery & Allied. The overall assessment score of MS Otorhinolaryngology was 73.3% followed by MS Obstetrics & Gynecology score of 71%. 

Conclusion: MD Medicine & Allied trainees must do hard work to enhance their medical knowledge while those of MS Surgery & Allied trainees should concentrate on enhancing their clinical competencies.

Assessments being an integral part of curriculum is of paramount significance to ensure trainees’ learning. This should be dealt with in a curriculum as a process of teaching and not as a separate entity [1]. Assessment should be ongoing to conform that teaching and learning is a dynamic process. Assessment along with specified learning outcomes of the curriculum and chalked out learning activities is imperative [2]. It is one of the pillars of education, the implementation of curriculum without which is perceived as impossible [3]. Certain guiding principles pertaining to assessment of trainees should strictly be followed for smooth execution of the whole educational process. Some of them are fairness of assessment, its communication to all students through various resource books and social links in addition to eligibility for assessment and provision of constructive feedback for improvement [4]. Although summative assessment is essential for awarding degree and certifying a student as competent; however, a there is a significant trend of emphasizing formative assessment of the students particularly postgraduate medical residents across the globe to promote their comprehensive learning.  Therefore, deliberating the content of curriculum especially that linked with higher medical education is essential [5]. Here the importance of curricular mapping cannot be overlooked that not only ensure alignment of learning outcomes with teaching strategies and assessment tools but also facilitate in searching gaps and verify the compliance with accreditation models [6]. 

There is diversity in assessments across different degree awarding medical colleges and universities worldwide. In training assessments taken by College of Physicians & Surgeons Pakistan (CPSP) is one of the components of summative assessment that consists of yearly written assessment of the trainees and their Workplace Based Assessment (WPBA). It is different from Intermediate Module (IMM) examination that is another component of summative assessment and is basically a mid-residency assessment [7]. On the other hand, Royal College of Physicians & Surgeons Canda (RCPSC) suggests regular execution of in training assessment of the postgraduate medical residents for their formative assessment. Although there is more focus on small module based regular assessments; it also has many lacunae that need attention of the competent authorities [8]. Although A study was done by Ringsted C et al to analyze the confidence level among junior doctors post-in training assessment; the data related to analysis of in training assessments is quite scarce [9]. Many aspects are essential to make in-training assessments impactful like comfortable environment and credible feedback both from supervisors as well as trainees [10]. In training assessments of postgraduate trainees enrolled for training at Rawalpindi Medical University and its allied hospitals is therefore accomplished by respective Deans to confirm assessments at familiar workplace. The present study is intended to comprehensively analyze the results of theoretical as well as clinical assessments of the trainees that were taken on completion of 1st year. This study would enable us to recognize the deficiencies of the trainees that would later be brought under consideration and subsequent modification by the stakeholders if required. 

A cross-sectional descriptive study was done to comprehensively analyze the results of in-training assessment-1st year that was taken during February 2024 by respective Deans on completion of 1st year of MS / MD training at Rawalpindi Medical University (RMU) & Allied Hospitals. Trainees enrolled in MS / MD training programs are enrolled through Central Induction Policy (CIP) and are known as “university residents”. In-training assessments of these residents are taken following official notification of assessments by Examination department of RMU. Plan of In-training assessments in accordance with Revised curriculum and assessment scheme 2021 [11] is tabulated below: 

Table 1: Revised Assessment Plan for university residents

In-training assessments at RMU & Allied hospitals are formative assessments with pass percentage of 50%. On getting score below 50%, trainees are not reassessed. However, two-way feedback is given due consideration and trainees with poor score are counseled by respective Deans for improvement. Total 56 trainees undergone in-training assessment-1st year that belonged to Medicine & Allied, Surgery & Allied, Obstetrics & Gynecology, Pediatrics, Radiology, Ophthalmology and Otorhinolaryngology. Their theoretical knowledge of “Knows How” level that primarily determines the application of knowledge as specified in Miller’s pyramid of assessment [12] was measured through Short Answer Questions (SAQs) and Short Essay Questions (SEQs) while clinical competencies were assessed by Objective Structured Clinical Examinations (OSCEs). However, trainees of MD Radiology were also subjected to Table viva apart from SAQs and OSCEs. The data analysis was done by Microsoft Excel 2016. Descriptive statistics were applied. Difference in mean scores of trainees from major specialties was statistically measured by applying independent sample t-test. P less than 0.05 was considered significant. 

Total 56 trainees were subjected to In-training assessment-1st year during February 2024 that were taken by their deans on completion of 1st year of training and fulfillment of respective requisites. Most of them (18) were from Medicine and Allied disciplines as revealed below in Figure 1. 

Figure 1: Program-wise depiction of trainees subjecting to In Training Assessment-1st year

Total 19 trainees enrolled in Medicine & Allied training programs had In Training assessment-1st year during February 2024 and highest score e was achieved by trainee of MD Psychiatry as illustrated below in Table 1. 

Table 2: In Training Assessment Results of Medicine & Allied trainees (n = 19)

Scores of medical knowledges as well as clinical competencies of Surgery & Allied trainees are presented below in Table 2.

Table 3: In Training Assessment Results of Surgery & Allied trainees (n = 16)

Clinical as well as theoretical assessment scores of the trainees other than those working in Medicin and Surgery & Allied disciplines is illustrated below in Table 3. 

Table 4: Assessment scores of the trainees from programs other than those of Medicine and Surgery & Allied training programs

Trainees of MD Radiology were also assessed through table viva apart from theory and OSCE as shown below in Table 4. 

Table 5: In Training assessment-1st year results of MD Radiology trainees

Trend of In-training assessment-1st year results of the trainees enrolled in teaching hospitals of RMU is revealed below in Figure 2.

Figure 2: Trend of in training Assessment-1st year results

As Medicine and Surgery & Allied trainees assessed constituted the majority, so comparison of their theory and OSCE mean scores achieved during in training assessment-1st year was carried out by applying independent sample t-test as depicted below in Table 5.

Table 6: Comparison of Theory and OSCE mean scores of Medicines and Surgery & Allied trainees

*Statistically significant difference 

Likewise, difference in mean scores of medical knowledge and clinical competencies of the trainees enrolled in MS Obstetrics & Gynecology and MD Pediatrics was also statistically tested but was found out to be insignificant as shown below in Table 6.

Table 7: Comparison of mean theoretical and clinical scores of MS Obstetrics & Gynecology and MD Pediatrics trainees

Difference in mean theory and OSCE scores of 2 major MD and 2 major MS programs’ trainees revealed highly significant difference (P = 0.0001) between mean theory and OSCE scores of MS Surgery & Allied and MS Obstetrics and Gynecology trainees only as depicted below in Table 7. 

Table 8: Statistical difference between theory and OSCE scores of 2 major MD and 2 major MS programs’ trainees

*Statistically significant difference 

On reviewing the trend of overall scores achieved during in-training assessment-1st year by university residents of RMU & Allied hospitals, the highest score (73.3%) was that of MS Otorhinolaryngology trainee followed by 71% of MS Obstetrics and Gynecology trainees and 70.2% MD Radiology trainees (Fig 2). On comparing the results of different assessment modalities, difference between theory and OSCE scores of MS Obstetrics and Gynecology trainees and MD Pediatrics trainees was statistically insignificant (Table 7). On the other hand, difference between OSCE scores only of MS Surgery & Allied trainees and MS Obstetrics and Gynecology was highly statistically significant as revealed in Table 8. On analyzing mean scores of the trainees enrolled in these programs, mean theory scores of MS Surgery & Allied was somewhat greater than those of MS Obstetrics and Gynecology while MS Obstetrics and Gynecology trainees got significantly higher score in OSCE depicting sufficient acquisition of clinical competencies by them (Table 8). As this was the assessment on completion of 1st year of training, this was in accordance with their Entrusted Professional Activities specified in their respective logbooks. A cross-sectional study carried out among Obstetrics & Gynecology trainees belonging to Denmark, Sweden and Norway revealed discrepancies between level of training and degree of confidence pertaining to doing ultrasound of the patients independently [13]. A study by Salvesen KA et al among European trainees revealed that deficits in promoting competency base medical education are attributed to lack of reliability in training standards across the countries [14]. The only way out to get rid of such discrepancies seems to be the clarity in curriculum regarding assessment Table of Assessment (TOS) planned for trainees of each year. This strategy would facilitate the trainees not only in attaining the relevant theoretical knowledge but will also prove useful in acquisition of intended clinical competencies. 

Overall score attained by trainees of MD Medicine & Allied programs was almost equal to that of MS Surgery & Allied in present study as shown in Fig 2. However, the difference in mean theory and assessment scores of both groups of trainees had statistically significant difference (table 6). Medicine & Allied trainees had comparatively deficient theoretical knowledge while those of Surgery & Allied had relatively inadequate clinical competencies. A systematic review by Pakkasjarvi N et al highlighted the need of competence-based training in all surgical disciplines [15]. Mean OSCE score of Surgery & Allied trainees during in-training assessment-1st year was 60.74 ± 3.8 while those of Medicine & Allied trainees was 67.3 ± 8.2. Although University Residency Program at RMU & Allied hospitals is based on Competency Based Medical Education (CBME) framework given by Accreditation Council for Graduate Medical Education (ACGME) [16]; the acquisition of the trainees with the required medical knowledge and clinical skills is a fact that should be deliberated and strategized by all stakeholders including institutional curriculum committee members and Deans for improvement and to bring the standard of our Higher Education Institutes (HEI) in par with international standards as specified by World Federation for Medical Education [17]. 

On detailed analysis of MD Medicine & Allied results, OSCE scores were comparatively better than those of theory (Table 2). This elucidates the need for more concentration on studying books, giving presentations and literature search for knowing subject related practices and advancements globally. According to Nagasaki Y et al, General Medicine in-training examination (GM-ITE) was perceived quite beneficial in determining clinical knowledge acquisition among Japanese postgraduate residents. This examination was intended to upgrade the training programs of the country. The study also verified positive correlation between GM-ITE and PLAB-I scores of those residents [18]. Degree Awarding Institutes (DAIs) across the world are not only responsible for taking examinations and awarding degrees but also meant to revolutionize the educational systems and training programs [19]. Contrary to MS Medicine & Allied residents, MS Surgery & Allied trainees scored better in theory than that of OSCE (Table 3). According to Harden et al, OSCE is a valid and beneficial tool for enhancing clinical competencies of the medical students as they go through the practice of history taking, clinical examination, counseling and case management in controlled environment during specified time frame [20]. Apart from assessing undergraduate medical students, OSCE has extensively been practiced for assessing diverse clinical competencies among postgraduate residents and the candidates appearing in licensing exams [21]. Despite the need for multiple resources to arrange 10-15 stations of OSCE, its complexity and being time-intensive activity, OSCE is still perceived as a valid and reliable tool and hence the gold standard for measuring clinical performance. It has been perceived optimistically by both examiners and students [22]. Latest research by Huang T et al among medical postgraduates emphasized the significance of team, case, lecture and evidence-based learning strategies in augmentation of clinical and research related competencies than those of lecture base learning alone [23]. A cross-sectional survey by Alam L et al pointed out many loopholes in competency-based training programs of Pakistan that should stringently be addressed by our policy makers for improvement [24]. Although short intensive preparatory courses have frequently been arranged on 6 monthly bases in addition to regular execution of 360-degree evaluation of university residents and their workplace-based assessments; in my opinion clinical assessments of trainees by employing multiple assessment tools like OSCE, long case, short case etc. at departmental level on monthly basis can prove advantageous in revealing the desired outcomes. 

There is dire need to boost medical knowledge and clinical skills by trainees of MD Medicine & Allied and MS Surgery & Allied respectively. Mock exams and preparatory drills should frequently be carried out at Deanery or departmental level to overcome the deficiencies. 

Conflicts of Interest

The authors declare no conflict of interest. 

Source of Funding

The author(s) received no financial support for the research, authorship and or publication of this article.