Research Article
Student and Teacher Satisfaction with Online Mode of Postgraduate Examination
Tamkin Khan ,Nasreen Noor*
Department of Obstetrics and Gynaecology, Jawaharlal Nehru Medical College, A.M.U., Aligarh 202002 (U.P.).
Received Date: 06/12/2022; Published Date: 09/01/2023.
*Corresponding author: *Nasreen Noor, Department of Obstetrics and Gynaecology, Jawaharlal Nehru Medical College, A.M.U., Aligarh 202002 (U.P.).
DOI: 10.55920/IJCIMR.2023.04.001134
Abstract
Background: With the advent of the COVID-19 pandemic medical residency training programs and other educational activities faced tremendous disruptions worldwide. It is important to provide continuity of teaching and learning as well as timely assessments so that certification for practice is not delayed for the benefit of the trainee. The conventional examination pattern involves clinical case presentations requiring face to face resident, assessor and patient interaction which increases the risk of transmission of infection. In addition external assessors were reluctant to travel due to the risk of exposure or due to covid induced lockdown. We were therefore forced to conduct an online examination in the virtual mode. Despite the use of this mode all around the globe, questions have been raised regarding the validity and reliability of the online mode as well as student and faculty satisfaction. Educational institutions suffered hiccups while going online. The availability of devices or lack of connectivity could put students at a disadvantage. Though online examination cannot replace the traditional assessment systems in clinical subjects many challenges can be met by careful planning and training of the faculty as well as students.
Aim: In this study, we tried to evaluate the student and faculty satisfaction with the online mode through a confidential questionnaire about PG practical examination in Obstetrics and Gynecology in our institution in June 2020 during the COVID pandemic.
Material and Methods: At the end of the online P.G examination, a questionnaire consisting of responses on a Likert scale and some open-ended questions were given to the students and the faculty members to assess the effectiveness, satisfaction, advantages, and challenges faced while conducting online PG practical examination.
Results: Practical examination was conducted in the virtual online mode on 18 postgraduate students by 04 examiners(2 internal and 2 external). All postgraduate students and all examiners expressed satisfaction with a virtual mode of online system of examination. A few short duration of interruptions due to internet connectivity issues was challenging.
Conclusions: Meticulous planning, prior student orientation and good internet connectivity for a virtual video conferencing platform appears to be an optimal alternative during the COVID pandemic for conducting online examinations.
Keywords: Students feedback, Faculty feedback, online examination, Postgraduate, Covid-19
Introduction
In the COVID- 19 era, learning organizations cannot lag in the use of new technologies[1]. Higher Education Institutions all over the world are increasingly adopting and implementing online modes.[2,3] Many institutions in developing countries are facing challenges with online examinations due to a lack of sensitization of students to online mode and lack of preparation of students, and lack of internet facilities in some areas.[4] In India, a nationwide lockdown was implemented due to the coronavirus disease (COVID-19) pandemic towards the end of March 2020 leading to immense disruption of routine hospital services and residency training. Since it was difficult to predict an end to the pandemic, educational activities gradually resumed normal teaching-learning activities and examination schedules in spite of the COVID-19 situation.[5] Like many other residency programs, our department continued the majority of educational activities during the pandemic through online mode[6,7] We also successfully conducted timely end-of-training evaluations of the final year M.S/ DGO graduating residents in the Department of OBGYN. In this article, we would like to share the perceptions of the postgraduate student and faculty members about their satisfaction, challenges, advantages, and disadvantages in conducting the final year P.G residency exit examination.
Material and Methods
Present observational study was conducted in the Department of Obstetrics and Gynaecology, Jawahar Lal Nehru Medical College, Aligarh Muslim University. The study was approved by the Institutional ethics committee. It was a questionnaire-based study consisting of questions on the Likert scale as well as open-ended questions. A pre-validated (face validity) questionnaire was used, ensuring anonymization. The questionnaire was delivered both M.S and DGO final year postgraduate students (n=18) during the academic year 2020. Feedback from the internal and external examiners and internal faculty members involved in the examination was also obtained through the same questionnaire.
The conventional examination pattern involves clinical case presentations and face to face viva voce with a set of examiners. The candidate was expected to elicit history, perform a clinical examination and present the cases to the examiners followed by questions. However, the Covid-19 pandemic posed the following challenges that made conventional practical exams difficult.
- To arrange patients with varied diagnoses who could be posted for examination for 18 postgraduate students. Due to the lockdown and conversion of our hospital into a covid hospital, we did not have any consultations in the OPD for gynecological cases except malignancies.
- To arrange a set of external examiners as most were reluctant to travel due to lockdown restrictions or their own safety concerns.
- To ensure the safety of examinees and examiners coming in contact with patients.
To overcome these challenges, we shifted to a virtual mode of examination. It was decided that the examination would be conducted through Skype and a group was created for this purpose.
A 90-minute time limit for 2 short gynecological paper cases and one long case obstetric case was set as agreed upon by the examiners. An orientation brief was given to the students and a mock test was conducted one week before the date of the examination so that the candidates and faculty became familiarised with the new virtual pattern and technological issues could be sorted out. A few days before the assessment dates the duly maintained PG logbook was sent to the external examiner as a scanned copy through e-mail.
Arrangements were made to ensure adequate social distancing and PPE was provided for each student during the obstetric patient workup. During the presentation, only the candidate, two internal examiners, one IT personnel, and a junior faculty for coordinating the examination were present following social distancing and COVID norms.
For obstetric long cases actual patients were allotted to each candidate by lottery method as we were continuing obstetric services and large variety of cases were available. PPE kits were provided to the examinees during the workup. After workup candidates presented the cases to the examiners through skype.
To replicate the conventional examination, for gynecology long cases case-based scenarios were prepared as computer assisted OSCE for uniformity by the faculty and were validated by external examiners. Photographs showing examination findings and photographs of imaging studies from patient records and case archives were used to build up the case scenarios.
Grand table viva including drugs, instruments, specimens, logbook, and maternity viva was also held through skype. The images, case summaries, instruments, and videos were displayed to all through screen sharing.
The assess the level of satisfaction amongst the examinees and the examiners, we designed a questionnaire aimed at assessing the challenges, advantages and disadvantages, quality of digital case scenarios, and, overall satisfaction with the exam pattern. The feedback questionnaire consisted of the following 07 open-ended questions. (Table 1). The responses were analyzed and categorized thematically
Results
Of all the questionnaires distributed, 100% were completed sufficiently for data analysis. A total of 18 residents and 10 faculty members participated in this study and filled up the feedback questionnaire. Amongst them, 08 were M.S students while 10 were DGO residents. Table II and Table III show the perception of students and faculty for conducting an online practical examination in the department of OBGYN.
Table I: Student Responses on the Feedback Questionnaire
Student Perspective
Table 1 shows student responses to the questions. The students reported stress and anxiety due to unfamiliar mode of examination 07 (38.8%) and said lack of prior sensitization to online mode of examination (11.1%) were the greatest challenges. Internet connectivity (38.8%) was another challenge experienced by them. Students felt that they felt less threatened by the examiner in the online mode (61.11%). They had difficulty in demonstration of skills on dummy pelvis (66.66%) and instruments and procedures (44.44%). Majority (88.88%). suggested good internet connectivity and prior practice (22.22%) could improve the process. Majority were completely satisfied by the online mode (66.66%) and agreed that during COVID-19 online examination is the only option for completing their assessment on time.
Faculty Perspective
Table II shows faculty responses on the feedback questionnaire. Internet connectivity (80 %) was the greatest challenge along with poor audiovisual aids (30 %) and need to repeat questions (20%). Online mode saved time (80%), need for travel (30%) and saved leave days (10%). Difficulty in assessment of skills in the virtual mode (30%), difficulty in demonstration of procedures (60%) and dummy pelvis (40%) were the greatest challenges. They felt that good internet connectivity (100%) increasing number of cameras (70%), better audiovisual aids and mannikins (70%) and prior practice (50%) could improve the process. Regarding the challenges faced in gynecological long cases as computer assisted OSCE majority felt that (80%) psychomotor and soft skills could not be assessed in this mode. However majority (90%) were satisfied with this mode.
Table 2: Faculty Responses on the Feedback Questionnaire
Discussion
As the novel coronavirus 2019 (COVID-19) continues its pandemic surge globally along with its social distancing norms, the physical conduct of practical examinations for medical graduates and postgraduates has become difficult. In this era of Covid 19 assessment moved to an online mode of examination, which has added immense psychological stress to students (8,9). Online assessment presents various challenges to the students, that can have a massive impact on their performance. Hence a meticulous preparation, familiarising students to this new mode with briefing and orientation sessions before the conduct of virtual examination is required. Though telemedicine sessions and online classes compensated for lack of training and clinical exposure in medical education, the conduct of practical examinations has been a challenge (10-12). Conducting practical examinations is more challenging because it requires technology, protection ( COVID risk), lack of clinical material in the form of patients (for case presentations), training of technical staff, and the need for interaction between examiners and students. Moreover, there are many impediments such as the imposition of lockdown; social distancing; restrictions on travel; limited healthcare resources; limited personnel; and limited personal protection equipment such as masks, sanitizers, gloves, and increased risk of exposure to COVID-19 (13-15).
Software-based systems such as medical software applications and social media platforms could provide alternative platforms for ensuring continuous medical education including evaluation of medical students (16,10,11,16). In our study, we employed professional video conferencing software on a desktop/laptop. Simultaneous involvement of students, examiners (internal and external), and technical assistants during virtual video conferencing tried to ensure there were no communication gaps.
The difficulties or problems, postgraduates faced during online mode were fear from disconnection of the internet or electricity and audiovisual aid (17). This can be especially true in countries with poor network connectivity. Lack of reliability can relate to the hardware, software, or even the power supply. We need to remember that since it is a technology-based system there is always a chance of technical failure. Though the only drawback encountered during the online evaluation was, a few interruptions due to varying internet speeds and data transfer, the sessions were manageable. All these internet interruptions were due to technical reasons such as ambient climatic conditions, distance from the tower of the internet service provider, electricity interruptions of internet modems, and varying internet speeds during the online examination. Despite these minor limitations, the online mode of examination was a far better option than tedious mobilization, excessive utilization of human/ healthcare/ patient/ instrumentation/ equipment resources, and need to travel required for a direct physical evaluation. Very few Indian studies addressed the satisfaction of students and faculty during virtual conduct of practical examinations of medical students during COVID times (18-20). The present study represents a one of its kind original research from India, with specific emphasis on the satisfaction of students with a virtual mode of practical examination.
Another major challenge is that of infrastructure. Investments must be made to ensure that a standard infrastructure is provided to institutions across the country. It is currently a subject of controversy in current research as to whether an online assessment is associated with higher costs in terms of initial investment than traditional forms of assessment, as it incurs its own unique set of costs (for example, software costs) Some educational software products are, in fact, available free of charge (21). As with other components costs overall should fall and value should increase in the long term (22). Currently, online assessment is used to assess knowledge (ideally, applied knowledge); however, newer technologies will enable the assessment of simulated clinical skills online (perhaps employing sophisticated laparoscopic simulators as suggested by some authors (23,24) Overall results indicated that all postgraduates and faculty are satisfied with their experience of online examination. We recommend periodic feedback from both students and faculty for improving the process of online assessment
Conclusion
Conducting postgraduate practical examinations online is still relatively new, however, it will become mainstream over the next decade worldwide. All postgraduate medical students and all examiners expressed satisfaction with online medical evaluation during this COVID pandemic. Our findings suggest that virtual conduction of practical medical examination through virtual video conferencing platforms appears to be an optimal alternative during the COVID pandemic.
Limitations
- Only one batch of PG students was taken in this study, however, it could be extended to all postgraduates for wider acceptability.
- Due to time constraints, the number of encounters were less so the study did not measure the validity and reliability.
- Only 10 faculty members were included in this study due to Covid-19 and social distancing
Recommendation
We recommend investment in technology and internet services and periodic feedback from both students and faculty for improving the process of online assessment
Acknowledgments
Special thanks are due to all students and faculty members of the Department of Obstetrics and Gynaecology J.N.M.C, A.M.U., Aligarh. U.P.-2002001 who participated in this study.
Conflict of Interests
The authors declare that there is no conflict of interest regarding the publication of this paper.
References
- Singer, M., Sarivan, L., Vries, P., & Muhren, A. (2010). Competent teachers for the knowledge society — A new master program. Buletinul, 62(1B), 107—116.
- Australian Flexible Learning Framework and National Quality Council (AFLFNQC). (2011, August 10). E-assessment guidelines for the VET sector. Australian Government Department of Education, Employment and Workplace Relations. Commonwealth of Australia, Canberra. Retrieved from flexiblelearning.net.au. Accessed on February 18, 2012.
- Aly, E. (2011, July 4—6). Students’ perceptions toward computer based tests: The case of science college. Proceedings of Edulearn 11 conference, Barcelona, Spain (pp. 1112—1122).
- Foxcroft, C., & Davies, C. (2006). Taking ownership of the ITC’s guidelines for computer- based and Internet-delivered testing: A South African application. International Journal of Testing, 6(2), 173—180.
- Agrawal V, Nath C, Mishra SK. Ethics committee meeting by video-conferencing during Covid-19. Indian J Med Ethics. 2020:1e2. https://doi.org/10.20529/ ijme.2020.062.
- Lal H, Sharma DK, Patralekh MK, Jain VK, Maini L. Out Patient Department practices in orthopaedics amidst COVID-19: the evolving model.JClinOrthopTrauma.2020.https://doi.org/10.1016/j.jcot.2020.05.009
- Sabharwal S, Ficke JR, Laporte DM. How we do it: modified residency pro- gramming and adoption of remote didactic curriculum during the COVID-19 pandemic. J Surg Educ. May 2020. https://doi.org/10.1016/j.jsurg.2020.05.026.
- Browning MHEM, Larson LR, Sharaievska I, et al. Psychological impacts from COVID-19 among university students: risk factors across seven states in the United States. PLoS ONE. 2021;16(1):e0245327. doi: 10.1371/journal.pone.0245327. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
- Son C, Hegde S, Smith A, et al. Effects of COVID-19 on College Students' Mental Health in the United States: interview survey study. J Med Internet Res. 2020;22(9):e21279. doi: 10.2196/21279. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
- Sharma D, Bhaskar S. Addressing Covid-19 burden on medical education and training: the role of telemedicine and tele-education during and beyond pandemic. Front Public Health. 2020;8:589669. doi: 10.3389/fpubh.2020.589669. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
- Megaloikonomos PD, Thaler M, Igoumenou VG, et al. Impact of the COVID-19 pandemic on orthopaedic and trauma surgery training in Europe. Int Orthop. 2020;44(9):1611–1619. doi: 10.1007/s00264-020-04742-3. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
- Chasset F, Barral M, Steichen O, et al. Immediate consequences and solutions used to maintain medical education during the COVID-19 pandemic for residents and medical students: a restricted review. Postgrad Med J. 2021 doi: 10.1136/postgradmedj-2021-139755. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
- Park SH. Personal protective equipment for healthcare workers during the COVID-19 pandemic. Infect Chemother. 2020;52:165–182. doi: 10.3947/ic.2020.52.2.165. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
- Goel S, Hawi S, Goel G, et al. Resilient and agile engineering solutions to address societal challenges such as coronavirus pandemic. Mater Today Chem. 2020;17:100300. doi: 10.1016/j.mtchem.2020.100300. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
- Panchangam RB, Kota SK, Mayilvaganan S. Letter to the Editor: endocrine and diabetes clinical practice during national lockdown and post lockdown period. Diabetes Metab Syndr. 2020;14:479. doi: 10.1016/j.dsx.2020.04.037. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
- Shekhar S, Wurth R, Kamilaris CDC, et al. Endocrine conditions and COVID-19. Horm Metab Res. 2020;52:471–484. doi: 10.1055/a-1172-1352. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
- Mohamed Abdelraouf Attia, Postgraduate students’ perceptions toward online assessment: The case of the faculty of education, Umm Al-Qura university Education for a Knowledge Society in Arabian Gulf Countries International Perspectives on Education and Society, May 2018 Volume 24, 151—173.
- Sadeesh T, Prabavathy G, Ganapathy A. Evaluation of undergraduate medical students' preference to human anatomy practical assessment methodology: a comparison between online and traditional methods. Surg Radiol Anat. 2021;43(4):531–535. doi: 10.1007/s00276-020-02637-x. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
- Pettit M, Shukla S, Zhang J, et al. Virtual exams: has COVID-19 provided the impetus to change assessment methods in medicine? Bone Jt Open. 2021;2(2):111–118. doi: 10.1302/2633-1462.22.BJO-2020-0142.R1. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
- Rajan R, Radhakrishnan DM, Srivastava AK, et al. Conduct of Virtual Neurology DM Final Examination during COVID-19 Pandemic. Ann Indian Acad Neurol. 2020;23(4):429–432. [PMC free article] [PubMed] [Google Scholar]
- Stockwell JW., Jr Free software in education: A case study of CWP/SU: Seismic Un*x. The Leading Edge. 1997;16(7):1045–1050. [Google Scholar]
- Walsh K, Jaye P. Cost and value in medical education. Educ Prim Care. 2013 Sep;24(6):391–393. [PubMed] [Google Scholar]
- Walsh K, Rafiq I, Hall R. Online educational tools developed by Heart improve the knowledge and skills of hospital doctors in cardiology. Postgraduate medical journal. 83(981):502–503. [PMC free article] [PubMed] [Google Scholar]
- Lewiss RE, Hoffmann B, Beaulieu Y, Phelan MB. Point-of-care ultrasound education: the increasing role of simulation and multimedia resources. J Ultrasound Med. 2014 Jan;33(1):27–32. [PubMed] [Google Scholar]
