 |
|
||||
|
|||||
| About | Latest Articles | Back-Issues | Articles
|
Search | Instructions | Online Submission | Subscribe | Etcetera | Contact |
| |||||||||||||||||||||
|
Integrating students' reflection-in-learning and examination performance as a method for providing educational feedback V Devi1, T Mandal1, S Kodidela1, V Pallath21 Department of Pharmacology, Melaka Manipal Medical College, Manipal Campus, Manipal University, Manipal, Karnataka, India 2 Department of Microbiology, Melaka Manipal Medical College, Manipal Campus, Manipal University, Manipal, Karnataka, India
Correspondence Address: Source of Support: None, Conflict of Interest: None DOI: 10.4103/0022-3859.105447
Background: Feedback provided by integrating students' examination performance and the reflection on learning process may assist medical students to identify the essential learning processes required for better understanding of learning material. Aims: To investigate the relationship between the student's learning process and examination performance and to explore students' perspectives on professional feedback given by integrating the self-reflection on learning process and individual examination performance. Settings and Design: At the end of every medical school block examination, faculty provided descriptive feedback to students regarding individual performance in each subject area. The study had a quasi-experimental design. Materials and Methods: Students' (n=153) self-reflection on learning process was collected using a reflection-in-learning scale. The learning process of the students categorized in fail, pass and first-class categories were compared. Students' self-reflection-in-learning and common mistakes found in answering essay questions were used to offer feedback. Students' perspectives regarding feedback were collected using a validated questionnaire. Statistical Analysis: Kruskal-Wallis test and Mann-Whitney test were used to analyze the data. A P<0.017 was considered as statistically significant. Results: The median score of each item related to reflection was ≤4 (Scale 1-7) in all categories of students (n=153) reflecting a low level of reflection. There was a statistically significant difference in total reflection scores between fail (n=46) and first class category (n=42) students (P=0.002). The majority of students agreed feedback assisted in examination preparation, enhanced individual learning and allowed examinations to be viewed as more objective. Conclusions: This study shows the usefulness of the integration of medical students' reflection-in-learning practices and examination performance in providing descriptive feedback. This modified feedback process may have improved students' awareness leading to acceptance and a conscious practice of self-reflection and enhanced students' examination performance. Keywords: Descriptive, examination performance, feedback, reflection-in-learning
Critical reflection is the process of analyzing and reframing an experience to make an assessment of it for learning (reflective learning) and/or to improve practice (reflective process). [1] During learning, reflective practice is helpful to understand the learning needs [2] required to adopt deep approaches to learning, to understand one's personal beliefs, attitudes and values to integrate them into learning [3] and to build integrated knowledge bases which involves linking new to existing knowledge. In students, reflective thinking at the deeper levels is associated with deep approaches to learning and meaning-making. [4] Sobral (2000) showed the practice of reflection-in-learning may induce readiness for self-regulation of learning, [5] where students' thoughts, feelings and actions are self-regulated for attaining academic goals. [6] Reflection-in-Learning scale (RLS) designed by Sobral (2000) [5] to apprise self-reflection in learning operates at 3 related areas, decision making process of initial learning, up gradation of learning and learner's self-appraisal. [7] The alpha reliability coefficient (internal consistency) for this scale was 0.81 and test-retest correlation was 0.65 after 3 months. The items in the RLS highlight processes required for effective learning, including students' search for integration of content, self-monitoring of learning and procedural adaptation. Thus RLS may be a useful tool for the appraisal of students' learning processes embedded in their learning profile. The study which investigated the relationship between measures of reflection and academic performance showed that high achievers scored high on the RLS, both at the start and at the end of the academic term. [7] Hence, assessment of self-reflection might be beneficial to help students in learning as its measurement might help us to understand the reason for differences in learning of high and low achievers. Feedback and reflection are two basic teaching-learning methods used in clinical setting. [8] Descriptive feedback, which conveys information of how one performs the task and a detailed way to overcome difficulties is more effective than evaluative feedback. [9] Hence, in this study it is hypothesized feedback given by integrating students' self- reflection on learning process with performance in examinations may help students to relate learning process to examination performance. This may lead them to consciously adopt better learning process for better performance in examination. Objectives of this study were: to investigate the relationship between the student's learning process and examination performance and to explore students' perspectives on professional feedback given by integrating the self-reflection on learning process and individual examination performance.
Educational context Our institution offers the Bachelors of Medicine and Bachelors of Surgery (MBBS) program at two campuses. Students admitted into the institute are initially placed at the campus in India (Phase I of the program). After successfully completing two and a half years of pre-clinical training, they return to their home country for the clinical training (Phase II of the program). Phase I of the program is divided into 3 stages, Stage I, Stage IIa and Stage IIb. Students study anatomy, physiology and biochemistry during Stage I. Pathology, pharmacology, microbiology and forensic medicine are taught during Stage IIa. In Stage IIb, clinical courses are taught, such as medicine, surgery etc. The courses are taught in four blocks during Stage I and II. At the end of each block a summative examination is provided and the average of the four block examination marks forms an internal assessment for the university examination. At the end of every block examination, faculty members provide a descriptive feedback to students regarding performance in each subject after having gone through the evaluated answer scripts. The present study was conducted during block 3 of stage IIa (August to October, 2010). Instruments To collect students' self-reflection on learning process, RLS (Sobral, 2000) was used following a required modification. [5] This questionnaire had 12 items. The alpha reliability coefficient (internal consistency) for the modified scale was 0.88. The RLS described by Sobral, 2000 was designed for the entire medical program. In this study, students learning process in pharmacology was explored. Moreover, our students understand MBBS program as course and various courses as subjects such as, pharmacology, pathology etc. Hence the RLS was suitably modified as suggested by experts during face and content validity. Items 8 and 12 of original RLS were deleted. Students were asked to respond in 7-point scale in RLS. Student feedback questionnaire was designed with open and closed ended questions to collect students' perspectives regarding examination feedback. Two faculty of the institution; with Foundation for Advancement of Medical Education and Research (FAIMER) fellowship degree and experience in validation, checked face and content validity of questionnaires. Students were asked to respond in 5-point Likert Scale to close ended questions. Study subjects 153 students of 21-23 years of age participated in this study before the examination feedback. Of these 49% were males and 51% females. Majority of the participants were of Malaysian origin constituting 92.8%, followed by 5.2% from Seychelles and the rest were from Sri Lanka. Study design The study design was prospective and quasi experimental [Figure 1]. Convenience sampling was completed without performing a power analysis. Following stage 2A, block 2 summative examinations in pharmacology, faculty evaluated students' restricted response essay answers using an answer key prepared by faculty members. The answer key provides the expected key points with the mark division to minimize subjectivity in assessment. Following the evaluation of answer scripts, common mistakes made by the students in answering the essay questions were noted. When students proceeded to block 3 of stage 2A, students' self-reflection on learning process was collected by providing the RLS. The data was analyzed and the learning process of the students in the fail with <50% of marks, pass with 50-64% marks and first class with ≥65% marks were compared.
Students' self-reflection on learning process and common mistakes made in answering essay questions were used to give descriptive feedback during the second week of block 3. During the feedback, the expected answers, answer key, interpretation of clinical case scenarios in the essay question paper, the level of reflection-in-learning for various categories of students, learning processes used and benefits of self-reflection-in-learning were offered to the students. Students were also encouraged to self-assess themselves by using previous year question papers and intended learning outcomes in pharmacology. Following Stage 2A, block 3 examinations, students' perspectives regarding feedback given at the beginning of block 3 were collected in the student feedback questionnaire. The study was approved by the Institutional Research Committee and the University Ethics Committee. Written informed consent was obtained from the study participants before obtaining any research-related responses. Statistical analysis The analyses were performed using the Statistical Package for the Social Sciences (SPSS) version 16. The categorical variables were described as median and interquartile range. Based on the total essay score, students were grouped into 3 categories: i) Fail- <50% of marks, ii) Pass- 50-64% marks, iii) First class- ≥65% marks. Comparison of self-reflection-in learning between the categories was completed by using Kruskal-Wallis test and Mann-Whitney test for pair-wise comparison. A P<0.017 was considered as statistically significant after a Bonferroni correction. Correlation of the category wise total reflection-in-learning score with the total essay marks of the student was performed using the Spearman Rho. Frequency analysis of responses to close-ended questions regarding feedback was done and expressed as a cumulative percentage of 'agree' and 'strongly-agree' responses. The students' responses to open-ended questions were tabulated in decreasing order of frequency of appearance. Later responses conveying same meaning were grouped under various themes independently by first two authors. Identified themes were finalized by consensus among authors [Table 1].
Students' response rates were 100% and 97.3% (149/153) before and after giving the examination feedback respectively. Total reflection score weakly correlated with the total essay marks of the students. The median score of each item related to reflection was ≤ 4 in all categories of students reflecting low level of reflection. There was a statistically significant difference in total reflection scores between 'fail' and 'first class' category students (fail: 40 (35, 47), first class: 50 (39, 56), P = 0.002) [Figure 2]. However, the difference between 'fail' versus 'pass' (45 (36, 52) and 'pass' versus 'first class' categories were not statistically significant [Figure 2]. First class category students more frequently planned the learning tasks carefully than 'pass' and 'fail' category students [Figure 3]. Moreover, 'first class' category students more often performed self-assessment of work, removed the negative feelings related to studying and mentally processed what they already knew and needed to know compared to the 'fail' category of students [Figure 4] and [Figure 5]. 74% of students agreed feedback helped in the preparation for the block 3 examination whereas 73% of the students opined feedback enhanced learning. 63% of students felt feedback made the examination more trustworthy and objective. [Table 1] shows the students' perceptions regarding the role of feedback in improving examination performance in decreasing order of frequency of their appearance.
The present study investigated the level of medical students' reflection-in-learning at the beginning of the 2 nd year of the curriculum. The study revealed the benefits of professional feedback for students when integrating students' reflection-in-learning and written examination performance. Due to this encouraging result, it appears worthwhile to study the extent to which feedback contributed toward improving students' overall performance. Therefore, the impact of feedback on students' performance needs to be explored. Low level of reflection-in-learning among students prior to giving feedback [Figure 3],[Figure 4], [Figure 5] suggests students were not aware of the learning benefits associated with reflection-in-learning. Introduction of reflective tasks at the beginning of undergraduate medical education may increase students' awareness and understanding of reflective practices. [10] First class category students carefully planned learning tasks and also mentally processed what was known and what still needed to be learned. These results indicate the students in the first class category were better in making decisions regarding the process of initial learning compared to pass and fail category students [Figure 3] and [Figure 4]. Moreover, self-monitoring of learning was happening more frequently in the 'first class' category students compared with the other category [Figure 5]. The highest total reflection-in-learning score for 'first class' category students suggest students in this category were more motivated to achieve goals than students in the other two categories. Despite these observed differences between different categories, integration of subjects/topics and procedural adaptation did not take place in any of the categories. The weak correlation between total reflection-in-learning scores and academic performance could be due to the complex demands of medical studies and various other external factors. Mann et al in their systematic review concluded that reflective practices need to be further developed and the ability to reflect enhanced while the presence of certain external stimuli exists. [11] In our study, feedback based on the students' reflection-in-learning might have stimulated them to practice reflection. Outcomes of reflection may include new perspectives on experience, readiness for application, and change in behavior or commitment to action. [2] These outcomes were evident from students' perspectives on feedback [Table 1]. Studies had shown a positive and significant relationship exists between reflection-in-learning and meaning orientation of approaches to study inventory. [7] The practice of reflection may allow students to realize the need of deeper approaches to study. [2],[10] This outcome of reflection also was evident in our study. Improved examination performance could be due to better self-monitoring of learning or better procedural adaptation as perceived by students. The feedback given connecting the level of reflection-in-learning and examination performance might have helped the students to understand the processes required during learning for better understanding of the learning content. The professional feedback given that integrated examination performance and reflection-in-learning allowed students to reflect both on the process and the content of learning, which in turn assisted students to have more conscious control over the learning process and influenced academic performance. The use of newly developed intended learning outcomes for self-monitoring could have contributed to improving students' academic performance. [12] It seems appropriate to further study the impact of feedback on students' reflection-in-learning and academic performance. The impact of reflection-in-learning on students' study approaches also may be explored in the future. This study's outcomes underline the importance of structured feedback coupled with self-reflection in improving the students' learning experience. Results also emphasize the importance of creating awareness about the benefits associated with self-reflection among medical students in order to improve and enhance overall learning. These interventions may positively serve in creating a better professional through medical education. The limitations associated with this study include the quasi-experimental design with convenient sampling. An experimental design with subjects randomized into case and control groups would have increased the validity of this study as it would have showed the extent to which the feedback contributed in improving students' examination performance. A research design with case and control groups would illuminate the impact of the intervention on students' learning. But this design requires the availability of students over a long period of time as the control group also needs to be benefited from the educational intervention to make sure the study is ethically sound. Furthermore, if an experimental design could be adopted, the control group would benefit from the intervention only following the examination. This design is not ethically desirable and the learning institution is not in favor with this research design proposal. The second limitation of the study is it was based partly on self-reported questionnaires. Students' portfolios are better source to assess the learning process. Furthermore, focus group discussions with students would allow students the opportunity to share their perceptions regarding the benefits of the educational intervention. In conclusion, examination feedback given based on reflection-in-learning shows students the way to reach correct solutions in the myriad of matters related to individual study habits. This study provides insights related to the integration of students' reflection-in-learning and examination performance that will improve students' awareness that may lead to acceptance and conscious practice of self-reflection. The practice of self-reflection may assist students to become active and lifelong learners.
Students of Batch 25 of Melaka Manipal Medical College, Manipal, India who participated in this study. Dejano T. Sobral, Department of Medicine, University of Brasilia, Faculty of Health Sciences, Medicine C.P. 15-3031, 70910-900 Brasilia, Brazil for permitting us to use the reflection-in-learning scale.
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5]
[Table 1]
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||
