Improving Medical Students' Written Communication Skills
Improving Medical Students' Written Communication Skills
The prospective, partially masked controlled trial was run for six consecutive Internal Medicine clerkship rotations, each lasting 6 weeks, from March 2012 to January 2013. We devised two educational programmes: one for medical students on communication skills (Box 1) and one for senior residents on feedback specifically addressing communication skills (Box 2). The study had three arms: control (arm 1), student communication tutorial only (arm 2) and student communication tutorial with senior resident feedback workshop (arm 3) (figure 1). To avoid contamination, study arms were separated by hospital site. A study assistant collected written consultation notes by the students on newly admitted patients. Notes were collected during the first week of the students' rotation, prior to the delivery of the educational programme, and again in week 6, the final week of their rotation. The study assistant removed all student and patient identifiers. The written consultation notes were scored using the independently validated Assessment Checklist (see below).
(Enlarge Image)
Figure 1.
Study design.
The study participants were 126 medical students on the Internal Medicine clinical clerkship rotation and 13 senior medical residents (Box 3). The study was conducted at the Hamilton General Hospital, St. Joseph's Healthcare Hamilton and the Juravinski Hospital, teaching hospitals affiliated with McMaster University in Hamilton, Ontario, Canada (Box 4).
An Assessment Checklist to evaluate medical student consultation notes was developed after thorough literature review followed by consensus of a focus group consisting of expert faculty and senior residents. The checklist was divided into subsections of common written consultation note headings (history, PE, assessment and plan), as well as handwriting, and consisted of dichotomous Yes/No questions and questions scored on a 7-point Likert Scale. The scale was anchored at 1 and 7 by 'Disagree' and 'Agree'. Each subsection had a Global Score assigned based on a 7-point Likert Scale ranging from 'Poor' to 'Excellent', and a Global Score on the same 7-point scale was assigned to the note as a whole.
The Assessment Checklist was validated using 50 randomly selected medical student consultation notes from the first 100 notes collected. The notes were scored by two raters, a combination of five residents and five staff from the Internal Medicine department. The internal consistency of the checklist was assessed using Cronbach's alpha and found to be 0.79 (95% lower confidence limit 0.73). The inter-rater reliability was calculated for the global consultation note score between all pairs of markers using a linear-weighted kappa score and was calculated to be 0.69. Raters completed a survey to determine the acceptability of the Assessment Checklist; 8/10 (80%) surveyed found the checklist somewhat easy or very easy to use. Through this process, we determined that the Assessment Checklist provided a portable and reliable method to evaluate the medical student consultation notes for our study.
The study assistant collected copies of consultation notes completed by the medical students on call the previous night during week 1 and week 6 of their Internal Medicine rotations from March 2012 to January 2013. Medical student consultation notes were identified via signatures. Student and patient information was removed from the notes, and then consultation notes were pooled. Masked faculty assessors not involved in the study or student evaluation performed consultation note scoring. Assessment Checklist scores were entered into a database.
Scores from the Assessment Checklist were analysed from week 1 to week 6 within study arms. Scores for each of the consultation note subsections (history, PE, assessment and plan), as well as overall scores for the entire consultation note, were analysed between week 1 and week 6. Analyses of improvement were conducted as an aggregate score of week 1 and week 6 students; pre-performance and post-performance of individual students were not studied as identifying student information was removed. Analyses were performed to determine correlations between handwriting scores and subsection scores with the overall global scores.
Global consultation note scores were designated as the primary outcome. An independent t test was used to compare scores between weeks 1 and 6 for each arm of the study. For the changes of global scores between arms, a generalised linear regression model was used, using pre-grouping–post-grouping and treatment arm as the independent variables. Pearson correlation was used to study associations between handwriting and subsection scores (history, PE and assessment and plan) with the overall global score. As no previous studies were available to inform this area, no sample size calculation was performed. Statistical analyses were performed by SPSS (V.20th, IBM Corp, Armonk, New York, USA).
Methods
Study Design
The prospective, partially masked controlled trial was run for six consecutive Internal Medicine clerkship rotations, each lasting 6 weeks, from March 2012 to January 2013. We devised two educational programmes: one for medical students on communication skills (Box 1) and one for senior residents on feedback specifically addressing communication skills (Box 2). The study had three arms: control (arm 1), student communication tutorial only (arm 2) and student communication tutorial with senior resident feedback workshop (arm 3) (figure 1). To avoid contamination, study arms were separated by hospital site. A study assistant collected written consultation notes by the students on newly admitted patients. Notes were collected during the first week of the students' rotation, prior to the delivery of the educational programme, and again in week 6, the final week of their rotation. The study assistant removed all student and patient identifiers. The written consultation notes were scored using the independently validated Assessment Checklist (see below).
(Enlarge Image)
Figure 1.
Study design.
Participants and Setting
The study participants were 126 medical students on the Internal Medicine clinical clerkship rotation and 13 senior medical residents (Box 3). The study was conducted at the Hamilton General Hospital, St. Joseph's Healthcare Hamilton and the Juravinski Hospital, teaching hospitals affiliated with McMaster University in Hamilton, Ontario, Canada (Box 4).
Instrument Design: Written Consultation Note Assessment Checklist
An Assessment Checklist to evaluate medical student consultation notes was developed after thorough literature review followed by consensus of a focus group consisting of expert faculty and senior residents. The checklist was divided into subsections of common written consultation note headings (history, PE, assessment and plan), as well as handwriting, and consisted of dichotomous Yes/No questions and questions scored on a 7-point Likert Scale. The scale was anchored at 1 and 7 by 'Disagree' and 'Agree'. Each subsection had a Global Score assigned based on a 7-point Likert Scale ranging from 'Poor' to 'Excellent', and a Global Score on the same 7-point scale was assigned to the note as a whole.
The Assessment Checklist was validated using 50 randomly selected medical student consultation notes from the first 100 notes collected. The notes were scored by two raters, a combination of five residents and five staff from the Internal Medicine department. The internal consistency of the checklist was assessed using Cronbach's alpha and found to be 0.79 (95% lower confidence limit 0.73). The inter-rater reliability was calculated for the global consultation note score between all pairs of markers using a linear-weighted kappa score and was calculated to be 0.69. Raters completed a survey to determine the acceptability of the Assessment Checklist; 8/10 (80%) surveyed found the checklist somewhat easy or very easy to use. Through this process, we determined that the Assessment Checklist provided a portable and reliable method to evaluate the medical student consultation notes for our study.
Data Collection
The study assistant collected copies of consultation notes completed by the medical students on call the previous night during week 1 and week 6 of their Internal Medicine rotations from March 2012 to January 2013. Medical student consultation notes were identified via signatures. Student and patient information was removed from the notes, and then consultation notes were pooled. Masked faculty assessors not involved in the study or student evaluation performed consultation note scoring. Assessment Checklist scores were entered into a database.
Data Analysis
Scores from the Assessment Checklist were analysed from week 1 to week 6 within study arms. Scores for each of the consultation note subsections (history, PE, assessment and plan), as well as overall scores for the entire consultation note, were analysed between week 1 and week 6. Analyses of improvement were conducted as an aggregate score of week 1 and week 6 students; pre-performance and post-performance of individual students were not studied as identifying student information was removed. Analyses were performed to determine correlations between handwriting scores and subsection scores with the overall global scores.
Statistical Analysis
Global consultation note scores were designated as the primary outcome. An independent t test was used to compare scores between weeks 1 and 6 for each arm of the study. For the changes of global scores between arms, a generalised linear regression model was used, using pre-grouping–post-grouping and treatment arm as the independent variables. Pearson correlation was used to study associations between handwriting and subsection scores (history, PE and assessment and plan) with the overall global score. As no previous studies were available to inform this area, no sample size calculation was performed. Statistical analyses were performed by SPSS (V.20th, IBM Corp, Armonk, New York, USA).
Source...