Approaches to TBL assessment in Medicine

The choice of delivering the module as TBL came from the need to assess team-working skills more effectively. Many high achieving students, while well-versed in medical knowledge, would fall short of working effectively in a team, which is an essential skill for doctors in a clinical environment. TBL had also been historically successfully used to encourage teamwork in BSc Biomedical Science and BSc Pharmacology students.  

Electronic delivery of assessments using Learning Activity Management System (LAMS) software provides real-time data as students go through the assessment. Electronic assessments also allow for automated marking of the iRAT and tRAT segment, which is important considering the huge undergraduate cohorts in Medicine of about 400 students per academic year. 

LAMS provides the ability to receive live data from student submissions, individually (iRAT) or as a group (tRAT, tAPP). The teacher or examiner has full control over the session and how students progress through the sequence of tasks. One further use of this data is to help mend and shape future sessions; if a question is only answered correctly by 20% of students, for example, then the data may suggest the question is misleading or too difficult. Examiners can then investigate this particular question, taking actions as necessary, as well as write future questions avoiding the same shortcomings. 

The iRAT and tRAT assessments consist of Single Best Answer (SBA) questions. In the iRAT, questions are designed to link more closely to the actual clinical and scientific teaching that students receive in the module. The tRAT, whilst also clinically relevant, is designed to encourage group communication as well. The tAPP focuses largely on developing team-working skills and important skills outside of clinical and scientific knowledge. The question design is therefore much broader and can be condensed into three main categories: data interpretation, infographics, and more complex clinical cases (see an example of a case study here). 

The module is constructed as a collaborative process between other modules. It is intended to connect content-wise with other parts of the curriculum so that students have a consistent knowledge base and understand what and why they are being taught the content in a team-working scenario.  

Some of the program level Intended Learning Outcomes have a heavy team-working aspect. The ability to work in a team is heavily weaved into this module’s specification and thus lends itself to these ILOs well.  

The module and module assessment continues over the three years of Phase 1. This means that students will develop a sense of consistency and familiarity with the assessment type, allowing for long-term progression of integrated knowledge from case-based learning in a team-based environment. 

To prepare students, there are formative TBL assessments in other modules that serve as preparatory work for CSI. Within CSI, a number of formative cases are constructed and then delivered via a Teams call to allow students to practice in their groups and to facilitate the process of the real assessment in-person. These formative cases are predominantly tAPPS, most likely due to their complexity for both students to complete and for staff to construct and mark. 

The iRAT and tRAT are auto-marked. The mark allocation is set up in a way that is as safe as possible for students; even if an individual student performed very poorly in the iRAT, the team-based component would protect them, making it virtually impossible for the student to fail the module.  That being said, for the tRAT, the marking system is quite punitive, with students being allowed four attempts to select the correct answer. 

The tAPP segment is marked by staff and is just as complex to mark as the questions themselves. There are usually more than eight individuals marking tAPPS, and they are all double marked.  

For the iRAT and tRAT, students have a post-assessment 24-hour window where they are given the opportunity to challenge questions for any reason. Once all challenges are accrued, the module staff will sit down, work through the challenges and collectively decide on their outcome. 

The tAPP feedback, being marked manually, generally takes longer. It is typically a 2 week turnaround.  

The module is managed by leads and fellows with one Science Lead (interviewee), one Clinical Lead, one Science Fellow and one Clinical Fellow. In a typical in-person session, there are five to six people who assist in delivering the content to students. These will be clinicians (i.e. medical practitioners), scientists (i.e., staff members involved in scientific research) etc. to encourage a range of perspectives for better delivery of content to students. Having clinicians and scientists in the room to assist in delivering sessions means that most questions asked by students can be answered.

During Covid the module and assessments were delivered online. Student groups were placed in breakout rooms with tutors invigilating by moving in and out of breakout rooms. Overall, student performance did not change, but student behaviour did change. Teams allowed students to turn off their cameras and microphones, which many of them did, allowing for student disengagement in groups.  

The student view of TBL is generally positive. There is a recognition of this way of learning being linked to real life practice in MDT (multidisciplinary) meetings where you are expected to express your opinion which is then discussed further with others. The students recognised that the varied tAPPs were a lot more difficult to prepare for. 

The actual style of the CSI questions were thought to be quite difficult.  There was a feeling of questions lacking clarity and that they were not really testing the knowledge, but testing students’ ability to pick up on those tricks.  

There also seemed to be a lot of disparity between the groups with some groups having 5 members and some seven. When it came to doing the tAPP tasks, smaller groups would often struggle because of less people working in a time pressure environment.