The goal of course assessment is to evaluate whether students have effectively fulfilled their learning objectives. Regardless of format, an assessment should allow for valid and reliable evidence of students’ achievement.
Debates about whether open-book (OBAs) or closed-book assessments (CBAs) are more likely to improve academic performance and student well-being are common. CBAs appear to be a more effective tool for testing the lower levels of Bloom’s taxonomy, such as remembering, and are associated with less student cheating. However, OBAs appear to be more productive for developing and testing higher levels of the Bloom’s taxonomy scale, in skills such as analysing and evaluating, and are associated with lower levels of student anxiety.
- 21st-century assessment: improve and update the way you evaluate
- Why you should write feedback to your students before they’ve submitted
- A step-by-step guide to designing rubrics that will save hours of marking time
As a traditional mode of assessment, CBAs can tend towards encouraging memorisation and recall rather than probing student mastery of knowledge and their capacity to apply this knowledge to novel situations.
In contrast, OBAs can, if designed effectively, encourage students to think in a more critical and analytical way, developing conceptual understanding and higher-order cognitive skills, which aligns well with the course outcomes that anchor these skills within the discipline. While OBAs are becoming more and more common in many disciplines, taking full advantage requires careful consideration of examination design and assessment intentions.
So, how can educators move from in-place CBAs to remote OBAs? This article aims to summarise conceptual considerations and practical tips in a short list of dos and don’ts of good practice:
Must-dos for introducing open-book assessment
1. Put an explanation in the syllabus or let students know well in advance that the assessment will be open book. Make it clear that their understanding of the course content, and not just recall of facts, will be addressed. Prepare a couple of mock exams that illustrate the kinds of knowledge and understanding that will be assessed, making sure students are aware of the learning objectives.
2. Time allocation for the exam should be realistic and manageable, while keeping the questions suitably challenging. Well-designed OBAs are typically more complex and require considerable thought. More time is needed to answer questions in an OBA than is needed for CBAs, which mainly rely on recall. Test timing in advance.
3. Let students know in advance the explicit criteria that the exam will be assessing – in particular, explain the expectations for referencing style and word length. For instance, economics application questions might require diagrams and algebra as well as longer, analytical answers. This may differ from pure mathematics, which puts more emphasis on direct deduction with algebra. In STEM subjects, share with students examples of exam questions and have a discussion about what model (or “good”) responses look like.
4. For students who have English as a second language, remember that lengthy, writing-heavy exams (such as essays) may pose challenges and therefore put such students at a disadvantage. It may be more appropriate to develop strategies such as including concise Q&A sections or short comment on cases to meet the goal of OBAs for those students. In economics, it is rare to challenge students by asking for long-form writing.
What not to do when introducing open-book assessment
1. Don’t assume that students are aware of the expectations and purpose of an OBA. Students may be confused about how to link the expectations of the exam or assessment to course learning outcomes.
2. Don’t set questions or tasks that require students to conduct extensive “new” (that is, unfamiliar to them) research into the literature. Students should be able to address exam questions using their notes and subject materials alone.
3. Don’t design questions that are easy to answer or too narrowly based; this increases the risk of plagiarism and collusion. Set divergent tasks instead, where students can take different approaches to demonstrate their achievement of learning objectives.
4. Don’t give students an unrealistic time limit either for in-class or take-home exams, in which they are not able to complete all the tasks with time to spare even if they have prepared well. Make sure the time limit is loose enough for students to perform well and reflect their learning.
Hongru Zhang is associate professor in the School of Liberal Arts at Macau University of Science and Technology.
If you found this interesting and want advice and insight from academics and university staff delivered direct to your inbox each week, sign up for the THE Campus newsletter.
comment