Technology’s influence on assessment design
|How technology shapes assessment design: Findings from a study of university teachers. British Journal of Educational Technology, 2016. Bennett, S., Dawson, P., Bearman, M., Molloy, E. & Boud, D.|
A wide range of technologies has been developed to enhance assessment, but adoption has been inconsistent. This is despite assessment being critical to student learning and certification. To understand why this is the case and how it can be addressed, we need to explore the perspectives of academics responsible for designing and implementing technology-supported assessment strategies. This paper reports on the experience of designing technology-supported assessment based on interviews with 33 Australian university teachers. The findings reveal the desire to achieve greater efficiencies and to be contemporary and innovative as key drivers of technology adoption for assessment. Participants sought to shape student behaviors through their designs and made adaptations in response to positive feedback and undesirable outcomes. Many designs required modification because of a lack of appropriate support, leading to compromise and, in some cases, abandonment. These findings highlight the challenges to effective technology-supported assessment design and demonstrate the difficulties university teachers face when attempting to negotiate mixed messages within institutions and the demands of design work. We use these findings to suggest opportunities to improve support by offering pedagogical guidance and technical help at critical stages of the design process and encouraging an iterative approach to design.
Electronic exam hacking
|Five ways to hack and cheat with bring-your-own-device electronic examinations British Journal of Educational Technology 2015. Dawson, P.|
Bring-your-own-device electronic examinations (BYOD e-exams) are a relatively new type of assessment where students sit an in-person exam under invigilated conditions with their own laptop. Special software restricts student access to prohibited computer functions and files, and provides access to any resources or software the examiner approves. In this study, the decades-old computer security principle that ‘software security depends on hardware security’ is applied to a range of BYOD e-exam tools. Five potential hacks are examined, four of which are confirmed to work against at least one BYOD e-exam tool. The consequences of these hacks are significant, ranging from removal of the exam paper from the venue through to receiving live assistance from an outside expert. Potential mitigation strategies are proposed; however, these are unlikely to completely protect the integrity of BYOD e-exams. Educational institutions are urged to balance the additional affordances of BYOD e-exams for examiners against the potential affordances for cheaters.
Online research ethics
|Our anonymous online research participants are not always anonymous: Is this a problem? British Journal of Educational Technology 2014. Dawson, P. Download PDF pre-print|
When educational research is conducted online, we sometimes promise our participants that they will be anonymous—but do we deliver on this promise? We have been warned since 1996 to be careful when using direct quotes in Internet research, as full-text web search engines make it easy to find chunks of text online. This paper details an empirical study into the prevalence of direct quotes from participants in a subset of the educational technology literature. Using basic web search techniques, the source of direct quotes could be found in 10 of 112 articles. Analysis of the articles revealed previously undiscussed threats from data triangulation and expert analysis/diagnosis. Issues of ethical obliviousness, obscurity and concern for future privacy-invasive technologies are also discussed. Recommendations for researchers, journals and institutional ethics review boards are made for how to better protect participants’ anonymity against current and future threats.
A follow-up blog post on that topic: Ethics in social media research: do we know what we are doing?
|Motivation and cognitive load in the flipped classroom: definition, rationale and a call for research Higher Education Research & Development, 2014. Abeysekera, L. & Dawson, P.|
Flipped classroom approaches remove the traditional transmissive lecture and replace it with active in-class tasks and pre-/post-class work. Despite the popularity of these approaches in the media, Google search, and casual hallway chats, there is very little evidence of effectiveness or consistency in understanding what a flipped classroom actually is. Although the flipped terminology is new, some of the approaches being labelled ‘flipped’ are actually much older. In this paper, we provide a catch-all definition for the flipped classroom, and attempt to retrofit it with a pedagogical rationale, which we articulate through six testable propositions. These propositions provide a potential agenda for research about flipped approaches and form the structure of our investigation. We construct a theoretical argument that flipped approaches might improve student motivation and help manage cognitive load. We conclude with a call for more specific types of research into the effectiveness of the flipped classroom approach.
- In 2012 I led a team that developed the Moodle ‘Engagement Analytics’ plugin. It’s now in use at a variety of universities. Here is a page with lots more info.
- I am Associate Editor on the open-access Journal of Learning Analytics.