Using Pretesting To Validate Assessment Products

Before I joined the ELT Assessment Department at Oxford University Press I was unaware of the lengths that this prestigious organisation goes to in order to ensure that the products it creates and releases to market are of the quality expected from such a huge brand. Upon joining this influential and acclaimed establishment I was instructed that one of my main duties was to ensure our pretesting targets were met and executed to the highest standards.

What in the world is pretesting?

In a nutshell, pretesting is the research activity we run to ensure that each and every question we include in any of our online assessment products (be it the Oxford Placement Test or the Oxford Test of English) undergoes rigorous analysis to ensure the questions are good enough to be included. This research is crucial in ensuring that we validate the ability of each question in terms of its capability to report on the CEFR correctly, as well as ensuring that questions are authentic and fair for students across a range of backgrounds, language groups and cultures.

So how do we do it?

Working with amazing teachers worldwide

Pretests themselves are a group of questions for a specific skill that our item writers have produced with a specific CEFR level in mind. In order to be sure that they really are measuring a student’s ability at the CEFR level in question, we need to get students to answer these questions and then use the data to see which questions we want to include and which we should discard.

In order to find these students our Pretesting Team contact our international research partners (teachers that are currently running English language courses) to see whether they have students available at the age-group and CEFR of interest. Once we know how many students are available we issue pretest licences that are valid for the period of a month. It is then up to our fantastic partners to decide when they will hold their online pretest session. They can do so at any time that suits them within that monthly period. We understand how the working day of a teacher is often unpredictable and so giving our partners this flexibility is key to a successful collaboration.

Most teachers are super busy. How do we convince them to get involved?

The majority of our teachers need no convincing! They already understand how valuable this kind of research is to advance assessment. And as the world around us changes and more of us than ever before are forced into the world of virtual learning and online testing, there is a lot of support for getting students comfortable taking tests online. It’s not always a pleasant feeling to talk at a computer, and anything we can do to give students more exposure to online testing will no doubt benefit them in their ongoing education and careers.

While pretest scores cannot be used to confirm a students’ ability level, nor to place students into a class of a specific CEFR level, they can be a useful guide for teachers to understand how each student in their class is performing against expectations. It can also highlight which skills a student may need focus on in order to improve their overall ability.

Register for the webinar to learn more about pretesting

Join me in the webinar to find out more about what pretesting is and what it is not, and consider whether it’s an activity that you could run with your students. It takes place all year round, even during the summer months, so there is plenty of time to register your interest and get pretesting!

Melissa Bacon entered the world of ELT when she began running residential English summer school courses in boarding schools across England in 2009. After 5 years she moved into a content creation role, writing English assessment questions for Wiloki, an online platform for French students that want to improve their English skills using a fun and interactive methodology. Since 2015 she has led the operational team in the Assessment department, focusing on pretesting research, test centre onboarding, and assessor management.