Shutterstock
<p>Multiple-choice questions are considered objective and easy to mark. But my <a href="https://melbourneinstitute.unimelb.edu.au/__data/assets/pdf_file/0003/3769050/ri2021n05.pdf">research shows</a> they give an advantage to males.</p>
<p>I compared around 500,000 test results of boys and girls who sat the same international test, but whose exam papers differed by detail (although not difficulty). The difference included a varied proportion of multiple-choice questions as opposed to open-ended questions.</p>
<p>I found the gender gap in math scores widened with the share of multiple-choice questions in the exam — advantaging males.</p>
<p>This shows the generally better performance of males in maths exams has to do more with the format of the test than their maths knowledge.</p>
<h2>How I conducted my research</h2>
<p>Standardised exams are widely used to test students and screen job candidates. Australians take several standardised tests throughout their education — such as the NAPLAN, High School Certificate (HSC) and the OECD’s Programme for International Student Assessment (PISA).</p>
<p>Such exams, especially when maths is involved, regularly include multiple-choice questions.</p>
<p>For example, more than 70% of <a href="https://www.acara.edu.au/assessment/naplan/naplan-2012-2016-test-papers">NAPLAN’s 2016 numeracy section</a> was made up of multiple-choice questions. Every year, the maths <a href="https://ace.nesa.nsw.edu.au/higher-school-certificate">HSC tests</a> include a section with multiple-choice questions.</p>
<p>These prompt students to identify the correct response from a set of possible answers.</p>
<p>I analysed data from PISA 2012 and 2015. <a href="https://www.oecd.org/pisa/">PISA</a> is the largest international standardised test in maths, reading and science. Every three years, more than 500,000 students aged 15, from more than 60 countries, including Australia, take the test.</p>
<p>Each student taking the PISA receives a different set of questions which are of similar context and difficulty. But there is a random variation in the proportion of multiple-choice questions each student gets in their test booklet.</p>
<p>For instance, in 2015, some students received an exam mostly made up of multiple-choice questions (70%), while other students’ exam papers contained only 30% multiple-choice questions.</p>
<p>I exploited this random variation in the proportion of multiple-choice questions to investigate how gender differences in maths performance vary.</p>
<h2>What I found</h2>
<p>Females performed worse than males on multiple-choice questions — this was especially the case when they received an exam booklet with 60% or more multiple-choice questions.</p>
<p>An increase in the share of multiple-choice questions by ten percetage points (such as from 50% to 60%) increased the gender gap in maths scores by 50% in favour of boys.</p>
<hr />
<p><iframe style="width: 100%; height: 600px;" title="Interactive or visual content" src="https://flo.uri.sh/visualisation/6272666/embed" width="100%" height="400" frameborder="0" scrolling="no" sandbox="allow-same-origin allow-forms allow-scripts allow-downloads allow-popups allow-popups-to-escape-sandbox allow-top-navigation-by-user-activation"></iframe></p>
<div ><a class="flourish-credit" href="https://public.flourish.studio/visualisation/6272666/?utm_source=embed&;utm_campaign=visualisation/6272666" target="_top" rel="noopener"><img src="https://public.flourish.studio/resources/made_with_flourish.svg" /> </a></div>
<hr />
<h2>Why is this happening?</h2>
<p>I also analysed how students approached the answers by tracking the time it took them to respond to a question, as well as the number of questions each student skipped.</p>
<p>PISA data allows me to identify students who answer questions too fast (say in under three seconds, which does not allow for careful reading of the question).</p>
<p>Answering questions too fast or skipping them entirely can be seen as a sign of low effort or inattentiveness.</p>
<p>I found a gender difference in the approach students took to answering questions.</p>
<p><iframe style="width: 100%; height: 600px;" title="Interactive or visual content" src="https://flo.uri.sh/visualisation/6272762/embed" width="100%" height="400" frameborder="0" scrolling="no" sandbox="allow-same-origin allow-forms allow-scripts allow-downloads allow-popups allow-popups-to-escape-sandbox allow-top-navigation-by-user-activation"></iframe></p>
<div ><a class="flourish-credit" href="https://public.flourish.studio/visualisation/6272762/?utm_source=embed&;utm_campaign=visualisation/6272762" target="_top" rel="noopener"><img src="https://public.flourish.studio/resources/made_with_flourish.svg" /> </a></div>
<p>Overall, boys were less engaged in the test than girls. They answered questions faster and skipped more of them. However, this difference started to reverse the more multiple-choice questions there were in the test.</p>
<p>Girls who received an exam with more multiple-choice questions were more likely to show a lack of effort than when there were more open-ended questions.</p>
<p><a href="https://www.tandfonline.com/doi/abs/10.1080/08957340902754635?casa_token=TQyOdyCueZUAAAAA:j-NU-mqczMCe2hOmiY5u4qa-XkSBA0sFqISwPjUiQvW1VbJpbWhxyQLsn8Wx4QFg9CjpS4Vpy6iyXKs">Previous research</a> supports the idea girls can be less engaged with multiple-choice questions. Girls tend to prefer questions that require more analysis and varied solutions while boys are more likely to just state their answers.</p>
<h2>Confidence matters too</h2>
<p>A student’s confidence in their maths knowledge can also play a part in their performance. For example, a higher level of confidence affects how fast students can rule out an incorrect responses.</p>
<p>PISA 2015 didn’t provide a measure of students’ levels of confidence.</p>
<p>However, <a href="https://bera-journals.onlinelibrary.wiley.com/doi/full/10.1002/berj.3329">previous research</a> has shown girls with mothers working in science, technology, engineering or maths (STEM) occupations are more confident in maths and less likely to believe the stereotypes boys are better than girls.</p>
<p>So, I used maternal occupation as a measure of girls’ level of confidence and beliefs in their maths abilities. I found the negative effect of multiple-choice questions on girls’ performance actually disappeared in girls whose mothers worked in STEM-related occupations.</p>
<p>These findings suggest multiple-choice exams may not be the most appropriate tools to measure students’ levels of knowledge.<!-- Below is The Conversation's page counter tag. Please DO NOT REMOVE. --><img style="border: none !important; box-shadow: none !important; margin: 0 !important; max-height: 1px !important; max-width: 1px !important; min-height: 1px !important; min-width: 1px !important; opacity: 0 !important; outline: none !important; padding: 0 !important; text-shadow: none !important;" src="https://counter.theconversation.com/content/160724/count.gif?distributor=republish-lightbox-basic" alt="The Conversation" width="1" height="1" /><!-- End of code. If you don't see any code above, please get new code from the Advanced tab after you click the republish button. The page counter does not collect any personal data. More info: https://theconversation.com/republishing-guidelines --></p>
<h6><a href="https://theconversation.com/profiles/silvia-griselda-896868">Silvia Griselda</a>, Postdoctoral research fellow, <em><a href="https://theconversation.com/institutions/the-university-of-melbourne-722">The University of Melbourne. </a></em>This article is republished from <a href="https://theconversation.com">The Conversation</a> under a Creative Commons license. Read the <a href="https://theconversation.com/multiple-choice-exams-favour-boys-over-girls-worsening-the-maths-gender-gap-160724">original article</a>.</h6>

A study investigating the realities of out-of-field teachers is seeking participants for groundbreaking research.
The resources are designed to support teachers to make sure all students are engaged in…
Tic disorders are far more common than many people realise, and are often misrepresented in…
The school library has long been a place of discovery, reflection, and learning. But as…
Is your school an inclusive community that empowers students to recognise bullying and to stand…
Performance indicators for the education and VET sectors have just been released with some encouraging…
This website uses cookies.