If you’re sitting an online or open book test, you’d expect it to be easier. But you’d be wrong.
Many students fall into the trap of easing their revision or feeling like their preparation is suddenly so much easier because the test is online. Unfortunately, there are three key reasons this doesn’t play out in real life.
In 2020, University of Auckland moved to online, 24-hour, open book testing. The result was massive variation in grades with the lowest medical admissions intake in the last 10 years. Assessment style was obviously not the only factor to influence the admission rate, but the amount of uncertainty, distress and regret from students who completely mis-prepared for their exams was higher than any year we had ever seen.
Why online or open-book tests can be harder
First of all, online testing is not the same as open-book testing, unless you plan to cheat.
Online testing can be invigilated to combat the rampant cheating and sharing of answers that you could expect students to engage in (unless you’re a horribly out of touch university of course…)
For example, many Australian universities hired exam supervisors who observed groups of students sitting their exam using Zoom, while requiring students to use certain apps that detect if they are accessing websites or other activity while sitting the exam.
If you are caught cheating, either in the exam or in retrospect through someone reporting you or in a review of your answers, you can guarantee that you will be unable to enter medical school ever again within the Australasia region. On top of completely undermining the integrity of the very profession you are seeking to enter into, it’s much more seriously punished in competitive degrees like premed.
Therefore, don’t cheat.
With that out of the way, here are three reasons why these tests are surprisingly difficult.
In the world of assessment writing, there are generally two modes of assessing: norm-referenced and criterion-referenced. Simply put, criterion-referenced testing requires you to meet certain metrics on a marking rubric. The standard is fixed and you need to reach it. Norm-referenced testing is based on your performance compared to other students. Your total “grade” is scaled against the performance of your peers. Each has its pro’s and con’s.
Many assessments in University are a combination of both where criterion needs to be met, then total scores are scaled based on normative data. On top of this, many Universities (the University of Auckland included) use multiple tests to balance the difficulty of the overall paper.
For example, if “test 1” is too easy, “test 2” or the exam is harder. This ensures that over- or under-difficulty tests are balanced across an entire paper.
This means that if you find a test easy, many other students are likely to have found it easy, resulting in partial scaling of the results and a more difficult second test or exam. A single test being “easy” may seem like a good thing, but actually the best scenario is for the test to be just the right amount of difficulty.
Compensated assessment style
Universities are not stupid. They know that students will be able to answer questions more easily if they have access to information, such as in an open-book format. As a result, almost all “open-book” style testing will feature a different level of questioning to test on higher levels of knowledge mastery.
There are multiple different frameworks that are used by assessment writers, depending on their guidelines, region or educational stage, however each framework shares the common trend that more difficult questions require you to use multiple concepts together and appreciate the information outside of isolation.
It’s not enough to know the details or be able to recall the facts at a university level. You’ll likely lose marks by not having strong inter-connectedness between ideas (which may have been taught to you in isolation).
Bloom’s taxonomy and SOLO taxonomy are two very common frameworks that are used. Think about what kind of questions might arise from an assessment writer aiming at the low levels vs what you might see in open-book testing, which are higher levels.
This harder level of assessment directly ties in to the final reason why underpreparation for open-book or online testing can be a game breaker…
When’s the last time you sat a difficult exam and found that you had lots of time to sit and ponder over every question?
Usually, exams are rushed and a certain “pace” is needed to get through everything to a sufficient quality. Online and open-book tests can be even worse. There are multiple reasons for this.
Firstly, because the test questions tend to be more difficult, not only do you need to spend more time answering them, but any deficiencies in knowledge take disproportionately longer to fill.
For example, if you don’t know the answer to a simple recall-style question, finding the answer only takes as long as looking it up. However if your question requires you to not only recall three separate facts, but additionally understand the relationship between them and the implications they have in relation to the question, a single question can easily take 5 to 10 minutes to answer. This is simply not viable.
Secondly, even the time it takes to look up basic facts can really stack up. If it takes you an average of 10 seconds to look up something you aren’t sure about, then you can easily waste 10 to 15 minutes on just looking up information. Given that the time it takes to answer difficult questions is longer, this is not time you can afford to waste.
As a result, students often remark that these kinds of tests are more stressful and pressuring than normal tests.
Prepare for online or open-book tests with the following guidelines:
- Come prepared. Don’t rely on looking anything up in an open-book test. It should be there for you just in case, not as a crutch. You should never rely on actually learning anything new during the exam. The chances that you can actually learn something and then apply the information accurately is too low to be safe.
- Offload irrelevant details, but still know the gist. If allowed to use a cheat sheet, use it for highly specific details such as cytokine numbers, reaction constants, equations, exact pressures in a blood vessel, etc. This will save time in revision so that you don’t need to memorise so many irrelevant details, but you must still have memorised the overall trend. Even if you don’t know the exact pressure or concentrations of something, you should have a ballpark figure and understand if it is higher or lower than something else.
- Don’t isolate information while revising. Double-down on inter-related understanding and practice generating and answering questions that force you to look at information in relation to other information. For the most difficult questions, testing isolated facts using things like flashcards will not get you very far (or you will need a lot of flashcards and a lot of time).
- Think like an evil examiner. Think of ways that an examiner could ask you about concepts that would make you stumble. Focus less on just having memorised the content and actively look for ways that you could be tested that expose gaps in your conceptual understanding, especially in regards to the implications that one concept can have on another. Don’t just reinforce what you already know; seek out your weaknesses in advance.
If you’re struggling with your studying or revision technique and you want to upskill, get in touch with our advisor to see if there are any options or courses that you either already have access to, or might want to consider purchasing.